2023-11-16T16:15:10.366 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-11-16T16:15:10.366 DEBUG:teuthology.run:Teuthology command: teuthology --description fs:workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/fs/misc}} --owner scheduled_choffman@teuthology --verbose --name choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi --archive /home/teuthworker/archive/choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi/7460919 -- /home/teuthworker/archive/choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi/7460919/orig.config.yaml 2023-11-16T16:15:10.419 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-16T16:15:10.502 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi/7460919 branch: wip-choffman-62793 description: fs:workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/fs/misc}} email: null first_in_suite: false job_id: '7460919' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-choffman-62793 ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 5e4a743ad7fd1422ebd6c4dc950a21a303279207 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: 5e4a743ad7fd1422ebd6c4dc950a21a303279207 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - wsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-choffman-62793 sha1: 5e4a743ad7fd1422ebd6c4dc950a21a303279207 owner: scheduled_choffman@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 4433 sha1: 5e4a743ad7fd1422ebd6c4dc950a21a303279207 sleep_before_teardown: 0 suite: fs:workload suite_branch: wip-choffman-62793 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 5e4a743ad7fd1422ebd6c4dc950a21a303279207 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi001.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi026.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi053.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1M - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6M3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - fs/misc teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 6899cd26fceddb2fec83dc1a1349394b28c8998e timestamp: 2023-11-16_16:09:17 tube: smithi user: choffman verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.1224587 2023-11-16T16:15:10.502 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa; will attempt to use it 2023-11-16T16:15:10.503 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa/tasks 2023-11-16T16:15:10.503 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-16T16:15:10.504 INFO:teuthology.task.internal:Checking packages... 2023-11-16T16:15:10.525 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '5e4a743ad7fd1422ebd6c4dc950a21a303279207' 2023-11-16T16:15:10.525 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-16T16:15:10.525 INFO:teuthology.packaging:ref: None 2023-11-16T16:15:10.525 INFO:teuthology.packaging:tag: None 2023-11-16T16:15:10.525 INFO:teuthology.packaging:branch: wip-choffman-62793 2023-11-16T16:15:10.525 INFO:teuthology.packaging:sha1: 5e4a743ad7fd1422ebd6c4dc950a21a303279207 2023-11-16T16:15:10.525 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-choffman-62793 2023-11-16T16:15:10.669 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-3905.g5e4a743a 2023-11-16T16:15:10.669 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-16T16:15:10.676 INFO:teuthology.task.internal:no buildpackages task found 2023-11-16T16:15:10.676 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-16T16:15:10.708 INFO:teuthology.task.internal:Saving configuration 2023-11-16T16:15:10.726 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-16T16:15:10.732 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-16T16:15:10.758 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi001.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi/7460919', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-16 16:10:26.161957', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-16T16:15:10.779 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi026.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi/7460919', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-16 16:10:26.160345', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-16T16:15:10.798 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi053.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi/7460919', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-16 16:10:26.158524', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-16T16:15:10.799 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-16T16:15:10.805 INFO:teuthology.task.internal:roles: ubuntu@smithi001.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-11-16T16:15:10.805 INFO:teuthology.task.internal:roles: ubuntu@smithi026.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-11-16T16:15:10.805 INFO:teuthology.task.internal:roles: ubuntu@smithi053.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-11-16T16:15:10.805 INFO:teuthology.run_tasks:Running task console_log... 2023-11-16T16:15:10.919 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8af2ae5b80>, signals=[15]) 2023-11-16T16:15:10.920 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-16T16:15:10.925 INFO:teuthology.task.internal:Opening connections... 2023-11-16T16:15:10.926 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi001.front.sepia.ceph.com 2023-11-16T16:15:10.927 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T16:15:10.995 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi026.front.sepia.ceph.com 2023-11-16T16:15:10.997 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T16:15:11.063 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi053.front.sepia.ceph.com 2023-11-16T16:15:11.064 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T16:15:11.130 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-16T16:15:11.137 DEBUG:teuthology.orchestra.run.smithi001:> uname -m 2023-11-16T16:15:11.221 INFO:teuthology.orchestra.run.smithi001.stdout:x86_64 2023-11-16T16:15:11.222 DEBUG:teuthology.orchestra.run.smithi001:> cat /etc/os-release 2023-11-16T16:15:11.240 INFO:teuthology.orchestra.run.smithi001.stdout:NAME="Red Hat Enterprise Linux" 2023-11-16T16:15:11.240 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION="8.6 (Ootpa)" 2023-11-16T16:15:11.241 INFO:teuthology.orchestra.run.smithi001.stdout:ID="rhel" 2023-11-16T16:15:11.241 INFO:teuthology.orchestra.run.smithi001.stdout:ID_LIKE="fedora" 2023-11-16T16:15:11.241 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION_ID="8.6" 2023-11-16T16:15:11.241 INFO:teuthology.orchestra.run.smithi001.stdout:PLATFORM_ID="platform:el8" 2023-11-16T16:15:11.241 INFO:teuthology.orchestra.run.smithi001.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-16T16:15:11.241 INFO:teuthology.orchestra.run.smithi001.stdout:ANSI_COLOR="0;31" 2023-11-16T16:15:11.241 INFO:teuthology.orchestra.run.smithi001.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-16T16:15:11.241 INFO:teuthology.orchestra.run.smithi001.stdout:HOME_URL="https://www.redhat.com/" 2023-11-16T16:15:11.241 INFO:teuthology.orchestra.run.smithi001.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-16T16:15:11.241 INFO:teuthology.orchestra.run.smithi001.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-16T16:15:11.241 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:15:11.241 INFO:teuthology.orchestra.run.smithi001.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-16T16:15:11.242 INFO:teuthology.orchestra.run.smithi001.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-16T16:15:11.242 INFO:teuthology.orchestra.run.smithi001.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-16T16:15:11.242 INFO:teuthology.orchestra.run.smithi001.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-16T16:15:11.242 INFO:teuthology.lock.ops:Updating smithi001.front.sepia.ceph.com on lock server 2023-11-16T16:15:11.267 DEBUG:teuthology.orchestra.run.smithi026:> uname -m 2023-11-16T16:15:11.288 INFO:teuthology.orchestra.run.smithi026.stdout:x86_64 2023-11-16T16:15:11.288 DEBUG:teuthology.orchestra.run.smithi026:> cat /etc/os-release 2023-11-16T16:15:11.348 INFO:teuthology.orchestra.run.smithi026.stdout:NAME="Red Hat Enterprise Linux" 2023-11-16T16:15:11.348 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION="8.6 (Ootpa)" 2023-11-16T16:15:11.348 INFO:teuthology.orchestra.run.smithi026.stdout:ID="rhel" 2023-11-16T16:15:11.348 INFO:teuthology.orchestra.run.smithi026.stdout:ID_LIKE="fedora" 2023-11-16T16:15:11.348 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION_ID="8.6" 2023-11-16T16:15:11.348 INFO:teuthology.orchestra.run.smithi026.stdout:PLATFORM_ID="platform:el8" 2023-11-16T16:15:11.348 INFO:teuthology.orchestra.run.smithi026.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-16T16:15:11.348 INFO:teuthology.orchestra.run.smithi026.stdout:ANSI_COLOR="0;31" 2023-11-16T16:15:11.348 INFO:teuthology.orchestra.run.smithi026.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-16T16:15:11.348 INFO:teuthology.orchestra.run.smithi026.stdout:HOME_URL="https://www.redhat.com/" 2023-11-16T16:15:11.349 INFO:teuthology.orchestra.run.smithi026.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-16T16:15:11.349 INFO:teuthology.orchestra.run.smithi026.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-16T16:15:11.349 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-11-16T16:15:11.349 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-16T16:15:11.349 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-16T16:15:11.349 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-16T16:15:11.349 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-16T16:15:11.349 INFO:teuthology.lock.ops:Updating smithi026.front.sepia.ceph.com on lock server 2023-11-16T16:15:11.367 DEBUG:teuthology.orchestra.run.smithi053:> uname -m 2023-11-16T16:15:11.387 INFO:teuthology.orchestra.run.smithi053.stdout:x86_64 2023-11-16T16:15:11.387 DEBUG:teuthology.orchestra.run.smithi053:> cat /etc/os-release 2023-11-16T16:15:11.448 INFO:teuthology.orchestra.run.smithi053.stdout:NAME="Red Hat Enterprise Linux" 2023-11-16T16:15:11.448 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION="8.6 (Ootpa)" 2023-11-16T16:15:11.448 INFO:teuthology.orchestra.run.smithi053.stdout:ID="rhel" 2023-11-16T16:15:11.448 INFO:teuthology.orchestra.run.smithi053.stdout:ID_LIKE="fedora" 2023-11-16T16:15:11.448 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_ID="8.6" 2023-11-16T16:15:11.448 INFO:teuthology.orchestra.run.smithi053.stdout:PLATFORM_ID="platform:el8" 2023-11-16T16:15:11.448 INFO:teuthology.orchestra.run.smithi053.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-16T16:15:11.448 INFO:teuthology.orchestra.run.smithi053.stdout:ANSI_COLOR="0;31" 2023-11-16T16:15:11.448 INFO:teuthology.orchestra.run.smithi053.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-16T16:15:11.448 INFO:teuthology.orchestra.run.smithi053.stdout:HOME_URL="https://www.redhat.com/" 2023-11-16T16:15:11.449 INFO:teuthology.orchestra.run.smithi053.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-16T16:15:11.449 INFO:teuthology.orchestra.run.smithi053.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-16T16:15:11.449 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-16T16:15:11.449 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-16T16:15:11.449 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-16T16:15:11.449 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-16T16:15:11.449 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-16T16:15:11.449 INFO:teuthology.lock.ops:Updating smithi053.front.sepia.ceph.com on lock server 2023-11-16T16:15:11.466 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-16T16:15:11.476 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-16T16:15:11.523 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-16T16:15:11.524 DEBUG:teuthology.orchestra.run.smithi001:> test '!' -e /home/ubuntu/cephtest 2023-11-16T16:15:11.526 DEBUG:teuthology.orchestra.run.smithi026:> test '!' -e /home/ubuntu/cephtest 2023-11-16T16:15:11.528 DEBUG:teuthology.orchestra.run.smithi053:> test '!' -e /home/ubuntu/cephtest 2023-11-16T16:15:11.547 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-16T16:15:11.554 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-16T16:15:11.554 DEBUG:teuthology.orchestra.run.smithi001:> test -z $(ls -A /var/lib/ceph) 2023-11-16T16:15:11.586 DEBUG:teuthology.orchestra.run.smithi026:> test -z $(ls -A /var/lib/ceph) 2023-11-16T16:15:11.588 DEBUG:teuthology.orchestra.run.smithi053:> test -z $(ls -A /var/lib/ceph) 2023-11-16T16:15:11.618 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-16T16:15:11.694 INFO:teuthology.run_tasks:Running task kernel... 2023-11-16T16:15:11.708 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2023-11-16T16:15:11.709 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2023-11-16T16:15:11.709 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2023-11-16T16:15:11.709 DEBUG:teuthology.orchestra.run.smithi001:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-16T16:15:11.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-16T16:15:11.729 DEBUG:teuthology.orchestra.run.smithi001:> uname -r 2023-11-16T16:15:11.788 INFO:teuthology.orchestra.run.smithi001.stdout:4.18.0-372.9.1.el8.x86_64 2023-11-16T16:15:11.789 INFO:teuthology.task.kernel:Running kernel on smithi001: 4.18.0-372.9.1.el8.x86_64 2023-11-16T16:15:11.789 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum install -y kernel 2023-11-16T16:15:13.028 INFO:teuthology.orchestra.run.smithi001.stdout:Updating Subscription Management repositories. 2023-11-16T16:15:13.029 INFO:teuthology.orchestra.run.smithi001.stdout:Unable to read consumer identity 2023-11-16T16:15:14.436 INFO:teuthology.orchestra.run.smithi001.stdout:Copr repo for python3-asyncssh owned by ceph 23 kB/s | 3.5 kB 00:00 2023-11-16T16:15:15.237 INFO:teuthology.orchestra.run.smithi001.stdout:Extra Packages for Enterprise Linux 20 MB/s | 16 MB 00:00 2023-11-16T16:15:19.037 INFO:teuthology.orchestra.run.smithi001.stdout:lab-extras 329 kB/s | 24 kB 00:00 2023-11-16T16:15:19.798 INFO:teuthology.orchestra.run.smithi001.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-11-16T16:15:19.850 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2023-11-16T16:15:19.851 INFO:teuthology.orchestra.run.smithi001.stdout:Nothing to do. 2023-11-16T16:15:19.851 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2023-11-16T16:15:20.006 DEBUG:teuthology.orchestra.run.smithi001:> echo no | sudo yum reinstall kernel || true 2023-11-16T16:15:20.358 INFO:teuthology.orchestra.run.smithi001.stdout:Updating Subscription Management repositories. 2023-11-16T16:15:20.359 INFO:teuthology.orchestra.run.smithi001.stdout:Unable to read consumer identity 2023-11-16T16:15:20.632 INFO:teuthology.orchestra.run.smithi001.stdout:Last metadata expiration check: 0:00:01 ago on Thu 16 Nov 2023 04:15:19 PM UTC. 2023-11-16T16:15:20.674 INFO:teuthology.orchestra.run.smithi001.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-16T16:15:20.734 INFO:teuthology.orchestra.run.smithi001.stderr:Error: No packages marked for reinstall. 2023-11-16T16:15:20.782 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum reinstall -y kernel || true 2023-11-16T16:15:21.133 INFO:teuthology.orchestra.run.smithi001.stdout:Updating Subscription Management repositories. 2023-11-16T16:15:21.133 INFO:teuthology.orchestra.run.smithi001.stdout:Unable to read consumer identity 2023-11-16T16:15:21.402 INFO:teuthology.orchestra.run.smithi001.stdout:Last metadata expiration check: 0:00:02 ago on Thu 16 Nov 2023 04:15:19 PM UTC. 2023-11-16T16:15:21.446 INFO:teuthology.orchestra.run.smithi001.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-16T16:15:21.474 INFO:teuthology.orchestra.run.smithi001.stderr:Error: No packages marked for reinstall. 2023-11-16T16:15:21.523 DEBUG:teuthology.orchestra.run.smithi001:> rpm -q kernel | sort -rV | head -n 1 2023-11-16T16:15:21.606 INFO:teuthology.orchestra.run.smithi001.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-11-16T16:15:21.606 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-11-16T16:15:21.606 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-11-16T16:15:21.607 DEBUG:teuthology.parallel:result is None 2023-11-16T16:15:21.607 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-16T16:15:21.615 INFO:teuthology.task.internal:Creating test directory... 2023-11-16T16:15:21.615 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-16T16:15:21.618 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-16T16:15:21.620 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-16T16:15:21.641 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-16T16:15:21.649 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-16T16:15:21.673 INFO:teuthology.task.internal:Creating archive directory... 2023-11-16T16:15:21.674 DEBUG:teuthology.orchestra.run.smithi001:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-16T16:15:21.676 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-16T16:15:21.682 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-16T16:15:21.715 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-16T16:15:21.722 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-16T16:15:21.722 DEBUG:teuthology.orchestra.run.smithi026:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-16T16:15:21.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-16T16:15:21.768 DEBUG:teuthology.orchestra.run.smithi053:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-16T16:15:21.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-16T16:15:21.788 DEBUG:teuthology.orchestra.run.smithi001:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-16T16:15:21.790 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-16T16:15:21.810 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-16T16:15:21.830 INFO:teuthology.orchestra.run.smithi001.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-16T16:15:21.848 INFO:teuthology.orchestra.run.smithi001.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-16T16:15:21.860 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-16T16:15:21.869 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-16T16:15:21.889 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-16T16:15:21.901 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-16T16:15:21.903 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-16T16:15:21.910 INFO:teuthology.task.internal:Configuring sudo... 2023-11-16T16:15:21.910 DEBUG:teuthology.orchestra.run.smithi001:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-16T16:15:21.912 DEBUG:teuthology.orchestra.run.smithi026:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-16T16:15:21.914 DEBUG:teuthology.orchestra.run.smithi053:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-16T16:15:21.962 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-16T16:15:21.970 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-16T16:15:21.971 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-16T16:15:21.980 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-16T16:15:21.985 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-16T16:15:22.022 DEBUG:teuthology.orchestra.run.smithi001:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-16T16:15:22.077 DEBUG:teuthology.orchestra.run.smithi001:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-16T16:15:22.125 DEBUG:teuthology.orchestra.run.smithi001:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-16T16:15:22.210 DEBUG:teuthology.orchestra.run.smithi001:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-16T16:15:22.251 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2023-11-16T16:15:22.252 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-16T16:15:22.320 DEBUG:teuthology.orchestra.run.smithi026:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-16T16:15:22.368 DEBUG:teuthology.orchestra.run.smithi026:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-16T16:15:22.425 DEBUG:teuthology.orchestra.run.smithi026:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-16T16:15:22.510 DEBUG:teuthology.orchestra.run.smithi026:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-16T16:15:22.559 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-11-16T16:15:22.559 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-16T16:15:22.653 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-16T16:15:22.702 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-16T16:15:22.756 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-16T16:15:22.843 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-16T16:15:22.891 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-16T16:15:22.891 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-16T16:15:22.962 DEBUG:teuthology.orchestra.run.smithi001:> sudo service rsyslog restart 2023-11-16T16:15:22.965 DEBUG:teuthology.orchestra.run.smithi026:> sudo service rsyslog restart 2023-11-16T16:15:22.968 DEBUG:teuthology.orchestra.run.smithi053:> sudo service rsyslog restart 2023-11-16T16:15:23.018 INFO:teuthology.orchestra.run.smithi001.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-16T16:15:23.027 INFO:teuthology.orchestra.run.smithi026.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-16T16:15:23.083 INFO:teuthology.orchestra.run.smithi053.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-16T16:15:23.447 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-16T16:15:23.456 INFO:teuthology.task.internal:Starting timer... 2023-11-16T16:15:23.456 INFO:teuthology.run_tasks:Running task pcp... 2023-11-16T16:15:23.514 INFO:teuthology.run_tasks:Running task selinux... 2023-11-16T16:15:23.576 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2023-11-16T16:15:23.577 DEBUG:teuthology.orchestra.run.smithi001:> sudo service auditd rotate 2023-11-16T16:15:23.650 INFO:teuthology.orchestra.run.smithi001.stdout:Rotating logs: 2023-11-16T16:15:23.652 DEBUG:teuthology.orchestra.run.smithi026:> sudo service auditd rotate 2023-11-16T16:15:23.833 INFO:teuthology.orchestra.run.smithi026.stdout:Rotating logs: 2023-11-16T16:15:23.835 DEBUG:teuthology.orchestra.run.smithi053:> sudo service auditd rotate 2023-11-16T16:15:23.905 INFO:teuthology.orchestra.run.smithi053.stdout:Rotating logs: 2023-11-16T16:15:23.907 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-16T16:15:23.907 DEBUG:teuthology.orchestra.run.smithi001:> /usr/sbin/getenforce 2023-11-16T16:15:23.936 INFO:teuthology.orchestra.run.smithi001.stdout:Permissive 2023-11-16T16:15:23.937 DEBUG:teuthology.orchestra.run.smithi026:> /usr/sbin/getenforce 2023-11-16T16:15:23.961 INFO:teuthology.orchestra.run.smithi026.stdout:Permissive 2023-11-16T16:15:23.962 DEBUG:teuthology.orchestra.run.smithi053:> /usr/sbin/getenforce 2023-11-16T16:15:23.987 INFO:teuthology.orchestra.run.smithi053.stdout:Permissive 2023-11-16T16:15:23.987 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi001.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi026.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi053.front.sepia.ceph.com': 'permissive'} 2023-11-16T16:15:23.987 DEBUG:teuthology.orchestra.run.smithi001:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-16T16:15:24.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-16T16:15:24.017 DEBUG:teuthology.orchestra.run.smithi026:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-16T16:15:24.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-16T16:15:24.046 DEBUG:teuthology.orchestra.run.smithi053:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-16T16:15:24.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-16T16:15:24.075 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-16T16:15:24.075 DEBUG:teuthology.orchestra.run.smithi001:> sudo /usr/sbin/setenforce permissive 2023-11-16T16:15:24.103 DEBUG:teuthology.orchestra.run.smithi026:> sudo /usr/sbin/setenforce permissive 2023-11-16T16:15:24.143 DEBUG:teuthology.orchestra.run.smithi053:> sudo /usr/sbin/setenforce permissive 2023-11-16T16:15:24.181 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-16T16:15:24.193 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-11-16T16:15:24.200 INFO:teuthology.repo_utils:Fetching main from origin 2023-11-16T16:15:24.352 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-16T16:15:24.369 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-16T16:15:24.370 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi001.front.sepia.ceph.com,smithi026.front.sepia.ceph.com,smithi053.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-16T16:25:26.327 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi001.front.sepia.ceph.com'), Remote(name='ubuntu@smithi026.front.sepia.ceph.com'), Remote(name='ubuntu@smithi053.front.sepia.ceph.com')] 2023-11-16T16:25:26.328 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2023-11-16T16:25:26.329 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T16:25:26.401 DEBUG:teuthology.orchestra.run.smithi001:> true 2023-11-16T16:25:26.473 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi001.front.sepia.ceph.com' 2023-11-16T16:25:26.473 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2023-11-16T16:25:26.474 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T16:25:26.544 DEBUG:teuthology.orchestra.run.smithi026:> true 2023-11-16T16:25:26.618 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi026.front.sepia.ceph.com' 2023-11-16T16:25:26.619 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2023-11-16T16:25:26.620 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-16T16:25:26.689 DEBUG:teuthology.orchestra.run.smithi053:> true 2023-11-16T16:25:26.765 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi053.front.sepia.ceph.com' 2023-11-16T16:25:26.766 INFO:teuthology.run_tasks:Running task clock... 2023-11-16T16:25:26.776 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-16T16:25:26.776 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-16T16:25:26.777 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-16T16:25:26.779 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-16T16:25:26.780 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-16T16:25:26.782 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-16T16:25:26.782 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-16T16:25:26.815 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-16T16:25:26.817 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-16T16:25:26.832 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-16T16:25:26.834 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-16T16:25:26.839 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-16T16:25:26.856 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-16T16:25:26.908 INFO:teuthology.orchestra.run.smithi001.stderr:sudo: ntpd: command not found 2023-11-16T16:25:26.918 INFO:teuthology.orchestra.run.smithi026.stderr:sudo: ntpd: command not found 2023-11-16T16:25:26.922 INFO:teuthology.orchestra.run.smithi001.stdout:506 Cannot talk to daemon 2023-11-16T16:25:26.924 INFO:teuthology.orchestra.run.smithi053.stderr:sudo: ntpd: command not found 2023-11-16T16:25:26.932 INFO:teuthology.orchestra.run.smithi026.stdout:506 Cannot talk to daemon 2023-11-16T16:25:26.936 INFO:teuthology.orchestra.run.smithi053.stdout:506 Cannot talk to daemon 2023-11-16T16:25:26.938 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-16T16:25:26.948 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-16T16:25:26.951 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-16T16:25:26.952 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-16T16:25:26.963 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-16T16:25:26.966 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-16T16:25:27.031 INFO:teuthology.orchestra.run.smithi026.stderr:bash: ntpq: command not found 2023-11-16T16:25:27.033 INFO:teuthology.orchestra.run.smithi001.stderr:bash: ntpq: command not found 2023-11-16T16:25:27.035 INFO:teuthology.orchestra.run.smithi026.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-16T16:25:27.035 INFO:teuthology.orchestra.run.smithi026.stdout:=============================================================================== 2023-11-16T16:25:27.035 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T16:25:27.035 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T16:25:27.035 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T16:25:27.035 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T16:25:27.037 INFO:teuthology.orchestra.run.smithi001.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-16T16:25:27.037 INFO:teuthology.orchestra.run.smithi001.stdout:=============================================================================== 2023-11-16T16:25:27.037 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T16:25:27.037 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T16:25:27.037 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T16:25:27.037 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T16:25:27.045 INFO:teuthology.orchestra.run.smithi053.stderr:bash: ntpq: command not found 2023-11-16T16:25:27.049 INFO:teuthology.orchestra.run.smithi053.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-16T16:25:27.049 INFO:teuthology.orchestra.run.smithi053.stdout:=============================================================================== 2023-11-16T16:25:27.049 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T16:25:27.049 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T16:25:27.049 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T16:25:27.049 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T16:25:27.050 INFO:teuthology.run_tasks:Running task pexec... 2023-11-16T16:25:27.060 INFO:teuthology.task.pexec:Executing custom commands... 2023-11-16T16:25:27.060 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi001.front.sepia.ceph.com 2023-11-16T16:25:27.060 DEBUG:teuthology.orchestra.run.smithi001:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-11-16T16:25:27.061 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi026.front.sepia.ceph.com 2023-11-16T16:25:27.061 DEBUG:teuthology.orchestra.run.smithi026:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-11-16T16:25:27.061 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi053.front.sepia.ceph.com 2023-11-16T16:25:27.061 DEBUG:teuthology.orchestra.run.smithi053:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-11-16T16:25:27.470 INFO:teuthology.orchestra.run.smithi026.stdout:Updating Subscription Management repositories. 2023-11-16T16:25:27.471 INFO:teuthology.orchestra.run.smithi001.stdout:Updating Subscription Management repositories. 2023-11-16T16:25:27.482 INFO:teuthology.orchestra.run.smithi053.stdout:Updating Subscription Management repositories. 2023-11-16T16:25:29.004 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:52 ago on Thu 16 Nov 2023 04:23:36 PM UTC. 2023-11-16T16:25:29.110 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:49 ago on Thu 16 Nov 2023 04:23:39 PM UTC. 2023-11-16T16:25:29.230 INFO:teuthology.orchestra.run.smithi001.stdout:Last metadata expiration check: 0:02:15 ago on Thu 16 Nov 2023 04:23:13 PM UTC. 2023-11-16T16:25:31.021 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2023-11-16T16:25:31.022 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2023-11-16T16:25:31.022 INFO:teuthology.orchestra.run.smithi053.stdout: Package Architecture Version Repository Size 2023-11-16T16:25:31.022 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2023-11-16T16:25:31.022 INFO:teuthology.orchestra.run.smithi053.stdout:Resetting modules: 2023-11-16T16:25:31.022 INFO:teuthology.orchestra.run.smithi053.stdout: container-tools 2023-11-16T16:25:31.022 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-16T16:25:31.022 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2023-11-16T16:25:31.022 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2023-11-16T16:25:31.023 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-16T16:25:31.088 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2023-11-16T16:25:31.089 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2023-11-16T16:25:31.089 INFO:teuthology.orchestra.run.smithi026.stdout: Package Architecture Version Repository Size 2023-11-16T16:25:31.089 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2023-11-16T16:25:31.089 INFO:teuthology.orchestra.run.smithi026.stdout:Resetting modules: 2023-11-16T16:25:31.089 INFO:teuthology.orchestra.run.smithi026.stdout: container-tools 2023-11-16T16:25:31.089 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-11-16T16:25:31.089 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2023-11-16T16:25:31.089 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2023-11-16T16:25:31.089 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-11-16T16:25:31.220 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2023-11-16T16:25:31.221 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2023-11-16T16:25:31.222 INFO:teuthology.orchestra.run.smithi001.stdout: Package Architecture Version Repository Size 2023-11-16T16:25:31.222 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2023-11-16T16:25:31.222 INFO:teuthology.orchestra.run.smithi001.stdout:Resetting modules: 2023-11-16T16:25:31.222 INFO:teuthology.orchestra.run.smithi001.stdout: container-tools 2023-11-16T16:25:31.222 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:25:31.222 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction Summary 2023-11-16T16:25:31.223 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2023-11-16T16:25:31.223 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:25:32.021 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:55 ago on Thu 16 Nov 2023 04:23:36 PM UTC. 2023-11-16T16:25:32.116 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:52 ago on Thu 16 Nov 2023 04:23:39 PM UTC. 2023-11-16T16:25:32.229 INFO:teuthology.orchestra.run.smithi001.stdout:Last metadata expiration check: 0:02:18 ago on Thu 16 Nov 2023 04:23:13 PM UTC. 2023-11-16T16:25:41.779 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2023-11-16T16:25:42.258 INFO:teuthology.orchestra.run.smithi001.stdout:Updating Subscription Management repositories. 2023-11-16T16:25:42.977 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2023-11-16T16:25:43.273 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2023-11-16T16:25:43.458 INFO:teuthology.orchestra.run.smithi026.stdout:Updating Subscription Management repositories. 2023-11-16T16:25:43.760 INFO:teuthology.orchestra.run.smithi053.stdout:Updating Subscription Management repositories. 2023-11-16T16:25:43.819 INFO:teuthology.orchestra.run.smithi001.stdout:Last metadata expiration check: 0:02:30 ago on Thu 16 Nov 2023 04:23:13 PM UTC. 2023-11-16T16:25:45.129 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:02:05 ago on Thu 16 Nov 2023 04:23:39 PM UTC. 2023-11-16T16:25:45.503 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:02:09 ago on Thu 16 Nov 2023 04:23:36 PM UTC. 2023-11-16T16:25:45.795 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2023-11-16T16:25:45.895 INFO:teuthology.orchestra.run.smithi001.stdout:============================================================================================================ 2023-11-16T16:25:45.895 INFO:teuthology.orchestra.run.smithi001.stdout: Package Arch Version Repository Size 2023-11-16T16:25:45.896 INFO:teuthology.orchestra.run.smithi001.stdout:============================================================================================================ 2023-11-16T16:25:45.896 INFO:teuthology.orchestra.run.smithi001.stdout:Installing group/module packages: 2023-11-16T16:25:45.896 INFO:teuthology.orchestra.run.smithi001.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-11-16T16:25:45.896 INFO:teuthology.orchestra.run.smithi001.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-11-16T16:25:45.896 INFO:teuthology.orchestra.run.smithi001.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-11-16T16:25:45.896 INFO:teuthology.orchestra.run.smithi001.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-11-16T16:25:45.896 INFO:teuthology.orchestra.run.smithi001.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-11-16T16:25:45.896 INFO:teuthology.orchestra.run.smithi001.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-11-16T16:25:45.896 INFO:teuthology.orchestra.run.smithi001.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-11-16T16:25:45.896 INFO:teuthology.orchestra.run.smithi001.stdout:Installing dependencies: 2023-11-16T16:25:45.896 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-11-16T16:25:45.896 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-11-16T16:25:45.896 INFO:teuthology.orchestra.run.smithi001.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-11-16T16:25:45.897 INFO:teuthology.orchestra.run.smithi001.stdout:Installing module profiles: 2023-11-16T16:25:45.897 INFO:teuthology.orchestra.run.smithi001.stdout: container-tools/common 2023-11-16T16:25:45.897 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:25:45.897 INFO:teuthology.orchestra.run.smithi001.stdout:Enabling module streams: 2023-11-16T16:25:45.897 INFO:teuthology.orchestra.run.smithi001.stdout: container-tools rhel8 2023-11-16T16:25:45.897 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:25:45.897 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction Summary 2023-11-16T16:25:45.897 INFO:teuthology.orchestra.run.smithi001.stdout:============================================================================================================ 2023-11-16T16:25:45.897 INFO:teuthology.orchestra.run.smithi001.stdout:Install 10 Packages 2023-11-16T16:25:45.897 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:25:45.898 INFO:teuthology.orchestra.run.smithi001.stdout:Total download size: 18 M 2023-11-16T16:25:45.898 INFO:teuthology.orchestra.run.smithi001.stdout:Installed size: 65 M 2023-11-16T16:25:45.898 INFO:teuthology.orchestra.run.smithi001.stdout:Downloading Packages: 2023-11-16T16:25:46.229 INFO:teuthology.orchestra.run.smithi001.stdout:(1/10): python3-pytoml-0.1.14-5.git7dea353.el8. 75 kB/s | 25 kB 00:00 2023-11-16T16:25:46.255 INFO:teuthology.orchestra.run.smithi001.stdout:(2/10): yajl-2.1.0-10.el8.x86_64.rpm 114 kB/s | 41 kB 00:00 2023-11-16T16:25:46.314 INFO:teuthology.orchestra.run.smithi001.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 226 kB/s | 94 kB 00:00 2023-11-16T16:25:46.555 INFO:teuthology.orchestra.run.smithi001.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 152 kB/s | 49 kB 00:00 2023-11-16T16:25:46.589 INFO:teuthology.orchestra.run.smithi001.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 1.5 MB/s | 524 kB 00:00 2023-11-16T16:25:46.764 INFO:teuthology.orchestra.run.smithi001.stdout:(6/10): buildah-1.26.2-1.module+el8.6.0+16771+2 18 MB/s | 8.0 MB 00:00 2023-11-16T16:25:46.789 INFO:teuthology.orchestra.run.smithi001.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 1.0 MB/s | 213 kB 00:00 2023-11-16T16:25:46.815 INFO:teuthology.orchestra.run.smithi001.stdout:(8/10): python3-podman-4.0.1-1.module+el8.6.0+1 577 kB/s | 149 kB 00:00 2023-11-16T16:25:47.114 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2023-11-16T16:25:47.198 INFO:teuthology.orchestra.run.smithi001.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 5.4 MB/s | 2.2 MB 00:00 2023-11-16T16:25:47.216 INFO:teuthology.orchestra.run.smithi026.stdout:============================================================================================================ 2023-11-16T16:25:47.216 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repository Size 2023-11-16T16:25:47.216 INFO:teuthology.orchestra.run.smithi026.stdout:============================================================================================================ 2023-11-16T16:25:47.216 INFO:teuthology.orchestra.run.smithi026.stdout:Installing group/module packages: 2023-11-16T16:25:47.216 INFO:teuthology.orchestra.run.smithi026.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-11-16T16:25:47.216 INFO:teuthology.orchestra.run.smithi026.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-11-16T16:25:47.217 INFO:teuthology.orchestra.run.smithi026.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-11-16T16:25:47.217 INFO:teuthology.orchestra.run.smithi026.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-11-16T16:25:47.217 INFO:teuthology.orchestra.run.smithi026.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-11-16T16:25:47.217 INFO:teuthology.orchestra.run.smithi026.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-11-16T16:25:47.217 INFO:teuthology.orchestra.run.smithi026.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-11-16T16:25:47.217 INFO:teuthology.orchestra.run.smithi026.stdout:Installing dependencies: 2023-11-16T16:25:47.217 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-11-16T16:25:47.217 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-11-16T16:25:47.217 INFO:teuthology.orchestra.run.smithi026.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-11-16T16:25:47.217 INFO:teuthology.orchestra.run.smithi026.stdout:Installing module profiles: 2023-11-16T16:25:47.217 INFO:teuthology.orchestra.run.smithi026.stdout: container-tools/common 2023-11-16T16:25:47.218 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-11-16T16:25:47.218 INFO:teuthology.orchestra.run.smithi026.stdout:Enabling module streams: 2023-11-16T16:25:47.218 INFO:teuthology.orchestra.run.smithi026.stdout: container-tools rhel8 2023-11-16T16:25:47.218 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-11-16T16:25:47.218 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2023-11-16T16:25:47.218 INFO:teuthology.orchestra.run.smithi026.stdout:============================================================================================================ 2023-11-16T16:25:47.218 INFO:teuthology.orchestra.run.smithi026.stdout:Install 10 Packages 2023-11-16T16:25:47.218 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-11-16T16:25:47.219 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 18 M 2023-11-16T16:25:47.219 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 65 M 2023-11-16T16:25:47.219 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2023-11-16T16:25:47.348 INFO:teuthology.orchestra.run.smithi001.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 12 MB/s | 7.0 MB 00:00 2023-11-16T16:25:47.349 INFO:teuthology.orchestra.run.smithi001.stdout:-------------------------------------------------------------------------------- 2023-11-16T16:25:47.349 INFO:teuthology.orchestra.run.smithi001.stdout:Total 13 MB/s | 18 MB 00:01 2023-11-16T16:25:47.381 INFO:teuthology.orchestra.run.smithi026.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 253 kB/s | 41 kB 00:00 2023-11-16T16:25:47.406 INFO:teuthology.orchestra.run.smithi026.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 133 kB/s | 25 kB 00:00 2023-11-16T16:25:47.461 INFO:teuthology.orchestra.run.smithi026.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 387 kB/s | 94 kB 00:00 2023-11-16T16:25:47.492 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2023-11-16T16:25:47.523 INFO:teuthology.orchestra.run.smithi026.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 347 kB/s | 49 kB 00:00 2023-11-16T16:25:47.557 INFO:teuthology.orchestra.run.smithi026.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 3.4 MB/s | 524 kB 00:00 2023-11-16T16:25:47.560 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction check 2023-11-16T16:25:47.580 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction check succeeded. 2023-11-16T16:25:47.580 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction test 2023-11-16T16:25:47.590 INFO:teuthology.orchestra.run.smithi026.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.2 MB/s | 149 kB 00:00 2023-11-16T16:25:47.596 INFO:teuthology.orchestra.run.smithi053.stdout:============================================================================================================ 2023-11-16T16:25:47.596 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2023-11-16T16:25:47.596 INFO:teuthology.orchestra.run.smithi053.stdout:============================================================================================================ 2023-11-16T16:25:47.596 INFO:teuthology.orchestra.run.smithi053.stdout:Installing group/module packages: 2023-11-16T16:25:47.596 INFO:teuthology.orchestra.run.smithi053.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-11-16T16:25:47.596 INFO:teuthology.orchestra.run.smithi053.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-11-16T16:25:47.596 INFO:teuthology.orchestra.run.smithi053.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-11-16T16:25:47.596 INFO:teuthology.orchestra.run.smithi053.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-11-16T16:25:47.597 INFO:teuthology.orchestra.run.smithi053.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-11-16T16:25:47.597 INFO:teuthology.orchestra.run.smithi053.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-11-16T16:25:47.597 INFO:teuthology.orchestra.run.smithi053.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-11-16T16:25:47.597 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2023-11-16T16:25:47.597 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-11-16T16:25:47.597 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-11-16T16:25:47.597 INFO:teuthology.orchestra.run.smithi053.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-11-16T16:25:47.597 INFO:teuthology.orchestra.run.smithi053.stdout:Installing module profiles: 2023-11-16T16:25:47.597 INFO:teuthology.orchestra.run.smithi053.stdout: container-tools/common 2023-11-16T16:25:47.597 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-16T16:25:47.598 INFO:teuthology.orchestra.run.smithi053.stdout:Enabling module streams: 2023-11-16T16:25:47.598 INFO:teuthology.orchestra.run.smithi053.stdout: container-tools rhel8 2023-11-16T16:25:47.598 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-16T16:25:47.598 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2023-11-16T16:25:47.598 INFO:teuthology.orchestra.run.smithi053.stdout:============================================================================================================ 2023-11-16T16:25:47.598 INFO:teuthology.orchestra.run.smithi053.stdout:Install 10 Packages 2023-11-16T16:25:47.598 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-16T16:25:47.599 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 18 M 2023-11-16T16:25:47.599 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 65 M 2023-11-16T16:25:47.599 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2023-11-16T16:25:47.657 INFO:teuthology.orchestra.run.smithi026.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 2.1 MB/s | 213 kB 00:00 2023-11-16T16:25:47.737 INFO:teuthology.orchestra.run.smithi053.stdout:(1/10): python3-pytoml-0.1.14-5.git7dea353.el8. 180 kB/s | 25 kB 00:00 2023-11-16T16:25:47.762 INFO:teuthology.orchestra.run.smithi053.stdout:(2/10): yajl-2.1.0-10.el8.x86_64.rpm 250 kB/s | 41 kB 00:00 2023-11-16T16:25:47.774 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction test succeeded. 2023-11-16T16:25:47.778 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction 2023-11-16T16:25:47.787 INFO:teuthology.orchestra.run.smithi053.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 498 kB/s | 94 kB 00:00 2023-11-16T16:25:47.816 INFO:teuthology.orchestra.run.smithi026.stdout:(8/10): buildah-1.26.2-1.module+el8.6.0+16771+2 23 MB/s | 8.0 MB 00:00 2023-11-16T16:25:47.863 INFO:teuthology.orchestra.run.smithi053.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 393 kB/s | 49 kB 00:00 2023-11-16T16:25:47.899 INFO:teuthology.orchestra.run.smithi026.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 9.1 MB/s | 2.2 MB 00:00 2023-11-16T16:25:47.904 INFO:teuthology.orchestra.run.smithi053.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 3.6 MB/s | 524 kB 00:00 2023-11-16T16:25:47.983 INFO:teuthology.orchestra.run.smithi026.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 18 MB/s | 7.0 MB 00:00 2023-11-16T16:25:47.984 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2023-11-16T16:25:47.984 INFO:teuthology.orchestra.run.smithi026.stdout:Total 24 MB/s | 18 MB 00:00 2023-11-16T16:25:48.005 INFO:teuthology.orchestra.run.smithi053.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 1.0 MB/s | 149 kB 00:00 2023-11-16T16:25:48.030 INFO:teuthology.orchestra.run.smithi053.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 1.7 MB/s | 213 kB 00:00 2023-11-16T16:25:48.131 INFO:teuthology.orchestra.run.smithi053.stdout:(8/10): buildah-1.26.2-1.module+el8.6.0+16771+2 23 MB/s | 8.0 MB 00:00 2023-11-16T16:25:48.191 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2023-11-16T16:25:48.212 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2023-11-16T16:25:48.212 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2023-11-16T16:25:48.231 INFO:teuthology.orchestra.run.smithi053.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 11 MB/s | 2.2 MB 00:00 2023-11-16T16:25:48.322 INFO:teuthology.orchestra.run.smithi001.stdout: Preparing : 1/1 2023-11-16T16:25:48.347 INFO:teuthology.orchestra.run.smithi053.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 20 MB/s | 7.0 MB 00:00 2023-11-16T16:25:48.348 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2023-11-16T16:25:48.348 INFO:teuthology.orchestra.run.smithi053.stdout:Total 24 MB/s | 18 MB 00:00 2023-11-16T16:25:48.405 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2023-11-16T16:25:48.409 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2023-11-16T16:25:48.506 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-11-16T16:25:48.555 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2023-11-16T16:25:48.575 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2023-11-16T16:25:48.575 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2023-11-16T16:25:48.669 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-16T16:25:48.769 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2023-11-16T16:25:48.774 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2023-11-16T16:25:48.894 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-11-16T16:25:48.934 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2023-11-16T16:25:49.077 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-11-16T16:25:49.144 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-11-16T16:25:49.298 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2023-11-16T16:25:49.303 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-16T16:25:49.434 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-11-16T16:25:49.483 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-11-16T16:25:49.506 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-11-16T16:25:49.683 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-16T16:25:49.709 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-11-16T16:25:49.880 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-11-16T16:25:50.076 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-11-16T16:25:50.111 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-11-16T16:25:50.385 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-11-16T16:25:50.481 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-11-16T16:25:50.865 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-11-16T16:25:51.288 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-11-16T16:25:51.305 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-11-16T16:25:51.539 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-11-16T16:25:51.742 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-11-16T16:25:51.762 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-11-16T16:25:51.910 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-16T16:25:51.999 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-11-16T16:25:52.159 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-11-16T16:25:52.189 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-16T16:25:52.189 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-11-16T16:25:52.190 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-16T16:25:52.190 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-11-16T16:25:52.190 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-11-16T16:25:52.190 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-11-16T16:25:52.190 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-11-16T16:25:52.190 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-11-16T16:25:52.190 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-11-16T16:25:52.190 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-11-16T16:25:52.194 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-11-16T16:25:52.335 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-16T16:25:52.423 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-11-16T16:25:52.572 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-11-16T16:25:52.626 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-16T16:25:52.626 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-11-16T16:25:52.626 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-16T16:25:52.627 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-11-16T16:25:52.627 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-11-16T16:25:52.627 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-11-16T16:25:52.627 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-11-16T16:25:52.627 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-11-16T16:25:52.627 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-11-16T16:25:52.627 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-11-16T16:25:52.724 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-16T16:25:53.027 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-16T16:25:53.027 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-11-16T16:25:53.027 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-16T16:25:53.027 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-11-16T16:25:53.028 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-11-16T16:25:53.028 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-11-16T16:25:53.028 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-11-16T16:25:53.028 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-11-16T16:25:53.028 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-11-16T16:25:53.028 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-11-16T16:25:53.367 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-11-16T16:25:53.367 INFO:teuthology.orchestra.run.smithi001.stdout:Last metadata expiration check: 0:02:39 ago on Thu 16 Nov 2023 04:23:13 PM UTC. 2023-11-16T16:25:53.847 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-11-16T16:25:53.848 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:02:14 ago on Thu 16 Nov 2023 04:23:39 PM UTC. 2023-11-16T16:25:54.235 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-11-16T16:25:54.235 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:02:17 ago on Thu 16 Nov 2023 04:23:36 PM UTC. 2023-11-16T16:26:03.380 INFO:teuthology.orchestra.run.smithi001.stdout:Installed products updated. 2023-11-16T16:26:03.591 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:26:03.591 INFO:teuthology.orchestra.run.smithi001.stdout:Installed: 2023-11-16T16:26:03.591 INFO:teuthology.orchestra.run.smithi001.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-16T16:26:03.591 INFO:teuthology.orchestra.run.smithi001.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-11-16T16:26:03.591 INFO:teuthology.orchestra.run.smithi001.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-16T16:26:03.591 INFO:teuthology.orchestra.run.smithi001.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-11-16T16:26:03.592 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-11-16T16:26:03.592 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-11-16T16:26:03.592 INFO:teuthology.orchestra.run.smithi001.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-16T16:26:03.592 INFO:teuthology.orchestra.run.smithi001.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-11-16T16:26:03.592 INFO:teuthology.orchestra.run.smithi001.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-11-16T16:26:03.592 INFO:teuthology.orchestra.run.smithi001.stdout: yajl-2.1.0-10.el8.x86_64 2023-11-16T16:26:03.592 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:26:03.592 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2023-11-16T16:26:03.739 DEBUG:teuthology.parallel:result is None 2023-11-16T16:26:06.178 INFO:teuthology.orchestra.run.smithi026.stdout:Installed products updated. 2023-11-16T16:26:06.396 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-11-16T16:26:06.396 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2023-11-16T16:26:06.397 INFO:teuthology.orchestra.run.smithi026.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-16T16:26:06.397 INFO:teuthology.orchestra.run.smithi026.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-11-16T16:26:06.397 INFO:teuthology.orchestra.run.smithi026.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-16T16:26:06.397 INFO:teuthology.orchestra.run.smithi026.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-11-16T16:26:06.397 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-11-16T16:26:06.397 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-11-16T16:26:06.397 INFO:teuthology.orchestra.run.smithi026.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-16T16:26:06.397 INFO:teuthology.orchestra.run.smithi026.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-11-16T16:26:06.397 INFO:teuthology.orchestra.run.smithi026.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-11-16T16:26:06.397 INFO:teuthology.orchestra.run.smithi026.stdout: yajl-2.1.0-10.el8.x86_64 2023-11-16T16:26:06.397 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-11-16T16:26:06.397 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2023-11-16T16:26:06.535 DEBUG:teuthology.parallel:result is None 2023-11-16T16:26:06.593 INFO:teuthology.orchestra.run.smithi053.stdout:Installed products updated. 2023-11-16T16:26:06.815 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-16T16:26:06.815 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2023-11-16T16:26:06.815 INFO:teuthology.orchestra.run.smithi053.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-16T16:26:06.815 INFO:teuthology.orchestra.run.smithi053.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-11-16T16:26:06.815 INFO:teuthology.orchestra.run.smithi053.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-16T16:26:06.815 INFO:teuthology.orchestra.run.smithi053.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-11-16T16:26:06.815 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-11-16T16:26:06.815 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-11-16T16:26:06.815 INFO:teuthology.orchestra.run.smithi053.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-16T16:26:06.815 INFO:teuthology.orchestra.run.smithi053.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-11-16T16:26:06.816 INFO:teuthology.orchestra.run.smithi053.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-11-16T16:26:06.816 INFO:teuthology.orchestra.run.smithi053.stdout: yajl-2.1.0-10.el8.x86_64 2023-11-16T16:26:06.816 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-16T16:26:06.816 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2023-11-16T16:26:06.965 DEBUG:teuthology.parallel:result is None 2023-11-16T16:26:06.965 INFO:teuthology.run_tasks:Running task install... 2023-11-16T16:26:06.974 DEBUG:teuthology.task.install:project ceph 2023-11-16T16:26:06.974 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '5e4a743ad7fd1422ebd6c4dc950a21a303279207'}} 2023-11-16T16:26:06.975 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '5e4a743ad7fd1422ebd6c4dc950a21a303279207'} 2023-11-16T16:26:06.975 INFO:teuthology.task.install:Using flavor: default 2023-11-16T16:26:06.981 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-16T16:26:06.981 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-16T16:26:06.981 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5e4a743ad7fd1422ebd6c4dc950a21a303279207', 'tag': None, 'wait_for_package': False} 2023-11-16T16:26:06.982 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=5e4a743ad7fd1422ebd6c4dc950a21a303279207 2023-11-16T16:26:06.983 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5e4a743ad7fd1422ebd6c4dc950a21a303279207', 'tag': None, 'wait_for_package': False} 2023-11-16T16:26:06.984 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=5e4a743ad7fd1422ebd6c4dc950a21a303279207 2023-11-16T16:26:06.985 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5e4a743ad7fd1422ebd6c4dc950a21a303279207', 'tag': None, 'wait_for_package': False} 2023-11-16T16:26:06.985 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=5e4a743ad7fd1422ebd6c4dc950a21a303279207 2023-11-16T16:26:07.310 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-choffman-62793/5e4a743ad7fd1422ebd6c4dc950a21a303279207/centos/8/flavors/default/ 2023-11-16T16:26:07.311 INFO:teuthology.task.install.rpm:Package version is 18.0.0-3905.g5e4a743a 2023-11-16T16:26:07.316 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-choffman-62793/5e4a743ad7fd1422ebd6c4dc950a21a303279207/centos/8/flavors/default/ 2023-11-16T16:26:07.316 INFO:teuthology.task.install.rpm:Package version is 18.0.0-3905.g5e4a743a 2023-11-16T16:26:07.320 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-choffman-62793/5e4a743ad7fd1422ebd6c4dc950a21a303279207/centos/8/flavors/default/ 2023-11-16T16:26:07.320 INFO:teuthology.task.install.rpm:Package version is 18.0.0-3905.g5e4a743a 2023-11-16T16:26:07.419 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-choffman-62793/5e4a743ad7fd1422ebd6c4dc950a21a303279207/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-choffman-62793/5e4a743ad7fd1422ebd6c4dc950a21a303279207/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-choffman-62793/5e4a743ad7fd1422ebd6c4dc950a21a303279207/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-16T16:26:07.419 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2023-11-16T16:26:07.419 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-16T16:26:07.430 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-choffman-62793/5e4a743ad7fd1422ebd6c4dc950a21a303279207/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-choffman-62793/5e4a743ad7fd1422ebd6c4dc950a21a303279207/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-choffman-62793/5e4a743ad7fd1422ebd6c4dc950a21a303279207/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-16T16:26:07.430 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-16T16:26:07.430 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-16T16:26:07.432 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-choffman-62793/5e4a743ad7fd1422ebd6c4dc950a21a303279207/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-choffman-62793/5e4a743ad7fd1422ebd6c4dc950a21a303279207/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-choffman-62793/5e4a743ad7fd1422ebd6c4dc950a21a303279207/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-16T16:26:07.432 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-11-16T16:26:07.432 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-16T16:26:07.454 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-16T16:26:07.454 DEBUG:teuthology.orchestra.run.smithi001:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5e4a743ad7fd1422ebd6c4dc950a21a303279207/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-16T16:26:07.463 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-16T16:26:07.463 DEBUG:teuthology.orchestra.run.smithi053:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5e4a743ad7fd1422ebd6c4dc950a21a303279207/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-16T16:26:07.466 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-16T16:26:07.466 DEBUG:teuthology.orchestra.run.smithi026:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5e4a743ad7fd1422ebd6c4dc950a21a303279207/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-16T16:26:07.526 DEBUG:teuthology.orchestra.run.smithi001:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-16T16:26:07.534 DEBUG:teuthology.orchestra.run.smithi053:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-16T16:26:07.540 DEBUG:teuthology.orchestra.run.smithi026:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-16T16:26:07.609 DEBUG:teuthology.orchestra.run.smithi001:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-16T16:26:07.614 DEBUG:teuthology.orchestra.run.smithi053:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-16T16:26:07.625 DEBUG:teuthology.orchestra.run.smithi026:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-16T16:26:07.677 INFO:teuthology.orchestra.run.smithi001.stdout:check_obsoletes = 1 2023-11-16T16:26:07.678 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum clean all 2023-11-16T16:26:07.683 INFO:teuthology.orchestra.run.smithi053.stdout:check_obsoletes = 1 2023-11-16T16:26:07.684 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum clean all 2023-11-16T16:26:07.695 INFO:teuthology.orchestra.run.smithi026.stdout:check_obsoletes = 1 2023-11-16T16:26:07.697 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum clean all 2023-11-16T16:26:08.074 INFO:teuthology.orchestra.run.smithi001.stdout:Updating Subscription Management repositories. 2023-11-16T16:26:08.078 INFO:teuthology.orchestra.run.smithi053.stdout:Updating Subscription Management repositories. 2023-11-16T16:26:08.093 INFO:teuthology.orchestra.run.smithi026.stdout:Updating Subscription Management repositories. 2023-11-16T16:26:08.909 INFO:teuthology.orchestra.run.smithi001.stdout:44 files removed 2023-11-16T16:26:08.926 INFO:teuthology.orchestra.run.smithi053.stdout:44 files removed 2023-11-16T16:26:08.962 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-16T16:26:08.976 INFO:teuthology.orchestra.run.smithi026.stdout:44 files removed 2023-11-16T16:26:08.982 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-16T16:26:09.031 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-16T16:26:09.316 INFO:teuthology.orchestra.run.smithi001.stdout:Updating Subscription Management repositories. 2023-11-16T16:26:09.336 INFO:teuthology.orchestra.run.smithi053.stdout:Updating Subscription Management repositories. 2023-11-16T16:26:09.387 INFO:teuthology.orchestra.run.smithi026.stdout:Updating Subscription Management repositories. 2023-11-16T16:26:10.933 INFO:teuthology.orchestra.run.smithi001.stdout:ceph packages for x86_64 206 kB/s | 80 kB 00:00 2023-11-16T16:26:10.974 INFO:teuthology.orchestra.run.smithi053.stdout:ceph packages for x86_64 183 kB/s | 80 kB 00:00 2023-11-16T16:26:11.002 INFO:teuthology.orchestra.run.smithi026.stdout:ceph packages for x86_64 169 kB/s | 80 kB 00:00 2023-11-16T16:26:11.311 INFO:teuthology.orchestra.run.smithi026.stdout:ceph noarch packages 34 kB/s | 9.4 kB 00:00 2023-11-16T16:26:11.342 INFO:teuthology.orchestra.run.smithi001.stdout:ceph noarch packages 25 kB/s | 9.4 kB 00:00 2023-11-16T16:26:11.358 INFO:teuthology.orchestra.run.smithi053.stdout:ceph noarch packages 27 kB/s | 9.4 kB 00:00 2023-11-16T16:26:11.620 INFO:teuthology.orchestra.run.smithi026.stdout:ceph source packages 5.2 kB/s | 1.5 kB 00:00 2023-11-16T16:26:11.701 INFO:teuthology.orchestra.run.smithi001.stdout:ceph source packages 4.4 kB/s | 1.5 kB 00:00 2023-11-16T16:26:11.734 INFO:teuthology.orchestra.run.smithi053.stdout:ceph source packages 4.2 kB/s | 1.5 kB 00:00 2023-11-16T16:26:11.788 INFO:teuthology.orchestra.run.smithi026.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2023-11-16T16:26:11.827 INFO:teuthology.orchestra.run.smithi001.stdout:Copr repo for python3-asyncssh owned by ceph 34 kB/s | 3.5 kB 00:00 2023-11-16T16:26:11.859 INFO:teuthology.orchestra.run.smithi053.stdout:Copr repo for python3-asyncssh owned by ceph 34 kB/s | 3.5 kB 00:00 2023-11-16T16:26:12.646 INFO:teuthology.orchestra.run.smithi001.stdout:Extra Packages for Enterprise Linux 20 MB/s | 16 MB 00:00 2023-11-16T16:26:13.338 INFO:teuthology.orchestra.run.smithi053.stdout:Extra Packages for Enterprise Linux 11 MB/s | 16 MB 00:01 2023-11-16T16:26:15.021 INFO:teuthology.orchestra.run.smithi026.stdout:Extra Packages for Enterprise Linux 5.0 MB/s | 16 MB 00:03 2023-11-16T16:26:16.386 INFO:teuthology.orchestra.run.smithi001.stdout:lab-extras 479 kB/s | 24 kB 00:00 2023-11-16T16:26:17.060 INFO:teuthology.orchestra.run.smithi053.stdout:lab-extras 419 kB/s | 24 kB 00:00 2023-11-16T16:26:17.872 INFO:teuthology.orchestra.run.smithi001.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 34 MB/s | 47 MB 00:01 2023-11-16T16:26:18.254 INFO:teuthology.orchestra.run.smithi053.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 42 MB/s | 47 MB 00:01 2023-11-16T16:26:18.721 INFO:teuthology.orchestra.run.smithi026.stdout:lab-extras 466 kB/s | 24 kB 00:00 2023-11-16T16:26:19.973 INFO:teuthology.orchestra.run.smithi026.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 40 MB/s | 47 MB 00:01 2023-11-16T16:26:26.490 INFO:teuthology.orchestra.run.smithi001.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 47 MB/s | 53 MB 00:01 2023-11-16T16:26:26.873 INFO:teuthology.orchestra.run.smithi053.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 47 MB/s | 53 MB 00:01 2023-11-16T16:26:28.708 INFO:teuthology.orchestra.run.smithi026.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 44 MB/s | 53 MB 00:01 2023-11-16T16:26:38.464 INFO:teuthology.orchestra.run.smithi001.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-16T16:26:38.464 INFO:teuthology.orchestra.run.smithi001.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-16T16:26:38.479 INFO:teuthology.orchestra.run.smithi001.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-16T16:26:38.480 INFO:teuthology.orchestra.run.smithi001.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-16T16:26:38.481 INFO:teuthology.orchestra.run.smithi001.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-16T16:26:38.482 INFO:teuthology.orchestra.run.smithi001.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-16T16:26:38.484 INFO:teuthology.orchestra.run.smithi001.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-16T16:26:38.487 INFO:teuthology.orchestra.run.smithi001.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-16T16:26:38.489 INFO:teuthology.orchestra.run.smithi001.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-16T16:26:38.632 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2023-11-16T16:26:38.645 INFO:teuthology.orchestra.run.smithi001.stdout:=========================================================================================================================== 2023-11-16T16:26:38.645 INFO:teuthology.orchestra.run.smithi001.stdout: Package Arch Version Repository Size 2023-11-16T16:26:38.645 INFO:teuthology.orchestra.run.smithi001.stdout:=========================================================================================================================== 2023-11-16T16:26:38.645 INFO:teuthology.orchestra.run.smithi001.stdout:Installing: 2023-11-16T16:26:38.645 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-16T16:26:38.645 INFO:teuthology.orchestra.run.smithi001.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-16T16:26:38.645 INFO:teuthology.orchestra.run.smithi001.stdout: ceph x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 6.4 k 2023-11-16T16:26:38.645 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-base x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 5.3 M 2023-11-16T16:26:38.646 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-fuse x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 847 k 2023-11-16T16:26:38.646 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 165 k 2023-11-16T16:26:38.646 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 1.5 M 2023-11-16T16:26:38.646 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-cephadm noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 127 k 2023-11-16T16:26:38.646 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-dashboard noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 1.7 M 2023-11-16T16:26:38.646 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 7.4 M 2023-11-16T16:26:38.646 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-rook noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 47 k 2023-11-16T16:26:38.646 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-radosgw x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 13 M 2023-11-16T16:26:38.646 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-test x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 41 M 2023-11-16T16:26:38.646 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-volume noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 265 k 2023-11-16T16:26:38.646 INFO:teuthology.orchestra.run.smithi001.stdout: cephadm noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 83 k 2023-11-16T16:26:38.646 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-mirror x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 258 k 2023-11-16T16:26:38.647 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-top noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 18 k 2023-11-16T16:26:38.647 INFO:teuthology.orchestra.run.smithi001.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-16T16:26:38.647 INFO:teuthology.orchestra.run.smithi001.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-16T16:26:38.647 INFO:teuthology.orchestra.run.smithi001.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-16T16:26:38.647 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs-devel x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 31 k 2023-11-16T16:26:38.647 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs2 x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 771 k 2023-11-16T16:26:38.647 INFO:teuthology.orchestra.run.smithi001.stdout: librados-devel x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 139 k 2023-11-16T16:26:38.647 INFO:teuthology.orchestra.run.smithi001.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-16T16:26:38.647 INFO:teuthology.orchestra.run.smithi001.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-16T16:26:38.647 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-16T16:26:38.647 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-16T16:26:38.647 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-16T16:26:38.648 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-16T16:26:38.648 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cephfs x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 197 k 2023-11-16T16:26:38.648 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rados x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 371 k 2023-11-16T16:26:38.648 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rbd x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 365 k 2023-11-16T16:26:38.648 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rgw x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 107 k 2023-11-16T16:26:38.648 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-fuse x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 94 k 2023-11-16T16:26:38.648 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-mirror x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 3.4 M 2023-11-16T16:26:38.648 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-nbd x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 180 k 2023-11-16T16:26:38.648 INFO:teuthology.orchestra.run.smithi001.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-16T16:26:38.648 INFO:teuthology.orchestra.run.smithi001.stdout:Upgrading: 2023-11-16T16:26:38.648 INFO:teuthology.orchestra.run.smithi001.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-16T16:26:38.648 INFO:teuthology.orchestra.run.smithi001.stdout: librados2 x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 3.7 M 2023-11-16T16:26:38.649 INFO:teuthology.orchestra.run.smithi001.stdout: librbd1 x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 3.6 M 2023-11-16T16:26:38.649 INFO:teuthology.orchestra.run.smithi001.stdout:Installing dependencies: 2023-11-16T16:26:38.649 INFO:teuthology.orchestra.run.smithi001.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-16T16:26:38.649 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-common x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 22 M 2023-11-16T16:26:38.649 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-grafana-dashboards noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 23 k 2023-11-16T16:26:38.649 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mds x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 2.3 M 2023-11-16T16:26:38.649 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-modules-core noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 240 k 2023-11-16T16:26:38.649 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mon x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 4.6 M 2023-11-16T16:26:38.649 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-osd x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 18 M 2023-11-16T16:26:38.649 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-prometheus-alerts noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 15 k 2023-11-16T16:26:38.649 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-selinux x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 24 k 2023-11-16T16:26:38.649 INFO:teuthology.orchestra.run.smithi001.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-16T16:26:38.649 INFO:teuthology.orchestra.run.smithi001.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-16T16:26:38.650 INFO:teuthology.orchestra.run.smithi001.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-16T16:26:38.650 INFO:teuthology.orchestra.run.smithi001.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-16T16:26:38.650 INFO:teuthology.orchestra.run.smithi001.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-16T16:26:38.650 INFO:teuthology.orchestra.run.smithi001.stdout: libcephsqlite x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 137 k 2023-11-16T16:26:38.650 INFO:teuthology.orchestra.run.smithi001.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-16T16:26:38.650 INFO:teuthology.orchestra.run.smithi001.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-16T16:26:38.650 INFO:teuthology.orchestra.run.smithi001.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-16T16:26:38.650 INFO:teuthology.orchestra.run.smithi001.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-16T16:26:38.650 INFO:teuthology.orchestra.run.smithi001.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-16T16:26:38.650 INFO:teuthology.orchestra.run.smithi001.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-16T16:26:38.650 INFO:teuthology.orchestra.run.smithi001.stdout: libradosstriper1 x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 473 k 2023-11-16T16:26:38.651 INFO:teuthology.orchestra.run.smithi001.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-16T16:26:38.651 INFO:teuthology.orchestra.run.smithi001.stdout: librgw2 x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 6.5 M 2023-11-16T16:26:38.651 INFO:teuthology.orchestra.run.smithi001.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-16T16:26:38.651 INFO:teuthology.orchestra.run.smithi001.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-16T16:26:38.651 INFO:teuthology.orchestra.run.smithi001.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-16T16:26:38.651 INFO:teuthology.orchestra.run.smithi001.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-16T16:26:38.651 INFO:teuthology.orchestra.run.smithi001.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-16T16:26:38.651 INFO:teuthology.orchestra.run.smithi001.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-16T16:26:38.651 INFO:teuthology.orchestra.run.smithi001.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-16T16:26:38.651 INFO:teuthology.orchestra.run.smithi001.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-16T16:26:38.651 INFO:teuthology.orchestra.run.smithi001.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-16T16:26:38.651 INFO:teuthology.orchestra.run.smithi001.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-16T16:26:38.652 INFO:teuthology.orchestra.run.smithi001.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-16T16:26:38.652 INFO:teuthology.orchestra.run.smithi001.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-16T16:26:38.652 INFO:teuthology.orchestra.run.smithi001.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-16T16:26:38.652 INFO:teuthology.orchestra.run.smithi001.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-16T16:26:38.652 INFO:teuthology.orchestra.run.smithi001.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-16T16:26:38.652 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-16T16:26:38.652 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-argparse x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 45 k 2023-11-16T16:26:38.652 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-common x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 119 k 2023-11-16T16:26:38.652 INFO:teuthology.orchestra.run.smithi001.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-16T16:26:38.652 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-16T16:26:38.652 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-16T16:26:38.652 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-16T16:26:38.653 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-16T16:26:38.653 INFO:teuthology.orchestra.run.smithi001.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-16T16:26:38.653 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-16T16:26:38.653 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-16T16:26:38.653 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-16T16:26:38.653 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-16T16:26:38.653 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-16T16:26:38.653 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-16T16:26:38.653 INFO:teuthology.orchestra.run.smithi001.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-16T16:26:38.653 INFO:teuthology.orchestra.run.smithi001.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-16T16:26:38.653 INFO:teuthology.orchestra.run.smithi001.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-16T16:26:38.653 INFO:teuthology.orchestra.run.smithi001.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-16T16:26:38.654 INFO:teuthology.orchestra.run.smithi001.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-16T16:26:38.654 INFO:teuthology.orchestra.run.smithi001.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-16T16:26:38.654 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-16T16:26:38.654 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-16T16:26:38.654 INFO:teuthology.orchestra.run.smithi001.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-16T16:26:38.654 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-16T16:26:38.654 INFO:teuthology.orchestra.run.smithi001.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-16T16:26:38.654 INFO:teuthology.orchestra.run.smithi001.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-16T16:26:38.654 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-16T16:26:38.654 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-16T16:26:38.654 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-16T16:26:38.655 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-16T16:26:38.655 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-16T16:26:38.655 INFO:teuthology.orchestra.run.smithi001.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-16T16:26:38.655 INFO:teuthology.orchestra.run.smithi001.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-16T16:26:38.655 INFO:teuthology.orchestra.run.smithi001.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-16T16:26:38.655 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-16T16:26:38.655 INFO:teuthology.orchestra.run.smithi001.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-16T16:26:38.655 INFO:teuthology.orchestra.run.smithi001.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-16T16:26:38.655 INFO:teuthology.orchestra.run.smithi001.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-16T16:26:38.655 INFO:teuthology.orchestra.run.smithi001.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-16T16:26:38.655 INFO:teuthology.orchestra.run.smithi001.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-16T16:26:38.655 INFO:teuthology.orchestra.run.smithi001.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-16T16:26:38.655 INFO:teuthology.orchestra.run.smithi001.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-16T16:26:38.656 INFO:teuthology.orchestra.run.smithi001.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-16T16:26:38.656 INFO:teuthology.orchestra.run.smithi001.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-16T16:26:38.656 INFO:teuthology.orchestra.run.smithi001.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-16T16:26:38.656 INFO:teuthology.orchestra.run.smithi001.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-16T16:26:38.656 INFO:teuthology.orchestra.run.smithi001.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-16T16:26:38.656 INFO:teuthology.orchestra.run.smithi001.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-16T16:26:38.656 INFO:teuthology.orchestra.run.smithi001.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-16T16:26:38.656 INFO:teuthology.orchestra.run.smithi001.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-16T16:26:38.656 INFO:teuthology.orchestra.run.smithi001.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-16T16:26:38.656 INFO:teuthology.orchestra.run.smithi001.stdout:Enabling module streams: 2023-11-16T16:26:38.656 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql 10 2023-11-16T16:26:38.656 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:26:38.656 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction Summary 2023-11-16T16:26:38.656 INFO:teuthology.orchestra.run.smithi001.stdout:=========================================================================================================================== 2023-11-16T16:26:38.657 INFO:teuthology.orchestra.run.smithi001.stdout:Install 128 Packages 2023-11-16T16:26:38.657 INFO:teuthology.orchestra.run.smithi001.stdout:Upgrade 3 Packages 2023-11-16T16:26:38.657 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:26:38.657 INFO:teuthology.orchestra.run.smithi001.stdout:Total download size: 197 M 2023-11-16T16:26:38.657 INFO:teuthology.orchestra.run.smithi001.stdout:Downloading Packages: 2023-11-16T16:26:38.828 INFO:teuthology.orchestra.run.smithi053.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-16T16:26:38.829 INFO:teuthology.orchestra.run.smithi053.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-16T16:26:38.844 INFO:teuthology.orchestra.run.smithi053.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-16T16:26:38.845 INFO:teuthology.orchestra.run.smithi053.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-16T16:26:38.846 INFO:teuthology.orchestra.run.smithi053.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-16T16:26:38.847 INFO:teuthology.orchestra.run.smithi053.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-16T16:26:38.849 INFO:teuthology.orchestra.run.smithi053.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-16T16:26:38.852 INFO:teuthology.orchestra.run.smithi053.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-16T16:26:38.854 INFO:teuthology.orchestra.run.smithi053.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-16T16:26:38.924 INFO:teuthology.orchestra.run.smithi001.stdout:(1/131): ceph-18.0.0-3905.g5e4a743a.el8.x86_64. 29 kB/s | 6.4 kB 00:00 2023-11-16T16:26:38.997 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2023-11-16T16:26:39.010 INFO:teuthology.orchestra.run.smithi053.stdout:=========================================================================================================================== 2023-11-16T16:26:39.010 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2023-11-16T16:26:39.010 INFO:teuthology.orchestra.run.smithi053.stdout:=========================================================================================================================== 2023-11-16T16:26:39.011 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2023-11-16T16:26:39.011 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-16T16:26:39.011 INFO:teuthology.orchestra.run.smithi053.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-16T16:26:39.011 INFO:teuthology.orchestra.run.smithi053.stdout: ceph x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 6.4 k 2023-11-16T16:26:39.011 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 5.3 M 2023-11-16T16:26:39.011 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-fuse x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 847 k 2023-11-16T16:26:39.011 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 165 k 2023-11-16T16:26:39.011 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 1.5 M 2023-11-16T16:26:39.011 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-cephadm noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 127 k 2023-11-16T16:26:39.011 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 1.7 M 2023-11-16T16:26:39.011 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 7.4 M 2023-11-16T16:26:39.012 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 47 k 2023-11-16T16:26:39.012 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-radosgw x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 13 M 2023-11-16T16:26:39.012 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-test x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 41 M 2023-11-16T16:26:39.012 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-volume noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 265 k 2023-11-16T16:26:39.012 INFO:teuthology.orchestra.run.smithi053.stdout: cephadm noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 83 k 2023-11-16T16:26:39.012 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-mirror x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 258 k 2023-11-16T16:26:39.012 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-top noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 18 k 2023-11-16T16:26:39.012 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-16T16:26:39.012 INFO:teuthology.orchestra.run.smithi053.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-16T16:26:39.013 INFO:teuthology.orchestra.run.smithi053.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-16T16:26:39.013 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-devel x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 31 k 2023-11-16T16:26:39.013 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs2 x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 771 k 2023-11-16T16:26:39.013 INFO:teuthology.orchestra.run.smithi053.stdout: librados-devel x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 139 k 2023-11-16T16:26:39.013 INFO:teuthology.orchestra.run.smithi053.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-16T16:26:39.013 INFO:teuthology.orchestra.run.smithi053.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-16T16:26:39.013 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-16T16:26:39.013 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-16T16:26:39.013 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-16T16:26:39.013 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-16T16:26:39.013 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cephfs x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 197 k 2023-11-16T16:26:39.013 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rados x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 371 k 2023-11-16T16:26:39.014 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rbd x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 365 k 2023-11-16T16:26:39.014 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rgw x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 107 k 2023-11-16T16:26:39.014 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-fuse x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 94 k 2023-11-16T16:26:39.014 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-mirror x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 3.4 M 2023-11-16T16:26:39.014 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-nbd x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 180 k 2023-11-16T16:26:39.014 INFO:teuthology.orchestra.run.smithi053.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-16T16:26:39.014 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrading: 2023-11-16T16:26:39.014 INFO:teuthology.orchestra.run.smithi053.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-16T16:26:39.014 INFO:teuthology.orchestra.run.smithi053.stdout: librados2 x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 3.7 M 2023-11-16T16:26:39.014 INFO:teuthology.orchestra.run.smithi053.stdout: librbd1 x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 3.6 M 2023-11-16T16:26:39.015 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2023-11-16T16:26:39.015 INFO:teuthology.orchestra.run.smithi053.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-16T16:26:39.015 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-common x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 22 M 2023-11-16T16:26:39.015 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-grafana-dashboards noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 23 k 2023-11-16T16:26:39.015 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mds x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 2.3 M 2023-11-16T16:26:39.015 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 240 k 2023-11-16T16:26:39.015 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mon x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 4.6 M 2023-11-16T16:26:39.015 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-osd x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 18 M 2023-11-16T16:26:39.015 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-prometheus-alerts noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 15 k 2023-11-16T16:26:39.015 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-selinux x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 24 k 2023-11-16T16:26:39.015 INFO:teuthology.orchestra.run.smithi053.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-16T16:26:39.016 INFO:teuthology.orchestra.run.smithi053.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-16T16:26:39.016 INFO:teuthology.orchestra.run.smithi053.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-16T16:26:39.016 INFO:teuthology.orchestra.run.smithi053.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-16T16:26:39.016 INFO:teuthology.orchestra.run.smithi053.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-16T16:26:39.016 INFO:teuthology.orchestra.run.smithi053.stdout: libcephsqlite x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 137 k 2023-11-16T16:26:39.016 INFO:teuthology.orchestra.run.smithi053.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-16T16:26:39.016 INFO:teuthology.orchestra.run.smithi053.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-16T16:26:39.016 INFO:teuthology.orchestra.run.smithi053.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-16T16:26:39.016 INFO:teuthology.orchestra.run.smithi053.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-16T16:26:39.016 INFO:teuthology.orchestra.run.smithi053.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-16T16:26:39.017 INFO:teuthology.orchestra.run.smithi053.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-16T16:26:39.017 INFO:teuthology.orchestra.run.smithi053.stdout: libradosstriper1 x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 473 k 2023-11-16T16:26:39.017 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-16T16:26:39.017 INFO:teuthology.orchestra.run.smithi053.stdout: librgw2 x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 6.5 M 2023-11-16T16:26:39.017 INFO:teuthology.orchestra.run.smithi053.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-16T16:26:39.017 INFO:teuthology.orchestra.run.smithi053.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-16T16:26:39.017 INFO:teuthology.orchestra.run.smithi053.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-16T16:26:39.017 INFO:teuthology.orchestra.run.smithi053.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-16T16:26:39.017 INFO:teuthology.orchestra.run.smithi053.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-16T16:26:39.017 INFO:teuthology.orchestra.run.smithi053.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-16T16:26:39.017 INFO:teuthology.orchestra.run.smithi053.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-16T16:26:39.018 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-16T16:26:39.018 INFO:teuthology.orchestra.run.smithi053.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-16T16:26:39.018 INFO:teuthology.orchestra.run.smithi053.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-16T16:26:39.018 INFO:teuthology.orchestra.run.smithi053.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-16T16:26:39.018 INFO:teuthology.orchestra.run.smithi053.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-16T16:26:39.018 INFO:teuthology.orchestra.run.smithi053.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-16T16:26:39.018 INFO:teuthology.orchestra.run.smithi053.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-16T16:26:39.018 INFO:teuthology.orchestra.run.smithi053.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-16T16:26:39.018 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-16T16:26:39.018 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 45 k 2023-11-16T16:26:39.019 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-common x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 119 k 2023-11-16T16:26:39.019 INFO:teuthology.orchestra.run.smithi053.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-16T16:26:39.019 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-16T16:26:39.019 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-16T16:26:39.019 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-16T16:26:39.019 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-16T16:26:39.019 INFO:teuthology.orchestra.run.smithi053.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-16T16:26:39.019 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-16T16:26:39.020 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-16T16:26:39.020 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-16T16:26:39.020 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-16T16:26:39.020 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-16T16:26:39.020 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-16T16:26:39.020 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-16T16:26:39.020 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-16T16:26:39.020 INFO:teuthology.orchestra.run.smithi053.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-16T16:26:39.021 INFO:teuthology.orchestra.run.smithi053.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-16T16:26:39.021 INFO:teuthology.orchestra.run.smithi053.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-16T16:26:39.021 INFO:teuthology.orchestra.run.smithi053.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-16T16:26:39.021 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-16T16:26:39.021 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-16T16:26:39.021 INFO:teuthology.orchestra.run.smithi053.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-16T16:26:39.021 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-16T16:26:39.021 INFO:teuthology.orchestra.run.smithi053.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-16T16:26:39.022 INFO:teuthology.orchestra.run.smithi053.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-16T16:26:39.022 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-16T16:26:39.022 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-16T16:26:39.022 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-16T16:26:39.022 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-16T16:26:39.022 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-16T16:26:39.022 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-16T16:26:39.022 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-16T16:26:39.022 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-16T16:26:39.023 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-16T16:26:39.023 INFO:teuthology.orchestra.run.smithi053.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-16T16:26:39.023 INFO:teuthology.orchestra.run.smithi053.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-16T16:26:39.023 INFO:teuthology.orchestra.run.smithi053.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-16T16:26:39.023 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-16T16:26:39.024 INFO:teuthology.orchestra.run.smithi053.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-16T16:26:39.024 INFO:teuthology.orchestra.run.smithi053.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-16T16:26:39.024 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-16T16:26:39.024 INFO:teuthology.orchestra.run.smithi053.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-16T16:26:39.024 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-16T16:26:39.025 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-16T16:26:39.025 INFO:teuthology.orchestra.run.smithi053.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-16T16:26:39.025 INFO:teuthology.orchestra.run.smithi053.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-16T16:26:39.025 INFO:teuthology.orchestra.run.smithi053.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-16T16:26:39.025 INFO:teuthology.orchestra.run.smithi053.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-16T16:26:39.025 INFO:teuthology.orchestra.run.smithi053.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-16T16:26:39.025 INFO:teuthology.orchestra.run.smithi053.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-16T16:26:39.025 INFO:teuthology.orchestra.run.smithi053.stdout:Enabling module streams: 2023-11-16T16:26:39.026 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql 10 2023-11-16T16:26:39.026 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-16T16:26:39.026 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2023-11-16T16:26:39.026 INFO:teuthology.orchestra.run.smithi053.stdout:=========================================================================================================================== 2023-11-16T16:26:39.026 INFO:teuthology.orchestra.run.smithi053.stdout:Install 128 Packages 2023-11-16T16:26:39.026 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrade 3 Packages 2023-11-16T16:26:39.026 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-16T16:26:39.027 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 197 M 2023-11-16T16:26:39.027 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2023-11-16T16:26:39.315 INFO:teuthology.orchestra.run.smithi001.stdout:(2/131): ceph-base-18.0.0-3905.g5e4a743a.el8.x8 8.7 MB/s | 5.3 MB 00:00 2023-11-16T16:26:39.339 INFO:teuthology.orchestra.run.smithi053.stdout:(1/131): ceph-18.0.0-3905.g5e4a743a.el8.x86_64. 43 kB/s | 6.4 kB 00:00 2023-11-16T16:26:39.424 INFO:teuthology.orchestra.run.smithi001.stdout:(3/131): ceph-immutable-object-cache-18.0.0-390 1.5 MB/s | 165 kB 00:00 2023-11-16T16:26:39.716 INFO:teuthology.orchestra.run.smithi001.stdout:(4/131): ceph-mds-18.0.0-3905.g5e4a743a.el8.x86 7.8 MB/s | 2.3 MB 00:00 2023-11-16T16:26:39.916 INFO:teuthology.orchestra.run.smithi001.stdout:(5/131): ceph-mgr-18.0.0-3905.g5e4a743a.el8.x86 7.7 MB/s | 1.5 MB 00:00 2023-11-16T16:26:39.949 INFO:teuthology.orchestra.run.smithi001.stdout:(6/131): ceph-fuse-18.0.0-3905.g5e4a743a.el8.x8 826 kB/s | 847 kB 00:01 2023-11-16T16:26:40.006 INFO:teuthology.orchestra.run.smithi053.stdout:(2/131): ceph-fuse-18.0.0-3905.g5e4a743a.el8.x8 1.2 MB/s | 847 kB 00:00 2023-11-16T16:26:40.198 INFO:teuthology.orchestra.run.smithi053.stdout:(3/131): ceph-immutable-object-cache-18.0.0-390 863 kB/s | 165 kB 00:00 2023-11-16T16:26:40.367 INFO:teuthology.orchestra.run.smithi001.stdout:(7/131): ceph-mon-18.0.0-3905.g5e4a743a.el8.x86 10 MB/s | 4.6 MB 00:00 2023-11-16T16:26:40.592 INFO:teuthology.orchestra.run.smithi001.stdout:(8/131): ceph-common-18.0.0-3905.g5e4a743a.el8. 11 MB/s | 22 MB 00:01 2023-11-16T16:26:40.625 INFO:teuthology.orchestra.run.smithi026.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-16T16:26:40.626 INFO:teuthology.orchestra.run.smithi026.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-16T16:26:40.642 INFO:teuthology.orchestra.run.smithi026.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-16T16:26:40.643 INFO:teuthology.orchestra.run.smithi026.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-16T16:26:40.644 INFO:teuthology.orchestra.run.smithi026.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-16T16:26:40.645 INFO:teuthology.orchestra.run.smithi026.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-16T16:26:40.648 INFO:teuthology.orchestra.run.smithi026.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-16T16:26:40.651 INFO:teuthology.orchestra.run.smithi026.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-16T16:26:40.653 INFO:teuthology.orchestra.run.smithi026.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-16T16:26:40.798 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2023-11-16T16:26:40.806 INFO:teuthology.orchestra.run.smithi053.stdout:(4/131): ceph-mds-18.0.0-3905.g5e4a743a.el8.x86 3.8 MB/s | 2.3 MB 00:00 2023-11-16T16:26:40.811 INFO:teuthology.orchestra.run.smithi026.stdout:=========================================================================================================================== 2023-11-16T16:26:40.811 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repository Size 2023-11-16T16:26:40.811 INFO:teuthology.orchestra.run.smithi026.stdout:=========================================================================================================================== 2023-11-16T16:26:40.811 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2023-11-16T16:26:40.812 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-16T16:26:40.812 INFO:teuthology.orchestra.run.smithi026.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-16T16:26:40.812 INFO:teuthology.orchestra.run.smithi026.stdout: ceph x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 6.4 k 2023-11-16T16:26:40.812 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-base x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 5.3 M 2023-11-16T16:26:40.812 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-fuse x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 847 k 2023-11-16T16:26:40.812 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 165 k 2023-11-16T16:26:40.812 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 1.5 M 2023-11-16T16:26:40.812 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-cephadm noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 127 k 2023-11-16T16:26:40.812 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-dashboard noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 1.7 M 2023-11-16T16:26:40.812 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 7.4 M 2023-11-16T16:26:40.812 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-rook noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 47 k 2023-11-16T16:26:40.812 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-radosgw x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 13 M 2023-11-16T16:26:40.812 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-test x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 41 M 2023-11-16T16:26:40.813 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-volume noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 265 k 2023-11-16T16:26:40.813 INFO:teuthology.orchestra.run.smithi026.stdout: cephadm noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 83 k 2023-11-16T16:26:40.813 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-mirror x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 258 k 2023-11-16T16:26:40.813 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-top noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 18 k 2023-11-16T16:26:40.813 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-16T16:26:40.813 INFO:teuthology.orchestra.run.smithi026.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-16T16:26:40.813 INFO:teuthology.orchestra.run.smithi026.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-16T16:26:40.813 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs-devel x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 31 k 2023-11-16T16:26:40.813 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs2 x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 771 k 2023-11-16T16:26:40.813 INFO:teuthology.orchestra.run.smithi026.stdout: librados-devel x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 139 k 2023-11-16T16:26:40.813 INFO:teuthology.orchestra.run.smithi026.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-16T16:26:40.813 INFO:teuthology.orchestra.run.smithi026.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-16T16:26:40.813 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-16T16:26:40.814 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-16T16:26:40.814 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-16T16:26:40.814 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-16T16:26:40.814 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cephfs x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 197 k 2023-11-16T16:26:40.814 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rados x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 371 k 2023-11-16T16:26:40.814 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rbd x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 365 k 2023-11-16T16:26:40.814 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rgw x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 107 k 2023-11-16T16:26:40.814 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-fuse x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 94 k 2023-11-16T16:26:40.814 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-mirror x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 3.4 M 2023-11-16T16:26:40.814 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-nbd x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 180 k 2023-11-16T16:26:40.814 INFO:teuthology.orchestra.run.smithi026.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-16T16:26:40.814 INFO:teuthology.orchestra.run.smithi026.stdout:Upgrading: 2023-11-16T16:26:40.814 INFO:teuthology.orchestra.run.smithi026.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-16T16:26:40.815 INFO:teuthology.orchestra.run.smithi026.stdout: librados2 x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 3.7 M 2023-11-16T16:26:40.815 INFO:teuthology.orchestra.run.smithi026.stdout: librbd1 x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 3.6 M 2023-11-16T16:26:40.815 INFO:teuthology.orchestra.run.smithi026.stdout:Installing dependencies: 2023-11-16T16:26:40.815 INFO:teuthology.orchestra.run.smithi026.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-16T16:26:40.815 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-common x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 22 M 2023-11-16T16:26:40.815 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-grafana-dashboards noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 23 k 2023-11-16T16:26:40.815 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mds x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 2.3 M 2023-11-16T16:26:40.815 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-modules-core noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 240 k 2023-11-16T16:26:40.815 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mon x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 4.6 M 2023-11-16T16:26:40.815 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-osd x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 18 M 2023-11-16T16:26:40.815 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-prometheus-alerts noarch 2:18.0.0-3905.g5e4a743a.el8 ceph-noarch 15 k 2023-11-16T16:26:40.815 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-selinux x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 24 k 2023-11-16T16:26:40.816 INFO:teuthology.orchestra.run.smithi026.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-16T16:26:40.816 INFO:teuthology.orchestra.run.smithi026.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-16T16:26:40.816 INFO:teuthology.orchestra.run.smithi026.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-16T16:26:40.816 INFO:teuthology.orchestra.run.smithi026.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-16T16:26:40.816 INFO:teuthology.orchestra.run.smithi026.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-16T16:26:40.816 INFO:teuthology.orchestra.run.smithi026.stdout: libcephsqlite x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 137 k 2023-11-16T16:26:40.816 INFO:teuthology.orchestra.run.smithi026.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-16T16:26:40.816 INFO:teuthology.orchestra.run.smithi026.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-16T16:26:40.816 INFO:teuthology.orchestra.run.smithi026.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-16T16:26:40.816 INFO:teuthology.orchestra.run.smithi026.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-16T16:26:40.816 INFO:teuthology.orchestra.run.smithi026.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-16T16:26:40.816 INFO:teuthology.orchestra.run.smithi026.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-16T16:26:40.816 INFO:teuthology.orchestra.run.smithi026.stdout: libradosstriper1 x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 473 k 2023-11-16T16:26:40.817 INFO:teuthology.orchestra.run.smithi026.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-16T16:26:40.817 INFO:teuthology.orchestra.run.smithi026.stdout: librgw2 x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 6.5 M 2023-11-16T16:26:40.817 INFO:teuthology.orchestra.run.smithi026.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-16T16:26:40.817 INFO:teuthology.orchestra.run.smithi026.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-16T16:26:40.817 INFO:teuthology.orchestra.run.smithi026.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-16T16:26:40.817 INFO:teuthology.orchestra.run.smithi026.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-16T16:26:40.817 INFO:teuthology.orchestra.run.smithi026.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-16T16:26:40.817 INFO:teuthology.orchestra.run.smithi026.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-16T16:26:40.817 INFO:teuthology.orchestra.run.smithi026.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-16T16:26:40.817 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-16T16:26:40.817 INFO:teuthology.orchestra.run.smithi026.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-16T16:26:40.817 INFO:teuthology.orchestra.run.smithi026.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-16T16:26:40.817 INFO:teuthology.orchestra.run.smithi026.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-16T16:26:40.818 INFO:teuthology.orchestra.run.smithi026.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-16T16:26:40.818 INFO:teuthology.orchestra.run.smithi026.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-16T16:26:40.818 INFO:teuthology.orchestra.run.smithi026.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-16T16:26:40.818 INFO:teuthology.orchestra.run.smithi026.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-16T16:26:40.818 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-16T16:26:40.818 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-argparse x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 45 k 2023-11-16T16:26:40.818 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-common x86_64 2:18.0.0-3905.g5e4a743a.el8 ceph 119 k 2023-11-16T16:26:40.818 INFO:teuthology.orchestra.run.smithi026.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-16T16:26:40.818 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-16T16:26:40.818 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-16T16:26:40.818 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-16T16:26:40.818 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-16T16:26:40.819 INFO:teuthology.orchestra.run.smithi026.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-16T16:26:40.819 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-16T16:26:40.819 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-16T16:26:40.819 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-16T16:26:40.819 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-16T16:26:40.819 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-16T16:26:40.819 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-16T16:26:40.819 INFO:teuthology.orchestra.run.smithi026.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-16T16:26:40.819 INFO:teuthology.orchestra.run.smithi026.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-16T16:26:40.819 INFO:teuthology.orchestra.run.smithi026.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-16T16:26:40.819 INFO:teuthology.orchestra.run.smithi026.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-16T16:26:40.819 INFO:teuthology.orchestra.run.smithi026.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-16T16:26:40.819 INFO:teuthology.orchestra.run.smithi026.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-16T16:26:40.819 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-16T16:26:40.820 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-16T16:26:40.820 INFO:teuthology.orchestra.run.smithi026.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-16T16:26:40.820 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-16T16:26:40.820 INFO:teuthology.orchestra.run.smithi026.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-16T16:26:40.820 INFO:teuthology.orchestra.run.smithi026.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-16T16:26:40.820 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-16T16:26:40.820 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-16T16:26:40.820 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-16T16:26:40.820 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-16T16:26:40.820 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-16T16:26:40.820 INFO:teuthology.orchestra.run.smithi026.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-16T16:26:40.820 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-16T16:26:40.820 INFO:teuthology.orchestra.run.smithi026.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-16T16:26:40.821 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-16T16:26:40.821 INFO:teuthology.orchestra.run.smithi026.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-16T16:26:40.821 INFO:teuthology.orchestra.run.smithi026.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-16T16:26:40.821 INFO:teuthology.orchestra.run.smithi026.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-16T16:26:40.821 INFO:teuthology.orchestra.run.smithi026.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-16T16:26:40.821 INFO:teuthology.orchestra.run.smithi026.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-16T16:26:40.821 INFO:teuthology.orchestra.run.smithi026.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-16T16:26:40.821 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-16T16:26:40.821 INFO:teuthology.orchestra.run.smithi026.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-16T16:26:40.821 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-16T16:26:40.821 INFO:teuthology.orchestra.run.smithi026.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-16T16:26:40.821 INFO:teuthology.orchestra.run.smithi026.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-16T16:26:40.821 INFO:teuthology.orchestra.run.smithi026.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-16T16:26:40.822 INFO:teuthology.orchestra.run.smithi026.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-16T16:26:40.822 INFO:teuthology.orchestra.run.smithi026.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-16T16:26:40.822 INFO:teuthology.orchestra.run.smithi026.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-16T16:26:40.822 INFO:teuthology.orchestra.run.smithi026.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-16T16:26:40.822 INFO:teuthology.orchestra.run.smithi026.stdout:Enabling module streams: 2023-11-16T16:26:40.822 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql 10 2023-11-16T16:26:40.822 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-11-16T16:26:40.822 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2023-11-16T16:26:40.822 INFO:teuthology.orchestra.run.smithi026.stdout:=========================================================================================================================== 2023-11-16T16:26:40.822 INFO:teuthology.orchestra.run.smithi026.stdout:Install 128 Packages 2023-11-16T16:26:40.822 INFO:teuthology.orchestra.run.smithi026.stdout:Upgrade 3 Packages 2023-11-16T16:26:40.822 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-11-16T16:26:40.823 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 197 M 2023-11-16T16:26:40.823 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2023-11-16T16:26:40.990 INFO:teuthology.orchestra.run.smithi053.stdout:(5/131): ceph-mgr-18.0.0-3905.g5e4a743a.el8.x86 8.5 MB/s | 1.5 MB 00:00 2023-11-16T16:26:41.084 INFO:teuthology.orchestra.run.smithi001.stdout:(9/131): ceph-selinux-18.0.0-3905.g5e4a743a.el8 49 kB/s | 24 kB 00:00 2023-11-16T16:26:41.208 INFO:teuthology.orchestra.run.smithi026.stdout:(1/131): ceph-18.0.0-3905.g5e4a743a.el8.x86_64. 28 kB/s | 6.4 kB 00:00 2023-11-16T16:26:41.707 INFO:teuthology.orchestra.run.smithi053.stdout:(6/131): ceph-mon-18.0.0-3905.g5e4a743a.el8.x86 6.4 MB/s | 4.6 MB 00:00 2023-11-16T16:26:41.934 INFO:teuthology.orchestra.run.smithi001.stdout:(10/131): ceph-radosgw-18.0.0-3905.g5e4a743a.el 8.3 MB/s | 13 MB 00:01 2023-11-16T16:26:42.115 INFO:teuthology.orchestra.run.smithi053.stdout:(7/131): ceph-base-18.0.0-3905.g5e4a743a.el8.x8 1.8 MB/s | 5.3 MB 00:02 2023-11-16T16:26:42.117 INFO:teuthology.orchestra.run.smithi001.stdout:(11/131): cephfs-mirror-18.0.0-3905.g5e4a743a.e 1.4 MB/s | 258 kB 00:00 2023-11-16T16:26:42.226 INFO:teuthology.orchestra.run.smithi001.stdout:(12/131): libcephfs-devel-18.0.0-3905.g5e4a743a 287 kB/s | 31 kB 00:00 2023-11-16T16:26:42.300 INFO:teuthology.orchestra.run.smithi026.stdout:(2/131): ceph-fuse-18.0.0-3905.g5e4a743a.el8.x8 776 kB/s | 847 kB 00:01 2023-11-16T16:26:42.492 INFO:teuthology.orchestra.run.smithi026.stdout:(3/131): ceph-immutable-object-cache-18.0.0-390 864 kB/s | 165 kB 00:00 2023-11-16T16:26:42.518 INFO:teuthology.orchestra.run.smithi001.stdout:(13/131): libcephfs2-18.0.0-3905.g5e4a743a.el8. 2.6 MB/s | 771 kB 00:00 2023-11-16T16:26:42.627 INFO:teuthology.orchestra.run.smithi001.stdout:(14/131): libcephsqlite-18.0.0-3905.g5e4a743a.e 1.2 MB/s | 137 kB 00:00 2023-11-16T16:26:42.752 INFO:teuthology.orchestra.run.smithi001.stdout:(15/131): librados-devel-18.0.0-3905.g5e4a743a. 1.1 MB/s | 139 kB 00:00 2023-11-16T16:26:42.928 INFO:teuthology.orchestra.run.smithi001.stdout:(16/131): libradosstriper1-18.0.0-3905.g5e4a743 2.6 MB/s | 473 kB 00:00 2023-11-16T16:26:44.015 INFO:teuthology.orchestra.run.smithi053.stdout:(8/131): ceph-osd-18.0.0-3905.g5e4a743a.el8.x86 7.7 MB/s | 18 MB 00:02 2023-11-16T16:26:44.091 INFO:teuthology.orchestra.run.smithi053.stdout:(9/131): ceph-selinux-18.0.0-3905.g5e4a743a.el8 326 kB/s | 24 kB 00:00 2023-11-16T16:26:44.259 INFO:teuthology.orchestra.run.smithi026.stdout:(4/131): ceph-mds-18.0.0-3905.g5e4a743a.el8.x86 1.3 MB/s | 2.3 MB 00:01 2023-11-16T16:26:44.394 INFO:teuthology.orchestra.run.smithi001.stdout:(17/131): ceph-osd-18.0.0-3905.g5e4a743a.el8.x8 4.0 MB/s | 18 MB 00:04 2023-11-16T16:26:44.495 INFO:teuthology.orchestra.run.smithi001.stdout:(18/131): librgw2-18.0.0-3905.g5e4a743a.el8.x86 4.2 MB/s | 6.5 MB 00:01 2023-11-16T16:26:44.541 INFO:teuthology.orchestra.run.smithi001.stdout:(19/131): python3-ceph-argparse-18.0.0-3905.g5e 307 kB/s | 45 kB 00:00 2023-11-16T16:26:44.629 INFO:teuthology.orchestra.run.smithi001.stdout:(20/131): python3-cephfs-18.0.0-3905.g5e4a743a. 2.2 MB/s | 197 kB 00:00 2023-11-16T16:26:44.770 INFO:teuthology.orchestra.run.smithi001.stdout:(21/131): python3-rados-18.0.0-3905.g5e4a743a.e 2.6 MB/s | 371 kB 00:00 2023-11-16T16:26:44.896 INFO:teuthology.orchestra.run.smithi001.stdout:(22/131): python3-rbd-18.0.0-3905.g5e4a743a.el8 2.9 MB/s | 365 kB 00:00 2023-11-16T16:26:45.021 INFO:teuthology.orchestra.run.smithi001.stdout:(23/131): python3-rgw-18.0.0-3905.g5e4a743a.el8 855 kB/s | 107 kB 00:00 2023-11-16T16:26:45.101 INFO:teuthology.orchestra.run.smithi026.stdout:(5/131): ceph-mgr-18.0.0-3905.g5e4a743a.el8.x86 1.8 MB/s | 1.5 MB 00:00 2023-11-16T16:26:45.155 INFO:teuthology.orchestra.run.smithi001.stdout:(24/131): rbd-fuse-18.0.0-3905.g5e4a743a.el8.x8 707 kB/s | 94 kB 00:00 2023-11-16T16:26:45.613 INFO:teuthology.orchestra.run.smithi001.stdout:(25/131): python3-ceph-common-18.0.0-3905.g5e4a 106 kB/s | 119 kB 00:01 2023-11-16T16:26:45.780 INFO:teuthology.orchestra.run.smithi001.stdout:(26/131): rbd-nbd-18.0.0-3905.g5e4a743a.el8.x86 1.1 MB/s | 180 kB 00:00 2023-11-16T16:26:45.872 INFO:teuthology.orchestra.run.smithi001.stdout:(27/131): rbd-mirror-18.0.0-3905.g5e4a743a.el8. 4.8 MB/s | 3.4 MB 00:00 2023-11-16T16:26:45.889 INFO:teuthology.orchestra.run.smithi001.stdout:(28/131): ceph-grafana-dashboards-18.0.0-3905.g 212 kB/s | 23 kB 00:00 2023-11-16T16:26:45.998 INFO:teuthology.orchestra.run.smithi001.stdout:(29/131): ceph-mgr-cephadm-18.0.0-3905.g5e4a743 1.0 MB/s | 127 kB 00:00 2023-11-16T16:26:46.331 INFO:teuthology.orchestra.run.smithi001.stdout:(30/131): ceph-mgr-dashboard-18.0.0-3905.g5e4a7 3.9 MB/s | 1.7 MB 00:00 2023-11-16T16:26:46.432 INFO:teuthology.orchestra.run.smithi001.stdout:(31/131): ceph-mgr-modules-core-18.0.0-3905.g5e 2.4 MB/s | 240 kB 00:00 2023-11-16T16:26:46.507 INFO:teuthology.orchestra.run.smithi001.stdout:(32/131): ceph-mgr-rook-18.0.0-3905.g5e4a743a.e 633 kB/s | 47 kB 00:00 2023-11-16T16:26:46.582 INFO:teuthology.orchestra.run.smithi001.stdout:(33/131): ceph-prometheus-alerts-18.0.0-3905.g5 194 kB/s | 15 kB 00:00 2023-11-16T16:26:46.683 INFO:teuthology.orchestra.run.smithi001.stdout:(34/131): ceph-volume-18.0.0-3905.g5e4a743a.el8 2.6 MB/s | 265 kB 00:00 2023-11-16T16:26:46.783 INFO:teuthology.orchestra.run.smithi001.stdout:(35/131): cephadm-18.0.0-3905.g5e4a743a.el8.noa 837 kB/s | 83 kB 00:00 2023-11-16T16:26:46.858 INFO:teuthology.orchestra.run.smithi001.stdout:(36/131): cephfs-top-18.0.0-3905.g5e4a743a.el8. 240 kB/s | 18 kB 00:00 2023-11-16T16:26:47.733 INFO:teuthology.orchestra.run.smithi001.stdout:(37/131): ceph-mgr-diskprediction-local-18.0.0- 4.3 MB/s | 7.4 MB 00:01 2023-11-16T16:26:47.758 INFO:teuthology.orchestra.run.smithi001.stdout:(38/131): gperftools-libs-2.7-9.el8.x86_64.rpm 340 kB/s | 306 kB 00:00 2023-11-16T16:26:47.776 INFO:teuthology.orchestra.run.smithi026.stdout:(6/131): ceph-common-18.0.0-3905.g5e4a743a.el8. 3.2 MB/s | 22 MB 00:06 2023-11-16T16:26:47.851 INFO:teuthology.orchestra.run.smithi026.stdout:(7/131): ceph-mon-18.0.0-3905.g5e4a743a.el8.x86 1.7 MB/s | 4.6 MB 00:02 2023-11-16T16:26:47.866 INFO:teuthology.orchestra.run.smithi053.stdout:(10/131): ceph-radosgw-18.0.0-3905.g5e4a743a.el 2.3 MB/s | 13 MB 00:05 2023-11-16T16:26:47.926 INFO:teuthology.orchestra.run.smithi001.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 450 kB/s | 75 kB 00:00 2023-11-16T16:26:47.951 INFO:teuthology.orchestra.run.smithi026.stdout:(8/131): ceph-base-18.0.0-3905.g5e4a743a.el8.x8 781 kB/s | 5.3 MB 00:06 2023-11-16T16:26:47.974 INFO:teuthology.orchestra.run.smithi053.stdout:(11/131): cephfs-mirror-18.0.0-3905.g5e4a743a.e 2.3 MB/s | 258 kB 00:00 2023-11-16T16:26:48.033 INFO:teuthology.orchestra.run.smithi053.stdout:(12/131): libcephfs-devel-18.0.0-3905.g5e4a743a 532 kB/s | 31 kB 00:00 2023-11-16T16:26:48.043 INFO:teuthology.orchestra.run.smithi026.stdout:(9/131): ceph-selinux-18.0.0-3905.g5e4a743a.el8 265 kB/s | 24 kB 00:00 2023-11-16T16:26:48.201 INFO:teuthology.orchestra.run.smithi001.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.7 MB/s | 476 kB 00:00 2023-11-16T16:26:48.260 INFO:teuthology.orchestra.run.smithi001.stdout:(41/131): liboath-2.6.2-3.el8.x86_64.rpm 112 kB/s | 59 kB 00:00 2023-11-16T16:26:48.317 INFO:teuthology.orchestra.run.smithi053.stdout:(13/131): libcephfs2-18.0.0-3905.g5e4a743a.el8. 2.7 MB/s | 771 kB 00:00 2023-11-16T16:26:48.343 INFO:teuthology.orchestra.run.smithi001.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 308 kB/s | 44 kB 00:00 2023-11-16T16:26:48.392 INFO:teuthology.orchestra.run.smithi053.stdout:(14/131): libcephsqlite-18.0.0-3905.g5e4a743a.e 1.8 MB/s | 137 kB 00:00 2023-11-16T16:26:48.467 INFO:teuthology.orchestra.run.smithi053.stdout:(15/131): librados-devel-18.0.0-3905.g5e4a743a. 1.8 MB/s | 139 kB 00:00 2023-11-16T16:26:48.568 INFO:teuthology.orchestra.run.smithi001.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 147 kB/s | 33 kB 00:00 2023-11-16T16:26:48.659 INFO:teuthology.orchestra.run.smithi053.stdout:(16/131): libradosstriper1-18.0.0-3905.g5e4a743 2.4 MB/s | 473 kB 00:00 2023-11-16T16:26:48.773 INFO:teuthology.orchestra.run.smithi001.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 361 kB/s | 185 kB 00:00 2023-11-16T16:26:48.828 INFO:teuthology.orchestra.run.smithi001.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 63 kB/s | 16 kB 00:00 2023-11-16T16:26:49.160 INFO:teuthology.orchestra.run.smithi053.stdout:(17/131): ceph-common-18.0.0-3905.g5e4a743a.el8 2.2 MB/s | 22 MB 00:09 2023-11-16T16:26:49.203 INFO:teuthology.orchestra.run.smithi001.stdout:(46/131): ceph-test-18.0.0-3905.g5e4a743a.el8.x 5.0 MB/s | 41 MB 00:08 2023-11-16T16:26:49.237 INFO:teuthology.orchestra.run.smithi001.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 373 kB/s | 173 kB 00:00 2023-11-16T16:26:49.262 INFO:teuthology.orchestra.run.smithi001.stdout:(48/131): python3-cherrypy-18.4.0-1.el8.noarch. 887 kB/s | 384 kB 00:00 2023-11-16T16:26:49.379 INFO:teuthology.orchestra.run.smithi001.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 78 kB/s | 11 kB 00:00 2023-11-16T16:26:49.404 INFO:teuthology.orchestra.run.smithi001.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 124 kB/s | 18 kB 00:00 2023-11-16T16:26:49.552 INFO:teuthology.orchestra.run.smithi053.stdout:(18/131): ceph-test-18.0.0-3905.g5e4a743a.el8.x 7.4 MB/s | 41 MB 00:05 2023-11-16T16:26:49.577 INFO:teuthology.orchestra.run.smithi053.stdout:(19/131): python3-ceph-argparse-18.0.0-3905.g5e 107 kB/s | 45 kB 00:00 2023-11-16T16:26:49.602 INFO:teuthology.orchestra.run.smithi053.stdout:(20/131): python3-ceph-common-18.0.0-3905.g5e4a 2.3 MB/s | 119 kB 00:00 2023-11-16T16:26:49.604 INFO:teuthology.orchestra.run.smithi001.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 244 kB/s | 49 kB 00:00 2023-11-16T16:26:49.686 INFO:teuthology.orchestra.run.smithi053.stdout:(21/131): python3-rados-18.0.0-3905.g5e4a743a.e 4.4 MB/s | 371 kB 00:00 2023-11-16T16:26:49.736 INFO:teuthology.orchestra.run.smithi053.stdout:(22/131): python3-cephfs-18.0.0-3905.g5e4a743a. 1.2 MB/s | 197 kB 00:00 2023-11-16T16:26:49.754 INFO:teuthology.orchestra.run.smithi001.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 392 kB/s | 59 kB 00:00 2023-11-16T16:26:49.761 INFO:teuthology.orchestra.run.smithi053.stdout:(23/131): python3-rbd-18.0.0-3905.g5e4a743a.el8 4.8 MB/s | 365 kB 00:00 2023-11-16T16:26:49.797 INFO:teuthology.orchestra.run.smithi001.stdout:(53/131): python3-kubernetes-11.0.0-6.el8.noarc 2.8 MB/s | 1.2 MB 00:00 2023-11-16T16:26:49.803 INFO:teuthology.orchestra.run.smithi053.stdout:(24/131): python3-rgw-18.0.0-3905.g5e4a743a.el8 1.6 MB/s | 107 kB 00:00 2023-11-16T16:26:49.822 INFO:teuthology.orchestra.run.smithi001.stdout:(54/131): python3-google-auth-1.1.1-10.el8.noar 131 kB/s | 81 kB 00:00 2023-11-16T16:26:49.828 INFO:teuthology.orchestra.run.smithi053.stdout:(25/131): rbd-fuse-18.0.0-3905.g5e4a743a.el8.x8 1.4 MB/s | 94 kB 00:00 2023-11-16T16:26:49.879 INFO:teuthology.orchestra.run.smithi053.stdout:(26/131): rbd-nbd-18.0.0-3905.g5e4a743a.el8.x86 3.5 MB/s | 180 kB 00:00 2023-11-16T16:26:49.889 INFO:teuthology.orchestra.run.smithi001.stdout:(55/131): python3-natsort-7.1.1-2.el8.noarch.rp 450 kB/s | 60 kB 00:00 2023-11-16T16:26:49.921 INFO:teuthology.orchestra.run.smithi053.stdout:(27/131): ceph-grafana-dashboards-18.0.0-3905.g 556 kB/s | 23 kB 00:00 2023-11-16T16:26:49.947 INFO:teuthology.orchestra.run.smithi001.stdout:(56/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.8 MB/s | 283 kB 00:00 2023-11-16T16:26:49.964 INFO:teuthology.orchestra.run.smithi001.stdout:(57/131): python3-portend-2.6-1.el8.noarch.rpm 109 kB/s | 16 kB 00:00 2023-11-16T16:26:49.971 INFO:teuthology.orchestra.run.smithi053.stdout:(28/131): ceph-mgr-cephadm-18.0.0-3905.g5e4a743 2.5 MB/s | 127 kB 00:00 2023-11-16T16:26:50.010 INFO:teuthology.orchestra.run.smithi026.stdout:(10/131): ceph-osd-18.0.0-3905.g5e4a743a.el8.x8 7.9 MB/s | 18 MB 00:02 2023-11-16T16:26:50.015 INFO:teuthology.orchestra.run.smithi001.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 268 kB/s | 34 kB 00:00 2023-11-16T16:26:50.073 INFO:teuthology.orchestra.run.smithi001.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 1.5 MB/s | 196 kB 00:00 2023-11-16T16:26:50.085 INFO:teuthology.orchestra.run.smithi026.stdout:(11/131): cephfs-mirror-18.0.0-3905.g5e4a743a.e 3.4 MB/s | 258 kB 00:00 2023-11-16T16:26:50.127 INFO:teuthology.orchestra.run.smithi026.stdout:(12/131): libcephfs-devel-18.0.0-3905.g5e4a743a 749 kB/s | 31 kB 00:00 2023-11-16T16:26:50.140 INFO:teuthology.orchestra.run.smithi001.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 149 kB/s | 19 kB 00:00 2023-11-16T16:26:50.165 INFO:teuthology.orchestra.run.smithi001.stdout:(61/131): python3-rsa-4.9-2.el8.noarch.rpm 320 kB/s | 64 kB 00:00 2023-11-16T16:26:50.180 INFO:teuthology.orchestra.run.smithi053.stdout:(29/131): librgw2-18.0.0-3905.g5e4a743a.el8.x86 4.3 MB/s | 6.5 MB 00:01 2023-11-16T16:26:50.191 INFO:teuthology.orchestra.run.smithi001.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 204 kB/s | 24 kB 00:00 2023-11-16T16:26:50.236 INFO:teuthology.orchestra.run.smithi026.stdout:(13/131): libcephfs2-18.0.0-3905.g5e4a743a.el8. 7.0 MB/s | 771 kB 00:00 2023-11-16T16:26:50.274 INFO:teuthology.orchestra.run.smithi001.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 218 kB/s | 29 kB 00:00 2023-11-16T16:26:50.286 INFO:teuthology.orchestra.run.smithi026.stdout:(14/131): libcephsqlite-18.0.0-3905.g5e4a743a.e 2.7 MB/s | 137 kB 00:00 2023-11-16T16:26:50.291 INFO:teuthology.orchestra.run.smithi001.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 219 kB/s | 27 kB 00:00 2023-11-16T16:26:50.333 INFO:teuthology.orchestra.run.smithi001.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 1.8 MB/s | 255 kB 00:00 2023-11-16T16:26:50.336 INFO:teuthology.orchestra.run.smithi026.stdout:(15/131): librados-devel-18.0.0-3905.g5e4a743a. 2.7 MB/s | 139 kB 00:00 2023-11-16T16:26:50.380 INFO:teuthology.orchestra.run.smithi053.stdout:(30/131): ceph-mgr-dashboard-18.0.0-3905.g5e4a7 4.2 MB/s | 1.7 MB 00:00 2023-11-16T16:26:50.428 INFO:teuthology.orchestra.run.smithi026.stdout:(16/131): libradosstriper1-18.0.0-3905.g5e4a743 5.0 MB/s | 473 kB 00:00 2023-11-16T16:26:50.438 INFO:teuthology.orchestra.run.smithi053.stdout:(31/131): ceph-mgr-modules-core-18.0.0-3905.g5e 4.0 MB/s | 240 kB 00:00 2023-11-16T16:26:50.475 INFO:teuthology.orchestra.run.smithi001.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.2 MB/s | 251 kB 00:00 2023-11-16T16:26:50.500 INFO:teuthology.orchestra.run.smithi001.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 293 kB/s | 61 kB 00:00 2023-11-16T16:26:50.505 INFO:teuthology.orchestra.run.smithi053.stdout:(32/131): ceph-mgr-rook-18.0.0-3905.g5e4a743a.e 711 kB/s | 47 kB 00:00 2023-11-16T16:26:50.525 INFO:teuthology.orchestra.run.smithi001.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 444 kB/s | 85 kB 00:00 2023-11-16T16:26:50.564 INFO:teuthology.orchestra.run.smithi053.stdout:(33/131): ceph-prometheus-alerts-18.0.0-3905.g5 250 kB/s | 15 kB 00:00 2023-11-16T16:26:50.576 INFO:teuthology.orchestra.run.smithi001.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2023-11-16T16:26:50.651 INFO:teuthology.orchestra.run.smithi001.stdout:(70/131): python3-zc-lockfile-2.0-2.el8.noarch. 130 kB/s | 23 kB 00:00 2023-11-16T16:26:50.656 INFO:teuthology.orchestra.run.smithi053.stdout:(34/131): ceph-volume-18.0.0-3905.g5e4a743a.el8 2.8 MB/s | 265 kB 00:00 2023-11-16T16:26:50.707 INFO:teuthology.orchestra.run.smithi053.stdout:(35/131): cephadm-18.0.0-3905.g5e4a743a.el8.noa 1.6 MB/s | 83 kB 00:00 2023-11-16T16:26:50.765 INFO:teuthology.orchestra.run.smithi053.stdout:(36/131): rbd-mirror-18.0.0-3905.g5e4a743a.el8. 3.6 MB/s | 3.4 MB 00:00 2023-11-16T16:26:50.790 INFO:teuthology.orchestra.run.smithi053.stdout:(37/131): cephfs-top-18.0.0-3905.g5e4a743a.el8. 215 kB/s | 18 kB 00:00 2023-11-16T16:26:51.069 INFO:teuthology.orchestra.run.smithi001.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 264 kB/s | 110 kB 00:00 2023-11-16T16:26:51.137 INFO:teuthology.orchestra.run.smithi026.stdout:(17/131): librgw2-18.0.0-3905.g5e4a743a.el8.x86 9.2 MB/s | 6.5 MB 00:00 2023-11-16T16:26:51.143 INFO:teuthology.orchestra.run.smithi001.stdout:(72/131): thrift-0.13.0-2.el8.x86_64.rpm 2.7 MB/s | 1.7 MB 00:00 2023-11-16T16:26:51.187 INFO:teuthology.orchestra.run.smithi026.stdout:(18/131): python3-ceph-argparse-18.0.0-3905.g5e 900 kB/s | 45 kB 00:00 2023-11-16T16:26:51.246 INFO:teuthology.orchestra.run.smithi026.stdout:(19/131): python3-ceph-common-18.0.0-3905.g5e4a 2.0 MB/s | 119 kB 00:00 2023-11-16T16:26:51.304 INFO:teuthology.orchestra.run.smithi026.stdout:(20/131): python3-cephfs-18.0.0-3905.g5e4a743a. 3.3 MB/s | 197 kB 00:00 2023-11-16T16:26:51.316 INFO:teuthology.orchestra.run.smithi053.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 112 kB/s | 59 kB 00:00 2023-11-16T16:26:51.319 INFO:teuthology.orchestra.run.smithi001.stdout:(73/131): python3-scipy-1.0.0-21.module+el8.5.0 18 MB/s | 14 MB 00:00 2023-11-16T16:26:51.344 INFO:teuthology.orchestra.run.smithi001.stdout:(74/131): python3-pytz-2017.2-9.el8.noarch.rpm 197 kB/s | 54 kB 00:00 2023-11-16T16:26:51.369 INFO:teuthology.orchestra.run.smithi001.stdout:(75/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.1 MB/s | 259 kB 00:00 2023-11-16T16:26:51.388 INFO:teuthology.orchestra.run.smithi026.stdout:(21/131): python3-rados-18.0.0-3905.g5e4a743a.e 4.4 MB/s | 371 kB 00:00 2023-11-16T16:26:51.466 INFO:teuthology.orchestra.run.smithi053.stdout:(39/131): ceph-mgr-diskprediction-local-18.0.0- 5.7 MB/s | 7.4 MB 00:01 2023-11-16T16:26:51.480 INFO:teuthology.orchestra.run.smithi026.stdout:(22/131): python3-rbd-18.0.0-3905.g5e4a743a.el8 3.9 MB/s | 365 kB 00:00 2023-11-16T16:26:51.530 INFO:teuthology.orchestra.run.smithi026.stdout:(23/131): python3-rgw-18.0.0-3905.g5e4a743a.el8 2.1 MB/s | 107 kB 00:00 2023-11-16T16:26:51.544 INFO:teuthology.orchestra.run.smithi001.stdout:(76/131): socat-1.7.4.1-1.el8.x86_64.rpm 1.4 MB/s | 323 kB 00:00 2023-11-16T16:26:51.550 INFO:teuthology.orchestra.run.smithi053.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 321 kB/s | 75 kB 00:00 2023-11-16T16:26:51.581 INFO:teuthology.orchestra.run.smithi026.stdout:(24/131): rbd-fuse-18.0.0-3905.g5e4a743a.el8.x8 1.8 MB/s | 94 kB 00:00 2023-11-16T16:26:51.595 INFO:teuthology.orchestra.run.smithi001.stdout:(77/131): python3-markupsafe-0.23-19.el8.x86_64 155 kB/s | 39 kB 00:00 2023-11-16T16:26:51.650 INFO:teuthology.orchestra.run.smithi053.stdout:(41/131): gperftools-libs-2.7-9.el8.x86_64.rpm 346 kB/s | 306 kB 00:00 2023-11-16T16:26:51.662 INFO:teuthology.orchestra.run.smithi001.stdout:(78/131): bison-3.0.4-10.el8.x86_64.rpm 2.3 MB/s | 688 kB 00:00 2023-11-16T16:26:51.675 INFO:teuthology.orchestra.run.smithi053.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 361 kB/s | 44 kB 00:00 2023-11-16T16:26:51.807 INFO:teuthology.orchestra.run.smithi053.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 250 kB/s | 33 kB 00:00 2023-11-16T16:26:51.829 INFO:teuthology.orchestra.run.smithi001.stdout:(79/131): python3-jinja2-2.10.1-3.el8.noarch.rp 1.8 MB/s | 538 kB 00:00 2023-11-16T16:26:51.862 INFO:teuthology.orchestra.run.smithi001.stdout:(80/131): python3-mako-1.0.6-13.el8.noarch.rpm 587 kB/s | 157 kB 00:00 2023-11-16T16:26:51.896 INFO:teuthology.orchestra.run.smithi001.stdout:(81/131): oniguruma-6.8.2-2.el8.x86_64.rpm 801 kB/s | 187 kB 00:00 2023-11-16T16:26:51.909 INFO:teuthology.orchestra.run.smithi053.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 161 kB/s | 16 kB 00:00 2023-11-16T16:26:51.934 INFO:teuthology.orchestra.run.smithi053.stdout:(45/131): python3-beautifulsoup4-4.6.3-2.el8.1. 653 kB/s | 185 kB 00:00 2023-11-16T16:26:51.980 INFO:teuthology.orchestra.run.smithi001.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 318 kB/s | 27 kB 00:00 2023-11-16T16:26:52.073 INFO:teuthology.orchestra.run.smithi026.stdout:(25/131): rbd-mirror-18.0.0-3905.g5e4a743a.el8. 7.0 MB/s | 3.4 MB 00:00 2023-11-16T16:26:52.105 INFO:teuthology.orchestra.run.smithi001.stdout:(83/131): automake-1.16.1-7.el8.noarch.rpm 2.5 MB/s | 713 kB 00:00 2023-11-16T16:26:52.176 INFO:teuthology.orchestra.run.smithi053.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 647 kB/s | 173 kB 00:00 2023-11-16T16:26:52.206 INFO:teuthology.orchestra.run.smithi026.stdout:(26/131): ceph-radosgw-18.0.0-3905.g5e4a743a.el 3.0 MB/s | 13 MB 00:04 2023-11-16T16:26:52.223 INFO:teuthology.orchestra.run.smithi026.stdout:(27/131): rbd-nbd-18.0.0-3905.g5e4a743a.el8.x86 1.2 MB/s | 180 kB 00:00 2023-11-16T16:26:52.238 INFO:teuthology.orchestra.run.smithi001.stdout:(84/131): flex-2.6.1-9.el8.x86_64.rpm 1.2 MB/s | 320 kB 00:00 2023-11-16T16:26:52.282 INFO:teuthology.orchestra.run.smithi026.stdout:(28/131): ceph-grafana-dashboards-18.0.0-3905.g 307 kB/s | 23 kB 00:00 2023-11-16T16:26:52.307 INFO:teuthology.orchestra.run.smithi026.stdout:(29/131): ceph-mgr-cephadm-18.0.0-3905.g5e4a743 1.5 MB/s | 127 kB 00:00 2023-11-16T16:26:52.314 INFO:teuthology.orchestra.run.smithi001.stdout:(85/131): python3-babel-2.5.1-7.el8.noarch.rpm 11 MB/s | 4.8 MB 00:00 2023-11-16T16:26:52.393 INFO:teuthology.orchestra.run.smithi053.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 375 kB/s | 81 kB 00:00 2023-11-16T16:26:52.430 INFO:teuthology.orchestra.run.smithi001.stdout:(86/131): python3-numpy-1.14.3-10.el8.x86_64.rp 19 MB/s | 3.7 MB 00:00 2023-11-16T16:26:52.443 INFO:teuthology.orchestra.run.smithi053.stdout:(48/131): python3-cherrypy-18.4.0-1.el8.noarch. 756 kB/s | 384 kB 00:00 2023-11-16T16:26:52.456 INFO:teuthology.orchestra.run.smithi001.stdout:(87/131): python3-prettytable-0.7.2-14.el8.noar 125 kB/s | 44 kB 00:00 2023-11-16T16:26:52.485 INFO:teuthology.orchestra.run.smithi053.stdout:(49/131): python3-asyncssh-2.7.0-2.el8.noarch.r 467 kB/s | 476 kB 00:01 2023-11-16T16:26:52.502 INFO:teuthology.orchestra.run.smithi053.stdout:(50/131): python3-jaraco-6.2-6.el8.noarch.rpm 102 kB/s | 11 kB 00:00 2023-11-16T16:26:52.540 INFO:teuthology.orchestra.run.smithi001.stdout:(88/131): python3-werkzeug-0.12.2-4.el8.noarch. 2.0 MB/s | 457 kB 00:00 2023-11-16T16:26:52.552 INFO:teuthology.orchestra.run.smithi053.stdout:(51/131): python3-jaraco-functools-2.0-4.el8.no 161 kB/s | 18 kB 00:00 2023-11-16T16:26:52.603 INFO:teuthology.orchestra.run.smithi053.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 485 kB/s | 49 kB 00:00 2023-11-16T16:26:52.656 INFO:teuthology.orchestra.run.smithi001.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 319 kB/s | 64 kB 00:00 2023-11-16T16:26:52.661 INFO:teuthology.orchestra.run.smithi053.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 540 kB/s | 59 kB 00:00 2023-11-16T16:26:52.682 INFO:teuthology.orchestra.run.smithi001.stdout:(90/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 898 kB/s | 225 kB 00:00 2023-11-16T16:26:52.712 INFO:teuthology.orchestra.run.smithi053.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 555 kB/s | 60 kB 00:00 2023-11-16T16:26:52.812 INFO:teuthology.orchestra.run.smithi053.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 156 kB/s | 16 kB 00:00 2023-11-16T16:26:52.857 INFO:teuthology.orchestra.run.smithi001.stdout:(91/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 398 kB/s | 126 kB 00:00 2023-11-16T16:26:52.891 INFO:teuthology.orchestra.run.smithi026.stdout:(30/131): ceph-mgr-dashboard-18.0.0-3905.g5e4a7 2.8 MB/s | 1.7 MB 00:00 2023-11-16T16:26:52.912 INFO:teuthology.orchestra.run.smithi053.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 336 kB/s | 34 kB 00:00 2023-11-16T16:26:52.941 INFO:teuthology.orchestra.run.smithi001.stdout:(92/131): python3-jsonpointer-1.10-11.el8.noarc 70 kB/s | 20 kB 00:00 2023-11-16T16:26:52.957 INFO:teuthology.orchestra.run.smithi001.stdout:(93/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 376 kB/s | 103 kB 00:00 2023-11-16T16:26:53.021 INFO:teuthology.orchestra.run.smithi053.stdout:(57/131): python3-pecan-1.3.2-9.el8.noarch.rpm 787 kB/s | 283 kB 00:00 2023-11-16T16:26:53.049 INFO:teuthology.orchestra.run.smithi026.stdout:(31/131): ceph-mgr-modules-core-18.0.0-3905.g5e 1.5 MB/s | 240 kB 00:00 2023-11-16T16:26:53.124 INFO:teuthology.orchestra.run.smithi026.stdout:(32/131): ceph-mgr-rook-18.0.0-3905.g5e4a743a.e 634 kB/s | 47 kB 00:00 2023-11-16T16:26:53.129 INFO:teuthology.orchestra.run.smithi053.stdout:(58/131): python3-rsa-4.9-2.el8.noarch.rpm 593 kB/s | 64 kB 00:00 2023-11-16T16:26:53.180 INFO:teuthology.orchestra.run.smithi053.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 735 kB/s | 196 kB 00:00 2023-11-16T16:26:53.191 INFO:teuthology.orchestra.run.smithi001.stdout:(94/131): libtool-2.4.6-25.el8.x86_64.rpm 2.1 MB/s | 709 kB 00:00 2023-11-16T16:26:53.208 INFO:teuthology.orchestra.run.smithi026.stdout:(33/131): ceph-prometheus-alerts-18.0.0-3905.g5 174 kB/s | 15 kB 00:00 2023-11-16T16:26:53.222 INFO:teuthology.orchestra.run.smithi053.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 203 kB/s | 19 kB 00:00 2023-11-16T16:26:53.225 INFO:teuthology.orchestra.run.smithi001.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 740 kB/s | 198 kB 00:00 2023-11-16T16:26:53.281 INFO:teuthology.orchestra.run.smithi053.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 238 kB/s | 24 kB 00:00 2023-11-16T16:26:53.322 INFO:teuthology.orchestra.run.smithi053.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 292 kB/s | 29 kB 00:00 2023-11-16T16:26:53.350 INFO:teuthology.orchestra.run.smithi001.stdout:(96/131): openblas-threads-0.3.15-3.el8.x86_64. 12 MB/s | 4.9 MB 00:00 2023-11-16T16:26:53.375 INFO:teuthology.orchestra.run.smithi026.stdout:(34/131): ceph-volume-18.0.0-3905.g5e4a743a.el8 1.6 MB/s | 265 kB 00:00 2023-11-16T16:26:53.389 INFO:teuthology.orchestra.run.smithi053.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 253 kB/s | 27 kB 00:00 2023-11-16T16:26:53.467 INFO:teuthology.orchestra.run.smithi026.stdout:(35/131): ceph-mgr-diskprediction-local-18.0.0- 6.4 MB/s | 7.4 MB 00:01 2023-11-16T16:26:53.492 INFO:teuthology.orchestra.run.smithi026.stdout:(36/131): cephadm-18.0.0-3905.g5e4a743a.el8.noa 714 kB/s | 83 kB 00:00 2023-11-16T16:26:53.542 INFO:teuthology.orchestra.run.smithi026.stdout:(37/131): cephfs-top-18.0.0-3905.g5e4a743a.el8. 239 kB/s | 18 kB 00:00 2023-11-16T16:26:53.600 INFO:teuthology.orchestra.run.smithi001.stdout:(97/131): openblas-0.3.15-3.el8.x86_64.rpm 12 MB/s | 4.7 MB 00:00 2023-11-16T16:26:53.625 INFO:teuthology.orchestra.run.smithi001.stdout:(98/131): jq-1.6-3.el8.x86_64.rpm 505 kB/s | 202 kB 00:00 2023-11-16T16:26:53.668 INFO:teuthology.orchestra.run.smithi001.stdout:(99/131): librdkafka-0.11.4-3.el8.x86_64.rpm 1.1 MB/s | 354 kB 00:00 2023-11-16T16:26:53.681 INFO:teuthology.orchestra.run.smithi053.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 712 kB/s | 255 kB 00:00 2023-11-16T16:26:53.709 INFO:teuthology.orchestra.run.smithi026.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 353 kB/s | 59 kB 00:00 2023-11-16T16:26:53.740 INFO:teuthology.orchestra.run.smithi053.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 717 kB/s | 251 kB 00:00 2023-11-16T16:26:53.743 INFO:teuthology.orchestra.run.smithi026.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.2 MB/s | 306 kB 00:00 2023-11-16T16:26:53.785 INFO:teuthology.orchestra.run.smithi026.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 997 kB/s | 75 kB 00:00 2023-11-16T16:26:53.799 INFO:teuthology.orchestra.run.smithi053.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 522 kB/s | 61 kB 00:00 2023-11-16T16:26:53.852 INFO:teuthology.orchestra.run.smithi026.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 654 kB/s | 44 kB 00:00 2023-11-16T16:26:53.877 INFO:teuthology.orchestra.run.smithi026.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 3.5 MB/s | 476 kB 00:00 2023-11-16T16:26:53.899 INFO:teuthology.orchestra.run.smithi053.stdout:(67/131): python3-zc-lockfile-2.0-2.el8.noarch. 230 kB/s | 23 kB 00:00 2023-11-16T16:26:53.924 INFO:teuthology.orchestra.run.smithi053.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 464 kB/s | 85 kB 00:00 2023-11-16T16:26:53.935 INFO:teuthology.orchestra.run.smithi001.stdout:(100/131): postgresql-contrib-10.21-2.module+el 2.6 MB/s | 808 kB 00:00 2023-11-16T16:26:53.936 INFO:teuthology.orchestra.run.smithi026.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 564 kB/s | 33 kB 00:00 2023-11-16T16:26:53.941 INFO:teuthology.orchestra.run.smithi053.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 4.1 MB/s | 70 kB 00:00 2023-11-16T16:26:53.961 INFO:teuthology.orchestra.run.smithi026.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.7 MB/s | 185 kB 00:00 2023-11-16T16:26:53.987 INFO:teuthology.orchestra.run.smithi026.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 324 kB/s | 16 kB 00:00 2023-11-16T16:26:54.026 INFO:teuthology.orchestra.run.smithi001.stdout:(101/131): postgresql-server-10.21-2.module+el8 12 MB/s | 5.1 MB 00:00 2023-11-16T16:26:54.037 INFO:teuthology.orchestra.run.smithi026.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.2 MB/s | 173 kB 00:00 2023-11-16T16:26:54.058 INFO:teuthology.orchestra.run.smithi053.stdout:(70/131): python3-kubernetes-11.0.0-6.el8.noarc 759 kB/s | 1.2 MB 00:01 2023-11-16T16:26:54.068 INFO:teuthology.orchestra.run.smithi001.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 3.8 MB/s | 1.5 MB 00:00 2023-11-16T16:26:54.094 INFO:teuthology.orchestra.run.smithi001.stdout:(103/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 295 kB/s | 47 kB 00:00 2023-11-16T16:26:54.096 INFO:teuthology.orchestra.run.smithi026.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 3.4 MB/s | 384 kB 00:00 2023-11-16T16:26:54.121 INFO:teuthology.orchestra.run.smithi026.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 974 kB/s | 81 kB 00:00 2023-11-16T16:26:54.154 INFO:teuthology.orchestra.run.smithi026.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 192 kB/s | 11 kB 00:00 2023-11-16T16:26:54.188 INFO:teuthology.orchestra.run.smithi026.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 264 kB/s | 18 kB 00:00 2023-11-16T16:26:54.255 INFO:teuthology.orchestra.run.smithi026.stdout:(51/131): python3-kubernetes-11.0.0-6.el8.noarc 12 MB/s | 1.2 MB 00:00 2023-11-16T16:26:54.281 INFO:teuthology.orchestra.run.smithi026.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 529 kB/s | 49 kB 00:00 2023-11-16T16:26:54.322 INFO:teuthology.orchestra.run.smithi026.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 878 kB/s | 59 kB 00:00 2023-11-16T16:26:54.348 INFO:teuthology.orchestra.run.smithi026.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 899 kB/s | 60 kB 00:00 2023-11-16T16:26:54.390 INFO:teuthology.orchestra.run.smithi026.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 4.1 MB/s | 283 kB 00:00 2023-11-16T16:26:54.394 INFO:teuthology.orchestra.run.smithi001.stdout:(104/131): libselinux-devel-2.9-5.el8.x86_64.rp 544 kB/s | 200 kB 00:00 2023-11-16T16:26:54.415 INFO:teuthology.orchestra.run.smithi026.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 233 kB/s | 16 kB 00:00 2023-11-16T16:26:54.419 INFO:teuthology.orchestra.run.smithi001.stdout:(105/131): python3-jwt-1.6.1-2.el8.noarch.rpm 121 kB/s | 43 kB 00:00 2023-11-16T16:26:54.425 INFO:teuthology.orchestra.run.smithi053.stdout:(71/131): python3-scipy-1.0.0-21.module+el8.5.0 28 MB/s | 14 MB 00:00 2023-11-16T16:26:54.440 INFO:teuthology.orchestra.run.smithi026.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 671 kB/s | 34 kB 00:00 2023-11-16T16:26:54.442 INFO:teuthology.orchestra.run.smithi053.stdout:(72/131): python3-pyasn1-modules-0.3.7-6.el8.no 287 kB/s | 110 kB 00:00 2023-11-16T16:26:54.478 INFO:teuthology.orchestra.run.smithi001.stdout:(106/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 574 kB/s | 220 kB 00:00 2023-11-16T16:26:54.516 INFO:teuthology.orchestra.run.smithi026.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.9 MB/s | 196 kB 00:00 2023-11-16T16:26:54.534 INFO:teuthology.orchestra.run.smithi053.stdout:(73/131): python3-pytz-2017.2-9.el8.noarch.rpm 500 kB/s | 54 kB 00:00 2023-11-16T16:26:54.549 INFO:teuthology.orchestra.run.smithi026.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 592 kB/s | 64 kB 00:00 2023-11-16T16:26:54.591 INFO:teuthology.orchestra.run.smithi026.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 251 kB/s | 19 kB 00:00 2023-11-16T16:26:54.592 INFO:teuthology.orchestra.run.smithi053.stdout:(74/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.7 MB/s | 259 kB 00:00 2023-11-16T16:26:54.616 INFO:teuthology.orchestra.run.smithi026.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 357 kB/s | 24 kB 00:00 2023-11-16T16:26:54.650 INFO:teuthology.orchestra.run.smithi026.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 501 kB/s | 29 kB 00:00 2023-11-16T16:26:54.675 INFO:teuthology.orchestra.run.smithi026.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 469 kB/s | 27 kB 00:00 2023-11-16T16:26:54.693 INFO:teuthology.orchestra.run.smithi053.stdout:(75/131): socat-1.7.4.1-1.el8.x86_64.rpm 2.0 MB/s | 323 kB 00:00 2023-11-16T16:26:54.725 INFO:teuthology.orchestra.run.smithi026.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 3.3 MB/s | 255 kB 00:00 2023-11-16T16:26:54.736 INFO:teuthology.orchestra.run.smithi001.stdout:(107/131): python3-requests-oauthlib-1.0.0-1.el 165 kB/s | 43 kB 00:00 2023-11-16T16:26:54.751 INFO:teuthology.orchestra.run.smithi026.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 3.3 MB/s | 251 kB 00:00 2023-11-16T16:26:54.762 INFO:teuthology.orchestra.run.smithi001.stdout:(108/131): keyutils-libs-devel-1.5.10-9.el8.x86 131 kB/s | 48 kB 00:00 2023-11-16T16:26:54.785 INFO:teuthology.orchestra.run.smithi053.stdout:(76/131): bison-3.0.4-10.el8.x86_64.rpm 7.3 MB/s | 688 kB 00:00 2023-11-16T16:26:54.787 INFO:teuthology.orchestra.run.smithi001.stdout:(109/131): libsepol-devel-2.9-3.el8.x86_64.rpm 237 kB/s | 87 kB 00:00 2023-11-16T16:26:54.793 INFO:teuthology.orchestra.run.smithi026.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 918 kB/s | 61 kB 00:00 2023-11-16T16:26:54.810 INFO:teuthology.orchestra.run.smithi053.stdout:(77/131): python3-markupsafe-0.23-19.el8.x86_64 178 kB/s | 39 kB 00:00 2023-11-16T16:26:54.818 INFO:teuthology.orchestra.run.smithi026.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.2 MB/s | 85 kB 00:00 2023-11-16T16:26:54.852 INFO:teuthology.orchestra.run.smithi026.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 392 kB/s | 23 kB 00:00 2023-11-16T16:26:54.877 INFO:teuthology.orchestra.run.smithi026.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2023-11-16T16:26:54.944 INFO:teuthology.orchestra.run.smithi053.stdout:(78/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.2 MB/s | 157 kB 00:00 2023-11-16T16:26:54.944 INFO:teuthology.orchestra.run.smithi026.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 14 MB/s | 1.7 MB 00:00 2023-11-16T16:26:54.954 INFO:teuthology.orchestra.run.smithi001.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.2 MB/s | 229 kB 00:00 2023-11-16T16:26:54.986 INFO:teuthology.orchestra.run.smithi053.stdout:(79/131): python3-jinja2-2.10.1-3.el8.noarch.rp 2.6 MB/s | 538 kB 00:00 2023-11-16T16:26:54.996 INFO:teuthology.orchestra.run.smithi001.stdout:(111/131): python3-pycparser-2.14-14.el8.noarch 421 kB/s | 109 kB 00:00 2023-11-16T16:26:55.071 INFO:teuthology.orchestra.run.smithi001.stdout:(112/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 1.9 MB/s | 560 kB 00:00 2023-11-16T16:26:55.119 INFO:teuthology.orchestra.run.smithi053.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.0 MB/s | 187 kB 00:00 2023-11-16T16:26:55.153 INFO:teuthology.orchestra.run.smithi053.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 4.2 MB/s | 713 kB 00:00 2023-11-16T16:26:55.161 INFO:teuthology.orchestra.run.smithi026.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 529 kB/s | 110 kB 00:00 2023-11-16T16:26:55.203 INFO:teuthology.orchestra.run.smithi053.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 533 kB/s | 27 kB 00:00 2023-11-16T16:26:55.213 INFO:teuthology.orchestra.run.smithi001.stdout:(113/131): libaio-devel-0.3.112-1.el8.x86_64.rp 72 kB/s | 19 kB 00:00 2023-11-16T16:26:55.305 INFO:teuthology.orchestra.run.smithi001.stdout:(114/131): python3-oauthlib-2.1.0-1.el8.noarch. 502 kB/s | 155 kB 00:00 2023-11-16T16:26:55.312 INFO:teuthology.orchestra.run.smithi053.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 2.9 MB/s | 320 kB 00:00 2023-11-16T16:26:55.347 INFO:teuthology.orchestra.run.smithi001.stdout:(115/131): python3-cryptography-3.2.1-5.el8.x86 2.0 MB/s | 559 kB 00:00 2023-11-16T16:26:55.414 INFO:teuthology.orchestra.run.smithi001.stdout:(116/131): libverto-devel-0.3.0-5.el8.x86_64.rp 89 kB/s | 18 kB 00:00 2023-11-16T16:26:55.437 INFO:teuthology.orchestra.run.smithi053.stdout:(84/131): python3-babel-2.5.1-7.el8.noarch.rpm 15 MB/s | 4.8 MB 00:00 2023-11-16T16:26:55.444 INFO:teuthology.orchestra.run.smithi026.stdout:(72/131): python3-scipy-1.0.0-21.module+el8.5.0 24 MB/s | 14 MB 00:00 2023-11-16T16:26:55.447 INFO:teuthology.orchestra.run.smithi001.stdout:(117/131): pcre2-devel-10.32-2.el8.x86_64.rpm 4.2 MB/s | 605 kB 00:00 2023-11-16T16:26:55.462 INFO:teuthology.orchestra.run.smithi053.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 293 kB/s | 44 kB 00:00 2023-11-16T16:26:55.470 INFO:teuthology.orchestra.run.smithi026.stdout:(73/131): python3-pytz-2017.2-9.el8.noarch.rpm 176 kB/s | 54 kB 00:00 2023-11-16T16:26:55.587 INFO:teuthology.orchestra.run.smithi026.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 2.7 MB/s | 323 kB 00:00 2023-11-16T16:26:55.588 INFO:teuthology.orchestra.run.smithi053.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 3.6 MB/s | 457 kB 00:00 2023-11-16T16:26:55.681 INFO:teuthology.orchestra.run.smithi001.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 712 kB/s | 238 kB 00:00 2023-11-16T16:26:55.767 INFO:teuthology.orchestra.run.smithi026.stdout:(75/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 805 kB/s | 259 kB 00:00 2023-11-16T16:26:55.796 INFO:teuthology.orchestra.run.smithi026.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 186 kB/s | 39 kB 00:00 2023-11-16T16:26:55.814 INFO:teuthology.orchestra.run.smithi001.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 197 kB/s | 72 kB 00:00 2023-11-16T16:26:55.890 INFO:teuthology.orchestra.run.smithi001.stdout:(120/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 4.9 MB/s | 2.3 MB 00:00 2023-11-16T16:26:55.991 INFO:teuthology.orchestra.run.smithi001.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 341 kB/s | 60 kB 00:00 2023-11-16T16:26:55.996 INFO:teuthology.orchestra.run.smithi026.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 2.9 MB/s | 688 kB 00:00 2023-11-16T16:26:56.049 INFO:teuthology.orchestra.run.smithi001.stdout:(122/131): fuse-2.9.7-15.el8.x86_64.rpm 225 kB/s | 83 kB 00:00 2023-11-16T16:26:56.063 INFO:teuthology.orchestra.run.smithi026.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 2.0 MB/s | 538 kB 00:00 2023-11-16T16:26:56.068 INFO:teuthology.orchestra.run.smithi053.stdout:(87/131): python3-numpy-1.14.3-10.el8.x86_64.rp 5.9 MB/s | 3.7 MB 00:00 2023-11-16T16:26:56.222 INFO:teuthology.orchestra.run.smithi026.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 696 kB/s | 157 kB 00:00 2023-11-16T16:26:56.238 INFO:teuthology.orchestra.run.smithi053.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 346 kB/s | 225 kB 00:00 2023-11-16T16:26:56.256 INFO:teuthology.orchestra.run.smithi026.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 975 kB/s | 187 kB 00:00 2023-11-16T16:26:56.257 INFO:teuthology.orchestra.run.smithi001.stdout:(123/131): libuuid-devel-2.32.1-35.el8.x86_64.r 267 kB/s | 98 kB 00:00 2023-11-16T16:26:56.280 INFO:teuthology.orchestra.run.smithi053.stdout:(89/131): thrift-0.13.0-2.el8.x86_64.rpm 746 kB/s | 1.7 MB 00:02 2023-11-16T16:26:56.282 INFO:teuthology.orchestra.run.smithi001.stdout:(124/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 197 kB/s | 58 kB 00:00 2023-11-16T16:26:56.306 INFO:teuthology.orchestra.run.smithi053.stdout:(90/131): uuid-1.6.2-43.el8.x86_64.rpm 271 kB/s | 64 kB 00:00 2023-11-16T16:26:56.339 INFO:teuthology.orchestra.run.smithi026.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 6.0 MB/s | 713 kB 00:00 2023-11-16T16:26:56.341 INFO:teuthology.orchestra.run.smithi001.stdout:(125/131): libcom_err-devel-1.45.6-4.el8.x86_64 132 kB/s | 39 kB 00:00 2023-11-16T16:26:56.372 INFO:teuthology.orchestra.run.smithi053.stdout:(91/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 945 kB/s | 126 kB 00:00 2023-11-16T16:26:56.440 INFO:teuthology.orchestra.run.smithi053.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 829 kB/s | 103 kB 00:00 2023-11-16T16:26:56.456 INFO:teuthology.orchestra.run.smithi026.stdout:(82/131): python3-babel-2.5.1-7.el8.noarch.rpm 24 MB/s | 4.8 MB 00:00 2023-11-16T16:26:56.481 INFO:teuthology.orchestra.run.smithi026.stdout:(83/131): python3-jsonpatch-1.21-2.el8.noarch.r 187 kB/s | 27 kB 00:00 2023-11-16T16:26:56.498 INFO:teuthology.orchestra.run.smithi053.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 5.5 MB/s | 709 kB 00:00 2023-11-16T16:26:56.515 INFO:teuthology.orchestra.run.smithi053.stdout:(94/131): python3-jsonpointer-1.10-11.el8.noarc 85 kB/s | 20 kB 00:00 2023-11-16T16:26:56.533 INFO:teuthology.orchestra.run.smithi001.stdout:(126/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 2.3 MB/s | 644 kB 00:00 2023-11-16T16:26:56.583 INFO:teuthology.orchestra.run.smithi001.stdout:(127/131): NetworkManager-initscripts-updown-1. 569 kB/s | 138 kB 00:00 2023-11-16T16:26:56.598 INFO:teuthology.orchestra.run.smithi026.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 376 kB/s | 44 kB 00:00 2023-11-16T16:26:56.667 INFO:teuthology.orchestra.run.smithi001.stdout:(128/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 446 kB/s | 171 kB 00:00 2023-11-16T16:26:56.774 INFO:teuthology.orchestra.run.smithi053.stdout:(95/131): openblas-0.3.15-3.el8.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2023-11-16T16:26:56.815 INFO:teuthology.orchestra.run.smithi026.stdout:(85/131): python3-numpy-1.14.3-10.el8.x86_64.rp 17 MB/s | 3.7 MB 00:00 2023-11-16T16:26:56.840 INFO:teuthology.orchestra.run.smithi026.stdout:(86/131): flex-2.6.1-9.el8.x86_64.rpm 832 kB/s | 320 kB 00:00 2023-11-16T16:26:56.841 INFO:teuthology.orchestra.run.smithi053.stdout:(96/131): openblas-threads-0.3.15-3.el8.x86_64. 12 MB/s | 4.9 MB 00:00 2023-11-16T16:26:56.866 INFO:teuthology.orchestra.run.smithi053.stdout:(97/131): libpq-13.5-1.el8.x86_64.rpm 538 kB/s | 198 kB 00:00 2023-11-16T16:26:56.899 INFO:teuthology.orchestra.run.smithi053.stdout:(98/131): jq-1.6-3.el8.x86_64.rpm 1.6 MB/s | 202 kB 00:00 2023-11-16T16:26:56.901 INFO:teuthology.orchestra.run.smithi001.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 3.3 MB/s | 798 kB 00:00 2023-11-16T16:26:56.932 INFO:teuthology.orchestra.run.smithi026.stdout:(87/131): python3-werkzeug-0.12.2-4.el8.noarch. 3.8 MB/s | 457 kB 00:00 2023-11-16T16:26:56.967 INFO:teuthology.orchestra.run.smithi053.stdout:(99/131): librdkafka-0.11.4-3.el8.x86_64.rpm 2.8 MB/s | 354 kB 00:00 2023-11-16T16:26:56.996 INFO:teuthology.orchestra.run.smithi026.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.4 MB/s | 225 kB 00:00 2023-11-16T16:26:57.034 INFO:teuthology.orchestra.run.smithi053.stdout:(100/131): postgresql-contrib-10.21-2.module+el 5.9 MB/s | 808 kB 00:00 2023-11-16T16:26:57.058 INFO:teuthology.orchestra.run.smithi026.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 511 kB/s | 64 kB 00:00 2023-11-16T16:26:57.117 INFO:teuthology.orchestra.run.smithi053.stdout:(101/131): postgresql-server-10.21-2.module+el8 20 MB/s | 5.1 MB 00:00 2023-11-16T16:26:57.133 INFO:teuthology.orchestra.run.smithi026.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 920 kB/s | 126 kB 00:00 2023-11-16T16:26:57.142 INFO:teuthology.orchestra.run.smithi053.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 432 kB/s | 47 kB 00:00 2023-11-16T16:26:57.201 INFO:teuthology.orchestra.run.smithi053.stdout:(103/131): postgresql-10.21-2.module+el8.6.0+15 6.4 MB/s | 1.5 MB 00:00 2023-11-16T16:26:57.234 INFO:teuthology.orchestra.run.smithi053.stdout:(104/131): libselinux-devel-2.9-5.el8.x86_64.rp 1.7 MB/s | 200 kB 00:00 2023-11-16T16:26:57.260 INFO:teuthology.orchestra.run.smithi053.stdout:(105/131): python3-jwt-1.6.1-2.el8.noarch.rpm 363 kB/s | 43 kB 00:00 2023-11-16T16:26:57.292 INFO:teuthology.orchestra.run.smithi026.stdout:(91/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 654 kB/s | 103 kB 00:00 2023-11-16T16:26:57.309 INFO:teuthology.orchestra.run.smithi026.stdout:(92/131): python3-jsonpointer-1.10-11.el8.noarc 80 kB/s | 20 kB 00:00 2023-11-16T16:26:57.327 INFO:teuthology.orchestra.run.smithi053.stdout:(106/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.7 MB/s | 220 kB 00:00 2023-11-16T16:26:57.352 INFO:teuthology.orchestra.run.smithi053.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 946 kB/s | 87 kB 00:00 2023-11-16T16:26:57.377 INFO:teuthology.orchestra.run.smithi053.stdout:(108/131): keyutils-libs-devel-1.5.10-9.el8.x86 338 kB/s | 48 kB 00:00 2023-11-16T16:26:57.469 INFO:teuthology.orchestra.run.smithi053.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 932 kB/s | 109 kB 00:00 2023-11-16T16:26:57.503 INFO:teuthology.orchestra.run.smithi053.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.8 MB/s | 229 kB 00:00 2023-11-16T16:26:57.518 INFO:teuthology.orchestra.run.smithi001.stdout:(130/131): librbd1-18.0.0-3905.g5e4a743a.el8.x8 3.8 MB/s | 3.6 MB 00:00 2023-11-16T16:26:57.542 INFO:teuthology.orchestra.run.smithi026.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 2.8 MB/s | 709 kB 00:00 2023-11-16T16:26:57.545 INFO:teuthology.orchestra.run.smithi053.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 7.3 MB/s | 560 kB 00:00 2023-11-16T16:26:57.570 INFO:teuthology.orchestra.run.smithi053.stdout:(112/131): python3-requests-oauthlib-1.0.0-1.el 176 kB/s | 43 kB 00:00 2023-11-16T16:26:57.626 INFO:teuthology.orchestra.run.smithi026.stdout:(94/131): openblas-threads-0.3.15-3.el8.x86_64. 15 MB/s | 4.9 MB 00:00 2023-11-16T16:26:57.629 INFO:teuthology.orchestra.run.smithi053.stdout:(113/131): libaio-devel-0.3.112-1.el8.x86_64.rp 149 kB/s | 19 kB 00:00 2023-11-16T16:26:57.693 INFO:teuthology.orchestra.run.smithi026.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 1.3 MB/s | 198 kB 00:00 2023-11-16T16:26:57.835 INFO:teuthology.orchestra.run.smithi026.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 1.4 MB/s | 202 kB 00:00 2023-11-16T16:26:57.859 INFO:teuthology.orchestra.run.smithi053.stdout:(114/131): python3-cryptography-3.2.1-5.el8.x86 1.9 MB/s | 559 kB 00:00 2023-11-16T16:26:57.888 INFO:teuthology.orchestra.run.smithi053.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 69 kB/s | 18 kB 00:00 2023-11-16T16:26:57.913 INFO:teuthology.orchestra.run.smithi053.stdout:(116/131): python3-oauthlib-2.1.0-1.el8.noarch. 421 kB/s | 155 kB 00:00 2023-11-16T16:26:57.927 INFO:teuthology.orchestra.run.smithi026.stdout:(97/131): openblas-0.3.15-3.el8.x86_64.rpm 16 MB/s | 4.7 MB 00:00 2023-11-16T16:26:57.971 INFO:teuthology.orchestra.run.smithi053.stdout:(117/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.3 MB/s | 605 kB 00:00 2023-11-16T16:26:57.977 INFO:teuthology.orchestra.run.smithi026.stdout:(98/131): librdkafka-0.11.4-3.el8.x86_64.rpm 2.4 MB/s | 354 kB 00:00 2023-11-16T16:26:57.997 INFO:teuthology.orchestra.run.smithi053.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.1 MB/s | 238 kB 00:00 2023-11-16T16:26:58.080 INFO:teuthology.orchestra.run.smithi053.stdout:(119/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 14 MB/s | 2.3 MB 00:00 2023-11-16T16:26:58.122 INFO:teuthology.orchestra.run.smithi053.stdout:(120/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 480 kB/s | 72 kB 00:00 2023-11-16T16:26:58.153 INFO:teuthology.orchestra.run.smithi026.stdout:(99/131): postgresql-server-10.21-2.module+el8. 22 MB/s | 5.1 MB 00:00 2023-11-16T16:26:58.168 INFO:teuthology.orchestra.run.smithi053.stdout:(121/131): fuse-2.9.7-15.el8.x86_64.rpm 495 kB/s | 83 kB 00:00 2023-11-16T16:26:58.186 INFO:teuthology.orchestra.run.smithi026.stdout:(100/131): postgresql-contrib-10.21-2.module+el 3.8 MB/s | 808 kB 00:00 2023-11-16T16:26:58.189 INFO:teuthology.orchestra.run.smithi053.stdout:(122/131): elfutils-libelf-devel-0.186-1.el8.x8 549 kB/s | 60 kB 00:00 2023-11-16T16:26:58.248 INFO:teuthology.orchestra.run.smithi053.stdout:(123/131): libuuid-devel-2.32.1-35.el8.x86_64.r 782 kB/s | 98 kB 00:00 2023-11-16T16:26:58.278 INFO:teuthology.orchestra.run.smithi026.stdout:(101/131): postgresql-10.21-2.module+el8.6.0+15 12 MB/s | 1.5 MB 00:00 2023-11-16T16:26:58.328 INFO:teuthology.orchestra.run.smithi026.stdout:(102/131): libselinux-devel-2.9-5.el8.x86_64.rp 3.9 MB/s | 200 kB 00:00 2023-11-16T16:26:58.332 INFO:teuthology.orchestra.run.smithi053.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 271 kB/s | 39 kB 00:00 2023-11-16T16:26:58.365 INFO:teuthology.orchestra.run.smithi053.stdout:(125/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 287 kB/s | 58 kB 00:00 2023-11-16T16:26:58.453 INFO:teuthology.orchestra.run.smithi053.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 1.4 MB/s | 171 kB 00:00 2023-11-16T16:26:58.533 INFO:teuthology.orchestra.run.smithi053.stdout:(127/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 2.2 MB/s | 644 kB 00:00 2023-11-16T16:26:58.537 INFO:teuthology.orchestra.run.smithi026.stdout:(103/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 133 kB/s | 47 kB 00:00 2023-11-16T16:26:58.595 INFO:teuthology.orchestra.run.smithi026.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 160 kB/s | 43 kB 00:00 2023-11-16T16:26:58.654 INFO:teuthology.orchestra.run.smithi026.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.8 MB/s | 220 kB 00:00 2023-11-16T16:26:58.658 INFO:teuthology.orchestra.run.smithi053.stdout:(128/131): NetworkManager-initscripts-updown-1. 471 kB/s | 138 kB 00:00 2023-11-16T16:26:58.713 INFO:teuthology.orchestra.run.smithi026.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 410 kB/s | 48 kB 00:00 2023-11-16T16:26:58.797 INFO:teuthology.orchestra.run.smithi026.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 612 kB/s | 87 kB 00:00 2023-11-16T16:26:58.822 INFO:teuthology.orchestra.run.smithi026.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 392 kB/s | 43 kB 00:00 2023-11-16T16:26:58.867 INFO:teuthology.orchestra.run.smithi053.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 3.7 MB/s | 798 kB 00:00 2023-11-16T16:26:58.947 INFO:teuthology.orchestra.run.smithi026.stdout:(109/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.8 MB/s | 229 kB 00:00 2023-11-16T16:26:58.972 INFO:teuthology.orchestra.run.smithi026.stdout:(110/131): python3-pycparser-2.14-14.el8.noarch 622 kB/s | 109 kB 00:00 2023-11-16T16:26:59.231 INFO:teuthology.orchestra.run.smithi026.stdout:(111/131): libaio-devel-0.3.112-1.el8.x86_64.rp 72 kB/s | 19 kB 00:00 2023-11-16T16:26:59.632 INFO:teuthology.orchestra.run.smithi026.stdout:(112/131): ceph-test-18.0.0-3905.g5e4a743a.el8. 3.5 MB/s | 41 MB 00:11 2023-11-16T16:26:59.648 INFO:teuthology.orchestra.run.smithi026.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 372 kB/s | 155 kB 00:00 2023-11-16T16:26:59.690 INFO:teuthology.orchestra.run.smithi026.stdout:(114/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 754 kB/s | 560 kB 00:00 2023-11-16T16:26:59.774 INFO:teuthology.orchestra.run.smithi026.stdout:(115/131): pcre2-devel-10.32-2.el8.x86_64.rpm 7.1 MB/s | 605 kB 00:00 2023-11-16T16:26:59.791 INFO:teuthology.orchestra.run.smithi026.stdout:(116/131): libverto-devel-0.3.0-5.el8.x86_64.rp 134 kB/s | 18 kB 00:00 2023-11-16T16:26:59.792 INFO:teuthology.orchestra.run.smithi053.stdout:(130/131): librados2-18.0.0-3905.g5e4a743a.el8. 2.7 MB/s | 3.7 MB 00:01 2023-11-16T16:26:59.824 INFO:teuthology.orchestra.run.smithi026.stdout:(117/131): python3-cryptography-3.2.1-5.el8.x86 2.8 MB/s | 559 kB 00:00 2023-11-16T16:27:00.066 INFO:teuthology.orchestra.run.smithi026.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 814 kB/s | 238 kB 00:00 2023-11-16T16:27:00.092 INFO:teuthology.orchestra.run.smithi026.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 271 kB/s | 72 kB 00:00 2023-11-16T16:27:00.158 INFO:teuthology.orchestra.run.smithi026.stdout:(120/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.3 MB/s | 2.3 MB 00:00 2023-11-16T16:27:00.201 INFO:teuthology.orchestra.run.smithi026.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 552 kB/s | 60 kB 00:00 2023-11-16T16:27:00.226 INFO:teuthology.orchestra.run.smithi026.stdout:(122/131): fuse-2.9.7-15.el8.x86_64.rpm 520 kB/s | 83 kB 00:00 2023-11-16T16:27:00.259 INFO:teuthology.orchestra.run.smithi026.stdout:(123/131): libuuid-devel-2.32.1-35.el8.x86_64.r 976 kB/s | 98 kB 00:00 2023-11-16T16:27:00.326 INFO:teuthology.orchestra.run.smithi026.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 385 kB/s | 39 kB 00:00 2023-11-16T16:27:00.343 INFO:teuthology.orchestra.run.smithi026.stdout:(125/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 405 kB/s | 58 kB 00:00 2023-11-16T16:27:00.685 INFO:teuthology.orchestra.run.smithi026.stdout:(126/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 1.5 MB/s | 644 kB 00:00 2023-11-16T16:27:00.710 INFO:teuthology.orchestra.run.smithi026.stdout:(127/131): NetworkManager-initscripts-updown-1. 375 kB/s | 138 kB 00:00 2023-11-16T16:27:00.736 INFO:teuthology.orchestra.run.smithi026.stdout:(128/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 418 kB/s | 171 kB 00:00 2023-11-16T16:27:00.928 INFO:teuthology.orchestra.run.smithi026.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 4.1 MB/s | 798 kB 00:00 2023-11-16T16:27:02.228 INFO:teuthology.orchestra.run.smithi026.stdout:(130/131): librbd1-18.0.0-3905.g5e4a743a.el8.x8 2.3 MB/s | 3.6 MB 00:01 2023-11-16T16:27:02.711 INFO:teuthology.orchestra.run.smithi026.stdout:(131/131): librados2-18.0.0-3905.g5e4a743a.el8. 1.8 MB/s | 3.7 MB 00:02 2023-11-16T16:27:02.715 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2023-11-16T16:27:02.715 INFO:teuthology.orchestra.run.smithi026.stdout:Total 9.0 MB/s | 197 MB 00:21 2023-11-16T16:27:03.546 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2023-11-16T16:27:03.677 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2023-11-16T16:27:03.677 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2023-11-16T16:27:04.092 INFO:teuthology.orchestra.run.smithi053.stdout:(131/131): librbd1-18.0.0-3905.g5e4a743a.el8.x8 656 kB/s | 3.6 MB 00:05 2023-11-16T16:27:04.096 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2023-11-16T16:27:04.096 INFO:teuthology.orchestra.run.smithi053.stdout:Total 7.8 MB/s | 197 MB 00:25 2023-11-16T16:27:04.826 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2023-11-16T16:27:04.831 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2023-11-16T16:27:04.929 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2023-11-16T16:27:05.061 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2023-11-16T16:27:05.062 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2023-11-16T16:27:05.159 INFO:teuthology.orchestra.run.smithi001.stdout:(131/131): librados2-18.0.0-3905.g5e4a743a.el8. 434 kB/s | 3.7 MB 00:08 2023-11-16T16:27:05.162 INFO:teuthology.orchestra.run.smithi001.stdout:-------------------------------------------------------------------------------- 2023-11-16T16:27:05.162 INFO:teuthology.orchestra.run.smithi001.stdout:Total 7.4 MB/s | 197 MB 00:26 2023-11-16T16:27:06.008 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction check 2023-11-16T16:27:06.142 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction check succeeded. 2023-11-16T16:27:06.142 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction test 2023-11-16T16:27:06.199 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2023-11-16T16:27:06.203 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2023-11-16T16:27:07.283 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction test succeeded. 2023-11-16T16:27:07.287 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction 2023-11-16T16:27:10.100 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2023-11-16T16:27:10.290 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-16T16:27:10.476 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-16T16:27:10.685 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-11-16T16:27:10.872 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-16T16:27:10.993 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-16T16:27:11.233 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-16T16:27:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2023-11-16T16:27:11.472 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-16T16:27:11.646 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-16T16:27:11.682 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-16T16:27:11.874 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-11-16T16:27:11.899 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : librados2-2:18.0.0-3905.g5e4a743a.el8.x86_64 6/134 2023-11-16T16:27:12.015 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-16T16:27:12.103 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-16T16:27:12.147 INFO:teuthology.orchestra.run.smithi001.stdout: Preparing : 1/1 2023-11-16T16:27:12.324 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librados2-2:18.0.0-3905.g5e4a743a.el8.x86_64 6/134 2023-11-16T16:27:12.342 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-16T16:27:12.382 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-16T16:27:12.525 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : librbd1-2:18.0.0-3905.g5e4a743a.el8.x86_64 7/134 2023-11-16T16:27:12.539 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-16T16:27:12.665 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librbd1-2:18.0.0-3905.g5e4a743a.el8.x86_64 7/134 2023-11-16T16:27:12.753 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-11-16T16:27:12.796 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-16T16:27:12.841 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephfs2-2:18.0.0-3905.g5e4a743a.el8.x86_64 8/134 2023-11-16T16:27:12.918 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-16T16:27:12.970 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libcephfs2-2:18.0.0-3905.g5e4a743a.el8.x86_64 8/134 2023-11-16T16:27:13.030 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-16T16:27:13.040 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : librados2-2:18.0.0-3905.g5e4a743a.el8.x86_64 6/134 2023-11-16T16:27:13.161 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rados-2:18.0.0-3905.g5e4a743a.el8.x86_64 9/134 2023-11-16T16:27:13.269 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-16T16:27:13.425 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-16T16:27:13.446 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados2-2:18.0.0-3905.g5e4a743a.el8.x86_64 6/134 2023-11-16T16:27:13.629 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-16T16:27:13.657 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : librbd1-2:18.0.0-3905.g5e4a743a.el8.x86_64 7/134 2023-11-16T16:27:13.695 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-16T16:27:13.788 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librbd1-2:18.0.0-3905.g5e4a743a.el8.x86_64 7/134 2023-11-16T16:27:13.901 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : librados2-2:18.0.0-3905.g5e4a743a.el8.x86_64 6/134 2023-11-16T16:27:14.013 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephfs2-2:18.0.0-3905.g5e4a743a.el8.x86_64 8/134 2023-11-16T16:27:14.067 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-16T16:27:14.116 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephfs2-2:18.0.0-3905.g5e4a743a.el8.x86_64 8/134 2023-11-16T16:27:14.292 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rados-2:18.0.0-3905.g5e4a743a.el8.x86_64 9/134 2023-11-16T16:27:14.320 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librados2-2:18.0.0-3905.g5e4a743a.el8.x86_64 6/134 2023-11-16T16:27:14.320 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-16T16:27:14.458 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephsqlite-2:18.0.0-3905.g5e4a743a.el8.x86_64 14/134 2023-11-16T16:27:14.493 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : librbd1-2:18.0.0-3905.g5e4a743a.el8.x86_64 7/134 2023-11-16T16:27:14.507 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-16T16:27:14.566 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libcephsqlite-2:18.0.0-3905.g5e4a743a.el8.x86_64 14/134 2023-11-16T16:27:14.621 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librbd1-2:18.0.0-3905.g5e4a743a.el8.x86_64 7/134 2023-11-16T16:27:14.744 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-16T16:27:14.749 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libradosstriper1-2:18.0.0-3905.g5e4a743a.el8.x86 15/134 2023-11-16T16:27:14.801 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libcephfs2-2:18.0.0-3905.g5e4a743a.el8.x86_64 8/134 2023-11-16T16:27:14.829 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libradosstriper1-2:18.0.0-3905.g5e4a743a.el8.x86 15/134 2023-11-16T16:27:14.925 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libcephfs2-2:18.0.0-3905.g5e4a743a.el8.x86_64 8/134 2023-11-16T16:27:14.957 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-16T16:27:15.108 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-16T16:27:15.117 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rados-2:18.0.0-3905.g5e4a743a.el8.x86_64 9/134 2023-11-16T16:27:15.131 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-16T16:27:15.274 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-16T16:27:15.352 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-16T16:27:15.386 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-16T16:27:15.557 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephsqlite-2:18.0.0-3905.g5e4a743a.el8.x86_64 14/134 2023-11-16T16:27:15.586 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-16T16:27:15.655 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephsqlite-2:18.0.0-3905.g5e4a743a.el8.x86_64 14/134 2023-11-16T16:27:15.832 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libradosstriper1-2:18.0.0-3905.g5e4a743a.el8.x86 15/134 2023-11-16T16:27:15.898 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libradosstriper1-2:18.0.0-3905.g5e4a743a.el8.x86 15/134 2023-11-16T16:27:15.933 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-16T16:27:16.007 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-16T16:27:16.039 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-16T16:27:16.099 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-16T16:27:16.184 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-16T16:27:16.214 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-16T16:27:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-16T16:27:16.376 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libcephsqlite-2:18.0.0-3905.g5e4a743a.el8.x86_64 14/134 2023-11-16T16:27:16.480 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libcephsqlite-2:18.0.0-3905.g5e4a743a.el8.x86_64 14/134 2023-11-16T16:27:16.626 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libradosstriper1-2:18.0.0-3905.g5e4a743a.el8.x86 15/134 2023-11-16T16:27:16.701 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libradosstriper1-2:18.0.0-3905.g5e4a743a.el8.x86 15/134 2023-11-16T16:27:16.730 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-16T16:27:16.817 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-16T16:27:16.954 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-16T16:27:17.033 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-16T16:27:17.165 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-11-16T16:27:17.189 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-16T16:27:17.193 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-16T16:27:17.811 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-16T16:27:17.839 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-16T16:27:17.916 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-11-16T16:27:17.990 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-11-16T16:27:18.009 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-16T16:27:18.181 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-11-16T16:27:18.433 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-11-16T16:27:18.647 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-16T16:27:18.670 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-11-16T16:27:18.762 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-11-16T16:27:18.834 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-11-16T16:27:18.890 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-11-16T16:27:19.013 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-11-16T16:27:19.103 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-11-16T16:27:19.240 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-11-16T16:27:19.390 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-16T16:27:19.452 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-11-16T16:27:19.594 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-11-16T16:27:19.598 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ceph-common-2:18.0.0-3905.g5e4a743a.el8. 26/134 2023-11-16T16:27:19.656 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-11-16T16:27:19.728 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ceph-argparse-2:18.0.0-3905.g5e4a743a.el 27/134 2023-11-16T16:27:19.874 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-11-16T16:27:19.894 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-11-16T16:27:20.092 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-11-16T16:27:20.146 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-16T16:27:20.289 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-11-16T16:27:20.355 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ceph-common-2:18.0.0-3905.g5e4a743a.el8. 26/134 2023-11-16T16:27:20.475 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-11-16T16:27:20.494 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ceph-argparse-2:18.0.0-3905.g5e4a743a.el 27/134 2023-11-16T16:27:20.675 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cephfs-2:18.0.0-3905.g5e4a743a.el8.x86_6 28/134 2023-11-16T16:27:20.686 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-11-16T16:27:21.029 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-11-16T16:27:21.174 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-16T16:27:21.313 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-11-16T16:27:21.375 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-ceph-common-2:18.0.0-3905.g5e4a743a.el8. 26/134 2023-11-16T16:27:21.443 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cephfs-2:18.0.0-3905.g5e4a743a.el8.x86_6 28/134 2023-11-16T16:27:21.513 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-ceph-argparse-2:18.0.0-3905.g5e4a743a.el 27/134 2023-11-16T16:27:21.575 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-11-16T16:27:21.788 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-11-16T16:27:21.915 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-16T16:27:22.307 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-11-16T16:27:22.468 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cephfs-2:18.0.0-3905.g5e4a743a.el8.x86_6 28/134 2023-11-16T16:27:22.567 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-11-16T16:27:22.781 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-16T16:27:22.808 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-11-16T16:27:23.074 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-11-16T16:27:23.324 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-11-16T16:27:23.492 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-16T16:27:23.803 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-16T16:27:24.143 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-11-16T16:27:24.371 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-16T16:27:24.577 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-16T16:27:24.676 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-16T16:27:24.796 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rbd-2:18.0.0-3905.g5e4a743a.el8.x86_64 36/134 2023-11-16T16:27:24.993 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librados-devel-2:18.0.0-3905.g5e4a743a.el8.x86_6 37/134 2023-11-16T16:27:25.017 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-11-16T16:27:25.130 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-11-16T16:27:25.238 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-16T16:27:25.317 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-11-16T16:27:25.363 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-16T16:27:25.418 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-16T16:27:25.534 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-11-16T16:27:25.646 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rbd-2:18.0.0-3905.g5e4a743a.el8.x86_64 36/134 2023-11-16T16:27:25.694 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-11-16T16:27:25.704 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-11-16T16:27:25.810 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librados-devel-2:18.0.0-3905.g5e4a743a.el8.x86_6 37/134 2023-11-16T16:27:25.907 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-16T16:27:25.933 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-11-16T16:27:25.972 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-11-16T16:27:26.113 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-16T16:27:26.132 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-11-16T16:27:26.159 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-11-16T16:27:26.333 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-11-16T16:27:26.357 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rbd-2:18.0.0-3905.g5e4a743a.el8.x86_64 36/134 2023-11-16T16:27:26.486 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-11-16T16:27:26.538 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librados-devel-2:18.0.0-3905.g5e4a743a.el8.x86_6 37/134 2023-11-16T16:27:26.544 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-11-16T16:27:26.701 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-11-16T16:27:26.741 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-11-16T16:27:26.870 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-11-16T16:27:26.885 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-11-16T16:27:26.940 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-11-16T16:27:27.094 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-11-16T16:27:27.109 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-11-16T16:27:27.239 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-11-16T16:27:27.325 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-11-16T16:27:27.326 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-11-16T16:27:27.469 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-11-16T16:27:27.580 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-11-16T16:27:27.601 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-11-16T16:27:27.686 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-11-16T16:27:27.792 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-11-16T16:27:27.902 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-11-16T16:27:28.041 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-11-16T16:27:28.066 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-11-16T16:27:28.106 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-11-16T16:27:28.262 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-11-16T16:27:28.322 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-11-16T16:27:28.363 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-11-16T16:27:28.556 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-11-16T16:27:28.561 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-11-16T16:27:28.604 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-11-16T16:27:28.779 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-11-16T16:27:28.786 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-11-16T16:27:28.805 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-11-16T16:27:28.952 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-11-16T16:27:29.054 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-11-16T16:27:29.065 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-11-16T16:27:29.263 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-11-16T16:27:29.370 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-11-16T16:27:29.397 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-11-16T16:27:29.444 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-11-16T16:27:29.558 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-11-16T16:27:29.660 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-11-16T16:27:29.719 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-11-16T16:27:29.778 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-11-16T16:27:29.980 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-11-16T16:27:29.991 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-11-16T16:27:30.012 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-11-16T16:27:30.155 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-11-16T16:27:30.237 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-11-16T16:27:30.286 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-11-16T16:27:30.321 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-11-16T16:27:30.438 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-11-16T16:27:30.530 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-11-16T16:27:30.591 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-11-16T16:27:30.659 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-11-16T16:27:30.763 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-11-16T16:27:30.770 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-11-16T16:27:30.854 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-11-16T16:27:30.897 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-11-16T16:27:30.937 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-11-16T16:27:31.090 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-11-16T16:27:31.097 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-11-16T16:27:31.133 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-11-16T16:27:31.313 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-11-16T16:27:31.332 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-11-16T16:27:31.335 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-11-16T16:27:31.385 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-11-16T16:27:31.488 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-16T16:27:31.490 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-11-16T16:27:31.558 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-16T16:27:31.618 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-11-16T16:27:31.644 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-11-16T16:27:31.799 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-11-16T16:27:31.806 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-11-16T16:27:31.817 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-11-16T16:27:31.886 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-11-16T16:27:32.004 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-16T16:27:32.089 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-16T16:27:32.141 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-11-16T16:27:32.153 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-11-16T16:27:32.290 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-16T16:27:32.320 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-11-16T16:27:32.321 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-11-16T16:27:32.458 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-16T16:27:32.471 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-16T16:27:32.515 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-11-16T16:27:32.565 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-16T16:27:32.680 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-16T16:27:32.758 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-11-16T16:27:32.769 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-16T16:27:32.869 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-16T16:27:32.918 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-11-16T16:27:33.021 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-16T16:27:33.076 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-16T16:27:33.150 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-16T16:27:33.168 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-16T16:27:33.290 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-16T16:27:33.345 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-16T16:27:33.387 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-16T16:27:33.527 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-16T16:27:33.542 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-16T16:27:33.573 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-16T16:27:33.728 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-16T16:27:33.734 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-16T16:27:33.774 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-16T16:27:33.921 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-16T16:27:33.942 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-16T16:27:33.944 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-16T16:27:34.088 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-16T16:27:34.100 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-16T16:27:34.136 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-16T16:27:34.270 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-16T16:27:34.313 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-16T16:27:34.343 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-16T16:27:34.503 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-16T16:27:34.532 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-16T16:27:34.532 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-16T16:27:34.678 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-16T16:27:34.738 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-16T16:27:34.751 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-16T16:27:34.911 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-16T16:27:34.927 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-16T16:27:34.935 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-16T16:27:35.047 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-16T16:27:35.176 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-16T16:27:35.192 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-16T16:27:35.281 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-16T16:27:35.402 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-16T16:27:35.458 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-16T16:27:35.529 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-16T16:27:35.630 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-16T16:27:35.649 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-16T16:27:35.731 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-16T16:27:35.854 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-16T16:27:35.865 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-16T16:27:35.997 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-16T16:27:36.055 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-16T16:27:36.266 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-16T16:27:36.275 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-16T16:27:36.442 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-16T16:27:36.671 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-16T16:27:37.034 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-16T16:27:37.310 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-16T16:27:37.468 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-16T16:27:37.581 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-16T16:27:37.766 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-16T16:27:37.851 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-16T16:27:37.933 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-16T16:27:38.099 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-16T16:27:38.105 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-16T16:27:38.187 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-16T16:27:38.368 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-16T16:27:38.413 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-16T16:27:38.459 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-16T16:27:38.686 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-16T16:27:38.722 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-16T16:27:38.955 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-16T16:27:39.047 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-16T16:27:39.196 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-16T16:27:39.234 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-16T16:27:39.362 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librgw2-2:18.0.0-3905.g5e4a743a.el8.x86_64 92/134 2023-11-16T16:27:39.426 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librgw2-2:18.0.0-3905.g5e4a743a.el8.x86_64 92/134 2023-11-16T16:27:39.621 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 93/134 2023-11-16T16:27:39.748 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-16T16:27:39.945 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librgw2-2:18.0.0-3905.g5e4a743a.el8.x86_64 92/134 2023-11-16T16:27:40.023 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-16T16:27:40.024 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librgw2-2:18.0.0-3905.g5e4a743a.el8.x86_64 92/134 2023-11-16T16:27:40.187 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 93/134 2023-11-16T16:27:40.206 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librgw2-2:18.0.0-3905.g5e4a743a.el8.x86_64 92/134 2023-11-16T16:27:40.317 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librgw2-2:18.0.0-3905.g5e4a743a.el8.x86_64 92/134 2023-11-16T16:27:40.490 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 93/134 2023-11-16T16:27:43.500 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-common-2:18.0.0-3905.g5e4a743a.el8.x86_64 94/134 2023-11-16T16:27:43.762 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-common-2:18.0.0-3905.g5e4a743a.el8.x86_64 94/134 2023-11-16T16:27:44.013 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-common-2:18.0.0-3905.g5e4a743a.el8.x86_64 94/134 2023-11-16T16:27:44.245 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-common-2:18.0.0-3905.g5e4a743a.el8.x86_64 94/134 2023-11-16T16:27:44.294 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-common-2:18.0.0-3905.g5e4a743a.el8.x86_64 94/134 2023-11-16T16:27:44.406 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-common-2:18.0.0-3905.g5e4a743a.el8.x86_64 94/134 2023-11-16T16:27:44.515 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-common-2:18.0.0-3905.g5e4a743a.el8.x86_64 94/134 2023-11-16T16:27:44.687 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-base-2:18.0.0-3905.g5e4a743a.el8.x86_64 95/134 2023-11-16T16:27:44.769 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-base-2:18.0.0-3905.g5e4a743a.el8.x86_64 95/134 2023-11-16T16:27:44.872 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-common-2:18.0.0-3905.g5e4a743a.el8.x86_64 94/134 2023-11-16T16:27:44.912 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-selinux-2:18.0.0-3905.g5e4a743a.el8.x86_64 96/134 2023-11-16T16:27:45.138 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-common-2:18.0.0-3905.g5e4a743a.el8.x86_64 94/134 2023-11-16T16:27:45.161 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-base-2:18.0.0-3905.g5e4a743a.el8.x86_64 95/134 2023-11-16T16:27:45.252 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-base-2:18.0.0-3905.g5e4a743a.el8.x86_64 95/134 2023-11-16T16:27:45.378 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-selinux-2:18.0.0-3905.g5e4a743a.el8.x86_64 96/134 2023-11-16T16:27:45.421 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-base-2:18.0.0-3905.g5e4a743a.el8.x86_64 95/134 2023-11-16T16:27:45.504 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-base-2:18.0.0-3905.g5e4a743a.el8.x86_64 95/134 2023-11-16T16:27:45.673 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-selinux-2:18.0.0-3905.g5e4a743a.el8.x86_64 96/134 2023-11-16T16:28:04.357 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-selinux-2:18.0.0-3905.g5e4a743a.el8.x86_64 96/134 2023-11-16T16:28:04.357 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /sys 2023-11-16T16:28:04.357 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /proc 2023-11-16T16:28:04.357 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /dev 2023-11-16T16:28:04.357 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /run 2023-11-16T16:28:04.357 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /mnt 2023-11-16T16:28:04.357 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /var/tmp 2023-11-16T16:28:04.357 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /home 2023-11-16T16:28:04.358 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /tmp 2023-11-16T16:28:04.358 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /dev 2023-11-16T16:28:04.358 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-11-16T16:28:04.742 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-volume-2:18.0.0-3905.g5e4a743a.el8.noarch 97/134 2023-11-16T16:28:04.750 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-selinux-2:18.0.0-3905.g5e4a743a.el8.x86_64 96/134 2023-11-16T16:28:04.750 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /sys 2023-11-16T16:28:04.751 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /proc 2023-11-16T16:28:04.751 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /dev 2023-11-16T16:28:04.751 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /run 2023-11-16T16:28:04.751 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /mnt 2023-11-16T16:28:04.751 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /var/tmp 2023-11-16T16:28:04.751 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /home 2023-11-16T16:28:04.751 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /tmp 2023-11-16T16:28:04.751 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /dev 2023-11-16T16:28:04.751 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-16T16:28:05.080 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-selinux-2:18.0.0-3905.g5e4a743a.el8.x86_64 96/134 2023-11-16T16:28:05.080 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /sys 2023-11-16T16:28:05.080 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /proc 2023-11-16T16:28:05.080 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /dev 2023-11-16T16:28:05.081 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /run 2023-11-16T16:28:05.081 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /mnt 2023-11-16T16:28:05.081 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /var/tmp 2023-11-16T16:28:05.081 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /home 2023-11-16T16:28:05.081 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /tmp 2023-11-16T16:28:05.081 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /dev 2023-11-16T16:28:05.081 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:28:05.134 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-volume-2:18.0.0-3905.g5e4a743a.el8.noarch 97/134 2023-11-16T16:28:05.420 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-volume-2:18.0.0-3905.g5e4a743a.el8.noarch 97/134 2023-11-16T16:28:06.405 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-volume-2:18.0.0-3905.g5e4a743a.el8.noarch 97/134 2023-11-16T16:28:06.567 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-osd-2:18.0.0-3905.g5e4a743a.el8.x86_64 98/134 2023-11-16T16:28:06.797 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-volume-2:18.0.0-3905.g5e4a743a.el8.noarch 97/134 2023-11-16T16:28:06.798 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-osd-2:18.0.0-3905.g5e4a743a.el8.x86_64 98/134 2023-11-16T16:28:06.958 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-osd-2:18.0.0-3905.g5e4a743a.el8.x86_64 98/134 2023-11-16T16:28:06.966 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mds-2:18.0.0-3905.g5e4a743a.el8.x86_64 99/134 2023-11-16T16:28:07.083 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-volume-2:18.0.0-3905.g5e4a743a.el8.noarch 97/134 2023-11-16T16:28:07.191 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-osd-2:18.0.0-3905.g5e4a743a.el8.x86_64 98/134 2023-11-16T16:28:07.277 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-osd-2:18.0.0-3905.g5e4a743a.el8.x86_64 98/134 2023-11-16T16:28:07.375 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mds-2:18.0.0-3905.g5e4a743a.el8.x86_64 99/134 2023-11-16T16:28:07.445 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mds-2:18.0.0-3905.g5e4a743a.el8.x86_64 99/134 2023-11-16T16:28:07.507 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-osd-2:18.0.0-3905.g5e4a743a.el8.x86_64 98/134 2023-11-16T16:28:07.625 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mon-2:18.0.0-3905.g5e4a743a.el8.x86_64 100/134 2023-11-16T16:28:07.643 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mon-2:18.0.0-3905.g5e4a743a.el8.x86_64 100/134 2023-11-16T16:28:07.686 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mds-2:18.0.0-3905.g5e4a743a.el8.x86_64 99/134 2023-11-16T16:28:07.854 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mds-2:18.0.0-3905.g5e4a743a.el8.x86_64 99/134 2023-11-16T16:28:08.008 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mon-2:18.0.0-3905.g5e4a743a.el8.x86_64 100/134 2023-11-16T16:28:08.026 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mon-2:18.0.0-3905.g5e4a743a.el8.x86_64 100/134 2023-11-16T16:28:08.164 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mds-2:18.0.0-3905.g5e4a743a.el8.x86_64 99/134 2023-11-16T16:28:08.312 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mon-2:18.0.0-3905.g5e4a743a.el8.x86_64 100/134 2023-11-16T16:28:08.330 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mon-2:18.0.0-3905.g5e4a743a.el8.x86_64 100/134 2023-11-16T16:28:09.442 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: cephadm-2:18.0.0-3905.g5e4a743a.el8.noarch 101/134 2023-11-16T16:28:09.652 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephadm-2:18.0.0-3905.g5e4a743a.el8.noarch 101/134 2023-11-16T16:28:09.772 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-3905.g5e4a743a.e 102/134 2023-11-16T16:28:09.966 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-3905.g5e4a743a. 103/134 2023-11-16T16:28:10.024 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cephadm-2:18.0.0-3905.g5e4a743a.el8.noarch 101/134 2023-11-16T16:28:10.176 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: cephadm-2:18.0.0-3905.g5e4a743a.el8.noarch 101/134 2023-11-16T16:28:10.182 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-3905.g5e4a743a.el8.noa 104/134 2023-11-16T16:28:10.252 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephadm-2:18.0.0-3905.g5e4a743a.el8.noarch 101/134 2023-11-16T16:28:10.364 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-3905.g5e4a743a.e 102/134 2023-11-16T16:28:10.371 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : cephadm-2:18.0.0-3905.g5e4a743a.el8.noarch 101/134 2023-11-16T16:28:10.402 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-3905.g5e4a743a.el8.noa 104/134 2023-11-16T16:28:10.476 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-3905.g5e4a743a.e 102/134 2023-11-16T16:28:10.573 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-3905.g5e4a743a. 103/134 2023-11-16T16:28:10.677 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-3905.g5e4a743a. 103/134 2023-11-16T16:28:10.699 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-3905.g5e4a743a.el8.n 105/134 2023-11-16T16:28:10.808 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-3905.g5e4a743a.el8.noa 104/134 2023-11-16T16:28:10.877 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-3905.g5e4a743a.el8.noa 104/134 2023-11-16T16:28:11.041 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-3905.g5e4a743a.el8.noa 104/134 2023-11-16T16:28:11.099 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-3905.g5e4a743a.el8.noa 104/134 2023-11-16T16:28:11.266 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-3905.g5e4a743a.el8.n 105/134 2023-11-16T16:28:11.350 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-3905.g5e4a743a.el8.n 105/134 2023-11-16T16:28:11.724 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-3905.g5e4a743a.el8.n 105/134 2023-11-16T16:28:11.890 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-3905.g5e4 106/134 2023-11-16T16:28:11.986 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-3905.g5e4 106/134 2023-11-16T16:28:12.294 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-3905.g5e4a743a.el8.n 105/134 2023-11-16T16:28:12.374 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-3905.g5e4a743a.el8.n 105/134 2023-11-16T16:28:12.378 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-3905.g5e4a743a.el 107/134 2023-11-16T16:28:12.491 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-3905.g5e4 106/134 2023-11-16T16:28:12.552 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-3905.g5e4 106/134 2023-11-16T16:28:12.589 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-3905.g5e4 106/134 2023-11-16T16:28:12.629 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-2:18.0.0-3905.g5e4a743a.el8.x86_64 108/134 2023-11-16T16:28:12.649 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-3905.g5e4 106/134 2023-11-16T16:28:12.666 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-2:18.0.0-3905.g5e4a743a.el8.x86_64 108/134 2023-11-16T16:28:12.823 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-rook-2:18.0.0-3905.g5e4a743a.el8.noarch 109/134 2023-11-16T16:28:12.835 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-3905.g5e4a743a.el8.noarch 109/134 2023-11-16T16:28:12.953 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-3905.g5e4a743a.el 107/134 2023-11-16T16:28:12.996 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-3905.g5e4a743a.el 107/134 2023-11-16T16:28:13.185 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-2:18.0.0-3905.g5e4a743a.el8.x86_64 108/134 2023-11-16T16:28:13.220 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-2:18.0.0-3905.g5e4a743a.el8.x86_64 108/134 2023-11-16T16:28:13.222 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-2:18.0.0-3905.g5e4a743a.el8.x86_64 108/134 2023-11-16T16:28:13.256 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-2:18.0.0-3905.g5e4a743a.el8.x86_64 108/134 2023-11-16T16:28:13.383 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-rook-2:18.0.0-3905.g5e4a743a.el8.noarch 109/134 2023-11-16T16:28:13.394 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-3905.g5e4a743a.el8.noarch 109/134 2023-11-16T16:28:13.411 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-rook-2:18.0.0-3905.g5e4a743a.el8.noarch 109/134 2023-11-16T16:28:13.422 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-3905.g5e4a743a.el8.noarch 109/134 2023-11-16T16:28:14.197 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-2:18.0.0-3905.g5e4a743a.el8.x86_64 110/134 2023-11-16T16:28:14.378 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-radosgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 111/134 2023-11-16T16:28:14.412 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 111/134 2023-11-16T16:28:14.582 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-3905.g5e4a7 112/134 2023-11-16T16:28:14.631 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-3905.g5e4a7 112/134 2023-11-16T16:28:14.754 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-2:18.0.0-3905.g5e4a743a.el8.x86_64 110/134 2023-11-16T16:28:14.761 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-2:18.0.0-3905.g5e4a743a.el8.x86_64 110/134 2023-11-16T16:28:14.773 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephfs-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 113/134 2023-11-16T16:28:14.927 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-radosgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 111/134 2023-11-16T16:28:14.961 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-radosgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 111/134 2023-11-16T16:28:14.962 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 111/134 2023-11-16T16:28:14.995 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 111/134 2023-11-16T16:28:15.127 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-3905.g5e4a7 112/134 2023-11-16T16:28:15.139 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 113/134 2023-11-16T16:28:15.157 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-3905.g5e4a7 112/134 2023-11-16T16:28:15.178 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-3905.g5e4a7 112/134 2023-11-16T16:28:15.207 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-3905.g5e4a7 112/134 2023-11-16T16:28:15.307 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 114/134 2023-11-16T16:28:15.310 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : cephfs-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 113/134 2023-11-16T16:28:15.399 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephfs-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 113/134 2023-11-16T16:28:15.677 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 113/134 2023-11-16T16:28:15.765 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 113/134 2023-11-16T16:28:15.886 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : rbd-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 114/134 2023-11-16T16:28:15.999 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 114/134 2023-11-16T16:28:19.469 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: rbd-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 114/134 2023-11-16T16:28:19.748 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-test-2:18.0.0-3905.g5e4a743a.el8.x86_64 115/134 2023-11-16T16:28:19.930 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-11-16T16:28:20.060 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: rbd-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 114/134 2023-11-16T16:28:20.144 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-11-16T16:28:20.177 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: rbd-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 114/134 2023-11-16T16:28:20.323 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-test-2:18.0.0-3905.g5e4a743a.el8.x86_64 115/134 2023-11-16T16:28:20.476 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-11-16T16:28:20.515 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-test-2:18.0.0-3905.g5e4a743a.el8.x86_64 115/134 2023-11-16T16:28:20.648 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-11-16T16:28:20.687 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-11-16T16:28:20.857 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-11-16T16:28:21.090 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-11-16T16:28:21.442 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-11-16T16:28:21.653 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-11-16T16:28:21.684 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-fuse-2:18.0.0-3905.g5e4a743a.el8.x86_64 119/134 2023-11-16T16:28:21.843 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-16T16:28:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-11-16T16:28:21.971 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-11-16T16:28:22.035 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephfs-devel-2:18.0.0-3905.g5e4a743a.el8.x86_ 121/134 2023-11-16T16:28:22.181 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-16T16:28:22.184 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-11-16T16:28:22.189 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-fuse-2:18.0.0-3905.g5e4a743a.el8.x86_64 119/134 2023-11-16T16:28:22.364 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-16T16:28:22.402 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-fuse-2:18.0.0-3905.g5e4a743a.el8.x86_64 119/134 2023-11-16T16:28:22.539 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libcephfs-devel-2:18.0.0-3905.g5e4a743a.el8.x86_ 121/134 2023-11-16T16:28:22.594 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-16T16:28:22.686 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-16T16:28:22.778 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephfs-devel-2:18.0.0-3905.g5e4a743a.el8.x86_ 121/134 2023-11-16T16:28:22.932 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-16T16:28:24.543 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-16T16:28:24.755 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-16T16:28:24.773 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-16T16:28:24.964 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephfs-top-2:18.0.0-3905.g5e4a743a.el8.noarch 124/134 2023-11-16T16:28:25.027 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-16T16:28:25.259 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-16T16:28:25.277 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-16T16:28:25.305 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-fuse-2:18.0.0-3905.g5e4a743a.el8.x86_64 125/134 2023-11-16T16:28:25.336 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-16T16:28:25.427 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : cephfs-top-2:18.0.0-3905.g5e4a743a.el8.noarch 124/134 2023-11-16T16:28:25.589 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-16T16:28:25.608 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-16T16:28:25.631 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-nbd-2:18.0.0-3905.g5e4a743a.el8.x86_64 126/134 2023-11-16T16:28:25.760 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : rbd-fuse-2:18.0.0-3905.g5e4a743a.el8.x86_64 125/134 2023-11-16T16:28:25.783 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephfs-top-2:18.0.0-3905.g5e4a743a.el8.noarch 124/134 2023-11-16T16:28:25.965 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-16T16:28:26.031 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-fuse-2:18.0.0-3905.g5e4a743a.el8.x86_64 125/134 2023-11-16T16:28:26.073 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : rbd-nbd-2:18.0.0-3905.g5e4a743a.el8.x86_64 126/134 2023-11-16T16:28:26.105 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-16T16:28:26.114 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-16T16:28:26.348 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-11-16T16:28:26.352 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-16T16:28:26.374 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-nbd-2:18.0.0-3905.g5e4a743a.el8.x86_64 126/134 2023-11-16T16:28:26.517 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-16T16:28:26.526 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-16T16:28:26.562 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-11-16T16:28:26.648 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-11-16T16:28:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-16T16:28:26.708 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-11-16T16:28:26.838 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-11-16T16:28:26.851 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-11-16T16:28:26.851 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-16T16:28:26.856 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-16T16:28:26.865 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-16T16:28:26.875 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-11-16T16:28:26.965 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-11-16T16:28:27.057 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-11-16T16:28:27.083 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-16T16:28:27.083 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-16T16:28:27.201 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-11-16T16:28:27.215 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-11-16T16:28:27.215 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-16T16:28:27.255 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-11-16T16:28:27.302 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-16T16:28:27.368 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-11-16T16:28:27.421 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-16T16:28:27.422 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-16T16:28:27.512 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-16T16:28:27.572 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-11-16T16:28:27.585 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-11-16T16:28:27.585 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-16T16:28:27.649 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-16T16:28:27.792 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-16T16:28:27.793 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-16T16:28:27.875 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-16T16:28:28.061 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-16T16:28:28.061 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-2:18.0.0-3905.g5e4a743a.el8.x86_64 1/134 2023-11-16T16:28:28.061 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-base-2:18.0.0-3905.g5e4a743a.el8.x86_64 2/134 2023-11-16T16:28:28.061 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-common-2:18.0.0-3905.g5e4a743a.el8.x86_64 3/134 2023-11-16T16:28:28.061 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-fuse-2:18.0.0-3905.g5e4a743a.el8.x86_64 4/134 2023-11-16T16:28:28.061 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-3905.g5e4a7 5/134 2023-11-16T16:28:28.061 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mds-2:18.0.0-3905.g5e4a743a.el8.x86_64 6/134 2023-11-16T16:28:28.061 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-2:18.0.0-3905.g5e4a743a.el8.x86_64 7/134 2023-11-16T16:28:28.061 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mon-2:18.0.0-3905.g5e4a743a.el8.x86_64 8/134 2023-11-16T16:28:28.061 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-osd-2:18.0.0-3905.g5e4a743a.el8.x86_64 9/134 2023-11-16T16:28:28.062 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-radosgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 10/134 2023-11-16T16:28:28.062 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-selinux-2:18.0.0-3905.g5e4a743a.el8.x86_64 11/134 2023-11-16T16:28:28.062 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-test-2:18.0.0-3905.g5e4a743a.el8.x86_64 12/134 2023-11-16T16:28:28.062 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephfs-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 13/134 2023-11-16T16:28:28.062 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephfs-devel-2:18.0.0-3905.g5e4a743a.el8.x86_ 14/134 2023-11-16T16:28:28.062 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephfs2-2:18.0.0-3905.g5e4a743a.el8.x86_64 15/134 2023-11-16T16:28:28.062 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephsqlite-2:18.0.0-3905.g5e4a743a.el8.x86_64 16/134 2023-11-16T16:28:28.062 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados-devel-2:18.0.0-3905.g5e4a743a.el8.x86_6 17/134 2023-11-16T16:28:28.062 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libradosstriper1-2:18.0.0-3905.g5e4a743a.el8.x86 18/134 2023-11-16T16:28:28.062 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librgw2-2:18.0.0-3905.g5e4a743a.el8.x86_64 19/134 2023-11-16T16:28:28.062 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ceph-argparse-2:18.0.0-3905.g5e4a743a.el 20/134 2023-11-16T16:28:28.063 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ceph-common-2:18.0.0-3905.g5e4a743a.el8. 21/134 2023-11-16T16:28:28.063 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cephfs-2:18.0.0-3905.g5e4a743a.el8.x86_6 22/134 2023-11-16T16:28:28.063 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rados-2:18.0.0-3905.g5e4a743a.el8.x86_64 23/134 2023-11-16T16:28:28.063 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rbd-2:18.0.0-3905.g5e4a743a.el8.x86_64 24/134 2023-11-16T16:28:28.063 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 25/134 2023-11-16T16:28:28.063 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-fuse-2:18.0.0-3905.g5e4a743a.el8.x86_64 26/134 2023-11-16T16:28:28.063 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 27/134 2023-11-16T16:28:28.063 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-nbd-2:18.0.0-3905.g5e4a743a.el8.x86_64 28/134 2023-11-16T16:28:28.063 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-3905.g5e4a743a. 29/134 2023-11-16T16:28:28.063 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-3905.g5e4a743a.el8.noa 30/134 2023-11-16T16:28:28.063 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-3905.g5e4a743a.el8.n 31/134 2023-11-16T16:28:28.064 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-3905.g5e4 32/134 2023-11-16T16:28:28.064 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-3905.g5e4a743a.el 33/134 2023-11-16T16:28:28.064 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-rook-2:18.0.0-3905.g5e4a743a.el8.noarch 34/134 2023-11-16T16:28:28.064 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-3905.g5e4a743a.e 35/134 2023-11-16T16:28:28.064 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-volume-2:18.0.0-3905.g5e4a743a.el8.noarch 36/134 2023-11-16T16:28:28.064 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephadm-2:18.0.0-3905.g5e4a743a.el8.noarch 37/134 2023-11-16T16:28:28.064 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephfs-top-2:18.0.0-3905.g5e4a743a.el8.noarch 38/134 2023-11-16T16:28:28.064 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-16T16:28:28.064 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-16T16:28:28.064 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-16T16:28:28.064 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-16T16:28:28.064 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-16T16:28:28.064 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-16T16:28:28.065 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-16T16:28:28.065 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-16T16:28:28.065 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-16T16:28:28.065 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-16T16:28:28.065 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-16T16:28:28.065 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-16T16:28:28.065 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-16T16:28:28.065 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-16T16:28:28.065 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-16T16:28:28.065 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-16T16:28:28.065 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-16T16:28:28.066 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-16T16:28:28.066 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-16T16:28:28.066 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-16T16:28:28.066 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-16T16:28:28.066 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-16T16:28:28.066 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-16T16:28:28.066 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-16T16:28:28.066 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-16T16:28:28.066 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-16T16:28:28.066 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-16T16:28:28.067 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-16T16:28:28.067 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-16T16:28:28.067 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-16T16:28:28.067 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-16T16:28:28.067 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-16T16:28:28.067 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-16T16:28:28.067 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-11-16T16:28:28.067 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-11-16T16:28:28.067 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-11-16T16:28:28.067 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-11-16T16:28:28.067 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-11-16T16:28:28.067 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-11-16T16:28:28.067 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-11-16T16:28:28.068 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-11-16T16:28:28.068 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-11-16T16:28:28.068 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-11-16T16:28:28.068 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-11-16T16:28:28.068 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-11-16T16:28:28.068 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-11-16T16:28:28.068 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-11-16T16:28:28.068 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-11-16T16:28:28.068 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-11-16T16:28:28.068 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-11-16T16:28:28.068 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-11-16T16:28:28.068 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-11-16T16:28:28.069 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-11-16T16:28:28.069 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-11-16T16:28:28.069 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-11-16T16:28:28.069 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-11-16T16:28:28.069 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-11-16T16:28:28.069 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-11-16T16:28:28.069 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-11-16T16:28:28.069 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-11-16T16:28:28.069 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-11-16T16:28:28.069 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-11-16T16:28:28.069 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-11-16T16:28:28.070 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-11-16T16:28:28.070 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-11-16T16:28:28.070 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-11-16T16:28:28.070 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-11-16T16:28:28.070 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-11-16T16:28:28.070 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-11-16T16:28:28.070 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-11-16T16:28:28.070 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-11-16T16:28:28.070 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-11-16T16:28:28.070 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-11-16T16:28:28.070 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-11-16T16:28:28.070 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-11-16T16:28:28.071 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-11-16T16:28:28.071 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-11-16T16:28:28.071 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-11-16T16:28:28.071 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-11-16T16:28:28.071 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-11-16T16:28:28.071 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-11-16T16:28:28.071 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-16T16:28:28.071 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-11-16T16:28:28.071 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-16T16:28:28.071 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-11-16T16:28:28.071 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-11-16T16:28:28.071 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-11-16T16:28:28.071 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-11-16T16:28:28.071 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-11-16T16:28:28.072 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-16T16:28:28.072 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados2-2:18.0.0-3905.g5e4a743a.el8.x86_64 129/134 2023-11-16T16:28:28.072 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-16T16:28:28.072 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librbd1-2:18.0.0-3905.g5e4a743a.el8.x86_64 131/134 2023-11-16T16:28:28.072 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-16T16:28:28.072 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-16T16:28:28.072 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-16T16:28:28.280 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-16T16:28:28.411 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-16T16:28:28.411 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-2:18.0.0-3905.g5e4a743a.el8.x86_64 1/134 2023-11-16T16:28:28.411 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-base-2:18.0.0-3905.g5e4a743a.el8.x86_64 2/134 2023-11-16T16:28:28.412 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-common-2:18.0.0-3905.g5e4a743a.el8.x86_64 3/134 2023-11-16T16:28:28.412 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-fuse-2:18.0.0-3905.g5e4a743a.el8.x86_64 4/134 2023-11-16T16:28:28.412 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-3905.g5e4a7 5/134 2023-11-16T16:28:28.412 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mds-2:18.0.0-3905.g5e4a743a.el8.x86_64 6/134 2023-11-16T16:28:28.412 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-2:18.0.0-3905.g5e4a743a.el8.x86_64 7/134 2023-11-16T16:28:28.412 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mon-2:18.0.0-3905.g5e4a743a.el8.x86_64 8/134 2023-11-16T16:28:28.412 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-osd-2:18.0.0-3905.g5e4a743a.el8.x86_64 9/134 2023-11-16T16:28:28.412 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-radosgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 10/134 2023-11-16T16:28:28.412 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-selinux-2:18.0.0-3905.g5e4a743a.el8.x86_64 11/134 2023-11-16T16:28:28.412 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-test-2:18.0.0-3905.g5e4a743a.el8.x86_64 12/134 2023-11-16T16:28:28.412 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : cephfs-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 13/134 2023-11-16T16:28:28.412 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libcephfs-devel-2:18.0.0-3905.g5e4a743a.el8.x86_ 14/134 2023-11-16T16:28:28.412 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libcephfs2-2:18.0.0-3905.g5e4a743a.el8.x86_64 15/134 2023-11-16T16:28:28.413 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libcephsqlite-2:18.0.0-3905.g5e4a743a.el8.x86_64 16/134 2023-11-16T16:28:28.413 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librados-devel-2:18.0.0-3905.g5e4a743a.el8.x86_6 17/134 2023-11-16T16:28:28.413 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libradosstriper1-2:18.0.0-3905.g5e4a743a.el8.x86 18/134 2023-11-16T16:28:28.413 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librgw2-2:18.0.0-3905.g5e4a743a.el8.x86_64 19/134 2023-11-16T16:28:28.413 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-ceph-argparse-2:18.0.0-3905.g5e4a743a.el 20/134 2023-11-16T16:28:28.413 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-ceph-common-2:18.0.0-3905.g5e4a743a.el8. 21/134 2023-11-16T16:28:28.413 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cephfs-2:18.0.0-3905.g5e4a743a.el8.x86_6 22/134 2023-11-16T16:28:28.413 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rados-2:18.0.0-3905.g5e4a743a.el8.x86_64 23/134 2023-11-16T16:28:28.413 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rbd-2:18.0.0-3905.g5e4a743a.el8.x86_64 24/134 2023-11-16T16:28:28.413 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 25/134 2023-11-16T16:28:28.413 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : rbd-fuse-2:18.0.0-3905.g5e4a743a.el8.x86_64 26/134 2023-11-16T16:28:28.413 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : rbd-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 27/134 2023-11-16T16:28:28.413 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : rbd-nbd-2:18.0.0-3905.g5e4a743a.el8.x86_64 28/134 2023-11-16T16:28:28.414 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-3905.g5e4a743a. 29/134 2023-11-16T16:28:28.414 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-3905.g5e4a743a.el8.noa 30/134 2023-11-16T16:28:28.414 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-3905.g5e4a743a.el8.n 31/134 2023-11-16T16:28:28.414 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-3905.g5e4 32/134 2023-11-16T16:28:28.414 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-3905.g5e4a743a.el 33/134 2023-11-16T16:28:28.414 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-rook-2:18.0.0-3905.g5e4a743a.el8.noarch 34/134 2023-11-16T16:28:28.414 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-3905.g5e4a743a.e 35/134 2023-11-16T16:28:28.414 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-volume-2:18.0.0-3905.g5e4a743a.el8.noarch 36/134 2023-11-16T16:28:28.414 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : cephadm-2:18.0.0-3905.g5e4a743a.el8.noarch 37/134 2023-11-16T16:28:28.414 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : cephfs-top-2:18.0.0-3905.g5e4a743a.el8.noarch 38/134 2023-11-16T16:28:28.414 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-16T16:28:28.415 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-16T16:28:28.415 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-16T16:28:28.415 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-16T16:28:28.415 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-16T16:28:28.415 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-16T16:28:28.415 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-16T16:28:28.415 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-16T16:28:28.415 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-16T16:28:28.415 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-16T16:28:28.415 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-16T16:28:28.416 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-16T16:28:28.416 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-16T16:28:28.416 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-16T16:28:28.416 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-16T16:28:28.416 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-16T16:28:28.416 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-16T16:28:28.416 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-16T16:28:28.416 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-16T16:28:28.416 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-16T16:28:28.416 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-16T16:28:28.416 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-16T16:28:28.416 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-16T16:28:28.416 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-16T16:28:28.417 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-16T16:28:28.417 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-16T16:28:28.417 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-16T16:28:28.417 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-16T16:28:28.417 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-16T16:28:28.417 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-16T16:28:28.417 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-16T16:28:28.417 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-16T16:28:28.417 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-16T16:28:28.417 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-11-16T16:28:28.417 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-11-16T16:28:28.417 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-11-16T16:28:28.417 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-11-16T16:28:28.418 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-11-16T16:28:28.418 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-11-16T16:28:28.418 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-11-16T16:28:28.418 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-11-16T16:28:28.418 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-11-16T16:28:28.418 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-11-16T16:28:28.418 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-11-16T16:28:28.418 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-11-16T16:28:28.418 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-11-16T16:28:28.418 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-11-16T16:28:28.418 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-11-16T16:28:28.418 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-11-16T16:28:28.419 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-11-16T16:28:28.419 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-11-16T16:28:28.419 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-11-16T16:28:28.419 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-11-16T16:28:28.419 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-11-16T16:28:28.419 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-11-16T16:28:28.419 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-11-16T16:28:28.419 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-11-16T16:28:28.419 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-11-16T16:28:28.419 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-11-16T16:28:28.419 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-11-16T16:28:28.419 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-11-16T16:28:28.419 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-11-16T16:28:28.420 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-11-16T16:28:28.420 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-11-16T16:28:28.420 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-11-16T16:28:28.420 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-11-16T16:28:28.420 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-11-16T16:28:28.420 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-11-16T16:28:28.420 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-11-16T16:28:28.420 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-11-16T16:28:28.420 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-11-16T16:28:28.420 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-11-16T16:28:28.420 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-11-16T16:28:28.420 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-11-16T16:28:28.421 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-11-16T16:28:28.421 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-11-16T16:28:28.421 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-11-16T16:28:28.421 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-11-16T16:28:28.421 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-11-16T16:28:28.421 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-11-16T16:28:28.421 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-11-16T16:28:28.421 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-16T16:28:28.421 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-11-16T16:28:28.421 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-16T16:28:28.421 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-11-16T16:28:28.421 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-11-16T16:28:28.422 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-11-16T16:28:28.422 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-11-16T16:28:28.422 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-11-16T16:28:28.422 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-16T16:28:28.422 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librados2-2:18.0.0-3905.g5e4a743a.el8.x86_64 129/134 2023-11-16T16:28:28.422 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-16T16:28:28.422 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librbd1-2:18.0.0-3905.g5e4a743a.el8.x86_64 131/134 2023-11-16T16:28:28.422 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-16T16:28:28.422 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-16T16:28:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-16T16:28:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-2:18.0.0-3905.g5e4a743a.el8.x86_64 1/134 2023-11-16T16:28:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-base-2:18.0.0-3905.g5e4a743a.el8.x86_64 2/134 2023-11-16T16:28:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-common-2:18.0.0-3905.g5e4a743a.el8.x86_64 3/134 2023-11-16T16:28:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-fuse-2:18.0.0-3905.g5e4a743a.el8.x86_64 4/134 2023-11-16T16:28:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-3905.g5e4a7 5/134 2023-11-16T16:28:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mds-2:18.0.0-3905.g5e4a743a.el8.x86_64 6/134 2023-11-16T16:28:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-2:18.0.0-3905.g5e4a743a.el8.x86_64 7/134 2023-11-16T16:28:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mon-2:18.0.0-3905.g5e4a743a.el8.x86_64 8/134 2023-11-16T16:28:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-osd-2:18.0.0-3905.g5e4a743a.el8.x86_64 9/134 2023-11-16T16:28:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-radosgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 10/134 2023-11-16T16:28:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-selinux-2:18.0.0-3905.g5e4a743a.el8.x86_64 11/134 2023-11-16T16:28:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-test-2:18.0.0-3905.g5e4a743a.el8.x86_64 12/134 2023-11-16T16:28:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephfs-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 13/134 2023-11-16T16:28:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephfs-devel-2:18.0.0-3905.g5e4a743a.el8.x86_ 14/134 2023-11-16T16:28:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephfs2-2:18.0.0-3905.g5e4a743a.el8.x86_64 15/134 2023-11-16T16:28:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephsqlite-2:18.0.0-3905.g5e4a743a.el8.x86_64 16/134 2023-11-16T16:28:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados-devel-2:18.0.0-3905.g5e4a743a.el8.x86_6 17/134 2023-11-16T16:28:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libradosstriper1-2:18.0.0-3905.g5e4a743a.el8.x86 18/134 2023-11-16T16:28:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librgw2-2:18.0.0-3905.g5e4a743a.el8.x86_64 19/134 2023-11-16T16:28:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ceph-argparse-2:18.0.0-3905.g5e4a743a.el 20/134 2023-11-16T16:28:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ceph-common-2:18.0.0-3905.g5e4a743a.el8. 21/134 2023-11-16T16:28:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cephfs-2:18.0.0-3905.g5e4a743a.el8.x86_6 22/134 2023-11-16T16:28:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rados-2:18.0.0-3905.g5e4a743a.el8.x86_64 23/134 2023-11-16T16:28:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rbd-2:18.0.0-3905.g5e4a743a.el8.x86_64 24/134 2023-11-16T16:28:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 25/134 2023-11-16T16:28:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-fuse-2:18.0.0-3905.g5e4a743a.el8.x86_64 26/134 2023-11-16T16:28:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 27/134 2023-11-16T16:28:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-nbd-2:18.0.0-3905.g5e4a743a.el8.x86_64 28/134 2023-11-16T16:28:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-3905.g5e4a743a. 29/134 2023-11-16T16:28:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-3905.g5e4a743a.el8.noa 30/134 2023-11-16T16:28:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-3905.g5e4a743a.el8.n 31/134 2023-11-16T16:28:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-3905.g5e4 32/134 2023-11-16T16:28:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-3905.g5e4a743a.el 33/134 2023-11-16T16:28:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-rook-2:18.0.0-3905.g5e4a743a.el8.noarch 34/134 2023-11-16T16:28:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-3905.g5e4a743a.e 35/134 2023-11-16T16:28:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-volume-2:18.0.0-3905.g5e4a743a.el8.noarch 36/134 2023-11-16T16:28:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephadm-2:18.0.0-3905.g5e4a743a.el8.noarch 37/134 2023-11-16T16:28:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephfs-top-2:18.0.0-3905.g5e4a743a.el8.noarch 38/134 2023-11-16T16:28:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-16T16:28:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-16T16:28:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-16T16:28:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-16T16:28:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-16T16:28:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-16T16:28:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-16T16:28:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-16T16:28:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-16T16:28:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-16T16:28:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-16T16:28:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-16T16:28:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-16T16:28:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-16T16:28:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-16T16:28:28.836 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-16T16:28:28.836 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-16T16:28:28.836 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-16T16:28:28.836 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-16T16:28:28.836 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-16T16:28:28.836 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-16T16:28:28.836 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-16T16:28:28.836 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-16T16:28:28.836 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-16T16:28:28.836 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-16T16:28:28.836 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-16T16:28:28.836 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-16T16:28:28.837 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-16T16:28:28.837 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-16T16:28:28.837 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-16T16:28:28.837 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-16T16:28:28.837 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-16T16:28:28.837 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-16T16:28:28.837 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-11-16T16:28:28.837 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-11-16T16:28:28.837 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-11-16T16:28:28.837 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-11-16T16:28:28.837 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-11-16T16:28:28.837 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-11-16T16:28:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-11-16T16:28:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-11-16T16:28:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-11-16T16:28:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-11-16T16:28:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-11-16T16:28:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-11-16T16:28:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-11-16T16:28:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-11-16T16:28:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-11-16T16:28:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-11-16T16:28:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-11-16T16:28:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-11-16T16:28:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-11-16T16:28:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-11-16T16:28:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-11-16T16:28:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-11-16T16:28:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-11-16T16:28:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-11-16T16:28:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-11-16T16:28:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-11-16T16:28:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-11-16T16:28:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-11-16T16:28:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-11-16T16:28:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-11-16T16:28:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-11-16T16:28:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-11-16T16:28:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-11-16T16:28:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-11-16T16:28:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-11-16T16:28:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-11-16T16:28:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-11-16T16:28:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-11-16T16:28:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-11-16T16:28:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-11-16T16:28:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-11-16T16:28:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-11-16T16:28:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-11-16T16:28:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-11-16T16:28:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-11-16T16:28:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-11-16T16:28:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-11-16T16:28:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-11-16T16:28:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-16T16:28:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-11-16T16:28:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-16T16:28:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-11-16T16:28:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-11-16T16:28:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-11-16T16:28:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-11-16T16:28:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-11-16T16:28:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-16T16:28:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados2-2:18.0.0-3905.g5e4a743a.el8.x86_64 129/134 2023-11-16T16:28:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-16T16:28:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librbd1-2:18.0.0-3905.g5e4a743a.el8.x86_64 131/134 2023-11-16T16:28:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-16T16:28:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-16T16:28:30.542 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-16T16:28:30.543 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:02:02 ago on Thu 16 Nov 2023 04:26:28 PM UTC. 2023-11-16T16:28:30.863 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-16T16:28:30.864 INFO:teuthology.orchestra.run.smithi001.stdout:Last metadata expiration check: 0:02:04 ago on Thu 16 Nov 2023 04:26:26 PM UTC. 2023-11-16T16:28:31.423 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-16T16:28:31.423 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:02:05 ago on Thu 16 Nov 2023 04:26:26 PM UTC. 2023-11-16T16:28:41.263 INFO:teuthology.orchestra.run.smithi026.stdout:Installed products updated. 2023-11-16T16:28:41.497 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-11-16T16:28:41.497 INFO:teuthology.orchestra.run.smithi026.stdout:Upgraded: 2023-11-16T16:28:41.497 INFO:teuthology.orchestra.run.smithi026.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-16T16:28:41.497 INFO:teuthology.orchestra.run.smithi026.stdout: librados2-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:41.497 INFO:teuthology.orchestra.run.smithi026.stdout: librbd1-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:41.498 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2023-11-16T16:28:41.498 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-16T16:28:41.498 INFO:teuthology.orchestra.run.smithi026.stdout: automake-1.16.1-7.el8.noarch 2023-11-16T16:28:41.498 INFO:teuthology.orchestra.run.smithi026.stdout: bison-3.0.4-10.el8.x86_64 2023-11-16T16:28:41.498 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:41.498 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-base-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:41.498 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-common-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:41.498 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-fuse-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:41.498 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-grafana-dashboards-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:28:41.498 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-immutable-object-cache-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:41.498 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mds-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:41.498 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:41.498 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-cephadm-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:28:41.499 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-dashboard-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:28:41.499 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-diskprediction-local-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:28:41.499 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-modules-core-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:28:41.499 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-rook-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:28:41.499 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mon-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:41.499 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-osd-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:41.499 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-prometheus-alerts-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:28:41.499 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-radosgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:41.499 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-selinux-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:41.499 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-test-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:41.499 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-volume-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:28:41.499 INFO:teuthology.orchestra.run.smithi026.stdout: cephadm-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:28:41.499 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:41.500 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-top-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:28:41.500 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-16T16:28:41.500 INFO:teuthology.orchestra.run.smithi026.stdout: flex-2.6.1-9.el8.x86_64 2023-11-16T16:28:41.500 INFO:teuthology.orchestra.run.smithi026.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-16T16:28:41.500 INFO:teuthology.orchestra.run.smithi026.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-16T16:28:41.500 INFO:teuthology.orchestra.run.smithi026.stdout: jq-1.6-3.el8.x86_64 2023-11-16T16:28:41.500 INFO:teuthology.orchestra.run.smithi026.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-16T16:28:41.500 INFO:teuthology.orchestra.run.smithi026.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-16T16:28:41.500 INFO:teuthology.orchestra.run.smithi026.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-16T16:28:41.500 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs-devel-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:41.500 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs2-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:41.500 INFO:teuthology.orchestra.run.smithi026.stdout: libcephsqlite-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:41.500 INFO:teuthology.orchestra.run.smithi026.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-16T16:28:41.501 INFO:teuthology.orchestra.run.smithi026.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-16T16:28:41.501 INFO:teuthology.orchestra.run.smithi026.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-16T16:28:41.501 INFO:teuthology.orchestra.run.smithi026.stdout: libpq-13.5-1.el8.x86_64 2023-11-16T16:28:41.501 INFO:teuthology.orchestra.run.smithi026.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-16T16:28:41.501 INFO:teuthology.orchestra.run.smithi026.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-16T16:28:41.501 INFO:teuthology.orchestra.run.smithi026.stdout: librados-devel-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:41.501 INFO:teuthology.orchestra.run.smithi026.stdout: libradosstriper1-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:41.501 INFO:teuthology.orchestra.run.smithi026.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-16T16:28:41.501 INFO:teuthology.orchestra.run.smithi026.stdout: librgw2-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:41.501 INFO:teuthology.orchestra.run.smithi026.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-16T16:28:41.501 INFO:teuthology.orchestra.run.smithi026.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-16T16:28:41.501 INFO:teuthology.orchestra.run.smithi026.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-16T16:28:41.501 INFO:teuthology.orchestra.run.smithi026.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-16T16:28:41.502 INFO:teuthology.orchestra.run.smithi026.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-16T16:28:41.502 INFO:teuthology.orchestra.run.smithi026.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-16T16:28:41.502 INFO:teuthology.orchestra.run.smithi026.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-16T16:28:41.502 INFO:teuthology.orchestra.run.smithi026.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-16T16:28:41.502 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-16T16:28:41.502 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-16T16:28:41.502 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-16T16:28:41.502 INFO:teuthology.orchestra.run.smithi026.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-16T16:28:41.502 INFO:teuthology.orchestra.run.smithi026.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-16T16:28:41.502 INFO:teuthology.orchestra.run.smithi026.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-16T16:28:41.502 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-16T16:28:41.502 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-16T16:28:41.502 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-16T16:28:41.502 INFO:teuthology.orchestra.run.smithi026.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-16T16:28:41.503 INFO:teuthology.orchestra.run.smithi026.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-16T16:28:41.503 INFO:teuthology.orchestra.run.smithi026.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-16T16:28:41.503 INFO:teuthology.orchestra.run.smithi026.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-16T16:28:41.503 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-16T16:28:41.503 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-argparse-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:41.503 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-common-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:41.503 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cephfs-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:41.503 INFO:teuthology.orchestra.run.smithi026.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-16T16:28:41.503 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-16T16:28:41.503 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-16T16:28:41.503 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-16T16:28:41.503 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-16T16:28:41.503 INFO:teuthology.orchestra.run.smithi026.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-16T16:28:41.503 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-16T16:28:41.504 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-16T16:28:41.504 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-16T16:28:41.504 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-16T16:28:41.504 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-16T16:28:41.504 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-16T16:28:41.504 INFO:teuthology.orchestra.run.smithi026.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-16T16:28:41.504 INFO:teuthology.orchestra.run.smithi026.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-16T16:28:41.504 INFO:teuthology.orchestra.run.smithi026.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-16T16:28:41.504 INFO:teuthology.orchestra.run.smithi026.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-16T16:28:41.504 INFO:teuthology.orchestra.run.smithi026.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-16T16:28:41.504 INFO:teuthology.orchestra.run.smithi026.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-16T16:28:41.504 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-16T16:28:41.504 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-16T16:28:41.504 INFO:teuthology.orchestra.run.smithi026.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-16T16:28:41.505 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-16T16:28:41.505 INFO:teuthology.orchestra.run.smithi026.stdout: python3-portend-2.6-1.el8.noarch 2023-11-16T16:28:41.505 INFO:teuthology.orchestra.run.smithi026.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-16T16:28:41.505 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-16T16:28:41.505 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-16T16:28:41.505 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-16T16:28:41.505 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-16T16:28:41.505 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-16T16:28:41.505 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rados-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:41.505 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rbd-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:41.505 INFO:teuthology.orchestra.run.smithi026.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-16T16:28:41.505 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-16T16:28:41.505 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:41.506 INFO:teuthology.orchestra.run.smithi026.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-16T16:28:41.506 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-16T16:28:41.506 INFO:teuthology.orchestra.run.smithi026.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-16T16:28:41.506 INFO:teuthology.orchestra.run.smithi026.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-16T16:28:41.506 INFO:teuthology.orchestra.run.smithi026.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-16T16:28:41.506 INFO:teuthology.orchestra.run.smithi026.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-16T16:28:41.506 INFO:teuthology.orchestra.run.smithi026.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-16T16:28:41.506 INFO:teuthology.orchestra.run.smithi026.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-16T16:28:41.506 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-16T16:28:41.506 INFO:teuthology.orchestra.run.smithi026.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-16T16:28:41.506 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-16T16:28:41.506 INFO:teuthology.orchestra.run.smithi026.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-16T16:28:41.506 INFO:teuthology.orchestra.run.smithi026.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-16T16:28:41.507 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-fuse-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:41.507 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:41.507 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-nbd-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:41.507 INFO:teuthology.orchestra.run.smithi026.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-16T16:28:41.507 INFO:teuthology.orchestra.run.smithi026.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-16T16:28:41.507 INFO:teuthology.orchestra.run.smithi026.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-16T16:28:41.507 INFO:teuthology.orchestra.run.smithi026.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-16T16:28:41.507 INFO:teuthology.orchestra.run.smithi026.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-16T16:28:41.507 INFO:teuthology.orchestra.run.smithi026.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-16T16:28:41.507 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-11-16T16:28:41.508 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2023-11-16T16:28:41.675 DEBUG:teuthology.parallel:result is None 2023-11-16T16:28:41.871 INFO:teuthology.orchestra.run.smithi001.stdout:Installed products updated. 2023-11-16T16:28:42.104 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:28:42.104 INFO:teuthology.orchestra.run.smithi001.stdout:Upgraded: 2023-11-16T16:28:42.105 INFO:teuthology.orchestra.run.smithi001.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-16T16:28:42.105 INFO:teuthology.orchestra.run.smithi001.stdout: librados2-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.105 INFO:teuthology.orchestra.run.smithi001.stdout: librbd1-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.105 INFO:teuthology.orchestra.run.smithi001.stdout:Installed: 2023-11-16T16:28:42.105 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-16T16:28:42.105 INFO:teuthology.orchestra.run.smithi001.stdout: automake-1.16.1-7.el8.noarch 2023-11-16T16:28:42.105 INFO:teuthology.orchestra.run.smithi001.stdout: bison-3.0.4-10.el8.x86_64 2023-11-16T16:28:42.105 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.105 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-base-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.105 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-common-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.105 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-fuse-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.105 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-grafana-dashboards-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:28:42.105 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-immutable-object-cache-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.106 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mds-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.106 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.106 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-cephadm-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:28:42.106 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-dashboard-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:28:42.106 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-diskprediction-local-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:28:42.106 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-modules-core-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:28:42.106 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-rook-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:28:42.106 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mon-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.106 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-osd-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.106 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-prometheus-alerts-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:28:42.106 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-radosgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.106 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-selinux-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.107 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-test-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.107 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-volume-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:28:42.107 INFO:teuthology.orchestra.run.smithi001.stdout: cephadm-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:28:42.107 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.107 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-top-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:28:42.107 INFO:teuthology.orchestra.run.smithi001.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-16T16:28:42.107 INFO:teuthology.orchestra.run.smithi001.stdout: flex-2.6.1-9.el8.x86_64 2023-11-16T16:28:42.107 INFO:teuthology.orchestra.run.smithi001.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-16T16:28:42.107 INFO:teuthology.orchestra.run.smithi001.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-16T16:28:42.107 INFO:teuthology.orchestra.run.smithi001.stdout: jq-1.6-3.el8.x86_64 2023-11-16T16:28:42.107 INFO:teuthology.orchestra.run.smithi001.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-16T16:28:42.107 INFO:teuthology.orchestra.run.smithi001.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-16T16:28:42.107 INFO:teuthology.orchestra.run.smithi001.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-16T16:28:42.107 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs-devel-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.108 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs2-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.108 INFO:teuthology.orchestra.run.smithi001.stdout: libcephsqlite-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.108 INFO:teuthology.orchestra.run.smithi001.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-16T16:28:42.108 INFO:teuthology.orchestra.run.smithi001.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-16T16:28:42.108 INFO:teuthology.orchestra.run.smithi001.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-16T16:28:42.108 INFO:teuthology.orchestra.run.smithi001.stdout: libpq-13.5-1.el8.x86_64 2023-11-16T16:28:42.108 INFO:teuthology.orchestra.run.smithi001.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-16T16:28:42.108 INFO:teuthology.orchestra.run.smithi001.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-16T16:28:42.108 INFO:teuthology.orchestra.run.smithi001.stdout: librados-devel-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.108 INFO:teuthology.orchestra.run.smithi001.stdout: libradosstriper1-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.108 INFO:teuthology.orchestra.run.smithi001.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-16T16:28:42.108 INFO:teuthology.orchestra.run.smithi001.stdout: librgw2-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.109 INFO:teuthology.orchestra.run.smithi001.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-16T16:28:42.109 INFO:teuthology.orchestra.run.smithi001.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-16T16:28:42.109 INFO:teuthology.orchestra.run.smithi001.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-16T16:28:42.109 INFO:teuthology.orchestra.run.smithi001.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-16T16:28:42.109 INFO:teuthology.orchestra.run.smithi001.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-16T16:28:42.109 INFO:teuthology.orchestra.run.smithi001.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-16T16:28:42.109 INFO:teuthology.orchestra.run.smithi001.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-16T16:28:42.109 INFO:teuthology.orchestra.run.smithi001.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-16T16:28:42.109 INFO:teuthology.orchestra.run.smithi001.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-16T16:28:42.109 INFO:teuthology.orchestra.run.smithi001.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-16T16:28:42.109 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-16T16:28:42.109 INFO:teuthology.orchestra.run.smithi001.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-16T16:28:42.110 INFO:teuthology.orchestra.run.smithi001.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-16T16:28:42.110 INFO:teuthology.orchestra.run.smithi001.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-16T16:28:42.110 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-16T16:28:42.110 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-16T16:28:42.110 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-16T16:28:42.110 INFO:teuthology.orchestra.run.smithi001.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-16T16:28:42.110 INFO:teuthology.orchestra.run.smithi001.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-16T16:28:42.110 INFO:teuthology.orchestra.run.smithi001.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-16T16:28:42.110 INFO:teuthology.orchestra.run.smithi001.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-16T16:28:42.110 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-16T16:28:42.110 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-argparse-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.110 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-common-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.111 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cephfs-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.111 INFO:teuthology.orchestra.run.smithi001.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-16T16:28:42.111 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-16T16:28:42.111 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-16T16:28:42.111 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-16T16:28:42.111 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-16T16:28:42.111 INFO:teuthology.orchestra.run.smithi001.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-16T16:28:42.111 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-16T16:28:42.111 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-16T16:28:42.111 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-16T16:28:42.111 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-16T16:28:42.111 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-16T16:28:42.111 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-16T16:28:42.111 INFO:teuthology.orchestra.run.smithi001.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-16T16:28:42.112 INFO:teuthology.orchestra.run.smithi001.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-16T16:28:42.112 INFO:teuthology.orchestra.run.smithi001.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-16T16:28:42.112 INFO:teuthology.orchestra.run.smithi001.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-16T16:28:42.112 INFO:teuthology.orchestra.run.smithi001.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-16T16:28:42.112 INFO:teuthology.orchestra.run.smithi001.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-16T16:28:42.112 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-16T16:28:42.112 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-16T16:28:42.112 INFO:teuthology.orchestra.run.smithi001.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-16T16:28:42.112 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-16T16:28:42.112 INFO:teuthology.orchestra.run.smithi001.stdout: python3-portend-2.6-1.el8.noarch 2023-11-16T16:28:42.112 INFO:teuthology.orchestra.run.smithi001.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-16T16:28:42.112 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-16T16:28:42.113 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-16T16:28:42.113 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-16T16:28:42.113 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-16T16:28:42.113 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-16T16:28:42.113 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rados-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.113 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rbd-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.113 INFO:teuthology.orchestra.run.smithi001.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-16T16:28:42.113 INFO:teuthology.orchestra.run.smithi001.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-16T16:28:42.113 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.113 INFO:teuthology.orchestra.run.smithi001.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-16T16:28:42.113 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-16T16:28:42.113 INFO:teuthology.orchestra.run.smithi001.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-16T16:28:42.114 INFO:teuthology.orchestra.run.smithi001.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-16T16:28:42.114 INFO:teuthology.orchestra.run.smithi001.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-16T16:28:42.114 INFO:teuthology.orchestra.run.smithi001.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-16T16:28:42.114 INFO:teuthology.orchestra.run.smithi001.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-16T16:28:42.114 INFO:teuthology.orchestra.run.smithi001.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-16T16:28:42.114 INFO:teuthology.orchestra.run.smithi001.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-16T16:28:42.114 INFO:teuthology.orchestra.run.smithi001.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-16T16:28:42.114 INFO:teuthology.orchestra.run.smithi001.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-16T16:28:42.114 INFO:teuthology.orchestra.run.smithi001.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-16T16:28:42.114 INFO:teuthology.orchestra.run.smithi001.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-16T16:28:42.114 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-fuse-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.114 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.115 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-nbd-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.115 INFO:teuthology.orchestra.run.smithi001.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-16T16:28:42.115 INFO:teuthology.orchestra.run.smithi001.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-16T16:28:42.115 INFO:teuthology.orchestra.run.smithi001.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-16T16:28:42.115 INFO:teuthology.orchestra.run.smithi001.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-16T16:28:42.115 INFO:teuthology.orchestra.run.smithi001.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-16T16:28:42.115 INFO:teuthology.orchestra.run.smithi053.stdout:Installed products updated. 2023-11-16T16:28:42.116 INFO:teuthology.orchestra.run.smithi001.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-16T16:28:42.116 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:28:42.116 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2023-11-16T16:28:42.276 DEBUG:teuthology.parallel:result is None 2023-11-16T16:28:42.345 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-16T16:28:42.345 INFO:teuthology.orchestra.run.smithi053.stdout:Upgraded: 2023-11-16T16:28:42.345 INFO:teuthology.orchestra.run.smithi053.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-16T16:28:42.345 INFO:teuthology.orchestra.run.smithi053.stdout: librados2-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.345 INFO:teuthology.orchestra.run.smithi053.stdout: librbd1-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.345 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2023-11-16T16:28:42.346 INFO:teuthology.orchestra.run.smithi053.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-16T16:28:42.346 INFO:teuthology.orchestra.run.smithi053.stdout: automake-1.16.1-7.el8.noarch 2023-11-16T16:28:42.346 INFO:teuthology.orchestra.run.smithi053.stdout: bison-3.0.4-10.el8.x86_64 2023-11-16T16:28:42.346 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.346 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.346 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-common-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.346 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-fuse-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.346 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-grafana-dashboards-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:28:42.346 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-immutable-object-cache-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.346 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mds-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.346 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.347 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-cephadm-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:28:42.347 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:28:42.347 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:28:42.347 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:28:42.347 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:28:42.347 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mon-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.347 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-osd-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.347 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-prometheus-alerts-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:28:42.347 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-radosgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.347 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-selinux-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.347 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-test-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.347 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-volume-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:28:42.348 INFO:teuthology.orchestra.run.smithi053.stdout: cephadm-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:28:42.348 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.348 INFO:teuthology.orchestra.run.smithi053.stdout: cephfs-top-2:18.0.0-3905.g5e4a743a.el8.noarch 2023-11-16T16:28:42.348 INFO:teuthology.orchestra.run.smithi053.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-16T16:28:42.348 INFO:teuthology.orchestra.run.smithi053.stdout: flex-2.6.1-9.el8.x86_64 2023-11-16T16:28:42.348 INFO:teuthology.orchestra.run.smithi053.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-16T16:28:42.348 INFO:teuthology.orchestra.run.smithi053.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-16T16:28:42.348 INFO:teuthology.orchestra.run.smithi053.stdout: jq-1.6-3.el8.x86_64 2023-11-16T16:28:42.348 INFO:teuthology.orchestra.run.smithi053.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-16T16:28:42.348 INFO:teuthology.orchestra.run.smithi053.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-16T16:28:42.348 INFO:teuthology.orchestra.run.smithi053.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-16T16:28:42.348 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-devel-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.349 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs2-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.349 INFO:teuthology.orchestra.run.smithi053.stdout: libcephsqlite-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.349 INFO:teuthology.orchestra.run.smithi053.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-16T16:28:42.349 INFO:teuthology.orchestra.run.smithi053.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-16T16:28:42.349 INFO:teuthology.orchestra.run.smithi053.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-16T16:28:42.349 INFO:teuthology.orchestra.run.smithi053.stdout: libpq-13.5-1.el8.x86_64 2023-11-16T16:28:42.349 INFO:teuthology.orchestra.run.smithi053.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-16T16:28:42.349 INFO:teuthology.orchestra.run.smithi053.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-16T16:28:42.349 INFO:teuthology.orchestra.run.smithi053.stdout: librados-devel-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.349 INFO:teuthology.orchestra.run.smithi053.stdout: libradosstriper1-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.349 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-16T16:28:42.350 INFO:teuthology.orchestra.run.smithi053.stdout: librgw2-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.350 INFO:teuthology.orchestra.run.smithi053.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-16T16:28:42.350 INFO:teuthology.orchestra.run.smithi053.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-16T16:28:42.350 INFO:teuthology.orchestra.run.smithi053.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-16T16:28:42.350 INFO:teuthology.orchestra.run.smithi053.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-16T16:28:42.350 INFO:teuthology.orchestra.run.smithi053.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-16T16:28:42.350 INFO:teuthology.orchestra.run.smithi053.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-16T16:28:42.350 INFO:teuthology.orchestra.run.smithi053.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-16T16:28:42.350 INFO:teuthology.orchestra.run.smithi053.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-16T16:28:42.350 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-16T16:28:42.350 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-16T16:28:42.350 INFO:teuthology.orchestra.run.smithi053.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-16T16:28:42.351 INFO:teuthology.orchestra.run.smithi053.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-16T16:28:42.351 INFO:teuthology.orchestra.run.smithi053.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-16T16:28:42.351 INFO:teuthology.orchestra.run.smithi053.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-16T16:28:42.351 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-16T16:28:42.351 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-16T16:28:42.351 INFO:teuthology.orchestra.run.smithi053.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-16T16:28:42.351 INFO:teuthology.orchestra.run.smithi053.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-16T16:28:42.351 INFO:teuthology.orchestra.run.smithi053.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-16T16:28:42.351 INFO:teuthology.orchestra.run.smithi053.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-16T16:28:42.351 INFO:teuthology.orchestra.run.smithi053.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-16T16:28:42.351 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-16T16:28:42.351 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.351 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-common-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.352 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cephfs-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.352 INFO:teuthology.orchestra.run.smithi053.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-16T16:28:42.352 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-16T16:28:42.352 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-16T16:28:42.352 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-16T16:28:42.352 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-16T16:28:42.352 INFO:teuthology.orchestra.run.smithi053.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-16T16:28:42.352 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-16T16:28:42.352 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-16T16:28:42.352 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-16T16:28:42.352 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-16T16:28:42.352 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-16T16:28:42.352 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-16T16:28:42.353 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-16T16:28:42.353 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-16T16:28:42.353 INFO:teuthology.orchestra.run.smithi053.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-16T16:28:42.353 INFO:teuthology.orchestra.run.smithi053.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-16T16:28:42.353 INFO:teuthology.orchestra.run.smithi053.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-16T16:28:42.353 INFO:teuthology.orchestra.run.smithi053.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-16T16:28:42.353 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-16T16:28:42.353 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-16T16:28:42.353 INFO:teuthology.orchestra.run.smithi053.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-16T16:28:42.353 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-16T16:28:42.353 INFO:teuthology.orchestra.run.smithi053.stdout: python3-portend-2.6-1.el8.noarch 2023-11-16T16:28:42.353 INFO:teuthology.orchestra.run.smithi053.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-16T16:28:42.354 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-16T16:28:42.354 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-16T16:28:42.354 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-16T16:28:42.354 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-16T16:28:42.354 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-16T16:28:42.354 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rados-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.354 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rbd-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.354 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-16T16:28:42.354 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-16T16:28:42.354 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rgw-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.354 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-16T16:28:42.355 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-16T16:28:42.355 INFO:teuthology.orchestra.run.smithi053.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-16T16:28:42.355 INFO:teuthology.orchestra.run.smithi053.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-16T16:28:42.355 INFO:teuthology.orchestra.run.smithi053.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-16T16:28:42.355 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-16T16:28:42.355 INFO:teuthology.orchestra.run.smithi053.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-16T16:28:42.355 INFO:teuthology.orchestra.run.smithi053.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-16T16:28:42.355 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-16T16:28:42.355 INFO:teuthology.orchestra.run.smithi053.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-16T16:28:42.355 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-16T16:28:42.355 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-16T16:28:42.355 INFO:teuthology.orchestra.run.smithi053.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-16T16:28:42.356 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-fuse-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.356 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-mirror-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.356 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-nbd-2:18.0.0-3905.g5e4a743a.el8.x86_64 2023-11-16T16:28:42.356 INFO:teuthology.orchestra.run.smithi053.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-16T16:28:42.356 INFO:teuthology.orchestra.run.smithi053.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-16T16:28:42.356 INFO:teuthology.orchestra.run.smithi053.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-16T16:28:42.356 INFO:teuthology.orchestra.run.smithi053.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-16T16:28:42.356 INFO:teuthology.orchestra.run.smithi053.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-16T16:28:42.356 INFO:teuthology.orchestra.run.smithi053.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-16T16:28:42.356 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-16T16:28:42.357 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2023-11-16T16:28:42.523 DEBUG:teuthology.parallel:result is None 2023-11-16T16:28:42.523 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=5e4a743ad7fd1422ebd6c4dc950a21a303279207 2023-11-16T16:28:42.738 DEBUG:teuthology.orchestra.run.smithi001:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-16T16:28:42.765 INFO:teuthology.orchestra.run.smithi001.stdout:18.0.0-3905.g5e4a743a.el8 2023-11-16T16:28:42.766 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-3905.g5e4a743a.el8 2023-11-16T16:28:42.766 INFO:teuthology.task.install:The correct ceph version 18.0.0-3905.g5e4a743a is installed. 2023-11-16T16:28:42.766 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=5e4a743ad7fd1422ebd6c4dc950a21a303279207 2023-11-16T16:28:42.976 DEBUG:teuthology.orchestra.run.smithi026:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-16T16:28:43.005 INFO:teuthology.orchestra.run.smithi026.stdout:18.0.0-3905.g5e4a743a.el8 2023-11-16T16:28:43.005 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-3905.g5e4a743a.el8 2023-11-16T16:28:43.005 INFO:teuthology.task.install:The correct ceph version 18.0.0-3905.g5e4a743a is installed. 2023-11-16T16:28:43.006 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=5e4a743ad7fd1422ebd6c4dc950a21a303279207 2023-11-16T16:28:43.141 DEBUG:teuthology.orchestra.run.smithi053:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-16T16:28:43.166 INFO:teuthology.orchestra.run.smithi053.stdout:18.0.0-3905.g5e4a743a.el8 2023-11-16T16:28:43.166 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-3905.g5e4a743a.el8 2023-11-16T16:28:43.166 INFO:teuthology.task.install:The correct ceph version 18.0.0-3905.g5e4a743a is installed. 2023-11-16T16:28:43.166 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-16T16:28:43.167 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2023-11-16T16:28:43.167 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-16T16:28:43.217 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-11-16T16:28:43.217 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-16T16:28:43.267 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-16T16:28:43.267 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-16T16:28:43.315 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-16T16:28:43.315 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2023-11-16T16:28:43.315 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/usr/bin/daemon-helper 2023-11-16T16:28:43.346 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-16T16:28:43.417 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-11-16T16:28:43.417 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/daemon-helper 2023-11-16T16:28:43.451 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-16T16:28:43.523 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-16T16:28:43.523 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/daemon-helper 2023-11-16T16:28:43.557 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-16T16:28:43.628 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-16T16:28:43.628 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2023-11-16T16:28:43.628 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-16T16:28:43.660 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-16T16:28:43.731 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-11-16T16:28:43.731 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-16T16:28:43.764 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-16T16:28:43.836 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-16T16:28:43.836 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-16T16:28:43.869 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-16T16:28:43.943 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-16T16:28:43.943 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2023-11-16T16:28:43.943 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/usr/bin/stdin-killer 2023-11-16T16:28:43.975 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-16T16:28:44.047 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-11-16T16:28:44.047 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/stdin-killer 2023-11-16T16:28:44.079 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-16T16:28:44.151 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-16T16:28:44.151 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/stdin-killer 2023-11-16T16:28:44.183 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-16T16:28:44.256 INFO:teuthology.run_tasks:Running task cephadm... 2023-11-16T16:28:44.324 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '5e4a743ad7fd1422ebd6c4dc950a21a303279207', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2023-11-16T16:28:44.325 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 2023-11-16T16:28:44.325 INFO:tasks.cephadm:Cluster fsid is 35d9b010-849d-11ee-8dbe-212e2dc638e7 2023-11-16T16:28:44.325 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-11-16T16:28:44.325 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.1', 'mon.b': '172.21.15.26', 'mon.c': '172.21.15.53'} 2023-11-16T16:28:44.326 INFO:tasks.cephadm:First mon is mon.a on smithi001 2023-11-16T16:28:44.326 INFO:tasks.cephadm:First mgr is x 2023-11-16T16:28:44.326 INFO:tasks.cephadm:Normalizing hostnames... 2023-11-16T16:28:44.326 DEBUG:teuthology.orchestra.run.smithi001:> sudo hostname $(hostname -s) 2023-11-16T16:28:44.360 DEBUG:teuthology.orchestra.run.smithi026:> sudo hostname $(hostname -s) 2023-11-16T16:28:44.393 DEBUG:teuthology.orchestra.run.smithi053:> sudo hostname $(hostname -s) 2023-11-16T16:28:44.427 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-11-16T16:28:44.428 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=5e4a743ad7fd1422ebd6c4dc950a21a303279207 2023-11-16T16:28:44.588 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '5e4a743ad7fd1422ebd6c4dc950a21a303279207', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/74186/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-3905-g5e4a743a', 'node_name': '172.21.5.32+adami02', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-3905.g5e4a743a'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-choffman-62793/5e4a743ad7fd1422ebd6c4dc950a21a303279207/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-11-16 00:50:05.467524', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-choffman-62793', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-choffman-62793/5e4a743ad7fd1422ebd6c4dc950a21a303279207/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-11-16T16:28:44.810 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-choffman-62793, sha1 5e4a743ad7fd1422ebd6c4dc950a21a303279207 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=5e4a743ad7fd1422ebd6c4dc950a21a303279207 2023-11-16T16:28:44.811 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-choffman-62793/5e4a743ad7fd1422ebd6c4dc950a21a303279207/centos/8/x86_64/flavors/default/cephadm 2023-11-16T16:28:44.811 DEBUG:teuthology.orchestra.run.smithi001:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-choffman-62793/5e4a743ad7fd1422ebd6c4dc950a21a303279207/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-11-16T16:28:45.778 INFO:teuthology.orchestra.run.smithi001.stdout:-rw-rw-r--. 1 ubuntu ubuntu 384133 Nov 16 16:28 /home/ubuntu/cephtest/cephadm 2023-11-16T16:28:45.779 DEBUG:teuthology.orchestra.run.smithi026:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-choffman-62793/5e4a743ad7fd1422ebd6c4dc950a21a303279207/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-11-16T16:28:46.818 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-rw-r--. 1 ubuntu ubuntu 384133 Nov 16 16:28 /home/ubuntu/cephtest/cephadm 2023-11-16T16:28:46.818 DEBUG:teuthology.orchestra.run.smithi053:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-choffman-62793/5e4a743ad7fd1422ebd6c4dc950a21a303279207/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-11-16T16:28:47.792 INFO:teuthology.orchestra.run.smithi053.stdout:-rw-rw-r--. 1 ubuntu ubuntu 384133 Nov 16 16:28 /home/ubuntu/cephtest/cephadm 2023-11-16T16:28:47.793 DEBUG:teuthology.orchestra.run.smithi001:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-11-16T16:28:47.815 DEBUG:teuthology.orchestra.run.smithi026:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-11-16T16:28:47.837 DEBUG:teuthology.orchestra.run.smithi053:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-11-16T16:28:47.873 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 on all hosts... 2023-11-16T16:28:47.873 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 pull 2023-11-16T16:28:47.875 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 pull 2023-11-16T16:28:47.880 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 pull 2023-11-16T16:28:50.884 INFO:teuthology.orchestra.run.smithi001.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207... 2023-11-16T16:28:51.477 INFO:teuthology.orchestra.run.smithi026.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207... 2023-11-16T16:28:51.486 INFO:teuthology.orchestra.run.smithi053.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207... 2023-11-16T16:29:29.010 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:29:29.011 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_version": "ceph version 18.0.0-3905-g5e4a743a (5e4a743ad7fd1422ebd6c4dc950a21a303279207) reef (dev)", 2023-11-16T16:29:29.011 INFO:teuthology.orchestra.run.smithi001.stdout: "image_id": "d5b1f852d682af5926771a6e4760786b6147f99518b76fd91b07a1b9e99bfe99", 2023-11-16T16:29:29.011 INFO:teuthology.orchestra.run.smithi001.stdout: "repo_digests": [ 2023-11-16T16:29:29.012 INFO:teuthology.orchestra.run.smithi001.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:209515a30c262c9be4300b2075ceb5fd8912e5a4eccd3c9021b5585705456b42" 2023-11-16T16:29:29.012 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2023-11-16T16:29:29.012 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:29:49.236 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-11-16T16:29:49.237 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_version": "ceph version 18.0.0-3905-g5e4a743a (5e4a743ad7fd1422ebd6c4dc950a21a303279207) reef (dev)", 2023-11-16T16:29:49.237 INFO:teuthology.orchestra.run.smithi026.stdout: "image_id": "d5b1f852d682af5926771a6e4760786b6147f99518b76fd91b07a1b9e99bfe99", 2023-11-16T16:29:49.237 INFO:teuthology.orchestra.run.smithi026.stdout: "repo_digests": [ 2023-11-16T16:29:49.237 INFO:teuthology.orchestra.run.smithi026.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:209515a30c262c9be4300b2075ceb5fd8912e5a4eccd3c9021b5585705456b42" 2023-11-16T16:29:49.237 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2023-11-16T16:29:49.237 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-11-16T16:29:53.658 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-16T16:29:53.658 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_version": "ceph version 18.0.0-3905-g5e4a743a (5e4a743ad7fd1422ebd6c4dc950a21a303279207) reef (dev)", 2023-11-16T16:29:53.658 INFO:teuthology.orchestra.run.smithi053.stdout: "image_id": "d5b1f852d682af5926771a6e4760786b6147f99518b76fd91b07a1b9e99bfe99", 2023-11-16T16:29:53.658 INFO:teuthology.orchestra.run.smithi053.stdout: "repo_digests": [ 2023-11-16T16:29:53.658 INFO:teuthology.orchestra.run.smithi053.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:209515a30c262c9be4300b2075ceb5fd8912e5a4eccd3c9021b5585705456b42" 2023-11-16T16:29:53.659 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2023-11-16T16:29:53.659 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-16T16:29:53.678 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /etc/ceph 2023-11-16T16:29:53.717 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /etc/ceph 2023-11-16T16:29:53.755 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /etc/ceph 2023-11-16T16:29:53.787 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 777 /etc/ceph 2023-11-16T16:29:53.819 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 777 /etc/ceph 2023-11-16T16:29:53.850 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 777 /etc/ceph 2023-11-16T16:29:53.879 INFO:tasks.cephadm:Writing seed config... 2023-11-16T16:29:53.880 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-11-16T16:29:53.881 INFO:tasks.cephadm: override: [client] debug client = 20 2023-11-16T16:29:53.881 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-11-16T16:29:53.881 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-11-16T16:29:53.881 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-11-16T16:29:53.881 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-11-16T16:29:53.881 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-11-16T16:29:53.881 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-11-16T16:29:53.881 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-11-16T16:29:53.881 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-11-16T16:29:53.881 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-11-16T16:29:53.881 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-11-16T16:29:53.882 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-11-16T16:29:53.882 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-11-16T16:29:53.882 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-11-16T16:29:53.882 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-11-16T16:29:53.882 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-11-16T16:29:53.882 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2023-11-16T16:29:53.882 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-11-16T16:29:53.882 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-11-16T16:29:53.882 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-11-16T16:29:53.882 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-11-16T16:29:53.882 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2023-11-16T16:29:53.882 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-11-16T16:29:53.882 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-11-16T16:29:53.883 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-11-16T16:29:53.883 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-11-16T16:29:53.883 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-11-16T16:29:53.883 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-11-16T16:29:53.883 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-11-16T16:29:53.883 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2023-11-16T16:29:53.883 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-11-16T16:29:53.883 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-11-16T16:29:53.883 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-11-16T16:29:53.883 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-11-16T16:29:53.883 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-11-16T16:29:53.883 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-11-16T16:29:53.883 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-11-16T16:29:53.884 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-11-16T16:29:53.884 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-11-16T16:29:53.884 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-11-16T16:29:53.884 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-11-16T16:29:53.884 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-11-16T16:29:53.884 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-11-16T16:29:53.884 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2023-11-16T16:29:53.885 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2023-11-16T16:29:53.885 DEBUG:teuthology.orchestra.run.smithi001:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-11-16T16:29:53.905 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 35d9b010-849d-11ee-8dbe-212e2dc638e7 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-11-16T16:29:53.905 DEBUG:teuthology.orchestra.run.smithi001:mon.a> sudo journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mon.a.service 2023-11-16T16:29:53.947 DEBUG:teuthology.orchestra.run.smithi001:mgr.x> sudo journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mgr.x.service 2023-11-16T16:29:53.978 INFO:journalctl@ceph.mon.a.smithi001.stdout:-- Logs begin at Thu 2023-11-16 16:14:27 UTC. -- 2023-11-16T16:29:53.979 INFO:tasks.cephadm:Bootstrapping... 2023-11-16T16:29:53.979 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 -v bootstrap --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.1 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-11-16T16:29:54.007 INFO:journalctl@ceph.mgr.x.smithi001.stdout:-- Logs begin at Thu 2023-11-16 16:14:27 UTC. -- 2023-11-16T16:29:54.246 INFO:teuthology.orchestra.run.smithi001.stdout:-------------------------------------------------------------------------------- 2023-11-16T16:29:54.246 INFO:teuthology.orchestra.run.smithi001.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207', '-v', 'bootstrap', '--fsid', '35d9b010-849d-11ee-8dbe-212e2dc638e7', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.1', '--skip-admin-label'] 2023-11-16T16:29:54.393 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stdout 4.1.1 2023-11-16T16:29:54.393 INFO:teuthology.orchestra.run.smithi001.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-11-16T16:29:54.394 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying podman|docker is present... 2023-11-16T16:29:54.540 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stdout 4.1.1 2023-11-16T16:29:54.540 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying lvm2 is present... 2023-11-16T16:29:54.540 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying time synchronization is in place... 2023-11-16T16:29:54.547 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-11-16T16:29:54.548 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-11-16T16:29:54.554 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-11-16T16:29:54.554 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2023-11-16T16:29:54.560 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout enabled 2023-11-16T16:29:54.566 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout active 2023-11-16T16:29:54.566 INFO:teuthology.orchestra.run.smithi001.stdout:Unit chronyd.service is enabled and running 2023-11-16T16:29:54.566 INFO:teuthology.orchestra.run.smithi001.stdout:Repeating the final host check... 2023-11-16T16:29:54.710 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stdout 4.1.1 2023-11-16T16:29:54.710 INFO:teuthology.orchestra.run.smithi001.stdout:podman (/bin/podman) version 4.1.1 is present 2023-11-16T16:29:54.710 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl is present 2023-11-16T16:29:54.710 INFO:teuthology.orchestra.run.smithi001.stdout:lvcreate is present 2023-11-16T16:29:54.716 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-11-16T16:29:54.716 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-11-16T16:29:54.722 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-11-16T16:29:54.722 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2023-11-16T16:29:54.728 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout enabled 2023-11-16T16:29:54.734 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout active 2023-11-16T16:29:54.734 INFO:teuthology.orchestra.run.smithi001.stdout:Unit chronyd.service is enabled and running 2023-11-16T16:29:54.734 INFO:teuthology.orchestra.run.smithi001.stdout:Host looks OK 2023-11-16T16:29:54.735 INFO:teuthology.orchestra.run.smithi001.stdout:Cluster fsid: 35d9b010-849d-11ee-8dbe-212e2dc638e7 2023-11-16T16:29:54.735 INFO:teuthology.orchestra.run.smithi001.stdout:Acquiring lock 140597836376552 on /run/cephadm/35d9b010-849d-11ee-8dbe-212e2dc638e7.lock 2023-11-16T16:29:54.735 INFO:teuthology.orchestra.run.smithi001.stdout:Lock 140597836376552 acquired on /run/cephadm/35d9b010-849d-11ee-8dbe-212e2dc638e7.lock 2023-11-16T16:29:54.735 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying IP 172.21.15.1 port 3300 ... 2023-11-16T16:29:54.736 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying IP 172.21.15.1 port 6789 ... 2023-11-16T16:29:54.736 INFO:teuthology.orchestra.run.smithi001.stdout:Base mon IP(s) is [172.21.15.1:3300, 172.21.15.1:6789], mon addrv is [v2:172.21.15.1:3300,v1:172.21.15.1:6789] 2023-11-16T16:29:54.740 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp metric 100 2023-11-16T16:29:54.740 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.1 metric 100 2023-11-16T16:29:54.743 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-11-16T16:29:54.743 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2023-11-16T16:29:54.743 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1625sec hoplimit 64 pref medium 2023-11-16T16:29:54.745 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-11-16T16:29:54.745 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-11-16T16:29:54.745 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-11-16T16:29:54.746 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2023-11-16T16:29:54.746 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15e8/64 scope link 2023-11-16T16:29:54.746 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-11-16T16:29:54.747 INFO:teuthology.orchestra.run.smithi001.stdout:Mon IP `172.21.15.1` is in CIDR network `172.21.0.0/20` 2023-11-16T16:29:54.747 INFO:teuthology.orchestra.run.smithi001.stdout:Mon IP `172.21.15.1` is in CIDR network `172.21.0.0/20` 2023-11-16T16:29:54.747 INFO:teuthology.orchestra.run.smithi001.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-11-16T16:29:54.747 INFO:teuthology.orchestra.run.smithi001.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-11-16T16:29:54.748 INFO:teuthology.orchestra.run.smithi001.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207... 2023-11-16T16:29:57.408 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stdout d5b1f852d682af5926771a6e4760786b6147f99518b76fd91b07a1b9e99bfe99 2023-11-16T16:29:57.408 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207... 2023-11-16T16:29:57.408 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Getting image source signatures 2023-11-16T16:29:57.408 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Copying blob sha256:3a8f6e475435f9e3b5d92795b454ceb30085f6221d261249c26042723227dea7 2023-11-16T16:29:57.409 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Copying blob sha256:837e66e3ff89804ed1bb42786f80e6c23534ff65fe48526d5007449f4e1f79b0 2023-11-16T16:29:57.409 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Copying config sha256:d5b1f852d682af5926771a6e4760786b6147f99518b76fd91b07a1b9e99bfe99 2023-11-16T16:29:57.409 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Writing manifest to image destination 2023-11-16T16:29:57.409 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Storing signatures 2023-11-16T16:29:59.269 INFO:teuthology.orchestra.run.smithi001.stdout:ceph: stdout ceph version 18.0.0-3905-g5e4a743a (5e4a743ad7fd1422ebd6c4dc950a21a303279207) reef (dev) 2023-11-16T16:29:59.269 INFO:teuthology.orchestra.run.smithi001.stdout:Ceph version: ceph version 18.0.0-3905-g5e4a743a (5e4a743ad7fd1422ebd6c4dc950a21a303279207) reef (dev) 2023-11-16T16:29:59.269 INFO:teuthology.orchestra.run.smithi001.stdout:Extracting ceph user uid/gid from container image... 2023-11-16T16:30:00.019 INFO:teuthology.orchestra.run.smithi001.stdout:stat: stdout 167 167 2023-11-16T16:30:00.019 INFO:teuthology.orchestra.run.smithi001.stdout:Creating initial keys... 2023-11-16T16:30:00.774 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph-authtool: stdout AQCIQ1ZlL50AGRAAxSpjzEOcwGJrsNq8XN5qLg== 2023-11-16T16:30:01.525 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph-authtool: stdout AQCJQ1ZlspS8CxAAKA1/EkscmV10n7wYrLHJKA== 2023-11-16T16:30:02.273 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph-authtool: stdout AQCJQ1ZlrMfyNhAAQCXu17GSxJDLuttPykHSwA== 2023-11-16T16:30:02.274 INFO:teuthology.orchestra.run.smithi001.stdout:Creating initial monmap... 2023-11-16T16:30:03.023 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-11-16T16:30:03.023 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-11-16T16:30:03.024 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 35d9b010-849d-11ee-8dbe-212e2dc638e7 2023-11-16T16:30:03.024 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-11-16T16:30:03.024 INFO:teuthology.orchestra.run.smithi001.stdout:monmaptool for a [v2:172.21.15.1:3300,v1:172.21.15.1:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-11-16T16:30:03.024 INFO:teuthology.orchestra.run.smithi001.stdout:setting min_mon_release = pacific 2023-11-16T16:30:03.024 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: set fsid to 35d9b010-849d-11ee-8dbe-212e2dc638e7 2023-11-16T16:30:03.024 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-11-16T16:30:03.024 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:30:03.024 INFO:teuthology.orchestra.run.smithi001.stdout:Creating mon... 2023-11-16T16:30:05.088 INFO:teuthology.orchestra.run.smithi001.stdout:create mon.a on 2023-11-16T16:30:05.199 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-11-16T16:30:05.294 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-11-16T16:30:05.388 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7.target → /etc/systemd/system/ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7.target. 2023-11-16T16:30:05.388 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7.target → /etc/systemd/system/ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7.target. 2023-11-16T16:30:05.522 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mon.a 2023-11-16T16:30:05.523 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Failed to reset failed state of unit ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mon.a.service: Unit ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mon.a.service not loaded. 2023-11-16T16:30:05.613 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7.target.wants/ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mon.a.service → /etc/systemd/system/ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@.service. 2023-11-16T16:30:05.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:05 smithi001 systemd[1]: Starting Ceph mon.a for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:30:06.221 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:05 smithi001 podman[98250]: 2023-11-16T16:30:06.221 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:06 smithi001 bash[98250]: b67563c5964a2279e851a7dd1421203acf367093b8e9c39ed0a4d7200a0c84bc 2023-11-16T16:30:06.221 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:06 smithi001 systemd[1]: Started Ceph mon.a for 35d9b010-849d-11ee-8dbe-212e2dc638e7. 2023-11-16T16:30:06.225 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-11-16T16:30:06.225 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout disabled 2023-11-16T16:30:06.233 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-11-16T16:30:06.233 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2023-11-16T16:30:06.233 INFO:teuthology.orchestra.run.smithi001.stdout:firewalld.service is not enabled 2023-11-16T16:30:06.233 INFO:teuthology.orchestra.run.smithi001.stdout:Not possible to enable service . firewalld.service is not available 2023-11-16T16:30:06.234 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mon to start... 2023-11-16T16:30:06.234 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mon... 2023-11-16T16:30:06.489 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:06 smithi001 ceph-mon[98290]: mkfs 35d9b010-849d-11ee-8dbe-212e2dc638e7 2023-11-16T16:30:06.489 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:06 smithi001 ceph-mon[98290]: mon.a is new leader, mons a in quorum (ranks 0) 2023-11-16T16:30:07.288 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout cluster: 2023-11-16T16:30:07.288 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout id: 35d9b010-849d-11ee-8dbe-212e2dc638e7 2023-11-16T16:30:07.289 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-11-16T16:30:07.289 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2023-11-16T16:30:07.289 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout services: 2023-11-16T16:30:07.289 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.723665s) 2023-11-16T16:30:07.289 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-11-16T16:30:07.289 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-11-16T16:30:07.289 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2023-11-16T16:30:07.289 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout data: 2023-11-16T16:30:07.289 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-11-16T16:30:07.289 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-11-16T16:30:07.289 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-11-16T16:30:07.289 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout pgs: 2023-11-16T16:30:07.289 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2023-11-16T16:30:07.289 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.800+0000 7fb1caee9700 1 Processor -- start 2023-11-16T16:30:07.289 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.800+0000 7fb1caee9700 1 -- start start 2023-11-16T16:30:07.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.800+0000 7fb1caee9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1c4104920 0x7fb1c4104d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:07.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.800+0000 7fb1caee9700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c4105300 con 0x7fb1c4104920 2023-11-16T16:30:07.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.800+0000 7fb1c8c85700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1c4104920 0x7fb1c4104d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:07.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.800+0000 7fb1c8c85700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1c4104920 0x7fb1c4104d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40094/0 (socket says 172.21.15.1:40094) 2023-11-16T16:30:07.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.800+0000 7fb1c8c85700 1 -- 172.21.15.1:0/2720330874 learned_addr learned my addr 172.21.15.1:0/2720330874 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:07.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.801+0000 7fb1c8c85700 1 -- 172.21.15.1:0/2720330874 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c4105b20 con 0x7fb1c4104920 2023-11-16T16:30:07.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.801+0000 7fb1c8c85700 1 --2- 172.21.15.1:0/2720330874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1c4104920 0x7fb1c4104d30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb1b80098a0 tx=0x7fb1b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f159f88ed08fa7 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:07.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.801+0000 7fb1c37fe700 1 -- 172.21.15.1:0/2720330874 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb1b8004160 con 0x7fb1c4104920 2023-11-16T16:30:07.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.801+0000 7fb1c37fe700 1 -- 172.21.15.1:0/2720330874 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b80042c0 con 0x7fb1c4104920 2023-11-16T16:30:07.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.801+0000 7fb1c37fe700 1 -- 172.21.15.1:0/2720330874 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb1b80045a0 con 0x7fb1c4104920 2023-11-16T16:30:07.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.801+0000 7fb1caee9700 1 -- 172.21.15.1:0/2720330874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1c4104920 msgr2=0x7fb1c4104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:07.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.801+0000 7fb1caee9700 1 --2- 172.21.15.1:0/2720330874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1c4104920 0x7fb1c4104d30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb1b80098a0 tx=0x7fb1b8009bb0 comp rx=0 tx=0).stop 2023-11-16T16:30:07.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.801+0000 7fb1caee9700 1 -- 172.21.15.1:0/2720330874 shutdown_connections 2023-11-16T16:30:07.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.801+0000 7fb1caee9700 1 --2- 172.21.15.1:0/2720330874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1c4104920 0x7fb1c4104d30 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:07.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.801+0000 7fb1caee9700 1 -- 172.21.15.1:0/2720330874 >> 172.21.15.1:0/2720330874 conn(0x7fb1c40ffed0 msgr2=0x7fb1c4102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:07.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.802+0000 7fb1caee9700 1 -- 172.21.15.1:0/2720330874 shutdown_connections 2023-11-16T16:30:07.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.802+0000 7fb1caee9700 1 -- 172.21.15.1:0/2720330874 wait complete. 2023-11-16T16:30:07.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.802+0000 7fb1caee9700 1 Processor -- start 2023-11-16T16:30:07.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.802+0000 7fb1caee9700 1 -- start start 2023-11-16T16:30:07.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.802+0000 7fb1caee9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1c4104920 0x7fb1c4198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:07.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.802+0000 7fb1caee9700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c41994a0 con 0x7fb1c4104920 2023-11-16T16:30:07.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.802+0000 7fb1c8c85700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1c4104920 0x7fb1c4198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:07.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.802+0000 7fb1c8c85700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1c4104920 0x7fb1c4198f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40096/0 (socket says 172.21.15.1:40096) 2023-11-16T16:30:07.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.803+0000 7fb1c8c85700 1 -- 172.21.15.1:0/2178956511 learned_addr learned my addr 172.21.15.1:0/2178956511 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:07.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.803+0000 7fb1c8c85700 1 -- 172.21.15.1:0/2178956511 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b8009550 con 0x7fb1c4104920 2023-11-16T16:30:07.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.803+0000 7fb1c8c85700 1 --2- 172.21.15.1:0/2178956511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1c4104920 0x7fb1c4198f60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8004000 tx=0x7fb1b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:07.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.803+0000 7fb1c1ffb700 1 -- 172.21.15.1:0/2178956511 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb1b8011630 con 0x7fb1c4104920 2023-11-16T16:30:07.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.803+0000 7fb1c1ffb700 1 -- 172.21.15.1:0/2178956511 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b8011790 con 0x7fb1c4104920 2023-11-16T16:30:07.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.803+0000 7fb1caee9700 1 -- 172.21.15.1:0/2178956511 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c4106020 con 0x7fb1c4104920 2023-11-16T16:30:07.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.803+0000 7fb1c1ffb700 1 -- 172.21.15.1:0/2178956511 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb1b8011a30 con 0x7fb1c4104920 2023-11-16T16:30:07.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.803+0000 7fb1caee9700 1 -- 172.21.15.1:0/2178956511 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c4105e90 con 0x7fb1c4104920 2023-11-16T16:30:07.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.803+0000 7fb1c1ffb700 1 -- 172.21.15.1:0/2178956511 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb1b8022070 con 0x7fb1c4104920 2023-11-16T16:30:07.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.804+0000 7fb1c1ffb700 1 -- 172.21.15.1:0/2178956511 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb1b801c450 con 0x7fb1c4104920 2023-11-16T16:30:07.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.804+0000 7fb1caee9700 1 -- 172.21.15.1:0/2178956511 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c41061b0 con 0x7fb1c4104920 2023-11-16T16:30:07.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.807+0000 7fb1c1ffb700 1 -- 172.21.15.1:0/2178956511 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7fb1b801b450 con 0x7fb1c4104920 2023-11-16T16:30:07.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.939+0000 7fb1caee9700 1 -- 172.21.15.1:0/2178956511 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fb1c4199b00 con 0x7fb1c4104920 2023-11-16T16:30:07.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.939+0000 7fb1c1ffb700 1 -- 172.21.15.1:0/2178956511 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7fb1b801b5f0 con 0x7fb1c4104920 2023-11-16T16:30:07.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.940+0000 7fb1caee9700 1 -- 172.21.15.1:0/2178956511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1c4104920 msgr2=0x7fb1c4198f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:07.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.940+0000 7fb1caee9700 1 --2- 172.21.15.1:0/2178956511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1c4104920 0x7fb1c4198f60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8004000 tx=0x7fb1b8004750 comp rx=0 tx=0).stop 2023-11-16T16:30:07.293 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.940+0000 7fb1caee9700 1 -- 172.21.15.1:0/2178956511 shutdown_connections 2023-11-16T16:30:07.293 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.940+0000 7fb1caee9700 1 --2- 172.21.15.1:0/2178956511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1c4104920 0x7fb1c4198f60 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:07.293 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.940+0000 7fb1caee9700 1 -- 172.21.15.1:0/2178956511 >> 172.21.15.1:0/2178956511 conn(0x7fb1c40ffed0 msgr2=0x7fb1c4079870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:07.293 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.940+0000 7fb1caee9700 1 -- 172.21.15.1:0/2178956511 shutdown_connections 2023-11-16T16:30:07.293 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:06.940+0000 7fb1caee9700 1 -- 172.21.15.1:0/2178956511 wait complete. 2023-11-16T16:30:07.293 INFO:teuthology.orchestra.run.smithi001.stdout:mon is available 2023-11-16T16:30:07.293 INFO:teuthology.orchestra.run.smithi001.stdout:Assimilating anything we can from ceph.conf... 2023-11-16T16:30:07.521 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:07 smithi001 ceph-mon[98290]: mon.a is new leader, mons a in quorum (ranks 0) 2023-11-16T16:30:07.521 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:07 smithi001 ceph-mon[98290]: monmap e1: 1 mons at {a=[v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0]} removed_ranks: {} 2023-11-16T16:30:07.521 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:07 smithi001 ceph-mon[98290]: fsmap 2023-11-16T16:30:07.521 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:07 smithi001 ceph-mon[98290]: osdmap e1: 0 total, 0 up, 0 in 2023-11-16T16:30:07.521 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:07 smithi001 ceph-mon[98290]: mgrmap e1: no daemons active 2023-11-16T16:30:07.522 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:07 smithi001 ceph-mon[98290]: from='client.? 172.21.15.1:0/2178956511' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-11-16T16:30:08.329 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2023-11-16T16:30:08.329 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [global] 2023-11-16T16:30:08.329 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout fsid = 35d9b010-849d-11ee-8dbe-212e2dc638e7 2023-11-16T16:30:08.330 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.1:3300,v1:172.21.15.1:6789] 2023-11-16T16:30:08.330 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-11-16T16:30:08.330 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-11-16T16:30:08.330 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-11-16T16:30:08.330 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-11-16T16:30:08.330 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2023-11-16T16:30:08.330 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [mgr] 2023-11-16T16:30:08.330 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:08 smithi001 ceph-mon[98290]: from='client.? 172.21.15.1:0/1596359353' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-11-16T16:30:08.331 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:08 smithi001 ceph-mon[98290]: from='client.? 172.21.15.1:0/1596359353' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-11-16T16:30:08.336 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-11-16T16:30:08.336 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2023-11-16T16:30:08.336 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [osd] 2023-11-16T16:30:08.336 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-11-16T16:30:08.336 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-11-16T16:30:08.336 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-11-16T16:30:08.336 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-11-16T16:30:13.604 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-11-16T16:30:13.604 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.818+0000 7f717a511700 1 Processor -- start 2023-11-16T16:30:13.604 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.819+0000 7f717a511700 1 -- start start 2023-11-16T16:30:13.604 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.819+0000 7f717a511700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7174104920 0x7f7174104d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:13.604 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.819+0000 7f717a511700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7174105300 con 0x7f7174104920 2023-11-16T16:30:13.604 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.819+0000 7f7173fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7174104920 0x7f7174104d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:13.605 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.819+0000 7f7173fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7174104920 0x7f7174104d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40098/0 (socket says 172.21.15.1:40098) 2023-11-16T16:30:13.605 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.819+0000 7f7173fff700 1 -- 172.21.15.1:0/1660602435 learned_addr learned my addr 172.21.15.1:0/1660602435 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:13.605 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.820+0000 7f7173fff700 1 -- 172.21.15.1:0/1660602435 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7174105b20 con 0x7f7174104920 2023-11-16T16:30:13.605 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.820+0000 7f7173fff700 1 --2- 172.21.15.1:0/1660602435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7174104920 0x7f7174104d30 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f71640098a0 tx=0x7f7164009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e6963ca18b25a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:13.605 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.820+0000 7f7172ffd700 1 -- 172.21.15.1:0/1660602435 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7164004160 con 0x7f7174104920 2023-11-16T16:30:13.605 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.820+0000 7f7172ffd700 1 -- 172.21.15.1:0/1660602435 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71640042c0 con 0x7f7174104920 2023-11-16T16:30:13.605 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.820+0000 7f7172ffd700 1 -- 172.21.15.1:0/1660602435 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f71640045a0 con 0x7f7174104920 2023-11-16T16:30:13.605 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.820+0000 7f717a511700 1 -- 172.21.15.1:0/1660602435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7174104920 msgr2=0x7f7174104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:13.605 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.820+0000 7f717a511700 1 --2- 172.21.15.1:0/1660602435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7174104920 0x7f7174104d30 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f71640098a0 tx=0x7f7164009bb0 comp rx=0 tx=0).stop 2023-11-16T16:30:13.605 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.821+0000 7f717a511700 1 -- 172.21.15.1:0/1660602435 shutdown_connections 2023-11-16T16:30:13.605 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.821+0000 7f717a511700 1 --2- 172.21.15.1:0/1660602435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7174104920 0x7f7174104d30 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:13.605 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.821+0000 7f717a511700 1 -- 172.21.15.1:0/1660602435 >> 172.21.15.1:0/1660602435 conn(0x7f71740ffed0 msgr2=0x7f7174102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:13.606 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.821+0000 7f717a511700 1 -- 172.21.15.1:0/1660602435 shutdown_connections 2023-11-16T16:30:13.606 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.821+0000 7f717a511700 1 -- 172.21.15.1:0/1660602435 wait complete. 2023-11-16T16:30:13.606 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.821+0000 7f717a511700 1 Processor -- start 2023-11-16T16:30:13.606 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.821+0000 7f717a511700 1 -- start start 2023-11-16T16:30:13.606 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.822+0000 7f717a511700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7174104920 0x7f717418d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:13.606 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.822+0000 7f717a511700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717418db00 con 0x7f7174104920 2023-11-16T16:30:13.606 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.822+0000 7f7173fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7174104920 0x7f717418d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:13.606 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.822+0000 7f7173fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7174104920 0x7f717418d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40100/0 (socket says 172.21.15.1:40100) 2023-11-16T16:30:13.606 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.822+0000 7f7173fff700 1 -- 172.21.15.1:0/1596359353 learned_addr learned my addr 172.21.15.1:0/1596359353 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:13.606 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.822+0000 7f7173fff700 1 -- 172.21.15.1:0/1596359353 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7164009550 con 0x7f7174104920 2023-11-16T16:30:13.606 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.822+0000 7f7173fff700 1 --2- 172.21.15.1:0/1596359353 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7174104920 0x7f717418d5c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7164004000 tx=0x7f7164004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:13.606 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.822+0000 7f71717fa700 1 -- 172.21.15.1:0/1596359353 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7164011630 con 0x7f7174104920 2023-11-16T16:30:13.606 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.822+0000 7f71717fa700 1 -- 172.21.15.1:0/1596359353 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7164011790 con 0x7f7174104920 2023-11-16T16:30:13.607 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.822+0000 7f717a511700 1 -- 172.21.15.1:0/1596359353 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7174106020 con 0x7f7174104920 2023-11-16T16:30:13.607 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.822+0000 7f71717fa700 1 -- 172.21.15.1:0/1596359353 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7164011a30 con 0x7f7174104920 2023-11-16T16:30:13.607 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.823+0000 7f717a511700 1 -- 172.21.15.1:0/1596359353 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7174105e90 con 0x7f7174104920 2023-11-16T16:30:13.607 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.823+0000 7f71717fa700 1 -- 172.21.15.1:0/1596359353 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7164022070 con 0x7f7174104920 2023-11-16T16:30:13.607 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.823+0000 7f71717fa700 1 -- 172.21.15.1:0/1596359353 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f716401c450 con 0x7f7174104920 2023-11-16T16:30:13.607 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.824+0000 7f717a511700 1 -- 172.21.15.1:0/1596359353 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7174062120 con 0x7f7174104920 2023-11-16T16:30:13.607 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.826+0000 7f71717fa700 1 -- 172.21.15.1:0/1596359353 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f716401b450 con 0x7f7174104920 2023-11-16T16:30:13.607 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.952+0000 7f717a511700 1 -- 172.21.15.1:0/1596359353 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f7174192180 con 0x7f7174104920 2023-11-16T16:30:13.607 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.956+0000 7f71717fa700 1 -- 172.21.15.1:0/1596359353 <== mon.0 v2:172.21.15.1:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f716401b5f0 con 0x7f7174104920 2023-11-16T16:30:13.607 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.956+0000 7f71717fa700 1 -- 172.21.15.1:0/1596359353 <== mon.0 v2:172.21.15.1:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+427 (secure 0 0 0) 0x7f71640375f0 con 0x7f7174104920 2023-11-16T16:30:13.607 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.957+0000 7f717a511700 1 -- 172.21.15.1:0/1596359353 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7174104920 msgr2=0x7f717418d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:13.607 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.957+0000 7f717a511700 1 --2- 172.21.15.1:0/1596359353 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7174104920 0x7f717418d5c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7164004000 tx=0x7f7164004750 comp rx=0 tx=0).stop 2023-11-16T16:30:13.608 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.957+0000 7f717a511700 1 -- 172.21.15.1:0/1596359353 shutdown_connections 2023-11-16T16:30:13.608 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.957+0000 7f717a511700 1 --2- 172.21.15.1:0/1596359353 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7174104920 0x7f717418d5c0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:13.608 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.957+0000 7f717a511700 1 -- 172.21.15.1:0/1596359353 >> 172.21.15.1:0/1596359353 conn(0x7f71740ffed0 msgr2=0x7f7174079830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:13.608 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.957+0000 7f717a511700 1 -- 172.21.15.1:0/1596359353 shutdown_connections 2023-11-16T16:30:13.608 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:07.957+0000 7f717a511700 1 -- 172.21.15.1:0/1596359353 wait complete. 2023-11-16T16:30:13.608 INFO:teuthology.orchestra.run.smithi001.stdout:Generating new minimal ceph.conf... 2023-11-16T16:30:13.613 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.834+0000 7f16c9d0d700 1 Processor -- start 2023-11-16T16:30:13.613 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.834+0000 7f16c9d0d700 1 -- start start 2023-11-16T16:30:13.613 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.835+0000 7f16c9d0d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16c4104920 0x7f16c4104d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:13.613 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.835+0000 7f16c9d0d700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c4105300 con 0x7f16c4104920 2023-11-16T16:30:13.613 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.835+0000 7f16c37fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16c4104920 0x7f16c4104d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:13.613 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.835+0000 7f16c37fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16c4104920 0x7f16c4104d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40102/0 (socket says 172.21.15.1:40102) 2023-11-16T16:30:13.613 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.835+0000 7f16c37fe700 1 -- 172.21.15.1:0/922794509 learned_addr learned my addr 172.21.15.1:0/922794509 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:13.613 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.835+0000 7f16c37fe700 1 -- 172.21.15.1:0/922794509 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c4105b20 con 0x7f16c4104920 2023-11-16T16:30:13.613 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.835+0000 7f16c37fe700 1 --2- 172.21.15.1:0/922794509 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16c4104920 0x7f16c4104d30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f16b40098a0 tx=0x7f16b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a304bf2b0d34f7d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:13.613 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.835+0000 7f16c27fc700 1 -- 172.21.15.1:0/922794509 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16b4004160 con 0x7f16c4104920 2023-11-16T16:30:13.613 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.836+0000 7f16c27fc700 1 -- 172.21.15.1:0/922794509 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f16b40042c0 con 0x7f16c4104920 2023-11-16T16:30:13.614 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.836+0000 7f16c27fc700 1 -- 172.21.15.1:0/922794509 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16b4011a00 con 0x7f16c4104920 2023-11-16T16:30:13.614 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.836+0000 7f16c9d0d700 1 -- 172.21.15.1:0/922794509 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16c4104920 msgr2=0x7f16c4104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:13.614 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.836+0000 7f16c9d0d700 1 --2- 172.21.15.1:0/922794509 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16c4104920 0x7f16c4104d30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f16b40098a0 tx=0x7f16b4009bb0 comp rx=0 tx=0).stop 2023-11-16T16:30:13.614 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.836+0000 7f16c9d0d700 1 -- 172.21.15.1:0/922794509 shutdown_connections 2023-11-16T16:30:13.614 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.836+0000 7f16c9d0d700 1 --2- 172.21.15.1:0/922794509 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16c4104920 0x7f16c4104d30 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:13.614 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.836+0000 7f16c9d0d700 1 -- 172.21.15.1:0/922794509 >> 172.21.15.1:0/922794509 conn(0x7f16c40ffed0 msgr2=0x7f16c4102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:13.614 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.836+0000 7f16c9d0d700 1 -- 172.21.15.1:0/922794509 shutdown_connections 2023-11-16T16:30:13.614 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.836+0000 7f16c9d0d700 1 -- 172.21.15.1:0/922794509 wait complete. 2023-11-16T16:30:13.614 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.837+0000 7f16c9d0d700 1 Processor -- start 2023-11-16T16:30:13.614 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.837+0000 7f16c9d0d700 1 -- start start 2023-11-16T16:30:13.614 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.837+0000 7f16c9d0d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16c4104920 0x7f16c4079b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:13.615 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.837+0000 7f16c9d0d700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c419c650 con 0x7f16c4104920 2023-11-16T16:30:13.615 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.837+0000 7f16c37fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16c4104920 0x7f16c4079b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:13.615 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.837+0000 7f16c37fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16c4104920 0x7f16c4079b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40104/0 (socket says 172.21.15.1:40104) 2023-11-16T16:30:13.615 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.837+0000 7f16c37fe700 1 -- 172.21.15.1:0/7548920 learned_addr learned my addr 172.21.15.1:0/7548920 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:13.615 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.838+0000 7f16c37fe700 1 -- 172.21.15.1:0/7548920 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b4009550 con 0x7f16c4104920 2023-11-16T16:30:13.615 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.838+0000 7f16c37fe700 1 --2- 172.21.15.1:0/7548920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16c4104920 0x7f16c4079b40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f16b4009fb0 tx=0x7f16b4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:13.615 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.838+0000 7f16c0ff9700 1 -- 172.21.15.1:0/7548920 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16b4003930 con 0x7f16c4104920 2023-11-16T16:30:13.615 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.838+0000 7f16c9d0d700 1 -- 172.21.15.1:0/7548920 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c4108d50 con 0x7f16c4104920 2023-11-16T16:30:13.615 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.838+0000 7f16c0ff9700 1 -- 172.21.15.1:0/7548920 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f16b4003a90 con 0x7f16c4104920 2023-11-16T16:30:13.615 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.838+0000 7f16c0ff9700 1 -- 172.21.15.1:0/7548920 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16b40195e0 con 0x7f16c4104920 2023-11-16T16:30:13.615 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.838+0000 7f16c9d0d700 1 -- 172.21.15.1:0/7548920 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c4108ee0 con 0x7f16c4104920 2023-11-16T16:30:13.615 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.838+0000 7f16c0ff9700 1 -- 172.21.15.1:0/7548920 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f16b4019bc0 con 0x7f16c4104920 2023-11-16T16:30:13.616 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.838+0000 7f16c0ff9700 1 -- 172.21.15.1:0/7548920 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f16b40243f0 con 0x7f16c4104920 2023-11-16T16:30:13.616 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.839+0000 7f16c9d0d700 1 -- 172.21.15.1:0/7548920 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16a4005320 con 0x7f16c4104920 2023-11-16T16:30:13.616 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.841+0000 7f16c0ff9700 1 -- 172.21.15.1:0/7548920 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f16b4003c00 con 0x7f16c4104920 2023-11-16T16:30:13.616 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.968+0000 7f16c9d0d700 1 -- 172.21.15.1:0/7548920 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f16a4005cc0 con 0x7f16c4104920 2023-11-16T16:30:13.616 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.968+0000 7f16c0ff9700 1 -- 172.21.15.1:0/7548920 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+173 (secure 0 0 0) 0x7f16b401a9d0 con 0x7f16c4104920 2023-11-16T16:30:13.616 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.969+0000 7f16c9d0d700 1 -- 172.21.15.1:0/7548920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16c4104920 msgr2=0x7f16c4079b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:13.616 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.969+0000 7f16c9d0d700 1 --2- 172.21.15.1:0/7548920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16c4104920 0x7f16c4079b40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f16b4009fb0 tx=0x7f16b4011ad0 comp rx=0 tx=0).stop 2023-11-16T16:30:13.616 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.969+0000 7f16c9d0d700 1 -- 172.21.15.1:0/7548920 shutdown_connections 2023-11-16T16:30:13.616 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.969+0000 7f16c9d0d700 1 --2- 172.21.15.1:0/7548920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16c4104920 0x7f16c4079b40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:13.616 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.969+0000 7f16c9d0d700 1 -- 172.21.15.1:0/7548920 >> 172.21.15.1:0/7548920 conn(0x7f16c40ffed0 msgr2=0x7f16c4101570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:13.616 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.969+0000 7f16c9d0d700 1 -- 172.21.15.1:0/7548920 shutdown_connections 2023-11-16T16:30:13.616 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:08.969+0000 7f16c9d0d700 1 -- 172.21.15.1:0/7548920 wait complete. 2023-11-16T16:30:13.617 INFO:teuthology.orchestra.run.smithi001.stdout:Restarting the monitor... 2023-11-16T16:30:13.618 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:09 smithi001 ceph-mon[98290]: from='client.? 172.21.15.1:0/7548920' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:30:13.618 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:09 smithi001 systemd[1]: Stopping Ceph mon.a for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:30:13.619 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:09 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mon-a[98266]: 2023-11-16T16:30:09.702+0000 7fc751fee700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:30:13.619 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:09 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mon-a[98266]: 2023-11-16T16:30:09.702+0000 7fc751fee700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-11-16T16:30:13.619 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:10 smithi001 bash[98629]: ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mon-a 2023-11-16T16:30:13.619 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:10 smithi001 systemd[1]: ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mon.a.service: Succeeded. 2023-11-16T16:30:13.619 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:10 smithi001 systemd[1]: Stopped Ceph mon.a for 35d9b010-849d-11ee-8dbe-212e2dc638e7. 2023-11-16T16:30:13.619 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:10 smithi001 systemd[1]: Starting Ceph mon.a for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:30:13.619 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:10 smithi001 podman[98760]: 2023-11-16T16:30:13.619 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: set uid:gid to 167:167 (ceph:ceph) 2023-11-16T16:30:13.619 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: ceph version 18.0.0-3905-g5e4a743a (5e4a743ad7fd1422ebd6c4dc950a21a303279207) reef (dev), process ceph-mon, pid 7 2023-11-16T16:30:13.619 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: pidfile_write: ignore empty --pid-file 2023-11-16T16:30:13.619 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 bash[98760]: 86dd16c97ab9d768f062d91539d07853f2b2ed0ad679ae614d18ba8e191887d2 2023-11-16T16:30:13.619 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: load: jerasure load: lrc 2023-11-16T16:30:13.620 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: RocksDB version: 7.9.2 2023-11-16T16:30:13.620 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Git sha 0 2023-11-16T16:30:13.620 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Compile date 2023-11-15 23:53:54 2023-11-16T16:30:13.620 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: DB SUMMARY 2023-11-16T16:30:13.620 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: DB Session ID: NVBFWPOMKGGEFWLICZGG 2023-11-16T16:30:13.620 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: CURRENT file: CURRENT 2023-11-16T16:30:13.620 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: IDENTITY file: IDENTITY 2023-11-16T16:30:13.620 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-11-16T16:30:13.620 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-11-16T16:30:13.620 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 98653 ; 2023-11-16T16:30:13.620 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.error_if_exists: 0 2023-11-16T16:30:13.620 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.create_if_missing: 0 2023-11-16T16:30:13.620 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.paranoid_checks: 1 2023-11-16T16:30:13.620 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.flush_verify_memtable_count: 1 2023-11-16T16:30:13.621 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-11-16T16:30:13.621 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-11-16T16:30:13.621 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.env: 0x55f55ddbe720 2023-11-16T16:30:13.621 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.fs: PosixFileSystem 2023-11-16T16:30:13.621 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.info_log: 0x55f55fba93c0 2023-11-16T16:30:13.621 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.max_file_opening_threads: 16 2023-11-16T16:30:13.621 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.statistics: (nil) 2023-11-16T16:30:13.621 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.use_fsync: 0 2023-11-16T16:30:13.621 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.max_log_file_size: 0 2023-11-16T16:30:13.621 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-11-16T16:30:13.621 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.log_file_time_to_roll: 0 2023-11-16T16:30:13.621 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.keep_log_file_num: 1000 2023-11-16T16:30:13.621 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.recycle_log_file_num: 0 2023-11-16T16:30:13.622 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.allow_fallocate: 1 2023-11-16T16:30:13.622 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.allow_mmap_reads: 0 2023-11-16T16:30:13.622 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.allow_mmap_writes: 0 2023-11-16T16:30:13.622 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.use_direct_reads: 0 2023-11-16T16:30:13.622 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-11-16T16:30:13.622 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.create_missing_column_families: 0 2023-11-16T16:30:13.622 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.db_log_dir: 2023-11-16T16:30:13.622 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.wal_dir: 2023-11-16T16:30:13.622 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.table_cache_numshardbits: 6 2023-11-16T16:30:13.622 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.WAL_ttl_seconds: 0 2023-11-16T16:30:13.622 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.WAL_size_limit_MB: 0 2023-11-16T16:30:13.622 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-11-16T16:30:13.622 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-11-16T16:30:13.623 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.is_fd_close_on_exec: 1 2023-11-16T16:30:13.623 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.advise_random_on_open: 1 2023-11-16T16:30:13.623 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.db_write_buffer_size: 0 2023-11-16T16:30:13.623 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.write_buffer_manager: 0x55f55ee1a3c0 2023-11-16T16:30:13.623 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-11-16T16:30:13.623 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-11-16T16:30:13.623 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.use_adaptive_mutex: 0 2023-11-16T16:30:13.623 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.rate_limiter: (nil) 2023-11-16T16:30:13.623 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-11-16T16:30:13.623 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.wal_recovery_mode: 2 2023-11-16T16:30:13.623 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.enable_thread_tracking: 0 2023-11-16T16:30:13.623 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.enable_pipelined_write: 0 2023-11-16T16:30:13.623 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.unordered_write: 0 2023-11-16T16:30:13.624 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-11-16T16:30:13.624 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-11-16T16:30:13.624 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-11-16T16:30:13.624 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-11-16T16:30:13.624 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.row_cache: None 2023-11-16T16:30:13.624 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.wal_filter: None 2023-11-16T16:30:13.624 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-11-16T16:30:13.624 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.allow_ingest_behind: 0 2023-11-16T16:30:13.624 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.two_write_queues: 0 2023-11-16T16:30:13.624 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.manual_wal_flush: 0 2023-11-16T16:30:13.624 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.wal_compression: 0 2023-11-16T16:30:13.624 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.atomic_flush: 0 2023-11-16T16:30:13.624 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-11-16T16:30:13.624 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.persist_stats_to_disk: 0 2023-11-16T16:30:13.625 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.write_dbid_to_manifest: 0 2023-11-16T16:30:13.625 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.log_readahead_size: 0 2023-11-16T16:30:13.625 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-11-16T16:30:13.625 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.best_efforts_recovery: 0 2023-11-16T16:30:13.625 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-11-16T16:30:13.625 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-11-16T16:30:13.625 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.allow_data_in_errors: 0 2023-11-16T16:30:13.625 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.db_host_id: __hostname__ 2023-11-16T16:30:13.625 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.enforce_single_del_contracts: true 2023-11-16T16:30:13.625 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.max_background_jobs: 2 2023-11-16T16:30:13.625 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.max_background_compactions: -1 2023-11-16T16:30:13.625 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.max_subcompactions: 1 2023-11-16T16:30:13.625 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-11-16T16:30:13.625 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-11-16T16:30:13.626 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.delayed_write_rate : 16777216 2023-11-16T16:30:13.626 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.max_total_wal_size: 0 2023-11-16T16:30:13.626 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-11-16T16:30:13.626 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.stats_dump_period_sec: 600 2023-11-16T16:30:13.626 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.stats_persist_period_sec: 600 2023-11-16T16:30:13.626 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-11-16T16:30:13.626 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.max_open_files: -1 2023-11-16T16:30:13.626 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.bytes_per_sync: 0 2023-11-16T16:30:13.626 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.wal_bytes_per_sync: 0 2023-11-16T16:30:13.626 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.strict_bytes_per_sync: 0 2023-11-16T16:30:13.626 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.compaction_readahead_size: 0 2023-11-16T16:30:13.626 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.max_background_flushes: -1 2023-11-16T16:30:13.626 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Compression algorithms supported: 2023-11-16T16:30:13.627 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-11-16T16:30:13.627 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: kZSTD supported: 0 2023-11-16T16:30:13.627 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: kXpressCompression supported: 0 2023-11-16T16:30:13.627 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: kLZ4HCCompression supported: 1 2023-11-16T16:30:13.627 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: kZlibCompression supported: 1 2023-11-16T16:30:13.627 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: kSnappyCompression supported: 1 2023-11-16T16:30:13.627 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: kLZ4Compression supported: 1 2023-11-16T16:30:13.627 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: kBZip2Compression supported: 0 2023-11-16T16:30:13.627 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Fast CRC32 supported: Supported on x86 2023-11-16T16:30:13.627 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: DMutex implementation: pthread_mutex_t 2023-11-16T16:30:13.627 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-11-16T16:30:13.627 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-11-16T16:30:13.627 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-11-16T16:30:13.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.merge_operator: 2023-11-16T16:30:13.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.compaction_filter: None 2023-11-16T16:30:13.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.compaction_filter_factory: None 2023-11-16T16:30:13.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.sst_partitioner_factory: None 2023-11-16T16:30:13.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.memtable_factory: SkipListFactory 2023-11-16T16:30:13.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.table_factory: BlockBasedTable 2023-11-16T16:30:13.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f55fba94e0) 2023-11-16T16:30:13.628 INFO:journalctl@ceph.mon.a.smithi001.stdout: cache_index_and_filter_blocks: 1 2023-11-16T16:30:13.628 INFO:journalctl@ceph.mon.a.smithi001.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-11-16T16:30:13.628 INFO:journalctl@ceph.mon.a.smithi001.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-11-16T16:30:13.628 INFO:journalctl@ceph.mon.a.smithi001.stdout: pin_top_level_index_and_filter: 1 2023-11-16T16:30:13.628 INFO:journalctl@ceph.mon.a.smithi001.stdout: index_type: 0 2023-11-16T16:30:13.628 INFO:journalctl@ceph.mon.a.smithi001.stdout: data_block_index_type: 0 2023-11-16T16:30:13.628 INFO:journalctl@ceph.mon.a.smithi001.stdout: index_shortening: 1 2023-11-16T16:30:13.629 INFO:journalctl@ceph.mon.a.smithi001.stdout: data_block_hash_table_util_ratio: 0.750000 2023-11-16T16:30:13.629 INFO:journalctl@ceph.mon.a.smithi001.stdout: checksum: 4 2023-11-16T16:30:13.629 INFO:journalctl@ceph.mon.a.smithi001.stdout: no_block_cache: 0 2023-11-16T16:30:13.629 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_cache: 0x55f55eebb1f0 2023-11-16T16:30:13.629 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_cache_name: BinnedLRUCache 2023-11-16T16:30:13.629 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_cache_options: 2023-11-16T16:30:13.629 INFO:journalctl@ceph.mon.a.smithi001.stdout: capacity : 536870912 2023-11-16T16:30:13.629 INFO:journalctl@ceph.mon.a.smithi001.stdout: num_shard_bits : 4 2023-11-16T16:30:13.629 INFO:journalctl@ceph.mon.a.smithi001.stdout: strict_capacity_limit : 0 2023-11-16T16:30:13.629 INFO:journalctl@ceph.mon.a.smithi001.stdout: high_pri_pool_ratio: 0.000 2023-11-16T16:30:13.629 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_cache_compressed: (nil) 2023-11-16T16:30:13.629 INFO:journalctl@ceph.mon.a.smithi001.stdout: persistent_cache: (nil) 2023-11-16T16:30:13.629 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_size: 4096 2023-11-16T16:30:13.630 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_size_deviation: 10 2023-11-16T16:30:13.630 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_restart_interval: 16 2023-11-16T16:30:13.630 INFO:journalctl@ceph.mon.a.smithi001.stdout: index_block_restart_interval: 1 2023-11-16T16:30:13.630 INFO:journalctl@ceph.mon.a.smithi001.stdout: metadata_block_size: 4096 2023-11-16T16:30:13.630 INFO:journalctl@ceph.mon.a.smithi001.stdout: partition_filters: 0 2023-11-16T16:30:13.630 INFO:journalctl@ceph.mon.a.smithi001.stdout: use_delta_encoding: 1 2023-11-16T16:30:13.630 INFO:journalctl@ceph.mon.a.smithi001.stdout: filter_policy: bloomfilter 2023-11-16T16:30:13.630 INFO:journalctl@ceph.mon.a.smithi001.stdout: whole_key_filtering: 1 2023-11-16T16:30:13.630 INFO:journalctl@ceph.mon.a.smithi001.stdout: verify_compression: 0 2023-11-16T16:30:13.630 INFO:journalctl@ceph.mon.a.smithi001.stdout: read_amp_bytes_per_bit: 0 2023-11-16T16:30:13.630 INFO:journalctl@ceph.mon.a.smithi001.stdout: format_version: 5 2023-11-16T16:30:13.630 INFO:journalctl@ceph.mon.a.smithi001.stdout: enable_index_compression: 1 2023-11-16T16:30:13.630 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_align: 0 2023-11-16T16:30:13.631 INFO:journalctl@ceph.mon.a.smithi001.stdout: max_auto_readahead_size: 262144 2023-11-16T16:30:13.631 INFO:journalctl@ceph.mon.a.smithi001.stdout: prepopulate_block_cache: 0 2023-11-16T16:30:13.631 INFO:journalctl@ceph.mon.a.smithi001.stdout: initial_auto_readahead_size: 8192 2023-11-16T16:30:13.631 INFO:journalctl@ceph.mon.a.smithi001.stdout: num_file_reads_for_auto_readahead: 2 2023-11-16T16:30:13.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.write_buffer_size: 33554432 2023-11-16T16:30:13.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.max_write_buffer_number: 2 2023-11-16T16:30:13.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.compression: NoCompression 2023-11-16T16:30:13.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.bottommost_compression: Disabled 2023-11-16T16:30:13.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.prefix_extractor: nullptr 2023-11-16T16:30:13.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-11-16T16:30:13.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.num_levels: 7 2023-11-16T16:30:13.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-11-16T16:30:13.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-11-16T16:30:13.632 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-11-16T16:30:13.632 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-11-16T16:30:13.632 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-11-16T16:30:13.632 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-11-16T16:30:13.632 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-11-16T16:30:13.632 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-11-16T16:30:13.632 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-11-16T16:30:13.632 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-11-16T16:30:13.632 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-11-16T16:30:13.632 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-11-16T16:30:13.632 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.compression_opts.window_bits: -14 2023-11-16T16:30:13.632 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.compression_opts.level: 32767 2023-11-16T16:30:13.632 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.compression_opts.strategy: 0 2023-11-16T16:30:13.632 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-11-16T16:30:13.633 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-11-16T16:30:13.633 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-11-16T16:30:13.633 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-11-16T16:30:13.633 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.compression_opts.enabled: false 2023-11-16T16:30:13.633 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-11-16T16:30:13.633 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-11-16T16:30:13.633 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-11-16T16:30:13.633 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-11-16T16:30:13.633 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.target_file_size_base: 67108864 2023-11-16T16:30:13.633 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.target_file_size_multiplier: 1 2023-11-16T16:30:13.633 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-11-16T16:30:13.633 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-11-16T16:30:13.633 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-11-16T16:30:13.633 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-11-16T16:30:13.634 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-11-16T16:30:13.634 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-11-16T16:30:13.634 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-11-16T16:30:13.634 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-11-16T16:30:13.634 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-11-16T16:30:13.634 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-11-16T16:30:13.634 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-11-16T16:30:13.634 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-11-16T16:30:13.634 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-11-16T16:30:13.634 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.arena_block_size: 1048576 2023-11-16T16:30:13.634 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-11-16T16:30:13.634 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-11-16T16:30:13.634 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.disable_auto_compactions: 0 2023-11-16T16:30:13.635 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-11-16T16:30:13.635 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-11-16T16:30:13.635 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-11-16T16:30:13.635 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-11-16T16:30:13.635 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-11-16T16:30:13.635 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-11-16T16:30:13.635 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-11-16T16:30:13.635 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-11-16T16:30:13.635 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-11-16T16:30:13.635 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-11-16T16:30:13.635 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.table_properties_collectors: 2023-11-16T16:30:13.635 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.inplace_update_support: 0 2023-11-16T16:30:13.635 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.inplace_update_num_locks: 10000 2023-11-16T16:30:13.635 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-11-16T16:30:13.636 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-11-16T16:30:13.636 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.memtable_huge_page_size: 0 2023-11-16T16:30:13.636 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.bloom_locality: 0 2023-11-16T16:30:13.636 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.max_successive_merges: 0 2023-11-16T16:30:13.636 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.optimize_filters_for_hits: 0 2023-11-16T16:30:13.636 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.paranoid_file_checks: 0 2023-11-16T16:30:13.636 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.force_consistency_checks: 1 2023-11-16T16:30:13.636 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.report_bg_io_stats: 0 2023-11-16T16:30:13.636 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.ttl: 2592000 2023-11-16T16:30:13.636 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.periodic_compaction_seconds: 0 2023-11-16T16:30:13.636 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-11-16T16:30:13.636 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-11-16T16:30:13.636 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.enable_blob_files: false 2023-11-16T16:30:13.637 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.min_blob_size: 0 2023-11-16T16:30:13.637 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.blob_file_size: 268435456 2023-11-16T16:30:13.637 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.blob_compression_type: NoCompression 2023-11-16T16:30:13.637 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.enable_blob_garbage_collection: false 2023-11-16T16:30:13.637 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-11-16T16:30:13.637 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-11-16T16:30:13.637 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-11-16T16:30:13.637 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.blob_file_starting_level: 0 2023-11-16T16:30:13.637 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-11-16T16:30:13.637 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-11-16T16:30:13.637 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-11-16T16:30:13.637 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 7f0c43dd-a1a1-40bd-8442-eb16ccee88e5 2023-11-16T16:30:13.638 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: EVENT_LOG_v1 {"time_micros": 1700152211057589, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-11-16T16:30:13.638 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-11-16T16:30:13.638 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: EVENT_LOG_v1 {"time_micros": 1700152211059279, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93906, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 314, "table_properties": {"data_size": 91905, "index_size": 283, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13658, "raw_average_key_size": 50, "raw_value_size": 84446, "raw_average_value_size": 309, "num_data_blocks": 11, "num_entries": 273, "num_filter_entries": 273, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1700152211, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "7f0c43dd-a1a1-40bd-8442-eb16ccee88e5", "db_session_id": "NVBFWPOMKGGEFWLICZGG", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-11-16T16:30:13.638 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: EVENT_LOG_v1 {"time_micros": 1700152211059381, "job": 1, "event": "recovery_finished"} 2023-11-16T16:30:13.638 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-11-16T16:30:13.638 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-11-16T16:30:13.638 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f55ef58000 2023-11-16T16:30:13.638 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: DB pointer 0x55f55ef44000 2023-11-16T16:30:13.638 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-11-16T16:30:13.638 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-11-16T16:30:13.638 INFO:journalctl@ceph.mon.a.smithi001.stdout: ** DB Stats ** 2023-11-16T16:30:13.638 INFO:journalctl@ceph.mon.a.smithi001.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-16T16:30:13.639 INFO:journalctl@ceph.mon.a.smithi001.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-11-16T16:30:13.639 INFO:journalctl@ceph.mon.a.smithi001.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-16T16:30:13.639 INFO:journalctl@ceph.mon.a.smithi001.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-16T16:30:13.639 INFO:journalctl@ceph.mon.a.smithi001.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-11-16T16:30:13.639 INFO:journalctl@ceph.mon.a.smithi001.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-16T16:30:13.639 INFO:journalctl@ceph.mon.a.smithi001.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-16T16:30:13.639 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2023-11-16T16:30:13.639 INFO:journalctl@ceph.mon.a.smithi001.stdout: ** Compaction Stats [default] ** 2023-11-16T16:30:13.639 INFO:journalctl@ceph.mon.a.smithi001.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-16T16:30:13.639 INFO:journalctl@ceph.mon.a.smithi001.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-11-16T16:30:13.639 INFO:journalctl@ceph.mon.a.smithi001.stdout: L0 2/0 93.54 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 88.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-11-16T16:30:13.639 INFO:journalctl@ceph.mon.a.smithi001.stdout: Sum 2/0 93.54 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 88.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-11-16T16:30:13.639 INFO:journalctl@ceph.mon.a.smithi001.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 88.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-11-16T16:30:13.640 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2023-11-16T16:30:13.640 INFO:journalctl@ceph.mon.a.smithi001.stdout: ** Compaction Stats [default] ** 2023-11-16T16:30:13.640 INFO:journalctl@ceph.mon.a.smithi001.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-16T16:30:13.640 INFO:journalctl@ceph.mon.a.smithi001.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-11-16T16:30:13.640 INFO:journalctl@ceph.mon.a.smithi001.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 88.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-11-16T16:30:13.640 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2023-11-16T16:30:13.640 INFO:journalctl@ceph.mon.a.smithi001.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-11-16T16:30:13.640 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2023-11-16T16:30:13.640 INFO:journalctl@ceph.mon.a.smithi001.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-16T16:30:13.640 INFO:journalctl@ceph.mon.a.smithi001.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-11-16T16:30:13.640 INFO:journalctl@ceph.mon.a.smithi001.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-11-16T16:30:13.640 INFO:journalctl@ceph.mon.a.smithi001.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-11-16T16:30:13.640 INFO:journalctl@ceph.mon.a.smithi001.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-11-16T16:30:13.640 INFO:journalctl@ceph.mon.a.smithi001.stdout: AddFile(Keys): cumulative 0, interval 0 2023-11-16T16:30:13.641 INFO:journalctl@ceph.mon.a.smithi001.stdout: Cumulative compaction: 0.00 GB write, 18.76 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-16T16:30:13.641 INFO:journalctl@ceph.mon.a.smithi001.stdout: Interval compaction: 0.00 GB write, 18.76 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-16T16:30:13.641 INFO:journalctl@ceph.mon.a.smithi001.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-11-16T16:30:13.641 INFO:journalctl@ceph.mon.a.smithi001.stdout: Block cache BinnedLRUCache@0x55f55eebb1f0#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-11-16T16:30:13.641 INFO:journalctl@ceph.mon.a.smithi001.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-11-16T16:30:13.641 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2023-11-16T16:30:13.641 INFO:journalctl@ceph.mon.a.smithi001.stdout: ** File Read Latency Histogram By Level [default] ** 2023-11-16T16:30:13.641 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: starting mon.a rank 0 at public addrs [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] at bind addrs [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 2023-11-16T16:30:13.641 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: mon.a@-1(???) e1 preinit fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 2023-11-16T16:30:13.641 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: mon.a@-1(???).mds e1 new map 2023-11-16T16:30:13.641 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: mon.a@-1(???).mds e1 print_map 2023-11-16T16:30:13.641 INFO:journalctl@ceph.mon.a.smithi001.stdout: e1 2023-11-16T16:30:13.641 INFO:journalctl@ceph.mon.a.smithi001.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-11-16T16:30:13.642 INFO:journalctl@ceph.mon.a.smithi001.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-11-16T16:30:13.642 INFO:journalctl@ceph.mon.a.smithi001.stdout: legacy client fscid: -1 2023-11-16T16:30:13.642 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2023-11-16T16:30:13.642 INFO:journalctl@ceph.mon.a.smithi001.stdout: No filesystems configured 2023-11-16T16:30:13.642 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-11-16T16:30:13.642 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-11-16T16:30:13.642 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-11-16T16:30:13.642 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-11-16T16:30:13.642 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-11-16T16:30:13.642 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: expand_channel_meta expand map: {default=false} 2023-11-16T16:30:13.642 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: expand_channel_meta from 'false' to 'false' 2023-11-16T16:30:13.642 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: expand_channel_meta expanded map: {default=false} 2023-11-16T16:30:13.642 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: expand_channel_meta expand map: {default=info} 2023-11-16T16:30:13.643 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: expand_channel_meta from 'info' to 'info' 2023-11-16T16:30:13.643 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: expand_channel_meta expanded map: {default=info} 2023-11-16T16:30:13.643 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: expand_channel_meta expand map: {default=daemon} 2023-11-16T16:30:13.643 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: expand_channel_meta from 'daemon' to 'daemon' 2023-11-16T16:30:13.643 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: expand_channel_meta expanded map: {default=daemon} 2023-11-16T16:30:13.643 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: expand_channel_meta expand map: {default=debug} 2023-11-16T16:30:13.643 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: expand_channel_meta from 'debug' to 'debug' 2023-11-16T16:30:13.643 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: expand_channel_meta expanded map: {default=debug} 2023-11-16T16:30:13.643 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: mon.a is new leader, mons a in quorum (ranks 0) 2023-11-16T16:30:13.643 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: monmap e1: 1 mons at {a=[v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0]} removed_ranks: {} 2023-11-16T16:30:13.643 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: fsmap 2023-11-16T16:30:13.643 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: osdmap e1: 0 total, 0 up, 0 in 2023-11-16T16:30:13.643 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 ceph-mon[98800]: mgrmap e1: no daemons active 2023-11-16T16:30:13.644 INFO:teuthology.orchestra.run.smithi001.stdout:Setting mon public_network to 172.21.0.0/20 2023-11-16T16:30:13.644 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:11 smithi001 systemd[1]: Started Ceph mon.a for 35d9b010-849d-11ee-8dbe-212e2dc638e7. 2023-11-16T16:30:13.648 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.643+0000 7f6640548700 1 Processor -- start 2023-11-16T16:30:13.648 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.643+0000 7f6640548700 1 -- start start 2023-11-16T16:30:13.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.644+0000 7f6640548700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6638104920 0x7f6638104d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:13.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.644+0000 7f6640548700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6638105300 con 0x7f6638104920 2023-11-16T16:30:13.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.644+0000 7f663e2e4700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6638104920 0x7f6638104d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:13.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.644+0000 7f663e2e4700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6638104920 0x7f6638104d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40106/0 (socket says 172.21.15.1:40106) 2023-11-16T16:30:13.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.644+0000 7f663e2e4700 1 -- 172.21.15.1:0/520227991 learned_addr learned my addr 172.21.15.1:0/520227991 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:13.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.645+0000 7f663e2e4700 1 -- 172.21.15.1:0/520227991 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6638105b20 con 0x7f6638104920 2023-11-16T16:30:13.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.645+0000 7f663e2e4700 1 --2- 172.21.15.1:0/520227991 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6638104920 0x7f6638104d30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f66340098a0 tx=0x7f6634009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ffedd48cae906a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:13.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.645+0000 7f663d2e2700 1 -- 172.21.15.1:0/520227991 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6634004160 con 0x7f6638104920 2023-11-16T16:30:13.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.645+0000 7f663d2e2700 1 -- 172.21.15.1:0/520227991 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f66340042c0 con 0x7f6638104920 2023-11-16T16:30:13.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.645+0000 7f663d2e2700 1 -- 172.21.15.1:0/520227991 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6634011a00 con 0x7f6638104920 2023-11-16T16:30:13.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.646+0000 7f6640548700 1 -- 172.21.15.1:0/520227991 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6638104920 msgr2=0x7f6638104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:13.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.646+0000 7f6640548700 1 --2- 172.21.15.1:0/520227991 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6638104920 0x7f6638104d30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f66340098a0 tx=0x7f6634009bb0 comp rx=0 tx=0).stop 2023-11-16T16:30:13.649 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.646+0000 7f6640548700 1 -- 172.21.15.1:0/520227991 shutdown_connections 2023-11-16T16:30:13.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.646+0000 7f6640548700 1 --2- 172.21.15.1:0/520227991 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6638104920 0x7f6638104d30 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:13.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.646+0000 7f6640548700 1 -- 172.21.15.1:0/520227991 >> 172.21.15.1:0/520227991 conn(0x7f66380ffed0 msgr2=0x7f6638102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:13.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.646+0000 7f6640548700 1 -- 172.21.15.1:0/520227991 shutdown_connections 2023-11-16T16:30:13.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.646+0000 7f6640548700 1 -- 172.21.15.1:0/520227991 wait complete. 2023-11-16T16:30:13.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.646+0000 7f6640548700 1 Processor -- start 2023-11-16T16:30:13.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.646+0000 7f6640548700 1 -- start start 2023-11-16T16:30:13.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.647+0000 7f6640548700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6638104920 0x7f663818fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:13.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.647+0000 7f6640548700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6638190260 con 0x7f6638104920 2023-11-16T16:30:13.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.647+0000 7f663e2e4700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6638104920 0x7f663818fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:13.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.647+0000 7f663e2e4700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6638104920 0x7f663818fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40108/0 (socket says 172.21.15.1:40108) 2023-11-16T16:30:13.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.647+0000 7f663e2e4700 1 -- 172.21.15.1:0/4196144284 learned_addr learned my addr 172.21.15.1:0/4196144284 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:13.650 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.647+0000 7f663e2e4700 1 -- 172.21.15.1:0/4196144284 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6634009550 con 0x7f6638104920 2023-11-16T16:30:13.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.647+0000 7f663e2e4700 1 --2- 172.21.15.1:0/4196144284 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6638104920 0x7f663818fd20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6634006af0 tx=0x7f6634011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:13.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.647+0000 7f662f7fe700 1 -- 172.21.15.1:0/4196144284 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6634003a40 con 0x7f6638104920 2023-11-16T16:30:13.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.647+0000 7f662f7fe700 1 -- 172.21.15.1:0/4196144284 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6634003ba0 con 0x7f6638104920 2023-11-16T16:30:13.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.647+0000 7f6640548700 1 -- 172.21.15.1:0/4196144284 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6638108d50 con 0x7f6638104920 2023-11-16T16:30:13.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.647+0000 7f662f7fe700 1 -- 172.21.15.1:0/4196144284 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f663401a700 con 0x7f6638104920 2023-11-16T16:30:13.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.647+0000 7f6640548700 1 -- 172.21.15.1:0/4196144284 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6638108ee0 con 0x7f6638104920 2023-11-16T16:30:13.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.648+0000 7f662f7fe700 1 -- 172.21.15.1:0/4196144284 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f663401ac30 con 0x7f6638104920 2023-11-16T16:30:13.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.648+0000 7f662f7fe700 1 -- 172.21.15.1:0/4196144284 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6634019700 con 0x7f6638104920 2023-11-16T16:30:13.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.648+0000 7f6640548700 1 -- 172.21.15.1:0/4196144284 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6638106990 con 0x7f6638104920 2023-11-16T16:30:13.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.651+0000 7f662f7fe700 1 -- 172.21.15.1:0/4196144284 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f6638106990 con 0x7f6638104920 2023-11-16T16:30:13.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.776+0000 7f6640548700 1 -- 172.21.15.1:0/4196144284 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f663818eb00 con 0x7f6638104920 2023-11-16T16:30:13.651 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.779+0000 7f662f7fe700 1 -- 172.21.15.1:0/4196144284 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f663402de40 con 0x7f6638104920 2023-11-16T16:30:13.652 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.780+0000 7f6640548700 1 -- 172.21.15.1:0/4196144284 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6638104920 msgr2=0x7f663818fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:13.652 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.780+0000 7f6640548700 1 --2- 172.21.15.1:0/4196144284 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6638104920 0x7f663818fd20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6634006af0 tx=0x7f6634011ad0 comp rx=0 tx=0).stop 2023-11-16T16:30:13.652 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.780+0000 7f6640548700 1 -- 172.21.15.1:0/4196144284 shutdown_connections 2023-11-16T16:30:13.652 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.780+0000 7f6640548700 1 --2- 172.21.15.1:0/4196144284 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6638104920 0x7f663818fd20 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:13.652 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.780+0000 7f6640548700 1 -- 172.21.15.1:0/4196144284 >> 172.21.15.1:0/4196144284 conn(0x7f66380ffed0 msgr2=0x7f6638079890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:13.652 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.780+0000 7f6640548700 1 -- 172.21.15.1:0/4196144284 shutdown_connections 2023-11-16T16:30:13.652 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:11.780+0000 7f6640548700 1 -- 172.21.15.1:0/4196144284 wait complete. 2023-11-16T16:30:13.652 INFO:teuthology.orchestra.run.smithi001.stdout:Wrote config to /etc/ceph/ceph.conf 2023-11-16T16:30:13.652 INFO:teuthology.orchestra.run.smithi001.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-11-16T16:30:13.652 INFO:teuthology.orchestra.run.smithi001.stdout:Creating mgr... 2023-11-16T16:30:13.652 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying port 9283 ... 2023-11-16T16:30:13.652 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying port 8765 ... 2023-11-16T16:30:13.652 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mgr.x 2023-11-16T16:30:13.652 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Failed to reset failed state of unit ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mgr.x.service: Unit ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mgr.x.service not loaded. 2023-11-16T16:30:13.653 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7.target.wants/ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mgr.x.service → /etc/systemd/system/ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@.service. 2023-11-16T16:30:13.653 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:12 smithi001 systemd[1]: Starting Ceph mgr.x for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:30:13.653 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:12 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/4196144284' entity='client.admin' 2023-11-16T16:30:13.653 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:12 smithi001 podman[99027]: 2023-11-16T16:30:13.655 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-11-16T16:30:13.655 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout disabled 2023-11-16T16:30:13.655 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-11-16T16:30:13.655 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2023-11-16T16:30:13.655 INFO:teuthology.orchestra.run.smithi001.stdout:firewalld.service is not enabled 2023-11-16T16:30:13.655 INFO:teuthology.orchestra.run.smithi001.stdout:Not possible to enable service . firewalld.service is not available 2023-11-16T16:30:13.655 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-11-16T16:30:13.656 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout disabled 2023-11-16T16:30:13.656 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-11-16T16:30:13.656 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2023-11-16T16:30:13.656 INFO:teuthology.orchestra.run.smithi001.stdout:firewalld.service is not enabled 2023-11-16T16:30:13.656 INFO:teuthology.orchestra.run.smithi001.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-11-16T16:30:13.656 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mgr to start... 2023-11-16T16:30:13.656 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mgr... 2023-11-16T16:30:13.656 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:12 smithi001 bash[99027]: f1661433f61118ae542974e22f1706bbc5d5818522f8d8620b05cc9d3dc21ba9 2023-11-16T16:30:13.656 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:12 smithi001 systemd[1]: Started Ceph mgr.x for 35d9b010-849d-11ee-8dbe-212e2dc638e7. 2023-11-16T16:30:13.656 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:13 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:13.052+0000 7f251662e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-16T16:30:13.657 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:13 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:13.148+0000 7f251662e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-16T16:30:13.657 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:13 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:13.262+0000 7f251662e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-16T16:30:13.657 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:13 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:13.375+0000 7f251662e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-16T16:30:13.657 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:13 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:13.586+0000 7f251662e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-16T16:30:13.890 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:13 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/2696675992' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-11-16T16:30:14.121 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2023-11-16T16:30:14.121 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2023-11-16T16:30:14.121 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsid": "35d9b010-849d-11ee-8dbe-212e2dc638e7", 2023-11-16T16:30:14.121 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "health": { 2023-11-16T16:30:14.121 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-11-16T16:30:14.121 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-11-16T16:30:14.121 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-11-16T16:30:14.121 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:30:14.121 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-11-16T16:30:14.122 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-11-16T16:30:14.122 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 0 2023-11-16T16:30:14.122 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2023-11-16T16:30:14.122 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-11-16T16:30:14.122 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "a" 2023-11-16T16:30:14.122 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2023-11-16T16:30:14.122 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2023-11-16T16:30:14.122 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "monmap": { 2023-11-16T16:30:14.122 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T16:30:14.122 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-11-16T16:30:14.122 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-11-16T16:30:14.122 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:30:14.123 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-11-16T16:30:14.123 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T16:30:14.123 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-11-16T16:30:14.123 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-11-16T16:30:14.123 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-11-16T16:30:14.123 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-11-16T16:30:14.130 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-11-16T16:30:14.130 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-11-16T16:30:14.130 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:30:14.130 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-11-16T16:30:14.130 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-11-16T16:30:14.130 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-11-16T16:30:14.130 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-11-16T16:30:14.130 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-11-16T16:30:14.130 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-11-16T16:30:14.130 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-11-16T16:30:14.131 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-11-16T16:30:14.131 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-11-16T16:30:14.131 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:30:14.131 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-11-16T16:30:14.131 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T16:30:14.131 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-11-16T16:30:14.131 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-11-16T16:30:14.131 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:30:14.131 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-11-16T16:30:14.131 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "available": false, 2023-11-16T16:30:14.131 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-11-16T16:30:14.131 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modules": [ 2023-11-16T16:30:14.131 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "iostat", 2023-11-16T16:30:14.132 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "nfs", 2023-11-16T16:30:14.132 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "restful" 2023-11-16T16:30:14.132 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2023-11-16T16:30:14.132 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-16T16:30:14.132 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:30:14.132 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-11-16T16:30:14.132 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T16:30:14.132 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modified": "2023-11-16T16:30:06.218445+0000", 2023-11-16T16:30:14.132 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-16T16:30:14.132 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:30:14.132 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-11-16T16:30:14.132 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2023-11-16T16:30:14.132 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.515+0000 7f719730f700 1 Processor -- start 2023-11-16T16:30:14.133 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.515+0000 7f719730f700 1 -- start start 2023-11-16T16:30:14.133 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.515+0000 7f719730f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7190106490 0x7f7190108870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:14.133 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.515+0000 7f719730f700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7190079e60 con 0x7f7190106490 2023-11-16T16:30:14.133 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.516+0000 7f71950ab700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7190106490 0x7f7190108870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:14.133 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.516+0000 7f71950ab700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7190106490 0x7f7190108870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40114/0 (socket says 172.21.15.1:40114) 2023-11-16T16:30:14.133 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.516+0000 7f71950ab700 1 -- 172.21.15.1:0/2806277274 learned_addr learned my addr 172.21.15.1:0/2806277274 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:14.133 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.516+0000 7f71950ab700 1 -- 172.21.15.1:0/2806277274 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7190108db0 con 0x7f7190106490 2023-11-16T16:30:14.133 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.516+0000 7f71950ab700 1 --2- 172.21.15.1:0/2806277274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7190106490 0x7f7190108870 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f718c0098a0 tx=0x7f718c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a00c3f4087f2eea server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:14.133 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.516+0000 7f7187fff700 1 -- 172.21.15.1:0/2806277274 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f718c004160 con 0x7f7190106490 2023-11-16T16:30:14.133 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.516+0000 7f7187fff700 1 -- 172.21.15.1:0/2806277274 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f718c0042c0 con 0x7f7190106490 2023-11-16T16:30:14.133 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.516+0000 7f7187fff700 1 -- 172.21.15.1:0/2806277274 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f718c011a00 con 0x7f7190106490 2023-11-16T16:30:14.133 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.517+0000 7f719730f700 1 -- 172.21.15.1:0/2806277274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7190106490 msgr2=0x7f7190108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:14.133 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.517+0000 7f719730f700 1 --2- 172.21.15.1:0/2806277274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7190106490 0x7f7190108870 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f718c0098a0 tx=0x7f718c009bb0 comp rx=0 tx=0).stop 2023-11-16T16:30:14.134 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.517+0000 7f719730f700 1 -- 172.21.15.1:0/2806277274 shutdown_connections 2023-11-16T16:30:14.134 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.517+0000 7f719730f700 1 --2- 172.21.15.1:0/2806277274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7190106490 0x7f7190108870 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:14.134 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.517+0000 7f719730f700 1 -- 172.21.15.1:0/2806277274 >> 172.21.15.1:0/2806277274 conn(0x7f71900ffed0 msgr2=0x7f7190102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:14.134 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.517+0000 7f719730f700 1 -- 172.21.15.1:0/2806277274 shutdown_connections 2023-11-16T16:30:14.134 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.517+0000 7f719730f700 1 -- 172.21.15.1:0/2806277274 wait complete. 2023-11-16T16:30:14.134 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.518+0000 7f719730f700 1 Processor -- start 2023-11-16T16:30:14.134 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.518+0000 7f719730f700 1 -- start start 2023-11-16T16:30:14.134 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.518+0000 7f719730f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7190106490 0x7f7190105df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:14.134 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.518+0000 7f719730f700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7190106330 con 0x7f7190106490 2023-11-16T16:30:14.134 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.518+0000 7f71950ab700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7190106490 0x7f7190105df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:14.134 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.518+0000 7f71950ab700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7190106490 0x7f7190105df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40116/0 (socket says 172.21.15.1:40116) 2023-11-16T16:30:14.135 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.518+0000 7f71950ab700 1 -- 172.21.15.1:0/2696675992 learned_addr learned my addr 172.21.15.1:0/2696675992 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:14.135 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.519+0000 7f71950ab700 1 -- 172.21.15.1:0/2696675992 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f718c009550 con 0x7f7190106490 2023-11-16T16:30:14.135 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.519+0000 7f71950ab700 1 --2- 172.21.15.1:0/2696675992 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7190106490 0x7f7190105df0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f718c009fb0 tx=0x7f718c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:14.135 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.519+0000 7f71867fc700 1 -- 172.21.15.1:0/2696675992 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f718c003930 con 0x7f7190106490 2023-11-16T16:30:14.135 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.519+0000 7f71867fc700 1 -- 172.21.15.1:0/2696675992 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f718c003a90 con 0x7f7190106490 2023-11-16T16:30:14.135 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.519+0000 7f719730f700 1 -- 172.21.15.1:0/2696675992 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7190109d40 con 0x7f7190106490 2023-11-16T16:30:14.135 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.519+0000 7f71867fc700 1 -- 172.21.15.1:0/2696675992 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f718c019610 con 0x7f7190106490 2023-11-16T16:30:14.135 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.519+0000 7f719730f700 1 -- 172.21.15.1:0/2696675992 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7190109bb0 con 0x7f7190106490 2023-11-16T16:30:14.135 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.519+0000 7f71867fc700 1 -- 172.21.15.1:0/2696675992 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f718c019a70 con 0x7f7190106490 2023-11-16T16:30:14.135 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.519+0000 7f71867fc700 1 -- 172.21.15.1:0/2696675992 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f718c0243f0 con 0x7f7190106490 2023-11-16T16:30:14.135 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.520+0000 7f719730f700 1 -- 172.21.15.1:0/2696675992 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7190062120 con 0x7f7190106490 2023-11-16T16:30:14.135 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.522+0000 7f71867fc700 1 -- 172.21.15.1:0/2696675992 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f718c003c00 con 0x7f7190106490 2023-11-16T16:30:14.135 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.656+0000 7f719730f700 1 -- 172.21.15.1:0/2696675992 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f7190109ed0 con 0x7f7190106490 2023-11-16T16:30:14.136 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.656+0000 7f71867fc700 1 -- 172.21.15.1:0/2696675992 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f718c01a9d0 con 0x7f7190106490 2023-11-16T16:30:14.136 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.657+0000 7f719730f700 1 -- 172.21.15.1:0/2696675992 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7190106490 msgr2=0x7f7190105df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:14.136 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.657+0000 7f719730f700 1 --2- 172.21.15.1:0/2696675992 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7190106490 0x7f7190105df0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f718c009fb0 tx=0x7f718c011ad0 comp rx=0 tx=0).stop 2023-11-16T16:30:14.136 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.657+0000 7f719730f700 1 -- 172.21.15.1:0/2696675992 shutdown_connections 2023-11-16T16:30:14.136 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.657+0000 7f719730f700 1 --2- 172.21.15.1:0/2696675992 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7190106490 0x7f7190105df0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:14.136 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.657+0000 7f719730f700 1 -- 172.21.15.1:0/2696675992 >> 172.21.15.1:0/2696675992 conn(0x7f71900ffed0 msgr2=0x7f7190100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:14.136 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.657+0000 7f719730f700 1 -- 172.21.15.1:0/2696675992 shutdown_connections 2023-11-16T16:30:14.136 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:13.657+0000 7f719730f700 1 -- 172.21.15.1:0/2696675992 wait complete. 2023-11-16T16:30:14.136 INFO:teuthology.orchestra.run.smithi001.stdout:mgr not available, waiting (1/15)... 2023-11-16T16:30:14.182 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:13 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:13.994+0000 7f251662e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-16T16:30:14.182 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:14 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:14.102+0000 7f251662e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-16T16:30:14.682 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:14 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:14.254+0000 7f251662e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-16T16:30:14.682 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:14 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:14.369+0000 7f251662e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-16T16:30:15.182 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:14 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:14.878+0000 7f251662e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-16T16:30:15.932 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:15 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:15.572+0000 7f251662e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-16T16:30:15.932 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:15 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:15.764+0000 7f251662e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-16T16:30:17.070 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:16 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:16.738+0000 7f251662e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-16T16:30:17.071 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:16 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:16.851+0000 7f251662e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-16T16:30:17.172 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2023-11-16T16:30:17.173 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2023-11-16T16:30:17.173 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsid": "35d9b010-849d-11ee-8dbe-212e2dc638e7", 2023-11-16T16:30:17.173 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "health": { 2023-11-16T16:30:17.173 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-11-16T16:30:17.173 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-11-16T16:30:17.174 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-11-16T16:30:17.174 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:30:17.174 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-11-16T16:30:17.174 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-11-16T16:30:17.174 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 0 2023-11-16T16:30:17.174 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2023-11-16T16:30:17.174 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-11-16T16:30:17.174 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "a" 2023-11-16T16:30:17.175 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2023-11-16T16:30:17.176 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2023-11-16T16:30:17.176 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "monmap": { 2023-11-16T16:30:17.176 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T16:30:17.176 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-11-16T16:30:17.176 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-11-16T16:30:17.176 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:30:17.176 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-11-16T16:30:17.176 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T16:30:17.176 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-11-16T16:30:17.176 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-11-16T16:30:17.176 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-11-16T16:30:17.184 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-11-16T16:30:17.184 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-11-16T16:30:17.184 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-11-16T16:30:17.184 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:30:17.184 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-11-16T16:30:17.184 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-11-16T16:30:17.185 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-11-16T16:30:17.185 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-11-16T16:30:17.185 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-11-16T16:30:17.185 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-11-16T16:30:17.185 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-11-16T16:30:17.185 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-11-16T16:30:17.185 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-11-16T16:30:17.185 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:30:17.185 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-11-16T16:30:17.185 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T16:30:17.185 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-11-16T16:30:17.185 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-11-16T16:30:17.185 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:30:17.186 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-11-16T16:30:17.186 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "available": false, 2023-11-16T16:30:17.186 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-11-16T16:30:17.186 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modules": [ 2023-11-16T16:30:17.186 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "iostat", 2023-11-16T16:30:17.186 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "nfs", 2023-11-16T16:30:17.186 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "restful" 2023-11-16T16:30:17.186 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2023-11-16T16:30:17.186 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-16T16:30:17.186 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:30:17.186 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-11-16T16:30:17.186 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T16:30:17.187 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modified": "2023-11-16T16:30:06.218445+0000", 2023-11-16T16:30:17.187 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-16T16:30:17.187 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:30:17.187 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-11-16T16:30:17.187 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2023-11-16T16:30:17.187 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.641+0000 7f992c871700 1 Processor -- start 2023-11-16T16:30:17.187 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.642+0000 7f992c871700 1 -- start start 2023-11-16T16:30:17.187 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.642+0000 7f992c871700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9928105a20 0x7f9928105e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:17.187 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.642+0000 7f992c871700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9928106400 con 0x7f9928105a20 2023-11-16T16:30:17.187 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.642+0000 7f992659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9928105a20 0x7f9928105e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:17.187 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.642+0000 7f992659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9928105a20 0x7f9928105e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40118/0 (socket says 172.21.15.1:40118) 2023-11-16T16:30:17.187 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.642+0000 7f992659c700 1 -- 172.21.15.1:0/691047371 learned_addr learned my addr 172.21.15.1:0/691047371 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:17.188 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.643+0000 7f992659c700 1 -- 172.21.15.1:0/691047371 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9928106c20 con 0x7f9928105a20 2023-11-16T16:30:17.188 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.643+0000 7f992659c700 1 --2- 172.21.15.1:0/691047371 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9928105a20 0x7f9928105e30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f99180098a0 tx=0x7f9918009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc1993caf6096e68 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:17.188 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.643+0000 7f992559a700 1 -- 172.21.15.1:0/691047371 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9918004160 con 0x7f9928105a20 2023-11-16T16:30:17.188 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.643+0000 7f992559a700 1 -- 172.21.15.1:0/691047371 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f99180042c0 con 0x7f9928105a20 2023-11-16T16:30:17.188 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.643+0000 7f992559a700 1 -- 172.21.15.1:0/691047371 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9918011a00 con 0x7f9928105a20 2023-11-16T16:30:17.188 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.644+0000 7f992c871700 1 -- 172.21.15.1:0/691047371 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9928105a20 msgr2=0x7f9928105e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:17.188 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.644+0000 7f992c871700 1 --2- 172.21.15.1:0/691047371 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9928105a20 0x7f9928105e30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f99180098a0 tx=0x7f9918009bb0 comp rx=0 tx=0).stop 2023-11-16T16:30:17.188 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.644+0000 7f992c871700 1 -- 172.21.15.1:0/691047371 shutdown_connections 2023-11-16T16:30:17.188 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.644+0000 7f992c871700 1 --2- 172.21.15.1:0/691047371 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9928105a20 0x7f9928105e30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:17.188 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.644+0000 7f992c871700 1 -- 172.21.15.1:0/691047371 >> 172.21.15.1:0/691047371 conn(0x7f9928100fd0 msgr2=0x7f9928103400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:17.188 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.644+0000 7f992c871700 1 -- 172.21.15.1:0/691047371 shutdown_connections 2023-11-16T16:30:17.189 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.644+0000 7f992c871700 1 -- 172.21.15.1:0/691047371 wait complete. 2023-11-16T16:30:17.189 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.645+0000 7f992c871700 1 Processor -- start 2023-11-16T16:30:17.189 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.645+0000 7f992c871700 1 -- start start 2023-11-16T16:30:17.189 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.645+0000 7f992c871700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9928105a20 0x7f9928199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:17.189 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.645+0000 7f992c871700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9928199e70 con 0x7f9928105a20 2023-11-16T16:30:17.189 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.645+0000 7f992659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9928105a20 0x7f9928199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:17.189 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.645+0000 7f992659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9928105a20 0x7f9928199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40120/0 (socket says 172.21.15.1:40120) 2023-11-16T16:30:17.189 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.645+0000 7f992659c700 1 -- 172.21.15.1:0/3422265321 learned_addr learned my addr 172.21.15.1:0/3422265321 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:17.189 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.645+0000 7f992659c700 1 -- 172.21.15.1:0/3422265321 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9918009550 con 0x7f9928105a20 2023-11-16T16:30:17.189 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.646+0000 7f992659c700 1 --2- 172.21.15.1:0/3422265321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9928105a20 0x7f9928199930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9918006af0 tx=0x7f9918011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:17.189 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.646+0000 7f99177fe700 1 -- 172.21.15.1:0/3422265321 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9918003a40 con 0x7f9928105a20 2023-11-16T16:30:17.189 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.646+0000 7f99177fe700 1 -- 172.21.15.1:0/3422265321 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9918003ba0 con 0x7f9928105a20 2023-11-16T16:30:17.190 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.646+0000 7f99177fe700 1 -- 172.21.15.1:0/3422265321 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f991801a700 con 0x7f9928105a20 2023-11-16T16:30:17.190 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.646+0000 7f992c871700 1 -- 172.21.15.1:0/3422265321 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9928109e50 con 0x7f9928105a20 2023-11-16T16:30:17.190 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.646+0000 7f992c871700 1 -- 172.21.15.1:0/3422265321 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9928109fe0 con 0x7f9928105a20 2023-11-16T16:30:17.190 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.646+0000 7f99177fe700 1 -- 172.21.15.1:0/3422265321 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f991801ac80 con 0x7f9928105a20 2023-11-16T16:30:17.190 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.646+0000 7f99177fe700 1 -- 172.21.15.1:0/3422265321 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9918019700 con 0x7f9928105a20 2023-11-16T16:30:17.190 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.647+0000 7f992c871700 1 -- 172.21.15.1:0/3422265321 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f992810a170 con 0x7f9928105a20 2023-11-16T16:30:17.190 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.649+0000 7f99177fe700 1 -- 172.21.15.1:0/3422265321 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f9918024ad0 con 0x7f9928105a20 2023-11-16T16:30:17.190 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.781+0000 7f992c871700 1 -- 172.21.15.1:0/3422265321 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f992819a3b0 con 0x7f9928105a20 2023-11-16T16:30:17.190 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.782+0000 7f99177fe700 1 -- 172.21.15.1:0/3422265321 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f992819a3b0 con 0x7f9928105a20 2023-11-16T16:30:17.190 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.783+0000 7f992c871700 1 -- 172.21.15.1:0/3422265321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9928105a20 msgr2=0x7f9928199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:17.190 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.783+0000 7f992c871700 1 --2- 172.21.15.1:0/3422265321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9928105a20 0x7f9928199930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9918006af0 tx=0x7f9918011ad0 comp rx=0 tx=0).stop 2023-11-16T16:30:17.191 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.783+0000 7f992c871700 1 -- 172.21.15.1:0/3422265321 shutdown_connections 2023-11-16T16:30:17.191 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.783+0000 7f992c871700 1 --2- 172.21.15.1:0/3422265321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9928105a20 0x7f9928199930 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:17.191 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.783+0000 7f992c871700 1 -- 172.21.15.1:0/3422265321 >> 172.21.15.1:0/3422265321 conn(0x7f9928100fd0 msgr2=0x7f9928102640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:17.191 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.783+0000 7f992c871700 1 -- 172.21.15.1:0/3422265321 shutdown_connections 2023-11-16T16:30:17.191 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:16.783+0000 7f992c871700 1 -- 172.21.15.1:0/3422265321 wait complete. 2023-11-16T16:30:17.191 INFO:teuthology.orchestra.run.smithi001.stdout:mgr not available, waiting (2/15)... 2023-11-16T16:30:17.425 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:17 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3422265321' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-11-16T16:30:17.425 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:17 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:17.072+0000 7f251662e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-16T16:30:17.675 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:17 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:17.422+0000 7f251662e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-16T16:30:17.932 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:17 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:17.673+0000 7f251662e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-16T16:30:17.932 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:17 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:17.853+0000 7f251662e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-16T16:30:18.432 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:18 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:18.179+0000 7f251662e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-16T16:30:19.408 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:18 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:18.997+0000 7f251662e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-16T16:30:19.408 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:19 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:19.153+0000 7f251662e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-16T16:30:19.682 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:19 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:19.405+0000 7f251662e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-16T16:30:19.682 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:19 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:19.547+0000 7f251662e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-16T16:30:20.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:19 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/516448842' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-11-16T16:30:20.183 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:19 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:19.820+0000 7f251662e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-16T16:30:20.345 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2023-11-16T16:30:20.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2023-11-16T16:30:20.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsid": "35d9b010-849d-11ee-8dbe-212e2dc638e7", 2023-11-16T16:30:20.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "health": { 2023-11-16T16:30:20.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-11-16T16:30:20.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-11-16T16:30:20.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-11-16T16:30:20.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:30:20.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-11-16T16:30:20.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-11-16T16:30:20.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 0 2023-11-16T16:30:20.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2023-11-16T16:30:20.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-11-16T16:30:20.347 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "a" 2023-11-16T16:30:20.347 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2023-11-16T16:30:20.347 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2023-11-16T16:30:20.347 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "monmap": { 2023-11-16T16:30:20.347 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T16:30:20.347 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-11-16T16:30:20.347 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-11-16T16:30:20.347 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:30:20.347 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-11-16T16:30:20.347 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T16:30:20.347 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-11-16T16:30:20.347 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-11-16T16:30:20.347 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-11-16T16:30:20.348 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-11-16T16:30:20.348 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-11-16T16:30:20.348 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-11-16T16:30:20.348 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:30:20.348 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-11-16T16:30:20.348 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-11-16T16:30:20.348 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-11-16T16:30:20.348 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-11-16T16:30:20.348 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-11-16T16:30:20.348 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-11-16T16:30:20.348 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-11-16T16:30:20.348 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-11-16T16:30:20.348 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-11-16T16:30:20.349 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:30:20.349 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-11-16T16:30:20.349 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T16:30:20.349 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-11-16T16:30:20.349 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-11-16T16:30:20.349 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:30:20.349 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-11-16T16:30:20.349 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "available": false, 2023-11-16T16:30:20.349 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-11-16T16:30:20.349 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modules": [ 2023-11-16T16:30:20.349 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "iostat", 2023-11-16T16:30:20.350 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "nfs", 2023-11-16T16:30:20.350 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "restful" 2023-11-16T16:30:20.350 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2023-11-16T16:30:20.350 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-16T16:30:20.350 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:30:20.350 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-11-16T16:30:20.350 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T16:30:20.350 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modified": "2023-11-16T16:30:06.218445+0000", 2023-11-16T16:30:20.350 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-16T16:30:20.350 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:30:20.350 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-11-16T16:30:20.351 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2023-11-16T16:30:20.351 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.756+0000 7f85592bf700 1 Processor -- start 2023-11-16T16:30:20.351 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.756+0000 7f85592bf700 1 -- start start 2023-11-16T16:30:20.351 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.756+0000 7f85592bf700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8554104920 0x7f8554104d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:20.351 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.756+0000 7f85592bf700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8554105300 con 0x7f8554104920 2023-11-16T16:30:20.351 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.756+0000 7f8552ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8554104920 0x7f8554104d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:20.351 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.756+0000 7f8552ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8554104920 0x7f8554104d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40122/0 (socket says 172.21.15.1:40122) 2023-11-16T16:30:20.351 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.756+0000 7f8552ffd700 1 -- 172.21.15.1:0/768515647 learned_addr learned my addr 172.21.15.1:0/768515647 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:20.351 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.757+0000 7f8552ffd700 1 -- 172.21.15.1:0/768515647 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8554105b20 con 0x7f8554104920 2023-11-16T16:30:20.351 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.757+0000 7f8552ffd700 1 --2- 172.21.15.1:0/768515647 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8554104920 0x7f8554104d30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f85440098a0 tx=0x7f8544009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f98bf318bfdde256 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:20.351 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.757+0000 7f8551ffb700 1 -- 172.21.15.1:0/768515647 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8544004160 con 0x7f8554104920 2023-11-16T16:30:20.351 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.757+0000 7f8551ffb700 1 -- 172.21.15.1:0/768515647 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f85440042c0 con 0x7f8554104920 2023-11-16T16:30:20.352 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.757+0000 7f8551ffb700 1 -- 172.21.15.1:0/768515647 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8544011a00 con 0x7f8554104920 2023-11-16T16:30:20.352 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.758+0000 7f85592bf700 1 -- 172.21.15.1:0/768515647 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8554104920 msgr2=0x7f8554104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:20.352 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.758+0000 7f85592bf700 1 --2- 172.21.15.1:0/768515647 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8554104920 0x7f8554104d30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f85440098a0 tx=0x7f8544009bb0 comp rx=0 tx=0).stop 2023-11-16T16:30:20.352 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.758+0000 7f85592bf700 1 -- 172.21.15.1:0/768515647 shutdown_connections 2023-11-16T16:30:20.352 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.758+0000 7f85592bf700 1 --2- 172.21.15.1:0/768515647 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8554104920 0x7f8554104d30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:20.352 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.758+0000 7f85592bf700 1 -- 172.21.15.1:0/768515647 >> 172.21.15.1:0/768515647 conn(0x7f85540ffed0 msgr2=0x7f8554102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:20.352 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.758+0000 7f85592bf700 1 -- 172.21.15.1:0/768515647 shutdown_connections 2023-11-16T16:30:20.352 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.758+0000 7f85592bf700 1 -- 172.21.15.1:0/768515647 wait complete. 2023-11-16T16:30:20.352 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.759+0000 7f85592bf700 1 Processor -- start 2023-11-16T16:30:20.352 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.759+0000 7f85592bf700 1 -- start start 2023-11-16T16:30:20.352 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.759+0000 7f85592bf700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8554104920 0x7f8554199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:20.352 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.759+0000 7f85592bf700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8554199e80 con 0x7f8554104920 2023-11-16T16:30:20.353 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.759+0000 7f8552ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8554104920 0x7f8554199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:20.353 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.759+0000 7f8552ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8554104920 0x7f8554199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40124/0 (socket says 172.21.15.1:40124) 2023-11-16T16:30:20.353 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.759+0000 7f8552ffd700 1 -- 172.21.15.1:0/516448842 learned_addr learned my addr 172.21.15.1:0/516448842 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:20.353 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.760+0000 7f8552ffd700 1 -- 172.21.15.1:0/516448842 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8544009550 con 0x7f8554104920 2023-11-16T16:30:20.353 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.760+0000 7f8552ffd700 1 --2- 172.21.15.1:0/516448842 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8554104920 0x7f8554199940 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8544006af0 tx=0x7f8544011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:20.353 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.760+0000 7f853bfff700 1 -- 172.21.15.1:0/516448842 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8544003a40 con 0x7f8554104920 2023-11-16T16:30:20.353 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.760+0000 7f85592bf700 1 -- 172.21.15.1:0/516448842 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8554108d50 con 0x7f8554104920 2023-11-16T16:30:20.353 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.760+0000 7f853bfff700 1 -- 172.21.15.1:0/516448842 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8544003ba0 con 0x7f8554104920 2023-11-16T16:30:20.353 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.760+0000 7f853bfff700 1 -- 172.21.15.1:0/516448842 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f854401a700 con 0x7f8554104920 2023-11-16T16:30:20.353 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.760+0000 7f85592bf700 1 -- 172.21.15.1:0/516448842 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8554108ee0 con 0x7f8554104920 2023-11-16T16:30:20.353 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.760+0000 7f853bfff700 1 -- 172.21.15.1:0/516448842 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f854401ac80 con 0x7f8554104920 2023-11-16T16:30:20.353 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.760+0000 7f853bfff700 1 -- 172.21.15.1:0/516448842 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8544019700 con 0x7f8554104920 2023-11-16T16:30:20.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.761+0000 7f85592bf700 1 -- 172.21.15.1:0/516448842 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8554106990 con 0x7f8554104920 2023-11-16T16:30:20.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.763+0000 7f853bfff700 1 -- 172.21.15.1:0/516448842 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f8554106990 con 0x7f8554104920 2023-11-16T16:30:20.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.896+0000 7f85592bf700 1 -- 172.21.15.1:0/516448842 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f855419a3c0 con 0x7f8554104920 2023-11-16T16:30:20.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.897+0000 7f853bfff700 1 -- 172.21.15.1:0/516448842 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f8544024ad0 con 0x7f8554104920 2023-11-16T16:30:20.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.898+0000 7f85592bf700 1 -- 172.21.15.1:0/516448842 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8554104920 msgr2=0x7f8554199940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:20.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.898+0000 7f85592bf700 1 --2- 172.21.15.1:0/516448842 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8554104920 0x7f8554199940 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8544006af0 tx=0x7f8544011ad0 comp rx=0 tx=0).stop 2023-11-16T16:30:20.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.898+0000 7f85592bf700 1 -- 172.21.15.1:0/516448842 shutdown_connections 2023-11-16T16:30:20.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.898+0000 7f85592bf700 1 --2- 172.21.15.1:0/516448842 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8554104920 0x7f8554199940 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:20.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.898+0000 7f85592bf700 1 -- 172.21.15.1:0/516448842 >> 172.21.15.1:0/516448842 conn(0x7f85540ffed0 msgr2=0x7f85540798b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:20.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.898+0000 7f85592bf700 1 -- 172.21.15.1:0/516448842 shutdown_connections 2023-11-16T16:30:20.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:19.898+0000 7f85592bf700 1 -- 172.21.15.1:0/516448842 wait complete. 2023-11-16T16:30:20.354 INFO:teuthology.orchestra.run.smithi001.stdout:mgr not available, waiting (3/15)... 2023-11-16T16:30:21.182 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:20 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:20.868+0000 7f251662e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-16T16:30:21.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:20 smithi001 ceph-mon[98800]: Activating manager daemon x 2023-11-16T16:30:21.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:20 smithi001 ceph-mon[98800]: mgrmap e2: x(active, starting, since 0.00308719s) 2023-11-16T16:30:21.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:20 smithi001 ceph-mon[98800]: from='mgr.14100 172.21.15.1:0/4119886884' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-11-16T16:30:21.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:20 smithi001 ceph-mon[98800]: from='mgr.14100 172.21.15.1:0/4119886884' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-11-16T16:30:21.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:20 smithi001 ceph-mon[98800]: from='mgr.14100 172.21.15.1:0/4119886884' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-11-16T16:30:21.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:20 smithi001 ceph-mon[98800]: from='mgr.14100 172.21.15.1:0/4119886884' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T16:30:21.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:20 smithi001 ceph-mon[98800]: from='mgr.14100 172.21.15.1:0/4119886884' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-11-16T16:30:21.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:20 smithi001 ceph-mon[98800]: Manager daemon x is now available 2023-11-16T16:30:21.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:20 smithi001 ceph-mon[98800]: from='mgr.14100 172.21.15.1:0/4119886884' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-11-16T16:30:21.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:20 smithi001 ceph-mon[98800]: from='mgr.14100 172.21.15.1:0/4119886884' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-11-16T16:30:21.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:20 smithi001 ceph-mon[98800]: from='mgr.14100 172.21.15.1:0/4119886884' entity='mgr.x' 2023-11-16T16:30:21.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:20 smithi001 ceph-mon[98800]: from='mgr.14100 172.21.15.1:0/4119886884' entity='mgr.x' 2023-11-16T16:30:21.184 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:20 smithi001 ceph-mon[98800]: from='mgr.14100 172.21.15.1:0/4119886884' entity='mgr.x' 2023-11-16T16:30:22.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:22 smithi001 ceph-mon[98800]: mgrmap e3: x(active, since 1.00751s) 2023-11-16T16:30:23.896 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2023-11-16T16:30:23.896 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2023-11-16T16:30:23.896 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsid": "35d9b010-849d-11ee-8dbe-212e2dc638e7", 2023-11-16T16:30:23.896 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "health": { 2023-11-16T16:30:23.896 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-11-16T16:30:23.896 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-11-16T16:30:23.896 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-11-16T16:30:23.896 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:30:23.897 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-11-16T16:30:23.897 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-11-16T16:30:23.897 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 0 2023-11-16T16:30:23.897 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2023-11-16T16:30:23.897 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-11-16T16:30:23.897 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "a" 2023-11-16T16:30:23.897 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2023-11-16T16:30:23.897 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2023-11-16T16:30:23.897 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "monmap": { 2023-11-16T16:30:23.898 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T16:30:23.898 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-11-16T16:30:23.898 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-11-16T16:30:23.898 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:30:23.898 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-11-16T16:30:23.898 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T16:30:23.898 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-11-16T16:30:23.898 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-11-16T16:30:23.898 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-11-16T16:30:23.906 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-11-16T16:30:23.906 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-11-16T16:30:23.906 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-11-16T16:30:23.906 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:30:23.906 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-11-16T16:30:23.906 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-11-16T16:30:23.906 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-11-16T16:30:23.906 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-11-16T16:30:23.907 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-11-16T16:30:23.907 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-11-16T16:30:23.907 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-11-16T16:30:23.907 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-11-16T16:30:23.907 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-11-16T16:30:23.907 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:30:23.907 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-11-16T16:30:23.907 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T16:30:23.907 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-11-16T16:30:23.907 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-11-16T16:30:23.907 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:30:23.907 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-11-16T16:30:23.908 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "available": true, 2023-11-16T16:30:23.908 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-11-16T16:30:23.908 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modules": [ 2023-11-16T16:30:23.908 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "iostat", 2023-11-16T16:30:23.908 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "nfs", 2023-11-16T16:30:23.908 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "restful" 2023-11-16T16:30:23.908 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2023-11-16T16:30:23.908 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-16T16:30:23.908 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:30:23.908 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-11-16T16:30:23.908 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-16T16:30:23.908 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modified": "2023-11-16T16:30:06.218445+0000", 2023-11-16T16:30:23.908 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-16T16:30:23.909 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2023-11-16T16:30:23.909 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-11-16T16:30:23.909 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2023-11-16T16:30:23.909 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.979+0000 7f5f83c8d700 1 Processor -- start 2023-11-16T16:30:23.909 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.979+0000 7f5f83c8d700 1 -- start start 2023-11-16T16:30:23.909 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.979+0000 7f5f83c8d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f7c104300 0x7f5f7c104710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:23.909 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.979+0000 7f5f83c8d700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c079de0 con 0x7f5f7c104300 2023-11-16T16:30:23.909 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.980+0000 7f5f81a29700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f7c104300 0x7f5f7c104710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:23.909 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.980+0000 7f5f81a29700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f7c104300 0x7f5f7c104710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40150/0 (socket says 172.21.15.1:40150) 2023-11-16T16:30:23.909 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.980+0000 7f5f81a29700 1 -- 172.21.15.1:0/1045201804 learned_addr learned my addr 172.21.15.1:0/1045201804 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:23.909 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.980+0000 7f5f81a29700 1 -- 172.21.15.1:0/1045201804 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7c104c50 con 0x7f5f7c104300 2023-11-16T16:30:23.909 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.980+0000 7f5f81a29700 1 --2- 172.21.15.1:0/1045201804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f7c104300 0x7f5f7c104710 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5f780098a0 tx=0x7f5f78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b99dc287e2289ee6 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:23.910 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.980+0000 7f5f80a27700 1 -- 172.21.15.1:0/1045201804 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f78004160 con 0x7f5f7c104300 2023-11-16T16:30:23.910 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.980+0000 7f5f80a27700 1 -- 172.21.15.1:0/1045201804 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5f780042c0 con 0x7f5f7c104300 2023-11-16T16:30:23.910 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.980+0000 7f5f80a27700 1 -- 172.21.15.1:0/1045201804 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f78011a00 con 0x7f5f7c104300 2023-11-16T16:30:23.910 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.981+0000 7f5f83c8d700 1 -- 172.21.15.1:0/1045201804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f7c104300 msgr2=0x7f5f7c104710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:23.910 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.981+0000 7f5f83c8d700 1 --2- 172.21.15.1:0/1045201804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f7c104300 0x7f5f7c104710 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5f780098a0 tx=0x7f5f78009bb0 comp rx=0 tx=0).stop 2023-11-16T16:30:23.910 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.981+0000 7f5f83c8d700 1 -- 172.21.15.1:0/1045201804 shutdown_connections 2023-11-16T16:30:23.910 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.981+0000 7f5f83c8d700 1 --2- 172.21.15.1:0/1045201804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f7c104300 0x7f5f7c104710 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:23.910 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.981+0000 7f5f83c8d700 1 -- 172.21.15.1:0/1045201804 >> 172.21.15.1:0/1045201804 conn(0x7f5f7c0ffed0 msgr2=0x7f5f7c102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:23.910 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.981+0000 7f5f83c8d700 1 -- 172.21.15.1:0/1045201804 shutdown_connections 2023-11-16T16:30:23.910 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.981+0000 7f5f83c8d700 1 -- 172.21.15.1:0/1045201804 wait complete. 2023-11-16T16:30:23.910 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.982+0000 7f5f83c8d700 1 Processor -- start 2023-11-16T16:30:23.911 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.982+0000 7f5f83c8d700 1 -- start start 2023-11-16T16:30:23.911 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.982+0000 7f5f83c8d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f7c104300 0x7f5f7c109e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:23.911 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.982+0000 7f5f83c8d700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c10a3c0 con 0x7f5f7c104300 2023-11-16T16:30:23.911 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.982+0000 7f5f81a29700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f7c104300 0x7f5f7c109e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:23.911 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.982+0000 7f5f81a29700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f7c104300 0x7f5f7c109e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40152/0 (socket says 172.21.15.1:40152) 2023-11-16T16:30:23.911 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.982+0000 7f5f81a29700 1 -- 172.21.15.1:0/1295864661 learned_addr learned my addr 172.21.15.1:0/1295864661 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:23.911 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.982+0000 7f5f81a29700 1 -- 172.21.15.1:0/1295864661 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f78009550 con 0x7f5f7c104300 2023-11-16T16:30:23.911 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.982+0000 7f5f81a29700 1 --2- 172.21.15.1:0/1295864661 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f7c104300 0x7f5f7c109e80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5f78009fb0 tx=0x7f5f78011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:23.911 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.983+0000 7f5f6effd700 1 -- 172.21.15.1:0/1295864661 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f78003930 con 0x7f5f7c104300 2023-11-16T16:30:23.911 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.983+0000 7f5f6effd700 1 -- 172.21.15.1:0/1295864661 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5f78003a90 con 0x7f5f7c104300 2023-11-16T16:30:23.911 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.983+0000 7f5f83c8d700 1 -- 172.21.15.1:0/1295864661 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f7c105c50 con 0x7f5f7c104300 2023-11-16T16:30:23.911 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.983+0000 7f5f6effd700 1 -- 172.21.15.1:0/1295864661 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f78019610 con 0x7f5f7c104300 2023-11-16T16:30:23.912 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.983+0000 7f5f83c8d700 1 -- 172.21.15.1:0/1295864661 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f7c105ac0 con 0x7f5f7c104300 2023-11-16T16:30:23.912 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.983+0000 7f5f6effd700 1 -- 172.21.15.1:0/1295864661 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 3) v1 ==== 45366+0+0 (secure 0 0 0) 0x7f5f780198d0 con 0x7f5f7c104300 2023-11-16T16:30:23.912 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.984+0000 7f5f6effd700 1 --2- 172.21.15.1:0/1295864661 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f5f68038550 0x7f5f6803aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:23.912 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.984+0000 7f5f6effd700 1 -- 172.21.15.1:0/1295864661 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5f7804c290 con 0x7f5f7c104300 2023-11-16T16:30:23.912 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.984+0000 7f5f81228700 1 --2- 172.21.15.1:0/1295864661 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f5f68038550 0x7f5f6803aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:23.912 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.984+0000 7f5f83c8d700 1 -- 172.21.15.1:0/1295864661 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f7c105de0 con 0x7f5f7c104300 2023-11-16T16:30:23.912 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.984+0000 7f5f81228700 1 --2- 172.21.15.1:0/1295864661 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f5f68038550 0x7f5f6803aa00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5f70006fd0 tx=0x7f5f70006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:23.912 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:22.989+0000 7f5f6effd700 1 -- 172.21.15.1:0/1295864661 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5f78019b80 con 0x7f5f7c104300 2023-11-16T16:30:23.912 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:23.501+0000 7f5f83c8d700 1 -- 172.21.15.1:0/1295864661 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f5f7c1074d0 con 0x7f5f7c104300 2023-11-16T16:30:23.912 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:23.502+0000 7f5f6effd700 1 -- 172.21.15.1:0/1295864661 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f5f78018b40 con 0x7f5f7c104300 2023-11-16T16:30:23.912 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:23.504+0000 7f5f83c8d700 1 -- 172.21.15.1:0/1295864661 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f5f68038550 msgr2=0x7f5f6803aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:23.913 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:23.504+0000 7f5f83c8d700 1 --2- 172.21.15.1:0/1295864661 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f5f68038550 0x7f5f6803aa00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5f70006fd0 tx=0x7f5f70006e40 comp rx=0 tx=0).stop 2023-11-16T16:30:23.913 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:23.504+0000 7f5f83c8d700 1 -- 172.21.15.1:0/1295864661 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f7c104300 msgr2=0x7f5f7c109e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:23.913 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:23.504+0000 7f5f83c8d700 1 --2- 172.21.15.1:0/1295864661 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f7c104300 0x7f5f7c109e80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5f78009fb0 tx=0x7f5f78011ad0 comp rx=0 tx=0).stop 2023-11-16T16:30:23.913 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:23.504+0000 7f5f83c8d700 1 -- 172.21.15.1:0/1295864661 shutdown_connections 2023-11-16T16:30:23.913 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:23.504+0000 7f5f83c8d700 1 --2- 172.21.15.1:0/1295864661 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f5f68038550 0x7f5f6803aa00 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:23.913 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:23.504+0000 7f5f83c8d700 1 --2- 172.21.15.1:0/1295864661 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f7c104300 0x7f5f7c109e80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:23.913 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:23.504+0000 7f5f83c8d700 1 -- 172.21.15.1:0/1295864661 >> 172.21.15.1:0/1295864661 conn(0x7f5f7c0ffed0 msgr2=0x7f5f7c18fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:23.913 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:23.504+0000 7f5f83c8d700 1 -- 172.21.15.1:0/1295864661 shutdown_connections 2023-11-16T16:30:23.913 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:23.504+0000 7f5f83c8d700 1 -- 172.21.15.1:0/1295864661 wait complete. 2023-11-16T16:30:23.913 INFO:teuthology.orchestra.run.smithi001.stdout:mgr is available 2023-11-16T16:30:24.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:23 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1295864661' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-11-16T16:30:25.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:24 smithi001 ceph-mon[98800]: mgrmap e4: x(active, since 3s) 2023-11-16T16:30:25.349 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2023-11-16T16:30:25.350 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [global] 2023-11-16T16:30:25.350 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout fsid = 35d9b010-849d-11ee-8dbe-212e2dc638e7 2023-11-16T16:30:25.350 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-11-16T16:30:25.350 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-11-16T16:30:25.350 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-11-16T16:30:25.350 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-11-16T16:30:25.350 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2023-11-16T16:30:25.350 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [mgr] 2023-11-16T16:30:25.350 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-11-16T16:30:25.351 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2023-11-16T16:30:25.351 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [osd] 2023-11-16T16:30:25.351 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-11-16T16:30:25.351 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-11-16T16:30:25.351 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-11-16T16:30:25.351 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-11-16T16:30:25.351 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-11-16T16:30:25.351 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.532+0000 7f8a658c6700 1 Processor -- start 2023-11-16T16:30:25.351 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.532+0000 7f8a658c6700 1 -- start start 2023-11-16T16:30:25.351 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.532+0000 7f8a658c6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a60106570 0x7f8a60108950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:25.352 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.532+0000 7f8a658c6700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a60079d80 con 0x7f8a60106570 2023-11-16T16:30:25.352 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.533+0000 7f8a5effd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a60106570 0x7f8a60108950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:25.352 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.533+0000 7f8a5effd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a60106570 0x7f8a60108950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40156/0 (socket says 172.21.15.1:40156) 2023-11-16T16:30:25.352 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.533+0000 7f8a5effd700 1 -- 172.21.15.1:0/1620592719 learned_addr learned my addr 172.21.15.1:0/1620592719 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:25.352 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.533+0000 7f8a5effd700 1 -- 172.21.15.1:0/1620592719 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a60079ec0 con 0x7f8a60106570 2023-11-16T16:30:25.352 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.533+0000 7f8a5effd700 1 --2- 172.21.15.1:0/1620592719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a60106570 0x7f8a60108950 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8a540098a0 tx=0x7f8a54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ed12124ac6b4e06 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:25.352 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.533+0000 7f8a5dffb700 1 -- 172.21.15.1:0/1620592719 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a54004160 con 0x7f8a60106570 2023-11-16T16:30:25.352 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.533+0000 7f8a5dffb700 1 -- 172.21.15.1:0/1620592719 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8a540042c0 con 0x7f8a60106570 2023-11-16T16:30:25.352 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.533+0000 7f8a5dffb700 1 -- 172.21.15.1:0/1620592719 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a54011a00 con 0x7f8a60106570 2023-11-16T16:30:25.352 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.534+0000 7f8a658c6700 1 -- 172.21.15.1:0/1620592719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a60106570 msgr2=0x7f8a60108950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:25.352 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.534+0000 7f8a658c6700 1 --2- 172.21.15.1:0/1620592719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a60106570 0x7f8a60108950 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8a540098a0 tx=0x7f8a54009bb0 comp rx=0 tx=0).stop 2023-11-16T16:30:25.352 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.534+0000 7f8a658c6700 1 -- 172.21.15.1:0/1620592719 shutdown_connections 2023-11-16T16:30:25.353 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.534+0000 7f8a658c6700 1 --2- 172.21.15.1:0/1620592719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a60106570 0x7f8a60108950 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:25.353 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.534+0000 7f8a658c6700 1 -- 172.21.15.1:0/1620592719 >> 172.21.15.1:0/1620592719 conn(0x7f8a600ffed0 msgr2=0x7f8a60102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:25.353 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.534+0000 7f8a658c6700 1 -- 172.21.15.1:0/1620592719 shutdown_connections 2023-11-16T16:30:25.353 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.534+0000 7f8a658c6700 1 -- 172.21.15.1:0/1620592719 wait complete. 2023-11-16T16:30:25.353 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.535+0000 7f8a658c6700 1 Processor -- start 2023-11-16T16:30:25.353 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.535+0000 7f8a658c6700 1 -- start start 2023-11-16T16:30:25.353 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.535+0000 7f8a658c6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a60106570 0x7f8a60199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:25.353 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.535+0000 7f8a658c6700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a60199ca0 con 0x7f8a60106570 2023-11-16T16:30:25.353 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.535+0000 7f8a5effd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a60106570 0x7f8a60199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:25.353 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.535+0000 7f8a5effd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a60106570 0x7f8a60199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40158/0 (socket says 172.21.15.1:40158) 2023-11-16T16:30:25.353 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.535+0000 7f8a5effd700 1 -- 172.21.15.1:0/2690453667 learned_addr learned my addr 172.21.15.1:0/2690453667 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:25.353 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.535+0000 7f8a5effd700 1 -- 172.21.15.1:0/2690453667 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a54009550 con 0x7f8a60106570 2023-11-16T16:30:25.353 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.536+0000 7f8a5effd700 1 --2- 172.21.15.1:0/2690453667 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a60106570 0x7f8a60199760 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8a54006af0 tx=0x7f8a54011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:25.353 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.536+0000 7f8a648c4700 1 -- 172.21.15.1:0/2690453667 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a54003a40 con 0x7f8a60106570 2023-11-16T16:30:25.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.536+0000 7f8a658c6700 1 -- 172.21.15.1:0/2690453667 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a60109d50 con 0x7f8a60106570 2023-11-16T16:30:25.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.536+0000 7f8a648c4700 1 -- 172.21.15.1:0/2690453667 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8a54003ba0 con 0x7f8a60106570 2023-11-16T16:30:25.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.536+0000 7f8a648c4700 1 -- 172.21.15.1:0/2690453667 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a5401a6e0 con 0x7f8a60106570 2023-11-16T16:30:25.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.536+0000 7f8a658c6700 1 -- 172.21.15.1:0/2690453667 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a60109bc0 con 0x7f8a60106570 2023-11-16T16:30:25.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.537+0000 7f8a648c4700 1 -- 172.21.15.1:0/2690453667 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7f8a5401a950 con 0x7f8a60106570 2023-11-16T16:30:25.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.537+0000 7f8a648c4700 1 --2- 172.21.15.1:0/2690453667 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f8a480386c0 0x7f8a4803ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:25.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.537+0000 7f8a648c4700 1 -- 172.21.15.1:0/2690453667 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8a5404c680 con 0x7f8a60106570 2023-11-16T16:30:25.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.537+0000 7f8a658c6700 1 -- 172.21.15.1:0/2690453667 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a60062120 con 0x7f8a60106570 2023-11-16T16:30:25.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.537+0000 7f8a5e7fc700 1 --2- 172.21.15.1:0/2690453667 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f8a480386c0 0x7f8a4803ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:25.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.538+0000 7f8a5e7fc700 1 --2- 172.21.15.1:0/2690453667 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f8a480386c0 0x7f8a4803ab70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8a50006fd0 tx=0x7f8a50006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:25.354 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.542+0000 7f8a648c4700 1 -- 172.21.15.1:0/2690453667 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8a5401ac00 con 0x7f8a60106570 2023-11-16T16:30:25.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.901+0000 7f8a658c6700 1 -- 172.21.15.1:0/2690453667 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f8a6019a1e0 con 0x7f8a60106570 2023-11-16T16:30:25.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.902+0000 7f8a648c4700 1 -- 172.21.15.1:0/2690453667 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f8a54018350 con 0x7f8a60106570 2023-11-16T16:30:25.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.904+0000 7f8a658c6700 1 -- 172.21.15.1:0/2690453667 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f8a480386c0 msgr2=0x7f8a4803ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:25.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.904+0000 7f8a658c6700 1 --2- 172.21.15.1:0/2690453667 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f8a480386c0 0x7f8a4803ab70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8a50006fd0 tx=0x7f8a50006e40 comp rx=0 tx=0).stop 2023-11-16T16:30:25.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.904+0000 7f8a658c6700 1 -- 172.21.15.1:0/2690453667 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a60106570 msgr2=0x7f8a60199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:25.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.904+0000 7f8a658c6700 1 --2- 172.21.15.1:0/2690453667 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a60106570 0x7f8a60199760 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8a54006af0 tx=0x7f8a54011ad0 comp rx=0 tx=0).stop 2023-11-16T16:30:25.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.905+0000 7f8a658c6700 1 -- 172.21.15.1:0/2690453667 shutdown_connections 2023-11-16T16:30:25.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.905+0000 7f8a658c6700 1 --2- 172.21.15.1:0/2690453667 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f8a480386c0 0x7f8a4803ab70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:25.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.905+0000 7f8a658c6700 1 --2- 172.21.15.1:0/2690453667 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8a60106570 0x7f8a60199760 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:25.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.905+0000 7f8a658c6700 1 -- 172.21.15.1:0/2690453667 >> 172.21.15.1:0/2690453667 conn(0x7f8a600ffed0 msgr2=0x7f8a60100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:25.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.905+0000 7f8a658c6700 1 -- 172.21.15.1:0/2690453667 shutdown_connections 2023-11-16T16:30:25.355 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:24.905+0000 7f8a658c6700 1 -- 172.21.15.1:0/2690453667 wait complete. 2023-11-16T16:30:25.356 INFO:teuthology.orchestra.run.smithi001.stdout:Enabling cephadm module... 2023-11-16T16:30:26.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:25 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/2690453667' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-11-16T16:30:27.364 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:27 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3929501152' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-11-16T16:30:27.365 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:27 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: ignoring --setuser ceph since I am not root 2023-11-16T16:30:27.365 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:27 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: ignoring --setgroup ceph since I am not root 2023-11-16T16:30:27.365 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:27 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:27.267+0000 7f05e9d1c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-16T16:30:27.496 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.821+0000 7f44c7b03700 1 Processor -- start 2023-11-16T16:30:27.496 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.821+0000 7f44c7b03700 1 -- start start 2023-11-16T16:30:27.496 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.821+0000 7f44c7b03700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44c01042e0 0x7f44c01046f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:27.496 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.821+0000 7f44c7b03700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c0079e00 con 0x7f44c01042e0 2023-11-16T16:30:27.496 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.821+0000 7f44c589f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44c01042e0 0x7f44c01046f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:27.496 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.821+0000 7f44c589f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44c01042e0 0x7f44c01046f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40162/0 (socket says 172.21.15.1:40162) 2023-11-16T16:30:27.496 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.821+0000 7f44c589f700 1 -- 172.21.15.1:0/2823955706 learned_addr learned my addr 172.21.15.1:0/2823955706 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:27.497 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.822+0000 7f44c589f700 1 -- 172.21.15.1:0/2823955706 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c0104c30 con 0x7f44c01042e0 2023-11-16T16:30:27.497 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.822+0000 7f44c589f700 1 --2- 172.21.15.1:0/2823955706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44c01042e0 0x7f44c01046f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f44b40098a0 tx=0x7f44b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e53b41b995c6a964 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:27.497 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.822+0000 7f44c489d700 1 -- 172.21.15.1:0/2823955706 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f44b4004160 con 0x7f44c01042e0 2023-11-16T16:30:27.497 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.822+0000 7f44c489d700 1 -- 172.21.15.1:0/2823955706 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f44b40042c0 con 0x7f44c01042e0 2023-11-16T16:30:27.497 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.822+0000 7f44c489d700 1 -- 172.21.15.1:0/2823955706 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f44b4011a00 con 0x7f44c01042e0 2023-11-16T16:30:27.497 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.823+0000 7f44c7b03700 1 -- 172.21.15.1:0/2823955706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44c01042e0 msgr2=0x7f44c01046f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:27.497 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.823+0000 7f44c7b03700 1 --2- 172.21.15.1:0/2823955706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44c01042e0 0x7f44c01046f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f44b40098a0 tx=0x7f44b4009bb0 comp rx=0 tx=0).stop 2023-11-16T16:30:27.497 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.823+0000 7f44c7b03700 1 -- 172.21.15.1:0/2823955706 shutdown_connections 2023-11-16T16:30:27.497 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.823+0000 7f44c7b03700 1 --2- 172.21.15.1:0/2823955706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44c01042e0 0x7f44c01046f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:27.497 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.823+0000 7f44c7b03700 1 -- 172.21.15.1:0/2823955706 >> 172.21.15.1:0/2823955706 conn(0x7f44c00ffed0 msgr2=0x7f44c0102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:27.498 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.823+0000 7f44c7b03700 1 -- 172.21.15.1:0/2823955706 shutdown_connections 2023-11-16T16:30:27.498 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.823+0000 7f44c7b03700 1 -- 172.21.15.1:0/2823955706 wait complete. 2023-11-16T16:30:27.498 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.824+0000 7f44c7b03700 1 Processor -- start 2023-11-16T16:30:27.498 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.824+0000 7f44c7b03700 1 -- start start 2023-11-16T16:30:27.498 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.824+0000 7f44c7b03700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44c01042e0 0x7f44c019b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:27.498 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.824+0000 7f44c7b03700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c019bec0 con 0x7f44c01042e0 2023-11-16T16:30:27.498 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.824+0000 7f44c589f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44c01042e0 0x7f44c019b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:27.498 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.824+0000 7f44c589f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44c01042e0 0x7f44c019b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40164/0 (socket says 172.21.15.1:40164) 2023-11-16T16:30:27.498 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.824+0000 7f44c589f700 1 -- 172.21.15.1:0/3929501152 learned_addr learned my addr 172.21.15.1:0/3929501152 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:27.498 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.824+0000 7f44c589f700 1 -- 172.21.15.1:0/3929501152 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b4009550 con 0x7f44c01042e0 2023-11-16T16:30:27.499 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.825+0000 7f44c589f700 1 --2- 172.21.15.1:0/3929501152 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44c01042e0 0x7f44c019b980 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f44b4006af0 tx=0x7f44b4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:27.499 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.825+0000 7f44b2ffd700 1 -- 172.21.15.1:0/3929501152 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f44b4003a40 con 0x7f44c01042e0 2023-11-16T16:30:27.499 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.825+0000 7f44b2ffd700 1 -- 172.21.15.1:0/3929501152 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f44b4003ba0 con 0x7f44c01042e0 2023-11-16T16:30:27.499 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.825+0000 7f44b2ffd700 1 -- 172.21.15.1:0/3929501152 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f44b401a6e0 con 0x7f44c01042e0 2023-11-16T16:30:27.499 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.825+0000 7f44c7b03700 1 -- 172.21.15.1:0/3929501152 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44c0105c30 con 0x7f44c01042e0 2023-11-16T16:30:27.499 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.825+0000 7f44c7b03700 1 -- 172.21.15.1:0/3929501152 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44c0105aa0 con 0x7f44c01042e0 2023-11-16T16:30:27.499 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.826+0000 7f44b2ffd700 1 -- 172.21.15.1:0/3929501152 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7f44b4024a60 con 0x7f44c01042e0 2023-11-16T16:30:27.499 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.826+0000 7f44b2ffd700 1 --2- 172.21.15.1:0/3929501152 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f44ac038710 0x7f44ac03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:27.499 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.826+0000 7f44b2ffd700 1 -- 172.21.15.1:0/3929501152 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f44b404c690 con 0x7f44c01042e0 2023-11-16T16:30:27.499 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.826+0000 7f44c7b03700 1 -- 172.21.15.1:0/3929501152 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44c01952a0 con 0x7f44c01042e0 2023-11-16T16:30:27.499 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.826+0000 7f44c509e700 1 --2- 172.21.15.1:0/3929501152 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f44ac038710 0x7f44ac03abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:27.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.827+0000 7f44c509e700 1 --2- 172.21.15.1:0/3929501152 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f44ac038710 0x7f44ac03abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f44bc006fd0 tx=0x7f44bc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:27.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:25.831+0000 7f44b2ffd700 1 -- 172.21.15.1:0/3929501152 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f44b4024430 con 0x7f44c01042e0 2023-11-16T16:30:27.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:26.305+0000 7f44c7b03700 1 -- 172.21.15.1:0/3929501152 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f44c002cfe0 con 0x7f44c01042e0 2023-11-16T16:30:27.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.075+0000 7f44b2ffd700 1 -- 172.21.15.1:0/3929501152 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f44b402d910 con 0x7f44c01042e0 2023-11-16T16:30:27.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.076+0000 7f44b2ffd700 1 -- 172.21.15.1:0/3929501152 <== mon.0 v2:172.21.15.1:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f44b4018db0 con 0x7f44c01042e0 2023-11-16T16:30:27.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.079+0000 7f44c7b03700 1 -- 172.21.15.1:0/3929501152 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f44ac038710 msgr2=0x7f44ac03abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:27.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.079+0000 7f44c7b03700 1 --2- 172.21.15.1:0/3929501152 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f44ac038710 0x7f44ac03abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f44bc006fd0 tx=0x7f44bc006e40 comp rx=0 tx=0).stop 2023-11-16T16:30:27.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.079+0000 7f44c7b03700 1 -- 172.21.15.1:0/3929501152 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44c01042e0 msgr2=0x7f44c019b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:27.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.079+0000 7f44c7b03700 1 --2- 172.21.15.1:0/3929501152 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44c01042e0 0x7f44c019b980 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f44b4006af0 tx=0x7f44b4011ad0 comp rx=0 tx=0).stop 2023-11-16T16:30:27.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.080+0000 7f44c7b03700 1 -- 172.21.15.1:0/3929501152 shutdown_connections 2023-11-16T16:30:27.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.080+0000 7f44c7b03700 1 --2- 172.21.15.1:0/3929501152 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f44ac038710 0x7f44ac03abc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:27.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.080+0000 7f44c7b03700 1 --2- 172.21.15.1:0/3929501152 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44c01042e0 0x7f44c019b980 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:27.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.080+0000 7f44c7b03700 1 -- 172.21.15.1:0/3929501152 >> 172.21.15.1:0/3929501152 conn(0x7f44c00ffed0 msgr2=0x7f44c018fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:27.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.080+0000 7f44c7b03700 1 -- 172.21.15.1:0/3929501152 shutdown_connections 2023-11-16T16:30:27.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.080+0000 7f44c7b03700 1 -- 172.21.15.1:0/3929501152 wait complete. 2023-11-16T16:30:27.682 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:27 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:27.363+0000 7f05e9d1c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-16T16:30:27.682 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:27 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:27.472+0000 7f05e9d1c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-16T16:30:27.682 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:27 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:27.588+0000 7f05e9d1c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-16T16:30:28.075 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:27 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:27.806+0000 7f05e9d1c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-16T16:30:28.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:28 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3929501152' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-11-16T16:30:28.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:28 smithi001 ceph-mon[98800]: mgrmap e5: x(active, since 6s) 2023-11-16T16:30:28.433 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:28 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:28.207+0000 7f05e9d1c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-16T16:30:28.433 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:28 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:28.316+0000 7f05e9d1c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-16T16:30:28.840 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2023-11-16T16:30:28.840 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-11-16T16:30:28.840 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "available": true, 2023-11-16T16:30:28.840 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-11-16T16:30:28.840 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-11-16T16:30:28.840 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2023-11-16T16:30:28.840 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.975+0000 7f0e19255700 1 Processor -- start 2023-11-16T16:30:28.840 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.976+0000 7f0e19255700 1 -- start start 2023-11-16T16:30:28.840 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.976+0000 7f0e19255700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e14106550 0x7f0e14108930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:28.841 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.976+0000 7f0e19255700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e14079da0 con 0x7f0e14106550 2023-11-16T16:30:28.841 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.976+0000 7f0e12d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e14106550 0x7f0e14108930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:28.841 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.976+0000 7f0e12d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e14106550 0x7f0e14108930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40178/0 (socket says 172.21.15.1:40178) 2023-11-16T16:30:28.841 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.976+0000 7f0e12d9d700 1 -- 172.21.15.1:0/209074137 learned_addr learned my addr 172.21.15.1:0/209074137 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:28.841 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.976+0000 7f0e12d9d700 1 -- 172.21.15.1:0/209074137 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e14079ee0 con 0x7f0e14106550 2023-11-16T16:30:28.841 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.976+0000 7f0e12d9d700 1 --2- 172.21.15.1:0/209074137 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e14106550 0x7f0e14108930 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0e040098a0 tx=0x7f0e04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37e3de4f42a1ae1c server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:28.841 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.977+0000 7f0e11d9b700 1 -- 172.21.15.1:0/209074137 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e04004160 con 0x7f0e14106550 2023-11-16T16:30:28.841 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.977+0000 7f0e11d9b700 1 -- 172.21.15.1:0/209074137 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0e040042c0 con 0x7f0e14106550 2023-11-16T16:30:28.841 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.977+0000 7f0e11d9b700 1 -- 172.21.15.1:0/209074137 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e04011a00 con 0x7f0e14106550 2023-11-16T16:30:28.841 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.977+0000 7f0e19255700 1 -- 172.21.15.1:0/209074137 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e14106550 msgr2=0x7f0e14108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:28.841 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.977+0000 7f0e19255700 1 --2- 172.21.15.1:0/209074137 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e14106550 0x7f0e14108930 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0e040098a0 tx=0x7f0e04009bb0 comp rx=0 tx=0).stop 2023-11-16T16:30:28.841 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.977+0000 7f0e19255700 1 -- 172.21.15.1:0/209074137 shutdown_connections 2023-11-16T16:30:28.842 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.977+0000 7f0e19255700 1 --2- 172.21.15.1:0/209074137 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e14106550 0x7f0e14108930 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:28.842 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.977+0000 7f0e19255700 1 -- 172.21.15.1:0/209074137 >> 172.21.15.1:0/209074137 conn(0x7f0e140ffed0 msgr2=0x7f0e14102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:28.842 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.977+0000 7f0e19255700 1 -- 172.21.15.1:0/209074137 shutdown_connections 2023-11-16T16:30:28.842 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.977+0000 7f0e19255700 1 -- 172.21.15.1:0/209074137 wait complete. 2023-11-16T16:30:28.842 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.978+0000 7f0e19255700 1 Processor -- start 2023-11-16T16:30:28.842 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.978+0000 7f0e19255700 1 -- start start 2023-11-16T16:30:28.842 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.979+0000 7f0e19255700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e14106550 0x7f0e14199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:28.842 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.979+0000 7f0e19255700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e14199cb0 con 0x7f0e14106550 2023-11-16T16:30:28.842 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.979+0000 7f0e12d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e14106550 0x7f0e14199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:28.842 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.979+0000 7f0e12d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e14106550 0x7f0e14199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40180/0 (socket says 172.21.15.1:40180) 2023-11-16T16:30:28.842 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.979+0000 7f0e12d9d700 1 -- 172.21.15.1:0/251044254 learned_addr learned my addr 172.21.15.1:0/251044254 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:28.843 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.979+0000 7f0e12d9d700 1 -- 172.21.15.1:0/251044254 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e04009550 con 0x7f0e14106550 2023-11-16T16:30:28.843 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.979+0000 7f0e12d9d700 1 --2- 172.21.15.1:0/251044254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e14106550 0x7f0e14199770 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0e04009fb0 tx=0x7f0e04011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:28.843 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.979+0000 7f0dfbfff700 1 -- 172.21.15.1:0/251044254 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e04003930 con 0x7f0e14106550 2023-11-16T16:30:28.843 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.979+0000 7f0dfbfff700 1 -- 172.21.15.1:0/251044254 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0e04003a90 con 0x7f0e14106550 2023-11-16T16:30:28.843 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.979+0000 7f0e19255700 1 -- 172.21.15.1:0/251044254 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e14109d30 con 0x7f0e14106550 2023-11-16T16:30:28.843 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.979+0000 7f0dfbfff700 1 -- 172.21.15.1:0/251044254 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e04019610 con 0x7f0e14106550 2023-11-16T16:30:28.843 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.979+0000 7f0e19255700 1 -- 172.21.15.1:0/251044254 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e14109ba0 con 0x7f0e14106550 2023-11-16T16:30:28.843 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.980+0000 7f0dfbfff700 1 -- 172.21.15.1:0/251044254 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f0e04019880 con 0x7f0e14106550 2023-11-16T16:30:28.843 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.981+0000 7f0e19255700 1 -- 172.21.15.1:0/251044254 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e14062120 con 0x7f0e14106550 2023-11-16T16:30:28.843 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.981+0000 7f0dfbfff700 1 --2- 172.21.15.1:0/251044254 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f0dfc038710 0x7f0dfc03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:28.843 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.981+0000 7f0e1259c700 1 -- 172.21.15.1:0/251044254 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f0dfc038710 msgr2=0x7f0dfc03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/1726282374 2023-11-16T16:30:28.843 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.981+0000 7f0dfbfff700 1 -- 172.21.15.1:0/251044254 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0e0404c4a0 con 0x7f0e14106550 2023-11-16T16:30:28.844 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.981+0000 7f0e1259c700 1 --2- 172.21.15.1:0/251044254 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f0dfc038710 0x7f0dfc03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-16T16:30:28.844 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:27.986+0000 7f0dfbfff700 1 -- 172.21.15.1:0/251044254 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0e0402d960 con 0x7f0e14106550 2023-11-16T16:30:28.844 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:28.182+0000 7f0e1259c700 1 -- 172.21.15.1:0/251044254 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f0dfc038710 msgr2=0x7f0dfc03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/1726282374 2023-11-16T16:30:28.844 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:28.182+0000 7f0e1259c700 1 --2- 172.21.15.1:0/251044254 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f0dfc038710 0x7f0dfc03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-16T16:30:28.844 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:28.455+0000 7f0e19255700 1 -- 172.21.15.1:0/251044254 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f0e1419a1f0 con 0x7f0e14106550 2023-11-16T16:30:28.844 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:28.455+0000 7f0dfbfff700 1 -- 172.21.15.1:0/251044254 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f0e04018b40 con 0x7f0e14106550 2023-11-16T16:30:28.844 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:28.457+0000 7f0e19255700 1 -- 172.21.15.1:0/251044254 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f0dfc038710 msgr2=0x7f0dfc03abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-16T16:30:28.844 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:28.457+0000 7f0e19255700 1 --2- 172.21.15.1:0/251044254 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f0dfc038710 0x7f0dfc03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:28.844 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:28.457+0000 7f0e19255700 1 -- 172.21.15.1:0/251044254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e14106550 msgr2=0x7f0e14199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:28.844 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:28.457+0000 7f0e19255700 1 --2- 172.21.15.1:0/251044254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e14106550 0x7f0e14199770 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0e04009fb0 tx=0x7f0e04011ad0 comp rx=0 tx=0).stop 2023-11-16T16:30:28.844 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:28.457+0000 7f0e19255700 1 -- 172.21.15.1:0/251044254 shutdown_connections 2023-11-16T16:30:28.844 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:28.457+0000 7f0e19255700 1 --2- 172.21.15.1:0/251044254 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f0dfc038710 0x7f0dfc03abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:28.845 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:28.457+0000 7f0e19255700 1 --2- 172.21.15.1:0/251044254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e14106550 0x7f0e14199770 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:28.845 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:28.457+0000 7f0e19255700 1 -- 172.21.15.1:0/251044254 >> 172.21.15.1:0/251044254 conn(0x7f0e140ffed0 msgr2=0x7f0e14101570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:28.845 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:28.458+0000 7f0e19255700 1 -- 172.21.15.1:0/251044254 shutdown_connections 2023-11-16T16:30:28.845 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:28.458+0000 7f0e19255700 1 -- 172.21.15.1:0/251044254 wait complete. 2023-11-16T16:30:28.845 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for the mgr to restart... 2023-11-16T16:30:28.845 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mgr epoch 5... 2023-11-16T16:30:28.845 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:28 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:28.468+0000 7f05e9d1c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-16T16:30:28.845 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:28 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:28.589+0000 7f05e9d1c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-16T16:30:29.098 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:29 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/251044254' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-11-16T16:30:29.432 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:29 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:29.110+0000 7f05e9d1c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-16T16:30:30.182 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:29 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:29.799+0000 7f05e9d1c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-16T16:30:30.182 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:29 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:29.991+0000 7f05e9d1c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-16T16:30:31.280 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:30 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:30.949+0000 7f05e9d1c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-16T16:30:31.280 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:31 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:31.060+0000 7f05e9d1c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-16T16:30:31.878 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:31 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:31.278+0000 7f05e9d1c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-16T16:30:31.878 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:31 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:31.625+0000 7f05e9d1c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-16T16:30:32.182 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:31 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:31.876+0000 7f05e9d1c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-16T16:30:32.182 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:32 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:32.057+0000 7f05e9d1c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-16T16:30:32.682 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:32 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:32.384+0000 7f05e9d1c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-16T16:30:33.608 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:33 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:33.207+0000 7f05e9d1c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-16T16:30:33.609 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:33 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:33.364+0000 7f05e9d1c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-16T16:30:33.932 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:33 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:33.606+0000 7f05e9d1c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-16T16:30:33.932 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:33 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:33.743+0000 7f05e9d1c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-16T16:30:34.433 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:34 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:34.015+0000 7f05e9d1c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-16T16:30:35.432 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:35 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:35.050+0000 7f05e9d1c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-16T16:30:35.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:35 smithi001 ceph-mon[98800]: Active manager daemon x restarted 2023-11-16T16:30:35.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:35 smithi001 ceph-mon[98800]: Activating manager daemon x 2023-11-16T16:30:35.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:35 smithi001 ceph-mon[98800]: osdmap e2: 0 total, 0 up, 0 in 2023-11-16T16:30:35.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:35 smithi001 ceph-mon[98800]: mgrmap e6: x(active, starting, since 0.00364719s) 2023-11-16T16:30:35.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:35 smithi001 ceph-mon[98800]: from='mgr.14120 172.21.15.1:0/1538801972' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T16:30:35.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:35 smithi001 ceph-mon[98800]: from='mgr.14120 172.21.15.1:0/1538801972' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-11-16T16:30:35.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:35 smithi001 ceph-mon[98800]: from='mgr.14120 172.21.15.1:0/1538801972' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-11-16T16:30:35.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:35 smithi001 ceph-mon[98800]: from='mgr.14120 172.21.15.1:0/1538801972' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-11-16T16:30:35.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:35 smithi001 ceph-mon[98800]: from='mgr.14120 172.21.15.1:0/1538801972' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-11-16T16:30:35.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:35 smithi001 ceph-mon[98800]: Manager daemon x is now available 2023-11-16T16:30:35.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:35 smithi001 ceph-mon[98800]: from='mgr.14120 172.21.15.1:0/1538801972' entity='mgr.x' 2023-11-16T16:30:35.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:35 smithi001 ceph-mon[98800]: from='mgr.14120 172.21.15.1:0/1538801972' entity='mgr.x' 2023-11-16T16:30:35.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:35 smithi001 ceph-mon[98800]: from='mgr.14120 172.21.15.1:0/1538801972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:30:35.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:35 smithi001 ceph-mon[98800]: from='mgr.14120 172.21.15.1:0/1538801972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:30:36.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:36 smithi001 ceph-mon[98800]: Found migration_current of "None". Setting to last migration. 2023-11-16T16:30:36.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:36 smithi001 ceph-mon[98800]: from='mgr.14120 172.21.15.1:0/1538801972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:30:36.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:36 smithi001 ceph-mon[98800]: from='mgr.14120 172.21.15.1:0/1538801972' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-11-16T16:30:36.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:36 smithi001 ceph-mon[98800]: from='mgr.14120 172.21.15.1:0/1538801972' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-11-16T16:30:36.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:36 smithi001 ceph-mon[98800]: mgrmap e7: x(active, since 1.00547s) 2023-11-16T16:30:36.499 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2023-11-16T16:30:36.499 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-11-16T16:30:36.499 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "initialized": true 2023-11-16T16:30:36.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2023-11-16T16:30:36.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.371+0000 7f2bbf560700 1 Processor -- start 2023-11-16T16:30:36.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.371+0000 7f2bbf560700 1 -- start start 2023-11-16T16:30:36.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.371+0000 7f2bbf560700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bb8106490 0x7f2bb8108870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:36.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.371+0000 7f2bbf560700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb8079e60 con 0x7f2bb8106490 2023-11-16T16:30:36.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.371+0000 7f2bbd2fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bb8106490 0x7f2bb8108870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:36.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.371+0000 7f2bbd2fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bb8106490 0x7f2bb8108870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40188/0 (socket says 172.21.15.1:40188) 2023-11-16T16:30:36.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.372+0000 7f2bbd2fc700 1 -- 172.21.15.1:0/1409708872 learned_addr learned my addr 172.21.15.1:0/1409708872 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:36.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.372+0000 7f2bbd2fc700 1 -- 172.21.15.1:0/1409708872 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb8108db0 con 0x7f2bb8106490 2023-11-16T16:30:36.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.372+0000 7f2bbd2fc700 1 --2- 172.21.15.1:0/1409708872 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bb8106490 0x7f2bb8108870 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2bb40098a0 tx=0x7f2bb4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c021baf4cc6e7dbe server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:36.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.372+0000 7f2baffff700 1 -- 172.21.15.1:0/1409708872 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2bb4004160 con 0x7f2bb8106490 2023-11-16T16:30:36.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.372+0000 7f2baffff700 1 -- 172.21.15.1:0/1409708872 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2bb40042c0 con 0x7f2bb8106490 2023-11-16T16:30:36.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.372+0000 7f2baffff700 1 -- 172.21.15.1:0/1409708872 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2bb4011a00 con 0x7f2bb8106490 2023-11-16T16:30:36.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.373+0000 7f2bbf560700 1 -- 172.21.15.1:0/1409708872 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bb8106490 msgr2=0x7f2bb8108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:36.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.373+0000 7f2bbf560700 1 --2- 172.21.15.1:0/1409708872 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bb8106490 0x7f2bb8108870 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2bb40098a0 tx=0x7f2bb4009bb0 comp rx=0 tx=0).stop 2023-11-16T16:30:36.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.373+0000 7f2bbf560700 1 -- 172.21.15.1:0/1409708872 shutdown_connections 2023-11-16T16:30:36.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.373+0000 7f2bbf560700 1 --2- 172.21.15.1:0/1409708872 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bb8106490 0x7f2bb8108870 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:36.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.373+0000 7f2bbf560700 1 -- 172.21.15.1:0/1409708872 >> 172.21.15.1:0/1409708872 conn(0x7f2bb80ffed0 msgr2=0x7f2bb8102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:36.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.373+0000 7f2bbf560700 1 -- 172.21.15.1:0/1409708872 shutdown_connections 2023-11-16T16:30:36.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.373+0000 7f2bbf560700 1 -- 172.21.15.1:0/1409708872 wait complete. 2023-11-16T16:30:36.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.374+0000 7f2bbf560700 1 Processor -- start 2023-11-16T16:30:36.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.374+0000 7f2bbf560700 1 -- start start 2023-11-16T16:30:36.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.374+0000 7f2bbf560700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bb8106490 0x7f2bb8105df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:36.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.374+0000 7f2bbf560700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb8106330 con 0x7f2bb8106490 2023-11-16T16:30:36.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.374+0000 7f2bbd2fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bb8106490 0x7f2bb8105df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:36.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.374+0000 7f2bbd2fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bb8106490 0x7f2bb8105df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40190/0 (socket says 172.21.15.1:40190) 2023-11-16T16:30:36.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.374+0000 7f2bbd2fc700 1 -- 172.21.15.1:0/2398950365 learned_addr learned my addr 172.21.15.1:0/2398950365 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:36.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.374+0000 7f2bbd2fc700 1 -- 172.21.15.1:0/2398950365 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb4009550 con 0x7f2bb8106490 2023-11-16T16:30:36.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.375+0000 7f2bbd2fc700 1 --2- 172.21.15.1:0/2398950365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bb8106490 0x7f2bb8105df0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4009fb0 tx=0x7f2bb4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:36.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.375+0000 7f2bae7fc700 1 -- 172.21.15.1:0/2398950365 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2bb4003930 con 0x7f2bb8106490 2023-11-16T16:30:36.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.375+0000 7f2bbf560700 1 -- 172.21.15.1:0/2398950365 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bb8109d40 con 0x7f2bb8106490 2023-11-16T16:30:36.503 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.375+0000 7f2bae7fc700 1 -- 172.21.15.1:0/2398950365 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2bb4003a90 con 0x7f2bb8106490 2023-11-16T16:30:36.503 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.375+0000 7f2bae7fc700 1 -- 172.21.15.1:0/2398950365 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2bb4019610 con 0x7f2bb8106490 2023-11-16T16:30:36.503 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.375+0000 7f2bbf560700 1 -- 172.21.15.1:0/2398950365 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bb8109bb0 con 0x7f2bb8106490 2023-11-16T16:30:36.503 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.376+0000 7f2bae7fc700 1 -- 172.21.15.1:0/2398950365 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f2bb40198d0 con 0x7f2bb8106490 2023-11-16T16:30:36.503 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.376+0000 7f2bae7fc700 1 --2- 172.21.15.1:0/2398950365 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f2ba40386c0 0x7f2ba403ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:36.503 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.376+0000 7f2ba3fff700 1 -- 172.21.15.1:0/2398950365 --> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2b98000d10 con 0x7f2ba40386c0 2023-11-16T16:30:36.503 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.376+0000 7f2bae7fc700 1 -- 172.21.15.1:0/2398950365 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2bb404c6c0 con 0x7f2bb8106490 2023-11-16T16:30:36.503 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.376+0000 7f2bbcafb700 1 -- 172.21.15.1:0/2398950365 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f2ba40386c0 msgr2=0x7f2ba403ab70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/1726282374 2023-11-16T16:30:36.503 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.376+0000 7f2bbcafb700 1 --2- 172.21.15.1:0/2398950365 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f2ba40386c0 0x7f2ba403ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-16T16:30:36.503 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.577+0000 7f2bbcafb700 1 -- 172.21.15.1:0/2398950365 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f2ba40386c0 msgr2=0x7f2ba403ab70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/1726282374 2023-11-16T16:30:36.503 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.577+0000 7f2bbcafb700 1 --2- 172.21.15.1:0/2398950365 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f2ba40386c0 0x7f2ba403ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-16T16:30:36.504 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.978+0000 7f2bbcafb700 1 -- 172.21.15.1:0/2398950365 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f2ba40386c0 msgr2=0x7f2ba403ab70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/1726282374 2023-11-16T16:30:36.504 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:29.978+0000 7f2bbcafb700 1 --2- 172.21.15.1:0/2398950365 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f2ba40386c0 0x7f2ba403ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-16T16:30:36.504 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:30.779+0000 7f2bbcafb700 1 -- 172.21.15.1:0/2398950365 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f2ba40386c0 msgr2=0x7f2ba403ab70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/1726282374 2023-11-16T16:30:36.504 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:30.779+0000 7f2bbcafb700 1 --2- 172.21.15.1:0/2398950365 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f2ba40386c0 0x7f2ba403ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-16T16:30:36.504 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:32.380+0000 7f2bbcafb700 1 -- 172.21.15.1:0/2398950365 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f2ba40386c0 msgr2=0x7f2ba403ab70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/1726282374 2023-11-16T16:30:36.504 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:32.380+0000 7f2bbcafb700 1 --2- 172.21.15.1:0/2398950365 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f2ba40386c0 0x7f2ba403ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-11-16T16:30:36.504 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:35.065+0000 7f2bae7fc700 1 -- 172.21.15.1:0/2398950365 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mgrmap(e 6) v1 ==== 45250+0+0 (secure 0 0 0) 0x7f2bb401a9d0 con 0x7f2bb8106490 2023-11-16T16:30:36.504 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:35.065+0000 7f2bae7fc700 1 -- 172.21.15.1:0/2398950365 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f2ba40386c0 msgr2=0x7f2ba403ab70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-16T16:30:36.504 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:35.065+0000 7f2bae7fc700 1 --2- 172.21.15.1:0/2398950365 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f2ba40386c0 0x7f2ba403ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:36.504 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:36.066+0000 7f2bae7fc700 1 -- 172.21.15.1:0/2398950365 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7f2bb404e250 con 0x7f2bb8106490 2023-11-16T16:30:36.504 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:36.066+0000 7f2bae7fc700 1 --2- 172.21.15.1:0/2398950365 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f2ba403be30 0x7f2ba403e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:36.504 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:36.066+0000 7f2bae7fc700 1 -- 172.21.15.1:0/2398950365 --> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2b98000d10 con 0x7f2ba403be30 2023-11-16T16:30:36.505 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:36.067+0000 7f2bbcafb700 1 --2- 172.21.15.1:0/2398950365 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f2ba403be30 0x7f2ba403e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:36.505 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:36.068+0000 7f2bbcafb700 1 --2- 172.21.15.1:0/2398950365 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f2ba403be30 0x7f2ba403e210 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8003a10 tx=0x7f2ba8009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:36.505 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:36.068+0000 7f2bae7fc700 1 -- 172.21.15.1:0/2398950365 <== mgr.14120 v2:172.21.15.1:6800/3137425292 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f2b98000d10 con 0x7f2ba403be30 2023-11-16T16:30:36.505 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:36.079+0000 7f2ba3fff700 1 -- 172.21.15.1:0/2398950365 --> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f2b980027d0 con 0x7f2ba403be30 2023-11-16T16:30:36.505 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:36.080+0000 7f2bae7fc700 1 -- 172.21.15.1:0/2398950365 <== mgr.14120 v2:172.21.15.1:6800/3137425292 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f2b980027d0 con 0x7f2ba403be30 2023-11-16T16:30:36.505 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:36.080+0000 7f2ba3fff700 1 -- 172.21.15.1:0/2398950365 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f2ba403be30 msgr2=0x7f2ba403e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:36.505 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:36.080+0000 7f2ba3fff700 1 --2- 172.21.15.1:0/2398950365 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f2ba403be30 0x7f2ba403e210 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8003a10 tx=0x7f2ba8009290 comp rx=0 tx=0).stop 2023-11-16T16:30:36.505 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:36.080+0000 7f2ba3fff700 1 -- 172.21.15.1:0/2398950365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bb8106490 msgr2=0x7f2bb8105df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:36.505 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:36.080+0000 7f2ba3fff700 1 --2- 172.21.15.1:0/2398950365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bb8106490 0x7f2bb8105df0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4009fb0 tx=0x7f2bb4011ad0 comp rx=0 tx=0).stop 2023-11-16T16:30:36.505 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:36.080+0000 7f2ba3fff700 1 -- 172.21.15.1:0/2398950365 shutdown_connections 2023-11-16T16:30:36.505 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:36.080+0000 7f2ba3fff700 1 --2- 172.21.15.1:0/2398950365 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f2ba403be30 0x7f2ba403e210 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:36.506 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:36.080+0000 7f2ba3fff700 1 --2- 172.21.15.1:0/2398950365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2bb8106490 0x7f2bb8105df0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:36.506 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:36.080+0000 7f2ba3fff700 1 --2- 172.21.15.1:0/2398950365 >> [v2:172.21.15.1:6800/1726282374,v1:172.21.15.1:6801/1726282374] conn(0x7f2ba40386c0 0x7f2ba403ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:36.506 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:36.080+0000 7f2ba3fff700 1 -- 172.21.15.1:0/2398950365 >> 172.21.15.1:0/2398950365 conn(0x7f2bb80ffed0 msgr2=0x7f2bb8100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:36.506 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:36.080+0000 7f2ba3fff700 1 -- 172.21.15.1:0/2398950365 shutdown_connections 2023-11-16T16:30:36.506 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:36.080+0000 7f2ba3fff700 1 -- 172.21.15.1:0/2398950365 wait complete. 2023-11-16T16:30:36.506 INFO:teuthology.orchestra.run.smithi001.stdout:mgr epoch 5 is available 2023-11-16T16:30:36.506 INFO:teuthology.orchestra.run.smithi001.stdout:Setting orchestrator backend to cephadm... 2023-11-16T16:30:37.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:37 smithi001 ceph-mon[98800]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-11-16T16:30:37.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:37 smithi001 ceph-mon[98800]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-11-16T16:30:37.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:37 smithi001 ceph-mon[98800]: from='mgr.14120 172.21.15.1:0/1538801972' entity='mgr.x' 2023-11-16T16:30:37.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:37 smithi001 ceph-mon[98800]: from='mgr.14120 172.21.15.1:0/1538801972' entity='mgr.x' 2023-11-16T16:30:37.844 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.030+0000 7ff556055700 1 Processor -- start 2023-11-16T16:30:37.844 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.030+0000 7ff556055700 1 -- start start 2023-11-16T16:30:37.844 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.030+0000 7ff556055700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff55007be70 0x7ff55007a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:37.844 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.030+0000 7ff556055700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55007a8a0 con 0x7ff55007be70 2023-11-16T16:30:37.845 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.030+0000 7ff54f7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff55007be70 0x7ff55007a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:37.845 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.030+0000 7ff54f7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff55007be70 0x7ff55007a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40234/0 (socket says 172.21.15.1:40234) 2023-11-16T16:30:37.845 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.030+0000 7ff54f7fe700 1 -- 172.21.15.1:0/4031979950 learned_addr learned my addr 172.21.15.1:0/4031979950 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:37.845 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.031+0000 7ff54f7fe700 1 -- 172.21.15.1:0/4031979950 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff55007a9e0 con 0x7ff55007be70 2023-11-16T16:30:37.845 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.031+0000 7ff54f7fe700 1 --2- 172.21.15.1:0/4031979950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff55007be70 0x7ff55007a2d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff5400098a0 tx=0x7ff540009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23c1ec62926405 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:37.845 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.031+0000 7ff54e7fc700 1 -- 172.21.15.1:0/4031979950 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff540004160 con 0x7ff55007be70 2023-11-16T16:30:37.845 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.031+0000 7ff54e7fc700 1 -- 172.21.15.1:0/4031979950 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff5400042c0 con 0x7ff55007be70 2023-11-16T16:30:37.845 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.031+0000 7ff54e7fc700 1 -- 172.21.15.1:0/4031979950 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff540011a00 con 0x7ff55007be70 2023-11-16T16:30:37.845 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.032+0000 7ff556055700 1 -- 172.21.15.1:0/4031979950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff55007be70 msgr2=0x7ff55007a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:37.845 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.032+0000 7ff556055700 1 --2- 172.21.15.1:0/4031979950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff55007be70 0x7ff55007a2d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff5400098a0 tx=0x7ff540009bb0 comp rx=0 tx=0).stop 2023-11-16T16:30:37.845 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.032+0000 7ff556055700 1 -- 172.21.15.1:0/4031979950 shutdown_connections 2023-11-16T16:30:37.845 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.032+0000 7ff556055700 1 --2- 172.21.15.1:0/4031979950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff55007be70 0x7ff55007a2d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:37.846 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.032+0000 7ff556055700 1 -- 172.21.15.1:0/4031979950 >> 172.21.15.1:0/4031979950 conn(0x7ff550100fd0 msgr2=0x7ff550103400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:37.846 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.032+0000 7ff556055700 1 -- 172.21.15.1:0/4031979950 shutdown_connections 2023-11-16T16:30:37.846 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.032+0000 7ff556055700 1 -- 172.21.15.1:0/4031979950 wait complete. 2023-11-16T16:30:37.846 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.033+0000 7ff556055700 1 Processor -- start 2023-11-16T16:30:37.846 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.033+0000 7ff556055700 1 -- start start 2023-11-16T16:30:37.846 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.033+0000 7ff556055700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff55007be70 0x7ff55019b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:37.846 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.033+0000 7ff556055700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55019bef0 con 0x7ff55007be70 2023-11-16T16:30:37.846 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.033+0000 7ff54f7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff55007be70 0x7ff55019b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:37.846 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.033+0000 7ff54f7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff55007be70 0x7ff55019b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40236/0 (socket says 172.21.15.1:40236) 2023-11-16T16:30:37.846 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.033+0000 7ff54f7fe700 1 -- 172.21.15.1:0/1867417702 learned_addr learned my addr 172.21.15.1:0/1867417702 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:37.846 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.033+0000 7ff54f7fe700 1 -- 172.21.15.1:0/1867417702 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff540009550 con 0x7ff55007be70 2023-11-16T16:30:37.846 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.034+0000 7ff54f7fe700 1 --2- 172.21.15.1:0/1867417702 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff55007be70 0x7ff55019b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff540009fb0 tx=0x7ff540011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:37.847 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.034+0000 7ff54cff9700 1 -- 172.21.15.1:0/1867417702 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff540003930 con 0x7ff55007be70 2023-11-16T16:30:37.847 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.034+0000 7ff54cff9700 1 -- 172.21.15.1:0/1867417702 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff540003a90 con 0x7ff55007be70 2023-11-16T16:30:37.847 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.034+0000 7ff556055700 1 -- 172.21.15.1:0/1867417702 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff55010bf60 con 0x7ff55007be70 2023-11-16T16:30:37.847 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.034+0000 7ff54cff9700 1 -- 172.21.15.1:0/1867417702 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff540019610 con 0x7ff55007be70 2023-11-16T16:30:37.847 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.034+0000 7ff556055700 1 -- 172.21.15.1:0/1867417702 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff55010bdd0 con 0x7ff55007be70 2023-11-16T16:30:37.847 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.035+0000 7ff54cff9700 1 -- 172.21.15.1:0/1867417702 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7ff5400198d0 con 0x7ff55007be70 2023-11-16T16:30:37.847 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.035+0000 7ff54cff9700 1 --2- 172.21.15.1:0/1867417702 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7ff5380385a0 0x7ff53803aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:37.847 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.035+0000 7ff54cff9700 1 -- 172.21.15.1:0/1867417702 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff54004c850 con 0x7ff55007be70 2023-11-16T16:30:37.847 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.035+0000 7ff556055700 1 -- 172.21.15.1:0/1867417702 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff550062120 con 0x7ff55007be70 2023-11-16T16:30:37.847 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.035+0000 7ff54effd700 1 --2- 172.21.15.1:0/1867417702 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7ff5380385a0 0x7ff53803aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:37.847 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.036+0000 7ff54effd700 1 --2- 172.21.15.1:0/1867417702 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7ff5380385a0 0x7ff53803aa50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff544006fd0 tx=0x7ff544006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:37.847 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.040+0000 7ff54cff9700 1 -- 172.21.15.1:0/1867417702 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff540019bd0 con 0x7ff55007be70 2023-11-16T16:30:37.847 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.426+0000 7ff556055700 1 -- 172.21.15.1:0/1867417702 --> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7ff5501032c0 con 0x7ff5380385a0 2023-11-16T16:30:37.848 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.432+0000 7ff54cff9700 1 -- 172.21.15.1:0/1867417702 <== mgr.14120 v2:172.21.15.1:6800/3137425292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff5501032c0 con 0x7ff5380385a0 2023-11-16T16:30:37.848 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.434+0000 7ff556055700 1 -- 172.21.15.1:0/1867417702 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7ff5380385a0 msgr2=0x7ff53803aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:37.848 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.434+0000 7ff556055700 1 --2- 172.21.15.1:0/1867417702 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7ff5380385a0 0x7ff53803aa50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff544006fd0 tx=0x7ff544006e40 comp rx=0 tx=0).stop 2023-11-16T16:30:37.848 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.434+0000 7ff556055700 1 -- 172.21.15.1:0/1867417702 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff55007be70 msgr2=0x7ff55019b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:37.848 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.434+0000 7ff556055700 1 --2- 172.21.15.1:0/1867417702 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff55007be70 0x7ff55019b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff540009fb0 tx=0x7ff540011ad0 comp rx=0 tx=0).stop 2023-11-16T16:30:37.848 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.435+0000 7ff556055700 1 -- 172.21.15.1:0/1867417702 shutdown_connections 2023-11-16T16:30:37.848 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.435+0000 7ff556055700 1 --2- 172.21.15.1:0/1867417702 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7ff5380385a0 0x7ff53803aa50 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:37.848 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.435+0000 7ff556055700 1 --2- 172.21.15.1:0/1867417702 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff55007be70 0x7ff55019b9b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:37.848 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.435+0000 7ff556055700 1 -- 172.21.15.1:0/1867417702 >> 172.21.15.1:0/1867417702 conn(0x7ff550100fd0 msgr2=0x7ff5501026a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:37.848 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.435+0000 7ff556055700 1 -- 172.21.15.1:0/1867417702 shutdown_connections 2023-11-16T16:30:37.848 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:37.435+0000 7ff556055700 1 -- 172.21.15.1:0/1867417702 wait complete. 2023-11-16T16:30:38.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:38 smithi001 ceph-mon[98800]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:30:38.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:38 smithi001 ceph-mon[98800]: from='mgr.14120 172.21.15.1:0/1538801972' entity='mgr.x' 2023-11-16T16:30:38.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:38 smithi001 ceph-mon[98800]: from='mgr.14120 172.21.15.1:0/1538801972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:30:38.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:38 smithi001 ceph-mon[98800]: mgrmap e8: x(active, since 2s) 2023-11-16T16:30:39.091 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout value unchanged 2023-11-16T16:30:39.091 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.363+0000 7f84e3720700 1 Processor -- start 2023-11-16T16:30:39.091 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.363+0000 7f84e3720700 1 -- start start 2023-11-16T16:30:39.091 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.364+0000 7f84e3720700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84dc106490 0x7f84dc108870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:39.091 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.364+0000 7f84e3720700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84dc079e60 con 0x7f84dc106490 2023-11-16T16:30:39.091 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.364+0000 7f84e14bc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84dc106490 0x7f84dc108870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:39.092 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.364+0000 7f84e14bc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84dc106490 0x7f84dc108870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40240/0 (socket says 172.21.15.1:40240) 2023-11-16T16:30:39.092 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.364+0000 7f84e14bc700 1 -- 172.21.15.1:0/4110543822 learned_addr learned my addr 172.21.15.1:0/4110543822 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:39.092 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.364+0000 7f84e14bc700 1 -- 172.21.15.1:0/4110543822 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84dc108db0 con 0x7f84dc106490 2023-11-16T16:30:39.092 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.364+0000 7f84e14bc700 1 --2- 172.21.15.1:0/4110543822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84dc106490 0x7f84dc108870 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f84d80098a0 tx=0x7f84d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d554b367024cfc16 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:39.092 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.365+0000 7f84d3fff700 1 -- 172.21.15.1:0/4110543822 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84d8004160 con 0x7f84dc106490 2023-11-16T16:30:39.092 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.365+0000 7f84d3fff700 1 -- 172.21.15.1:0/4110543822 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f84d80042c0 con 0x7f84dc106490 2023-11-16T16:30:39.092 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.365+0000 7f84d3fff700 1 -- 172.21.15.1:0/4110543822 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84d8011a00 con 0x7f84dc106490 2023-11-16T16:30:39.093 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.365+0000 7f84e3720700 1 -- 172.21.15.1:0/4110543822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84dc106490 msgr2=0x7f84dc108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:39.093 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.365+0000 7f84e3720700 1 --2- 172.21.15.1:0/4110543822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84dc106490 0x7f84dc108870 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f84d80098a0 tx=0x7f84d8009bb0 comp rx=0 tx=0).stop 2023-11-16T16:30:39.093 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.365+0000 7f84e3720700 1 -- 172.21.15.1:0/4110543822 shutdown_connections 2023-11-16T16:30:39.093 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.365+0000 7f84e3720700 1 --2- 172.21.15.1:0/4110543822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84dc106490 0x7f84dc108870 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:39.093 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.365+0000 7f84e3720700 1 -- 172.21.15.1:0/4110543822 >> 172.21.15.1:0/4110543822 conn(0x7f84dc0ffed0 msgr2=0x7f84dc102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:39.093 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.365+0000 7f84e3720700 1 -- 172.21.15.1:0/4110543822 shutdown_connections 2023-11-16T16:30:39.093 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.365+0000 7f84e3720700 1 -- 172.21.15.1:0/4110543822 wait complete. 2023-11-16T16:30:39.094 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.366+0000 7f84e3720700 1 Processor -- start 2023-11-16T16:30:39.094 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.366+0000 7f84e3720700 1 -- start start 2023-11-16T16:30:39.094 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.366+0000 7f84e3720700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84dc106490 0x7f84dc105df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:39.094 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.366+0000 7f84e3720700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84dc106330 con 0x7f84dc106490 2023-11-16T16:30:39.094 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.366+0000 7f84e14bc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84dc106490 0x7f84dc105df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:39.094 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.366+0000 7f84e14bc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84dc106490 0x7f84dc105df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40242/0 (socket says 172.21.15.1:40242) 2023-11-16T16:30:39.095 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.366+0000 7f84e14bc700 1 -- 172.21.15.1:0/4143415421 learned_addr learned my addr 172.21.15.1:0/4143415421 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:39.095 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.367+0000 7f84e14bc700 1 -- 172.21.15.1:0/4143415421 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d8009550 con 0x7f84dc106490 2023-11-16T16:30:39.095 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.367+0000 7f84e14bc700 1 --2- 172.21.15.1:0/4143415421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84dc106490 0x7f84dc105df0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f84d8006af0 tx=0x7f84d8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:39.095 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.367+0000 7f84d27fc700 1 -- 172.21.15.1:0/4143415421 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84d8003a40 con 0x7f84dc106490 2023-11-16T16:30:39.095 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.367+0000 7f84d27fc700 1 -- 172.21.15.1:0/4143415421 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f84d8003ba0 con 0x7f84dc106490 2023-11-16T16:30:39.095 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.367+0000 7f84e3720700 1 -- 172.21.15.1:0/4143415421 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84dc109d40 con 0x7f84dc106490 2023-11-16T16:30:39.095 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.367+0000 7f84d27fc700 1 -- 172.21.15.1:0/4143415421 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84d801a6e0 con 0x7f84dc106490 2023-11-16T16:30:39.095 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.367+0000 7f84e3720700 1 -- 172.21.15.1:0/4143415421 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84dc109bb0 con 0x7f84dc106490 2023-11-16T16:30:39.096 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.368+0000 7f84d27fc700 1 -- 172.21.15.1:0/4143415421 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f84d801a950 con 0x7f84dc106490 2023-11-16T16:30:39.096 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.368+0000 7f84d27fc700 1 --2- 172.21.15.1:0/4143415421 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f84c8038710 0x7f84c803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:39.096 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.368+0000 7f84d27fc700 1 -- 172.21.15.1:0/4143415421 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f84d804c780 con 0x7f84dc106490 2023-11-16T16:30:39.096 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.368+0000 7f84e0cbb700 1 --2- 172.21.15.1:0/4143415421 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f84c8038710 0x7f84c803abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:39.096 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.368+0000 7f84e3720700 1 -- 172.21.15.1:0/4143415421 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84dc062120 con 0x7f84dc106490 2023-11-16T16:30:39.096 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.369+0000 7f84e0cbb700 1 --2- 172.21.15.1:0/4143415421 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f84c8038710 0x7f84c803abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f84cc006fd0 tx=0x7f84cc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:39.097 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.373+0000 7f84d27fc700 1 -- 172.21.15.1:0/4143415421 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f84d801ac00 con 0x7f84dc106490 2023-11-16T16:30:39.097 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.739+0000 7f84e3720700 1 -- 172.21.15.1:0/4143415421 --> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f84dc105310 con 0x7f84c8038710 2023-11-16T16:30:39.097 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.740+0000 7f84d27fc700 1 -- 172.21.15.1:0/4143415421 <== mgr.14120 v2:172.21.15.1:6800/3137425292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f84dc105310 con 0x7f84c8038710 2023-11-16T16:30:39.097 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.742+0000 7f84e3720700 1 -- 172.21.15.1:0/4143415421 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f84c8038710 msgr2=0x7f84c803abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:39.097 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.742+0000 7f84e3720700 1 --2- 172.21.15.1:0/4143415421 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f84c8038710 0x7f84c803abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f84cc006fd0 tx=0x7f84cc006e40 comp rx=0 tx=0).stop 2023-11-16T16:30:39.097 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.743+0000 7f84e3720700 1 -- 172.21.15.1:0/4143415421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84dc106490 msgr2=0x7f84dc105df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:39.098 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.743+0000 7f84e3720700 1 --2- 172.21.15.1:0/4143415421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84dc106490 0x7f84dc105df0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f84d8006af0 tx=0x7f84d8011ad0 comp rx=0 tx=0).stop 2023-11-16T16:30:39.098 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.743+0000 7f84e3720700 1 -- 172.21.15.1:0/4143415421 shutdown_connections 2023-11-16T16:30:39.098 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.743+0000 7f84e3720700 1 --2- 172.21.15.1:0/4143415421 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f84c8038710 0x7f84c803abc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:39.098 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.743+0000 7f84e3720700 1 --2- 172.21.15.1:0/4143415421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84dc106490 0x7f84dc105df0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:39.098 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.743+0000 7f84e3720700 1 -- 172.21.15.1:0/4143415421 >> 172.21.15.1:0/4143415421 conn(0x7f84dc0ffed0 msgr2=0x7f84dc100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:39.098 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.743+0000 7f84e3720700 1 -- 172.21.15.1:0/4143415421 shutdown_connections 2023-11-16T16:30:39.098 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:38.743+0000 7f84e3720700 1 -- 172.21.15.1:0/4143415421 wait complete. 2023-11-16T16:30:39.099 INFO:teuthology.orchestra.run.smithi001.stdout:Generating ssh key... 2023-11-16T16:30:39.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:39 smithi001 ceph-mon[98800]: [16/Nov/2023:16:30:38] ENGINE Bus STARTING 2023-11-16T16:30:39.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:39 smithi001 ceph-mon[98800]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:30:39.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:39 smithi001 ceph-mon[98800]: [16/Nov/2023:16:30:38] ENGINE Serving on https://172.21.15.1:7150 2023-11-16T16:30:39.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:39 smithi001 ceph-mon[98800]: [16/Nov/2023:16:30:38] ENGINE Serving on http://172.21.15.1:8765 2023-11-16T16:30:39.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:39 smithi001 ceph-mon[98800]: [16/Nov/2023:16:30:38] ENGINE Bus STARTED 2023-11-16T16:30:39.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:39 smithi001 ceph-mon[98800]: from='mgr.14120 172.21.15.1:0/1538801972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:30:40.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:40 smithi001 ceph-mon[98800]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:30:40.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:40 smithi001 ceph-mon[98800]: Generating ssh key... 2023-11-16T16:30:40.702 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:40 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: Generating public/private rsa key pair. 2023-11-16T16:30:40.702 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:40 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: Your identification has been saved in /tmp/tmpdz26szn0/key. 2023-11-16T16:30:40.702 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:40 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: Your public key has been saved in /tmp/tmpdz26szn0/key.pub. 2023-11-16T16:30:40.702 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:40 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: The key fingerprint is: 2023-11-16T16:30:40.702 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:40 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: SHA256:DLSKHPS4xje41GWsTz4Skc5yUgjEKQp05yDV8rSLrxU ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7 2023-11-16T16:30:40.702 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:40 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: The key's randomart image is: 2023-11-16T16:30:40.702 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:40 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: +---[RSA 3072]----+ 2023-11-16T16:30:40.702 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:40 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: |=+++... | 2023-11-16T16:30:40.702 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:40 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: |o=o=+* . | 2023-11-16T16:30:40.703 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:40 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: |+ + O.B | 2023-11-16T16:30:40.703 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:40 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: |.o X O o | 2023-11-16T16:30:40.703 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:40 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: | @ %Eo S | 2023-11-16T16:30:40.703 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:40 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: | o B B. | 2023-11-16T16:30:40.703 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:40 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: | . o.+ | 2023-11-16T16:30:40.703 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:40 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: | .o . | 2023-11-16T16:30:40.703 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:40 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: | .. | 2023-11-16T16:30:40.703 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:40 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: +----[SHA256]-----+ 2023-11-16T16:30:41.033 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.591+0000 7f750cd08700 1 Processor -- start 2023-11-16T16:30:41.038 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.591+0000 7f750cd08700 1 -- start start 2023-11-16T16:30:41.038 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.592+0000 7f750cd08700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75081042c0 0x7f75081046d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:41.038 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.592+0000 7f750cd08700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7508079e20 con 0x7f75081042c0 2023-11-16T16:30:41.038 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.592+0000 7f750659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75081042c0 0x7f75081046d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:41.038 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.592+0000 7f750659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75081042c0 0x7f75081046d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40254/0 (socket says 172.21.15.1:40254) 2023-11-16T16:30:41.038 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.592+0000 7f750659c700 1 -- 172.21.15.1:0/3418510125 learned_addr learned my addr 172.21.15.1:0/3418510125 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:41.038 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.592+0000 7f750659c700 1 -- 172.21.15.1:0/3418510125 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7508104c10 con 0x7f75081042c0 2023-11-16T16:30:41.039 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.593+0000 7f750659c700 1 --2- 172.21.15.1:0/3418510125 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75081042c0 0x7f75081046d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f74f80098a0 tx=0x7f74f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d9d99a3356c895 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:41.039 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.593+0000 7f750559a700 1 -- 172.21.15.1:0/3418510125 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74f8004160 con 0x7f75081042c0 2023-11-16T16:30:41.039 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.593+0000 7f750559a700 1 -- 172.21.15.1:0/3418510125 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f74f80042c0 con 0x7f75081042c0 2023-11-16T16:30:41.039 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.593+0000 7f750559a700 1 -- 172.21.15.1:0/3418510125 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74f8011a00 con 0x7f75081042c0 2023-11-16T16:30:41.039 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.593+0000 7f750cd08700 1 -- 172.21.15.1:0/3418510125 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75081042c0 msgr2=0x7f75081046d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:41.039 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.593+0000 7f750cd08700 1 --2- 172.21.15.1:0/3418510125 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75081042c0 0x7f75081046d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f74f80098a0 tx=0x7f74f8009bb0 comp rx=0 tx=0).stop 2023-11-16T16:30:41.040 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.593+0000 7f750cd08700 1 -- 172.21.15.1:0/3418510125 shutdown_connections 2023-11-16T16:30:41.040 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.593+0000 7f750cd08700 1 --2- 172.21.15.1:0/3418510125 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75081042c0 0x7f75081046d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:41.040 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.593+0000 7f750cd08700 1 -- 172.21.15.1:0/3418510125 >> 172.21.15.1:0/3418510125 conn(0x7f75080ffed0 msgr2=0x7f7508102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:41.040 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.594+0000 7f750cd08700 1 -- 172.21.15.1:0/3418510125 shutdown_connections 2023-11-16T16:30:41.040 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.594+0000 7f750cd08700 1 -- 172.21.15.1:0/3418510125 wait complete. 2023-11-16T16:30:41.040 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.594+0000 7f750cd08700 1 Processor -- start 2023-11-16T16:30:41.040 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.594+0000 7f750cd08700 1 -- start start 2023-11-16T16:30:41.041 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.595+0000 7f750cd08700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75081042c0 0x7f750819b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:41.041 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.595+0000 7f750cd08700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f750819bec0 con 0x7f75081042c0 2023-11-16T16:30:41.041 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.595+0000 7f750659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75081042c0 0x7f750819b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:41.041 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.595+0000 7f750659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75081042c0 0x7f750819b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40256/0 (socket says 172.21.15.1:40256) 2023-11-16T16:30:41.041 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.595+0000 7f750659c700 1 -- 172.21.15.1:0/773833813 learned_addr learned my addr 172.21.15.1:0/773833813 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:41.042 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.595+0000 7f750659c700 1 -- 172.21.15.1:0/773833813 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f8009550 con 0x7f75081042c0 2023-11-16T16:30:41.042 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.595+0000 7f750659c700 1 --2- 172.21.15.1:0/773833813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75081042c0 0x7f750819b980 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f74f8009fb0 tx=0x7f74f8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:41.042 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.595+0000 7f74ff7fe700 1 -- 172.21.15.1:0/773833813 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74f8003930 con 0x7f75081042c0 2023-11-16T16:30:41.042 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.595+0000 7f74ff7fe700 1 -- 172.21.15.1:0/773833813 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f74f8003a90 con 0x7f75081042c0 2023-11-16T16:30:41.042 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.595+0000 7f74ff7fe700 1 -- 172.21.15.1:0/773833813 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74f8019610 con 0x7f75081042c0 2023-11-16T16:30:41.042 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.595+0000 7f750cd08700 1 -- 172.21.15.1:0/773833813 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7508105c10 con 0x7f75081042c0 2023-11-16T16:30:41.042 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.595+0000 7f750cd08700 1 -- 172.21.15.1:0/773833813 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7508105a80 con 0x7f75081042c0 2023-11-16T16:30:41.043 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.596+0000 7f74ff7fe700 1 -- 172.21.15.1:0/773833813 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f74f8019880 con 0x7f75081042c0 2023-11-16T16:30:41.043 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.596+0000 7f74ff7fe700 1 --2- 172.21.15.1:0/773833813 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f74f4038760 0x7f74f403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:41.043 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.596+0000 7f74ff7fe700 1 -- 172.21.15.1:0/773833813 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f74f804c970 con 0x7f75081042c0 2023-11-16T16:30:41.043 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.597+0000 7f7505d9b700 1 --2- 172.21.15.1:0/773833813 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f74f4038760 0x7f74f403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:41.043 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.597+0000 7f74fd7fa700 1 -- 172.21.15.1:0/773833813 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74ec0052f0 con 0x7f75081042c0 2023-11-16T16:30:41.043 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.597+0000 7f7505d9b700 1 --2- 172.21.15.1:0/773833813 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f74f4038760 0x7f74f403ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f74f0006fd0 tx=0x7f74f0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:41.044 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.602+0000 7f74ff7fe700 1 -- 172.21.15.1:0/773833813 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f74f8019b80 con 0x7f75081042c0 2023-11-16T16:30:41.044 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:39.963+0000 7f74fd7fa700 1 -- 172.21.15.1:0/773833813 --> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f74ec000bc0 con 0x7f74f4038760 2023-11-16T16:30:41.044 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:40.675+0000 7f74ff7fe700 1 -- 172.21.15.1:0/773833813 <== mgr.14120 v2:172.21.15.1:6800/3137425292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f74ec000bc0 con 0x7f74f4038760 2023-11-16T16:30:41.044 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:40.677+0000 7f74fd7fa700 1 -- 172.21.15.1:0/773833813 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f74f4038760 msgr2=0x7f74f403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:41.044 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:40.677+0000 7f74fd7fa700 1 --2- 172.21.15.1:0/773833813 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f74f4038760 0x7f74f403ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f74f0006fd0 tx=0x7f74f0006e40 comp rx=0 tx=0).stop 2023-11-16T16:30:41.044 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:40.677+0000 7f74fd7fa700 1 -- 172.21.15.1:0/773833813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75081042c0 msgr2=0x7f750819b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:41.044 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:40.677+0000 7f74fd7fa700 1 --2- 172.21.15.1:0/773833813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75081042c0 0x7f750819b980 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f74f8009fb0 tx=0x7f74f8011ad0 comp rx=0 tx=0).stop 2023-11-16T16:30:41.045 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:40.677+0000 7f74fd7fa700 1 -- 172.21.15.1:0/773833813 shutdown_connections 2023-11-16T16:30:41.045 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:40.677+0000 7f74fd7fa700 1 --2- 172.21.15.1:0/773833813 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f74f4038760 0x7f74f403ac10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:41.045 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:40.677+0000 7f74fd7fa700 1 --2- 172.21.15.1:0/773833813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75081042c0 0x7f750819b980 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:41.045 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:40.677+0000 7f74fd7fa700 1 -- 172.21.15.1:0/773833813 >> 172.21.15.1:0/773833813 conn(0x7f75080ffed0 msgr2=0x7f750818fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:41.045 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:40.677+0000 7f74fd7fa700 1 -- 172.21.15.1:0/773833813 shutdown_connections 2023-11-16T16:30:41.045 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:40.677+0000 7f74fd7fa700 1 -- 172.21.15.1:0/773833813 wait complete. 2023-11-16T16:30:41.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:41 smithi001 ceph-mon[98800]: from='mgr.14120 172.21.15.1:0/1538801972' entity='mgr.x' 2023-11-16T16:30:41.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:41 smithi001 ceph-mon[98800]: from='mgr.14120 172.21.15.1:0/1538801972' entity='mgr.x' 2023-11-16T16:30:42.285 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwDkmaltnqApgAGAS1JnIiIaLi1tcfzyeCKYojJPYSAjZ3kQY/cIaE5Enj75HRJvWi/1CEVH/+L807c40k3RQ1mB1sA3M/zcxGrDppM06TYmbUKSyXX96bfeUYzSngBT//vIKrKUgzBOpiumnCJ9B2u86Jv58CrUTj5yl91jFMzKZjrLBEQojhGapfLCtbtz4jZSCpcQy94WsfMg38zGMQ2YEpvQYGMjdn2t9r2AhCfckae81EnPwPE7/5oBNAw1yo0GN7woIK/BRGZodi6l2Yl3Nujrf73OaZQSyZRSp7O1wYZ2ckOIvQ5VfeaIF0j1q4pFdhUUSgNikj4ImrMfkJOW+7QVBvFzYPokmUyiDemmPNCBAuG8tx59TqCwgGsTh7P+zZ7yCXGgTuxLpVKj/fyiB1VUzMQEBtniq82FW9+gbgTarf6+6vVtjkXEWiUAO1W/f6AG9HUCkeJYJysIvZyRXv5TH07Ab+b3WvVaZQc92TeD8EAyKuHuNmGtzDZkM= ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7 2023-11-16T16:30:42.286 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.525+0000 7f1fab83b700 1 Processor -- start 2023-11-16T16:30:42.286 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.525+0000 7f1fab83b700 1 -- start start 2023-11-16T16:30:42.286 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.526+0000 7f1fab83b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fa4103d10 0x7f1fa4104120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:42.286 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.526+0000 7f1fab83b700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa41046f0 con 0x7f1fa4103d10 2023-11-16T16:30:42.286 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.526+0000 7f1fa95d7700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fa4103d10 0x7f1fa4104120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:42.286 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.526+0000 7f1fa95d7700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fa4103d10 0x7f1fa4104120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40260/0 (socket says 172.21.15.1:40260) 2023-11-16T16:30:42.287 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.526+0000 7f1fa95d7700 1 -- 172.21.15.1:0/338559954 learned_addr learned my addr 172.21.15.1:0/338559954 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:42.287 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.526+0000 7f1fa95d7700 1 -- 172.21.15.1:0/338559954 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa4104830 con 0x7f1fa4103d10 2023-11-16T16:30:42.287 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.526+0000 7f1fa95d7700 1 --2- 172.21.15.1:0/338559954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fa4103d10 0x7f1fa4104120 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1fa00098a0 tx=0x7f1fa0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=306cf7e9425cdaf8 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:42.287 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.526+0000 7f1f9bfff700 1 -- 172.21.15.1:0/338559954 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1fa0004160 con 0x7f1fa4103d10 2023-11-16T16:30:42.287 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.526+0000 7f1f9bfff700 1 -- 172.21.15.1:0/338559954 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1fa00042c0 con 0x7f1fa4103d10 2023-11-16T16:30:42.287 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.526+0000 7f1f9bfff700 1 -- 172.21.15.1:0/338559954 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1fa0011a00 con 0x7f1fa4103d10 2023-11-16T16:30:42.288 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.527+0000 7f1fab83b700 1 -- 172.21.15.1:0/338559954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fa4103d10 msgr2=0x7f1fa4104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:42.288 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.527+0000 7f1fab83b700 1 --2- 172.21.15.1:0/338559954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fa4103d10 0x7f1fa4104120 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1fa00098a0 tx=0x7f1fa0009bb0 comp rx=0 tx=0).stop 2023-11-16T16:30:42.288 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.527+0000 7f1fab83b700 1 -- 172.21.15.1:0/338559954 shutdown_connections 2023-11-16T16:30:42.288 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.527+0000 7f1fab83b700 1 --2- 172.21.15.1:0/338559954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fa4103d10 0x7f1fa4104120 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:42.288 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.527+0000 7f1fab83b700 1 -- 172.21.15.1:0/338559954 >> 172.21.15.1:0/338559954 conn(0x7f1fa40ff650 msgr2=0x7f1fa4101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:42.289 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.527+0000 7f1fab83b700 1 -- 172.21.15.1:0/338559954 shutdown_connections 2023-11-16T16:30:42.289 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.527+0000 7f1fab83b700 1 -- 172.21.15.1:0/338559954 wait complete. 2023-11-16T16:30:42.289 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.528+0000 7f1fab83b700 1 Processor -- start 2023-11-16T16:30:42.289 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.528+0000 7f1fab83b700 1 -- start start 2023-11-16T16:30:42.289 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.528+0000 7f1fab83b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fa4103d10 0x7f1fa4199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:42.289 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.528+0000 7f1fab83b700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa4199c50 con 0x7f1fa4103d10 2023-11-16T16:30:42.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.528+0000 7f1fa95d7700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fa4103d10 0x7f1fa4199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:42.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.528+0000 7f1fa95d7700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fa4103d10 0x7f1fa4199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40262/0 (socket says 172.21.15.1:40262) 2023-11-16T16:30:42.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.528+0000 7f1fa95d7700 1 -- 172.21.15.1:0/2939691030 learned_addr learned my addr 172.21.15.1:0/2939691030 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:42.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.528+0000 7f1fa95d7700 1 -- 172.21.15.1:0/2939691030 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa0009550 con 0x7f1fa4103d10 2023-11-16T16:30:42.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.528+0000 7f1fa95d7700 1 --2- 172.21.15.1:0/2939691030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fa4103d10 0x7f1fa4199710 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0009fb0 tx=0x7f1fa0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:42.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.529+0000 7f1f9a7fc700 1 -- 172.21.15.1:0/2939691030 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1fa0003930 con 0x7f1fa4103d10 2023-11-16T16:30:42.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.529+0000 7f1f9a7fc700 1 -- 172.21.15.1:0/2939691030 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1fa0003a90 con 0x7f1fa4103d10 2023-11-16T16:30:42.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.529+0000 7f1fab83b700 1 -- 172.21.15.1:0/2939691030 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fa4107ae0 con 0x7f1fa4103d10 2023-11-16T16:30:42.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.529+0000 7f1f9a7fc700 1 -- 172.21.15.1:0/2939691030 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1fa00195e0 con 0x7f1fa4103d10 2023-11-16T16:30:42.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.529+0000 7f1fab83b700 1 -- 172.21.15.1:0/2939691030 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa4107950 con 0x7f1fa4103d10 2023-11-16T16:30:42.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.530+0000 7f1fab83b700 1 -- 172.21.15.1:0/2939691030 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fa4062120 con 0x7f1fa4103d10 2023-11-16T16:30:42.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.530+0000 7f1f9a7fc700 1 -- 172.21.15.1:0/2939691030 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f1fa001a9d0 con 0x7f1fa4103d10 2023-11-16T16:30:42.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.530+0000 7f1f9a7fc700 1 --2- 172.21.15.1:0/2939691030 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f1f90038710 0x7f1f9003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:42.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.530+0000 7f1f9a7fc700 1 -- 172.21.15.1:0/2939691030 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1fa004ccf0 con 0x7f1fa4103d10 2023-11-16T16:30:42.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.531+0000 7f1fa8dd6700 1 --2- 172.21.15.1:0/2939691030 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f1f90038710 0x7f1f9003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:42.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.531+0000 7f1fa8dd6700 1 --2- 172.21.15.1:0/2939691030 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f1f90038710 0x7f1f9003abc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1f94006fd0 tx=0x7f1f94006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:42.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.535+0000 7f1f9a7fc700 1 -- 172.21.15.1:0/2939691030 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1fa0023b80 con 0x7f1fa4103d10 2023-11-16T16:30:42.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.896+0000 7f1fab83b700 1 -- 172.21.15.1:0/2939691030 --> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f1fa4101910 con 0x7f1f90038710 2023-11-16T16:30:42.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.897+0000 7f1f9a7fc700 1 -- 172.21.15.1:0/2939691030 <== mgr.14120 v2:172.21.15.1:6800/3137425292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f1fa4101910 con 0x7f1f90038710 2023-11-16T16:30:42.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.899+0000 7f1fab83b700 1 -- 172.21.15.1:0/2939691030 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f1f90038710 msgr2=0x7f1f9003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:42.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.899+0000 7f1fab83b700 1 --2- 172.21.15.1:0/2939691030 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f1f90038710 0x7f1f9003abc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1f94006fd0 tx=0x7f1f94006e40 comp rx=0 tx=0).stop 2023-11-16T16:30:42.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.899+0000 7f1fab83b700 1 -- 172.21.15.1:0/2939691030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fa4103d10 msgr2=0x7f1fa4199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:42.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.899+0000 7f1fab83b700 1 --2- 172.21.15.1:0/2939691030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fa4103d10 0x7f1fa4199710 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0009fb0 tx=0x7f1fa0011ad0 comp rx=0 tx=0).stop 2023-11-16T16:30:42.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.899+0000 7f1fab83b700 1 -- 172.21.15.1:0/2939691030 shutdown_connections 2023-11-16T16:30:42.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.899+0000 7f1fab83b700 1 --2- 172.21.15.1:0/2939691030 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f1f90038710 0x7f1f9003abc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:42.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.899+0000 7f1fab83b700 1 --2- 172.21.15.1:0/2939691030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fa4103d10 0x7f1fa4199710 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:42.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.899+0000 7f1fab83b700 1 -- 172.21.15.1:0/2939691030 >> 172.21.15.1:0/2939691030 conn(0x7f1fa40ff650 msgr2=0x7f1fa4100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:42.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.899+0000 7f1fab83b700 1 -- 172.21.15.1:0/2939691030 shutdown_connections 2023-11-16T16:30:42.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:41.899+0000 7f1fab83b700 1 -- 172.21.15.1:0/2939691030 wait complete. 2023-11-16T16:30:42.292 INFO:teuthology.orchestra.run.smithi001.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-11-16T16:30:42.292 INFO:teuthology.orchestra.run.smithi001.stdout:Adding key to root@localhost authorized_keys... 2023-11-16T16:30:42.292 INFO:teuthology.orchestra.run.smithi001.stdout:Adding host smithi001... 2023-11-16T16:30:42.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:42 smithi001 ceph-mon[98800]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:30:44.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:44 smithi001 ceph-mon[98800]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi001", "addr": "172.21.15.1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:30:44.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:44 smithi001 ceph-mon[98800]: Deploying cephadm binary to smithi001 2023-11-16T16:30:45.939 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout Added host 'smithi001' with addr '172.21.15.1' 2023-11-16T16:30:45.939 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.782+0000 7f8ddc143700 1 Processor -- start 2023-11-16T16:30:45.939 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.782+0000 7f8ddc143700 1 -- start start 2023-11-16T16:30:45.939 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.782+0000 7f8ddc143700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8dd4105ac0 0x7f8dd4105ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:45.939 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.782+0000 7f8ddc143700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd4106410 con 0x7f8dd4105ac0 2023-11-16T16:30:45.939 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.782+0000 7f8dd9edf700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8dd4105ac0 0x7f8dd4105ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:45.939 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.782+0000 7f8dd9edf700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8dd4105ac0 0x7f8dd4105ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40266/0 (socket says 172.21.15.1:40266) 2023-11-16T16:30:45.939 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.782+0000 7f8dd9edf700 1 -- 172.21.15.1:0/728059499 learned_addr learned my addr 172.21.15.1:0/728059499 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:45.939 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.783+0000 7f8dd9edf700 1 -- 172.21.15.1:0/728059499 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd4106550 con 0x7f8dd4105ac0 2023-11-16T16:30:45.939 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.783+0000 7f8dd9edf700 1 --2- 172.21.15.1:0/728059499 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8dd4105ac0 0x7f8dd4105ed0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8dc80098a0 tx=0x7f8dc8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e92dff7671093d13 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:45.940 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.783+0000 7f8dd8edd700 1 -- 172.21.15.1:0/728059499 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8dc8004160 con 0x7f8dd4105ac0 2023-11-16T16:30:45.940 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.783+0000 7f8dd8edd700 1 -- 172.21.15.1:0/728059499 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8dc80042c0 con 0x7f8dd4105ac0 2023-11-16T16:30:45.940 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.783+0000 7f8dd8edd700 1 -- 172.21.15.1:0/728059499 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8dc8011a00 con 0x7f8dd4105ac0 2023-11-16T16:30:45.940 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.784+0000 7f8ddc143700 1 -- 172.21.15.1:0/728059499 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8dd4105ac0 msgr2=0x7f8dd4105ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:45.940 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.784+0000 7f8ddc143700 1 --2- 172.21.15.1:0/728059499 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8dd4105ac0 0x7f8dd4105ed0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8dc80098a0 tx=0x7f8dc8009bb0 comp rx=0 tx=0).stop 2023-11-16T16:30:45.940 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.784+0000 7f8ddc143700 1 -- 172.21.15.1:0/728059499 shutdown_connections 2023-11-16T16:30:45.940 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.784+0000 7f8ddc143700 1 --2- 172.21.15.1:0/728059499 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8dd4105ac0 0x7f8dd4105ed0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:45.940 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.784+0000 7f8ddc143700 1 -- 172.21.15.1:0/728059499 >> 172.21.15.1:0/728059499 conn(0x7f8dd4101170 msgr2=0x7f8dd4103580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:45.940 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.784+0000 7f8ddc143700 1 -- 172.21.15.1:0/728059499 shutdown_connections 2023-11-16T16:30:45.940 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.784+0000 7f8ddc143700 1 -- 172.21.15.1:0/728059499 wait complete. 2023-11-16T16:30:45.940 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.784+0000 7f8ddc143700 1 Processor -- start 2023-11-16T16:30:45.941 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.784+0000 7f8ddc143700 1 -- start start 2023-11-16T16:30:45.941 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.785+0000 7f8ddc143700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8dd4105ac0 0x7f8dd4199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:45.941 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.785+0000 7f8ddc143700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd4199d60 con 0x7f8dd4105ac0 2023-11-16T16:30:45.941 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.785+0000 7f8dd9edf700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8dd4105ac0 0x7f8dd4199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:45.941 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.785+0000 7f8dd9edf700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8dd4105ac0 0x7f8dd4199820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40268/0 (socket says 172.21.15.1:40268) 2023-11-16T16:30:45.941 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.785+0000 7f8dd9edf700 1 -- 172.21.15.1:0/1161837753 learned_addr learned my addr 172.21.15.1:0/1161837753 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:45.941 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.785+0000 7f8dd9edf700 1 -- 172.21.15.1:0/1161837753 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc8009550 con 0x7f8dd4105ac0 2023-11-16T16:30:45.941 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.785+0000 7f8dd9edf700 1 --2- 172.21.15.1:0/1161837753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8dd4105ac0 0x7f8dd4199820 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8006af0 tx=0x7f8dc8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:45.941 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.785+0000 7f8dc6ffd700 1 -- 172.21.15.1:0/1161837753 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8dc8003a40 con 0x7f8dd4105ac0 2023-11-16T16:30:45.941 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.785+0000 7f8dc6ffd700 1 -- 172.21.15.1:0/1161837753 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8dc8003ba0 con 0x7f8dd4105ac0 2023-11-16T16:30:45.941 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.785+0000 7f8ddc143700 1 -- 172.21.15.1:0/1161837753 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dd4107b10 con 0x7f8dd4105ac0 2023-11-16T16:30:45.942 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.785+0000 7f8dc6ffd700 1 -- 172.21.15.1:0/1161837753 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8dc801a6e0 con 0x7f8dd4105ac0 2023-11-16T16:30:45.942 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.785+0000 7f8ddc143700 1 -- 172.21.15.1:0/1161837753 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dd4107980 con 0x7f8dd4105ac0 2023-11-16T16:30:45.942 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.786+0000 7f8dc6ffd700 1 -- 172.21.15.1:0/1161837753 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f8dc8024a60 con 0x7f8dd4105ac0 2023-11-16T16:30:45.942 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.786+0000 7f8dc6ffd700 1 --2- 172.21.15.1:0/1161837753 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f8dc0038710 0x7f8dc003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:45.942 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.786+0000 7f8dc6ffd700 1 -- 172.21.15.1:0/1161837753 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f8dc804c770 con 0x7f8dd4105ac0 2023-11-16T16:30:45.942 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.786+0000 7f8ddc143700 1 -- 172.21.15.1:0/1161837753 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dd4109f90 con 0x7f8dd4105ac0 2023-11-16T16:30:45.942 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.787+0000 7f8dd96de700 1 --2- 172.21.15.1:0/1161837753 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f8dc0038710 0x7f8dc003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:45.942 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.787+0000 7f8dd96de700 1 --2- 172.21.15.1:0/1161837753 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f8dc0038710 0x7f8dc003abc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0006fd0 tx=0x7f8dd0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:45.942 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:42.791+0000 7f8dc6ffd700 1 -- 172.21.15.1:0/1161837753 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8dc8024430 con 0x7f8dd4105ac0 2023-11-16T16:30:45.942 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:43.160+0000 7f8ddc143700 1 -- 172.21.15.1:0/1161837753 --> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi001", "addr": "172.21.15.1", "target": ["mon-mgr", ""]}) v1 -- 0x7f8dd4102630 con 0x7f8dc0038710 2023-11-16T16:30:45.942 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:45.565+0000 7f8dc6ffd700 1 -- 172.21.15.1:0/1161837753 <== mgr.14120 v2:172.21.15.1:6800/3137425292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f8dd4102630 con 0x7f8dc0038710 2023-11-16T16:30:45.943 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:45.568+0000 7f8ddc143700 1 -- 172.21.15.1:0/1161837753 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f8dc0038710 msgr2=0x7f8dc003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:45.943 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:45.568+0000 7f8ddc143700 1 --2- 172.21.15.1:0/1161837753 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f8dc0038710 0x7f8dc003abc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0006fd0 tx=0x7f8dd0006e40 comp rx=0 tx=0).stop 2023-11-16T16:30:45.943 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:45.568+0000 7f8ddc143700 1 -- 172.21.15.1:0/1161837753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8dd4105ac0 msgr2=0x7f8dd4199820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:45.943 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:45.568+0000 7f8ddc143700 1 --2- 172.21.15.1:0/1161837753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8dd4105ac0 0x7f8dd4199820 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8006af0 tx=0x7f8dc8011ad0 comp rx=0 tx=0).stop 2023-11-16T16:30:45.943 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:45.569+0000 7f8ddc143700 1 -- 172.21.15.1:0/1161837753 shutdown_connections 2023-11-16T16:30:45.943 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:45.569+0000 7f8ddc143700 1 --2- 172.21.15.1:0/1161837753 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f8dc0038710 0x7f8dc003abc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:45.943 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:45.569+0000 7f8ddc143700 1 --2- 172.21.15.1:0/1161837753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8dd4105ac0 0x7f8dd4199820 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:45.943 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:45.569+0000 7f8ddc143700 1 -- 172.21.15.1:0/1161837753 >> 172.21.15.1:0/1161837753 conn(0x7f8dd4101170 msgr2=0x7f8dd4101f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:45.943 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:45.569+0000 7f8ddc143700 1 -- 172.21.15.1:0/1161837753 shutdown_connections 2023-11-16T16:30:45.943 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:45.569+0000 7f8ddc143700 1 -- 172.21.15.1:0/1161837753 wait complete. 2023-11-16T16:30:45.943 INFO:teuthology.orchestra.run.smithi001.stdout:Deploying unmanaged mon service... 2023-11-16T16:30:46.851 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:46 smithi001 ceph-mon[98800]: from='mgr.14120 172.21.15.1:0/1538801972' entity='mgr.x' 2023-11-16T16:30:46.851 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:46 smithi001 ceph-mon[98800]: Added host smithi001 2023-11-16T16:30:46.851 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:46 smithi001 ceph-mon[98800]: from='mgr.14120 172.21.15.1:0/1538801972' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:30:47.684 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-11-16T16:30:47.685 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.421+0000 7f15abd3a700 1 Processor -- start 2023-11-16T16:30:47.685 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.422+0000 7f15abd3a700 1 -- start start 2023-11-16T16:30:47.685 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.422+0000 7f15abd3a700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15a4103d10 0x7f15a4104120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:47.685 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.422+0000 7f15abd3a700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a41046f0 con 0x7f15a4103d10 2023-11-16T16:30:47.685 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.422+0000 7f15a9ad6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15a4103d10 0x7f15a4104120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:47.685 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.422+0000 7f15a9ad6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15a4103d10 0x7f15a4104120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40300/0 (socket says 172.21.15.1:40300) 2023-11-16T16:30:47.685 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.422+0000 7f15a9ad6700 1 -- 172.21.15.1:0/1791040919 learned_addr learned my addr 172.21.15.1:0/1791040919 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:47.685 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.423+0000 7f15a9ad6700 1 -- 172.21.15.1:0/1791040919 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a4106ca0 con 0x7f15a4103d10 2023-11-16T16:30:47.685 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.423+0000 7f15a9ad6700 1 --2- 172.21.15.1:0/1791040919 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15a4103d10 0x7f15a4104120 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f15a00098a0 tx=0x7f15a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3205540caf8d67b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:47.685 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.423+0000 7f15a8ad4700 1 -- 172.21.15.1:0/1791040919 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15a0004160 con 0x7f15a4103d10 2023-11-16T16:30:47.685 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.423+0000 7f15a8ad4700 1 -- 172.21.15.1:0/1791040919 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f15a00042c0 con 0x7f15a4103d10 2023-11-16T16:30:47.685 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.423+0000 7f15a8ad4700 1 -- 172.21.15.1:0/1791040919 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15a0011a00 con 0x7f15a4103d10 2023-11-16T16:30:47.686 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.423+0000 7f15abd3a700 1 -- 172.21.15.1:0/1791040919 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15a4103d10 msgr2=0x7f15a4104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:47.686 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.424+0000 7f15abd3a700 1 --2- 172.21.15.1:0/1791040919 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15a4103d10 0x7f15a4104120 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f15a00098a0 tx=0x7f15a0009bb0 comp rx=0 tx=0).stop 2023-11-16T16:30:47.686 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.424+0000 7f15abd3a700 1 -- 172.21.15.1:0/1791040919 shutdown_connections 2023-11-16T16:30:47.686 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.424+0000 7f15abd3a700 1 --2- 172.21.15.1:0/1791040919 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15a4103d10 0x7f15a4104120 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:47.686 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.424+0000 7f15abd3a700 1 -- 172.21.15.1:0/1791040919 >> 172.21.15.1:0/1791040919 conn(0x7f15a40ff650 msgr2=0x7f15a4101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:47.686 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.424+0000 7f15abd3a700 1 -- 172.21.15.1:0/1791040919 shutdown_connections 2023-11-16T16:30:47.686 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.424+0000 7f15abd3a700 1 -- 172.21.15.1:0/1791040919 wait complete. 2023-11-16T16:30:47.686 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.424+0000 7f15abd3a700 1 Processor -- start 2023-11-16T16:30:47.686 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.424+0000 7f15abd3a700 1 -- start start 2023-11-16T16:30:47.686 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.425+0000 7f15abd3a700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15a4103d10 0x7f15a4106600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:47.686 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.425+0000 7f15abd3a700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a4106b40 con 0x7f15a4103d10 2023-11-16T16:30:47.686 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.425+0000 7f15a9ad6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15a4103d10 0x7f15a4106600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:47.687 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.425+0000 7f15a9ad6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15a4103d10 0x7f15a4106600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40302/0 (socket says 172.21.15.1:40302) 2023-11-16T16:30:47.687 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.425+0000 7f15a9ad6700 1 -- 172.21.15.1:0/2907316760 learned_addr learned my addr 172.21.15.1:0/2907316760 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:47.687 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.425+0000 7f15a9ad6700 1 -- 172.21.15.1:0/2907316760 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a0009550 con 0x7f15a4103d10 2023-11-16T16:30:47.687 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.425+0000 7f15a9ad6700 1 --2- 172.21.15.1:0/2907316760 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15a4103d10 0x7f15a4106600 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f15a0009fb0 tx=0x7f15a0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:47.687 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.425+0000 7f1596ffd700 1 -- 172.21.15.1:0/2907316760 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15a0003930 con 0x7f15a4103d10 2023-11-16T16:30:47.687 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.425+0000 7f1596ffd700 1 -- 172.21.15.1:0/2907316760 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f15a0003a90 con 0x7f15a4103d10 2023-11-16T16:30:47.687 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.425+0000 7f15abd3a700 1 -- 172.21.15.1:0/2907316760 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a4109ed0 con 0x7f15a4103d10 2023-11-16T16:30:47.687 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.425+0000 7f1596ffd700 1 -- 172.21.15.1:0/2907316760 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15a00195e0 con 0x7f15a4103d10 2023-11-16T16:30:47.687 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.425+0000 7f15abd3a700 1 -- 172.21.15.1:0/2907316760 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a410a060 con 0x7f15a4103d10 2023-11-16T16:30:47.687 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.426+0000 7f1596ffd700 1 -- 172.21.15.1:0/2907316760 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f15a0019850 con 0x7f15a4103d10 2023-11-16T16:30:47.687 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.426+0000 7f1596ffd700 1 --2- 172.21.15.1:0/2907316760 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f1590038710 0x7f159003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:47.687 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.426+0000 7f1596ffd700 1 -- 172.21.15.1:0/2907316760 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f15a004c860 con 0x7f15a4103d10 2023-11-16T16:30:47.688 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.426+0000 7f15abd3a700 1 -- 172.21.15.1:0/2907316760 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a4107b10 con 0x7f15a4103d10 2023-11-16T16:30:47.688 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.426+0000 7f15a92d5700 1 --2- 172.21.15.1:0/2907316760 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f1590038710 0x7f159003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:47.688 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.427+0000 7f15a92d5700 1 --2- 172.21.15.1:0/2907316760 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f1590038710 0x7f159003abc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1598006fd0 tx=0x7f1598006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:47.688 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.431+0000 7f1596ffd700 1 -- 172.21.15.1:0/2907316760 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f15a4107b10 con 0x7f15a4103d10 2023-11-16T16:30:47.688 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.816+0000 7f15abd3a700 1 -- 172.21.15.1:0/2907316760 --> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f15a4107b10 con 0x7f1590038710 2023-11-16T16:30:47.688 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.821+0000 7f1596ffd700 1 -- 172.21.15.1:0/2907316760 <== mgr.14120 v2:172.21.15.1:6800/3137425292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f15a4107b10 con 0x7f1590038710 2023-11-16T16:30:47.688 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.823+0000 7f15abd3a700 1 -- 172.21.15.1:0/2907316760 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f1590038710 msgr2=0x7f159003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:47.688 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.823+0000 7f15abd3a700 1 --2- 172.21.15.1:0/2907316760 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f1590038710 0x7f159003abc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1598006fd0 tx=0x7f1598006e40 comp rx=0 tx=0).stop 2023-11-16T16:30:47.688 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.823+0000 7f15abd3a700 1 -- 172.21.15.1:0/2907316760 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15a4103d10 msgr2=0x7f15a4106600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:47.688 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.823+0000 7f15abd3a700 1 --2- 172.21.15.1:0/2907316760 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15a4103d10 0x7f15a4106600 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f15a0009fb0 tx=0x7f15a0011ad0 comp rx=0 tx=0).stop 2023-11-16T16:30:47.688 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.824+0000 7f15abd3a700 1 -- 172.21.15.1:0/2907316760 shutdown_connections 2023-11-16T16:30:47.688 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.824+0000 7f15abd3a700 1 --2- 172.21.15.1:0/2907316760 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f1590038710 0x7f159003abc0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:47.689 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.824+0000 7f15abd3a700 1 --2- 172.21.15.1:0/2907316760 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15a4103d10 0x7f15a4106600 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:47.689 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.824+0000 7f15abd3a700 1 -- 172.21.15.1:0/2907316760 >> 172.21.15.1:0/2907316760 conn(0x7f15a40ff650 msgr2=0x7f15a4100410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:47.689 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.824+0000 7f15abd3a700 1 -- 172.21.15.1:0/2907316760 shutdown_connections 2023-11-16T16:30:47.689 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:46.824+0000 7f15abd3a700 1 -- 172.21.15.1:0/2907316760 wait complete. 2023-11-16T16:30:47.689 INFO:teuthology.orchestra.run.smithi001.stdout:Deploying unmanaged mgr service... 2023-11-16T16:30:47.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:47 smithi001 ceph-mon[98800]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:30:47.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:47 smithi001 ceph-mon[98800]: Saving service mon spec with placement count:5 2023-11-16T16:30:47.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:47 smithi001 ceph-mon[98800]: from='mgr.14120 172.21.15.1:0/1538801972' entity='mgr.x' 2023-11-16T16:30:49.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:49 smithi001 ceph-mon[98800]: from='mgr.14120 172.21.15.1:0/1538801972' entity='mgr.x' 2023-11-16T16:30:49.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:49 smithi001 ceph-mon[98800]: from='mgr.14120 172.21.15.1:0/1538801972' entity='mgr.x' 2023-11-16T16:30:49.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:49 smithi001 ceph-mon[98800]: from='mgr.14120 172.21.15.1:0/1538801972' entity='mgr.x' 2023-11-16T16:30:49.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-11-16T16:30:49.290 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.477+0000 7f4b84429700 1 Processor -- start 2023-11-16T16:30:49.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.478+0000 7f4b84429700 1 -- start start 2023-11-16T16:30:49.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.478+0000 7f4b84429700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b7c104320 0x7f4b7c104730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:49.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.478+0000 7f4b84429700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c079dc0 con 0x7f4b7c104320 2023-11-16T16:30:49.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.478+0000 7f4b821c5700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b7c104320 0x7f4b7c104730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:49.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.478+0000 7f4b821c5700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b7c104320 0x7f4b7c104730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40308/0 (socket says 172.21.15.1:40308) 2023-11-16T16:30:49.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.478+0000 7f4b821c5700 1 -- 172.21.15.1:0/3544524800 learned_addr learned my addr 172.21.15.1:0/3544524800 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:49.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.478+0000 7f4b821c5700 1 -- 172.21.15.1:0/3544524800 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b7c079f00 con 0x7f4b7c104320 2023-11-16T16:30:49.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.479+0000 7f4b821c5700 1 --2- 172.21.15.1:0/3544524800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b7c104320 0x7f4b7c104730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4b780098a0 tx=0x7f4b78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f60404052ef747 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:49.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.479+0000 7f4b811c3700 1 -- 172.21.15.1:0/3544524800 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b78004160 con 0x7f4b7c104320 2023-11-16T16:30:49.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.479+0000 7f4b811c3700 1 -- 172.21.15.1:0/3544524800 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b780042c0 con 0x7f4b7c104320 2023-11-16T16:30:49.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.479+0000 7f4b811c3700 1 -- 172.21.15.1:0/3544524800 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b780119b0 con 0x7f4b7c104320 2023-11-16T16:30:49.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.479+0000 7f4b84429700 1 -- 172.21.15.1:0/3544524800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b7c104320 msgr2=0x7f4b7c104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:49.291 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.479+0000 7f4b84429700 1 --2- 172.21.15.1:0/3544524800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b7c104320 0x7f4b7c104730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4b780098a0 tx=0x7f4b78009bb0 comp rx=0 tx=0).stop 2023-11-16T16:30:49.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.479+0000 7f4b84429700 1 -- 172.21.15.1:0/3544524800 shutdown_connections 2023-11-16T16:30:49.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.479+0000 7f4b84429700 1 --2- 172.21.15.1:0/3544524800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b7c104320 0x7f4b7c104730 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:49.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.479+0000 7f4b84429700 1 -- 172.21.15.1:0/3544524800 >> 172.21.15.1:0/3544524800 conn(0x7f4b7c0ffed0 msgr2=0x7f4b7c102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:49.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.480+0000 7f4b84429700 1 -- 172.21.15.1:0/3544524800 shutdown_connections 2023-11-16T16:30:49.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.480+0000 7f4b84429700 1 -- 172.21.15.1:0/3544524800 wait complete. 2023-11-16T16:30:49.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.480+0000 7f4b84429700 1 Processor -- start 2023-11-16T16:30:49.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.480+0000 7f4b84429700 1 -- start start 2023-11-16T16:30:49.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.480+0000 7f4b84429700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b7c104320 0x7f4b7c19b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:49.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.480+0000 7f4b84429700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c19bed0 con 0x7f4b7c104320 2023-11-16T16:30:49.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.480+0000 7f4b821c5700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b7c104320 0x7f4b7c19b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:49.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.480+0000 7f4b821c5700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b7c104320 0x7f4b7c19b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40310/0 (socket says 172.21.15.1:40310) 2023-11-16T16:30:49.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.480+0000 7f4b821c5700 1 -- 172.21.15.1:0/2621276881 learned_addr learned my addr 172.21.15.1:0/2621276881 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:49.292 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.481+0000 7f4b821c5700 1 -- 172.21.15.1:0/2621276881 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b78009550 con 0x7f4b7c104320 2023-11-16T16:30:49.293 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.481+0000 7f4b821c5700 1 --2- 172.21.15.1:0/2621276881 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b7c104320 0x7f4b7c19b990 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4b78011980 tx=0x7f4b78011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:49.293 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.481+0000 7f4b6f7fe700 1 -- 172.21.15.1:0/2621276881 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b78003a60 con 0x7f4b7c104320 2023-11-16T16:30:49.293 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.481+0000 7f4b6f7fe700 1 -- 172.21.15.1:0/2621276881 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b78003bc0 con 0x7f4b7c104320 2023-11-16T16:30:49.293 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.481+0000 7f4b84429700 1 -- 172.21.15.1:0/2621276881 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b7c105b20 con 0x7f4b7c104320 2023-11-16T16:30:49.293 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.481+0000 7f4b6f7fe700 1 -- 172.21.15.1:0/2621276881 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b7801a670 con 0x7f4b7c104320 2023-11-16T16:30:49.293 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.481+0000 7f4b84429700 1 -- 172.21.15.1:0/2621276881 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b7c105990 con 0x7f4b7c104320 2023-11-16T16:30:49.293 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.482+0000 7f4b6f7fe700 1 -- 172.21.15.1:0/2621276881 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f4b78024a00 con 0x7f4b7c104320 2023-11-16T16:30:49.293 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.482+0000 7f4b84429700 1 -- 172.21.15.1:0/2621276881 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b7c105cb0 con 0x7f4b7c104320 2023-11-16T16:30:49.293 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.482+0000 7f4b6f7fe700 1 --2- 172.21.15.1:0/2621276881 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f4b68038760 0x7f4b6803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:49.293 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.483+0000 7f4b6f7fe700 1 -- 172.21.15.1:0/2621276881 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f4b7804c860 con 0x7f4b7c104320 2023-11-16T16:30:49.293 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.483+0000 7f4b819c4700 1 --2- 172.21.15.1:0/2621276881 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f4b68038760 0x7f4b6803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:49.293 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.483+0000 7f4b819c4700 1 --2- 172.21.15.1:0/2621276881 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f4b68038760 0x7f4b6803ac10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4b70006fd0 tx=0x7f4b70006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:49.293 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.487+0000 7f4b6f7fe700 1 -- 172.21.15.1:0/2621276881 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4b78018b80 con 0x7f4b7c104320 2023-11-16T16:30:49.294 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.874+0000 7f4b84429700 1 -- 172.21.15.1:0/2621276881 --> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f4b7c060f30 con 0x7f4b68038760 2023-11-16T16:30:49.294 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.878+0000 7f4b6f7fe700 1 -- 172.21.15.1:0/2621276881 <== mgr.14120 v2:172.21.15.1:6800/3137425292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f4b7c060f30 con 0x7f4b68038760 2023-11-16T16:30:49.294 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.880+0000 7f4b84429700 1 -- 172.21.15.1:0/2621276881 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f4b68038760 msgr2=0x7f4b6803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:49.294 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.880+0000 7f4b84429700 1 --2- 172.21.15.1:0/2621276881 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f4b68038760 0x7f4b6803ac10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4b70006fd0 tx=0x7f4b70006e40 comp rx=0 tx=0).stop 2023-11-16T16:30:49.294 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.881+0000 7f4b84429700 1 -- 172.21.15.1:0/2621276881 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b7c104320 msgr2=0x7f4b7c19b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:49.294 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.881+0000 7f4b84429700 1 --2- 172.21.15.1:0/2621276881 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b7c104320 0x7f4b7c19b990 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4b78011980 tx=0x7f4b78011a60 comp rx=0 tx=0).stop 2023-11-16T16:30:49.294 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.881+0000 7f4b84429700 1 -- 172.21.15.1:0/2621276881 shutdown_connections 2023-11-16T16:30:49.294 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.881+0000 7f4b84429700 1 --2- 172.21.15.1:0/2621276881 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f4b68038760 0x7f4b6803ac10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:49.294 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.881+0000 7f4b84429700 1 --2- 172.21.15.1:0/2621276881 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b7c104320 0x7f4b7c19b990 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:49.294 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.881+0000 7f4b84429700 1 -- 172.21.15.1:0/2621276881 >> 172.21.15.1:0/2621276881 conn(0x7f4b7c0ffed0 msgr2=0x7f4b7c18fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:49.294 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.881+0000 7f4b84429700 1 -- 172.21.15.1:0/2621276881 shutdown_connections 2023-11-16T16:30:49.295 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:48.881+0000 7f4b84429700 1 -- 172.21.15.1:0/2621276881 wait complete. 2023-11-16T16:30:50.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:50 smithi001 ceph-mon[98800]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:30:50.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:50 smithi001 ceph-mon[98800]: Saving service mgr spec with placement count:2 2023-11-16T16:30:50.639 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.769+0000 7f10bb5c8700 1 Processor -- start 2023-11-16T16:30:50.639 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.769+0000 7f10bb5c8700 1 -- start start 2023-11-16T16:30:50.639 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.769+0000 7f10bb5c8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f10b4104280 0x7f10b4104690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:50.639 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.769+0000 7f10bb5c8700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b4079e60 con 0x7f10b4104280 2023-11-16T16:30:50.639 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.769+0000 7f10b9364700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f10b4104280 0x7f10b4104690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:50.639 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.769+0000 7f10b9364700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f10b4104280 0x7f10b4104690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40314/0 (socket says 172.21.15.1:40314) 2023-11-16T16:30:50.639 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.769+0000 7f10b9364700 1 -- 172.21.15.1:0/2238066198 learned_addr learned my addr 172.21.15.1:0/2238066198 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:50.639 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.770+0000 7f10b9364700 1 -- 172.21.15.1:0/2238066198 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b4104bd0 con 0x7f10b4104280 2023-11-16T16:30:50.639 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.770+0000 7f10b9364700 1 --2- 172.21.15.1:0/2238066198 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f10b4104280 0x7f10b4104690 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f10b00098a0 tx=0x7f10b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7263277fce316a server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:50.639 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.770+0000 7f10abfff700 1 -- 172.21.15.1:0/2238066198 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f10b0004160 con 0x7f10b4104280 2023-11-16T16:30:50.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.770+0000 7f10abfff700 1 -- 172.21.15.1:0/2238066198 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10b00042c0 con 0x7f10b4104280 2023-11-16T16:30:50.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.770+0000 7f10abfff700 1 -- 172.21.15.1:0/2238066198 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f10b00119b0 con 0x7f10b4104280 2023-11-16T16:30:50.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.770+0000 7f10bb5c8700 1 -- 172.21.15.1:0/2238066198 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f10b4104280 msgr2=0x7f10b4104690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:50.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.770+0000 7f10bb5c8700 1 --2- 172.21.15.1:0/2238066198 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f10b4104280 0x7f10b4104690 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f10b00098a0 tx=0x7f10b0009bb0 comp rx=0 tx=0).stop 2023-11-16T16:30:50.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.770+0000 7f10bb5c8700 1 -- 172.21.15.1:0/2238066198 shutdown_connections 2023-11-16T16:30:50.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.770+0000 7f10bb5c8700 1 --2- 172.21.15.1:0/2238066198 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f10b4104280 0x7f10b4104690 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:50.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.770+0000 7f10bb5c8700 1 -- 172.21.15.1:0/2238066198 >> 172.21.15.1:0/2238066198 conn(0x7f10b40ffed0 msgr2=0x7f10b4102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:50.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.771+0000 7f10bb5c8700 1 -- 172.21.15.1:0/2238066198 shutdown_connections 2023-11-16T16:30:50.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.771+0000 7f10bb5c8700 1 -- 172.21.15.1:0/2238066198 wait complete. 2023-11-16T16:30:50.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.771+0000 7f10bb5c8700 1 Processor -- start 2023-11-16T16:30:50.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.771+0000 7f10bb5c8700 1 -- start start 2023-11-16T16:30:50.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.771+0000 7f10bb5c8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f10b4104280 0x7f10b419b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:50.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.771+0000 7f10bb5c8700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b419bef0 con 0x7f10b4104280 2023-11-16T16:30:50.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.771+0000 7f10b9364700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f10b4104280 0x7f10b419b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:50.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.772+0000 7f10b9364700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f10b4104280 0x7f10b419b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40316/0 (socket says 172.21.15.1:40316) 2023-11-16T16:30:50.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.772+0000 7f10b9364700 1 -- 172.21.15.1:0/2314020605 learned_addr learned my addr 172.21.15.1:0/2314020605 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:50.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.772+0000 7f10b9364700 1 -- 172.21.15.1:0/2314020605 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b0009550 con 0x7f10b4104280 2023-11-16T16:30:50.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.772+0000 7f10b9364700 1 --2- 172.21.15.1:0/2314020605 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f10b4104280 0x7f10b419b9b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f10b0011980 tx=0x7f10b0011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:50.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.772+0000 7f10aa7fc700 1 -- 172.21.15.1:0/2314020605 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f10b0003a60 con 0x7f10b4104280 2023-11-16T16:30:50.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.772+0000 7f10aa7fc700 1 -- 172.21.15.1:0/2314020605 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10b0003bc0 con 0x7f10b4104280 2023-11-16T16:30:50.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.772+0000 7f10bb5c8700 1 -- 172.21.15.1:0/2314020605 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10b4105c10 con 0x7f10b4104280 2023-11-16T16:30:50.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.772+0000 7f10aa7fc700 1 -- 172.21.15.1:0/2314020605 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f10b001a670 con 0x7f10b4104280 2023-11-16T16:30:50.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.772+0000 7f10bb5c8700 1 -- 172.21.15.1:0/2314020605 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10b4105a80 con 0x7f10b4104280 2023-11-16T16:30:50.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.773+0000 7f10aa7fc700 1 -- 172.21.15.1:0/2314020605 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f10b001a930 con 0x7f10b4104280 2023-11-16T16:30:50.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.773+0000 7f10aa7fc700 1 --2- 172.21.15.1:0/2314020605 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f10a0038710 0x7f10a003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:50.642 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.773+0000 7f10aa7fc700 1 -- 172.21.15.1:0/2314020605 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f10b004c860 con 0x7f10b4104280 2023-11-16T16:30:50.642 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.773+0000 7f10b8b63700 1 --2- 172.21.15.1:0/2314020605 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f10a0038710 0x7f10a003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:50.642 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.773+0000 7f10bb5c8700 1 -- 172.21.15.1:0/2314020605 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10b4105da0 con 0x7f10b4104280 2023-11-16T16:30:50.642 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.774+0000 7f10b8b63700 1 --2- 172.21.15.1:0/2314020605 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f10a0038710 0x7f10a003abc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f10a4006fd0 tx=0x7f10a4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:50.642 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:49.778+0000 7f10aa7fc700 1 -- 172.21.15.1:0/2314020605 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f10b001abe0 con 0x7f10b4104280 2023-11-16T16:30:50.642 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:50.152+0000 7f10bb5c8700 1 -- 172.21.15.1:0/2314020605 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f10b4195240 con 0x7f10b4104280 2023-11-16T16:30:50.642 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:50.156+0000 7f10aa7fc700 1 -- 172.21.15.1:0/2314020605 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f10b0018350 con 0x7f10b4104280 2023-11-16T16:30:50.642 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:50.159+0000 7f10bb5c8700 1 -- 172.21.15.1:0/2314020605 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f10a0038710 msgr2=0x7f10a003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:50.643 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:50.159+0000 7f10bb5c8700 1 --2- 172.21.15.1:0/2314020605 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f10a0038710 0x7f10a003abc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f10a4006fd0 tx=0x7f10a4006e40 comp rx=0 tx=0).stop 2023-11-16T16:30:50.643 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:50.159+0000 7f10bb5c8700 1 -- 172.21.15.1:0/2314020605 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f10b4104280 msgr2=0x7f10b419b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:50.643 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:50.159+0000 7f10bb5c8700 1 --2- 172.21.15.1:0/2314020605 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f10b4104280 0x7f10b419b9b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f10b0011980 tx=0x7f10b0011a60 comp rx=0 tx=0).stop 2023-11-16T16:30:50.643 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:50.159+0000 7f10bb5c8700 1 -- 172.21.15.1:0/2314020605 shutdown_connections 2023-11-16T16:30:50.643 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:50.159+0000 7f10bb5c8700 1 --2- 172.21.15.1:0/2314020605 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f10a0038710 0x7f10a003abc0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:50.643 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:50.159+0000 7f10bb5c8700 1 --2- 172.21.15.1:0/2314020605 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f10b4104280 0x7f10b419b9b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:50.643 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:50.159+0000 7f10bb5c8700 1 -- 172.21.15.1:0/2314020605 >> 172.21.15.1:0/2314020605 conn(0x7f10b40ffed0 msgr2=0x7f10b418fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:50.643 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:50.160+0000 7f10bb5c8700 1 -- 172.21.15.1:0/2314020605 shutdown_connections 2023-11-16T16:30:50.643 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:50.160+0000 7f10bb5c8700 1 -- 172.21.15.1:0/2314020605 wait complete. 2023-11-16T16:30:51.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:51 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/2314020605' entity='client.admin' 2023-11-16T16:30:51.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:51 smithi001 ceph-mon[98800]: from='mgr.14120 172.21.15.1:0/1538801972' entity='mgr.x' 2023-11-16T16:30:51.885 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.140+0000 7fb4b49e2700 1 Processor -- start 2023-11-16T16:30:51.885 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.140+0000 7fb4b49e2700 1 -- start start 2023-11-16T16:30:51.885 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.141+0000 7fb4b49e2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4b0104920 0x7fb4b0104d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:51.885 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.141+0000 7fb4b49e2700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b0105300 con 0x7fb4b0104920 2023-11-16T16:30:51.885 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.141+0000 7fb4ae59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4b0104920 0x7fb4b0104d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:51.885 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.141+0000 7fb4ae59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4b0104920 0x7fb4b0104d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40320/0 (socket says 172.21.15.1:40320) 2023-11-16T16:30:51.885 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.141+0000 7fb4ae59c700 1 -- 172.21.15.1:0/273051014 learned_addr learned my addr 172.21.15.1:0/273051014 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:51.885 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.141+0000 7fb4ae59c700 1 -- 172.21.15.1:0/273051014 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b0105b20 con 0x7fb4b0104920 2023-11-16T16:30:51.886 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.141+0000 7fb4ae59c700 1 --2- 172.21.15.1:0/273051014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4b0104920 0x7fb4b0104d30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb4a00098a0 tx=0x7fb4a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d8b1f4fdb3eb5bc server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:51.886 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.141+0000 7fb4ad59a700 1 -- 172.21.15.1:0/273051014 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4a0004160 con 0x7fb4b0104920 2023-11-16T16:30:51.886 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.142+0000 7fb4ad59a700 1 -- 172.21.15.1:0/273051014 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4a00042c0 con 0x7fb4b0104920 2023-11-16T16:30:51.886 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.142+0000 7fb4ad59a700 1 -- 172.21.15.1:0/273051014 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4a00119b0 con 0x7fb4b0104920 2023-11-16T16:30:51.886 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.142+0000 7fb4b49e2700 1 -- 172.21.15.1:0/273051014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4b0104920 msgr2=0x7fb4b0104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:51.886 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.142+0000 7fb4b49e2700 1 --2- 172.21.15.1:0/273051014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4b0104920 0x7fb4b0104d30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb4a00098a0 tx=0x7fb4a0009bb0 comp rx=0 tx=0).stop 2023-11-16T16:30:51.886 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.142+0000 7fb4b49e2700 1 -- 172.21.15.1:0/273051014 shutdown_connections 2023-11-16T16:30:51.886 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.142+0000 7fb4b49e2700 1 --2- 172.21.15.1:0/273051014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4b0104920 0x7fb4b0104d30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:51.886 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.142+0000 7fb4b49e2700 1 -- 172.21.15.1:0/273051014 >> 172.21.15.1:0/273051014 conn(0x7fb4b00ffed0 msgr2=0x7fb4b0102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:51.886 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.142+0000 7fb4b49e2700 1 -- 172.21.15.1:0/273051014 shutdown_connections 2023-11-16T16:30:51.886 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.142+0000 7fb4b49e2700 1 -- 172.21.15.1:0/273051014 wait complete. 2023-11-16T16:30:51.887 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.143+0000 7fb4b49e2700 1 Processor -- start 2023-11-16T16:30:51.887 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.143+0000 7fb4b49e2700 1 -- start start 2023-11-16T16:30:51.887 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.143+0000 7fb4b49e2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4b0104920 0x7fb4b0079ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:51.887 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.143+0000 7fb4b49e2700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b019a2f0 con 0x7fb4b0104920 2023-11-16T16:30:51.887 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.144+0000 7fb4ae59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4b0104920 0x7fb4b0079ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:51.887 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.144+0000 7fb4ae59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4b0104920 0x7fb4b0079ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40322/0 (socket says 172.21.15.1:40322) 2023-11-16T16:30:51.887 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.144+0000 7fb4ae59c700 1 -- 172.21.15.1:0/1783863179 learned_addr learned my addr 172.21.15.1:0/1783863179 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:51.887 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.144+0000 7fb4ae59c700 1 -- 172.21.15.1:0/1783863179 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a0009550 con 0x7fb4b0104920 2023-11-16T16:30:51.887 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.144+0000 7fb4ae59c700 1 --2- 172.21.15.1:0/1783863179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4b0104920 0x7fb4b0079ae0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0000c00 tx=0x7fb4a0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:51.887 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.144+0000 7fb49f7fe700 1 -- 172.21.15.1:0/1783863179 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4a00039f0 con 0x7fb4b0104920 2023-11-16T16:30:51.887 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.144+0000 7fb49f7fe700 1 -- 172.21.15.1:0/1783863179 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4a0003b50 con 0x7fb4b0104920 2023-11-16T16:30:51.887 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.144+0000 7fb4b49e2700 1 -- 172.21.15.1:0/1783863179 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4b0108ee0 con 0x7fb4b0104920 2023-11-16T16:30:51.888 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.144+0000 7fb49f7fe700 1 -- 172.21.15.1:0/1783863179 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4a001a780 con 0x7fb4b0104920 2023-11-16T16:30:51.888 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.144+0000 7fb4b49e2700 1 -- 172.21.15.1:0/1783863179 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4b0108d50 con 0x7fb4b0104920 2023-11-16T16:30:51.888 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.146+0000 7fb4b49e2700 1 -- 172.21.15.1:0/1783863179 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4b0109070 con 0x7fb4b0104920 2023-11-16T16:30:51.888 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.146+0000 7fb49f7fe700 1 -- 172.21.15.1:0/1783863179 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fb4a0024440 con 0x7fb4b0104920 2023-11-16T16:30:51.888 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.146+0000 7fb49f7fe700 1 --2- 172.21.15.1:0/1783863179 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7fb498038760 0x7fb49803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:51.888 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.146+0000 7fb49f7fe700 1 -- 172.21.15.1:0/1783863179 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb4a004c770 con 0x7fb4b0104920 2023-11-16T16:30:51.888 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.146+0000 7fb4add9b700 1 --2- 172.21.15.1:0/1783863179 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7fb498038760 0x7fb49803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:51.888 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.147+0000 7fb4add9b700 1 --2- 172.21.15.1:0/1783863179 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7fb498038760 0x7fb49803ac10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4006fd0 tx=0x7fb4a4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:51.888 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.151+0000 7fb49f7fe700 1 -- 172.21.15.1:0/1783863179 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb4a00246f0 con 0x7fb4b0104920 2023-11-16T16:30:51.888 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.513+0000 7fb4b49e2700 1 -- 172.21.15.1:0/1783863179 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fb4b019f650 con 0x7fb4b0104920 2023-11-16T16:30:51.888 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.516+0000 7fb49f7fe700 1 -- 172.21.15.1:0/1783863179 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fb4a0018350 con 0x7fb4b0104920 2023-11-16T16:30:51.889 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.518+0000 7fb4b49e2700 1 -- 172.21.15.1:0/1783863179 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7fb498038760 msgr2=0x7fb49803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:51.889 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.518+0000 7fb4b49e2700 1 --2- 172.21.15.1:0/1783863179 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7fb498038760 0x7fb49803ac10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4006fd0 tx=0x7fb4a4006e40 comp rx=0 tx=0).stop 2023-11-16T16:30:51.889 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.518+0000 7fb4b49e2700 1 -- 172.21.15.1:0/1783863179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4b0104920 msgr2=0x7fb4b0079ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:51.889 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.518+0000 7fb4b49e2700 1 --2- 172.21.15.1:0/1783863179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4b0104920 0x7fb4b0079ae0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0000c00 tx=0x7fb4a0011a30 comp rx=0 tx=0).stop 2023-11-16T16:30:51.889 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.519+0000 7fb4b49e2700 1 -- 172.21.15.1:0/1783863179 shutdown_connections 2023-11-16T16:30:51.889 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.519+0000 7fb4b49e2700 1 --2- 172.21.15.1:0/1783863179 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7fb498038760 0x7fb49803ac10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:51.889 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.519+0000 7fb4b49e2700 1 --2- 172.21.15.1:0/1783863179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4b0104920 0x7fb4b0079ae0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:51.889 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.519+0000 7fb4b49e2700 1 -- 172.21.15.1:0/1783863179 >> 172.21.15.1:0/1783863179 conn(0x7fb4b00ffed0 msgr2=0x7fb4b0101580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:51.889 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.519+0000 7fb4b49e2700 1 -- 172.21.15.1:0/1783863179 shutdown_connections 2023-11-16T16:30:51.889 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:51.519+0000 7fb4b49e2700 1 -- 172.21.15.1:0/1783863179 wait complete. 2023-11-16T16:30:51.889 INFO:teuthology.orchestra.run.smithi001.stdout:Enabling the dashboard module... 2023-11-16T16:30:52.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:52 smithi001 ceph-mon[98800]: from='mgr.14120 172.21.15.1:0/1538801972' entity='mgr.x' 2023-11-16T16:30:52.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:52 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1783863179' entity='client.admin' 2023-11-16T16:30:53.655 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:53 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1094375263' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-11-16T16:30:53.655 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:53 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: ignoring --setuser ceph since I am not root 2023-11-16T16:30:53.655 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:53 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: ignoring --setgroup ceph since I am not root 2023-11-16T16:30:53.655 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:53 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:53.556+0000 7f94084e3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-16T16:30:53.656 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:53 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:53.652+0000 7f94084e3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-16T16:30:53.834 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.376+0000 7fc8580f8700 1 Processor -- start 2023-11-16T16:30:53.834 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.376+0000 7fc8580f8700 1 -- start start 2023-11-16T16:30:53.834 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.377+0000 7fc8580f8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc850104920 0x7fc850104d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:53.834 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.377+0000 7fc8580f8700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc850105300 con 0x7fc850104920 2023-11-16T16:30:53.834 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.377+0000 7fc855e94700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc850104920 0x7fc850104d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:53.834 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.377+0000 7fc855e94700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc850104920 0x7fc850104d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40326/0 (socket says 172.21.15.1:40326) 2023-11-16T16:30:53.834 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.377+0000 7fc855e94700 1 -- 172.21.15.1:0/833531126 learned_addr learned my addr 172.21.15.1:0/833531126 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:53.835 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.377+0000 7fc855e94700 1 -- 172.21.15.1:0/833531126 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc850105b20 con 0x7fc850104920 2023-11-16T16:30:53.835 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.377+0000 7fc855e94700 1 --2- 172.21.15.1:0/833531126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc850104920 0x7fc850104d30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc84c0098a0 tx=0x7fc84c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a3215373355e52 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:53.835 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.377+0000 7fc854e92700 1 -- 172.21.15.1:0/833531126 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc84c004160 con 0x7fc850104920 2023-11-16T16:30:53.835 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.377+0000 7fc854e92700 1 -- 172.21.15.1:0/833531126 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc84c0042c0 con 0x7fc850104920 2023-11-16T16:30:53.835 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.377+0000 7fc854e92700 1 -- 172.21.15.1:0/833531126 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc84c0119b0 con 0x7fc850104920 2023-11-16T16:30:53.835 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.378+0000 7fc8580f8700 1 -- 172.21.15.1:0/833531126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc850104920 msgr2=0x7fc850104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:53.835 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.378+0000 7fc8580f8700 1 --2- 172.21.15.1:0/833531126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc850104920 0x7fc850104d30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc84c0098a0 tx=0x7fc84c009bb0 comp rx=0 tx=0).stop 2023-11-16T16:30:53.835 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.379+0000 7fc8580f8700 1 -- 172.21.15.1:0/833531126 shutdown_connections 2023-11-16T16:30:53.835 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.379+0000 7fc8580f8700 1 --2- 172.21.15.1:0/833531126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc850104920 0x7fc850104d30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:53.835 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.379+0000 7fc8580f8700 1 -- 172.21.15.1:0/833531126 >> 172.21.15.1:0/833531126 conn(0x7fc8500ffed0 msgr2=0x7fc850102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:53.835 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.379+0000 7fc8580f8700 1 -- 172.21.15.1:0/833531126 shutdown_connections 2023-11-16T16:30:53.835 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.379+0000 7fc8580f8700 1 -- 172.21.15.1:0/833531126 wait complete. 2023-11-16T16:30:53.836 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.379+0000 7fc8580f8700 1 Processor -- start 2023-11-16T16:30:53.836 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.379+0000 7fc8580f8700 1 -- start start 2023-11-16T16:30:53.836 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.379+0000 7fc8580f8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc850104920 0x7fc850079ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:53.836 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.380+0000 7fc8580f8700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85019a2f0 con 0x7fc850104920 2023-11-16T16:30:53.836 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.380+0000 7fc855e94700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc850104920 0x7fc850079ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:53.836 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.380+0000 7fc855e94700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc850104920 0x7fc850079ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40328/0 (socket says 172.21.15.1:40328) 2023-11-16T16:30:53.836 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.380+0000 7fc855e94700 1 -- 172.21.15.1:0/1094375263 learned_addr learned my addr 172.21.15.1:0/1094375263 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:53.836 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.380+0000 7fc855e94700 1 -- 172.21.15.1:0/1094375263 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc84c009550 con 0x7fc850104920 2023-11-16T16:30:53.836 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.380+0000 7fc855e94700 1 --2- 172.21.15.1:0/1094375263 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc850104920 0x7fc850079ae0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc84c003e60 tx=0x7fc84c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:53.836 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.380+0000 7fc846ffd700 1 -- 172.21.15.1:0/1094375263 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc84c003970 con 0x7fc850104920 2023-11-16T16:30:53.836 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.380+0000 7fc846ffd700 1 -- 172.21.15.1:0/1094375263 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc84c003ad0 con 0x7fc850104920 2023-11-16T16:30:53.836 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.380+0000 7fc8580f8700 1 -- 172.21.15.1:0/1094375263 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc850108ee0 con 0x7fc850104920 2023-11-16T16:30:53.836 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.380+0000 7fc846ffd700 1 -- 172.21.15.1:0/1094375263 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc84c0195e0 con 0x7fc850104920 2023-11-16T16:30:53.837 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.381+0000 7fc8580f8700 1 -- 172.21.15.1:0/1094375263 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc850108d50 con 0x7fc850104920 2023-11-16T16:30:53.837 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.381+0000 7fc846ffd700 1 -- 172.21.15.1:0/1094375263 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fc84c0198a0 con 0x7fc850104920 2023-11-16T16:30:53.837 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.382+0000 7fc846ffd700 1 --2- 172.21.15.1:0/1094375263 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7fc83c0386c0 0x7fc83c03ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:53.837 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.382+0000 7fc8580f8700 1 -- 172.21.15.1:0/1094375263 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc850109070 con 0x7fc850104920 2023-11-16T16:30:53.837 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.382+0000 7fc846ffd700 1 -- 172.21.15.1:0/1094375263 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc84c04ca50 con 0x7fc850104920 2023-11-16T16:30:53.837 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.382+0000 7fc855693700 1 --2- 172.21.15.1:0/1094375263 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7fc83c0386c0 0x7fc83c03ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:53.837 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.382+0000 7fc855693700 1 --2- 172.21.15.1:0/1094375263 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7fc83c0386c0 0x7fc83c03ab70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc840006fd0 tx=0x7fc840006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:53.837 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.387+0000 7fc846ffd700 1 -- 172.21.15.1:0/1094375263 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc84c019ba0 con 0x7fc850104920 2023-11-16T16:30:53.837 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:52.870+0000 7fc8580f8700 1 -- 172.21.15.1:0/1094375263 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fc85019ef50 con 0x7fc850104920 2023-11-16T16:30:53.838 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:53.344+0000 7fc846ffd700 1 -- 172.21.15.1:0/1094375263 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fc84c018d20 con 0x7fc850104920 2023-11-16T16:30:53.838 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:53.345+0000 7fc846ffd700 1 -- 172.21.15.1:0/1094375263 <== mon.0 v2:172.21.15.1:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fc84c00bc10 con 0x7fc850104920 2023-11-16T16:30:53.838 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:53.349+0000 7fc8580f8700 1 -- 172.21.15.1:0/1094375263 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7fc83c0386c0 msgr2=0x7fc83c03ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:53.838 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:53.349+0000 7fc8580f8700 1 --2- 172.21.15.1:0/1094375263 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7fc83c0386c0 0x7fc83c03ab70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc840006fd0 tx=0x7fc840006e40 comp rx=0 tx=0).stop 2023-11-16T16:30:53.838 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:53.349+0000 7fc8580f8700 1 -- 172.21.15.1:0/1094375263 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc850104920 msgr2=0x7fc850079ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:53.838 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:53.349+0000 7fc8580f8700 1 --2- 172.21.15.1:0/1094375263 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc850104920 0x7fc850079ae0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc84c003e60 tx=0x7fc84c011aa0 comp rx=0 tx=0).stop 2023-11-16T16:30:53.838 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:53.349+0000 7fc8580f8700 1 -- 172.21.15.1:0/1094375263 shutdown_connections 2023-11-16T16:30:53.838 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:53.349+0000 7fc8580f8700 1 --2- 172.21.15.1:0/1094375263 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7fc83c0386c0 0x7fc83c03ab70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:53.838 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:53.349+0000 7fc8580f8700 1 --2- 172.21.15.1:0/1094375263 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc850104920 0x7fc850079ae0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:53.838 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:53.349+0000 7fc8580f8700 1 -- 172.21.15.1:0/1094375263 >> 172.21.15.1:0/1094375263 conn(0x7fc8500ffed0 msgr2=0x7fc850101580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:53.838 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:53.349+0000 7fc8580f8700 1 -- 172.21.15.1:0/1094375263 shutdown_connections 2023-11-16T16:30:53.838 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:53.349+0000 7fc8580f8700 1 -- 172.21.15.1:0/1094375263 wait complete. 2023-11-16T16:30:53.932 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:53 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:53.762+0000 7f94084e3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-16T16:30:53.932 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:53 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:53.877+0000 7f94084e3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-16T16:30:54.345 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:54 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:54.094+0000 7f94084e3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-16T16:30:54.612 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:54 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1094375263' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-11-16T16:30:54.612 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:54 smithi001 ceph-mon[98800]: mgrmap e9: x(active, since 18s) 2023-11-16T16:30:54.612 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:54 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:54.502+0000 7f94084e3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-16T16:30:54.880 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:54 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:54.610+0000 7f94084e3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-16T16:30:54.881 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:54 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:54.762+0000 7f94084e3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-16T16:30:55.182 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:54 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:54.878+0000 7f94084e3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-16T16:30:55.273 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2023-11-16T16:30:55.279 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-11-16T16:30:55.279 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "available": true, 2023-11-16T16:30:55.279 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-11-16T16:30:55.279 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-11-16T16:30:55.280 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2023-11-16T16:30:55.280 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.355+0000 7fbf9022c700 1 Processor -- start 2023-11-16T16:30:55.280 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.355+0000 7fbf9022c700 1 -- start start 2023-11-16T16:30:55.280 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.355+0000 7fbf9022c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf88104280 0x7fbf88104690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:55.280 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.355+0000 7fbf9022c700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf88079e60 con 0x7fbf88104280 2023-11-16T16:30:55.280 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.355+0000 7fbf8dfc8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf88104280 0x7fbf88104690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:55.280 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.355+0000 7fbf8dfc8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf88104280 0x7fbf88104690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40342/0 (socket says 172.21.15.1:40342) 2023-11-16T16:30:55.280 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.355+0000 7fbf8dfc8700 1 -- 172.21.15.1:0/4204530256 learned_addr learned my addr 172.21.15.1:0/4204530256 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:55.280 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.356+0000 7fbf8dfc8700 1 -- 172.21.15.1:0/4204530256 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf88104bd0 con 0x7fbf88104280 2023-11-16T16:30:55.280 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.356+0000 7fbf8dfc8700 1 --2- 172.21.15.1:0/4204530256 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf88104280 0x7fbf88104690 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbf840098a0 tx=0x7fbf84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb65d427e715202 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:55.280 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.356+0000 7fbf8cfc6700 1 -- 172.21.15.1:0/4204530256 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf84004160 con 0x7fbf88104280 2023-11-16T16:30:55.280 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.356+0000 7fbf8cfc6700 1 -- 172.21.15.1:0/4204530256 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf840042c0 con 0x7fbf88104280 2023-11-16T16:30:55.281 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.356+0000 7fbf8cfc6700 1 -- 172.21.15.1:0/4204530256 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf840119b0 con 0x7fbf88104280 2023-11-16T16:30:55.281 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.357+0000 7fbf9022c700 1 -- 172.21.15.1:0/4204530256 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf88104280 msgr2=0x7fbf88104690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:55.281 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.357+0000 7fbf9022c700 1 --2- 172.21.15.1:0/4204530256 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf88104280 0x7fbf88104690 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbf840098a0 tx=0x7fbf84009bb0 comp rx=0 tx=0).stop 2023-11-16T16:30:55.281 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.357+0000 7fbf9022c700 1 -- 172.21.15.1:0/4204530256 shutdown_connections 2023-11-16T16:30:55.281 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.357+0000 7fbf9022c700 1 --2- 172.21.15.1:0/4204530256 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf88104280 0x7fbf88104690 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:55.281 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.357+0000 7fbf9022c700 1 -- 172.21.15.1:0/4204530256 >> 172.21.15.1:0/4204530256 conn(0x7fbf880ffed0 msgr2=0x7fbf88102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:55.281 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.357+0000 7fbf9022c700 1 -- 172.21.15.1:0/4204530256 shutdown_connections 2023-11-16T16:30:55.281 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.357+0000 7fbf9022c700 1 -- 172.21.15.1:0/4204530256 wait complete. 2023-11-16T16:30:55.281 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.357+0000 7fbf9022c700 1 Processor -- start 2023-11-16T16:30:55.281 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.358+0000 7fbf9022c700 1 -- start start 2023-11-16T16:30:55.281 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.358+0000 7fbf9022c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf88104280 0x7fbf8819b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:55.281 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.358+0000 7fbf9022c700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8819beb0 con 0x7fbf88104280 2023-11-16T16:30:55.281 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.358+0000 7fbf8dfc8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf88104280 0x7fbf8819b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:30:55.282 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.358+0000 7fbf8dfc8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf88104280 0x7fbf8819b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40344/0 (socket says 172.21.15.1:40344) 2023-11-16T16:30:55.282 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.358+0000 7fbf8dfc8700 1 -- 172.21.15.1:0/970634685 learned_addr learned my addr 172.21.15.1:0/970634685 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:30:55.282 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.358+0000 7fbf8dfc8700 1 -- 172.21.15.1:0/970634685 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf84009550 con 0x7fbf88104280 2023-11-16T16:30:55.282 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.358+0000 7fbf8dfc8700 1 --2- 172.21.15.1:0/970634685 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf88104280 0x7fbf8819b970 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbf84003e60 tx=0x7fbf84011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:30:55.282 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.359+0000 7fbf7effd700 1 -- 172.21.15.1:0/970634685 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf84003970 con 0x7fbf88104280 2023-11-16T16:30:55.282 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.359+0000 7fbf7effd700 1 -- 172.21.15.1:0/970634685 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf84003ad0 con 0x7fbf88104280 2023-11-16T16:30:55.282 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.359+0000 7fbf9022c700 1 -- 172.21.15.1:0/970634685 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf88105c10 con 0x7fbf88104280 2023-11-16T16:30:55.282 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.359+0000 7fbf7effd700 1 -- 172.21.15.1:0/970634685 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf840195e0 con 0x7fbf88104280 2023-11-16T16:30:55.282 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.359+0000 7fbf9022c700 1 -- 172.21.15.1:0/970634685 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf88105a80 con 0x7fbf88104280 2023-11-16T16:30:55.282 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.359+0000 7fbf7effd700 1 -- 172.21.15.1:0/970634685 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fbf84019850 con 0x7fbf88104280 2023-11-16T16:30:55.282 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.360+0000 7fbf7effd700 1 --2- 172.21.15.1:0/970634685 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7fbf74038760 0x7fbf7403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:30:55.282 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.360+0000 7fbf7effd700 1 -- 172.21.15.1:0/970634685 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fbf8404c890 con 0x7fbf88104280 2023-11-16T16:30:55.283 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.360+0000 7fbf8d7c7700 1 -- 172.21.15.1:0/970634685 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7fbf74038760 msgr2=0x7fbf7403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/3137425292 2023-11-16T16:30:55.283 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.360+0000 7fbf8d7c7700 1 --2- 172.21.15.1:0/970634685 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7fbf74038760 0x7fbf7403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-16T16:30:55.283 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.360+0000 7fbf9022c700 1 -- 172.21.15.1:0/970634685 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf88105da0 con 0x7fbf88104280 2023-11-16T16:30:55.283 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.365+0000 7fbf7effd700 1 -- 172.21.15.1:0/970634685 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbf84019ba0 con 0x7fbf88104280 2023-11-16T16:30:55.283 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.561+0000 7fbf8d7c7700 1 -- 172.21.15.1:0/970634685 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7fbf74038760 msgr2=0x7fbf7403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/3137425292 2023-11-16T16:30:55.283 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.561+0000 7fbf8d7c7700 1 --2- 172.21.15.1:0/970634685 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7fbf74038760 0x7fbf7403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-16T16:30:55.283 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.838+0000 7fbf9022c700 1 -- 172.21.15.1:0/970634685 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fbf88195220 con 0x7fbf88104280 2023-11-16T16:30:55.283 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.839+0000 7fbf7effd700 1 -- 172.21.15.1:0/970634685 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7fbf84018b40 con 0x7fbf88104280 2023-11-16T16:30:55.283 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.841+0000 7fbf9022c700 1 -- 172.21.15.1:0/970634685 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7fbf74038760 msgr2=0x7fbf7403ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-16T16:30:55.283 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.841+0000 7fbf9022c700 1 --2- 172.21.15.1:0/970634685 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7fbf74038760 0x7fbf7403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:55.283 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.841+0000 7fbf9022c700 1 -- 172.21.15.1:0/970634685 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf88104280 msgr2=0x7fbf8819b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:30:55.283 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.841+0000 7fbf9022c700 1 --2- 172.21.15.1:0/970634685 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf88104280 0x7fbf8819b970 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbf84003e60 tx=0x7fbf84011aa0 comp rx=0 tx=0).stop 2023-11-16T16:30:55.284 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.841+0000 7fbf9022c700 1 -- 172.21.15.1:0/970634685 shutdown_connections 2023-11-16T16:30:55.284 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.841+0000 7fbf9022c700 1 --2- 172.21.15.1:0/970634685 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7fbf74038760 0x7fbf7403ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:55.284 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.841+0000 7fbf9022c700 1 --2- 172.21.15.1:0/970634685 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf88104280 0x7fbf8819b970 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:30:55.284 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.841+0000 7fbf9022c700 1 -- 172.21.15.1:0/970634685 >> 172.21.15.1:0/970634685 conn(0x7fbf880ffed0 msgr2=0x7fbf8818fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:30:55.284 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.841+0000 7fbf9022c700 1 -- 172.21.15.1:0/970634685 shutdown_connections 2023-11-16T16:30:55.284 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:54.841+0000 7fbf9022c700 1 -- 172.21.15.1:0/970634685 wait complete. 2023-11-16T16:30:55.284 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for the mgr to restart... 2023-11-16T16:30:55.284 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mgr epoch 9... 2023-11-16T16:30:55.457 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:30:55 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/970634685' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-11-16T16:30:55.457 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:55 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:55.408+0000 7f94084e3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-16T16:30:56.432 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:56 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:56.104+0000 7f94084e3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-16T16:30:56.432 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:56 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:56.297+0000 7f94084e3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-16T16:30:57.591 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:57 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:57.259+0000 7f94084e3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-16T16:30:57.592 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:57 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:57.370+0000 7f94084e3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-16T16:30:58.191 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:57 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:57.589+0000 7f94084e3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-16T16:30:58.191 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:57 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:57.937+0000 7f94084e3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-16T16:30:58.682 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:58 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:58.188+0000 7f94084e3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-16T16:30:58.682 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:58 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:58.370+0000 7f94084e3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-16T16:30:59.182 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:58 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:58.701+0000 7f94084e3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-16T16:30:59.931 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:59 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:59.528+0000 7f94084e3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-16T16:30:59.931 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:59 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:59.686+0000 7f94084e3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-16T16:31:00.182 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:30:59 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:30:59.929+0000 7f94084e3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-16T16:31:00.182 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:31:00 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:31:00.065+0000 7f94084e3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-16T16:31:00.682 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:31:00 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:31:00.337+0000 7f94084e3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-16T16:31:01.682 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:31:01 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:31:01.377+0000 7f94084e3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-16T16:31:02.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:02 smithi001 ceph-mon[98800]: Active manager daemon x restarted 2023-11-16T16:31:02.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:02 smithi001 ceph-mon[98800]: Activating manager daemon x 2023-11-16T16:31:02.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:02 smithi001 ceph-mon[98800]: osdmap e3: 0 total, 0 up, 0 in 2023-11-16T16:31:02.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:02 smithi001 ceph-mon[98800]: mgrmap e10: x(active, starting, since 0.00353313s) 2023-11-16T16:31:02.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:02 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T16:31:02.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:02 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-11-16T16:31:02.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:02 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-11-16T16:31:02.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:02 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-11-16T16:31:02.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:02 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-11-16T16:31:02.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:02 smithi001 ceph-mon[98800]: Manager daemon x is now available 2023-11-16T16:31:02.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:02 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:31:02.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:02 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:31:02.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:02 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-11-16T16:31:02.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:02 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-11-16T16:31:02.843 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2023-11-16T16:31:02.843 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-11-16T16:31:02.843 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "initialized": true 2023-11-16T16:31:02.843 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2023-11-16T16:31:02.843 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.742+0000 7f5891160700 1 Processor -- start 2023-11-16T16:31:02.843 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.742+0000 7f5891160700 1 -- start start 2023-11-16T16:31:02.843 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.742+0000 7f5891160700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f588c106510 0x7f588c1088f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:02.843 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.742+0000 7f5891160700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588c079de0 con 0x7f588c106510 2023-11-16T16:31:02.843 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.742+0000 7f588ad9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f588c106510 0x7f588c1088f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:02.843 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.742+0000 7f588ad9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f588c106510 0x7f588c1088f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40352/0 (socket says 172.21.15.1:40352) 2023-11-16T16:31:02.843 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.742+0000 7f588ad9d700 1 -- 172.21.15.1:0/4123176501 learned_addr learned my addr 172.21.15.1:0/4123176501 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:31:02.844 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.743+0000 7f588ad9d700 1 -- 172.21.15.1:0/4123176501 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f588c108e30 con 0x7f588c106510 2023-11-16T16:31:02.844 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.743+0000 7f588ad9d700 1 --2- 172.21.15.1:0/4123176501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f588c106510 0x7f588c1088f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f587c0098a0 tx=0x7f587c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4333b0d9e071b6e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:02.844 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.743+0000 7f5889d9b700 1 -- 172.21.15.1:0/4123176501 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f587c004160 con 0x7f588c106510 2023-11-16T16:31:02.844 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.743+0000 7f5889d9b700 1 -- 172.21.15.1:0/4123176501 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f587c0042c0 con 0x7f588c106510 2023-11-16T16:31:02.844 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.743+0000 7f5889d9b700 1 -- 172.21.15.1:0/4123176501 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f587c0119b0 con 0x7f588c106510 2023-11-16T16:31:02.844 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.744+0000 7f5891160700 1 -- 172.21.15.1:0/4123176501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f588c106510 msgr2=0x7f588c1088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:02.844 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.744+0000 7f5891160700 1 --2- 172.21.15.1:0/4123176501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f588c106510 0x7f588c1088f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f587c0098a0 tx=0x7f587c009bb0 comp rx=0 tx=0).stop 2023-11-16T16:31:02.844 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.744+0000 7f5891160700 1 -- 172.21.15.1:0/4123176501 shutdown_connections 2023-11-16T16:31:02.844 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.744+0000 7f5891160700 1 --2- 172.21.15.1:0/4123176501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f588c106510 0x7f588c1088f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:02.844 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.744+0000 7f5891160700 1 -- 172.21.15.1:0/4123176501 >> 172.21.15.1:0/4123176501 conn(0x7f588c0ffed0 msgr2=0x7f588c102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:31:02.844 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.744+0000 7f5891160700 1 -- 172.21.15.1:0/4123176501 shutdown_connections 2023-11-16T16:31:02.844 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.744+0000 7f5891160700 1 -- 172.21.15.1:0/4123176501 wait complete. 2023-11-16T16:31:02.845 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.745+0000 7f5891160700 1 Processor -- start 2023-11-16T16:31:02.845 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.745+0000 7f5891160700 1 -- start start 2023-11-16T16:31:02.845 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.745+0000 7f5891160700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f588c106510 0x7f588c199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:02.845 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.745+0000 7f5891160700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588c199cd0 con 0x7f588c106510 2023-11-16T16:31:02.845 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.745+0000 7f588ad9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f588c106510 0x7f588c199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:02.845 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.746+0000 7f588ad9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f588c106510 0x7f588c199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40354/0 (socket says 172.21.15.1:40354) 2023-11-16T16:31:02.845 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.746+0000 7f588ad9d700 1 -- 172.21.15.1:0/3371189591 learned_addr learned my addr 172.21.15.1:0/3371189591 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:31:02.845 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.746+0000 7f588ad9d700 1 -- 172.21.15.1:0/3371189591 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f587c009550 con 0x7f588c106510 2023-11-16T16:31:02.845 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.746+0000 7f588ad9d700 1 --2- 172.21.15.1:0/3371189591 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f588c106510 0x7f588c199790 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f587c011980 tx=0x7f587c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:02.845 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.746+0000 7f5873fff700 1 -- 172.21.15.1:0/3371189591 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f587c003a60 con 0x7f588c106510 2023-11-16T16:31:02.845 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.746+0000 7f5873fff700 1 -- 172.21.15.1:0/3371189591 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f587c003bc0 con 0x7f588c106510 2023-11-16T16:31:02.845 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.746+0000 7f5891160700 1 -- 172.21.15.1:0/3371189591 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f588c109d50 con 0x7f588c106510 2023-11-16T16:31:02.846 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.746+0000 7f5873fff700 1 -- 172.21.15.1:0/3371189591 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f587c01a670 con 0x7f588c106510 2023-11-16T16:31:02.846 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.746+0000 7f5891160700 1 -- 172.21.15.1:0/3371189591 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f588c109bc0 con 0x7f588c106510 2023-11-16T16:31:02.846 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.747+0000 7f5873fff700 1 -- 172.21.15.1:0/3371189591 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f587c01a930 con 0x7f588c106510 2023-11-16T16:31:02.846 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.747+0000 7f5873fff700 1 --2- 172.21.15.1:0/3371189591 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f5874038760 0x7f587403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:02.846 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.747+0000 7f5873fff700 1 -- 172.21.15.1:0/3371189591 --> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f587403b320 con 0x7f5874038760 2023-11-16T16:31:02.846 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.748+0000 7f588a59c700 1 -- 172.21.15.1:0/3371189591 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f5874038760 msgr2=0x7f587403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/3137425292 2023-11-16T16:31:02.846 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.748+0000 7f5873fff700 1 -- 172.21.15.1:0/3371189591 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f587c04c830 con 0x7f588c106510 2023-11-16T16:31:02.846 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.748+0000 7f588a59c700 1 --2- 172.21.15.1:0/3371189591 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f5874038760 0x7f587403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-16T16:31:02.846 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.949+0000 7f588a59c700 1 -- 172.21.15.1:0/3371189591 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f5874038760 msgr2=0x7f587403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/3137425292 2023-11-16T16:31:02.846 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:55.949+0000 7f588a59c700 1 --2- 172.21.15.1:0/3371189591 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f5874038760 0x7f587403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-16T16:31:02.846 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:56.350+0000 7f588a59c700 1 -- 172.21.15.1:0/3371189591 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f5874038760 msgr2=0x7f587403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/3137425292 2023-11-16T16:31:02.847 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:56.350+0000 7f588a59c700 1 --2- 172.21.15.1:0/3371189591 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f5874038760 0x7f587403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-16T16:31:02.847 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:57.151+0000 7f588a59c700 1 -- 172.21.15.1:0/3371189591 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f5874038760 msgr2=0x7f587403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/3137425292 2023-11-16T16:31:02.847 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:57.151+0000 7f588a59c700 1 --2- 172.21.15.1:0/3371189591 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f5874038760 0x7f587403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-16T16:31:02.847 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:58.752+0000 7f588a59c700 1 -- 172.21.15.1:0/3371189591 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f5874038760 msgr2=0x7f587403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/3137425292 2023-11-16T16:31:02.847 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:30:58.752+0000 7f588a59c700 1 --2- 172.21.15.1:0/3371189591 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f5874038760 0x7f587403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-11-16T16:31:02.847 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:01.392+0000 7f5873fff700 1 -- 172.21.15.1:0/3371189591 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mgrmap(e 10) v1 ==== 45263+0+0 (secure 0 0 0) 0x7f587c02d960 con 0x7f588c106510 2023-11-16T16:31:02.847 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:01.392+0000 7f5873fff700 1 -- 172.21.15.1:0/3371189591 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f5874038760 msgr2=0x7f587403ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-16T16:31:02.847 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:01.392+0000 7f5873fff700 1 --2- 172.21.15.1:0/3371189591 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f5874038760 0x7f587403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:02.847 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:02.394+0000 7f5873fff700 1 -- 172.21.15.1:0/3371189591 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 11) v1 ==== 45390+0+0 (secure 0 0 0) 0x7f587c04e400 con 0x7f588c106510 2023-11-16T16:31:02.847 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:02.394+0000 7f5873fff700 1 --2- 172.21.15.1:0/3371189591 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f587403c220 0x7f587403e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:02.847 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:02.394+0000 7f5873fff700 1 -- 172.21.15.1:0/3371189591 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f587403b320 con 0x7f587403c220 2023-11-16T16:31:02.848 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:02.395+0000 7f588a59c700 1 --2- 172.21.15.1:0/3371189591 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f587403c220 0x7f587403e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:02.848 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:02.395+0000 7f588a59c700 1 --2- 172.21.15.1:0/3371189591 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f587403c220 0x7f587403e600 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5880003a10 tx=0x7f58800092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:02.848 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:02.396+0000 7f5873fff700 1 -- 172.21.15.1:0/3371189591 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f587403b320 con 0x7f587403c220 2023-11-16T16:31:02.848 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:02.407+0000 7f5891160700 1 -- 172.21.15.1:0/3371189591 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f588c19aad0 con 0x7f587403c220 2023-11-16T16:31:02.848 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:02.408+0000 7f5873fff700 1 -- 172.21.15.1:0/3371189591 <== mgr.14152 v2:172.21.15.1:6800/2490729488 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f588c19aad0 con 0x7f587403c220 2023-11-16T16:31:02.848 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:02.408+0000 7f5891160700 1 -- 172.21.15.1:0/3371189591 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f587403c220 msgr2=0x7f587403e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:02.848 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:02.408+0000 7f5891160700 1 --2- 172.21.15.1:0/3371189591 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f587403c220 0x7f587403e600 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5880003a10 tx=0x7f58800092b0 comp rx=0 tx=0).stop 2023-11-16T16:31:02.848 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:02.408+0000 7f5891160700 1 -- 172.21.15.1:0/3371189591 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f588c106510 msgr2=0x7f588c199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:02.848 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:02.408+0000 7f5891160700 1 --2- 172.21.15.1:0/3371189591 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f588c106510 0x7f588c199790 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f587c011980 tx=0x7f587c011a60 comp rx=0 tx=0).stop 2023-11-16T16:31:02.848 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:02.408+0000 7f5891160700 1 -- 172.21.15.1:0/3371189591 shutdown_connections 2023-11-16T16:31:02.848 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:02.408+0000 7f5891160700 1 --2- 172.21.15.1:0/3371189591 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f587403c220 0x7f587403e600 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:02.848 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:02.408+0000 7f5891160700 1 --2- 172.21.15.1:0/3371189591 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f588c106510 0x7f588c199790 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:02.849 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:02.408+0000 7f5891160700 1 --2- 172.21.15.1:0/3371189591 >> [v2:172.21.15.1:6800/3137425292,v1:172.21.15.1:6801/3137425292] conn(0x7f5874038760 0x7f587403ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:02.849 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:02.408+0000 7f5891160700 1 -- 172.21.15.1:0/3371189591 >> 172.21.15.1:0/3371189591 conn(0x7f588c0ffed0 msgr2=0x7f588c100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:31:02.849 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:02.409+0000 7f5891160700 1 -- 172.21.15.1:0/3371189591 shutdown_connections 2023-11-16T16:31:02.849 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:02.409+0000 7f5891160700 1 -- 172.21.15.1:0/3371189591 wait complete. 2023-11-16T16:31:02.849 INFO:teuthology.orchestra.run.smithi001.stdout:mgr epoch 9 is available 2023-11-16T16:31:02.849 INFO:teuthology.orchestra.run.smithi001.stdout:Generating a dashboard self-signed certificate... 2023-11-16T16:31:03.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:03 smithi001 ceph-mon[98800]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-11-16T16:31:03.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:03 smithi001 ceph-mon[98800]: mgrmap e11: x(active, since 1.00679s) 2023-11-16T16:31:03.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:03 smithi001 ceph-mon[98800]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-11-16T16:31:04.440 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-11-16T16:31:04.440 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.564+0000 7f7338262700 1 Processor -- start 2023-11-16T16:31:04.440 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.565+0000 7f7338262700 1 -- start start 2023-11-16T16:31:04.440 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.565+0000 7f7338262700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7330104920 0x7f7330104d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:04.441 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.565+0000 7f7338262700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7330105300 con 0x7f7330104920 2023-11-16T16:31:04.441 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.565+0000 7f7335ffe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7330104920 0x7f7330104d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:04.441 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.565+0000 7f7335ffe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7330104920 0x7f7330104d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40408/0 (socket says 172.21.15.1:40408) 2023-11-16T16:31:04.441 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.565+0000 7f7335ffe700 1 -- 172.21.15.1:0/4249777323 learned_addr learned my addr 172.21.15.1:0/4249777323 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:31:04.441 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.566+0000 7f7335ffe700 1 -- 172.21.15.1:0/4249777323 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7330105b20 con 0x7f7330104920 2023-11-16T16:31:04.441 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.566+0000 7f7335ffe700 1 --2- 172.21.15.1:0/4249777323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7330104920 0x7f7330104d30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f732c0098a0 tx=0x7f732c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=280487386281ac2e server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:04.441 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.566+0000 7f7334ffc700 1 -- 172.21.15.1:0/4249777323 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f732c004160 con 0x7f7330104920 2023-11-16T16:31:04.441 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.566+0000 7f7334ffc700 1 -- 172.21.15.1:0/4249777323 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f732c0042c0 con 0x7f7330104920 2023-11-16T16:31:04.441 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.566+0000 7f7334ffc700 1 -- 172.21.15.1:0/4249777323 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f732c0119b0 con 0x7f7330104920 2023-11-16T16:31:04.441 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.567+0000 7f7338262700 1 -- 172.21.15.1:0/4249777323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7330104920 msgr2=0x7f7330104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:04.441 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.567+0000 7f7338262700 1 --2- 172.21.15.1:0/4249777323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7330104920 0x7f7330104d30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f732c0098a0 tx=0x7f732c009bb0 comp rx=0 tx=0).stop 2023-11-16T16:31:04.441 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.567+0000 7f7338262700 1 -- 172.21.15.1:0/4249777323 shutdown_connections 2023-11-16T16:31:04.442 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.567+0000 7f7338262700 1 --2- 172.21.15.1:0/4249777323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7330104920 0x7f7330104d30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:04.442 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.567+0000 7f7338262700 1 -- 172.21.15.1:0/4249777323 >> 172.21.15.1:0/4249777323 conn(0x7f73300ffed0 msgr2=0x7f7330102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:31:04.442 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.567+0000 7f7338262700 1 -- 172.21.15.1:0/4249777323 shutdown_connections 2023-11-16T16:31:04.442 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.567+0000 7f7338262700 1 -- 172.21.15.1:0/4249777323 wait complete. 2023-11-16T16:31:04.442 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.568+0000 7f7338262700 1 Processor -- start 2023-11-16T16:31:04.442 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.568+0000 7f7338262700 1 -- start start 2023-11-16T16:31:04.442 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.568+0000 7f7338262700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7330104920 0x7f733018fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:04.442 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.568+0000 7f7338262700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73301900d0 con 0x7f7330104920 2023-11-16T16:31:04.442 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.568+0000 7f7335ffe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7330104920 0x7f733018fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:04.442 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.568+0000 7f7335ffe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7330104920 0x7f733018fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40410/0 (socket says 172.21.15.1:40410) 2023-11-16T16:31:04.442 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.568+0000 7f7335ffe700 1 -- 172.21.15.1:0/243360448 learned_addr learned my addr 172.21.15.1:0/243360448 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:31:04.443 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.568+0000 7f7335ffe700 1 -- 172.21.15.1:0/243360448 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f732c009550 con 0x7f7330104920 2023-11-16T16:31:04.443 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.568+0000 7f7335ffe700 1 --2- 172.21.15.1:0/243360448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7330104920 0x7f733018fb90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f732c000c00 tx=0x7f732c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:04.443 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.568+0000 7f7326ffd700 1 -- 172.21.15.1:0/243360448 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f732c0039f0 con 0x7f7330104920 2023-11-16T16:31:04.443 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.569+0000 7f7326ffd700 1 -- 172.21.15.1:0/243360448 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f732c003b50 con 0x7f7330104920 2023-11-16T16:31:04.443 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.569+0000 7f7338262700 1 -- 172.21.15.1:0/243360448 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7330108ee0 con 0x7f7330104920 2023-11-16T16:31:04.443 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.569+0000 7f7326ffd700 1 -- 172.21.15.1:0/243360448 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f732c01a780 con 0x7f7330104920 2023-11-16T16:31:04.443 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.569+0000 7f7338262700 1 -- 172.21.15.1:0/243360448 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7330108d50 con 0x7f7330104920 2023-11-16T16:31:04.443 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.570+0000 7f7326ffd700 1 -- 172.21.15.1:0/243360448 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 11) v1 ==== 45390+0+0 (secure 0 0 0) 0x7f732c024440 con 0x7f7330104920 2023-11-16T16:31:04.443 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.570+0000 7f7338262700 1 -- 172.21.15.1:0/243360448 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7330062190 con 0x7f7330104920 2023-11-16T16:31:04.443 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.570+0000 7f7326ffd700 1 --2- 172.21.15.1:0/243360448 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f731c038630 0x7f731c03aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:04.443 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.570+0000 7f7326ffd700 1 -- 172.21.15.1:0/243360448 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f732c04c6e0 con 0x7f7330104920 2023-11-16T16:31:04.444 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.570+0000 7f73357fd700 1 --2- 172.21.15.1:0/243360448 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f731c038630 0x7f731c03aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:04.444 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.571+0000 7f73357fd700 1 --2- 172.21.15.1:0/243360448 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f731c038630 0x7f731c03aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7320006fd0 tx=0x7f7320006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:04.444 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.575+0000 7f7326ffd700 1 -- 172.21.15.1:0/243360448 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7330062190 con 0x7f7330104920 2023-11-16T16:31:04.444 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:03.948+0000 7f7338262700 1 -- 172.21.15.1:0/243360448 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f7330062190 con 0x7f731c038630 2023-11-16T16:31:04.444 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:04.038+0000 7f7326ffd700 1 -- 172.21.15.1:0/243360448 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f7330062190 con 0x7f731c038630 2023-11-16T16:31:04.444 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:04.040+0000 7f7338262700 1 -- 172.21.15.1:0/243360448 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f731c038630 msgr2=0x7f731c03aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:04.444 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:04.040+0000 7f7338262700 1 --2- 172.21.15.1:0/243360448 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f731c038630 0x7f731c03aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7320006fd0 tx=0x7f7320006e40 comp rx=0 tx=0).stop 2023-11-16T16:31:04.444 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:04.040+0000 7f7338262700 1 -- 172.21.15.1:0/243360448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7330104920 msgr2=0x7f733018fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:04.444 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:04.040+0000 7f7338262700 1 --2- 172.21.15.1:0/243360448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7330104920 0x7f733018fb90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f732c000c00 tx=0x7f732c011a30 comp rx=0 tx=0).stop 2023-11-16T16:31:04.444 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:04.040+0000 7f7338262700 1 -- 172.21.15.1:0/243360448 shutdown_connections 2023-11-16T16:31:04.444 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:04.040+0000 7f7338262700 1 --2- 172.21.15.1:0/243360448 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f731c038630 0x7f731c03aae0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:04.444 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:04.040+0000 7f7338262700 1 --2- 172.21.15.1:0/243360448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7330104920 0x7f733018fb90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:04.445 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:04.040+0000 7f7338262700 1 -- 172.21.15.1:0/243360448 >> 172.21.15.1:0/243360448 conn(0x7f73300ffed0 msgr2=0x7f7330101580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:31:04.445 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:04.041+0000 7f7338262700 1 -- 172.21.15.1:0/243360448 shutdown_connections 2023-11-16T16:31:04.445 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:04.041+0000 7f7338262700 1 -- 172.21.15.1:0/243360448 wait complete. 2023-11-16T16:31:04.445 INFO:teuthology.orchestra.run.smithi001.stdout:Creating initial admin user... 2023-11-16T16:31:04.672 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:04 smithi001 ceph-mon[98800]: [16/Nov/2023:16:31:02] ENGINE Bus STARTING 2023-11-16T16:31:04.672 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:04 smithi001 ceph-mon[98800]: [16/Nov/2023:16:31:02] ENGINE Serving on http://172.21.15.1:8765 2023-11-16T16:31:04.673 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:04 smithi001 ceph-mon[98800]: [16/Nov/2023:16:31:02] ENGINE Serving on https://172.21.15.1:7150 2023-11-16T16:31:04.673 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:04 smithi001 ceph-mon[98800]: [16/Nov/2023:16:31:02] ENGINE Bus STARTED 2023-11-16T16:31:04.673 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:04 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:04.673 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:04 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:04.673 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:04 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:05.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:05 smithi001 ceph-mon[98800]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:31:05.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:05 smithi001 ceph-mon[98800]: mgrmap e12: x(active, since 3s) 2023-11-16T16:31:05.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:05 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:06.388 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$jLDVY4EN0vs9G7giJ0UQNexAsrhtniE4ujbgGdDz7nNyxQ5qLBbwO", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1700152265, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-11-16T16:31:06.389 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.010+0000 7f51b5c7e700 1 Processor -- start 2023-11-16T16:31:06.389 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.010+0000 7f51b5c7e700 1 -- start start 2023-11-16T16:31:06.389 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.011+0000 7f51b5c7e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b01064d0 0x7f51b01088b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:06.389 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.011+0000 7f51b5c7e700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b0079e20 con 0x7f51b01064d0 2023-11-16T16:31:06.389 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.011+0000 7f51af7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b01064d0 0x7f51b01088b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:06.389 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.011+0000 7f51af7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b01064d0 0x7f51b01088b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40414/0 (socket says 172.21.15.1:40414) 2023-11-16T16:31:06.389 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.011+0000 7f51af7fe700 1 -- 172.21.15.1:0/437469090 learned_addr learned my addr 172.21.15.1:0/437469090 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:31:06.389 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.011+0000 7f51af7fe700 1 -- 172.21.15.1:0/437469090 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b0108df0 con 0x7f51b01064d0 2023-11-16T16:31:06.389 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.011+0000 7f51af7fe700 1 --2- 172.21.15.1:0/437469090 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b01064d0 0x7f51b01088b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f51a00098a0 tx=0x7f51a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80b06690f77be6d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:06.389 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.012+0000 7f51ae7fc700 1 -- 172.21.15.1:0/437469090 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f51a0004160 con 0x7f51b01064d0 2023-11-16T16:31:06.390 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.012+0000 7f51ae7fc700 1 -- 172.21.15.1:0/437469090 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51a00042c0 con 0x7f51b01064d0 2023-11-16T16:31:06.390 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.012+0000 7f51ae7fc700 1 -- 172.21.15.1:0/437469090 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f51a00119b0 con 0x7f51b01064d0 2023-11-16T16:31:06.390 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.012+0000 7f51b5c7e700 1 -- 172.21.15.1:0/437469090 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b01064d0 msgr2=0x7f51b01088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:06.390 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.012+0000 7f51b5c7e700 1 --2- 172.21.15.1:0/437469090 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b01064d0 0x7f51b01088b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f51a00098a0 tx=0x7f51a0009bb0 comp rx=0 tx=0).stop 2023-11-16T16:31:06.390 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.012+0000 7f51b5c7e700 1 -- 172.21.15.1:0/437469090 shutdown_connections 2023-11-16T16:31:06.390 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.012+0000 7f51b5c7e700 1 --2- 172.21.15.1:0/437469090 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b01064d0 0x7f51b01088b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:06.390 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.012+0000 7f51b5c7e700 1 -- 172.21.15.1:0/437469090 >> 172.21.15.1:0/437469090 conn(0x7f51b00ffed0 msgr2=0x7f51b0102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:31:06.390 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.012+0000 7f51b5c7e700 1 -- 172.21.15.1:0/437469090 shutdown_connections 2023-11-16T16:31:06.390 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.012+0000 7f51b5c7e700 1 -- 172.21.15.1:0/437469090 wait complete. 2023-11-16T16:31:06.390 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.013+0000 7f51b5c7e700 1 Processor -- start 2023-11-16T16:31:06.390 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.013+0000 7f51b5c7e700 1 -- start start 2023-11-16T16:31:06.391 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.013+0000 7f51b5c7e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b01064d0 0x7f51b01997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:06.391 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.013+0000 7f51b5c7e700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b0199ce0 con 0x7f51b01064d0 2023-11-16T16:31:06.391 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.014+0000 7f51af7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b01064d0 0x7f51b01997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:06.391 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.014+0000 7f51af7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b01064d0 0x7f51b01997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40416/0 (socket says 172.21.15.1:40416) 2023-11-16T16:31:06.391 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.014+0000 7f51af7fe700 1 -- 172.21.15.1:0/4127015563 learned_addr learned my addr 172.21.15.1:0/4127015563 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:31:06.391 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.014+0000 7f51af7fe700 1 -- 172.21.15.1:0/4127015563 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a0009550 con 0x7f51b01064d0 2023-11-16T16:31:06.391 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.014+0000 7f51af7fe700 1 --2- 172.21.15.1:0/4127015563 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b01064d0 0x7f51b01997a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f51a0003e60 tx=0x7f51a0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:06.391 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.014+0000 7f51acff9700 1 -- 172.21.15.1:0/4127015563 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f51a0003970 con 0x7f51b01064d0 2023-11-16T16:31:06.391 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.014+0000 7f51acff9700 1 -- 172.21.15.1:0/4127015563 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51a0003ad0 con 0x7f51b01064d0 2023-11-16T16:31:06.391 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.014+0000 7f51b5c7e700 1 -- 172.21.15.1:0/4127015563 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b0109d60 con 0x7f51b01064d0 2023-11-16T16:31:06.391 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.014+0000 7f51acff9700 1 -- 172.21.15.1:0/4127015563 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f51a00195e0 con 0x7f51b01064d0 2023-11-16T16:31:06.392 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.014+0000 7f51b5c7e700 1 -- 172.21.15.1:0/4127015563 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b0109bd0 con 0x7f51b01064d0 2023-11-16T16:31:06.392 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.015+0000 7f51acff9700 1 -- 172.21.15.1:0/4127015563 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f51a00198e0 con 0x7f51b01064d0 2023-11-16T16:31:06.392 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.015+0000 7f51acff9700 1 --2- 172.21.15.1:0/4127015563 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f519c038710 0x7f519c03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:06.392 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.015+0000 7f51acff9700 1 -- 172.21.15.1:0/4127015563 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f51a004cc50 con 0x7f51b01064d0 2023-11-16T16:31:06.392 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.015+0000 7f51aeffd700 1 --2- 172.21.15.1:0/4127015563 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f519c038710 0x7f519c03abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:06.392 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.015+0000 7f51b5c7e700 1 -- 172.21.15.1:0/4127015563 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51b0109ef0 con 0x7f51b01064d0 2023-11-16T16:31:06.392 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.016+0000 7f51aeffd700 1 --2- 172.21.15.1:0/4127015563 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f519c038710 0x7f519c03abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5198006fd0 tx=0x7f5198006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:06.392 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.020+0000 7f51acff9700 1 -- 172.21.15.1:0/4127015563 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f51a0019c30 con 0x7f51b01064d0 2023-11-16T16:31:06.392 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.416+0000 7f51b5c7e700 1 -- 172.21.15.1:0/4127015563 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f51b019a4a0 con 0x7f519c038710 2023-11-16T16:31:06.392 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.664+0000 7f51acff9700 1 -- 172.21.15.1:0/4127015563 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f51b019a4a0 con 0x7f519c038710 2023-11-16T16:31:06.392 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.666+0000 7f51b5c7e700 1 -- 172.21.15.1:0/4127015563 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f519c038710 msgr2=0x7f519c03abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:06.392 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.666+0000 7f51b5c7e700 1 --2- 172.21.15.1:0/4127015563 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f519c038710 0x7f519c03abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5198006fd0 tx=0x7f5198006e40 comp rx=0 tx=0).stop 2023-11-16T16:31:06.393 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.666+0000 7f51b5c7e700 1 -- 172.21.15.1:0/4127015563 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b01064d0 msgr2=0x7f51b01997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:06.393 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.666+0000 7f51b5c7e700 1 --2- 172.21.15.1:0/4127015563 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b01064d0 0x7f51b01997a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f51a0003e60 tx=0x7f51a0011aa0 comp rx=0 tx=0).stop 2023-11-16T16:31:06.393 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.666+0000 7f51b5c7e700 1 -- 172.21.15.1:0/4127015563 shutdown_connections 2023-11-16T16:31:06.393 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.666+0000 7f51b5c7e700 1 --2- 172.21.15.1:0/4127015563 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f519c038710 0x7f519c03abc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:06.393 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.666+0000 7f51b5c7e700 1 --2- 172.21.15.1:0/4127015563 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b01064d0 0x7f51b01997a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:06.393 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.666+0000 7f51b5c7e700 1 -- 172.21.15.1:0/4127015563 >> 172.21.15.1:0/4127015563 conn(0x7f51b00ffed0 msgr2=0x7f51b01015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:31:06.393 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.666+0000 7f51b5c7e700 1 -- 172.21.15.1:0/4127015563 shutdown_connections 2023-11-16T16:31:06.393 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:05.666+0000 7f51b5c7e700 1 -- 172.21.15.1:0/4127015563 wait complete. 2023-11-16T16:31:06.393 INFO:teuthology.orchestra.run.smithi001.stdout:Fetching dashboard port number... 2023-11-16T16:31:06.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:06 smithi001 ceph-mon[98800]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:31:06.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:06 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:07.629 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 8443 2023-11-16T16:31:07.634 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.836+0000 7fb00d9a2700 1 Processor -- start 2023-11-16T16:31:07.634 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.836+0000 7fb00d9a2700 1 -- start start 2023-11-16T16:31:07.634 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.836+0000 7fb00d9a2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb0081064f0 0x7fb0081088d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:07.634 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.836+0000 7fb00d9a2700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb008079e00 con 0x7fb0081064f0 2023-11-16T16:31:07.634 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.836+0000 7fb006ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb0081064f0 0x7fb0081088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:07.634 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.836+0000 7fb006ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb0081064f0 0x7fb0081088d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40424/0 (socket says 172.21.15.1:40424) 2023-11-16T16:31:07.634 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.836+0000 7fb006ffd700 1 -- 172.21.15.1:0/4056742461 learned_addr learned my addr 172.21.15.1:0/4056742461 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:31:07.634 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.837+0000 7fb006ffd700 1 -- 172.21.15.1:0/4056742461 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb008108e10 con 0x7fb0081064f0 2023-11-16T16:31:07.634 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.837+0000 7fb006ffd700 1 --2- 172.21.15.1:0/4056742461 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb0081064f0 0x7fb0081088d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faff80098a0 tx=0x7faff8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c296bb2763eae3 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:07.634 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.837+0000 7fb0067fc700 1 -- 172.21.15.1:0/4056742461 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faff8004160 con 0x7fb0081064f0 2023-11-16T16:31:07.635 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.837+0000 7fb0067fc700 1 -- 172.21.15.1:0/4056742461 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faff80042c0 con 0x7fb0081064f0 2023-11-16T16:31:07.635 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.837+0000 7fb0067fc700 1 -- 172.21.15.1:0/4056742461 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faff80119b0 con 0x7fb0081064f0 2023-11-16T16:31:07.635 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.838+0000 7fb00d9a2700 1 -- 172.21.15.1:0/4056742461 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb0081064f0 msgr2=0x7fb0081088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:07.635 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.838+0000 7fb00d9a2700 1 --2- 172.21.15.1:0/4056742461 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb0081064f0 0x7fb0081088d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faff80098a0 tx=0x7faff8009bb0 comp rx=0 tx=0).stop 2023-11-16T16:31:07.635 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.838+0000 7fb00d9a2700 1 -- 172.21.15.1:0/4056742461 shutdown_connections 2023-11-16T16:31:07.635 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.838+0000 7fb00d9a2700 1 --2- 172.21.15.1:0/4056742461 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb0081064f0 0x7fb0081088d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:07.635 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.838+0000 7fb00d9a2700 1 -- 172.21.15.1:0/4056742461 >> 172.21.15.1:0/4056742461 conn(0x7fb0080ffed0 msgr2=0x7fb008102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:31:07.635 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.838+0000 7fb00d9a2700 1 -- 172.21.15.1:0/4056742461 shutdown_connections 2023-11-16T16:31:07.635 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.838+0000 7fb00d9a2700 1 -- 172.21.15.1:0/4056742461 wait complete. 2023-11-16T16:31:07.635 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.838+0000 7fb00d9a2700 1 Processor -- start 2023-11-16T16:31:07.635 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.838+0000 7fb00d9a2700 1 -- start start 2023-11-16T16:31:07.635 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.838+0000 7fb00d9a2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb0081064f0 0x7fb008105e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:07.636 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.838+0000 7fb00d9a2700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb008106390 con 0x7fb0081064f0 2023-11-16T16:31:07.636 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.839+0000 7fb006ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb0081064f0 0x7fb008105e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:07.636 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.839+0000 7fb006ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb0081064f0 0x7fb008105e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40426/0 (socket says 172.21.15.1:40426) 2023-11-16T16:31:07.636 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.839+0000 7fb006ffd700 1 -- 172.21.15.1:0/236747140 learned_addr learned my addr 172.21.15.1:0/236747140 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:31:07.636 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.839+0000 7fb006ffd700 1 -- 172.21.15.1:0/236747140 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff8009550 con 0x7fb0081064f0 2023-11-16T16:31:07.636 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.839+0000 7fb006ffd700 1 --2- 172.21.15.1:0/236747140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb0081064f0 0x7fb008105e50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faff8011980 tx=0x7faff8011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:07.636 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.839+0000 7fb00c9a0700 1 -- 172.21.15.1:0/236747140 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faff8003a60 con 0x7fb0081064f0 2023-11-16T16:31:07.636 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.839+0000 7fb00c9a0700 1 -- 172.21.15.1:0/236747140 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faff8003bc0 con 0x7fb0081064f0 2023-11-16T16:31:07.636 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.839+0000 7fb00d9a2700 1 -- 172.21.15.1:0/236747140 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb008109d80 con 0x7fb0081064f0 2023-11-16T16:31:07.636 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.839+0000 7fb00c9a0700 1 -- 172.21.15.1:0/236747140 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faff801a670 con 0x7fb0081064f0 2023-11-16T16:31:07.636 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.839+0000 7fb00d9a2700 1 -- 172.21.15.1:0/236747140 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb008109bf0 con 0x7fb0081064f0 2023-11-16T16:31:07.636 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.840+0000 7fb00c9a0700 1 -- 172.21.15.1:0/236747140 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7faff801a930 con 0x7fb0081064f0 2023-11-16T16:31:07.637 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.840+0000 7fb00c9a0700 1 --2- 172.21.15.1:0/236747140 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7faff4038760 0x7faff403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:07.637 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.840+0000 7fb00c9a0700 1 -- 172.21.15.1:0/236747140 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7faff804cb00 con 0x7fb0081064f0 2023-11-16T16:31:07.637 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.840+0000 7fb00d9a2700 1 -- 172.21.15.1:0/236747140 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb008062120 con 0x7fb0081064f0 2023-11-16T16:31:07.637 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.841+0000 7faffffff700 1 --2- 172.21.15.1:0/236747140 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7faff4038760 0x7faff403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:07.637 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.841+0000 7faffffff700 1 --2- 172.21.15.1:0/236747140 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7faff4038760 0x7faff403ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7faff0006fd0 tx=0x7faff0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:07.637 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:06.845+0000 7fb00c9a0700 1 -- 172.21.15.1:0/236747140 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7faff8018db0 con 0x7fb0081064f0 2023-11-16T16:31:07.637 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:07.204+0000 7fb00d9a2700 1 -- 172.21.15.1:0/236747140 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fb0081052b0 con 0x7fb0081064f0 2023-11-16T16:31:07.637 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:07.205+0000 7fb00c9a0700 1 -- 172.21.15.1:0/236747140 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7faff8018350 con 0x7fb0081064f0 2023-11-16T16:31:07.637 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:07.207+0000 7fb00d9a2700 1 -- 172.21.15.1:0/236747140 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7faff4038760 msgr2=0x7faff403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:07.637 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:07.207+0000 7fb00d9a2700 1 --2- 172.21.15.1:0/236747140 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7faff4038760 0x7faff403ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7faff0006fd0 tx=0x7faff0006e40 comp rx=0 tx=0).stop 2023-11-16T16:31:07.637 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:07.207+0000 7fb00d9a2700 1 -- 172.21.15.1:0/236747140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb0081064f0 msgr2=0x7fb008105e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:07.637 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:07.207+0000 7fb00d9a2700 1 --2- 172.21.15.1:0/236747140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb0081064f0 0x7fb008105e50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faff8011980 tx=0x7faff8011a60 comp rx=0 tx=0).stop 2023-11-16T16:31:07.637 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:07.207+0000 7fb00d9a2700 1 -- 172.21.15.1:0/236747140 shutdown_connections 2023-11-16T16:31:07.638 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:07.207+0000 7fb00d9a2700 1 --2- 172.21.15.1:0/236747140 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7faff4038760 0x7faff403ac10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:07.638 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:07.207+0000 7fb00d9a2700 1 --2- 172.21.15.1:0/236747140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb0081064f0 0x7fb008105e50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:07.638 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:07.207+0000 7fb00d9a2700 1 -- 172.21.15.1:0/236747140 >> 172.21.15.1:0/236747140 conn(0x7fb0080ffed0 msgr2=0x7fb0081015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:31:07.638 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:07.207+0000 7fb00d9a2700 1 -- 172.21.15.1:0/236747140 shutdown_connections 2023-11-16T16:31:07.638 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:07.207+0000 7fb00d9a2700 1 -- 172.21.15.1:0/236747140 wait complete. 2023-11-16T16:31:07.643 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-11-16T16:31:07.643 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout disabled 2023-11-16T16:31:07.650 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-11-16T16:31:07.650 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2023-11-16T16:31:07.650 INFO:teuthology.orchestra.run.smithi001.stdout:firewalld.service is not enabled 2023-11-16T16:31:07.650 INFO:teuthology.orchestra.run.smithi001.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-11-16T16:31:07.652 INFO:teuthology.orchestra.run.smithi001.stdout:Ceph Dashboard is now available at: 2023-11-16T16:31:07.652 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:31:07.652 INFO:teuthology.orchestra.run.smithi001.stdout: URL: https://smithi001:8443/ 2023-11-16T16:31:07.652 INFO:teuthology.orchestra.run.smithi001.stdout: User: admin 2023-11-16T16:31:07.652 INFO:teuthology.orchestra.run.smithi001.stdout: Password: 2ry7666vzc 2023-11-16T16:31:07.652 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:31:07.653 INFO:teuthology.orchestra.run.smithi001.stdout:Saving cluster configuration to /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config directory 2023-11-16T16:31:07.653 INFO:teuthology.orchestra.run.smithi001.stdout:Enabling autotune for osd_memory_target 2023-11-16T16:31:07.871 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:07 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/236747140' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-11-16T16:31:08.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:08 smithi001 ceph-mon[98800]: mgrmap e13: x(active, since 6s) 2023-11-16T16:31:09.001 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.159+0000 7fbfcaa44700 1 Processor -- start 2023-11-16T16:31:09.001 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.159+0000 7fbfcaa44700 1 -- start start 2023-11-16T16:31:09.001 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.159+0000 7fbfcaa44700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfc4106490 0x7fbfc4108870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:09.001 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.159+0000 7fbfcaa44700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc4079e60 con 0x7fbfc4106490 2023-11-16T16:31:09.001 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.159+0000 7fbfc3fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfc4106490 0x7fbfc4108870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:09.001 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.159+0000 7fbfc3fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfc4106490 0x7fbfc4108870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40430/0 (socket says 172.21.15.1:40430) 2023-11-16T16:31:09.001 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.159+0000 7fbfc3fff700 1 -- 172.21.15.1:0/1514652975 learned_addr learned my addr 172.21.15.1:0/1514652975 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:31:09.001 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.159+0000 7fbfc3fff700 1 -- 172.21.15.1:0/1514652975 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc4108db0 con 0x7fbfc4106490 2023-11-16T16:31:09.001 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.160+0000 7fbfc3fff700 1 --2- 172.21.15.1:0/1514652975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfc4106490 0x7fbfc4108870 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbfb40098a0 tx=0x7fbfb4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ebcae55efd2383 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:09.002 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.160+0000 7fbfc2ffd700 1 -- 172.21.15.1:0/1514652975 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbfb4004160 con 0x7fbfc4106490 2023-11-16T16:31:09.002 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.160+0000 7fbfc2ffd700 1 -- 172.21.15.1:0/1514652975 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfb40042c0 con 0x7fbfc4106490 2023-11-16T16:31:09.002 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.160+0000 7fbfc2ffd700 1 -- 172.21.15.1:0/1514652975 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbfb40119b0 con 0x7fbfc4106490 2023-11-16T16:31:09.002 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.160+0000 7fbfcaa44700 1 -- 172.21.15.1:0/1514652975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfc4106490 msgr2=0x7fbfc4108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:09.002 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.160+0000 7fbfcaa44700 1 --2- 172.21.15.1:0/1514652975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfc4106490 0x7fbfc4108870 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbfb40098a0 tx=0x7fbfb4009bb0 comp rx=0 tx=0).stop 2023-11-16T16:31:09.002 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.161+0000 7fbfcaa44700 1 -- 172.21.15.1:0/1514652975 shutdown_connections 2023-11-16T16:31:09.002 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.161+0000 7fbfcaa44700 1 --2- 172.21.15.1:0/1514652975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfc4106490 0x7fbfc4108870 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:09.002 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.161+0000 7fbfcaa44700 1 -- 172.21.15.1:0/1514652975 >> 172.21.15.1:0/1514652975 conn(0x7fbfc40ffed0 msgr2=0x7fbfc4102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:31:09.002 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.161+0000 7fbfcaa44700 1 -- 172.21.15.1:0/1514652975 shutdown_connections 2023-11-16T16:31:09.002 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.161+0000 7fbfcaa44700 1 -- 172.21.15.1:0/1514652975 wait complete. 2023-11-16T16:31:09.002 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.161+0000 7fbfcaa44700 1 Processor -- start 2023-11-16T16:31:09.003 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.161+0000 7fbfcaa44700 1 -- start start 2023-11-16T16:31:09.003 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.161+0000 7fbfcaa44700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfc4106490 0x7fbfc4199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:09.003 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.162+0000 7fbfcaa44700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc4199cc0 con 0x7fbfc4106490 2023-11-16T16:31:09.003 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.162+0000 7fbfc3fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfc4106490 0x7fbfc4199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:09.003 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.162+0000 7fbfc3fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfc4106490 0x7fbfc4199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40432/0 (socket says 172.21.15.1:40432) 2023-11-16T16:31:09.003 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.162+0000 7fbfc3fff700 1 -- 172.21.15.1:0/3077205095 learned_addr learned my addr 172.21.15.1:0/3077205095 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:31:09.003 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.162+0000 7fbfc3fff700 1 -- 172.21.15.1:0/3077205095 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb4009550 con 0x7fbfc4106490 2023-11-16T16:31:09.003 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.162+0000 7fbfc3fff700 1 --2- 172.21.15.1:0/3077205095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfc4106490 0x7fbfc4199780 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4011980 tx=0x7fbfb4011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:09.003 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.162+0000 7fbfc17fa700 1 -- 172.21.15.1:0/3077205095 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbfb4003a60 con 0x7fbfc4106490 2023-11-16T16:31:09.003 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.162+0000 7fbfc17fa700 1 -- 172.21.15.1:0/3077205095 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfb4003bc0 con 0x7fbfc4106490 2023-11-16T16:31:09.003 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.162+0000 7fbfcaa44700 1 -- 172.21.15.1:0/3077205095 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc4109d80 con 0x7fbfc4106490 2023-11-16T16:31:09.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.162+0000 7fbfc17fa700 1 -- 172.21.15.1:0/3077205095 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbfb401a670 con 0x7fbfc4106490 2023-11-16T16:31:09.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.162+0000 7fbfcaa44700 1 -- 172.21.15.1:0/3077205095 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc4109bf0 con 0x7fbfc4106490 2023-11-16T16:31:09.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.163+0000 7fbfc17fa700 1 -- 172.21.15.1:0/3077205095 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7fbfb401a930 con 0x7fbfc4106490 2023-11-16T16:31:09.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.163+0000 7fbfc17fa700 1 --2- 172.21.15.1:0/3077205095 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbfac038800 0x7fbfac03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:09.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.163+0000 7fbfc17fa700 1 -- 172.21.15.1:0/3077205095 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbfb404ce70 con 0x7fbfc4106490 2023-11-16T16:31:09.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.164+0000 7fbfc37fe700 1 --2- 172.21.15.1:0/3077205095 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbfac038800 0x7fbfac03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:09.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.164+0000 7fbfcaa44700 1 -- 172.21.15.1:0/3077205095 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfc4109f10 con 0x7fbfc4106490 2023-11-16T16:31:09.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.164+0000 7fbfc37fe700 1 --2- 172.21.15.1:0/3077205095 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbfac038800 0x7fbfac03acb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8006fd0 tx=0x7fbfb8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:09.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.168+0000 7fbfc17fa700 1 -- 172.21.15.1:0/3077205095 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbfb401ac30 con 0x7fbfc4106490 2023-11-16T16:31:09.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.525+0000 7fbfcaa44700 1 -- 172.21.15.1:0/3077205095 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fbfc4062120 con 0x7fbfc4106490 2023-11-16T16:31:09.004 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.525+0000 7fbfc17fa700 1 -- 172.21.15.1:0/3077205095 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fbfb4018b00 con 0x7fbfc4106490 2023-11-16T16:31:09.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.527+0000 7fbfcaa44700 1 -- 172.21.15.1:0/3077205095 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbfac038800 msgr2=0x7fbfac03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:09.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.527+0000 7fbfcaa44700 1 --2- 172.21.15.1:0/3077205095 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbfac038800 0x7fbfac03acb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8006fd0 tx=0x7fbfb8006e40 comp rx=0 tx=0).stop 2023-11-16T16:31:09.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.528+0000 7fbfcaa44700 1 -- 172.21.15.1:0/3077205095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfc4106490 msgr2=0x7fbfc4199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:09.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.528+0000 7fbfcaa44700 1 --2- 172.21.15.1:0/3077205095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfc4106490 0x7fbfc4199780 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4011980 tx=0x7fbfb4011a60 comp rx=0 tx=0).stop 2023-11-16T16:31:09.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.528+0000 7fbfcaa44700 1 -- 172.21.15.1:0/3077205095 shutdown_connections 2023-11-16T16:31:09.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.528+0000 7fbfcaa44700 1 --2- 172.21.15.1:0/3077205095 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbfac038800 0x7fbfac03acb0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:09.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.528+0000 7fbfcaa44700 1 --2- 172.21.15.1:0/3077205095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfc4106490 0x7fbfc4199780 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:09.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.528+0000 7fbfcaa44700 1 -- 172.21.15.1:0/3077205095 >> 172.21.15.1:0/3077205095 conn(0x7fbfc40ffed0 msgr2=0x7fbfc4100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:31:09.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.528+0000 7fbfcaa44700 1 -- 172.21.15.1:0/3077205095 shutdown_connections 2023-11-16T16:31:09.005 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:08.528+0000 7fbfcaa44700 1 -- 172.21.15.1:0/3077205095 wait complete. 2023-11-16T16:31:10.548 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.500+0000 7f94d04af700 1 Processor -- start 2023-11-16T16:31:10.548 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.500+0000 7f94d04af700 1 -- start start 2023-11-16T16:31:10.549 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.500+0000 7f94d04af700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94c8106490 0x7f94c8108870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:10.549 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.500+0000 7f94d04af700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c8079e60 con 0x7f94c8106490 2023-11-16T16:31:10.549 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.500+0000 7f94ce24b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94c8106490 0x7f94c8108870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:10.549 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.500+0000 7f94ce24b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94c8106490 0x7f94c8108870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40436/0 (socket says 172.21.15.1:40436) 2023-11-16T16:31:10.549 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.500+0000 7f94ce24b700 1 -- 172.21.15.1:0/857886476 learned_addr learned my addr 172.21.15.1:0/857886476 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:31:10.549 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.500+0000 7f94ce24b700 1 -- 172.21.15.1:0/857886476 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c8108db0 con 0x7f94c8106490 2023-11-16T16:31:10.549 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.501+0000 7f94ce24b700 1 --2- 172.21.15.1:0/857886476 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94c8106490 0x7f94c8108870 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f94c40098a0 tx=0x7f94c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15fff442d5fb3c96 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:10.549 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.501+0000 7f94cd249700 1 -- 172.21.15.1:0/857886476 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94c4004160 con 0x7f94c8106490 2023-11-16T16:31:10.549 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.501+0000 7f94cd249700 1 -- 172.21.15.1:0/857886476 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94c40042c0 con 0x7f94c8106490 2023-11-16T16:31:10.550 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.501+0000 7f94cd249700 1 -- 172.21.15.1:0/857886476 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94c40119b0 con 0x7f94c8106490 2023-11-16T16:31:10.550 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.501+0000 7f94d04af700 1 -- 172.21.15.1:0/857886476 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94c8106490 msgr2=0x7f94c8108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:10.550 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.501+0000 7f94d04af700 1 --2- 172.21.15.1:0/857886476 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94c8106490 0x7f94c8108870 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f94c40098a0 tx=0x7f94c4009bb0 comp rx=0 tx=0).stop 2023-11-16T16:31:10.550 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.502+0000 7f94d04af700 1 -- 172.21.15.1:0/857886476 shutdown_connections 2023-11-16T16:31:10.550 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.502+0000 7f94d04af700 1 --2- 172.21.15.1:0/857886476 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94c8106490 0x7f94c8108870 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:10.550 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.502+0000 7f94d04af700 1 -- 172.21.15.1:0/857886476 >> 172.21.15.1:0/857886476 conn(0x7f94c80ffed0 msgr2=0x7f94c8102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:31:10.550 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.502+0000 7f94d04af700 1 -- 172.21.15.1:0/857886476 shutdown_connections 2023-11-16T16:31:10.550 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.502+0000 7f94d04af700 1 -- 172.21.15.1:0/857886476 wait complete. 2023-11-16T16:31:10.550 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.502+0000 7f94d04af700 1 Processor -- start 2023-11-16T16:31:10.551 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.502+0000 7f94d04af700 1 -- start start 2023-11-16T16:31:10.551 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.502+0000 7f94d04af700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94c8106490 0x7f94c8105df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:10.551 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.502+0000 7f94d04af700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c8106330 con 0x7f94c8106490 2023-11-16T16:31:10.551 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.502+0000 7f94ce24b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94c8106490 0x7f94c8105df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:10.551 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.502+0000 7f94ce24b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94c8106490 0x7f94c8105df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40438/0 (socket says 172.21.15.1:40438) 2023-11-16T16:31:10.551 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.503+0000 7f94ce24b700 1 -- 172.21.15.1:0/3230049143 learned_addr learned my addr 172.21.15.1:0/3230049143 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:31:10.551 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.503+0000 7f94ce24b700 1 -- 172.21.15.1:0/3230049143 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c4009550 con 0x7f94c8106490 2023-11-16T16:31:10.551 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.503+0000 7f94ce24b700 1 --2- 172.21.15.1:0/3230049143 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94c8106490 0x7f94c8105df0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f94c4011980 tx=0x7f94c4011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:10.551 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.503+0000 7f94bb7fe700 1 -- 172.21.15.1:0/3230049143 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94c4003a60 con 0x7f94c8106490 2023-11-16T16:31:10.551 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.503+0000 7f94bb7fe700 1 -- 172.21.15.1:0/3230049143 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94c4003bc0 con 0x7f94c8106490 2023-11-16T16:31:10.552 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.503+0000 7f94bb7fe700 1 -- 172.21.15.1:0/3230049143 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94c401a670 con 0x7f94c8106490 2023-11-16T16:31:10.552 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.503+0000 7f94d04af700 1 -- 172.21.15.1:0/3230049143 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c8109d80 con 0x7f94c8106490 2023-11-16T16:31:10.552 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.503+0000 7f94d04af700 1 -- 172.21.15.1:0/3230049143 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c8109bf0 con 0x7f94c8106490 2023-11-16T16:31:10.552 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.504+0000 7f94bb7fe700 1 -- 172.21.15.1:0/3230049143 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f94c4024a00 con 0x7f94c8106490 2023-11-16T16:31:10.552 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.504+0000 7f94d04af700 1 -- 172.21.15.1:0/3230049143 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94c8062120 con 0x7f94c8106490 2023-11-16T16:31:10.552 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.504+0000 7f94bb7fe700 1 --2- 172.21.15.1:0/3230049143 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f94b4038850 0x7f94b403ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:10.552 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.504+0000 7f94bb7fe700 1 -- 172.21.15.1:0/3230049143 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f94c404ca70 con 0x7f94c8106490 2023-11-16T16:31:10.552 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.504+0000 7f94cda4a700 1 --2- 172.21.15.1:0/3230049143 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f94b4038850 0x7f94b403ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:10.552 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.505+0000 7f94cda4a700 1 --2- 172.21.15.1:0/3230049143 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f94b4038850 0x7f94b403ad00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f94bc006fd0 tx=0x7f94bc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:10.553 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:09.509+0000 7f94bb7fe700 1 -- 172.21.15.1:0/3230049143 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f94c4024440 con 0x7f94c8106490 2023-11-16T16:31:10.553 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:10.079+0000 7f94d04af700 1 -- 172.21.15.1:0/3230049143 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f94c8104ba0 con 0x7f94c8106490 2023-11-16T16:31:10.553 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:10.082+0000 7f94bb7fe700 1 -- 172.21.15.1:0/3230049143 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f94c4018350 con 0x7f94c8106490 2023-11-16T16:31:10.553 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:10.084+0000 7f94d04af700 1 -- 172.21.15.1:0/3230049143 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f94b4038850 msgr2=0x7f94b403ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:10.553 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:10.084+0000 7f94d04af700 1 --2- 172.21.15.1:0/3230049143 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f94b4038850 0x7f94b403ad00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f94bc006fd0 tx=0x7f94bc006e40 comp rx=0 tx=0).stop 2023-11-16T16:31:10.553 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:10.084+0000 7f94d04af700 1 -- 172.21.15.1:0/3230049143 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94c8106490 msgr2=0x7f94c8105df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:10.553 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:10.084+0000 7f94d04af700 1 --2- 172.21.15.1:0/3230049143 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94c8106490 0x7f94c8105df0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f94c4011980 tx=0x7f94c4011a60 comp rx=0 tx=0).stop 2023-11-16T16:31:10.553 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:10.084+0000 7f94d04af700 1 -- 172.21.15.1:0/3230049143 shutdown_connections 2023-11-16T16:31:10.553 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:10.084+0000 7f94d04af700 1 --2- 172.21.15.1:0/3230049143 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f94b4038850 0x7f94b403ad00 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:10.553 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:10.084+0000 7f94d04af700 1 --2- 172.21.15.1:0/3230049143 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f94c8106490 0x7f94c8105df0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:10.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:10.084+0000 7f94d04af700 1 -- 172.21.15.1:0/3230049143 >> 172.21.15.1:0/3230049143 conn(0x7f94c80ffed0 msgr2=0x7f94c8100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:31:10.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:10.084+0000 7f94d04af700 1 -- 172.21.15.1:0/3230049143 shutdown_connections 2023-11-16T16:31:10.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2023-11-16T16:31:10.084+0000 7f94d04af700 1 -- 172.21.15.1:0/3230049143 wait complete. 2023-11-16T16:31:10.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-11-16T16:31:10.554 INFO:teuthology.orchestra.run.smithi001.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-11-16T16:31:10.554 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:31:10.554 INFO:teuthology.orchestra.run.smithi001.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-11-16T16:31:10.554 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:31:10.554 INFO:teuthology.orchestra.run.smithi001.stdout:Or, if you are only running a single cluster on this host: 2023-11-16T16:31:10.554 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:31:10.555 INFO:teuthology.orchestra.run.smithi001.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-11-16T16:31:10.555 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:31:10.555 INFO:teuthology.orchestra.run.smithi001.stdout:Please consider enabling telemetry to help improve Ceph: 2023-11-16T16:31:10.555 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:31:10.555 INFO:teuthology.orchestra.run.smithi001.stdout: ceph telemetry on 2023-11-16T16:31:10.555 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:31:10.555 INFO:teuthology.orchestra.run.smithi001.stdout:For more information see: 2023-11-16T16:31:10.555 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:31:10.555 INFO:teuthology.orchestra.run.smithi001.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-11-16T16:31:10.555 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:31:10.555 INFO:teuthology.orchestra.run.smithi001.stdout:Bootstrap complete. 2023-11-16T16:31:10.583 INFO:tasks.cephadm:Fetching config... 2023-11-16T16:31:10.583 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2023-11-16T16:31:10.583 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-11-16T16:31:10.600 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-11-16T16:31:10.601 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2023-11-16T16:31:10.601 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-11-16T16:31:10.659 INFO:tasks.cephadm:Fetching mon keyring... 2023-11-16T16:31:10.659 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2023-11-16T16:31:10.659 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/keyring of=/dev/stdout 2023-11-16T16:31:10.728 INFO:tasks.cephadm:Fetching pub ssh key... 2023-11-16T16:31:10.729 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2023-11-16T16:31:10.729 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-11-16T16:31:10.786 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-11-16T16:31:10.786 DEBUG:teuthology.orchestra.run.smithi001:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwDkmaltnqApgAGAS1JnIiIaLi1tcfzyeCKYojJPYSAjZ3kQY/cIaE5Enj75HRJvWi/1CEVH/+L807c40k3RQ1mB1sA3M/zcxGrDppM06TYmbUKSyXX96bfeUYzSngBT//vIKrKUgzBOpiumnCJ9B2u86Jv58CrUTj5yl91jFMzKZjrLBEQojhGapfLCtbtz4jZSCpcQy94WsfMg38zGMQ2YEpvQYGMjdn2t9r2AhCfckae81EnPwPE7/5oBNAw1yo0GN7woIK/BRGZodi6l2Yl3Nujrf73OaZQSyZRSp7O1wYZ2ckOIvQ5VfeaIF0j1q4pFdhUUSgNikj4ImrMfkJOW+7QVBvFzYPokmUyiDemmPNCBAuG8tx59TqCwgGsTh7P+zZ7yCXGgTuxLpVKj/fyiB1VUzMQEBtniq82FW9+gbgTarf6+6vVtjkXEWiUAO1W/f6AG9HUCkeJYJysIvZyRXv5TH07Ab+b3WvVaZQc92TeD8EAyKuHuNmGtzDZkM= ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-11-16T16:31:10.866 INFO:teuthology.orchestra.run.smithi001.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwDkmaltnqApgAGAS1JnIiIaLi1tcfzyeCKYojJPYSAjZ3kQY/cIaE5Enj75HRJvWi/1CEVH/+L807c40k3RQ1mB1sA3M/zcxGrDppM06TYmbUKSyXX96bfeUYzSngBT//vIKrKUgzBOpiumnCJ9B2u86Jv58CrUTj5yl91jFMzKZjrLBEQojhGapfLCtbtz4jZSCpcQy94WsfMg38zGMQ2YEpvQYGMjdn2t9r2AhCfckae81EnPwPE7/5oBNAw1yo0GN7woIK/BRGZodi6l2Yl3Nujrf73OaZQSyZRSp7O1wYZ2ckOIvQ5VfeaIF0j1q4pFdhUUSgNikj4ImrMfkJOW+7QVBvFzYPokmUyiDemmPNCBAuG8tx59TqCwgGsTh7P+zZ7yCXGgTuxLpVKj/fyiB1VUzMQEBtniq82FW9+gbgTarf6+6vVtjkXEWiUAO1W/f6AG9HUCkeJYJysIvZyRXv5TH07Ab+b3WvVaZQc92TeD8EAyKuHuNmGtzDZkM= ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7 2023-11-16T16:31:10.880 DEBUG:teuthology.orchestra.run.smithi026:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwDkmaltnqApgAGAS1JnIiIaLi1tcfzyeCKYojJPYSAjZ3kQY/cIaE5Enj75HRJvWi/1CEVH/+L807c40k3RQ1mB1sA3M/zcxGrDppM06TYmbUKSyXX96bfeUYzSngBT//vIKrKUgzBOpiumnCJ9B2u86Jv58CrUTj5yl91jFMzKZjrLBEQojhGapfLCtbtz4jZSCpcQy94WsfMg38zGMQ2YEpvQYGMjdn2t9r2AhCfckae81EnPwPE7/5oBNAw1yo0GN7woIK/BRGZodi6l2Yl3Nujrf73OaZQSyZRSp7O1wYZ2ckOIvQ5VfeaIF0j1q4pFdhUUSgNikj4ImrMfkJOW+7QVBvFzYPokmUyiDemmPNCBAuG8tx59TqCwgGsTh7P+zZ7yCXGgTuxLpVKj/fyiB1VUzMQEBtniq82FW9+gbgTarf6+6vVtjkXEWiUAO1W/f6AG9HUCkeJYJysIvZyRXv5TH07Ab+b3WvVaZQc92TeD8EAyKuHuNmGtzDZkM= ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-11-16T16:31:10.927 INFO:teuthology.orchestra.run.smithi026.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwDkmaltnqApgAGAS1JnIiIaLi1tcfzyeCKYojJPYSAjZ3kQY/cIaE5Enj75HRJvWi/1CEVH/+L807c40k3RQ1mB1sA3M/zcxGrDppM06TYmbUKSyXX96bfeUYzSngBT//vIKrKUgzBOpiumnCJ9B2u86Jv58CrUTj5yl91jFMzKZjrLBEQojhGapfLCtbtz4jZSCpcQy94WsfMg38zGMQ2YEpvQYGMjdn2t9r2AhCfckae81EnPwPE7/5oBNAw1yo0GN7woIK/BRGZodi6l2Yl3Nujrf73OaZQSyZRSp7O1wYZ2ckOIvQ5VfeaIF0j1q4pFdhUUSgNikj4ImrMfkJOW+7QVBvFzYPokmUyiDemmPNCBAuG8tx59TqCwgGsTh7P+zZ7yCXGgTuxLpVKj/fyiB1VUzMQEBtniq82FW9+gbgTarf6+6vVtjkXEWiUAO1W/f6AG9HUCkeJYJysIvZyRXv5TH07Ab+b3WvVaZQc92TeD8EAyKuHuNmGtzDZkM= ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7 2023-11-16T16:31:10.941 DEBUG:teuthology.orchestra.run.smithi053:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwDkmaltnqApgAGAS1JnIiIaLi1tcfzyeCKYojJPYSAjZ3kQY/cIaE5Enj75HRJvWi/1CEVH/+L807c40k3RQ1mB1sA3M/zcxGrDppM06TYmbUKSyXX96bfeUYzSngBT//vIKrKUgzBOpiumnCJ9B2u86Jv58CrUTj5yl91jFMzKZjrLBEQojhGapfLCtbtz4jZSCpcQy94WsfMg38zGMQ2YEpvQYGMjdn2t9r2AhCfckae81EnPwPE7/5oBNAw1yo0GN7woIK/BRGZodi6l2Yl3Nujrf73OaZQSyZRSp7O1wYZ2ckOIvQ5VfeaIF0j1q4pFdhUUSgNikj4ImrMfkJOW+7QVBvFzYPokmUyiDemmPNCBAuG8tx59TqCwgGsTh7P+zZ7yCXGgTuxLpVKj/fyiB1VUzMQEBtniq82FW9+gbgTarf6+6vVtjkXEWiUAO1W/f6AG9HUCkeJYJysIvZyRXv5TH07Ab+b3WvVaZQc92TeD8EAyKuHuNmGtzDZkM= ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-11-16T16:31:10.990 INFO:teuthology.orchestra.run.smithi053.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwDkmaltnqApgAGAS1JnIiIaLi1tcfzyeCKYojJPYSAjZ3kQY/cIaE5Enj75HRJvWi/1CEVH/+L807c40k3RQ1mB1sA3M/zcxGrDppM06TYmbUKSyXX96bfeUYzSngBT//vIKrKUgzBOpiumnCJ9B2u86Jv58CrUTj5yl91jFMzKZjrLBEQojhGapfLCtbtz4jZSCpcQy94WsfMg38zGMQ2YEpvQYGMjdn2t9r2AhCfckae81EnPwPE7/5oBNAw1yo0GN7woIK/BRGZodi6l2Yl3Nujrf73OaZQSyZRSp7O1wYZ2ckOIvQ5VfeaIF0j1q4pFdhUUSgNikj4ImrMfkJOW+7QVBvFzYPokmUyiDemmPNCBAuG8tx59TqCwgGsTh7P+zZ7yCXGgTuxLpVKj/fyiB1VUzMQEBtniq82FW9+gbgTarf6+6vVtjkXEWiUAO1W/f6AG9HUCkeJYJysIvZyRXv5TH07Ab+b3WvVaZQc92TeD8EAyKuHuNmGtzDZkM= ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7 2023-11-16T16:31:11.003 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-11-16T16:31:11.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:11 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3230049143' entity='client.admin' 2023-11-16T16:31:11.832 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:31:12.871 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:12 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:12.871 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:12 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:12.872 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:12 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi001", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:31:12.872 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:12 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:12.872 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:12 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:31:12.872 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:12 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:13.553 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.549+0000 7f08ff2d6700 1 -- 172.21.15.1:0/42887562 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08f8101300 msgr2=0x7f08f8101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:13.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.549+0000 7f08ff2d6700 1 --2- 172.21.15.1:0/42887562 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08f8101300 0x7f08f8101710 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f08f4009910 tx=0x7f08f4009c20 comp rx=0 tx=0).stop 2023-11-16T16:31:13.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.549+0000 7f08ff2d6700 1 -- 172.21.15.1:0/42887562 shutdown_connections 2023-11-16T16:31:13.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.549+0000 7f08ff2d6700 1 --2- 172.21.15.1:0/42887562 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08f8101300 0x7f08f8101710 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:13.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.549+0000 7f08ff2d6700 1 -- 172.21.15.1:0/42887562 >> 172.21.15.1:0/42887562 conn(0x7f08f80f9930 msgr2=0x7f08f80fbd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:31:13.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.550+0000 7f08ff2d6700 1 -- 172.21.15.1:0/42887562 shutdown_connections 2023-11-16T16:31:13.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.550+0000 7f08ff2d6700 1 -- 172.21.15.1:0/42887562 wait complete. 2023-11-16T16:31:13.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.550+0000 7f08ff2d6700 1 Processor -- start 2023-11-16T16:31:13.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.550+0000 7f08ff2d6700 1 -- start start 2023-11-16T16:31:13.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.550+0000 7f08ff2d6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08f8101300 0x7f08f8194e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:13.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.550+0000 7f08ff2d6700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f8195360 con 0x7f08f8101300 2023-11-16T16:31:13.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.550+0000 7f08fd072700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08f8101300 0x7f08f8194e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:13.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.550+0000 7f08fd072700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08f8101300 0x7f08f8194e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40444/0 (socket says 172.21.15.1:40444) 2023-11-16T16:31:13.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.550+0000 7f08fd072700 1 -- 172.21.15.1:0/3370556188 learned_addr learned my addr 172.21.15.1:0/3370556188 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:31:13.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.551+0000 7f08fd072700 1 -- 172.21.15.1:0/3370556188 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f40095f0 con 0x7f08f8101300 2023-11-16T16:31:13.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.551+0000 7f08fd072700 1 --2- 172.21.15.1:0/3370556188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08f8101300 0x7f08f8194e20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f08f4006010 tx=0x7f08f40057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:13.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.551+0000 7f08ee7fc700 1 -- 172.21.15.1:0/3370556188 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f08f401d070 con 0x7f08f8101300 2023-11-16T16:31:13.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.551+0000 7f08ee7fc700 1 -- 172.21.15.1:0/3370556188 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08f40044e0 con 0x7f08f8101300 2023-11-16T16:31:13.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.551+0000 7f08ff2d6700 1 -- 172.21.15.1:0/3370556188 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08f8105bf0 con 0x7f08f8101300 2023-11-16T16:31:13.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.551+0000 7f08ee7fc700 1 -- 172.21.15.1:0/3370556188 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f08f40039b0 con 0x7f08f8101300 2023-11-16T16:31:13.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.551+0000 7f08ff2d6700 1 -- 172.21.15.1:0/3370556188 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08f8105a60 con 0x7f08f8101300 2023-11-16T16:31:13.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.552+0000 7f08ee7fc700 1 -- 172.21.15.1:0/3370556188 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f08f4003c20 con 0x7f08f8101300 2023-11-16T16:31:13.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.552+0000 7f08ff2d6700 1 -- 172.21.15.1:0/3370556188 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08f8105d80 con 0x7f08f8101300 2023-11-16T16:31:13.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.552+0000 7f08ee7fc700 1 --2- 172.21.15.1:0/3370556188 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f08e40388c0 0x7f08e403ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:13.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.552+0000 7f08ee7fc700 1 -- 172.21.15.1:0/3370556188 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f08f404c8a0 con 0x7f08f8101300 2023-11-16T16:31:13.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.552+0000 7f08fc871700 1 --2- 172.21.15.1:0/3370556188 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f08e40388c0 0x7f08e403ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:13.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.553+0000 7f08fc871700 1 --2- 172.21.15.1:0/3370556188 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f08e40388c0 0x7f08e403ad70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f08e8006fd0 tx=0x7f08e8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:13.559 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.557+0000 7f08ee7fc700 1 -- 172.21.15.1:0/3370556188 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f08f4020d40 con 0x7f08f8101300 2023-11-16T16:31:13.756 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.754+0000 7f08ff2d6700 1 -- 172.21.15.1:0/3370556188 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f08f8195950 con 0x7f08f8101300 2023-11-16T16:31:13.759 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.758+0000 7f08ee7fc700 1 -- 172.21.15.1:0/3370556188 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f08f4020b60 con 0x7f08f8101300 2023-11-16T16:31:13.762 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.760+0000 7f08ff2d6700 1 -- 172.21.15.1:0/3370556188 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f08e40388c0 msgr2=0x7f08e403ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:13.762 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.760+0000 7f08ff2d6700 1 --2- 172.21.15.1:0/3370556188 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f08e40388c0 0x7f08e403ad70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f08e8006fd0 tx=0x7f08e8006e40 comp rx=0 tx=0).stop 2023-11-16T16:31:13.762 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.760+0000 7f08ff2d6700 1 -- 172.21.15.1:0/3370556188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08f8101300 msgr2=0x7f08f8194e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:13.762 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.760+0000 7f08ff2d6700 1 --2- 172.21.15.1:0/3370556188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08f8101300 0x7f08f8194e20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f08f4006010 tx=0x7f08f40057c0 comp rx=0 tx=0).stop 2023-11-16T16:31:13.762 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.760+0000 7f08ff2d6700 1 -- 172.21.15.1:0/3370556188 shutdown_connections 2023-11-16T16:31:13.762 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.760+0000 7f08ff2d6700 1 --2- 172.21.15.1:0/3370556188 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f08e40388c0 0x7f08e403ad70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:13.762 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.760+0000 7f08ff2d6700 1 --2- 172.21.15.1:0/3370556188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08f8101300 0x7f08f8194e20 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:13.762 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.760+0000 7f08ff2d6700 1 -- 172.21.15.1:0/3370556188 >> 172.21.15.1:0/3370556188 conn(0x7f08f80f9930 msgr2=0x7f08f80fa620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:31:13.762 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.760+0000 7f08ff2d6700 1 -- 172.21.15.1:0/3370556188 shutdown_connections 2023-11-16T16:31:13.762 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:13.760+0000 7f08ff2d6700 1 -- 172.21.15.1:0/3370556188 wait complete. 2023-11-16T16:31:14.192 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-11-16T16:31:14.193 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-11-16T16:31:14.844 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:31:15.063 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:14 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3370556188' entity='client.admin' 2023-11-16T16:31:15.063 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:14 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:31:15.063 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:14 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:16.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.235+0000 7f709d82d700 1 -- 172.21.15.1:0/977082910 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70981001b0 msgr2=0x7f70981005c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:16.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.235+0000 7f709d82d700 1 --2- 172.21.15.1:0/977082910 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70981001b0 0x7f70981005c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7088009910 tx=0x7f7088009c20 comp rx=0 tx=0).stop 2023-11-16T16:31:16.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.235+0000 7f709d82d700 1 -- 172.21.15.1:0/977082910 shutdown_connections 2023-11-16T16:31:16.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.235+0000 7f709d82d700 1 --2- 172.21.15.1:0/977082910 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70981001b0 0x7f70981005c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:16.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.235+0000 7f709d82d700 1 -- 172.21.15.1:0/977082910 >> 172.21.15.1:0/977082910 conn(0x7f70980f9930 msgr2=0x7f70980fbd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:31:16.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.235+0000 7f709d82d700 1 -- 172.21.15.1:0/977082910 shutdown_connections 2023-11-16T16:31:16.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.235+0000 7f709d82d700 1 -- 172.21.15.1:0/977082910 wait complete. 2023-11-16T16:31:16.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.235+0000 7f709d82d700 1 Processor -- start 2023-11-16T16:31:16.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.236+0000 7f709d82d700 1 -- start start 2023-11-16T16:31:16.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.236+0000 7f709d82d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70981001b0 0x7f70980fe960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:16.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.236+0000 7f709d82d700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7098197b40 con 0x7f70981001b0 2023-11-16T16:31:16.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.236+0000 7f7096ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70981001b0 0x7f70980fe960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:16.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.236+0000 7f7096ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70981001b0 0x7f70980fe960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40450/0 (socket says 172.21.15.1:40450) 2023-11-16T16:31:16.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.236+0000 7f7096ffd700 1 -- 172.21.15.1:0/427223548 learned_addr learned my addr 172.21.15.1:0/427223548 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:31:16.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.236+0000 7f7096ffd700 1 -- 172.21.15.1:0/427223548 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70880095f0 con 0x7f70981001b0 2023-11-16T16:31:16.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.236+0000 7f7096ffd700 1 --2- 172.21.15.1:0/427223548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70981001b0 0x7f70980fe960 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7088006010 tx=0x7f70880057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:16.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.237+0000 7f709c82b700 1 -- 172.21.15.1:0/427223548 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f708801d070 con 0x7f70981001b0 2023-11-16T16:31:16.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.237+0000 7f709c82b700 1 -- 172.21.15.1:0/427223548 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70880044e0 con 0x7f70981001b0 2023-11-16T16:31:16.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.237+0000 7f709d82d700 1 -- 172.21.15.1:0/427223548 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7098104af0 con 0x7f70981001b0 2023-11-16T16:31:16.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.237+0000 7f709c82b700 1 -- 172.21.15.1:0/427223548 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70880039b0 con 0x7f70981001b0 2023-11-16T16:31:16.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.237+0000 7f709d82d700 1 -- 172.21.15.1:0/427223548 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7098104960 con 0x7f70981001b0 2023-11-16T16:31:16.242 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.237+0000 7f709c82b700 1 -- 172.21.15.1:0/427223548 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f7088003c70 con 0x7f70981001b0 2023-11-16T16:31:16.242 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.238+0000 7f709d82d700 1 -- 172.21.15.1:0/427223548 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f709818d820 con 0x7f70981001b0 2023-11-16T16:31:16.242 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.238+0000 7f709c82b700 1 --2- 172.21.15.1:0/427223548 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f7080038870 0x7f708003ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:16.242 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.238+0000 7f709c82b700 1 -- 172.21.15.1:0/427223548 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f708804c940 con 0x7f70981001b0 2023-11-16T16:31:16.242 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.238+0000 7f70967fc700 1 --2- 172.21.15.1:0/427223548 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f7080038870 0x7f708003ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:16.242 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.238+0000 7f70967fc700 1 --2- 172.21.15.1:0/427223548 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f7080038870 0x7f708003ad20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f708c006fd0 tx=0x7f708c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:16.244 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.243+0000 7f709c82b700 1 -- 172.21.15.1:0/427223548 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f708802a950 con 0x7f70981001b0 2023-11-16T16:31:16.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.446+0000 7f709d82d700 1 -- 172.21.15.1:0/427223548 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f7098061b20 con 0x7f7080038870 2023-11-16T16:31:16.451 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.449+0000 7f709c82b700 1 -- 172.21.15.1:0/427223548 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7098061b20 con 0x7f7080038870 2023-11-16T16:31:16.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.451+0000 7f709d82d700 1 -- 172.21.15.1:0/427223548 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f7080038870 msgr2=0x7f708003ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:16.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.451+0000 7f709d82d700 1 --2- 172.21.15.1:0/427223548 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f7080038870 0x7f708003ad20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f708c006fd0 tx=0x7f708c006e40 comp rx=0 tx=0).stop 2023-11-16T16:31:16.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.451+0000 7f709d82d700 1 -- 172.21.15.1:0/427223548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70981001b0 msgr2=0x7f70980fe960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:16.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.451+0000 7f709d82d700 1 --2- 172.21.15.1:0/427223548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70981001b0 0x7f70980fe960 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7088006010 tx=0x7f70880057c0 comp rx=0 tx=0).stop 2023-11-16T16:31:16.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.451+0000 7f709d82d700 1 -- 172.21.15.1:0/427223548 shutdown_connections 2023-11-16T16:31:16.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.451+0000 7f709d82d700 1 --2- 172.21.15.1:0/427223548 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f7080038870 0x7f708003ad20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:16.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.451+0000 7f709d82d700 1 --2- 172.21.15.1:0/427223548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70981001b0 0x7f70980fe960 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:16.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.451+0000 7f709d82d700 1 -- 172.21.15.1:0/427223548 >> 172.21.15.1:0/427223548 conn(0x7f70980f9930 msgr2=0x7f70980fa620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:31:16.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.451+0000 7f709d82d700 1 -- 172.21.15.1:0/427223548 shutdown_connections 2023-11-16T16:31:16.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:16.451+0000 7f709d82d700 1 -- 172.21.15.1:0/427223548 wait complete. 2023-11-16T16:31:16.906 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi026 2023-11-16T16:31:16.906 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-11-16T16:31:16.906 DEBUG:teuthology.orchestra.run.smithi026:> dd of=/etc/ceph/ceph.conf 2023-11-16T16:31:16.927 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-11-16T16:31:16.928 DEBUG:teuthology.orchestra.run.smithi026:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-11-16T16:31:16.987 INFO:tasks.cephadm:Adding host smithi026 to orchestrator... 2023-11-16T16:31:16.988 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph orch host add smithi026 2023-11-16T16:31:17.653 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:31:17.838 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:17 smithi001 ceph-mon[98800]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:31:17.838 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:17 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:17.839 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:17 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:31:17.839 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:17 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:31:17.839 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:17 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:31:17.839 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:17 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:31:17.839 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:17 smithi001 ceph-mon[98800]: Updating smithi001:/etc/ceph/ceph.conf 2023-11-16T16:31:17.839 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:17 smithi001 ceph-mon[98800]: Updating smithi001:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:31:17.839 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:17 smithi001 ceph-mon[98800]: Updating smithi001:/etc/ceph/ceph.client.admin.keyring 2023-11-16T16:31:17.839 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:17 smithi001 ceph-mon[98800]: Updating smithi001:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-16T16:31:18.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:18 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:18.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:18 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:18.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:18 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:19.188 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:19.185+0000 7f7670048700 1 -- 172.21.15.1:0/535141801 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76680fef80 msgr2=0x7f76680ff350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:19.188 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:19.185+0000 7f7670048700 1 --2- 172.21.15.1:0/535141801 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76680fef80 0x7f76680ff350 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7664009910 tx=0x7f7664009c20 comp rx=0 tx=0).stop 2023-11-16T16:31:19.188 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:19.186+0000 7f7670048700 1 -- 172.21.15.1:0/535141801 shutdown_connections 2023-11-16T16:31:19.188 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:19.186+0000 7f7670048700 1 --2- 172.21.15.1:0/535141801 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76680fef80 0x7f76680ff350 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:19.188 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:19.186+0000 7f7670048700 1 -- 172.21.15.1:0/535141801 >> 172.21.15.1:0/535141801 conn(0x7f76680fa880 msgr2=0x7f76680fcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:31:19.188 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:19.186+0000 7f7670048700 1 -- 172.21.15.1:0/535141801 shutdown_connections 2023-11-16T16:31:19.188 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:19.186+0000 7f7670048700 1 -- 172.21.15.1:0/535141801 wait complete. 2023-11-16T16:31:19.190 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:19.186+0000 7f7670048700 1 Processor -- start 2023-11-16T16:31:19.191 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:19.186+0000 7f7670048700 1 -- start start 2023-11-16T16:31:19.191 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:19.186+0000 7f7670048700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76680fef80 0x7f7668195180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:19.191 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:19.186+0000 7f7670048700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76681956c0 con 0x7f76680fef80 2023-11-16T16:31:19.191 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:19.187+0000 7f766dde4700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76680fef80 0x7f7668195180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:19.191 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:19.187+0000 7f766dde4700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76680fef80 0x7f7668195180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40456/0 (socket says 172.21.15.1:40456) 2023-11-16T16:31:19.191 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:19.187+0000 7f766dde4700 1 -- 172.21.15.1:0/2976757759 learned_addr learned my addr 172.21.15.1:0/2976757759 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:31:19.191 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:19.187+0000 7f766dde4700 1 -- 172.21.15.1:0/2976757759 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76640095f0 con 0x7f76680fef80 2023-11-16T16:31:19.191 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:19.187+0000 7f766dde4700 1 --2- 172.21.15.1:0/2976757759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76680fef80 0x7f7668195180 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7664006010 tx=0x7f7664004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:19.191 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:19.187+0000 7f765affd700 1 -- 172.21.15.1:0/2976757759 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f766401d070 con 0x7f76680fef80 2023-11-16T16:31:19.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:19.187+0000 7f765affd700 1 -- 172.21.15.1:0/2976757759 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f766400bc10 con 0x7f76680fef80 2023-11-16T16:31:19.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:19.187+0000 7f7670048700 1 -- 172.21.15.1:0/2976757759 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f766810a260 con 0x7f76680fef80 2023-11-16T16:31:19.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:19.187+0000 7f765affd700 1 -- 172.21.15.1:0/2976757759 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7664003910 con 0x7f76680fef80 2023-11-16T16:31:19.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:19.187+0000 7f7670048700 1 -- 172.21.15.1:0/2976757759 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f766810a0d0 con 0x7f76680fef80 2023-11-16T16:31:19.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:19.188+0000 7f765affd700 1 -- 172.21.15.1:0/2976757759 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f766400bd80 con 0x7f76680fef80 2023-11-16T16:31:19.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:19.188+0000 7f7670048700 1 -- 172.21.15.1:0/2976757759 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f766810a3f0 con 0x7f76680fef80 2023-11-16T16:31:19.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:19.188+0000 7f765affd700 1 --2- 172.21.15.1:0/2976757759 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f76540388c0 0x7f765403ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:19.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:19.188+0000 7f765affd700 1 -- 172.21.15.1:0/2976757759 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f766404c4a0 con 0x7f76680fef80 2023-11-16T16:31:19.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:19.188+0000 7f766d5e3700 1 --2- 172.21.15.1:0/2976757759 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f76540388c0 0x7f765403ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:19.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:19.189+0000 7f766d5e3700 1 --2- 172.21.15.1:0/2976757759 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f76540388c0 0x7f765403ad70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f765c006fd0 tx=0x7f765c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:19.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:19.193+0000 7f765affd700 1 -- 172.21.15.1:0/2976757759 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7664004390 con 0x7f76680fef80 2023-11-16T16:31:19.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:19.393+0000 7f7670048700 1 -- 172.21.15.1:0/2976757759 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi026", "target": ["mon-mgr", ""]}) v1 -- 0x7f76681070c0 con 0x7f76540388c0 2023-11-16T16:31:19.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:19 smithi001 ceph-mon[98800]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi026", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:31:20.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:20 smithi001 ceph-mon[98800]: Deploying cephadm binary to smithi026 2023-11-16T16:31:21.876 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:21.874+0000 7f765affd700 1 -- 172.21.15.1:0/2976757759 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f76681070c0 con 0x7f76540388c0 2023-11-16T16:31:21.876 INFO:teuthology.orchestra.run.smithi001.stdout:Added host 'smithi026' with addr '172.21.15.26' 2023-11-16T16:31:21.878 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:21.876+0000 7f7670048700 1 -- 172.21.15.1:0/2976757759 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f76540388c0 msgr2=0x7f765403ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:21.878 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:21.876+0000 7f7670048700 1 --2- 172.21.15.1:0/2976757759 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f76540388c0 0x7f765403ad70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f765c006fd0 tx=0x7f765c006e40 comp rx=0 tx=0).stop 2023-11-16T16:31:21.879 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:21.876+0000 7f7670048700 1 -- 172.21.15.1:0/2976757759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76680fef80 msgr2=0x7f7668195180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:21.879 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:21.876+0000 7f7670048700 1 --2- 172.21.15.1:0/2976757759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76680fef80 0x7f7668195180 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7664006010 tx=0x7f7664004ff0 comp rx=0 tx=0).stop 2023-11-16T16:31:21.879 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:21.876+0000 7f7670048700 1 -- 172.21.15.1:0/2976757759 shutdown_connections 2023-11-16T16:31:21.879 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:21.876+0000 7f7670048700 1 --2- 172.21.15.1:0/2976757759 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f76540388c0 0x7f765403ad70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:21.879 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:21.876+0000 7f7670048700 1 --2- 172.21.15.1:0/2976757759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76680fef80 0x7f7668195180 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:21.879 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:21.876+0000 7f7670048700 1 -- 172.21.15.1:0/2976757759 >> 172.21.15.1:0/2976757759 conn(0x7f76680fa880 msgr2=0x7f7668103160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:31:21.879 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:21.877+0000 7f7670048700 1 -- 172.21.15.1:0/2976757759 shutdown_connections 2023-11-16T16:31:21.879 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:21.877+0000 7f7670048700 1 -- 172.21.15.1:0/2976757759 wait complete. 2023-11-16T16:31:22.323 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph orch host ls --format=json 2023-11-16T16:31:22.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:22 smithi001 ceph-mon[98800]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:22.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:22 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:22.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:22 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:31:22.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:22 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:22.985 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:31:23.522 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:23 smithi001 ceph-mon[98800]: Added host smithi026 2023-11-16T16:31:23.522 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:23 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:24.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.276+0000 7fc27e47d700 1 -- 172.21.15.1:0/676420816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2780fef00 msgr2=0x7fc2780ff2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:24.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.276+0000 7fc27e47d700 1 --2- 172.21.15.1:0/676420816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2780fef00 0x7fc2780ff2d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc268009910 tx=0x7fc268009c20 comp rx=0 tx=0).stop 2023-11-16T16:31:24.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.276+0000 7fc27e47d700 1 -- 172.21.15.1:0/676420816 shutdown_connections 2023-11-16T16:31:24.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.276+0000 7fc27e47d700 1 --2- 172.21.15.1:0/676420816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2780fef00 0x7fc2780ff2d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:24.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.276+0000 7fc27e47d700 1 -- 172.21.15.1:0/676420816 >> 172.21.15.1:0/676420816 conn(0x7fc278074930 msgr2=0x7fc278074d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:31:24.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.276+0000 7fc27e47d700 1 -- 172.21.15.1:0/676420816 shutdown_connections 2023-11-16T16:31:24.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.276+0000 7fc27e47d700 1 -- 172.21.15.1:0/676420816 wait complete. 2023-11-16T16:31:24.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.276+0000 7fc27e47d700 1 Processor -- start 2023-11-16T16:31:24.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.277+0000 7fc27e47d700 1 -- start start 2023-11-16T16:31:24.283 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.277+0000 7fc27e47d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2780fef00 0x7fc278196030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:24.283 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.277+0000 7fc27e47d700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc278196570 con 0x7fc2780fef00 2023-11-16T16:31:24.283 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.277+0000 7fc277fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2780fef00 0x7fc278196030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:24.283 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.277+0000 7fc277fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2780fef00 0x7fc278196030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40482/0 (socket says 172.21.15.1:40482) 2023-11-16T16:31:24.283 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.277+0000 7fc277fff700 1 -- 172.21.15.1:0/2366305026 learned_addr learned my addr 172.21.15.1:0/2366305026 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:31:24.283 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.277+0000 7fc277fff700 1 -- 172.21.15.1:0/2366305026 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2680095f0 con 0x7fc2780fef00 2023-11-16T16:31:24.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.277+0000 7fc277fff700 1 --2- 172.21.15.1:0/2366305026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2780fef00 0x7fc278196030 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc268004f40 tx=0x7fc268005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:24.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.278+0000 7fc2757fa700 1 -- 172.21.15.1:0/2366305026 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc26801d070 con 0x7fc2780fef00 2023-11-16T16:31:24.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.278+0000 7fc2757fa700 1 -- 172.21.15.1:0/2366305026 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc268004370 con 0x7fc2780fef00 2023-11-16T16:31:24.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.278+0000 7fc27e47d700 1 -- 172.21.15.1:0/2366305026 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc278109090 con 0x7fc2780fef00 2023-11-16T16:31:24.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.278+0000 7fc27e47d700 1 -- 172.21.15.1:0/2366305026 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc278108f00 con 0x7fc2780fef00 2023-11-16T16:31:24.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.278+0000 7fc2757fa700 1 -- 172.21.15.1:0/2366305026 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc2680039b0 con 0x7fc2780fef00 2023-11-16T16:31:24.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.278+0000 7fc2757fa700 1 -- 172.21.15.1:0/2366305026 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7fc268003c20 con 0x7fc2780fef00 2023-11-16T16:31:24.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.279+0000 7fc2757fa700 1 --2- 172.21.15.1:0/2366305026 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc2600388c0 0x7fc26003ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:24.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.279+0000 7fc27e47d700 1 -- 172.21.15.1:0/2366305026 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc264005320 con 0x7fc2780fef00 2023-11-16T16:31:24.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.279+0000 7fc2757fa700 1 -- 172.21.15.1:0/2366305026 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc26804c730 con 0x7fc2780fef00 2023-11-16T16:31:24.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.279+0000 7fc2777fe700 1 --2- 172.21.15.1:0/2366305026 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc2600388c0 0x7fc26003ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:24.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.279+0000 7fc2777fe700 1 --2- 172.21.15.1:0/2366305026 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc2600388c0 0x7fc26003ad70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc26c006fd0 tx=0x7fc26c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:24.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.284+0000 7fc2757fa700 1 -- 172.21.15.1:0/2366305026 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc268029770 con 0x7fc2780fef00 2023-11-16T16:31:24.489 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.487+0000 7fc27e47d700 1 -- 172.21.15.1:0/2366305026 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc264000bf0 con 0x7fc2600388c0 2023-11-16T16:31:24.490 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.488+0000 7fc2757fa700 1 -- 172.21.15.1:0/2366305026 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+158 (secure 0 0 0) 0x7fc264000bf0 con 0x7fc2600388c0 2023-11-16T16:31:24.490 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:31:24.491 INFO:teuthology.orchestra.run.smithi001.stdout:[{"addr": "172.21.15.1", "hostname": "smithi001", "labels": [], "status": ""}, {"addr": "172.21.15.26", "hostname": "smithi026", "labels": [], "status": ""}] 2023-11-16T16:31:24.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.490+0000 7fc27e47d700 1 -- 172.21.15.1:0/2366305026 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc2600388c0 msgr2=0x7fc26003ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:24.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.490+0000 7fc27e47d700 1 --2- 172.21.15.1:0/2366305026 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc2600388c0 0x7fc26003ad70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc26c006fd0 tx=0x7fc26c006e40 comp rx=0 tx=0).stop 2023-11-16T16:31:24.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.490+0000 7fc27e47d700 1 -- 172.21.15.1:0/2366305026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2780fef00 msgr2=0x7fc278196030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:24.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.490+0000 7fc27e47d700 1 --2- 172.21.15.1:0/2366305026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2780fef00 0x7fc278196030 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc268004f40 tx=0x7fc268005020 comp rx=0 tx=0).stop 2023-11-16T16:31:24.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.491+0000 7fc27e47d700 1 -- 172.21.15.1:0/2366305026 shutdown_connections 2023-11-16T16:31:24.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.491+0000 7fc27e47d700 1 --2- 172.21.15.1:0/2366305026 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc2600388c0 0x7fc26003ad70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:24.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.491+0000 7fc27e47d700 1 --2- 172.21.15.1:0/2366305026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2780fef00 0x7fc278196030 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:24.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.491+0000 7fc27e47d700 1 -- 172.21.15.1:0/2366305026 >> 172.21.15.1:0/2366305026 conn(0x7fc278074930 msgr2=0x7fc278103090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:31:24.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.491+0000 7fc27e47d700 1 -- 172.21.15.1:0/2366305026 shutdown_connections 2023-11-16T16:31:24.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:24.491+0000 7fc27e47d700 1 -- 172.21.15.1:0/2366305026 wait complete. 2023-11-16T16:31:24.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:24 smithi001 ceph-mon[98800]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:24.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:24 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:24.948 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi053 2023-11-16T16:31:24.948 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-16T16:31:24.948 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/etc/ceph/ceph.conf 2023-11-16T16:31:24.970 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-16T16:31:24.970 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-11-16T16:31:25.030 INFO:tasks.cephadm:Adding host smithi053 to orchestrator... 2023-11-16T16:31:25.031 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph orch host add smithi053 2023-11-16T16:31:25.690 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:31:26.525 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:26 smithi001 ceph-mon[98800]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:31:26.526 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:26 smithi001 ceph-mon[98800]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:26.997 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:26.995+0000 7fa21cde3700 1 -- 172.21.15.1:0/1776878545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2180fef00 msgr2=0x7fa2180ff2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:26.998 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:26.995+0000 7fa21cde3700 1 --2- 172.21.15.1:0/1776878545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2180fef00 0x7fa2180ff2d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa208009910 tx=0x7fa208009c20 comp rx=0 tx=0).stop 2023-11-16T16:31:26.998 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:26.995+0000 7fa21cde3700 1 -- 172.21.15.1:0/1776878545 shutdown_connections 2023-11-16T16:31:26.998 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:26.995+0000 7fa21cde3700 1 --2- 172.21.15.1:0/1776878545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2180fef00 0x7fa2180ff2d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:26.998 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:26.995+0000 7fa21cde3700 1 -- 172.21.15.1:0/1776878545 >> 172.21.15.1:0/1776878545 conn(0x7fa218074930 msgr2=0x7fa218074d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:31:26.998 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:26.995+0000 7fa21cde3700 1 -- 172.21.15.1:0/1776878545 shutdown_connections 2023-11-16T16:31:26.998 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:26.995+0000 7fa21cde3700 1 -- 172.21.15.1:0/1776878545 wait complete. 2023-11-16T16:31:26.999 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:26.996+0000 7fa21cde3700 1 Processor -- start 2023-11-16T16:31:26.999 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:26.996+0000 7fa21cde3700 1 -- start start 2023-11-16T16:31:26.999 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:26.996+0000 7fa21cde3700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2180fef00 0x7fa2181008c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:27.000 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:26.996+0000 7fa21cde3700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa218102840 con 0x7fa2180fef00 2023-11-16T16:31:27.000 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:26.996+0000 7fa21659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2180fef00 0x7fa2181008c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:27.000 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:26.997+0000 7fa21659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2180fef00 0x7fa2181008c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40488/0 (socket says 172.21.15.1:40488) 2023-11-16T16:31:27.000 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:26.997+0000 7fa21659c700 1 -- 172.21.15.1:0/1970729299 learned_addr learned my addr 172.21.15.1:0/1970729299 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:31:27.000 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:26.997+0000 7fa21659c700 1 -- 172.21.15.1:0/1970729299 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2080095f0 con 0x7fa2180fef00 2023-11-16T16:31:27.000 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:26.997+0000 7fa21659c700 1 --2- 172.21.15.1:0/1970729299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2180fef00 0x7fa2181008c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa208004f40 tx=0x7fa208005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:27.001 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:26.997+0000 7fa2077fe700 1 -- 172.21.15.1:0/1970729299 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa20801d070 con 0x7fa2180fef00 2023-11-16T16:31:27.001 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:26.997+0000 7fa2077fe700 1 -- 172.21.15.1:0/1970729299 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa208004370 con 0x7fa2180fef00 2023-11-16T16:31:27.001 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:26.997+0000 7fa21cde3700 1 -- 172.21.15.1:0/1970729299 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa218109090 con 0x7fa2180fef00 2023-11-16T16:31:27.001 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:26.997+0000 7fa2077fe700 1 -- 172.21.15.1:0/1970729299 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa2080039b0 con 0x7fa2180fef00 2023-11-16T16:31:27.001 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:26.997+0000 7fa21cde3700 1 -- 172.21.15.1:0/1970729299 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa218108f00 con 0x7fa2180fef00 2023-11-16T16:31:27.002 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:26.998+0000 7fa21cde3700 1 -- 172.21.15.1:0/1970729299 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa218109220 con 0x7fa2180fef00 2023-11-16T16:31:27.002 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:26.998+0000 7fa2077fe700 1 -- 172.21.15.1:0/1970729299 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7fa208003c70 con 0x7fa2180fef00 2023-11-16T16:31:27.002 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:26.998+0000 7fa2077fe700 1 --2- 172.21.15.1:0/1970729299 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fa200038870 0x7fa20003ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:27.002 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:26.999+0000 7fa215d9b700 1 --2- 172.21.15.1:0/1970729299 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fa200038870 0x7fa20003ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:27.002 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:26.999+0000 7fa215d9b700 1 --2- 172.21.15.1:0/1970729299 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fa200038870 0x7fa20003ad20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa20c006fd0 tx=0x7fa20c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:27.002 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:27.000+0000 7fa2077fe700 1 -- 172.21.15.1:0/1970729299 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa20800ece0 con 0x7fa2180fef00 2023-11-16T16:31:27.006 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:27.005+0000 7fa2077fe700 1 -- 172.21.15.1:0/1970729299 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa20807e730 con 0x7fa2180fef00 2023-11-16T16:31:27.210 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:27.208+0000 7fa21cde3700 1 -- 172.21.15.1:0/1970729299 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi053", "target": ["mon-mgr", ""]}) v1 -- 0x7fa218101630 con 0x7fa200038870 2023-11-16T16:31:28.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:27 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:28.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:27 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:28.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:27 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:28.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:27 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:28.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:27 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi026", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:31:28.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:27 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:31:28.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:27 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:31:28.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:27 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:31:28.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:27 smithi001 ceph-mon[98800]: Updating smithi026:/etc/ceph/ceph.conf 2023-11-16T16:31:28.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:27 smithi001 ceph-mon[98800]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi053", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:31:28.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:27 smithi001 ceph-mon[98800]: Updating smithi026:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:31:28.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:27 smithi001 ceph-mon[98800]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:29.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:29 smithi001 ceph-mon[98800]: Updating smithi026:/etc/ceph/ceph.client.admin.keyring 2023-11-16T16:31:29.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:29 smithi001 ceph-mon[98800]: Deploying cephadm binary to smithi053 2023-11-16T16:31:29.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:29 smithi001 ceph-mon[98800]: Updating smithi026:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-16T16:31:29.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:29 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:29.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:29 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:29.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:29 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:29.679 INFO:teuthology.orchestra.run.smithi001.stdout:Added host 'smithi053' with addr '172.21.15.53' 2023-11-16T16:31:29.679 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:29.677+0000 7fa2077fe700 1 -- 172.21.15.1:0/1970729299 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7fa218101630 con 0x7fa200038870 2023-11-16T16:31:29.681 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:29.679+0000 7fa21cde3700 1 -- 172.21.15.1:0/1970729299 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fa200038870 msgr2=0x7fa20003ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:29.681 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:29.679+0000 7fa21cde3700 1 --2- 172.21.15.1:0/1970729299 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fa200038870 0x7fa20003ad20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa20c006fd0 tx=0x7fa20c006e40 comp rx=0 tx=0).stop 2023-11-16T16:31:29.681 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:29.679+0000 7fa21cde3700 1 -- 172.21.15.1:0/1970729299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2180fef00 msgr2=0x7fa2181008c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:29.681 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:29.679+0000 7fa21cde3700 1 --2- 172.21.15.1:0/1970729299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2180fef00 0x7fa2181008c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa208004f40 tx=0x7fa208005020 comp rx=0 tx=0).stop 2023-11-16T16:31:29.681 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:29.679+0000 7fa21cde3700 1 -- 172.21.15.1:0/1970729299 shutdown_connections 2023-11-16T16:31:29.681 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:29.679+0000 7fa21cde3700 1 --2- 172.21.15.1:0/1970729299 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fa200038870 0x7fa20003ad20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:29.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:29.679+0000 7fa21cde3700 1 --2- 172.21.15.1:0/1970729299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa2180fef00 0x7fa2181008c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:29.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:29.679+0000 7fa21cde3700 1 -- 172.21.15.1:0/1970729299 >> 172.21.15.1:0/1970729299 conn(0x7fa218074930 msgr2=0x7fa218067cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:31:29.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:29.679+0000 7fa21cde3700 1 -- 172.21.15.1:0/1970729299 shutdown_connections 2023-11-16T16:31:29.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:29.679+0000 7fa21cde3700 1 -- 172.21.15.1:0/1970729299 wait complete. 2023-11-16T16:31:30.158 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph orch host ls --format=json 2023-11-16T16:31:30.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:30 smithi001 ceph-mon[98800]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:30.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:30 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:30.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:30 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:31:30.816 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:31:31.338 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:31 smithi001 ceph-mon[98800]: Added host smithi053 2023-11-16T16:31:31.338 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:31 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:31.338 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:31 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:32.036 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.033+0000 7f6ea6839700 1 -- 172.21.15.1:0/2370894233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ea00fef80 msgr2=0x7f6ea00ff350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:32.036 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.033+0000 7f6ea6839700 1 --2- 172.21.15.1:0/2370894233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ea00fef80 0x7f6ea00ff350 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6e90009910 tx=0x7f6e90009c20 comp rx=0 tx=0).stop 2023-11-16T16:31:32.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.033+0000 7f6ea6839700 1 -- 172.21.15.1:0/2370894233 shutdown_connections 2023-11-16T16:31:32.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.033+0000 7f6ea6839700 1 --2- 172.21.15.1:0/2370894233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ea00fef80 0x7f6ea00ff350 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:32.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.033+0000 7f6ea6839700 1 -- 172.21.15.1:0/2370894233 >> 172.21.15.1:0/2370894233 conn(0x7f6ea00fa880 msgr2=0x7f6ea00fcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:31:32.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.033+0000 7f6ea6839700 1 -- 172.21.15.1:0/2370894233 shutdown_connections 2023-11-16T16:31:32.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.033+0000 7f6ea6839700 1 -- 172.21.15.1:0/2370894233 wait complete. 2023-11-16T16:31:32.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.033+0000 7f6ea6839700 1 Processor -- start 2023-11-16T16:31:32.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.033+0000 7f6ea6839700 1 -- start start 2023-11-16T16:31:32.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.034+0000 7f6ea6839700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ea00fef80 0x7f6ea0195180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:32.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.034+0000 7f6ea6839700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea01956c0 con 0x7f6ea00fef80 2023-11-16T16:31:32.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.034+0000 7f6e9ffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ea00fef80 0x7f6ea0195180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:32.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.034+0000 7f6e9ffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ea00fef80 0x7f6ea0195180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40514/0 (socket says 172.21.15.1:40514) 2023-11-16T16:31:32.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.034+0000 7f6e9ffff700 1 -- 172.21.15.1:0/1608566867 learned_addr learned my addr 172.21.15.1:0/1608566867 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:31:32.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.034+0000 7f6e9ffff700 1 -- 172.21.15.1:0/1608566867 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e900095f0 con 0x7f6ea00fef80 2023-11-16T16:31:32.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.034+0000 7f6e9ffff700 1 --2- 172.21.15.1:0/1608566867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ea00fef80 0x7f6ea0195180 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6e90006010 tx=0x7f6e90004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:32.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.034+0000 7f6e9d7fa700 1 -- 172.21.15.1:0/1608566867 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e9001d070 con 0x7f6ea00fef80 2023-11-16T16:31:32.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.034+0000 7f6e9d7fa700 1 -- 172.21.15.1:0/1608566867 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e9000bc10 con 0x7f6ea00fef80 2023-11-16T16:31:32.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.035+0000 7f6e9d7fa700 1 -- 172.21.15.1:0/1608566867 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e90003910 con 0x7f6ea00fef80 2023-11-16T16:31:32.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.035+0000 7f6ea6839700 1 -- 172.21.15.1:0/1608566867 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ea0109160 con 0x7f6ea00fef80 2023-11-16T16:31:32.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.035+0000 7f6ea6839700 1 -- 172.21.15.1:0/1608566867 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ea0108fd0 con 0x7f6ea00fef80 2023-11-16T16:31:32.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.035+0000 7f6e9d7fa700 1 -- 172.21.15.1:0/1608566867 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f6e90003b80 con 0x7f6ea00fef80 2023-11-16T16:31:32.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.035+0000 7f6ea6839700 1 -- 172.21.15.1:0/1608566867 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ea01092f0 con 0x7f6ea00fef80 2023-11-16T16:31:32.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.036+0000 7f6e9d7fa700 1 --2- 172.21.15.1:0/1608566867 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f6e880388c0 0x7f6e8803ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:32.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.036+0000 7f6e9d7fa700 1 -- 172.21.15.1:0/1608566867 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6e9004c460 con 0x7f6ea00fef80 2023-11-16T16:31:32.040 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.036+0000 7f6e9f7fe700 1 --2- 172.21.15.1:0/1608566867 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f6e880388c0 0x7f6e8803ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:32.040 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.036+0000 7f6e9f7fe700 1 --2- 172.21.15.1:0/1608566867 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f6e880388c0 0x7f6e8803ad70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6e94006fd0 tx=0x7f6e94006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:32.042 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.041+0000 7f6e9d7fa700 1 -- 172.21.15.1:0/1608566867 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6e9000ba70 con 0x7f6ea00fef80 2023-11-16T16:31:32.244 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.242+0000 7f6ea6839700 1 -- 172.21.15.1:0/1608566867 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6ea0195ca0 con 0x7f6e880388c0 2023-11-16T16:31:32.245 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.243+0000 7f6e9d7fa700 1 -- 172.21.15.1:0/1608566867 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+237 (secure 0 0 0) 0x7f6ea0195ca0 con 0x7f6e880388c0 2023-11-16T16:31:32.245 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:31:32.245 INFO:teuthology.orchestra.run.smithi001.stdout:[{"addr": "172.21.15.1", "hostname": "smithi001", "labels": [], "status": ""}, {"addr": "172.21.15.26", "hostname": "smithi026", "labels": [], "status": ""}, {"addr": "172.21.15.53", "hostname": "smithi053", "labels": [], "status": ""}] 2023-11-16T16:31:32.247 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.245+0000 7f6ea6839700 1 -- 172.21.15.1:0/1608566867 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f6e880388c0 msgr2=0x7f6e8803ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:32.247 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.245+0000 7f6ea6839700 1 --2- 172.21.15.1:0/1608566867 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f6e880388c0 0x7f6e8803ad70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6e94006fd0 tx=0x7f6e94006e40 comp rx=0 tx=0).stop 2023-11-16T16:31:32.247 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.245+0000 7f6ea6839700 1 -- 172.21.15.1:0/1608566867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ea00fef80 msgr2=0x7f6ea0195180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:32.247 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.245+0000 7f6ea6839700 1 --2- 172.21.15.1:0/1608566867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ea00fef80 0x7f6ea0195180 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6e90006010 tx=0x7f6e90004ff0 comp rx=0 tx=0).stop 2023-11-16T16:31:32.248 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.245+0000 7f6ea6839700 1 -- 172.21.15.1:0/1608566867 shutdown_connections 2023-11-16T16:31:32.248 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.245+0000 7f6ea6839700 1 --2- 172.21.15.1:0/1608566867 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f6e880388c0 0x7f6e8803ad70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:32.248 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.245+0000 7f6ea6839700 1 --2- 172.21.15.1:0/1608566867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ea00fef80 0x7f6ea0195180 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:32.248 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.245+0000 7f6ea6839700 1 -- 172.21.15.1:0/1608566867 >> 172.21.15.1:0/1608566867 conn(0x7f6ea00fa880 msgr2=0x7f6ea0068580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:31:32.248 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.245+0000 7f6ea6839700 1 -- 172.21.15.1:0/1608566867 shutdown_connections 2023-11-16T16:31:32.248 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:32.245+0000 7f6ea6839700 1 -- 172.21.15.1:0/1608566867 wait complete. 2023-11-16T16:31:32.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:32 smithi001 ceph-mon[98800]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:32.676 INFO:tasks.cephadm:Setting crush tunables to default 2023-11-16T16:31:32.676 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph osd crush tunables default 2023-11-16T16:31:33.331 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:31:33.563 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:33 smithi001 ceph-mon[98800]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:31:33.563 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:33 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:34.563 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:34.559+0000 7efdf8d23700 1 -- 172.21.15.1:0/27767334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efdf40fff90 msgr2=0x7efdf4100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:34.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:34.559+0000 7efdf8d23700 1 --2- 172.21.15.1:0/27767334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efdf40fff90 0x7efdf4100360 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efde8009910 tx=0x7efde8009c20 comp rx=0 tx=0).stop 2023-11-16T16:31:34.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:34.559+0000 7efdf8d23700 1 -- 172.21.15.1:0/27767334 shutdown_connections 2023-11-16T16:31:34.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:34.559+0000 7efdf8d23700 1 --2- 172.21.15.1:0/27767334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efdf40fff90 0x7efdf4100360 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:34.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:34.559+0000 7efdf8d23700 1 -- 172.21.15.1:0/27767334 >> 172.21.15.1:0/27767334 conn(0x7efdf40fb870 msgr2=0x7efdf40fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:31:34.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:34.559+0000 7efdf8d23700 1 -- 172.21.15.1:0/27767334 shutdown_connections 2023-11-16T16:31:34.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:34.559+0000 7efdf8d23700 1 -- 172.21.15.1:0/27767334 wait complete. 2023-11-16T16:31:34.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:34.560+0000 7efdf8d23700 1 Processor -- start 2023-11-16T16:31:34.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:34.560+0000 7efdf8d23700 1 -- start start 2023-11-16T16:31:34.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:34.560+0000 7efdf8d23700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efdf40fff90 0x7efdf4195180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:34.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:34.560+0000 7efdf8d23700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf41956c0 con 0x7efdf40fff90 2023-11-16T16:31:34.602 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:34.561+0000 7efdf259c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efdf40fff90 0x7efdf4195180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:34.602 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:34.561+0000 7efdf259c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efdf40fff90 0x7efdf4195180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40520/0 (socket says 172.21.15.1:40520) 2023-11-16T16:31:34.602 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:34.561+0000 7efdf259c700 1 -- 172.21.15.1:0/1139665089 learned_addr learned my addr 172.21.15.1:0/1139665089 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:31:34.602 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:34.561+0000 7efdf259c700 1 -- 172.21.15.1:0/1139665089 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde80095f0 con 0x7efdf40fff90 2023-11-16T16:31:34.602 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:34.561+0000 7efdf259c700 1 --2- 172.21.15.1:0/1139665089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efdf40fff90 0x7efdf4195180 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efde8006010 tx=0x7efde8004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:34.602 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:34.561+0000 7efde37fe700 1 -- 172.21.15.1:0/1139665089 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efde801d070 con 0x7efdf40fff90 2023-11-16T16:31:34.602 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:34.561+0000 7efde37fe700 1 -- 172.21.15.1:0/1139665089 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efde800bc10 con 0x7efdf40fff90 2023-11-16T16:31:34.602 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:34.561+0000 7efdf8d23700 1 -- 172.21.15.1:0/1139665089 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdf410a170 con 0x7efdf40fff90 2023-11-16T16:31:34.602 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:34.561+0000 7efde37fe700 1 -- 172.21.15.1:0/1139665089 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efde8003910 con 0x7efdf40fff90 2023-11-16T16:31:34.602 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:34.561+0000 7efdf8d23700 1 -- 172.21.15.1:0/1139665089 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdf4109fe0 con 0x7efdf40fff90 2023-11-16T16:31:34.603 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:34.562+0000 7efde37fe700 1 -- 172.21.15.1:0/1139665089 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7efde8003b80 con 0x7efdf40fff90 2023-11-16T16:31:34.603 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:34.562+0000 7efdf8d23700 1 -- 172.21.15.1:0/1139665089 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdf410a300 con 0x7efdf40fff90 2023-11-16T16:31:34.603 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:34.562+0000 7efde37fe700 1 --2- 172.21.15.1:0/1139665089 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7efddc0388c0 0x7efddc03ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:34.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:34.562+0000 7efde37fe700 1 -- 172.21.15.1:0/1139665089 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efde804c5c0 con 0x7efdf40fff90 2023-11-16T16:31:34.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:34.562+0000 7efdf1d9b700 1 --2- 172.21.15.1:0/1139665089 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7efddc0388c0 0x7efddc03ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:34.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:34.563+0000 7efdf1d9b700 1 --2- 172.21.15.1:0/1139665089 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7efddc0388c0 0x7efddc03ad70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efde4006fd0 tx=0x7efde4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:34.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:34.567+0000 7efde37fe700 1 -- 172.21.15.1:0/1139665089 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7efde802aba0 con 0x7efdf40fff90 2023-11-16T16:31:34.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:34 smithi001 ceph-mon[98800]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:34.771 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:34.770+0000 7efdf8d23700 1 -- 172.21.15.1:0/1139665089 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7efdf4062120 con 0x7efdf40fff90 2023-11-16T16:31:35.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:35.281+0000 7efde37fe700 1 -- 172.21.15.1:0/1139665089 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7efde8026070 con 0x7efdf40fff90 2023-11-16T16:31:35.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:35.284+0000 7efdf8d23700 1 -- 172.21.15.1:0/1139665089 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7efddc0388c0 msgr2=0x7efddc03ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:35.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:35.284+0000 7efdf8d23700 1 --2- 172.21.15.1:0/1139665089 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7efddc0388c0 0x7efddc03ad70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efde4006fd0 tx=0x7efde4006e40 comp rx=0 tx=0).stop 2023-11-16T16:31:35.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:35.284+0000 7efdf8d23700 1 -- 172.21.15.1:0/1139665089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efdf40fff90 msgr2=0x7efdf4195180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:35.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:35.284+0000 7efdf8d23700 1 --2- 172.21.15.1:0/1139665089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efdf40fff90 0x7efdf4195180 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efde8006010 tx=0x7efde8004ff0 comp rx=0 tx=0).stop 2023-11-16T16:31:35.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:35.284+0000 7efdf8d23700 1 -- 172.21.15.1:0/1139665089 shutdown_connections 2023-11-16T16:31:35.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:35.284+0000 7efdf8d23700 1 --2- 172.21.15.1:0/1139665089 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7efddc0388c0 0x7efddc03ad70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:35.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:35.284+0000 7efdf8d23700 1 --2- 172.21.15.1:0/1139665089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efdf40fff90 0x7efdf4195180 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:35.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:35.284+0000 7efdf8d23700 1 -- 172.21.15.1:0/1139665089 >> 172.21.15.1:0/1139665089 conn(0x7efdf40fb870 msgr2=0x7efdf4104170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:31:35.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:35.284+0000 7efdf8d23700 1 -- 172.21.15.1:0/1139665089 shutdown_connections 2023-11-16T16:31:35.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:35.284+0000 7efdf8d23700 1 -- 172.21.15.1:0/1139665089 wait complete. 2023-11-16T16:31:35.287 INFO:teuthology.orchestra.run.smithi001.stderr:adjusted tunables profile to default 2023-11-16T16:31:35.668 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:35 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1139665089' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-11-16T16:31:35.669 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:35 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:35.669 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:35 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:35.669 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:35 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:35.669 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:35 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:35.669 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:35 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi053", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:31:35.669 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:35 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:31:35.669 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:35 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:31:35.685 INFO:tasks.cephadm:Adding mon.a on smithi001 2023-11-16T16:31:35.685 INFO:tasks.cephadm:Adding mon.b on smithi026 2023-11-16T16:31:35.685 INFO:tasks.cephadm:Adding mon.c on smithi053 2023-11-16T16:31:35.685 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph orch apply mon '3;smithi001:172.21.15.1=a;smithi026:172.21.15.26=b;smithi053:172.21.15.53=c' 2023-11-16T16:31:36.379 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:31:36.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:36 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:31:36.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:36 smithi001 ceph-mon[98800]: Updating smithi053:/etc/ceph/ceph.conf 2023-11-16T16:31:36.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:36 smithi001 ceph-mon[98800]: Updating smithi053:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:31:36.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:36 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1139665089' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-11-16T16:31:36.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:36 smithi001 ceph-mon[98800]: osdmap e4: 0 total, 0 up, 0 in 2023-11-16T16:31:36.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:36 smithi001 ceph-mon[98800]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:36.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:36 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:36.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:36 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:36.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:36 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:37.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:37 smithi001 ceph-mon[98800]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2023-11-16T16:31:37.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:37 smithi001 ceph-mon[98800]: Updating smithi053:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-16T16:31:37.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.694+0000 7f9d7a228700 1 -- 172.21.15.53:0/2612544125 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d740fff90 msgr2=0x7f9d74100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:37.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.694+0000 7f9d7a228700 1 --2- 172.21.15.53:0/2612544125 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d740fff90 0x7f9d74100360 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9d68009910 tx=0x7f9d68009c20 comp rx=0 tx=0).stop 2023-11-16T16:31:37.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.694+0000 7f9d7a228700 1 -- 172.21.15.53:0/2612544125 shutdown_connections 2023-11-16T16:31:37.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.694+0000 7f9d7a228700 1 --2- 172.21.15.53:0/2612544125 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d740fff90 0x7f9d74100360 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:37.696 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.694+0000 7f9d7a228700 1 -- 172.21.15.53:0/2612544125 >> 172.21.15.53:0/2612544125 conn(0x7f9d740fb870 msgr2=0x7f9d740fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:31:37.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.694+0000 7f9d7a228700 1 -- 172.21.15.53:0/2612544125 shutdown_connections 2023-11-16T16:31:37.697 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.694+0000 7f9d7a228700 1 -- 172.21.15.53:0/2612544125 wait complete. 2023-11-16T16:31:37.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.695+0000 7f9d7a228700 1 Processor -- start 2023-11-16T16:31:37.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.695+0000 7f9d7a228700 1 -- start start 2023-11-16T16:31:37.698 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.695+0000 7f9d7a228700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d740fff90 0x7f9d741951b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:37.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.695+0000 7f9d7a228700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d741956f0 con 0x7f9d740fff90 2023-11-16T16:31:37.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.696+0000 7f9d737fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d740fff90 0x7f9d741951b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:37.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.696+0000 7f9d737fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d740fff90 0x7f9d741951b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.53:46156/0 (socket says 172.21.15.53:46156) 2023-11-16T16:31:37.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.696+0000 7f9d737fe700 1 -- 172.21.15.53:0/64701335 learned_addr learned my addr 172.21.15.53:0/64701335 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-16T16:31:37.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.696+0000 7f9d737fe700 1 -- 172.21.15.53:0/64701335 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d680095f0 con 0x7f9d740fff90 2023-11-16T16:31:37.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.696+0000 7f9d737fe700 1 --2- 172.21.15.53:0/64701335 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d740fff90 0x7f9d741951b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9d68004f40 tx=0x7f9d68005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:37.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.696+0000 7f9d70ff9700 1 -- 172.21.15.53:0/64701335 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9d6801d070 con 0x7f9d740fff90 2023-11-16T16:31:37.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.696+0000 7f9d7a228700 1 -- 172.21.15.53:0/64701335 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d7410a170 con 0x7f9d740fff90 2023-11-16T16:31:37.699 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.696+0000 7f9d70ff9700 1 -- 172.21.15.53:0/64701335 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d68004370 con 0x7f9d740fff90 2023-11-16T16:31:37.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.696+0000 7f9d70ff9700 1 -- 172.21.15.53:0/64701335 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9d680039b0 con 0x7f9d740fff90 2023-11-16T16:31:37.700 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.697+0000 7f9d7a228700 1 -- 172.21.15.53:0/64701335 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d74109fe0 con 0x7f9d740fff90 2023-11-16T16:31:37.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.697+0000 7f9d7a228700 1 -- 172.21.15.53:0/64701335 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d7410a300 con 0x7f9d740fff90 2023-11-16T16:31:37.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.697+0000 7f9d70ff9700 1 -- 172.21.15.53:0/64701335 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f9d68003c70 con 0x7f9d740fff90 2023-11-16T16:31:37.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.698+0000 7f9d70ff9700 1 --2- 172.21.15.53:0/64701335 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f9d5c038870 0x7f9d5c03ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:37.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.698+0000 7f9d70ff9700 1 -- 172.21.15.53:0/64701335 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9d6804c8a0 con 0x7f9d740fff90 2023-11-16T16:31:37.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.698+0000 7f9d72ffd700 1 --2- 172.21.15.53:0/64701335 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f9d5c038870 0x7f9d5c03ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:37.701 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.698+0000 7f9d72ffd700 1 --2- 172.21.15.53:0/64701335 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f9d5c038870 0x7f9d5c03ad20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9d64006fd0 tx=0x7f9d64006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:37.704 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.703+0000 7f9d70ff9700 1 -- 172.21.15.53:0/64701335 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9d68029360 con 0x7f9d740fff90 2023-11-16T16:31:37.911 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.910+0000 7f9d7a228700 1 -- 172.21.15.53:0/64701335 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi001:172.21.15.1=a;smithi026:172.21.15.26=b;smithi053:172.21.15.53=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d74108180 con 0x7f9d5c038870 2023-11-16T16:31:37.918 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.918+0000 7f9d70ff9700 1 -- 172.21.15.53:0/64701335 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f9d74108180 con 0x7f9d5c038870 2023-11-16T16:31:37.919 INFO:teuthology.orchestra.run.smithi053.stdout:Scheduled mon update... 2023-11-16T16:31:37.920 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.919+0000 7f9d7a228700 1 -- 172.21.15.53:0/64701335 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f9d5c038870 msgr2=0x7f9d5c03ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:37.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.919+0000 7f9d7a228700 1 --2- 172.21.15.53:0/64701335 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f9d5c038870 0x7f9d5c03ad20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9d64006fd0 tx=0x7f9d64006e40 comp rx=0 tx=0).stop 2023-11-16T16:31:37.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.919+0000 7f9d7a228700 1 -- 172.21.15.53:0/64701335 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d740fff90 msgr2=0x7f9d741951b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:37.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.919+0000 7f9d7a228700 1 --2- 172.21.15.53:0/64701335 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d740fff90 0x7f9d741951b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9d68004f40 tx=0x7f9d68005020 comp rx=0 tx=0).stop 2023-11-16T16:31:37.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.919+0000 7f9d7a228700 1 -- 172.21.15.53:0/64701335 shutdown_connections 2023-11-16T16:31:37.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.919+0000 7f9d7a228700 1 --2- 172.21.15.53:0/64701335 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f9d5c038870 0x7f9d5c03ad20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:37.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.919+0000 7f9d7a228700 1 --2- 172.21.15.53:0/64701335 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9d740fff90 0x7f9d741951b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:37.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.919+0000 7f9d7a228700 1 -- 172.21.15.53:0/64701335 >> 172.21.15.53:0/64701335 conn(0x7f9d740fb870 msgr2=0x7f9d74104170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:31:37.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.919+0000 7f9d7a228700 1 -- 172.21.15.53:0/64701335 shutdown_connections 2023-11-16T16:31:37.921 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:37.919+0000 7f9d7a228700 1 -- 172.21.15.53:0/64701335 wait complete. 2023-11-16T16:31:38.439 DEBUG:teuthology.orchestra.run.smithi026:mon.b> sudo journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mon.b.service 2023-11-16T16:31:38.442 DEBUG:teuthology.orchestra.run.smithi053:mon.c> sudo journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mon.c.service 2023-11-16T16:31:38.445 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-11-16T16:31:38.445 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph mon dump -f json 2023-11-16T16:31:38.477 INFO:journalctl@ceph.mon.c.smithi053.stdout:-- Logs begin at Thu 2023-11-16 16:14:15 UTC. -- 2023-11-16T16:31:38.479 INFO:journalctl@ceph.mon.b.smithi026.stdout:-- Logs begin at Thu 2023-11-16 16:14:13 UTC. -- 2023-11-16T16:31:38.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:38 smithi001 ceph-mon[98800]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:38.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:38 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:38.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:38 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:31:38.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:38 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:31:38.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:38 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:31:38.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:38 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:38.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:38 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-16T16:31:38.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:38 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:31:39.375 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:31:39.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:39 smithi001 ceph-mon[98800]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi001:172.21.15.1=a;smithi026:172.21.15.26=b;smithi053:172.21.15.53=c", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:31:39.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:39 smithi001 ceph-mon[98800]: Saving service mon spec with placement smithi001:172.21.15.1=a;smithi026:172.21.15.26=b;smithi053:172.21.15.53=c;count:3 2023-11-16T16:31:39.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:39 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:31:39.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:39 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:31:39.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:39 smithi001 ceph-mon[98800]: Deploying daemon mon.c on smithi053 2023-11-16T16:31:40.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:40 smithi001 ceph-mon[98800]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:41.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.204+0000 7f053f176700 1 -- 172.21.15.53:0/624507000 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05380fff90 msgr2=0x7f0538100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:41.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.204+0000 7f053f176700 1 --2- 172.21.15.53:0/624507000 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05380fff90 0x7f0538100360 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f052c009910 tx=0x7f052c009c20 comp rx=0 tx=0).stop 2023-11-16T16:31:41.207 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.204+0000 7f053f176700 1 -- 172.21.15.53:0/624507000 shutdown_connections 2023-11-16T16:31:41.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.204+0000 7f053f176700 1 --2- 172.21.15.53:0/624507000 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05380fff90 0x7f0538100360 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:41.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.204+0000 7f053f176700 1 -- 172.21.15.53:0/624507000 >> 172.21.15.53:0/624507000 conn(0x7f05380fb870 msgr2=0x7f05380fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:31:41.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.204+0000 7f053f176700 1 -- 172.21.15.53:0/624507000 shutdown_connections 2023-11-16T16:31:41.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.204+0000 7f053f176700 1 -- 172.21.15.53:0/624507000 wait complete. 2023-11-16T16:31:41.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.205+0000 7f053f176700 1 Processor -- start 2023-11-16T16:31:41.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.205+0000 7f053f176700 1 -- start start 2023-11-16T16:31:41.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.205+0000 7f053f176700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05380fff90 0x7f0538195180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:41.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.205+0000 7f053f176700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05381956c0 con 0x7f05380fff90 2023-11-16T16:31:41.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.206+0000 7f053cf12700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05380fff90 0x7f0538195180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:41.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.206+0000 7f053cf12700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05380fff90 0x7f0538195180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.53:46162/0 (socket says 172.21.15.53:46162) 2023-11-16T16:31:41.208 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.206+0000 7f053cf12700 1 -- 172.21.15.53:0/3037447447 learned_addr learned my addr 172.21.15.53:0/3037447447 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-16T16:31:41.209 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.206+0000 7f053cf12700 1 -- 172.21.15.53:0/3037447447 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f052c0095f0 con 0x7f05380fff90 2023-11-16T16:31:41.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.206+0000 7f053cf12700 1 --2- 172.21.15.53:0/3037447447 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05380fff90 0x7f0538195180 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f052c004f40 tx=0x7f052c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:41.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.207+0000 7f0535ffb700 1 -- 172.21.15.53:0/3037447447 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f052c01d070 con 0x7f05380fff90 2023-11-16T16:31:41.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.207+0000 7f053f176700 1 -- 172.21.15.53:0/3037447447 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f053810a170 con 0x7f05380fff90 2023-11-16T16:31:41.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.207+0000 7f0535ffb700 1 -- 172.21.15.53:0/3037447447 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f052c004370 con 0x7f05380fff90 2023-11-16T16:31:41.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.207+0000 7f0535ffb700 1 -- 172.21.15.53:0/3037447447 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f052c0039b0 con 0x7f05380fff90 2023-11-16T16:31:41.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.207+0000 7f053f176700 1 -- 172.21.15.53:0/3037447447 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0538109fe0 con 0x7f05380fff90 2023-11-16T16:31:41.210 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.208+0000 7f053f176700 1 -- 172.21.15.53:0/3037447447 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f053810a300 con 0x7f05380fff90 2023-11-16T16:31:41.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.208+0000 7f0535ffb700 1 -- 172.21.15.53:0/3037447447 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f052c003c70 con 0x7f05380fff90 2023-11-16T16:31:41.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.208+0000 7f0535ffb700 1 --2- 172.21.15.53:0/3037447447 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f0528038870 0x7f052803ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:41.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.208+0000 7f0535ffb700 1 -- 172.21.15.53:0/3037447447 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f052c04c590 con 0x7f05380fff90 2023-11-16T16:31:41.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.209+0000 7f0537fff700 1 --2- 172.21.15.53:0/3037447447 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f0528038870 0x7f052803ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:41.211 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.209+0000 7f0537fff700 1 --2- 172.21.15.53:0/3037447447 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f0528038870 0x7f052803ad20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0524006fd0 tx=0x7f0524006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:41.216 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.215+0000 7f0535ffb700 1 -- 172.21.15.53:0/3037447447 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f052c029770 con 0x7f05380fff90 2023-11-16T16:31:41.511 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.509+0000 7f053f176700 1 -- 172.21.15.53:0/3037447447 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0538062120 con 0x7f05380fff90 2023-11-16T16:31:41.512 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.510+0000 7f0535ffb700 1 -- 172.21.15.53:0/3037447447 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+732 (secure 0 0 0) 0x7f052c026070 con 0x7f05380fff90 2023-11-16T16:31:41.512 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-16T16:31:41.512 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":1,"fsid":"35d9b010-849d-11ee-8dbe-212e2dc638e7","modified":"2023-11-16T16:30:02.658115Z","created":"2023-11-16T16:30:02.658115Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-11-16T16:31:41.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.512+0000 7f053f176700 1 -- 172.21.15.53:0/3037447447 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f0528038870 msgr2=0x7f052803ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:41.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.512+0000 7f053f176700 1 --2- 172.21.15.53:0/3037447447 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f0528038870 0x7f052803ad20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0524006fd0 tx=0x7f0524006e40 comp rx=0 tx=0).stop 2023-11-16T16:31:41.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.512+0000 7f053f176700 1 -- 172.21.15.53:0/3037447447 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05380fff90 msgr2=0x7f0538195180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:41.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.512+0000 7f053f176700 1 --2- 172.21.15.53:0/3037447447 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05380fff90 0x7f0538195180 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f052c004f40 tx=0x7f052c005020 comp rx=0 tx=0).stop 2023-11-16T16:31:41.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.512+0000 7f053f176700 1 -- 172.21.15.53:0/3037447447 shutdown_connections 2023-11-16T16:31:41.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.512+0000 7f053f176700 1 --2- 172.21.15.53:0/3037447447 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f0528038870 0x7f052803ad20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:41.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.512+0000 7f053f176700 1 --2- 172.21.15.53:0/3037447447 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05380fff90 0x7f0538195180 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:41.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.512+0000 7f053f176700 1 -- 172.21.15.53:0/3037447447 >> 172.21.15.53:0/3037447447 conn(0x7f05380fb870 msgr2=0x7f0538104170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:31:41.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.512+0000 7f053f176700 1 -- 172.21.15.53:0/3037447447 shutdown_connections 2023-11-16T16:31:41.514 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:41.512+0000 7f053f176700 1 -- 172.21.15.53:0/3037447447 wait complete. 2023-11-16T16:31:41.515 INFO:teuthology.orchestra.run.smithi053.stderr:dumped monmap epoch 1 2023-11-16T16:31:41.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 podman[100576]: 2023-11-16T16:31:42.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 bash[100576]: f3182d4a36b4733789ee0f66de46d91fa9444af4dc8e625edcd669468b233bff 2023-11-16T16:31:42.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: set uid:gid to 167:167 (ceph:ceph) 2023-11-16T16:31:42.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: ceph version 18.0.0-3905-g5e4a743a (5e4a743ad7fd1422ebd6c4dc950a21a303279207) reef (dev), process ceph-mon, pid 7 2023-11-16T16:31:42.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: pidfile_write: ignore empty --pid-file 2023-11-16T16:31:42.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: load: jerasure load: lrc 2023-11-16T16:31:42.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: RocksDB version: 7.9.2 2023-11-16T16:31:42.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Git sha 0 2023-11-16T16:31:42.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Compile date 2023-11-15 23:53:54 2023-11-16T16:31:42.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: DB SUMMARY 2023-11-16T16:31:42.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: DB Session ID: SF2HXNL51FSYEKN7ZUOX 2023-11-16T16:31:42.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: CURRENT file: CURRENT 2023-11-16T16:31:42.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: IDENTITY file: IDENTITY 2023-11-16T16:31:42.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-11-16T16:31:42.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-11-16T16:31:42.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-11-16T16:31:42.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.error_if_exists: 0 2023-11-16T16:31:42.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.create_if_missing: 0 2023-11-16T16:31:42.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.paranoid_checks: 1 2023-11-16T16:31:42.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.flush_verify_memtable_count: 1 2023-11-16T16:31:42.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-11-16T16:31:42.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-11-16T16:31:42.334 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.env: 0x555b81aa3720 2023-11-16T16:31:42.334 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.fs: PosixFileSystem 2023-11-16T16:31:42.334 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.info_log: 0x555b839e73c0 2023-11-16T16:31:42.334 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.max_file_opening_threads: 16 2023-11-16T16:31:42.334 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.statistics: (nil) 2023-11-16T16:31:42.334 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.use_fsync: 0 2023-11-16T16:31:42.334 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.max_log_file_size: 0 2023-11-16T16:31:42.334 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-11-16T16:31:42.334 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.log_file_time_to_roll: 0 2023-11-16T16:31:42.334 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.keep_log_file_num: 1000 2023-11-16T16:31:42.334 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.recycle_log_file_num: 0 2023-11-16T16:31:42.334 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.allow_fallocate: 1 2023-11-16T16:31:42.334 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.allow_mmap_reads: 0 2023-11-16T16:31:42.335 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.allow_mmap_writes: 0 2023-11-16T16:31:42.335 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.use_direct_reads: 0 2023-11-16T16:31:42.335 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-11-16T16:31:42.335 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.create_missing_column_families: 0 2023-11-16T16:31:42.335 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.db_log_dir: 2023-11-16T16:31:42.335 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.wal_dir: 2023-11-16T16:31:42.335 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.table_cache_numshardbits: 6 2023-11-16T16:31:42.335 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.WAL_ttl_seconds: 0 2023-11-16T16:31:42.335 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.WAL_size_limit_MB: 0 2023-11-16T16:31:42.335 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-11-16T16:31:42.335 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-11-16T16:31:42.335 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.is_fd_close_on_exec: 1 2023-11-16T16:31:42.335 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.advise_random_on_open: 1 2023-11-16T16:31:42.335 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.db_write_buffer_size: 0 2023-11-16T16:31:42.335 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.write_buffer_manager: 0x555b82c583c0 2023-11-16T16:31:42.336 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-11-16T16:31:42.336 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-11-16T16:31:42.336 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.use_adaptive_mutex: 0 2023-11-16T16:31:42.336 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.rate_limiter: (nil) 2023-11-16T16:31:42.336 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-11-16T16:31:42.336 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.wal_recovery_mode: 2 2023-11-16T16:31:42.336 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.enable_thread_tracking: 0 2023-11-16T16:31:42.336 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.enable_pipelined_write: 0 2023-11-16T16:31:42.336 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.unordered_write: 0 2023-11-16T16:31:42.336 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-11-16T16:31:42.336 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-11-16T16:31:42.336 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-11-16T16:31:42.336 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-11-16T16:31:42.336 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.row_cache: None 2023-11-16T16:31:42.336 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.wal_filter: None 2023-11-16T16:31:42.337 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-11-16T16:31:42.337 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.allow_ingest_behind: 0 2023-11-16T16:31:42.337 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.two_write_queues: 0 2023-11-16T16:31:42.337 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.manual_wal_flush: 0 2023-11-16T16:31:42.337 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.wal_compression: 0 2023-11-16T16:31:42.337 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.atomic_flush: 0 2023-11-16T16:31:42.337 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-11-16T16:31:42.337 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.persist_stats_to_disk: 0 2023-11-16T16:31:42.337 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.write_dbid_to_manifest: 0 2023-11-16T16:31:42.337 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.log_readahead_size: 0 2023-11-16T16:31:42.337 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-11-16T16:31:42.337 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.best_efforts_recovery: 0 2023-11-16T16:31:42.337 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-11-16T16:31:42.337 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-11-16T16:31:42.338 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.allow_data_in_errors: 0 2023-11-16T16:31:42.338 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.db_host_id: __hostname__ 2023-11-16T16:31:42.338 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.enforce_single_del_contracts: true 2023-11-16T16:31:42.338 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.max_background_jobs: 2 2023-11-16T16:31:42.338 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.max_background_compactions: -1 2023-11-16T16:31:42.338 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.max_subcompactions: 1 2023-11-16T16:31:42.338 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-11-16T16:31:42.338 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-11-16T16:31:42.338 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.delayed_write_rate : 16777216 2023-11-16T16:31:42.338 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.max_total_wal_size: 0 2023-11-16T16:31:42.338 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-11-16T16:31:42.338 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.stats_dump_period_sec: 600 2023-11-16T16:31:42.338 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.stats_persist_period_sec: 600 2023-11-16T16:31:42.339 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-11-16T16:31:42.339 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.max_open_files: -1 2023-11-16T16:31:42.339 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.bytes_per_sync: 0 2023-11-16T16:31:42.339 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.wal_bytes_per_sync: 0 2023-11-16T16:31:42.339 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.strict_bytes_per_sync: 0 2023-11-16T16:31:42.339 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.compaction_readahead_size: 0 2023-11-16T16:31:42.339 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.max_background_flushes: -1 2023-11-16T16:31:42.339 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Compression algorithms supported: 2023-11-16T16:31:42.339 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-11-16T16:31:42.339 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: kZSTD supported: 0 2023-11-16T16:31:42.339 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: kXpressCompression supported: 0 2023-11-16T16:31:42.339 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: kLZ4HCCompression supported: 1 2023-11-16T16:31:42.339 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: kZlibCompression supported: 1 2023-11-16T16:31:42.339 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: kSnappyCompression supported: 1 2023-11-16T16:31:42.340 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: kLZ4Compression supported: 1 2023-11-16T16:31:42.340 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: kBZip2Compression supported: 0 2023-11-16T16:31:42.340 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Fast CRC32 supported: Supported on x86 2023-11-16T16:31:42.340 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: DMutex implementation: pthread_mutex_t 2023-11-16T16:31:42.340 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2023-11-16T16:31:42.340 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-11-16T16:31:42.340 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-11-16T16:31:42.340 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.merge_operator: 2023-11-16T16:31:42.340 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.compaction_filter: None 2023-11-16T16:31:42.340 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.compaction_filter_factory: None 2023-11-16T16:31:42.340 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.sst_partitioner_factory: None 2023-11-16T16:31:42.340 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.memtable_factory: SkipListFactory 2023-11-16T16:31:42.340 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.table_factory: BlockBasedTable 2023-11-16T16:31:42.341 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x555b839e74e0) 2023-11-16T16:31:42.341 INFO:journalctl@ceph.mon.c.smithi053.stdout: cache_index_and_filter_blocks: 1 2023-11-16T16:31:42.341 INFO:journalctl@ceph.mon.c.smithi053.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-11-16T16:31:42.341 INFO:journalctl@ceph.mon.c.smithi053.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-11-16T16:31:42.341 INFO:journalctl@ceph.mon.c.smithi053.stdout: pin_top_level_index_and_filter: 1 2023-11-16T16:31:42.341 INFO:journalctl@ceph.mon.c.smithi053.stdout: index_type: 0 2023-11-16T16:31:42.341 INFO:journalctl@ceph.mon.c.smithi053.stdout: data_block_index_type: 0 2023-11-16T16:31:42.341 INFO:journalctl@ceph.mon.c.smithi053.stdout: index_shortening: 1 2023-11-16T16:31:42.341 INFO:journalctl@ceph.mon.c.smithi053.stdout: data_block_hash_table_util_ratio: 0.750000 2023-11-16T16:31:42.341 INFO:journalctl@ceph.mon.c.smithi053.stdout: checksum: 4 2023-11-16T16:31:42.341 INFO:journalctl@ceph.mon.c.smithi053.stdout: no_block_cache: 0 2023-11-16T16:31:42.341 INFO:journalctl@ceph.mon.c.smithi053.stdout: block_cache: 0x555b82cf91f0 2023-11-16T16:31:42.341 INFO:journalctl@ceph.mon.c.smithi053.stdout: block_cache_name: BinnedLRUCache 2023-11-16T16:31:42.341 INFO:journalctl@ceph.mon.c.smithi053.stdout: block_cache_options: 2023-11-16T16:31:42.341 INFO:journalctl@ceph.mon.c.smithi053.stdout: capacity : 536870912 2023-11-16T16:31:42.342 INFO:journalctl@ceph.mon.c.smithi053.stdout: num_shard_bits : 4 2023-11-16T16:31:42.342 INFO:journalctl@ceph.mon.c.smithi053.stdout: strict_capacity_limit : 0 2023-11-16T16:31:42.342 INFO:journalctl@ceph.mon.c.smithi053.stdout: high_pri_pool_ratio: 0.000 2023-11-16T16:31:42.342 INFO:journalctl@ceph.mon.c.smithi053.stdout: block_cache_compressed: (nil) 2023-11-16T16:31:42.342 INFO:journalctl@ceph.mon.c.smithi053.stdout: persistent_cache: (nil) 2023-11-16T16:31:42.342 INFO:journalctl@ceph.mon.c.smithi053.stdout: block_size: 4096 2023-11-16T16:31:42.342 INFO:journalctl@ceph.mon.c.smithi053.stdout: block_size_deviation: 10 2023-11-16T16:31:42.342 INFO:journalctl@ceph.mon.c.smithi053.stdout: block_restart_interval: 16 2023-11-16T16:31:42.342 INFO:journalctl@ceph.mon.c.smithi053.stdout: index_block_restart_interval: 1 2023-11-16T16:31:42.342 INFO:journalctl@ceph.mon.c.smithi053.stdout: metadata_block_size: 4096 2023-11-16T16:31:42.342 INFO:journalctl@ceph.mon.c.smithi053.stdout: partition_filters: 0 2023-11-16T16:31:42.342 INFO:journalctl@ceph.mon.c.smithi053.stdout: use_delta_encoding: 1 2023-11-16T16:31:42.342 INFO:journalctl@ceph.mon.c.smithi053.stdout: filter_policy: bloomfilter 2023-11-16T16:31:42.342 INFO:journalctl@ceph.mon.c.smithi053.stdout: whole_key_filtering: 1 2023-11-16T16:31:42.343 INFO:journalctl@ceph.mon.c.smithi053.stdout: verify_compression: 0 2023-11-16T16:31:42.343 INFO:journalctl@ceph.mon.c.smithi053.stdout: read_amp_bytes_per_bit: 0 2023-11-16T16:31:42.343 INFO:journalctl@ceph.mon.c.smithi053.stdout: format_version: 5 2023-11-16T16:31:42.343 INFO:journalctl@ceph.mon.c.smithi053.stdout: enable_index_compression: 1 2023-11-16T16:31:42.343 INFO:journalctl@ceph.mon.c.smithi053.stdout: block_align: 0 2023-11-16T16:31:42.343 INFO:journalctl@ceph.mon.c.smithi053.stdout: max_auto_readahead_size: 262144 2023-11-16T16:31:42.343 INFO:journalctl@ceph.mon.c.smithi053.stdout: prepopulate_block_cache: 0 2023-11-16T16:31:42.343 INFO:journalctl@ceph.mon.c.smithi053.stdout: initial_auto_readahead_size: 8192 2023-11-16T16:31:42.343 INFO:journalctl@ceph.mon.c.smithi053.stdout: num_file_reads_for_auto_readahead: 2 2023-11-16T16:31:42.343 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.write_buffer_size: 33554432 2023-11-16T16:31:42.343 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.max_write_buffer_number: 2 2023-11-16T16:31:42.343 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.compression: NoCompression 2023-11-16T16:31:42.343 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.bottommost_compression: Disabled 2023-11-16T16:31:42.343 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.prefix_extractor: nullptr 2023-11-16T16:31:42.343 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-11-16T16:31:42.344 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.num_levels: 7 2023-11-16T16:31:42.344 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-11-16T16:31:42.344 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-11-16T16:31:42.344 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-11-16T16:31:42.344 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-11-16T16:31:42.344 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-11-16T16:31:42.344 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-11-16T16:31:42.344 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-11-16T16:31:42.344 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-11-16T16:31:42.344 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-11-16T16:31:42.344 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-11-16T16:31:42.344 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-11-16T16:31:42.344 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-11-16T16:31:42.344 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.compression_opts.window_bits: -14 2023-11-16T16:31:42.344 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.compression_opts.level: 32767 2023-11-16T16:31:42.345 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.compression_opts.strategy: 0 2023-11-16T16:31:42.345 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-11-16T16:31:42.345 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-11-16T16:31:42.345 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-11-16T16:31:42.345 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-11-16T16:31:42.345 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.compression_opts.enabled: false 2023-11-16T16:31:42.345 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-11-16T16:31:42.345 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-11-16T16:31:42.345 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-11-16T16:31:42.345 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-11-16T16:31:42.345 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.target_file_size_base: 67108864 2023-11-16T16:31:42.345 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.target_file_size_multiplier: 1 2023-11-16T16:31:42.346 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-11-16T16:31:42.346 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-11-16T16:31:42.346 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-11-16T16:31:42.346 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-11-16T16:31:42.346 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-11-16T16:31:42.346 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-11-16T16:31:42.346 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-11-16T16:31:42.346 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-11-16T16:31:42.346 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-11-16T16:31:42.346 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-11-16T16:31:42.346 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-11-16T16:31:42.346 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-11-16T16:31:42.346 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-11-16T16:31:42.346 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.arena_block_size: 1048576 2023-11-16T16:31:42.347 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-11-16T16:31:42.347 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-11-16T16:31:42.347 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.disable_auto_compactions: 0 2023-11-16T16:31:42.348 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-11-16T16:31:42.348 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-11-16T16:31:42.348 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-11-16T16:31:42.348 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-11-16T16:31:42.348 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-11-16T16:31:42.348 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-11-16T16:31:42.348 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-11-16T16:31:42.348 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-11-16T16:31:42.348 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-11-16T16:31:42.348 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-11-16T16:31:42.348 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.table_properties_collectors: 2023-11-16T16:31:42.348 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.inplace_update_support: 0 2023-11-16T16:31:42.348 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.inplace_update_num_locks: 10000 2023-11-16T16:31:42.349 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-11-16T16:31:42.349 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-11-16T16:31:42.349 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.memtable_huge_page_size: 0 2023-11-16T16:31:42.349 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.bloom_locality: 0 2023-11-16T16:31:42.349 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.max_successive_merges: 0 2023-11-16T16:31:42.349 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.optimize_filters_for_hits: 0 2023-11-16T16:31:42.349 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.paranoid_file_checks: 0 2023-11-16T16:31:42.349 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.force_consistency_checks: 1 2023-11-16T16:31:42.349 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.report_bg_io_stats: 0 2023-11-16T16:31:42.349 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.ttl: 2592000 2023-11-16T16:31:42.349 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.periodic_compaction_seconds: 0 2023-11-16T16:31:42.349 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-11-16T16:31:42.349 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-11-16T16:31:42.349 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.enable_blob_files: false 2023-11-16T16:31:42.350 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.min_blob_size: 0 2023-11-16T16:31:42.350 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.blob_file_size: 268435456 2023-11-16T16:31:42.350 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.blob_compression_type: NoCompression 2023-11-16T16:31:42.350 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.enable_blob_garbage_collection: false 2023-11-16T16:31:42.350 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-11-16T16:31:42.350 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-11-16T16:31:42.350 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-11-16T16:31:42.350 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.blob_file_starting_level: 0 2023-11-16T16:31:42.350 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-11-16T16:31:42.350 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-11-16T16:31:42.350 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-11-16T16:31:42.350 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 324a0f4c-5cd9-4bdd-b449-f22c44f85a56 2023-11-16T16:31:42.350 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: EVENT_LOG_v1 {"time_micros": 1700152301895072, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-11-16T16:31:42.351 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-11-16T16:31:42.351 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: EVENT_LOG_v1 {"time_micros": 1700152301895499, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1700152301, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "324a0f4c-5cd9-4bdd-b449-f22c44f85a56", "db_session_id": "SF2HXNL51FSYEKN7ZUOX", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-11-16T16:31:42.351 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: EVENT_LOG_v1 {"time_micros": 1700152301895577, "job": 1, "event": "recovery_finished"} 2023-11-16T16:31:42.351 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-11-16T16:31:42.351 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-11-16T16:31:42.351 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x555b82d96000 2023-11-16T16:31:42.351 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: DB pointer 0x555b82d82000 2023-11-16T16:31:42.351 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-11-16T16:31:42.351 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-11-16T16:31:42.351 INFO:journalctl@ceph.mon.c.smithi053.stdout: ** DB Stats ** 2023-11-16T16:31:42.351 INFO:journalctl@ceph.mon.c.smithi053.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-16T16:31:42.351 INFO:journalctl@ceph.mon.c.smithi053.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-11-16T16:31:42.351 INFO:journalctl@ceph.mon.c.smithi053.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-16T16:31:42.351 INFO:journalctl@ceph.mon.c.smithi053.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-16T16:31:42.352 INFO:journalctl@ceph.mon.c.smithi053.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-11-16T16:31:42.352 INFO:journalctl@ceph.mon.c.smithi053.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-16T16:31:42.352 INFO:journalctl@ceph.mon.c.smithi053.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-16T16:31:42.352 INFO:journalctl@ceph.mon.c.smithi053.stdout: 2023-11-16T16:31:42.352 INFO:journalctl@ceph.mon.c.smithi053.stdout: ** Compaction Stats [default] ** 2023-11-16T16:31:42.352 INFO:journalctl@ceph.mon.c.smithi053.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-16T16:31:42.352 INFO:journalctl@ceph.mon.c.smithi053.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-11-16T16:31:42.352 INFO:journalctl@ceph.mon.c.smithi053.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-16T16:31:42.352 INFO:journalctl@ceph.mon.c.smithi053.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-16T16:31:42.352 INFO:journalctl@ceph.mon.c.smithi053.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-16T16:31:42.352 INFO:journalctl@ceph.mon.c.smithi053.stdout: 2023-11-16T16:31:42.352 INFO:journalctl@ceph.mon.c.smithi053.stdout: ** Compaction Stats [default] ** 2023-11-16T16:31:42.352 INFO:journalctl@ceph.mon.c.smithi053.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-16T16:31:42.352 INFO:journalctl@ceph.mon.c.smithi053.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-11-16T16:31:42.353 INFO:journalctl@ceph.mon.c.smithi053.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-16T16:31:42.353 INFO:journalctl@ceph.mon.c.smithi053.stdout: 2023-11-16T16:31:42.353 INFO:journalctl@ceph.mon.c.smithi053.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-11-16T16:31:42.353 INFO:journalctl@ceph.mon.c.smithi053.stdout: 2023-11-16T16:31:42.353 INFO:journalctl@ceph.mon.c.smithi053.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-16T16:31:42.353 INFO:journalctl@ceph.mon.c.smithi053.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-11-16T16:31:42.353 INFO:journalctl@ceph.mon.c.smithi053.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-11-16T16:31:42.353 INFO:journalctl@ceph.mon.c.smithi053.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-11-16T16:31:42.353 INFO:journalctl@ceph.mon.c.smithi053.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-11-16T16:31:42.353 INFO:journalctl@ceph.mon.c.smithi053.stdout: AddFile(Keys): cumulative 0, interval 0 2023-11-16T16:31:42.353 INFO:journalctl@ceph.mon.c.smithi053.stdout: Cumulative compaction: 0.00 GB write, 0.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-16T16:31:42.353 INFO:journalctl@ceph.mon.c.smithi053.stdout: Interval compaction: 0.00 GB write, 0.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-16T16:31:42.353 INFO:journalctl@ceph.mon.c.smithi053.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-11-16T16:31:42.353 INFO:journalctl@ceph.mon.c.smithi053.stdout: Block cache BinnedLRUCache@0x555b82cf91f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-11-16T16:31:42.353 INFO:journalctl@ceph.mon.c.smithi053.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-11-16T16:31:42.354 INFO:journalctl@ceph.mon.c.smithi053.stdout: 2023-11-16T16:31:42.354 INFO:journalctl@ceph.mon.c.smithi053.stdout: ** File Read Latency Histogram By Level [default] ** 2023-11-16T16:31:42.354 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-11-16T16:31:42.354 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: using public_addr v2:172.21.15.53:0/0 -> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] 2023-11-16T16:31:42.354 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: starting mon.c rank -1 at public addrs [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] at bind addrs [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 2023-11-16T16:31:42.354 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: mon.c@-1(???) e0 preinit fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 2023-11-16T16:31:42.354 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: mon.c@-1(synchronizing).mds e1 new map 2023-11-16T16:31:42.354 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: mon.c@-1(synchronizing).mds e1 print_map 2023-11-16T16:31:42.354 INFO:journalctl@ceph.mon.c.smithi053.stdout: e1 2023-11-16T16:31:42.354 INFO:journalctl@ceph.mon.c.smithi053.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-11-16T16:31:42.354 INFO:journalctl@ceph.mon.c.smithi053.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-11-16T16:31:42.354 INFO:journalctl@ceph.mon.c.smithi053.stdout: legacy client fscid: -1 2023-11-16T16:31:42.354 INFO:journalctl@ceph.mon.c.smithi053.stdout: 2023-11-16T16:31:42.354 INFO:journalctl@ceph.mon.c.smithi053.stdout: No filesystems configured 2023-11-16T16:31:42.355 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-11-16T16:31:42.355 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-11-16T16:31:42.355 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-11-16T16:31:42.355 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-11-16T16:31:42.355 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-11-16T16:31:42.355 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-11-16T16:31:42.355 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-11-16T16:31:42.355 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-11-16T16:31:42.355 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-11-16T16:31:42.355 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-11-16T16:31:42.355 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: Added host smithi026 2023-11-16T16:31:42.355 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:42.355 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:42.355 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:42.355 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:31:42.356 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:42.356 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:42.356 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:42.356 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:42.356 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:42.356 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi026", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:31:42.356 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:31:42.356 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:31:42.356 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:31:42.356 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: Updating smithi026:/etc/ceph/ceph.conf 2023-11-16T16:31:42.356 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi053", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:31:42.356 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: Updating smithi026:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:31:42.356 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:42.356 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: Updating smithi026:/etc/ceph/ceph.client.admin.keyring 2023-11-16T16:31:42.357 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: Deploying cephadm binary to smithi053 2023-11-16T16:31:42.357 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: Updating smithi026:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-16T16:31:42.357 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:42.357 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:42.357 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:42.357 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:42.357 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:42.357 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:31:42.357 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: Added host smithi053 2023-11-16T16:31:42.357 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:42.357 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:42.357 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:42.357 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:31:42.357 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:42.358 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:42.358 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1139665089' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-11-16T16:31:42.358 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:42.358 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:42.358 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:42.358 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:42.358 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi053", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:31:42.358 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:31:42.358 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:31:42.358 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:31:42.358 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: Updating smithi053:/etc/ceph/ceph.conf 2023-11-16T16:31:42.358 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: Updating smithi053:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:31:42.358 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1139665089' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-11-16T16:31:42.358 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: osdmap e4: 0 total, 0 up, 0 in 2023-11-16T16:31:42.359 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:42.359 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:42.359 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:42.359 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:42.359 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2023-11-16T16:31:42.359 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: Updating smithi053:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-16T16:31:42.359 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:42.359 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:42.359 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:31:42.359 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:31:42.359 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:31:42.359 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:42.359 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-16T16:31:42.359 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:31:42.360 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi001:172.21.15.1=a;smithi026:172.21.15.26=b;smithi053:172.21.15.53=c", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:31:42.360 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: Saving service mon spec with placement smithi001:172.21.15.1=a;smithi026:172.21.15.26=b;smithi053:172.21.15.53=c;count:3 2023-11-16T16:31:42.360 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:31:42.360 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:31:42.360 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: Deploying daemon mon.c on smithi053 2023-11-16T16:31:42.527 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:42.528 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-11-16T16:31:42.528 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 ceph-mon[100638]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-11-16T16:31:42.528 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:41 smithi053 systemd[1]: Started Ceph mon.c for 35d9b010-849d-11ee-8dbe-212e2dc638e7. 2023-11-16T16:31:42.999 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-11-16T16:31:42.999 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph mon dump -f json 2023-11-16T16:31:43.664 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-16T16:31:47.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:47 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:31:47.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:47 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T16:31:47.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:47 smithi053 ceph-mon[100638]: mon.a calling monitor election 2023-11-16T16:31:47.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:47 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:31:47.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:47 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:31:47.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:47 smithi053 ceph-mon[100638]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:47.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:47 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:31:47.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:47 smithi053 ceph-mon[100638]: mon.c calling monitor election 2023-11-16T16:31:47.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:47 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:31:47.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:47 smithi053 ceph-mon[100638]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:47.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:47 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:31:47.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:47 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:31:47.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:47 smithi053 ceph-mon[100638]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-11-16T16:31:47.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:47 smithi053 ceph-mon[100638]: monmap e2: 2 mons at {a=[v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0],c=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0]} removed_ranks: {} 2023-11-16T16:31:47.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:47 smithi053 ceph-mon[100638]: fsmap 2023-11-16T16:31:47.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:47 smithi053 ceph-mon[100638]: osdmap e4: 0 total, 0 up, 0 in 2023-11-16T16:31:47.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:47 smithi053 ceph-mon[100638]: mgrmap e13: x(active, since 45s) 2023-11-16T16:31:47.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:47 smithi053 ceph-mon[100638]: overall HEALTH_OK 2023-11-16T16:31:47.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:47 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:47.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:47 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:47.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:47 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:47.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:47 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-16T16:31:47.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:47 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:31:47.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:47 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:31:47.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:47 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T16:31:47.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:47 smithi001 ceph-mon[98800]: mon.a calling monitor election 2023-11-16T16:31:47.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:47 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:31:47.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:47 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:31:47.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:47 smithi001 ceph-mon[98800]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:47.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:47 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:31:47.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:47 smithi001 ceph-mon[98800]: mon.c calling monitor election 2023-11-16T16:31:47.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:47 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:31:47.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:47 smithi001 ceph-mon[98800]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:47.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:47 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:31:47.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:47 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:31:47.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:47 smithi001 ceph-mon[98800]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-11-16T16:31:47.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:47 smithi001 ceph-mon[98800]: monmap e2: 2 mons at {a=[v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0],c=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0]} removed_ranks: {} 2023-11-16T16:31:47.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:47 smithi001 ceph-mon[98800]: fsmap 2023-11-16T16:31:47.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:47 smithi001 ceph-mon[98800]: osdmap e4: 0 total, 0 up, 0 in 2023-11-16T16:31:47.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:47 smithi001 ceph-mon[98800]: mgrmap e13: x(active, since 45s) 2023-11-16T16:31:47.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:47 smithi001 ceph-mon[98800]: overall HEALTH_OK 2023-11-16T16:31:47.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:47 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:47.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:47 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:47.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:47 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:47.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:47 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-16T16:31:47.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:47 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:31:47.979 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:47.976+0000 7f6aa54bc700 1 -- 172.21.15.53:0/1145901058 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a880035c0 msgr2=0x7f6a88005a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:47.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:47.976+0000 7f6aa54bc700 1 --2- 172.21.15.53:0/1145901058 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a880035c0 0x7f6a88005a40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6aa00fec70 tx=0x7f6a9000feb0 comp rx=0 tx=0).stop 2023-11-16T16:31:47.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:47.977+0000 7f6aa54bc700 1 -- 172.21.15.53:0/1145901058 shutdown_connections 2023-11-16T16:31:47.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:47.977+0000 7f6aa54bc700 1 --2- 172.21.15.53:0/1145901058 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a880035c0 0x7f6a88005a40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:47.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:47.977+0000 7f6aa54bc700 1 -- 172.21.15.53:0/1145901058 >> 172.21.15.53:0/1145901058 conn(0x7f6aa00fa8a0 msgr2=0x7f6aa00fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:31:47.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:47.977+0000 7f6aa54bc700 1 -- 172.21.15.53:0/1145901058 shutdown_connections 2023-11-16T16:31:47.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:47.977+0000 7f6aa54bc700 1 -- 172.21.15.53:0/1145901058 wait complete. 2023-11-16T16:31:47.980 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:47.978+0000 7f6aa54bc700 1 Processor -- start 2023-11-16T16:31:47.981 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:47.978+0000 7f6aa54bc700 1 -- start start 2023-11-16T16:31:47.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:47.979+0000 7f6aa54bc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a880035c0 0x7f6aa0196140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:47.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:47.979+0000 7f6aa54bc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aa0196680 0x7f6aa0196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:47.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:47.979+0000 7f6aa54bc700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa019f1b0 con 0x7f6a880035c0 2023-11-16T16:31:47.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:47.979+0000 7f6aa54bc700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa019f2f0 con 0x7f6aa0196680 2023-11-16T16:31:47.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:47.979+0000 7f6a9e7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aa0196680 0x7f6aa0196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:47.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:47.979+0000 7f6a9effd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a880035c0 0x7f6aa0196140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:47.982 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:47.979+0000 7f6a9effd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a880035c0 0x7f6aa0196140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.53:46194/0 (socket says 172.21.15.53:46194) 2023-11-16T16:31:47.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:47.979+0000 7f6a9effd700 1 -- 172.21.15.53:0/3436161807 learned_addr learned my addr 172.21.15.53:0/3436161807 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-16T16:31:47.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:47.979+0000 7f6a9e7fc700 1 -- 172.21.15.53:0/3436161807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aa0196680 msgr2=0x7f6aa0196af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-11-16T16:31:47.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:47.979+0000 7f6a9e7fc700 1 -- 172.21.15.53:0/3436161807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aa0196680 msgr2=0x7f6aa0196af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-11-16T16:31:47.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:47.979+0000 7f6a9e7fc700 1 --2- 172.21.15.53:0/3436161807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aa0196680 0x7f6aa0196af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-16T16:31:47.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:47.979+0000 7f6a9e7fc700 1 --2- 172.21.15.53:0/3436161807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aa0196680 0x7f6aa0196af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-16T16:31:47.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:47.979+0000 7f6a9effd700 1 -- 172.21.15.53:0/3436161807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aa0196680 msgr2=0x7f6aa0196af0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-16T16:31:47.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:47.979+0000 7f6a9effd700 1 --2- 172.21.15.53:0/3436161807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aa0196680 0x7f6aa0196af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:47.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:47.979+0000 7f6a9effd700 1 -- 172.21.15.53:0/3436161807 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9000f900 con 0x7f6a880035c0 2023-11-16T16:31:47.983 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:47.980+0000 7f6a9effd700 1 --2- 172.21.15.53:0/3436161807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a880035c0 0x7f6aa0196140 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6a9000f800 tx=0x7f6a900049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:47.984 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:47.980+0000 7f6a87fff700 1 -- 172.21.15.53:0/3436161807 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f6a90013d10 con 0x7f6a880035c0 2023-11-16T16:31:47.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:47.980+0000 7f6a87fff700 1 -- 172.21.15.53:0/3436161807 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a90005160 con 0x7f6a880035c0 2023-11-16T16:31:47.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:47.980+0000 7f6aa54bc700 1 -- 172.21.15.53:0/3436161807 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa0106b90 con 0x7f6a880035c0 2023-11-16T16:31:47.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:47.980+0000 7f6a87fff700 1 -- 172.21.15.53:0/3436161807 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f6a90003d50 con 0x7f6a880035c0 2023-11-16T16:31:47.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:47.980+0000 7f6aa54bc700 1 -- 172.21.15.53:0/3436161807 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa0106a00 con 0x7f6a880035c0 2023-11-16T16:31:47.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:47.981+0000 7f6a87fff700 1 -- 172.21.15.53:0/3436161807 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f6a90011470 con 0x7f6a880035c0 2023-11-16T16:31:47.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:47.981+0000 7f6a87fff700 1 --2- 172.21.15.53:0/3436161807 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f6a8c0386b0 0x7f6a8c03ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:47.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:47.981+0000 7f6a87fff700 1 -- 172.21.15.53:0/3436161807 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6a9004ecc0 con 0x7f6a880035c0 2023-11-16T16:31:47.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:47.981+0000 7f6a9e7fc700 1 --2- 172.21.15.53:0/3436161807 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f6a8c0386b0 0x7f6a8c03ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:47.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:47.981+0000 7f6aa54bc700 1 -- 172.21.15.53:0/3436161807 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aa0106d20 con 0x7f6a880035c0 2023-11-16T16:31:47.985 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:47.982+0000 7f6a9e7fc700 1 --2- 172.21.15.53:0/3436161807 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f6a8c0386b0 0x7f6a8c03ab60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0069bb0 tx=0x7f6a94008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:47.989 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:47.987+0000 7f6a87fff700 1 -- 172.21.15.53:0/3436161807 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6a900109c0 con 0x7f6a880035c0 2023-11-16T16:31:48.277 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:48.274+0000 7f6aa54bc700 1 -- 172.21.15.53:0/3436161807 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6aa019f430 con 0x7f6a880035c0 2023-11-16T16:31:48.278 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:48.275+0000 7f6a87fff700 1 -- 172.21.15.53:0/3436161807 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+998 (secure 0 0 0) 0x7f6a900103d0 con 0x7f6a880035c0 2023-11-16T16:31:48.278 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-16T16:31:48.278 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":2,"fsid":"35d9b010-849d-11ee-8dbe-212e2dc638e7","modified":"2023-11-16T16:31:41.918366Z","created":"2023-11-16T16:30:02.658115Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-11-16T16:31:48.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:48.277+0000 7f6aa54bc700 1 -- 172.21.15.53:0/3436161807 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f6a8c0386b0 msgr2=0x7f6a8c03ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:48.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:48.277+0000 7f6aa54bc700 1 --2- 172.21.15.53:0/3436161807 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f6a8c0386b0 0x7f6a8c03ab60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0069bb0 tx=0x7f6a94008040 comp rx=0 tx=0).stop 2023-11-16T16:31:48.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:48.277+0000 7f6aa54bc700 1 -- 172.21.15.53:0/3436161807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a880035c0 msgr2=0x7f6aa0196140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:48.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:48.277+0000 7f6aa54bc700 1 --2- 172.21.15.53:0/3436161807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a880035c0 0x7f6aa0196140 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6a9000f800 tx=0x7f6a900049d0 comp rx=0 tx=0).stop 2023-11-16T16:31:48.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:48.277+0000 7f6aa54bc700 1 -- 172.21.15.53:0/3436161807 shutdown_connections 2023-11-16T16:31:48.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:48.277+0000 7f6aa54bc700 1 --2- 172.21.15.53:0/3436161807 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f6a8c0386b0 0x7f6a8c03ab60 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:48.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:48.277+0000 7f6aa54bc700 1 --2- 172.21.15.53:0/3436161807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a880035c0 0x7f6aa0196140 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:48.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:48.277+0000 7f6aa54bc700 1 --2- 172.21.15.53:0/3436161807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6aa0196680 0x7f6aa0196af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:48.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:48.277+0000 7f6aa54bc700 1 -- 172.21.15.53:0/3436161807 >> 172.21.15.53:0/3436161807 conn(0x7f6aa00fa8a0 msgr2=0x7f6aa01008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:31:48.280 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:48.277+0000 7f6aa54bc700 1 -- 172.21.15.53:0/3436161807 shutdown_connections 2023-11-16T16:31:48.281 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:48.277+0000 7f6aa54bc700 1 -- 172.21.15.53:0/3436161807 wait complete. 2023-11-16T16:31:48.281 INFO:teuthology.orchestra.run.smithi053.stderr:dumped monmap epoch 2 2023-11-16T16:31:48.301 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:48 smithi053 ceph-mon[100638]: Deploying daemon mon.b on smithi026 2023-11-16T16:31:48.301 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:48 smithi053 ceph-mon[100638]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:48.301 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:48 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:31:48.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:48 smithi001 ceph-mon[98800]: Deploying daemon mon.b on smithi026 2023-11-16T16:31:48.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:48 smithi001 ceph-mon[98800]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:48.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:48 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:31:49.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:49 smithi053 ceph-mon[100638]: from='client.? 172.21.15.53:0/3436161807' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-16T16:31:49.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:49 smithi001 ceph-mon[98800]: from='client.? 172.21.15.53:0/3436161807' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-16T16:31:49.735 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-11-16T16:31:49.735 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph mon dump -f json 2023-11-16T16:31:49.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:49 smithi026 systemd[1]: Starting Ceph mon.b for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:31:50.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:49 smithi026 podman[100073]: 2023-11-16T16:31:50.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 bash[100073]: a9021255676c4f0d73509b9feb727fbec3b6d0dc153102c9e6b0c4b178ab2f34 2023-11-16T16:31:50.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: set uid:gid to 167:167 (ceph:ceph) 2023-11-16T16:31:50.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: ceph version 18.0.0-3905-g5e4a743a (5e4a743ad7fd1422ebd6c4dc950a21a303279207) reef (dev), process ceph-mon, pid 7 2023-11-16T16:31:50.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: pidfile_write: ignore empty --pid-file 2023-11-16T16:31:50.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: load: jerasure load: lrc 2023-11-16T16:31:50.260 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: RocksDB version: 7.9.2 2023-11-16T16:31:50.260 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Git sha 0 2023-11-16T16:31:50.260 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Compile date 2023-11-15 23:53:54 2023-11-16T16:31:50.260 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: DB SUMMARY 2023-11-16T16:31:50.260 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: DB Session ID: 7R0ZL9ZEWEL3HQ80D88E 2023-11-16T16:31:50.260 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: CURRENT file: CURRENT 2023-11-16T16:31:50.260 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: IDENTITY file: IDENTITY 2023-11-16T16:31:50.260 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-11-16T16:31:50.260 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-11-16T16:31:50.260 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-11-16T16:31:50.260 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.error_if_exists: 0 2023-11-16T16:31:50.261 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.create_if_missing: 0 2023-11-16T16:31:50.261 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.paranoid_checks: 1 2023-11-16T16:31:50.261 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.flush_verify_memtable_count: 1 2023-11-16T16:31:50.261 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 systemd[1]: Started Ceph mon.b for 35d9b010-849d-11ee-8dbe-212e2dc638e7. 2023-11-16T16:31:50.261 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-11-16T16:31:50.261 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-11-16T16:31:50.261 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.env: 0x55f583573720 2023-11-16T16:31:50.261 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.fs: PosixFileSystem 2023-11-16T16:31:50.261 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.info_log: 0x55f584fb73c0 2023-11-16T16:31:50.261 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.max_file_opening_threads: 16 2023-11-16T16:31:50.261 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.statistics: (nil) 2023-11-16T16:31:50.262 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.use_fsync: 0 2023-11-16T16:31:50.262 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.max_log_file_size: 0 2023-11-16T16:31:50.262 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-11-16T16:31:50.262 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.log_file_time_to_roll: 0 2023-11-16T16:31:50.262 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.keep_log_file_num: 1000 2023-11-16T16:31:50.262 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.recycle_log_file_num: 0 2023-11-16T16:31:50.262 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.allow_fallocate: 1 2023-11-16T16:31:50.262 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.allow_mmap_reads: 0 2023-11-16T16:31:50.262 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.allow_mmap_writes: 0 2023-11-16T16:31:50.262 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.use_direct_reads: 0 2023-11-16T16:31:50.263 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-11-16T16:31:50.263 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.create_missing_column_families: 0 2023-11-16T16:31:50.263 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.db_log_dir: 2023-11-16T16:31:50.263 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.wal_dir: 2023-11-16T16:31:50.263 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.table_cache_numshardbits: 6 2023-11-16T16:31:50.263 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.WAL_ttl_seconds: 0 2023-11-16T16:31:50.263 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.WAL_size_limit_MB: 0 2023-11-16T16:31:50.263 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-11-16T16:31:50.263 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-11-16T16:31:50.263 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.is_fd_close_on_exec: 1 2023-11-16T16:31:50.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.advise_random_on_open: 1 2023-11-16T16:31:50.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.db_write_buffer_size: 0 2023-11-16T16:31:50.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.write_buffer_manager: 0x55f5842283c0 2023-11-16T16:31:50.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-11-16T16:31:50.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-11-16T16:31:50.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.use_adaptive_mutex: 0 2023-11-16T16:31:50.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.rate_limiter: (nil) 2023-11-16T16:31:50.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-11-16T16:31:50.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.wal_recovery_mode: 2 2023-11-16T16:31:50.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.enable_thread_tracking: 0 2023-11-16T16:31:50.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.enable_pipelined_write: 0 2023-11-16T16:31:50.264 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.unordered_write: 0 2023-11-16T16:31:50.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-11-16T16:31:50.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-11-16T16:31:50.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-11-16T16:31:50.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-11-16T16:31:50.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.row_cache: None 2023-11-16T16:31:50.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.wal_filter: None 2023-11-16T16:31:50.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-11-16T16:31:50.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.allow_ingest_behind: 0 2023-11-16T16:31:50.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.two_write_queues: 0 2023-11-16T16:31:50.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.manual_wal_flush: 0 2023-11-16T16:31:50.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.wal_compression: 0 2023-11-16T16:31:50.265 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.atomic_flush: 0 2023-11-16T16:31:50.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-11-16T16:31:50.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.persist_stats_to_disk: 0 2023-11-16T16:31:50.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.write_dbid_to_manifest: 0 2023-11-16T16:31:50.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.log_readahead_size: 0 2023-11-16T16:31:50.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-11-16T16:31:50.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.best_efforts_recovery: 0 2023-11-16T16:31:50.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-11-16T16:31:50.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-11-16T16:31:50.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.allow_data_in_errors: 0 2023-11-16T16:31:50.266 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.db_host_id: __hostname__ 2023-11-16T16:31:50.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.enforce_single_del_contracts: true 2023-11-16T16:31:50.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.max_background_jobs: 2 2023-11-16T16:31:50.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.max_background_compactions: -1 2023-11-16T16:31:50.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.max_subcompactions: 1 2023-11-16T16:31:50.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-11-16T16:31:50.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-11-16T16:31:50.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.delayed_write_rate : 16777216 2023-11-16T16:31:50.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.max_total_wal_size: 0 2023-11-16T16:31:50.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-11-16T16:31:50.267 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.stats_dump_period_sec: 600 2023-11-16T16:31:50.268 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.stats_persist_period_sec: 600 2023-11-16T16:31:50.268 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-11-16T16:31:50.268 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.max_open_files: -1 2023-11-16T16:31:50.268 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.bytes_per_sync: 0 2023-11-16T16:31:50.268 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.wal_bytes_per_sync: 0 2023-11-16T16:31:50.268 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.strict_bytes_per_sync: 0 2023-11-16T16:31:50.268 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.compaction_readahead_size: 0 2023-11-16T16:31:50.268 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.max_background_flushes: -1 2023-11-16T16:31:50.268 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Compression algorithms supported: 2023-11-16T16:31:50.268 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-11-16T16:31:50.268 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: kZSTD supported: 0 2023-11-16T16:31:50.268 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: kXpressCompression supported: 0 2023-11-16T16:31:50.268 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: kLZ4HCCompression supported: 1 2023-11-16T16:31:50.269 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: kZlibCompression supported: 1 2023-11-16T16:31:50.269 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: kSnappyCompression supported: 1 2023-11-16T16:31:50.269 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: kLZ4Compression supported: 1 2023-11-16T16:31:50.269 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: kBZip2Compression supported: 0 2023-11-16T16:31:50.269 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Fast CRC32 supported: Supported on x86 2023-11-16T16:31:50.269 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: DMutex implementation: pthread_mutex_t 2023-11-16T16:31:50.269 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-11-16T16:31:50.269 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-11-16T16:31:50.269 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-11-16T16:31:50.269 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.merge_operator: 2023-11-16T16:31:50.269 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.compaction_filter: None 2023-11-16T16:31:50.269 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.compaction_filter_factory: None 2023-11-16T16:31:50.270 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.sst_partitioner_factory: None 2023-11-16T16:31:50.270 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.memtable_factory: SkipListFactory 2023-11-16T16:31:50.270 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.table_factory: BlockBasedTable 2023-11-16T16:31:50.270 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f584fb74e0) 2023-11-16T16:31:50.270 INFO:journalctl@ceph.mon.b.smithi026.stdout: cache_index_and_filter_blocks: 1 2023-11-16T16:31:50.270 INFO:journalctl@ceph.mon.b.smithi026.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-11-16T16:31:50.270 INFO:journalctl@ceph.mon.b.smithi026.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-11-16T16:31:50.270 INFO:journalctl@ceph.mon.b.smithi026.stdout: pin_top_level_index_and_filter: 1 2023-11-16T16:31:50.270 INFO:journalctl@ceph.mon.b.smithi026.stdout: index_type: 0 2023-11-16T16:31:50.270 INFO:journalctl@ceph.mon.b.smithi026.stdout: data_block_index_type: 0 2023-11-16T16:31:50.270 INFO:journalctl@ceph.mon.b.smithi026.stdout: index_shortening: 1 2023-11-16T16:31:50.271 INFO:journalctl@ceph.mon.b.smithi026.stdout: data_block_hash_table_util_ratio: 0.750000 2023-11-16T16:31:50.271 INFO:journalctl@ceph.mon.b.smithi026.stdout: checksum: 4 2023-11-16T16:31:50.271 INFO:journalctl@ceph.mon.b.smithi026.stdout: no_block_cache: 0 2023-11-16T16:31:50.271 INFO:journalctl@ceph.mon.b.smithi026.stdout: block_cache: 0x55f5842c91f0 2023-11-16T16:31:50.271 INFO:journalctl@ceph.mon.b.smithi026.stdout: block_cache_name: BinnedLRUCache 2023-11-16T16:31:50.271 INFO:journalctl@ceph.mon.b.smithi026.stdout: block_cache_options: 2023-11-16T16:31:50.271 INFO:journalctl@ceph.mon.b.smithi026.stdout: capacity : 536870912 2023-11-16T16:31:50.271 INFO:journalctl@ceph.mon.b.smithi026.stdout: num_shard_bits : 4 2023-11-16T16:31:50.271 INFO:journalctl@ceph.mon.b.smithi026.stdout: strict_capacity_limit : 0 2023-11-16T16:31:50.271 INFO:journalctl@ceph.mon.b.smithi026.stdout: high_pri_pool_ratio: 0.000 2023-11-16T16:31:50.271 INFO:journalctl@ceph.mon.b.smithi026.stdout: block_cache_compressed: (nil) 2023-11-16T16:31:50.271 INFO:journalctl@ceph.mon.b.smithi026.stdout: persistent_cache: (nil) 2023-11-16T16:31:50.272 INFO:journalctl@ceph.mon.b.smithi026.stdout: block_size: 4096 2023-11-16T16:31:50.272 INFO:journalctl@ceph.mon.b.smithi026.stdout: block_size_deviation: 10 2023-11-16T16:31:50.272 INFO:journalctl@ceph.mon.b.smithi026.stdout: block_restart_interval: 16 2023-11-16T16:31:50.272 INFO:journalctl@ceph.mon.b.smithi026.stdout: index_block_restart_interval: 1 2023-11-16T16:31:50.272 INFO:journalctl@ceph.mon.b.smithi026.stdout: metadata_block_size: 4096 2023-11-16T16:31:50.272 INFO:journalctl@ceph.mon.b.smithi026.stdout: partition_filters: 0 2023-11-16T16:31:50.272 INFO:journalctl@ceph.mon.b.smithi026.stdout: use_delta_encoding: 1 2023-11-16T16:31:50.272 INFO:journalctl@ceph.mon.b.smithi026.stdout: filter_policy: bloomfilter 2023-11-16T16:31:50.272 INFO:journalctl@ceph.mon.b.smithi026.stdout: whole_key_filtering: 1 2023-11-16T16:31:50.272 INFO:journalctl@ceph.mon.b.smithi026.stdout: verify_compression: 0 2023-11-16T16:31:50.272 INFO:journalctl@ceph.mon.b.smithi026.stdout: read_amp_bytes_per_bit: 0 2023-11-16T16:31:50.272 INFO:journalctl@ceph.mon.b.smithi026.stdout: format_version: 5 2023-11-16T16:31:50.272 INFO:journalctl@ceph.mon.b.smithi026.stdout: enable_index_compression: 1 2023-11-16T16:31:50.273 INFO:journalctl@ceph.mon.b.smithi026.stdout: block_align: 0 2023-11-16T16:31:50.273 INFO:journalctl@ceph.mon.b.smithi026.stdout: max_auto_readahead_size: 262144 2023-11-16T16:31:50.273 INFO:journalctl@ceph.mon.b.smithi026.stdout: prepopulate_block_cache: 0 2023-11-16T16:31:50.273 INFO:journalctl@ceph.mon.b.smithi026.stdout: initial_auto_readahead_size: 8192 2023-11-16T16:31:50.273 INFO:journalctl@ceph.mon.b.smithi026.stdout: num_file_reads_for_auto_readahead: 2 2023-11-16T16:31:50.273 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.write_buffer_size: 33554432 2023-11-16T16:31:50.273 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.max_write_buffer_number: 2 2023-11-16T16:31:50.273 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.compression: NoCompression 2023-11-16T16:31:50.273 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.bottommost_compression: Disabled 2023-11-16T16:31:50.274 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.prefix_extractor: nullptr 2023-11-16T16:31:50.274 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-11-16T16:31:50.274 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.num_levels: 7 2023-11-16T16:31:50.274 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-11-16T16:31:50.274 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-11-16T16:31:50.274 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-11-16T16:31:50.274 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-11-16T16:31:50.274 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-11-16T16:31:50.275 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-11-16T16:31:50.275 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-11-16T16:31:50.276 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-11-16T16:31:50.276 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-11-16T16:31:50.276 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-11-16T16:31:50.276 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-11-16T16:31:50.276 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-11-16T16:31:50.276 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.compression_opts.window_bits: -14 2023-11-16T16:31:50.276 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.compression_opts.level: 32767 2023-11-16T16:31:50.276 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.compression_opts.strategy: 0 2023-11-16T16:31:50.276 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-11-16T16:31:50.276 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-11-16T16:31:50.276 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-11-16T16:31:50.276 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-11-16T16:31:50.276 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.compression_opts.enabled: false 2023-11-16T16:31:50.277 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-11-16T16:31:50.277 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-11-16T16:31:50.277 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-11-16T16:31:50.277 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-11-16T16:31:50.277 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.target_file_size_base: 67108864 2023-11-16T16:31:50.277 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.target_file_size_multiplier: 1 2023-11-16T16:31:50.277 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-11-16T16:31:50.277 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-11-16T16:31:50.277 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-11-16T16:31:50.277 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-11-16T16:31:50.277 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-11-16T16:31:50.277 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-11-16T16:31:50.277 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-11-16T16:31:50.278 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-11-16T16:31:50.278 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-11-16T16:31:50.278 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-11-16T16:31:50.278 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-11-16T16:31:50.278 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-11-16T16:31:50.278 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-11-16T16:31:50.278 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.arena_block_size: 1048576 2023-11-16T16:31:50.278 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-11-16T16:31:50.278 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-11-16T16:31:50.278 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.disable_auto_compactions: 0 2023-11-16T16:31:50.278 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-11-16T16:31:50.279 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-11-16T16:31:50.279 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-11-16T16:31:50.279 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-11-16T16:31:50.279 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-11-16T16:31:50.279 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-11-16T16:31:50.279 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-11-16T16:31:50.279 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-11-16T16:31:50.279 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-11-16T16:31:50.279 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-11-16T16:31:50.279 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.table_properties_collectors: 2023-11-16T16:31:50.279 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.inplace_update_support: 0 2023-11-16T16:31:50.279 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.inplace_update_num_locks: 10000 2023-11-16T16:31:50.279 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-11-16T16:31:50.280 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-11-16T16:31:50.280 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.memtable_huge_page_size: 0 2023-11-16T16:31:50.280 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.bloom_locality: 0 2023-11-16T16:31:50.280 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.max_successive_merges: 0 2023-11-16T16:31:50.280 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.optimize_filters_for_hits: 0 2023-11-16T16:31:50.280 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.paranoid_file_checks: 0 2023-11-16T16:31:50.280 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.force_consistency_checks: 1 2023-11-16T16:31:50.280 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.report_bg_io_stats: 0 2023-11-16T16:31:50.280 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.ttl: 2592000 2023-11-16T16:31:50.280 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.periodic_compaction_seconds: 0 2023-11-16T16:31:50.280 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-11-16T16:31:50.280 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-11-16T16:31:50.280 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.enable_blob_files: false 2023-11-16T16:31:50.281 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.min_blob_size: 0 2023-11-16T16:31:50.281 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.blob_file_size: 268435456 2023-11-16T16:31:50.281 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.blob_compression_type: NoCompression 2023-11-16T16:31:50.281 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.enable_blob_garbage_collection: false 2023-11-16T16:31:50.281 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-11-16T16:31:50.281 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-11-16T16:31:50.281 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-11-16T16:31:50.281 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.blob_file_starting_level: 0 2023-11-16T16:31:50.281 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-11-16T16:31:50.281 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-11-16T16:31:50.281 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-11-16T16:31:50.281 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 5c168fcb-1259-45f3-b3df-e08e92ea94ca 2023-11-16T16:31:50.281 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: EVENT_LOG_v1 {"time_micros": 1700152310160104, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-11-16T16:31:50.282 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-11-16T16:31:50.282 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: EVENT_LOG_v1 {"time_micros": 1700152310160723, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1700152310, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "5c168fcb-1259-45f3-b3df-e08e92ea94ca", "db_session_id": "7R0ZL9ZEWEL3HQ80D88E", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-11-16T16:31:50.282 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: EVENT_LOG_v1 {"time_micros": 1700152310160821, "job": 1, "event": "recovery_finished"} 2023-11-16T16:31:50.282 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-11-16T16:31:50.282 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-11-16T16:31:50.282 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f584366000 2023-11-16T16:31:50.282 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: DB pointer 0x55f584352000 2023-11-16T16:31:50.282 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-11-16T16:31:50.282 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-11-16T16:31:50.282 INFO:journalctl@ceph.mon.b.smithi026.stdout: ** DB Stats ** 2023-11-16T16:31:50.282 INFO:journalctl@ceph.mon.b.smithi026.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-16T16:31:50.282 INFO:journalctl@ceph.mon.b.smithi026.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-11-16T16:31:50.283 INFO:journalctl@ceph.mon.b.smithi026.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-16T16:31:50.283 INFO:journalctl@ceph.mon.b.smithi026.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-16T16:31:50.283 INFO:journalctl@ceph.mon.b.smithi026.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-11-16T16:31:50.283 INFO:journalctl@ceph.mon.b.smithi026.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-16T16:31:50.283 INFO:journalctl@ceph.mon.b.smithi026.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-16T16:31:50.283 INFO:journalctl@ceph.mon.b.smithi026.stdout: 2023-11-16T16:31:50.283 INFO:journalctl@ceph.mon.b.smithi026.stdout: ** Compaction Stats [default] ** 2023-11-16T16:31:50.283 INFO:journalctl@ceph.mon.b.smithi026.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-16T16:31:50.283 INFO:journalctl@ceph.mon.b.smithi026.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-11-16T16:31:50.283 INFO:journalctl@ceph.mon.b.smithi026.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-11-16T16:31:50.283 INFO:journalctl@ceph.mon.b.smithi026.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-11-16T16:31:50.284 INFO:journalctl@ceph.mon.b.smithi026.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-11-16T16:31:50.284 INFO:journalctl@ceph.mon.b.smithi026.stdout: 2023-11-16T16:31:50.284 INFO:journalctl@ceph.mon.b.smithi026.stdout: ** Compaction Stats [default] ** 2023-11-16T16:31:50.284 INFO:journalctl@ceph.mon.b.smithi026.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-16T16:31:50.284 INFO:journalctl@ceph.mon.b.smithi026.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-11-16T16:31:50.284 INFO:journalctl@ceph.mon.b.smithi026.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-11-16T16:31:50.284 INFO:journalctl@ceph.mon.b.smithi026.stdout: 2023-11-16T16:31:50.284 INFO:journalctl@ceph.mon.b.smithi026.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-11-16T16:31:50.284 INFO:journalctl@ceph.mon.b.smithi026.stdout: 2023-11-16T16:31:50.284 INFO:journalctl@ceph.mon.b.smithi026.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-16T16:31:50.284 INFO:journalctl@ceph.mon.b.smithi026.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-11-16T16:31:50.284 INFO:journalctl@ceph.mon.b.smithi026.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-11-16T16:31:50.284 INFO:journalctl@ceph.mon.b.smithi026.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-11-16T16:31:50.285 INFO:journalctl@ceph.mon.b.smithi026.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-11-16T16:31:50.285 INFO:journalctl@ceph.mon.b.smithi026.stdout: AddFile(Keys): cumulative 0, interval 0 2023-11-16T16:31:50.285 INFO:journalctl@ceph.mon.b.smithi026.stdout: Cumulative compaction: 0.00 GB write, 0.43 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-16T16:31:50.285 INFO:journalctl@ceph.mon.b.smithi026.stdout: Interval compaction: 0.00 GB write, 0.43 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-16T16:31:50.285 INFO:journalctl@ceph.mon.b.smithi026.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-11-16T16:31:50.285 INFO:journalctl@ceph.mon.b.smithi026.stdout: Block cache BinnedLRUCache@0x55f5842c91f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.3e-05 secs_since: 0 2023-11-16T16:31:50.285 INFO:journalctl@ceph.mon.b.smithi026.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-11-16T16:31:50.285 INFO:journalctl@ceph.mon.b.smithi026.stdout: 2023-11-16T16:31:50.285 INFO:journalctl@ceph.mon.b.smithi026.stdout: ** File Read Latency Histogram By Level [default] ** 2023-11-16T16:31:50.285 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-11-16T16:31:50.285 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: using public_addr v2:172.21.15.26:0/0 -> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] 2023-11-16T16:31:50.285 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: starting mon.b rank -1 at public addrs [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] at bind addrs [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 2023-11-16T16:31:50.285 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: mon.b@-1(???) e0 preinit fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 2023-11-16T16:31:50.286 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: mon.b@-1(synchronizing).mds e1 new map 2023-11-16T16:31:50.286 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: mon.b@-1(synchronizing).mds e1 print_map 2023-11-16T16:31:50.286 INFO:journalctl@ceph.mon.b.smithi026.stdout: e1 2023-11-16T16:31:50.286 INFO:journalctl@ceph.mon.b.smithi026.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-11-16T16:31:50.286 INFO:journalctl@ceph.mon.b.smithi026.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-11-16T16:31:50.286 INFO:journalctl@ceph.mon.b.smithi026.stdout: legacy client fscid: -1 2023-11-16T16:31:50.286 INFO:journalctl@ceph.mon.b.smithi026.stdout: 2023-11-16T16:31:50.286 INFO:journalctl@ceph.mon.b.smithi026.stdout: No filesystems configured 2023-11-16T16:31:50.286 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-11-16T16:31:50.286 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-11-16T16:31:50.286 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-11-16T16:31:50.286 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-11-16T16:31:50.287 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-11-16T16:31:50.287 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-11-16T16:31:50.287 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-11-16T16:31:50.287 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-11-16T16:31:50.287 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-11-16T16:31:50.287 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-11-16T16:31:50.287 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: Added host smithi026 2023-11-16T16:31:50.287 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:50.287 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:50.287 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:50.287 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:31:50.287 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:50.287 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:50.288 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:50.288 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:50.288 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:50.288 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi026", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:31:50.288 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:31:50.288 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:31:50.288 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:31:50.288 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: Updating smithi026:/etc/ceph/ceph.conf 2023-11-16T16:31:50.288 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi053", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:31:50.288 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: Updating smithi026:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:31:50.288 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:50.288 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: Updating smithi026:/etc/ceph/ceph.client.admin.keyring 2023-11-16T16:31:50.289 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: Deploying cephadm binary to smithi053 2023-11-16T16:31:50.289 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: Updating smithi026:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-16T16:31:50.289 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:50.289 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:50.289 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:50.289 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:50.289 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:50.289 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:31:50.289 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: Added host smithi053 2023-11-16T16:31:50.289 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:50.289 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:50.289 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:50.289 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:31:50.290 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:50.290 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:50.290 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1139665089' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-11-16T16:31:50.290 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:50.290 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:50.290 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:50.290 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:50.290 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi053", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:31:50.290 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:31:50.290 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:31:50.290 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:31:50.290 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: Updating smithi053:/etc/ceph/ceph.conf 2023-11-16T16:31:50.291 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: Updating smithi053:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:31:50.291 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1139665089' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-11-16T16:31:50.291 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: osdmap e4: 0 total, 0 up, 0 in 2023-11-16T16:31:50.291 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:50.291 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:50.291 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:50.291 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:50.291 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2023-11-16T16:31:50.291 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: Updating smithi053:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-16T16:31:50.291 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:50.291 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:50.291 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:31:50.291 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:31:50.292 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:31:50.292 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:50.292 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-16T16:31:50.292 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:31:50.292 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi001:172.21.15.1=a;smithi026:172.21.15.26=b;smithi053:172.21.15.53=c", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:31:50.292 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: Saving service mon spec with placement smithi001:172.21.15.1=a;smithi026:172.21.15.26=b;smithi053:172.21.15.53=c;count:3 2023-11-16T16:31:50.292 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:31:50.292 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:31:50.292 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: Deploying daemon mon.c on smithi053 2023-11-16T16:31:50.292 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:50.292 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:31:50.292 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T16:31:50.292 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: mon.a calling monitor election 2023-11-16T16:31:50.293 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:31:50.293 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:31:50.293 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:50.293 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:31:50.293 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: mon.c calling monitor election 2023-11-16T16:31:50.293 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:31:50.293 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:50.293 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:31:50.293 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:31:50.293 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-11-16T16:31:50.293 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: monmap e2: 2 mons at {a=[v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0],c=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0]} removed_ranks: {} 2023-11-16T16:31:50.293 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: fsmap 2023-11-16T16:31:50.293 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: osdmap e4: 0 total, 0 up, 0 in 2023-11-16T16:31:50.293 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: mgrmap e13: x(active, since 45s) 2023-11-16T16:31:50.294 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: overall HEALTH_OK 2023-11-16T16:31:50.294 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:50.294 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:50.294 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:50.294 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-16T16:31:50.294 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:31:50.294 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: Deploying daemon mon.b on smithi026 2023-11-16T16:31:50.294 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:50.294 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:31:50.295 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: from='client.? 172.21.15.53:0/3436161807' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-16T16:31:50.295 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:50.295 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-11-16T16:31:50.295 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-11-16T16:31:50.295 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: expand_channel_meta expand map: {default=false} 2023-11-16T16:31:50.295 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: expand_channel_meta from 'false' to 'false' 2023-11-16T16:31:50.295 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: expand_channel_meta expanded map: {default=false} 2023-11-16T16:31:50.295 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: expand_channel_meta expand map: {default=info} 2023-11-16T16:31:50.295 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: expand_channel_meta from 'info' to 'info' 2023-11-16T16:31:50.295 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: expand_channel_meta expanded map: {default=info} 2023-11-16T16:31:50.295 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: expand_channel_meta expand map: {default=daemon} 2023-11-16T16:31:50.295 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: expand_channel_meta from 'daemon' to 'daemon' 2023-11-16T16:31:50.295 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: expand_channel_meta expanded map: {default=daemon} 2023-11-16T16:31:50.296 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: expand_channel_meta expand map: {default=debug} 2023-11-16T16:31:50.296 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: expand_channel_meta from 'debug' to 'debug' 2023-11-16T16:31:50.296 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:50 smithi026 ceph-mon[100112]: expand_channel_meta expanded map: {default=debug} 2023-11-16T16:31:50.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:50 smithi053 ceph-mon[100638]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:50.399 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-16T16:31:50.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:50 smithi001 ceph-mon[98800]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:55.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.219+0000 7f8b80808700 1 -- 172.21.15.53:0/2068948782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b640035c0 msgr2=0x7f8b64005a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:55.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.219+0000 7f8b80808700 1 --2- 172.21.15.53:0/2068948782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b640035c0 0x7f8b64005a40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c0fecb0 tx=0x7f8b70014ec0 comp rx=0 tx=0).stop 2023-11-16T16:31:55.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.220+0000 7f8b80808700 1 -- 172.21.15.53:0/2068948782 shutdown_connections 2023-11-16T16:31:55.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.220+0000 7f8b80808700 1 --2- 172.21.15.53:0/2068948782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b640035c0 0x7f8b64005a40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:55.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.220+0000 7f8b80808700 1 -- 172.21.15.53:0/2068948782 >> 172.21.15.53:0/2068948782 conn(0x7f8b7c074ab0 msgr2=0x7f8b7c074eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:31:55.223 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.220+0000 7f8b80808700 1 -- 172.21.15.53:0/2068948782 shutdown_connections 2023-11-16T16:31:55.224 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.220+0000 7f8b80808700 1 -- 172.21.15.53:0/2068948782 wait complete. 2023-11-16T16:31:55.225 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.221+0000 7f8b80808700 1 Processor -- start 2023-11-16T16:31:55.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.221+0000 7f8b80808700 1 -- start start 2023-11-16T16:31:55.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.222+0000 7f8b80808700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b640035c0 0x7f8b7c195400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:55.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.222+0000 7f8b80808700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b7c195940 0x7f8b7c199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:55.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.222+0000 7f8b80808700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b7c19a2a0 0x7f8b7c19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:55.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.222+0000 7f8b80808700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c196010 con 0x7f8b7c19a2a0 2023-11-16T16:31:55.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.222+0000 7f8b80808700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c1a2010 con 0x7f8b640035c0 2023-11-16T16:31:55.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.222+0000 7f8b80808700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c1a2150 con 0x7f8b7c195940 2023-11-16T16:31:55.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.222+0000 7f8b79d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b7c195940 0x7f8b7c199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:55.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.222+0000 7f8b79d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b7c195940 0x7f8b7c199d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57566/0 (socket says 172.21.15.53:57566) 2023-11-16T16:31:55.226 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.222+0000 7f8b79d9b700 1 -- 172.21.15.53:0/2304191404 learned_addr learned my addr 172.21.15.53:0/2304191404 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-16T16:31:55.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.222+0000 7f8b7a59c700 1 --2- 172.21.15.53:0/2304191404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b640035c0 0x7f8b7c195400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:55.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.222+0000 7f8b7ad9d700 1 --2- 172.21.15.53:0/2304191404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b7c19a2a0 0x7f8b7c19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:55.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.222+0000 7f8b79d9b700 1 -- 172.21.15.53:0/2304191404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b7c195940 msgr2=0x7f8b7c199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2023-11-16T16:31:55.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.222+0000 7f8b79d9b700 1 -- 172.21.15.53:0/2304191404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b7c195940 msgr2=0x7f8b7c199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-11-16T16:31:55.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.222+0000 7f8b79d9b700 1 --2- 172.21.15.53:0/2304191404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b7c195940 0x7f8b7c199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-16T16:31:55.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.222+0000 7f8b7a59c700 1 -- 172.21.15.53:0/2304191404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b640035c0 msgr2=0x7f8b7c195400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-11-16T16:31:55.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.222+0000 7f8b7a59c700 1 -- 172.21.15.53:0/2304191404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b640035c0 msgr2=0x7f8b7c195400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-11-16T16:31:55.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.222+0000 7f8b7a59c700 1 --2- 172.21.15.53:0/2304191404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b640035c0 0x7f8b7c195400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-16T16:31:55.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.222+0000 7f8b79d9b700 1 --2- 172.21.15.53:0/2304191404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b7c195940 0x7f8b7c199d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-16T16:31:55.227 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.222+0000 7f8b7a59c700 1 --2- 172.21.15.53:0/2304191404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b640035c0 0x7f8b7c195400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-16T16:31:55.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.222+0000 7f8b7ad9d700 1 -- 172.21.15.53:0/2304191404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b7c195940 msgr2=0x7f8b7c199d60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-16T16:31:55.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.222+0000 7f8b7ad9d700 1 --2- 172.21.15.53:0/2304191404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b7c195940 0x7f8b7c199d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:55.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.222+0000 7f8b7ad9d700 1 -- 172.21.15.53:0/2304191404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b640035c0 msgr2=0x7f8b7c195400 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-16T16:31:55.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.222+0000 7f8b7ad9d700 1 --2- 172.21.15.53:0/2304191404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b640035c0 0x7f8b7c195400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:55.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.222+0000 7f8b7ad9d700 1 -- 172.21.15.53:0/2304191404 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b70014910 con 0x7f8b7c19a2a0 2023-11-16T16:31:55.228 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.223+0000 7f8b7ad9d700 1 --2- 172.21.15.53:0/2304191404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b7c19a2a0 0x7f8b7c19a740 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8b7400bd80 tx=0x7f8b7400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:55.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.224+0000 7f8b6b7fe700 1 -- 172.21.15.53:0/2304191404 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7400cd10 con 0x7f8b7c19a2a0 2023-11-16T16:31:55.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.224+0000 7f8b80808700 1 -- 172.21.15.53:0/2304191404 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b7c106b80 con 0x7f8b7c19a2a0 2023-11-16T16:31:55.229 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.224+0000 7f8b80808700 1 -- 172.21.15.53:0/2304191404 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b7c1069f0 con 0x7f8b7c19a2a0 2023-11-16T16:31:55.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.224+0000 7f8b6b7fe700 1 -- 172.21.15.53:0/2304191404 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b7400ce70 con 0x7f8b7c19a2a0 2023-11-16T16:31:55.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.224+0000 7f8b6b7fe700 1 -- 172.21.15.53:0/2304191404 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7401d810 con 0x7f8b7c19a2a0 2023-11-16T16:31:55.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.225+0000 7f8b6b7fe700 1 -- 172.21.15.53:0/2304191404 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f8b7401d990 con 0x7f8b7c19a2a0 2023-11-16T16:31:55.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.225+0000 7f8b6b7fe700 1 --2- 172.21.15.53:0/2304191404 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8b5c038d40 0x7f8b5c03b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:55.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.225+0000 7f8b6b7fe700 1 -- 172.21.15.53:0/2304191404 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8b7404bd10 con 0x7f8b7c19a2a0 2023-11-16T16:31:55.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.225+0000 7f8b7a59c700 1 --2- 172.21.15.53:0/2304191404 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8b5c038d40 0x7f8b5c03b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:55.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.225+0000 7f8b80808700 1 -- 172.21.15.53:0/2304191404 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b7c106d10 con 0x7f8b7c19a2a0 2023-11-16T16:31:55.230 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.226+0000 7f8b7a59c700 1 --2- 172.21.15.53:0/2304191404 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8b5c038d40 0x7f8b5c03b1f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8b70010010 tx=0x7f8b70001e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:55.233 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.231+0000 7f8b6b7fe700 1 -- 172.21.15.53:0/2304191404 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8b7401c360 con 0x7f8b7c19a2a0 2023-11-16T16:31:55.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.530+0000 7f8b80808700 1 -- 172.21.15.53:0/2304191404 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8b7c1967b0 con 0x7f8b7c19a2a0 2023-11-16T16:31:55.533 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.531+0000 7f8b6b7fe700 1 -- 172.21.15.53:0/2304191404 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1262 (secure 0 0 0) 0x7f8b74007290 con 0x7f8b7c19a2a0 2023-11-16T16:31:55.534 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-11-16T16:31:55.534 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":3,"fsid":"35d9b010-849d-11ee-8dbe-212e2dc638e7","modified":"2023-11-16T16:31:50.186996Z","created":"2023-11-16T16:30:02.658115Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:3300","nonce":0},{"type":"v1","addr":"172.21.15.53:6789","nonce":0}]},"addr":"172.21.15.53:6789/0","public_addr":"172.21.15.53:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:3300","nonce":0},{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-11-16T16:31:55.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.533+0000 7f8b80808700 1 -- 172.21.15.53:0/2304191404 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8b5c038d40 msgr2=0x7f8b5c03b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:55.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.533+0000 7f8b80808700 1 --2- 172.21.15.53:0/2304191404 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8b5c038d40 0x7f8b5c03b1f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8b70010010 tx=0x7f8b70001e50 comp rx=0 tx=0).stop 2023-11-16T16:31:55.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.533+0000 7f8b80808700 1 -- 172.21.15.53:0/2304191404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b7c19a2a0 msgr2=0x7f8b7c19a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:55.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.533+0000 7f8b80808700 1 --2- 172.21.15.53:0/2304191404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b7c19a2a0 0x7f8b7c19a740 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8b7400bd80 tx=0x7f8b7400be60 comp rx=0 tx=0).stop 2023-11-16T16:31:55.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.533+0000 7f8b80808700 1 -- 172.21.15.53:0/2304191404 shutdown_connections 2023-11-16T16:31:55.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.533+0000 7f8b80808700 1 --2- 172.21.15.53:0/2304191404 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8b5c038d40 0x7f8b5c03b1f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:55.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.533+0000 7f8b80808700 1 --2- 172.21.15.53:0/2304191404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b7c19a2a0 0x7f8b7c19a740 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:55.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.533+0000 7f8b80808700 1 --2- 172.21.15.53:0/2304191404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b640035c0 0x7f8b7c195400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:55.536 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.533+0000 7f8b80808700 1 --2- 172.21.15.53:0/2304191404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8b7c195940 0x7f8b7c199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:55.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.533+0000 7f8b80808700 1 -- 172.21.15.53:0/2304191404 >> 172.21.15.53:0/2304191404 conn(0x7f8b7c074ab0 msgr2=0x7f8b7c068240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:31:55.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.533+0000 7f8b80808700 1 -- 172.21.15.53:0/2304191404 shutdown_connections 2023-11-16T16:31:55.537 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:31:55.533+0000 7f8b80808700 1 -- 172.21.15.53:0/2304191404 wait complete. 2023-11-16T16:31:55.537 INFO:teuthology.orchestra.run.smithi053.stderr:dumped monmap epoch 3 2023-11-16T16:31:55.558 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:55 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T16:31:55.558 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:55 smithi053 ceph-mon[100638]: mon.a calling monitor election 2023-11-16T16:31:55.558 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:55 smithi053 ceph-mon[100638]: mon.c calling monitor election 2023-11-16T16:31:55.558 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:55 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:31:55.559 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:55 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:31:55.559 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:55 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:31:55.559 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:55 smithi053 ceph-mon[100638]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:55.559 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:55 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:31:55.559 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:55 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:31:55.559 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:55 smithi053 ceph-mon[100638]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:55.559 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:55 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:31:55.559 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:55 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:31:55.559 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:55 smithi053 ceph-mon[100638]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-11-16T16:31:55.559 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:55 smithi053 ceph-mon[100638]: monmap e3: 3 mons at {a=[v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0],b=[v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0],c=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0]} removed_ranks: {} 2023-11-16T16:31:55.559 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:55 smithi053 ceph-mon[100638]: fsmap 2023-11-16T16:31:55.560 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:55 smithi053 ceph-mon[100638]: osdmap e4: 0 total, 0 up, 0 in 2023-11-16T16:31:55.560 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:55 smithi053 ceph-mon[100638]: mgrmap e13: x(active, since 53s) 2023-11-16T16:31:55.560 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:55 smithi053 ceph-mon[100638]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-11-16T16:31:55.560 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:55 smithi053 ceph-mon[100638]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-11-16T16:31:55.560 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:55 smithi053 ceph-mon[100638]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-11-16T16:31:55.560 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:55 smithi053 ceph-mon[100638]: mon.b (rank 2) addr [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] is down (out of quorum) 2023-11-16T16:31:55.560 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:55 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:55.560 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:55 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:55.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:55 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T16:31:55.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:55 smithi001 ceph-mon[98800]: mon.a calling monitor election 2023-11-16T16:31:55.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:55 smithi001 ceph-mon[98800]: mon.c calling monitor election 2023-11-16T16:31:55.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:55 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:31:55.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:55 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:31:55.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:55 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:31:55.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:55 smithi001 ceph-mon[98800]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:55.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:55 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:31:55.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:55 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:31:55.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:55 smithi001 ceph-mon[98800]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:55.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:55 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:31:55.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:55 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:31:55.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:55 smithi001 ceph-mon[98800]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-11-16T16:31:55.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:55 smithi001 ceph-mon[98800]: monmap e3: 3 mons at {a=[v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0],b=[v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0],c=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0]} removed_ranks: {} 2023-11-16T16:31:55.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:55 smithi001 ceph-mon[98800]: fsmap 2023-11-16T16:31:55.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:55 smithi001 ceph-mon[98800]: osdmap e4: 0 total, 0 up, 0 in 2023-11-16T16:31:55.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:55 smithi001 ceph-mon[98800]: mgrmap e13: x(active, since 53s) 2023-11-16T16:31:55.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:55 smithi001 ceph-mon[98800]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-11-16T16:31:55.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:55 smithi001 ceph-mon[98800]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-11-16T16:31:55.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:55 smithi001 ceph-mon[98800]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-11-16T16:31:55.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:55 smithi001 ceph-mon[98800]: mon.b (rank 2) addr [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] is down (out of quorum) 2023-11-16T16:31:55.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:55 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:55.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:55 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:55.980 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-11-16T16:31:55.980 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph config generate-minimal-conf 2023-11-16T16:31:56.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:56 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:56.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:56 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:56.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:56 smithi053 ceph-mon[100638]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:56.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:56 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:31:56.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:56 smithi053 ceph-mon[100638]: from='client.? 172.21.15.53:0/2304191404' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-16T16:31:56.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:56 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:31:56.649 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:31:56.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:56 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:56.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:56 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:56.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:56 smithi001 ceph-mon[98800]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:56.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:56 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:31:56.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:56 smithi001 ceph-mon[98800]: from='client.? 172.21.15.53:0/2304191404' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-16T16:31:56.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:56 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:31:57.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:57 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T16:31:57.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:57 smithi026 ceph-mon[100112]: mon.a calling monitor election 2023-11-16T16:31:57.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:57 smithi026 ceph-mon[100112]: mon.c calling monitor election 2023-11-16T16:31:57.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:57 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:31:57.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:57 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:31:57.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:57 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:31:57.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:57 smithi026 ceph-mon[100112]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:57.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:57 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:31:57.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:57 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:31:57.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:57 smithi026 ceph-mon[100112]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:57.510 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:57 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:31:57.510 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:57 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:31:57.510 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:57 smithi026 ceph-mon[100112]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-11-16T16:31:57.510 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:57 smithi026 ceph-mon[100112]: monmap e3: 3 mons at {a=[v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0],b=[v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0],c=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0]} removed_ranks: {} 2023-11-16T16:31:57.510 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:57 smithi026 ceph-mon[100112]: fsmap 2023-11-16T16:31:57.510 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:57 smithi026 ceph-mon[100112]: osdmap e4: 0 total, 0 up, 0 in 2023-11-16T16:31:57.510 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:57 smithi026 ceph-mon[100112]: mgrmap e13: x(active, since 53s) 2023-11-16T16:31:57.510 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:57 smithi026 ceph-mon[100112]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-11-16T16:31:57.511 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:57 smithi026 ceph-mon[100112]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-11-16T16:31:57.511 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:57 smithi026 ceph-mon[100112]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-11-16T16:31:57.511 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:57 smithi026 ceph-mon[100112]: mon.b (rank 2) addr [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] is down (out of quorum) 2023-11-16T16:31:57.511 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:57 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:57.511 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:57 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:57.511 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:57 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:57.511 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:57 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:57.511 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:57 smithi026 ceph-mon[100112]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:57.511 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:57 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:31:57.511 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:57 smithi026 ceph-mon[100112]: from='client.? 172.21.15.53:0/2304191404' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-16T16:31:57.512 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:57 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:31:57.928 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:57.925+0000 7f06424a0700 1 -- 172.21.15.1:0/2727969369 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f063c0feb50 msgr2=0x7f063c0fef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:57.928 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:57.925+0000 7f06424a0700 1 --2- 172.21.15.1:0/2727969369 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f063c0feb50 0x7f063c0fef20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f062c009910 tx=0x7f062c009c20 comp rx=0 tx=0).stop 2023-11-16T16:31:57.928 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:57.925+0000 7f06424a0700 1 -- 172.21.15.1:0/2727969369 shutdown_connections 2023-11-16T16:31:57.928 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:57.926+0000 7f06424a0700 1 --2- 172.21.15.1:0/2727969369 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f063c0feb50 0x7f063c0fef20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:57.928 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:57.926+0000 7f06424a0700 1 -- 172.21.15.1:0/2727969369 >> 172.21.15.1:0/2727969369 conn(0x7f063c074930 msgr2=0x7f063c074d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:31:57.928 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:57.926+0000 7f06424a0700 1 -- 172.21.15.1:0/2727969369 shutdown_connections 2023-11-16T16:31:57.929 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:57.926+0000 7f06424a0700 1 -- 172.21.15.1:0/2727969369 wait complete. 2023-11-16T16:31:57.932 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:57.926+0000 7f06424a0700 1 Processor -- start 2023-11-16T16:31:57.932 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:57.926+0000 7f06424a0700 1 -- start start 2023-11-16T16:31:57.932 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:57.927+0000 7f06424a0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f063c0feb50 0x7f063c197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:57.932 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:57.927+0000 7f06424a0700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f063c197a60 0x7f063c19bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:57.932 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:57.927+0000 7f06424a0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f063c19c410 0x7f063c19c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:57.933 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:57.927+0000 7f06424a0700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063c198130 con 0x7f063c0feb50 2023-11-16T16:31:57.933 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:57.927+0000 7f06424a0700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063c1a4180 con 0x7f063c19c410 2023-11-16T16:31:57.933 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:57.927+0000 7f06424a0700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063c1a42c0 con 0x7f063c197a60 2023-11-16T16:31:57.933 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:57.927+0000 7f063bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f063c0feb50 0x7f063c197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:57.933 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:57.927+0000 7f063bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f063c0feb50 0x7f063c197520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40526/0 (socket says 172.21.15.1:40526) 2023-11-16T16:31:57.933 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:57.927+0000 7f063bfff700 1 -- 172.21.15.1:0/956431820 learned_addr learned my addr 172.21.15.1:0/956431820 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:31:57.933 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:57.927+0000 7f063b7fe700 1 --2- 172.21.15.1:0/956431820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f063c197a60 0x7f063c19bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:57.933 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:57.927+0000 7f0640a3d700 1 --2- 172.21.15.1:0/956431820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f063c19c410 0x7f063c19c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:57.933 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:57.927+0000 7f063bfff700 1 -- 172.21.15.1:0/956431820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f063c197a60 msgr2=0x7f063c19bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:57.934 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:57.927+0000 7f063bfff700 1 --2- 172.21.15.1:0/956431820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f063c197a60 0x7f063c19bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:57.934 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:57.927+0000 7f063bfff700 1 -- 172.21.15.1:0/956431820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f063c19c410 msgr2=0x7f063c19c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:57.934 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:57.927+0000 7f063bfff700 1 --2- 172.21.15.1:0/956431820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f063c19c410 0x7f063c19c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:57.934 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:57.927+0000 7f063bfff700 1 -- 172.21.15.1:0/956431820 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f062c0095f0 con 0x7f063c0feb50 2023-11-16T16:31:57.934 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:57.927+0000 7f0640a3d700 1 --2- 172.21.15.1:0/956431820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f063c19c410 0x7f063c19c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:31:57.934 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:57.927+0000 7f063bfff700 1 --2- 172.21.15.1:0/956431820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f063c0feb50 0x7f063c197520 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f062c004ca0 tx=0x7f062c004cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:57.934 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:57.927+0000 7f06397fa700 1 -- 172.21.15.1:0/956431820 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c01d070 con 0x7f063c0feb50 2023-11-16T16:31:57.935 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:57.928+0000 7f06397fa700 1 -- 172.21.15.1:0/956431820 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f062c005000 con 0x7f063c0feb50 2023-11-16T16:31:57.935 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:57.928+0000 7f06424a0700 1 -- 172.21.15.1:0/956431820 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f063c10c3a0 con 0x7f063c0feb50 2023-11-16T16:31:57.935 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:57.928+0000 7f06397fa700 1 -- 172.21.15.1:0/956431820 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c022e30 con 0x7f063c0feb50 2023-11-16T16:31:57.935 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:57.928+0000 7f06424a0700 1 -- 172.21.15.1:0/956431820 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f063c10c210 con 0x7f063c0feb50 2023-11-16T16:31:57.935 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:57.928+0000 7f06397fa700 1 -- 172.21.15.1:0/956431820 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f062c00f460 con 0x7f063c0feb50 2023-11-16T16:31:57.935 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:57.929+0000 7f06397fa700 1 --2- 172.21.15.1:0/956431820 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f0624038d40 0x7f062403b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:31:57.935 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:57.929+0000 7f06397fa700 1 -- 172.21.15.1:0/956431820 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f062c04c600 con 0x7f063c0feb50 2023-11-16T16:31:57.935 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:57.929+0000 7f063b7fe700 1 --2- 172.21.15.1:0/956431820 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f0624038d40 0x7f062403b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:31:57.936 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:57.929+0000 7f06424a0700 1 -- 172.21.15.1:0/956431820 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f063c10c530 con 0x7f063c0feb50 2023-11-16T16:31:57.936 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:57.929+0000 7f063b7fe700 1 --2- 172.21.15.1:0/956431820 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f0624038d40 0x7f062403b1f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0630009ad0 tx=0x7f0630009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:31:57.936 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:57.934+0000 7f06397fa700 1 -- 172.21.15.1:0/956431820 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f062c027030 con 0x7f063c0feb50 2023-11-16T16:31:58.144 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:58.141+0000 7f06424a0700 1 -- 172.21.15.1:0/956431820 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f063c198920 con 0x7f063c0feb50 2023-11-16T16:31:58.144 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:58.142+0000 7f06397fa700 1 -- 172.21.15.1:0/956431820 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+269 (secure 0 0 0) 0x7f062c027030 con 0x7f063c0feb50 2023-11-16T16:31:58.144 INFO:teuthology.orchestra.run.smithi001.stdout:# minimal ceph.conf for 35d9b010-849d-11ee-8dbe-212e2dc638e7 2023-11-16T16:31:58.145 INFO:teuthology.orchestra.run.smithi001.stdout:[global] 2023-11-16T16:31:58.145 INFO:teuthology.orchestra.run.smithi001.stdout: fsid = 35d9b010-849d-11ee-8dbe-212e2dc638e7 2023-11-16T16:31:58.145 INFO:teuthology.orchestra.run.smithi001.stdout: mon_host = [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] 2023-11-16T16:31:58.146 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:58.144+0000 7f06424a0700 1 -- 172.21.15.1:0/956431820 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f0624038d40 msgr2=0x7f062403b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:58.146 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:58.144+0000 7f06424a0700 1 --2- 172.21.15.1:0/956431820 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f0624038d40 0x7f062403b1f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0630009ad0 tx=0x7f0630009210 comp rx=0 tx=0).stop 2023-11-16T16:31:58.147 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:58.144+0000 7f06424a0700 1 -- 172.21.15.1:0/956431820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f063c0feb50 msgr2=0x7f063c197520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:31:58.147 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:58.144+0000 7f06424a0700 1 --2- 172.21.15.1:0/956431820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f063c0feb50 0x7f063c197520 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f062c004ca0 tx=0x7f062c004cd0 comp rx=0 tx=0).stop 2023-11-16T16:31:58.147 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:58.144+0000 7f06424a0700 1 -- 172.21.15.1:0/956431820 shutdown_connections 2023-11-16T16:31:58.147 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:58.144+0000 7f06424a0700 1 --2- 172.21.15.1:0/956431820 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f0624038d40 0x7f062403b1f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:58.147 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:58.144+0000 7f06424a0700 1 --2- 172.21.15.1:0/956431820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f063c19c410 0x7f063c19c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:58.147 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:58.144+0000 7f06424a0700 1 --2- 172.21.15.1:0/956431820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f063c0feb50 0x7f063c197520 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:58.147 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:58.144+0000 7f06424a0700 1 --2- 172.21.15.1:0/956431820 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f063c197a60 0x7f063c19bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:31:58.147 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:58.144+0000 7f06424a0700 1 -- 172.21.15.1:0/956431820 >> 172.21.15.1:0/956431820 conn(0x7f063c074930 msgr2=0x7f063c107180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:31:58.147 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:58.144+0000 7f06424a0700 1 -- 172.21.15.1:0/956431820 shutdown_connections 2023-11-16T16:31:58.148 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:31:58.144+0000 7f06424a0700 1 -- 172.21.15.1:0/956431820 wait complete. 2023-11-16T16:31:58.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:58 smithi026 ceph-mon[100112]: mon.b calling monitor election 2023-11-16T16:31:58.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:58 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:31:58.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:58 smithi026 ceph-mon[100112]: Updating smithi001:/etc/ceph/ceph.conf 2023-11-16T16:31:58.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:58 smithi026 ceph-mon[100112]: Updating smithi026:/etc/ceph/ceph.conf 2023-11-16T16:31:58.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:58 smithi026 ceph-mon[100112]: Updating smithi053:/etc/ceph/ceph.conf 2023-11-16T16:31:58.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:58 smithi026 ceph-mon[100112]: mon.b calling monitor election 2023-11-16T16:31:58.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:58 smithi026 ceph-mon[100112]: mon.c calling monitor election 2023-11-16T16:31:58.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:58 smithi026 ceph-mon[100112]: mon.a calling monitor election 2023-11-16T16:31:58.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:58 smithi026 ceph-mon[100112]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-11-16T16:31:58.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:58 smithi026 ceph-mon[100112]: monmap e3: 3 mons at {a=[v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0],b=[v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0],c=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0]} removed_ranks: {} 2023-11-16T16:31:58.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:58 smithi026 ceph-mon[100112]: fsmap 2023-11-16T16:31:58.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:58 smithi026 ceph-mon[100112]: osdmap e4: 0 total, 0 up, 0 in 2023-11-16T16:31:58.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:58 smithi026 ceph-mon[100112]: mgrmap e13: x(active, since 55s) 2023-11-16T16:31:58.260 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:58 smithi026 ceph-mon[100112]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-11-16T16:31:58.260 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:58 smithi026 ceph-mon[100112]: Cluster is now healthy 2023-11-16T16:31:58.260 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:58 smithi026 ceph-mon[100112]: overall HEALTH_OK 2023-11-16T16:31:58.260 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:58 smithi026 ceph-mon[100112]: Updating smithi001:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:31:58.260 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:58 smithi026 ceph-mon[100112]: Updating smithi026:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:31:58.260 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:58 smithi026 ceph-mon[100112]: Updating smithi053:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:31:58.260 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:58 smithi026 ceph-mon[100112]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:58.260 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:58 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:58.260 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:58 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:58.260 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:58 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:58.260 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:58 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:58.261 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:58 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:58.261 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:58 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:58.261 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:58 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:58.261 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:58 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-16T16:31:58.261 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:58 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-16T16:31:58.261 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:58 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:31:58.326 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:58 smithi001 ceph-mon[98800]: mon.b calling monitor election 2023-11-16T16:31:58.326 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:58 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:31:58.326 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:58 smithi001 ceph-mon[98800]: Updating smithi001:/etc/ceph/ceph.conf 2023-11-16T16:31:58.326 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:58 smithi001 ceph-mon[98800]: Updating smithi026:/etc/ceph/ceph.conf 2023-11-16T16:31:58.326 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:58 smithi001 ceph-mon[98800]: Updating smithi053:/etc/ceph/ceph.conf 2023-11-16T16:31:58.326 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:58 smithi001 ceph-mon[98800]: mon.b calling monitor election 2023-11-16T16:31:58.327 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:58 smithi001 ceph-mon[98800]: mon.c calling monitor election 2023-11-16T16:31:58.327 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:58 smithi001 ceph-mon[98800]: mon.a calling monitor election 2023-11-16T16:31:58.327 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:58 smithi001 ceph-mon[98800]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-11-16T16:31:58.327 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:58 smithi001 ceph-mon[98800]: monmap e3: 3 mons at {a=[v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0],b=[v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0],c=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0]} removed_ranks: {} 2023-11-16T16:31:58.327 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:58 smithi001 ceph-mon[98800]: fsmap 2023-11-16T16:31:58.327 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:58 smithi001 ceph-mon[98800]: osdmap e4: 0 total, 0 up, 0 in 2023-11-16T16:31:58.327 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:58 smithi001 ceph-mon[98800]: mgrmap e13: x(active, since 55s) 2023-11-16T16:31:58.327 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:58 smithi001 ceph-mon[98800]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-11-16T16:31:58.327 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:58 smithi001 ceph-mon[98800]: Cluster is now healthy 2023-11-16T16:31:58.327 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:58 smithi001 ceph-mon[98800]: overall HEALTH_OK 2023-11-16T16:31:58.327 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:58 smithi001 ceph-mon[98800]: Updating smithi001:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:31:58.328 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:58 smithi001 ceph-mon[98800]: Updating smithi026:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:31:58.328 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:58 smithi001 ceph-mon[98800]: Updating smithi053:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:31:58.328 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:58 smithi001 ceph-mon[98800]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:58.328 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:58 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:58.328 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:58 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:58.328 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:58 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:58.328 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:58 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:58.328 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:58 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:58.328 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:58 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:58.328 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:58 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:58.328 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:58 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-16T16:31:58.329 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:58 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-16T16:31:58.329 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:58 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:31:58.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:58 smithi053 ceph-mon[100638]: mon.b calling monitor election 2023-11-16T16:31:58.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:58 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:31:58.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:58 smithi053 ceph-mon[100638]: Updating smithi001:/etc/ceph/ceph.conf 2023-11-16T16:31:58.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:58 smithi053 ceph-mon[100638]: Updating smithi026:/etc/ceph/ceph.conf 2023-11-16T16:31:58.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:58 smithi053 ceph-mon[100638]: Updating smithi053:/etc/ceph/ceph.conf 2023-11-16T16:31:58.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:58 smithi053 ceph-mon[100638]: mon.b calling monitor election 2023-11-16T16:31:58.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:58 smithi053 ceph-mon[100638]: mon.c calling monitor election 2023-11-16T16:31:58.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:58 smithi053 ceph-mon[100638]: mon.a calling monitor election 2023-11-16T16:31:58.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:58 smithi053 ceph-mon[100638]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-11-16T16:31:58.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:58 smithi053 ceph-mon[100638]: monmap e3: 3 mons at {a=[v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0],b=[v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0],c=[v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0]} removed_ranks: {} 2023-11-16T16:31:58.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:58 smithi053 ceph-mon[100638]: fsmap 2023-11-16T16:31:58.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:58 smithi053 ceph-mon[100638]: osdmap e4: 0 total, 0 up, 0 in 2023-11-16T16:31:58.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:58 smithi053 ceph-mon[100638]: mgrmap e13: x(active, since 55s) 2023-11-16T16:31:58.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:58 smithi053 ceph-mon[100638]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-11-16T16:31:58.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:58 smithi053 ceph-mon[100638]: Cluster is now healthy 2023-11-16T16:31:58.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:58 smithi053 ceph-mon[100638]: overall HEALTH_OK 2023-11-16T16:31:58.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:58 smithi053 ceph-mon[100638]: Updating smithi001:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:31:58.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:58 smithi053 ceph-mon[100638]: Updating smithi026:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:31:58.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:58 smithi053 ceph-mon[100638]: Updating smithi053:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:31:58.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:58 smithi053 ceph-mon[100638]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:31:58.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:58 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:58.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:58 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:58.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:58 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:58.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:58 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:58.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:58 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:58.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:58 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:58.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:58 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:31:58.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:58 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-16T16:31:58.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:58 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-16T16:31:58.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:58 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:31:58.667 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-11-16T16:31:58.667 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2023-11-16T16:31:58.667 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/ceph/ceph.conf 2023-11-16T16:31:58.697 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2023-11-16T16:31:58.697 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-11-16T16:31:58.778 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-11-16T16:31:58.778 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/ceph/ceph.conf 2023-11-16T16:31:58.818 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-11-16T16:31:58.818 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-11-16T16:31:58.891 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-16T16:31:58.891 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.conf 2023-11-16T16:31:58.928 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-16T16:31:58.928 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-11-16T16:31:59.000 INFO:tasks.cephadm:Adding mgr.x on smithi001 2023-11-16T16:31:59.001 INFO:tasks.cephadm:Adding mgr.y on smithi026 2023-11-16T16:31:59.001 INFO:tasks.cephadm:Adding mgr.z on smithi053 2023-11-16T16:31:59.001 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph orch apply mgr '3;smithi001=x;smithi026=y;smithi053=z' 2023-11-16T16:31:59.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:59 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:31:59.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:59 smithi026 ceph-mon[100112]: Reconfiguring mon.a (unknown last config time)... 2023-11-16T16:31:59.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:59 smithi026 ceph-mon[100112]: Reconfiguring daemon mon.a on smithi001 2023-11-16T16:31:59.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:59 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/956431820' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:31:59.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:31:59 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:31:59.279 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:59 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:31:59.279 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:59 smithi001 ceph-mon[98800]: Reconfiguring mon.a (unknown last config time)... 2023-11-16T16:31:59.279 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:59 smithi001 ceph-mon[98800]: Reconfiguring daemon mon.a on smithi001 2023-11-16T16:31:59.280 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:59 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/956431820' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:31:59.280 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:31:59 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:31:59.330 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:59 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:31:59.330 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:59 smithi053 ceph-mon[100638]: Reconfiguring mon.a (unknown last config time)... 2023-11-16T16:31:59.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:59 smithi053 ceph-mon[100638]: Reconfiguring daemon mon.a on smithi001 2023-11-16T16:31:59.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:59 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/956431820' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:31:59.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:31:59 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:31:59.696 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-16T16:32:00.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:00 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:00.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:00 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:00.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:00 smithi053 ceph-mon[100638]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:00.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:00 smithi053 ceph-mon[100638]: Reconfiguring mon.b (monmap changed)... 2023-11-16T16:32:00.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:00 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-16T16:32:00.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:00 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-16T16:32:00.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:00 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:00.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:00 smithi053 ceph-mon[100638]: Reconfiguring daemon mon.b on smithi026 2023-11-16T16:32:00.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:00 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:00.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:00 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:00.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:00 smithi001 ceph-mon[98800]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:00.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:00 smithi001 ceph-mon[98800]: Reconfiguring mon.b (monmap changed)... 2023-11-16T16:32:00.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:00 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-16T16:32:00.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:00 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-16T16:32:00.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:00 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:00.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:00 smithi001 ceph-mon[98800]: Reconfiguring daemon mon.b on smithi026 2023-11-16T16:32:00.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:00 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:00.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:00 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:00.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:00 smithi026 ceph-mon[100112]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:00.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:00 smithi026 ceph-mon[100112]: Reconfiguring mon.b (monmap changed)... 2023-11-16T16:32:00.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:00 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-16T16:32:00.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:00 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-16T16:32:00.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:00 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:00.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:00 smithi026 ceph-mon[100112]: Reconfiguring daemon mon.b on smithi026 2023-11-16T16:32:00.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.932+0000 7fb8f6578700 1 -- 172.21.15.53:0/3717353644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8f0100920 msgr2=0x7fb8f0102d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:32:00.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.932+0000 7fb8f6578700 1 --2- 172.21.15.53:0/3717353644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8f0100920 0x7fb8f0102d00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0009910 tx=0x7fb8e0009c20 comp rx=0 tx=0).stop 2023-11-16T16:32:00.934 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.932+0000 7fb8f6578700 1 -- 172.21.15.53:0/3717353644 shutdown_connections 2023-11-16T16:32:00.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.932+0000 7fb8f6578700 1 --2- 172.21.15.53:0/3717353644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8f0100920 0x7fb8f0102d00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:32:00.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.932+0000 7fb8f6578700 1 -- 172.21.15.53:0/3717353644 >> 172.21.15.53:0/3717353644 conn(0x7fb8f00fa2c0 msgr2=0x7fb8f00fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:32:00.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.932+0000 7fb8f6578700 1 -- 172.21.15.53:0/3717353644 shutdown_connections 2023-11-16T16:32:00.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.932+0000 7fb8f6578700 1 -- 172.21.15.53:0/3717353644 wait complete. 2023-11-16T16:32:00.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.933+0000 7fb8f6578700 1 Processor -- start 2023-11-16T16:32:00.935 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.933+0000 7fb8f6578700 1 -- start start 2023-11-16T16:32:00.937 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.933+0000 7fb8f6578700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8f0100920 0x7fb8f0195040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:32:00.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.933+0000 7fb8f6578700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8f0195580 0x7fb8f019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:32:00.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.933+0000 7fb8f6578700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8f01959f0 0x7fb8f019cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:32:00.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.933+0000 7fb8f6578700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f019d300 con 0x7fb8f0195580 2023-11-16T16:32:00.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.933+0000 7fb8f6578700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f019d470 con 0x7fb8f01959f0 2023-11-16T16:32:00.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.933+0000 7fb8f6578700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f019d630 con 0x7fb8f0100920 2023-11-16T16:32:00.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.933+0000 7fb8effff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8f0100920 0x7fb8f0195040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:32:00.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.933+0000 7fb8ef7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8f0195580 0x7fb8f019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:32:00.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.933+0000 7fb8effff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8f0100920 0x7fb8f0195040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57572/0 (socket says 172.21.15.53:57572) 2023-11-16T16:32:00.938 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.933+0000 7fb8effff700 1 -- 172.21.15.53:0/1809594862 learned_addr learned my addr 172.21.15.53:0/1809594862 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-16T16:32:00.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.933+0000 7fb8ef7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8f0195580 0x7fb8f019a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.53:46228/0 (socket says 172.21.15.53:46228) 2023-11-16T16:32:00.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.933+0000 7fb8f4b15700 1 --2- 172.21.15.53:0/1809594862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8f01959f0 0x7fb8f019cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:32:00.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.934+0000 7fb8effff700 1 -- 172.21.15.53:0/1809594862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8f01959f0 msgr2=0x7fb8f019cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:32:00.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.934+0000 7fb8effff700 1 --2- 172.21.15.53:0/1809594862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8f01959f0 0x7fb8f019cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:32:00.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.934+0000 7fb8effff700 1 -- 172.21.15.53:0/1809594862 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8f0195580 msgr2=0x7fb8f019a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:32:00.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.934+0000 7fb8effff700 1 --2- 172.21.15.53:0/1809594862 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8f0195580 0x7fb8f019a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:32:00.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.934+0000 7fb8effff700 1 -- 172.21.15.53:0/1809594862 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e00095f0 con 0x7fb8f0100920 2023-11-16T16:32:00.939 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.934+0000 7fb8f4b15700 1 --2- 172.21.15.53:0/1809594862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8f01959f0 0x7fb8f019cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:32:00.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.934+0000 7fb8ef7fe700 1 --2- 172.21.15.53:0/1809594862 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8f0195580 0x7fb8f019a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:32:00.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.934+0000 7fb8effff700 1 --2- 172.21.15.53:0/1809594862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8f0100920 0x7fb8f0195040 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0005950 tx=0x7fb8e0004cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:32:00.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.934+0000 7fb8ed7fa700 1 -- 172.21.15.53:0/1809594862 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e001d070 con 0x7fb8f0100920 2023-11-16T16:32:00.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.934+0000 7fb8ed7fa700 1 -- 172.21.15.53:0/1809594862 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8e000be20 con 0x7fb8f0100920 2023-11-16T16:32:00.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.934+0000 7fb8ed7fa700 1 -- 172.21.15.53:0/1809594862 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0022e90 con 0x7fb8f0100920 2023-11-16T16:32:00.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.934+0000 7fb8f6578700 1 -- 172.21.15.53:0/1809594862 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8f0105d00 con 0x7fb8f0100920 2023-11-16T16:32:00.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.934+0000 7fb8f6578700 1 -- 172.21.15.53:0/1809594862 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8f0105b70 con 0x7fb8f0100920 2023-11-16T16:32:00.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.935+0000 7fb8ed7fa700 1 -- 172.21.15.53:0/1809594862 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7fb8e000f460 con 0x7fb8f0100920 2023-11-16T16:32:00.940 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.935+0000 7fb8f6578700 1 -- 172.21.15.53:0/1809594862 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8f018ea30 con 0x7fb8f0100920 2023-11-16T16:32:00.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.935+0000 7fb8ed7fa700 1 --2- 172.21.15.53:0/1809594862 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fb8d8038d40 0x7fb8d803b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:32:00.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.935+0000 7fb8ed7fa700 1 -- 172.21.15.53:0/1809594862 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb8e004c7f0 con 0x7fb8f0100920 2023-11-16T16:32:00.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.936+0000 7fb8ef7fe700 1 --2- 172.21.15.53:0/1809594862 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fb8d8038d40 0x7fb8d803b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:32:00.941 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.936+0000 7fb8ef7fe700 1 --2- 172.21.15.53:0/1809594862 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fb8d8038d40 0x7fb8d803b1f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4009ec0 tx=0x7fb8e4009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:32:00.942 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:00.940+0000 7fb8ed7fa700 1 -- 172.21.15.53:0/1809594862 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb8e002a770 con 0x7fb8f0100920 2023-11-16T16:32:01.171 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:01.169+0000 7fb8f6578700 1 -- 172.21.15.53:0/1809594862 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi001=x;smithi026=y;smithi053=z", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8f0061ae0 con 0x7fb8d8038d40 2023-11-16T16:32:01.178 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:01.176+0000 7fb8ed7fa700 1 -- 172.21.15.53:0/1809594862 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fb8f0061ae0 con 0x7fb8d8038d40 2023-11-16T16:32:01.178 INFO:teuthology.orchestra.run.smithi053.stdout:Scheduled mgr update... 2023-11-16T16:32:01.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:01.178+0000 7fb8f6578700 1 -- 172.21.15.53:0/1809594862 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fb8d8038d40 msgr2=0x7fb8d803b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:32:01.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:01.178+0000 7fb8f6578700 1 --2- 172.21.15.53:0/1809594862 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fb8d8038d40 0x7fb8d803b1f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4009ec0 tx=0x7fb8e4009210 comp rx=0 tx=0).stop 2023-11-16T16:32:01.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:01.178+0000 7fb8f6578700 1 -- 172.21.15.53:0/1809594862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8f0100920 msgr2=0x7fb8f0195040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:32:01.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:01.178+0000 7fb8f6578700 1 --2- 172.21.15.53:0/1809594862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8f0100920 0x7fb8f0195040 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0005950 tx=0x7fb8e0004cb0 comp rx=0 tx=0).stop 2023-11-16T16:32:01.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:01.178+0000 7fb8f6578700 1 -- 172.21.15.53:0/1809594862 shutdown_connections 2023-11-16T16:32:01.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:01.178+0000 7fb8f6578700 1 --2- 172.21.15.53:0/1809594862 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fb8d8038d40 0x7fb8d803b1f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:32:01.181 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:01.178+0000 7fb8f6578700 1 --2- 172.21.15.53:0/1809594862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8f01959f0 0x7fb8f019cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:32:01.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:01.178+0000 7fb8f6578700 1 --2- 172.21.15.53:0/1809594862 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb8f0100920 0x7fb8f0195040 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:32:01.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:01.178+0000 7fb8f6578700 1 --2- 172.21.15.53:0/1809594862 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb8f0195580 0x7fb8f019a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:32:01.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:01.179+0000 7fb8f6578700 1 -- 172.21.15.53:0/1809594862 >> 172.21.15.53:0/1809594862 conn(0x7fb8f00fa2c0 msgr2=0x7fb8f00faf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:32:01.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:01.179+0000 7fb8f6578700 1 -- 172.21.15.53:0/1809594862 shutdown_connections 2023-11-16T16:32:01.182 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:01.179+0000 7fb8f6578700 1 -- 172.21.15.53:0/1809594862 wait complete. 2023-11-16T16:32:01.658 DEBUG:teuthology.orchestra.run.smithi026:mgr.y> sudo journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mgr.y.service 2023-11-16T16:32:01.661 DEBUG:teuthology.orchestra.run.smithi053:mgr.z> sudo journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mgr.z.service 2023-11-16T16:32:01.664 INFO:tasks.cephadm:Deploying OSDs... 2023-11-16T16:32:01.664 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2023-11-16T16:32:01.664 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/scratch_devs of=/dev/stdout 2023-11-16T16:32:01.686 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-16T16:32:01.686 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_1 2023-11-16T16:32:01.693 INFO:journalctl@ceph.mgr.z.smithi053.stdout:-- Logs begin at Thu 2023-11-16 16:14:15 UTC. -- 2023-11-16T16:32:01.696 INFO:journalctl@ceph.mgr.y.smithi026.stdout:-- Logs begin at Thu 2023-11-16 16:14:13 UTC. -- 2023-11-16T16:32:01.748 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-16T16:32:01.748 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-16T16:32:01.748 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 6h/6d Inode: 381545 Links: 1 2023-11-16T16:32:01.748 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-16T16:32:01.748 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2023-11-16T16:32:01.748 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2023-11-16 16:31:12.803461656 +0000 2023-11-16T16:32:01.748 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2023-11-16 16:24:16.790236464 +0000 2023-11-16T16:32:01.748 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2023-11-16 16:24:16.790236464 +0000 2023-11-16T16:32:01.748 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: - 2023-11-16T16:32:01.749 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-16T16:32:01.820 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2023-11-16T16:32:01.820 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2023-11-16T16:32:01.820 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000130265 s, 3.9 MB/s 2023-11-16T16:32:01.821 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-16T16:32:01.882 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_2 2023-11-16T16:32:01.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:01 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:01.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:01 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:01.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:01 smithi001 ceph-mon[98800]: Reconfiguring mon.c (monmap changed)... 2023-11-16T16:32:01.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:01 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-16T16:32:01.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:01 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-16T16:32:01.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:01 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:01.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:01 smithi001 ceph-mon[98800]: Reconfiguring daemon mon.c on smithi053 2023-11-16T16:32:01.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:01 smithi001 ceph-mon[98800]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi001=x;smithi026=y;smithi053=z", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:32:01.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:01 smithi001 ceph-mon[98800]: Saving service mgr spec with placement smithi001=x;smithi026=y;smithi053=z;count:3 2023-11-16T16:32:01.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:01 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:01.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:01 smithi001 ceph-mon[98800]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:01.942 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-16T16:32:01.942 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-16T16:32:01.942 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 6h/6d Inode: 388639 Links: 1 2023-11-16T16:32:01.943 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-16T16:32:01.943 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2023-11-16T16:32:01.943 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2023-11-16 16:31:12.803461656 +0000 2023-11-16T16:32:01.943 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2023-11-16 16:24:17.301230140 +0000 2023-11-16T16:32:01.943 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2023-11-16 16:24:17.301230140 +0000 2023-11-16T16:32:01.943 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: - 2023-11-16T16:32:01.943 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-16T16:32:02.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:01 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:02.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:01 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:02.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:01 smithi053 ceph-mon[100638]: Reconfiguring mon.c (monmap changed)... 2023-11-16T16:32:02.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:01 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-16T16:32:02.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:01 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-16T16:32:02.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:01 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:02.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:01 smithi053 ceph-mon[100638]: Reconfiguring daemon mon.c on smithi053 2023-11-16T16:32:02.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:01 smithi053 ceph-mon[100638]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi001=x;smithi026=y;smithi053=z", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:32:02.007 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:01 smithi053 ceph-mon[100638]: Saving service mgr spec with placement smithi001=x;smithi026=y;smithi053=z;count:3 2023-11-16T16:32:02.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:01 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:02.008 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:01 smithi053 ceph-mon[100638]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:02.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:01 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:02.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:01 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:02.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:01 smithi026 ceph-mon[100112]: Reconfiguring mon.c (monmap changed)... 2023-11-16T16:32:02.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:01 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-16T16:32:02.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:01 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-16T16:32:02.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:01 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:02.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:01 smithi026 ceph-mon[100112]: Reconfiguring daemon mon.c on smithi053 2023-11-16T16:32:02.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:01 smithi026 ceph-mon[100112]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi001=x;smithi026=y;smithi053=z", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:32:02.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:01 smithi026 ceph-mon[100112]: Saving service mgr spec with placement smithi001=x;smithi026=y;smithi053=z;count:3 2023-11-16T16:32:02.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:01 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:02.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:01 smithi026 ceph-mon[100112]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:02.012 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2023-11-16T16:32:02.012 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2023-11-16T16:32:02.012 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000120645 s, 4.2 MB/s 2023-11-16T16:32:02.014 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-16T16:32:02.073 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_3 2023-11-16T16:32:02.132 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-16T16:32:02.132 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-16T16:32:02.132 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 6h/6d Inode: 389510 Links: 1 2023-11-16T16:32:02.132 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-16T16:32:02.132 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2023-11-16T16:32:02.132 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2023-11-16 16:31:12.803461656 +0000 2023-11-16T16:32:02.132 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2023-11-16 16:24:17.821223705 +0000 2023-11-16T16:32:02.132 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2023-11-16 16:24:17.821223705 +0000 2023-11-16T16:32:02.133 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: - 2023-11-16T16:32:02.133 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-16T16:32:02.200 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2023-11-16T16:32:02.200 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2023-11-16T16:32:02.201 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.00011613 s, 4.4 MB/s 2023-11-16T16:32:02.201 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-16T16:32:02.260 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_4 2023-11-16T16:32:02.318 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-16T16:32:02.318 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-16T16:32:02.319 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 6h/6d Inode: 393890 Links: 1 2023-11-16T16:32:02.319 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-16T16:32:02.319 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2023-11-16T16:32:02.319 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2023-11-16 16:31:12.803461656 +0000 2023-11-16T16:32:02.319 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2023-11-16 16:24:18.347217195 +0000 2023-11-16T16:32:02.319 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2023-11-16 16:24:18.347217195 +0000 2023-11-16T16:32:02.319 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: - 2023-11-16T16:32:02.319 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-16T16:32:02.385 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2023-11-16T16:32:02.385 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2023-11-16T16:32:02.385 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.00010512 s, 4.9 MB/s 2023-11-16T16:32:02.386 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-16T16:32:02.445 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-11-16T16:32:02.445 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/scratch_devs of=/dev/stdout 2023-11-16T16:32:02.467 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-16T16:32:02.467 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_1 2023-11-16T16:32:02.529 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-16T16:32:02.529 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-16T16:32:02.529 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 6h/6d Inode: 379815 Links: 1 2023-11-16T16:32:02.529 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-16T16:32:02.529 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2023-11-16T16:32:02.529 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2023-11-16 16:31:49.618888653 +0000 2023-11-16T16:32:02.529 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2023-11-16 16:24:56.362994663 +0000 2023-11-16T16:32:02.529 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2023-11-16 16:24:56.362994663 +0000 2023-11-16T16:32:02.529 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: - 2023-11-16T16:32:02.530 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-16T16:32:02.601 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2023-11-16T16:32:02.601 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2023-11-16T16:32:02.601 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000127729 s, 4.0 MB/s 2023-11-16T16:32:02.602 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-16T16:32:02.664 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_2 2023-11-16T16:32:02.725 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-16T16:32:02.725 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-16T16:32:02.725 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 6h/6d Inode: 385718 Links: 1 2023-11-16T16:32:02.726 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-16T16:32:02.726 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2023-11-16T16:32:02.726 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2023-11-16 16:31:49.618888653 +0000 2023-11-16T16:32:02.726 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2023-11-16 16:24:56.887986518 +0000 2023-11-16T16:32:02.726 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2023-11-16 16:24:56.887986518 +0000 2023-11-16T16:32:02.726 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: - 2023-11-16T16:32:02.726 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-16T16:32:02.796 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2023-11-16T16:32:02.796 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2023-11-16T16:32:02.797 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000124003 s, 4.1 MB/s 2023-11-16T16:32:02.798 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-16T16:32:02.859 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_3 2023-11-16T16:32:02.919 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-16T16:32:02.920 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-16T16:32:02.920 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 6h/6d Inode: 387718 Links: 1 2023-11-16T16:32:02.920 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-16T16:32:02.920 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2023-11-16T16:32:02.920 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2023-11-16 16:31:49.618888653 +0000 2023-11-16T16:32:02.920 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2023-11-16 16:24:57.403978513 +0000 2023-11-16T16:32:02.920 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2023-11-16 16:24:57.403978513 +0000 2023-11-16T16:32:02.920 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: - 2023-11-16T16:32:02.920 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-16T16:32:02.991 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2023-11-16T16:32:02.991 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2023-11-16T16:32:02.991 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000119135 s, 4.3 MB/s 2023-11-16T16:32:02.993 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-16T16:32:03.055 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_4 2023-11-16T16:32:03.117 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-16T16:32:03.117 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-16T16:32:03.117 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 6h/6d Inode: 388674 Links: 1 2023-11-16T16:32:03.117 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-16T16:32:03.117 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2023-11-16T16:32:03.117 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2023-11-16 16:31:49.618888653 +0000 2023-11-16T16:32:03.117 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2023-11-16 16:24:57.911970633 +0000 2023-11-16T16:32:03.117 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2023-11-16 16:24:57.911970633 +0000 2023-11-16T16:32:03.117 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: - 2023-11-16T16:32:03.118 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-16T16:32:03.188 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2023-11-16T16:32:03.188 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2023-11-16T16:32:03.189 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000130716 s, 3.9 MB/s 2023-11-16T16:32:03.190 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-16T16:32:03.251 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2023-11-16T16:32:03.251 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/scratch_devs of=/dev/stdout 2023-11-16T16:32:03.269 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-16T16:32:03.269 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_1 2023-11-16T16:32:03.327 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-16T16:32:03.327 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-16T16:32:03.327 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 6h/6d Inode: 390647 Links: 1 2023-11-16T16:32:03.327 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-16T16:32:03.327 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2023-11-16T16:32:03.327 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2023-11-16 16:31:40.950030954 +0000 2023-11-16T16:32:03.327 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2023-11-16 16:24:50.051096647 +0000 2023-11-16T16:32:03.327 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2023-11-16 16:24:50.051096647 +0000 2023-11-16T16:32:03.328 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: - 2023-11-16T16:32:03.328 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-16T16:32:03.398 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2023-11-16T16:32:03.398 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2023-11-16T16:32:03.399 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 9.9942e-05 s, 5.1 MB/s 2023-11-16T16:32:03.400 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-16T16:32:03.459 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_2 2023-11-16T16:32:03.518 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-16T16:32:03.518 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-16T16:32:03.518 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 6h/6d Inode: 388336 Links: 1 2023-11-16T16:32:03.519 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-16T16:32:03.519 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2023-11-16T16:32:03.519 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2023-11-16 16:31:40.950030954 +0000 2023-11-16T16:32:03.519 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2023-11-16 16:24:50.651085691 +0000 2023-11-16T16:32:03.519 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2023-11-16 16:24:50.651085691 +0000 2023-11-16T16:32:03.519 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: - 2023-11-16T16:32:03.519 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-16T16:32:03.584 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2023-11-16T16:32:03.584 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2023-11-16T16:32:03.584 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 9.3686e-05 s, 5.5 MB/s 2023-11-16T16:32:03.585 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-16T16:32:03.643 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_3 2023-11-16T16:32:03.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:03 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:03.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:03 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:03.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:03 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:32:03.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:03 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:03.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:03 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:32:03.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:03 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:03.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:03 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-16T16:32:03.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:03 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-16T16:32:03.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:03 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-16T16:32:03.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:03 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:03.701 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-16T16:32:03.701 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-16T16:32:03.701 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 6h/6d Inode: 394245 Links: 1 2023-11-16T16:32:03.701 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-16T16:32:03.701 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2023-11-16T16:32:03.701 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2023-11-16 16:31:40.951030935 +0000 2023-11-16T16:32:03.701 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2023-11-16 16:24:51.200075666 +0000 2023-11-16T16:32:03.701 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2023-11-16 16:24:51.200075666 +0000 2023-11-16T16:32:03.701 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: - 2023-11-16T16:32:03.701 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-16T16:32:03.716 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:03 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:03.716 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:03 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:03.716 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:03 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:32:03.716 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:03 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:03.716 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:03 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:32:03.716 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:03 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:03.716 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:03 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-16T16:32:03.716 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:03 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-16T16:32:03.716 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:03 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-16T16:32:03.717 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:03 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:03.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:03 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:03.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:03 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:03.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:03 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:32:03.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:03 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:03.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:03 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:32:03.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:03 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:03.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:03 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-16T16:32:03.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:03 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-16T16:32:03.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:03 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-16T16:32:03.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:03 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:03.821 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2023-11-16T16:32:03.821 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2023-11-16T16:32:03.821 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000118158 s, 4.3 MB/s 2023-11-16T16:32:03.822 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-16T16:32:03.840 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_4 2023-11-16T16:32:03.900 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-16T16:32:03.901 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-16T16:32:03.901 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 6h/6d Inode: 395285 Links: 1 2023-11-16T16:32:03.901 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-16T16:32:03.901 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2023-11-16T16:32:03.901 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2023-11-16 16:32:03.766588203 +0000 2023-11-16T16:32:03.901 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2023-11-16 16:24:51.724066098 +0000 2023-11-16T16:32:03.901 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2023-11-16 16:24:51.724066098 +0000 2023-11-16T16:32:03.901 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: - 2023-11-16T16:32:03.901 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-16T16:32:03.966 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:32:03 smithi053 systemd[1]: Starting Ceph mgr.z for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:32:03.969 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2023-11-16T16:32:03.969 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2023-11-16T16:32:03.969 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000115088 s, 4.4 MB/s 2023-11-16T16:32:03.971 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-16T16:32:04.032 INFO:tasks.cephadm:Deploying osd.0 on smithi001 with /dev/vg_nvme/lv_4... 2023-11-16T16:32:04.032 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-11-16T16:32:04.501 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:04 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:32:04.501 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:04 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:32:04.501 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:04 smithi053 ceph-mon[100638]: Deploying daemon mgr.z on smithi053 2023-11-16T16:32:04.502 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:04 smithi053 ceph-mon[100638]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:04.502 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:32:04 smithi053 podman[102737]: 2023-11-16T16:32:04.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:04 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:32:04.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:04 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:32:04.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:04 smithi001 ceph-mon[98800]: Deploying daemon mgr.z on smithi053 2023-11-16T16:32:04.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:04 smithi001 ceph-mon[98800]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:04.694 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:32:04.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:04 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:32:04.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:04 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:32:04.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:04 smithi026 ceph-mon[100112]: Deploying daemon mgr.z on smithi053 2023-11-16T16:32:04.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:04 smithi026 ceph-mon[100112]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:04.831 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:32:04 smithi053 ceph-mgr[102777]: -- 172.21.15.53:0/3763860418 <== mon.1 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d2af3b9520 con 0x55d2af38e800 2023-11-16T16:32:04.831 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:32:04 smithi053 bash[102737]: c996c0f1fc57dca5041ceb639b384f48ea5171ef211fb298f4beafcdbd718e99 2023-11-16T16:32:04.831 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:32:04 smithi053 systemd[1]: Started Ceph mgr.z for 35d9b010-849d-11ee-8dbe-212e2dc638e7. 2023-11-16T16:32:04.831 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:32:04 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:32:04.641+0000 7f84f9d5b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-16T16:32:04.831 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:32:04 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:32:04.736+0000 7f84f9d5b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-16T16:32:05.160 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:32:04 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:32:04.843+0000 7f84f9d5b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-16T16:32:05.160 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:32:04 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:32:04.953+0000 7f84f9d5b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-16T16:32:05.554 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:32:05 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:32:05.158+0000 7f84f9d5b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-16T16:32:05.807 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:05 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:05.807 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:05 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:05.807 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:05 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:05.807 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:05 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-16T16:32:05.807 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:05 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-16T16:32:05.807 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:05 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-16T16:32:05.807 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:05 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:05.808 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:05 smithi001 ceph-mon[98800]: Deploying daemon mgr.y on smithi026 2023-11-16T16:32:05.808 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:05 smithi001 ceph-mon[98800]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:05.821 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:32:05 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:32:05.552+0000 7f84f9d5b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-16T16:32:05.821 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:32:05 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:32:05.661+0000 7f84f9d5b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-16T16:32:05.821 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:05 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:05.821 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:05 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:05.821 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:05 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:05.821 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:05 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-16T16:32:05.821 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:05 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-16T16:32:05.822 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:05 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-16T16:32:05.822 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:05 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:05.822 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:05 smithi053 ceph-mon[100638]: Deploying daemon mgr.y on smithi026 2023-11-16T16:32:05.822 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:05 smithi053 ceph-mon[100638]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:05.873 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:05 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:05.873 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:05 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:05.874 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:05 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:05.874 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:05 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-16T16:32:05.874 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:05 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-16T16:32:05.874 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:05 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-16T16:32:05.874 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:05 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:05.874 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:05 smithi026 ceph-mon[100112]: Deploying daemon mgr.y on smithi026 2023-11-16T16:32:05.874 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:05 smithi026 ceph-mon[100112]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:06.080 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:32:05 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:32:05.819+0000 7f84f9d5b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-16T16:32:06.081 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:32:05 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:32:05.935+0000 7f84f9d5b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-16T16:32:06.139 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:32:06 smithi026 systemd[1]: Starting Ceph mgr.y for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:32:06.307 INFO:teuthology.orchestra.run.smithi001.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-11-16T16:32:06.307 INFO:teuthology.orchestra.run.smithi001.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-11-16T16:32:06.307 INFO:teuthology.orchestra.run.smithi001.stderr: stderr: 10+0 records in 2023-11-16T16:32:06.307 INFO:teuthology.orchestra.run.smithi001.stderr:10+0 records out 2023-11-16T16:32:06.307 INFO:teuthology.orchestra.run.smithi001.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00849035 s, 1.2 GB/s 2023-11-16T16:32:06.307 INFO:teuthology.orchestra.run.smithi001.stderr:--> Zapping successful for: 2023-11-16T16:32:06.656 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph orch daemon add osd smithi001:vg_nvme/lv_4 2023-11-16T16:32:06.716 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:32:06 smithi026 podman[101550]: 2023-11-16T16:32:06.830 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:32:06 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:32:06.456+0000 7f84f9d5b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-16T16:32:06.967 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:32:06 smithi026 bash[101550]: 298db30201e1462feac2d3ebd192df8c96867da410a81b81dfc66db0a482a2d1 2023-11-16T16:32:06.967 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:32:06 smithi026 ceph-mgr[101590]: -- 172.21.15.26:0/3031877357 <== mon.2 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5601bf22f520 con 0x5601bf204c00 2023-11-16T16:32:06.968 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:32:06 smithi026 systemd[1]: Started Ceph mgr.y for 35d9b010-849d-11ee-8dbe-212e2dc638e7. 2023-11-16T16:32:06.968 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:32:06 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:32:06.870+0000 7f10a96ee200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-16T16:32:06.968 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:32:06 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:32:06.966+0000 7f10a96ee200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-16T16:32:07.258 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:32:07 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:32:07.073+0000 7f10a96ee200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-16T16:32:07.258 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:32:07 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:32:07.188+0000 7f10a96ee200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-16T16:32:07.316 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:32:07.581 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:32:07 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:32:07.164+0000 7f84f9d5b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-16T16:32:07.581 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:32:07 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:32:07.360+0000 7f84f9d5b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-16T16:32:07.758 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:32:07 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:32:07.403+0000 7f10a96ee200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-16T16:32:07.916 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:07 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:07.917 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:07 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:07.917 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:07 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:07.917 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:07 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:07.917 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:07 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:32:07.917 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:07 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:32:07.918 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:07 smithi001 ceph-mon[98800]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:08.072 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:32:07 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:32:07.807+0000 7f10a96ee200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-16T16:32:08.072 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:32:07 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:32:07.916+0000 7f10a96ee200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-16T16:32:08.072 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:07 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:08.073 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:07 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:08.073 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:07 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:08.073 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:07 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:08.073 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:07 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:32:08.073 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:07 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:32:08.074 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:07 smithi026 ceph-mon[100112]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:08.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:07 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:08.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:07 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:08.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:07 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:08.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:07 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:08.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:07 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:32:08.082 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:07 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:32:08.082 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:07 smithi053 ceph-mon[100638]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:08.508 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:32:08 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:32:08.070+0000 7f10a96ee200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-16T16:32:08.508 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:32:08 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:32:08.190+0000 7f10a96ee200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-16T16:32:08.651 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.647+0000 7f8ccbbd3700 1 -- 172.21.15.1:0/2393388333 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cc41003e0 msgr2=0x7f8cc4100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:32:08.652 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.647+0000 7f8ccbbd3700 1 --2- 172.21.15.1:0/2393388333 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cc41003e0 0x7f8cc4100830 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4009990 tx=0x7f8cb4009ca0 comp rx=0 tx=0).stop 2023-11-16T16:32:08.652 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.647+0000 7f8ccbbd3700 1 -- 172.21.15.1:0/2393388333 shutdown_connections 2023-11-16T16:32:08.652 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.647+0000 7f8ccbbd3700 1 --2- 172.21.15.1:0/2393388333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cc4100d70 0x7f8cc4107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:32:08.652 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.647+0000 7f8ccbbd3700 1 --2- 172.21.15.1:0/2393388333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cc40ff1e0 0x7f8cc40ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:32:08.652 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.647+0000 7f8ccbbd3700 1 --2- 172.21.15.1:0/2393388333 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cc41003e0 0x7f8cc4100830 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:32:08.652 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.647+0000 7f8ccbbd3700 1 -- 172.21.15.1:0/2393388333 >> 172.21.15.1:0/2393388333 conn(0x7f8cc40fa7f0 msgr2=0x7f8cc40fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:32:08.653 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.647+0000 7f8ccbbd3700 1 -- 172.21.15.1:0/2393388333 shutdown_connections 2023-11-16T16:32:08.653 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.647+0000 7f8ccbbd3700 1 -- 172.21.15.1:0/2393388333 wait complete. 2023-11-16T16:32:08.653 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.648+0000 7f8ccbbd3700 1 Processor -- start 2023-11-16T16:32:08.653 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.648+0000 7f8ccbbd3700 1 -- start start 2023-11-16T16:32:08.654 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.648+0000 7f8ccbbd3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cc40ff1e0 0x7f8cc4196060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:32:08.654 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.648+0000 7f8ccbbd3700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cc41003e0 0x7f8cc41965a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:32:08.654 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.648+0000 7f8ccbbd3700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cc4100d70 0x7f8cc419d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:32:08.654 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.648+0000 7f8ccbbd3700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc419dbf0 con 0x7f8cc41003e0 2023-11-16T16:32:08.654 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.648+0000 7f8ccbbd3700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc419dd30 con 0x7f8cc40ff1e0 2023-11-16T16:32:08.654 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.648+0000 7f8ccbbd3700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc419dec0 con 0x7f8cc4100d70 2023-11-16T16:32:08.655 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.648+0000 7f8cc916e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cc41003e0 0x7f8cc41965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:32:08.655 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.648+0000 7f8cc916e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cc41003e0 0x7f8cc41965a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40540/0 (socket says 172.21.15.1:40540) 2023-11-16T16:32:08.655 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.648+0000 7f8cc916e700 1 -- 172.21.15.1:0/4114200644 learned_addr learned my addr 172.21.15.1:0/4114200644 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:32:08.655 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.648+0000 7f8cc996f700 1 --2- 172.21.15.1:0/4114200644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cc40ff1e0 0x7f8cc4196060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:32:08.655 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.648+0000 7f8cca170700 1 --2- 172.21.15.1:0/4114200644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cc4100d70 0x7f8cc419d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:32:08.655 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.649+0000 7f8cc916e700 1 -- 172.21.15.1:0/4114200644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cc4100d70 msgr2=0x7f8cc419d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:32:08.656 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.649+0000 7f8cc916e700 1 --2- 172.21.15.1:0/4114200644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cc4100d70 0x7f8cc419d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:32:08.656 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.649+0000 7f8cc916e700 1 -- 172.21.15.1:0/4114200644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cc40ff1e0 msgr2=0x7f8cc4196060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:32:08.656 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.649+0000 7f8cc916e700 1 --2- 172.21.15.1:0/4114200644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cc40ff1e0 0x7f8cc4196060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:32:08.656 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.649+0000 7f8cc916e700 1 -- 172.21.15.1:0/4114200644 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb40095f0 con 0x7f8cc41003e0 2023-11-16T16:32:08.656 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.649+0000 7f8cc996f700 1 --2- 172.21.15.1:0/4114200644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cc40ff1e0 0x7f8cc4196060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:32:08.656 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.649+0000 7f8cc916e700 1 --2- 172.21.15.1:0/4114200644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cc41003e0 0x7f8cc41965a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4005910 tx=0x7f8cb4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:32:08.657 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.649+0000 7f8cbaffd700 1 -- 172.21.15.1:0/4114200644 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb401d070 con 0x7f8cc41003e0 2023-11-16T16:32:08.657 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.649+0000 7f8cbaffd700 1 -- 172.21.15.1:0/4114200644 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8cb4022470 con 0x7f8cc41003e0 2023-11-16T16:32:08.657 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.649+0000 7f8ccbbd3700 1 -- 172.21.15.1:0/4114200644 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc410ab40 con 0x7f8cc41003e0 2023-11-16T16:32:08.657 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.649+0000 7f8cbaffd700 1 -- 172.21.15.1:0/4114200644 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb4017650 con 0x7f8cc41003e0 2023-11-16T16:32:08.657 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.649+0000 7f8ccbbd3700 1 -- 172.21.15.1:0/4114200644 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc410a9b0 con 0x7f8cc41003e0 2023-11-16T16:32:08.657 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.650+0000 7f8cbaffd700 1 -- 172.21.15.1:0/4114200644 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f8cb40177b0 con 0x7f8cc41003e0 2023-11-16T16:32:08.658 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.650+0000 7f8ccbbd3700 1 -- 172.21.15.1:0/4114200644 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cc4107320 con 0x7f8cc41003e0 2023-11-16T16:32:08.659 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.650+0000 7f8cbaffd700 1 --2- 172.21.15.1:0/4114200644 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8cb00389e0 0x7f8cb003ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:32:08.659 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.650+0000 7f8cbaffd700 1 -- 172.21.15.1:0/4114200644 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8cb404c7e0 con 0x7f8cc41003e0 2023-11-16T16:32:08.659 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.650+0000 7f8cc996f700 1 --2- 172.21.15.1:0/4114200644 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8cb00389e0 0x7f8cb003ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:32:08.659 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.651+0000 7f8cc996f700 1 --2- 172.21.15.1:0/4114200644 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8cb00389e0 0x7f8cb003ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4100240 tx=0x7f8cc0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:32:08.660 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.655+0000 7f8cbaffd700 1 -- 172.21.15.1:0/4114200644 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8cb4016b90 con 0x7f8cc41003e0 2023-11-16T16:32:08.700 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:32:08 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:32:08.363+0000 7f84f9d5b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-16T16:32:08.700 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:32:08 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:32:08.475+0000 7f84f9d5b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-16T16:32:08.867 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:08.864+0000 7f8ccbbd3700 1 -- 172.21.15.1:0/4114200644 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f8cc4061c10 con 0x7f8cb00389e0 2023-11-16T16:32:09.008 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:32:08 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:32:08.721+0000 7f10a96ee200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-16T16:32:09.047 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:32:08 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:32:08.699+0000 7f84f9d5b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-16T16:32:09.581 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:32:09 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:32:09.046+0000 7f84f9d5b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-16T16:32:09.581 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:32:09 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:32:09.297+0000 7f84f9d5b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-16T16:32:09.581 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:32:09 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:32:09.477+0000 7f84f9d5b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-16T16:32:09.729 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:32:09 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:32:09.417+0000 7f10a96ee200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-16T16:32:09.729 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:32:09 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:32:09.608+0000 7f10a96ee200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-16T16:32:09.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:09 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:09.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:09 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:09.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:09 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:09.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:09 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:09.934 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:09 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:09.934 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:09 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:32:09.934 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:09 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:32:09.935 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:09 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:09.935 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:09 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:32:09.935 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:09 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:32:09.936 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:09 smithi001 ceph-mon[98800]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:32:09.936 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:09 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:32:09.937 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:09 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:32:09.937 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:09 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:09.937 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:09 smithi001 ceph-mon[98800]: Reconfiguring mgr.x (unknown last config time)... 2023-11-16T16:32:09.938 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:09 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-16T16:32:09.938 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:09 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-16T16:32:09.938 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:09 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:09.939 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:09 smithi001 ceph-mon[98800]: Reconfiguring daemon mgr.x on smithi001 2023-11-16T16:32:09.939 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:09 smithi001 ceph-mon[98800]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:10.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:09 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:10.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:09 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:10.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:09 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:10.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:09 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:10.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:09 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:10.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:09 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:32:10.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:09 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:32:10.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:09 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:10.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:09 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:32:10.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:09 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:32:10.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:09 smithi026 ceph-mon[100112]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:32:10.010 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:09 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:32:10.010 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:09 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:32:10.010 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:09 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:10.010 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:09 smithi026 ceph-mon[100112]: Reconfiguring mgr.x (unknown last config time)... 2023-11-16T16:32:10.010 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:09 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-16T16:32:10.010 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:09 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-16T16:32:10.010 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:09 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:10.010 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:09 smithi026 ceph-mon[100112]: Reconfiguring daemon mgr.x on smithi001 2023-11-16T16:32:10.010 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:09 smithi026 ceph-mon[100112]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:10.081 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:32:09 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:32:09.802+0000 7f84f9d5b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-16T16:32:10.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:09 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:10.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:09 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:10.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:09 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:10.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:09 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:10.082 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:09 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:10.082 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:09 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:32:10.082 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:09 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:32:10.082 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:09 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:10.082 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:09 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:32:10.083 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:09 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:32:10.083 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:09 smithi053 ceph-mon[100638]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:32:10.083 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:09 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:32:10.083 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:09 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:32:10.083 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:09 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:10.083 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:09 smithi053 ceph-mon[100638]: Reconfiguring mgr.x (unknown last config time)... 2023-11-16T16:32:10.083 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:09 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-16T16:32:10.083 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:09 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-16T16:32:10.083 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:09 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:10.083 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:09 smithi053 ceph-mon[100638]: Reconfiguring daemon mgr.x on smithi001 2023-11-16T16:32:10.083 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:09 smithi053 ceph-mon[100638]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:10.894 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:32:10 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:32:10.564+0000 7f10a96ee200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-16T16:32:10.894 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:32:10 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:32:10.674+0000 7f10a96ee200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-16T16:32:11.024 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:32:10 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:32:10.624+0000 7f84f9d5b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-16T16:32:11.024 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:32:10 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:32:10.781+0000 7f84f9d5b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-16T16:32:11.241 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:32:10 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:32:10.892+0000 7f10a96ee200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-16T16:32:11.331 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:32:11 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:32:11.022+0000 7f84f9d5b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-16T16:32:11.331 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:32:11 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:32:11.158+0000 7f84f9d5b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-16T16:32:11.493 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:32:11 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:32:11.239+0000 7f10a96ee200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-16T16:32:11.758 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:32:11 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:32:11.492+0000 7f10a96ee200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-16T16:32:11.758 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:32:11 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:32:11.677+0000 7f10a96ee200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-16T16:32:11.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:11 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:11.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:11 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:11.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:11 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:32:11.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:11 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:11.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:11 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:32:11.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:11 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:32:11.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:11 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:11.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:11 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:32:11.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:11 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:32:11.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:11 smithi026 ceph-mon[100112]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:11.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:11 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:11.831 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:32:11 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:32:11.428+0000 7f84f9d5b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-16T16:32:11.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:11 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:11.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:11 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:11.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:11 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:32:11.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:11 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:11.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:11 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:32:11.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:11 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:32:11.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:11 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:11.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:11 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:32:11.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:11 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:32:11.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:11 smithi053 ceph-mon[100638]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:11.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:11 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:11.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:11 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:11.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:11 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:11.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:11 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:32:11.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:11 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:11.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:11 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:32:11.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:11 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:32:11.934 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:11 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:11.934 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:11 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:32:11.934 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:11 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:32:11.934 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:11 smithi001 ceph-mon[98800]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:11.934 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:11 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:12.258 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:32:12 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:32:12.002+0000 7f10a96ee200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-16T16:32:12.825 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:12 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/564125151' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8480507-5275-4e52-953b-626110beb436"}]: dispatch 2023-11-16T16:32:12.825 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:12 smithi026 ceph-mon[100112]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8480507-5275-4e52-953b-626110beb436"}]: dispatch 2023-11-16T16:32:12.826 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:12 smithi026 ceph-mon[100112]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8480507-5275-4e52-953b-626110beb436"}]': finished 2023-11-16T16:32:12.826 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:12 smithi026 ceph-mon[100112]: osdmap e5: 1 total, 0 up, 1 in 2023-11-16T16:32:12.826 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:12 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-16T16:32:12.826 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:12 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/2605425405' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:32:12.826 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:12 smithi026 ceph-mon[100112]: Standby manager daemon z started 2023-11-16T16:32:12.827 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:12 smithi026 ceph-mon[100112]: from='mgr.? 172.21.15.53:0/1755785105' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-11-16T16:32:12.827 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:12 smithi026 ceph-mon[100112]: from='mgr.? 172.21.15.53:0/1755785105' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-16T16:32:12.827 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:12 smithi026 ceph-mon[100112]: from='mgr.? 172.21.15.53:0/1755785105' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-11-16T16:32:12.827 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:12 smithi026 ceph-mon[100112]: from='mgr.? 172.21.15.53:0/1755785105' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-16T16:32:12.831 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:32:12 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:32:12.467+0000 7f84f9d5b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-16T16:32:12.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:12 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/564125151' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8480507-5275-4e52-953b-626110beb436"}]: dispatch 2023-11-16T16:32:12.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:12 smithi053 ceph-mon[100638]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8480507-5275-4e52-953b-626110beb436"}]: dispatch 2023-11-16T16:32:12.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:12 smithi053 ceph-mon[100638]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8480507-5275-4e52-953b-626110beb436"}]': finished 2023-11-16T16:32:12.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:12 smithi053 ceph-mon[100638]: osdmap e5: 1 total, 0 up, 1 in 2023-11-16T16:32:12.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:12 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-16T16:32:12.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:12 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/2605425405' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:32:12.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:12 smithi053 ceph-mon[100638]: Standby manager daemon z started 2023-11-16T16:32:12.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:12 smithi053 ceph-mon[100638]: from='mgr.? 172.21.15.53:0/1755785105' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-11-16T16:32:12.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:12 smithi053 ceph-mon[100638]: from='mgr.? 172.21.15.53:0/1755785105' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-16T16:32:12.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:12 smithi053 ceph-mon[100638]: from='mgr.? 172.21.15.53:0/1755785105' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-11-16T16:32:12.833 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:12 smithi053 ceph-mon[100638]: from='mgr.? 172.21.15.53:0/1755785105' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-16T16:32:12.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:12 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/564125151' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8480507-5275-4e52-953b-626110beb436"}]: dispatch 2023-11-16T16:32:12.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:12 smithi001 ceph-mon[98800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8480507-5275-4e52-953b-626110beb436"}]: dispatch 2023-11-16T16:32:12.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:12 smithi001 ceph-mon[98800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8480507-5275-4e52-953b-626110beb436"}]': finished 2023-11-16T16:32:12.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:12 smithi001 ceph-mon[98800]: osdmap e5: 1 total, 0 up, 1 in 2023-11-16T16:32:12.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:12 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-16T16:32:12.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:12 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/2605425405' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:32:12.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:12 smithi001 ceph-mon[98800]: Standby manager daemon z started 2023-11-16T16:32:12.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:12 smithi001 ceph-mon[98800]: from='mgr.? 172.21.15.53:0/1755785105' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-11-16T16:32:12.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:12 smithi001 ceph-mon[98800]: from='mgr.? 172.21.15.53:0/1755785105' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-16T16:32:12.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:12 smithi001 ceph-mon[98800]: from='mgr.? 172.21.15.53:0/1755785105' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-11-16T16:32:12.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:12 smithi001 ceph-mon[98800]: from='mgr.? 172.21.15.53:0/1755785105' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-16T16:32:13.095 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:13.093+0000 7f8cbaffd700 1 -- 172.21.15.1:0/4114200644 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 14) v1 ==== 90598+0+0 (secure 0 0 0) 0x7f8cb40163a0 con 0x7f8cc41003e0 2023-11-16T16:32:13.225 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:32:12 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:32:12.823+0000 7f10a96ee200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-16T16:32:13.225 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:32:12 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:32:12.980+0000 7f10a96ee200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-16T16:32:13.508 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:32:13 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:32:13.223+0000 7f10a96ee200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-16T16:32:13.508 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:32:13 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:32:13.359+0000 7f10a96ee200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-16T16:32:14.008 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:32:13 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:32:13.630+0000 7f10a96ee200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-16T16:32:14.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:14 smithi001 ceph-mon[98800]: mgrmap e14: x(active, since 71s), standbys: z 2023-11-16T16:32:14.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:14 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-11-16T16:32:14.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:14 smithi001 ceph-mon[98800]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:14.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:14 smithi026 ceph-mon[100112]: mgrmap e14: x(active, since 71s), standbys: z 2023-11-16T16:32:14.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:14 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-11-16T16:32:14.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:14 smithi026 ceph-mon[100112]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:14.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:14 smithi053 ceph-mon[100638]: mgrmap e14: x(active, since 71s), standbys: z 2023-11-16T16:32:14.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:14 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-11-16T16:32:14.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:14 smithi053 ceph-mon[100638]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:15.008 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:32:14 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:32:14.668+0000 7f10a96ee200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-16T16:32:15.109 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:15.107+0000 7f8cbaffd700 1 -- 172.21.15.1:0/4114200644 <== mon.0 v2:172.21.15.1:3300/0 8 ==== mgrmap(e 15) v1 ==== 135658+0+0 (secure 0 0 0) 0x7f8cb40292c0 con 0x7f8cc41003e0 2023-11-16T16:32:15.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:15 smithi001 ceph-mon[98800]: Standby manager daemon y started 2023-11-16T16:32:15.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:15 smithi001 ceph-mon[98800]: from='mgr.? 172.21.15.26:0/1841963184' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-11-16T16:32:15.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:15 smithi001 ceph-mon[98800]: from='mgr.? 172.21.15.26:0/1841963184' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-16T16:32:15.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:15 smithi001 ceph-mon[98800]: from='mgr.? 172.21.15.26:0/1841963184' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-11-16T16:32:15.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:15 smithi001 ceph-mon[98800]: from='mgr.? 172.21.15.26:0/1841963184' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-16T16:32:15.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:15 smithi026 ceph-mon[100112]: Standby manager daemon y started 2023-11-16T16:32:15.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:15 smithi026 ceph-mon[100112]: from='mgr.? 172.21.15.26:0/1841963184' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-11-16T16:32:15.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:15 smithi026 ceph-mon[100112]: from='mgr.? 172.21.15.26:0/1841963184' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-16T16:32:15.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:15 smithi026 ceph-mon[100112]: from='mgr.? 172.21.15.26:0/1841963184' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-11-16T16:32:15.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:15 smithi026 ceph-mon[100112]: from='mgr.? 172.21.15.26:0/1841963184' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-16T16:32:15.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:15 smithi053 ceph-mon[100638]: Standby manager daemon y started 2023-11-16T16:32:15.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:15 smithi053 ceph-mon[100638]: from='mgr.? 172.21.15.26:0/1841963184' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-11-16T16:32:15.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:15 smithi053 ceph-mon[100638]: from='mgr.? 172.21.15.26:0/1841963184' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-16T16:32:15.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:15 smithi053 ceph-mon[100638]: from='mgr.? 172.21.15.26:0/1841963184' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-11-16T16:32:15.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:15 smithi053 ceph-mon[100638]: from='mgr.? 172.21.15.26:0/1841963184' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-16T16:32:16.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:16 smithi001 ceph-mon[98800]: mgrmap e15: x(active, since 73s), standbys: y, z 2023-11-16T16:32:16.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:16 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-11-16T16:32:16.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:16 smithi001 ceph-mon[98800]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:16.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:16 smithi026 ceph-mon[100112]: mgrmap e15: x(active, since 73s), standbys: y, z 2023-11-16T16:32:16.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:16 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-11-16T16:32:16.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:16 smithi026 ceph-mon[100112]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:16.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:16 smithi053 ceph-mon[100638]: mgrmap e15: x(active, since 73s), standbys: y, z 2023-11-16T16:32:16.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:16 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-11-16T16:32:16.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:16 smithi053 ceph-mon[100638]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:17.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:17 smithi026 ceph-mon[100112]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:17.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:17 smithi053 ceph-mon[100638]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:17.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:17 smithi001 ceph-mon[98800]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:18.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:18 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-11-16T16:32:18.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:18 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:19.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:18 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-11-16T16:32:19.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:18 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:19.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:18 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-11-16T16:32:19.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:18 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:19.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:19 smithi001 ceph-mon[98800]: Deploying daemon osd.0 on smithi001 2023-11-16T16:32:19.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:19 smithi001 ceph-mon[98800]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:20.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:19 smithi026 ceph-mon[100112]: Deploying daemon osd.0 on smithi001 2023-11-16T16:32:20.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:19 smithi026 ceph-mon[100112]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:20.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:19 smithi053 ceph-mon[100638]: Deploying daemon osd.0 on smithi001 2023-11-16T16:32:20.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:19 smithi053 ceph-mon[100638]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:22.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:22 smithi001 ceph-mon[98800]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:22.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:22 smithi026 ceph-mon[100112]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:22.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:22 smithi053 ceph-mon[100638]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:24.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:24 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:24.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:24 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:24.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:24 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:32:24.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:24 smithi001 ceph-mon[98800]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:24.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:24 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:24.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:24 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:24.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:24 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:32:24.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:24 smithi026 ceph-mon[100112]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:24.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:24 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:24.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:24 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:24.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:24 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:32:24.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:24 smithi053 ceph-mon[100638]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:25.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:25 smithi026 ceph-mon[100112]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:25.814 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:25 smithi001 ceph-mon[98800]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:25.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:25 smithi053 ceph-mon[100638]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:26.170 INFO:teuthology.orchestra.run.smithi001.stdout:Created osd(s) 0 on host 'smithi001' 2023-11-16T16:32:26.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:26.168+0000 7f8cbaffd700 1 -- 172.21.15.1:0/4114200644 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8cc4061c10 con 0x7f8cb00389e0 2023-11-16T16:32:26.173 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:26.170+0000 7f8ccbbd3700 1 -- 172.21.15.1:0/4114200644 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8cb00389e0 msgr2=0x7f8cb003ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:32:26.173 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:26.170+0000 7f8ccbbd3700 1 --2- 172.21.15.1:0/4114200644 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8cb00389e0 0x7f8cb003ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4100240 tx=0x7f8cc0009260 comp rx=0 tx=0).stop 2023-11-16T16:32:26.173 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:26.170+0000 7f8ccbbd3700 1 -- 172.21.15.1:0/4114200644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cc41003e0 msgr2=0x7f8cc41965a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:32:26.173 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:26.170+0000 7f8ccbbd3700 1 --2- 172.21.15.1:0/4114200644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cc41003e0 0x7f8cc41965a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4005910 tx=0x7f8cb4009580 comp rx=0 tx=0).stop 2023-11-16T16:32:26.173 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:26.171+0000 7f8ccbbd3700 1 -- 172.21.15.1:0/4114200644 shutdown_connections 2023-11-16T16:32:26.173 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:26.171+0000 7f8ccbbd3700 1 --2- 172.21.15.1:0/4114200644 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8cb00389e0 0x7f8cb003ae90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:32:26.174 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:26.171+0000 7f8ccbbd3700 1 --2- 172.21.15.1:0/4114200644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8cc4100d70 0x7f8cc419d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:32:26.174 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:26.171+0000 7f8ccbbd3700 1 --2- 172.21.15.1:0/4114200644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8cc40ff1e0 0x7f8cc4196060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:32:26.174 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:26.171+0000 7f8ccbbd3700 1 --2- 172.21.15.1:0/4114200644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cc41003e0 0x7f8cc41965a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:32:26.174 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:26.171+0000 7f8ccbbd3700 1 -- 172.21.15.1:0/4114200644 >> 172.21.15.1:0/4114200644 conn(0x7f8cc40fa7f0 msgr2=0x7f8cc40fca70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:32:26.174 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:26.171+0000 7f8ccbbd3700 1 -- 172.21.15.1:0/4114200644 shutdown_connections 2023-11-16T16:32:26.174 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:32:26.171+0000 7f8ccbbd3700 1 -- 172.21.15.1:0/4114200644 wait complete. 2023-11-16T16:32:26.622 DEBUG:teuthology.orchestra.run.smithi001:osd.0> sudo journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@osd.0.service 2023-11-16T16:32:26.624 INFO:tasks.cephadm:Deploying osd.1 on smithi026 with /dev/vg_nvme/lv_4... 2023-11-16T16:32:26.625 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-11-16T16:32:26.658 INFO:journalctl@ceph.osd.0.smithi001.stdout:-- Logs begin at Thu 2023-11-16 16:14:27 UTC. -- 2023-11-16T16:32:26.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:26 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:26.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:26 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:26.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:26 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:26.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:26 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:32:26.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:26 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:32:26.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:26 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:26.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:26 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:32:26.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:26 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:32:26.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:26 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:26.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:26 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:26.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:26 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:32:27.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:26 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:27.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:26 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:27.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:26 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:27.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:26 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:32:27.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:26 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:32:27.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:26 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:27.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:26 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:32:27.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:26 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:32:27.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:26 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:27.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:26 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:27.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:26 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:32:27.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:26 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:27.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:26 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:27.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:26 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:27.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:26 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:32:27.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:26 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:32:27.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:26 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:27.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:26 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:32:27.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:26 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:32:27.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:26 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:27.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:26 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:27.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:26 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:32:27.276 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.b/config 2023-11-16T16:32:27.432 INFO:journalctl@ceph.osd.0.smithi001.stdout:Nov 16 16:32:27 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-0[111670]: 2023-11-16T16:32:27.134+0000 7fab06810640 -1 osd.0 0 log_to_monitors true 2023-11-16T16:32:27.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:27 smithi026 ceph-mon[100112]: from='osd.0 [v2:172.21.15.1:6802/2469671609,v1:172.21.15.1:6803/2469671609]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-11-16T16:32:27.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:27 smithi026 ceph-mon[100112]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:27.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:27 smithi001 ceph-mon[98800]: from='osd.0 [v2:172.21.15.1:6802/2469671609,v1:172.21.15.1:6803/2469671609]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-11-16T16:32:27.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:27 smithi001 ceph-mon[98800]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:28.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:27 smithi053 ceph-mon[100638]: from='osd.0 [v2:172.21.15.1:6802/2469671609,v1:172.21.15.1:6803/2469671609]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-11-16T16:32:28.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:27 smithi053 ceph-mon[100638]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:28.805 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-11-16T16:32:28.806 INFO:teuthology.orchestra.run.smithi026.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-11-16T16:32:28.806 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10+0 records in 2023-11-16T16:32:28.806 INFO:teuthology.orchestra.run.smithi026.stderr:10+0 records out 2023-11-16T16:32:28.806 INFO:teuthology.orchestra.run.smithi026.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00810459 s, 1.3 GB/s 2023-11-16T16:32:28.806 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping successful for: 2023-11-16T16:32:28.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:28 smithi001 ceph-mon[98800]: from='osd.0 [v2:172.21.15.1:6802/2469671609,v1:172.21.15.1:6803/2469671609]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-11-16T16:32:28.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:28 smithi001 ceph-mon[98800]: osdmap e6: 1 total, 0 up, 1 in 2023-11-16T16:32:28.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:28 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-16T16:32:28.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:28 smithi001 ceph-mon[98800]: from='osd.0 [v2:172.21.15.1:6802/2469671609,v1:172.21.15.1:6803/2469671609]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2023-11-16T16:32:28.933 INFO:journalctl@ceph.osd.0.smithi001.stdout:Nov 16 16:32:28 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-0[111670]: 2023-11-16T16:32:28.622+0000 7faafbe87700 -1 osd.0 0 waiting for initial osdmap 2023-11-16T16:32:28.933 INFO:journalctl@ceph.osd.0.smithi001.stdout:Nov 16 16:32:28 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-0[111670]: 2023-11-16T16:32:28.625+0000 7faaf5c78700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-16T16:32:28.997 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:28 smithi026 ceph-mon[100112]: from='osd.0 [v2:172.21.15.1:6802/2469671609,v1:172.21.15.1:6803/2469671609]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-11-16T16:32:28.997 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:28 smithi026 ceph-mon[100112]: osdmap e6: 1 total, 0 up, 1 in 2023-11-16T16:32:28.997 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:28 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-16T16:32:28.997 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:28 smithi026 ceph-mon[100112]: from='osd.0 [v2:172.21.15.1:6802/2469671609,v1:172.21.15.1:6803/2469671609]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2023-11-16T16:32:29.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:28 smithi053 ceph-mon[100638]: from='osd.0 [v2:172.21.15.1:6802/2469671609,v1:172.21.15.1:6803/2469671609]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-11-16T16:32:29.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:28 smithi053 ceph-mon[100638]: osdmap e6: 1 total, 0 up, 1 in 2023-11-16T16:32:29.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:28 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-16T16:32:29.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:28 smithi053 ceph-mon[100638]: from='osd.0 [v2:172.21.15.1:6802/2469671609,v1:172.21.15.1:6803/2469671609]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2023-11-16T16:32:29.233 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph orch daemon add osd smithi026:vg_nvme/lv_4 2023-11-16T16:32:29.884 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.b/config 2023-11-16T16:32:29.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:29 smithi001 ceph-mon[98800]: from='osd.0 [v2:172.21.15.1:6802/2469671609,v1:172.21.15.1:6803/2469671609]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2023-11-16T16:32:29.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:29 smithi001 ceph-mon[98800]: osdmap e7: 1 total, 0 up, 1 in 2023-11-16T16:32:29.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:29 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-16T16:32:29.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:29 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-16T16:32:29.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:29 smithi001 ceph-mon[98800]: Detected new or changed devices on smithi001 2023-11-16T16:32:29.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:29 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:29.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:29 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:29.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:29 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:32:29.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:29 smithi001 ceph-mon[98800]: Adjusting osd_memory_target on smithi001 to 17173M 2023-11-16T16:32:29.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:29 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:29.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:29 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:29.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:29 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:32:29.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:29 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:32:29.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:29 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:29.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:29 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:32:29.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:29 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:32:29.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:29 smithi001 ceph-mon[98800]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:30.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:29 smithi026 ceph-mon[100112]: from='osd.0 [v2:172.21.15.1:6802/2469671609,v1:172.21.15.1:6803/2469671609]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2023-11-16T16:32:30.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:29 smithi026 ceph-mon[100112]: osdmap e7: 1 total, 0 up, 1 in 2023-11-16T16:32:30.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:29 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-16T16:32:30.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:29 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-16T16:32:30.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:29 smithi026 ceph-mon[100112]: Detected new or changed devices on smithi001 2023-11-16T16:32:30.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:29 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:30.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:29 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:30.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:29 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:32:30.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:29 smithi026 ceph-mon[100112]: Adjusting osd_memory_target on smithi001 to 17173M 2023-11-16T16:32:30.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:29 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:30.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:29 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:30.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:29 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:32:30.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:29 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:32:30.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:29 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:30.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:29 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:32:30.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:29 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:32:30.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:29 smithi026 ceph-mon[100112]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:30.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:29 smithi053 ceph-mon[100638]: from='osd.0 [v2:172.21.15.1:6802/2469671609,v1:172.21.15.1:6803/2469671609]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2023-11-16T16:32:30.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:29 smithi053 ceph-mon[100638]: osdmap e7: 1 total, 0 up, 1 in 2023-11-16T16:32:30.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:29 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-16T16:32:30.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:29 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-16T16:32:30.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:29 smithi053 ceph-mon[100638]: Detected new or changed devices on smithi001 2023-11-16T16:32:30.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:29 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:30.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:29 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:30.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:29 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:32:30.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:29 smithi053 ceph-mon[100638]: Adjusting osd_memory_target on smithi001 to 17173M 2023-11-16T16:32:30.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:29 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:30.082 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:29 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:30.082 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:29 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:32:30.082 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:29 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:32:30.082 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:29 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:30.082 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:29 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:32:30.082 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:29 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:32:30.082 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:29 smithi053 ceph-mon[100638]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-16T16:32:30.888 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:30 smithi026 ceph-mon[100112]: purged_snaps scrub starts 2023-11-16T16:32:30.888 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:30 smithi026 ceph-mon[100112]: purged_snaps scrub ok 2023-11-16T16:32:30.888 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:30 smithi026 ceph-mon[100112]: osd.0 [v2:172.21.15.1:6802/2469671609,v1:172.21.15.1:6803/2469671609] boot 2023-11-16T16:32:30.888 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:30 smithi026 ceph-mon[100112]: osdmap e8: 1 total, 1 up, 1 in 2023-11-16T16:32:30.888 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:30 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-16T16:32:30.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:30 smithi001 ceph-mon[98800]: purged_snaps scrub starts 2023-11-16T16:32:30.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:30 smithi001 ceph-mon[98800]: purged_snaps scrub ok 2023-11-16T16:32:30.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:30 smithi001 ceph-mon[98800]: osd.0 [v2:172.21.15.1:6802/2469671609,v1:172.21.15.1:6803/2469671609] boot 2023-11-16T16:32:30.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:30 smithi001 ceph-mon[98800]: osdmap e8: 1 total, 1 up, 1 in 2023-11-16T16:32:30.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:30 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-16T16:32:31.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.069+0000 7f159b037700 1 -- 172.21.15.26:0/4166443348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1594101ea0 msgr2=0x7f1594108760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:32:31.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.069+0000 7f159b037700 1 --2- 172.21.15.26:0/4166443348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1594101ea0 0x7f1594108760 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f158c00b0a0 tx=0x7f158c00b3b0 comp rx=0 tx=0).stop 2023-11-16T16:32:31.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.070+0000 7f159b037700 1 -- 172.21.15.26:0/4166443348 shutdown_connections 2023-11-16T16:32:31.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.070+0000 7f159b037700 1 --2- 172.21.15.26:0/4166443348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1594101ea0 0x7f1594108760 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:32:31.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.070+0000 7f159b037700 1 --2- 172.21.15.26:0/4166443348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1594100310 0x7f1594100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:32:31.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.070+0000 7f159b037700 1 --2- 172.21.15.26:0/4166443348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1594101510 0x7f1594101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:32:31.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.070+0000 7f159b037700 1 -- 172.21.15.26:0/4166443348 >> 172.21.15.26:0/4166443348 conn(0x7f15940fb960 msgr2=0x7f15940fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:32:31.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.070+0000 7f159b037700 1 -- 172.21.15.26:0/4166443348 shutdown_connections 2023-11-16T16:32:31.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.070+0000 7f159b037700 1 -- 172.21.15.26:0/4166443348 wait complete. 2023-11-16T16:32:31.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.070+0000 7f159b037700 1 Processor -- start 2023-11-16T16:32:31.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.070+0000 7f159b037700 1 -- start start 2023-11-16T16:32:31.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.071+0000 7f159b037700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1594100310 0x7f15941961d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:32:31.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.071+0000 7f159b037700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1594101510 0x7f1594196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:32:31.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.071+0000 7f159b037700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1594101ea0 0x7f159419d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:32:31.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.071+0000 7f159b037700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159419dd60 con 0x7f1594101ea0 2023-11-16T16:32:31.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.071+0000 7f159b037700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159419dea0 con 0x7f1594100310 2023-11-16T16:32:31.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.071+0000 7f159b037700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159419e030 con 0x7f1594101510 2023-11-16T16:32:31.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.071+0000 7f1598dd3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1594100310 0x7f15941961d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:32:31.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.071+0000 7f1598dd3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1594100310 0x7f15941961d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54982/0 (socket says 172.21.15.26:54982) 2023-11-16T16:32:31.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.071+0000 7f1598dd3700 1 -- 172.21.15.26:0/2803817109 learned_addr learned my addr 172.21.15.26:0/2803817109 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-11-16T16:32:31.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.071+0000 7f1593fff700 1 --2- 172.21.15.26:0/2803817109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1594101510 0x7f1594196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:32:31.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.071+0000 7f15995d4700 1 --2- 172.21.15.26:0/2803817109 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1594101ea0 0x7f159419d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:32:31.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.071+0000 7f1598dd3700 1 -- 172.21.15.26:0/2803817109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1594101510 msgr2=0x7f1594196710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:32:31.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.071+0000 7f1598dd3700 1 --2- 172.21.15.26:0/2803817109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1594101510 0x7f1594196710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:32:31.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.071+0000 7f1598dd3700 1 -- 172.21.15.26:0/2803817109 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1594101ea0 msgr2=0x7f159419d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:32:31.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.071+0000 7f1598dd3700 1 --2- 172.21.15.26:0/2803817109 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1594101ea0 0x7f159419d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:32:31.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.071+0000 7f1598dd3700 1 -- 172.21.15.26:0/2803817109 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f158c009d00 con 0x7f1594100310 2023-11-16T16:32:31.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.071+0000 7f1598dd3700 1 --2- 172.21.15.26:0/2803817109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1594100310 0x7f15941961d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f158800b500 tx=0x7f158800b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:32:31.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.071+0000 7f1591ffb700 1 -- 172.21.15.26:0/2803817109 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1588011830 con 0x7f1594100310 2023-11-16T16:32:31.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.071+0000 7f1591ffb700 1 -- 172.21.15.26:0/2803817109 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1588011e50 con 0x7f1594100310 2023-11-16T16:32:31.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.071+0000 7f159b037700 1 -- 172.21.15.26:0/2803817109 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f159410bd60 con 0x7f1594100310 2023-11-16T16:32:31.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.071+0000 7f1591ffb700 1 -- 172.21.15.26:0/2803817109 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158800f2e0 con 0x7f1594100310 2023-11-16T16:32:31.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.072+0000 7f159b037700 1 -- 172.21.15.26:0/2803817109 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f159410bbd0 con 0x7f1594100310 2023-11-16T16:32:31.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.072+0000 7f159b037700 1 -- 172.21.15.26:0/2803817109 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1594107860 con 0x7f1594100310 2023-11-16T16:32:31.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.073+0000 7f1591ffb700 1 -- 172.21.15.26:0/2803817109 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 135658+0+0 (secure 0 0 0) 0x7f1588010580 con 0x7f1594100310 2023-11-16T16:32:31.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.074+0000 7f1591ffb700 1 --2- 172.21.15.26:0/2803817109 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f15840a1140 0x7f15840a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:32:31.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.074+0000 7f1591ffb700 1 -- 172.21.15.26:0/2803817109 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f15880ca8b0 con 0x7f1594100310 2023-11-16T16:32:31.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.074+0000 7f1593fff700 1 --2- 172.21.15.26:0/2803817109 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f15840a1140 0x7f15840a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:32:31.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.075+0000 7f1593fff700 1 --2- 172.21.15.26:0/2803817109 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f15840a1140 0x7f15840a35f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1580005950 tx=0x7f1580005cf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:32:31.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.079+0000 7f1591ffb700 1 -- 172.21.15.26:0/2803817109 <== mon.2 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1588095a70 con 0x7f1594100310 2023-11-16T16:32:31.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:30 smithi053 ceph-mon[100638]: purged_snaps scrub starts 2023-11-16T16:32:31.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:30 smithi053 ceph-mon[100638]: purged_snaps scrub ok 2023-11-16T16:32:31.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:30 smithi053 ceph-mon[100638]: osd.0 [v2:172.21.15.1:6802/2469671609,v1:172.21.15.1:6803/2469671609] boot 2023-11-16T16:32:31.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:30 smithi053 ceph-mon[100638]: osdmap e8: 1 total, 1 up, 1 in 2023-11-16T16:32:31.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:30 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-16T16:32:31.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:31.282+0000 7f159b037700 1 -- 172.21.15.26:0/2803817109 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f1594061c80 con 0x7f15840a1140 2023-11-16T16:32:32.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:32 smithi053 ceph-mon[100638]: osdmap e9: 1 total, 1 up, 1 in 2023-11-16T16:32:32.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:32 smithi053 ceph-mon[100638]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:32:32.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:32 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:32:32.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:32 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:32:32.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:32 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:32.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:32 smithi053 ceph-mon[100638]: pgmap v44: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:32:32.352 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:32 smithi026 ceph-mon[100112]: osdmap e9: 1 total, 1 up, 1 in 2023-11-16T16:32:32.352 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:32 smithi026 ceph-mon[100112]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:32:32.352 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:32 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:32:32.352 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:32 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:32:32.352 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:32 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:32.352 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:32 smithi026 ceph-mon[100112]: pgmap v44: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:32:32.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:32 smithi001 ceph-mon[98800]: osdmap e9: 1 total, 1 up, 1 in 2023-11-16T16:32:32.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:32 smithi001 ceph-mon[98800]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:32:32.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:32 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:32:32.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:32 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:32:32.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:32 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:32.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:32 smithi001 ceph-mon[98800]: pgmap v44: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:32:33.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:33 smithi026 ceph-mon[100112]: pgmap v45: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:32:33.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:33 smithi053 ceph-mon[100638]: pgmap v45: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:32:33.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:33 smithi001 ceph-mon[98800]: pgmap v45: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:32:34.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:34 smithi026 ceph-mon[100112]: from='client.? 172.21.15.26:0/4286879134' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d011db2-99d1-4575-bdc5-51f88a308fdc"}]: dispatch 2023-11-16T16:32:34.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:34 smithi026 ceph-mon[100112]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d011db2-99d1-4575-bdc5-51f88a308fdc"}]: dispatch 2023-11-16T16:32:34.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:34 smithi026 ceph-mon[100112]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d011db2-99d1-4575-bdc5-51f88a308fdc"}]': finished 2023-11-16T16:32:34.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:34 smithi026 ceph-mon[100112]: osdmap e10: 2 total, 1 up, 2 in 2023-11-16T16:32:34.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:34 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-16T16:32:34.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:34 smithi026 ceph-mon[100112]: from='client.? 172.21.15.26:0/2550227702' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:32:34.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:34 smithi053 ceph-mon[100638]: from='client.? 172.21.15.26:0/4286879134' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d011db2-99d1-4575-bdc5-51f88a308fdc"}]: dispatch 2023-11-16T16:32:34.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:34 smithi053 ceph-mon[100638]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d011db2-99d1-4575-bdc5-51f88a308fdc"}]: dispatch 2023-11-16T16:32:34.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:34 smithi053 ceph-mon[100638]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d011db2-99d1-4575-bdc5-51f88a308fdc"}]': finished 2023-11-16T16:32:34.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:34 smithi053 ceph-mon[100638]: osdmap e10: 2 total, 1 up, 2 in 2023-11-16T16:32:34.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:34 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-16T16:32:34.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:34 smithi053 ceph-mon[100638]: from='client.? 172.21.15.26:0/2550227702' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:32:34.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:34 smithi001 ceph-mon[98800]: from='client.? 172.21.15.26:0/4286879134' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d011db2-99d1-4575-bdc5-51f88a308fdc"}]: dispatch 2023-11-16T16:32:34.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:34 smithi001 ceph-mon[98800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d011db2-99d1-4575-bdc5-51f88a308fdc"}]: dispatch 2023-11-16T16:32:34.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:34 smithi001 ceph-mon[98800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d011db2-99d1-4575-bdc5-51f88a308fdc"}]': finished 2023-11-16T16:32:34.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:34 smithi001 ceph-mon[98800]: osdmap e10: 2 total, 1 up, 2 in 2023-11-16T16:32:34.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:34 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-16T16:32:34.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:34 smithi001 ceph-mon[98800]: from='client.? 172.21.15.26:0/2550227702' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:32:35.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:35 smithi026 ceph-mon[100112]: pgmap v47: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:32:35.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:35 smithi053 ceph-mon[100638]: pgmap v47: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:32:35.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:35 smithi001 ceph-mon[98800]: pgmap v47: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:32:37.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:37 smithi026 ceph-mon[100112]: pgmap v48: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:32:37.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:37 smithi053 ceph-mon[100638]: pgmap v48: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:32:37.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:37 smithi001 ceph-mon[98800]: pgmap v48: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:32:39.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:39 smithi026 ceph-mon[100112]: pgmap v49: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:32:39.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:39 smithi053 ceph-mon[100638]: pgmap v49: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:32:39.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:39 smithi001 ceph-mon[98800]: pgmap v49: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:32:40.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:40 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-11-16T16:32:40.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:40 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:40.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:40 smithi026 ceph-mon[100112]: Deploying daemon osd.1 on smithi026 2023-11-16T16:32:40.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:40 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-11-16T16:32:40.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:40 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:40.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:40 smithi053 ceph-mon[100638]: Deploying daemon osd.1 on smithi026 2023-11-16T16:32:40.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:40 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-11-16T16:32:40.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:40 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:40.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:40 smithi001 ceph-mon[98800]: Deploying daemon osd.1 on smithi026 2023-11-16T16:32:42.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:42 smithi026 ceph-mon[100112]: pgmap v50: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:32:42.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:42 smithi001 ceph-mon[98800]: pgmap v50: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:32:42.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:42 smithi053 ceph-mon[100638]: pgmap v50: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:32:43.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:43 smithi026 ceph-mon[100112]: pgmap v51: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:32:43.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:43 smithi053 ceph-mon[100638]: pgmap v51: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:32:43.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:43 smithi001 ceph-mon[98800]: pgmap v51: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:32:46.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:45 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:46.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:45 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:46.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:45 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:32:46.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:45 smithi026 ceph-mon[100112]: pgmap v52: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:32:46.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:45 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:46.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:45 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:46.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:45 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:32:46.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:45 smithi053 ceph-mon[100638]: pgmap v52: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:32:46.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:45 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:46.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:45 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:46.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:45 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:32:46.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:45 smithi001 ceph-mon[98800]: pgmap v52: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:32:47.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:47.853+0000 7f1591ffb700 1 -- 172.21.15.26:0/2803817109 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1594061c80 con 0x7f15840a1140 2023-11-16T16:32:47.855 INFO:teuthology.orchestra.run.smithi026.stdout:Created osd(s) 1 on host 'smithi026' 2023-11-16T16:32:47.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:47.857+0000 7f159b037700 1 -- 172.21.15.26:0/2803817109 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f15840a1140 msgr2=0x7f15840a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:32:47.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:47.857+0000 7f159b037700 1 --2- 172.21.15.26:0/2803817109 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f15840a1140 0x7f15840a35f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1580005950 tx=0x7f1580005cf0 comp rx=0 tx=0).stop 2023-11-16T16:32:47.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:47.857+0000 7f159b037700 1 -- 172.21.15.26:0/2803817109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1594100310 msgr2=0x7f15941961d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:32:47.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:47.857+0000 7f159b037700 1 --2- 172.21.15.26:0/2803817109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1594100310 0x7f15941961d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f158800b500 tx=0x7f158800b8c0 comp rx=0 tx=0).stop 2023-11-16T16:32:47.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:47.857+0000 7f159b037700 1 -- 172.21.15.26:0/2803817109 shutdown_connections 2023-11-16T16:32:47.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:47.857+0000 7f159b037700 1 --2- 172.21.15.26:0/2803817109 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f15840a1140 0x7f15840a35f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:32:47.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:47.857+0000 7f159b037700 1 --2- 172.21.15.26:0/2803817109 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1594101ea0 0x7f159419d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:32:47.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:47.857+0000 7f159b037700 1 --2- 172.21.15.26:0/2803817109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1594100310 0x7f15941961d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:32:47.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:47.857+0000 7f159b037700 1 --2- 172.21.15.26:0/2803817109 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1594101510 0x7f1594196710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:32:47.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:47.857+0000 7f159b037700 1 -- 172.21.15.26:0/2803817109 >> 172.21.15.26:0/2803817109 conn(0x7f15940fb960 msgr2=0x7f15941049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:32:47.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:47.857+0000 7f159b037700 1 -- 172.21.15.26:0/2803817109 shutdown_connections 2023-11-16T16:32:47.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:32:47.857+0000 7f159b037700 1 -- 172.21.15.26:0/2803817109 wait complete. 2023-11-16T16:32:48.182 DEBUG:teuthology.orchestra.run.smithi026:osd.1> sudo journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@osd.1.service 2023-11-16T16:32:48.184 INFO:tasks.cephadm:Deploying osd.2 on smithi053 with /dev/vg_nvme/lv_4... 2023-11-16T16:32:48.185 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-11-16T16:32:48.221 INFO:journalctl@ceph.osd.1.smithi026.stdout:-- Logs begin at Thu 2023-11-16 16:14:13 UTC. -- 2023-11-16T16:32:48.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:48 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:48.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:48 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:48.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:48 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:48.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:48 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:32:48.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:48 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:32:48.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:48 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:48.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:48 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:32:48.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:48 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:32:48.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:48 smithi053 ceph-mon[100638]: pgmap v53: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:32:48.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:48 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:48.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:48 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:48.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:48 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:32:48.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:48 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:48.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:48 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:48.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:48 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:48.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:48 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:32:48.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:48 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:32:48.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:48 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:48.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:48 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:32:48.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:48 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:32:48.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:48 smithi001 ceph-mon[98800]: pgmap v53: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:32:48.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:48 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:48.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:48 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:48.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:48 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:32:48.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:48 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:48.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:48 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:48.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:48 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:48.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:48 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:32:48.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:48 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:32:48.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:48 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:48.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:48 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:32:48.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:48 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:32:48.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:48 smithi026 ceph-mon[100112]: pgmap v53: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:32:48.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:48 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:48.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:48 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:48.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:48 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:32:48.847 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-16T16:32:49.008 INFO:journalctl@ceph.osd.1.smithi026.stdout:Nov 16 16:32:48 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-1[105219]: 2023-11-16T16:32:48.922+0000 7fee2b1b7640 -1 osd.1 0 log_to_monitors true 2023-11-16T16:32:49.281 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:49 smithi026 ceph-mon[100112]: from='osd.1 [v2:172.21.15.26:6800/247994796,v1:172.21.15.26:6801/247994796]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-16T16:32:49.281 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:49 smithi026 ceph-mon[100112]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-16T16:32:49.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:49 smithi053 ceph-mon[100638]: from='osd.1 [v2:172.21.15.26:6800/247994796,v1:172.21.15.26:6801/247994796]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-16T16:32:49.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:49 smithi053 ceph-mon[100638]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-16T16:32:49.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:49 smithi001 ceph-mon[98800]: from='osd.1 [v2:172.21.15.26:6800/247994796,v1:172.21.15.26:6801/247994796]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-16T16:32:49.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:49 smithi001 ceph-mon[98800]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-16T16:32:50.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:50 smithi053 ceph-mon[100638]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-11-16T16:32:50.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:50 smithi053 ceph-mon[100638]: osdmap e11: 2 total, 1 up, 2 in 2023-11-16T16:32:50.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:50 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-16T16:32:50.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:50 smithi053 ceph-mon[100638]: from='osd.1 [v2:172.21.15.26:6800/247994796,v1:172.21.15.26:6801/247994796]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-11-16T16:32:50.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:50 smithi053 ceph-mon[100638]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-11-16T16:32:50.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:50 smithi053 ceph-mon[100638]: pgmap v55: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:32:50.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:50 smithi001 ceph-mon[98800]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-11-16T16:32:50.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:50 smithi001 ceph-mon[98800]: osdmap e11: 2 total, 1 up, 2 in 2023-11-16T16:32:50.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:50 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-16T16:32:50.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:50 smithi001 ceph-mon[98800]: from='osd.1 [v2:172.21.15.26:6800/247994796,v1:172.21.15.26:6801/247994796]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-11-16T16:32:50.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:50 smithi001 ceph-mon[98800]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-11-16T16:32:50.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:50 smithi001 ceph-mon[98800]: pgmap v55: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:32:50.456 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-11-16T16:32:50.456 INFO:teuthology.orchestra.run.smithi053.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-11-16T16:32:50.456 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10+0 records in 2023-11-16T16:32:50.457 INFO:teuthology.orchestra.run.smithi053.stderr:10+0 records out 2023-11-16T16:32:50.457 INFO:teuthology.orchestra.run.smithi053.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00798461 s, 1.3 GB/s 2023-11-16T16:32:50.457 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping successful for: 2023-11-16T16:32:50.508 INFO:journalctl@ceph.osd.1.smithi026.stdout:Nov 16 16:32:50 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-1[105219]: 2023-11-16T16:32:50.082+0000 7fee2082e700 -1 osd.1 0 waiting for initial osdmap 2023-11-16T16:32:50.509 INFO:journalctl@ceph.osd.1.smithi026.stdout:Nov 16 16:32:50 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-1[105219]: 2023-11-16T16:32:50.086+0000 7fee1a61f700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-16T16:32:50.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:50 smithi026 ceph-mon[100112]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-11-16T16:32:50.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:50 smithi026 ceph-mon[100112]: osdmap e11: 2 total, 1 up, 2 in 2023-11-16T16:32:50.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-16T16:32:50.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:50 smithi026 ceph-mon[100112]: from='osd.1 [v2:172.21.15.26:6800/247994796,v1:172.21.15.26:6801/247994796]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-11-16T16:32:50.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:50 smithi026 ceph-mon[100112]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-11-16T16:32:50.510 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:50 smithi026 ceph-mon[100112]: pgmap v55: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-16T16:32:50.905 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph orch daemon add osd smithi053:vg_nvme/lv_4 2023-11-16T16:32:51.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:51 smithi053 ceph-mon[100638]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2023-11-16T16:32:51.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:51 smithi053 ceph-mon[100638]: osdmap e12: 2 total, 1 up, 2 in 2023-11-16T16:32:51.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:51 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-16T16:32:51.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:51 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-16T16:32:51.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:51 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:51.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:51 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:51.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:51 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:32:51.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:51 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:51.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:51 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:51.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:51 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:32:51.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:51 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:51.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:51 smithi001 ceph-mon[98800]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2023-11-16T16:32:51.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:51 smithi001 ceph-mon[98800]: osdmap e12: 2 total, 1 up, 2 in 2023-11-16T16:32:51.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:51 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-16T16:32:51.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:51 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-16T16:32:51.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:51 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:51.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:51 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:51.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:51 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:32:51.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:51 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:51.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:51 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:51.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:51 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:32:51.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:51 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:51.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:51 smithi026 ceph-mon[100112]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2023-11-16T16:32:51.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:51 smithi026 ceph-mon[100112]: osdmap e12: 2 total, 1 up, 2 in 2023-11-16T16:32:51.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:51 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-16T16:32:51.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:51 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-16T16:32:51.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:51 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:51.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:51 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:51.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:51 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:32:51.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:51 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:51.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:51 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:51.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:51 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:32:51.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:51 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:32:51.556 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-16T16:32:52.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:52 smithi053 ceph-mon[100638]: purged_snaps scrub starts 2023-11-16T16:32:52.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:52 smithi053 ceph-mon[100638]: purged_snaps scrub ok 2023-11-16T16:32:52.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:52 smithi053 ceph-mon[100638]: Detected new or changed devices on smithi026 2023-11-16T16:32:52.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:52 smithi053 ceph-mon[100638]: Adjusting osd_memory_target on smithi026 to 17173M 2023-11-16T16:32:52.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:52 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:32:52.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:52 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:32:52.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:52 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:32:52.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:52 smithi053 ceph-mon[100638]: osd.1 [v2:172.21.15.26:6800/247994796,v1:172.21.15.26:6801/247994796] boot 2023-11-16T16:32:52.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:52 smithi053 ceph-mon[100638]: osdmap e13: 2 total, 2 up, 2 in 2023-11-16T16:32:52.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:52 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-16T16:32:52.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:52 smithi053 ceph-mon[100638]: pgmap v58: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:32:52.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:52 smithi001 ceph-mon[98800]: purged_snaps scrub starts 2023-11-16T16:32:52.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:52 smithi001 ceph-mon[98800]: purged_snaps scrub ok 2023-11-16T16:32:52.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:52 smithi001 ceph-mon[98800]: Detected new or changed devices on smithi026 2023-11-16T16:32:52.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:52 smithi001 ceph-mon[98800]: Adjusting osd_memory_target on smithi026 to 17173M 2023-11-16T16:32:52.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:52 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:32:52.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:52 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:32:52.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:52 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:32:52.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:52 smithi001 ceph-mon[98800]: osd.1 [v2:172.21.15.26:6800/247994796,v1:172.21.15.26:6801/247994796] boot 2023-11-16T16:32:52.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:52 smithi001 ceph-mon[98800]: osdmap e13: 2 total, 2 up, 2 in 2023-11-16T16:32:52.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:52 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-16T16:32:52.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:52 smithi001 ceph-mon[98800]: pgmap v58: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:32:52.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:52 smithi026 ceph-mon[100112]: purged_snaps scrub starts 2023-11-16T16:32:52.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:52 smithi026 ceph-mon[100112]: purged_snaps scrub ok 2023-11-16T16:32:52.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:52 smithi026 ceph-mon[100112]: Detected new or changed devices on smithi026 2023-11-16T16:32:52.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:52 smithi026 ceph-mon[100112]: Adjusting osd_memory_target on smithi026 to 17173M 2023-11-16T16:32:52.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:52 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:32:52.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:52 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:32:52.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:52 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:32:52.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:52 smithi026 ceph-mon[100112]: osd.1 [v2:172.21.15.26:6800/247994796,v1:172.21.15.26:6801/247994796] boot 2023-11-16T16:32:52.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:52 smithi026 ceph-mon[100112]: osdmap e13: 2 total, 2 up, 2 in 2023-11-16T16:32:52.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:52 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-16T16:32:52.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:52 smithi026 ceph-mon[100112]: pgmap v58: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:32:52.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.857+0000 7f3982b9d700 1 -- 172.21.15.53:0/2253771865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f397c106010 msgr2=0x7f397c108410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:32:52.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.857+0000 7f3982b9d700 1 --2- 172.21.15.53:0/2253771865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f397c106010 0x7f397c108410 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f397400b0a0 tx=0x7f397400b3b0 comp rx=0 tx=0).stop 2023-11-16T16:32:52.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.857+0000 7f3982b9d700 1 -- 172.21.15.53:0/2253771865 shutdown_connections 2023-11-16T16:32:52.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.857+0000 7f3982b9d700 1 --2- 172.21.15.53:0/2253771865 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f397c106010 0x7f397c108410 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:32:52.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.857+0000 7f3982b9d700 1 --2- 172.21.15.53:0/2253771865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f397c100dd0 0x7f397c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:32:52.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.857+0000 7f3982b9d700 1 --2- 172.21.15.53:0/2253771865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f397c1036f0 0x7f397c105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:32:52.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.857+0000 7f3982b9d700 1 -- 172.21.15.53:0/2253771865 >> 172.21.15.53:0/2253771865 conn(0x7f397c0fa7f0 msgr2=0x7f397c0fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:32:52.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.857+0000 7f3982b9d700 1 -- 172.21.15.53:0/2253771865 shutdown_connections 2023-11-16T16:32:52.859 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.857+0000 7f3982b9d700 1 -- 172.21.15.53:0/2253771865 wait complete. 2023-11-16T16:32:52.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.858+0000 7f3982b9d700 1 Processor -- start 2023-11-16T16:32:52.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.858+0000 7f3982b9d700 1 -- start start 2023-11-16T16:32:52.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.858+0000 7f3982b9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f397c100dd0 0x7f397c100730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:32:52.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.858+0000 7f3982b9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f397c1036f0 0x7f397c0fed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:32:52.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.858+0000 7f3982b9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f397c106010 0x7f397c0ff2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:32:52.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.858+0000 7f3982b9d700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397c0ffa40 con 0x7f397c100dd0 2023-11-16T16:32:52.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.858+0000 7f3982b9d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397c0ffbb0 con 0x7f397c106010 2023-11-16T16:32:52.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.858+0000 7f3982b9d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397c1a01c0 con 0x7f397c1036f0 2023-11-16T16:32:52.861 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.858+0000 7f397bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f397c1036f0 0x7f397c0fed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:32:52.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.858+0000 7f397bfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f397c1036f0 0x7f397c0fed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57604/0 (socket says 172.21.15.53:57604) 2023-11-16T16:32:52.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.858+0000 7f397bfff700 1 -- 172.21.15.53:0/1398812252 learned_addr learned my addr 172.21.15.53:0/1398812252 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-16T16:32:52.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.858+0000 7f3980939700 1 --2- 172.21.15.53:0/1398812252 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f397c100dd0 0x7f397c100730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:32:52.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.858+0000 7f398113a700 1 --2- 172.21.15.53:0/1398812252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f397c106010 0x7f397c0ff2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:32:52.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.858+0000 7f397bfff700 1 -- 172.21.15.53:0/1398812252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f397c106010 msgr2=0x7f397c0ff2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:32:52.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.858+0000 7f397bfff700 1 --2- 172.21.15.53:0/1398812252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f397c106010 0x7f397c0ff2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:32:52.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.858+0000 7f397bfff700 1 -- 172.21.15.53:0/1398812252 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f397c100dd0 msgr2=0x7f397c100730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:32:52.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.858+0000 7f397bfff700 1 --2- 172.21.15.53:0/1398812252 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f397c100dd0 0x7f397c100730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:32:52.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.858+0000 7f397bfff700 1 -- 172.21.15.53:0/1398812252 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3974009d00 con 0x7f397c1036f0 2023-11-16T16:32:52.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.858+0000 7f397bfff700 1 --2- 172.21.15.53:0/1398812252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f397c1036f0 0x7f397c0fed80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f396800d950 tx=0x7f396800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:32:52.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.859+0000 7f3979ffb700 1 -- 172.21.15.53:0/1398812252 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396800f890 con 0x7f397c1036f0 2023-11-16T16:32:52.862 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.859+0000 7f3979ffb700 1 -- 172.21.15.53:0/1398812252 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3968010460 con 0x7f397c1036f0 2023-11-16T16:32:52.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.859+0000 7f3982b9d700 1 -- 172.21.15.53:0/1398812252 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f397c10bc50 con 0x7f397c1036f0 2023-11-16T16:32:52.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.859+0000 7f3979ffb700 1 -- 172.21.15.53:0/1398812252 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396800e610 con 0x7f397c1036f0 2023-11-16T16:32:52.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.859+0000 7f3982b9d700 1 -- 172.21.15.53:0/1398812252 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f397c10bac0 con 0x7f397c1036f0 2023-11-16T16:32:52.863 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.860+0000 7f3982b9d700 1 -- 172.21.15.53:0/1398812252 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f397c0fc360 con 0x7f397c1036f0 2023-11-16T16:32:52.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.861+0000 7f3979ffb700 1 -- 172.21.15.53:0/1398812252 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 15) v1 ==== 135658+0+0 (secure 0 0 0) 0x7f39680108d0 con 0x7f397c1036f0 2023-11-16T16:32:52.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.861+0000 7f3979ffb700 1 --2- 172.21.15.53:0/1398812252 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f396c0a1140 0x7f396c0a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:32:52.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.862+0000 7f3979ffb700 1 -- 172.21.15.53:0/1398812252 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f39680cadc0 con 0x7f397c1036f0 2023-11-16T16:32:52.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.862+0000 7f3980939700 1 --2- 172.21.15.53:0/1398812252 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f396c0a1140 0x7f396c0a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:32:52.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.862+0000 7f3980939700 1 --2- 172.21.15.53:0/1398812252 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f396c0a1140 0x7f396c0a35f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3970005950 tx=0x7f397000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:32:52.868 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:52.866+0000 7f3979ffb700 1 -- 172.21.15.53:0/1398812252 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3968095d60 con 0x7f397c1036f0 2023-11-16T16:32:53.069 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:32:53.068+0000 7f3982b9d700 1 -- 172.21.15.53:0/1398812252 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f397c061a50 con 0x7f396c0a1140 2023-11-16T16:32:53.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:53 smithi001 ceph-mon[98800]: osdmap e14: 2 total, 2 up, 2 in 2023-11-16T16:32:53.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:53 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:32:53.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:53 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:32:53.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:53 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:53.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:53 smithi026 ceph-mon[100112]: osdmap e14: 2 total, 2 up, 2 in 2023-11-16T16:32:53.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:53 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:32:53.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:53 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:32:53.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:53 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:53.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:53 smithi053 ceph-mon[100638]: osdmap e14: 2 total, 2 up, 2 in 2023-11-16T16:32:53.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:53 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:32:53.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:53 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:32:53.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:53 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:32:54.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:54 smithi053 ceph-mon[100638]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:32:54.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:54 smithi053 ceph-mon[100638]: pgmap v60: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:32:54.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:54 smithi001 ceph-mon[98800]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:32:54.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:54 smithi001 ceph-mon[98800]: pgmap v60: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:32:54.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:54 smithi026 ceph-mon[100112]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:32:54.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:54 smithi026 ceph-mon[100112]: pgmap v60: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:32:55.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:55 smithi026 ceph-mon[100112]: pgmap v61: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:32:55.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:55 smithi053 ceph-mon[100638]: pgmap v61: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:32:55.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:55 smithi001 ceph-mon[98800]: pgmap v61: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:32:56.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:56 smithi026 ceph-mon[100112]: from='client.? 172.21.15.53:0/7714634' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d7a2bd1-3c8e-4546-a243-f0354473b031"}]: dispatch 2023-11-16T16:32:56.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:56 smithi026 ceph-mon[100112]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d7a2bd1-3c8e-4546-a243-f0354473b031"}]: dispatch 2023-11-16T16:32:56.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:56 smithi026 ceph-mon[100112]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4d7a2bd1-3c8e-4546-a243-f0354473b031"}]': finished 2023-11-16T16:32:56.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:56 smithi026 ceph-mon[100112]: osdmap e15: 3 total, 2 up, 3 in 2023-11-16T16:32:56.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:56 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-16T16:32:56.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:56 smithi026 ceph-mon[100112]: from='client.? 172.21.15.53:0/2521581892' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:32:56.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:56 smithi053 ceph-mon[100638]: from='client.? 172.21.15.53:0/7714634' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d7a2bd1-3c8e-4546-a243-f0354473b031"}]: dispatch 2023-11-16T16:32:56.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:56 smithi053 ceph-mon[100638]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d7a2bd1-3c8e-4546-a243-f0354473b031"}]: dispatch 2023-11-16T16:32:56.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:56 smithi053 ceph-mon[100638]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4d7a2bd1-3c8e-4546-a243-f0354473b031"}]': finished 2023-11-16T16:32:56.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:56 smithi053 ceph-mon[100638]: osdmap e15: 3 total, 2 up, 3 in 2023-11-16T16:32:56.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:56 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-16T16:32:56.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:56 smithi053 ceph-mon[100638]: from='client.? 172.21.15.53:0/2521581892' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:32:56.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:56 smithi001 ceph-mon[98800]: from='client.? 172.21.15.53:0/7714634' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d7a2bd1-3c8e-4546-a243-f0354473b031"}]: dispatch 2023-11-16T16:32:56.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:56 smithi001 ceph-mon[98800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d7a2bd1-3c8e-4546-a243-f0354473b031"}]: dispatch 2023-11-16T16:32:56.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:56 smithi001 ceph-mon[98800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4d7a2bd1-3c8e-4546-a243-f0354473b031"}]': finished 2023-11-16T16:32:56.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:56 smithi001 ceph-mon[98800]: osdmap e15: 3 total, 2 up, 3 in 2023-11-16T16:32:56.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:56 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-16T16:32:56.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:56 smithi001 ceph-mon[98800]: from='client.? 172.21.15.53:0/2521581892' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:32:57.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:57 smithi026 ceph-mon[100112]: pgmap v63: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:32:57.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:57 smithi053 ceph-mon[100638]: pgmap v63: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:32:57.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:57 smithi001 ceph-mon[98800]: pgmap v63: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:32:59.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:32:59 smithi026 ceph-mon[100112]: pgmap v64: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:32:59.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:32:59 smithi053 ceph-mon[100638]: pgmap v64: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:32:59.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:32:59 smithi001 ceph-mon[98800]: pgmap v64: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:33:02.370 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:02 smithi053 ceph-mon[100638]: pgmap v65: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:33:02.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:02 smithi001 ceph-mon[98800]: pgmap v65: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:33:02.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:02 smithi026 ceph-mon[100112]: pgmap v65: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:33:03.418 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:03 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-11-16T16:33:03.418 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:03 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:33:03.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:03 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-11-16T16:33:03.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:03 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:33:03.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:03 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-11-16T16:33:03.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:03 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:33:04.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:04 smithi001 ceph-mon[98800]: Deploying daemon osd.2 on smithi053 2023-11-16T16:33:04.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:04 smithi001 ceph-mon[98800]: pgmap v66: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:33:04.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:04 smithi026 ceph-mon[100112]: Deploying daemon osd.2 on smithi053 2023-11-16T16:33:04.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:04 smithi026 ceph-mon[100112]: pgmap v66: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:33:04.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:04 smithi053 ceph-mon[100638]: Deploying daemon osd.2 on smithi053 2023-11-16T16:33:04.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:04 smithi053 ceph-mon[100638]: pgmap v66: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:33:05.630 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:05 smithi053 ceph-mon[100638]: pgmap v67: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:33:05.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:05 smithi026 ceph-mon[100112]: pgmap v67: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:33:05.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:05 smithi001 ceph-mon[98800]: pgmap v67: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:33:07.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:07 smithi053 ceph-mon[100638]: pgmap v68: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:33:07.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:07 smithi026 ceph-mon[100112]: pgmap v68: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:33:07.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:07 smithi001 ceph-mon[98800]: pgmap v68: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:33:08.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:08 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:08.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:08 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:08.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:08 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:33:08.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:08 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:08.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:08 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:08.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:08 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:33:09.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:08 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:09.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:08 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:09.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:08 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:33:09.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:09 smithi053 ceph-mon[100638]: pgmap v69: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:33:09.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:09 smithi001 ceph-mon[98800]: pgmap v69: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:33:10.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:09 smithi026 ceph-mon[100112]: pgmap v69: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:33:10.336 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:10.334+0000 7f3979ffb700 1 -- 172.21.15.53:0/1398812252 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f397c061a50 con 0x7f396c0a1140 2023-11-16T16:33:10.336 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 2 on host 'smithi053' 2023-11-16T16:33:10.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:10.336+0000 7f3982b9d700 1 -- 172.21.15.53:0/1398812252 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f396c0a1140 msgr2=0x7f396c0a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:33:10.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:10.336+0000 7f3982b9d700 1 --2- 172.21.15.53:0/1398812252 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f396c0a1140 0x7f396c0a35f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3970005950 tx=0x7f397000b410 comp rx=0 tx=0).stop 2023-11-16T16:33:10.338 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:10.337+0000 7f3982b9d700 1 -- 172.21.15.53:0/1398812252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f397c1036f0 msgr2=0x7f397c0fed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:33:10.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:10.337+0000 7f3982b9d700 1 --2- 172.21.15.53:0/1398812252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f397c1036f0 0x7f397c0fed80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f396800d950 tx=0x7f396800dc60 comp rx=0 tx=0).stop 2023-11-16T16:33:10.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:10.337+0000 7f3982b9d700 1 -- 172.21.15.53:0/1398812252 shutdown_connections 2023-11-16T16:33:10.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:10.337+0000 7f3982b9d700 1 --2- 172.21.15.53:0/1398812252 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f396c0a1140 0x7f396c0a35f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:33:10.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:10.337+0000 7f3982b9d700 1 --2- 172.21.15.53:0/1398812252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f397c106010 0x7f397c0ff2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:33:10.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:10.337+0000 7f3982b9d700 1 --2- 172.21.15.53:0/1398812252 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f397c100dd0 0x7f397c100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:33:10.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:10.337+0000 7f3982b9d700 1 --2- 172.21.15.53:0/1398812252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f397c1036f0 0x7f397c0fed80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:33:10.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:10.337+0000 7f3982b9d700 1 -- 172.21.15.53:0/1398812252 >> 172.21.15.53:0/1398812252 conn(0x7f397c0fa7f0 msgr2=0x7f397c101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:33:10.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:10.337+0000 7f3982b9d700 1 -- 172.21.15.53:0/1398812252 shutdown_connections 2023-11-16T16:33:10.339 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:10.337+0000 7f3982b9d700 1 -- 172.21.15.53:0/1398812252 wait complete. 2023-11-16T16:33:10.745 DEBUG:teuthology.orchestra.run.smithi053:osd.2> sudo journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@osd.2.service 2023-11-16T16:33:10.747 INFO:tasks.cephadm:Deploying osd.3 on smithi001 with /dev/vg_nvme/lv_3... 2023-11-16T16:33:10.747 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-11-16T16:33:10.781 INFO:journalctl@ceph.osd.2.smithi053.stdout:-- Logs begin at Thu 2023-11-16 16:14:15 UTC. -- 2023-11-16T16:33:11.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:10 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:11.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:10 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:11.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:10 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:33:11.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:10 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:33:11.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:10 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:33:11.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:10 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:11.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:10 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:33:11.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:10 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:33:11.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:10 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:11.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:10 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:11.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:10 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:33:11.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:10 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:11.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:10 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:11.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:10 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:33:11.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:10 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:33:11.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:10 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:33:11.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:10 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:11.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:10 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:33:11.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:10 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:33:11.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:10 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:11.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:10 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:11.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:10 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:33:11.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:10 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:11.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:10 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:11.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:10 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:33:11.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:10 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:33:11.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:10 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:33:11.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:10 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:11.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:10 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:33:11.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:10 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:33:11.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:10 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:11.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:10 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:11.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:10 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:33:11.413 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:33:11.694 INFO:journalctl@ceph.osd.2.smithi053.stdout:Nov 16 16:33:11 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-2[106394]: 2023-11-16T16:33:11.537+0000 7f44cb9ab640 -1 osd.2 0 log_to_monitors true 2023-11-16T16:33:12.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:12 smithi001 ceph-mon[98800]: pgmap v70: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:33:12.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:12 smithi001 ceph-mon[98800]: from='osd.2 [v2:172.21.15.53:6800/1659316290,v1:172.21.15.53:6801/1659316290]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-11-16T16:33:12.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:12 smithi001 ceph-mon[98800]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-11-16T16:33:12.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:12 smithi026 ceph-mon[100112]: pgmap v70: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:33:12.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:12 smithi026 ceph-mon[100112]: from='osd.2 [v2:172.21.15.53:6800/1659316290,v1:172.21.15.53:6801/1659316290]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-11-16T16:33:12.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:12 smithi026 ceph-mon[100112]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-11-16T16:33:12.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:12 smithi053 ceph-mon[100638]: pgmap v70: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:33:12.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:12 smithi053 ceph-mon[100638]: from='osd.2 [v2:172.21.15.53:6800/1659316290,v1:172.21.15.53:6801/1659316290]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-11-16T16:33:12.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:12 smithi053 ceph-mon[100638]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-11-16T16:33:12.902 INFO:teuthology.orchestra.run.smithi001.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-11-16T16:33:12.902 INFO:teuthology.orchestra.run.smithi001.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-11-16T16:33:12.902 INFO:teuthology.orchestra.run.smithi001.stderr: stderr: 10+0 records in 2023-11-16T16:33:12.902 INFO:teuthology.orchestra.run.smithi001.stderr:10+0 records out 2023-11-16T16:33:12.902 INFO:teuthology.orchestra.run.smithi001.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00819214 s, 1.3 GB/s 2023-11-16T16:33:12.902 INFO:teuthology.orchestra.run.smithi001.stderr:--> Zapping successful for: 2023-11-16T16:33:13.258 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:13 smithi001 ceph-mon[98800]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-11-16T16:33:13.259 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:13 smithi001 ceph-mon[98800]: osdmap e16: 3 total, 2 up, 3 in 2023-11-16T16:33:13.259 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:13 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-16T16:33:13.259 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:13 smithi001 ceph-mon[98800]: from='osd.2 [v2:172.21.15.53:6800/1659316290,v1:172.21.15.53:6801/1659316290]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2023-11-16T16:33:13.259 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:13 smithi001 ceph-mon[98800]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2023-11-16T16:33:13.474 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph orch daemon add osd smithi001:vg_nvme/lv_3 2023-11-16T16:33:13.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:13 smithi026 ceph-mon[100112]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-11-16T16:33:13.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:13 smithi026 ceph-mon[100112]: osdmap e16: 3 total, 2 up, 3 in 2023-11-16T16:33:13.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:13 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-16T16:33:13.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:13 smithi026 ceph-mon[100112]: from='osd.2 [v2:172.21.15.53:6800/1659316290,v1:172.21.15.53:6801/1659316290]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2023-11-16T16:33:13.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:13 smithi026 ceph-mon[100112]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2023-11-16T16:33:13.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:13 smithi053 ceph-mon[100638]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-11-16T16:33:13.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:13 smithi053 ceph-mon[100638]: osdmap e16: 3 total, 2 up, 3 in 2023-11-16T16:33:13.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:13 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-16T16:33:13.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:13 smithi053 ceph-mon[100638]: from='osd.2 [v2:172.21.15.53:6800/1659316290,v1:172.21.15.53:6801/1659316290]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2023-11-16T16:33:13.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:13 smithi053 ceph-mon[100638]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2023-11-16T16:33:13.582 INFO:journalctl@ceph.osd.2.smithi053.stdout:Nov 16 16:33:13 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-2[106394]: 2023-11-16T16:33:13.114+0000 7f44c1022700 -1 osd.2 0 waiting for initial osdmap 2023-11-16T16:33:13.582 INFO:journalctl@ceph.osd.2.smithi053.stdout:Nov 16 16:33:13 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-2[106394]: 2023-11-16T16:33:13.119+0000 7f44b9e11700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-16T16:33:14.127 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:33:14.423 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:14 smithi001 ceph-mon[98800]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2023-11-16T16:33:14.423 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:14 smithi001 ceph-mon[98800]: osdmap e17: 3 total, 2 up, 3 in 2023-11-16T16:33:14.423 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:14 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-16T16:33:14.423 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:14 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-16T16:33:14.423 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:14 smithi001 ceph-mon[98800]: Detected new or changed devices on smithi053 2023-11-16T16:33:14.423 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:14 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:14.423 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:14 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:14.423 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:14 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:33:14.424 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:14 smithi001 ceph-mon[98800]: Adjusting osd_memory_target on smithi053 to 17173M 2023-11-16T16:33:14.424 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:14 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:14.424 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:14 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:33:14.424 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:14 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:33:14.424 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:14 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:33:14.424 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:14 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:14.424 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:14 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:33:14.425 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:14 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:33:14.425 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:14 smithi001 ceph-mon[98800]: pgmap v73: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:33:14.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:14 smithi026 ceph-mon[100112]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2023-11-16T16:33:14.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:14 smithi026 ceph-mon[100112]: osdmap e17: 3 total, 2 up, 3 in 2023-11-16T16:33:14.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:14 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-16T16:33:14.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:14 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-16T16:33:14.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:14 smithi026 ceph-mon[100112]: Detected new or changed devices on smithi053 2023-11-16T16:33:14.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:14 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:14.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:14 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:14.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:14 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:33:14.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:14 smithi026 ceph-mon[100112]: Adjusting osd_memory_target on smithi053 to 17173M 2023-11-16T16:33:14.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:14 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:14.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:14 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:33:14.510 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:14 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:33:14.510 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:14 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:33:14.510 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:14 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:14.510 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:14 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:33:14.510 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:14 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:33:14.510 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:14 smithi026 ceph-mon[100112]: pgmap v73: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:33:14.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:14 smithi053 ceph-mon[100638]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2023-11-16T16:33:14.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:14 smithi053 ceph-mon[100638]: osdmap e17: 3 total, 2 up, 3 in 2023-11-16T16:33:14.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:14 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-16T16:33:14.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:14 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-16T16:33:14.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:14 smithi053 ceph-mon[100638]: Detected new or changed devices on smithi053 2023-11-16T16:33:14.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:14 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:14.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:14 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:14.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:14 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:33:14.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:14 smithi053 ceph-mon[100638]: Adjusting osd_memory_target on smithi053 to 17173M 2023-11-16T16:33:14.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:14 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:14.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:14 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:33:14.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:14 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:33:14.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:14 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:33:14.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:14 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:14.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:14 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:33:14.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:14 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:33:14.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:14 smithi053 ceph-mon[100638]: pgmap v73: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-16T16:33:15.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:15 smithi001 ceph-mon[98800]: purged_snaps scrub starts 2023-11-16T16:33:15.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:15 smithi001 ceph-mon[98800]: purged_snaps scrub ok 2023-11-16T16:33:15.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:15 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-16T16:33:15.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:15 smithi001 ceph-mon[98800]: osd.2 [v2:172.21.15.53:6800/1659316290,v1:172.21.15.53:6801/1659316290] boot 2023-11-16T16:33:15.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:15 smithi001 ceph-mon[98800]: osdmap e18: 3 total, 3 up, 3 in 2023-11-16T16:33:15.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:15 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-16T16:33:15.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:15 smithi026 ceph-mon[100112]: purged_snaps scrub starts 2023-11-16T16:33:15.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:15 smithi026 ceph-mon[100112]: purged_snaps scrub ok 2023-11-16T16:33:15.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:15 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-16T16:33:15.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:15 smithi026 ceph-mon[100112]: osd.2 [v2:172.21.15.53:6800/1659316290,v1:172.21.15.53:6801/1659316290] boot 2023-11-16T16:33:15.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:15 smithi026 ceph-mon[100112]: osdmap e18: 3 total, 3 up, 3 in 2023-11-16T16:33:15.510 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:15 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-16T16:33:15.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.535+0000 7fe95599f700 1 -- 172.21.15.1:0/3452768042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9500feb60 msgr2=0x7fe9500fef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:33:15.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.535+0000 7fe95599f700 1 --2- 172.21.15.1:0/3452768042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9500feb60 0x7fe9500fef70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe940009990 tx=0x7fe940009ca0 comp rx=0 tx=0).stop 2023-11-16T16:33:15.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.535+0000 7fe95599f700 1 -- 172.21.15.1:0/3452768042 shutdown_connections 2023-11-16T16:33:15.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.535+0000 7fe95599f700 1 --2- 172.21.15.1:0/3452768042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9500ffe60 0x7fe9501002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:33:15.538 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.535+0000 7fe95599f700 1 --2- 172.21.15.1:0/3452768042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9500feb60 0x7fe9500fef70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:33:15.538 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.535+0000 7fe95599f700 1 --2- 172.21.15.1:0/3452768042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9500ff4b0 0x7fe9500ff920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:33:15.538 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.535+0000 7fe95599f700 1 -- 172.21.15.1:0/3452768042 >> 172.21.15.1:0/3452768042 conn(0x7fe9500fa7f0 msgr2=0x7fe9500fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:33:15.538 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.535+0000 7fe95599f700 1 -- 172.21.15.1:0/3452768042 shutdown_connections 2023-11-16T16:33:15.538 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.535+0000 7fe95599f700 1 -- 172.21.15.1:0/3452768042 wait complete. 2023-11-16T16:33:15.538 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.535+0000 7fe95599f700 1 Processor -- start 2023-11-16T16:33:15.538 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.536+0000 7fe95599f700 1 -- start start 2023-11-16T16:33:15.540 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.536+0000 7fe95599f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9500feb60 0x7fe950198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:33:15.540 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.536+0000 7fe95599f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9500ff4b0 0x7fe950198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:33:15.540 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.536+0000 7fe95599f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9500ffe60 0x7fe95019f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:33:15.540 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.536+0000 7fe95599f700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95019fdb0 con 0x7fe9500ff4b0 2023-11-16T16:33:15.540 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.536+0000 7fe95599f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95019fef0 con 0x7fe9500ffe60 2023-11-16T16:33:15.541 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.536+0000 7fe95599f700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9501a0080 con 0x7fe9500feb60 2023-11-16T16:33:15.541 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.536+0000 7fe94e7fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9500ff4b0 0x7fe950198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:33:15.541 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.536+0000 7fe94e7fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9500ff4b0 0x7fe950198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40610/0 (socket says 172.21.15.1:40610) 2023-11-16T16:33:15.541 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.536+0000 7fe94e7fc700 1 -- 172.21.15.1:0/3759038836 learned_addr learned my addr 172.21.15.1:0/3759038836 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:33:15.541 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.536+0000 7fe94effd700 1 --2- 172.21.15.1:0/3759038836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9500feb60 0x7fe950198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:33:15.541 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.536+0000 7fe94f7fe700 1 --2- 172.21.15.1:0/3759038836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9500ffe60 0x7fe95019f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:33:15.541 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.536+0000 7fe94e7fc700 1 -- 172.21.15.1:0/3759038836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9500feb60 msgr2=0x7fe950198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:33:15.541 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.536+0000 7fe94e7fc700 1 --2- 172.21.15.1:0/3759038836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9500feb60 0x7fe950198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:33:15.542 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.536+0000 7fe94e7fc700 1 -- 172.21.15.1:0/3759038836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9500ffe60 msgr2=0x7fe95019f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:33:15.542 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.536+0000 7fe94e7fc700 1 --2- 172.21.15.1:0/3759038836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9500ffe60 0x7fe95019f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:33:15.542 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.536+0000 7fe94e7fc700 1 -- 172.21.15.1:0/3759038836 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9400095f0 con 0x7fe9500ff4b0 2023-11-16T16:33:15.542 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.536+0000 7fe94f7fe700 1 --2- 172.21.15.1:0/3759038836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9500ffe60 0x7fe95019f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:33:15.542 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.537+0000 7fe94e7fc700 1 --2- 172.21.15.1:0/3759038836 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9500ff4b0 0x7fe950198760 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe94400e9d0 tx=0x7fe94400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:33:15.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:15 smithi053 ceph-mon[100638]: purged_snaps scrub starts 2023-11-16T16:33:15.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:15 smithi053 ceph-mon[100638]: purged_snaps scrub ok 2023-11-16T16:33:15.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:15 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-16T16:33:15.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:15 smithi053 ceph-mon[100638]: osd.2 [v2:172.21.15.53:6800/1659316290,v1:172.21.15.53:6801/1659316290] boot 2023-11-16T16:33:15.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:15 smithi053 ceph-mon[100638]: osdmap e18: 3 total, 3 up, 3 in 2023-11-16T16:33:15.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:15 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-16T16:33:15.917 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.915+0000 7fe95499d700 1 -- 172.21.15.1:0/3759038836 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe944009790 con 0x7fe9500ff4b0 2023-11-16T16:33:15.918 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.915+0000 7fe95499d700 1 -- 172.21.15.1:0/3759038836 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe94400cbd0 con 0x7fe9500ff4b0 2023-11-16T16:33:15.918 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.915+0000 7fe95599f700 1 -- 172.21.15.1:0/3759038836 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe95010de20 con 0x7fe9500ff4b0 2023-11-16T16:33:15.918 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.915+0000 7fe95499d700 1 -- 172.21.15.1:0/3759038836 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe944018680 con 0x7fe9500ff4b0 2023-11-16T16:33:15.918 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.916+0000 7fe95599f700 1 -- 172.21.15.1:0/3759038836 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe95010dc90 con 0x7fe9500ff4b0 2023-11-16T16:33:15.918 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.916+0000 7fe95599f700 1 -- 172.21.15.1:0/3759038836 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe950109940 con 0x7fe9500ff4b0 2023-11-16T16:33:15.920 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.918+0000 7fe95499d700 1 -- 172.21.15.1:0/3759038836 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 15) v1 ==== 135658+0+0 (secure 0 0 0) 0x7fe94400cd40 con 0x7fe9500ff4b0 2023-11-16T16:33:15.921 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.919+0000 7fe95499d700 1 --2- 172.21.15.1:0/3759038836 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fe9380a1140 0x7fe9380a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:33:15.921 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.919+0000 7fe94effd700 1 --2- 172.21.15.1:0/3759038836 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fe9380a1140 0x7fe9380a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:33:15.921 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.919+0000 7fe95499d700 1 -- 172.21.15.1:0/3759038836 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 2851+0+0 (secure 0 0 0) 0x7fe944014070 con 0x7fe9500ff4b0 2023-11-16T16:33:15.921 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.919+0000 7fe94effd700 1 --2- 172.21.15.1:0/3759038836 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fe9380a1140 0x7fe9380a35f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe94000b2e0 tx=0x7fe940005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:33:15.924 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:15.922+0000 7fe95499d700 1 -- 172.21.15.1:0/3759038836 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe944096560 con 0x7fe9500ff4b0 2023-11-16T16:33:16.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:16.128+0000 7fe95599f700 1 -- 172.21.15.1:0/3759038836 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fe950061c10 con 0x7fe9380a1140 2023-11-16T16:33:16.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:16 smithi026 ceph-mon[100112]: osdmap e19: 3 total, 3 up, 3 in 2023-11-16T16:33:16.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:16 smithi026 ceph-mon[100112]: pgmap v76: 0 pgs: ; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-16T16:33:16.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:16 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-11-16T16:33:16.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:16 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:33:16.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:16 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:33:16.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:16 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:33:16.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:16 smithi053 ceph-mon[100638]: osdmap e19: 3 total, 3 up, 3 in 2023-11-16T16:33:16.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:16 smithi053 ceph-mon[100638]: pgmap v76: 0 pgs: ; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-16T16:33:16.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:16 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-11-16T16:33:16.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:16 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:33:16.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:16 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:33:16.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:16 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:33:16.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:16 smithi001 ceph-mon[98800]: osdmap e19: 3 total, 3 up, 3 in 2023-11-16T16:33:16.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:16 smithi001 ceph-mon[98800]: pgmap v76: 0 pgs: ; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-16T16:33:16.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:16 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-11-16T16:33:16.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:16 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:33:16.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:16 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:33:16.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:16 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:33:17.341 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:17 smithi001 ceph-mon[98800]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:33:17.341 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:17 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-11-16T16:33:17.341 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:17 smithi001 ceph-mon[98800]: osdmap e20: 3 total, 3 up, 3 in 2023-11-16T16:33:17.342 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:17 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-11-16T16:33:17.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:17 smithi026 ceph-mon[100112]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:33:17.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:17 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-11-16T16:33:17.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:17 smithi026 ceph-mon[100112]: osdmap e20: 3 total, 3 up, 3 in 2023-11-16T16:33:17.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:17 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-11-16T16:33:17.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:17 smithi053 ceph-mon[100638]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:33:17.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:17 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-11-16T16:33:17.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:17 smithi053 ceph-mon[100638]: osdmap e20: 3 total, 3 up, 3 in 2023-11-16T16:33:17.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:17 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-11-16T16:33:18.182 INFO:journalctl@ceph.osd.0.smithi001.stdout:Nov 16 16:33:17 smithi001 sudo[114853]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-16T16:33:18.182 INFO:journalctl@ceph.osd.0.smithi001.stdout:Nov 16 16:33:17 smithi001 sudo[114853]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-16T16:33:18.183 INFO:journalctl@ceph.osd.0.smithi001.stdout:Nov 16 16:33:17 smithi001 sudo[114853]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-16T16:33:18.183 INFO:journalctl@ceph.osd.0.smithi001.stdout:Nov 16 16:33:17 smithi001 sudo[114853]: pam_unix(sudo:session): session closed for user root 2023-11-16T16:33:18.203 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:18.201+0000 7fe95499d700 1 -- 172.21.15.1:0/3759038836 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fe944096930 con 0x7fe9500ff4b0 2023-11-16T16:33:18.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:18 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-11-16T16:33:18.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:18 smithi026 ceph-mon[100112]: osdmap e21: 3 total, 3 up, 3 in 2023-11-16T16:33:18.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:18 smithi026 ceph-mon[100112]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-16T16:33:18.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:18 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-11-16T16:33:18.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:18 smithi053 ceph-mon[100638]: osdmap e21: 3 total, 3 up, 3 in 2023-11-16T16:33:18.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:18 smithi053 ceph-mon[100638]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-16T16:33:18.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:18 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-11-16T16:33:18.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:18 smithi001 ceph-mon[98800]: osdmap e21: 3 total, 3 up, 3 in 2023-11-16T16:33:18.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:18 smithi001 ceph-mon[98800]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-16T16:33:18.683 INFO:journalctl@ceph.osd.0.smithi001.stdout:Nov 16 16:33:18 smithi001 sudo[114889]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-16T16:33:18.683 INFO:journalctl@ceph.osd.0.smithi001.stdout:Nov 16 16:33:18 smithi001 sudo[114889]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-16T16:33:18.683 INFO:journalctl@ceph.osd.0.smithi001.stdout:Nov 16 16:33:18 smithi001 sudo[114889]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-16T16:33:18.683 INFO:journalctl@ceph.osd.0.smithi001.stdout:Nov 16 16:33:18 smithi001 sudo[114889]: pam_unix(sudo:session): session closed for user root 2023-11-16T16:33:19.008 INFO:journalctl@ceph.osd.1.smithi026.stdout:Nov 16 16:33:18 smithi026 sudo[107526]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-16T16:33:19.008 INFO:journalctl@ceph.osd.1.smithi026.stdout:Nov 16 16:33:18 smithi026 sudo[107526]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-16T16:33:19.008 INFO:journalctl@ceph.osd.1.smithi026.stdout:Nov 16 16:33:18 smithi026 sudo[107526]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-16T16:33:19.008 INFO:journalctl@ceph.osd.1.smithi026.stdout:Nov 16 16:33:18 smithi026 sudo[107526]: pam_unix(sudo:session): session closed for user root 2023-11-16T16:33:19.508 INFO:journalctl@ceph.osd.1.smithi026.stdout:Nov 16 16:33:19 smithi026 sudo[107529]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-16T16:33:19.508 INFO:journalctl@ceph.osd.1.smithi026.stdout:Nov 16 16:33:19 smithi026 sudo[107529]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-16T16:33:19.508 INFO:journalctl@ceph.osd.1.smithi026.stdout:Nov 16 16:33:19 smithi026 sudo[107529]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-16T16:33:19.508 INFO:journalctl@ceph.osd.1.smithi026.stdout:Nov 16 16:33:19 smithi026 sudo[107529]: pam_unix(sudo:session): session closed for user root 2023-11-16T16:33:19.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:19 smithi026 ceph-mon[100112]: osdmap e22: 3 total, 3 up, 3 in 2023-11-16T16:33:19.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:19 smithi026 ceph-mon[100112]: mgrmap e16: x(active, since 2m), standbys: y, z 2023-11-16T16:33:19.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:19 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:19.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:19 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/4113691785' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9c290fd9-e121-4a81-9687-b0eb93debd86"}]: dispatch 2023-11-16T16:33:19.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:19 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/4113691785' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9c290fd9-e121-4a81-9687-b0eb93debd86"}]': finished 2023-11-16T16:33:19.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:19 smithi026 ceph-mon[100112]: osdmap e23: 4 total, 3 up, 4 in 2023-11-16T16:33:19.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:19 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-16T16:33:19.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:19 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:19.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:19 smithi053 ceph-mon[100638]: osdmap e22: 3 total, 3 up, 3 in 2023-11-16T16:33:19.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:19 smithi053 ceph-mon[100638]: mgrmap e16: x(active, since 2m), standbys: y, z 2023-11-16T16:33:19.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:19 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:19.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:19 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/4113691785' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9c290fd9-e121-4a81-9687-b0eb93debd86"}]: dispatch 2023-11-16T16:33:19.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:19 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/4113691785' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9c290fd9-e121-4a81-9687-b0eb93debd86"}]': finished 2023-11-16T16:33:19.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:19 smithi053 ceph-mon[100638]: osdmap e23: 4 total, 3 up, 4 in 2023-11-16T16:33:19.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:19 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-16T16:33:19.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:19 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:19.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:19 smithi001 ceph-mon[98800]: osdmap e22: 3 total, 3 up, 3 in 2023-11-16T16:33:19.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:19 smithi001 ceph-mon[98800]: mgrmap e16: x(active, since 2m), standbys: y, z 2023-11-16T16:33:19.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:19 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:19.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:19 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/4113691785' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9c290fd9-e121-4a81-9687-b0eb93debd86"}]: dispatch 2023-11-16T16:33:19.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:19 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/4113691785' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9c290fd9-e121-4a81-9687-b0eb93debd86"}]': finished 2023-11-16T16:33:19.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:19 smithi001 ceph-mon[98800]: osdmap e23: 4 total, 3 up, 4 in 2023-11-16T16:33:19.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:19 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-16T16:33:19.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:19 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:20.041 INFO:journalctl@ceph.osd.2.smithi053.stdout:Nov 16 16:33:19 smithi053 sudo[108630]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-16T16:33:20.041 INFO:journalctl@ceph.osd.2.smithi053.stdout:Nov 16 16:33:19 smithi053 sudo[108630]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-16T16:33:20.041 INFO:journalctl@ceph.osd.2.smithi053.stdout:Nov 16 16:33:19 smithi053 sudo[108630]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-16T16:33:20.041 INFO:journalctl@ceph.osd.2.smithi053.stdout:Nov 16 16:33:19 smithi053 sudo[108630]: pam_unix(sudo:session): session closed for user root 2023-11-16T16:33:20.330 INFO:journalctl@ceph.osd.2.smithi053.stdout:Nov 16 16:33:20 smithi053 sudo[108633]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-16T16:33:20.331 INFO:journalctl@ceph.osd.2.smithi053.stdout:Nov 16 16:33:20 smithi053 sudo[108633]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-16T16:33:20.331 INFO:journalctl@ceph.osd.2.smithi053.stdout:Nov 16 16:33:20 smithi053 sudo[108633]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-16T16:33:20.331 INFO:journalctl@ceph.osd.2.smithi053.stdout:Nov 16 16:33:20 smithi053 sudo[108633]: pam_unix(sudo:session): session closed for user root 2023-11-16T16:33:20.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:20 smithi001 sudo[115382]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-16T16:33:20.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:20 smithi001 sudo[115382]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-16T16:33:20.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:20 smithi001 sudo[115382]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-16T16:33:20.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:20 smithi001 sudo[115382]: pam_unix(sudo:session): session closed for user root 2023-11-16T16:33:21.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:20 smithi001 sudo[115754]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-16T16:33:21.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:20 smithi001 sudo[115754]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-16T16:33:21.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:20 smithi001 sudo[115754]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-16T16:33:21.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:20 smithi001 sudo[115754]: pam_unix(sudo:session): session closed for user root 2023-11-16T16:33:21.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:21 smithi053 sudo[108636]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-16T16:33:21.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:21 smithi053 sudo[108636]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-16T16:33:21.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:21 smithi053 sudo[108636]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-16T16:33:21.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:21 smithi053 sudo[108636]: pam_unix(sudo:session): session closed for user root 2023-11-16T16:33:22.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:21 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/2688130338' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:33:22.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:21 smithi001 ceph-mon[98800]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-16T16:33:22.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:21 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:22.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:21 smithi001 ceph-mon[98800]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-16T16:33:22.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:21 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/2688130338' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:33:22.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:21 smithi026 ceph-mon[100112]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-16T16:33:22.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:21 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:22.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:21 smithi026 ceph-mon[100112]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-16T16:33:22.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:21 smithi053 sudo[108639]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-16T16:33:22.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:21 smithi053 sudo[108639]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-16T16:33:22.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:21 smithi053 sudo[108639]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-16T16:33:22.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:21 smithi053 sudo[108639]: pam_unix(sudo:session): session closed for user root 2023-11-16T16:33:22.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:21 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/2688130338' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:33:22.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:21 smithi053 ceph-mon[100638]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-16T16:33:22.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:21 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:22.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:21 smithi053 ceph-mon[100638]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-16T16:33:22.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:22 smithi026 sudo[107540]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-16T16:33:22.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:22 smithi026 sudo[107540]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-16T16:33:22.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:22 smithi026 sudo[107540]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-16T16:33:22.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:22 smithi026 sudo[107540]: pam_unix(sudo:session): session closed for user root 2023-11-16T16:33:23.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:22 smithi001 ceph-mon[98800]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-16T16:33:23.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:22 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T16:33:23.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:22 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:33:23.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:22 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:33:23.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:22 smithi001 ceph-mon[98800]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-16T16:33:23.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:22 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T16:33:23.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:22 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:33:23.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:22 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:33:23.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:22 smithi001 ceph-mon[98800]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-16T16:33:23.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:22 smithi001 ceph-mon[98800]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-16T16:33:23.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:22 smithi001 ceph-mon[98800]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-16T16:33:23.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:22 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T16:33:23.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:22 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:33:23.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:22 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:33:23.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:22 smithi001 ceph-mon[98800]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-16T16:33:23.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:22 smithi026 sudo[107543]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-16T16:33:23.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:22 smithi026 sudo[107543]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-16T16:33:23.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:22 smithi026 sudo[107543]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-16T16:33:23.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:22 smithi026 sudo[107543]: pam_unix(sudo:session): session closed for user root 2023-11-16T16:33:23.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:22 smithi026 ceph-mon[100112]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-16T16:33:23.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:22 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T16:33:23.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:22 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:33:23.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:22 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:33:23.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:22 smithi026 ceph-mon[100112]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-16T16:33:23.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:22 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T16:33:23.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:22 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:33:23.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:22 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:33:23.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:22 smithi026 ceph-mon[100112]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-16T16:33:23.260 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:22 smithi026 ceph-mon[100112]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-16T16:33:23.260 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:22 smithi026 ceph-mon[100112]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-16T16:33:23.260 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:22 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T16:33:23.260 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:22 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:33:23.260 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:22 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:33:23.260 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:22 smithi026 ceph-mon[100112]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-16T16:33:23.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:22 smithi053 ceph-mon[100638]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-16T16:33:23.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:22 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T16:33:23.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:22 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:33:23.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:22 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:33:23.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:22 smithi053 ceph-mon[100638]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-16T16:33:23.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:22 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T16:33:23.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:22 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:33:23.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:22 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:33:23.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:22 smithi053 ceph-mon[100638]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-16T16:33:23.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:22 smithi053 ceph-mon[100638]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-16T16:33:23.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:22 smithi053 ceph-mon[100638]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-16T16:33:23.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:22 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T16:33:23.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:22 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:33:23.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:22 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:33:23.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:22 smithi053 ceph-mon[100638]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-16T16:33:24.006 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:23 smithi001 ceph-mon[98800]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-16T16:33:24.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:23 smithi026 ceph-mon[100112]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-16T16:33:24.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:23 smithi053 ceph-mon[100638]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-16T16:33:25.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:25 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-11-16T16:33:25.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:25 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:33:25.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:25 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-11-16T16:33:25.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:25 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:33:25.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:25 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-11-16T16:33:25.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:25 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:33:26.680 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:26 smithi001 ceph-mon[98800]: Deploying daemon osd.3 on smithi001 2023-11-16T16:33:26.680 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:26 smithi001 ceph-mon[98800]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-16T16:33:26.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:26 smithi026 ceph-mon[100112]: Deploying daemon osd.3 on smithi001 2023-11-16T16:33:26.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:26 smithi026 ceph-mon[100112]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-16T16:33:26.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:26 smithi053 ceph-mon[100638]: Deploying daemon osd.3 on smithi001 2023-11-16T16:33:26.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:26 smithi053 ceph-mon[100638]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-11-16T16:33:27.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:27 smithi026 ceph-mon[100112]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2023-11-16T16:33:27.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:27 smithi053 ceph-mon[100638]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2023-11-16T16:33:27.871 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:27 smithi001 ceph-mon[98800]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2023-11-16T16:33:29.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:29 smithi026 ceph-mon[100112]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2023-11-16T16:33:29.789 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:29 smithi001 ceph-mon[98800]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2023-11-16T16:33:29.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:29 smithi053 ceph-mon[100638]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2023-11-16T16:33:31.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:30 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:31.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:30 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:31.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:30 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:33:31.330 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:30 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:31.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:30 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:31.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:30 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:33:31.347 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:30 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:31.347 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:30 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:31.347 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:30 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:33:32.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:32 smithi001 ceph-mon[98800]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-11-16T16:33:32.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:32 smithi026 ceph-mon[100112]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-11-16T16:33:32.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:32 smithi053 ceph-mon[100638]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-11-16T16:33:33.036 INFO:teuthology.orchestra.run.smithi001.stdout:Created osd(s) 3 on host 'smithi001' 2023-11-16T16:33:33.036 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:33.033+0000 7fe95499d700 1 -- 172.21.15.1:0/3759038836 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe950061c10 con 0x7fe9380a1140 2023-11-16T16:33:33.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:33.035+0000 7fe95599f700 1 -- 172.21.15.1:0/3759038836 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fe9380a1140 msgr2=0x7fe9380a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:33:33.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:33.035+0000 7fe95599f700 1 --2- 172.21.15.1:0/3759038836 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fe9380a1140 0x7fe9380a35f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe94000b2e0 tx=0x7fe940005fd0 comp rx=0 tx=0).stop 2023-11-16T16:33:33.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:33.035+0000 7fe95599f700 1 -- 172.21.15.1:0/3759038836 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9500ff4b0 msgr2=0x7fe950198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:33:33.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:33.035+0000 7fe95599f700 1 --2- 172.21.15.1:0/3759038836 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9500ff4b0 0x7fe950198760 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe94400e9d0 tx=0x7fe94400ece0 comp rx=0 tx=0).stop 2023-11-16T16:33:33.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:33.035+0000 7fe95599f700 1 -- 172.21.15.1:0/3759038836 shutdown_connections 2023-11-16T16:33:33.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:33.035+0000 7fe95599f700 1 --2- 172.21.15.1:0/3759038836 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fe9380a1140 0x7fe9380a35f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:33:33.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:33.035+0000 7fe95599f700 1 --2- 172.21.15.1:0/3759038836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9500ffe60 0x7fe95019f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:33:33.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:33.035+0000 7fe95599f700 1 --2- 172.21.15.1:0/3759038836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe9500feb60 0x7fe950198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:33:33.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:33.036+0000 7fe95599f700 1 --2- 172.21.15.1:0/3759038836 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe9500ff4b0 0x7fe950198760 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:33:33.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:33.036+0000 7fe95599f700 1 -- 172.21.15.1:0/3759038836 >> 172.21.15.1:0/3759038836 conn(0x7fe9500fa7f0 msgr2=0x7fe9500fc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:33:33.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:33.036+0000 7fe95599f700 1 -- 172.21.15.1:0/3759038836 shutdown_connections 2023-11-16T16:33:33.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:33:33.036+0000 7fe95599f700 1 -- 172.21.15.1:0/3759038836 wait complete. 2023-11-16T16:33:33.526 DEBUG:teuthology.orchestra.run.smithi001:osd.3> sudo journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@osd.3.service 2023-11-16T16:33:33.528 INFO:tasks.cephadm:Deploying osd.4 on smithi026 with /dev/vg_nvme/lv_3... 2023-11-16T16:33:33.529 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-11-16T16:33:33.560 INFO:journalctl@ceph.osd.3.smithi001.stdout:-- Logs begin at Thu 2023-11-16 16:14:27 UTC. -- 2023-11-16T16:33:33.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:33 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:33.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:33 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:33.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:33 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:33:33.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:33 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:33:33.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:33 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:33:33.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:33 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:33.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:33 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:33:33.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:33 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:33:33.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:33 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:33.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:33 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:33.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:33 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:33:33.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:33 smithi001 ceph-mon[98800]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-11-16T16:33:33.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:33 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:33.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:33 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:33.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:33 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:33:33.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:33 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:33:33.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:33 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:33:33.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:33 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:33.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:33 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:33:33.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:33 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:33:33.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:33 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:33.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:33 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:33.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:33 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:33:33.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:33 smithi026 ceph-mon[100112]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-11-16T16:33:33.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:33 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:33.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:33 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:33.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:33 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:33:33.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:33 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:33:33.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:33 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:33:33.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:33 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:33.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:33 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:33:33.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:33 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:33:33.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:33 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:33.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:33 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:33.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:33 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:33:33.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:33 smithi053 ceph-mon[100638]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-11-16T16:33:34.190 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.b/config 2023-11-16T16:33:34.241 INFO:journalctl@ceph.osd.3.smithi001.stdout:Nov 16 16:33:33 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-3[117210]: 2023-11-16T16:33:33.971+0000 7fa7124e7640 -1 osd.3 0 log_to_monitors true 2023-11-16T16:33:34.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:34 smithi026 ceph-mon[100112]: from='osd.3 [v2:172.21.15.1:6810/2728987281,v1:172.21.15.1:6811/2728987281]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-11-16T16:33:34.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:34 smithi053 ceph-mon[100638]: from='osd.3 [v2:172.21.15.1:6810/2728987281,v1:172.21.15.1:6811/2728987281]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-11-16T16:33:34.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:34 smithi001 ceph-mon[98800]: from='osd.3 [v2:172.21.15.1:6810/2728987281,v1:172.21.15.1:6811/2728987281]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-11-16T16:33:35.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:35 smithi026 ceph-mon[100112]: from='osd.3 [v2:172.21.15.1:6810/2728987281,v1:172.21.15.1:6811/2728987281]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-11-16T16:33:35.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:35 smithi026 ceph-mon[100112]: osdmap e24: 4 total, 3 up, 4 in 2023-11-16T16:33:35.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:35 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-16T16:33:35.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:35 smithi026 ceph-mon[100112]: from='osd.3 [v2:172.21.15.1:6810/2728987281,v1:172.21.15.1:6811/2728987281]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2023-11-16T16:33:35.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:35 smithi026 ceph-mon[100112]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-11-16T16:33:35.796 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:35 smithi001 ceph-mon[98800]: from='osd.3 [v2:172.21.15.1:6810/2728987281,v1:172.21.15.1:6811/2728987281]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-11-16T16:33:35.796 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:35 smithi001 ceph-mon[98800]: osdmap e24: 4 total, 3 up, 4 in 2023-11-16T16:33:35.797 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:35 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-16T16:33:35.797 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:35 smithi001 ceph-mon[98800]: from='osd.3 [v2:172.21.15.1:6810/2728987281,v1:172.21.15.1:6811/2728987281]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2023-11-16T16:33:35.797 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:35 smithi001 ceph-mon[98800]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-11-16T16:33:35.797 INFO:journalctl@ceph.osd.3.smithi001.stdout:Nov 16 16:33:35 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-3[117210]: 2023-11-16T16:33:35.510+0000 7fa707b5e700 -1 osd.3 0 waiting for initial osdmap 2023-11-16T16:33:35.797 INFO:journalctl@ceph.osd.3.smithi001.stdout:Nov 16 16:33:35 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-3[117210]: 2023-11-16T16:33:35.521+0000 7fa70014c700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-16T16:33:35.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:35 smithi053 ceph-mon[100638]: from='osd.3 [v2:172.21.15.1:6810/2728987281,v1:172.21.15.1:6811/2728987281]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-11-16T16:33:35.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:35 smithi053 ceph-mon[100638]: osdmap e24: 4 total, 3 up, 4 in 2023-11-16T16:33:35.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:35 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-16T16:33:35.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:35 smithi053 ceph-mon[100638]: from='osd.3 [v2:172.21.15.1:6810/2728987281,v1:172.21.15.1:6811/2728987281]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2023-11-16T16:33:35.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:35 smithi053 ceph-mon[100638]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2023-11-16T16:33:35.887 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-11-16T16:33:35.887 INFO:teuthology.orchestra.run.smithi026.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-11-16T16:33:35.888 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10+0 records in 2023-11-16T16:33:35.888 INFO:teuthology.orchestra.run.smithi026.stderr:10+0 records out 2023-11-16T16:33:35.888 INFO:teuthology.orchestra.run.smithi026.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00801505 s, 1.3 GB/s 2023-11-16T16:33:35.888 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping successful for: 2023-11-16T16:33:36.236 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph orch daemon add osd smithi026:vg_nvme/lv_3 2023-11-16T16:33:36.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:36 smithi053 ceph-mon[100638]: from='osd.3 [v2:172.21.15.1:6810/2728987281,v1:172.21.15.1:6811/2728987281]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2023-11-16T16:33:36.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:36 smithi053 ceph-mon[100638]: osdmap e25: 4 total, 3 up, 4 in 2023-11-16T16:33:36.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:36 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-16T16:33:36.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:36 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-16T16:33:36.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:36 smithi053 ceph-mon[100638]: Detected new or changed devices on smithi001 2023-11-16T16:33:36.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:36 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:36.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:36 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:36.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:36 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:33:36.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:36 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:33:36.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:36 smithi053 ceph-mon[100638]: Adjusting osd_memory_target on smithi001 to 8586M 2023-11-16T16:33:36.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:36 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:36.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:36 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:33:36.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:36 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:33:36.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:36 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:33:36.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:36 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:36.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:36 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:33:36.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:36 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:33:36.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:36 smithi053 ceph-mon[100638]: osd.3 [v2:172.21.15.1:6810/2728987281,v1:172.21.15.1:6811/2728987281] boot 2023-11-16T16:33:36.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:36 smithi053 ceph-mon[100638]: osdmap e26: 4 total, 4 up, 4 in 2023-11-16T16:33:36.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:36 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-16T16:33:36.891 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.b/config 2023-11-16T16:33:36.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:36 smithi001 ceph-mon[98800]: from='osd.3 [v2:172.21.15.1:6810/2728987281,v1:172.21.15.1:6811/2728987281]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2023-11-16T16:33:36.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:36 smithi001 ceph-mon[98800]: osdmap e25: 4 total, 3 up, 4 in 2023-11-16T16:33:36.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:36 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-16T16:33:36.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:36 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-16T16:33:36.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:36 smithi001 ceph-mon[98800]: Detected new or changed devices on smithi001 2023-11-16T16:33:36.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:36 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:36.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:36 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:36.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:36 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:33:36.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:36 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:33:36.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:36 smithi001 ceph-mon[98800]: Adjusting osd_memory_target on smithi001 to 8586M 2023-11-16T16:33:36.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:36 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:36.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:36 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:33:36.934 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:36 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:33:36.934 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:36 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:33:36.934 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:36 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:36.934 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:36 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:33:36.934 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:36 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:33:36.934 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:36 smithi001 ceph-mon[98800]: osd.3 [v2:172.21.15.1:6810/2728987281,v1:172.21.15.1:6811/2728987281] boot 2023-11-16T16:33:36.935 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:36 smithi001 ceph-mon[98800]: osdmap e26: 4 total, 4 up, 4 in 2023-11-16T16:33:36.935 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:36 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-16T16:33:37.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:36 smithi026 ceph-mon[100112]: from='osd.3 [v2:172.21.15.1:6810/2728987281,v1:172.21.15.1:6811/2728987281]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2023-11-16T16:33:37.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:36 smithi026 ceph-mon[100112]: osdmap e25: 4 total, 3 up, 4 in 2023-11-16T16:33:37.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:36 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-16T16:33:37.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:36 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-16T16:33:37.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:36 smithi026 ceph-mon[100112]: Detected new or changed devices on smithi001 2023-11-16T16:33:37.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:36 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:37.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:36 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:37.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:36 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:33:37.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:36 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:33:37.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:36 smithi026 ceph-mon[100112]: Adjusting osd_memory_target on smithi001 to 8586M 2023-11-16T16:33:37.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:36 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:37.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:36 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:33:37.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:36 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:33:37.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:36 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:33:37.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:36 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:37.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:36 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:33:37.010 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:36 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:33:37.010 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:36 smithi026 ceph-mon[100112]: osd.3 [v2:172.21.15.1:6810/2728987281,v1:172.21.15.1:6811/2728987281] boot 2023-11-16T16:33:37.010 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:36 smithi026 ceph-mon[100112]: osdmap e26: 4 total, 4 up, 4 in 2023-11-16T16:33:37.010 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:36 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-16T16:33:37.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:37 smithi026 ceph-mon[100112]: purged_snaps scrub starts 2023-11-16T16:33:37.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:37 smithi026 ceph-mon[100112]: purged_snaps scrub ok 2023-11-16T16:33:37.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:37 smithi026 ceph-mon[100112]: osdmap e27: 4 total, 4 up, 4 in 2023-11-16T16:33:37.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:37 smithi026 ceph-mon[100112]: pgmap v95: 1 pgs: 1 peering; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail 2023-11-16T16:33:37.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:37 smithi053 ceph-mon[100638]: purged_snaps scrub starts 2023-11-16T16:33:37.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:37 smithi053 ceph-mon[100638]: purged_snaps scrub ok 2023-11-16T16:33:37.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:37 smithi053 ceph-mon[100638]: osdmap e27: 4 total, 4 up, 4 in 2023-11-16T16:33:37.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:37 smithi053 ceph-mon[100638]: pgmap v95: 1 pgs: 1 peering; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail 2023-11-16T16:33:37.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:37 smithi001 ceph-mon[98800]: purged_snaps scrub starts 2023-11-16T16:33:37.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:37 smithi001 ceph-mon[98800]: purged_snaps scrub ok 2023-11-16T16:33:37.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:37 smithi001 ceph-mon[98800]: osdmap e27: 4 total, 4 up, 4 in 2023-11-16T16:33:37.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:37 smithi001 ceph-mon[98800]: pgmap v95: 1 pgs: 1 peering; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail 2023-11-16T16:33:38.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.130+0000 7f8668818700 1 -- 172.21.15.26:0/3002045635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86640ff1e0 msgr2=0x7f86640ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:33:38.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.130+0000 7f8668818700 1 --2- 172.21.15.26:0/3002045635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86640ff1e0 0x7f86640ff5f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8654009990 tx=0x7f8654009ca0 comp rx=0 tx=0).stop 2023-11-16T16:33:38.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.130+0000 7f8668818700 1 -- 172.21.15.26:0/3002045635 shutdown_connections 2023-11-16T16:33:38.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.130+0000 7f8668818700 1 --2- 172.21.15.26:0/3002045635 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8664100d70 0x7f8664107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:33:38.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.130+0000 7f8668818700 1 --2- 172.21.15.26:0/3002045635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86640ff1e0 0x7f86640ff5f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:33:38.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.130+0000 7f8668818700 1 --2- 172.21.15.26:0/3002045635 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86641003e0 0x7f8664100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:33:38.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.130+0000 7f8668818700 1 -- 172.21.15.26:0/3002045635 >> 172.21.15.26:0/3002045635 conn(0x7f86640fa7f0 msgr2=0x7f86640fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:33:38.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.130+0000 7f8668818700 1 -- 172.21.15.26:0/3002045635 shutdown_connections 2023-11-16T16:33:38.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.130+0000 7f8668818700 1 -- 172.21.15.26:0/3002045635 wait complete. 2023-11-16T16:33:38.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.131+0000 7f8668818700 1 Processor -- start 2023-11-16T16:33:38.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.131+0000 7f8668818700 1 -- start start 2023-11-16T16:33:38.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.131+0000 7f8668818700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86640ff1e0 0x7f86641982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:33:38.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.131+0000 7f8668818700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86641003e0 0x7f8664198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:33:38.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.131+0000 7f8668818700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8664100d70 0x7f866419f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:33:38.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.131+0000 7f8668818700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866419fe80 con 0x7f86641003e0 2023-11-16T16:33:38.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.131+0000 7f8668818700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866419ffc0 con 0x7f8664100d70 2023-11-16T16:33:38.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.131+0000 7f8668818700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86641a0150 con 0x7f86640ff1e0 2023-11-16T16:33:38.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.131+0000 7f8662d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8664100d70 0x7f866419f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:33:38.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.131+0000 7f8662d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8664100d70 0x7f866419f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55060/0 (socket says 172.21.15.26:55060) 2023-11-16T16:33:38.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.131+0000 7f8662d9d700 1 -- 172.21.15.26:0/4175327305 learned_addr learned my addr 172.21.15.26:0/4175327305 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-11-16T16:33:38.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.131+0000 7f8661d9b700 1 --2- 172.21.15.26:0/4175327305 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86641003e0 0x7f8664198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:33:38.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.132+0000 7f866259c700 1 --2- 172.21.15.26:0/4175327305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86640ff1e0 0x7f86641982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:33:38.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.132+0000 7f8662d9d700 1 -- 172.21.15.26:0/4175327305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86640ff1e0 msgr2=0x7f86641982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:33:38.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.132+0000 7f8662d9d700 1 --2- 172.21.15.26:0/4175327305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86640ff1e0 0x7f86641982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:33:38.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.132+0000 7f8662d9d700 1 -- 172.21.15.26:0/4175327305 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86641003e0 msgr2=0x7f8664198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:33:38.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.132+0000 7f8662d9d700 1 --2- 172.21.15.26:0/4175327305 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86641003e0 0x7f8664198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:33:38.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.132+0000 7f8662d9d700 1 -- 172.21.15.26:0/4175327305 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86540095f0 con 0x7f8664100d70 2023-11-16T16:33:38.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.132+0000 7f8662d9d700 1 --2- 172.21.15.26:0/4175327305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8664100d70 0x7f866419f8b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f865c00b970 tx=0x7f865c00bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:33:38.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.132+0000 7f86537fe700 1 -- 172.21.15.26:0/4175327305 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c00d860 con 0x7f8664100d70 2023-11-16T16:33:38.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.132+0000 7f86537fe700 1 -- 172.21.15.26:0/4175327305 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f865c00de80 con 0x7f8664100d70 2023-11-16T16:33:38.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.132+0000 7f8668818700 1 -- 172.21.15.26:0/4175327305 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f866410ac30 con 0x7f8664100d70 2023-11-16T16:33:38.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.132+0000 7f86537fe700 1 -- 172.21.15.26:0/4175327305 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c017400 con 0x7f8664100d70 2023-11-16T16:33:38.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.132+0000 7f8668818700 1 -- 172.21.15.26:0/4175327305 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f866410aaa0 con 0x7f8664100d70 2023-11-16T16:33:38.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.133+0000 7f8668818700 1 -- 172.21.15.26:0/4175327305 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86641a0910 con 0x7f8664100d70 2023-11-16T16:33:38.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.134+0000 7f86537fe700 1 -- 172.21.15.26:0/4175327305 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f865c00e630 con 0x7f8664100d70 2023-11-16T16:33:38.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.135+0000 7f86537fe700 1 --2- 172.21.15.26:0/4175327305 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f864c0a1150 0x7f864c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:33:38.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.135+0000 7f86537fe700 1 -- 172.21.15.26:0/4175327305 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f865c006e40 con 0x7f8664100d70 2023-11-16T16:33:38.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.135+0000 7f866259c700 1 --2- 172.21.15.26:0/4175327305 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f864c0a1150 0x7f864c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:33:38.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.136+0000 7f866259c700 1 --2- 172.21.15.26:0/4175327305 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f864c0a1150 0x7f864c0a3600 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f865400b5c0 tx=0x7f8654005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:33:38.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.139+0000 7f86537fe700 1 -- 172.21.15.26:0/4175327305 <== mon.2 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f865c095bc0 con 0x7f8664100d70 2023-11-16T16:33:38.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:38.343+0000 7f8668818700 1 -- 172.21.15.26:0/4175327305 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f86641a0ec0 con 0x7f864c0a1150 2023-11-16T16:33:39.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:39 smithi001 ceph-mon[98800]: osdmap e28: 4 total, 4 up, 4 in 2023-11-16T16:33:39.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:39 smithi001 ceph-mon[98800]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:33:39.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:39 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:33:39.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:39 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:33:39.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:39 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:33:39.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:39 smithi026 ceph-mon[100112]: osdmap e28: 4 total, 4 up, 4 in 2023-11-16T16:33:39.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:39 smithi026 ceph-mon[100112]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:33:39.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:39 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:33:39.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:39 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:33:39.451 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:39 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:33:39.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:39 smithi053 ceph-mon[100638]: osdmap e28: 4 total, 4 up, 4 in 2023-11-16T16:33:39.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:39 smithi053 ceph-mon[100638]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:33:39.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:39 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:33:39.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:39 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:33:39.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:39 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:33:40.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:40 smithi001 ceph-mon[98800]: osdmap e29: 4 total, 4 up, 4 in 2023-11-16T16:33:40.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:40 smithi001 ceph-mon[98800]: pgmap v98: 1 pgs: 1 peering; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail 2023-11-16T16:33:40.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:40 smithi026 ceph-mon[100112]: osdmap e29: 4 total, 4 up, 4 in 2023-11-16T16:33:40.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:40 smithi026 ceph-mon[100112]: pgmap v98: 1 pgs: 1 peering; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail 2023-11-16T16:33:40.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:40 smithi053 ceph-mon[100638]: osdmap e29: 4 total, 4 up, 4 in 2023-11-16T16:33:40.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:40 smithi053 ceph-mon[100638]: pgmap v98: 1 pgs: 1 peering; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail 2023-11-16T16:33:41.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:41 smithi001 ceph-mon[98800]: osdmap e30: 4 total, 4 up, 4 in 2023-11-16T16:33:41.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:41 smithi001 ceph-mon[98800]: from='client.? 172.21.15.26:0/1333025644' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "533226bc-cc57-4f15-9e29-820294d57e25"}]: dispatch 2023-11-16T16:33:41.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:41 smithi001 ceph-mon[98800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "533226bc-cc57-4f15-9e29-820294d57e25"}]: dispatch 2023-11-16T16:33:41.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:41 smithi001 ceph-mon[98800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "533226bc-cc57-4f15-9e29-820294d57e25"}]': finished 2023-11-16T16:33:41.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:41 smithi001 ceph-mon[98800]: osdmap e31: 5 total, 4 up, 5 in 2023-11-16T16:33:41.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:41 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-16T16:33:41.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:41 smithi026 ceph-mon[100112]: osdmap e30: 4 total, 4 up, 4 in 2023-11-16T16:33:41.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:41 smithi026 ceph-mon[100112]: from='client.? 172.21.15.26:0/1333025644' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "533226bc-cc57-4f15-9e29-820294d57e25"}]: dispatch 2023-11-16T16:33:41.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:41 smithi026 ceph-mon[100112]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "533226bc-cc57-4f15-9e29-820294d57e25"}]: dispatch 2023-11-16T16:33:41.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:41 smithi026 ceph-mon[100112]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "533226bc-cc57-4f15-9e29-820294d57e25"}]': finished 2023-11-16T16:33:41.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:41 smithi026 ceph-mon[100112]: osdmap e31: 5 total, 4 up, 5 in 2023-11-16T16:33:41.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:41 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-16T16:33:41.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:41 smithi053 ceph-mon[100638]: osdmap e30: 4 total, 4 up, 4 in 2023-11-16T16:33:41.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:41 smithi053 ceph-mon[100638]: from='client.? 172.21.15.26:0/1333025644' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "533226bc-cc57-4f15-9e29-820294d57e25"}]: dispatch 2023-11-16T16:33:41.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:41 smithi053 ceph-mon[100638]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "533226bc-cc57-4f15-9e29-820294d57e25"}]: dispatch 2023-11-16T16:33:41.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:41 smithi053 ceph-mon[100638]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "533226bc-cc57-4f15-9e29-820294d57e25"}]': finished 2023-11-16T16:33:41.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:41 smithi053 ceph-mon[100638]: osdmap e31: 5 total, 4 up, 5 in 2023-11-16T16:33:41.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:41 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-16T16:33:42.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:42 smithi001 ceph-mon[98800]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-16T16:33:42.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:42 smithi001 ceph-mon[98800]: from='client.? 172.21.15.26:0/2773148781' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:33:42.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:42 smithi026 ceph-mon[100112]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-16T16:33:42.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:42 smithi026 ceph-mon[100112]: from='client.? 172.21.15.26:0/2773148781' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:33:42.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:42 smithi053 ceph-mon[100638]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-16T16:33:42.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:42 smithi053 ceph-mon[100638]: from='client.? 172.21.15.26:0/2773148781' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:33:43.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:43 smithi026 ceph-mon[100112]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 109 KiB/s, 0 objects/s recovering 2023-11-16T16:33:43.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:43 smithi053 ceph-mon[100638]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 109 KiB/s, 0 objects/s recovering 2023-11-16T16:33:43.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:43 smithi001 ceph-mon[98800]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 109 KiB/s, 0 objects/s recovering 2023-11-16T16:33:45.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:45 smithi026 ceph-mon[100112]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2023-11-16T16:33:45.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:45 smithi053 ceph-mon[100638]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2023-11-16T16:33:45.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:45 smithi001 ceph-mon[98800]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2023-11-16T16:33:47.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:47 smithi026 ceph-mon[100112]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-16T16:33:47.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:47 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-11-16T16:33:47.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:47 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:33:47.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:47 smithi053 ceph-mon[100638]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-16T16:33:47.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:47 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-11-16T16:33:47.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:47 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:33:47.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:47 smithi001 ceph-mon[98800]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-16T16:33:47.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:47 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-11-16T16:33:47.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:47 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:33:48.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:48 smithi026 ceph-mon[100112]: Deploying daemon osd.4 on smithi026 2023-11-16T16:33:48.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:48 smithi053 ceph-mon[100638]: Deploying daemon osd.4 on smithi026 2023-11-16T16:33:48.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:48 smithi001 ceph-mon[98800]: Deploying daemon osd.4 on smithi026 2023-11-16T16:33:49.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:49 smithi026 ceph-mon[100112]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2023-11-16T16:33:49.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:49 smithi053 ceph-mon[100638]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2023-11-16T16:33:49.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:49 smithi001 ceph-mon[98800]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2023-11-16T16:33:52.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:52 smithi001 ceph-mon[98800]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2023-11-16T16:33:52.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:52 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:52.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:52 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:52.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:52 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:33:52.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:52 smithi026 ceph-mon[100112]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2023-11-16T16:33:52.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:52 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:52.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:52 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:52.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:52 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:33:52.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:52 smithi053 ceph-mon[100638]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2023-11-16T16:33:52.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:52 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:52.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:52 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:52.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:52 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:33:53.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:53 smithi026 ceph-mon[100112]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-11-16T16:33:53.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:53 smithi053 ceph-mon[100638]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-11-16T16:33:53.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:53 smithi001 ceph-mon[98800]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-11-16T16:33:54.764 INFO:teuthology.orchestra.run.smithi026.stdout:Created osd(s) 4 on host 'smithi026' 2023-11-16T16:33:54.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:54.761+0000 7f86537fe700 1 -- 172.21.15.26:0/4175327305 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f86641a0ec0 con 0x7f864c0a1150 2023-11-16T16:33:54.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:54.763+0000 7f8668818700 1 -- 172.21.15.26:0/4175327305 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f864c0a1150 msgr2=0x7f864c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:33:54.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:54.763+0000 7f8668818700 1 --2- 172.21.15.26:0/4175327305 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f864c0a1150 0x7f864c0a3600 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f865400b5c0 tx=0x7f8654005fd0 comp rx=0 tx=0).stop 2023-11-16T16:33:54.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:54.763+0000 7f8668818700 1 -- 172.21.15.26:0/4175327305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8664100d70 msgr2=0x7f866419f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:33:54.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:54.763+0000 7f8668818700 1 --2- 172.21.15.26:0/4175327305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8664100d70 0x7f866419f8b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f865c00b970 tx=0x7f865c00bc80 comp rx=0 tx=0).stop 2023-11-16T16:33:54.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:54.764+0000 7f8668818700 1 -- 172.21.15.26:0/4175327305 shutdown_connections 2023-11-16T16:33:54.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:54.764+0000 7f8668818700 1 --2- 172.21.15.26:0/4175327305 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f864c0a1150 0x7f864c0a3600 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:33:54.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:54.764+0000 7f8668818700 1 --2- 172.21.15.26:0/4175327305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8664100d70 0x7f866419f8b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:33:54.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:54.764+0000 7f8668818700 1 --2- 172.21.15.26:0/4175327305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f86640ff1e0 0x7f86641982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:33:54.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:54.764+0000 7f8668818700 1 --2- 172.21.15.26:0/4175327305 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86641003e0 0x7f8664198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:33:54.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:54.764+0000 7f8668818700 1 -- 172.21.15.26:0/4175327305 >> 172.21.15.26:0/4175327305 conn(0x7f86640fa7f0 msgr2=0x7f8664068ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:33:54.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:54.764+0000 7f8668818700 1 -- 172.21.15.26:0/4175327305 shutdown_connections 2023-11-16T16:33:54.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:33:54.764+0000 7f8668818700 1 -- 172.21.15.26:0/4175327305 wait complete. 2023-11-16T16:33:55.173 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:55 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:55.173 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:55 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:55.173 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:55 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:33:55.173 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:55 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:33:55.173 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:55 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:33:55.173 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:55 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:55.173 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:55 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:33:55.173 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:55 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:33:55.174 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:55 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:55.174 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:55 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:55.174 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:55 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:33:55.175 DEBUG:teuthology.orchestra.run.smithi026:osd.4> sudo journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@osd.4.service 2023-11-16T16:33:55.178 INFO:tasks.cephadm:Deploying osd.5 on smithi053 with /dev/vg_nvme/lv_3... 2023-11-16T16:33:55.178 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-11-16T16:33:55.220 INFO:journalctl@ceph.osd.4.smithi026.stdout:-- Logs begin at Thu 2023-11-16 16:14:13 UTC. -- 2023-11-16T16:33:55.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:55 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:55.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:55 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:55.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:55 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:33:55.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:55 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:33:55.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:55 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:33:55.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:55 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:55.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:55 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:33:55.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:55 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:33:55.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:55 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:55.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:55 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:55.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:55 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:33:55.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:55 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:55.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:55 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:55.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:55 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:33:55.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:55 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:33:55.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:55 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:33:55.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:55 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:55.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:55 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:33:55.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:55 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:33:55.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:55 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:55.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:55 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:55.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:55 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:33:55.852 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-16T16:33:56.008 INFO:journalctl@ceph.osd.4.smithi026.stdout:Nov 16 16:33:55 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-4[110709]: 2023-11-16T16:33:55.783+0000 7f842ddc4640 -1 osd.4 0 log_to_monitors true 2023-11-16T16:33:56.330 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:56 smithi053 ceph-mon[100638]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-11-16T16:33:56.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:56 smithi053 ceph-mon[100638]: from='osd.4 [v2:172.21.15.26:6808/2954740024,v1:172.21.15.26:6809/2954740024]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-16T16:33:56.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:56 smithi053 ceph-mon[100638]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-16T16:33:56.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:56 smithi001 ceph-mon[98800]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-11-16T16:33:56.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:56 smithi001 ceph-mon[98800]: from='osd.4 [v2:172.21.15.26:6808/2954740024,v1:172.21.15.26:6809/2954740024]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-16T16:33:56.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:56 smithi001 ceph-mon[98800]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-16T16:33:56.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:56 smithi026 ceph-mon[100112]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-11-16T16:33:56.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:56 smithi026 ceph-mon[100112]: from='osd.4 [v2:172.21.15.26:6808/2954740024,v1:172.21.15.26:6809/2954740024]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-16T16:33:56.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:56 smithi026 ceph-mon[100112]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-16T16:33:57.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:57 smithi053 ceph-mon[100638]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-11-16T16:33:57.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:57 smithi053 ceph-mon[100638]: osdmap e32: 5 total, 4 up, 5 in 2023-11-16T16:33:57.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:57 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-16T16:33:57.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:57 smithi053 ceph-mon[100638]: from='osd.4 [v2:172.21.15.26:6808/2954740024,v1:172.21.15.26:6809/2954740024]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-11-16T16:33:57.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:57 smithi053 ceph-mon[100638]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-11-16T16:33:57.425 INFO:journalctl@ceph.osd.4.smithi026.stdout:Nov 16 16:33:57 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-4[110709]: 2023-11-16T16:33:57.044+0000 7f842343b700 -1 osd.4 0 waiting for initial osdmap 2023-11-16T16:33:57.425 INFO:journalctl@ceph.osd.4.smithi026.stdout:Nov 16 16:33:57 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-4[110709]: 2023-11-16T16:33:57.051+0000 7f841ba29700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-16T16:33:57.426 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:57 smithi026 ceph-mon[100112]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-11-16T16:33:57.426 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:57 smithi026 ceph-mon[100112]: osdmap e32: 5 total, 4 up, 5 in 2023-11-16T16:33:57.426 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:57 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-16T16:33:57.426 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:57 smithi026 ceph-mon[100112]: from='osd.4 [v2:172.21.15.26:6808/2954740024,v1:172.21.15.26:6809/2954740024]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-11-16T16:33:57.426 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:57 smithi026 ceph-mon[100112]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-11-16T16:33:57.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:57 smithi001 ceph-mon[98800]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-11-16T16:33:57.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:57 smithi001 ceph-mon[98800]: osdmap e32: 5 total, 4 up, 5 in 2023-11-16T16:33:57.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:57 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-16T16:33:57.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:57 smithi001 ceph-mon[98800]: from='osd.4 [v2:172.21.15.26:6808/2954740024,v1:172.21.15.26:6809/2954740024]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-11-16T16:33:57.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:57 smithi001 ceph-mon[98800]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-11-16T16:33:57.591 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-11-16T16:33:57.591 INFO:teuthology.orchestra.run.smithi053.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-11-16T16:33:57.591 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10+0 records in 2023-11-16T16:33:57.591 INFO:teuthology.orchestra.run.smithi053.stderr:10+0 records out 2023-11-16T16:33:57.591 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00802395 s, 1.3 GB/s 2023-11-16T16:33:57.592 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping successful for: 2023-11-16T16:33:58.009 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph orch daemon add osd smithi053:vg_nvme/lv_3 2023-11-16T16:33:58.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:58 smithi053 ceph-mon[100638]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2023-11-16T16:33:58.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:58 smithi053 ceph-mon[100638]: osdmap e33: 5 total, 4 up, 5 in 2023-11-16T16:33:58.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:58 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-16T16:33:58.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:58 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-16T16:33:58.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:58 smithi053 ceph-mon[100638]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-11-16T16:33:58.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:58 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:58.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:58 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:58.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:58 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:33:58.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:58 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:33:58.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:58 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:58.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:58 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:33:58.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:58 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:33:58.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:58 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:58.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:58 smithi001 ceph-mon[98800]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2023-11-16T16:33:58.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:58 smithi001 ceph-mon[98800]: osdmap e33: 5 total, 4 up, 5 in 2023-11-16T16:33:58.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:58 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-16T16:33:58.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:58 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-16T16:33:58.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:58 smithi001 ceph-mon[98800]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-11-16T16:33:58.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:58 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:58.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:58 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:58.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:58 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:33:58.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:58 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:33:58.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:58 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:58.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:58 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:33:58.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:58 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:33:58.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:58 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:58.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:58 smithi026 ceph-mon[100112]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2023-11-16T16:33:58.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:58 smithi026 ceph-mon[100112]: osdmap e33: 5 total, 4 up, 5 in 2023-11-16T16:33:58.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:58 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-16T16:33:58.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:58 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-16T16:33:58.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:58 smithi026 ceph-mon[100112]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-11-16T16:33:58.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:58 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:58.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:58 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:58.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:58 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:33:58.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:58 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:33:58.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:58 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:58.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:58 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:33:58.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:58 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:33:58.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:58 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:33:58.661 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-16T16:33:59.121 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:59 smithi053 ceph-mon[100638]: purged_snaps scrub starts 2023-11-16T16:33:59.122 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:59 smithi053 ceph-mon[100638]: purged_snaps scrub ok 2023-11-16T16:33:59.122 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:59 smithi053 ceph-mon[100638]: Detected new or changed devices on smithi026 2023-11-16T16:33:59.122 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:59 smithi053 ceph-mon[100638]: Adjusting osd_memory_target on smithi026 to 8586M 2023-11-16T16:33:59.122 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:59 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:33:59.122 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:59 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:33:59.122 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:59 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:33:59.123 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:59 smithi053 ceph-mon[100638]: osd.4 [v2:172.21.15.26:6808/2954740024,v1:172.21.15.26:6809/2954740024] boot 2023-11-16T16:33:59.123 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:59 smithi053 ceph-mon[100638]: osdmap e34: 5 total, 5 up, 5 in 2023-11-16T16:33:59.123 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:33:59 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-16T16:33:59.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:59 smithi001 ceph-mon[98800]: purged_snaps scrub starts 2023-11-16T16:33:59.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:59 smithi001 ceph-mon[98800]: purged_snaps scrub ok 2023-11-16T16:33:59.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:59 smithi001 ceph-mon[98800]: Detected new or changed devices on smithi026 2023-11-16T16:33:59.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:59 smithi001 ceph-mon[98800]: Adjusting osd_memory_target on smithi026 to 8586M 2023-11-16T16:33:59.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:59 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:33:59.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:59 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:33:59.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:59 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:33:59.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:59 smithi001 ceph-mon[98800]: osd.4 [v2:172.21.15.26:6808/2954740024,v1:172.21.15.26:6809/2954740024] boot 2023-11-16T16:33:59.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:59 smithi001 ceph-mon[98800]: osdmap e34: 5 total, 5 up, 5 in 2023-11-16T16:33:59.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:33:59 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-16T16:33:59.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:59 smithi026 ceph-mon[100112]: purged_snaps scrub starts 2023-11-16T16:33:59.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:59 smithi026 ceph-mon[100112]: purged_snaps scrub ok 2023-11-16T16:33:59.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:59 smithi026 ceph-mon[100112]: Detected new or changed devices on smithi026 2023-11-16T16:33:59.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:59 smithi026 ceph-mon[100112]: Adjusting osd_memory_target on smithi026 to 8586M 2023-11-16T16:33:59.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:59 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:33:59.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:59 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:33:59.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:59 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:33:59.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:59 smithi026 ceph-mon[100112]: osd.4 [v2:172.21.15.26:6808/2954740024,v1:172.21.15.26:6809/2954740024] boot 2023-11-16T16:33:59.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:59 smithi026 ceph-mon[100112]: osdmap e34: 5 total, 5 up, 5 in 2023-11-16T16:33:59.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:33:59 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-16T16:33:59.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.827+0000 7fa00456a700 1 -- 172.21.15.53:0/3217100415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ffc100d70 msgr2=0x7f9ffc107630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:33:59.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.827+0000 7fa00456a700 1 --2- 172.21.15.53:0/3217100415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ffc100d70 0x7f9ffc107630 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9ff400b0a0 tx=0x7f9ff400b3b0 comp rx=0 tx=0).stop 2023-11-16T16:33:59.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.827+0000 7fa00456a700 1 -- 172.21.15.53:0/3217100415 shutdown_connections 2023-11-16T16:33:59.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.827+0000 7fa00456a700 1 --2- 172.21.15.53:0/3217100415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ffc100d70 0x7f9ffc107630 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:33:59.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.827+0000 7fa00456a700 1 --2- 172.21.15.53:0/3217100415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ffc0ff1e0 0x7f9ffc0ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:33:59.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.827+0000 7fa00456a700 1 --2- 172.21.15.53:0/3217100415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ffc1003e0 0x7f9ffc100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:33:59.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.827+0000 7fa00456a700 1 -- 172.21.15.53:0/3217100415 >> 172.21.15.53:0/3217100415 conn(0x7f9ffc0fa7f0 msgr2=0x7f9ffc0fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:33:59.829 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.827+0000 7fa00456a700 1 -- 172.21.15.53:0/3217100415 shutdown_connections 2023-11-16T16:33:59.830 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.827+0000 7fa00456a700 1 -- 172.21.15.53:0/3217100415 wait complete. 2023-11-16T16:33:59.831 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.828+0000 7fa00456a700 1 Processor -- start 2023-11-16T16:33:59.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.828+0000 7fa00456a700 1 -- start start 2023-11-16T16:33:59.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.828+0000 7fa00456a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ffc0ff1e0 0x7f9ffc196140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:33:59.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.828+0000 7fa00456a700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ffc1003e0 0x7f9ffc196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:33:59.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.828+0000 7fa00456a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ffc100d70 0x7f9ffc19d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:33:59.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.828+0000 7fa00456a700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ffc19dcd0 con 0x7f9ffc1003e0 2023-11-16T16:33:59.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.828+0000 7fa00456a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ffc19de10 con 0x7f9ffc100d70 2023-11-16T16:33:59.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.828+0000 7fa00456a700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ffc19dfa0 con 0x7f9ffc0ff1e0 2023-11-16T16:33:59.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.828+0000 7fa002306700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ffc0ff1e0 0x7f9ffc196140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:33:59.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.828+0000 7fa002306700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ffc0ff1e0 0x7f9ffc196140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57684/0 (socket says 172.21.15.53:57684) 2023-11-16T16:33:59.832 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.828+0000 7fa002306700 1 -- 172.21.15.53:0/211826638 learned_addr learned my addr 172.21.15.53:0/211826638 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-16T16:33:59.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.828+0000 7fa001b05700 1 --2- 172.21.15.53:0/211826638 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ffc1003e0 0x7f9ffc196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:33:59.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.829+0000 7fa002b07700 1 --2- 172.21.15.53:0/211826638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ffc100d70 0x7f9ffc19d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:33:59.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.829+0000 7fa002306700 1 -- 172.21.15.53:0/211826638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ffc100d70 msgr2=0x7f9ffc19d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:33:59.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.829+0000 7fa002306700 1 --2- 172.21.15.53:0/211826638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ffc100d70 0x7f9ffc19d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:33:59.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.829+0000 7fa002306700 1 -- 172.21.15.53:0/211826638 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ffc1003e0 msgr2=0x7f9ffc196680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:33:59.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.829+0000 7fa002306700 1 --2- 172.21.15.53:0/211826638 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ffc1003e0 0x7f9ffc196680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:33:59.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.829+0000 7fa002306700 1 -- 172.21.15.53:0/211826638 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff4009d00 con 0x7f9ffc0ff1e0 2023-11-16T16:33:59.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.829+0000 7fa002b07700 1 --2- 172.21.15.53:0/211826638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ffc100d70 0x7f9ffc19d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:33:59.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.829+0000 7fa002306700 1 --2- 172.21.15.53:0/211826638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ffc0ff1e0 0x7f9ffc196140 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9ff000c9a0 tx=0x7f9ff000ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:33:59.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.829+0000 7f9fef7fe700 1 -- 172.21.15.53:0/211826638 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0004220 con 0x7f9ffc0ff1e0 2023-11-16T16:33:59.833 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.829+0000 7f9fef7fe700 1 -- 172.21.15.53:0/211826638 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ff0007d20 con 0x7f9ffc0ff1e0 2023-11-16T16:33:59.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.829+0000 7fa00456a700 1 -- 172.21.15.53:0/211826638 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ffc10ab30 con 0x7f9ffc0ff1e0 2023-11-16T16:33:59.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.829+0000 7f9fef7fe700 1 -- 172.21.15.53:0/211826638 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0003c00 con 0x7f9ffc0ff1e0 2023-11-16T16:33:59.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.829+0000 7fa00456a700 1 -- 172.21.15.53:0/211826638 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ffc10a9a0 con 0x7f9ffc0ff1e0 2023-11-16T16:33:59.834 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.830+0000 7fa00456a700 1 -- 172.21.15.53:0/211826638 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ffc1072d0 con 0x7f9ffc0ff1e0 2023-11-16T16:33:59.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.831+0000 7f9fef7fe700 1 -- 172.21.15.53:0/211826638 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f9ff0007860 con 0x7f9ffc0ff1e0 2023-11-16T16:33:59.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.832+0000 7f9fef7fe700 1 --2- 172.21.15.53:0/211826638 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f9fe80a11d0 0x7f9fe80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:33:59.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.832+0000 7f9fef7fe700 1 -- 172.21.15.53:0/211826638 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f9ff00cb4c0 con 0x7f9ffc0ff1e0 2023-11-16T16:33:59.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.832+0000 7fa001b05700 1 --2- 172.21.15.53:0/211826638 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f9fe80a11d0 0x7f9fe80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:33:59.835 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.833+0000 7fa001b05700 1 --2- 172.21.15.53:0/211826638 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f9fe80a11d0 0x7f9fe80a3680 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8005950 tx=0x7f9ff800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:33:59.837 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:33:59.836+0000 7f9fef7fe700 1 -- 172.21.15.53:0/211826638 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9ff0095be0 con 0x7f9ffc0ff1e0 2023-11-16T16:34:00.043 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:34:00.042+0000 7fa00456a700 1 -- 172.21.15.53:0/211826638 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ffc061b00 con 0x7f9fe80a11d0 2023-11-16T16:34:00.330 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:00 smithi053 ceph-mon[100638]: osdmap e35: 5 total, 5 up, 5 in 2023-11-16T16:34:00.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:00 smithi053 ceph-mon[100638]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:34:00.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:00 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:34:00.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:00 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:34:00.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:00 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:34:00.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:00 smithi001 ceph-mon[98800]: osdmap e35: 5 total, 5 up, 5 in 2023-11-16T16:34:00.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:00 smithi001 ceph-mon[98800]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:34:00.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:00 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:34:00.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:00 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:34:00.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:00 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:34:00.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:00 smithi026 ceph-mon[100112]: osdmap e35: 5 total, 5 up, 5 in 2023-11-16T16:34:00.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:00 smithi026 ceph-mon[100112]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:34:00.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:00 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:34:00.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:00 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:34:00.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:00 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:34:01.230 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:01 smithi053 ceph-mon[100638]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:34:01.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:01 smithi001 ceph-mon[98800]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:34:01.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:01 smithi026 ceph-mon[100112]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:34:02.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:02 smithi001 ceph-mon[98800]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:34:02.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:02 smithi026 ceph-mon[100112]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:34:02.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:02 smithi053 ceph-mon[100638]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:34:03.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:03 smithi001 ceph-mon[98800]: from='client.? 172.21.15.53:0/1009841731' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3ab25eb-e1d4-4557-bc89-efc9072401ce"}]: dispatch 2023-11-16T16:34:03.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:03 smithi001 ceph-mon[98800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3ab25eb-e1d4-4557-bc89-efc9072401ce"}]: dispatch 2023-11-16T16:34:03.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:03 smithi001 ceph-mon[98800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d3ab25eb-e1d4-4557-bc89-efc9072401ce"}]': finished 2023-11-16T16:34:03.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:03 smithi001 ceph-mon[98800]: osdmap e36: 6 total, 5 up, 6 in 2023-11-16T16:34:03.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:03 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-16T16:34:03.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:03 smithi001 ceph-mon[98800]: from='client.? 172.21.15.53:0/88857165' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:34:03.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:03 smithi026 ceph-mon[100112]: from='client.? 172.21.15.53:0/1009841731' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3ab25eb-e1d4-4557-bc89-efc9072401ce"}]: dispatch 2023-11-16T16:34:03.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:03 smithi026 ceph-mon[100112]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3ab25eb-e1d4-4557-bc89-efc9072401ce"}]: dispatch 2023-11-16T16:34:03.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:03 smithi026 ceph-mon[100112]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d3ab25eb-e1d4-4557-bc89-efc9072401ce"}]': finished 2023-11-16T16:34:03.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:03 smithi026 ceph-mon[100112]: osdmap e36: 6 total, 5 up, 6 in 2023-11-16T16:34:03.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:03 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-16T16:34:03.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:03 smithi026 ceph-mon[100112]: from='client.? 172.21.15.53:0/88857165' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:34:03.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:03 smithi053 ceph-mon[100638]: from='client.? 172.21.15.53:0/1009841731' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3ab25eb-e1d4-4557-bc89-efc9072401ce"}]: dispatch 2023-11-16T16:34:03.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:03 smithi053 ceph-mon[100638]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d3ab25eb-e1d4-4557-bc89-efc9072401ce"}]: dispatch 2023-11-16T16:34:03.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:03 smithi053 ceph-mon[100638]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d3ab25eb-e1d4-4557-bc89-efc9072401ce"}]': finished 2023-11-16T16:34:03.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:03 smithi053 ceph-mon[100638]: osdmap e36: 6 total, 5 up, 6 in 2023-11-16T16:34:03.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:03 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-16T16:34:03.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:03 smithi053 ceph-mon[100638]: from='client.? 172.21.15.53:0/88857165' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:34:04.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:04 smithi001 ceph-mon[98800]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:34:04.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:04 smithi026 ceph-mon[100112]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:34:04.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:04 smithi053 ceph-mon[100638]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:34:05.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:05 smithi026 ceph-mon[100112]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:34:05.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:05 smithi053 ceph-mon[100638]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:34:05.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:05 smithi001 ceph-mon[98800]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:34:07.711 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:07 smithi053 ceph-mon[100638]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:34:07.757 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:07 smithi026 ceph-mon[100112]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:34:07.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:07 smithi001 ceph-mon[98800]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:34:09.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:09 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-11-16T16:34:09.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:09 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:34:09.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:09 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-11-16T16:34:09.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:09 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:34:09.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:09 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-11-16T16:34:09.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:09 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:34:10.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:10 smithi053 ceph-mon[100638]: Deploying daemon osd.5 on smithi053 2023-11-16T16:34:10.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:10 smithi053 ceph-mon[100638]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:34:10.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:10 smithi001 ceph-mon[98800]: Deploying daemon osd.5 on smithi053 2023-11-16T16:34:10.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:10 smithi001 ceph-mon[98800]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:34:10.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:10 smithi026 ceph-mon[100112]: Deploying daemon osd.5 on smithi053 2023-11-16T16:34:10.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:10 smithi026 ceph-mon[100112]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:34:12.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:12 smithi053 ceph-mon[100638]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:34:12.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:12 smithi001 ceph-mon[98800]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:34:12.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:12 smithi026 ceph-mon[100112]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:34:14.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:14 smithi001 ceph-mon[98800]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:34:14.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:14 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:14.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:14 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:14.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:14 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:34:14.702 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:14 smithi053 ceph-mon[100638]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:34:14.702 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:14 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:14.702 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:14 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:14.702 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:14 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:34:14.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:14 smithi026 ceph-mon[100112]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:34:14.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:14 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:14.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:14 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:14.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:14 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:34:16.361 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 5 on host 'smithi053' 2023-11-16T16:34:16.362 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:34:16.359+0000 7f9fef7fe700 1 -- 172.21.15.53:0/211826638 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9ffc061b00 con 0x7f9fe80a11d0 2023-11-16T16:34:16.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:34:16.362+0000 7fa00456a700 1 -- 172.21.15.53:0/211826638 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f9fe80a11d0 msgr2=0x7f9fe80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:34:16.363 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:34:16.362+0000 7fa00456a700 1 --2- 172.21.15.53:0/211826638 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f9fe80a11d0 0x7f9fe80a3680 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8005950 tx=0x7f9ff800b410 comp rx=0 tx=0).stop 2023-11-16T16:34:16.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:34:16.362+0000 7fa00456a700 1 -- 172.21.15.53:0/211826638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ffc0ff1e0 msgr2=0x7f9ffc196140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:34:16.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:34:16.362+0000 7fa00456a700 1 --2- 172.21.15.53:0/211826638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ffc0ff1e0 0x7f9ffc196140 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9ff000c9a0 tx=0x7f9ff000ccb0 comp rx=0 tx=0).stop 2023-11-16T16:34:16.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:34:16.362+0000 7fa00456a700 1 -- 172.21.15.53:0/211826638 shutdown_connections 2023-11-16T16:34:16.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:34:16.362+0000 7fa00456a700 1 --2- 172.21.15.53:0/211826638 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f9fe80a11d0 0x7f9fe80a3680 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:34:16.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:34:16.362+0000 7fa00456a700 1 --2- 172.21.15.53:0/211826638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ffc100d70 0x7f9ffc19d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:34:16.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:34:16.362+0000 7fa00456a700 1 --2- 172.21.15.53:0/211826638 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ffc0ff1e0 0x7f9ffc196140 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:34:16.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:34:16.362+0000 7fa00456a700 1 --2- 172.21.15.53:0/211826638 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ffc1003e0 0x7f9ffc196680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:34:16.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:34:16.362+0000 7fa00456a700 1 -- 172.21.15.53:0/211826638 >> 172.21.15.53:0/211826638 conn(0x7f9ffc0fa7f0 msgr2=0x7f9ffc0fca40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:34:16.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:34:16.362+0000 7fa00456a700 1 -- 172.21.15.53:0/211826638 shutdown_connections 2023-11-16T16:34:16.364 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:34:16.362+0000 7fa00456a700 1 -- 172.21.15.53:0/211826638 wait complete. 2023-11-16T16:34:16.552 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:16 smithi053 ceph-mon[100638]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:34:16.552 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:16 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:16.553 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:16 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:16.553 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:16 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:34:16.553 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:16 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:34:16.553 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:16 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:16.553 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:16 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:16.553 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:16 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:16.553 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:16 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:34:16.752 DEBUG:teuthology.orchestra.run.smithi053:osd.5> sudo journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@osd.5.service 2023-11-16T16:34:16.754 INFO:tasks.cephadm:Deploying osd.6 on smithi001 with /dev/vg_nvme/lv_2... 2023-11-16T16:34:16.755 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-11-16T16:34:16.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:16 smithi026 ceph-mon[100112]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:34:16.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:16 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:16.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:16 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:16.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:16 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:34:16.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:16 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:34:16.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:16 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:16.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:16 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:16.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:16 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:16.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:16 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:34:16.785 INFO:journalctl@ceph.osd.5.smithi053.stdout:-- Logs begin at Thu 2023-11-16 16:14:15 UTC. -- 2023-11-16T16:34:16.802 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:16 smithi001 ceph-mon[98800]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:34:16.802 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:16 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:16.802 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:16 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:16.802 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:16 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:34:16.802 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:16 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:34:16.802 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:16 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:16.802 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:16 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:16.802 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:16 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:16.802 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:16 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:34:17.433 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:34:17.543 INFO:journalctl@ceph.osd.5.smithi053.stdout:Nov 16 16:34:17 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-5[111878]: 2023-11-16T16:34:17.429+0000 7fdbd2cf8640 -1 osd.5 0 log_to_monitors true 2023-11-16T16:34:17.544 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:17 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:34:17.544 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:17 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:34:17.544 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:17 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:34:17.544 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:17 smithi053 ceph-mon[100638]: from='osd.5 [v2:172.21.15.53:6808/3966666318,v1:172.21.15.53:6809/3966666318]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-11-16T16:34:17.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:17 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:34:17.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:17 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:34:17.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:17 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:34:17.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:17 smithi001 ceph-mon[98800]: from='osd.5 [v2:172.21.15.53:6808/3966666318,v1:172.21.15.53:6809/3966666318]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-11-16T16:34:17.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:17 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:34:17.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:17 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:34:17.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:17 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:34:17.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:17 smithi026 ceph-mon[100112]: from='osd.5 [v2:172.21.15.53:6808/3966666318,v1:172.21.15.53:6809/3966666318]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-11-16T16:34:18.474 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:18 smithi001 ceph-mon[98800]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:34:18.474 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:18 smithi001 ceph-mon[98800]: from='osd.5 [v2:172.21.15.53:6808/3966666318,v1:172.21.15.53:6809/3966666318]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-11-16T16:34:18.474 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:18 smithi001 ceph-mon[98800]: osdmap e37: 6 total, 5 up, 6 in 2023-11-16T16:34:18.474 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:18 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-16T16:34:18.474 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:18 smithi001 ceph-mon[98800]: from='osd.5 [v2:172.21.15.53:6808/3966666318,v1:172.21.15.53:6809/3966666318]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2023-11-16T16:34:18.710 INFO:journalctl@ceph.osd.5.smithi053.stdout:Nov 16 16:34:18 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-5[111878]: 2023-11-16T16:34:18.445+0000 7fdbc836f700 -1 osd.5 0 waiting for initial osdmap 2023-11-16T16:34:18.711 INFO:journalctl@ceph.osd.5.smithi053.stdout:Nov 16 16:34:18 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-5[111878]: 2023-11-16T16:34:18.453+0000 7fdbc4164700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-16T16:34:18.711 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:18 smithi053 ceph-mon[100638]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:34:18.711 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:18 smithi053 ceph-mon[100638]: from='osd.5 [v2:172.21.15.53:6808/3966666318,v1:172.21.15.53:6809/3966666318]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-11-16T16:34:18.711 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:18 smithi053 ceph-mon[100638]: osdmap e37: 6 total, 5 up, 6 in 2023-11-16T16:34:18.711 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:18 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-16T16:34:18.711 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:18 smithi053 ceph-mon[100638]: from='osd.5 [v2:172.21.15.53:6808/3966666318,v1:172.21.15.53:6809/3966666318]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2023-11-16T16:34:18.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:18 smithi026 ceph-mon[100112]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:34:18.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:18 smithi026 ceph-mon[100112]: from='osd.5 [v2:172.21.15.53:6808/3966666318,v1:172.21.15.53:6809/3966666318]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-11-16T16:34:18.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:18 smithi026 ceph-mon[100112]: osdmap e37: 6 total, 5 up, 6 in 2023-11-16T16:34:18.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:18 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-16T16:34:18.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:18 smithi026 ceph-mon[100112]: from='osd.5 [v2:172.21.15.53:6808/3966666318,v1:172.21.15.53:6809/3966666318]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2023-11-16T16:34:19.008 INFO:teuthology.orchestra.run.smithi001.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-11-16T16:34:19.009 INFO:teuthology.orchestra.run.smithi001.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-11-16T16:34:19.009 INFO:teuthology.orchestra.run.smithi001.stderr: stderr: 10+0 records in 2023-11-16T16:34:19.009 INFO:teuthology.orchestra.run.smithi001.stderr:10+0 records out 2023-11-16T16:34:19.009 INFO:teuthology.orchestra.run.smithi001.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00808246 s, 1.3 GB/s 2023-11-16T16:34:19.009 INFO:teuthology.orchestra.run.smithi001.stderr:--> Zapping successful for: 2023-11-16T16:34:19.483 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph orch daemon add osd smithi001:vg_nvme/lv_2 2023-11-16T16:34:19.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:19 smithi001 ceph-mon[98800]: from='osd.5 [v2:172.21.15.53:6808/3966666318,v1:172.21.15.53:6809/3966666318]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2023-11-16T16:34:19.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:19 smithi001 ceph-mon[98800]: osdmap e38: 6 total, 5 up, 6 in 2023-11-16T16:34:19.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:19 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-16T16:34:19.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:19 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-16T16:34:19.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:19 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:19.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:19 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:19.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:19 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:34:19.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:19 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:34:19.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:19 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:19.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:19 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:34:19.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:19 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:34:19.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:19 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:19.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:19 smithi026 ceph-mon[100112]: from='osd.5 [v2:172.21.15.53:6808/3966666318,v1:172.21.15.53:6809/3966666318]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2023-11-16T16:34:19.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:19 smithi026 ceph-mon[100112]: osdmap e38: 6 total, 5 up, 6 in 2023-11-16T16:34:19.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:19 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-16T16:34:19.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:19 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-16T16:34:19.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:19 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:19.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:19 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:19.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:19 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:34:19.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:19 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:34:19.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:19 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:19.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:19 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:34:19.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:19 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:34:19.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:19 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:19.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:19 smithi053 ceph-mon[100638]: from='osd.5 [v2:172.21.15.53:6808/3966666318,v1:172.21.15.53:6809/3966666318]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2023-11-16T16:34:19.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:19 smithi053 ceph-mon[100638]: osdmap e38: 6 total, 5 up, 6 in 2023-11-16T16:34:19.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:19 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-16T16:34:19.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:19 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-16T16:34:19.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:19 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:19.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:19 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:19.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:19 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:34:19.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:19 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:34:19.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:19 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:19.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:19 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:34:19.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:19 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:34:19.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:19 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:20.134 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:34:20.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:20 smithi001 ceph-mon[98800]: purged_snaps scrub starts 2023-11-16T16:34:20.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:20 smithi001 ceph-mon[98800]: purged_snaps scrub ok 2023-11-16T16:34:20.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:20 smithi001 ceph-mon[98800]: Detected new or changed devices on smithi053 2023-11-16T16:34:20.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:20 smithi001 ceph-mon[98800]: Adjusting osd_memory_target on smithi053 to 8586M 2023-11-16T16:34:20.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:20 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:34:20.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:20 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:34:20.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:20 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:34:20.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:20 smithi001 ceph-mon[98800]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:34:20.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:20 smithi001 ceph-mon[98800]: osd.5 [v2:172.21.15.53:6808/3966666318,v1:172.21.15.53:6809/3966666318] boot 2023-11-16T16:34:20.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:20 smithi001 ceph-mon[98800]: osdmap e39: 6 total, 6 up, 6 in 2023-11-16T16:34:20.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:20 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-16T16:34:20.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:20 smithi026 ceph-mon[100112]: purged_snaps scrub starts 2023-11-16T16:34:20.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:20 smithi026 ceph-mon[100112]: purged_snaps scrub ok 2023-11-16T16:34:20.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:20 smithi026 ceph-mon[100112]: Detected new or changed devices on smithi053 2023-11-16T16:34:20.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:20 smithi026 ceph-mon[100112]: Adjusting osd_memory_target on smithi053 to 8586M 2023-11-16T16:34:20.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:20 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:34:20.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:20 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:34:20.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:20 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:34:20.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:20 smithi026 ceph-mon[100112]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:34:20.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:20 smithi026 ceph-mon[100112]: osd.5 [v2:172.21.15.53:6808/3966666318,v1:172.21.15.53:6809/3966666318] boot 2023-11-16T16:34:20.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:20 smithi026 ceph-mon[100112]: osdmap e39: 6 total, 6 up, 6 in 2023-11-16T16:34:20.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:20 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-16T16:34:20.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:20 smithi053 ceph-mon[100638]: purged_snaps scrub starts 2023-11-16T16:34:20.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:20 smithi053 ceph-mon[100638]: purged_snaps scrub ok 2023-11-16T16:34:20.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:20 smithi053 ceph-mon[100638]: Detected new or changed devices on smithi053 2023-11-16T16:34:20.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:20 smithi053 ceph-mon[100638]: Adjusting osd_memory_target on smithi053 to 8586M 2023-11-16T16:34:20.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:20 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:34:20.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:20 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:34:20.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:20 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:34:20.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:20 smithi053 ceph-mon[100638]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-11-16T16:34:20.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:20 smithi053 ceph-mon[100638]: osd.5 [v2:172.21.15.53:6808/3966666318,v1:172.21.15.53:6809/3966666318] boot 2023-11-16T16:34:20.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:20 smithi053 ceph-mon[100638]: osdmap e39: 6 total, 6 up, 6 in 2023-11-16T16:34:20.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:20 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-16T16:34:21.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.495+0000 7fdb63134700 1 -- 172.21.15.1:0/2557819887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb5c0ff1e0 msgr2=0x7fdb5c0ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:34:21.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.495+0000 7fdb63134700 1 --2- 172.21.15.1:0/2557819887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb5c0ff1e0 0x7fdb5c0ff5f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdb50009990 tx=0x7fdb50009ca0 comp rx=0 tx=0).stop 2023-11-16T16:34:21.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.495+0000 7fdb63134700 1 -- 172.21.15.1:0/2557819887 shutdown_connections 2023-11-16T16:34:21.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.495+0000 7fdb63134700 1 --2- 172.21.15.1:0/2557819887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb5c100d70 0x7fdb5c107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:34:21.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.495+0000 7fdb63134700 1 --2- 172.21.15.1:0/2557819887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb5c0ff1e0 0x7fdb5c0ff5f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:34:21.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.495+0000 7fdb63134700 1 --2- 172.21.15.1:0/2557819887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb5c1003e0 0x7fdb5c100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:34:21.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.495+0000 7fdb63134700 1 -- 172.21.15.1:0/2557819887 >> 172.21.15.1:0/2557819887 conn(0x7fdb5c0fa7f0 msgr2=0x7fdb5c0fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:34:21.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.495+0000 7fdb63134700 1 -- 172.21.15.1:0/2557819887 shutdown_connections 2023-11-16T16:34:21.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.495+0000 7fdb63134700 1 -- 172.21.15.1:0/2557819887 wait complete. 2023-11-16T16:34:21.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.496+0000 7fdb63134700 1 Processor -- start 2023-11-16T16:34:21.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.496+0000 7fdb63134700 1 -- start start 2023-11-16T16:34:21.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.496+0000 7fdb63134700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb5c0ff1e0 0x7fdb5c069170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:34:21.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.496+0000 7fdb63134700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb5c1003e0 0x7fdb5c19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:34:21.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.496+0000 7fdb63134700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb5c100d70 0x7fdb5c19d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:34:21.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.496+0000 7fdb63134700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c19dbe0 con 0x7fdb5c1003e0 2023-11-16T16:34:21.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.496+0000 7fdb63134700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c19dd50 con 0x7fdb5c100d70 2023-11-16T16:34:21.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.496+0000 7fdb63134700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c19df10 con 0x7fdb5c0ff1e0 2023-11-16T16:34:21.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.496+0000 7fdb5bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb5c1003e0 0x7fdb5c19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:34:21.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.496+0000 7fdb5bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb5c1003e0 0x7fdb5c19adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40748/0 (socket says 172.21.15.1:40748) 2023-11-16T16:34:21.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.496+0000 7fdb5bfff700 1 -- 172.21.15.1:0/3268126385 learned_addr learned my addr 172.21.15.1:0/3268126385 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:34:21.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.496+0000 7fdb60ed0700 1 --2- 172.21.15.1:0/3268126385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb5c0ff1e0 0x7fdb5c069170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:34:21.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.496+0000 7fdb616d1700 1 --2- 172.21.15.1:0/3268126385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb5c100d70 0x7fdb5c19d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:34:21.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.496+0000 7fdb5bfff700 1 -- 172.21.15.1:0/3268126385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb5c0ff1e0 msgr2=0x7fdb5c069170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:34:21.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.496+0000 7fdb5bfff700 1 --2- 172.21.15.1:0/3268126385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb5c0ff1e0 0x7fdb5c069170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:34:21.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.496+0000 7fdb5bfff700 1 -- 172.21.15.1:0/3268126385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb5c100d70 msgr2=0x7fdb5c19d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:34:21.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.496+0000 7fdb5bfff700 1 --2- 172.21.15.1:0/3268126385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb5c100d70 0x7fdb5c19d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:34:21.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.496+0000 7fdb5bfff700 1 -- 172.21.15.1:0/3268126385 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb500095f0 con 0x7fdb5c1003e0 2023-11-16T16:34:21.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.496+0000 7fdb616d1700 1 --2- 172.21.15.1:0/3268126385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb5c100d70 0x7fdb5c19d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:34:21.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.497+0000 7fdb5bfff700 1 --2- 172.21.15.1:0/3268126385 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb5c1003e0 0x7fdb5c19adb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800d950 tx=0x7fdb4800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:34:21.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.497+0000 7fdb59ffb700 1 -- 172.21.15.1:0/3268126385 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb48009940 con 0x7fdb5c1003e0 2023-11-16T16:34:21.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.497+0000 7fdb59ffb700 1 -- 172.21.15.1:0/3268126385 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb48010460 con 0x7fdb5c1003e0 2023-11-16T16:34:21.502 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.497+0000 7fdb59ffb700 1 -- 172.21.15.1:0/3268126385 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4800f620 con 0x7fdb5c1003e0 2023-11-16T16:34:21.502 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.497+0000 7fdb63134700 1 -- 172.21.15.1:0/3268126385 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb5c10ab30 con 0x7fdb5c1003e0 2023-11-16T16:34:21.502 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.497+0000 7fdb63134700 1 -- 172.21.15.1:0/3268126385 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb5c10a9a0 con 0x7fdb5c1003e0 2023-11-16T16:34:21.502 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.498+0000 7fdb63134700 1 -- 172.21.15.1:0/3268126385 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb5c19e5f0 con 0x7fdb5c1003e0 2023-11-16T16:34:21.503 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.499+0000 7fdb59ffb700 1 -- 172.21.15.1:0/3268126385 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fdb480108d0 con 0x7fdb5c1003e0 2023-11-16T16:34:21.503 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.499+0000 7fdb59ffb700 1 --2- 172.21.15.1:0/3268126385 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fdb4c0a11d0 0x7fdb4c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:34:21.503 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.500+0000 7fdb60ed0700 1 --2- 172.21.15.1:0/3268126385 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fdb4c0a11d0 0x7fdb4c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:34:21.503 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.500+0000 7fdb59ffb700 1 -- 172.21.15.1:0/3268126385 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4562+0+0 (secure 0 0 0) 0x7fdb480cb910 con 0x7fdb5c1003e0 2023-11-16T16:34:21.503 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.500+0000 7fdb60ed0700 1 --2- 172.21.15.1:0/3268126385 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fdb4c0a11d0 0x7fdb4c0a3680 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdb5000b5c0 tx=0x7fdb50005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:34:21.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.504+0000 7fdb59ffb700 1 -- 172.21.15.1:0/3268126385 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fdb48095e60 con 0x7fdb5c1003e0 2023-11-16T16:34:21.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:21 smithi001 ceph-mon[98800]: osdmap e40: 6 total, 6 up, 6 in 2023-11-16T16:34:21.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:21 smithi001 ceph-mon[98800]: pgmap v130: 1 pgs: 1 remapped+peering; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:34:21.710 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:21.709+0000 7fdb63134700 1 -- 172.21.15.1:0/3268126385 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb5c060f30 con 0x7fdb4c0a11d0 2023-11-16T16:34:21.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:21 smithi026 ceph-mon[100112]: osdmap e40: 6 total, 6 up, 6 in 2023-11-16T16:34:21.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:21 smithi026 ceph-mon[100112]: pgmap v130: 1 pgs: 1 remapped+peering; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:34:21.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:21 smithi053 ceph-mon[100638]: osdmap e40: 6 total, 6 up, 6 in 2023-11-16T16:34:21.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:21 smithi053 ceph-mon[100638]: pgmap v130: 1 pgs: 1 remapped+peering; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:34:22.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:22 smithi026 ceph-mon[100112]: osdmap e41: 6 total, 6 up, 6 in 2023-11-16T16:34:22.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:22 smithi026 ceph-mon[100112]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:34:22.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:22 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:34:22.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:22 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:34:22.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:22 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:34:22.819 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:22 smithi001 ceph-mon[98800]: osdmap e41: 6 total, 6 up, 6 in 2023-11-16T16:34:22.819 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:22 smithi001 ceph-mon[98800]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:34:22.820 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:22 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:34:22.820 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:22 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:34:22.820 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:22 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:34:22.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:22 smithi053 ceph-mon[100638]: osdmap e41: 6 total, 6 up, 6 in 2023-11-16T16:34:22.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:22 smithi053 ceph-mon[100638]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:34:22.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:22 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:34:22.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:22 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:34:22.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:22 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:34:24.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:24 smithi001 ceph-mon[98800]: osdmap e42: 6 total, 6 up, 6 in 2023-11-16T16:34:24.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:24 smithi001 ceph-mon[98800]: pgmap v133: 1 pgs: 1 remapped+peering; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:34:24.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:24 smithi026 ceph-mon[100112]: osdmap e42: 6 total, 6 up, 6 in 2023-11-16T16:34:24.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:24 smithi026 ceph-mon[100112]: pgmap v133: 1 pgs: 1 remapped+peering; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:34:24.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:24 smithi053 ceph-mon[100638]: osdmap e42: 6 total, 6 up, 6 in 2023-11-16T16:34:24.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:24 smithi053 ceph-mon[100638]: pgmap v133: 1 pgs: 1 remapped+peering; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:34:25.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:25 smithi001 ceph-mon[98800]: osdmap e43: 6 total, 6 up, 6 in 2023-11-16T16:34:25.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:25 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/4263159579' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "180f506e-afb8-4043-b0b4-2dc86f6d4bc3"}]: dispatch 2023-11-16T16:34:25.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:25 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/4263159579' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "180f506e-afb8-4043-b0b4-2dc86f6d4bc3"}]': finished 2023-11-16T16:34:25.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:25 smithi001 ceph-mon[98800]: osdmap e44: 7 total, 6 up, 7 in 2023-11-16T16:34:25.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:25 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-16T16:34:25.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:25 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/454745243' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:34:25.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:25 smithi026 ceph-mon[100112]: osdmap e43: 6 total, 6 up, 6 in 2023-11-16T16:34:25.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:25 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/4263159579' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "180f506e-afb8-4043-b0b4-2dc86f6d4bc3"}]: dispatch 2023-11-16T16:34:25.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:25 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/4263159579' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "180f506e-afb8-4043-b0b4-2dc86f6d4bc3"}]': finished 2023-11-16T16:34:25.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:25 smithi026 ceph-mon[100112]: osdmap e44: 7 total, 6 up, 7 in 2023-11-16T16:34:25.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:25 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-16T16:34:25.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:25 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/454745243' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:34:25.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:25 smithi053 ceph-mon[100638]: osdmap e43: 6 total, 6 up, 6 in 2023-11-16T16:34:25.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:25 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/4263159579' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "180f506e-afb8-4043-b0b4-2dc86f6d4bc3"}]: dispatch 2023-11-16T16:34:25.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:25 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/4263159579' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "180f506e-afb8-4043-b0b4-2dc86f6d4bc3"}]': finished 2023-11-16T16:34:25.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:25 smithi053 ceph-mon[100638]: osdmap e44: 7 total, 6 up, 7 in 2023-11-16T16:34:25.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:25 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-16T16:34:25.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:25 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/454745243' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:34:26.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:26 smithi001 ceph-mon[98800]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:34:26.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:26 smithi026 ceph-mon[100112]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:34:26.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:26 smithi053 ceph-mon[100638]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:34:27.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:27 smithi026 ceph-mon[100112]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2023-11-16T16:34:27.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:27 smithi053 ceph-mon[100638]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2023-11-16T16:34:27.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:27 smithi001 ceph-mon[98800]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2023-11-16T16:34:29.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:29 smithi026 ceph-mon[100112]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2023-11-16T16:34:29.812 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:29 smithi001 ceph-mon[98800]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2023-11-16T16:34:29.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:29 smithi053 ceph-mon[100638]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2023-11-16T16:34:31.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:30 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-11-16T16:34:31.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:30 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:34:31.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:30 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-11-16T16:34:31.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:30 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:34:31.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:30 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-11-16T16:34:31.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:30 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:34:32.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:32 smithi001 ceph-mon[98800]: Deploying daemon osd.6 on smithi001 2023-11-16T16:34:32.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:32 smithi001 ceph-mon[98800]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-16T16:34:32.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:32 smithi026 ceph-mon[100112]: Deploying daemon osd.6 on smithi001 2023-11-16T16:34:32.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:32 smithi026 ceph-mon[100112]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-16T16:34:32.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:32 smithi053 ceph-mon[100638]: Deploying daemon osd.6 on smithi001 2023-11-16T16:34:32.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:32 smithi053 ceph-mon[100638]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-16T16:34:33.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:33 smithi001 ceph-mon[98800]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2023-11-16T16:34:33.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:33 smithi026 ceph-mon[100112]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2023-11-16T16:34:33.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:33 smithi053 ceph-mon[100638]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2023-11-16T16:34:35.606 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:35 smithi001 ceph-mon[98800]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-11-16T16:34:35.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:35 smithi026 ceph-mon[100112]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-11-16T16:34:35.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:35 smithi053 ceph-mon[100638]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-11-16T16:34:36.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:36 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:36.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:36 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:36.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:36 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:34:37.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:36 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:37.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:36 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:37.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:36 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:34:37.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:36 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:37.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:36 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:37.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:36 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:34:37.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:37 smithi001 ceph-mon[98800]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-11-16T16:34:38.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:37 smithi026 ceph-mon[100112]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-11-16T16:34:38.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:37 smithi053 ceph-mon[100638]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-11-16T16:34:38.668 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:38.666+0000 7fdb59ffb700 1 -- 172.21.15.1:0/3268126385 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fdb5c060f30 con 0x7fdb4c0a11d0 2023-11-16T16:34:38.668 INFO:teuthology.orchestra.run.smithi001.stdout:Created osd(s) 6 on host 'smithi001' 2023-11-16T16:34:38.671 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:38.668+0000 7fdb63134700 1 -- 172.21.15.1:0/3268126385 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fdb4c0a11d0 msgr2=0x7fdb4c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:34:38.671 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:38.668+0000 7fdb63134700 1 --2- 172.21.15.1:0/3268126385 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fdb4c0a11d0 0x7fdb4c0a3680 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdb5000b5c0 tx=0x7fdb50005fb0 comp rx=0 tx=0).stop 2023-11-16T16:34:38.671 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:38.668+0000 7fdb63134700 1 -- 172.21.15.1:0/3268126385 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb5c1003e0 msgr2=0x7fdb5c19adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:34:38.671 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:38.668+0000 7fdb63134700 1 --2- 172.21.15.1:0/3268126385 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb5c1003e0 0x7fdb5c19adb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800d950 tx=0x7fdb4800dc60 comp rx=0 tx=0).stop 2023-11-16T16:34:38.671 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:38.668+0000 7fdb63134700 1 -- 172.21.15.1:0/3268126385 shutdown_connections 2023-11-16T16:34:38.671 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:38.668+0000 7fdb63134700 1 --2- 172.21.15.1:0/3268126385 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fdb4c0a11d0 0x7fdb4c0a3680 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:34:38.672 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:38.668+0000 7fdb63134700 1 --2- 172.21.15.1:0/3268126385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb5c100d70 0x7fdb5c19d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:34:38.672 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:38.668+0000 7fdb63134700 1 --2- 172.21.15.1:0/3268126385 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdb5c0ff1e0 0x7fdb5c069170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:34:38.672 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:38.668+0000 7fdb63134700 1 --2- 172.21.15.1:0/3268126385 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb5c1003e0 0x7fdb5c19adb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:34:38.672 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:38.668+0000 7fdb63134700 1 -- 172.21.15.1:0/3268126385 >> 172.21.15.1:0/3268126385 conn(0x7fdb5c0fa7f0 msgr2=0x7fdb5c0fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:34:38.672 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:38.668+0000 7fdb63134700 1 -- 172.21.15.1:0/3268126385 shutdown_connections 2023-11-16T16:34:38.672 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:34:38.668+0000 7fdb63134700 1 -- 172.21.15.1:0/3268126385 wait complete. 2023-11-16T16:34:39.132 DEBUG:teuthology.orchestra.run.smithi001:osd.6> sudo journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@osd.6.service 2023-11-16T16:34:39.135 INFO:tasks.cephadm:Deploying osd.7 on smithi026 with /dev/vg_nvme/lv_2... 2023-11-16T16:34:39.135 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-11-16T16:34:39.167 INFO:journalctl@ceph.osd.6.smithi001.stdout:-- Logs begin at Thu 2023-11-16 16:14:27 UTC. -- 2023-11-16T16:34:39.723 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:34:39 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:34:39.568+0000 7fc3ed12a640 -1 osd.6 0 log_to_monitors true 2023-11-16T16:34:39.723 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:39 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:39.723 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:39 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:39.723 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:39 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:34:39.723 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:39 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:34:39.723 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:39 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:34:39.723 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:39 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:39.723 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:39 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:34:39.724 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:39 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:34:39.724 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:39 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:39.724 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:39 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:39.724 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:39 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:34:39.724 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:39 smithi001 ceph-mon[98800]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:34:39.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:39 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:39.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:39 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:39.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:39 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:34:39.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:39 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:34:39.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:39 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:34:39.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:39 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:39.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:39 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:34:39.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:39 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:34:39.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:39 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:39.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:39 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:39.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:39 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:34:39.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:39 smithi026 ceph-mon[100112]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:34:39.801 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.b/config 2023-11-16T16:34:39.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:39 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:39.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:39 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:39.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:39 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:34:39.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:39 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:34:39.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:39 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:34:39.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:39 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:39.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:39 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:34:39.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:39 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:34:39.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:39 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:39.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:39 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:39.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:39 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:34:39.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:39 smithi053 ceph-mon[100638]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:34:40.539 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:40 smithi001 ceph-mon[98800]: from='osd.6 [v2:172.21.15.1:6818/3364032646,v1:172.21.15.1:6819/3364032646]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-11-16T16:34:40.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:40 smithi026 ceph-mon[100112]: from='osd.6 [v2:172.21.15.1:6818/3364032646,v1:172.21.15.1:6819/3364032646]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-11-16T16:34:40.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:40 smithi053 ceph-mon[100638]: from='osd.6 [v2:172.21.15.1:6818/3364032646,v1:172.21.15.1:6819/3364032646]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-11-16T16:34:41.298 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-11-16T16:34:41.298 INFO:teuthology.orchestra.run.smithi026.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-11-16T16:34:41.299 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10+0 records in 2023-11-16T16:34:41.299 INFO:teuthology.orchestra.run.smithi026.stderr:10+0 records out 2023-11-16T16:34:41.299 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0136844 s, 766 MB/s 2023-11-16T16:34:41.299 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping successful for: 2023-11-16T16:34:41.623 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:41 smithi026 ceph-mon[100112]: from='osd.6 [v2:172.21.15.1:6818/3364032646,v1:172.21.15.1:6819/3364032646]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-11-16T16:34:41.623 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:41 smithi026 ceph-mon[100112]: osdmap e45: 7 total, 6 up, 7 in 2023-11-16T16:34:41.624 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:41 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-16T16:34:41.624 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:41 smithi026 ceph-mon[100112]: from='osd.6 [v2:172.21.15.1:6818/3364032646,v1:172.21.15.1:6819/3364032646]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2023-11-16T16:34:41.624 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:41 smithi026 ceph-mon[100112]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:34:41.647 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph orch daemon add osd smithi026:vg_nvme/lv_2 2023-11-16T16:34:41.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:41 smithi053 ceph-mon[100638]: from='osd.6 [v2:172.21.15.1:6818/3364032646,v1:172.21.15.1:6819/3364032646]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-11-16T16:34:41.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:41 smithi053 ceph-mon[100638]: osdmap e45: 7 total, 6 up, 7 in 2023-11-16T16:34:41.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:41 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-16T16:34:41.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:41 smithi053 ceph-mon[100638]: from='osd.6 [v2:172.21.15.1:6818/3364032646,v1:172.21.15.1:6819/3364032646]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2023-11-16T16:34:41.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:41 smithi053 ceph-mon[100638]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:34:41.932 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:34:41 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:34:41.483+0000 7fc3e27a1700 -1 osd.6 0 waiting for initial osdmap 2023-11-16T16:34:41.932 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:34:41 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:34:41.494+0000 7fc3dc592700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-16T16:34:41.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:41 smithi001 ceph-mon[98800]: from='osd.6 [v2:172.21.15.1:6818/3364032646,v1:172.21.15.1:6819/3364032646]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-11-16T16:34:41.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:41 smithi001 ceph-mon[98800]: osdmap e45: 7 total, 6 up, 7 in 2023-11-16T16:34:41.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:41 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-16T16:34:41.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:41 smithi001 ceph-mon[98800]: from='osd.6 [v2:172.21.15.1:6818/3364032646,v1:172.21.15.1:6819/3364032646]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2023-11-16T16:34:41.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:41 smithi001 ceph-mon[98800]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:34:42.298 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.b/config 2023-11-16T16:34:42.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:42 smithi026 ceph-mon[100112]: from='osd.6 [v2:172.21.15.1:6818/3364032646,v1:172.21.15.1:6819/3364032646]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2023-11-16T16:34:42.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:42 smithi026 ceph-mon[100112]: osdmap e46: 7 total, 6 up, 7 in 2023-11-16T16:34:42.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:42 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-16T16:34:42.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:42 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-16T16:34:42.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:42 smithi026 ceph-mon[100112]: Detected new or changed devices on smithi001 2023-11-16T16:34:42.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:42 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:42.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:42 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:42.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:42 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:34:42.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:42 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:34:42.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:42 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:34:42.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:42 smithi026 ceph-mon[100112]: Adjusting osd_memory_target on smithi001 to 5724M 2023-11-16T16:34:42.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:42 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:42.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:42 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:34:42.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:42 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:34:42.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:42 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:34:42.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:42 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:42.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:42 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:34:42.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:42 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:34:42.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:42 smithi053 ceph-mon[100638]: from='osd.6 [v2:172.21.15.1:6818/3364032646,v1:172.21.15.1:6819/3364032646]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2023-11-16T16:34:42.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:42 smithi053 ceph-mon[100638]: osdmap e46: 7 total, 6 up, 7 in 2023-11-16T16:34:42.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:42 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-16T16:34:42.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:42 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-16T16:34:42.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:42 smithi053 ceph-mon[100638]: Detected new or changed devices on smithi001 2023-11-16T16:34:42.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:42 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:42.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:42 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:42.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:42 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:34:42.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:42 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:34:42.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:42 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:34:42.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:42 smithi053 ceph-mon[100638]: Adjusting osd_memory_target on smithi001 to 5724M 2023-11-16T16:34:42.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:42 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:42.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:42 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:34:42.833 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:42 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:34:42.833 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:42 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:34:42.833 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:42 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:42.833 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:42 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:34:42.833 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:42 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:34:42.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:42 smithi001 ceph-mon[98800]: from='osd.6 [v2:172.21.15.1:6818/3364032646,v1:172.21.15.1:6819/3364032646]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2023-11-16T16:34:42.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:42 smithi001 ceph-mon[98800]: osdmap e46: 7 total, 6 up, 7 in 2023-11-16T16:34:42.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:42 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-16T16:34:42.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:42 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-16T16:34:42.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:42 smithi001 ceph-mon[98800]: Detected new or changed devices on smithi001 2023-11-16T16:34:42.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:42 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:42.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:42 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:42.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:42 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:34:42.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:42 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:34:42.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:42 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:34:42.934 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:42 smithi001 ceph-mon[98800]: Adjusting osd_memory_target on smithi001 to 5724M 2023-11-16T16:34:42.934 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:42 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:42.934 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:42 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:34:42.934 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:42 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:34:42.934 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:42 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:34:42.934 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:42 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:42.934 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:42 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:34:42.934 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:42 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:34:43.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:34:43.494+0000 7f2542aba700 1 -- 172.21.15.26:0/4120757971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f253c0ff1e0 msgr2=0x7f253c0ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:34:43.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:34:43.494+0000 7f2542aba700 1 --2- 172.21.15.26:0/4120757971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f253c0ff1e0 0x7f253c0ff5f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2530009990 tx=0x7f2530009ca0 comp rx=0 tx=0).stop 2023-11-16T16:34:43.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:34:43.494+0000 7f2542aba700 1 -- 172.21.15.26:0/4120757971 shutdown_connections 2023-11-16T16:34:43.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:34:43.494+0000 7f2542aba700 1 --2- 172.21.15.26:0/4120757971 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f253c100d70 0x7f253c107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:34:43.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:34:43.494+0000 7f2542aba700 1 --2- 172.21.15.26:0/4120757971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f253c0ff1e0 0x7f253c0ff5f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:34:43.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:34:43.494+0000 7f2542aba700 1 --2- 172.21.15.26:0/4120757971 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f253c1003e0 0x7f253c100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:34:43.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:34:43.494+0000 7f2542aba700 1 -- 172.21.15.26:0/4120757971 >> 172.21.15.26:0/4120757971 conn(0x7f253c0fa7f0 msgr2=0x7f253c0fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:34:43.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:34:43.494+0000 7f2542aba700 1 -- 172.21.15.26:0/4120757971 shutdown_connections 2023-11-16T16:34:43.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:34:43.494+0000 7f2542aba700 1 -- 172.21.15.26:0/4120757971 wait complete. 2023-11-16T16:34:43.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:34:43.495+0000 7f2542aba700 1 Processor -- start 2023-11-16T16:34:43.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:34:43.495+0000 7f2542aba700 1 -- start start 2023-11-16T16:34:43.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:34:43.496+0000 7f2542aba700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f253c0ff1e0 0x7f253c194f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:34:43.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:34:43.496+0000 7f2542aba700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f253c1003e0 0x7f253c1954a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:34:43.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:34:43.496+0000 7f2542aba700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f253c100d70 0x7f253c19c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:34:43.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:34:43.496+0000 7f2542aba700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c19caf0 con 0x7f253c1003e0 2023-11-16T16:34:43.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:34:43.496+0000 7f2542aba700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c19cc30 con 0x7f253c0ff1e0 2023-11-16T16:34:43.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:34:43.496+0000 7f2542aba700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c19cdc0 con 0x7f253c100d70 2023-11-16T16:34:43.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:34:43.496+0000 7f2540856700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f253c0ff1e0 0x7f253c194f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:34:43.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:34:43.496+0000 7f2540856700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f253c0ff1e0 0x7f253c194f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55170/0 (socket says 172.21.15.26:55170) 2023-11-16T16:34:43.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:34:43.496+0000 7f2540856700 1 -- 172.21.15.26:0/2992316864 learned_addr learned my addr 172.21.15.26:0/2992316864 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-11-16T16:34:43.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:34:43.496+0000 7f2540856700 1 -- 172.21.15.26:0/2992316864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f253c100d70 msgr2=0x7f253c19c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:34:43.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:34:43.496+0000 7f2540856700 1 --2- 172.21.15.26:0/2992316864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f253c100d70 0x7f253c19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:34:43.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:34:43.496+0000 7f2540856700 1 -- 172.21.15.26:0/2992316864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f253c1003e0 msgr2=0x7f253c1954a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:34:43.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:34:43.496+0000 7f2540856700 1 --2- 172.21.15.26:0/2992316864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f253c1003e0 0x7f253c1954a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:34:43.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:34:43.496+0000 7f2540856700 1 -- 172.21.15.26:0/2992316864 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25300095f0 con 0x7f253c0ff1e0 2023-11-16T16:34:43.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:34:43.496+0000 7f2540856700 1 --2- 172.21.15.26:0/2992316864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f253c0ff1e0 0x7f253c194f60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f253000f6c0 tx=0x7f253000f6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:34:43.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:34:43.496+0000 7f2539ffb700 1 -- 172.21.15.26:0/2992316864 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253001d070 con 0x7f253c0ff1e0 2023-11-16T16:34:43.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:34:43.497+0000 7f2539ffb700 1 -- 172.21.15.26:0/2992316864 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2530004cf0 con 0x7f253c0ff1e0 2023-11-16T16:34:43.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:34:43.497+0000 7f2542aba700 1 -- 172.21.15.26:0/2992316864 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f253c10ab90 con 0x7f253c0ff1e0 2023-11-16T16:34:43.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:34:43.497+0000 7f2539ffb700 1 -- 172.21.15.26:0/2992316864 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2530005680 con 0x7f253c0ff1e0 2023-11-16T16:34:43.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:34:43.497+0000 7f2542aba700 1 -- 172.21.15.26:0/2992316864 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f253c10aa00 con 0x7f253c0ff1e0 2023-11-16T16:34:43.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:34:43.498+0000 7f2542aba700 1 -- 172.21.15.26:0/2992316864 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f253c1067d0 con 0x7f253c0ff1e0 2023-11-16T16:34:43.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:34:43.499+0000 7f2539ffb700 1 -- 172.21.15.26:0/2992316864 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f253000fc90 con 0x7f253c0ff1e0 2023-11-16T16:34:43.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:34:43.499+0000 7f2539ffb700 1 --2- 172.21.15.26:0/2992316864 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f252c0a1150 0x7f252c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:34:43.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:34:43.499+0000 7f2539ffb700 1 -- 172.21.15.26:0/2992316864 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f25300cd020 con 0x7f253c0ff1e0 2023-11-16T16:34:43.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:34:43.500+0000 7f253bfff700 1 --2- 172.21.15.26:0/2992316864 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f252c0a1150 0x7f252c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:34:43.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:34:43.500+0000 7f253bfff700 1 --2- 172.21.15.26:0/2992316864 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f252c0a1150 0x7f252c0a3600 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2528005fd0 tx=0x7f2528005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:34:43.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:34:43.504+0000 7f2539ffb700 1 -- 172.21.15.26:0/2992316864 <== mon.2 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f253005b730 con 0x7f253c0ff1e0 2023-11-16T16:34:43.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:34:43.708+0000 7f2542aba700 1 -- 172.21.15.26:0/2992316864 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f253c061c00 con 0x7f252c0a1150 2023-11-16T16:34:43.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:43 smithi026 ceph-mon[100112]: purged_snaps scrub starts 2023-11-16T16:34:43.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:43 smithi026 ceph-mon[100112]: purged_snaps scrub ok 2023-11-16T16:34:43.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:43 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-16T16:34:43.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:43 smithi026 ceph-mon[100112]: osd.6 [v2:172.21.15.1:6818/3364032646,v1:172.21.15.1:6819/3364032646] boot 2023-11-16T16:34:43.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:43 smithi026 ceph-mon[100112]: osdmap e47: 7 total, 7 up, 7 in 2023-11-16T16:34:43.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:43 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-16T16:34:43.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:43 smithi026 ceph-mon[100112]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:34:43.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:43 smithi053 ceph-mon[100638]: purged_snaps scrub starts 2023-11-16T16:34:43.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:43 smithi053 ceph-mon[100638]: purged_snaps scrub ok 2023-11-16T16:34:43.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:43 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-16T16:34:43.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:43 smithi053 ceph-mon[100638]: osd.6 [v2:172.21.15.1:6818/3364032646,v1:172.21.15.1:6819/3364032646] boot 2023-11-16T16:34:43.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:43 smithi053 ceph-mon[100638]: osdmap e47: 7 total, 7 up, 7 in 2023-11-16T16:34:43.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:43 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-16T16:34:43.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:43 smithi053 ceph-mon[100638]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:34:43.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:43 smithi001 ceph-mon[98800]: purged_snaps scrub starts 2023-11-16T16:34:43.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:43 smithi001 ceph-mon[98800]: purged_snaps scrub ok 2023-11-16T16:34:43.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:43 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-16T16:34:43.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:43 smithi001 ceph-mon[98800]: osd.6 [v2:172.21.15.1:6818/3364032646,v1:172.21.15.1:6819/3364032646] boot 2023-11-16T16:34:43.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:43 smithi001 ceph-mon[98800]: osdmap e47: 7 total, 7 up, 7 in 2023-11-16T16:34:43.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:43 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-16T16:34:43.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:43 smithi001 ceph-mon[98800]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-11-16T16:34:44.801 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:44 smithi026 ceph-mon[100112]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:34:44.801 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:44 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:34:44.801 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:44 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:34:44.801 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:44 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:34:44.801 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:44 smithi026 ceph-mon[100112]: osdmap e48: 7 total, 7 up, 7 in 2023-11-16T16:34:44.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:44 smithi053 ceph-mon[100638]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:34:44.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:44 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:34:44.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:44 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:34:44.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:44 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:34:44.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:44 smithi053 ceph-mon[100638]: osdmap e48: 7 total, 7 up, 7 in 2023-11-16T16:34:44.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:44 smithi001 ceph-mon[98800]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:34:44.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:44 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:34:44.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:44 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:34:44.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:44 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:34:44.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:44 smithi001 ceph-mon[98800]: osdmap e48: 7 total, 7 up, 7 in 2023-11-16T16:34:45.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:45 smithi053 ceph-mon[100638]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:34:45.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:45 smithi001 ceph-mon[98800]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:34:46.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:45 smithi026 ceph-mon[100112]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:34:47.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:47 smithi001 ceph-mon[98800]: from='client.? 172.21.15.26:0/89940300' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d837bac-194f-498d-b7fb-b3147e992329"}]: dispatch 2023-11-16T16:34:47.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:47 smithi001 ceph-mon[98800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d837bac-194f-498d-b7fb-b3147e992329"}]: dispatch 2023-11-16T16:34:47.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:47 smithi001 ceph-mon[98800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d837bac-194f-498d-b7fb-b3147e992329"}]': finished 2023-11-16T16:34:47.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:47 smithi001 ceph-mon[98800]: osdmap e49: 8 total, 7 up, 8 in 2023-11-16T16:34:47.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:47 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-16T16:34:47.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:47 smithi001 ceph-mon[98800]: from='client.? 172.21.15.26:0/4202875637' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:34:47.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:47 smithi026 ceph-mon[100112]: from='client.? 172.21.15.26:0/89940300' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d837bac-194f-498d-b7fb-b3147e992329"}]: dispatch 2023-11-16T16:34:47.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:47 smithi026 ceph-mon[100112]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d837bac-194f-498d-b7fb-b3147e992329"}]: dispatch 2023-11-16T16:34:47.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:47 smithi026 ceph-mon[100112]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d837bac-194f-498d-b7fb-b3147e992329"}]': finished 2023-11-16T16:34:47.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:47 smithi026 ceph-mon[100112]: osdmap e49: 8 total, 7 up, 8 in 2023-11-16T16:34:47.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:47 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-16T16:34:47.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:47 smithi026 ceph-mon[100112]: from='client.? 172.21.15.26:0/4202875637' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:34:47.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:47 smithi053 ceph-mon[100638]: from='client.? 172.21.15.26:0/89940300' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d837bac-194f-498d-b7fb-b3147e992329"}]: dispatch 2023-11-16T16:34:47.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:47 smithi053 ceph-mon[100638]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d837bac-194f-498d-b7fb-b3147e992329"}]: dispatch 2023-11-16T16:34:47.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:47 smithi053 ceph-mon[100638]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d837bac-194f-498d-b7fb-b3147e992329"}]': finished 2023-11-16T16:34:47.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:47 smithi053 ceph-mon[100638]: osdmap e49: 8 total, 7 up, 8 in 2023-11-16T16:34:47.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:47 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-16T16:34:47.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:47 smithi053 ceph-mon[100638]: from='client.? 172.21.15.26:0/4202875637' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:34:48.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:48 smithi001 ceph-mon[98800]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:34:48.507 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:48 smithi026 ceph-mon[100112]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:34:48.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:48 smithi053 ceph-mon[100638]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:34:49.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:49 smithi053 ceph-mon[100638]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:34:49.928 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:49 smithi026 ceph-mon[100112]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:34:49.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:49 smithi001 ceph-mon[98800]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:34:52.408 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:52 smithi026 ceph-mon[100112]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:34:52.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:52 smithi001 ceph-mon[98800]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:34:52.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:52 smithi053 ceph-mon[100638]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:34:53.422 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:53 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-11-16T16:34:53.422 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:53 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:34:53.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:53 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-11-16T16:34:53.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:53 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:34:53.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:53 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-11-16T16:34:53.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:53 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:34:54.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:54 smithi001 ceph-mon[98800]: Deploying daemon osd.7 on smithi026 2023-11-16T16:34:54.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:54 smithi001 ceph-mon[98800]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:34:54.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:54 smithi026 ceph-mon[100112]: Deploying daemon osd.7 on smithi026 2023-11-16T16:34:54.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:54 smithi026 ceph-mon[100112]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:34:54.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:54 smithi053 ceph-mon[100638]: Deploying daemon osd.7 on smithi026 2023-11-16T16:34:54.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:54 smithi053 ceph-mon[100638]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:34:55.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:55 smithi026 ceph-mon[100112]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:34:55.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:55 smithi053 ceph-mon[100638]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:34:55.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:55 smithi001 ceph-mon[98800]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:34:58.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:58 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:58.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:58 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:58.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:58 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:34:58.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:58 smithi001 ceph-mon[98800]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:34:58.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:58 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:58.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:58 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:58.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:58 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:34:58.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:58 smithi026 ceph-mon[100112]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:34:58.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:58 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:58.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:58 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:34:58.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:58 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:34:58.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:58 smithi053 ceph-mon[100638]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:34:59.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:34:59 smithi053 ceph-mon[100638]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:34:59.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:34:59 smithi001 ceph-mon[98800]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:34:59.937 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:34:59 smithi026 ceph-mon[100112]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:35:00.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:00.380+0000 7f2539ffb700 1 -- 172.21.15.26:0/2992316864 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f253c061c00 con 0x7f252c0a1150 2023-11-16T16:35:00.383 INFO:teuthology.orchestra.run.smithi026.stdout:Created osd(s) 7 on host 'smithi026' 2023-11-16T16:35:00.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:00.382+0000 7f2542aba700 1 -- 172.21.15.26:0/2992316864 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f252c0a1150 msgr2=0x7f252c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:35:00.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:00.382+0000 7f2542aba700 1 --2- 172.21.15.26:0/2992316864 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f252c0a1150 0x7f252c0a3600 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2528005fd0 tx=0x7f2528005d00 comp rx=0 tx=0).stop 2023-11-16T16:35:00.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:00.382+0000 7f2542aba700 1 -- 172.21.15.26:0/2992316864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f253c0ff1e0 msgr2=0x7f253c194f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:35:00.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:00.382+0000 7f2542aba700 1 --2- 172.21.15.26:0/2992316864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f253c0ff1e0 0x7f253c194f60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f253000f6c0 tx=0x7f253000f6f0 comp rx=0 tx=0).stop 2023-11-16T16:35:00.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:00.382+0000 7f2542aba700 1 -- 172.21.15.26:0/2992316864 shutdown_connections 2023-11-16T16:35:00.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:00.382+0000 7f2542aba700 1 --2- 172.21.15.26:0/2992316864 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f252c0a1150 0x7f252c0a3600 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:35:00.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:00.382+0000 7f2542aba700 1 --2- 172.21.15.26:0/2992316864 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f253c100d70 0x7f253c19c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:35:00.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:00.382+0000 7f2542aba700 1 --2- 172.21.15.26:0/2992316864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f253c0ff1e0 0x7f253c194f60 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:35:00.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:00.382+0000 7f2542aba700 1 --2- 172.21.15.26:0/2992316864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f253c1003e0 0x7f253c1954a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:35:00.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:00.382+0000 7f2542aba700 1 -- 172.21.15.26:0/2992316864 >> 172.21.15.26:0/2992316864 conn(0x7f253c0fa7f0 msgr2=0x7f253c103910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:35:00.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:00.382+0000 7f2542aba700 1 -- 172.21.15.26:0/2992316864 shutdown_connections 2023-11-16T16:35:00.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:00.382+0000 7f2542aba700 1 -- 172.21.15.26:0/2992316864 wait complete. 2023-11-16T16:35:00.793 DEBUG:teuthology.orchestra.run.smithi026:osd.7> sudo journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@osd.7.service 2023-11-16T16:35:00.795 INFO:tasks.cephadm:Deploying osd.8 on smithi053 with /dev/vg_nvme/lv_2... 2023-11-16T16:35:00.796 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-11-16T16:35:00.827 INFO:journalctl@ceph.osd.7.smithi026.stdout:-- Logs begin at Thu 2023-11-16 16:14:13 UTC. -- 2023-11-16T16:35:01.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:00 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:01.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:00 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:01.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:00 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:35:01.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:00 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:35:01.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:00 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:35:01.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:00 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:01.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:00 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:35:01.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:00 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:35:01.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:00 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:01.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:00 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:01.082 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:00 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:35:01.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:00 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:01.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:00 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:01.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:00 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:35:01.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:00 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:35:01.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:00 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:35:01.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:00 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:01.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:00 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:35:01.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:00 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:35:01.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:00 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:01.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:00 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:01.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:00 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:35:01.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:00 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:01.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:00 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:01.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:00 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:35:01.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:00 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:35:01.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:00 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:35:01.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:00 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:01.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:00 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:35:01.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:00 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:35:01.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:00 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:01.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:00 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:01.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:00 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:35:01.460 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-16T16:35:01.606 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:35:01 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:35:01.266+0000 7fb3324a1640 -1 osd.7 0 log_to_monitors true 2023-11-16T16:35:02.176 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:02 smithi053 ceph-mon[100638]: from='osd.7 [v2:172.21.15.26:6816/3766901391,v1:172.21.15.26:6817/3766901391]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-16T16:35:02.176 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:02 smithi053 ceph-mon[100638]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-16T16:35:02.176 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:02 smithi053 ceph-mon[100638]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:35:02.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:02 smithi001 ceph-mon[98800]: from='osd.7 [v2:172.21.15.26:6816/3766901391,v1:172.21.15.26:6817/3766901391]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-16T16:35:02.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:02 smithi001 ceph-mon[98800]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-16T16:35:02.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:02 smithi001 ceph-mon[98800]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:35:02.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:02 smithi026 ceph-mon[100112]: from='osd.7 [v2:172.21.15.26:6816/3766901391,v1:172.21.15.26:6817/3766901391]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-16T16:35:02.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:02 smithi026 ceph-mon[100112]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-16T16:35:02.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:02 smithi026 ceph-mon[100112]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:35:02.915 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-11-16T16:35:02.916 INFO:teuthology.orchestra.run.smithi053.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-11-16T16:35:02.916 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10+0 records in 2023-11-16T16:35:02.916 INFO:teuthology.orchestra.run.smithi053.stderr:10+0 records out 2023-11-16T16:35:02.916 INFO:teuthology.orchestra.run.smithi053.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00797906 s, 1.3 GB/s 2023-11-16T16:35:02.916 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping successful for: 2023-11-16T16:35:03.307 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:03 smithi053 ceph-mon[100638]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-11-16T16:35:03.307 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:03 smithi053 ceph-mon[100638]: osdmap e50: 8 total, 7 up, 8 in 2023-11-16T16:35:03.307 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:03 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-16T16:35:03.307 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:03 smithi053 ceph-mon[100638]: from='osd.7 [v2:172.21.15.26:6816/3766901391,v1:172.21.15.26:6817/3766901391]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-11-16T16:35:03.307 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:03 smithi053 ceph-mon[100638]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-11-16T16:35:03.307 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:03 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:03.307 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:03 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:03.308 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:03 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:35:03.308 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:03 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:35:03.308 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:03 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:35:03.308 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:03 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:03.308 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:03 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:35:03.308 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:03 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:35:03.308 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:03 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:03.309 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph orch daemon add osd smithi053:vg_nvme/lv_2 2023-11-16T16:35:03.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:03 smithi001 ceph-mon[98800]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-11-16T16:35:03.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:03 smithi001 ceph-mon[98800]: osdmap e50: 8 total, 7 up, 8 in 2023-11-16T16:35:03.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:03 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-16T16:35:03.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:03 smithi001 ceph-mon[98800]: from='osd.7 [v2:172.21.15.26:6816/3766901391,v1:172.21.15.26:6817/3766901391]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-11-16T16:35:03.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:03 smithi001 ceph-mon[98800]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-11-16T16:35:03.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:03 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:03.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:03 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:03.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:03 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:35:03.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:03 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:35:03.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:03 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:35:03.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:03 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:03.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:03 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:35:03.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:03 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:35:03.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:03 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:03.508 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:35:03 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:35:03.143+0000 7fb327b18700 -1 osd.7 0 waiting for initial osdmap 2023-11-16T16:35:03.508 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:35:03 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:35:03.154+0000 7fb320106700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-16T16:35:03.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:03 smithi026 ceph-mon[100112]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-11-16T16:35:03.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:03 smithi026 ceph-mon[100112]: osdmap e50: 8 total, 7 up, 8 in 2023-11-16T16:35:03.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:03 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-16T16:35:03.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:03 smithi026 ceph-mon[100112]: from='osd.7 [v2:172.21.15.26:6816/3766901391,v1:172.21.15.26:6817/3766901391]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-11-16T16:35:03.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:03 smithi026 ceph-mon[100112]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-11-16T16:35:03.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:03 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:03.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:03 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:03.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:03 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:35:03.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:03 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:35:03.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:03 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:35:03.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:03 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:03.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:03 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:35:03.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:03 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:35:03.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:03 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:03.961 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-16T16:35:04.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:04 smithi001 ceph-mon[98800]: Detected new or changed devices on smithi026 2023-11-16T16:35:04.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:04 smithi001 ceph-mon[98800]: Adjusting osd_memory_target on smithi026 to 5724M 2023-11-16T16:35:04.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:04 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:35:04.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:04 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:35:04.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:04 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:35:04.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:04 smithi001 ceph-mon[98800]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2023-11-16T16:35:04.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:04 smithi001 ceph-mon[98800]: osdmap e51: 8 total, 7 up, 8 in 2023-11-16T16:35:04.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:04 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-16T16:35:04.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:04 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-16T16:35:04.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:04 smithi001 ceph-mon[98800]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:35:04.462 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:04 smithi053 ceph-mon[100638]: Detected new or changed devices on smithi026 2023-11-16T16:35:04.462 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:04 smithi053 ceph-mon[100638]: Adjusting osd_memory_target on smithi026 to 5724M 2023-11-16T16:35:04.462 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:04 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:35:04.462 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:04 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:35:04.463 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:04 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:35:04.463 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:04 smithi053 ceph-mon[100638]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2023-11-16T16:35:04.463 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:04 smithi053 ceph-mon[100638]: osdmap e51: 8 total, 7 up, 8 in 2023-11-16T16:35:04.463 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:04 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-16T16:35:04.463 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:04 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-16T16:35:04.463 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:04 smithi053 ceph-mon[100638]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:35:04.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:04 smithi026 ceph-mon[100112]: Detected new or changed devices on smithi026 2023-11-16T16:35:04.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:04 smithi026 ceph-mon[100112]: Adjusting osd_memory_target on smithi026 to 5724M 2023-11-16T16:35:04.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:04 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:35:04.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:04 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:35:04.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:04 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:35:04.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:04 smithi026 ceph-mon[100112]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2023-11-16T16:35:04.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:04 smithi026 ceph-mon[100112]: osdmap e51: 8 total, 7 up, 8 in 2023-11-16T16:35:04.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:04 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-16T16:35:04.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:04 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-16T16:35:04.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:04 smithi026 ceph-mon[100112]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-11-16T16:35:05.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.178+0000 7f17bf6ac700 1 -- 172.21.15.53:0/1059182915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17b80ff1e0 msgr2=0x7f17b80ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:35:05.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.178+0000 7f17bf6ac700 1 --2- 172.21.15.53:0/1059182915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17b80ff1e0 0x7f17b80ff5f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f17b4009990 tx=0x7f17b4009ca0 comp rx=0 tx=0).stop 2023-11-16T16:35:05.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.178+0000 7f17bf6ac700 1 -- 172.21.15.53:0/1059182915 shutdown_connections 2023-11-16T16:35:05.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.178+0000 7f17bf6ac700 1 --2- 172.21.15.53:0/1059182915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17b8100d70 0x7f17b8107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:35:05.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.178+0000 7f17bf6ac700 1 --2- 172.21.15.53:0/1059182915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17b80ff1e0 0x7f17b80ff5f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:35:05.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.178+0000 7f17bf6ac700 1 --2- 172.21.15.53:0/1059182915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17b81003e0 0x7f17b8100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:35:05.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.178+0000 7f17bf6ac700 1 -- 172.21.15.53:0/1059182915 >> 172.21.15.53:0/1059182915 conn(0x7f17b80fa7f0 msgr2=0x7f17b80fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:35:05.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.178+0000 7f17bf6ac700 1 -- 172.21.15.53:0/1059182915 shutdown_connections 2023-11-16T16:35:05.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.179+0000 7f17bf6ac700 1 -- 172.21.15.53:0/1059182915 wait complete. 2023-11-16T16:35:05.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.179+0000 7f17bf6ac700 1 Processor -- start 2023-11-16T16:35:05.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.179+0000 7f17bf6ac700 1 -- start start 2023-11-16T16:35:05.183 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.179+0000 7f17bf6ac700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17b80ff1e0 0x7f17b8069170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:35:05.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.179+0000 7f17bf6ac700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17b81003e0 0x7f17b819adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:35:05.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.179+0000 7f17bf6ac700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17b8100d70 0x7f17b819d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:35:05.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.179+0000 7f17bf6ac700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b819dc20 con 0x7f17b81003e0 2023-11-16T16:35:05.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.179+0000 7f17bf6ac700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b819dd90 con 0x7f17b80ff1e0 2023-11-16T16:35:05.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.179+0000 7f17bf6ac700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b819df50 con 0x7f17b8100d70 2023-11-16T16:35:05.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.179+0000 7f17bdc49700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17b8100d70 0x7f17b819d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:35:05.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.179+0000 7f17bdc49700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17b8100d70 0x7f17b819d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57798/0 (socket says 172.21.15.53:57798) 2023-11-16T16:35:05.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.179+0000 7f17bdc49700 1 -- 172.21.15.53:0/3189514724 learned_addr learned my addr 172.21.15.53:0/3189514724 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-16T16:35:05.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.180+0000 7f17bd448700 1 --2- 172.21.15.53:0/3189514724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17b80ff1e0 0x7f17b8069170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:35:05.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.180+0000 7f17bcc47700 1 --2- 172.21.15.53:0/3189514724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17b81003e0 0x7f17b819adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:35:05.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.180+0000 7f17bdc49700 1 -- 172.21.15.53:0/3189514724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17b80ff1e0 msgr2=0x7f17b8069170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:35:05.184 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.180+0000 7f17bdc49700 1 --2- 172.21.15.53:0/3189514724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17b80ff1e0 0x7f17b8069170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:35:05.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.180+0000 7f17bdc49700 1 -- 172.21.15.53:0/3189514724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17b81003e0 msgr2=0x7f17b819adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:35:05.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.180+0000 7f17bdc49700 1 --2- 172.21.15.53:0/3189514724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17b81003e0 0x7f17b819adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:35:05.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.180+0000 7f17bdc49700 1 -- 172.21.15.53:0/3189514724 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b40095f0 con 0x7f17b8100d70 2023-11-16T16:35:05.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.180+0000 7f17bcc47700 1 --2- 172.21.15.53:0/3189514724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17b81003e0 0x7f17b819adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:35:05.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.180+0000 7f17bd448700 1 --2- 172.21.15.53:0/3189514724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17b80ff1e0 0x7f17b8069170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:35:05.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.180+0000 7f17bdc49700 1 --2- 172.21.15.53:0/3189514724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17b8100d70 0x7f17b819d480 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f17b0007d70 tx=0x7f17b000a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:35:05.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.180+0000 7f17aa7fc700 1 -- 172.21.15.53:0/3189514724 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b000ad40 con 0x7f17b8100d70 2023-11-16T16:35:05.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.180+0000 7f17aa7fc700 1 -- 172.21.15.53:0/3189514724 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17b00128e0 con 0x7f17b8100d70 2023-11-16T16:35:05.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.180+0000 7f17bf6ac700 1 -- 172.21.15.53:0/3189514724 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17b810ab90 con 0x7f17b8100d70 2023-11-16T16:35:05.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.180+0000 7f17aa7fc700 1 -- 172.21.15.53:0/3189514724 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b0011780 con 0x7f17b8100d70 2023-11-16T16:35:05.185 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.180+0000 7f17bf6ac700 1 -- 172.21.15.53:0/3189514724 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b810aa00 con 0x7f17b8100d70 2023-11-16T16:35:05.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.181+0000 7f17bf6ac700 1 -- 172.21.15.53:0/3189514724 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17b8106830 con 0x7f17b8100d70 2023-11-16T16:35:05.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.182+0000 7f17aa7fc700 1 -- 172.21.15.53:0/3189514724 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f17b0012a50 con 0x7f17b8100d70 2023-11-16T16:35:05.186 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.183+0000 7f17aa7fc700 1 --2- 172.21.15.53:0/3189514724 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f17a40a1150 0x7f17a40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:35:05.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.183+0000 7f17aa7fc700 1 -- 172.21.15.53:0/3189514724 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5426+0+0 (secure 0 0 0) 0x7f17b00cc570 con 0x7f17b8100d70 2023-11-16T16:35:05.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.183+0000 7f17bd448700 1 --2- 172.21.15.53:0/3189514724 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f17a40a1150 0x7f17a40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:35:05.187 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.184+0000 7f17bd448700 1 --2- 172.21.15.53:0/3189514724 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f17a40a1150 0x7f17a40a3600 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f17b400b5c0 tx=0x7f17b4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:35:05.188 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.186+0000 7f17aa7fc700 1 -- 172.21.15.53:0/3189514724 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f17b0096760 con 0x7f17b8100d70 2023-11-16T16:35:05.330 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:05 smithi053 ceph-mon[100638]: purged_snaps scrub starts 2023-11-16T16:35:05.330 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:05 smithi053 ceph-mon[100638]: purged_snaps scrub ok 2023-11-16T16:35:05.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:05 smithi053 ceph-mon[100638]: osd.7 [v2:172.21.15.26:6816/3766901391,v1:172.21.15.26:6817/3766901391] boot 2023-11-16T16:35:05.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:05 smithi053 ceph-mon[100638]: osdmap e52: 8 total, 8 up, 8 in 2023-11-16T16:35:05.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:05 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-16T16:35:05.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:05 smithi053 ceph-mon[100638]: osdmap e53: 8 total, 8 up, 8 in 2023-11-16T16:35:05.394 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:05.392+0000 7f17bf6ac700 1 -- 172.21.15.53:0/3189514724 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f17b819e110 con 0x7f17a40a1150 2023-11-16T16:35:05.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:05 smithi001 ceph-mon[98800]: purged_snaps scrub starts 2023-11-16T16:35:05.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:05 smithi001 ceph-mon[98800]: purged_snaps scrub ok 2023-11-16T16:35:05.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:05 smithi001 ceph-mon[98800]: osd.7 [v2:172.21.15.26:6816/3766901391,v1:172.21.15.26:6817/3766901391] boot 2023-11-16T16:35:05.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:05 smithi001 ceph-mon[98800]: osdmap e52: 8 total, 8 up, 8 in 2023-11-16T16:35:05.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:05 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-16T16:35:05.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:05 smithi001 ceph-mon[98800]: osdmap e53: 8 total, 8 up, 8 in 2023-11-16T16:35:05.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:05 smithi026 ceph-mon[100112]: purged_snaps scrub starts 2023-11-16T16:35:05.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:05 smithi026 ceph-mon[100112]: purged_snaps scrub ok 2023-11-16T16:35:05.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:05 smithi026 ceph-mon[100112]: osd.7 [v2:172.21.15.26:6816/3766901391,v1:172.21.15.26:6817/3766901391] boot 2023-11-16T16:35:05.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:05 smithi026 ceph-mon[100112]: osdmap e52: 8 total, 8 up, 8 in 2023-11-16T16:35:05.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:05 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-16T16:35:05.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:05 smithi026 ceph-mon[100112]: osdmap e53: 8 total, 8 up, 8 in 2023-11-16T16:35:06.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:06 smithi001 ceph-mon[98800]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:35:06.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:06 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:35:06.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:06 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:35:06.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:06 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:35:06.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:06 smithi001 ceph-mon[98800]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:35:06.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:06 smithi001 ceph-mon[98800]: osdmap e54: 8 total, 8 up, 8 in 2023-11-16T16:35:06.449 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:06 smithi053 ceph-mon[100638]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:35:06.449 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:06 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:35:06.449 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:06 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:35:06.449 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:06 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:35:06.449 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:06 smithi053 ceph-mon[100638]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:35:06.449 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:06 smithi053 ceph-mon[100638]: osdmap e54: 8 total, 8 up, 8 in 2023-11-16T16:35:06.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:06 smithi026 ceph-mon[100112]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:35:06.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:06 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:35:06.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:06 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:35:06.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:06 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:35:06.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:06 smithi026 ceph-mon[100112]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:35:06.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:06 smithi026 ceph-mon[100112]: osdmap e54: 8 total, 8 up, 8 in 2023-11-16T16:35:08.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:08 smithi001 ceph-mon[98800]: osdmap e55: 8 total, 8 up, 8 in 2023-11-16T16:35:08.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:08 smithi001 ceph-mon[98800]: pgmap v168: 1 pgs: 1 remapped+peering; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:35:08.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:08 smithi001 ceph-mon[98800]: from='client.? 172.21.15.53:0/2334859561' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "933f231a-7454-47b0-9291-f1fa64f0baec"}]: dispatch 2023-11-16T16:35:08.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:08 smithi001 ceph-mon[98800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "933f231a-7454-47b0-9291-f1fa64f0baec"}]: dispatch 2023-11-16T16:35:08.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:08 smithi001 ceph-mon[98800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "933f231a-7454-47b0-9291-f1fa64f0baec"}]': finished 2023-11-16T16:35:08.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:08 smithi001 ceph-mon[98800]: osdmap e56: 9 total, 8 up, 9 in 2023-11-16T16:35:08.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:08 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-16T16:35:08.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:08 smithi026 ceph-mon[100112]: osdmap e55: 8 total, 8 up, 8 in 2023-11-16T16:35:08.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:08 smithi026 ceph-mon[100112]: pgmap v168: 1 pgs: 1 remapped+peering; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:35:08.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:08 smithi026 ceph-mon[100112]: from='client.? 172.21.15.53:0/2334859561' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "933f231a-7454-47b0-9291-f1fa64f0baec"}]: dispatch 2023-11-16T16:35:08.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:08 smithi026 ceph-mon[100112]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "933f231a-7454-47b0-9291-f1fa64f0baec"}]: dispatch 2023-11-16T16:35:08.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:08 smithi026 ceph-mon[100112]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "933f231a-7454-47b0-9291-f1fa64f0baec"}]': finished 2023-11-16T16:35:08.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:08 smithi026 ceph-mon[100112]: osdmap e56: 9 total, 8 up, 9 in 2023-11-16T16:35:08.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:08 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-16T16:35:08.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:08 smithi053 ceph-mon[100638]: osdmap e55: 8 total, 8 up, 8 in 2023-11-16T16:35:08.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:08 smithi053 ceph-mon[100638]: pgmap v168: 1 pgs: 1 remapped+peering; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:35:08.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:08 smithi053 ceph-mon[100638]: from='client.? 172.21.15.53:0/2334859561' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "933f231a-7454-47b0-9291-f1fa64f0baec"}]: dispatch 2023-11-16T16:35:08.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:08 smithi053 ceph-mon[100638]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "933f231a-7454-47b0-9291-f1fa64f0baec"}]: dispatch 2023-11-16T16:35:08.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:08 smithi053 ceph-mon[100638]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "933f231a-7454-47b0-9291-f1fa64f0baec"}]': finished 2023-11-16T16:35:08.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:08 smithi053 ceph-mon[100638]: osdmap e56: 9 total, 8 up, 9 in 2023-11-16T16:35:08.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:08 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-16T16:35:09.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:09 smithi001 ceph-mon[98800]: from='client.? 172.21.15.53:0/2410458067' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:35:09.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:09 smithi026 ceph-mon[100112]: from='client.? 172.21.15.53:0/2410458067' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:35:09.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:09 smithi053 ceph-mon[100638]: from='client.? 172.21.15.53:0/2410458067' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:35:10.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:10 smithi001 ceph-mon[98800]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:35:10.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:10 smithi026 ceph-mon[100112]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:35:10.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:10 smithi053 ceph-mon[100638]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:35:12.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:12 smithi053 ceph-mon[100638]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:35:12.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:12 smithi001 ceph-mon[98800]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:35:12.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:12 smithi026 ceph-mon[100112]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:35:13.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:13 smithi053 ceph-mon[100638]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:35:13.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:13 smithi001 ceph-mon[98800]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:35:14.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:13 smithi026 ceph-mon[100112]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:35:14.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:14 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-11-16T16:35:14.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:14 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:35:14.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:14 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-11-16T16:35:14.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:14 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:35:15.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:14 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-11-16T16:35:15.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:14 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:35:15.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:15 smithi053 ceph-mon[100638]: Deploying daemon osd.8 on smithi053 2023-11-16T16:35:15.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:15 smithi053 ceph-mon[100638]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:35:15.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:15 smithi001 ceph-mon[98800]: Deploying daemon osd.8 on smithi053 2023-11-16T16:35:15.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:15 smithi001 ceph-mon[98800]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:35:16.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:15 smithi026 ceph-mon[100112]: Deploying daemon osd.8 on smithi053 2023-11-16T16:35:16.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:15 smithi026 ceph-mon[100112]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:35:17.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:17 smithi053 ceph-mon[100638]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:35:17.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:17 smithi001 ceph-mon[98800]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:35:18.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:17 smithi026 ceph-mon[100112]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:35:20.152 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:19 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:20.152 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:19 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:35:20.153 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:19 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:20.153 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:19 smithi053 ceph-mon[100638]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:35:20.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:19 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:20.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:19 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:35:20.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:19 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:20.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:19 smithi001 ceph-mon[98800]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:35:20.257 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:19 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:20.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:19 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:35:20.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:19 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:20.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:19 smithi026 ceph-mon[100112]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:35:21.662 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:21.659+0000 7f17aa7fc700 1 -- 172.21.15.53:0/3189514724 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f17b819e110 con 0x7f17a40a1150 2023-11-16T16:35:21.662 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 8 on host 'smithi053' 2023-11-16T16:35:21.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:21.661+0000 7f17bf6ac700 1 -- 172.21.15.53:0/3189514724 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f17a40a1150 msgr2=0x7f17a40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:35:21.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:21.661+0000 7f17bf6ac700 1 --2- 172.21.15.53:0/3189514724 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f17a40a1150 0x7f17a40a3600 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f17b400b5c0 tx=0x7f17b4005fb0 comp rx=0 tx=0).stop 2023-11-16T16:35:21.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:21.661+0000 7f17bf6ac700 1 -- 172.21.15.53:0/3189514724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17b8100d70 msgr2=0x7f17b819d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:35:21.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:21.661+0000 7f17bf6ac700 1 --2- 172.21.15.53:0/3189514724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17b8100d70 0x7f17b819d480 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f17b0007d70 tx=0x7f17b000a340 comp rx=0 tx=0).stop 2023-11-16T16:35:21.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:21.661+0000 7f17bf6ac700 1 -- 172.21.15.53:0/3189514724 shutdown_connections 2023-11-16T16:35:21.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:21.661+0000 7f17bf6ac700 1 --2- 172.21.15.53:0/3189514724 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f17a40a1150 0x7f17a40a3600 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:35:21.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:21.661+0000 7f17bf6ac700 1 --2- 172.21.15.53:0/3189514724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f17b8100d70 0x7f17b819d480 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:35:21.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:21.661+0000 7f17bf6ac700 1 --2- 172.21.15.53:0/3189514724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17b80ff1e0 0x7f17b8069170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:35:21.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:21.661+0000 7f17bf6ac700 1 --2- 172.21.15.53:0/3189514724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17b81003e0 0x7f17b819adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:35:21.664 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:21.661+0000 7f17bf6ac700 1 -- 172.21.15.53:0/3189514724 >> 172.21.15.53:0/3189514724 conn(0x7f17b80fa7f0 msgr2=0x7f17b8103970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:35:21.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:21.661+0000 7f17bf6ac700 1 -- 172.21.15.53:0/3189514724 shutdown_connections 2023-11-16T16:35:21.665 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:35:21.661+0000 7f17bf6ac700 1 -- 172.21.15.53:0/3189514724 wait complete. 2023-11-16T16:35:22.065 DEBUG:teuthology.orchestra.run.smithi053:osd.8> sudo journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@osd.8.service 2023-11-16T16:35:22.067 INFO:tasks.cephadm:Deploying osd.9 on smithi001 with /dev/vg_nvme/lv_1... 2023-11-16T16:35:22.067 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-11-16T16:35:22.106 INFO:journalctl@ceph.osd.8.smithi053.stdout:-- Logs begin at Thu 2023-11-16 16:14:15 UTC. -- 2023-11-16T16:35:22.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:22 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:22.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:22 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:22.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:22 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:35:22.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:22 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:35:22.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:22 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:35:22.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:22 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:22.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:22 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:35:22.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:22 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:35:22.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:22 smithi001 ceph-mon[98800]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:35:22.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:22 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:22.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:22 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:22.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:22 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:35:22.695 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:22 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:22.695 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:22 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:22.695 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:22 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:35:22.695 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:22 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:35:22.695 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:22 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:35:22.695 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:22 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:22.696 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:22 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:35:22.696 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:22 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:35:22.696 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:22 smithi053 ceph-mon[100638]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:35:22.696 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:22 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:22.696 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:22 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:22.696 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:22 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:35:22.732 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:35:22.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:22 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:22.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:22 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:22.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:22 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:35:22.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:22 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:35:22.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:22 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:35:22.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:22 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:22.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:22 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:35:22.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:22 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:35:22.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:22 smithi026 ceph-mon[100112]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:35:22.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:22 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:22.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:22 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:22.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:22 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:35:23.037 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:35:22 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:35:22.821+0000 7fc0389ce640 -1 osd.8 0 log_to_monitors true 2023-11-16T16:35:23.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:23 smithi053 ceph-mon[100638]: from='osd.8 [v2:172.21.15.53:6816/3224335871,v1:172.21.15.53:6817/3224335871]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-16T16:35:23.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:23 smithi053 ceph-mon[100638]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-16T16:35:23.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:23 smithi001 ceph-mon[98800]: from='osd.8 [v2:172.21.15.53:6816/3224335871,v1:172.21.15.53:6817/3224335871]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-16T16:35:23.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:23 smithi001 ceph-mon[98800]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-16T16:35:23.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:23 smithi026 ceph-mon[100112]: from='osd.8 [v2:172.21.15.53:6816/3224335871,v1:172.21.15.53:6817/3224335871]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-16T16:35:23.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:23 smithi026 ceph-mon[100112]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-16T16:35:24.360 INFO:teuthology.orchestra.run.smithi001.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-11-16T16:35:24.360 INFO:teuthology.orchestra.run.smithi001.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-11-16T16:35:24.360 INFO:teuthology.orchestra.run.smithi001.stderr: stderr: 10+0 records in 2023-11-16T16:35:24.360 INFO:teuthology.orchestra.run.smithi001.stderr:10+0 records out 2023-11-16T16:35:24.360 INFO:teuthology.orchestra.run.smithi001.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00849534 s, 1.2 GB/s 2023-11-16T16:35:24.360 INFO:teuthology.orchestra.run.smithi001.stderr:--> Zapping successful for: 2023-11-16T16:35:24.426 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:24 smithi053 ceph-mon[100638]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-11-16T16:35:24.426 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:24 smithi053 ceph-mon[100638]: osdmap e57: 9 total, 8 up, 9 in 2023-11-16T16:35:24.426 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:24 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-16T16:35:24.426 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:24 smithi053 ceph-mon[100638]: from='osd.8 [v2:172.21.15.53:6816/3224335871,v1:172.21.15.53:6817/3224335871]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2023-11-16T16:35:24.426 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:24 smithi053 ceph-mon[100638]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2023-11-16T16:35:24.426 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:24 smithi053 ceph-mon[100638]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:35:24.426 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:24 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:24.426 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:24 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:24.426 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:24 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:35:24.427 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:24 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:35:24.427 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:24 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:35:24.427 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:24 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:24.427 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:24 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:35:24.427 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:24 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:35:24.427 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:24 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:24.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:24 smithi001 ceph-mon[98800]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-11-16T16:35:24.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:24 smithi001 ceph-mon[98800]: osdmap e57: 9 total, 8 up, 9 in 2023-11-16T16:35:24.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:24 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-16T16:35:24.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:24 smithi001 ceph-mon[98800]: from='osd.8 [v2:172.21.15.53:6816/3224335871,v1:172.21.15.53:6817/3224335871]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2023-11-16T16:35:24.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:24 smithi001 ceph-mon[98800]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2023-11-16T16:35:24.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:24 smithi001 ceph-mon[98800]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:35:24.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:24 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:24.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:24 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:24.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:24 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:35:24.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:24 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:35:24.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:24 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:35:24.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:24 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:24.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:24 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:35:24.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:24 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:35:24.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:24 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:24.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:24 smithi026 ceph-mon[100112]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-11-16T16:35:24.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:24 smithi026 ceph-mon[100112]: osdmap e57: 9 total, 8 up, 9 in 2023-11-16T16:35:24.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:24 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-16T16:35:24.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:24 smithi026 ceph-mon[100112]: from='osd.8 [v2:172.21.15.53:6816/3224335871,v1:172.21.15.53:6817/3224335871]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2023-11-16T16:35:24.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:24 smithi026 ceph-mon[100112]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2023-11-16T16:35:24.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:24 smithi026 ceph-mon[100112]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:35:24.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:24 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:24.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:24 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:24.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:24 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:35:24.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:24 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:35:24.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:24 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:35:24.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:24 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:24.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:24 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:35:24.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:24 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:35:24.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:24 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:24.779 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph orch daemon add osd smithi001:vg_nvme/lv_1 2023-11-16T16:35:24.830 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:35:24 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:35:24.422+0000 7fc02e045700 -1 osd.8 0 waiting for initial osdmap 2023-11-16T16:35:24.830 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:35:24 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:35:24.435+0000 7fc029639700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-16T16:35:25.437 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:35:25.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:25 smithi001 ceph-mon[98800]: Detected new or changed devices on smithi053 2023-11-16T16:35:25.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:25 smithi001 ceph-mon[98800]: Adjusting osd_memory_target on smithi053 to 5724M 2023-11-16T16:35:25.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:25 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:35:25.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:25 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:35:25.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:25 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:35:25.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:25 smithi001 ceph-mon[98800]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2023-11-16T16:35:25.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:25 smithi001 ceph-mon[98800]: osdmap e58: 9 total, 8 up, 9 in 2023-11-16T16:35:25.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:25 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-16T16:35:25.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:25 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-16T16:35:25.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:25 smithi026 ceph-mon[100112]: Detected new or changed devices on smithi053 2023-11-16T16:35:25.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:25 smithi026 ceph-mon[100112]: Adjusting osd_memory_target on smithi053 to 5724M 2023-11-16T16:35:25.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:25 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:35:25.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:25 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:35:25.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:25 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:35:25.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:25 smithi026 ceph-mon[100112]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2023-11-16T16:35:25.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:25 smithi026 ceph-mon[100112]: osdmap e58: 9 total, 8 up, 9 in 2023-11-16T16:35:25.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:25 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-16T16:35:25.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:25 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-16T16:35:25.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:25 smithi053 ceph-mon[100638]: Detected new or changed devices on smithi053 2023-11-16T16:35:25.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:25 smithi053 ceph-mon[100638]: Adjusting osd_memory_target on smithi053 to 5724M 2023-11-16T16:35:25.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:25 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:35:25.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:25 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:35:25.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:25 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:35:25.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:25 smithi053 ceph-mon[100638]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2023-11-16T16:35:25.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:25 smithi053 ceph-mon[100638]: osdmap e58: 9 total, 8 up, 9 in 2023-11-16T16:35:25.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:25 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-16T16:35:25.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:25 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-16T16:35:26.648 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:26 smithi001 ceph-mon[98800]: purged_snaps scrub starts 2023-11-16T16:35:26.648 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:26 smithi001 ceph-mon[98800]: purged_snaps scrub ok 2023-11-16T16:35:26.648 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:26 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-16T16:35:26.648 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:26 smithi001 ceph-mon[98800]: osd.8 [v2:172.21.15.53:6816/3224335871,v1:172.21.15.53:6817/3224335871] boot 2023-11-16T16:35:26.648 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:26 smithi001 ceph-mon[98800]: osdmap e59: 9 total, 9 up, 9 in 2023-11-16T16:35:26.648 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:26 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-16T16:35:26.648 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:26 smithi001 ceph-mon[98800]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:35:26.648 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:26 smithi001 ceph-mon[98800]: osdmap e60: 9 total, 9 up, 9 in 2023-11-16T16:35:26.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:26 smithi026 ceph-mon[100112]: purged_snaps scrub starts 2023-11-16T16:35:26.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:26 smithi026 ceph-mon[100112]: purged_snaps scrub ok 2023-11-16T16:35:26.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:26 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-16T16:35:26.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:26 smithi026 ceph-mon[100112]: osd.8 [v2:172.21.15.53:6816/3224335871,v1:172.21.15.53:6817/3224335871] boot 2023-11-16T16:35:26.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:26 smithi026 ceph-mon[100112]: osdmap e59: 9 total, 9 up, 9 in 2023-11-16T16:35:26.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:26 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-16T16:35:26.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:26 smithi026 ceph-mon[100112]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:35:26.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:26 smithi026 ceph-mon[100112]: osdmap e60: 9 total, 9 up, 9 in 2023-11-16T16:35:26.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:26 smithi053 ceph-mon[100638]: purged_snaps scrub starts 2023-11-16T16:35:26.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:26 smithi053 ceph-mon[100638]: purged_snaps scrub ok 2023-11-16T16:35:26.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:26 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-16T16:35:26.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:26 smithi053 ceph-mon[100638]: osd.8 [v2:172.21.15.53:6816/3224335871,v1:172.21.15.53:6817/3224335871] boot 2023-11-16T16:35:26.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:26 smithi053 ceph-mon[100638]: osdmap e59: 9 total, 9 up, 9 in 2023-11-16T16:35:26.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:26 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-16T16:35:26.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:26 smithi053 ceph-mon[100638]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-11-16T16:35:26.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:26 smithi053 ceph-mon[100638]: osdmap e60: 9 total, 9 up, 9 in 2023-11-16T16:35:26.851 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.847+0000 7feabad61700 1 -- 172.21.15.1:0/430344750 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feab4100d70 msgr2=0x7feab4107630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:35:26.851 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.847+0000 7feabad61700 1 --2- 172.21.15.1:0/430344750 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feab4100d70 0x7feab4107630 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7feaac00b0a0 tx=0x7feaac00b3b0 comp rx=0 tx=0).stop 2023-11-16T16:35:26.851 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.847+0000 7feabad61700 1 -- 172.21.15.1:0/430344750 shutdown_connections 2023-11-16T16:35:26.851 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.847+0000 7feabad61700 1 --2- 172.21.15.1:0/430344750 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feab4100d70 0x7feab4107630 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:35:26.851 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.847+0000 7feabad61700 1 --2- 172.21.15.1:0/430344750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feab40ff1e0 0x7feab40ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:35:26.851 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.847+0000 7feabad61700 1 --2- 172.21.15.1:0/430344750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feab41003e0 0x7feab4100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:35:26.851 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.847+0000 7feabad61700 1 -- 172.21.15.1:0/430344750 >> 172.21.15.1:0/430344750 conn(0x7feab40fa7f0 msgr2=0x7feab40fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:35:26.851 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.847+0000 7feabad61700 1 -- 172.21.15.1:0/430344750 shutdown_connections 2023-11-16T16:35:26.851 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.847+0000 7feabad61700 1 -- 172.21.15.1:0/430344750 wait complete. 2023-11-16T16:35:26.851 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.848+0000 7feabad61700 1 Processor -- start 2023-11-16T16:35:26.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.848+0000 7feabad61700 1 -- start start 2023-11-16T16:35:26.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.848+0000 7feabad61700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feab40ff1e0 0x7feab4103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:35:26.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.848+0000 7feabad61700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feab41003e0 0x7feab41017c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:35:26.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.848+0000 7feabad61700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feab4100d70 0x7feab4101d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:35:26.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.848+0000 7feabad61700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab4102480 con 0x7feab41003e0 2023-11-16T16:35:26.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.848+0000 7feabad61700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab41025f0 con 0x7feab40ff1e0 2023-11-16T16:35:26.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.848+0000 7feabad61700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab41a01c0 con 0x7feab4100d70 2023-11-16T16:35:26.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.848+0000 7feab3fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feab41003e0 0x7feab41017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:35:26.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.848+0000 7feab3fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feab41003e0 0x7feab41017c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40890/0 (socket says 172.21.15.1:40890) 2023-11-16T16:35:26.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.848+0000 7feab3fff700 1 -- 172.21.15.1:0/406848875 learned_addr learned my addr 172.21.15.1:0/406848875 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:35:26.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.848+0000 7feab8afd700 1 --2- 172.21.15.1:0/406848875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feab40ff1e0 0x7feab4103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:35:26.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.848+0000 7feab92fe700 1 --2- 172.21.15.1:0/406848875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feab4100d70 0x7feab4101d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:35:26.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.848+0000 7feab3fff700 1 -- 172.21.15.1:0/406848875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feab4100d70 msgr2=0x7feab4101d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:35:26.853 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.848+0000 7feab3fff700 1 --2- 172.21.15.1:0/406848875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feab4100d70 0x7feab4101d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:35:26.853 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.848+0000 7feab3fff700 1 -- 172.21.15.1:0/406848875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feab40ff1e0 msgr2=0x7feab4103170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:35:26.853 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.848+0000 7feab3fff700 1 --2- 172.21.15.1:0/406848875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feab40ff1e0 0x7feab4103170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:35:26.853 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.848+0000 7feab3fff700 1 -- 172.21.15.1:0/406848875 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaac009d00 con 0x7feab41003e0 2023-11-16T16:35:26.853 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.848+0000 7feab92fe700 1 --2- 172.21.15.1:0/406848875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feab4100d70 0x7feab4101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:35:26.853 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.849+0000 7feab3fff700 1 --2- 172.21.15.1:0/406848875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feab41003e0 0x7feab41017c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7feaa000d980 tx=0x7feaa000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:35:26.853 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.849+0000 7feab1ffb700 1 -- 172.21.15.1:0/406848875 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa000bb60 con 0x7feab41003e0 2023-11-16T16:35:26.853 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.849+0000 7feab1ffb700 1 -- 172.21.15.1:0/406848875 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feaa0007640 con 0x7feab41003e0 2023-11-16T16:35:26.853 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.849+0000 7feabad61700 1 -- 172.21.15.1:0/406848875 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feab410ab30 con 0x7feab41003e0 2023-11-16T16:35:26.853 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.849+0000 7feab1ffb700 1 -- 172.21.15.1:0/406848875 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa0004e90 con 0x7feab41003e0 2023-11-16T16:35:26.853 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.849+0000 7feabad61700 1 -- 172.21.15.1:0/406848875 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab410a9a0 con 0x7feab41003e0 2023-11-16T16:35:26.854 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.850+0000 7feabad61700 1 -- 172.21.15.1:0/406848875 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feab4066ea0 con 0x7feab41003e0 2023-11-16T16:35:26.854 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.851+0000 7feab1ffb700 1 -- 172.21.15.1:0/406848875 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7feaa000bcc0 con 0x7feab41003e0 2023-11-16T16:35:26.854 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.851+0000 7feab1ffb700 1 --2- 172.21.15.1:0/406848875 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7feaa40a1150 0x7feaa40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:35:26.854 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.852+0000 7feab1ffb700 1 -- 172.21.15.1:0/406848875 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5858+0+0 (secure 0 0 0) 0x7feaa00cbe40 con 0x7feab41003e0 2023-11-16T16:35:26.854 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.852+0000 7feab8afd700 1 --2- 172.21.15.1:0/406848875 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7feaa40a1150 0x7feaa40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:35:26.855 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.852+0000 7feab8afd700 1 --2- 172.21.15.1:0/406848875 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7feaa40a1150 0x7feaa40a3600 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7feaa8005fd0 tx=0x7feaa8005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:35:26.858 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:26.856+0000 7feab1ffb700 1 -- 172.21.15.1:0/406848875 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7feaa0095f20 con 0x7feab41003e0 2023-11-16T16:35:27.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:27.061+0000 7feabad61700 1 -- 172.21.15.1:0/406848875 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7feab4068d20 con 0x7feaa40a1150 2023-11-16T16:35:27.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:27 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:35:27.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:27 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:35:27.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:27 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:35:27.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:27 smithi001 ceph-mon[98800]: osdmap e61: 9 total, 9 up, 9 in 2023-11-16T16:35:27.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:27 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:35:27.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:27 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:35:27.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:27 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:35:27.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:27 smithi026 ceph-mon[100112]: osdmap e61: 9 total, 9 up, 9 in 2023-11-16T16:35:27.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:27 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:35:27.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:27 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:35:27.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:27 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:35:27.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:27 smithi053 ceph-mon[100638]: osdmap e61: 9 total, 9 up, 9 in 2023-11-16T16:35:28.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:28 smithi001 ceph-mon[98800]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:35:28.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:28 smithi001 ceph-mon[98800]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:35:28.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:28 smithi026 ceph-mon[100112]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:35:28.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:28 smithi026 ceph-mon[100112]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:35:28.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:28 smithi053 ceph-mon[100638]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:35:28.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:28 smithi053 ceph-mon[100638]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:35:29.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:29 smithi026 ceph-mon[100112]: osdmap e62: 9 total, 9 up, 9 in 2023-11-16T16:35:29.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:29 smithi053 ceph-mon[100638]: osdmap e62: 9 total, 9 up, 9 in 2023-11-16T16:35:29.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:29 smithi001 ceph-mon[98800]: osdmap e62: 9 total, 9 up, 9 in 2023-11-16T16:35:30.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:30 smithi026 ceph-mon[100112]: osdmap e63: 9 total, 9 up, 9 in 2023-11-16T16:35:30.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:30 smithi026 ceph-mon[100112]: pgmap v187: 1 pgs: 1 peering; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-16T16:35:30.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:30 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1607396295' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "245fcbb3-c651-4703-ab80-309dc98bd983"}]: dispatch 2023-11-16T16:35:30.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:30 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1607396295' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "245fcbb3-c651-4703-ab80-309dc98bd983"}]': finished 2023-11-16T16:35:30.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:30 smithi026 ceph-mon[100112]: osdmap e64: 10 total, 9 up, 10 in 2023-11-16T16:35:30.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:30 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-16T16:35:30.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:30 smithi053 ceph-mon[100638]: osdmap e63: 9 total, 9 up, 9 in 2023-11-16T16:35:30.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:30 smithi053 ceph-mon[100638]: pgmap v187: 1 pgs: 1 peering; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-16T16:35:30.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:30 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1607396295' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "245fcbb3-c651-4703-ab80-309dc98bd983"}]: dispatch 2023-11-16T16:35:30.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:30 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1607396295' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "245fcbb3-c651-4703-ab80-309dc98bd983"}]': finished 2023-11-16T16:35:30.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:30 smithi053 ceph-mon[100638]: osdmap e64: 10 total, 9 up, 10 in 2023-11-16T16:35:30.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:30 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-16T16:35:30.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:30 smithi001 ceph-mon[98800]: osdmap e63: 9 total, 9 up, 9 in 2023-11-16T16:35:30.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:30 smithi001 ceph-mon[98800]: pgmap v187: 1 pgs: 1 peering; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-16T16:35:30.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:30 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1607396295' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "245fcbb3-c651-4703-ab80-309dc98bd983"}]: dispatch 2023-11-16T16:35:30.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:30 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1607396295' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "245fcbb3-c651-4703-ab80-309dc98bd983"}]': finished 2023-11-16T16:35:30.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:30 smithi001 ceph-mon[98800]: osdmap e64: 10 total, 9 up, 10 in 2023-11-16T16:35:30.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:30 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-16T16:35:31.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:31 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1452099332' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:35:31.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:31 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1452099332' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:35:31.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:31 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1452099332' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:35:32.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:32 smithi026 ceph-mon[100112]: pgmap v189: 1 pgs: 1 peering; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 134 KiB/s, 0 objects/s recovering 2023-11-16T16:35:32.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:32 smithi053 ceph-mon[100638]: pgmap v189: 1 pgs: 1 peering; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 134 KiB/s, 0 objects/s recovering 2023-11-16T16:35:32.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:32 smithi001 ceph-mon[98800]: pgmap v189: 1 pgs: 1 peering; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 134 KiB/s, 0 objects/s recovering 2023-11-16T16:35:34.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:34 smithi026 ceph-mon[100112]: pgmap v190: 1 pgs: 1 peering; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-11-16T16:35:34.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:34 smithi053 ceph-mon[100638]: pgmap v190: 1 pgs: 1 peering; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-11-16T16:35:34.840 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:34 smithi001 ceph-mon[98800]: pgmap v190: 1 pgs: 1 peering; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-11-16T16:35:36.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:36 smithi026 ceph-mon[100112]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2023-11-16T16:35:36.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:36 smithi053 ceph-mon[100638]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2023-11-16T16:35:36.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:36 smithi001 ceph-mon[98800]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 82 KiB/s, 0 objects/s recovering 2023-11-16T16:35:37.648 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:37 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-11-16T16:35:37.648 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:37 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:35:37.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:37 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-11-16T16:35:37.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:37 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:35:37.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:37 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-11-16T16:35:37.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:37 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:35:38.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:38 smithi026 ceph-mon[100112]: Deploying daemon osd.9 on smithi001 2023-11-16T16:35:38.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:38 smithi026 ceph-mon[100112]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:35:38.764 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:38 smithi001 ceph-mon[98800]: Deploying daemon osd.9 on smithi001 2023-11-16T16:35:38.764 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:38 smithi001 ceph-mon[98800]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:35:38.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:38 smithi053 ceph-mon[100638]: Deploying daemon osd.9 on smithi001 2023-11-16T16:35:38.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:38 smithi053 ceph-mon[100638]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:35:40.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:40 smithi001 ceph-mon[98800]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:35:40.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:40 smithi026 ceph-mon[100112]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:35:40.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:40 smithi053 ceph-mon[100638]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:35:42.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:42 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:42.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:42 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:42.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:42 smithi001 ceph-mon[98800]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:35:42.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:42 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:35:42.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:42 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:42.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:42 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:42.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:42 smithi026 ceph-mon[100112]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:35:42.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:42 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:35:42.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:42 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:42.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:42 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:42.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:42 smithi053 ceph-mon[100638]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:35:42.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:42 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:35:43.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:43 smithi053 ceph-mon[100638]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:35:43.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:43 smithi001 ceph-mon[98800]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:35:44.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:43 smithi026 ceph-mon[100112]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:35:44.526 INFO:teuthology.orchestra.run.smithi001.stdout:Created osd(s) 9 on host 'smithi001' 2023-11-16T16:35:44.526 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:44.523+0000 7feab1ffb700 1 -- 172.21.15.1:0/406848875 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7feab4068d20 con 0x7feaa40a1150 2023-11-16T16:35:44.528 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:44.526+0000 7feabad61700 1 -- 172.21.15.1:0/406848875 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7feaa40a1150 msgr2=0x7feaa40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:35:44.528 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:44.526+0000 7feabad61700 1 --2- 172.21.15.1:0/406848875 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7feaa40a1150 0x7feaa40a3600 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7feaa8005fd0 tx=0x7feaa8005dd0 comp rx=0 tx=0).stop 2023-11-16T16:35:44.528 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:44.526+0000 7feabad61700 1 -- 172.21.15.1:0/406848875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feab41003e0 msgr2=0x7feab41017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:35:44.528 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:44.526+0000 7feabad61700 1 --2- 172.21.15.1:0/406848875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feab41003e0 0x7feab41017c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7feaa000d980 tx=0x7feaa000dd40 comp rx=0 tx=0).stop 2023-11-16T16:35:44.528 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:44.526+0000 7feabad61700 1 -- 172.21.15.1:0/406848875 shutdown_connections 2023-11-16T16:35:44.528 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:44.526+0000 7feabad61700 1 --2- 172.21.15.1:0/406848875 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7feaa40a1150 0x7feaa40a3600 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:35:44.529 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:44.526+0000 7feabad61700 1 --2- 172.21.15.1:0/406848875 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feab4100d70 0x7feab4101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:35:44.529 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:44.526+0000 7feabad61700 1 --2- 172.21.15.1:0/406848875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feab40ff1e0 0x7feab4103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:35:44.529 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:44.526+0000 7feabad61700 1 --2- 172.21.15.1:0/406848875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feab41003e0 0x7feab41017c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:35:44.529 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:44.526+0000 7feabad61700 1 -- 172.21.15.1:0/406848875 >> 172.21.15.1:0/406848875 conn(0x7feab40fa7f0 msgr2=0x7feab4068610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:35:44.529 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:44.526+0000 7feabad61700 1 -- 172.21.15.1:0/406848875 shutdown_connections 2023-11-16T16:35:44.529 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:35:44.526+0000 7feabad61700 1 -- 172.21.15.1:0/406848875 wait complete. 2023-11-16T16:35:44.928 DEBUG:teuthology.orchestra.run.smithi001:osd.9> sudo journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@osd.9.service 2023-11-16T16:35:44.930 INFO:tasks.cephadm:Deploying osd.10 on smithi026 with /dev/vg_nvme/lv_1... 2023-11-16T16:35:44.930 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-11-16T16:35:44.961 INFO:journalctl@ceph.osd.9.smithi001.stdout:-- Logs begin at Thu 2023-11-16 16:14:27 UTC. -- 2023-11-16T16:35:45.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:45 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:45.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:45 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:45.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:45 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:35:45.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:45 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:35:45.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:45 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:35:45.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:45 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:45.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:45 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:35:45.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:45 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:35:45.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:45 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:45.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:45 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:45.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:45 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:35:45.312 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:45 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:45.313 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:45 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:45.313 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:45 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:35:45.313 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:45 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:35:45.313 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:45 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:35:45.313 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:45 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:45.313 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:45 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:35:45.313 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:45 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:35:45.313 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:45 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:45.313 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:45 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:45.314 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:45 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:35:45.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:45 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:45.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:45 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:45.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:45 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:35:45.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:45 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:35:45.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:45 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:35:45.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:45 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:45.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:45 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:35:45.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:45 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:35:45.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:45 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:45.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:45 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:45.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:45 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:35:45.593 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.b/config 2023-11-16T16:35:45.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:35:45 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:35:45.309+0000 7f9ef6311640 -1 osd.9 0 log_to_monitors true 2023-11-16T16:35:46.158 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:46 smithi026 ceph-mon[100112]: from='osd.9 [v2:172.21.15.1:6826/1391199844,v1:172.21.15.1:6827/1391199844]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-11-16T16:35:46.158 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:46 smithi026 ceph-mon[100112]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:35:46.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:46 smithi053 ceph-mon[100638]: from='osd.9 [v2:172.21.15.1:6826/1391199844,v1:172.21.15.1:6827/1391199844]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-11-16T16:35:46.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:46 smithi053 ceph-mon[100638]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:35:46.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:46 smithi001 ceph-mon[98800]: from='osd.9 [v2:172.21.15.1:6826/1391199844,v1:172.21.15.1:6827/1391199844]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-11-16T16:35:46.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:46 smithi001 ceph-mon[98800]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:35:47.226 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-11-16T16:35:47.226 INFO:teuthology.orchestra.run.smithi026.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-11-16T16:35:47.226 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10+0 records in 2023-11-16T16:35:47.227 INFO:teuthology.orchestra.run.smithi026.stderr:10+0 records out 2023-11-16T16:35:47.227 INFO:teuthology.orchestra.run.smithi026.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0133268 s, 787 MB/s 2023-11-16T16:35:47.227 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 2023-11-16T16:35:47.227 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping successful for: 2023-11-16T16:35:47.259 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:47 smithi001 ceph-mon[98800]: from='osd.9 [v2:172.21.15.1:6826/1391199844,v1:172.21.15.1:6827/1391199844]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-11-16T16:35:47.259 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:47 smithi001 ceph-mon[98800]: osdmap e65: 10 total, 9 up, 10 in 2023-11-16T16:35:47.260 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:47 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-16T16:35:47.260 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:47 smithi001 ceph-mon[98800]: from='osd.9 [v2:172.21.15.1:6826/1391199844,v1:172.21.15.1:6827/1391199844]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2023-11-16T16:35:47.260 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:35:47 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:35:47.048+0000 7f9eeb988700 -1 osd.9 0 waiting for initial osdmap 2023-11-16T16:35:47.260 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:35:47 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:35:47.061+0000 7f9ee3775700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-16T16:35:47.330 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:47 smithi053 ceph-mon[100638]: from='osd.9 [v2:172.21.15.1:6826/1391199844,v1:172.21.15.1:6827/1391199844]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-11-16T16:35:47.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:47 smithi053 ceph-mon[100638]: osdmap e65: 10 total, 9 up, 10 in 2023-11-16T16:35:47.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:47 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-16T16:35:47.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:47 smithi053 ceph-mon[100638]: from='osd.9 [v2:172.21.15.1:6826/1391199844,v1:172.21.15.1:6827/1391199844]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2023-11-16T16:35:47.448 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:47 smithi026 ceph-mon[100112]: from='osd.9 [v2:172.21.15.1:6826/1391199844,v1:172.21.15.1:6827/1391199844]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-11-16T16:35:47.449 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:47 smithi026 ceph-mon[100112]: osdmap e65: 10 total, 9 up, 10 in 2023-11-16T16:35:47.449 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:47 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-16T16:35:47.449 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:47 smithi026 ceph-mon[100112]: from='osd.9 [v2:172.21.15.1:6826/1391199844,v1:172.21.15.1:6827/1391199844]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2023-11-16T16:35:47.653 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph orch daemon add osd smithi026:vg_nvme/lv_1 2023-11-16T16:35:48.307 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.b/config 2023-11-16T16:35:48.330 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:48 smithi053 ceph-mon[100638]: from='osd.9 [v2:172.21.15.1:6826/1391199844,v1:172.21.15.1:6827/1391199844]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2023-11-16T16:35:48.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:48 smithi053 ceph-mon[100638]: osdmap e66: 10 total, 9 up, 10 in 2023-11-16T16:35:48.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:48 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-16T16:35:48.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:48 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-16T16:35:48.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:48 smithi053 ceph-mon[100638]: Detected new or changed devices on smithi001 2023-11-16T16:35:48.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:48 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:48.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:48 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:48.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:48 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:35:48.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:48 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:35:48.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:48 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:35:48.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:48 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:35:48.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:48 smithi053 ceph-mon[100638]: Adjusting osd_memory_target on smithi001 to 4293M 2023-11-16T16:35:48.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:48 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:48.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:48 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:35:48.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:48 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:35:48.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:48 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:35:48.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:48 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:48.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:48 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:35:48.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:48 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:35:48.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:48 smithi053 ceph-mon[100638]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:35:48.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:48 smithi001 ceph-mon[98800]: from='osd.9 [v2:172.21.15.1:6826/1391199844,v1:172.21.15.1:6827/1391199844]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2023-11-16T16:35:48.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:48 smithi001 ceph-mon[98800]: osdmap e66: 10 total, 9 up, 10 in 2023-11-16T16:35:48.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:48 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-16T16:35:48.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:48 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-16T16:35:48.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:48 smithi001 ceph-mon[98800]: Detected new or changed devices on smithi001 2023-11-16T16:35:48.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:48 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:48.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:48 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:48.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:48 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:35:48.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:48 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:35:48.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:48 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:35:48.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:48 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:35:48.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:48 smithi001 ceph-mon[98800]: Adjusting osd_memory_target on smithi001 to 4293M 2023-11-16T16:35:48.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:48 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:48.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:48 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:35:48.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:48 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:35:48.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:48 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:35:48.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:48 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:48.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:48 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:35:48.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:48 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:35:48.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:48 smithi001 ceph-mon[98800]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:35:48.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:48 smithi026 ceph-mon[100112]: from='osd.9 [v2:172.21.15.1:6826/1391199844,v1:172.21.15.1:6827/1391199844]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2023-11-16T16:35:48.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:48 smithi026 ceph-mon[100112]: osdmap e66: 10 total, 9 up, 10 in 2023-11-16T16:35:48.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:48 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-16T16:35:48.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:48 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-16T16:35:48.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:48 smithi026 ceph-mon[100112]: Detected new or changed devices on smithi001 2023-11-16T16:35:48.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:48 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:48.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:48 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:48.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:48 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:35:48.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:48 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:35:48.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:48 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:35:48.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:48 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:35:48.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:48 smithi026 ceph-mon[100112]: Adjusting osd_memory_target on smithi001 to 4293M 2023-11-16T16:35:48.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:48 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:48.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:48 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:35:48.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:48 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:35:48.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:48 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:35:48.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:48 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:35:48.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:48 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:35:48.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:48 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:35:48.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:48 smithi026 ceph-mon[100112]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-11-16T16:35:49.118 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:49 smithi026 ceph-mon[100112]: purged_snaps scrub starts 2023-11-16T16:35:49.118 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:49 smithi026 ceph-mon[100112]: purged_snaps scrub ok 2023-11-16T16:35:49.118 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:49 smithi026 ceph-mon[100112]: osd.9 [v2:172.21.15.1:6826/1391199844,v1:172.21.15.1:6827/1391199844] boot 2023-11-16T16:35:49.118 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:49 smithi026 ceph-mon[100112]: osdmap e67: 10 total, 10 up, 10 in 2023-11-16T16:35:49.118 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:49 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-16T16:35:49.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:49 smithi053 ceph-mon[100638]: purged_snaps scrub starts 2023-11-16T16:35:49.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:49 smithi053 ceph-mon[100638]: purged_snaps scrub ok 2023-11-16T16:35:49.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:49 smithi053 ceph-mon[100638]: osd.9 [v2:172.21.15.1:6826/1391199844,v1:172.21.15.1:6827/1391199844] boot 2023-11-16T16:35:49.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:49 smithi053 ceph-mon[100638]: osdmap e67: 10 total, 10 up, 10 in 2023-11-16T16:35:49.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:49 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-16T16:35:49.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:49 smithi001 ceph-mon[98800]: purged_snaps scrub starts 2023-11-16T16:35:49.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:49 smithi001 ceph-mon[98800]: purged_snaps scrub ok 2023-11-16T16:35:49.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:49 smithi001 ceph-mon[98800]: osd.9 [v2:172.21.15.1:6826/1391199844,v1:172.21.15.1:6827/1391199844] boot 2023-11-16T16:35:49.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:49 smithi001 ceph-mon[98800]: osdmap e67: 10 total, 10 up, 10 in 2023-11-16T16:35:49.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:49 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-16T16:35:49.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.534+0000 7f62443b5700 1 -- 172.21.15.26:0/3759862626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f623c1002f0 msgr2=0x7f623c100700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:35:49.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.534+0000 7f62443b5700 1 --2- 172.21.15.26:0/3759862626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f623c1002f0 0x7f623c100700 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6238009990 tx=0x7f6238009ca0 comp rx=0 tx=0).stop 2023-11-16T16:35:49.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.534+0000 7f62443b5700 1 -- 172.21.15.26:0/3759862626 shutdown_connections 2023-11-16T16:35:49.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.534+0000 7f62443b5700 1 --2- 172.21.15.26:0/3759862626 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f623c101e80 0x7f623c108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:35:49.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.534+0000 7f62443b5700 1 --2- 172.21.15.26:0/3759862626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f623c1002f0 0x7f623c100700 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:35:49.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.534+0000 7f62443b5700 1 --2- 172.21.15.26:0/3759862626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f623c1014f0 0x7f623c101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:35:49.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.534+0000 7f62443b5700 1 -- 172.21.15.26:0/3759862626 >> 172.21.15.26:0/3759862626 conn(0x7f623c0fb900 msgr2=0x7f623c0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:35:49.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.534+0000 7f62443b5700 1 -- 172.21.15.26:0/3759862626 shutdown_connections 2023-11-16T16:35:49.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.534+0000 7f62443b5700 1 -- 172.21.15.26:0/3759862626 wait complete. 2023-11-16T16:35:49.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.534+0000 7f62443b5700 1 Processor -- start 2023-11-16T16:35:49.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.534+0000 7f62443b5700 1 -- start start 2023-11-16T16:35:49.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.535+0000 7f62443b5700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f623c1002f0 0x7f623c074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:35:49.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.535+0000 7f62443b5700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f623c1014f0 0x7f623c072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:35:49.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.535+0000 7f62443b5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f623c101e80 0x7f623c073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:35:49.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.535+0000 7f62443b5700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c073ba0 con 0x7f623c1002f0 2023-11-16T16:35:49.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.535+0000 7f62443b5700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c073d10 con 0x7f623c101e80 2023-11-16T16:35:49.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.535+0000 7f62443b5700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c1a0160 con 0x7f623c1014f0 2023-11-16T16:35:49.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.535+0000 7f6242952700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f623c101e80 0x7f623c073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:35:49.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.535+0000 7f6242952700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f623c101e80 0x7f623c073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55316/0 (socket says 172.21.15.26:55316) 2023-11-16T16:35:49.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.535+0000 7f6242952700 1 -- 172.21.15.26:0/1847973781 learned_addr learned my addr 172.21.15.26:0/1847973781 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-11-16T16:35:49.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.535+0000 7f6242151700 1 --2- 172.21.15.26:0/1847973781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f623c1002f0 0x7f623c074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:35:49.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.535+0000 7f6241950700 1 --2- 172.21.15.26:0/1847973781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f623c1014f0 0x7f623c072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:35:49.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.535+0000 7f6242952700 1 -- 172.21.15.26:0/1847973781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f623c1014f0 msgr2=0x7f623c072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:35:49.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.535+0000 7f6242952700 1 --2- 172.21.15.26:0/1847973781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f623c1014f0 0x7f623c072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:35:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.535+0000 7f6242952700 1 -- 172.21.15.26:0/1847973781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f623c1002f0 msgr2=0x7f623c074890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:35:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.535+0000 7f6242952700 1 --2- 172.21.15.26:0/1847973781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f623c1002f0 0x7f623c074890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:35:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.535+0000 7f6242952700 1 -- 172.21.15.26:0/1847973781 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62380095f0 con 0x7f623c101e80 2023-11-16T16:35:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.535+0000 7f6242952700 1 --2- 172.21.15.26:0/1847973781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f623c101e80 0x7f623c073420 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f623400b970 tx=0x7f623400bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:35:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.535+0000 7f62337fe700 1 -- 172.21.15.26:0/1847973781 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623400c930 con 0x7f623c101e80 2023-11-16T16:35:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.535+0000 7f62337fe700 1 -- 172.21.15.26:0/1847973781 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6234014440 con 0x7f623c101e80 2023-11-16T16:35:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.535+0000 7f62337fe700 1 -- 172.21.15.26:0/1847973781 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62340075d0 con 0x7f623c101e80 2023-11-16T16:35:49.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.535+0000 7f62443b5700 1 -- 172.21.15.26:0/1847973781 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f623c10bd40 con 0x7f623c101e80 2023-11-16T16:35:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.536+0000 7f62443b5700 1 -- 172.21.15.26:0/1847973781 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f623c10bbb0 con 0x7f623c101e80 2023-11-16T16:35:49.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.537+0000 7f62443b5700 1 -- 172.21.15.26:0/1847973781 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f623c1078e0 con 0x7f623c101e80 2023-11-16T16:35:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.538+0000 7f62337fe700 1 -- 172.21.15.26:0/1847973781 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f62340145b0 con 0x7f623c101e80 2023-11-16T16:35:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.538+0000 7f62337fe700 1 --2- 172.21.15.26:0/1847973781 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f62280a11d0 0x7f62280a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:35:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.538+0000 7f62337fe700 1 -- 172.21.15.26:0/1847973781 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f6234006e40 con 0x7f623c101e80 2023-11-16T16:35:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.538+0000 7f6242151700 1 --2- 172.21.15.26:0/1847973781 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f62280a11d0 0x7f62280a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:35:49.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.539+0000 7f6242151700 1 --2- 172.21.15.26:0/1847973781 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f62280a11d0 0x7f62280a3680 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6238006010 tx=0x7f6238005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:35:49.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.543+0000 7f62337fe700 1 -- 172.21.15.26:0/1847973781 <== mon.2 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f623404e730 con 0x7f623c101e80 2023-11-16T16:35:49.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:35:49.745+0000 7f62443b5700 1 -- 172.21.15.26:0/1847973781 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f623c061c70 con 0x7f62280a11d0 2023-11-16T16:35:50.681 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:50 smithi026 ceph-mon[100112]: osdmap e68: 10 total, 10 up, 10 in 2023-11-16T16:35:50.681 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:50 smithi026 ceph-mon[100112]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:35:50.681 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:35:50.681 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:35:50.681 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:35:50.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:50 smithi001 ceph-mon[98800]: osdmap e68: 10 total, 10 up, 10 in 2023-11-16T16:35:50.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:50 smithi001 ceph-mon[98800]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:35:50.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:50 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:35:50.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:50 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:35:50.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:50 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:35:50.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:50 smithi053 ceph-mon[100638]: osdmap e68: 10 total, 10 up, 10 in 2023-11-16T16:35:50.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:50 smithi053 ceph-mon[100638]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:35:50.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:50 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:35:50.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:50 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:35:50.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:50 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:35:51.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:51 smithi026 ceph-mon[100112]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:35:51.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:51 smithi001 ceph-mon[98800]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:35:51.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:51 smithi053 ceph-mon[100638]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:35:52.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:52 smithi001 ceph-mon[98800]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:35:52.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:52 smithi001 ceph-mon[98800]: from='client.? 172.21.15.26:0/1366728645' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c090e7a-5240-4d52-ae4f-e94fe05383d9"}]: dispatch 2023-11-16T16:35:52.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:52 smithi001 ceph-mon[98800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c090e7a-5240-4d52-ae4f-e94fe05383d9"}]: dispatch 2023-11-16T16:35:52.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:52 smithi001 ceph-mon[98800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c090e7a-5240-4d52-ae4f-e94fe05383d9"}]': finished 2023-11-16T16:35:52.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:52 smithi001 ceph-mon[98800]: osdmap e69: 11 total, 10 up, 11 in 2023-11-16T16:35:52.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:52 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-16T16:35:52.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:52 smithi026 ceph-mon[100112]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:35:52.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:52 smithi026 ceph-mon[100112]: from='client.? 172.21.15.26:0/1366728645' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c090e7a-5240-4d52-ae4f-e94fe05383d9"}]: dispatch 2023-11-16T16:35:52.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:52 smithi026 ceph-mon[100112]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c090e7a-5240-4d52-ae4f-e94fe05383d9"}]: dispatch 2023-11-16T16:35:52.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:52 smithi026 ceph-mon[100112]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c090e7a-5240-4d52-ae4f-e94fe05383d9"}]': finished 2023-11-16T16:35:52.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:52 smithi026 ceph-mon[100112]: osdmap e69: 11 total, 10 up, 11 in 2023-11-16T16:35:52.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:52 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-16T16:35:52.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:52 smithi053 ceph-mon[100638]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:35:52.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:52 smithi053 ceph-mon[100638]: from='client.? 172.21.15.26:0/1366728645' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c090e7a-5240-4d52-ae4f-e94fe05383d9"}]: dispatch 2023-11-16T16:35:52.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:52 smithi053 ceph-mon[100638]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c090e7a-5240-4d52-ae4f-e94fe05383d9"}]: dispatch 2023-11-16T16:35:52.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:52 smithi053 ceph-mon[100638]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c090e7a-5240-4d52-ae4f-e94fe05383d9"}]': finished 2023-11-16T16:35:52.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:52 smithi053 ceph-mon[100638]: osdmap e69: 11 total, 10 up, 11 in 2023-11-16T16:35:52.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:52 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-16T16:35:53.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:53 smithi001 ceph-mon[98800]: from='client.? 172.21.15.26:0/1327041406' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:35:53.757 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:53 smithi026 ceph-mon[100112]: from='client.? 172.21.15.26:0/1327041406' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:35:53.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:53 smithi053 ceph-mon[100638]: from='client.? 172.21.15.26:0/1327041406' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:35:54.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:54 smithi001 ceph-mon[98800]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:35:54.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:54 smithi026 ceph-mon[100112]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:35:54.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:54 smithi053 ceph-mon[100638]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:35:55.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:55 smithi053 ceph-mon[100638]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:35:55.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:55 smithi001 ceph-mon[98800]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:35:56.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:55 smithi026 ceph-mon[100112]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:35:57.697 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:57 smithi026 ceph-mon[100112]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:35:57.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:57 smithi053 ceph-mon[100638]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:35:57.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:57 smithi001 ceph-mon[98800]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:35:59.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:59 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-11-16T16:35:59.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:35:59 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:35:59.330 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:59 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-11-16T16:35:59.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:35:59 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:35:59.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:59 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-11-16T16:35:59.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:35:59 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:00.256 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:00 smithi026 ceph-mon[100112]: Deploying daemon osd.10 on smithi026 2023-11-16T16:36:00.257 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:00 smithi026 ceph-mon[100112]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:36:00.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:00 smithi053 ceph-mon[100638]: Deploying daemon osd.10 on smithi026 2023-11-16T16:36:00.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:00 smithi053 ceph-mon[100638]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:36:00.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:00 smithi001 ceph-mon[98800]: Deploying daemon osd.10 on smithi026 2023-11-16T16:36:00.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:00 smithi001 ceph-mon[98800]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:36:02.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:02 smithi001 ceph-mon[98800]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:36:02.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:02 smithi026 ceph-mon[100112]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:36:02.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:02 smithi053 ceph-mon[100638]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:36:03.575 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:03 smithi026 ceph-mon[100112]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:36:03.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:03 smithi053 ceph-mon[100638]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:36:03.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:03 smithi001 ceph-mon[98800]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:36:04.921 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:04 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:04.921 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:04 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:04.921 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:04 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:36:05.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:04 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:05.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:04 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:05.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:04 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:36:05.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:04 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:05.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:04 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:05.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:04 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:36:06.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:05 smithi026 ceph-mon[100112]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:36:06.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:05 smithi053 ceph-mon[100638]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:36:06.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:05 smithi001 ceph-mon[98800]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:36:06.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:36:06.463+0000 7f62337fe700 1 -- 172.21.15.26:0/1847973781 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f623c061c70 con 0x7f62280a11d0 2023-11-16T16:36:06.466 INFO:teuthology.orchestra.run.smithi026.stdout:Created osd(s) 10 on host 'smithi026' 2023-11-16T16:36:06.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:36:06.465+0000 7f62443b5700 1 -- 172.21.15.26:0/1847973781 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f62280a11d0 msgr2=0x7f62280a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:06.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:36:06.465+0000 7f62443b5700 1 --2- 172.21.15.26:0/1847973781 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f62280a11d0 0x7f62280a3680 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6238006010 tx=0x7f6238005960 comp rx=0 tx=0).stop 2023-11-16T16:36:06.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:36:06.465+0000 7f62443b5700 1 -- 172.21.15.26:0/1847973781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f623c101e80 msgr2=0x7f623c073420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:06.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:36:06.465+0000 7f62443b5700 1 --2- 172.21.15.26:0/1847973781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f623c101e80 0x7f623c073420 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f623400b970 tx=0x7f623400bc80 comp rx=0 tx=0).stop 2023-11-16T16:36:06.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:36:06.465+0000 7f62443b5700 1 -- 172.21.15.26:0/1847973781 shutdown_connections 2023-11-16T16:36:06.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:36:06.465+0000 7f62443b5700 1 --2- 172.21.15.26:0/1847973781 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f62280a11d0 0x7f62280a3680 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:06.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:36:06.465+0000 7f62443b5700 1 --2- 172.21.15.26:0/1847973781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f623c101e80 0x7f623c073420 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:06.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:36:06.465+0000 7f62443b5700 1 --2- 172.21.15.26:0/1847973781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f623c1002f0 0x7f623c074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:06.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:36:06.465+0000 7f62443b5700 1 --2- 172.21.15.26:0/1847973781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f623c1014f0 0x7f623c072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:06.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:36:06.465+0000 7f62443b5700 1 -- 172.21.15.26:0/1847973781 >> 172.21.15.26:0/1847973781 conn(0x7f623c0fb900 msgr2=0x7f623c104a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:36:06.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:36:06.465+0000 7f62443b5700 1 -- 172.21.15.26:0/1847973781 shutdown_connections 2023-11-16T16:36:06.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-11-16T16:36:06.465+0000 7f62443b5700 1 -- 172.21.15.26:0/1847973781 wait complete. 2023-11-16T16:36:06.903 DEBUG:teuthology.orchestra.run.smithi026:osd.10> sudo journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@osd.10.service 2023-11-16T16:36:06.906 INFO:tasks.cephadm:Deploying osd.11 on smithi053 with /dev/vg_nvme/lv_1... 2023-11-16T16:36:06.906 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-11-16T16:36:06.935 INFO:journalctl@ceph.osd.10.smithi026.stdout:-- Logs begin at Thu 2023-11-16 16:14:13 UTC. -- 2023-11-16T16:36:07.570 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-16T16:36:07.742 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:07 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:07.742 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:07 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:07.742 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:07 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:07.743 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:07 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:07.769 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:07 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:07.769 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:07 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:07.769 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:07 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:07.769 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:07 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:07.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:07 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:07.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:07 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:07.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:07 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:07.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:07 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:08.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:36:07 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:36:07.739+0000 7f4bbe4f0640 -1 osd.10 0 log_to_monitors true 2023-11-16T16:36:08.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:08 smithi026 ceph-mon[100112]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:36:08.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:08 smithi026 ceph-mon[100112]: from='osd.10 [v2:172.21.15.26:6824/1773545447,v1:172.21.15.26:6825/1773545447]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-16T16:36:08.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:08 smithi026 ceph-mon[100112]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-16T16:36:08.702 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:08 smithi053 ceph-mon[100638]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:36:08.702 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:08 smithi053 ceph-mon[100638]: from='osd.10 [v2:172.21.15.26:6824/1773545447,v1:172.21.15.26:6825/1773545447]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-16T16:36:08.702 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:08 smithi053 ceph-mon[100638]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-16T16:36:08.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:08 smithi001 ceph-mon[98800]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:36:08.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:08 smithi001 ceph-mon[98800]: from='osd.10 [v2:172.21.15.26:6824/1773545447,v1:172.21.15.26:6825/1773545447]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-16T16:36:08.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:08 smithi001 ceph-mon[98800]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-16T16:36:09.159 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-11-16T16:36:09.159 INFO:teuthology.orchestra.run.smithi053.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-11-16T16:36:09.159 INFO:teuthology.orchestra.run.smithi053.stderr: stderr: 10+0 records in 2023-11-16T16:36:09.159 INFO:teuthology.orchestra.run.smithi053.stderr:10+0 records out 2023-11-16T16:36:09.159 INFO:teuthology.orchestra.run.smithi053.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00802675 s, 1.3 GB/s 2023-11-16T16:36:09.160 INFO:teuthology.orchestra.run.smithi053.stderr:--> Zapping successful for: 2023-11-16T16:36:09.636 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph orch daemon add osd smithi053:vg_nvme/lv_1 2023-11-16T16:36:09.758 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:36:09 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:36:09.448+0000 7f4bb3b67700 -1 osd.10 0 waiting for initial osdmap 2023-11-16T16:36:09.758 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:36:09 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:36:09.462+0000 7f4bab954700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-16T16:36:09.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:09 smithi026 ceph-mon[100112]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-11-16T16:36:09.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:09 smithi026 ceph-mon[100112]: osdmap e70: 11 total, 10 up, 11 in 2023-11-16T16:36:09.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:09 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-16T16:36:09.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:09 smithi026 ceph-mon[100112]: from='osd.10 [v2:172.21.15.26:6824/1773545447,v1:172.21.15.26:6825/1773545447]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-11-16T16:36:09.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:09 smithi026 ceph-mon[100112]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-11-16T16:36:09.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:09 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:09.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:09 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:09.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:09 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:36:09.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:09 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:36:09.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:09 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:36:09.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:09 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:36:09.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:09 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:09.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:09 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:09.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:09 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:36:09.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:09 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:09.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:09 smithi053 ceph-mon[100638]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-11-16T16:36:09.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:09 smithi053 ceph-mon[100638]: osdmap e70: 11 total, 10 up, 11 in 2023-11-16T16:36:09.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:09 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-16T16:36:09.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:09 smithi053 ceph-mon[100638]: from='osd.10 [v2:172.21.15.26:6824/1773545447,v1:172.21.15.26:6825/1773545447]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-11-16T16:36:09.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:09 smithi053 ceph-mon[100638]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-11-16T16:36:09.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:09 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:09.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:09 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:09.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:09 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:36:09.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:09 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:36:09.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:09 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:36:09.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:09 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:36:09.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:09 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:09.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:09 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:09.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:09 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:36:09.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:09 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:09.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:09 smithi001 ceph-mon[98800]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-11-16T16:36:09.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:09 smithi001 ceph-mon[98800]: osdmap e70: 11 total, 10 up, 11 in 2023-11-16T16:36:09.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:09 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-16T16:36:09.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:09 smithi001 ceph-mon[98800]: from='osd.10 [v2:172.21.15.26:6824/1773545447,v1:172.21.15.26:6825/1773545447]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-11-16T16:36:09.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:09 smithi001 ceph-mon[98800]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-11-16T16:36:09.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:09 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:09.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:09 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:09.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:09 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:36:09.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:09 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:36:09.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:09 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:36:09.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:09 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:36:09.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:09 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:09.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:09 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:09.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:09 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:36:09.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:09 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:10.290 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-16T16:36:10.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:10 smithi026 ceph-mon[100112]: Detected new or changed devices on smithi026 2023-11-16T16:36:10.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:10 smithi026 ceph-mon[100112]: Adjusting osd_memory_target on smithi026 to 4293M 2023-11-16T16:36:10.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:10 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:36:10.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:10 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:36:10.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:10 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:36:10.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:10 smithi026 ceph-mon[100112]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2023-11-16T16:36:10.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:10 smithi026 ceph-mon[100112]: osdmap e71: 11 total, 10 up, 11 in 2023-11-16T16:36:10.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:10 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-16T16:36:10.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:10 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-16T16:36:10.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:10 smithi026 ceph-mon[100112]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:36:10.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:10 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:36:10.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:10 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:10.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:10 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:36:10.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:10 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:10.762 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:10 smithi053 ceph-mon[100638]: Detected new or changed devices on smithi026 2023-11-16T16:36:10.762 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:10 smithi053 ceph-mon[100638]: Adjusting osd_memory_target on smithi026 to 4293M 2023-11-16T16:36:10.762 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:10 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:36:10.762 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:10 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:36:10.762 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:10 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:36:10.763 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:10 smithi053 ceph-mon[100638]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2023-11-16T16:36:10.763 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:10 smithi053 ceph-mon[100638]: osdmap e71: 11 total, 10 up, 11 in 2023-11-16T16:36:10.763 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:10 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-16T16:36:10.763 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:10 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-16T16:36:10.763 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:10 smithi053 ceph-mon[100638]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:36:10.763 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:10 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:36:10.763 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:10 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:10.764 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:10 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:36:10.764 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:10 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:10.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:10 smithi001 ceph-mon[98800]: Detected new or changed devices on smithi026 2023-11-16T16:36:10.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:10 smithi001 ceph-mon[98800]: Adjusting osd_memory_target on smithi026 to 4293M 2023-11-16T16:36:10.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:10 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:36:10.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:10 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:36:10.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:10 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:36:10.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:10 smithi001 ceph-mon[98800]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2023-11-16T16:36:10.934 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:10 smithi001 ceph-mon[98800]: osdmap e71: 11 total, 10 up, 11 in 2023-11-16T16:36:10.934 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:10 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-16T16:36:10.934 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:10 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-16T16:36:10.934 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:10 smithi001 ceph-mon[98800]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-16T16:36:10.935 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:10 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:36:10.935 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:10 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:10.935 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:10 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:36:10.935 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:10 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:11.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.408+0000 7f1fc3168700 1 -- 172.21.15.53:0/1133851619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fbc103690 msgr2=0x7f1fbc105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:11.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.408+0000 7f1fc3168700 1 --2- 172.21.15.53:0/1133851619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fbc103690 0x7f1fbc105a70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8009990 tx=0x7f1fa8009ca0 comp rx=0 tx=0).stop 2023-11-16T16:36:11.411 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.408+0000 7f1fc3168700 1 -- 172.21.15.53:0/1133851619 shutdown_connections 2023-11-16T16:36:11.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.408+0000 7f1fc3168700 1 --2- 172.21.15.53:0/1133851619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fbc105fb0 0x7f1fbc108410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:11.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.408+0000 7f1fc3168700 1 --2- 172.21.15.53:0/1133851619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fbc100d70 0x7f1fbc103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:11.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.408+0000 7f1fc3168700 1 --2- 172.21.15.53:0/1133851619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fbc103690 0x7f1fbc105a70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:11.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.408+0000 7f1fc3168700 1 -- 172.21.15.53:0/1133851619 >> 172.21.15.53:0/1133851619 conn(0x7f1fbc0fa7f0 msgr2=0x7f1fbc0fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:36:11.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.408+0000 7f1fc3168700 1 -- 172.21.15.53:0/1133851619 shutdown_connections 2023-11-16T16:36:11.412 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.408+0000 7f1fc3168700 1 -- 172.21.15.53:0/1133851619 wait complete. 2023-11-16T16:36:11.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.409+0000 7f1fc3168700 1 Processor -- start 2023-11-16T16:36:11.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.409+0000 7f1fc3168700 1 -- start start 2023-11-16T16:36:11.415 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.409+0000 7f1fc3168700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fbc100d70 0x7f1fbc1960b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:11.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.409+0000 7f1fc3168700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fbc103690 0x7f1fbc1965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:11.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.409+0000 7f1fc3168700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fbc105fb0 0x7f1fbc19d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:11.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.409+0000 7f1fc3168700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fbc19dc40 con 0x7f1fbc100d70 2023-11-16T16:36:11.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.409+0000 7f1fc3168700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fbc19dd80 con 0x7f1fbc105fb0 2023-11-16T16:36:11.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.409+0000 7f1fc3168700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fbc19df10 con 0x7f1fbc103690 2023-11-16T16:36:11.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.409+0000 7f1fbbfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fbc103690 0x7f1fbc1965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:11.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.409+0000 7f1fbbfff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fbc103690 0x7f1fbc1965f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:57938/0 (socket says 172.21.15.53:57938) 2023-11-16T16:36:11.416 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.409+0000 7f1fbbfff700 1 -- 172.21.15.53:0/4069568068 learned_addr learned my addr 172.21.15.53:0/4069568068 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-16T16:36:11.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.410+0000 7f1fc1705700 1 --2- 172.21.15.53:0/4069568068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fbc105fb0 0x7f1fbc19d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:11.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.410+0000 7f1fc0f04700 1 --2- 172.21.15.53:0/4069568068 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fbc100d70 0x7f1fbc1960b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:11.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.410+0000 7f1fbbfff700 1 -- 172.21.15.53:0/4069568068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fbc105fb0 msgr2=0x7f1fbc19d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:11.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.410+0000 7f1fbbfff700 1 --2- 172.21.15.53:0/4069568068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fbc105fb0 0x7f1fbc19d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:11.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.410+0000 7f1fbbfff700 1 -- 172.21.15.53:0/4069568068 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fbc100d70 msgr2=0x7f1fbc1960b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:11.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.410+0000 7f1fbbfff700 1 --2- 172.21.15.53:0/4069568068 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fbc100d70 0x7f1fbc1960b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:11.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.410+0000 7f1fbbfff700 1 -- 172.21.15.53:0/4069568068 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa80095f0 con 0x7f1fbc103690 2023-11-16T16:36:11.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.410+0000 7f1fc0f04700 1 --2- 172.21.15.53:0/4069568068 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fbc100d70 0x7f1fbc1960b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:36:11.417 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.410+0000 7f1fbbfff700 1 --2- 172.21.15.53:0/4069568068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fbc103690 0x7f1fbc1965f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8006010 tx=0x7f1fa8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:36:11.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.410+0000 7f1fb9ffb700 1 -- 172.21.15.53:0/4069568068 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa801d070 con 0x7f1fbc103690 2023-11-16T16:36:11.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.410+0000 7f1fb9ffb700 1 -- 172.21.15.53:0/4069568068 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fa800fb80 con 0x7f1fbc103690 2023-11-16T16:36:11.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.410+0000 7f1fc3168700 1 -- 172.21.15.53:0/4069568068 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fbc10bcd0 con 0x7f1fbc103690 2023-11-16T16:36:11.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.410+0000 7f1fb9ffb700 1 -- 172.21.15.53:0/4069568068 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa8017650 con 0x7f1fbc103690 2023-11-16T16:36:11.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.410+0000 7f1fc3168700 1 -- 172.21.15.53:0/4069568068 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fbc10bb40 con 0x7f1fbc103690 2023-11-16T16:36:11.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.411+0000 7f1fc3168700 1 -- 172.21.15.53:0/4069568068 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fbc0fc360 con 0x7f1fbc103690 2023-11-16T16:36:11.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.412+0000 7f1fb9ffb700 1 -- 172.21.15.53:0/4069568068 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f1fa8021410 con 0x7f1fbc103690 2023-11-16T16:36:11.418 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.413+0000 7f1fb9ffb700 1 --2- 172.21.15.53:0/4069568068 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f1fac0a1150 0x7f1fac0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:11.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.413+0000 7f1fb9ffb700 1 -- 172.21.15.53:0/4069568068 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f1fa80cd440 con 0x7f1fbc103690 2023-11-16T16:36:11.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.413+0000 7f1fc0f04700 1 --2- 172.21.15.53:0/4069568068 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f1fac0a1150 0x7f1fac0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:11.419 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.414+0000 7f1fc0f04700 1 --2- 172.21.15.53:0/4069568068 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f1fac0a1150 0x7f1fac0a3600 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0009be0 tx=0x7f1fb0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:36:11.420 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.417+0000 7f1fb9ffb700 1 -- 172.21.15.53:0/4069568068 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1fa8097250 con 0x7f1fbc103690 2023-11-16T16:36:11.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:11 smithi053 ceph-mon[100638]: purged_snaps scrub starts 2023-11-16T16:36:11.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:11 smithi053 ceph-mon[100638]: purged_snaps scrub ok 2023-11-16T16:36:11.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:11 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:36:11.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:11 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:36:11.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:11 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:36:11.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:11 smithi053 ceph-mon[100638]: osd.10 [v2:172.21.15.26:6824/1773545447,v1:172.21.15.26:6825/1773545447] boot 2023-11-16T16:36:11.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:11 smithi053 ceph-mon[100638]: osdmap e72: 11 total, 11 up, 11 in 2023-11-16T16:36:11.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:11 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-16T16:36:11.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:11 smithi053 ceph-mon[100638]: osdmap e73: 11 total, 11 up, 11 in 2023-11-16T16:36:11.626 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:11.624+0000 7f1fc3168700 1 -- 172.21.15.53:0/4069568068 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f1fbc061a20 con 0x7f1fac0a1150 2023-11-16T16:36:11.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:11 smithi026 ceph-mon[100112]: purged_snaps scrub starts 2023-11-16T16:36:11.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:11 smithi026 ceph-mon[100112]: purged_snaps scrub ok 2023-11-16T16:36:11.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:11 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:36:11.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:11 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:36:11.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:11 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:36:11.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:11 smithi026 ceph-mon[100112]: osd.10 [v2:172.21.15.26:6824/1773545447,v1:172.21.15.26:6825/1773545447] boot 2023-11-16T16:36:11.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:11 smithi026 ceph-mon[100112]: osdmap e72: 11 total, 11 up, 11 in 2023-11-16T16:36:11.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:11 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-16T16:36:11.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:11 smithi026 ceph-mon[100112]: osdmap e73: 11 total, 11 up, 11 in 2023-11-16T16:36:11.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:11 smithi001 ceph-mon[98800]: purged_snaps scrub starts 2023-11-16T16:36:11.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:11 smithi001 ceph-mon[98800]: purged_snaps scrub ok 2023-11-16T16:36:11.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:11 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:36:11.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:11 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:36:11.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:11 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:36:11.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:11 smithi001 ceph-mon[98800]: osd.10 [v2:172.21.15.26:6824/1773545447,v1:172.21.15.26:6825/1773545447] boot 2023-11-16T16:36:11.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:11 smithi001 ceph-mon[98800]: osdmap e72: 11 total, 11 up, 11 in 2023-11-16T16:36:11.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:11 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-16T16:36:11.934 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:11 smithi001 ceph-mon[98800]: osdmap e73: 11 total, 11 up, 11 in 2023-11-16T16:36:12.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:12 smithi026 ceph-mon[100112]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 65 MiB used, 983 GiB / 983 GiB avail 2023-11-16T16:36:12.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:12 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:36:12.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:12 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:36:12.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:12 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:12.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:12 smithi026 ceph-mon[100112]: osdmap e74: 11 total, 11 up, 11 in 2023-11-16T16:36:12.779 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:12 smithi053 ceph-mon[100638]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 65 MiB used, 983 GiB / 983 GiB avail 2023-11-16T16:36:12.779 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:12 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:36:12.779 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:12 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:36:12.779 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:12 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:12.779 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:12 smithi053 ceph-mon[100638]: osdmap e74: 11 total, 11 up, 11 in 2023-11-16T16:36:12.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:12 smithi001 ceph-mon[98800]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 65 MiB used, 983 GiB / 983 GiB avail 2023-11-16T16:36:12.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:12 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-16T16:36:12.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:12 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-16T16:36:12.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:12 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:12.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:12 smithi001 ceph-mon[98800]: osdmap e74: 11 total, 11 up, 11 in 2023-11-16T16:36:13.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:13 smithi053 ceph-mon[100638]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:36:13.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:13 smithi026 ceph-mon[100112]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:36:13.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:13 smithi001 ceph-mon[98800]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi053:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:36:14.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:14 smithi026 ceph-mon[100112]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 65 MiB used, 983 GiB / 983 GiB avail 2023-11-16T16:36:14.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:14 smithi026 ceph-mon[100112]: osdmap e75: 11 total, 11 up, 11 in 2023-11-16T16:36:14.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:14 smithi026 ceph-mon[100112]: from='client.? 172.21.15.53:0/1115732053' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4511e32d-e469-45e1-8310-0ff400cb7b54"}]: dispatch 2023-11-16T16:36:14.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:14 smithi026 ceph-mon[100112]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4511e32d-e469-45e1-8310-0ff400cb7b54"}]: dispatch 2023-11-16T16:36:14.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:14 smithi026 ceph-mon[100112]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4511e32d-e469-45e1-8310-0ff400cb7b54"}]': finished 2023-11-16T16:36:14.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:14 smithi026 ceph-mon[100112]: osdmap e76: 12 total, 11 up, 12 in 2023-11-16T16:36:14.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:14 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-16T16:36:14.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:14 smithi053 ceph-mon[100638]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 65 MiB used, 983 GiB / 983 GiB avail 2023-11-16T16:36:14.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:14 smithi053 ceph-mon[100638]: osdmap e75: 11 total, 11 up, 11 in 2023-11-16T16:36:14.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:14 smithi053 ceph-mon[100638]: from='client.? 172.21.15.53:0/1115732053' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4511e32d-e469-45e1-8310-0ff400cb7b54"}]: dispatch 2023-11-16T16:36:14.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:14 smithi053 ceph-mon[100638]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4511e32d-e469-45e1-8310-0ff400cb7b54"}]: dispatch 2023-11-16T16:36:14.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:14 smithi053 ceph-mon[100638]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4511e32d-e469-45e1-8310-0ff400cb7b54"}]': finished 2023-11-16T16:36:14.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:14 smithi053 ceph-mon[100638]: osdmap e76: 12 total, 11 up, 12 in 2023-11-16T16:36:14.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:14 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-16T16:36:14.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:14 smithi001 ceph-mon[98800]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 65 MiB used, 983 GiB / 983 GiB avail 2023-11-16T16:36:14.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:14 smithi001 ceph-mon[98800]: osdmap e75: 11 total, 11 up, 11 in 2023-11-16T16:36:14.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:14 smithi001 ceph-mon[98800]: from='client.? 172.21.15.53:0/1115732053' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4511e32d-e469-45e1-8310-0ff400cb7b54"}]: dispatch 2023-11-16T16:36:14.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:14 smithi001 ceph-mon[98800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4511e32d-e469-45e1-8310-0ff400cb7b54"}]: dispatch 2023-11-16T16:36:14.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:14 smithi001 ceph-mon[98800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4511e32d-e469-45e1-8310-0ff400cb7b54"}]': finished 2023-11-16T16:36:14.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:14 smithi001 ceph-mon[98800]: osdmap e76: 12 total, 11 up, 12 in 2023-11-16T16:36:14.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:14 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-16T16:36:15.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:15 smithi026 ceph-mon[100112]: from='client.? 172.21.15.53:0/481945056' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:36:15.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:15 smithi053 ceph-mon[100638]: from='client.? 172.21.15.53:0/481945056' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:36:15.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:15 smithi001 ceph-mon[98800]: from='client.? 172.21.15.53:0/481945056' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-16T16:36:16.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:16 smithi026 ceph-mon[100112]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail; 134 KiB/s, 0 objects/s recovering 2023-11-16T16:36:16.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:16 smithi053 ceph-mon[100638]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail; 134 KiB/s, 0 objects/s recovering 2023-11-16T16:36:16.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:16 smithi001 ceph-mon[98800]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail; 134 KiB/s, 0 objects/s recovering 2023-11-16T16:36:17.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:17 smithi026 ceph-mon[100112]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-11-16T16:36:17.789 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:17 smithi053 ceph-mon[100638]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-11-16T16:36:17.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:17 smithi001 ceph-mon[98800]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-11-16T16:36:19.647 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:19 smithi053 ceph-mon[100638]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2023-11-16T16:36:19.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:19 smithi001 ceph-mon[98800]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2023-11-16T16:36:20.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:19 smithi026 ceph-mon[100112]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2023-11-16T16:36:21.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:20 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-11-16T16:36:21.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:20 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:21.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:20 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-11-16T16:36:21.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:20 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:21.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:20 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-11-16T16:36:21.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:20 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:22.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:22 smithi001 ceph-mon[98800]: Deploying daemon osd.11 on smithi053 2023-11-16T16:36:22.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:22 smithi001 ceph-mon[98800]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-16T16:36:22.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:22 smithi026 ceph-mon[100112]: Deploying daemon osd.11 on smithi053 2023-11-16T16:36:22.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:22 smithi026 ceph-mon[100112]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-16T16:36:22.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:22 smithi053 ceph-mon[100638]: Deploying daemon osd.11 on smithi053 2023-11-16T16:36:22.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:22 smithi053 ceph-mon[100638]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-16T16:36:23.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:23 smithi053 ceph-mon[100638]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-11-16T16:36:23.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:23 smithi001 ceph-mon[98800]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-11-16T16:36:24.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:23 smithi026 ceph-mon[100112]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-11-16T16:36:26.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:26 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:26.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:26 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:26.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:26 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:36:26.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:26 smithi053 ceph-mon[100638]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-11-16T16:36:26.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:26 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:26.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:26 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:26.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:26 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:36:26.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:26 smithi001 ceph-mon[98800]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-11-16T16:36:26.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:26 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:26.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:26 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:26.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:26 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:36:26.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:26 smithi026 ceph-mon[100112]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-11-16T16:36:27.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:27 smithi053 ceph-mon[100638]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-11-16T16:36:27.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:27 smithi001 ceph-mon[98800]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-11-16T16:36:28.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:27 smithi026 ceph-mon[100112]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-11-16T16:36:28.098 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:28.096+0000 7f1fb9ffb700 1 -- 172.21.15.53:0/4069568068 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f1fbc061a20 con 0x7f1fac0a1150 2023-11-16T16:36:28.098 INFO:teuthology.orchestra.run.smithi053.stdout:Created osd(s) 11 on host 'smithi053' 2023-11-16T16:36:28.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:28.098+0000 7f1fc3168700 1 -- 172.21.15.53:0/4069568068 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f1fac0a1150 msgr2=0x7f1fac0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:28.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:28.098+0000 7f1fc3168700 1 --2- 172.21.15.53:0/4069568068 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f1fac0a1150 0x7f1fac0a3600 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0009be0 tx=0x7f1fb0009260 comp rx=0 tx=0).stop 2023-11-16T16:36:28.100 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:28.098+0000 7f1fc3168700 1 -- 172.21.15.53:0/4069568068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fbc103690 msgr2=0x7f1fbc1965f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:28.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:28.098+0000 7f1fc3168700 1 --2- 172.21.15.53:0/4069568068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fbc103690 0x7f1fbc1965f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8006010 tx=0x7f1fa8004730 comp rx=0 tx=0).stop 2023-11-16T16:36:28.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:28.098+0000 7f1fc3168700 1 -- 172.21.15.53:0/4069568068 shutdown_connections 2023-11-16T16:36:28.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:28.098+0000 7f1fc3168700 1 --2- 172.21.15.53:0/4069568068 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f1fac0a1150 0x7f1fac0a3600 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:28.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:28.098+0000 7f1fc3168700 1 --2- 172.21.15.53:0/4069568068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fbc105fb0 0x7f1fbc19d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:28.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:28.098+0000 7f1fc3168700 1 --2- 172.21.15.53:0/4069568068 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1fbc100d70 0x7f1fbc1960b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:28.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:28.098+0000 7f1fc3168700 1 --2- 172.21.15.53:0/4069568068 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1fbc103690 0x7f1fbc1965f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:28.101 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:28.098+0000 7f1fc3168700 1 -- 172.21.15.53:0/4069568068 >> 172.21.15.53:0/4069568068 conn(0x7f1fbc0fa7f0 msgr2=0x7f1fbc1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:36:28.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:28.098+0000 7f1fc3168700 1 -- 172.21.15.53:0/4069568068 shutdown_connections 2023-11-16T16:36:28.102 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:28.098+0000 7f1fc3168700 1 -- 172.21.15.53:0/4069568068 wait complete. 2023-11-16T16:36:28.488 DEBUG:teuthology.orchestra.run.smithi053:osd.11> sudo journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@osd.11.service 2023-11-16T16:36:28.490 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-11-16T16:36:28.491 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph osd stat -f json 2023-11-16T16:36:28.520 INFO:journalctl@ceph.osd.11.smithi053.stdout:-- Logs begin at Thu 2023-11-16 16:14:15 UTC. -- 2023-11-16T16:36:28.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:28 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:28.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:28 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:28.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:28 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:28.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:28 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:36:28.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:28 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:36:28.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:28 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:28.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:28 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:36:28.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:28 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:36:28.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:28 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:28.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:28 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:28.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:28 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:36:28.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:28 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:28.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:28 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:28.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:28 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:28.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:28 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:36:28.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:28 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:36:28.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:28 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:28.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:28 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:36:28.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:28 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:36:28.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:28 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:28.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:28 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:28.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:28 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:36:29.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:28 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:29.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:28 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:29.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:28 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:29.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:28 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:36:29.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:28 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:36:29.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:28 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:29.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:28 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:36:29.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:28 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:36:29.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:28 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:29.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:28 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:29.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:28 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:36:29.157 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:36:29.309 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:36:29 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:36:29.079+0000 7f22efeda640 -1 osd.11 0 log_to_monitors true 2023-11-16T16:36:29.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:29 smithi053 ceph-mon[100638]: from='osd.11 [v2:172.21.15.53:6824/2564669546,v1:172.21.15.53:6825/2564669546]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-16T16:36:29.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:29 smithi053 ceph-mon[100638]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-16T16:36:29.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:29 smithi053 ceph-mon[100638]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-11-16T16:36:30.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:29 smithi026 ceph-mon[100112]: from='osd.11 [v2:172.21.15.53:6824/2564669546,v1:172.21.15.53:6825/2564669546]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-16T16:36:30.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:29 smithi026 ceph-mon[100112]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-16T16:36:30.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:29 smithi026 ceph-mon[100112]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-11-16T16:36:30.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:29 smithi001 ceph-mon[98800]: from='osd.11 [v2:172.21.15.53:6824/2564669546,v1:172.21.15.53:6825/2564669546]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-16T16:36:30.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:29 smithi001 ceph-mon[98800]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-16T16:36:30.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:29 smithi001 ceph-mon[98800]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-11-16T16:36:30.490 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.486+0000 7f43d2376700 1 -- 172.21.15.1:0/3337069336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43cc100310 msgr2=0x7f43cc100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:30.490 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.486+0000 7f43d2376700 1 --2- 172.21.15.1:0/3337069336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43cc100310 0x7f43cc100720 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f43bc009990 tx=0x7f43bc009ca0 comp rx=0 tx=0).stop 2023-11-16T16:36:30.490 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.486+0000 7f43d2376700 1 -- 172.21.15.1:0/3337069336 shutdown_connections 2023-11-16T16:36:30.490 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.486+0000 7f43d2376700 1 --2- 172.21.15.1:0/3337069336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43cc101ea0 0x7f43cc108760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:30.490 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.486+0000 7f43d2376700 1 --2- 172.21.15.1:0/3337069336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43cc100310 0x7f43cc100720 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:30.490 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.486+0000 7f43d2376700 1 --2- 172.21.15.1:0/3337069336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43cc101510 0x7f43cc101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:30.490 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.486+0000 7f43d2376700 1 -- 172.21.15.1:0/3337069336 >> 172.21.15.1:0/3337069336 conn(0x7f43cc0fb960 msgr2=0x7f43cc0fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:36:30.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.486+0000 7f43d2376700 1 -- 172.21.15.1:0/3337069336 shutdown_connections 2023-11-16T16:36:30.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.486+0000 7f43d2376700 1 -- 172.21.15.1:0/3337069336 wait complete. 2023-11-16T16:36:30.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.486+0000 7f43d2376700 1 Processor -- start 2023-11-16T16:36:30.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.486+0000 7f43d2376700 1 -- start start 2023-11-16T16:36:30.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.487+0000 7f43d2376700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43cc100310 0x7f43cc196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:30.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.487+0000 7f43d2376700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43cc101510 0x7f43cc196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:30.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.487+0000 7f43d2376700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43cc101ea0 0x7f43cc19d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:30.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.487+0000 7f43d2376700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc19db90 con 0x7f43cc101ea0 2023-11-16T16:36:30.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.487+0000 7f43d2376700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc19dcd0 con 0x7f43cc101510 2023-11-16T16:36:30.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.487+0000 7f43d2376700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc19de60 con 0x7f43cc100310 2023-11-16T16:36:30.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.487+0000 7f43d0913700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43cc101ea0 0x7f43cc19d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:30.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.487+0000 7f43d0913700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43cc101ea0 0x7f43cc19d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41048/0 (socket says 172.21.15.1:41048) 2023-11-16T16:36:30.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.487+0000 7f43d0913700 1 -- 172.21.15.1:0/1178173166 learned_addr learned my addr 172.21.15.1:0/1178173166 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:36:30.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.487+0000 7f43cbfff700 1 --2- 172.21.15.1:0/1178173166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43cc100310 0x7f43cc196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:30.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.487+0000 7f43cb7fe700 1 --2- 172.21.15.1:0/1178173166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43cc101510 0x7f43cc196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:30.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.487+0000 7f43d0913700 1 -- 172.21.15.1:0/1178173166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43cc100310 msgr2=0x7f43cc196000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:30.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.487+0000 7f43d0913700 1 --2- 172.21.15.1:0/1178173166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43cc100310 0x7f43cc196000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:30.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.487+0000 7f43d0913700 1 -- 172.21.15.1:0/1178173166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43cc101510 msgr2=0x7f43cc196540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:30.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.487+0000 7f43d0913700 1 --2- 172.21.15.1:0/1178173166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43cc101510 0x7f43cc196540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:30.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.487+0000 7f43d0913700 1 -- 172.21.15.1:0/1178173166 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43bc0095f0 con 0x7f43cc101ea0 2023-11-16T16:36:30.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.487+0000 7f43d0913700 1 --2- 172.21.15.1:0/1178173166 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43cc101ea0 0x7f43cc19d5c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f43c4007f10 tx=0x7f43c400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:36:30.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.488+0000 7f43c97fa700 1 -- 172.21.15.1:0/1178173166 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c4017070 con 0x7f43cc101ea0 2023-11-16T16:36:30.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.488+0000 7f43c97fa700 1 -- 172.21.15.1:0/1178173166 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43c400f040 con 0x7f43cc101ea0 2023-11-16T16:36:30.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.488+0000 7f43d2376700 1 -- 172.21.15.1:0/1178173166 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43cc10bc60 con 0x7f43cc101ea0 2023-11-16T16:36:30.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.488+0000 7f43c97fa700 1 -- 172.21.15.1:0/1178173166 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c4005740 con 0x7f43cc101ea0 2023-11-16T16:36:30.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.488+0000 7f43d2376700 1 -- 172.21.15.1:0/1178173166 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43cc10bad0 con 0x7f43cc101ea0 2023-11-16T16:36:30.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.489+0000 7f43d2376700 1 -- 172.21.15.1:0/1178173166 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43cc1078b0 con 0x7f43cc101ea0 2023-11-16T16:36:30.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.490+0000 7f43c97fa700 1 -- 172.21.15.1:0/1178173166 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f43c40058a0 con 0x7f43cc101ea0 2023-11-16T16:36:30.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.490+0000 7f43c97fa700 1 --2- 172.21.15.1:0/1178173166 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f43b40a1150 0x7f43b40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:30.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.490+0000 7f43cbfff700 1 --2- 172.21.15.1:0/1178173166 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f43b40a1150 0x7f43b40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:30.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.490+0000 7f43c97fa700 1 -- 172.21.15.1:0/1178173166 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f43c40cbc40 con 0x7f43cc101ea0 2023-11-16T16:36:30.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.491+0000 7f43cbfff700 1 --2- 172.21.15.1:0/1178173166 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f43b40a1150 0x7f43b40a3600 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f43bc006010 tx=0x7f43bc00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:36:30.496 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.495+0000 7f43c97fa700 1 -- 172.21.15.1:0/1178173166 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f43c4095960 con 0x7f43cc101ea0 2023-11-16T16:36:30.700 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.698+0000 7f43d2376700 1 -- 172.21.15.1:0/1178173166 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f43cc02cfe0 con 0x7f43cc101ea0 2023-11-16T16:36:30.700 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.699+0000 7f43c97fa700 1 -- 172.21.15.1:0/1178173166 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v77) v1 ==== 74+0+133 (secure 0 0 0) 0x7f43c4098f80 con 0x7f43cc101ea0 2023-11-16T16:36:30.701 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:36:30.703 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.701+0000 7f43d2376700 1 -- 172.21.15.1:0/1178173166 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f43b40a1150 msgr2=0x7f43b40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:30.703 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.701+0000 7f43d2376700 1 --2- 172.21.15.1:0/1178173166 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f43b40a1150 0x7f43b40a3600 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f43bc006010 tx=0x7f43bc00b540 comp rx=0 tx=0).stop 2023-11-16T16:36:30.703 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.701+0000 7f43d2376700 1 -- 172.21.15.1:0/1178173166 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43cc101ea0 msgr2=0x7f43cc19d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:30.703 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.701+0000 7f43d2376700 1 --2- 172.21.15.1:0/1178173166 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43cc101ea0 0x7f43cc19d5c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f43c4007f10 tx=0x7f43c400d3b0 comp rx=0 tx=0).stop 2023-11-16T16:36:30.703 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.701+0000 7f43d2376700 1 -- 172.21.15.1:0/1178173166 shutdown_connections 2023-11-16T16:36:30.703 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.701+0000 7f43d2376700 1 --2- 172.21.15.1:0/1178173166 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f43b40a1150 0x7f43b40a3600 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:30.703 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.701+0000 7f43d2376700 1 --2- 172.21.15.1:0/1178173166 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f43cc101ea0 0x7f43cc19d5c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:30.703 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.701+0000 7f43d2376700 1 --2- 172.21.15.1:0/1178173166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f43cc100310 0x7f43cc196000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:30.704 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.701+0000 7f43d2376700 1 --2- 172.21.15.1:0/1178173166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43cc101510 0x7f43cc196540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:30.704 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.701+0000 7f43d2376700 1 -- 172.21.15.1:0/1178173166 >> 172.21.15.1:0/1178173166 conn(0x7f43cc0fb960 msgr2=0x7f43cc1049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:36:30.704 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.701+0000 7f43d2376700 1 -- 172.21.15.1:0/1178173166 shutdown_connections 2023-11-16T16:36:30.704 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:30.701+0000 7f43d2376700 1 -- 172.21.15.1:0/1178173166 wait complete. 2023-11-16T16:36:31.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:30 smithi026 ceph-mon[100112]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-11-16T16:36:31.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:30 smithi026 ceph-mon[100112]: osdmap e77: 12 total, 11 up, 12 in 2023-11-16T16:36:31.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:30 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-16T16:36:31.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:30 smithi026 ceph-mon[100112]: from='osd.11 [v2:172.21.15.53:6824/2564669546,v1:172.21.15.53:6825/2564669546]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2023-11-16T16:36:31.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:30 smithi026 ceph-mon[100112]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2023-11-16T16:36:31.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:30 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1178173166' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-11-16T16:36:31.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:30 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:31.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:30 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:31.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:30 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:36:31.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:30 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:36:31.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:30 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:36:31.080 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:36:30 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:36:30.727+0000 7f22e5551700 -1 osd.11 0 waiting for initial osdmap 2023-11-16T16:36:31.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:36:30 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:36:30.745+0000 7f22dfb43700 -1 osd.11 78 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-16T16:36:31.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:30 smithi053 ceph-mon[100638]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-11-16T16:36:31.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:30 smithi053 ceph-mon[100638]: osdmap e77: 12 total, 11 up, 12 in 2023-11-16T16:36:31.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:30 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-16T16:36:31.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:30 smithi053 ceph-mon[100638]: from='osd.11 [v2:172.21.15.53:6824/2564669546,v1:172.21.15.53:6825/2564669546]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2023-11-16T16:36:31.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:30 smithi053 ceph-mon[100638]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2023-11-16T16:36:31.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:30 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1178173166' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-11-16T16:36:31.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:30 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:31.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:30 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:31.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:30 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:36:31.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:30 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:36:31.082 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:30 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:36:31.103 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:30 smithi001 ceph-mon[98800]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-11-16T16:36:31.103 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:30 smithi001 ceph-mon[98800]: osdmap e77: 12 total, 11 up, 12 in 2023-11-16T16:36:31.103 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:30 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-16T16:36:31.103 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:30 smithi001 ceph-mon[98800]: from='osd.11 [v2:172.21.15.53:6824/2564669546,v1:172.21.15.53:6825/2564669546]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2023-11-16T16:36:31.104 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:30 smithi001 ceph-mon[98800]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]: dispatch 2023-11-16T16:36:31.104 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:30 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1178173166' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-11-16T16:36:31.104 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:30 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:31.104 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:30 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:31.104 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:30 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:36:31.104 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:30 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:36:31.104 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:30 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:36:31.108 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":77,"num_osds":12,"num_up_osds":11,"osd_up_since":1700152570,"num_in_osds":12,"osd_in_since":1700152574,"num_remapped_pgs":0} 2023-11-16T16:36:32.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:31 smithi026 ceph-mon[100112]: Detected new or changed devices on smithi053 2023-11-16T16:36:32.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:31 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:36:32.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:31 smithi026 ceph-mon[100112]: Adjusting osd_memory_target on smithi053 to 4293M 2023-11-16T16:36:32.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:31 smithi026 ceph-mon[100112]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2023-11-16T16:36:32.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:31 smithi026 ceph-mon[100112]: osdmap e78: 12 total, 11 up, 12 in 2023-11-16T16:36:32.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:31 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-16T16:36:32.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:31 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-16T16:36:32.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:31 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:32.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:31 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:32.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:31 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:36:32.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:31 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:36:32.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:31 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:32.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:31 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:36:32.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:31 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:36:32.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:31 smithi026 ceph-mon[100112]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-11-16T16:36:32.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:31 smithi053 ceph-mon[100638]: Detected new or changed devices on smithi053 2023-11-16T16:36:32.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:31 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:36:32.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:31 smithi053 ceph-mon[100638]: Adjusting osd_memory_target on smithi053 to 4293M 2023-11-16T16:36:32.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:31 smithi053 ceph-mon[100638]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2023-11-16T16:36:32.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:31 smithi053 ceph-mon[100638]: osdmap e78: 12 total, 11 up, 12 in 2023-11-16T16:36:32.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:31 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-16T16:36:32.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:31 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-16T16:36:32.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:31 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:32.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:31 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:32.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:31 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:36:32.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:31 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:36:32.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:31 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:32.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:31 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:36:32.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:31 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:36:32.082 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:31 smithi053 ceph-mon[100638]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-11-16T16:36:32.109 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph osd stat -f json 2023-11-16T16:36:32.141 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:31 smithi001 ceph-mon[98800]: Detected new or changed devices on smithi053 2023-11-16T16:36:32.141 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:31 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:36:32.142 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:31 smithi001 ceph-mon[98800]: Adjusting osd_memory_target on smithi053 to 4293M 2023-11-16T16:36:32.142 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:31 smithi001 ceph-mon[98800]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi053", "root=default"]}]': finished 2023-11-16T16:36:32.142 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:31 smithi001 ceph-mon[98800]: osdmap e78: 12 total, 11 up, 12 in 2023-11-16T16:36:32.142 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:31 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-16T16:36:32.142 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:31 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-16T16:36:32.142 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:31 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:32.142 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:31 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:32.142 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:31 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:36:32.142 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:31 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:36:32.142 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:31 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:32.143 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:31 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:36:32.143 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:31 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:36:32.143 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:31 smithi001 ceph-mon[98800]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-11-16T16:36:32.776 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:36:32.980 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:32 smithi001 ceph-mon[98800]: purged_snaps scrub starts 2023-11-16T16:36:32.980 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:32 smithi001 ceph-mon[98800]: purged_snaps scrub ok 2023-11-16T16:36:32.980 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:32 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-16T16:36:32.980 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:32 smithi001 ceph-mon[98800]: osd.11 [v2:172.21.15.53:6824/2564669546,v1:172.21.15.53:6825/2564669546] boot 2023-11-16T16:36:32.980 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:32 smithi001 ceph-mon[98800]: osdmap e79: 12 total, 12 up, 12 in 2023-11-16T16:36:32.980 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:32 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-16T16:36:33.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:32 smithi026 ceph-mon[100112]: purged_snaps scrub starts 2023-11-16T16:36:33.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:32 smithi026 ceph-mon[100112]: purged_snaps scrub ok 2023-11-16T16:36:33.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:32 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-16T16:36:33.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:32 smithi026 ceph-mon[100112]: osd.11 [v2:172.21.15.53:6824/2564669546,v1:172.21.15.53:6825/2564669546] boot 2023-11-16T16:36:33.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:32 smithi026 ceph-mon[100112]: osdmap e79: 12 total, 12 up, 12 in 2023-11-16T16:36:33.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:32 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-16T16:36:33.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:32 smithi053 ceph-mon[100638]: purged_snaps scrub starts 2023-11-16T16:36:33.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:32 smithi053 ceph-mon[100638]: purged_snaps scrub ok 2023-11-16T16:36:33.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:32 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-16T16:36:33.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:32 smithi053 ceph-mon[100638]: osd.11 [v2:172.21.15.53:6824/2564669546,v1:172.21.15.53:6825/2564669546] boot 2023-11-16T16:36:33.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:32 smithi053 ceph-mon[100638]: osdmap e79: 12 total, 12 up, 12 in 2023-11-16T16:36:33.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:32 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-16T16:36:33.963 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.960+0000 7f16cd186700 1 -- 172.21.15.1:0/1595861364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16c8101e90 msgr2=0x7f16c8108750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:33.963 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.960+0000 7f16cd186700 1 --2- 172.21.15.1:0/1595861364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16c8101e90 0x7f16c8108750 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f16c000b0a0 tx=0x7f16c000b3b0 comp rx=0 tx=0).stop 2023-11-16T16:36:33.963 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.960+0000 7f16cd186700 1 -- 172.21.15.1:0/1595861364 shutdown_connections 2023-11-16T16:36:33.963 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.960+0000 7f16cd186700 1 --2- 172.21.15.1:0/1595861364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16c8101e90 0x7f16c8108750 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:33.963 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.960+0000 7f16cd186700 1 --2- 172.21.15.1:0/1595861364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16c8100300 0x7f16c8100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:33.963 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.960+0000 7f16cd186700 1 --2- 172.21.15.1:0/1595861364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16c8101500 0x7f16c8101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:33.963 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.960+0000 7f16cd186700 1 -- 172.21.15.1:0/1595861364 >> 172.21.15.1:0/1595861364 conn(0x7f16c80fb910 msgr2=0x7f16c80fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:36:33.963 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.960+0000 7f16cd186700 1 -- 172.21.15.1:0/1595861364 shutdown_connections 2023-11-16T16:36:33.963 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.960+0000 7f16cd186700 1 -- 172.21.15.1:0/1595861364 wait complete. 2023-11-16T16:36:33.965 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.961+0000 7f16cd186700 1 Processor -- start 2023-11-16T16:36:33.965 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.962+0000 7f16cd186700 1 -- start start 2023-11-16T16:36:33.965 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.962+0000 7f16cd186700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16c8100300 0x7f16c81961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:33.966 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.962+0000 7f16cd186700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16c8101500 0x7f16c81966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:33.966 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.962+0000 7f16cd186700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16c8101e90 0x7f16c819d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:33.966 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.962+0000 7f16cd186700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c819dd40 con 0x7f16c8101500 2023-11-16T16:36:33.966 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.962+0000 7f16cd186700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c819de80 con 0x7f16c8100300 2023-11-16T16:36:33.966 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.962+0000 7f16cd186700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c819e010 con 0x7f16c8101e90 2023-11-16T16:36:33.966 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.962+0000 7f16c659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16c8101500 0x7f16c81966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:33.966 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.962+0000 7f16c659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16c8101500 0x7f16c81966f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41068/0 (socket says 172.21.15.1:41068) 2023-11-16T16:36:33.966 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.962+0000 7f16c659c700 1 -- 172.21.15.1:0/646298120 learned_addr learned my addr 172.21.15.1:0/646298120 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:36:33.966 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.962+0000 7f16c6d9d700 1 --2- 172.21.15.1:0/646298120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16c8100300 0x7f16c81961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:33.966 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.962+0000 7f16c759e700 1 --2- 172.21.15.1:0/646298120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16c8101e90 0x7f16c819d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:33.966 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.962+0000 7f16c659c700 1 -- 172.21.15.1:0/646298120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16c8101e90 msgr2=0x7f16c819d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:33.966 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.962+0000 7f16c659c700 1 --2- 172.21.15.1:0/646298120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16c8101e90 0x7f16c819d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:33.967 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.962+0000 7f16c659c700 1 -- 172.21.15.1:0/646298120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16c8100300 msgr2=0x7f16c81961b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:33.967 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.962+0000 7f16c659c700 1 --2- 172.21.15.1:0/646298120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16c8100300 0x7f16c81961b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:33.967 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.962+0000 7f16c659c700 1 -- 172.21.15.1:0/646298120 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c0009d00 con 0x7f16c8101500 2023-11-16T16:36:33.967 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.962+0000 7f16c659c700 1 --2- 172.21.15.1:0/646298120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16c8101500 0x7f16c81966f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f16bc00d980 tx=0x7f16bc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:36:33.967 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.963+0000 7f16affff700 1 -- 172.21.15.1:0/646298120 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16bc009b40 con 0x7f16c8101500 2023-11-16T16:36:33.967 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.963+0000 7f16affff700 1 -- 172.21.15.1:0/646298120 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16bc00b6e0 con 0x7f16c8101500 2023-11-16T16:36:33.967 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.963+0000 7f16cd186700 1 -- 172.21.15.1:0/646298120 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c810bcf0 con 0x7f16c8101500 2023-11-16T16:36:33.967 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.963+0000 7f16affff700 1 -- 172.21.15.1:0/646298120 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16bc00f620 con 0x7f16c8101500 2023-11-16T16:36:33.967 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.963+0000 7f16cd186700 1 -- 172.21.15.1:0/646298120 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c810bb60 con 0x7f16c8101500 2023-11-16T16:36:33.967 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.964+0000 7f16cd186700 1 -- 172.21.15.1:0/646298120 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16c8066ea0 con 0x7f16c8101500 2023-11-16T16:36:33.968 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.965+0000 7f16affff700 1 -- 172.21.15.1:0/646298120 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f16bc010460 con 0x7f16c8101500 2023-11-16T16:36:33.968 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.965+0000 7f16affff700 1 --2- 172.21.15.1:0/646298120 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f16b00a1150 0x7f16b00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:33.968 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.965+0000 7f16c6d9d700 1 --2- 172.21.15.1:0/646298120 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f16b00a1150 0x7f16b00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:33.968 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.965+0000 7f16affff700 1 -- 172.21.15.1:0/646298120 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f16bc0cc710 con 0x7f16c8101500 2023-11-16T16:36:33.968 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.966+0000 7f16c6d9d700 1 --2- 172.21.15.1:0/646298120 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f16b00a1150 0x7f16b00a3600 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f16c8101360 tx=0x7f16b8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:36:33.971 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:33.969+0000 7f16affff700 1 -- 172.21.15.1:0/646298120 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f16bc096300 con 0x7f16c8101500 2023-11-16T16:36:34.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:33 smithi026 ceph-mon[100112]: osdmap e80: 12 total, 12 up, 12 in 2023-11-16T16:36:34.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:33 smithi026 ceph-mon[100112]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:36:34.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:33 smithi053 ceph-mon[100638]: osdmap e80: 12 total, 12 up, 12 in 2023-11-16T16:36:34.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:33 smithi053 ceph-mon[100638]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:36:34.168 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:36:34.169 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:34.166+0000 7f16cd186700 1 -- 172.21.15.1:0/646298120 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f16c81071c0 con 0x7f16c8101500 2023-11-16T16:36:34.169 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:34.166+0000 7f16affff700 1 -- 172.21.15.1:0/646298120 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) v1 ==== 74+0+133 (secure 0 0 0) 0x7f16bc099920 con 0x7f16c8101500 2023-11-16T16:36:34.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:34.168+0000 7f16cd186700 1 -- 172.21.15.1:0/646298120 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f16b00a1150 msgr2=0x7f16b00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:34.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:34.168+0000 7f16cd186700 1 --2- 172.21.15.1:0/646298120 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f16b00a1150 0x7f16b00a3600 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f16c8101360 tx=0x7f16b8009190 comp rx=0 tx=0).stop 2023-11-16T16:36:34.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:34.168+0000 7f16cd186700 1 -- 172.21.15.1:0/646298120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16c8101500 msgr2=0x7f16c81966f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:34.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:34.168+0000 7f16cd186700 1 --2- 172.21.15.1:0/646298120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16c8101500 0x7f16c81966f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f16bc00d980 tx=0x7f16bc00dc90 comp rx=0 tx=0).stop 2023-11-16T16:36:34.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:34.168+0000 7f16cd186700 1 -- 172.21.15.1:0/646298120 shutdown_connections 2023-11-16T16:36:34.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:34.168+0000 7f16cd186700 1 --2- 172.21.15.1:0/646298120 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f16b00a1150 0x7f16b00a3600 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:34.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:34.168+0000 7f16cd186700 1 --2- 172.21.15.1:0/646298120 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16c8101e90 0x7f16c819d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:34.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:34.168+0000 7f16cd186700 1 --2- 172.21.15.1:0/646298120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16c8100300 0x7f16c81961b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:34.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:34.168+0000 7f16cd186700 1 --2- 172.21.15.1:0/646298120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16c8101500 0x7f16c81966f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:34.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:34.168+0000 7f16cd186700 1 -- 172.21.15.1:0/646298120 >> 172.21.15.1:0/646298120 conn(0x7f16c80fb910 msgr2=0x7f16c80fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:36:34.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:34.169+0000 7f16cd186700 1 -- 172.21.15.1:0/646298120 shutdown_connections 2023-11-16T16:36:34.172 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:34.169+0000 7f16cd186700 1 -- 172.21.15.1:0/646298120 wait complete. 2023-11-16T16:36:34.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:33 smithi001 ceph-mon[98800]: osdmap e80: 12 total, 12 up, 12 in 2023-11-16T16:36:34.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:33 smithi001 ceph-mon[98800]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:36:34.623 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1700152591,"num_in_osds":12,"osd_in_since":1700152574,"num_remapped_pgs":0} 2023-11-16T16:36:34.624 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph osd dump --format=json 2023-11-16T16:36:34.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:34 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/646298120' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-11-16T16:36:35.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:34 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/646298120' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-11-16T16:36:35.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:34 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/646298120' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-11-16T16:36:35.318 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:36:35.832 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:35 smithi001 ceph-mon[98800]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:36:36.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:35 smithi026 ceph-mon[100112]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:36:36.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:35 smithi053 ceph-mon[100638]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:36:36.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.532+0000 7f254acb7700 1 -- 172.21.15.1:0/2057650414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2544104510 msgr2=0x7f254410c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:36.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.532+0000 7f254acb7700 1 --2- 172.21.15.1:0/2057650414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2544104510 0x7f254410c090 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f253c00b0d0 tx=0x7f253c00b3e0 comp rx=0 tx=0).stop 2023-11-16T16:36:36.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.532+0000 7f254acb7700 1 -- 172.21.15.1:0/2057650414 shutdown_connections 2023-11-16T16:36:36.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.532+0000 7f254acb7700 1 --2- 172.21.15.1:0/2057650414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2544104510 0x7f254410c090 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:36.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.532+0000 7f254acb7700 1 --2- 172.21.15.1:0/2057650414 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25440ff470 0x7f25440ff840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:36.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.532+0000 7f254acb7700 1 --2- 172.21.15.1:0/2057650414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25440ffe10 0x7f2544103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:36.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.532+0000 7f254acb7700 1 -- 172.21.15.1:0/2057650414 >> 172.21.15.1:0/2057650414 conn(0x7f25440770d0 msgr2=0x7f25440774d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:36:36.538 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.532+0000 7f254acb7700 1 -- 172.21.15.1:0/2057650414 shutdown_connections 2023-11-16T16:36:36.538 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.532+0000 7f254acb7700 1 -- 172.21.15.1:0/2057650414 wait complete. 2023-11-16T16:36:36.538 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.533+0000 7f254acb7700 1 Processor -- start 2023-11-16T16:36:36.538 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.533+0000 7f254acb7700 1 -- start start 2023-11-16T16:36:36.538 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.533+0000 7f254acb7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25440ff470 0x7f2544196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:36.538 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.533+0000 7f254acb7700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25440ffe10 0x7f2544196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:36.538 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.533+0000 7f254acb7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2544104510 0x7f254419aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:36.539 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.533+0000 7f254acb7700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254419b4e0 con 0x7f25440ffe10 2023-11-16T16:36:36.539 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.533+0000 7f254acb7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254419b650 con 0x7f25440ff470 2023-11-16T16:36:36.539 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.533+0000 7f254acb7700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254419b810 con 0x7f2544104510 2023-11-16T16:36:36.539 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.533+0000 7f2543fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25440ffe10 0x7f2544196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:36.539 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.533+0000 7f2543fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25440ffe10 0x7f2544196a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41080/0 (socket says 172.21.15.1:41080) 2023-11-16T16:36:36.539 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.533+0000 7f2543fff700 1 -- 172.21.15.1:0/3653156766 learned_addr learned my addr 172.21.15.1:0/3653156766 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:36:36.539 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.533+0000 7f2548a53700 1 --2- 172.21.15.1:0/3653156766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25440ff470 0x7f2544196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:36.540 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.533+0000 7f2549254700 1 --2- 172.21.15.1:0/3653156766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2544104510 0x7f254419aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:36.540 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.533+0000 7f2543fff700 1 -- 172.21.15.1:0/3653156766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2544104510 msgr2=0x7f254419aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:36.540 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.533+0000 7f2543fff700 1 --2- 172.21.15.1:0/3653156766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2544104510 0x7f254419aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:36.540 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.533+0000 7f2543fff700 1 -- 172.21.15.1:0/3653156766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25440ff470 msgr2=0x7f2544196520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:36.540 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.533+0000 7f2543fff700 1 --2- 172.21.15.1:0/3653156766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25440ff470 0x7f2544196520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:36.540 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.533+0000 7f2543fff700 1 -- 172.21.15.1:0/3653156766 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253c009d00 con 0x7f25440ffe10 2023-11-16T16:36:36.540 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.534+0000 7f2543fff700 1 --2- 172.21.15.1:0/3653156766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25440ffe10 0x7f2544196a60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f253000d980 tx=0x7f253000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:36:36.540 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.534+0000 7f2541ffb700 1 -- 172.21.15.1:0/3653156766 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2530009930 con 0x7f25440ffe10 2023-11-16T16:36:36.541 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.534+0000 7f2541ffb700 1 -- 172.21.15.1:0/3653156766 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2530010460 con 0x7f25440ffe10 2023-11-16T16:36:36.541 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.534+0000 7f254acb7700 1 -- 172.21.15.1:0/3653156766 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f254410f2c0 con 0x7f25440ffe10 2023-11-16T16:36:36.541 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.534+0000 7f2541ffb700 1 -- 172.21.15.1:0/3653156766 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253000f620 con 0x7f25440ffe10 2023-11-16T16:36:36.541 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.534+0000 7f254acb7700 1 -- 172.21.15.1:0/3653156766 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f254410f130 con 0x7f25440ffe10 2023-11-16T16:36:36.541 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.535+0000 7f254acb7700 1 -- 172.21.15.1:0/3653156766 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f254410f450 con 0x7f25440ffe10 2023-11-16T16:36:36.542 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.536+0000 7f2541ffb700 1 -- 172.21.15.1:0/3653156766 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f25300108d0 con 0x7f25440ffe10 2023-11-16T16:36:36.542 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.536+0000 7f2541ffb700 1 --2- 172.21.15.1:0/3653156766 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f25340a11d0 0x7f25340a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:36.543 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.537+0000 7f2541ffb700 1 -- 172.21.15.1:0/3653156766 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f25300cbe20 con 0x7f25440ffe10 2023-11-16T16:36:36.543 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.537+0000 7f2548a53700 1 --2- 172.21.15.1:0/3653156766 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f25340a11d0 0x7f25340a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:36.543 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.537+0000 7f2548a53700 1 --2- 172.21.15.1:0/3653156766 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f25340a11d0 0x7f25340a3680 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2538005950 tx=0x7f253800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:36:36.543 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.541+0000 7f2541ffb700 1 -- 172.21.15.1:0/3653156766 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2530095a10 con 0x7f25440ffe10 2023-11-16T16:36:36.740 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.738+0000 7f254acb7700 1 -- 172.21.15.1:0/3653156766 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2544066ea0 con 0x7f25440ffe10 2023-11-16T16:36:36.742 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.739+0000 7f2541ffb700 1 -- 172.21.15.1:0/3653156766 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18291 (secure 0 0 0) 0x7f2530099030 con 0x7f25440ffe10 2023-11-16T16:36:36.742 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:36:36.742 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":80,"fsid":"35d9b010-849d-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:30:06.217778+0000","modified":"2023-11-16T16:36:32.736561+0000","last_up_change":"2023-11-16T16:36:31.735403+0000","last_in_change":"2023-11-16T16:36:14.083021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:33:15.916723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a8480507-5275-4e52-953b-626110beb436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6803","nonce":2469671609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6805","nonce":2469671609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6809","nonce":2469671609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6807","nonce":2469671609}]},"public_addr":"172.21.15.1:6803/2469671609","cluster_addr":"172.21.15.1:6805/2469671609","heartbeat_back_addr":"172.21.15.1:6809/2469671609","heartbeat_front_addr":"172.21.15.1:6807/2469671609","state":["exists","up"]},{"osd":1,"uuid":"1d011db2-99d1-4575-bdc5-51f88a308fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6801","nonce":247994796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6803","nonce":247994796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6807","nonce":247994796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6805","nonce":247994796}]},"public_addr":"172.21.15.26:6801/247994796","cluster_addr":"172.21.15.26:6803/247994796","heartbeat_back_addr":"172.21.15.26:6807/247994796","heartbeat_front_addr":"172.21.15.26:6805/247994796","state":["exists","up"]},{"osd":2,"uuid":"4d7a2bd1-3c8e-4546-a243-f0354473b031","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6801","nonce":1659316290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6803","nonce":1659316290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6807","nonce":1659316290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6805","nonce":1659316290}]},"public_addr":"172.21.15.53:6801/1659316290","cluster_addr":"172.21.15.53:6803/1659316290","heartbeat_back_addr":"172.21.15.53:6807/1659316290","heartbeat_front_addr":"172.21.15.53:6805/1659316290","state":["exists","up"]},{"osd":3,"uuid":"9c290fd9-e121-4a81-9687-b0eb93debd86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6811","nonce":2728987281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6813","nonce":2728987281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6817","nonce":2728987281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6815","nonce":2728987281}]},"public_addr":"172.21.15.1:6811/2728987281","cluster_addr":"172.21.15.1:6813/2728987281","heartbeat_back_addr":"172.21.15.1:6817/2728987281","heartbeat_front_addr":"172.21.15.1:6815/2728987281","state":["exists","up"]},{"osd":4,"uuid":"533226bc-cc57-4f15-9e29-820294d57e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6809","nonce":2954740024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6811","nonce":2954740024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6815","nonce":2954740024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6813","nonce":2954740024}]},"public_addr":"172.21.15.26:6809/2954740024","cluster_addr":"172.21.15.26:6811/2954740024","heartbeat_back_addr":"172.21.15.26:6815/2954740024","heartbeat_front_addr":"172.21.15.26:6813/2954740024","state":["exists","up"]},{"osd":5,"uuid":"d3ab25eb-e1d4-4557-bc89-efc9072401ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6809","nonce":3966666318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6811","nonce":3966666318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6815","nonce":3966666318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6813","nonce":3966666318}]},"public_addr":"172.21.15.53:6809/3966666318","cluster_addr":"172.21.15.53:6811/3966666318","heartbeat_back_addr":"172.21.15.53:6815/3966666318","heartbeat_front_addr":"172.21.15.53:6813/3966666318","state":["exists","up"]},{"osd":6,"uuid":"180f506e-afb8-4043-b0b4-2dc86f6d4bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6819","nonce":3364032646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6821","nonce":3364032646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6825","nonce":3364032646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6823","nonce":3364032646}]},"public_addr":"172.21.15.1:6819/3364032646","cluster_addr":"172.21.15.1:6821/3364032646","heartbeat_back_addr":"172.21.15.1:6825/3364032646","heartbeat_front_addr":"172.21.15.1:6823/3364032646","state":["exists","up"]},{"osd":7,"uuid":"0d837bac-194f-498d-b7fb-b3147e992329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6817","nonce":3766901391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6819","nonce":3766901391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6823","nonce":3766901391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6821","nonce":3766901391}]},"public_addr":"172.21.15.26:6817/3766901391","cluster_addr":"172.21.15.26:6819/3766901391","heartbeat_back_addr":"172.21.15.26:6823/3766901391","heartbeat_front_addr":"172.21.15.26:6821/3766901391","state":["exists","up"]},{"osd":8,"uuid":"933f231a-7454-47b0-9291-f1fa64f0baec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6817","nonce":3224335871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6819","nonce":3224335871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6823","nonce":3224335871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6821","nonce":3224335871}]},"public_addr":"172.21.15.53:6817/3224335871","cluster_addr":"172.21.15.53:6819/3224335871","heartbeat_back_addr":"172.21.15.53:6823/3224335871","heartbeat_front_addr":"172.21.15.53:6821/3224335871","state":["exists","up"]},{"osd":9,"uuid":"245fcbb3-c651-4703-ab80-309dc98bd983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6827","nonce":1391199844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6829","nonce":1391199844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6833","nonce":1391199844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6831","nonce":1391199844}]},"public_addr":"172.21.15.1:6827/1391199844","cluster_addr":"172.21.15.1:6829/1391199844","heartbeat_back_addr":"172.21.15.1:6833/1391199844","heartbeat_front_addr":"172.21.15.1:6831/1391199844","state":["exists","up"]},{"osd":10,"uuid":"2c090e7a-5240-4d52-ae4f-e94fe05383d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6825","nonce":1773545447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6827","nonce":1773545447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6831","nonce":1773545447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6829","nonce":1773545447}]},"public_addr":"172.21.15.26:6825/1773545447","cluster_addr":"172.21.15.26:6827/1773545447","heartbeat_back_addr":"172.21.15.26:6831/1773545447","heartbeat_front_addr":"172.21.15.26:6829/1773545447","state":["exists","up"]},{"osd":11,"uuid":"4511e32d-e469-45e1-8310-0ff400cb7b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6825","nonce":2564669546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6827","nonce":2564669546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6831","nonce":2564669546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6829","nonce":2564669546}]},"public_addr":"172.21.15.53:6825/2564669546","cluster_addr":"172.21.15.53:6827/2564669546","heartbeat_back_addr":"172.21.15.53:6831/2564669546","heartbeat_front_addr":"172.21.15.53:6829/2564669546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:28.140250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:49.885461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:12.521487+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:34.926197+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:56.815594+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:18.387751+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:40.587106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:02.319370+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:23.832298+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:46.338040+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:08.730414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:30.050070+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1494336948":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6801/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6800/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6800/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/1641976669":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/2895923638":"2023-11-17T16:30:35.064289+0000","172.21.15.1:6801/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/78811711":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/4131957708":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/344403698":"2023-11-17T16:30:35.064289+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:36:36.744 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.742+0000 7f254acb7700 1 -- 172.21.15.1:0/3653156766 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f25340a11d0 msgr2=0x7f25340a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:36.744 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.742+0000 7f254acb7700 1 --2- 172.21.15.1:0/3653156766 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f25340a11d0 0x7f25340a3680 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2538005950 tx=0x7f253800a300 comp rx=0 tx=0).stop 2023-11-16T16:36:36.744 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.742+0000 7f254acb7700 1 -- 172.21.15.1:0/3653156766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25440ffe10 msgr2=0x7f2544196a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:36.744 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.742+0000 7f254acb7700 1 --2- 172.21.15.1:0/3653156766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25440ffe10 0x7f2544196a60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f253000d980 tx=0x7f253000dc90 comp rx=0 tx=0).stop 2023-11-16T16:36:36.744 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.742+0000 7f254acb7700 1 -- 172.21.15.1:0/3653156766 shutdown_connections 2023-11-16T16:36:36.744 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.742+0000 7f254acb7700 1 --2- 172.21.15.1:0/3653156766 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f25340a11d0 0x7f25340a3680 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:36.745 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.742+0000 7f254acb7700 1 --2- 172.21.15.1:0/3653156766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2544104510 0x7f254419aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:36.745 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.742+0000 7f254acb7700 1 --2- 172.21.15.1:0/3653156766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25440ff470 0x7f2544196520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:36.745 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.742+0000 7f254acb7700 1 --2- 172.21.15.1:0/3653156766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25440ffe10 0x7f2544196a60 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:36.745 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.742+0000 7f254acb7700 1 -- 172.21.15.1:0/3653156766 >> 172.21.15.1:0/3653156766 conn(0x7f25440770d0 msgr2=0x7f2544068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:36:36.745 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.742+0000 7f254acb7700 1 -- 172.21.15.1:0/3653156766 shutdown_connections 2023-11-16T16:36:36.745 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:36.742+0000 7f254acb7700 1 -- 172.21.15.1:0/3653156766 wait complete. 2023-11-16T16:36:37.177 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-16T16:33:15.916723+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-16T16:36:37.178 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph osd pool get .mgr pg_num 2023-11-16T16:36:37.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:37 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3653156766' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:36:37.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:37 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/3653156766' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:36:37.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:37 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/3653156766' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:36:37.833 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:36:38.398 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:38 smithi001 ceph-mon[98800]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:36:38.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:38 smithi026 ceph-mon[100112]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:36:38.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:38 smithi053 ceph-mon[100638]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:36:39.147 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.143+0000 7f5b0eacd700 1 -- 172.21.15.1:0/3888959873 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b08100ee0 msgr2=0x7f5b08104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:39.147 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.143+0000 7f5b0eacd700 1 --2- 172.21.15.1:0/3888959873 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b08100ee0 0x7f5b08104eb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5af40099c0 tx=0x7f5af4009cd0 comp rx=0 tx=0).stop 2023-11-16T16:36:39.147 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.143+0000 7f5b0eacd700 1 -- 172.21.15.1:0/3888959873 shutdown_connections 2023-11-16T16:36:39.147 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.143+0000 7f5b0eacd700 1 --2- 172.21.15.1:0/3888959873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b081055e0 0x7f5b0810d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:39.148 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.143+0000 7f5b0eacd700 1 --2- 172.21.15.1:0/3888959873 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b08100540 0x7f5b08100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:39.148 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.143+0000 7f5b0eacd700 1 --2- 172.21.15.1:0/3888959873 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b08100ee0 0x7f5b08104eb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:39.148 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.143+0000 7f5b0eacd700 1 -- 172.21.15.1:0/3888959873 >> 172.21.15.1:0/3888959873 conn(0x7f5b080fbdc0 msgr2=0x7f5b080fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:36:39.148 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.143+0000 7f5b0eacd700 1 -- 172.21.15.1:0/3888959873 shutdown_connections 2023-11-16T16:36:39.148 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.143+0000 7f5b0eacd700 1 -- 172.21.15.1:0/3888959873 wait complete. 2023-11-16T16:36:39.148 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.143+0000 7f5b0eacd700 1 Processor -- start 2023-11-16T16:36:39.148 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.144+0000 7f5b0eacd700 1 -- start start 2023-11-16T16:36:39.148 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.144+0000 7f5b0eacd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b08100540 0x7f5b08072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:39.148 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.144+0000 7f5b0eacd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b08100ee0 0x7f5b08073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:39.148 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.144+0000 7f5b0eacd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b081055e0 0x7f5b080738b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:39.148 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.144+0000 7f5b0eacd700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b081a4840 con 0x7f5b081055e0 2023-11-16T16:36:39.148 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.144+0000 7f5b0eacd700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b081a49b0 con 0x7f5b08100540 2023-11-16T16:36:39.149 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.144+0000 7f5b0eacd700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b081a4b70 con 0x7f5b08100ee0 2023-11-16T16:36:39.149 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.144+0000 7f5b0d06a700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b081055e0 0x7f5b080738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:39.149 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.144+0000 7f5b0d06a700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b081055e0 0x7f5b080738b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41094/0 (socket says 172.21.15.1:41094) 2023-11-16T16:36:39.149 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.144+0000 7f5b0d06a700 1 -- 172.21.15.1:0/3810616614 learned_addr learned my addr 172.21.15.1:0/3810616614 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:36:39.149 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.144+0000 7f5b0c869700 1 --2- 172.21.15.1:0/3810616614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b08100540 0x7f5b08072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:39.149 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.144+0000 7f5b07fff700 1 --2- 172.21.15.1:0/3810616614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b08100ee0 0x7f5b08073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:39.149 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.144+0000 7f5b0d06a700 1 -- 172.21.15.1:0/3810616614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b08100ee0 msgr2=0x7f5b08073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:39.149 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.144+0000 7f5b0d06a700 1 --2- 172.21.15.1:0/3810616614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b08100ee0 0x7f5b08073330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:39.149 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.144+0000 7f5b0d06a700 1 -- 172.21.15.1:0/3810616614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b08100540 msgr2=0x7f5b08072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:39.149 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.144+0000 7f5b0d06a700 1 --2- 172.21.15.1:0/3810616614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b08100540 0x7f5b08072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:39.149 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.144+0000 7f5b0d06a700 1 -- 172.21.15.1:0/3810616614 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af40095f0 con 0x7f5b081055e0 2023-11-16T16:36:39.149 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.144+0000 7f5b0d06a700 1 --2- 172.21.15.1:0/3810616614 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b081055e0 0x7f5b080738b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5b00007ee0 tx=0x7f5b0000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:36:39.150 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.145+0000 7f5b05ffb700 1 -- 172.21.15.1:0/3810616614 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b00017070 con 0x7f5b081055e0 2023-11-16T16:36:39.150 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.145+0000 7f5b05ffb700 1 -- 172.21.15.1:0/3810616614 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b0000f040 con 0x7f5b081055e0 2023-11-16T16:36:39.150 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.145+0000 7f5b0eacd700 1 -- 172.21.15.1:0/3810616614 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b08110360 con 0x7f5b081055e0 2023-11-16T16:36:39.150 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.145+0000 7f5b05ffb700 1 -- 172.21.15.1:0/3810616614 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b00005740 con 0x7f5b081055e0 2023-11-16T16:36:39.150 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.145+0000 7f5b0eacd700 1 -- 172.21.15.1:0/3810616614 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b081101d0 con 0x7f5b081055e0 2023-11-16T16:36:39.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.146+0000 7f5b0eacd700 1 -- 172.21.15.1:0/3810616614 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b081104f0 con 0x7f5b081055e0 2023-11-16T16:36:39.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.147+0000 7f5b05ffb700 1 -- 172.21.15.1:0/3810616614 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f5b0000dc60 con 0x7f5b081055e0 2023-11-16T16:36:39.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.148+0000 7f5b05ffb700 1 --2- 172.21.15.1:0/3810616614 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f5af80a11d0 0x7f5af80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:39.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.148+0000 7f5b0c869700 1 --2- 172.21.15.1:0/3810616614 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f5af80a11d0 0x7f5af80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:39.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.148+0000 7f5b05ffb700 1 -- 172.21.15.1:0/3810616614 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5b000cbb70 con 0x7f5b081055e0 2023-11-16T16:36:39.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.148+0000 7f5b0c869700 1 --2- 172.21.15.1:0/3810616614 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f5af80a11d0 0x7f5af80a3680 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5afc005950 tx=0x7f5afc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:36:39.154 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.152+0000 7f5b05ffb700 1 -- 172.21.15.1:0/3810616614 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5b00095760 con 0x7f5b081055e0 2023-11-16T16:36:39.352 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.350+0000 7f5b0eacd700 1 -- 172.21.15.1:0/3810616614 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5b08074320 con 0x7f5b081055e0 2023-11-16T16:36:39.353 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.351+0000 7f5b05ffb700 1 -- 172.21.15.1:0/3810616614 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5b00098d80 con 0x7f5b081055e0 2023-11-16T16:36:39.353 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2023-11-16T16:36:39.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.353+0000 7f5b0eacd700 1 -- 172.21.15.1:0/3810616614 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f5af80a11d0 msgr2=0x7f5af80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:39.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.353+0000 7f5b0eacd700 1 --2- 172.21.15.1:0/3810616614 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f5af80a11d0 0x7f5af80a3680 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5afc005950 tx=0x7f5afc00b410 comp rx=0 tx=0).stop 2023-11-16T16:36:39.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.353+0000 7f5b0eacd700 1 -- 172.21.15.1:0/3810616614 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b081055e0 msgr2=0x7f5b080738b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:39.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.353+0000 7f5b0eacd700 1 --2- 172.21.15.1:0/3810616614 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b081055e0 0x7f5b080738b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5b00007ee0 tx=0x7f5b0000d3b0 comp rx=0 tx=0).stop 2023-11-16T16:36:39.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.353+0000 7f5b0eacd700 1 -- 172.21.15.1:0/3810616614 shutdown_connections 2023-11-16T16:36:39.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.353+0000 7f5b0eacd700 1 --2- 172.21.15.1:0/3810616614 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f5af80a11d0 0x7f5af80a3680 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:39.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.353+0000 7f5b0eacd700 1 --2- 172.21.15.1:0/3810616614 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b081055e0 0x7f5b080738b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:39.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.353+0000 7f5b0eacd700 1 --2- 172.21.15.1:0/3810616614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b08100540 0x7f5b08072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:39.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.353+0000 7f5b0eacd700 1 --2- 172.21.15.1:0/3810616614 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b08100ee0 0x7f5b08073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:39.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.353+0000 7f5b0eacd700 1 -- 172.21.15.1:0/3810616614 >> 172.21.15.1:0/3810616614 conn(0x7f5b080fbdc0 msgr2=0x7f5b080fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:36:39.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.353+0000 7f5b0eacd700 1 -- 172.21.15.1:0/3810616614 shutdown_connections 2023-11-16T16:36:39.356 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:39.353+0000 7f5b0eacd700 1 -- 172.21.15.1:0/3810616614 wait complete. 2023-11-16T16:36:39.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:39 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3810616614' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:36:39.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:39 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/3810616614' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:36:39.791 INFO:tasks.cephadm:Adding mds.a on smithi001 2023-11-16T16:36:39.791 INFO:tasks.cephadm:Adding mds.d on smithi001 2023-11-16T16:36:39.791 INFO:tasks.cephadm:Adding mds.g on smithi001 2023-11-16T16:36:39.791 INFO:tasks.cephadm:Adding mds.j on smithi001 2023-11-16T16:36:39.791 INFO:tasks.cephadm:Adding mds.b on smithi026 2023-11-16T16:36:39.791 INFO:tasks.cephadm:Adding mds.e on smithi026 2023-11-16T16:36:39.791 INFO:tasks.cephadm:Adding mds.h on smithi026 2023-11-16T16:36:39.791 INFO:tasks.cephadm:Adding mds.k on smithi026 2023-11-16T16:36:39.791 INFO:tasks.cephadm:Adding mds.c on smithi053 2023-11-16T16:36:39.792 INFO:tasks.cephadm:Adding mds.f on smithi053 2023-11-16T16:36:39.792 INFO:tasks.cephadm:Adding mds.i on smithi053 2023-11-16T16:36:39.792 INFO:tasks.cephadm:Adding mds.l on smithi053 2023-11-16T16:36:39.792 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph orch apply mds all '12;smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi026=b;smithi026=e;smithi026=h;smithi026=k;smithi053=c;smithi053=f;smithi053=i;smithi053=l' 2023-11-16T16:36:39.826 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:39 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/3810616614' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:36:40.463 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-16T16:36:40.673 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:40 smithi053 ceph-mon[100638]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:36:40.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:40 smithi001 ceph-mon[98800]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:36:40.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:40 smithi026 ceph-mon[100112]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:36:41.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.648+0000 7f3ed0179700 1 -- 172.21.15.53:0/1343489880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ec8100cf0 msgr2=0x7f3ec81075b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:41.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.648+0000 7f3ed0179700 1 --2- 172.21.15.53:0/1343489880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ec8100cf0 0x7f3ec81075b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3ec000b0a0 tx=0x7f3ec000b3b0 comp rx=0 tx=0).stop 2023-11-16T16:36:41.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.648+0000 7f3ed0179700 1 -- 172.21.15.53:0/1343489880 shutdown_connections 2023-11-16T16:36:41.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.648+0000 7f3ed0179700 1 --2- 172.21.15.53:0/1343489880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ec8100cf0 0x7f3ec81075b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:41.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.648+0000 7f3ed0179700 1 --2- 172.21.15.53:0/1343489880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ec80ff160 0x7f3ec80ff570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:41.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.648+0000 7f3ed0179700 1 --2- 172.21.15.53:0/1343489880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ec8100360 0x7f3ec81007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:41.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.648+0000 7f3ed0179700 1 -- 172.21.15.53:0/1343489880 >> 172.21.15.53:0/1343489880 conn(0x7f3ec80fa790 msgr2=0x7f3ec80fcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:36:41.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.648+0000 7f3ed0179700 1 -- 172.21.15.53:0/1343489880 shutdown_connections 2023-11-16T16:36:41.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.648+0000 7f3ed0179700 1 -- 172.21.15.53:0/1343489880 wait complete. 2023-11-16T16:36:41.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.648+0000 7f3ed0179700 1 Processor -- start 2023-11-16T16:36:41.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.648+0000 7f3ed0179700 1 -- start start 2023-11-16T16:36:41.651 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.648+0000 7f3ed0179700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ec80ff160 0x7f3ec81030f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:41.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.648+0000 7f3ed0179700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ec8100360 0x7f3ec8101740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:41.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.649+0000 7f3ed0179700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ec8100cf0 0x7f3ec8101c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:41.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.649+0000 7f3ed0179700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec8102400 con 0x7f3ec80ff160 2023-11-16T16:36:41.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.649+0000 7f3ed0179700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec8102570 con 0x7f3ec8100360 2023-11-16T16:36:41.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.649+0000 7f3ed0179700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec81a0120 con 0x7f3ec8100cf0 2023-11-16T16:36:41.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.649+0000 7f3ece716700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ec8100cf0 0x7f3ec8101c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:41.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.649+0000 7f3ece716700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ec8100cf0 0x7f3ec8101c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.53:58074/0 (socket says 172.21.15.53:58074) 2023-11-16T16:36:41.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.649+0000 7f3ece716700 1 -- 172.21.15.53:0/283373101 learned_addr learned my addr 172.21.15.53:0/283373101 (peer_addr_for_me v2:172.21.15.53:0/0) 2023-11-16T16:36:41.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.649+0000 7f3ecd714700 1 --2- 172.21.15.53:0/283373101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ec8100360 0x7f3ec8101740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:41.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.649+0000 7f3ecdf15700 1 --2- 172.21.15.53:0/283373101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ec80ff160 0x7f3ec81030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:41.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.649+0000 7f3ece716700 1 -- 172.21.15.53:0/283373101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ec8100360 msgr2=0x7f3ec8101740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:41.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.649+0000 7f3ece716700 1 --2- 172.21.15.53:0/283373101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ec8100360 0x7f3ec8101740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:41.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.649+0000 7f3ece716700 1 -- 172.21.15.53:0/283373101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ec80ff160 msgr2=0x7f3ec81030f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:41.652 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.649+0000 7f3ece716700 1 --2- 172.21.15.53:0/283373101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ec80ff160 0x7f3ec81030f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:41.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.649+0000 7f3ece716700 1 -- 172.21.15.53:0/283373101 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec0009d00 con 0x7f3ec8100cf0 2023-11-16T16:36:41.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.649+0000 7f3ece716700 1 --2- 172.21.15.53:0/283373101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ec8100cf0 0x7f3ec8101c80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3ec00062a0 tx=0x7f3ec0004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:36:41.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.649+0000 7f3ebeffd700 1 -- 172.21.15.53:0/283373101 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec000e040 con 0x7f3ec8100cf0 2023-11-16T16:36:41.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.649+0000 7f3ed0179700 1 -- 172.21.15.53:0/283373101 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ec810aac0 con 0x7f3ec8100cf0 2023-11-16T16:36:41.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.649+0000 7f3ebeffd700 1 -- 172.21.15.53:0/283373101 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ec000cec0 con 0x7f3ec8100cf0 2023-11-16T16:36:41.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.649+0000 7f3ebeffd700 1 -- 172.21.15.53:0/283373101 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec001baf0 con 0x7f3ec8100cf0 2023-11-16T16:36:41.653 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.649+0000 7f3ed0179700 1 -- 172.21.15.53:0/283373101 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ec810a930 con 0x7f3ec8100cf0 2023-11-16T16:36:41.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.650+0000 7f3ed0179700 1 -- 172.21.15.53:0/283373101 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ec804e930 con 0x7f3ec8100cf0 2023-11-16T16:36:41.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.652+0000 7f3ebeffd700 1 -- 172.21.15.53:0/283373101 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f3ec001bc50 con 0x7f3ec8100cf0 2023-11-16T16:36:41.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.652+0000 7f3ebeffd700 1 --2- 172.21.15.53:0/283373101 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f3eb40a11d0 0x7f3eb40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:41.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.652+0000 7f3ebeffd700 1 -- 172.21.15.53:0/283373101 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3ec00ce5a0 con 0x7f3ec8100cf0 2023-11-16T16:36:41.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.652+0000 7f3ecdf15700 1 --2- 172.21.15.53:0/283373101 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f3eb40a11d0 0x7f3eb40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:41.654 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.653+0000 7f3ecdf15700 1 --2- 172.21.15.53:0/283373101 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f3eb40a11d0 0x7f3eb40a3680 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3ec81001c0 tx=0x7f3eb8007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:36:41.657 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.656+0000 7f3ebeffd700 1 -- 172.21.15.53:0/283373101 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3ec0098220 con 0x7f3ec8100cf0 2023-11-16T16:36:41.864 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.863+0000 7f3ed0179700 1 -- 172.21.15.53:0/283373101 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi026=b;smithi026=e;smithi026=h;smithi026=k;smithi053=c;smithi053=f;smithi053=i;smithi053=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ec8069650 con 0x7f3eb40a11d0 2023-11-16T16:36:41.873 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.872+0000 7f3ebeffd700 1 -- 172.21.15.53:0/283373101 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f3ec8069650 con 0x7f3eb40a11d0 2023-11-16T16:36:41.873 INFO:teuthology.orchestra.run.smithi053.stdout:Scheduled mds.all update... 2023-11-16T16:36:41.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.874+0000 7f3ed0179700 1 -- 172.21.15.53:0/283373101 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f3eb40a11d0 msgr2=0x7f3eb40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:41.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.874+0000 7f3ed0179700 1 --2- 172.21.15.53:0/283373101 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f3eb40a11d0 0x7f3eb40a3680 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3ec81001c0 tx=0x7f3eb8007400 comp rx=0 tx=0).stop 2023-11-16T16:36:41.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.874+0000 7f3ed0179700 1 -- 172.21.15.53:0/283373101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ec8100cf0 msgr2=0x7f3ec8101c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:41.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.874+0000 7f3ed0179700 1 --2- 172.21.15.53:0/283373101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ec8100cf0 0x7f3ec8101c80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3ec00062a0 tx=0x7f3ec0004d00 comp rx=0 tx=0).stop 2023-11-16T16:36:41.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.874+0000 7f3ed0179700 1 -- 172.21.15.53:0/283373101 shutdown_connections 2023-11-16T16:36:41.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.874+0000 7f3ed0179700 1 --2- 172.21.15.53:0/283373101 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f3eb40a11d0 0x7f3eb40a3680 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:41.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.874+0000 7f3ed0179700 1 --2- 172.21.15.53:0/283373101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3ec8100cf0 0x7f3ec8101c80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:41.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.874+0000 7f3ed0179700 1 --2- 172.21.15.53:0/283373101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ec80ff160 0x7f3ec81030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:41.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.874+0000 7f3ed0179700 1 --2- 172.21.15.53:0/283373101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ec8100360 0x7f3ec8101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:41.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.874+0000 7f3ed0179700 1 -- 172.21.15.53:0/283373101 >> 172.21.15.53:0/283373101 conn(0x7f3ec80fa790 msgr2=0x7f3ec8068f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:36:41.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.874+0000 7f3ed0179700 1 -- 172.21.15.53:0/283373101 shutdown_connections 2023-11-16T16:36:41.876 INFO:teuthology.orchestra.run.smithi053.stderr:2023-11-16T16:36:41.874+0000 7f3ed0179700 1 -- 172.21.15.53:0/283373101 wait complete. 2023-11-16T16:36:42.319 DEBUG:teuthology.orchestra.run.smithi001:mds.a> sudo journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mds.a.service 2023-11-16T16:36:42.322 DEBUG:teuthology.orchestra.run.smithi001:mds.d> sudo journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mds.d.service 2023-11-16T16:36:42.358 INFO:journalctl@ceph.mds.a.smithi001.stdout:-- Logs begin at Thu 2023-11-16 16:14:27 UTC. -- 2023-11-16T16:36:42.359 DEBUG:teuthology.orchestra.run.smithi001:mds.g> sudo journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mds.g.service 2023-11-16T16:36:42.390 INFO:journalctl@ceph.mds.d.smithi001.stdout:-- Logs begin at Thu 2023-11-16 16:14:27 UTC. -- 2023-11-16T16:36:42.392 DEBUG:teuthology.orchestra.run.smithi001:mds.j> sudo journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mds.j.service 2023-11-16T16:36:42.415 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:42 smithi001 ceph-mon[98800]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:36:42.415 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:42 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:42.415 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:42 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:36:42.415 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:42 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:42.415 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:42 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:36:42.415 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:42 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:42.416 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:42 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:36:42.416 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:42 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:36:42.416 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:42 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:42.416 DEBUG:teuthology.orchestra.run.smithi026:mds.b> sudo journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mds.b.service 2023-11-16T16:36:42.419 DEBUG:teuthology.orchestra.run.smithi026:mds.e> sudo journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mds.e.service 2023-11-16T16:36:42.421 INFO:journalctl@ceph.mds.g.smithi001.stdout:-- Logs begin at Thu 2023-11-16 16:14:27 UTC. -- 2023-11-16T16:36:42.446 INFO:journalctl@ceph.mds.j.smithi001.stdout:-- Logs begin at Thu 2023-11-16 16:14:27 UTC. -- 2023-11-16T16:36:42.449 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:42 smithi026 ceph-mon[100112]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:36:42.449 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:42 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:42.449 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:42 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:36:42.449 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:42 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:42.450 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:42 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:36:42.450 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:42 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:42.450 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:42 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:36:42.450 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:42 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:36:42.450 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:42 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:42.450 DEBUG:teuthology.orchestra.run.smithi026:mds.h> sudo journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mds.h.service 2023-11-16T16:36:42.457 INFO:journalctl@ceph.mds.b.smithi026.stdout:-- Logs begin at Thu 2023-11-16 16:14:13 UTC. -- 2023-11-16T16:36:42.458 DEBUG:teuthology.orchestra.run.smithi026:mds.k> sudo journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mds.k.service 2023-11-16T16:36:42.482 INFO:journalctl@ceph.mds.e.smithi026.stdout:-- Logs begin at Thu 2023-11-16 16:14:13 UTC. -- 2023-11-16T16:36:42.483 DEBUG:teuthology.orchestra.run.smithi053:mds.c> sudo journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mds.c.service 2023-11-16T16:36:42.485 DEBUG:teuthology.orchestra.run.smithi053:mds.f> sudo journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mds.f.service 2023-11-16T16:36:42.488 INFO:journalctl@ceph.mds.h.smithi026.stdout:-- Logs begin at Thu 2023-11-16 16:14:13 UTC. -- 2023-11-16T16:36:42.510 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:42 smithi053 ceph-mon[100638]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:36:42.510 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:42 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:42.510 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:42 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:36:42.510 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:42 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:42.510 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:42 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:36:42.510 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:42 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:42.510 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:42 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:36:42.510 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:42 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:36:42.511 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:42 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:42.511 DEBUG:teuthology.orchestra.run.smithi053:mds.i> sudo journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mds.i.service 2023-11-16T16:36:42.512 INFO:journalctl@ceph.mds.k.smithi026.stdout:-- Logs begin at Thu 2023-11-16 16:14:13 UTC. -- 2023-11-16T16:36:42.516 INFO:journalctl@ceph.mds.c.smithi053.stdout:-- Logs begin at Thu 2023-11-16 16:14:15 UTC. -- 2023-11-16T16:36:42.517 DEBUG:teuthology.orchestra.run.smithi053:mds.l> sudo journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mds.l.service 2023-11-16T16:36:42.539 INFO:journalctl@ceph.mds.f.smithi053.stdout:-- Logs begin at Thu 2023-11-16 16:14:15 UTC. -- 2023-11-16T16:36:42.539 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-11-16T16:36:42.540 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-16T16:36:42.544 INFO:journalctl@ceph.mds.i.smithi053.stdout:-- Logs begin at Thu 2023-11-16 16:14:15 UTC. -- 2023-11-16T16:36:42.566 INFO:journalctl@ceph.mds.l.smithi053.stdout:-- Logs begin at Thu 2023-11-16 16:14:15 UTC. -- 2023-11-16T16:36:42.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.680+0000 7f28578bb700 1 -- 172.21.15.1:0/352999702 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28501003b0 msgr2=0x7f2850100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:42.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.680+0000 7f28578bb700 1 --2- 172.21.15.1:0/352999702 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28501003b0 0x7f2850100800 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f284c009990 tx=0x7f284c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:36:42.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.680+0000 7f28578bb700 1 -- 172.21.15.1:0/352999702 shutdown_connections 2023-11-16T16:36:42.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.680+0000 7f28578bb700 1 --2- 172.21.15.1:0/352999702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2850100d40 0x7f2850107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:42.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.680+0000 7f28578bb700 1 --2- 172.21.15.1:0/352999702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28500ff1c0 0x7f28500ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:42.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.680+0000 7f28578bb700 1 --2- 172.21.15.1:0/352999702 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28501003b0 0x7f2850100800 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:42.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.680+0000 7f28578bb700 1 -- 172.21.15.1:0/352999702 >> 172.21.15.1:0/352999702 conn(0x7f28500fa7a0 msgr2=0x7f28500fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:36:42.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.681+0000 7f28578bb700 1 -- 172.21.15.1:0/352999702 shutdown_connections 2023-11-16T16:36:42.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.681+0000 7f28578bb700 1 -- 172.21.15.1:0/352999702 wait complete. 2023-11-16T16:36:42.685 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.681+0000 7f28578bb700 1 Processor -- start 2023-11-16T16:36:42.685 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.681+0000 7f28578bb700 1 -- start start 2023-11-16T16:36:42.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.682+0000 7f28578bb700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28500ff1c0 0x7f2850069c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:42.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.682+0000 7f28578bb700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28501003b0 0x7f285019add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:42.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.682+0000 7f28578bb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2850100d40 0x7f285019d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:42.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.682+0000 7f28578bb700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285019dbf0 con 0x7f28500ff1c0 2023-11-16T16:36:42.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.682+0000 7f28578bb700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285019dd60 con 0x7f2850100d40 2023-11-16T16:36:42.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.682+0000 7f28578bb700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285019df20 con 0x7f28501003b0 2023-11-16T16:36:42.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.682+0000 7f2855657700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28500ff1c0 0x7f2850069c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:42.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.682+0000 7f2855657700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28500ff1c0 0x7f2850069c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41104/0 (socket says 172.21.15.1:41104) 2023-11-16T16:36:42.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.682+0000 7f2855657700 1 -- 172.21.15.1:0/1592555686 learned_addr learned my addr 172.21.15.1:0/1592555686 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:36:42.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.682+0000 7f2854e56700 1 --2- 172.21.15.1:0/1592555686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28501003b0 0x7f285019add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:42.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.682+0000 7f2855e58700 1 --2- 172.21.15.1:0/1592555686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2850100d40 0x7f285019d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:42.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.682+0000 7f2855657700 1 -- 172.21.15.1:0/1592555686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28501003b0 msgr2=0x7f285019add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:42.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.682+0000 7f2855657700 1 --2- 172.21.15.1:0/1592555686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28501003b0 0x7f285019add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:42.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.682+0000 7f2855657700 1 -- 172.21.15.1:0/1592555686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2850100d40 msgr2=0x7f285019d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:42.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.682+0000 7f2855657700 1 --2- 172.21.15.1:0/1592555686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2850100d40 0x7f285019d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:42.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.682+0000 7f2855657700 1 -- 172.21.15.1:0/1592555686 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f284c0095f0 con 0x7f28500ff1c0 2023-11-16T16:36:42.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.682+0000 7f2855657700 1 --2- 172.21.15.1:0/1592555686 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28500ff1c0 0x7f2850069c10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f284000b500 tx=0x7f284000b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:36:42.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.682+0000 7f28467fc700 1 -- 172.21.15.1:0/1592555686 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2840011830 con 0x7f28500ff1c0 2023-11-16T16:36:42.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.682+0000 7f28578bb700 1 -- 172.21.15.1:0/1592555686 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f285010abc0 con 0x7f28500ff1c0 2023-11-16T16:36:42.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.682+0000 7f28467fc700 1 -- 172.21.15.1:0/1592555686 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2840011e50 con 0x7f28500ff1c0 2023-11-16T16:36:42.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.682+0000 7f28467fc700 1 -- 172.21.15.1:0/1592555686 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2840010dc0 con 0x7f28500ff1c0 2023-11-16T16:36:42.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.683+0000 7f28578bb700 1 -- 172.21.15.1:0/1592555686 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f285010aa30 con 0x7f28500ff1c0 2023-11-16T16:36:42.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.684+0000 7f28578bb700 1 -- 172.21.15.1:0/1592555686 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2850106110 con 0x7f28500ff1c0 2023-11-16T16:36:42.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.685+0000 7f28467fc700 1 -- 172.21.15.1:0/1592555686 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f2840010380 con 0x7f28500ff1c0 2023-11-16T16:36:42.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.686+0000 7f28467fc700 1 --2- 172.21.15.1:0/1592555686 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f283c0a1150 0x7f283c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:42.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.686+0000 7f2854e56700 1 --2- 172.21.15.1:0/1592555686 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f283c0a1150 0x7f283c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:42.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.686+0000 7f28467fc700 1 -- 172.21.15.1:0/1592555686 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f28400cc410 con 0x7f28500ff1c0 2023-11-16T16:36:42.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.686+0000 7f2854e56700 1 --2- 172.21.15.1:0/1592555686 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f283c0a1150 0x7f283c0a3600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f284c00b2e0 tx=0x7f284c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:36:42.690 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.689+0000 7f28467fc700 1 -- 172.21.15.1:0/1592555686 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2840096000 con 0x7f28500ff1c0 2023-11-16T16:36:42.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.887+0000 7f28578bb700 1 -- 172.21.15.1:0/1592555686 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f285002cfe0 con 0x7f28500ff1c0 2023-11-16T16:36:42.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.888+0000 7f28467fc700 1 -- 172.21.15.1:0/1592555686 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18291 (secure 0 0 0) 0x7f2840099620 con 0x7f28500ff1c0 2023-11-16T16:36:42.891 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:36:42.891 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":80,"fsid":"35d9b010-849d-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:30:06.217778+0000","modified":"2023-11-16T16:36:32.736561+0000","last_up_change":"2023-11-16T16:36:31.735403+0000","last_in_change":"2023-11-16T16:36:14.083021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:33:15.916723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a8480507-5275-4e52-953b-626110beb436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6803","nonce":2469671609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6805","nonce":2469671609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6809","nonce":2469671609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6807","nonce":2469671609}]},"public_addr":"172.21.15.1:6803/2469671609","cluster_addr":"172.21.15.1:6805/2469671609","heartbeat_back_addr":"172.21.15.1:6809/2469671609","heartbeat_front_addr":"172.21.15.1:6807/2469671609","state":["exists","up"]},{"osd":1,"uuid":"1d011db2-99d1-4575-bdc5-51f88a308fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6801","nonce":247994796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6803","nonce":247994796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6807","nonce":247994796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6805","nonce":247994796}]},"public_addr":"172.21.15.26:6801/247994796","cluster_addr":"172.21.15.26:6803/247994796","heartbeat_back_addr":"172.21.15.26:6807/247994796","heartbeat_front_addr":"172.21.15.26:6805/247994796","state":["exists","up"]},{"osd":2,"uuid":"4d7a2bd1-3c8e-4546-a243-f0354473b031","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6801","nonce":1659316290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6803","nonce":1659316290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6807","nonce":1659316290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6805","nonce":1659316290}]},"public_addr":"172.21.15.53:6801/1659316290","cluster_addr":"172.21.15.53:6803/1659316290","heartbeat_back_addr":"172.21.15.53:6807/1659316290","heartbeat_front_addr":"172.21.15.53:6805/1659316290","state":["exists","up"]},{"osd":3,"uuid":"9c290fd9-e121-4a81-9687-b0eb93debd86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6811","nonce":2728987281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6813","nonce":2728987281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6817","nonce":2728987281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6815","nonce":2728987281}]},"public_addr":"172.21.15.1:6811/2728987281","cluster_addr":"172.21.15.1:6813/2728987281","heartbeat_back_addr":"172.21.15.1:6817/2728987281","heartbeat_front_addr":"172.21.15.1:6815/2728987281","state":["exists","up"]},{"osd":4,"uuid":"533226bc-cc57-4f15-9e29-820294d57e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6809","nonce":2954740024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6811","nonce":2954740024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6815","nonce":2954740024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6813","nonce":2954740024}]},"public_addr":"172.21.15.26:6809/2954740024","cluster_addr":"172.21.15.26:6811/2954740024","heartbeat_back_addr":"172.21.15.26:6815/2954740024","heartbeat_front_addr":"172.21.15.26:6813/2954740024","state":["exists","up"]},{"osd":5,"uuid":"d3ab25eb-e1d4-4557-bc89-efc9072401ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6809","nonce":3966666318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6811","nonce":3966666318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6815","nonce":3966666318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6813","nonce":3966666318}]},"public_addr":"172.21.15.53:6809/3966666318","cluster_addr":"172.21.15.53:6811/3966666318","heartbeat_back_addr":"172.21.15.53:6815/3966666318","heartbeat_front_addr":"172.21.15.53:6813/3966666318","state":["exists","up"]},{"osd":6,"uuid":"180f506e-afb8-4043-b0b4-2dc86f6d4bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6819","nonce":3364032646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6821","nonce":3364032646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6825","nonce":3364032646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6823","nonce":3364032646}]},"public_addr":"172.21.15.1:6819/3364032646","cluster_addr":"172.21.15.1:6821/3364032646","heartbeat_back_addr":"172.21.15.1:6825/3364032646","heartbeat_front_addr":"172.21.15.1:6823/3364032646","state":["exists","up"]},{"osd":7,"uuid":"0d837bac-194f-498d-b7fb-b3147e992329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6817","nonce":3766901391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6819","nonce":3766901391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6823","nonce":3766901391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6821","nonce":3766901391}]},"public_addr":"172.21.15.26:6817/3766901391","cluster_addr":"172.21.15.26:6819/3766901391","heartbeat_back_addr":"172.21.15.26:6823/3766901391","heartbeat_front_addr":"172.21.15.26:6821/3766901391","state":["exists","up"]},{"osd":8,"uuid":"933f231a-7454-47b0-9291-f1fa64f0baec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6817","nonce":3224335871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6819","nonce":3224335871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6823","nonce":3224335871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6821","nonce":3224335871}]},"public_addr":"172.21.15.53:6817/3224335871","cluster_addr":"172.21.15.53:6819/3224335871","heartbeat_back_addr":"172.21.15.53:6823/3224335871","heartbeat_front_addr":"172.21.15.53:6821/3224335871","state":["exists","up"]},{"osd":9,"uuid":"245fcbb3-c651-4703-ab80-309dc98bd983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6827","nonce":1391199844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6829","nonce":1391199844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6833","nonce":1391199844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6831","nonce":1391199844}]},"public_addr":"172.21.15.1:6827/1391199844","cluster_addr":"172.21.15.1:6829/1391199844","heartbeat_back_addr":"172.21.15.1:6833/1391199844","heartbeat_front_addr":"172.21.15.1:6831/1391199844","state":["exists","up"]},{"osd":10,"uuid":"2c090e7a-5240-4d52-ae4f-e94fe05383d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6825","nonce":1773545447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6827","nonce":1773545447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6831","nonce":1773545447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6829","nonce":1773545447}]},"public_addr":"172.21.15.26:6825/1773545447","cluster_addr":"172.21.15.26:6827/1773545447","heartbeat_back_addr":"172.21.15.26:6831/1773545447","heartbeat_front_addr":"172.21.15.26:6829/1773545447","state":["exists","up"]},{"osd":11,"uuid":"4511e32d-e469-45e1-8310-0ff400cb7b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6825","nonce":2564669546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6827","nonce":2564669546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6831","nonce":2564669546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6829","nonce":2564669546}]},"public_addr":"172.21.15.53:6825/2564669546","cluster_addr":"172.21.15.53:6827/2564669546","heartbeat_back_addr":"172.21.15.53:6831/2564669546","heartbeat_front_addr":"172.21.15.53:6829/2564669546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:28.140250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:49.885461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:12.521487+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:34.926197+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:56.815594+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:18.387751+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:40.587106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:02.319370+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:23.832298+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:46.338040+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:08.730414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:30.050070+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1494336948":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6801/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6800/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6800/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/1641976669":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/2895923638":"2023-11-17T16:30:35.064289+0000","172.21.15.1:6801/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/78811711":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/4131957708":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/344403698":"2023-11-17T16:30:35.064289+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:36:42.892 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.890+0000 7f28578bb700 1 -- 172.21.15.1:0/1592555686 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f283c0a1150 msgr2=0x7f283c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:42.893 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.890+0000 7f28578bb700 1 --2- 172.21.15.1:0/1592555686 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f283c0a1150 0x7f283c0a3600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f284c00b2e0 tx=0x7f284c005960 comp rx=0 tx=0).stop 2023-11-16T16:36:42.893 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.890+0000 7f28578bb700 1 -- 172.21.15.1:0/1592555686 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28500ff1c0 msgr2=0x7f2850069c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:42.893 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.890+0000 7f28578bb700 1 --2- 172.21.15.1:0/1592555686 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28500ff1c0 0x7f2850069c10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f284000b500 tx=0x7f284000b810 comp rx=0 tx=0).stop 2023-11-16T16:36:42.893 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.891+0000 7f28578bb700 1 -- 172.21.15.1:0/1592555686 shutdown_connections 2023-11-16T16:36:42.893 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.891+0000 7f28578bb700 1 --2- 172.21.15.1:0/1592555686 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f283c0a1150 0x7f283c0a3600 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:42.893 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.891+0000 7f28578bb700 1 --2- 172.21.15.1:0/1592555686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2850100d40 0x7f285019d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:42.893 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.891+0000 7f28578bb700 1 --2- 172.21.15.1:0/1592555686 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28500ff1c0 0x7f2850069c10 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:42.893 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.891+0000 7f28578bb700 1 --2- 172.21.15.1:0/1592555686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28501003b0 0x7f285019add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:42.893 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.891+0000 7f28578bb700 1 -- 172.21.15.1:0/1592555686 >> 172.21.15.1:0/1592555686 conn(0x7f28500fa7a0 msgr2=0x7f28500fca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:36:42.893 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.891+0000 7f28578bb700 1 -- 172.21.15.1:0/1592555686 shutdown_connections 2023-11-16T16:36:42.893 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:42.891+0000 7f28578bb700 1 -- 172.21.15.1:0/1592555686 wait complete. 2023-11-16T16:36:42.905 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-16T16:33:15.916723+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-16T16:36:42.906 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-16T16:36:43.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.033+0000 7f71c0518700 1 -- 172.21.15.1:0/3009134356 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71b8100290 msgr2=0x7f71b81006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:43.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.033+0000 7f71c0518700 1 --2- 172.21.15.1:0/3009134356 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71b8100290 0x7f71b81006a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f71ac009990 tx=0x7f71ac009ca0 comp rx=0 tx=0).stop 2023-11-16T16:36:43.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.033+0000 7f71c0518700 1 -- 172.21.15.1:0/3009134356 shutdown_connections 2023-11-16T16:36:43.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.033+0000 7f71c0518700 1 --2- 172.21.15.1:0/3009134356 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71b8101e10 0x7f71b81086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:43.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.033+0000 7f71c0518700 1 --2- 172.21.15.1:0/3009134356 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71b8100290 0x7f71b81006a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:43.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.033+0000 7f71c0518700 1 --2- 172.21.15.1:0/3009134356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71b8101480 0x7f71b81018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:43.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.033+0000 7f71c0518700 1 -- 172.21.15.1:0/3009134356 >> 172.21.15.1:0/3009134356 conn(0x7f71b80fb870 msgr2=0x7f71b80fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:36:43.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.033+0000 7f71c0518700 1 -- 172.21.15.1:0/3009134356 shutdown_connections 2023-11-16T16:36:43.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.033+0000 7f71c0518700 1 -- 172.21.15.1:0/3009134356 wait complete. 2023-11-16T16:36:43.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.034+0000 7f71c0518700 1 Processor -- start 2023-11-16T16:36:43.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.034+0000 7f71c0518700 1 -- start start 2023-11-16T16:36:43.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.034+0000 7f71c0518700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71b8100290 0x7f71b80748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:43.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.034+0000 7f71c0518700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71b8101480 0x7f71b8072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:43.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.034+0000 7f71c0518700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71b8101e10 0x7f71b8073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:43.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.034+0000 7f71c0518700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b8073bc0 con 0x7f71b8101e10 2023-11-16T16:36:43.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.034+0000 7f71c0518700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b8073d30 con 0x7f71b8100290 2023-11-16T16:36:43.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.034+0000 7f71c0518700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b81a0010 con 0x7f71b8101480 2023-11-16T16:36:43.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.034+0000 7f71beab5700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71b8101e10 0x7f71b8073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:43.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.034+0000 7f71beab5700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71b8101e10 0x7f71b8073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41122/0 (socket says 172.21.15.1:41122) 2023-11-16T16:36:43.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.034+0000 7f71beab5700 1 -- 172.21.15.1:0/176766192 learned_addr learned my addr 172.21.15.1:0/176766192 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:36:43.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.034+0000 7f71be2b4700 1 --2- 172.21.15.1:0/176766192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71b8100290 0x7f71b80748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:43.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.034+0000 7f71bdab3700 1 --2- 172.21.15.1:0/176766192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71b8101480 0x7f71b8072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:43.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.034+0000 7f71beab5700 1 -- 172.21.15.1:0/176766192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71b8101480 msgr2=0x7f71b8072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:43.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.034+0000 7f71beab5700 1 --2- 172.21.15.1:0/176766192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71b8101480 0x7f71b8072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:43.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.034+0000 7f71beab5700 1 -- 172.21.15.1:0/176766192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71b8100290 msgr2=0x7f71b80748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:43.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.034+0000 7f71beab5700 1 --2- 172.21.15.1:0/176766192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71b8100290 0x7f71b80748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:43.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.034+0000 7f71beab5700 1 -- 172.21.15.1:0/176766192 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71ac0095f0 con 0x7f71b8101e10 2023-11-16T16:36:43.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.034+0000 7f71beab5700 1 --2- 172.21.15.1:0/176766192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71b8101e10 0x7f71b8073420 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f71b000ef30 tx=0x7f71b000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:36:43.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.035+0000 7f71ab7fe700 1 -- 172.21.15.1:0/176766192 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b0018070 con 0x7f71b8101e10 2023-11-16T16:36:43.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.035+0000 7f71ab7fe700 1 -- 172.21.15.1:0/176766192 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71b000f040 con 0x7f71b8101e10 2023-11-16T16:36:43.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.035+0000 7f71ab7fe700 1 -- 172.21.15.1:0/176766192 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b000cd00 con 0x7f71b8101e10 2023-11-16T16:36:43.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.035+0000 7f71c0518700 1 -- 172.21.15.1:0/176766192 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b810bbf0 con 0x7f71b8101e10 2023-11-16T16:36:43.040 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.035+0000 7f71c0518700 1 -- 172.21.15.1:0/176766192 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b810ba60 con 0x7f71b8101e10 2023-11-16T16:36:43.040 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.036+0000 7f71c0518700 1 -- 172.21.15.1:0/176766192 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71b81a0650 con 0x7f71b8101e10 2023-11-16T16:36:43.041 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.037+0000 7f71ab7fe700 1 -- 172.21.15.1:0/176766192 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f71b0003a00 con 0x7f71b8101e10 2023-11-16T16:36:43.041 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.037+0000 7f71ab7fe700 1 --2- 172.21.15.1:0/176766192 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f71a40a11d0 0x7f71a40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:43.041 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.037+0000 7f71be2b4700 1 --2- 172.21.15.1:0/176766192 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f71a40a11d0 0x7f71a40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:43.041 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.038+0000 7f71ab7fe700 1 -- 172.21.15.1:0/176766192 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f71b00cbf60 con 0x7f71b8101e10 2023-11-16T16:36:43.041 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.038+0000 7f71be2b4700 1 --2- 172.21.15.1:0/176766192 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f71a40a11d0 0x7f71a40a3680 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f71ac006010 tx=0x7f71ac005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:36:43.043 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.042+0000 7f71ab7fe700 1 -- 172.21.15.1:0/176766192 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f71b0095be0 con 0x7f71b8101e10 2023-11-16T16:36:43.245 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.244+0000 7f71c0518700 1 -- 172.21.15.1:0/176766192 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f71b8107250 con 0x7f71b8101e10 2023-11-16T16:36:43.246 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.244+0000 7f71ab7fe700 1 -- 172.21.15.1:0/176766192 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f71b0099200 con 0x7f71b8101e10 2023-11-16T16:36:43.246 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2023-11-16T16:36:43.248 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.246+0000 7f71c0518700 1 -- 172.21.15.1:0/176766192 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f71a40a11d0 msgr2=0x7f71a40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:43.248 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.246+0000 7f71c0518700 1 --2- 172.21.15.1:0/176766192 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f71a40a11d0 0x7f71a40a3680 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f71ac006010 tx=0x7f71ac005960 comp rx=0 tx=0).stop 2023-11-16T16:36:43.248 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.246+0000 7f71c0518700 1 -- 172.21.15.1:0/176766192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71b8101e10 msgr2=0x7f71b8073420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:43.249 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.246+0000 7f71c0518700 1 --2- 172.21.15.1:0/176766192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71b8101e10 0x7f71b8073420 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f71b000ef30 tx=0x7f71b000c3a0 comp rx=0 tx=0).stop 2023-11-16T16:36:43.249 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.246+0000 7f71c0518700 1 -- 172.21.15.1:0/176766192 shutdown_connections 2023-11-16T16:36:43.249 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.246+0000 7f71c0518700 1 --2- 172.21.15.1:0/176766192 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f71a40a11d0 0x7f71a40a3680 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:43.249 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.246+0000 7f71c0518700 1 --2- 172.21.15.1:0/176766192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71b8101e10 0x7f71b8073420 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:43.249 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.246+0000 7f71c0518700 1 --2- 172.21.15.1:0/176766192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71b8100290 0x7f71b80748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:43.249 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.246+0000 7f71c0518700 1 --2- 172.21.15.1:0/176766192 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71b8101480 0x7f71b8072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:43.249 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.246+0000 7f71c0518700 1 -- 172.21.15.1:0/176766192 >> 172.21.15.1:0/176766192 conn(0x7f71b80fb870 msgr2=0x7f71b80fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:36:43.249 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.246+0000 7f71c0518700 1 -- 172.21.15.1:0/176766192 shutdown_connections 2023-11-16T16:36:43.249 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.246+0000 7f71c0518700 1 -- 172.21.15.1:0/176766192 wait complete. 2023-11-16T16:36:43.260 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:36:43.387 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.385+0000 7fcd48e6d700 1 -- 172.21.15.1:0/1034553599 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd44100dc0 msgr2=0x7fcd441031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:43.387 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.385+0000 7fcd48e6d700 1 --2- 172.21.15.1:0/1034553599 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd44100dc0 0x7fcd441031a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcd34009990 tx=0x7fcd34009ca0 comp rx=0 tx=0).stop 2023-11-16T16:36:43.387 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.385+0000 7fcd48e6d700 1 -- 172.21.15.1:0/1034553599 shutdown_connections 2023-11-16T16:36:43.387 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.385+0000 7fcd48e6d700 1 --2- 172.21.15.1:0/1034553599 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd44106000 0x7fcd441083e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:43.387 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.385+0000 7fcd48e6d700 1 --2- 172.21.15.1:0/1034553599 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd44100dc0 0x7fcd441031a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:43.387 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.385+0000 7fcd48e6d700 1 --2- 172.21.15.1:0/1034553599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd441036e0 0x7fcd44105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:43.387 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.385+0000 7fcd48e6d700 1 -- 172.21.15.1:0/1034553599 >> 172.21.15.1:0/1034553599 conn(0x7fcd440fa7c0 msgr2=0x7fcd440fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:36:43.387 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.385+0000 7fcd48e6d700 1 -- 172.21.15.1:0/1034553599 shutdown_connections 2023-11-16T16:36:43.387 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.385+0000 7fcd48e6d700 1 -- 172.21.15.1:0/1034553599 wait complete. 2023-11-16T16:36:43.390 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.386+0000 7fcd48e6d700 1 Processor -- start 2023-11-16T16:36:43.390 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.386+0000 7fcd48e6d700 1 -- start start 2023-11-16T16:36:43.390 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.386+0000 7fcd48e6d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd44100dc0 0x7fcd44196010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:43.390 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.386+0000 7fcd48e6d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd441036e0 0x7fcd44196550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:43.390 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.386+0000 7fcd48e6d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd44106000 0x7fcd4419d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:43.390 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.386+0000 7fcd48e6d700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4419db90 con 0x7fcd44106000 2023-11-16T16:36:43.390 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.386+0000 7fcd48e6d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4419dcd0 con 0x7fcd44100dc0 2023-11-16T16:36:43.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.386+0000 7fcd48e6d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4419de60 con 0x7fcd441036e0 2023-11-16T16:36:43.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.386+0000 7fcd42d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd44106000 0x7fcd4419d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:43.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.386+0000 7fcd42d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd44106000 0x7fcd4419d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41136/0 (socket says 172.21.15.1:41136) 2023-11-16T16:36:43.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.386+0000 7fcd42d9d700 1 -- 172.21.15.1:0/1580553721 learned_addr learned my addr 172.21.15.1:0/1580553721 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:36:43.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.386+0000 7fcd41d9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd441036e0 0x7fcd44196550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:43.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.386+0000 7fcd4259c700 1 --2- 172.21.15.1:0/1580553721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd44100dc0 0x7fcd44196010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:43.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.386+0000 7fcd42d9d700 1 -- 172.21.15.1:0/1580553721 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd441036e0 msgr2=0x7fcd44196550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:43.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.386+0000 7fcd42d9d700 1 --2- 172.21.15.1:0/1580553721 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd441036e0 0x7fcd44196550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:43.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.386+0000 7fcd42d9d700 1 -- 172.21.15.1:0/1580553721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd44100dc0 msgr2=0x7fcd44196010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:43.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.386+0000 7fcd42d9d700 1 --2- 172.21.15.1:0/1580553721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd44100dc0 0x7fcd44196010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:43.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.386+0000 7fcd42d9d700 1 -- 172.21.15.1:0/1580553721 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd340095f0 con 0x7fcd44106000 2023-11-16T16:36:43.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.386+0000 7fcd4259c700 1 --2- 172.21.15.1:0/1580553721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd44100dc0 0x7fcd44196010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:36:43.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.387+0000 7fcd42d9d700 1 --2- 172.21.15.1:0/1580553721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd44106000 0x7fcd4419d5c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c00b970 tx=0x7fcd3c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:36:43.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.387+0000 7fcd337fe700 1 -- 172.21.15.1:0/1580553721 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c00d860 con 0x7fcd44106000 2023-11-16T16:36:43.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.387+0000 7fcd337fe700 1 -- 172.21.15.1:0/1580553721 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd3c00de80 con 0x7fcd44106000 2023-11-16T16:36:43.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.387+0000 7fcd48e6d700 1 -- 172.21.15.1:0/1580553721 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd4410bc80 con 0x7fcd44106000 2023-11-16T16:36:43.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.387+0000 7fcd337fe700 1 -- 172.21.15.1:0/1580553721 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c017400 con 0x7fcd44106000 2023-11-16T16:36:43.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.387+0000 7fcd48e6d700 1 -- 172.21.15.1:0/1580553721 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd4410baf0 con 0x7fcd44106000 2023-11-16T16:36:43.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.388+0000 7fcd48e6d700 1 -- 172.21.15.1:0/1580553721 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd440fc330 con 0x7fcd44106000 2023-11-16T16:36:43.393 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.389+0000 7fcd337fe700 1 -- 172.21.15.1:0/1580553721 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fcd3c00e630 con 0x7fcd44106000 2023-11-16T16:36:43.393 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.390+0000 7fcd337fe700 1 --2- 172.21.15.1:0/1580553721 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fcd2c0a1150 0x7fcd2c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:43.393 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.390+0000 7fcd4259c700 1 --2- 172.21.15.1:0/1580553721 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fcd2c0a1150 0x7fcd2c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:43.393 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.390+0000 7fcd337fe700 1 -- 172.21.15.1:0/1580553721 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcd3c006e40 con 0x7fcd44106000 2023-11-16T16:36:43.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.390+0000 7fcd4259c700 1 --2- 172.21.15.1:0/1580553721 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fcd2c0a1150 0x7fcd2c0a3600 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcd3400b5c0 tx=0x7fcd34005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:36:43.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.393+0000 7fcd337fe700 1 -- 172.21.15.1:0/1580553721 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fcd3c095e70 con 0x7fcd44106000 2023-11-16T16:36:43.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:43 smithi001 ceph-mon[98800]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi026=b;smithi026=e;smithi026=h;smithi026=k;smithi053=c;smithi053=f;smithi053=i;smithi053=l", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:36:43.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:43 smithi001 ceph-mon[98800]: Saving service mds.all spec with placement smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi026=b;smithi026=e;smithi026=h;smithi026=k;smithi053=c;smithi053=f;smithi053=i;smithi053=l;count:12 2023-11-16T16:36:43.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:43 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:36:43.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:43 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:36:43.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:43 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:36:43.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:43 smithi001 ceph-mon[98800]: Service mds.all has no related daemon already placed. 2023-11-16T16:36:43.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:43 smithi001 ceph-mon[98800]: Deploying daemon mds.l on smithi053 2023-11-16T16:36:43.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:43 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1592555686' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:36:43.469 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:43 smithi053 ceph-mon[100638]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi026=b;smithi026=e;smithi026=h;smithi026=k;smithi053=c;smithi053=f;smithi053=i;smithi053=l", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:36:43.469 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:43 smithi053 ceph-mon[100638]: Saving service mds.all spec with placement smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi026=b;smithi026=e;smithi026=h;smithi026=k;smithi053=c;smithi053=f;smithi053=i;smithi053=l;count:12 2023-11-16T16:36:43.469 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:43 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:36:43.469 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:43 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:36:43.469 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:43 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:36:43.469 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:43 smithi053 ceph-mon[100638]: Service mds.all has no related daemon already placed. 2023-11-16T16:36:43.470 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:43 smithi053 ceph-mon[100638]: Deploying daemon mds.l on smithi053 2023-11-16T16:36:43.470 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:43 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1592555686' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:36:43.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:43 smithi026 ceph-mon[100112]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi026=b;smithi026=e;smithi026=h;smithi026=k;smithi053=c;smithi053=f;smithi053=i;smithi053=l", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:36:43.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:43 smithi026 ceph-mon[100112]: Saving service mds.all spec with placement smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi026=b;smithi026=e;smithi026=h;smithi026=k;smithi053=c;smithi053=f;smithi053=i;smithi053=l;count:12 2023-11-16T16:36:43.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:43 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:36:43.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:43 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:36:43.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:43 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:36:43.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:43 smithi026 ceph-mon[100112]: Service mds.all has no related daemon already placed. 2023-11-16T16:36:43.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:43 smithi026 ceph-mon[100112]: Deploying daemon mds.l on smithi053 2023-11-16T16:36:43.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:43 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1592555686' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:36:43.638 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.637+0000 7fcd48e6d700 1 -- 172.21.15.1:0/1580553721 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd4402cfe0 con 0x7fcd44106000 2023-11-16T16:36:43.639 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.637+0000 7fcd337fe700 1 -- 172.21.15.1:0/1580553721 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7fcd3c099490 con 0x7fcd44106000 2023-11-16T16:36:43.639 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:36:43.639 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-11-16T16:36:43.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.639+0000 7fcd48e6d700 1 -- 172.21.15.1:0/1580553721 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fcd2c0a1150 msgr2=0x7fcd2c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:43.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.639+0000 7fcd48e6d700 1 --2- 172.21.15.1:0/1580553721 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fcd2c0a1150 0x7fcd2c0a3600 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcd3400b5c0 tx=0x7fcd34005fb0 comp rx=0 tx=0).stop 2023-11-16T16:36:43.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.639+0000 7fcd48e6d700 1 -- 172.21.15.1:0/1580553721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd44106000 msgr2=0x7fcd4419d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:43.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.639+0000 7fcd48e6d700 1 --2- 172.21.15.1:0/1580553721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd44106000 0x7fcd4419d5c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c00b970 tx=0x7fcd3c00bc80 comp rx=0 tx=0).stop 2023-11-16T16:36:43.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.639+0000 7fcd48e6d700 1 -- 172.21.15.1:0/1580553721 shutdown_connections 2023-11-16T16:36:43.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.639+0000 7fcd48e6d700 1 --2- 172.21.15.1:0/1580553721 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fcd2c0a1150 0x7fcd2c0a3600 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:43.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.639+0000 7fcd48e6d700 1 --2- 172.21.15.1:0/1580553721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd44106000 0x7fcd4419d5c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:43.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.639+0000 7fcd48e6d700 1 --2- 172.21.15.1:0/1580553721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd44100dc0 0x7fcd44196010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:43.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.639+0000 7fcd48e6d700 1 --2- 172.21.15.1:0/1580553721 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcd441036e0 0x7fcd44196550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:43.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.639+0000 7fcd48e6d700 1 -- 172.21.15.1:0/1580553721 >> 172.21.15.1:0/1580553721 conn(0x7fcd440fa7c0 msgr2=0x7fcd44101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:36:43.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.639+0000 7fcd48e6d700 1 -- 172.21.15.1:0/1580553721 shutdown_connections 2023-11-16T16:36:43.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:43.639+0000 7fcd48e6d700 1 -- 172.21.15.1:0/1580553721 wait complete. 2023-11-16T16:36:43.642 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 1 2023-11-16T16:36:43.747 INFO:journalctl@ceph.mds.l.smithi053.stdout:Nov 16 16:36:43 smithi053 systemd[1]: Starting Ceph mds.l for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:36:44.014 INFO:journalctl@ceph.mds.l.smithi053.stdout:Nov 16 16:36:43 smithi053 podman[125562]: 2023-11-16T16:36:44.015 INFO:journalctl@ceph.mds.l.smithi053.stdout:Nov 16 16:36:43 smithi053 ceph-mds[125602]: -- 172.21.15.53:0/1646964255 <== mon.1 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559a9f62f520 con 0x559a9f604c00 2023-11-16T16:36:44.015 INFO:journalctl@ceph.mds.l.smithi053.stdout:Nov 16 16:36:43 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-l[125578]: starting mds.l at 2023-11-16T16:36:44.331 INFO:journalctl@ceph.mds.l.smithi053.stdout:Nov 16 16:36:44 smithi053 bash[125562]: 88cb45acd15cdc3c3436aa30e926245ecc2f90358ec867ce1a0816aa66e921bf 2023-11-16T16:36:44.331 INFO:journalctl@ceph.mds.l.smithi053.stdout:Nov 16 16:36:44 smithi053 systemd[1]: Started Ceph mds.l for 35d9b010-849d-11ee-8dbe-212e2dc638e7. 2023-11-16T16:36:44.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:44 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/176766192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:36:44.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:44 smithi053 ceph-mon[100638]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:36:44.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:44 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1580553721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:36:44.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:44 smithi053 ceph-mon[100638]: mds.? [v2:172.21.15.53:6832/3693905267,v1:172.21.15.53:6833/3693905267] up:boot 2023-11-16T16:36:44.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:44 smithi053 ceph-mon[100638]: fsmap 1 up:standby 2023-11-16T16:36:44.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:44 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-11-16T16:36:44.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:44 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:44.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:44 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:44.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:44 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/176766192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:36:44.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:44 smithi001 ceph-mon[98800]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:36:44.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:44 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1580553721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:36:44.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:44 smithi001 ceph-mon[98800]: mds.? [v2:172.21.15.53:6832/3693905267,v1:172.21.15.53:6833/3693905267] up:boot 2023-11-16T16:36:44.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:44 smithi001 ceph-mon[98800]: fsmap 1 up:standby 2023-11-16T16:36:44.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:44 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-11-16T16:36:44.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:44 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:44.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:44 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:44.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:44 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/176766192' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:36:44.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:44 smithi026 ceph-mon[100112]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:36:44.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:44 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1580553721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:36:44.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:44 smithi026 ceph-mon[100112]: mds.? [v2:172.21.15.53:6832/3693905267,v1:172.21.15.53:6833/3693905267] up:boot 2023-11-16T16:36:44.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:44 smithi026 ceph-mon[100112]: fsmap 1 up:standby 2023-11-16T16:36:44.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:44 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-11-16T16:36:44.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:44 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:44.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:44 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:45.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:45 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:45.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:45 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:36:45.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:45 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:36:45.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:45 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:45.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:45 smithi001 ceph-mon[98800]: Deploying daemon mds.k on smithi026 2023-11-16T16:36:45.443 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:45 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:45.443 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:45 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:36:45.444 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:45 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:36:45.444 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:45 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:45.444 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:45 smithi026 ceph-mon[100112]: Deploying daemon mds.k on smithi026 2023-11-16T16:36:45.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:45 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:45.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:45 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:36:45.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:45 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:36:45.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:45 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:45.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:45 smithi053 ceph-mon[100638]: Deploying daemon mds.k on smithi026 2023-11-16T16:36:45.654 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:36:45.794 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.791+0000 7fbcc98ee700 1 -- 172.21.15.1:0/3100755106 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbcc41003d0 msgr2=0x7fbcc4100820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:45.794 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.791+0000 7fbcc98ee700 1 --2- 172.21.15.1:0/3100755106 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbcc41003d0 0x7fbcc4100820 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8009990 tx=0x7fbcb8009ca0 comp rx=0 tx=0).stop 2023-11-16T16:36:45.794 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.792+0000 7fbcc98ee700 1 -- 172.21.15.1:0/3100755106 shutdown_connections 2023-11-16T16:36:45.794 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.792+0000 7fbcc98ee700 1 --2- 172.21.15.1:0/3100755106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcc4100d60 0x7fbcc4107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:45.794 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.792+0000 7fbcc98ee700 1 --2- 172.21.15.1:0/3100755106 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcc40ff1e0 0x7fbcc40ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:45.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.792+0000 7fbcc98ee700 1 --2- 172.21.15.1:0/3100755106 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbcc41003d0 0x7fbcc4100820 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:45.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.792+0000 7fbcc98ee700 1 -- 172.21.15.1:0/3100755106 >> 172.21.15.1:0/3100755106 conn(0x7fbcc40fa7c0 msgr2=0x7fbcc40fcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:36:45.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.792+0000 7fbcc98ee700 1 -- 172.21.15.1:0/3100755106 shutdown_connections 2023-11-16T16:36:45.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.792+0000 7fbcc98ee700 1 -- 172.21.15.1:0/3100755106 wait complete. 2023-11-16T16:36:45.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.792+0000 7fbcc98ee700 1 Processor -- start 2023-11-16T16:36:45.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.792+0000 7fbcc98ee700 1 -- start start 2023-11-16T16:36:45.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.793+0000 7fbcc98ee700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcc40ff1e0 0x7fbcc4069880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:45.798 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.793+0000 7fbcc98ee700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbcc41003d0 0x7fbcc419acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:45.798 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.793+0000 7fbcc98ee700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcc4100d60 0x7fbcc419d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:45.798 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.793+0000 7fbcc98ee700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc419daa0 con 0x7fbcc41003d0 2023-11-16T16:36:45.798 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.793+0000 7fbcc98ee700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc419dc10 con 0x7fbcc40ff1e0 2023-11-16T16:36:45.798 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.793+0000 7fbcc98ee700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc419ddd0 con 0x7fbcc4100d60 2023-11-16T16:36:45.798 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.793+0000 7fbcc27fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbcc41003d0 0x7fbcc419acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:45.798 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.793+0000 7fbcc27fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbcc41003d0 0x7fbcc419acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41148/0 (socket says 172.21.15.1:41148) 2023-11-16T16:36:45.798 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.793+0000 7fbcc27fc700 1 -- 172.21.15.1:0/1300013297 learned_addr learned my addr 172.21.15.1:0/1300013297 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:36:45.798 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.793+0000 7fbcc2ffd700 1 --2- 172.21.15.1:0/1300013297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcc40ff1e0 0x7fbcc4069880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:45.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.793+0000 7fbcc37fe700 1 --2- 172.21.15.1:0/1300013297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcc4100d60 0x7fbcc419d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:45.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.793+0000 7fbcc27fc700 1 -- 172.21.15.1:0/1300013297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcc4100d60 msgr2=0x7fbcc419d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:45.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.793+0000 7fbcc27fc700 1 --2- 172.21.15.1:0/1300013297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcc4100d60 0x7fbcc419d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:45.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.793+0000 7fbcc27fc700 1 -- 172.21.15.1:0/1300013297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcc40ff1e0 msgr2=0x7fbcc4069880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:45.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.793+0000 7fbcc27fc700 1 --2- 172.21.15.1:0/1300013297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcc40ff1e0 0x7fbcc4069880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:45.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.793+0000 7fbcc27fc700 1 -- 172.21.15.1:0/1300013297 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb80095f0 con 0x7fbcc41003d0 2023-11-16T16:36:45.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.793+0000 7fbcc37fe700 1 --2- 172.21.15.1:0/1300013297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcc4100d60 0x7fbcc419d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:36:45.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.793+0000 7fbcc27fc700 1 --2- 172.21.15.1:0/1300013297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbcc41003d0 0x7fbcc419acc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8004760 tx=0x7fbcb8004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:36:45.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.793+0000 7fbcc88ec700 1 -- 172.21.15.1:0/1300013297 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb801d070 con 0x7fbcc41003d0 2023-11-16T16:36:45.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.794+0000 7fbcc88ec700 1 -- 172.21.15.1:0/1300013297 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbcb8004cf0 con 0x7fbcc41003d0 2023-11-16T16:36:45.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.794+0000 7fbcc98ee700 1 -- 172.21.15.1:0/1300013297 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcc410ab50 con 0x7fbcc41003d0 2023-11-16T16:36:45.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.794+0000 7fbcc88ec700 1 -- 172.21.15.1:0/1300013297 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb8005280 con 0x7fbcc41003d0 2023-11-16T16:36:45.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.794+0000 7fbcc98ee700 1 -- 172.21.15.1:0/1300013297 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcc410a9c0 con 0x7fbcc41003d0 2023-11-16T16:36:45.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.794+0000 7fbcc98ee700 1 -- 172.21.15.1:0/1300013297 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcc4106020 con 0x7fbcc41003d0 2023-11-16T16:36:45.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.796+0000 7fbcc88ec700 1 -- 172.21.15.1:0/1300013297 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fbcb80053e0 con 0x7fbcc41003d0 2023-11-16T16:36:45.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.796+0000 7fbcc88ec700 1 --2- 172.21.15.1:0/1300013297 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbcac0a11d0 0x7fbcac0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:45.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.796+0000 7fbcc2ffd700 1 --2- 172.21.15.1:0/1300013297 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbcac0a11d0 0x7fbcac0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:45.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.796+0000 7fbcc88ec700 1 -- 172.21.15.1:0/1300013297 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbcb80cd950 con 0x7fbcc41003d0 2023-11-16T16:36:45.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.797+0000 7fbcc2ffd700 1 --2- 172.21.15.1:0/1300013297 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbcac0a11d0 0x7fbcac0a3680 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4100230 tx=0x7fbcb400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:36:45.802 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:45.800+0000 7fbcc88ec700 1 -- 172.21.15.1:0/1300013297 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbcb80975d0 con 0x7fbcc41003d0 2023-11-16T16:36:46.001 INFO:journalctl@ceph.mds.k.smithi026.stdout:Nov 16 16:36:45 smithi026 systemd[1]: Starting Ceph mds.k for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:36:46.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:46.045+0000 7fbcc98ee700 1 -- 172.21.15.1:0/1300013297 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbcc4061700 con 0x7fbcc41003d0 2023-11-16T16:36:46.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:46.045+0000 7fbcc88ec700 1 -- 172.21.15.1:0/1300013297 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1257 (secure 0 0 0) 0x7fbcb809abf0 con 0x7fbcc41003d0 2023-11-16T16:36:46.047 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:36:46.047 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3693905267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3693905267},{"type":"v1","addr":"172.21.15.53:6833","nonce":3693905267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-11-16T16:36:46.049 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:46.047+0000 7fbcc98ee700 1 -- 172.21.15.1:0/1300013297 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbcac0a11d0 msgr2=0x7fbcac0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:46.049 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:46.047+0000 7fbcc98ee700 1 --2- 172.21.15.1:0/1300013297 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbcac0a11d0 0x7fbcac0a3680 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4100230 tx=0x7fbcb400b410 comp rx=0 tx=0).stop 2023-11-16T16:36:46.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:46.047+0000 7fbcc98ee700 1 -- 172.21.15.1:0/1300013297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbcc41003d0 msgr2=0x7fbcc419acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:46.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:46.047+0000 7fbcc98ee700 1 --2- 172.21.15.1:0/1300013297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbcc41003d0 0x7fbcc419acc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8004760 tx=0x7fbcb8004790 comp rx=0 tx=0).stop 2023-11-16T16:36:46.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:46.047+0000 7fbcc98ee700 1 -- 172.21.15.1:0/1300013297 shutdown_connections 2023-11-16T16:36:46.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:46.047+0000 7fbcc98ee700 1 --2- 172.21.15.1:0/1300013297 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbcac0a11d0 0x7fbcac0a3680 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:46.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:46.047+0000 7fbcc98ee700 1 --2- 172.21.15.1:0/1300013297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcc4100d60 0x7fbcc419d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:46.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:46.047+0000 7fbcc98ee700 1 --2- 172.21.15.1:0/1300013297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcc40ff1e0 0x7fbcc4069880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:46.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:46.047+0000 7fbcc98ee700 1 --2- 172.21.15.1:0/1300013297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbcc41003d0 0x7fbcc419acc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:46.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:46.047+0000 7fbcc98ee700 1 -- 172.21.15.1:0/1300013297 >> 172.21.15.1:0/1300013297 conn(0x7fbcc40fa7c0 msgr2=0x7fbcc40fc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:36:46.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:46.047+0000 7fbcc98ee700 1 -- 172.21.15.1:0/1300013297 shutdown_connections 2023-11-16T16:36:46.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:46.048+0000 7fbcc98ee700 1 -- 172.21.15.1:0/1300013297 wait complete. 2023-11-16T16:36:46.051 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 2 2023-11-16T16:36:46.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:46 smithi026 ceph-mon[100112]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:36:46.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:46 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1300013297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:36:46.258 INFO:journalctl@ceph.mds.k.smithi026.stdout:Nov 16 16:36:46 smithi026 podman[124359]: 2023-11-16T16:36:46.258 INFO:journalctl@ceph.mds.k.smithi026.stdout:Nov 16 16:36:46 smithi026 bash[124359]: b53348018d34ee4d3986a1c84f21a9fe221990d624e8c2b3b60dd2cc6b979ab1 2023-11-16T16:36:46.258 INFO:journalctl@ceph.mds.k.smithi026.stdout:Nov 16 16:36:46 smithi026 ceph-mds[124399]: -- 172.21.15.26:0/64248516 <== mon.2 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559538b25520 con 0x559538afb000 2023-11-16T16:36:46.258 INFO:journalctl@ceph.mds.k.smithi026.stdout:Nov 16 16:36:46 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-k[124375]: starting mds.k at 2023-11-16T16:36:46.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:46 smithi001 ceph-mon[98800]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:36:46.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:46 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1300013297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:36:46.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:46 smithi053 ceph-mon[100638]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:36:46.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:46 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1300013297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:36:46.758 INFO:journalctl@ceph.mds.k.smithi026.stdout:Nov 16 16:36:46 smithi026 systemd[1]: Started Ceph mds.k for 35d9b010-849d-11ee-8dbe-212e2dc638e7. 2023-11-16T16:36:47.480 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:47 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:47.480 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:47 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:47.480 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:47 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:47.480 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:47 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:36:47.480 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:47 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:36:47.480 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:47 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:47.480 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:47 smithi001 ceph-mon[98800]: Deploying daemon mds.j on smithi001 2023-11-16T16:36:47.481 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:47 smithi001 ceph-mon[98800]: mds.? [v2:172.21.15.26:6832/164668033,v1:172.21.15.26:6833/164668033] up:boot 2023-11-16T16:36:47.481 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:47 smithi001 ceph-mon[98800]: fsmap 2 up:standby 2023-11-16T16:36:47.481 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:47 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-11-16T16:36:47.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:47 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:47.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:47 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:47.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:47 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:47.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:47 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:36:47.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:47 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:36:47.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:47 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:47.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:47 smithi026 ceph-mon[100112]: Deploying daemon mds.j on smithi001 2023-11-16T16:36:47.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:47 smithi026 ceph-mon[100112]: mds.? [v2:172.21.15.26:6832/164668033,v1:172.21.15.26:6833/164668033] up:boot 2023-11-16T16:36:47.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:47 smithi026 ceph-mon[100112]: fsmap 2 up:standby 2023-11-16T16:36:47.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:47 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-11-16T16:36:47.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:47 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:47.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:47 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:47.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:47 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:47.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:47 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:36:47.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:47 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:36:47.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:47 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:47.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:47 smithi053 ceph-mon[100638]: Deploying daemon mds.j on smithi001 2023-11-16T16:36:47.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:47 smithi053 ceph-mon[100638]: mds.? [v2:172.21.15.26:6832/164668033,v1:172.21.15.26:6833/164668033] up:boot 2023-11-16T16:36:47.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:47 smithi053 ceph-mon[100638]: fsmap 2 up:standby 2023-11-16T16:36:47.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:47 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-11-16T16:36:48.006 INFO:journalctl@ceph.mds.j.smithi001.stdout:Nov 16 16:36:48 smithi001 systemd[1]: Starting Ceph mds.j for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:36:48.063 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:36:48.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.209+0000 7fdbff9dc700 1 -- 172.21.15.1:0/836318807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbf80fe350 msgr2=0x7fdbf80fe7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:48.212 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.209+0000 7fdbff9dc700 1 --2- 172.21.15.1:0/836318807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbf80fe350 0x7fdbf80fe7a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8009990 tx=0x7fdbe8009ca0 comp rx=0 tx=0).stop 2023-11-16T16:36:48.212 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.210+0000 7fdbff9dc700 1 -- 172.21.15.1:0/836318807 shutdown_connections 2023-11-16T16:36:48.212 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.210+0000 7fdbff9dc700 1 --2- 172.21.15.1:0/836318807 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbf80fece0 0x7fdbf81088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:48.212 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.210+0000 7fdbff9dc700 1 --2- 172.21.15.1:0/836318807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbf81018e0 0x7fdbf8101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:48.212 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.210+0000 7fdbff9dc700 1 --2- 172.21.15.1:0/836318807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbf80fe350 0x7fdbf80fe7a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:48.212 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.210+0000 7fdbff9dc700 1 -- 172.21.15.1:0/836318807 >> 172.21.15.1:0/836318807 conn(0x7fdbf80f9f80 msgr2=0x7fdbf80fc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:36:48.212 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.210+0000 7fdbff9dc700 1 -- 172.21.15.1:0/836318807 shutdown_connections 2023-11-16T16:36:48.212 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.210+0000 7fdbff9dc700 1 -- 172.21.15.1:0/836318807 wait complete. 2023-11-16T16:36:48.214 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.210+0000 7fdbff9dc700 1 Processor -- start 2023-11-16T16:36:48.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.210+0000 7fdbff9dc700 1 -- start start 2023-11-16T16:36:48.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.211+0000 7fdbff9dc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbf80fe350 0x7fdbf8195fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:48.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.211+0000 7fdbff9dc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbf80fece0 0x7fdbf8196510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:48.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.211+0000 7fdbff9dc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbf81018e0 0x7fdbf819d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:48.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.211+0000 7fdbff9dc700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf819db50 con 0x7fdbf80fe350 2023-11-16T16:36:48.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.211+0000 7fdbff9dc700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf819dc90 con 0x7fdbf80fece0 2023-11-16T16:36:48.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.211+0000 7fdbff9dc700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf819de20 con 0x7fdbf81018e0 2023-11-16T16:36:48.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.211+0000 7fdbfd778700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbf80fe350 0x7fdbf8195fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:48.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.211+0000 7fdbfd778700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbf80fe350 0x7fdbf8195fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41160/0 (socket says 172.21.15.1:41160) 2023-11-16T16:36:48.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.211+0000 7fdbfd778700 1 -- 172.21.15.1:0/616158115 learned_addr learned my addr 172.21.15.1:0/616158115 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:36:48.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.211+0000 7fdbfdf79700 1 --2- 172.21.15.1:0/616158115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbf81018e0 0x7fdbf819d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:48.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.211+0000 7fdbfd778700 1 -- 172.21.15.1:0/616158115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbf81018e0 msgr2=0x7fdbf819d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:48.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.211+0000 7fdbfd778700 1 --2- 172.21.15.1:0/616158115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbf81018e0 0x7fdbf819d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:48.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.211+0000 7fdbfcf77700 1 --2- 172.21.15.1:0/616158115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbf80fece0 0x7fdbf8196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:48.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.211+0000 7fdbfd778700 1 -- 172.21.15.1:0/616158115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbf80fece0 msgr2=0x7fdbf8196510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:48.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.211+0000 7fdbfd778700 1 --2- 172.21.15.1:0/616158115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbf80fece0 0x7fdbf8196510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:48.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.211+0000 7fdbfd778700 1 -- 172.21.15.1:0/616158115 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe80095f0 con 0x7fdbf80fe350 2023-11-16T16:36:48.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.211+0000 7fdbfdf79700 1 --2- 172.21.15.1:0/616158115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbf81018e0 0x7fdbf819d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:36:48.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.211+0000 7fdbfcf77700 1 --2- 172.21.15.1:0/616158115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbf80fece0 0x7fdbf8196510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:36:48.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.211+0000 7fdbfd778700 1 --2- 172.21.15.1:0/616158115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbf80fe350 0x7fdbf8195fd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdbf400c970 tx=0x7fdbf400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:36:48.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.211+0000 7fdbee7fc700 1 -- 172.21.15.1:0/616158115 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf4004220 con 0x7fdbf80fe350 2023-11-16T16:36:48.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.212+0000 7fdbff9dc700 1 -- 172.21.15.1:0/616158115 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbf810bc50 con 0x7fdbf80fe350 2023-11-16T16:36:48.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.212+0000 7fdbee7fc700 1 -- 172.21.15.1:0/616158115 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbf4007d60 con 0x7fdbf80fe350 2023-11-16T16:36:48.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.212+0000 7fdbee7fc700 1 -- 172.21.15.1:0/616158115 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf4003d00 con 0x7fdbf80fe350 2023-11-16T16:36:48.217 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.212+0000 7fdbff9dc700 1 -- 172.21.15.1:0/616158115 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbf810bac0 con 0x7fdbf80fe350 2023-11-16T16:36:48.217 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.212+0000 7fdbff9dc700 1 -- 172.21.15.1:0/616158115 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbf8067f40 con 0x7fdbf80fe350 2023-11-16T16:36:48.217 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.214+0000 7fdbee7fc700 1 -- 172.21.15.1:0/616158115 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fdbf4004380 con 0x7fdbf80fe350 2023-11-16T16:36:48.217 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.214+0000 7fdbee7fc700 1 --2- 172.21.15.1:0/616158115 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fdbe40a1150 0x7fdbe40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:48.218 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.214+0000 7fdbfcf77700 1 --2- 172.21.15.1:0/616158115 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fdbe40a1150 0x7fdbe40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:48.218 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.214+0000 7fdbee7fc700 1 -- 172.21.15.1:0/616158115 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdbf40cc400 con 0x7fdbf80fe350 2023-11-16T16:36:48.218 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.215+0000 7fdbfcf77700 1 --2- 172.21.15.1:0/616158115 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fdbe40a1150 0x7fdbe40a3600 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdbe800b5c0 tx=0x7fdbe8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:36:48.220 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.219+0000 7fdbee7fc700 1 -- 172.21.15.1:0/616158115 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fdbf4095ff0 con 0x7fdbf80fe350 2023-11-16T16:36:48.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:48 smithi001 ceph-mon[98800]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:36:48.433 INFO:journalctl@ceph.mds.j.smithi001.stdout:Nov 16 16:36:48 smithi001 podman[132169]: 2023-11-16T16:36:48.472 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.471+0000 7fdbff9dc700 1 -- 172.21.15.1:0/616158115 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdbf81002d0 con 0x7fdbf80fe350 2023-11-16T16:36:48.473 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.471+0000 7fdbee7fc700 1 -- 172.21.15.1:0/616158115 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2011 (secure 0 0 0) 0x7fdbf4099610 con 0x7fdbf80fe350 2023-11-16T16:36:48.473 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:36:48.474 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3693905267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3693905267},{"type":"v1","addr":"172.21.15.53:6833","nonce":3693905267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/164668033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":164668033},{"type":"v1","addr":"172.21.15.26:6833","nonce":164668033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-11-16T16:36:48.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.474+0000 7fdbff9dc700 1 -- 172.21.15.1:0/616158115 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fdbe40a1150 msgr2=0x7fdbe40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:48.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.474+0000 7fdbff9dc700 1 --2- 172.21.15.1:0/616158115 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fdbe40a1150 0x7fdbe40a3600 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdbe800b5c0 tx=0x7fdbe8005fb0 comp rx=0 tx=0).stop 2023-11-16T16:36:48.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.474+0000 7fdbff9dc700 1 -- 172.21.15.1:0/616158115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbf80fe350 msgr2=0x7fdbf8195fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:48.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.474+0000 7fdbff9dc700 1 --2- 172.21.15.1:0/616158115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbf80fe350 0x7fdbf8195fd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdbf400c970 tx=0x7fdbf400cd30 comp rx=0 tx=0).stop 2023-11-16T16:36:48.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.474+0000 7fdbff9dc700 1 -- 172.21.15.1:0/616158115 shutdown_connections 2023-11-16T16:36:48.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.474+0000 7fdbff9dc700 1 --2- 172.21.15.1:0/616158115 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fdbe40a1150 0x7fdbe40a3600 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:48.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.474+0000 7fdbff9dc700 1 --2- 172.21.15.1:0/616158115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdbf81018e0 0x7fdbf819d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:48.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.474+0000 7fdbff9dc700 1 --2- 172.21.15.1:0/616158115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdbf80fe350 0x7fdbf8195fd0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:48.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.474+0000 7fdbff9dc700 1 --2- 172.21.15.1:0/616158115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbf80fece0 0x7fdbf8196510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:48.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.474+0000 7fdbff9dc700 1 -- 172.21.15.1:0/616158115 >> 172.21.15.1:0/616158115 conn(0x7fdbf80f9f80 msgr2=0x7fdbf80fc180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:36:48.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.474+0000 7fdbff9dc700 1 -- 172.21.15.1:0/616158115 shutdown_connections 2023-11-16T16:36:48.477 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:48.474+0000 7fdbff9dc700 1 -- 172.21.15.1:0/616158115 wait complete. 2023-11-16T16:36:48.477 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 3 2023-11-16T16:36:48.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:48 smithi026 ceph-mon[100112]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:36:48.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:48 smithi053 ceph-mon[100638]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:36:48.932 INFO:journalctl@ceph.mds.j.smithi001.stdout:Nov 16 16:36:48 smithi001 bash[132169]: d89b1b8e5201b1539a14d770256b9bd33c6d0704833c6500511d933f36441d4d 2023-11-16T16:36:48.932 INFO:journalctl@ceph.mds.j.smithi001.stdout:Nov 16 16:36:48 smithi001 ceph-mds[132231]: -- 172.21.15.1:0/3844627345 <== mon.0 v2:172.21.15.1:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ef3a945520 con 0x55ef3a91b000 2023-11-16T16:36:48.932 INFO:journalctl@ceph.mds.j.smithi001.stdout:Nov 16 16:36:48 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-j[132205]: starting mds.j at 2023-11-16T16:36:48.932 INFO:journalctl@ceph.mds.j.smithi001.stdout:Nov 16 16:36:48 smithi001 systemd[1]: Started Ceph mds.j for 35d9b010-849d-11ee-8dbe-212e2dc638e7. 2023-11-16T16:36:49.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:49 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/616158115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:36:49.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:49 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:49.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:49 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:49.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:49 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:49.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:49 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:36:49.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:49 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:36:49.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:49 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:49.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:49 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/616158115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:36:49.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:49 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:49.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:49 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:49.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:49 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:49.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:49 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:36:49.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:49 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:36:49.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:49 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:49.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:49 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/616158115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:36:49.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:49 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:49.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:49 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:49.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:49 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:49.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:49 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:36:49.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:49 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:36:49.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:49 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:50.353 INFO:journalctl@ceph.mds.d.smithi001.stdout:Nov 16 16:36:50 smithi001 systemd[1]: Starting Ceph mds.d for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:36:50.354 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:50 smithi001 ceph-mon[98800]: Deploying daemon mds.d on smithi001 2023-11-16T16:36:50.354 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:50 smithi001 ceph-mon[98800]: mds.? [v2:172.21.15.1:6834/2587563684,v1:172.21.15.1:6835/2587563684] up:boot 2023-11-16T16:36:50.354 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:50 smithi001 ceph-mon[98800]: fsmap 3 up:standby 2023-11-16T16:36:50.354 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:50 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-11-16T16:36:50.355 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:50 smithi001 ceph-mon[98800]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:36:50.488 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:36:50.609 INFO:journalctl@ceph.mds.d.smithi001.stdout:Nov 16 16:36:50 smithi001 podman[132530]: 2023-11-16T16:36:50.629 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.624+0000 7ff9874d9700 1 -- 172.21.15.1:0/268345709 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9800ffb50 msgr2=0x7ff9800fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:50.629 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.624+0000 7ff9874d9700 1 --2- 172.21.15.1:0/268345709 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9800ffb50 0x7ff9800fffa0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff970009990 tx=0x7ff970009ca0 comp rx=0 tx=0).stop 2023-11-16T16:36:50.629 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.624+0000 7ff9874d9700 1 -- 172.21.15.1:0/268345709 shutdown_connections 2023-11-16T16:36:50.629 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.624+0000 7ff9874d9700 1 --2- 172.21.15.1:0/268345709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9801004e0 0x7ff980106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:50.629 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.624+0000 7ff9874d9700 1 --2- 172.21.15.1:0/268345709 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9800fe960 0x7ff9800fed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:50.629 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.624+0000 7ff9874d9700 1 --2- 172.21.15.1:0/268345709 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9800ffb50 0x7ff9800fffa0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:50.629 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.624+0000 7ff9874d9700 1 -- 172.21.15.1:0/268345709 >> 172.21.15.1:0/268345709 conn(0x7ff9800f9f40 msgr2=0x7ff9800fc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:36:50.629 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.624+0000 7ff9874d9700 1 -- 172.21.15.1:0/268345709 shutdown_connections 2023-11-16T16:36:50.629 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.624+0000 7ff9874d9700 1 -- 172.21.15.1:0/268345709 wait complete. 2023-11-16T16:36:50.630 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.625+0000 7ff9874d9700 1 Processor -- start 2023-11-16T16:36:50.630 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.625+0000 7ff9874d9700 1 -- start start 2023-11-16T16:36:50.630 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.625+0000 7ff9874d9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9800fe960 0x7ff98010b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:50.630 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.625+0000 7ff9874d9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9800ffb50 0x7ff98019ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:50.630 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.625+0000 7ff9874d9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9801004e0 0x7ff98019d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:50.630 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.625+0000 7ff9874d9700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98019db60 con 0x7ff9801004e0 2023-11-16T16:36:50.630 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.625+0000 7ff9874d9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98019dcd0 con 0x7ff9800fe960 2023-11-16T16:36:50.630 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.625+0000 7ff9874d9700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98019de90 con 0x7ff9800ffb50 2023-11-16T16:36:50.631 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.625+0000 7ff985a76700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9801004e0 0x7ff98019d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:50.631 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.625+0000 7ff985a76700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9801004e0 0x7ff98019d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41188/0 (socket says 172.21.15.1:41188) 2023-11-16T16:36:50.631 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.625+0000 7ff985a76700 1 -- 172.21.15.1:0/280403420 learned_addr learned my addr 172.21.15.1:0/280403420 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:36:50.631 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.625+0000 7ff985275700 1 --2- 172.21.15.1:0/280403420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9800fe960 0x7ff98010b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:50.631 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.625+0000 7ff984a74700 1 --2- 172.21.15.1:0/280403420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9800ffb50 0x7ff98019ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:50.631 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.625+0000 7ff985a76700 1 -- 172.21.15.1:0/280403420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9800ffb50 msgr2=0x7ff98019ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:50.631 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.625+0000 7ff985a76700 1 --2- 172.21.15.1:0/280403420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9800ffb50 0x7ff98019ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:50.631 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.625+0000 7ff985a76700 1 -- 172.21.15.1:0/280403420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9800fe960 msgr2=0x7ff98010b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:50.631 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.625+0000 7ff985a76700 1 --2- 172.21.15.1:0/280403420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9800fe960 0x7ff98010b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:50.631 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.625+0000 7ff985a76700 1 -- 172.21.15.1:0/280403420 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9700095f0 con 0x7ff9801004e0 2023-11-16T16:36:50.632 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.626+0000 7ff985a76700 1 --2- 172.21.15.1:0/280403420 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9801004e0 0x7ff98019d390 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff97800ef30 tx=0x7ff97800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:36:50.632 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.626+0000 7ff9767fc700 1 -- 172.21.15.1:0/280403420 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff978018070 con 0x7ff9801004e0 2023-11-16T16:36:50.632 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.626+0000 7ff9767fc700 1 -- 172.21.15.1:0/280403420 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff97800f040 con 0x7ff9801004e0 2023-11-16T16:36:50.632 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.626+0000 7ff9874d9700 1 -- 172.21.15.1:0/280403420 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff98010a2c0 con 0x7ff9801004e0 2023-11-16T16:36:50.632 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.626+0000 7ff9767fc700 1 -- 172.21.15.1:0/280403420 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97800cd00 con 0x7ff9801004e0 2023-11-16T16:36:50.632 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.626+0000 7ff9874d9700 1 -- 172.21.15.1:0/280403420 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff98010a130 con 0x7ff9801004e0 2023-11-16T16:36:50.632 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.627+0000 7ff9874d9700 1 -- 172.21.15.1:0/280403420 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff980105530 con 0x7ff9801004e0 2023-11-16T16:36:50.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.628+0000 7ff9767fc700 1 -- 172.21.15.1:0/280403420 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7ff978003a00 con 0x7ff9801004e0 2023-11-16T16:36:50.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.628+0000 7ff9767fc700 1 --2- 172.21.15.1:0/280403420 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7ff96c0a1150 0x7ff96c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:50.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.629+0000 7ff985275700 1 --2- 172.21.15.1:0/280403420 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7ff96c0a1150 0x7ff96c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:50.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.629+0000 7ff9767fc700 1 -- 172.21.15.1:0/280403420 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff9780cc060 con 0x7ff9801004e0 2023-11-16T16:36:50.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.629+0000 7ff985275700 1 --2- 172.21.15.1:0/280403420 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7ff96c0a1150 0x7ff96c0a3600 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff9800ff9b0 tx=0x7ff97c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:36:50.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.633+0000 7ff9767fc700 1 -- 172.21.15.1:0/280403420 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff978095c40 con 0x7ff9801004e0 2023-11-16T16:36:50.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:50 smithi026 ceph-mon[100112]: Deploying daemon mds.d on smithi001 2023-11-16T16:36:50.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:50 smithi026 ceph-mon[100112]: mds.? [v2:172.21.15.1:6834/2587563684,v1:172.21.15.1:6835/2587563684] up:boot 2023-11-16T16:36:50.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:50 smithi026 ceph-mon[100112]: fsmap 3 up:standby 2023-11-16T16:36:50.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:50 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-11-16T16:36:50.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:50 smithi026 ceph-mon[100112]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:36:50.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:50 smithi053 ceph-mon[100638]: Deploying daemon mds.d on smithi001 2023-11-16T16:36:50.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:50 smithi053 ceph-mon[100638]: mds.? [v2:172.21.15.1:6834/2587563684,v1:172.21.15.1:6835/2587563684] up:boot 2023-11-16T16:36:50.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:50 smithi053 ceph-mon[100638]: fsmap 3 up:standby 2023-11-16T16:36:50.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:50 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-11-16T16:36:50.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:50 smithi053 ceph-mon[100638]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:36:50.883 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.881+0000 7ff9874d9700 1 -- 172.21.15.1:0/280403420 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff980061700 con 0x7ff9801004e0 2023-11-16T16:36:50.884 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.882+0000 7ff9767fc700 1 -- 172.21.15.1:0/280403420 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2765 (secure 0 0 0) 0x7ff978099260 con 0x7ff9801004e0 2023-11-16T16:36:50.884 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:36:50.884 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2587563684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2587563684},{"type":"v1","addr":"172.21.15.1:6835","nonce":2587563684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3693905267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3693905267},{"type":"v1","addr":"172.21.15.53:6833","nonce":3693905267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/164668033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":164668033},{"type":"v1","addr":"172.21.15.26:6833","nonce":164668033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-11-16T16:36:50.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.884+0000 7ff9874d9700 1 -- 172.21.15.1:0/280403420 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7ff96c0a1150 msgr2=0x7ff96c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:50.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.884+0000 7ff9874d9700 1 --2- 172.21.15.1:0/280403420 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7ff96c0a1150 0x7ff96c0a3600 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff9800ff9b0 tx=0x7ff97c00a380 comp rx=0 tx=0).stop 2023-11-16T16:36:50.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.884+0000 7ff9874d9700 1 -- 172.21.15.1:0/280403420 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9801004e0 msgr2=0x7ff98019d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:50.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.884+0000 7ff9874d9700 1 --2- 172.21.15.1:0/280403420 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9801004e0 0x7ff98019d390 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff97800ef30 tx=0x7ff97800c3a0 comp rx=0 tx=0).stop 2023-11-16T16:36:50.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.884+0000 7ff9874d9700 1 -- 172.21.15.1:0/280403420 shutdown_connections 2023-11-16T16:36:50.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.884+0000 7ff9874d9700 1 --2- 172.21.15.1:0/280403420 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7ff96c0a1150 0x7ff96c0a3600 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:50.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.884+0000 7ff9874d9700 1 --2- 172.21.15.1:0/280403420 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9801004e0 0x7ff98019d390 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:50.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.884+0000 7ff9874d9700 1 --2- 172.21.15.1:0/280403420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9800fe960 0x7ff98010b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:50.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.884+0000 7ff9874d9700 1 --2- 172.21.15.1:0/280403420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9800ffb50 0x7ff98019ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:50.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.884+0000 7ff9874d9700 1 -- 172.21.15.1:0/280403420 >> 172.21.15.1:0/280403420 conn(0x7ff9800f9f40 msgr2=0x7ff9800fb8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:36:50.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.885+0000 7ff9874d9700 1 -- 172.21.15.1:0/280403420 shutdown_connections 2023-11-16T16:36:50.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:50.885+0000 7ff9874d9700 1 -- 172.21.15.1:0/280403420 wait complete. 2023-11-16T16:36:50.889 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 4 2023-11-16T16:36:51.182 INFO:journalctl@ceph.mds.d.smithi001.stdout:Nov 16 16:36:50 smithi001 ceph-mds[132614]: -- 172.21.15.1:0/3710247553 <== mon.1 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558f5de7f520 con 0x558f5de54800 2023-11-16T16:36:51.182 INFO:journalctl@ceph.mds.d.smithi001.stdout:Nov 16 16:36:50 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-d[132569]: starting mds.d at 2023-11-16T16:36:51.183 INFO:journalctl@ceph.mds.d.smithi001.stdout:Nov 16 16:36:50 smithi001 bash[132530]: c10f9f6ec7650b4d3bb17d33182289a22285081f53560e5893be007890409cb9 2023-11-16T16:36:51.183 INFO:journalctl@ceph.mds.d.smithi001.stdout:Nov 16 16:36:50 smithi001 systemd[1]: Started Ceph mds.d for 35d9b010-849d-11ee-8dbe-212e2dc638e7. 2023-11-16T16:36:51.922 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:51 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/280403420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:36:51.922 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:51 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:51.922 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:51 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:51.922 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:51 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:51.922 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:51 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:36:51.923 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:51 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:36:51.923 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:51 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:51.923 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:51 smithi026 ceph-mon[100112]: Deploying daemon mds.e on smithi026 2023-11-16T16:36:51.923 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:51 smithi026 ceph-mon[100112]: mds.? [v2:172.21.15.1:6836/277936168,v1:172.21.15.1:6837/277936168] up:boot 2023-11-16T16:36:51.923 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:51 smithi026 ceph-mon[100112]: fsmap 4 up:standby 2023-11-16T16:36:51.923 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:51 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-11-16T16:36:51.923 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:51 smithi026 ceph-mon[100112]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:36:52.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:51 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/280403420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:36:52.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:51 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:52.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:51 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:52.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:51 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:52.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:51 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:36:52.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:51 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:36:52.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:51 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:52.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:51 smithi053 ceph-mon[100638]: Deploying daemon mds.e on smithi026 2023-11-16T16:36:52.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:51 smithi053 ceph-mon[100638]: mds.? [v2:172.21.15.1:6836/277936168,v1:172.21.15.1:6837/277936168] up:boot 2023-11-16T16:36:52.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:51 smithi053 ceph-mon[100638]: fsmap 4 up:standby 2023-11-16T16:36:52.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:51 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-11-16T16:36:52.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:51 smithi053 ceph-mon[100638]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:36:52.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:51 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/280403420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:36:52.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:51 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:52.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:51 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:52.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:51 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:52.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:51 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:36:52.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:51 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:36:52.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:51 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:52.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:51 smithi001 ceph-mon[98800]: Deploying daemon mds.e on smithi026 2023-11-16T16:36:52.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:51 smithi001 ceph-mon[98800]: mds.? [v2:172.21.15.1:6836/277936168,v1:172.21.15.1:6837/277936168] up:boot 2023-11-16T16:36:52.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:51 smithi001 ceph-mon[98800]: fsmap 4 up:standby 2023-11-16T16:36:52.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:51 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-11-16T16:36:52.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:51 smithi001 ceph-mon[98800]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:36:52.597 INFO:journalctl@ceph.mds.e.smithi026.stdout:Nov 16 16:36:52 smithi026 systemd[1]: Starting Ceph mds.e for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:36:52.902 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:36:53.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.041+0000 7fc0e4c66700 1 -- 172.21.15.1:0/3976117084 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0e00ff1c0 msgr2=0x7fc0e00ff5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:53.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.041+0000 7fc0e4c66700 1 --2- 172.21.15.1:0/3976117084 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0e00ff1c0 0x7fc0e00ff5d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0009990 tx=0x7fc0d0009ca0 comp rx=0 tx=0).stop 2023-11-16T16:36:53.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.041+0000 7fc0e4c66700 1 -- 172.21.15.1:0/3976117084 shutdown_connections 2023-11-16T16:36:53.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.042+0000 7fc0e4c66700 1 --2- 172.21.15.1:0/3976117084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0e0100d40 0x7fc0e0107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:53.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.042+0000 7fc0e4c66700 1 --2- 172.21.15.1:0/3976117084 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0e00ff1c0 0x7fc0e00ff5d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:53.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.042+0000 7fc0e4c66700 1 --2- 172.21.15.1:0/3976117084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0e01003b0 0x7fc0e0100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:53.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.042+0000 7fc0e4c66700 1 -- 172.21.15.1:0/3976117084 >> 172.21.15.1:0/3976117084 conn(0x7fc0e00fa7c0 msgr2=0x7fc0e00fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:36:53.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.042+0000 7fc0e4c66700 1 -- 172.21.15.1:0/3976117084 shutdown_connections 2023-11-16T16:36:53.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.042+0000 7fc0e4c66700 1 -- 172.21.15.1:0/3976117084 wait complete. 2023-11-16T16:36:53.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.042+0000 7fc0e4c66700 1 Processor -- start 2023-11-16T16:36:53.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.042+0000 7fc0e4c66700 1 -- start start 2023-11-16T16:36:53.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.043+0000 7fc0e4c66700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0e00ff1c0 0x7fc0e0196080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:53.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.043+0000 7fc0e4c66700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0e01003b0 0x7fc0e01965c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:53.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.043+0000 7fc0e4c66700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0e0100d40 0x7fc0e019d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:53.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.043+0000 7fc0e4c66700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e019dc00 con 0x7fc0e01003b0 2023-11-16T16:36:53.049 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.043+0000 7fc0e4c66700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e019dd40 con 0x7fc0e0100d40 2023-11-16T16:36:53.049 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.043+0000 7fc0e4c66700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e019ded0 con 0x7fc0e00ff1c0 2023-11-16T16:36:53.049 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.043+0000 7fc0ddd9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0e01003b0 0x7fc0e01965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:53.049 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.043+0000 7fc0ddd9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0e01003b0 0x7fc0e01965c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41210/0 (socket says 172.21.15.1:41210) 2023-11-16T16:36:53.049 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.043+0000 7fc0ddd9b700 1 -- 172.21.15.1:0/2306784051 learned_addr learned my addr 172.21.15.1:0/2306784051 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:36:53.049 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.043+0000 7fc0de59c700 1 --2- 172.21.15.1:0/2306784051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0e00ff1c0 0x7fc0e0196080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:53.049 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.043+0000 7fc0ded9d700 1 --2- 172.21.15.1:0/2306784051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0e0100d40 0x7fc0e019d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:53.049 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.043+0000 7fc0ddd9b700 1 -- 172.21.15.1:0/2306784051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0e00ff1c0 msgr2=0x7fc0e0196080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:53.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.043+0000 7fc0ddd9b700 1 --2- 172.21.15.1:0/2306784051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0e00ff1c0 0x7fc0e0196080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:53.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.043+0000 7fc0ddd9b700 1 -- 172.21.15.1:0/2306784051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0e0100d40 msgr2=0x7fc0e019d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:53.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.043+0000 7fc0ddd9b700 1 --2- 172.21.15.1:0/2306784051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0e0100d40 0x7fc0e019d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:53.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.043+0000 7fc0ddd9b700 1 -- 172.21.15.1:0/2306784051 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d00095f0 con 0x7fc0e01003b0 2023-11-16T16:36:53.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.043+0000 7fc0ded9d700 1 --2- 172.21.15.1:0/2306784051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0e0100d40 0x7fc0e019d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:36:53.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.043+0000 7fc0ddd9b700 1 --2- 172.21.15.1:0/2306784051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0e01003b0 0x7fc0e01965c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc0c800c9b0 tx=0x7fc0c800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:36:53.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.043+0000 7fc0d77fe700 1 -- 172.21.15.1:0/2306784051 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c8007b00 con 0x7fc0e01003b0 2023-11-16T16:36:53.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.043+0000 7fc0d77fe700 1 -- 172.21.15.1:0/2306784051 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0c8007c60 con 0x7fc0e01003b0 2023-11-16T16:36:53.051 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.044+0000 7fc0e4c66700 1 -- 172.21.15.1:0/2306784051 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0e010ab80 con 0x7fc0e01003b0 2023-11-16T16:36:53.051 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.044+0000 7fc0d77fe700 1 -- 172.21.15.1:0/2306784051 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c8018660 con 0x7fc0e01003b0 2023-11-16T16:36:53.051 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.044+0000 7fc0e4c66700 1 -- 172.21.15.1:0/2306784051 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0e010a9f0 con 0x7fc0e01003b0 2023-11-16T16:36:53.051 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.044+0000 7fc0e4c66700 1 -- 172.21.15.1:0/2306784051 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0e019e620 con 0x7fc0e01003b0 2023-11-16T16:36:53.052 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.046+0000 7fc0d77fe700 1 -- 172.21.15.1:0/2306784051 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fc0c800f450 con 0x7fc0e01003b0 2023-11-16T16:36:53.052 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.046+0000 7fc0d77fe700 1 --2- 172.21.15.1:0/2306784051 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc0cc0a1150 0x7fc0cc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:53.052 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.046+0000 7fc0de59c700 1 --2- 172.21.15.1:0/2306784051 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc0cc0a1150 0x7fc0cc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:53.052 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.046+0000 7fc0d77fe700 1 -- 172.21.15.1:0/2306784051 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc0c80cc700 con 0x7fc0e01003b0 2023-11-16T16:36:53.053 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.047+0000 7fc0de59c700 1 --2- 172.21.15.1:0/2306784051 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc0cc0a1150 0x7fc0cc0a3600 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc0d00036c0 tx=0x7fc0d000b560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:36:53.053 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.050+0000 7fc0d77fe700 1 -- 172.21.15.1:0/2306784051 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc0c8096380 con 0x7fc0e01003b0 2023-11-16T16:36:53.164 INFO:journalctl@ceph.mds.e.smithi026.stdout:Nov 16 16:36:52 smithi026 podman[124701]: 2023-11-16T16:36:53.299 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.298+0000 7fc0e4c66700 1 -- 172.21.15.1:0/2306784051 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc0e019ebd0 con 0x7fc0e01003b0 2023-11-16T16:36:53.300 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.298+0000 7fc0d77fe700 1 -- 172.21.15.1:0/2306784051 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3516 (secure 0 0 0) 0x7fc0c80999a0 con 0x7fc0e01003b0 2023-11-16T16:36:53.300 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:36:53.300 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2587563684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2587563684},{"type":"v1","addr":"172.21.15.1:6835","nonce":2587563684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/277936168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":277936168},{"type":"v1","addr":"172.21.15.1:6837","nonce":277936168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3693905267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3693905267},{"type":"v1","addr":"172.21.15.53:6833","nonce":3693905267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/164668033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":164668033},{"type":"v1","addr":"172.21.15.26:6833","nonce":164668033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-11-16T16:36:53.302 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.300+0000 7fc0e4c66700 1 -- 172.21.15.1:0/2306784051 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc0cc0a1150 msgr2=0x7fc0cc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:53.302 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.300+0000 7fc0e4c66700 1 --2- 172.21.15.1:0/2306784051 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc0cc0a1150 0x7fc0cc0a3600 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc0d00036c0 tx=0x7fc0d000b560 comp rx=0 tx=0).stop 2023-11-16T16:36:53.303 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.300+0000 7fc0e4c66700 1 -- 172.21.15.1:0/2306784051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0e01003b0 msgr2=0x7fc0e01965c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:53.303 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.300+0000 7fc0e4c66700 1 --2- 172.21.15.1:0/2306784051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0e01003b0 0x7fc0e01965c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc0c800c9b0 tx=0x7fc0c800ccc0 comp rx=0 tx=0).stop 2023-11-16T16:36:53.303 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.300+0000 7fc0e4c66700 1 -- 172.21.15.1:0/2306784051 shutdown_connections 2023-11-16T16:36:53.303 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.300+0000 7fc0e4c66700 1 --2- 172.21.15.1:0/2306784051 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc0cc0a1150 0x7fc0cc0a3600 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:53.303 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.300+0000 7fc0e4c66700 1 --2- 172.21.15.1:0/2306784051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0e0100d40 0x7fc0e019d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:53.303 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.300+0000 7fc0e4c66700 1 --2- 172.21.15.1:0/2306784051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc0e00ff1c0 0x7fc0e0196080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:53.303 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.300+0000 7fc0e4c66700 1 --2- 172.21.15.1:0/2306784051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0e01003b0 0x7fc0e01965c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:53.303 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.300+0000 7fc0e4c66700 1 -- 172.21.15.1:0/2306784051 >> 172.21.15.1:0/2306784051 conn(0x7fc0e00fa7c0 msgr2=0x7fc0e0067de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:36:53.303 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.300+0000 7fc0e4c66700 1 -- 172.21.15.1:0/2306784051 shutdown_connections 2023-11-16T16:36:53.303 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:53.301+0000 7fc0e4c66700 1 -- 172.21.15.1:0/2306784051 wait complete. 2023-11-16T16:36:53.304 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 5 2023-11-16T16:36:53.508 INFO:journalctl@ceph.mds.e.smithi026.stdout:Nov 16 16:36:53 smithi026 ceph-mds[124741]: -- 172.21.15.26:0/2111024237 <== mon.2 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561530fcd520 con 0x561530fa2800 2023-11-16T16:36:53.508 INFO:journalctl@ceph.mds.e.smithi026.stdout:Nov 16 16:36:53 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-e[124717]: starting mds.e at 2023-11-16T16:36:53.508 INFO:journalctl@ceph.mds.e.smithi026.stdout:Nov 16 16:36:53 smithi026 bash[124701]: 288fc61e40ca6dd3d5779bbe6f3715aefd18a0c2fcce005bd8602da9ff51121f 2023-11-16T16:36:53.508 INFO:journalctl@ceph.mds.e.smithi026.stdout:Nov 16 16:36:53 smithi026 systemd[1]: Started Ceph mds.e for 35d9b010-849d-11ee-8dbe-212e2dc638e7. 2023-11-16T16:36:54.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:54 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:54.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:54 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:54.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:54 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:54.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:54 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:36:54.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:54 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:36:54.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:54 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:54.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:54 smithi053 ceph-mon[100638]: Deploying daemon mds.f on smithi053 2023-11-16T16:36:54.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:54 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/2306784051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:36:54.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:54 smithi053 ceph-mon[100638]: mds.? [v2:172.21.15.26:6834/1010760275,v1:172.21.15.26:6835/1010760275] up:boot 2023-11-16T16:36:54.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:54 smithi053 ceph-mon[100638]: fsmap 5 up:standby 2023-11-16T16:36:54.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:54 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-16T16:36:54.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:54 smithi053 ceph-mon[100638]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:36:54.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:54 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:54.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:54 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:54.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:54 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:54.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:54 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:36:54.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:54 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:36:54.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:54 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:54.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:54 smithi026 ceph-mon[100112]: Deploying daemon mds.f on smithi053 2023-11-16T16:36:54.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:54 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/2306784051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:36:54.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:54 smithi026 ceph-mon[100112]: mds.? [v2:172.21.15.26:6834/1010760275,v1:172.21.15.26:6835/1010760275] up:boot 2023-11-16T16:36:54.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:54 smithi026 ceph-mon[100112]: fsmap 5 up:standby 2023-11-16T16:36:54.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:54 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-16T16:36:54.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:54 smithi026 ceph-mon[100112]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:36:54.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:54 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:54.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:54 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:54.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:54 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:54.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:54 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:36:54.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:54 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:36:54.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:54 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:54.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:54 smithi001 ceph-mon[98800]: Deploying daemon mds.f on smithi053 2023-11-16T16:36:54.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:54 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/2306784051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:36:54.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:54 smithi001 ceph-mon[98800]: mds.? [v2:172.21.15.26:6834/1010760275,v1:172.21.15.26:6835/1010760275] up:boot 2023-11-16T16:36:54.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:54 smithi001 ceph-mon[98800]: fsmap 5 up:standby 2023-11-16T16:36:54.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:54 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-16T16:36:54.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:54 smithi001 ceph-mon[98800]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:36:55.081 INFO:journalctl@ceph.mds.f.smithi053.stdout:Nov 16 16:36:54 smithi053 systemd[1]: Starting Ceph mds.f for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:36:55.315 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:36:55.402 INFO:journalctl@ceph.mds.f.smithi053.stdout:Nov 16 16:36:55 smithi053 podman[125906]: 2023-11-16T16:36:55.402 INFO:journalctl@ceph.mds.f.smithi053.stdout:Nov 16 16:36:55 smithi053 ceph-mds[125946]: -- 172.21.15.53:0/1313312112 <== mon.1 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fb9a249520 con 0x55fb9a21e800 2023-11-16T16:36:55.402 INFO:journalctl@ceph.mds.f.smithi053.stdout:Nov 16 16:36:55 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-f[125922]: starting mds.f at 2023-11-16T16:36:55.402 INFO:journalctl@ceph.mds.f.smithi053.stdout:Nov 16 16:36:55 smithi053 bash[125906]: 2cc821f71aaa3d7782a3c7bbefe7a1b76afa8e28cd7bc4e609beacc0cda39d42 2023-11-16T16:36:55.458 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.456+0000 7fb059008700 1 -- 172.21.15.1:0/328444793 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb0540ff1c0 msgr2=0x7fb0540ff5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:55.459 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.456+0000 7fb059008700 1 --2- 172.21.15.1:0/328444793 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb0540ff1c0 0x7fb0540ff5d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb048009990 tx=0x7fb048009ca0 comp rx=0 tx=0).stop 2023-11-16T16:36:55.459 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.456+0000 7fb059008700 1 -- 172.21.15.1:0/328444793 shutdown_connections 2023-11-16T16:36:55.459 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.456+0000 7fb059008700 1 --2- 172.21.15.1:0/328444793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb054100d40 0x7fb054107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:55.459 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.456+0000 7fb059008700 1 --2- 172.21.15.1:0/328444793 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb0540ff1c0 0x7fb0540ff5d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:55.459 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.456+0000 7fb059008700 1 --2- 172.21.15.1:0/328444793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0541003b0 0x7fb054100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:55.459 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.456+0000 7fb059008700 1 -- 172.21.15.1:0/328444793 >> 172.21.15.1:0/328444793 conn(0x7fb0540fa7c0 msgr2=0x7fb0540fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:36:55.459 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.456+0000 7fb059008700 1 -- 172.21.15.1:0/328444793 shutdown_connections 2023-11-16T16:36:55.459 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.456+0000 7fb059008700 1 -- 172.21.15.1:0/328444793 wait complete. 2023-11-16T16:36:55.459 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.457+0000 7fb059008700 1 Processor -- start 2023-11-16T16:36:55.459 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.457+0000 7fb059008700 1 -- start start 2023-11-16T16:36:55.462 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.457+0000 7fb059008700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0540ff1c0 0x7fb0541982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:55.462 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.457+0000 7fb059008700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb0541003b0 0x7fb054198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:55.462 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.457+0000 7fb059008700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb054100d40 0x7fb05419f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:55.462 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.457+0000 7fb059008700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05419fe70 con 0x7fb0541003b0 2023-11-16T16:36:55.462 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.457+0000 7fb059008700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05419ffb0 con 0x7fb054100d40 2023-11-16T16:36:55.462 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.457+0000 7fb059008700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0541a0140 con 0x7fb0540ff1c0 2023-11-16T16:36:55.462 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.457+0000 7fb05259c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb0541003b0 0x7fb054198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:55.462 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.457+0000 7fb05259c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb0541003b0 0x7fb054198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41222/0 (socket says 172.21.15.1:41222) 2023-11-16T16:36:55.463 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.457+0000 7fb05259c700 1 -- 172.21.15.1:0/127928791 learned_addr learned my addr 172.21.15.1:0/127928791 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:36:55.463 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.457+0000 7fb052d9d700 1 --2- 172.21.15.1:0/127928791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0540ff1c0 0x7fb0541982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:55.463 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.457+0000 7fb05359e700 1 --2- 172.21.15.1:0/127928791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb054100d40 0x7fb05419f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:55.463 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.457+0000 7fb05259c700 1 -- 172.21.15.1:0/127928791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0540ff1c0 msgr2=0x7fb0541982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:55.463 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.457+0000 7fb05259c700 1 --2- 172.21.15.1:0/127928791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0540ff1c0 0x7fb0541982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:55.463 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.457+0000 7fb05259c700 1 -- 172.21.15.1:0/127928791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb054100d40 msgr2=0x7fb05419f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:55.463 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.457+0000 7fb05259c700 1 --2- 172.21.15.1:0/127928791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb054100d40 0x7fb05419f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:55.463 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.457+0000 7fb05259c700 1 -- 172.21.15.1:0/127928791 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0480095f0 con 0x7fb0541003b0 2023-11-16T16:36:55.463 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.458+0000 7fb05259c700 1 --2- 172.21.15.1:0/127928791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb0541003b0 0x7fb054198830 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb04400c920 tx=0x7fb04400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:36:55.463 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.458+0000 7fb03bfff700 1 -- 172.21.15.1:0/127928791 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb044004a30 con 0x7fb0541003b0 2023-11-16T16:36:55.464 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.458+0000 7fb059008700 1 -- 172.21.15.1:0/127928791 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb05410ab80 con 0x7fb0541003b0 2023-11-16T16:36:55.464 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.458+0000 7fb03bfff700 1 -- 172.21.15.1:0/127928791 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0440054e0 con 0x7fb0541003b0 2023-11-16T16:36:55.464 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.458+0000 7fb03bfff700 1 -- 172.21.15.1:0/127928791 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb044007bd0 con 0x7fb0541003b0 2023-11-16T16:36:55.464 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.458+0000 7fb059008700 1 -- 172.21.15.1:0/127928791 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb05410a9f0 con 0x7fb0541003b0 2023-11-16T16:36:55.464 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.459+0000 7fb059008700 1 -- 172.21.15.1:0/127928791 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0541a0890 con 0x7fb0541003b0 2023-11-16T16:36:55.464 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.460+0000 7fb03bfff700 1 -- 172.21.15.1:0/127928791 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fb044007d50 con 0x7fb0541003b0 2023-11-16T16:36:55.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.460+0000 7fb03bfff700 1 --2- 172.21.15.1:0/127928791 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fb03c0a1150 0x7fb03c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:55.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.461+0000 7fb052d9d700 1 --2- 172.21.15.1:0/127928791 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fb03c0a1150 0x7fb03c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:55.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.461+0000 7fb03bfff700 1 -- 172.21.15.1:0/127928791 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb0440cc3b0 con 0x7fb0541003b0 2023-11-16T16:36:55.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.461+0000 7fb052d9d700 1 --2- 172.21.15.1:0/127928791 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fb03c0a1150 0x7fb03c0a3600 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb0480036c0 tx=0x7fb048005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:36:55.466 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.464+0000 7fb03bfff700 1 -- 172.21.15.1:0/127928791 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb044095fa0 con 0x7fb0541003b0 2023-11-16T16:36:55.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.714+0000 7fb059008700 1 -- 172.21.15.1:0/127928791 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0541a0e40 con 0x7fb0541003b0 2023-11-16T16:36:55.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.714+0000 7fb03bfff700 1 -- 172.21.15.1:0/127928791 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5030 (secure 0 0 0) 0x7fb0440995c0 con 0x7fb0541003b0 2023-11-16T16:36:55.716 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:36:55.716 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2587563684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2587563684},{"type":"v1","addr":"172.21.15.1:6835","nonce":2587563684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/277936168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":277936168},{"type":"v1","addr":"172.21.15.1:6837","nonce":277936168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3693905267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3693905267},{"type":"v1","addr":"172.21.15.53:6833","nonce":3693905267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/164668033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":164668033},{"type":"v1","addr":"172.21.15.26:6833","nonce":164668033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2140542192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2140542192},{"type":"v1","addr":"172.21.15.53:6835","nonce":2140542192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1010760275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1010760275},{"type":"v1","addr":"172.21.15.26:6835","nonce":1010760275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2023-11-16T16:36:55.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.716+0000 7fb059008700 1 -- 172.21.15.1:0/127928791 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fb03c0a1150 msgr2=0x7fb03c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:55.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.716+0000 7fb059008700 1 --2- 172.21.15.1:0/127928791 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fb03c0a1150 0x7fb03c0a3600 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb0480036c0 tx=0x7fb048005ee0 comp rx=0 tx=0).stop 2023-11-16T16:36:55.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.716+0000 7fb059008700 1 -- 172.21.15.1:0/127928791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb0541003b0 msgr2=0x7fb054198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:55.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.716+0000 7fb059008700 1 --2- 172.21.15.1:0/127928791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb0541003b0 0x7fb054198830 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb04400c920 tx=0x7fb04400cc30 comp rx=0 tx=0).stop 2023-11-16T16:36:55.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.716+0000 7fb059008700 1 -- 172.21.15.1:0/127928791 shutdown_connections 2023-11-16T16:36:55.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.716+0000 7fb059008700 1 --2- 172.21.15.1:0/127928791 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fb03c0a1150 0x7fb03c0a3600 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:55.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.716+0000 7fb059008700 1 --2- 172.21.15.1:0/127928791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb054100d40 0x7fb05419f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:55.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.716+0000 7fb059008700 1 --2- 172.21.15.1:0/127928791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0540ff1c0 0x7fb0541982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:55.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.716+0000 7fb059008700 1 --2- 172.21.15.1:0/127928791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb0541003b0 0x7fb054198830 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:55.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.716+0000 7fb059008700 1 -- 172.21.15.1:0/127928791 >> 172.21.15.1:0/127928791 conn(0x7fb0540fa7c0 msgr2=0x7fb054067de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:36:55.720 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.717+0000 7fb059008700 1 -- 172.21.15.1:0/127928791 shutdown_connections 2023-11-16T16:36:55.720 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:55.717+0000 7fb059008700 1 -- 172.21.15.1:0/127928791 wait complete. 2023-11-16T16:36:55.720 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 7 2023-11-16T16:36:55.831 INFO:journalctl@ceph.mds.f.smithi053.stdout:Nov 16 16:36:55 smithi053 systemd[1]: Started Ceph mds.f for 35d9b010-849d-11ee-8dbe-212e2dc638e7. 2023-11-16T16:36:56.514 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:56 smithi053 ceph-mon[100638]: mds.? [v2:172.21.15.53:6834/2140542192,v1:172.21.15.53:6835/2140542192] up:boot 2023-11-16T16:36:56.514 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:56 smithi053 ceph-mon[100638]: fsmap 6 up:standby 2023-11-16T16:36:56.514 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:56 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-16T16:36:56.515 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:56 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:56.515 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:56 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:56.515 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:56 smithi053 ceph-mon[100638]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:36:56.515 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:56 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:56.515 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:56 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:36:56.515 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:56 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:36:56.515 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:56 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:56.515 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:56 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/127928791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:36:56.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:56 smithi001 ceph-mon[98800]: mds.? [v2:172.21.15.53:6834/2140542192,v1:172.21.15.53:6835/2140542192] up:boot 2023-11-16T16:36:56.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:56 smithi001 ceph-mon[98800]: fsmap 6 up:standby 2023-11-16T16:36:56.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:56 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-16T16:36:56.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:56 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:56.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:56 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:56.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:56 smithi001 ceph-mon[98800]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:36:56.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:56 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:56.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:56 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:36:56.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:56 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:36:56.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:56 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:56.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:56 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/127928791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:36:56.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:56 smithi026 ceph-mon[100112]: mds.? [v2:172.21.15.53:6834/2140542192,v1:172.21.15.53:6835/2140542192] up:boot 2023-11-16T16:36:56.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:56 smithi026 ceph-mon[100112]: fsmap 6 up:standby 2023-11-16T16:36:56.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:56 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-16T16:36:56.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:56 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:56.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:56 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:56.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:56 smithi026 ceph-mon[100112]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:36:56.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:56 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:56.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:56 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:36:56.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:56 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:36:56.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:56 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:56.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:56 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/127928791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:36:57.029 INFO:journalctl@ceph.mds.c.smithi053.stdout:Nov 16 16:36:57 smithi053 systemd[1]: Starting Ceph mds.c for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:36:57.619 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:57 smithi053 ceph-mon[100638]: Deploying daemon mds.c on smithi053 2023-11-16T16:36:57.619 INFO:journalctl@ceph.mds.c.smithi053.stdout:Nov 16 16:36:57 smithi053 podman[126250]: 2023-11-16T16:36:57.620 INFO:journalctl@ceph.mds.c.smithi053.stdout:Nov 16 16:36:57 smithi053 ceph-mds[126290]: -- 172.21.15.53:0/2802668201 <== mon.1 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563ee4659520 con 0x563ee462f000 2023-11-16T16:36:57.620 INFO:journalctl@ceph.mds.c.smithi053.stdout:Nov 16 16:36:57 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-c[126266]: starting mds.c at 2023-11-16T16:36:57.620 INFO:journalctl@ceph.mds.c.smithi053.stdout:Nov 16 16:36:57 smithi053 bash[126250]: 3ad3d48099007c6396a85f80439dde533b444b01a55e5a2beb154273256b97f7 2023-11-16T16:36:57.732 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:36:57.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:57 smithi026 ceph-mon[100112]: Deploying daemon mds.c on smithi053 2023-11-16T16:36:57.761 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:57 smithi001 ceph-mon[98800]: Deploying daemon mds.c on smithi053 2023-11-16T16:36:57.872 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.867+0000 7f676ccac700 1 -- 172.21.15.1:0/131039225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6768103730 msgr2=0x7f6768105b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:57.872 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.867+0000 7f676ccac700 1 --2- 172.21.15.1:0/131039225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6768103730 0x7f6768105b10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f675c009990 tx=0x7f675c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:36:57.872 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.867+0000 7f676ccac700 1 -- 172.21.15.1:0/131039225 shutdown_connections 2023-11-16T16:36:57.873 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.867+0000 7f676ccac700 1 --2- 172.21.15.1:0/131039225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6768106050 0x7f6768108430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:57.873 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.867+0000 7f676ccac700 1 --2- 172.21.15.1:0/131039225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6768100e10 0x7f67681031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:57.873 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.867+0000 7f676ccac700 1 --2- 172.21.15.1:0/131039225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6768103730 0x7f6768105b10 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:57.873 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.867+0000 7f676ccac700 1 -- 172.21.15.1:0/131039225 >> 172.21.15.1:0/131039225 conn(0x7f67680fa7d0 msgr2=0x7f67680fcc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:36:57.873 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.868+0000 7f676ccac700 1 -- 172.21.15.1:0/131039225 shutdown_connections 2023-11-16T16:36:57.873 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.868+0000 7f676ccac700 1 -- 172.21.15.1:0/131039225 wait complete. 2023-11-16T16:36:57.873 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.868+0000 7f676ccac700 1 Processor -- start 2023-11-16T16:36:57.873 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.868+0000 7f676ccac700 1 -- start start 2023-11-16T16:36:57.873 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.868+0000 7f676ccac700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6768100e10 0x7f6768100780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:57.874 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.868+0000 7f676ccac700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6768103730 0x7f67680fedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:57.874 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.868+0000 7f676ccac700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6768106050 0x7f67680ff300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:57.874 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.868+0000 7f676ccac700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67680ffaa0 con 0x7f6768100e10 2023-11-16T16:36:57.874 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.868+0000 7f676ccac700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67680ffc10 con 0x7f6768103730 2023-11-16T16:36:57.874 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.868+0000 7f676ccac700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67681a0000 con 0x7f6768106050 2023-11-16T16:36:57.874 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.869+0000 7f676659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6768100e10 0x7f6768100780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:57.874 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.869+0000 7f676659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6768100e10 0x7f6768100780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41236/0 (socket says 172.21.15.1:41236) 2023-11-16T16:36:57.875 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.869+0000 7f676659c700 1 -- 172.21.15.1:0/1783157016 learned_addr learned my addr 172.21.15.1:0/1783157016 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:36:57.875 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.869+0000 7f6765d9b700 1 --2- 172.21.15.1:0/1783157016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6768103730 0x7f67680fedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:57.875 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.869+0000 7f6766d9d700 1 --2- 172.21.15.1:0/1783157016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6768106050 0x7f67680ff300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:57.875 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.869+0000 7f676659c700 1 -- 172.21.15.1:0/1783157016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6768106050 msgr2=0x7f67680ff300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:57.875 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.869+0000 7f676659c700 1 --2- 172.21.15.1:0/1783157016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6768106050 0x7f67680ff300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:57.875 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.869+0000 7f676659c700 1 -- 172.21.15.1:0/1783157016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6768103730 msgr2=0x7f67680fedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:57.875 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.869+0000 7f676659c700 1 --2- 172.21.15.1:0/1783157016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6768103730 0x7f67680fedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:57.875 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.869+0000 7f676659c700 1 -- 172.21.15.1:0/1783157016 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f675c0095f0 con 0x7f6768100e10 2023-11-16T16:36:57.876 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.869+0000 7f676659c700 1 --2- 172.21.15.1:0/1783157016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6768100e10 0x7f6768100780 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f675800d980 tx=0x7f675800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:36:57.876 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.869+0000 7f67577fe700 1 -- 172.21.15.1:0/1783157016 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6758004220 con 0x7f6768100e10 2023-11-16T16:36:57.876 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.869+0000 7f676ccac700 1 -- 172.21.15.1:0/1783157016 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f676810bd00 con 0x7f6768100e10 2023-11-16T16:36:57.876 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.869+0000 7f67577fe700 1 -- 172.21.15.1:0/1783157016 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6758007640 con 0x7f6768100e10 2023-11-16T16:36:57.876 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.869+0000 7f67577fe700 1 -- 172.21.15.1:0/1783157016 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6758005690 con 0x7f6768100e10 2023-11-16T16:36:57.876 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.869+0000 7f676ccac700 1 -- 172.21.15.1:0/1783157016 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f676810bb70 con 0x7f6768100e10 2023-11-16T16:36:57.876 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.870+0000 7f676ccac700 1 -- 172.21.15.1:0/1783157016 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67680fc340 con 0x7f6768100e10 2023-11-16T16:36:57.877 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.872+0000 7f67577fe700 1 -- 172.21.15.1:0/1783157016 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f6758004750 con 0x7f6768100e10 2023-11-16T16:36:57.877 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.873+0000 7f67577fe700 1 --2- 172.21.15.1:0/1783157016 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f67500a1150 0x7f67500a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:36:57.877 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.873+0000 7f6765d9b700 1 --2- 172.21.15.1:0/1783157016 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f67500a1150 0x7f67500a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:36:57.878 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.873+0000 7f67577fe700 1 -- 172.21.15.1:0/1783157016 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f67580cc870 con 0x7f6768100e10 2023-11-16T16:36:57.878 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.873+0000 7f6765d9b700 1 --2- 172.21.15.1:0/1783157016 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f67500a1150 0x7f67500a3600 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f675c00b5c0 tx=0x7f675c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:36:57.878 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:57.876+0000 7f67577fe700 1 -- 172.21.15.1:0/1783157016 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f67580963c0 con 0x7f6768100e10 2023-11-16T16:36:58.080 INFO:journalctl@ceph.mds.c.smithi053.stdout:Nov 16 16:36:57 smithi053 systemd[1]: Started Ceph mds.c for 35d9b010-849d-11ee-8dbe-212e2dc638e7. 2023-11-16T16:36:58.122 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:58.120+0000 7f676ccac700 1 -- 172.21.15.1:0/1783157016 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f676802cfe0 con 0x7f6768100e10 2023-11-16T16:36:58.122 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:58.120+0000 7f67577fe700 1 -- 172.21.15.1:0/1783157016 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5030 (secure 0 0 0) 0x7f67580999e0 con 0x7f6768100e10 2023-11-16T16:36:58.122 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:36:58.123 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2587563684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2587563684},{"type":"v1","addr":"172.21.15.1:6835","nonce":2587563684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/277936168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":277936168},{"type":"v1","addr":"172.21.15.1:6837","nonce":277936168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3693905267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3693905267},{"type":"v1","addr":"172.21.15.53:6833","nonce":3693905267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/164668033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":164668033},{"type":"v1","addr":"172.21.15.26:6833","nonce":164668033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2140542192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2140542192},{"type":"v1","addr":"172.21.15.53:6835","nonce":2140542192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1010760275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1010760275},{"type":"v1","addr":"172.21.15.26:6835","nonce":1010760275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2023-11-16T16:36:58.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:58.122+0000 7f676ccac700 1 -- 172.21.15.1:0/1783157016 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f67500a1150 msgr2=0x7f67500a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:58.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:58.122+0000 7f676ccac700 1 --2- 172.21.15.1:0/1783157016 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f67500a1150 0x7f67500a3600 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f675c00b5c0 tx=0x7f675c005fd0 comp rx=0 tx=0).stop 2023-11-16T16:36:58.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:58.122+0000 7f676ccac700 1 -- 172.21.15.1:0/1783157016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6768100e10 msgr2=0x7f6768100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:36:58.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:58.122+0000 7f676ccac700 1 --2- 172.21.15.1:0/1783157016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6768100e10 0x7f6768100780 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f675800d980 tx=0x7f675800dc90 comp rx=0 tx=0).stop 2023-11-16T16:36:58.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:58.122+0000 7f676ccac700 1 -- 172.21.15.1:0/1783157016 shutdown_connections 2023-11-16T16:36:58.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:58.122+0000 7f676ccac700 1 --2- 172.21.15.1:0/1783157016 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f67500a1150 0x7f67500a3600 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:58.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:58.122+0000 7f676ccac700 1 --2- 172.21.15.1:0/1783157016 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6768106050 0x7f67680ff300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:58.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:58.122+0000 7f676ccac700 1 --2- 172.21.15.1:0/1783157016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6768100e10 0x7f6768100780 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:58.126 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:58.122+0000 7f676ccac700 1 --2- 172.21.15.1:0/1783157016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6768103730 0x7f67680fedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:36:58.126 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:58.122+0000 7f676ccac700 1 -- 172.21.15.1:0/1783157016 >> 172.21.15.1:0/1783157016 conn(0x7f67680fa7d0 msgr2=0x7f6768101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:36:58.126 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:58.122+0000 7f676ccac700 1 -- 172.21.15.1:0/1783157016 shutdown_connections 2023-11-16T16:36:58.126 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:36:58.122+0000 7f676ccac700 1 -- 172.21.15.1:0/1783157016 wait complete. 2023-11-16T16:36:58.126 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 7 2023-11-16T16:36:58.746 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:58 smithi026 ceph-mon[100112]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:36:58.746 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:58 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:58.747 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:58 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:58.747 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:58 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:58.747 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:58 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:36:58.747 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:58 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:36:58.747 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:58 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:58.747 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:58 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1783157016' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:36:58.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:58 smithi053 ceph-mon[100638]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:36:58.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:58 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:58.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:58 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:58.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:58 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:58.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:58 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:36:58.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:58 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:36:58.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:58 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:58.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:58 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1783157016' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:36:58.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:58 smithi001 ceph-mon[98800]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:36:58.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:58 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:58.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:58 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:58.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:58 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:36:58.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:58 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:36:58.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:58 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:36:58.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:58 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:36:58.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:58 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1783157016' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:36:59.613 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:59 smithi026 ceph-mon[100112]: Deploying daemon mds.h on smithi026 2023-11-16T16:36:59.613 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:59 smithi026 ceph-mon[100112]: mds.? [v2:172.21.15.53:6836/3197060302,v1:172.21.15.53:6837/3197060302] up:boot 2023-11-16T16:36:59.614 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:59 smithi026 ceph-mon[100112]: fsmap 7 up:standby 2023-11-16T16:36:59.614 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:36:59 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-16T16:36:59.614 INFO:journalctl@ceph.mds.h.smithi026.stdout:Nov 16 16:36:59 smithi026 systemd[1]: Starting Ceph mds.h for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:36:59.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:59 smithi053 ceph-mon[100638]: Deploying daemon mds.h on smithi026 2023-11-16T16:36:59.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:59 smithi053 ceph-mon[100638]: mds.? [v2:172.21.15.53:6836/3197060302,v1:172.21.15.53:6837/3197060302] up:boot 2023-11-16T16:36:59.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:59 smithi053 ceph-mon[100638]: fsmap 7 up:standby 2023-11-16T16:36:59.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:36:59 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-16T16:36:59.891 INFO:journalctl@ceph.mds.h.smithi026.stdout:Nov 16 16:36:59 smithi026 podman[125043]: 2023-11-16T16:36:59.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:59 smithi001 ceph-mon[98800]: Deploying daemon mds.h on smithi026 2023-11-16T16:36:59.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:59 smithi001 ceph-mon[98800]: mds.? [v2:172.21.15.53:6836/3197060302,v1:172.21.15.53:6837/3197060302] up:boot 2023-11-16T16:36:59.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:59 smithi001 ceph-mon[98800]: fsmap 7 up:standby 2023-11-16T16:36:59.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:36:59 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-16T16:37:00.138 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:37:00.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.280+0000 7f6405fda700 1 -- 172.21.15.1:0/1233686428 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64000ff1b0 msgr2=0x7f64000ff5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:00.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.280+0000 7f6405fda700 1 --2- 172.21.15.1:0/1233686428 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64000ff1b0 0x7f64000ff5c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f63e8009990 tx=0x7f63e8009ca0 comp rx=0 tx=0).stop 2023-11-16T16:37:00.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.280+0000 7f6405fda700 1 -- 172.21.15.1:0/1233686428 shutdown_connections 2023-11-16T16:37:00.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.280+0000 7f6405fda700 1 --2- 172.21.15.1:0/1233686428 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6400100df0 0x7f6400107690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:00.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.280+0000 7f6405fda700 1 --2- 172.21.15.1:0/1233686428 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64000ff1b0 0x7f64000ff5c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:00.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.280+0000 7f6405fda700 1 --2- 172.21.15.1:0/1233686428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6400100440 0x7f64001008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:00.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.280+0000 7f6405fda700 1 -- 172.21.15.1:0/1233686428 >> 172.21.15.1:0/1233686428 conn(0x7f64000fa7d0 msgr2=0x7f64000fcc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:00.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.280+0000 7f6405fda700 1 -- 172.21.15.1:0/1233686428 shutdown_connections 2023-11-16T16:37:00.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.280+0000 7f6405fda700 1 -- 172.21.15.1:0/1233686428 wait complete. 2023-11-16T16:37:00.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.280+0000 7f6405fda700 1 Processor -- start 2023-11-16T16:37:00.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.281+0000 7f6405fda700 1 -- start start 2023-11-16T16:37:00.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.281+0000 7f6405fda700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64000ff1b0 0x7f6400069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:00.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.281+0000 7f6405fda700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6400100440 0x7f640019ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:00.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.281+0000 7f6405fda700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6400100df0 0x7f640019d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:00.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.281+0000 7f6405fda700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640019dbc0 con 0x7f6400100df0 2023-11-16T16:37:00.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.281+0000 7f6405fda700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640019dd30 con 0x7f6400100440 2023-11-16T16:37:00.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.281+0000 7f6405fda700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640019def0 con 0x7f64000ff1b0 2023-11-16T16:37:00.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.281+0000 7f63fffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6400100df0 0x7f640019d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:00.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.281+0000 7f63ff7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64000ff1b0 0x7f6400069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:00.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.281+0000 7f63fffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6400100df0 0x7f640019d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41254/0 (socket says 172.21.15.1:41254) 2023-11-16T16:37:00.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.281+0000 7f63fffff700 1 -- 172.21.15.1:0/2921708294 learned_addr learned my addr 172.21.15.1:0/2921708294 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:00.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.281+0000 7f63feffd700 1 --2- 172.21.15.1:0/2921708294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6400100440 0x7f640019ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:00.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.281+0000 7f63fffff700 1 -- 172.21.15.1:0/2921708294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64000ff1b0 msgr2=0x7f6400069cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:00.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.281+0000 7f63fffff700 1 --2- 172.21.15.1:0/2921708294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64000ff1b0 0x7f6400069cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:00.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.281+0000 7f63fffff700 1 -- 172.21.15.1:0/2921708294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6400100440 msgr2=0x7f640019ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:00.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.281+0000 7f63fffff700 1 --2- 172.21.15.1:0/2921708294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6400100440 0x7f640019ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:00.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.281+0000 7f63fffff700 1 -- 172.21.15.1:0/2921708294 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e80095f0 con 0x7f6400100df0 2023-11-16T16:37:00.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.281+0000 7f63feffd700 1 --2- 172.21.15.1:0/2921708294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6400100440 0x7f640019ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:37:00.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.282+0000 7f63fffff700 1 --2- 172.21.15.1:0/2921708294 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6400100df0 0x7f640019d3f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f63f8007ee0 tx=0x7f63f800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:00.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.282+0000 7f63fcff9700 1 -- 172.21.15.1:0/2921708294 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f8017070 con 0x7f6400100df0 2023-11-16T16:37:00.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.282+0000 7f63fcff9700 1 -- 172.21.15.1:0/2921708294 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63f800f040 con 0x7f6400100df0 2023-11-16T16:37:00.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.282+0000 7f6405fda700 1 -- 172.21.15.1:0/2921708294 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f640010ab50 con 0x7f6400100df0 2023-11-16T16:37:00.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.282+0000 7f63fcff9700 1 -- 172.21.15.1:0/2921708294 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f8005580 con 0x7f6400100df0 2023-11-16T16:37:00.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.282+0000 7f6405fda700 1 -- 172.21.15.1:0/2921708294 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f640010a9c0 con 0x7f6400100df0 2023-11-16T16:37:00.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.283+0000 7f6405fda700 1 -- 172.21.15.1:0/2921708294 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6400106200 con 0x7f6400100df0 2023-11-16T16:37:00.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.284+0000 7f63fcff9700 1 -- 172.21.15.1:0/2921708294 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f63f800db20 con 0x7f6400100df0 2023-11-16T16:37:00.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.285+0000 7f63fcff9700 1 --2- 172.21.15.1:0/2921708294 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f63ec0a1150 0x7f63ec0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:00.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.285+0000 7f63ff7fe700 1 --2- 172.21.15.1:0/2921708294 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f63ec0a1150 0x7f63ec0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:00.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.285+0000 7f63fcff9700 1 -- 172.21.15.1:0/2921708294 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f63f80cc250 con 0x7f6400100df0 2023-11-16T16:37:00.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.285+0000 7f63ff7fe700 1 --2- 172.21.15.1:0/2921708294 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f63ec0a1150 0x7f63ec0a3600 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f63e800b2e0 tx=0x7f63e8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:00.290 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.288+0000 7f63fcff9700 1 -- 172.21.15.1:0/2921708294 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f63f8095e40 con 0x7f6400100df0 2023-11-16T16:37:00.461 INFO:journalctl@ceph.mds.h.smithi026.stdout:Nov 16 16:37:00 smithi026 ceph-mds[125083]: -- 172.21.15.26:0/2962438091 <== mon.2 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56048ec01520 con 0x56048ebd6c00 2023-11-16T16:37:00.462 INFO:journalctl@ceph.mds.h.smithi026.stdout:Nov 16 16:37:00 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-h[125059]: starting mds.h at 2023-11-16T16:37:00.462 INFO:journalctl@ceph.mds.h.smithi026.stdout:Nov 16 16:37:00 smithi026 bash[125043]: 0b2aed59634cc21a38afa9576295a9496aa6283cfc6e1465d8c2c3dc4bda262f 2023-11-16T16:37:00.462 INFO:journalctl@ceph.mds.h.smithi026.stdout:Nov 16 16:37:00 smithi026 systemd[1]: Started Ceph mds.h for 35d9b010-849d-11ee-8dbe-212e2dc638e7. 2023-11-16T16:37:00.535 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.534+0000 7f6405fda700 1 -- 172.21.15.1:0/2921708294 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6400061700 con 0x7f6400100df0 2023-11-16T16:37:00.536 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.534+0000 7f63fcff9700 1 -- 172.21.15.1:0/2921708294 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6544 (secure 0 0 0) 0x7f63f8099460 con 0x7f6400100df0 2023-11-16T16:37:00.536 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:37:00.536 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2587563684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2587563684},{"type":"v1","addr":"172.21.15.1:6835","nonce":2587563684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/277936168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":277936168},{"type":"v1","addr":"172.21.15.1:6837","nonce":277936168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3693905267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3693905267},{"type":"v1","addr":"172.21.15.53:6833","nonce":3693905267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/164668033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":164668033},{"type":"v1","addr":"172.21.15.26:6833","nonce":164668033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2140542192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2140542192},{"type":"v1","addr":"172.21.15.53:6835","nonce":2140542192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1010760275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1010760275},{"type":"v1","addr":"172.21.15.26:6835","nonce":1010760275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3197060302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3197060302},{"type":"v1","addr":"172.21.15.53:6837","nonce":3197060302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1453504863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1453504863},{"type":"v1","addr":"172.21.15.26:6837","nonce":1453504863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-11-16T16:37:00.538 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.536+0000 7f6405fda700 1 -- 172.21.15.1:0/2921708294 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f63ec0a1150 msgr2=0x7f63ec0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:00.538 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.536+0000 7f6405fda700 1 --2- 172.21.15.1:0/2921708294 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f63ec0a1150 0x7f63ec0a3600 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f63e800b2e0 tx=0x7f63e8005fd0 comp rx=0 tx=0).stop 2023-11-16T16:37:00.539 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.536+0000 7f6405fda700 1 -- 172.21.15.1:0/2921708294 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6400100df0 msgr2=0x7f640019d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:00.539 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.536+0000 7f6405fda700 1 --2- 172.21.15.1:0/2921708294 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6400100df0 0x7f640019d3f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f63f8007ee0 tx=0x7f63f800d3b0 comp rx=0 tx=0).stop 2023-11-16T16:37:00.539 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.536+0000 7f6405fda700 1 -- 172.21.15.1:0/2921708294 shutdown_connections 2023-11-16T16:37:00.539 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.536+0000 7f6405fda700 1 --2- 172.21.15.1:0/2921708294 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f63ec0a1150 0x7f63ec0a3600 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:00.539 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.536+0000 7f6405fda700 1 --2- 172.21.15.1:0/2921708294 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6400100df0 0x7f640019d3f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:00.539 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.536+0000 7f6405fda700 1 --2- 172.21.15.1:0/2921708294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f64000ff1b0 0x7f6400069cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:00.539 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.536+0000 7f6405fda700 1 --2- 172.21.15.1:0/2921708294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6400100440 0x7f640019ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:00.539 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.536+0000 7f6405fda700 1 -- 172.21.15.1:0/2921708294 >> 172.21.15.1:0/2921708294 conn(0x7f64000fa7d0 msgr2=0x7f64000fcab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:00.539 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.536+0000 7f6405fda700 1 -- 172.21.15.1:0/2921708294 shutdown_connections 2023-11-16T16:37:00.539 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:00.536+0000 7f6405fda700 1 -- 172.21.15.1:0/2921708294 wait complete. 2023-11-16T16:37:00.540 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 9 2023-11-16T16:37:00.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:00 smithi026 ceph-mon[100112]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:37:00.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:00 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:00.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:00 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:00.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:00 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:00.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:00 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:37:00.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:00 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:37:00.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:00 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:37:00.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:00 smithi053 ceph-mon[100638]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:37:00.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:00 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:00.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:00 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:00.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:00 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:00.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:00 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:37:00.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:00 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:37:00.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:00 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:37:00.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:00 smithi001 ceph-mon[98800]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:37:00.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:00 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:00.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:00 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:00.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:00 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:00.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:00 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:37:00.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:00 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:37:00.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:00 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:37:01.493 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:01 smithi053 ceph-mon[100638]: Deploying daemon mds.i on smithi053 2023-11-16T16:37:01.493 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:01 smithi053 ceph-mon[100638]: mds.? [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] up:boot 2023-11-16T16:37:01.493 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:01 smithi053 ceph-mon[100638]: fsmap 8 up:standby 2023-11-16T16:37:01.493 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:01 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-16T16:37:01.493 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:01 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/2921708294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:37:01.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:01 smithi026 ceph-mon[100112]: Deploying daemon mds.i on smithi053 2023-11-16T16:37:01.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:01 smithi026 ceph-mon[100112]: mds.? [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] up:boot 2023-11-16T16:37:01.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:01 smithi026 ceph-mon[100112]: fsmap 8 up:standby 2023-11-16T16:37:01.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:01 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-16T16:37:01.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:01 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/2921708294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:37:01.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:01 smithi001 ceph-mon[98800]: Deploying daemon mds.i on smithi053 2023-11-16T16:37:01.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:01 smithi001 ceph-mon[98800]: mds.? [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] up:boot 2023-11-16T16:37:01.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:01 smithi001 ceph-mon[98800]: fsmap 8 up:standby 2023-11-16T16:37:01.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:01 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-16T16:37:01.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:01 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/2921708294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:37:02.057 INFO:journalctl@ceph.mds.i.smithi053.stdout:Nov 16 16:37:01 smithi053 systemd[1]: Starting Ceph mds.i for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:37:02.330 INFO:journalctl@ceph.mds.i.smithi053.stdout:Nov 16 16:37:02 smithi053 podman[126591]: 2023-11-16T16:37:02.331 INFO:journalctl@ceph.mds.i.smithi053.stdout:Nov 16 16:37:02 smithi053 bash[126591]: 9e6d073b0887fb943fadabad3c1a7beea318f790fa5e090c993af6663081a29e 2023-11-16T16:37:02.331 INFO:journalctl@ceph.mds.i.smithi053.stdout:Nov 16 16:37:02 smithi053 ceph-mds[126631]: -- 172.21.15.53:0/350709691 <== mon.1 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562f3d85d520 con 0x562f3d833000 2023-11-16T16:37:02.331 INFO:journalctl@ceph.mds.i.smithi053.stdout:Nov 16 16:37:02 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-i[126607]: starting mds.i at 2023-11-16T16:37:02.551 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:37:02.685 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.680+0000 7f5e45255700 1 -- 172.21.15.1:0/3208622480 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e40100e00 msgr2=0x7f5e401031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:02.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.680+0000 7f5e45255700 1 --2- 172.21.15.1:0/3208622480 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e40100e00 0x7f5e401031e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5e30009990 tx=0x7f5e30009ca0 comp rx=0 tx=0).stop 2023-11-16T16:37:02.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.680+0000 7f5e45255700 1 -- 172.21.15.1:0/3208622480 shutdown_connections 2023-11-16T16:37:02.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.680+0000 7f5e45255700 1 --2- 172.21.15.1:0/3208622480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e40106040 0x7f5e40108420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:02.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.680+0000 7f5e45255700 1 --2- 172.21.15.1:0/3208622480 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e40100e00 0x7f5e401031e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:02.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.680+0000 7f5e45255700 1 --2- 172.21.15.1:0/3208622480 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e40103720 0x7f5e40105b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:02.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.680+0000 7f5e45255700 1 -- 172.21.15.1:0/3208622480 >> 172.21.15.1:0/3208622480 conn(0x7f5e400fa7c0 msgr2=0x7f5e400fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:02.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.680+0000 7f5e45255700 1 -- 172.21.15.1:0/3208622480 shutdown_connections 2023-11-16T16:37:02.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.680+0000 7f5e45255700 1 -- 172.21.15.1:0/3208622480 wait complete. 2023-11-16T16:37:02.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.681+0000 7f5e45255700 1 Processor -- start 2023-11-16T16:37:02.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.681+0000 7f5e45255700 1 -- start start 2023-11-16T16:37:02.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.681+0000 7f5e45255700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e40100e00 0x7f5e401960f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:02.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.681+0000 7f5e45255700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e40103720 0x7f5e40196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:02.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.681+0000 7f5e45255700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e40106040 0x7f5e4019d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:02.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.681+0000 7f5e45255700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4019dc70 con 0x7f5e40100e00 2023-11-16T16:37:02.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.681+0000 7f5e45255700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4019ddb0 con 0x7f5e40106040 2023-11-16T16:37:02.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.681+0000 7f5e45255700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4019df40 con 0x7f5e40103720 2023-11-16T16:37:02.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.682+0000 7f5e3ed9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e40100e00 0x7f5e401960f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:02.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.682+0000 7f5e3ed9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e40100e00 0x7f5e401960f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41264/0 (socket says 172.21.15.1:41264) 2023-11-16T16:37:02.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.682+0000 7f5e3ed9d700 1 -- 172.21.15.1:0/3298128135 learned_addr learned my addr 172.21.15.1:0/3298128135 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:02.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.682+0000 7f5e3f59e700 1 --2- 172.21.15.1:0/3298128135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e40106040 0x7f5e4019d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:02.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.682+0000 7f5e3e59c700 1 --2- 172.21.15.1:0/3298128135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e40103720 0x7f5e40196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:02.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.682+0000 7f5e3ed9d700 1 -- 172.21.15.1:0/3298128135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e40103720 msgr2=0x7f5e40196630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:02.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.682+0000 7f5e3ed9d700 1 --2- 172.21.15.1:0/3298128135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e40103720 0x7f5e40196630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:02.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.682+0000 7f5e3ed9d700 1 -- 172.21.15.1:0/3298128135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e40106040 msgr2=0x7f5e4019d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:02.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.682+0000 7f5e3ed9d700 1 --2- 172.21.15.1:0/3298128135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e40106040 0x7f5e4019d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:02.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.682+0000 7f5e3ed9d700 1 -- 172.21.15.1:0/3298128135 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e300095f0 con 0x7f5e40100e00 2023-11-16T16:37:02.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.682+0000 7f5e3ed9d700 1 --2- 172.21.15.1:0/3298128135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e40100e00 0x7f5e401960f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5e3000b680 tx=0x7f5e30009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:02.690 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.682+0000 7f5e27fff700 1 -- 172.21.15.1:0/3298128135 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3001d070 con 0x7f5e40100e00 2023-11-16T16:37:02.690 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.682+0000 7f5e27fff700 1 -- 172.21.15.1:0/3298128135 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e30021410 con 0x7f5e40100e00 2023-11-16T16:37:02.690 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.682+0000 7f5e45255700 1 -- 172.21.15.1:0/3298128135 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e4010bcc0 con 0x7f5e40100e00 2023-11-16T16:37:02.690 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.682+0000 7f5e27fff700 1 -- 172.21.15.1:0/3298128135 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e30017650 con 0x7f5e40100e00 2023-11-16T16:37:02.690 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.682+0000 7f5e45255700 1 -- 172.21.15.1:0/3298128135 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e4010bb30 con 0x7f5e40100e00 2023-11-16T16:37:02.690 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.684+0000 7f5e45255700 1 -- 172.21.15.1:0/3298128135 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e400fc330 con 0x7f5e40100e00 2023-11-16T16:37:02.691 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.684+0000 7f5e27fff700 1 -- 172.21.15.1:0/3298128135 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f5e30021580 con 0x7f5e40100e00 2023-11-16T16:37:02.692 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.685+0000 7f5e27fff700 1 --2- 172.21.15.1:0/3298128135 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f5e280a11d0 0x7f5e280a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:02.692 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.685+0000 7f5e3e59c700 1 --2- 172.21.15.1:0/3298128135 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f5e280a11d0 0x7f5e280a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:02.692 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.685+0000 7f5e27fff700 1 -- 172.21.15.1:0/3298128135 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5e300cd2b0 con 0x7f5e40100e00 2023-11-16T16:37:02.692 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.686+0000 7f5e3e59c700 1 --2- 172.21.15.1:0/3298128135 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f5e280a11d0 0x7f5e280a3680 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5e34009bf0 tx=0x7f5e34009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:02.692 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.689+0000 7f5e27fff700 1 -- 172.21.15.1:0/3298128135 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5e30096f30 con 0x7f5e40100e00 2023-11-16T16:37:02.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:02 smithi026 ceph-mon[100112]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:37:02.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:02 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:02.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:02 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:02.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:02 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:02.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:02 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:37:02.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:02 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:37:02.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:02 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:37:02.831 INFO:journalctl@ceph.mds.i.smithi053.stdout:Nov 16 16:37:02 smithi053 systemd[1]: Started Ceph mds.i for 35d9b010-849d-11ee-8dbe-212e2dc638e7. 2023-11-16T16:37:02.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:02 smithi053 ceph-mon[100638]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:37:02.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:02 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:02.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:02 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:02.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:02 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:02.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:02 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:37:02.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:02 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:37:02.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:02 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:37:02.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:02 smithi001 ceph-mon[98800]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:37:02.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:02 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:02.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:02 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:02.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:02 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:02.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:02 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:37:02.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:02 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:37:02.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:02 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:37:02.962 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.960+0000 7f5e45255700 1 -- 172.21.15.1:0/3298128135 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e40067af0 con 0x7f5e40100e00 2023-11-16T16:37:02.963 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.961+0000 7f5e27fff700 1 -- 172.21.15.1:0/3298128135 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7303 (secure 0 0 0) 0x7f5e3009a550 con 0x7f5e40100e00 2023-11-16T16:37:02.963 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:37:02.963 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2587563684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2587563684},{"type":"v1","addr":"172.21.15.1:6835","nonce":2587563684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/277936168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":277936168},{"type":"v1","addr":"172.21.15.1:6837","nonce":277936168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3693905267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3693905267},{"type":"v1","addr":"172.21.15.53:6833","nonce":3693905267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/164668033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":164668033},{"type":"v1","addr":"172.21.15.26:6833","nonce":164668033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2140542192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2140542192},{"type":"v1","addr":"172.21.15.53:6835","nonce":2140542192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1010760275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1010760275},{"type":"v1","addr":"172.21.15.26:6835","nonce":1010760275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3197060302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3197060302},{"type":"v1","addr":"172.21.15.53:6837","nonce":3197060302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1453504863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1453504863},{"type":"v1","addr":"172.21.15.26:6837","nonce":1453504863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6839/1044753024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1044753024},{"type":"v1","addr":"172.21.15.53:6839","nonce":1044753024}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-11-16T16:37:02.965 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.963+0000 7f5e45255700 1 -- 172.21.15.1:0/3298128135 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f5e280a11d0 msgr2=0x7f5e280a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:02.965 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.963+0000 7f5e45255700 1 --2- 172.21.15.1:0/3298128135 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f5e280a11d0 0x7f5e280a3680 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5e34009bf0 tx=0x7f5e34009260 comp rx=0 tx=0).stop 2023-11-16T16:37:02.966 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.963+0000 7f5e45255700 1 -- 172.21.15.1:0/3298128135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e40100e00 msgr2=0x7f5e401960f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:02.966 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.963+0000 7f5e45255700 1 --2- 172.21.15.1:0/3298128135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e40100e00 0x7f5e401960f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5e3000b680 tx=0x7f5e30009580 comp rx=0 tx=0).stop 2023-11-16T16:37:02.966 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.963+0000 7f5e45255700 1 -- 172.21.15.1:0/3298128135 shutdown_connections 2023-11-16T16:37:02.966 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.963+0000 7f5e45255700 1 --2- 172.21.15.1:0/3298128135 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f5e280a11d0 0x7f5e280a3680 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:02.967 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.963+0000 7f5e45255700 1 --2- 172.21.15.1:0/3298128135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e40106040 0x7f5e4019d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:02.967 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.963+0000 7f5e45255700 1 --2- 172.21.15.1:0/3298128135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e40100e00 0x7f5e401960f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:02.967 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.963+0000 7f5e45255700 1 --2- 172.21.15.1:0/3298128135 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e40103720 0x7f5e40196630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:02.967 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.963+0000 7f5e45255700 1 -- 172.21.15.1:0/3298128135 >> 172.21.15.1:0/3298128135 conn(0x7f5e400fa7c0 msgr2=0x7f5e40101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:02.967 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.963+0000 7f5e45255700 1 -- 172.21.15.1:0/3298128135 shutdown_connections 2023-11-16T16:37:02.967 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:02.963+0000 7f5e45255700 1 -- 172.21.15.1:0/3298128135 wait complete. 2023-11-16T16:37:02.968 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 10 2023-11-16T16:37:03.528 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:03 smithi001 ceph-mon[98800]: Deploying daemon mds.g on smithi001 2023-11-16T16:37:03.528 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:03 smithi001 ceph-mon[98800]: mds.? [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] up:boot 2023-11-16T16:37:03.529 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:03 smithi001 ceph-mon[98800]: fsmap 9 up:standby 2023-11-16T16:37:03.529 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:03 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-16T16:37:03.529 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:03 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3298128135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:37:03.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:03 smithi026 ceph-mon[100112]: Deploying daemon mds.g on smithi001 2023-11-16T16:37:03.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:03 smithi026 ceph-mon[100112]: mds.? [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] up:boot 2023-11-16T16:37:03.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:03 smithi026 ceph-mon[100112]: fsmap 9 up:standby 2023-11-16T16:37:03.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:03 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-16T16:37:03.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:03 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/3298128135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:37:03.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:03 smithi053 ceph-mon[100638]: Deploying daemon mds.g on smithi001 2023-11-16T16:37:03.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:03 smithi053 ceph-mon[100638]: mds.? [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] up:boot 2023-11-16T16:37:03.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:03 smithi053 ceph-mon[100638]: fsmap 9 up:standby 2023-11-16T16:37:03.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:03 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-16T16:37:03.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:03 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/3298128135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:37:04.087 INFO:journalctl@ceph.mds.g.smithi001.stdout:Nov 16 16:37:04 smithi001 systemd[1]: Starting Ceph mds.g for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:37:04.662 INFO:journalctl@ceph.mds.g.smithi001.stdout:Nov 16 16:37:04 smithi001 podman[133137]: 2023-11-16T16:37:04.662 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:04 smithi001 ceph-mon[98800]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:37:04.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:04 smithi026 ceph-mon[100112]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:37:04.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:04 smithi053 ceph-mon[100638]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:37:04.932 INFO:journalctl@ceph.mds.g.smithi001.stdout:Nov 16 16:37:04 smithi001 ceph-mds[133177]: -- 172.21.15.1:0/4133508660 <== mon.0 v2:172.21.15.1:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5646f98af520 con 0x5646f9884c00 2023-11-16T16:37:04.932 INFO:journalctl@ceph.mds.g.smithi001.stdout:Nov 16 16:37:04 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-g[133153]: starting mds.g at 2023-11-16T16:37:04.932 INFO:journalctl@ceph.mds.g.smithi001.stdout:Nov 16 16:37:04 smithi001 bash[133137]: 3b7913e030f8fa3b2a71423a1abd46f76ade09254ff6b16df354bcaaa71481c9 2023-11-16T16:37:04.932 INFO:journalctl@ceph.mds.g.smithi001.stdout:Nov 16 16:37:04 smithi001 systemd[1]: Started Ceph mds.g for 35d9b010-849d-11ee-8dbe-212e2dc638e7. 2023-11-16T16:37:04.984 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:37:05.117 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.115+0000 7f9334cb3700 1 -- 172.21.15.1:0/1961810119 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9330100430 msgr2=0x7f93301008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:05.117 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.115+0000 7f9334cb3700 1 --2- 172.21.15.1:0/1961810119 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9330100430 0x7f93301008a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9318009990 tx=0x7f9318009ca0 comp rx=0 tx=0).stop 2023-11-16T16:37:05.117 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.115+0000 7f9334cb3700 1 -- 172.21.15.1:0/1961810119 shutdown_connections 2023-11-16T16:37:05.117 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.115+0000 7f9334cb3700 1 --2- 172.21.15.1:0/1961810119 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9330100de0 0x7f9330107680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:05.118 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.115+0000 7f9334cb3700 1 --2- 172.21.15.1:0/1961810119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93300ff720 0x7f93300ffb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:05.118 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.115+0000 7f9334cb3700 1 --2- 172.21.15.1:0/1961810119 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9330100430 0x7f93301008a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:05.118 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.115+0000 7f9334cb3700 1 -- 172.21.15.1:0/1961810119 >> 172.21.15.1:0/1961810119 conn(0x7f93300f9f80 msgr2=0x7f93300fc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:05.118 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.115+0000 7f9334cb3700 1 -- 172.21.15.1:0/1961810119 shutdown_connections 2023-11-16T16:37:05.118 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.115+0000 7f9334cb3700 1 -- 172.21.15.1:0/1961810119 wait complete. 2023-11-16T16:37:05.120 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.116+0000 7f9334cb3700 1 Processor -- start 2023-11-16T16:37:05.121 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.116+0000 7f9334cb3700 1 -- start start 2023-11-16T16:37:05.121 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.116+0000 7f9334cb3700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93300ff720 0x7f9330195020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:05.121 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.116+0000 7f9334cb3700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9330100430 0x7f9330195560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:05.121 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.116+0000 7f9334cb3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9330100de0 0x7f933019c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:05.121 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.116+0000 7f9334cb3700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933019cba0 con 0x7f9330100430 2023-11-16T16:37:05.121 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.116+0000 7f9334cb3700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933019cce0 con 0x7f9330100de0 2023-11-16T16:37:05.121 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.116+0000 7f9334cb3700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933019ce70 con 0x7f93300ff720 2023-11-16T16:37:05.121 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.116+0000 7f9327fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9330100430 0x7f9330195560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:05.121 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.116+0000 7f9327fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9330100430 0x7f9330195560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41292/0 (socket says 172.21.15.1:41292) 2023-11-16T16:37:05.121 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.116+0000 7f9327fff700 1 -- 172.21.15.1:0/1779334606 learned_addr learned my addr 172.21.15.1:0/1779334606 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:05.122 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.116+0000 7f932e59c700 1 --2- 172.21.15.1:0/1779334606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93300ff720 0x7f9330195020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:05.122 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.116+0000 7f932ed9d700 1 --2- 172.21.15.1:0/1779334606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9330100de0 0x7f933019c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:05.122 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.116+0000 7f9327fff700 1 -- 172.21.15.1:0/1779334606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93300ff720 msgr2=0x7f9330195020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:05.122 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.116+0000 7f9327fff700 1 --2- 172.21.15.1:0/1779334606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93300ff720 0x7f9330195020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:05.122 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.116+0000 7f9327fff700 1 -- 172.21.15.1:0/1779334606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9330100de0 msgr2=0x7f933019c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:05.122 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.116+0000 7f9327fff700 1 --2- 172.21.15.1:0/1779334606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9330100de0 0x7f933019c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:05.122 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.116+0000 7f9327fff700 1 -- 172.21.15.1:0/1779334606 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93180095f0 con 0x7f9330100430 2023-11-16T16:37:05.122 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.117+0000 7f9327fff700 1 --2- 172.21.15.1:0/1779334606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9330100430 0x7f9330195560 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f931800f730 tx=0x7f931800f760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:05.122 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.117+0000 7f93277fe700 1 -- 172.21.15.1:0/1779334606 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931801d070 con 0x7f9330100430 2023-11-16T16:37:05.122 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.117+0000 7f9334cb3700 1 -- 172.21.15.1:0/1779334606 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f933010abd0 con 0x7f9330100430 2023-11-16T16:37:05.123 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.117+0000 7f93277fe700 1 -- 172.21.15.1:0/1779334606 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9318004ca0 con 0x7f9330100430 2023-11-16T16:37:05.123 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.117+0000 7f93277fe700 1 -- 172.21.15.1:0/1779334606 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9318005570 con 0x7f9330100430 2023-11-16T16:37:05.123 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.117+0000 7f9334cb3700 1 -- 172.21.15.1:0/1779334606 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f933010aa40 con 0x7f9330100430 2023-11-16T16:37:05.123 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.118+0000 7f9334cb3700 1 -- 172.21.15.1:0/1779334606 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9330106780 con 0x7f9330100430 2023-11-16T16:37:05.123 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.119+0000 7f93277fe700 1 -- 172.21.15.1:0/1779334606 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f931800fbd0 con 0x7f9330100430 2023-11-16T16:37:05.124 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.119+0000 7f93277fe700 1 --2- 172.21.15.1:0/1779334606 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f931c0a1150 0x7f931c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:05.124 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.119+0000 7f93277fe700 1 -- 172.21.15.1:0/1779334606 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f93180cd580 con 0x7f9330100430 2023-11-16T16:37:05.124 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.119+0000 7f932e59c700 1 --2- 172.21.15.1:0/1779334606 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f931c0a1150 0x7f931c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:05.124 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.120+0000 7f932e59c700 1 --2- 172.21.15.1:0/1779334606 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f931c0a1150 0x7f931c0a3600 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9330068c70 tx=0x7f932000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:05.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.124+0000 7f93277fe700 1 -- 172.21.15.1:0/1779334606 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9318097200 con 0x7f9330100430 2023-11-16T16:37:05.378 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.376+0000 7f9334cb3700 1 -- 172.21.15.1:0/1779334606 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9330061700 con 0x7f9330100430 2023-11-16T16:37:05.378 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.377+0000 7f93277fe700 1 -- 172.21.15.1:0/1779334606 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7303 (secure 0 0 0) 0x7f931809a820 con 0x7f9330100430 2023-11-16T16:37:05.379 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:37:05.379 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2587563684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2587563684},{"type":"v1","addr":"172.21.15.1:6835","nonce":2587563684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/277936168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":277936168},{"type":"v1","addr":"172.21.15.1:6837","nonce":277936168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3693905267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3693905267},{"type":"v1","addr":"172.21.15.53:6833","nonce":3693905267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/164668033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":164668033},{"type":"v1","addr":"172.21.15.26:6833","nonce":164668033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2140542192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2140542192},{"type":"v1","addr":"172.21.15.53:6835","nonce":2140542192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1010760275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1010760275},{"type":"v1","addr":"172.21.15.26:6835","nonce":1010760275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3197060302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3197060302},{"type":"v1","addr":"172.21.15.53:6837","nonce":3197060302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1453504863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1453504863},{"type":"v1","addr":"172.21.15.26:6837","nonce":1453504863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6839/1044753024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1044753024},{"type":"v1","addr":"172.21.15.53:6839","nonce":1044753024}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-11-16T16:37:05.381 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.379+0000 7f9334cb3700 1 -- 172.21.15.1:0/1779334606 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f931c0a1150 msgr2=0x7f931c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:05.381 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.379+0000 7f9334cb3700 1 --2- 172.21.15.1:0/1779334606 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f931c0a1150 0x7f931c0a3600 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9330068c70 tx=0x7f932000a380 comp rx=0 tx=0).stop 2023-11-16T16:37:05.381 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.379+0000 7f9334cb3700 1 -- 172.21.15.1:0/1779334606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9330100430 msgr2=0x7f9330195560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:05.381 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.379+0000 7f9334cb3700 1 --2- 172.21.15.1:0/1779334606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9330100430 0x7f9330195560 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f931800f730 tx=0x7f931800f760 comp rx=0 tx=0).stop 2023-11-16T16:37:05.381 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.379+0000 7f9334cb3700 1 -- 172.21.15.1:0/1779334606 shutdown_connections 2023-11-16T16:37:05.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.379+0000 7f9334cb3700 1 --2- 172.21.15.1:0/1779334606 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f931c0a1150 0x7f931c0a3600 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:05.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.379+0000 7f9334cb3700 1 --2- 172.21.15.1:0/1779334606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9330100de0 0x7f933019c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:05.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.379+0000 7f9334cb3700 1 --2- 172.21.15.1:0/1779334606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93300ff720 0x7f9330195020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:05.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.379+0000 7f9334cb3700 1 --2- 172.21.15.1:0/1779334606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9330100430 0x7f9330195560 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:05.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.379+0000 7f9334cb3700 1 -- 172.21.15.1:0/1779334606 >> 172.21.15.1:0/1779334606 conn(0x7f93300f9f80 msgr2=0x7f93301038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:05.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.379+0000 7f9334cb3700 1 -- 172.21.15.1:0/1779334606 shutdown_connections 2023-11-16T16:37:05.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:05.379+0000 7f9334cb3700 1 -- 172.21.15.1:0/1779334606 wait complete. 2023-11-16T16:37:05.382 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 10 2023-11-16T16:37:06.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:05 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:06.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:05 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:06.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:05 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:06.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:05 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:37:06.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:05 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:37:06.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:05 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:37:06.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:05 smithi026 ceph-mon[100112]: Deploying daemon mds.a on smithi001 2023-11-16T16:37:06.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:05 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1779334606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:37:06.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:05 smithi026 ceph-mon[100112]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:37:06.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:05 smithi026 ceph-mon[100112]: mds.? [v2:172.21.15.1:6838/1172575045,v1:172.21.15.1:6839/1172575045] up:boot 2023-11-16T16:37:06.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:05 smithi026 ceph-mon[100112]: fsmap 10 up:standby 2023-11-16T16:37:06.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:05 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-11-16T16:37:06.036 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:05 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:06.036 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:05 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:06.037 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:05 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:06.037 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:05 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:37:06.037 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:05 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:37:06.037 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:05 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:37:06.037 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:05 smithi001 ceph-mon[98800]: Deploying daemon mds.a on smithi001 2023-11-16T16:37:06.037 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:05 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1779334606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:37:06.037 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:05 smithi001 ceph-mon[98800]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:37:06.037 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:05 smithi001 ceph-mon[98800]: mds.? [v2:172.21.15.1:6838/1172575045,v1:172.21.15.1:6839/1172575045] up:boot 2023-11-16T16:37:06.037 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:05 smithi001 ceph-mon[98800]: fsmap 10 up:standby 2023-11-16T16:37:06.037 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:05 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-11-16T16:37:06.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:05 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:06.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:05 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:06.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:05 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:06.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:05 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:37:06.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:05 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:37:06.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:05 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:37:06.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:05 smithi053 ceph-mon[100638]: Deploying daemon mds.a on smithi001 2023-11-16T16:37:06.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:05 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1779334606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:37:06.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:05 smithi053 ceph-mon[100638]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:37:06.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:05 smithi053 ceph-mon[100638]: mds.? [v2:172.21.15.1:6838/1172575045,v1:172.21.15.1:6839/1172575045] up:boot 2023-11-16T16:37:06.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:05 smithi053 ceph-mon[100638]: fsmap 10 up:standby 2023-11-16T16:37:06.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:05 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-11-16T16:37:06.579 INFO:journalctl@ceph.mds.a.smithi001.stdout:Nov 16 16:37:06 smithi001 systemd[1]: Starting Ceph mds.a for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:37:06.862 INFO:journalctl@ceph.mds.a.smithi001.stdout:Nov 16 16:37:06 smithi001 podman[133528]: 2023-11-16T16:37:06.862 INFO:journalctl@ceph.mds.a.smithi001.stdout:Nov 16 16:37:06 smithi001 ceph-mds[133568]: -- 172.21.15.1:0/2099141208 <== mon.0 v2:172.21.15.1:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fd0feab520 con 0x55fd0fe81000 2023-11-16T16:37:07.183 INFO:journalctl@ceph.mds.a.smithi001.stdout:Nov 16 16:37:06 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-a[133544]: starting mds.a at 2023-11-16T16:37:07.183 INFO:journalctl@ceph.mds.a.smithi001.stdout:Nov 16 16:37:06 smithi001 bash[133528]: e59f1e9a7cf31105773973b1892be2df37c750631684da5e841495b7ac44a8a5 2023-11-16T16:37:07.183 INFO:journalctl@ceph.mds.a.smithi001.stdout:Nov 16 16:37:06 smithi001 systemd[1]: Started Ceph mds.a for 35d9b010-849d-11ee-8dbe-212e2dc638e7. 2023-11-16T16:37:07.394 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:37:07.532 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.530+0000 7fd2af7b6700 1 -- 172.21.15.1:0/4175478213 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2a80ff1c0 msgr2=0x7fd2a80ff5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:07.532 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.530+0000 7fd2af7b6700 1 --2- 172.21.15.1:0/4175478213 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2a80ff1c0 0x7fd2a80ff5d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd298009990 tx=0x7fd298009ca0 comp rx=0 tx=0).stop 2023-11-16T16:37:07.532 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.530+0000 7fd2af7b6700 1 -- 172.21.15.1:0/4175478213 shutdown_connections 2023-11-16T16:37:07.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.530+0000 7fd2af7b6700 1 --2- 172.21.15.1:0/4175478213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2a8100d40 0x7fd2a8107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:07.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.530+0000 7fd2af7b6700 1 --2- 172.21.15.1:0/4175478213 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2a80ff1c0 0x7fd2a80ff5d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:07.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.530+0000 7fd2af7b6700 1 --2- 172.21.15.1:0/4175478213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2a81003b0 0x7fd2a8100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:07.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.530+0000 7fd2af7b6700 1 -- 172.21.15.1:0/4175478213 >> 172.21.15.1:0/4175478213 conn(0x7fd2a80fa7c0 msgr2=0x7fd2a80fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:07.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.530+0000 7fd2af7b6700 1 -- 172.21.15.1:0/4175478213 shutdown_connections 2023-11-16T16:37:07.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.530+0000 7fd2af7b6700 1 -- 172.21.15.1:0/4175478213 wait complete. 2023-11-16T16:37:07.535 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.531+0000 7fd2af7b6700 1 Processor -- start 2023-11-16T16:37:07.535 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.531+0000 7fd2af7b6700 1 -- start start 2023-11-16T16:37:07.535 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.531+0000 7fd2af7b6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2a80ff1c0 0x7fd2a8103450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:07.536 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.531+0000 7fd2af7b6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2a81003b0 0x7fd2a8101a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:07.536 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.531+0000 7fd2af7b6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2a8100d40 0x7fd2a8101fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:07.536 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.531+0000 7fd2af7b6700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a8102770 con 0x7fd2a80ff1c0 2023-11-16T16:37:07.536 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.531+0000 7fd2af7b6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a81028e0 con 0x7fd2a81003b0 2023-11-16T16:37:07.536 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.531+0000 7fd2af7b6700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a81a0180 con 0x7fd2a8100d40 2023-11-16T16:37:07.536 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.531+0000 7fd2ad552700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2a80ff1c0 0x7fd2a8103450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:07.536 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.531+0000 7fd2ad552700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2a80ff1c0 0x7fd2a8103450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41318/0 (socket says 172.21.15.1:41318) 2023-11-16T16:37:07.536 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.531+0000 7fd2ad552700 1 -- 172.21.15.1:0/874807529 learned_addr learned my addr 172.21.15.1:0/874807529 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:07.536 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.531+0000 7fd2acd51700 1 --2- 172.21.15.1:0/874807529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2a81003b0 0x7fd2a8101a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:07.536 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.532+0000 7fd2add53700 1 --2- 172.21.15.1:0/874807529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2a8100d40 0x7fd2a8101fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:07.536 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.532+0000 7fd2ad552700 1 -- 172.21.15.1:0/874807529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2a8100d40 msgr2=0x7fd2a8101fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:07.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.532+0000 7fd2ad552700 1 --2- 172.21.15.1:0/874807529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2a8100d40 0x7fd2a8101fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:07.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.532+0000 7fd2ad552700 1 -- 172.21.15.1:0/874807529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2a81003b0 msgr2=0x7fd2a8101a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:07.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.532+0000 7fd2ad552700 1 --2- 172.21.15.1:0/874807529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2a81003b0 0x7fd2a8101a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:07.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.532+0000 7fd2ad552700 1 -- 172.21.15.1:0/874807529 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2980095f0 con 0x7fd2a80ff1c0 2023-11-16T16:37:07.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.532+0000 7fd2acd51700 1 --2- 172.21.15.1:0/874807529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2a81003b0 0x7fd2a8101a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:37:07.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.532+0000 7fd2ad552700 1 --2- 172.21.15.1:0/874807529 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2a80ff1c0 0x7fd2a8103450 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd298006010 tx=0x7fd298005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:07.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.532+0000 7fd29e7fc700 1 -- 172.21.15.1:0/874807529 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29801d070 con 0x7fd2a80ff1c0 2023-11-16T16:37:07.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.532+0000 7fd29e7fc700 1 -- 172.21.15.1:0/874807529 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2980040e0 con 0x7fd2a80ff1c0 2023-11-16T16:37:07.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.532+0000 7fd2af7b6700 1 -- 172.21.15.1:0/874807529 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a810aba0 con 0x7fd2a80ff1c0 2023-11-16T16:37:07.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.532+0000 7fd29e7fc700 1 -- 172.21.15.1:0/874807529 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd298022e70 con 0x7fd2a80ff1c0 2023-11-16T16:37:07.538 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.532+0000 7fd2af7b6700 1 -- 172.21.15.1:0/874807529 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a810aa10 con 0x7fd2a80ff1c0 2023-11-16T16:37:07.538 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.533+0000 7fd2af7b6700 1 -- 172.21.15.1:0/874807529 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2a8107290 con 0x7fd2a80ff1c0 2023-11-16T16:37:07.538 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.535+0000 7fd29e7fc700 1 -- 172.21.15.1:0/874807529 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fd298022570 con 0x7fd2a80ff1c0 2023-11-16T16:37:07.539 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.535+0000 7fd29e7fc700 1 --2- 172.21.15.1:0/874807529 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd2940a1150 0x7fd2940a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:07.539 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.535+0000 7fd29e7fc700 1 -- 172.21.15.1:0/874807529 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd2980cdc60 con 0x7fd2a80ff1c0 2023-11-16T16:37:07.539 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.536+0000 7fd2acd51700 1 --2- 172.21.15.1:0/874807529 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd2940a1150 0x7fd2940a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:07.539 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.536+0000 7fd2acd51700 1 --2- 172.21.15.1:0/874807529 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd2940a1150 0x7fd2940a3600 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd2a40099d0 tx=0x7fd2a4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:07.540 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.539+0000 7fd29e7fc700 1 -- 172.21.15.1:0/874807529 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd298097840 con 0x7fd2a80ff1c0 2023-11-16T16:37:07.789 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.787+0000 7fd2af7b6700 1 -- 172.21.15.1:0/874807529 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2a802cfe0 con 0x7fd2a80ff1c0 2023-11-16T16:37:07.790 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.788+0000 7fd29e7fc700 1 -- 172.21.15.1:0/874807529 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8813 (secure 0 0 0) 0x7fd2980b2280 con 0x7fd2a80ff1c0 2023-11-16T16:37:07.790 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:37:07.790 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2587563684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2587563684},{"type":"v1","addr":"172.21.15.1:6835","nonce":2587563684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/277936168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":277936168},{"type":"v1","addr":"172.21.15.1:6837","nonce":277936168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/1172575045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1172575045},{"type":"v1","addr":"172.21.15.1:6839","nonce":1172575045}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/2631175899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2631175899},{"type":"v1","addr":"172.21.15.1:6841","nonce":2631175899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3693905267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3693905267},{"type":"v1","addr":"172.21.15.53:6833","nonce":3693905267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/164668033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":164668033},{"type":"v1","addr":"172.21.15.26:6833","nonce":164668033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2140542192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2140542192},{"type":"v1","addr":"172.21.15.53:6835","nonce":2140542192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1010760275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1010760275},{"type":"v1","addr":"172.21.15.26:6835","nonce":1010760275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3197060302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3197060302},{"type":"v1","addr":"172.21.15.53:6837","nonce":3197060302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1453504863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1453504863},{"type":"v1","addr":"172.21.15.26:6837","nonce":1453504863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6839/1044753024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1044753024},{"type":"v1","addr":"172.21.15.53:6839","nonce":1044753024}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-11-16T16:37:07.792 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.790+0000 7fd2af7b6700 1 -- 172.21.15.1:0/874807529 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd2940a1150 msgr2=0x7fd2940a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:07.792 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.790+0000 7fd2af7b6700 1 --2- 172.21.15.1:0/874807529 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd2940a1150 0x7fd2940a3600 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd2a40099d0 tx=0x7fd2a4008040 comp rx=0 tx=0).stop 2023-11-16T16:37:07.793 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.790+0000 7fd2af7b6700 1 -- 172.21.15.1:0/874807529 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2a80ff1c0 msgr2=0x7fd2a8103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:07.793 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.790+0000 7fd2af7b6700 1 --2- 172.21.15.1:0/874807529 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2a80ff1c0 0x7fd2a8103450 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd298006010 tx=0x7fd298005ef0 comp rx=0 tx=0).stop 2023-11-16T16:37:07.793 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.790+0000 7fd2af7b6700 1 -- 172.21.15.1:0/874807529 shutdown_connections 2023-11-16T16:37:07.793 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.790+0000 7fd2af7b6700 1 --2- 172.21.15.1:0/874807529 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd2940a1150 0x7fd2940a3600 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:07.793 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.790+0000 7fd2af7b6700 1 --2- 172.21.15.1:0/874807529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd2a8100d40 0x7fd2a8101fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:07.793 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.790+0000 7fd2af7b6700 1 --2- 172.21.15.1:0/874807529 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd2a80ff1c0 0x7fd2a8103450 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:07.793 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.790+0000 7fd2af7b6700 1 --2- 172.21.15.1:0/874807529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2a81003b0 0x7fd2a8101a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:07.793 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.790+0000 7fd2af7b6700 1 -- 172.21.15.1:0/874807529 >> 172.21.15.1:0/874807529 conn(0x7fd2a80fa7c0 msgr2=0x7fd2a80fca50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:07.793 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.790+0000 7fd2af7b6700 1 -- 172.21.15.1:0/874807529 shutdown_connections 2023-11-16T16:37:07.793 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:07.790+0000 7fd2af7b6700 1 -- 172.21.15.1:0/874807529 wait complete. 2023-11-16T16:37:07.794 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 12 2023-11-16T16:37:07.994 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:07 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:07.995 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:07 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:07.995 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:07 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:07.995 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:07 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:37:07.995 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:07 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:37:07.995 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:07 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:37:07.995 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:07 smithi026 ceph-mon[100112]: Deploying daemon mds.b on smithi026 2023-11-16T16:37:07.995 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:07 smithi026 ceph-mon[100112]: mds.? [v2:172.21.15.1:6840/2631175899,v1:172.21.15.1:6841/2631175899] up:boot 2023-11-16T16:37:07.995 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:07 smithi026 ceph-mon[100112]: fsmap 11 up:standby 2023-11-16T16:37:07.995 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:07 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-11-16T16:37:07.995 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:07 smithi026 ceph-mon[100112]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:37:07.995 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:07 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/874807529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:37:08.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:07 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:08.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:07 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:08.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:07 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:08.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:07 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:37:08.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:07 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:37:08.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:07 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:37:08.184 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:07 smithi001 ceph-mon[98800]: Deploying daemon mds.b on smithi026 2023-11-16T16:37:08.184 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:07 smithi001 ceph-mon[98800]: mds.? [v2:172.21.15.1:6840/2631175899,v1:172.21.15.1:6841/2631175899] up:boot 2023-11-16T16:37:08.184 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:07 smithi001 ceph-mon[98800]: fsmap 11 up:standby 2023-11-16T16:37:08.184 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:07 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-11-16T16:37:08.184 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:07 smithi001 ceph-mon[98800]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:37:08.184 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:07 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/874807529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:37:08.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:07 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:08.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:07 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:08.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:07 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:08.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:07 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-16T16:37:08.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:07 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-16T16:37:08.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:07 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:37:08.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:07 smithi053 ceph-mon[100638]: Deploying daemon mds.b on smithi026 2023-11-16T16:37:08.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:07 smithi053 ceph-mon[100638]: mds.? [v2:172.21.15.1:6840/2631175899,v1:172.21.15.1:6841/2631175899] up:boot 2023-11-16T16:37:08.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:07 smithi053 ceph-mon[100638]: fsmap 11 up:standby 2023-11-16T16:37:08.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:07 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-11-16T16:37:08.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:07 smithi053 ceph-mon[100638]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:37:08.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:07 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/874807529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:37:08.758 INFO:journalctl@ceph.mds.b.smithi026.stdout:Nov 16 16:37:08 smithi026 systemd[1]: Starting Ceph mds.b for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:37:09.176 INFO:journalctl@ceph.mds.b.smithi026.stdout:Nov 16 16:37:08 smithi026 podman[125381]: 2023-11-16T16:37:09.508 INFO:journalctl@ceph.mds.b.smithi026.stdout:Nov 16 16:37:09 smithi026 bash[125381]: d1169a3a3254914c4874b97a7681c2fa5d2fa8b2f3e5d02de8fa82dbdb0aa42b 2023-11-16T16:37:09.508 INFO:journalctl@ceph.mds.b.smithi026.stdout:Nov 16 16:37:09 smithi026 ceph-mds[125422]: -- 172.21.15.26:0/244200164 <== mon.2 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558a5dc23520 con 0x558a5dbf8c00 2023-11-16T16:37:09.508 INFO:journalctl@ceph.mds.b.smithi026.stdout:Nov 16 16:37:09 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-b[125397]: starting mds.b at 2023-11-16T16:37:09.508 INFO:journalctl@ceph.mds.b.smithi026.stdout:Nov 16 16:37:09 smithi026 systemd[1]: Started Ceph mds.b for 35d9b010-849d-11ee-8dbe-212e2dc638e7. 2023-11-16T16:37:09.805 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:37:09.942 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.940+0000 7f3219dc8700 1 -- 172.21.15.1:0/1108412945 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32140ffe00 msgr2=0x7f32141076b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:09.942 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.940+0000 7f3219dc8700 1 --2- 172.21.15.1:0/1108412945 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32140ffe00 0x7f32141076b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f320c00b0a0 tx=0x7f320c00b3b0 comp rx=0 tx=0).stop 2023-11-16T16:37:09.942 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.940+0000 7f3219dc8700 1 -- 172.21.15.1:0/1108412945 shutdown_connections 2023-11-16T16:37:09.942 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.940+0000 7f3219dc8700 1 --2- 172.21.15.1:0/1108412945 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32140ffe00 0x7f32141076b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:09.942 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.940+0000 7f3219dc8700 1 --2- 172.21.15.1:0/1108412945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3214100800 0x7f3214100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:09.942 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.940+0000 7f3219dc8700 1 --2- 172.21.15.1:0/1108412945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32140ff450 0x7f32140ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:09.942 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.940+0000 7f3219dc8700 1 -- 172.21.15.1:0/1108412945 >> 172.21.15.1:0/1108412945 conn(0x7f32140f9f70 msgr2=0x7f32140fc3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:09.942 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.940+0000 7f3219dc8700 1 -- 172.21.15.1:0/1108412945 shutdown_connections 2023-11-16T16:37:09.943 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.940+0000 7f3219dc8700 1 -- 172.21.15.1:0/1108412945 wait complete. 2023-11-16T16:37:09.943 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.940+0000 7f3219dc8700 1 Processor -- start 2023-11-16T16:37:09.944 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.940+0000 7f3219dc8700 1 -- start start 2023-11-16T16:37:09.945 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.941+0000 7f3219dc8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32140ff450 0x7f3214103200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:09.945 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.941+0000 7f3219dc8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32140ffe00 0x7f3214101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:09.945 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.941+0000 7f3219dc8700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3214100800 0x7f3214101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:09.945 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.941+0000 7f3219dc8700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3214199fe0 con 0x7f32140ff450 2023-11-16T16:37:09.945 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.941+0000 7f3219dc8700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32141022c0 con 0x7f32140ffe00 2023-11-16T16:37:09.945 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.941+0000 7f3219dc8700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3214102480 con 0x7f3214100800 2023-11-16T16:37:09.945 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.941+0000 7f32137fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32140ff450 0x7f3214103200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:09.945 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.941+0000 7f32137fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32140ff450 0x7f3214103200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41332/0 (socket says 172.21.15.1:41332) 2023-11-16T16:37:09.945 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.941+0000 7f32137fe700 1 -- 172.21.15.1:0/4028287245 learned_addr learned my addr 172.21.15.1:0/4028287245 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:09.946 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.941+0000 7f3212ffd700 1 --2- 172.21.15.1:0/4028287245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32140ffe00 0x7f3214101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:09.946 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.941+0000 7f3213fff700 1 --2- 172.21.15.1:0/4028287245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3214100800 0x7f3214101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:09.946 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.941+0000 7f32137fe700 1 -- 172.21.15.1:0/4028287245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3214100800 msgr2=0x7f3214101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:09.946 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.941+0000 7f32137fe700 1 --2- 172.21.15.1:0/4028287245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3214100800 0x7f3214101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:09.946 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.941+0000 7f32137fe700 1 -- 172.21.15.1:0/4028287245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32140ffe00 msgr2=0x7f3214101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:09.946 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.941+0000 7f32137fe700 1 --2- 172.21.15.1:0/4028287245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32140ffe00 0x7f3214101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:09.946 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.941+0000 7f32137fe700 1 -- 172.21.15.1:0/4028287245 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f320c009d00 con 0x7f32140ff450 2023-11-16T16:37:09.946 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.941+0000 7f32137fe700 1 --2- 172.21.15.1:0/4028287245 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32140ff450 0x7f3214103200 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f31fc00e970 tx=0x7f31fc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:09.946 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.941+0000 7f3210ff9700 1 -- 172.21.15.1:0/4028287245 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc009790 con 0x7f32140ff450 2023-11-16T16:37:09.946 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.941+0000 7f3219dc8700 1 -- 172.21.15.1:0/4028287245 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f321410ac20 con 0x7f32140ff450 2023-11-16T16:37:09.946 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.941+0000 7f3210ff9700 1 -- 172.21.15.1:0/4028287245 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31fc00eec0 con 0x7f32140ff450 2023-11-16T16:37:09.946 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.941+0000 7f3210ff9700 1 -- 172.21.15.1:0/4028287245 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc003cf0 con 0x7f32140ff450 2023-11-16T16:37:09.947 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.941+0000 7f3219dc8700 1 -- 172.21.15.1:0/4028287245 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f321410aa90 con 0x7f32140ff450 2023-11-16T16:37:09.947 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.942+0000 7f3219dc8700 1 -- 172.21.15.1:0/4028287245 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32141a4d20 con 0x7f32140ff450 2023-11-16T16:37:09.947 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.943+0000 7f3210ff9700 1 -- 172.21.15.1:0/4028287245 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f31fc00ccd0 con 0x7f32140ff450 2023-11-16T16:37:09.947 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.944+0000 7f3210ff9700 1 --2- 172.21.15.1:0/4028287245 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f32000a1150 0x7f32000a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:09.948 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.944+0000 7f3210ff9700 1 -- 172.21.15.1:0/4028287245 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f31fc014070 con 0x7f32140ff450 2023-11-16T16:37:09.948 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.944+0000 7f3212ffd700 1 --2- 172.21.15.1:0/4028287245 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f32000a1150 0x7f32000a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:09.948 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.945+0000 7f3212ffd700 1 --2- 172.21.15.1:0/4028287245 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f32000a1150 0x7f32000a3600 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3204009c60 tx=0x7f3204009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:09.949 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:09.947+0000 7f3210ff9700 1 -- 172.21.15.1:0/4028287245 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f31fc096430 con 0x7f32140ff450 2023-11-16T16:37:10.213 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.211+0000 7f3219dc8700 1 -- 172.21.15.1:0/4028287245 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32141a52d0 con 0x7f32140ff450 2023-11-16T16:37:10.214 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.212+0000 7f3210ff9700 1 -- 172.21.15.1:0/4028287245 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9571 (secure 0 0 0) 0x7f31fc099a50 con 0x7f32140ff450 2023-11-16T16:37:10.214 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:37:10.214 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2587563684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2587563684},{"type":"v1","addr":"172.21.15.1:6835","nonce":2587563684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/277936168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":277936168},{"type":"v1","addr":"172.21.15.1:6837","nonce":277936168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/1172575045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1172575045},{"type":"v1","addr":"172.21.15.1:6839","nonce":1172575045}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/2631175899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2631175899},{"type":"v1","addr":"172.21.15.1:6841","nonce":2631175899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3693905267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3693905267},{"type":"v1","addr":"172.21.15.53:6833","nonce":3693905267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/164668033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":164668033},{"type":"v1","addr":"172.21.15.26:6833","nonce":164668033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6835/2140542192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2140542192},{"type":"v1","addr":"172.21.15.53:6835","nonce":2140542192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1010760275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1010760275},{"type":"v1","addr":"172.21.15.26:6835","nonce":1010760275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6837/3197060302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3197060302},{"type":"v1","addr":"172.21.15.53:6837","nonce":3197060302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/1453504863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1453504863},{"type":"v1","addr":"172.21.15.26:6837","nonce":1453504863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6839/1044753024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1044753024},{"type":"v1","addr":"172.21.15.53:6839","nonce":1044753024}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24521,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/2192205949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2192205949},{"type":"v1","addr":"172.21.15.26:6839","nonce":2192205949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2023-11-16T16:37:10.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.214+0000 7f3219dc8700 1 -- 172.21.15.1:0/4028287245 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f32000a1150 msgr2=0x7f32000a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:10.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.214+0000 7f3219dc8700 1 --2- 172.21.15.1:0/4028287245 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f32000a1150 0x7f32000a3600 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3204009c60 tx=0x7f3204009260 comp rx=0 tx=0).stop 2023-11-16T16:37:10.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.214+0000 7f3219dc8700 1 -- 172.21.15.1:0/4028287245 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32140ff450 msgr2=0x7f3214103200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:10.217 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.214+0000 7f3219dc8700 1 --2- 172.21.15.1:0/4028287245 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32140ff450 0x7f3214103200 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f31fc00e970 tx=0x7f31fc00ec80 comp rx=0 tx=0).stop 2023-11-16T16:37:10.217 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.214+0000 7f3219dc8700 1 -- 172.21.15.1:0/4028287245 shutdown_connections 2023-11-16T16:37:10.217 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.214+0000 7f3219dc8700 1 --2- 172.21.15.1:0/4028287245 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f32000a1150 0x7f32000a3600 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:10.217 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.214+0000 7f3219dc8700 1 --2- 172.21.15.1:0/4028287245 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3214100800 0x7f3214101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:10.217 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.214+0000 7f3219dc8700 1 --2- 172.21.15.1:0/4028287245 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32140ff450 0x7f3214103200 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:10.217 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.214+0000 7f3219dc8700 1 --2- 172.21.15.1:0/4028287245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32140ffe00 0x7f3214101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:10.217 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.214+0000 7f3219dc8700 1 -- 172.21.15.1:0/4028287245 >> 172.21.15.1:0/4028287245 conn(0x7f32140f9f70 msgr2=0x7f32140fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:10.217 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.214+0000 7f3219dc8700 1 -- 172.21.15.1:0/4028287245 shutdown_connections 2023-11-16T16:37:10.217 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.214+0000 7f3219dc8700 1 -- 172.21.15.1:0/4028287245 wait complete. 2023-11-16T16:37:10.217 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 13 2023-11-16T16:37:10.229 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-16T16:37:10.361 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.359+0000 7f1a7605a700 1 -- 172.21.15.1:0/3493184222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a70100310 msgr2=0x7f1a70100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:10.361 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.359+0000 7f1a7605a700 1 --2- 172.21.15.1:0/3493184222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a70100310 0x7f1a70100720 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1a60009990 tx=0x7f1a60009ca0 comp rx=0 tx=0).stop 2023-11-16T16:37:10.361 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.359+0000 7f1a7605a700 1 -- 172.21.15.1:0/3493184222 shutdown_connections 2023-11-16T16:37:10.361 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.359+0000 7f1a7605a700 1 --2- 172.21.15.1:0/3493184222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a70101e90 0x7f1a70108750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:10.361 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.359+0000 7f1a7605a700 1 --2- 172.21.15.1:0/3493184222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a70100310 0x7f1a70100720 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:10.361 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.359+0000 7f1a7605a700 1 --2- 172.21.15.1:0/3493184222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a70101500 0x7f1a70101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:10.361 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.359+0000 7f1a7605a700 1 -- 172.21.15.1:0/3493184222 >> 172.21.15.1:0/3493184222 conn(0x7f1a700fb930 msgr2=0x7f1a700fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:10.362 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.359+0000 7f1a7605a700 1 -- 172.21.15.1:0/3493184222 shutdown_connections 2023-11-16T16:37:10.362 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.359+0000 7f1a7605a700 1 -- 172.21.15.1:0/3493184222 wait complete. 2023-11-16T16:37:10.364 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.360+0000 7f1a7605a700 1 Processor -- start 2023-11-16T16:37:10.364 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.360+0000 7f1a7605a700 1 -- start start 2023-11-16T16:37:10.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.360+0000 7f1a7605a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a70100310 0x7f1a70196050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:10.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.360+0000 7f1a7605a700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a70101500 0x7f1a70196590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:10.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.360+0000 7f1a7605a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a70101e90 0x7f1a7019d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:10.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.360+0000 7f1a7605a700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7019dbd0 con 0x7f1a70101500 2023-11-16T16:37:10.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.360+0000 7f1a7605a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7019dd10 con 0x7f1a70100310 2023-11-16T16:37:10.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.360+0000 7f1a7605a700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7019dea0 con 0x7f1a70101e90 2023-11-16T16:37:10.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.360+0000 7f1a6effd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a70101500 0x7f1a70196590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:10.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.360+0000 7f1a6effd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a70101500 0x7f1a70196590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41348/0 (socket says 172.21.15.1:41348) 2023-11-16T16:37:10.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.360+0000 7f1a6effd700 1 -- 172.21.15.1:0/990486725 learned_addr learned my addr 172.21.15.1:0/990486725 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:10.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.360+0000 7f1a6f7fe700 1 --2- 172.21.15.1:0/990486725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a70100310 0x7f1a70196050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:10.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.360+0000 7f1a6ffff700 1 --2- 172.21.15.1:0/990486725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a70101e90 0x7f1a7019d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:10.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.361+0000 7f1a6effd700 1 -- 172.21.15.1:0/990486725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a70101e90 msgr2=0x7f1a7019d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:10.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.361+0000 7f1a6effd700 1 --2- 172.21.15.1:0/990486725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a70101e90 0x7f1a7019d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:10.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.361+0000 7f1a6effd700 1 -- 172.21.15.1:0/990486725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a70100310 msgr2=0x7f1a70196050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:10.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.361+0000 7f1a6effd700 1 --2- 172.21.15.1:0/990486725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a70100310 0x7f1a70196050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:10.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.361+0000 7f1a6effd700 1 -- 172.21.15.1:0/990486725 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a600095f0 con 0x7f1a70101500 2023-11-16T16:37:10.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.361+0000 7f1a6effd700 1 --2- 172.21.15.1:0/990486725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a70101500 0x7f1a70196590 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1a6400d950 tx=0x7f1a6400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:10.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.361+0000 7f1a6cff9700 1 -- 172.21.15.1:0/990486725 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6400f890 con 0x7f1a70101500 2023-11-16T16:37:10.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.361+0000 7f1a6cff9700 1 -- 172.21.15.1:0/990486725 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a64010460 con 0x7f1a70101500 2023-11-16T16:37:10.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.361+0000 7f1a7605a700 1 -- 172.21.15.1:0/990486725 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a7010bc90 con 0x7f1a70101500 2023-11-16T16:37:10.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.361+0000 7f1a6cff9700 1 -- 172.21.15.1:0/990486725 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6400e610 con 0x7f1a70101500 2023-11-16T16:37:10.367 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.361+0000 7f1a7605a700 1 -- 172.21.15.1:0/990486725 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a7010bb00 con 0x7f1a70101500 2023-11-16T16:37:10.367 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.362+0000 7f1a7605a700 1 -- 172.21.15.1:0/990486725 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a701078b0 con 0x7f1a70101500 2023-11-16T16:37:10.367 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.363+0000 7f1a6cff9700 1 -- 172.21.15.1:0/990486725 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f1a640108d0 con 0x7f1a70101500 2023-11-16T16:37:10.367 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.364+0000 7f1a6cff9700 1 --2- 172.21.15.1:0/990486725 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f1a580a1150 0x7f1a580a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:10.368 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.364+0000 7f1a6f7fe700 1 --2- 172.21.15.1:0/990486725 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f1a580a1150 0x7f1a580a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:10.368 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.364+0000 7f1a6cff9700 1 -- 172.21.15.1:0/990486725 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1a640cc2f0 con 0x7f1a70101500 2023-11-16T16:37:10.368 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.364+0000 7f1a6f7fe700 1 --2- 172.21.15.1:0/990486725 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f1a580a1150 0x7f1a580a3600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1a60006010 tx=0x7f1a60005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:10.370 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.368+0000 7f1a6cff9700 1 -- 172.21.15.1:0/990486725 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1a64095e40 con 0x7f1a70101500 2023-11-16T16:37:10.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:10 smithi026 ceph-mon[100112]: mds.? [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] up:boot 2023-11-16T16:37:10.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:10 smithi026 ceph-mon[100112]: fsmap 12 up:standby 2023-11-16T16:37:10.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:10 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-16T16:37:10.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:10 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:10.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:10 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:10.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:10 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:10.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:10 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:10.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:10 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:10.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:10 smithi026 ceph-mon[100112]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:37:10.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:10 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:37:10.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:10 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/4028287245' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:37:10.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:10 smithi053 ceph-mon[100638]: mds.? [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] up:boot 2023-11-16T16:37:10.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:10 smithi053 ceph-mon[100638]: fsmap 12 up:standby 2023-11-16T16:37:10.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:10 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-16T16:37:10.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:10 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:10.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:10 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:10.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:10 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:10.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:10 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:10.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:10 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:10.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:10 smithi053 ceph-mon[100638]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:37:10.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:10 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:37:10.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:10 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/4028287245' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:37:10.597 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.595+0000 7f1a7605a700 1 -- 172.21.15.1:0/990486725 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a70061850 con 0x7f1a70101500 2023-11-16T16:37:10.598 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.596+0000 7f1a6cff9700 1 -- 172.21.15.1:0/990486725 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18291 (secure 0 0 0) 0x7f1a64099460 con 0x7f1a70101500 2023-11-16T16:37:10.599 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:37:10.599 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":80,"fsid":"35d9b010-849d-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:30:06.217778+0000","modified":"2023-11-16T16:36:32.736561+0000","last_up_change":"2023-11-16T16:36:31.735403+0000","last_in_change":"2023-11-16T16:36:14.083021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:33:15.916723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a8480507-5275-4e52-953b-626110beb436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6803","nonce":2469671609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6805","nonce":2469671609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6809","nonce":2469671609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6807","nonce":2469671609}]},"public_addr":"172.21.15.1:6803/2469671609","cluster_addr":"172.21.15.1:6805/2469671609","heartbeat_back_addr":"172.21.15.1:6809/2469671609","heartbeat_front_addr":"172.21.15.1:6807/2469671609","state":["exists","up"]},{"osd":1,"uuid":"1d011db2-99d1-4575-bdc5-51f88a308fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6801","nonce":247994796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6803","nonce":247994796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6807","nonce":247994796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6805","nonce":247994796}]},"public_addr":"172.21.15.26:6801/247994796","cluster_addr":"172.21.15.26:6803/247994796","heartbeat_back_addr":"172.21.15.26:6807/247994796","heartbeat_front_addr":"172.21.15.26:6805/247994796","state":["exists","up"]},{"osd":2,"uuid":"4d7a2bd1-3c8e-4546-a243-f0354473b031","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6801","nonce":1659316290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6803","nonce":1659316290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6807","nonce":1659316290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6805","nonce":1659316290}]},"public_addr":"172.21.15.53:6801/1659316290","cluster_addr":"172.21.15.53:6803/1659316290","heartbeat_back_addr":"172.21.15.53:6807/1659316290","heartbeat_front_addr":"172.21.15.53:6805/1659316290","state":["exists","up"]},{"osd":3,"uuid":"9c290fd9-e121-4a81-9687-b0eb93debd86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6811","nonce":2728987281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6813","nonce":2728987281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6817","nonce":2728987281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6815","nonce":2728987281}]},"public_addr":"172.21.15.1:6811/2728987281","cluster_addr":"172.21.15.1:6813/2728987281","heartbeat_back_addr":"172.21.15.1:6817/2728987281","heartbeat_front_addr":"172.21.15.1:6815/2728987281","state":["exists","up"]},{"osd":4,"uuid":"533226bc-cc57-4f15-9e29-820294d57e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6809","nonce":2954740024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6811","nonce":2954740024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6815","nonce":2954740024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6813","nonce":2954740024}]},"public_addr":"172.21.15.26:6809/2954740024","cluster_addr":"172.21.15.26:6811/2954740024","heartbeat_back_addr":"172.21.15.26:6815/2954740024","heartbeat_front_addr":"172.21.15.26:6813/2954740024","state":["exists","up"]},{"osd":5,"uuid":"d3ab25eb-e1d4-4557-bc89-efc9072401ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6809","nonce":3966666318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6811","nonce":3966666318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6815","nonce":3966666318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6813","nonce":3966666318}]},"public_addr":"172.21.15.53:6809/3966666318","cluster_addr":"172.21.15.53:6811/3966666318","heartbeat_back_addr":"172.21.15.53:6815/3966666318","heartbeat_front_addr":"172.21.15.53:6813/3966666318","state":["exists","up"]},{"osd":6,"uuid":"180f506e-afb8-4043-b0b4-2dc86f6d4bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6819","nonce":3364032646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6821","nonce":3364032646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6825","nonce":3364032646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6823","nonce":3364032646}]},"public_addr":"172.21.15.1:6819/3364032646","cluster_addr":"172.21.15.1:6821/3364032646","heartbeat_back_addr":"172.21.15.1:6825/3364032646","heartbeat_front_addr":"172.21.15.1:6823/3364032646","state":["exists","up"]},{"osd":7,"uuid":"0d837bac-194f-498d-b7fb-b3147e992329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6817","nonce":3766901391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6819","nonce":3766901391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6823","nonce":3766901391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6821","nonce":3766901391}]},"public_addr":"172.21.15.26:6817/3766901391","cluster_addr":"172.21.15.26:6819/3766901391","heartbeat_back_addr":"172.21.15.26:6823/3766901391","heartbeat_front_addr":"172.21.15.26:6821/3766901391","state":["exists","up"]},{"osd":8,"uuid":"933f231a-7454-47b0-9291-f1fa64f0baec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6817","nonce":3224335871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6819","nonce":3224335871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6823","nonce":3224335871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6821","nonce":3224335871}]},"public_addr":"172.21.15.53:6817/3224335871","cluster_addr":"172.21.15.53:6819/3224335871","heartbeat_back_addr":"172.21.15.53:6823/3224335871","heartbeat_front_addr":"172.21.15.53:6821/3224335871","state":["exists","up"]},{"osd":9,"uuid":"245fcbb3-c651-4703-ab80-309dc98bd983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6827","nonce":1391199844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6829","nonce":1391199844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6833","nonce":1391199844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6831","nonce":1391199844}]},"public_addr":"172.21.15.1:6827/1391199844","cluster_addr":"172.21.15.1:6829/1391199844","heartbeat_back_addr":"172.21.15.1:6833/1391199844","heartbeat_front_addr":"172.21.15.1:6831/1391199844","state":["exists","up"]},{"osd":10,"uuid":"2c090e7a-5240-4d52-ae4f-e94fe05383d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6825","nonce":1773545447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6827","nonce":1773545447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6831","nonce":1773545447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6829","nonce":1773545447}]},"public_addr":"172.21.15.26:6825/1773545447","cluster_addr":"172.21.15.26:6827/1773545447","heartbeat_back_addr":"172.21.15.26:6831/1773545447","heartbeat_front_addr":"172.21.15.26:6829/1773545447","state":["exists","up"]},{"osd":11,"uuid":"4511e32d-e469-45e1-8310-0ff400cb7b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6825","nonce":2564669546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6827","nonce":2564669546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6831","nonce":2564669546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6829","nonce":2564669546}]},"public_addr":"172.21.15.53:6825/2564669546","cluster_addr":"172.21.15.53:6827/2564669546","heartbeat_back_addr":"172.21.15.53:6831/2564669546","heartbeat_front_addr":"172.21.15.53:6829/2564669546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:28.140250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:49.885461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:12.521487+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:34.926197+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:56.815594+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:18.387751+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:40.587106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:02.319370+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:23.832298+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:46.338040+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:08.730414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:30.050070+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1494336948":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6801/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6800/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6800/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/1641976669":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/2895923638":"2023-11-17T16:30:35.064289+0000","172.21.15.1:6801/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/78811711":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/4131957708":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/344403698":"2023-11-17T16:30:35.064289+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:37:10.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.599+0000 7f1a7605a700 1 -- 172.21.15.1:0/990486725 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f1a580a1150 msgr2=0x7f1a580a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:10.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.599+0000 7f1a7605a700 1 --2- 172.21.15.1:0/990486725 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f1a580a1150 0x7f1a580a3600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1a60006010 tx=0x7f1a60005960 comp rx=0 tx=0).stop 2023-11-16T16:37:10.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.599+0000 7f1a7605a700 1 -- 172.21.15.1:0/990486725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a70101500 msgr2=0x7f1a70196590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:10.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.599+0000 7f1a7605a700 1 --2- 172.21.15.1:0/990486725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a70101500 0x7f1a70196590 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1a6400d950 tx=0x7f1a6400dc60 comp rx=0 tx=0).stop 2023-11-16T16:37:10.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.599+0000 7f1a7605a700 1 -- 172.21.15.1:0/990486725 shutdown_connections 2023-11-16T16:37:10.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.599+0000 7f1a7605a700 1 --2- 172.21.15.1:0/990486725 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f1a580a1150 0x7f1a580a3600 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:10.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.599+0000 7f1a7605a700 1 --2- 172.21.15.1:0/990486725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1a70101e90 0x7f1a7019d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:10.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.599+0000 7f1a7605a700 1 --2- 172.21.15.1:0/990486725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a70100310 0x7f1a70196050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:10.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.599+0000 7f1a7605a700 1 --2- 172.21.15.1:0/990486725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a70101500 0x7f1a70196590 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:10.602 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.599+0000 7f1a7605a700 1 -- 172.21.15.1:0/990486725 >> 172.21.15.1:0/990486725 conn(0x7f1a700fb930 msgr2=0x7f1a701049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:10.602 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.599+0000 7f1a7605a700 1 -- 172.21.15.1:0/990486725 shutdown_connections 2023-11-16T16:37:10.602 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.599+0000 7f1a7605a700 1 -- 172.21.15.1:0/990486725 wait complete. 2023-11-16T16:37:10.614 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:10 smithi001 ceph-mon[98800]: mds.? [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] up:boot 2023-11-16T16:37:10.614 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:10 smithi001 ceph-mon[98800]: fsmap 12 up:standby 2023-11-16T16:37:10.614 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:10 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-16T16:37:10.614 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:10 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:10.614 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:10 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:10.614 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:10 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:10.614 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:10 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:10.615 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:10 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:10.615 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:10 smithi001 ceph-mon[98800]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:37:10.615 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:10 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:37:10.615 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:10 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/4028287245' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:37:10.616 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-16T16:33:15.916723+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-16T16:37:10.616 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-16T16:37:10.755 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.752+0000 7fea7f21d700 1 -- 172.21.15.1:0/3670687399 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea78100270 msgr2=0x7fea781006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:10.755 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.752+0000 7fea7f21d700 1 --2- 172.21.15.1:0/3670687399 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea78100270 0x7fea781006e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fea68009990 tx=0x7fea68009ca0 comp rx=0 tx=0).stop 2023-11-16T16:37:10.755 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.752+0000 7fea7f21d700 1 -- 172.21.15.1:0/3670687399 shutdown_connections 2023-11-16T16:37:10.755 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.752+0000 7fea7f21d700 1 --2- 172.21.15.1:0/3670687399 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea78100c20 0x7fea781074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:10.755 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.752+0000 7fea7f21d700 1 --2- 172.21.15.1:0/3670687399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea780fefe0 0x7fea780ff3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:10.755 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.752+0000 7fea7f21d700 1 --2- 172.21.15.1:0/3670687399 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea78100270 0x7fea781006e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:10.755 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.752+0000 7fea7f21d700 1 -- 172.21.15.1:0/3670687399 >> 172.21.15.1:0/3670687399 conn(0x7fea780fa600 msgr2=0x7fea780fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:10.755 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.752+0000 7fea7f21d700 1 -- 172.21.15.1:0/3670687399 shutdown_connections 2023-11-16T16:37:10.756 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.752+0000 7fea7f21d700 1 -- 172.21.15.1:0/3670687399 wait complete. 2023-11-16T16:37:10.756 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.753+0000 7fea7f21d700 1 Processor -- start 2023-11-16T16:37:10.756 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.753+0000 7fea7f21d700 1 -- start start 2023-11-16T16:37:10.757 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.753+0000 7fea7f21d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea780fefe0 0x7fea78195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:10.757 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.753+0000 7fea7f21d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea78100270 0x7fea781962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:10.757 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.753+0000 7fea7f21d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea78100c20 0x7fea7819d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:10.757 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.753+0000 7fea7f21d700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7819d8f0 con 0x7fea780fefe0 2023-11-16T16:37:10.757 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.753+0000 7fea7f21d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7819da30 con 0x7fea78100c20 2023-11-16T16:37:10.758 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.753+0000 7fea7f21d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7819dbc0 con 0x7fea78100270 2023-11-16T16:37:10.758 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.753+0000 7fea7e21b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea780fefe0 0x7fea78195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:10.758 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.753+0000 7fea7e21b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea780fefe0 0x7fea78195d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41360/0 (socket says 172.21.15.1:41360) 2023-11-16T16:37:10.758 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.753+0000 7fea7e21b700 1 -- 172.21.15.1:0/520958629 learned_addr learned my addr 172.21.15.1:0/520958629 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:10.758 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.753+0000 7fea7da1a700 1 --2- 172.21.15.1:0/520958629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea78100270 0x7fea781962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:10.758 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.753+0000 7fea7ea1c700 1 --2- 172.21.15.1:0/520958629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea78100c20 0x7fea7819d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:10.758 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.754+0000 7fea7e21b700 1 -- 172.21.15.1:0/520958629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea78100270 msgr2=0x7fea781962b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:10.758 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.754+0000 7fea7e21b700 1 --2- 172.21.15.1:0/520958629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea78100270 0x7fea781962b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:10.758 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.754+0000 7fea7e21b700 1 -- 172.21.15.1:0/520958629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea78100c20 msgr2=0x7fea7819d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:10.758 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.754+0000 7fea7e21b700 1 --2- 172.21.15.1:0/520958629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea78100c20 0x7fea7819d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:10.759 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.754+0000 7fea7e21b700 1 -- 172.21.15.1:0/520958629 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea680095f0 con 0x7fea780fefe0 2023-11-16T16:37:10.759 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.754+0000 7fea7e21b700 1 --2- 172.21.15.1:0/520958629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea780fefe0 0x7fea78195d70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fea7400d980 tx=0x7fea7400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:10.759 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.754+0000 7fea6f7fe700 1 -- 172.21.15.1:0/520958629 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea74004220 con 0x7fea780fefe0 2023-11-16T16:37:10.759 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.754+0000 7fea7f21d700 1 -- 172.21.15.1:0/520958629 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea78069a10 con 0x7fea780fefe0 2023-11-16T16:37:10.759 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.754+0000 7fea6f7fe700 1 -- 172.21.15.1:0/520958629 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea74007640 con 0x7fea780fefe0 2023-11-16T16:37:10.759 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.754+0000 7fea6f7fe700 1 -- 172.21.15.1:0/520958629 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea74005690 con 0x7fea780fefe0 2023-11-16T16:37:10.759 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.754+0000 7fea7f21d700 1 -- 172.21.15.1:0/520958629 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea78069880 con 0x7fea780fefe0 2023-11-16T16:37:10.759 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.755+0000 7fea7f21d700 1 -- 172.21.15.1:0/520958629 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea78106520 con 0x7fea780fefe0 2023-11-16T16:37:10.760 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.756+0000 7fea6f7fe700 1 -- 172.21.15.1:0/520958629 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fea74004750 con 0x7fea780fefe0 2023-11-16T16:37:10.760 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.757+0000 7fea6f7fe700 1 --2- 172.21.15.1:0/520958629 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fea640a11f0 0x7fea640a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:10.760 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.757+0000 7fea6f7fe700 1 -- 172.21.15.1:0/520958629 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fea740cc780 con 0x7fea780fefe0 2023-11-16T16:37:10.760 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.757+0000 7fea7da1a700 1 --2- 172.21.15.1:0/520958629 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fea640a11f0 0x7fea640a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:10.760 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.757+0000 7fea7da1a700 1 --2- 172.21.15.1:0/520958629 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fea640a11f0 0x7fea640a36a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fea6800b5c0 tx=0x7fea68005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:10.762 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.761+0000 7fea6f7fe700 1 -- 172.21.15.1:0/520958629 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fea74096370 con 0x7fea780fefe0 2023-11-16T16:37:10.980 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.978+0000 7fea7f21d700 1 -- 172.21.15.1:0/520958629 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fea78061930 con 0x7fea780fefe0 2023-11-16T16:37:10.980 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.979+0000 7fea6f7fe700 1 -- 172.21.15.1:0/520958629 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7fea74099990 con 0x7fea780fefe0 2023-11-16T16:37:10.981 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2023-11-16T16:37:10.983 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.981+0000 7fea7f21d700 1 -- 172.21.15.1:0/520958629 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fea640a11f0 msgr2=0x7fea640a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:10.983 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.981+0000 7fea7f21d700 1 --2- 172.21.15.1:0/520958629 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fea640a11f0 0x7fea640a36a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fea6800b5c0 tx=0x7fea68005fb0 comp rx=0 tx=0).stop 2023-11-16T16:37:10.983 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.981+0000 7fea7f21d700 1 -- 172.21.15.1:0/520958629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea780fefe0 msgr2=0x7fea78195d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:10.983 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.981+0000 7fea7f21d700 1 --2- 172.21.15.1:0/520958629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea780fefe0 0x7fea78195d70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fea7400d980 tx=0x7fea7400dc90 comp rx=0 tx=0).stop 2023-11-16T16:37:10.983 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.981+0000 7fea7f21d700 1 -- 172.21.15.1:0/520958629 shutdown_connections 2023-11-16T16:37:10.983 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.981+0000 7fea7f21d700 1 --2- 172.21.15.1:0/520958629 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fea640a11f0 0x7fea640a36a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:10.983 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.981+0000 7fea7f21d700 1 --2- 172.21.15.1:0/520958629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea78100c20 0x7fea7819d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:10.983 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.981+0000 7fea7f21d700 1 --2- 172.21.15.1:0/520958629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea780fefe0 0x7fea78195d70 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:10.984 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.981+0000 7fea7f21d700 1 --2- 172.21.15.1:0/520958629 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea78100270 0x7fea781962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:10.984 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.981+0000 7fea7f21d700 1 -- 172.21.15.1:0/520958629 >> 172.21.15.1:0/520958629 conn(0x7fea780fa600 msgr2=0x7fea78103660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:10.984 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.982+0000 7fea7f21d700 1 -- 172.21.15.1:0/520958629 shutdown_connections 2023-11-16T16:37:10.984 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:10.982+0000 7fea7f21d700 1 -- 172.21.15.1:0/520958629 wait complete. 2023-11-16T16:37:10.998 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-16T16:37:11.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.128+0000 7f3525f9f700 1 -- 172.21.15.1:0/3975872652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3520100d40 msgr2=0x7f3520107600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:11.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.128+0000 7f3525f9f700 1 --2- 172.21.15.1:0/3975872652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3520100d40 0x7f3520107600 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f351800b0a0 tx=0x7f351800b3b0 comp rx=0 tx=0).stop 2023-11-16T16:37:11.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.128+0000 7f3525f9f700 1 -- 172.21.15.1:0/3975872652 shutdown_connections 2023-11-16T16:37:11.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.128+0000 7f3525f9f700 1 --2- 172.21.15.1:0/3975872652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3520100d40 0x7f3520107600 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:11.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.128+0000 7f3525f9f700 1 --2- 172.21.15.1:0/3975872652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35200ff1c0 0x7f35200ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:11.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.128+0000 7f3525f9f700 1 --2- 172.21.15.1:0/3975872652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35201003b0 0x7f3520100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:11.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.128+0000 7f3525f9f700 1 -- 172.21.15.1:0/3975872652 >> 172.21.15.1:0/3975872652 conn(0x7f35200fa7a0 msgr2=0x7f35200fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:11.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.128+0000 7f3525f9f700 1 -- 172.21.15.1:0/3975872652 shutdown_connections 2023-11-16T16:37:11.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.128+0000 7f3525f9f700 1 -- 172.21.15.1:0/3975872652 wait complete. 2023-11-16T16:37:11.133 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.129+0000 7f3525f9f700 1 Processor -- start 2023-11-16T16:37:11.133 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.129+0000 7f3525f9f700 1 -- start start 2023-11-16T16:37:11.133 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.129+0000 7f3525f9f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35200ff1c0 0x7f3520196150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:11.133 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.129+0000 7f3525f9f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35201003b0 0x7f3520196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:11.133 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.129+0000 7f3525f9f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3520100d40 0x7f352019d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:11.133 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.129+0000 7f3525f9f700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352019dcd0 con 0x7f35201003b0 2023-11-16T16:37:11.133 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.129+0000 7f3525f9f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352019de10 con 0x7f35200ff1c0 2023-11-16T16:37:11.133 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.129+0000 7f3525f9f700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352019dfa0 con 0x7f3520100d40 2023-11-16T16:37:11.133 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.129+0000 7f351effd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35201003b0 0x7f3520196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:11.134 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.129+0000 7f351f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35200ff1c0 0x7f3520196150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:11.134 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.129+0000 7f351effd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35201003b0 0x7f3520196690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41374/0 (socket says 172.21.15.1:41374) 2023-11-16T16:37:11.134 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.129+0000 7f351f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35200ff1c0 0x7f3520196150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.1:35560/0 (socket says 172.21.15.1:35560) 2023-11-16T16:37:11.134 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.129+0000 7f351effd700 1 -- 172.21.15.1:0/1952540826 learned_addr learned my addr 172.21.15.1:0/1952540826 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:11.134 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.129+0000 7f351ffff700 1 --2- 172.21.15.1:0/1952540826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3520100d40 0x7f352019d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:11.134 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.130+0000 7f351f7fe700 1 -- 172.21.15.1:0/1952540826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3520100d40 msgr2=0x7f352019d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:11.134 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.130+0000 7f351f7fe700 1 --2- 172.21.15.1:0/1952540826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3520100d40 0x7f352019d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:11.134 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.130+0000 7f351f7fe700 1 -- 172.21.15.1:0/1952540826 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35201003b0 msgr2=0x7f3520196690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:11.134 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.130+0000 7f351f7fe700 1 --2- 172.21.15.1:0/1952540826 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35201003b0 0x7f3520196690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:11.134 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.130+0000 7f351f7fe700 1 -- 172.21.15.1:0/1952540826 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3518009d00 con 0x7f35200ff1c0 2023-11-16T16:37:11.134 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.130+0000 7f351f7fe700 1 --2- 172.21.15.1:0/1952540826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35200ff1c0 0x7f3520196150 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f351000e9c0 tx=0x7f351000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:11.134 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.130+0000 7f351cff9700 1 -- 172.21.15.1:0/1952540826 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3510004220 con 0x7f35200ff1c0 2023-11-16T16:37:11.135 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.130+0000 7f351cff9700 1 -- 172.21.15.1:0/1952540826 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3510004380 con 0x7f35200ff1c0 2023-11-16T16:37:11.135 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.130+0000 7f3525f9f700 1 -- 172.21.15.1:0/1952540826 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f352010ab80 con 0x7f35200ff1c0 2023-11-16T16:37:11.135 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.130+0000 7f351cff9700 1 -- 172.21.15.1:0/1952540826 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3510003be0 con 0x7f35200ff1c0 2023-11-16T16:37:11.135 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.130+0000 7f3525f9f700 1 -- 172.21.15.1:0/1952540826 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f352010a9f0 con 0x7f35200ff1c0 2023-11-16T16:37:11.135 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.131+0000 7f3525f9f700 1 -- 172.21.15.1:0/1952540826 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f352019e740 con 0x7f35200ff1c0 2023-11-16T16:37:11.135 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.133+0000 7f351cff9700 1 -- 172.21.15.1:0/1952540826 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f3510003d40 con 0x7f35200ff1c0 2023-11-16T16:37:11.136 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.133+0000 7f351cff9700 1 --2- 172.21.15.1:0/1952540826 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f35080a11d0 0x7f35080a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:11.136 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.133+0000 7f351effd700 1 --2- 172.21.15.1:0/1952540826 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f35080a11d0 0x7f35080a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:11.136 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.133+0000 7f351cff9700 1 -- 172.21.15.1:0/1952540826 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3510014070 con 0x7f35200ff1c0 2023-11-16T16:37:11.136 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.134+0000 7f351effd700 1 --2- 172.21.15.1:0/1952540826 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f35080a11d0 0x7f35080a3680 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3514009d20 tx=0x7f3514009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:11.138 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.136+0000 7f351cff9700 1 -- 172.21.15.1:0/1952540826 <== mon.2 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3510096880 con 0x7f35200ff1c0 2023-11-16T16:37:11.354 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.353+0000 7f3525f9f700 1 -- 172.21.15.1:0/1952540826 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f352019ecf0 con 0x7f35200ff1c0 2023-11-16T16:37:11.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.353+0000 7f351cff9700 1 -- 172.21.15.1:0/1952540826 <== mon.2 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v80) v1 ==== 84+0+65 (secure 0 0 0) 0x7f351001f340 con 0x7f35200ff1c0 2023-11-16T16:37:11.355 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:37:11.355 INFO:teuthology.orchestra.run.smithi001.stdout:[ 2023-11-16T16:37:11.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:37:11.356 INFO:teuthology.orchestra.run.smithi001.stdout: "poolnum": 1, 2023-11-16T16:37:11.356 INFO:teuthology.orchestra.run.smithi001.stdout: "poolname": ".mgr" 2023-11-16T16:37:11.356 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:37:11.356 INFO:teuthology.orchestra.run.smithi001.stdout:] 2023-11-16T16:37:11.358 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.356+0000 7f3525f9f700 1 -- 172.21.15.1:0/1952540826 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f35080a11d0 msgr2=0x7f35080a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:11.358 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.356+0000 7f3525f9f700 1 --2- 172.21.15.1:0/1952540826 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f35080a11d0 0x7f35080a3680 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3514009d20 tx=0x7f3514009260 comp rx=0 tx=0).stop 2023-11-16T16:37:11.358 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.356+0000 7f3525f9f700 1 -- 172.21.15.1:0/1952540826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35200ff1c0 msgr2=0x7f3520196150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:11.358 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.356+0000 7f3525f9f700 1 --2- 172.21.15.1:0/1952540826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35200ff1c0 0x7f3520196150 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f351000e9c0 tx=0x7f351000ecd0 comp rx=0 tx=0).stop 2023-11-16T16:37:11.358 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.356+0000 7f3525f9f700 1 -- 172.21.15.1:0/1952540826 shutdown_connections 2023-11-16T16:37:11.358 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.356+0000 7f3525f9f700 1 --2- 172.21.15.1:0/1952540826 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f35080a11d0 0x7f35080a3680 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:11.358 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.356+0000 7f3525f9f700 1 --2- 172.21.15.1:0/1952540826 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3520100d40 0x7f352019d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:11.358 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.356+0000 7f3525f9f700 1 --2- 172.21.15.1:0/1952540826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35200ff1c0 0x7f3520196150 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:11.358 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.356+0000 7f3525f9f700 1 --2- 172.21.15.1:0/1952540826 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35201003b0 0x7f3520196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:11.358 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.356+0000 7f3525f9f700 1 -- 172.21.15.1:0/1952540826 >> 172.21.15.1:0/1952540826 conn(0x7f35200fa7a0 msgr2=0x7f3520069160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:11.359 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.356+0000 7f3525f9f700 1 -- 172.21.15.1:0/1952540826 shutdown_connections 2023-11-16T16:37:11.359 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.356+0000 7f3525f9f700 1 -- 172.21.15.1:0/1952540826 wait complete. 2023-11-16T16:37:11.370 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-16T16:37:11.370 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-16T16:37:11.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.503+0000 7f1c52353700 1 -- 172.21.15.1:0/503429137 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c4c100290 msgr2=0x7f1c4c1006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:11.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.503+0000 7f1c52353700 1 --2- 172.21.15.1:0/503429137 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c4c100290 0x7f1c4c1006a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c009990 tx=0x7f1c3c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:37:11.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.504+0000 7f1c52353700 1 -- 172.21.15.1:0/503429137 shutdown_connections 2023-11-16T16:37:11.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.504+0000 7f1c52353700 1 --2- 172.21.15.1:0/503429137 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c4c101e10 0x7f1c4c1086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:11.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.504+0000 7f1c52353700 1 --2- 172.21.15.1:0/503429137 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c4c100290 0x7f1c4c1006a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:11.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.504+0000 7f1c52353700 1 --2- 172.21.15.1:0/503429137 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c4c101480 0x7f1c4c1018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:11.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.504+0000 7f1c52353700 1 -- 172.21.15.1:0/503429137 >> 172.21.15.1:0/503429137 conn(0x7f1c4c0fb8b0 msgr2=0x7f1c4c0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:11.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.504+0000 7f1c52353700 1 -- 172.21.15.1:0/503429137 shutdown_connections 2023-11-16T16:37:11.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.504+0000 7f1c52353700 1 -- 172.21.15.1:0/503429137 wait complete. 2023-11-16T16:37:11.507 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.504+0000 7f1c52353700 1 Processor -- start 2023-11-16T16:37:11.507 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.504+0000 7f1c52353700 1 -- start start 2023-11-16T16:37:11.508 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.504+0000 7f1c52353700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c4c100290 0x7f1c4c1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:11.509 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.505+0000 7f1c52353700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c4c101480 0x7f1c4c1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:11.509 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.505+0000 7f1c52353700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c4c101e10 0x7f1c4c19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:11.509 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.505+0000 7f1c52353700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c19fd20 con 0x7f1c4c101480 2023-11-16T16:37:11.509 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.505+0000 7f1c52353700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c19fe60 con 0x7f1c4c100290 2023-11-16T16:37:11.509 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.505+0000 7f1c52353700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c19fff0 con 0x7f1c4c101e10 2023-11-16T16:37:11.509 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.505+0000 7f1c4b7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c4c101480 0x7f1c4c1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:11.509 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.505+0000 7f1c4b7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c4c101480 0x7f1c4c1986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41390/0 (socket says 172.21.15.1:41390) 2023-11-16T16:37:11.509 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.505+0000 7f1c4b7fe700 1 -- 172.21.15.1:0/3241942687 learned_addr learned my addr 172.21.15.1:0/3241942687 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:11.509 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.505+0000 7f1c508f0700 1 --2- 172.21.15.1:0/3241942687 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c4c101e10 0x7f1c4c19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:11.509 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.505+0000 7f1c4bfff700 1 --2- 172.21.15.1:0/3241942687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c4c100290 0x7f1c4c1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:11.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.505+0000 7f1c4b7fe700 1 -- 172.21.15.1:0/3241942687 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c4c101e10 msgr2=0x7f1c4c19f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:11.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.505+0000 7f1c4b7fe700 1 --2- 172.21.15.1:0/3241942687 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c4c101e10 0x7f1c4c19f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:11.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.505+0000 7f1c4b7fe700 1 -- 172.21.15.1:0/3241942687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c4c100290 msgr2=0x7f1c4c1981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:11.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.505+0000 7f1c4b7fe700 1 --2- 172.21.15.1:0/3241942687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c4c100290 0x7f1c4c1981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:11.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.505+0000 7f1c4b7fe700 1 -- 172.21.15.1:0/3241942687 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c3c0095f0 con 0x7f1c4c101480 2023-11-16T16:37:11.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.505+0000 7f1c4bfff700 1 --2- 172.21.15.1:0/3241942687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c4c100290 0x7f1c4c1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:37:11.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.505+0000 7f1c4b7fe700 1 --2- 172.21.15.1:0/3241942687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c4c101480 0x7f1c4c1986e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1c4000e9a0 tx=0x7f1c4000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:11.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.505+0000 7f1c497fa700 1 -- 172.21.15.1:0/3241942687 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c40009790 con 0x7f1c4c101480 2023-11-16T16:37:11.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.505+0000 7f1c497fa700 1 -- 172.21.15.1:0/3241942687 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c4000cbd0 con 0x7f1c4c101480 2023-11-16T16:37:11.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.505+0000 7f1c52353700 1 -- 172.21.15.1:0/3241942687 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c4c10bbf0 con 0x7f1c4c101480 2023-11-16T16:37:11.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.505+0000 7f1c497fa700 1 -- 172.21.15.1:0/3241942687 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c40018680 con 0x7f1c4c101480 2023-11-16T16:37:11.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.505+0000 7f1c52353700 1 -- 172.21.15.1:0/3241942687 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c4c10ba60 con 0x7f1c4c101480 2023-11-16T16:37:11.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.506+0000 7f1c52353700 1 -- 172.21.15.1:0/3241942687 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c4c10bd80 con 0x7f1c4c101480 2023-11-16T16:37:11.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.507+0000 7f1c497fa700 1 -- 172.21.15.1:0/3241942687 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f1c4000cd40 con 0x7f1c4c101480 2023-11-16T16:37:11.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.508+0000 7f1c497fa700 1 --2- 172.21.15.1:0/3241942687 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f1c340a1150 0x7f1c340a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:11.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.508+0000 7f1c4bfff700 1 --2- 172.21.15.1:0/3241942687 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f1c340a1150 0x7f1c340a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:11.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.508+0000 7f1c497fa700 1 -- 172.21.15.1:0/3241942687 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1c40014070 con 0x7f1c4c101480 2023-11-16T16:37:11.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.508+0000 7f1c4bfff700 1 --2- 172.21.15.1:0/3241942687 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f1c340a1150 0x7f1c340a3600 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c00b2e0 tx=0x7f1c3c005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:11.513 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.511+0000 7f1c497fa700 1 -- 172.21.15.1:0/3241942687 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1c40096860 con 0x7f1c4c101480 2023-11-16T16:37:11.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:11 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/990486725' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:37:11.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:11 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/520958629' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:37:11.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:11 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1952540826' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-11-16T16:37:11.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:11.717+0000 7f1c52353700 1 -- 172.21.15.1:0/3241942687 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f1c4c067210 con 0x7f1c4c101480 2023-11-16T16:37:11.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:11 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/990486725' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:37:11.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:11 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/520958629' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:37:11.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:11 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1952540826' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-11-16T16:37:11.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:11 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/990486725' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:37:11.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:11 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/520958629' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:37:11.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:11 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1952540826' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-11-16T16:37:12.480 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.478+0000 7f1c497fa700 1 -- 172.21.15.1:0/3241942687 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v81) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1c40099e80 con 0x7f1c4c101480 2023-11-16T16:37:12.484 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.482+0000 7f1c52353700 1 -- 172.21.15.1:0/3241942687 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f1c340a1150 msgr2=0x7f1c340a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:12.484 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.482+0000 7f1c52353700 1 --2- 172.21.15.1:0/3241942687 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f1c340a1150 0x7f1c340a3600 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c00b2e0 tx=0x7f1c3c005e80 comp rx=0 tx=0).stop 2023-11-16T16:37:12.484 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.482+0000 7f1c52353700 1 -- 172.21.15.1:0/3241942687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c4c101480 msgr2=0x7f1c4c1986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:12.484 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.482+0000 7f1c52353700 1 --2- 172.21.15.1:0/3241942687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c4c101480 0x7f1c4c1986e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1c4000e9a0 tx=0x7f1c4000ed60 comp rx=0 tx=0).stop 2023-11-16T16:37:12.484 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.482+0000 7f1c52353700 1 -- 172.21.15.1:0/3241942687 shutdown_connections 2023-11-16T16:37:12.484 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.482+0000 7f1c52353700 1 --2- 172.21.15.1:0/3241942687 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f1c340a1150 0x7f1c340a3600 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:12.484 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.482+0000 7f1c52353700 1 --2- 172.21.15.1:0/3241942687 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1c4c101e10 0x7f1c4c19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:12.484 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.482+0000 7f1c52353700 1 --2- 172.21.15.1:0/3241942687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c4c100290 0x7f1c4c1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:12.484 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.482+0000 7f1c52353700 1 --2- 172.21.15.1:0/3241942687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c4c101480 0x7f1c4c1986e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:12.485 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.482+0000 7f1c52353700 1 -- 172.21.15.1:0/3241942687 >> 172.21.15.1:0/3241942687 conn(0x7f1c4c0fb8b0 msgr2=0x7f1c4c0fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:12.485 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.482+0000 7f1c52353700 1 -- 172.21.15.1:0/3241942687 shutdown_connections 2023-11-16T16:37:12.485 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.482+0000 7f1c52353700 1 -- 172.21.15.1:0/3241942687 wait complete. 2023-11-16T16:37:12.486 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_metadata' created 2023-11-16T16:37:12.497 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-16T16:37:12.632 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.629+0000 7fcfc304f700 1 -- 172.21.15.1:0/2048071549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfbc100d10 msgr2=0x7fcfbc1075d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:12.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.629+0000 7fcfc304f700 1 --2- 172.21.15.1:0/2048071549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfbc100d10 0x7fcfbc1075d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcfb400b0a0 tx=0x7fcfb400b3b0 comp rx=0 tx=0).stop 2023-11-16T16:37:12.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.629+0000 7fcfc304f700 1 -- 172.21.15.1:0/2048071549 shutdown_connections 2023-11-16T16:37:12.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.629+0000 7fcfc304f700 1 --2- 172.21.15.1:0/2048071549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfbc100d10 0x7fcfbc1075d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:12.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.629+0000 7fcfc304f700 1 --2- 172.21.15.1:0/2048071549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfbc0ff190 0x7fcfbc0ff5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:12.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.629+0000 7fcfc304f700 1 --2- 172.21.15.1:0/2048071549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfbc100380 0x7fcfbc1007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:12.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.629+0000 7fcfc304f700 1 -- 172.21.15.1:0/2048071549 >> 172.21.15.1:0/2048071549 conn(0x7fcfbc0fa770 msgr2=0x7fcfbc0fcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:12.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.629+0000 7fcfc304f700 1 -- 172.21.15.1:0/2048071549 shutdown_connections 2023-11-16T16:37:12.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.629+0000 7fcfc304f700 1 -- 172.21.15.1:0/2048071549 wait complete. 2023-11-16T16:37:12.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.629+0000 7fcfc304f700 1 Processor -- start 2023-11-16T16:37:12.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.630+0000 7fcfc304f700 1 -- start start 2023-11-16T16:37:12.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.630+0000 7fcfc304f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfbc0ff190 0x7fcfbc195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:12.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.630+0000 7fcfc304f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfbc100380 0x7fcfbc1964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:12.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.630+0000 7fcfc304f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfbc100d10 0x7fcfbc19d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:12.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.630+0000 7fcfc304f700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc19db30 con 0x7fcfbc100380 2023-11-16T16:37:12.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.630+0000 7fcfc304f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc19dc70 con 0x7fcfbc0ff190 2023-11-16T16:37:12.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.630+0000 7fcfc304f700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc19de00 con 0x7fcfbc100d10 2023-11-16T16:37:12.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.630+0000 7fcfbbfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfbc100380 0x7fcfbc1964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:12.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.630+0000 7fcfbbfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfbc100380 0x7fcfbc1964f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41404/0 (socket says 172.21.15.1:41404) 2023-11-16T16:37:12.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.630+0000 7fcfbbfff700 1 -- 172.21.15.1:0/2372051516 learned_addr learned my addr 172.21.15.1:0/2372051516 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:12.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.630+0000 7fcfc0deb700 1 --2- 172.21.15.1:0/2372051516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfbc0ff190 0x7fcfbc195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:12.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.630+0000 7fcfc15ec700 1 --2- 172.21.15.1:0/2372051516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfbc100d10 0x7fcfbc19d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:12.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.630+0000 7fcfbbfff700 1 -- 172.21.15.1:0/2372051516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfbc100d10 msgr2=0x7fcfbc19d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:12.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.630+0000 7fcfbbfff700 1 --2- 172.21.15.1:0/2372051516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfbc100d10 0x7fcfbc19d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:12.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.630+0000 7fcfbbfff700 1 -- 172.21.15.1:0/2372051516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfbc0ff190 msgr2=0x7fcfbc195fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:12.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.630+0000 7fcfbbfff700 1 --2- 172.21.15.1:0/2372051516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfbc0ff190 0x7fcfbc195fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:12.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.630+0000 7fcfbbfff700 1 -- 172.21.15.1:0/2372051516 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb4009d00 con 0x7fcfbc100380 2023-11-16T16:37:12.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.630+0000 7fcfc0deb700 1 --2- 172.21.15.1:0/2372051516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfbc0ff190 0x7fcfbc195fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:37:12.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.630+0000 7fcfbbfff700 1 --2- 172.21.15.1:0/2372051516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfbc100380 0x7fcfbc1964f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcfa800e9a0 tx=0x7fcfa800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:12.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.630+0000 7fcfb9ffb700 1 -- 172.21.15.1:0/2372051516 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8009790 con 0x7fcfbc100380 2023-11-16T16:37:12.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.631+0000 7fcfb9ffb700 1 -- 172.21.15.1:0/2372051516 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcfa8010430 con 0x7fcfbc100380 2023-11-16T16:37:12.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.631+0000 7fcfc304f700 1 -- 172.21.15.1:0/2372051516 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfbc10ab10 con 0x7fcfbc100380 2023-11-16T16:37:12.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.631+0000 7fcfb9ffb700 1 -- 172.21.15.1:0/2372051516 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8018680 con 0x7fcfbc100380 2023-11-16T16:37:12.636 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.631+0000 7fcfc304f700 1 -- 172.21.15.1:0/2372051516 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfbc10a980 con 0x7fcfbc100380 2023-11-16T16:37:12.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.632+0000 7fcfc304f700 1 -- 172.21.15.1:0/2372051516 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfbc19e4f0 con 0x7fcfbc100380 2023-11-16T16:37:12.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.633+0000 7fcfb9ffb700 1 -- 172.21.15.1:0/2372051516 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fcfa80105a0 con 0x7fcfbc100380 2023-11-16T16:37:12.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.633+0000 7fcfb9ffb700 1 --2- 172.21.15.1:0/2372051516 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fcfac0a1150 0x7fcfac0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:12.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.634+0000 7fcfc0deb700 1 --2- 172.21.15.1:0/2372051516 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fcfac0a1150 0x7fcfac0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:12.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.634+0000 7fcfb9ffb700 1 -- 172.21.15.1:0/2372051516 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fcfa8014070 con 0x7fcfbc100380 2023-11-16T16:37:12.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.634+0000 7fcfc0deb700 1 --2- 172.21.15.1:0/2372051516 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fcfac0a1150 0x7fcfac0a3600 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc1001e0 tx=0x7fcfb0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:12.639 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.637+0000 7fcfb9ffb700 1 -- 172.21.15.1:0/2372051516 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fcfa8096410 con 0x7fcfbc100380 2023-11-16T16:37:12.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:12 smithi026 ceph-mon[100112]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:37:12.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:12 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:12.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:12 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/3241942687' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-11-16T16:37:12.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:12 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:12.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:12 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:12.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:12 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:12.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:12 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:12.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:12 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:12.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:12 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:12.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:12 smithi053 ceph-mon[100638]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:37:12.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:12 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:12.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:12 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/3241942687' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-11-16T16:37:12.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:12 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:12.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:12 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:12.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:12 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:12.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:12 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:12.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:12 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:12.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:12 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:12.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:12.838+0000 7fcfc304f700 1 -- 172.21.15.1:0/2372051516 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fcfbc19eaa0 con 0x7fcfbc100380 2023-11-16T16:37:12.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:12 smithi001 ceph-mon[98800]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:37:12.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:12 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:12.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:12 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3241942687' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-11-16T16:37:12.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:12 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:12.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:12 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:12.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:12 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:12.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:12 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:12.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:12 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:12.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:12 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:13.485 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.483+0000 7fcfb9ffb700 1 -- 172.21.15.1:0/2372051516 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v82) v1 ==== 170+0+0 (secure 0 0 0) 0x7fcfa8099a30 con 0x7fcfbc100380 2023-11-16T16:37:13.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.496+0000 7fcfc304f700 1 -- 172.21.15.1:0/2372051516 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fcfac0a1150 msgr2=0x7fcfac0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:13.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.496+0000 7fcfc304f700 1 --2- 172.21.15.1:0/2372051516 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fcfac0a1150 0x7fcfac0a3600 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc1001e0 tx=0x7fcfb0009260 comp rx=0 tx=0).stop 2023-11-16T16:37:13.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.496+0000 7fcfc304f700 1 -- 172.21.15.1:0/2372051516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfbc100380 msgr2=0x7fcfbc1964f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:13.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.496+0000 7fcfc304f700 1 --2- 172.21.15.1:0/2372051516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfbc100380 0x7fcfbc1964f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcfa800e9a0 tx=0x7fcfa800ecb0 comp rx=0 tx=0).stop 2023-11-16T16:37:13.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.497+0000 7fcfc304f700 1 -- 172.21.15.1:0/2372051516 shutdown_connections 2023-11-16T16:37:13.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.498+0000 7fcfc304f700 1 --2- 172.21.15.1:0/2372051516 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fcfac0a1150 0x7fcfac0a3600 secure :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc1001e0 tx=0x7fcfb0009260 comp rx=0 tx=0).stop 2023-11-16T16:37:13.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.498+0000 7fcfc304f700 1 --2- 172.21.15.1:0/2372051516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcfbc100d10 0x7fcfbc19d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:13.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.498+0000 7fcfc304f700 1 --2- 172.21.15.1:0/2372051516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfbc0ff190 0x7fcfbc195fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:13.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.498+0000 7fcfc304f700 1 --2- 172.21.15.1:0/2372051516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfbc100380 0x7fcfbc1964f0 secure :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcfa800e9a0 tx=0x7fcfa800ecb0 comp rx=0 tx=0).stop 2023-11-16T16:37:13.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.498+0000 7fcfc304f700 1 -- 172.21.15.1:0/2372051516 >> 172.21.15.1:0/2372051516 conn(0x7fcfbc0fa770 msgr2=0x7fcfbc068ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:13.503 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.501+0000 7fcfc304f700 1 -- 172.21.15.1:0/2372051516 shutdown_connections 2023-11-16T16:37:13.503 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.501+0000 7fcfc304f700 1 -- 172.21.15.1:0/2372051516 wait complete. 2023-11-16T16:37:13.505 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data' created 2023-11-16T16:37:13.522 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-16T16:37:13.660 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.655+0000 7f8330a69700 1 -- 172.21.15.1:0/1107943054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f832c1035e0 msgr2=0x7f832c1059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:13.660 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.655+0000 7f8330a69700 1 --2- 172.21.15.1:0/1107943054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f832c1035e0 0x7f832c1059c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8320009990 tx=0x7f8320009ca0 comp rx=0 tx=0).stop 2023-11-16T16:37:13.661 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.655+0000 7f8330a69700 1 -- 172.21.15.1:0/1107943054 shutdown_connections 2023-11-16T16:37:13.661 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.655+0000 7f8330a69700 1 --2- 172.21.15.1:0/1107943054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f832c105f00 0x7f832c1082e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:13.661 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.655+0000 7f8330a69700 1 --2- 172.21.15.1:0/1107943054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f832c100cc0 0x7f832c1030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:13.661 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.655+0000 7f8330a69700 1 --2- 172.21.15.1:0/1107943054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f832c1035e0 0x7f832c1059c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:13.661 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.655+0000 7f8330a69700 1 -- 172.21.15.1:0/1107943054 >> 172.21.15.1:0/1107943054 conn(0x7f832c0fa680 msgr2=0x7f832c0fcaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:13.661 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.655+0000 7f8330a69700 1 -- 172.21.15.1:0/1107943054 shutdown_connections 2023-11-16T16:37:13.661 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.655+0000 7f8330a69700 1 -- 172.21.15.1:0/1107943054 wait complete. 2023-11-16T16:37:13.661 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.656+0000 7f8330a69700 1 Processor -- start 2023-11-16T16:37:13.661 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.656+0000 7f8330a69700 1 -- start start 2023-11-16T16:37:13.661 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.656+0000 7f8330a69700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f832c100cc0 0x7f832c195f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:13.662 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.656+0000 7f8330a69700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f832c1035e0 0x7f832c196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:13.662 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.656+0000 7f8330a69700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f832c105f00 0x7f832c19d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:13.662 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.656+0000 7f8330a69700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c19dac0 con 0x7f832c1035e0 2023-11-16T16:37:13.662 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.656+0000 7f8330a69700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c19dc00 con 0x7f832c105f00 2023-11-16T16:37:13.662 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.656+0000 7f8330a69700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c19dd90 con 0x7f832c100cc0 2023-11-16T16:37:13.662 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.656+0000 7f8329d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f832c1035e0 0x7f832c196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:13.662 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.656+0000 7f8329d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f832c1035e0 0x7f832c196480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41422/0 (socket says 172.21.15.1:41422) 2023-11-16T16:37:13.662 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.656+0000 7f8329d9b700 1 -- 172.21.15.1:0/1102501970 learned_addr learned my addr 172.21.15.1:0/1102501970 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:13.663 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.656+0000 7f832a59c700 1 --2- 172.21.15.1:0/1102501970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f832c100cc0 0x7f832c195f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:13.663 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.656+0000 7f832ad9d700 1 --2- 172.21.15.1:0/1102501970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f832c105f00 0x7f832c19d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:13.663 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.657+0000 7f8329d9b700 1 -- 172.21.15.1:0/1102501970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f832c100cc0 msgr2=0x7f832c195f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:13.663 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.657+0000 7f8329d9b700 1 --2- 172.21.15.1:0/1102501970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f832c100cc0 0x7f832c195f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:13.663 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.657+0000 7f8329d9b700 1 -- 172.21.15.1:0/1102501970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f832c105f00 msgr2=0x7f832c19d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:13.663 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.657+0000 7f8329d9b700 1 --2- 172.21.15.1:0/1102501970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f832c105f00 0x7f832c19d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:13.663 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.657+0000 7f8329d9b700 1 -- 172.21.15.1:0/1102501970 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83200095f0 con 0x7f832c1035e0 2023-11-16T16:37:13.663 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.657+0000 7f8329d9b700 1 --2- 172.21.15.1:0/1102501970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f832c1035e0 0x7f832c196480 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8320005910 tx=0x7f832000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:13.663 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.657+0000 7f831b7fe700 1 -- 172.21.15.1:0/1102501970 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832001d070 con 0x7f832c1035e0 2023-11-16T16:37:13.663 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.657+0000 7f831b7fe700 1 -- 172.21.15.1:0/1102501970 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8320022470 con 0x7f832c1035e0 2023-11-16T16:37:13.664 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.657+0000 7f8330a69700 1 -- 172.21.15.1:0/1102501970 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f832c10bb50 con 0x7f832c1035e0 2023-11-16T16:37:13.664 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.657+0000 7f831b7fe700 1 -- 172.21.15.1:0/1102501970 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8320017650 con 0x7f832c1035e0 2023-11-16T16:37:13.664 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.657+0000 7f8330a69700 1 -- 172.21.15.1:0/1102501970 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f832c10b9c0 con 0x7f832c1035e0 2023-11-16T16:37:13.664 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.658+0000 7f8330a69700 1 -- 172.21.15.1:0/1102501970 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f832c19e330 con 0x7f832c1035e0 2023-11-16T16:37:13.665 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.659+0000 7f831b7fe700 1 -- 172.21.15.1:0/1102501970 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f832000fad0 con 0x7f832c1035e0 2023-11-16T16:37:13.665 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.660+0000 7f831b7fe700 1 --2- 172.21.15.1:0/1102501970 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f83140a1150 0x7f83140a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:13.665 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.660+0000 7f831b7fe700 1 -- 172.21.15.1:0/1102501970 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f83200cdf00 con 0x7f832c1035e0 2023-11-16T16:37:13.665 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.660+0000 7f832a59c700 1 --2- 172.21.15.1:0/1102501970 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f83140a1150 0x7f83140a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:13.665 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.660+0000 7f832a59c700 1 --2- 172.21.15.1:0/1102501970 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f83140a1150 0x7f83140a3600 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f831c005950 tx=0x7f831c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:13.666 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.664+0000 7f831b7fe700 1 -- 172.21.15.1:0/1102501970 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8320097890 con 0x7f832c1035e0 2023-11-16T16:37:13.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:13 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/3241942687' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-11-16T16:37:13.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:13 smithi026 ceph-mon[100112]: osdmap e81: 12 total, 12 up, 12 in 2023-11-16T16:37:13.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:13 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/2372051516' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-11-16T16:37:13.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:13 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:37:13.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:13 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:37:13.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:13 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:13.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:13 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/3241942687' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-11-16T16:37:13.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:13 smithi053 ceph-mon[100638]: osdmap e81: 12 total, 12 up, 12 in 2023-11-16T16:37:13.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:13 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/2372051516' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-11-16T16:37:13.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:13 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:37:13.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:13 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:37:13.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:13 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:13.918 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.917+0000 7f8330a69700 1 -- 172.21.15.1:0/1102501970 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f832c19e8e0 con 0x7f832c1035e0 2023-11-16T16:37:13.925 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.923+0000 7f831b7fe700 1 -- 172.21.15.1:0/1102501970 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f832009aeb0 con 0x7f832c1035e0 2023-11-16T16:37:13.932 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.930+0000 7f8330a69700 1 -- 172.21.15.1:0/1102501970 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f83140a1150 msgr2=0x7f83140a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:13.932 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.930+0000 7f8330a69700 1 --2- 172.21.15.1:0/1102501970 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f83140a1150 0x7f83140a3600 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f831c005950 tx=0x7f831c00b410 comp rx=0 tx=0).stop 2023-11-16T16:37:13.932 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.930+0000 7f8330a69700 1 -- 172.21.15.1:0/1102501970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f832c1035e0 msgr2=0x7f832c196480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:13.932 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.930+0000 7f8330a69700 1 --2- 172.21.15.1:0/1102501970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f832c1035e0 0x7f832c196480 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8320005910 tx=0x7f832000f690 comp rx=0 tx=0).stop 2023-11-16T16:37:13.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:13 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3241942687' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-11-16T16:37:13.934 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:13 smithi001 ceph-mon[98800]: osdmap e81: 12 total, 12 up, 12 in 2023-11-16T16:37:13.934 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:13 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/2372051516' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-11-16T16:37:13.934 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:13 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:37:13.934 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:13 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:37:13.934 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:13 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:13.940 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.938+0000 7f8330a69700 1 -- 172.21.15.1:0/1102501970 shutdown_connections 2023-11-16T16:37:13.940 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.938+0000 7f8330a69700 1 --2- 172.21.15.1:0/1102501970 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f83140a1150 0x7f83140a3600 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:13.940 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.938+0000 7f8330a69700 1 --2- 172.21.15.1:0/1102501970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f832c105f00 0x7f832c19d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:13.941 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.938+0000 7f8330a69700 1 --2- 172.21.15.1:0/1102501970 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f832c100cc0 0x7f832c195f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:13.941 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.938+0000 7f8330a69700 1 --2- 172.21.15.1:0/1102501970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f832c1035e0 0x7f832c196480 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:13.941 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.938+0000 7f8330a69700 1 -- 172.21.15.1:0/1102501970 >> 172.21.15.1:0/1102501970 conn(0x7f832c0fa680 msgr2=0x7f832c102120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:13.944 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.941+0000 7f8330a69700 1 -- 172.21.15.1:0/1102501970 shutdown_connections 2023-11-16T16:37:13.944 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:13.941+0000 7f8330a69700 1 -- 172.21.15.1:0/1102501970 wait complete. 2023-11-16T16:37:13.954 INFO:teuthology.orchestra.run.smithi001.stderr:new fs with metadata pool 2 and data pool 3 2023-11-16T16:37:13.966 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-16T16:37:14.096 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.093+0000 7fadd6608700 1 -- 172.21.15.1:0/3287565061 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fadd0100d90 msgr2=0x7fadd0103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:14.096 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.093+0000 7fadd6608700 1 --2- 172.21.15.1:0/3287565061 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fadd0100d90 0x7fadd0103170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fadc0009990 tx=0x7fadc0009ca0 comp rx=0 tx=0).stop 2023-11-16T16:37:14.097 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.093+0000 7fadd6608700 1 -- 172.21.15.1:0/3287565061 shutdown_connections 2023-11-16T16:37:14.097 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.093+0000 7fadd6608700 1 --2- 172.21.15.1:0/3287565061 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadd0105fd0 0x7fadd0108430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:14.097 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.093+0000 7fadd6608700 1 --2- 172.21.15.1:0/3287565061 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fadd0100d90 0x7fadd0103170 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:14.097 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.093+0000 7fadd6608700 1 --2- 172.21.15.1:0/3287565061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadd01036b0 0x7fadd0105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:14.097 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.093+0000 7fadd6608700 1 -- 172.21.15.1:0/3287565061 >> 172.21.15.1:0/3287565061 conn(0x7fadd00fa810 msgr2=0x7fadd00fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:14.097 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.093+0000 7fadd6608700 1 -- 172.21.15.1:0/3287565061 shutdown_connections 2023-11-16T16:37:14.097 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.093+0000 7fadd6608700 1 -- 172.21.15.1:0/3287565061 wait complete. 2023-11-16T16:37:14.097 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.094+0000 7fadd6608700 1 Processor -- start 2023-11-16T16:37:14.098 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.094+0000 7fadd6608700 1 -- start start 2023-11-16T16:37:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.095+0000 7fadd6608700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fadd0100d90 0x7fadd0196030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.095+0000 7fadd6608700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadd01036b0 0x7fadd0196570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.095+0000 7fadd6608700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadd0105fd0 0x7fadd019d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.095+0000 7fadd6608700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd019dbb0 con 0x7fadd0100d90 2023-11-16T16:37:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.095+0000 7fadd6608700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd019dcf0 con 0x7fadd0105fd0 2023-11-16T16:37:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.095+0000 7fadd6608700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd019de80 con 0x7fadd01036b0 2023-11-16T16:37:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.095+0000 7fadcffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fadd0100d90 0x7fadd0196030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:14.103 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.095+0000 7fadcffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fadd0100d90 0x7fadd0196030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41438/0 (socket says 172.21.15.1:41438) 2023-11-16T16:37:14.103 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.095+0000 7fadcffff700 1 -- 172.21.15.1:0/3786954012 learned_addr learned my addr 172.21.15.1:0/3786954012 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:14.103 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.095+0000 7fadcf7fe700 1 --2- 172.21.15.1:0/3786954012 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadd01036b0 0x7fadd0196570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:14.103 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.095+0000 7fadd4ba5700 1 --2- 172.21.15.1:0/3786954012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadd0105fd0 0x7fadd019d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:14.103 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.095+0000 7fadd4ba5700 1 -- 172.21.15.1:0/3786954012 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadd01036b0 msgr2=0x7fadd0196570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:14.103 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.095+0000 7fadd4ba5700 1 --2- 172.21.15.1:0/3786954012 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadd01036b0 0x7fadd0196570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:14.103 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.095+0000 7fadd4ba5700 1 -- 172.21.15.1:0/3786954012 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fadd0100d90 msgr2=0x7fadd0196030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:14.104 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.095+0000 7fadd4ba5700 1 --2- 172.21.15.1:0/3786954012 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fadd0100d90 0x7fadd0196030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:14.104 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.095+0000 7fadd4ba5700 1 -- 172.21.15.1:0/3786954012 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc00095f0 con 0x7fadd0105fd0 2023-11-16T16:37:14.104 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.095+0000 7fadcffff700 1 --2- 172.21.15.1:0/3786954012 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fadd0100d90 0x7fadd0196030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:37:14.104 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.095+0000 7fadcf7fe700 1 --2- 172.21.15.1:0/3786954012 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadd01036b0 0x7fadd0196570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:37:14.104 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.095+0000 7fadd4ba5700 1 --2- 172.21.15.1:0/3786954012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadd0105fd0 0x7fadd019d5e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fadc8007e90 tx=0x7fadc800f740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:14.104 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.096+0000 7fadcd7fa700 1 -- 172.21.15.1:0/3786954012 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc8014070 con 0x7fadd0105fd0 2023-11-16T16:37:14.104 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.096+0000 7fadcd7fa700 1 -- 172.21.15.1:0/3786954012 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadc8009e30 con 0x7fadd0105fd0 2023-11-16T16:37:14.104 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.096+0000 7fadcd7fa700 1 -- 172.21.15.1:0/3786954012 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc8004320 con 0x7fadd0105fd0 2023-11-16T16:37:14.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.096+0000 7fadd6608700 1 -- 172.21.15.1:0/3786954012 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadd010bc70 con 0x7fadd0105fd0 2023-11-16T16:37:14.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.096+0000 7fadd6608700 1 -- 172.21.15.1:0/3786954012 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd010bae0 con 0x7fadd0105fd0 2023-11-16T16:37:14.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.097+0000 7fadd6608700 1 -- 172.21.15.1:0/3786954012 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadd00fc380 con 0x7fadd0105fd0 2023-11-16T16:37:14.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.098+0000 7fadcd7fa700 1 -- 172.21.15.1:0/3786954012 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fadc8018070 con 0x7fadd0105fd0 2023-11-16T16:37:14.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.098+0000 7fadcd7fa700 1 --2- 172.21.15.1:0/3786954012 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fadb80a1150 0x7fadb80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:14.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.099+0000 7fadcffff700 1 --2- 172.21.15.1:0/3786954012 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fadb80a1150 0x7fadb80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:14.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.099+0000 7fadcd7fa700 1 -- 172.21.15.1:0/3786954012 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fadc80cfc00 con 0x7fadd0105fd0 2023-11-16T16:37:14.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.099+0000 7fadcffff700 1 --2- 172.21.15.1:0/3786954012 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fadb80a1150 0x7fadb80a3600 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fadc000b2e0 tx=0x7fadc0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:14.106 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.103+0000 7fadcd7fa700 1 -- 172.21.15.1:0/3786954012 <== mon.2 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fadc8099480 con 0x7fadd0105fd0 2023-11-16T16:37:14.325 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.323+0000 7fadd6608700 1 -- 172.21.15.1:0/3786954012 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fadd002cfe0 con 0x7fadd0105fd0 2023-11-16T16:37:14.327 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.325+0000 7fadcd7fa700 1 -- 172.21.15.1:0/3786954012 <== mon.2 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22001 (secure 0 0 0) 0x7fadc809caa0 con 0x7fadd0105fd0 2023-11-16T16:37:14.328 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:37:14.328 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":83,"fsid":"35d9b010-849d-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:30:06.217778+0000","modified":"2023-11-16T16:37:13.918565+0000","last_up_change":"2023-11-16T16:36:31.735403+0000","last_in_change":"2023-11-16T16:36:14.083021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:33:15.916723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-16T16:37:12.158031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-16T16:37:13.272343+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a8480507-5275-4e52-953b-626110beb436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6803","nonce":2469671609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6805","nonce":2469671609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6809","nonce":2469671609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6807","nonce":2469671609}]},"public_addr":"172.21.15.1:6803/2469671609","cluster_addr":"172.21.15.1:6805/2469671609","heartbeat_back_addr":"172.21.15.1:6809/2469671609","heartbeat_front_addr":"172.21.15.1:6807/2469671609","state":["exists","up"]},{"osd":1,"uuid":"1d011db2-99d1-4575-bdc5-51f88a308fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6801","nonce":247994796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6803","nonce":247994796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6807","nonce":247994796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6805","nonce":247994796}]},"public_addr":"172.21.15.26:6801/247994796","cluster_addr":"172.21.15.26:6803/247994796","heartbeat_back_addr":"172.21.15.26:6807/247994796","heartbeat_front_addr":"172.21.15.26:6805/247994796","state":["exists","up"]},{"osd":2,"uuid":"4d7a2bd1-3c8e-4546-a243-f0354473b031","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6801","nonce":1659316290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6803","nonce":1659316290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6807","nonce":1659316290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6805","nonce":1659316290}]},"public_addr":"172.21.15.53:6801/1659316290","cluster_addr":"172.21.15.53:6803/1659316290","heartbeat_back_addr":"172.21.15.53:6807/1659316290","heartbeat_front_addr":"172.21.15.53:6805/1659316290","state":["exists","up"]},{"osd":3,"uuid":"9c290fd9-e121-4a81-9687-b0eb93debd86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6811","nonce":2728987281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6813","nonce":2728987281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6817","nonce":2728987281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6815","nonce":2728987281}]},"public_addr":"172.21.15.1:6811/2728987281","cluster_addr":"172.21.15.1:6813/2728987281","heartbeat_back_addr":"172.21.15.1:6817/2728987281","heartbeat_front_addr":"172.21.15.1:6815/2728987281","state":["exists","up"]},{"osd":4,"uuid":"533226bc-cc57-4f15-9e29-820294d57e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6809","nonce":2954740024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6811","nonce":2954740024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6815","nonce":2954740024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6813","nonce":2954740024}]},"public_addr":"172.21.15.26:6809/2954740024","cluster_addr":"172.21.15.26:6811/2954740024","heartbeat_back_addr":"172.21.15.26:6815/2954740024","heartbeat_front_addr":"172.21.15.26:6813/2954740024","state":["exists","up"]},{"osd":5,"uuid":"d3ab25eb-e1d4-4557-bc89-efc9072401ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6809","nonce":3966666318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6811","nonce":3966666318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6815","nonce":3966666318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6813","nonce":3966666318}]},"public_addr":"172.21.15.53:6809/3966666318","cluster_addr":"172.21.15.53:6811/3966666318","heartbeat_back_addr":"172.21.15.53:6815/3966666318","heartbeat_front_addr":"172.21.15.53:6813/3966666318","state":["exists","up"]},{"osd":6,"uuid":"180f506e-afb8-4043-b0b4-2dc86f6d4bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6819","nonce":3364032646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6821","nonce":3364032646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6825","nonce":3364032646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6823","nonce":3364032646}]},"public_addr":"172.21.15.1:6819/3364032646","cluster_addr":"172.21.15.1:6821/3364032646","heartbeat_back_addr":"172.21.15.1:6825/3364032646","heartbeat_front_addr":"172.21.15.1:6823/3364032646","state":["exists","up"]},{"osd":7,"uuid":"0d837bac-194f-498d-b7fb-b3147e992329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6817","nonce":3766901391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6819","nonce":3766901391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6823","nonce":3766901391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6821","nonce":3766901391}]},"public_addr":"172.21.15.26:6817/3766901391","cluster_addr":"172.21.15.26:6819/3766901391","heartbeat_back_addr":"172.21.15.26:6823/3766901391","heartbeat_front_addr":"172.21.15.26:6821/3766901391","state":["exists","up"]},{"osd":8,"uuid":"933f231a-7454-47b0-9291-f1fa64f0baec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6817","nonce":3224335871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6819","nonce":3224335871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6823","nonce":3224335871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6821","nonce":3224335871}]},"public_addr":"172.21.15.53:6817/3224335871","cluster_addr":"172.21.15.53:6819/3224335871","heartbeat_back_addr":"172.21.15.53:6823/3224335871","heartbeat_front_addr":"172.21.15.53:6821/3224335871","state":["exists","up"]},{"osd":9,"uuid":"245fcbb3-c651-4703-ab80-309dc98bd983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6827","nonce":1391199844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6829","nonce":1391199844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6833","nonce":1391199844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6831","nonce":1391199844}]},"public_addr":"172.21.15.1:6827/1391199844","cluster_addr":"172.21.15.1:6829/1391199844","heartbeat_back_addr":"172.21.15.1:6833/1391199844","heartbeat_front_addr":"172.21.15.1:6831/1391199844","state":["exists","up"]},{"osd":10,"uuid":"2c090e7a-5240-4d52-ae4f-e94fe05383d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6825","nonce":1773545447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6827","nonce":1773545447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6831","nonce":1773545447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6829","nonce":1773545447}]},"public_addr":"172.21.15.26:6825/1773545447","cluster_addr":"172.21.15.26:6827/1773545447","heartbeat_back_addr":"172.21.15.26:6831/1773545447","heartbeat_front_addr":"172.21.15.26:6829/1773545447","state":["exists","up"]},{"osd":11,"uuid":"4511e32d-e469-45e1-8310-0ff400cb7b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6825","nonce":2564669546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6827","nonce":2564669546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6831","nonce":2564669546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6829","nonce":2564669546}]},"public_addr":"172.21.15.53:6825/2564669546","cluster_addr":"172.21.15.53:6827/2564669546","heartbeat_back_addr":"172.21.15.53:6831/2564669546","heartbeat_front_addr":"172.21.15.53:6829/2564669546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:28.140250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:49.885461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:12.521487+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:34.926197+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:56.815594+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:18.387751+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:40.587106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:02.319370+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:23.832298+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:46.338040+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:08.730414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:30.050070+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1494336948":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6801/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6800/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6800/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/1641976669":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/2895923638":"2023-11-17T16:30:35.064289+0000","172.21.15.1:6801/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/78811711":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/4131957708":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/344403698":"2023-11-17T16:30:35.064289+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:37:14.330 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.327+0000 7fadd6608700 1 -- 172.21.15.1:0/3786954012 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fadb80a1150 msgr2=0x7fadb80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:14.330 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.327+0000 7fadd6608700 1 --2- 172.21.15.1:0/3786954012 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fadb80a1150 0x7fadb80a3600 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fadc000b2e0 tx=0x7fadc0005960 comp rx=0 tx=0).stop 2023-11-16T16:37:14.330 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.327+0000 7fadd6608700 1 -- 172.21.15.1:0/3786954012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadd0105fd0 msgr2=0x7fadd019d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:14.330 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.327+0000 7fadd6608700 1 --2- 172.21.15.1:0/3786954012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadd0105fd0 0x7fadd019d5e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fadc8007e90 tx=0x7fadc800f740 comp rx=0 tx=0).stop 2023-11-16T16:37:14.330 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.327+0000 7fadd6608700 1 -- 172.21.15.1:0/3786954012 shutdown_connections 2023-11-16T16:37:14.330 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.327+0000 7fadd6608700 1 --2- 172.21.15.1:0/3786954012 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fadb80a1150 0x7fadb80a3600 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:14.330 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.327+0000 7fadd6608700 1 --2- 172.21.15.1:0/3786954012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadd0105fd0 0x7fadd019d5e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:14.331 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.327+0000 7fadd6608700 1 --2- 172.21.15.1:0/3786954012 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fadd0100d90 0x7fadd0196030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:14.331 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.327+0000 7fadd6608700 1 --2- 172.21.15.1:0/3786954012 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fadd01036b0 0x7fadd0196570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:14.331 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.327+0000 7fadd6608700 1 -- 172.21.15.1:0/3786954012 >> 172.21.15.1:0/3786954012 conn(0x7fadd00fa810 msgr2=0x7fadd01019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:14.331 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.327+0000 7fadd6608700 1 -- 172.21.15.1:0/3786954012 shutdown_connections 2023-11-16T16:37:14.331 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.327+0000 7fadd6608700 1 -- 172.21.15.1:0/3786954012 wait complete. 2023-11-16T16:37:14.342 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-16T16:37:14.475 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.470+0000 7fb3dc6c4700 1 -- 172.21.15.1:0/2005341830 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3d4100d40 msgr2=0x7fb3d4107600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:14.475 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.470+0000 7fb3dc6c4700 1 --2- 172.21.15.1:0/2005341830 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3d4100d40 0x7fb3d4107600 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc00b0a0 tx=0x7fb3cc00b3b0 comp rx=0 tx=0).stop 2023-11-16T16:37:14.475 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.470+0000 7fb3dc6c4700 1 -- 172.21.15.1:0/2005341830 shutdown_connections 2023-11-16T16:37:14.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.470+0000 7fb3dc6c4700 1 --2- 172.21.15.1:0/2005341830 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3d4100d40 0x7fb3d4107600 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:14.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.470+0000 7fb3dc6c4700 1 --2- 172.21.15.1:0/2005341830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3d40ff1c0 0x7fb3d40ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:14.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.470+0000 7fb3dc6c4700 1 --2- 172.21.15.1:0/2005341830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3d41003b0 0x7fb3d4100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:14.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.470+0000 7fb3dc6c4700 1 -- 172.21.15.1:0/2005341830 >> 172.21.15.1:0/2005341830 conn(0x7fb3d40fa7a0 msgr2=0x7fb3d40fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:14.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.470+0000 7fb3dc6c4700 1 -- 172.21.15.1:0/2005341830 shutdown_connections 2023-11-16T16:37:14.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.470+0000 7fb3dc6c4700 1 -- 172.21.15.1:0/2005341830 wait complete. 2023-11-16T16:37:14.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.471+0000 7fb3dc6c4700 1 Processor -- start 2023-11-16T16:37:14.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.471+0000 7fb3dc6c4700 1 -- start start 2023-11-16T16:37:14.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.471+0000 7fb3dc6c4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3d40ff1c0 0x7fb3d41960d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:14.477 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.471+0000 7fb3dc6c4700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3d41003b0 0x7fb3d4196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:14.477 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.471+0000 7fb3dc6c4700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3d4100d40 0x7fb3d419d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:14.477 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.471+0000 7fb3dc6c4700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d419dc50 con 0x7fb3d41003b0 2023-11-16T16:37:14.477 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.471+0000 7fb3dc6c4700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d419dd90 con 0x7fb3d40ff1c0 2023-11-16T16:37:14.477 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.471+0000 7fb3dc6c4700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d419df20 con 0x7fb3d4100d40 2023-11-16T16:37:14.477 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.471+0000 7fb3d9c5f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3d41003b0 0x7fb3d4196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:14.477 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.471+0000 7fb3d9c5f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3d41003b0 0x7fb3d4196610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41454/0 (socket says 172.21.15.1:41454) 2023-11-16T16:37:14.477 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.471+0000 7fb3dac61700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3d4100d40 0x7fb3d419d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:14.478 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.471+0000 7fb3d9c5f700 1 -- 172.21.15.1:0/2125693226 learned_addr learned my addr 172.21.15.1:0/2125693226 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:14.478 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.471+0000 7fb3da460700 1 --2- 172.21.15.1:0/2125693226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3d40ff1c0 0x7fb3d41960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:14.478 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.471+0000 7fb3dac61700 1 -- 172.21.15.1:0/2125693226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3d40ff1c0 msgr2=0x7fb3d41960d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:14.478 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.471+0000 7fb3dac61700 1 --2- 172.21.15.1:0/2125693226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3d40ff1c0 0x7fb3d41960d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:14.478 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.471+0000 7fb3dac61700 1 -- 172.21.15.1:0/2125693226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3d41003b0 msgr2=0x7fb3d4196610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:14.478 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.471+0000 7fb3dac61700 1 --2- 172.21.15.1:0/2125693226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3d41003b0 0x7fb3d4196610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:14.478 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.471+0000 7fb3dac61700 1 -- 172.21.15.1:0/2125693226 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3cc009d00 con 0x7fb3d4100d40 2023-11-16T16:37:14.478 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.471+0000 7fb3d9c5f700 1 --2- 172.21.15.1:0/2125693226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3d41003b0 0x7fb3d4196610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:37:14.479 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.472+0000 7fb3dac61700 1 --2- 172.21.15.1:0/2125693226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3d4100d40 0x7fb3d419d680 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc00b0a0 tx=0x7fb3cc004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:14.479 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.472+0000 7fb3c77fe700 1 -- 172.21.15.1:0/2125693226 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc00e040 con 0x7fb3d4100d40 2023-11-16T16:37:14.479 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.472+0000 7fb3dc6c4700 1 -- 172.21.15.1:0/2125693226 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3d410ab40 con 0x7fb3d4100d40 2023-11-16T16:37:14.479 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.472+0000 7fb3c77fe700 1 -- 172.21.15.1:0/2125693226 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3cc01db50 con 0x7fb3d4100d40 2023-11-16T16:37:14.479 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.472+0000 7fb3c77fe700 1 -- 172.21.15.1:0/2125693226 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc01bba0 con 0x7fb3d4100d40 2023-11-16T16:37:14.479 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.472+0000 7fb3dc6c4700 1 -- 172.21.15.1:0/2125693226 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3d410a9b0 con 0x7fb3d4100d40 2023-11-16T16:37:14.479 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.473+0000 7fb3dc6c4700 1 -- 172.21.15.1:0/2125693226 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3d4067260 con 0x7fb3d4100d40 2023-11-16T16:37:14.480 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.474+0000 7fb3c77fe700 1 -- 172.21.15.1:0/2125693226 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fb3cc01dcc0 con 0x7fb3d4100d40 2023-11-16T16:37:14.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.474+0000 7fb3c77fe700 1 --2- 172.21.15.1:0/2125693226 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fb3c00a1150 0x7fb3c00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:14.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.475+0000 7fb3da460700 1 --2- 172.21.15.1:0/2125693226 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fb3c00a1150 0x7fb3c00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:14.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.475+0000 7fb3c77fe700 1 -- 172.21.15.1:0/2125693226 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb3cc0ce8f0 con 0x7fb3d4100d40 2023-11-16T16:37:14.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.475+0000 7fb3da460700 1 --2- 172.21.15.1:0/2125693226 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fb3c00a1150 0x7fb3c00a3600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4100210 tx=0x7fb3d000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:14.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.478+0000 7fb3c77fe700 1 -- 172.21.15.1:0/2125693226 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb3cc098180 con 0x7fb3d4100d40 2023-11-16T16:37:14.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:14 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:37:14.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:14 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:37:14.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:14 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:37:14.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:14 smithi001 ceph-mon[98800]: Service mds.all has no related daemon already placed. 2023-11-16T16:37:14.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:14 smithi001 ceph-mon[98800]: pgmap v257: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:37:14.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:14 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:37:14.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:14 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/2372051516' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-11-16T16:37:14.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:14 smithi001 ceph-mon[98800]: osdmap e82: 12 total, 12 up, 12 in 2023-11-16T16:37:14.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:14 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1102501970' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-11-16T16:37:14.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:14 smithi001 ceph-mon[98800]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-11-16T16:37:14.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:14 smithi001 ceph-mon[98800]: osdmap e83: 12 total, 12 up, 12 in 2023-11-16T16:37:14.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:14 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1102501970' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-11-16T16:37:14.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:14 smithi001 ceph-mon[98800]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-11-16T16:37:14.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:14 smithi001 ceph-mon[98800]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-11-16T16:37:14.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:14 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3786954012' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:37:14.691 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.689+0000 7fb3dc6c4700 1 -- 172.21.15.1:0/2125693226 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3d419ea30 con 0x7fb3d4100d40 2023-11-16T16:37:14.692 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.692+0000 7fb3c77fe700 1 -- 172.21.15.1:0/2125693226 <== mon.1 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22001 (secure 0 0 0) 0x7fb3cc021090 con 0x7fb3d4100d40 2023-11-16T16:37:14.693 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:37:14.693 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":83,"fsid":"35d9b010-849d-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:30:06.217778+0000","modified":"2023-11-16T16:37:13.918565+0000","last_up_change":"2023-11-16T16:36:31.735403+0000","last_in_change":"2023-11-16T16:36:14.083021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:33:15.916723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-16T16:37:12.158031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-16T16:37:13.272343+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a8480507-5275-4e52-953b-626110beb436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6803","nonce":2469671609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6805","nonce":2469671609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6809","nonce":2469671609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6807","nonce":2469671609}]},"public_addr":"172.21.15.1:6803/2469671609","cluster_addr":"172.21.15.1:6805/2469671609","heartbeat_back_addr":"172.21.15.1:6809/2469671609","heartbeat_front_addr":"172.21.15.1:6807/2469671609","state":["exists","up"]},{"osd":1,"uuid":"1d011db2-99d1-4575-bdc5-51f88a308fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6801","nonce":247994796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6803","nonce":247994796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6807","nonce":247994796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6805","nonce":247994796}]},"public_addr":"172.21.15.26:6801/247994796","cluster_addr":"172.21.15.26:6803/247994796","heartbeat_back_addr":"172.21.15.26:6807/247994796","heartbeat_front_addr":"172.21.15.26:6805/247994796","state":["exists","up"]},{"osd":2,"uuid":"4d7a2bd1-3c8e-4546-a243-f0354473b031","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6801","nonce":1659316290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6803","nonce":1659316290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6807","nonce":1659316290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6805","nonce":1659316290}]},"public_addr":"172.21.15.53:6801/1659316290","cluster_addr":"172.21.15.53:6803/1659316290","heartbeat_back_addr":"172.21.15.53:6807/1659316290","heartbeat_front_addr":"172.21.15.53:6805/1659316290","state":["exists","up"]},{"osd":3,"uuid":"9c290fd9-e121-4a81-9687-b0eb93debd86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6811","nonce":2728987281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6813","nonce":2728987281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6817","nonce":2728987281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6815","nonce":2728987281}]},"public_addr":"172.21.15.1:6811/2728987281","cluster_addr":"172.21.15.1:6813/2728987281","heartbeat_back_addr":"172.21.15.1:6817/2728987281","heartbeat_front_addr":"172.21.15.1:6815/2728987281","state":["exists","up"]},{"osd":4,"uuid":"533226bc-cc57-4f15-9e29-820294d57e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6809","nonce":2954740024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6811","nonce":2954740024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6815","nonce":2954740024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6813","nonce":2954740024}]},"public_addr":"172.21.15.26:6809/2954740024","cluster_addr":"172.21.15.26:6811/2954740024","heartbeat_back_addr":"172.21.15.26:6815/2954740024","heartbeat_front_addr":"172.21.15.26:6813/2954740024","state":["exists","up"]},{"osd":5,"uuid":"d3ab25eb-e1d4-4557-bc89-efc9072401ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6809","nonce":3966666318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6811","nonce":3966666318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6815","nonce":3966666318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6813","nonce":3966666318}]},"public_addr":"172.21.15.53:6809/3966666318","cluster_addr":"172.21.15.53:6811/3966666318","heartbeat_back_addr":"172.21.15.53:6815/3966666318","heartbeat_front_addr":"172.21.15.53:6813/3966666318","state":["exists","up"]},{"osd":6,"uuid":"180f506e-afb8-4043-b0b4-2dc86f6d4bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6819","nonce":3364032646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6821","nonce":3364032646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6825","nonce":3364032646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6823","nonce":3364032646}]},"public_addr":"172.21.15.1:6819/3364032646","cluster_addr":"172.21.15.1:6821/3364032646","heartbeat_back_addr":"172.21.15.1:6825/3364032646","heartbeat_front_addr":"172.21.15.1:6823/3364032646","state":["exists","up"]},{"osd":7,"uuid":"0d837bac-194f-498d-b7fb-b3147e992329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6817","nonce":3766901391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6819","nonce":3766901391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6823","nonce":3766901391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6821","nonce":3766901391}]},"public_addr":"172.21.15.26:6817/3766901391","cluster_addr":"172.21.15.26:6819/3766901391","heartbeat_back_addr":"172.21.15.26:6823/3766901391","heartbeat_front_addr":"172.21.15.26:6821/3766901391","state":["exists","up"]},{"osd":8,"uuid":"933f231a-7454-47b0-9291-f1fa64f0baec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6817","nonce":3224335871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6819","nonce":3224335871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6823","nonce":3224335871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6821","nonce":3224335871}]},"public_addr":"172.21.15.53:6817/3224335871","cluster_addr":"172.21.15.53:6819/3224335871","heartbeat_back_addr":"172.21.15.53:6823/3224335871","heartbeat_front_addr":"172.21.15.53:6821/3224335871","state":["exists","up"]},{"osd":9,"uuid":"245fcbb3-c651-4703-ab80-309dc98bd983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6827","nonce":1391199844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6829","nonce":1391199844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6833","nonce":1391199844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6831","nonce":1391199844}]},"public_addr":"172.21.15.1:6827/1391199844","cluster_addr":"172.21.15.1:6829/1391199844","heartbeat_back_addr":"172.21.15.1:6833/1391199844","heartbeat_front_addr":"172.21.15.1:6831/1391199844","state":["exists","up"]},{"osd":10,"uuid":"2c090e7a-5240-4d52-ae4f-e94fe05383d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6825","nonce":1773545447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6827","nonce":1773545447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6831","nonce":1773545447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6829","nonce":1773545447}]},"public_addr":"172.21.15.26:6825/1773545447","cluster_addr":"172.21.15.26:6827/1773545447","heartbeat_back_addr":"172.21.15.26:6831/1773545447","heartbeat_front_addr":"172.21.15.26:6829/1773545447","state":["exists","up"]},{"osd":11,"uuid":"4511e32d-e469-45e1-8310-0ff400cb7b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6825","nonce":2564669546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6827","nonce":2564669546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6831","nonce":2564669546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6829","nonce":2564669546}]},"public_addr":"172.21.15.53:6825/2564669546","cluster_addr":"172.21.15.53:6827/2564669546","heartbeat_back_addr":"172.21.15.53:6831/2564669546","heartbeat_front_addr":"172.21.15.53:6829/2564669546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:28.140250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:49.885461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:12.521487+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:34.926197+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:56.815594+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:18.387751+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:40.587106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:02.319370+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:23.832298+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:46.338040+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:08.730414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:30.050070+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1494336948":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6801/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6800/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6800/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/1641976669":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/2895923638":"2023-11-17T16:30:35.064289+0000","172.21.15.1:6801/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/78811711":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/4131957708":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/344403698":"2023-11-17T16:30:35.064289+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:37:14.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.693+0000 7fb3dc6c4700 1 -- 172.21.15.1:0/2125693226 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fb3c00a1150 msgr2=0x7fb3c00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:14.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.693+0000 7fb3dc6c4700 1 --2- 172.21.15.1:0/2125693226 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fb3c00a1150 0x7fb3c00a3600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4100210 tx=0x7fb3d000b410 comp rx=0 tx=0).stop 2023-11-16T16:37:14.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.693+0000 7fb3dc6c4700 1 -- 172.21.15.1:0/2125693226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3d4100d40 msgr2=0x7fb3d419d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:14.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.693+0000 7fb3dc6c4700 1 --2- 172.21.15.1:0/2125693226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3d4100d40 0x7fb3d419d680 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc00b0a0 tx=0x7fb3cc004d00 comp rx=0 tx=0).stop 2023-11-16T16:37:14.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.693+0000 7fb3dc6c4700 1 -- 172.21.15.1:0/2125693226 shutdown_connections 2023-11-16T16:37:14.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.693+0000 7fb3dc6c4700 1 --2- 172.21.15.1:0/2125693226 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fb3c00a1150 0x7fb3c00a3600 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:14.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.693+0000 7fb3dc6c4700 1 --2- 172.21.15.1:0/2125693226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb3d4100d40 0x7fb3d419d680 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:14.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.693+0000 7fb3dc6c4700 1 --2- 172.21.15.1:0/2125693226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3d40ff1c0 0x7fb3d41960d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:14.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.693+0000 7fb3dc6c4700 1 --2- 172.21.15.1:0/2125693226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3d41003b0 0x7fb3d4196610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:14.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.693+0000 7fb3dc6c4700 1 -- 172.21.15.1:0/2125693226 >> 172.21.15.1:0/2125693226 conn(0x7fb3d40fa7a0 msgr2=0x7fb3d4068e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:14.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.693+0000 7fb3dc6c4700 1 -- 172.21.15.1:0/2125693226 shutdown_connections 2023-11-16T16:37:14.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.693+0000 7fb3dc6c4700 1 -- 172.21.15.1:0/2125693226 wait complete. 2023-11-16T16:37:14.708 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-16T16:37:14.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:14 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:37:14.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:14 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:37:14.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:14 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:37:14.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:14 smithi026 ceph-mon[100112]: Service mds.all has no related daemon already placed. 2023-11-16T16:37:14.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:14 smithi026 ceph-mon[100112]: pgmap v257: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:37:14.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:14 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:37:14.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:14 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/2372051516' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-11-16T16:37:14.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:14 smithi026 ceph-mon[100112]: osdmap e82: 12 total, 12 up, 12 in 2023-11-16T16:37:14.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:14 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1102501970' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-11-16T16:37:14.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:14 smithi026 ceph-mon[100112]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-11-16T16:37:14.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:14 smithi026 ceph-mon[100112]: osdmap e83: 12 total, 12 up, 12 in 2023-11-16T16:37:14.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:14 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1102501970' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-11-16T16:37:14.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:14 smithi026 ceph-mon[100112]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-11-16T16:37:14.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:14 smithi026 ceph-mon[100112]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-11-16T16:37:14.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:14 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/3786954012' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:37:14.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:14 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:37:14.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:14 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:37:14.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:14 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:37:14.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:14 smithi053 ceph-mon[100638]: Service mds.all has no related daemon already placed. 2023-11-16T16:37:14.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:14 smithi053 ceph-mon[100638]: pgmap v257: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:37:14.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:14 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:37:14.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:14 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/2372051516' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-11-16T16:37:14.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:14 smithi053 ceph-mon[100638]: osdmap e82: 12 total, 12 up, 12 in 2023-11-16T16:37:14.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:14 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1102501970' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-11-16T16:37:14.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:14 smithi053 ceph-mon[100638]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-11-16T16:37:14.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:14 smithi053 ceph-mon[100638]: osdmap e83: 12 total, 12 up, 12 in 2023-11-16T16:37:14.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:14 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1102501970' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-11-16T16:37:14.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:14 smithi053 ceph-mon[100638]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-11-16T16:37:14.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:14 smithi053 ceph-mon[100638]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-11-16T16:37:14.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:14 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/3786954012' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:37:14.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.840+0000 7f320655c700 1 -- 172.21.15.1:0/114476174 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3200100d90 msgr2=0x7f3200103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:14.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.840+0000 7f320655c700 1 --2- 172.21.15.1:0/114476174 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3200100d90 0x7f3200103170 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f31f0009990 tx=0x7f31f0009ca0 comp rx=0 tx=0).stop 2023-11-16T16:37:14.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.840+0000 7f320655c700 1 -- 172.21.15.1:0/114476174 shutdown_connections 2023-11-16T16:37:14.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.840+0000 7f320655c700 1 --2- 172.21.15.1:0/114476174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3200105fd0 0x7f32001083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:14.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.840+0000 7f320655c700 1 --2- 172.21.15.1:0/114476174 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3200100d90 0x7f3200103170 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:14.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.840+0000 7f320655c700 1 --2- 172.21.15.1:0/114476174 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f32001036b0 0x7f3200105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:14.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.840+0000 7f320655c700 1 -- 172.21.15.1:0/114476174 >> 172.21.15.1:0/114476174 conn(0x7f32000fa750 msgr2=0x7f32000fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:14.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.840+0000 7f320655c700 1 -- 172.21.15.1:0/114476174 shutdown_connections 2023-11-16T16:37:14.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.840+0000 7f320655c700 1 -- 172.21.15.1:0/114476174 wait complete. 2023-11-16T16:37:14.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.841+0000 7f320655c700 1 Processor -- start 2023-11-16T16:37:14.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.841+0000 7f320655c700 1 -- start start 2023-11-16T16:37:14.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.841+0000 7f320655c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3200100d90 0x7f3200195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:14.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.841+0000 7f320655c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32001036b0 0x7f32001964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:14.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.841+0000 7f320655c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3200105fd0 0x7f320019d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:14.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.842+0000 7f320655c700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320019db10 con 0x7f3200105fd0 2023-11-16T16:37:14.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.842+0000 7f320655c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320019dc50 con 0x7f32001036b0 2023-11-16T16:37:14.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.842+0000 7f320655c700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320019dde0 con 0x7f3200100d90 2023-11-16T16:37:14.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.842+0000 7f3204af9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3200105fd0 0x7f320019d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:14.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.842+0000 7f31ff7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32001036b0 0x7f32001964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:14.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.842+0000 7f3204af9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3200105fd0 0x7f320019d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41470/0 (socket says 172.21.15.1:41470) 2023-11-16T16:37:14.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.842+0000 7f31ff7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32001036b0 0x7f32001964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.1:35654/0 (socket says 172.21.15.1:35654) 2023-11-16T16:37:14.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.842+0000 7f3204af9700 1 -- 172.21.15.1:0/1382598123 learned_addr learned my addr 172.21.15.1:0/1382598123 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:14.847 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.842+0000 7f31fffff700 1 --2- 172.21.15.1:0/1382598123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3200100d90 0x7f3200195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:14.847 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.842+0000 7f3204af9700 1 -- 172.21.15.1:0/1382598123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3200100d90 msgr2=0x7f3200195f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:14.847 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.842+0000 7f3204af9700 1 --2- 172.21.15.1:0/1382598123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3200100d90 0x7f3200195f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:14.847 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.842+0000 7f3204af9700 1 -- 172.21.15.1:0/1382598123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32001036b0 msgr2=0x7f32001964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:14.847 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.842+0000 7f3204af9700 1 --2- 172.21.15.1:0/1382598123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32001036b0 0x7f32001964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:14.847 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.842+0000 7f3204af9700 1 -- 172.21.15.1:0/1382598123 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f00095f0 con 0x7f3200105fd0 2023-11-16T16:37:14.847 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.842+0000 7f31ff7fe700 1 --2- 172.21.15.1:0/1382598123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32001036b0 0x7f32001964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:37:14.847 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.842+0000 7f3204af9700 1 --2- 172.21.15.1:0/1382598123 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3200105fd0 0x7f320019d540 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f31f800efc0 tx=0x7f31f800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:14.847 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.842+0000 7f31fd7fa700 1 -- 172.21.15.1:0/1382598123 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f8018070 con 0x7f3200105fd0 2023-11-16T16:37:14.847 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.842+0000 7f320655c700 1 -- 172.21.15.1:0/1382598123 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f320010bbe0 con 0x7f3200105fd0 2023-11-16T16:37:14.847 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.842+0000 7f31fd7fa700 1 -- 172.21.15.1:0/1382598123 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31f800f040 con 0x7f3200105fd0 2023-11-16T16:37:14.847 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.842+0000 7f31fd7fa700 1 -- 172.21.15.1:0/1382598123 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f800cca0 con 0x7f3200105fd0 2023-11-16T16:37:14.848 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.842+0000 7f320655c700 1 -- 172.21.15.1:0/1382598123 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f320010ba50 con 0x7f3200105fd0 2023-11-16T16:37:14.848 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.843+0000 7f320655c700 1 -- 172.21.15.1:0/1382598123 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32000fc2c0 con 0x7f3200105fd0 2023-11-16T16:37:14.848 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.844+0000 7f31fd7fa700 1 -- 172.21.15.1:0/1382598123 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f31f8003a00 con 0x7f3200105fd0 2023-11-16T16:37:14.848 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.845+0000 7f31fd7fa700 1 --2- 172.21.15.1:0/1382598123 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f31e80a1150 0x7f31e80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:14.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.845+0000 7f31fffff700 1 --2- 172.21.15.1:0/1382598123 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f31e80a1150 0x7f31e80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:14.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.845+0000 7f31fd7fa700 1 -- 172.21.15.1:0/1382598123 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f31f80cc2b0 con 0x7f3200105fd0 2023-11-16T16:37:14.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.845+0000 7f31fffff700 1 --2- 172.21.15.1:0/1382598123 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f31e80a1150 0x7f31e80a3600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f31f000b680 tx=0x7f31f0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:14.851 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:14.850+0000 7f31fd7fa700 1 -- 172.21.15.1:0/1382598123 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f31f8095b40 con 0x7f3200105fd0 2023-11-16T16:37:15.104 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:15.103+0000 7f320655c700 1 -- 172.21.15.1:0/1382598123 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f320002cfe0 con 0x7f3200105fd0 2023-11-16T16:37:15.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:15 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/2125693226' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:37:15.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:15 smithi026 ceph-mon[100112]: osdmap e84: 12 total, 12 up, 12 in 2023-11-16T16:37:15.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:15 smithi026 ceph-mon[100112]: mds.? [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] up:active 2023-11-16T16:37:15.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:15 smithi026 ceph-mon[100112]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-16T16:37:15.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:15 smithi026 ceph-mon[100112]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-16T16:37:15.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:15 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1382598123' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-11-16T16:37:15.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:15 smithi026 ceph-mon[100112]: pgmap v261: 97 pgs: 46 active+clean, 6 creating+peering, 3 creating+activating, 42 unknown; 579 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 5 op/s 2023-11-16T16:37:15.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:15 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/2125693226' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:37:15.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:15 smithi053 ceph-mon[100638]: osdmap e84: 12 total, 12 up, 12 in 2023-11-16T16:37:15.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:15 smithi053 ceph-mon[100638]: mds.? [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] up:active 2023-11-16T16:37:15.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:15 smithi053 ceph-mon[100638]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-16T16:37:15.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:15 smithi053 ceph-mon[100638]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-16T16:37:15.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:15 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1382598123' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-11-16T16:37:15.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:15 smithi053 ceph-mon[100638]: pgmap v261: 97 pgs: 46 active+clean, 6 creating+peering, 3 creating+activating, 42 unknown; 579 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 5 op/s 2023-11-16T16:37:15.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:15 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/2125693226' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:37:15.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:15 smithi001 ceph-mon[98800]: osdmap e84: 12 total, 12 up, 12 in 2023-11-16T16:37:15.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:15 smithi001 ceph-mon[98800]: mds.? [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] up:active 2023-11-16T16:37:15.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:15 smithi001 ceph-mon[98800]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-16T16:37:15.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:15 smithi001 ceph-mon[98800]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-16T16:37:15.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:15 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1382598123' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-11-16T16:37:15.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:15 smithi001 ceph-mon[98800]: pgmap v261: 97 pgs: 46 active+clean, 6 creating+peering, 3 creating+activating, 42 unknown; 579 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 5 op/s 2023-11-16T16:37:15.936 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:15.935+0000 7f31fd7fa700 1 -- 172.21.15.1:0/1382598123 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f31f8099160 con 0x7f3200105fd0 2023-11-16T16:37:15.939 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:15.937+0000 7f320655c700 1 -- 172.21.15.1:0/1382598123 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f31e80a1150 msgr2=0x7f31e80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:15.939 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:15.937+0000 7f320655c700 1 --2- 172.21.15.1:0/1382598123 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f31e80a1150 0x7f31e80a3600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f31f000b680 tx=0x7f31f0005fd0 comp rx=0 tx=0).stop 2023-11-16T16:37:15.939 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:15.937+0000 7f320655c700 1 -- 172.21.15.1:0/1382598123 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3200105fd0 msgr2=0x7f320019d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:15.939 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:15.937+0000 7f320655c700 1 --2- 172.21.15.1:0/1382598123 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3200105fd0 0x7f320019d540 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f31f800efc0 tx=0x7f31f800c3a0 comp rx=0 tx=0).stop 2023-11-16T16:37:15.939 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:15.937+0000 7f320655c700 1 -- 172.21.15.1:0/1382598123 shutdown_connections 2023-11-16T16:37:15.939 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:15.937+0000 7f320655c700 1 --2- 172.21.15.1:0/1382598123 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f31e80a1150 0x7f31e80a3600 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:15.939 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:15.937+0000 7f320655c700 1 --2- 172.21.15.1:0/1382598123 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3200105fd0 0x7f320019d540 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:15.940 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:15.937+0000 7f320655c700 1 --2- 172.21.15.1:0/1382598123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3200100d90 0x7f3200195f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:15.940 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:15.937+0000 7f320655c700 1 --2- 172.21.15.1:0/1382598123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32001036b0 0x7f32001964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:15.940 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:15.937+0000 7f320655c700 1 -- 172.21.15.1:0/1382598123 >> 172.21.15.1:0/1382598123 conn(0x7f32000fa750 msgr2=0x7f32001019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:15.940 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:15.937+0000 7f320655c700 1 -- 172.21.15.1:0/1382598123 shutdown_connections 2023-11-16T16:37:15.940 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:15.937+0000 7f320655c700 1 -- 172.21.15.1:0/1382598123 wait complete. 2023-11-16T16:37:15.950 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2023-11-16T16:37:15.951 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-11-16T16:37:16.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.085+0000 7f8494829700 1 -- 172.21.15.1:0/260738418 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8490100400 msgr2=0x7f8490100870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:16.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.085+0000 7f8494829700 1 --2- 172.21.15.1:0/260738418 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8490100400 0x7f8490100870 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8480009990 tx=0x7f8480009ca0 comp rx=0 tx=0).stop 2023-11-16T16:37:16.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.085+0000 7f8494829700 1 -- 172.21.15.1:0/260738418 shutdown_connections 2023-11-16T16:37:16.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.085+0000 7f8494829700 1 --2- 172.21.15.1:0/260738418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8490100db0 0x7f8490107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:16.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.085+0000 7f8494829700 1 --2- 172.21.15.1:0/260738418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84900ff170 0x7f84900ff580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:16.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.085+0000 7f8494829700 1 --2- 172.21.15.1:0/260738418 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8490100400 0x7f8490100870 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:16.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.085+0000 7f8494829700 1 -- 172.21.15.1:0/260738418 >> 172.21.15.1:0/260738418 conn(0x7f84900fa750 msgr2=0x7f84900fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:16.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.085+0000 7f8494829700 1 -- 172.21.15.1:0/260738418 shutdown_connections 2023-11-16T16:37:16.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.085+0000 7f8494829700 1 -- 172.21.15.1:0/260738418 wait complete. 2023-11-16T16:37:16.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.086+0000 7f8494829700 1 Processor -- start 2023-11-16T16:37:16.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.086+0000 7f8494829700 1 -- start start 2023-11-16T16:37:16.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.086+0000 7f8494829700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84900ff170 0x7f8490068950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:16.091 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.086+0000 7f8494829700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8490100400 0x7f849019ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:16.091 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.086+0000 7f8494829700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8490100db0 0x7f849019d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:16.091 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.086+0000 7f8494829700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849019db30 con 0x7f84900ff170 2023-11-16T16:37:16.091 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.086+0000 7f8494829700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849019dca0 con 0x7f8490100400 2023-11-16T16:37:16.091 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.086+0000 7f8494829700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849019de60 con 0x7f8490100db0 2023-11-16T16:37:16.091 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.086+0000 7f848e59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84900ff170 0x7f8490068950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:16.091 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.086+0000 7f848e59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84900ff170 0x7f8490068950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41480/0 (socket says 172.21.15.1:41480) 2023-11-16T16:37:16.091 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.086+0000 7f848e59c700 1 -- 172.21.15.1:0/1258227699 learned_addr learned my addr 172.21.15.1:0/1258227699 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:16.092 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.086+0000 7f848ed9d700 1 --2- 172.21.15.1:0/1258227699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8490100db0 0x7f849019d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:16.092 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.086+0000 7f848dd9b700 1 --2- 172.21.15.1:0/1258227699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8490100400 0x7f849019ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:16.092 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.086+0000 7f848e59c700 1 -- 172.21.15.1:0/1258227699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8490100db0 msgr2=0x7f849019d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:16.092 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.086+0000 7f848e59c700 1 --2- 172.21.15.1:0/1258227699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8490100db0 0x7f849019d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:16.092 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.086+0000 7f848e59c700 1 -- 172.21.15.1:0/1258227699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8490100400 msgr2=0x7f849019ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:16.092 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.087+0000 7f848e59c700 1 --2- 172.21.15.1:0/1258227699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8490100400 0x7f849019ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:16.092 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.087+0000 7f848e59c700 1 -- 172.21.15.1:0/1258227699 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84800095f0 con 0x7f84900ff170 2023-11-16T16:37:16.092 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.087+0000 7f848e59c700 1 --2- 172.21.15.1:0/1258227699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84900ff170 0x7f8490068950 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f848400e9a0 tx=0x7f848400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:16.093 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.087+0000 7f847f7fe700 1 -- 172.21.15.1:0/1258227699 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8484009790 con 0x7f84900ff170 2023-11-16T16:37:16.093 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.087+0000 7f847f7fe700 1 -- 172.21.15.1:0/1258227699 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8484010430 con 0x7f84900ff170 2023-11-16T16:37:16.093 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.087+0000 7f8494829700 1 -- 172.21.15.1:0/1258227699 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f849010abc0 con 0x7f84900ff170 2023-11-16T16:37:16.093 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.087+0000 7f847f7fe700 1 -- 172.21.15.1:0/1258227699 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8484018680 con 0x7f84900ff170 2023-11-16T16:37:16.093 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.087+0000 7f8494829700 1 -- 172.21.15.1:0/1258227699 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f849010aa30 con 0x7f84900ff170 2023-11-16T16:37:16.093 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.088+0000 7f8494829700 1 -- 172.21.15.1:0/1258227699 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f849019e550 con 0x7f84900ff170 2023-11-16T16:37:16.094 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.089+0000 7f847f7fe700 1 -- 172.21.15.1:0/1258227699 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f84840105a0 con 0x7f84900ff170 2023-11-16T16:37:16.095 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.089+0000 7f847f7fe700 1 --2- 172.21.15.1:0/1258227699 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f84780a1150 0x7f84780a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:16.095 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.090+0000 7f848dd9b700 1 --2- 172.21.15.1:0/1258227699 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f84780a1150 0x7f84780a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:16.095 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.090+0000 7f847f7fe700 1 -- 172.21.15.1:0/1258227699 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8484014070 con 0x7f84900ff170 2023-11-16T16:37:16.095 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.090+0000 7f848dd9b700 1 --2- 172.21.15.1:0/1258227699 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f84780a1150 0x7f84780a3600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f84900686c0 tx=0x7f8480005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:16.095 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.093+0000 7f847f7fe700 1 -- 172.21.15.1:0/1258227699 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8484096410 con 0x7f84900ff170 2023-11-16T16:37:16.353 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:16.353+0000 7f8494829700 1 -- 172.21.15.1:0/1258227699 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f8490106140 con 0x7f84900ff170 2023-11-16T16:37:17.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:16 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1382598123' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-11-16T16:37:17.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:16 smithi026 ceph-mon[100112]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-16T16:37:17.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:16 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:17.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:16 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:17.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:16 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1258227699' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-11-16T16:37:17.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:16 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:17.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:16 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:17.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:16 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:17.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:16 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:17.260 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:16 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:37:17.260 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:16 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:37:17.260 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:16 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:17.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.274+0000 7f847f7fe700 1 -- 172.21.15.1:0/1258227699 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7f84840260b0 con 0x7f84900ff170 2023-11-16T16:37:17.277 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.276+0000 7f8494829700 1 -- 172.21.15.1:0/1258227699 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f84780a1150 msgr2=0x7f84780a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:17.277 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.276+0000 7f8494829700 1 --2- 172.21.15.1:0/1258227699 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f84780a1150 0x7f84780a3600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f84900686c0 tx=0x7f8480005fb0 comp rx=0 tx=0).stop 2023-11-16T16:37:17.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.276+0000 7f8494829700 1 -- 172.21.15.1:0/1258227699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84900ff170 msgr2=0x7f8490068950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:17.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.276+0000 7f8494829700 1 --2- 172.21.15.1:0/1258227699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84900ff170 0x7f8490068950 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f848400e9a0 tx=0x7f848400ecb0 comp rx=0 tx=0).stop 2023-11-16T16:37:17.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.276+0000 7f8494829700 1 -- 172.21.15.1:0/1258227699 shutdown_connections 2023-11-16T16:37:17.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.276+0000 7f8494829700 1 --2- 172.21.15.1:0/1258227699 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f84780a1150 0x7f84780a3600 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:17.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.276+0000 7f8494829700 1 --2- 172.21.15.1:0/1258227699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8490100db0 0x7f849019d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:17.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.276+0000 7f8494829700 1 --2- 172.21.15.1:0/1258227699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84900ff170 0x7f8490068950 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:17.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.276+0000 7f8494829700 1 --2- 172.21.15.1:0/1258227699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8490100400 0x7f849019ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:17.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.276+0000 7f8494829700 1 -- 172.21.15.1:0/1258227699 >> 172.21.15.1:0/1258227699 conn(0x7f84900fa750 msgr2=0x7f84900fca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:17.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.276+0000 7f8494829700 1 -- 172.21.15.1:0/1258227699 shutdown_connections 2023-11-16T16:37:17.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.276+0000 7f8494829700 1 -- 172.21.15.1:0/1258227699 wait complete. 2023-11-16T16:37:17.294 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:16 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1382598123' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-11-16T16:37:17.294 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:16 smithi001 ceph-mon[98800]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-16T16:37:17.294 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:16 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:17.294 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:16 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:17.294 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:16 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1258227699' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-11-16T16:37:17.294 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:16 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:17.295 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:16 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:17.295 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:16 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:17.295 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:16 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:17.295 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:16 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:37:17.295 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:16 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:37:17.295 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:16 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:17.296 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-11-16T16:37:17.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:16 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1382598123' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-11-16T16:37:17.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:16 smithi053 ceph-mon[100638]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-16T16:37:17.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:16 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:17.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:16 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:17.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:16 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1258227699' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-11-16T16:37:17.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:16 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:17.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:16 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:17.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:16 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:17.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:16 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:17.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:16 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:37:17.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:16 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:37:17.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:16 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:37:17.432 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.429+0000 7ff1f7827700 1 -- 172.21.15.1:0/2393612756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1f0100400 msgr2=0x7ff1f0100870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:17.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.429+0000 7ff1f7827700 1 --2- 172.21.15.1:0/2393612756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1f0100400 0x7ff1f0100870 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009990 tx=0x7ff1e0009ca0 comp rx=0 tx=0).stop 2023-11-16T16:37:17.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.429+0000 7ff1f7827700 1 -- 172.21.15.1:0/2393612756 shutdown_connections 2023-11-16T16:37:17.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.429+0000 7ff1f7827700 1 --2- 172.21.15.1:0/2393612756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1f0100db0 0x7ff1f0107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:17.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.429+0000 7ff1f7827700 1 --2- 172.21.15.1:0/2393612756 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1f00ff170 0x7ff1f00ff580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:17.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.429+0000 7ff1f7827700 1 --2- 172.21.15.1:0/2393612756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1f0100400 0x7ff1f0100870 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:17.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.429+0000 7ff1f7827700 1 -- 172.21.15.1:0/2393612756 >> 172.21.15.1:0/2393612756 conn(0x7ff1f00fa750 msgr2=0x7ff1f00fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:17.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.429+0000 7ff1f7827700 1 -- 172.21.15.1:0/2393612756 shutdown_connections 2023-11-16T16:37:17.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.429+0000 7ff1f7827700 1 -- 172.21.15.1:0/2393612756 wait complete. 2023-11-16T16:37:17.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.430+0000 7ff1f7827700 1 Processor -- start 2023-11-16T16:37:17.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.430+0000 7ff1f7827700 1 -- start start 2023-11-16T16:37:17.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.430+0000 7ff1f7827700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1f00ff170 0x7ff1f0195fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:17.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.430+0000 7ff1f7827700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1f0100400 0x7ff1f01964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:17.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.430+0000 7ff1f7827700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1f0100db0 0x7ff1f019d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:17.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.430+0000 7ff1f7827700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f019db20 con 0x7ff1f0100db0 2023-11-16T16:37:17.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.430+0000 7ff1f7827700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f019dc60 con 0x7ff1f00ff170 2023-11-16T16:37:17.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.430+0000 7ff1f7827700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f019ddf0 con 0x7ff1f0100400 2023-11-16T16:37:17.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.430+0000 7ff1f5dc4700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1f0100db0 0x7ff1f019d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:17.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.430+0000 7ff1f4dc2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1f0100400 0x7ff1f01964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:17.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.430+0000 7ff1f5dc4700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1f0100db0 0x7ff1f019d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41498/0 (socket says 172.21.15.1:41498) 2023-11-16T16:37:17.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.430+0000 7ff1f5dc4700 1 -- 172.21.15.1:0/3093891486 learned_addr learned my addr 172.21.15.1:0/3093891486 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:17.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.430+0000 7ff1f55c3700 1 --2- 172.21.15.1:0/3093891486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1f00ff170 0x7ff1f0195fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:17.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.430+0000 7ff1f5dc4700 1 -- 172.21.15.1:0/3093891486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1f0100400 msgr2=0x7ff1f01964e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:17.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.430+0000 7ff1f5dc4700 1 --2- 172.21.15.1:0/3093891486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1f0100400 0x7ff1f01964e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:17.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.430+0000 7ff1f5dc4700 1 -- 172.21.15.1:0/3093891486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1f00ff170 msgr2=0x7ff1f0195fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:17.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.430+0000 7ff1f5dc4700 1 --2- 172.21.15.1:0/3093891486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1f00ff170 0x7ff1f0195fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:17.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.430+0000 7ff1f5dc4700 1 -- 172.21.15.1:0/3093891486 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e00095f0 con 0x7ff1f0100db0 2023-11-16T16:37:17.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.431+0000 7ff1f5dc4700 1 --2- 172.21.15.1:0/3093891486 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1f0100db0 0x7ff1f019d550 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800b970 tx=0x7ff1e800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:17.436 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.431+0000 7ff1e67fc700 1 -- 172.21.15.1:0/3093891486 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e800d860 con 0x7ff1f0100db0 2023-11-16T16:37:17.436 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.431+0000 7ff1e67fc700 1 -- 172.21.15.1:0/3093891486 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1e800de80 con 0x7ff1f0100db0 2023-11-16T16:37:17.436 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.431+0000 7ff1f7827700 1 -- 172.21.15.1:0/3093891486 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1f010ab70 con 0x7ff1f0100db0 2023-11-16T16:37:17.436 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.431+0000 7ff1e67fc700 1 -- 172.21.15.1:0/3093891486 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8017400 con 0x7ff1f0100db0 2023-11-16T16:37:17.436 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.431+0000 7ff1f7827700 1 -- 172.21.15.1:0/3093891486 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1f010a9e0 con 0x7ff1f0100db0 2023-11-16T16:37:17.436 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.432+0000 7ff1f7827700 1 -- 172.21.15.1:0/3093891486 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1f019e540 con 0x7ff1f0100db0 2023-11-16T16:37:17.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.433+0000 7ff1e67fc700 1 -- 172.21.15.1:0/3093891486 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7ff1e800e630 con 0x7ff1f0100db0 2023-11-16T16:37:17.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.433+0000 7ff1e67fc700 1 --2- 172.21.15.1:0/3093891486 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7ff1dc0a11d0 0x7ff1dc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:17.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.434+0000 7ff1e67fc700 1 -- 172.21.15.1:0/3093891486 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff1e8006e40 con 0x7ff1f0100db0 2023-11-16T16:37:17.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.434+0000 7ff1f55c3700 1 --2- 172.21.15.1:0/3093891486 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7ff1dc0a11d0 0x7ff1dc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:17.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.434+0000 7ff1f55c3700 1 --2- 172.21.15.1:0/3093891486 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7ff1dc0a11d0 0x7ff1dc0a3680 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec005950 tx=0x7ff1ec00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:17.438 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.437+0000 7ff1e67fc700 1 -- 172.21.15.1:0/3093891486 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff1e8095e20 con 0x7ff1f0100db0 2023-11-16T16:37:17.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:17.686+0000 7ff1f7827700 1 -- 172.21.15.1:0/3093891486 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7ff1f019eaf0 con 0x7ff1f0100db0 2023-11-16T16:37:18.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:17 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:37:18.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:17 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:37:18.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:17 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:37:18.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:17 smithi026 ceph-mon[100112]: Service mds.all has no related daemon already placed. 2023-11-16T16:37:18.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:17 smithi026 ceph-mon[100112]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-11-16T16:37:18.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:17 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1258227699' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-11-16T16:37:18.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:17 smithi026 ceph-mon[100112]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-11-16T16:37:18.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:17 smithi026 ceph-mon[100112]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-16T16:37:18.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:17 smithi026 ceph-mon[100112]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-11-16T16:37:18.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:17 smithi026 ceph-mon[100112]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-11-16T16:37:18.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:17 smithi026 ceph-mon[100112]: pgmap v262: 97 pgs: 81 active+clean, 6 creating+peering, 3 creating+activating, 7 unknown; 579 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 8 op/s 2023-11-16T16:37:18.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:17 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/3093891486' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-11-16T16:37:18.283 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.282+0000 7ff1e67fc700 1 -- 172.21.15.1:0/3093891486 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7ff1e8099440 con 0x7ff1f0100db0 2023-11-16T16:37:18.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.285+0000 7ff1f7827700 1 -- 172.21.15.1:0/3093891486 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7ff1dc0a11d0 msgr2=0x7ff1dc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:18.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.285+0000 7ff1f7827700 1 --2- 172.21.15.1:0/3093891486 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7ff1dc0a11d0 0x7ff1dc0a3680 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec005950 tx=0x7ff1ec00a300 comp rx=0 tx=0).stop 2023-11-16T16:37:18.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.285+0000 7ff1f7827700 1 -- 172.21.15.1:0/3093891486 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1f0100db0 msgr2=0x7ff1f019d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:18.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.285+0000 7ff1f7827700 1 --2- 172.21.15.1:0/3093891486 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1f0100db0 0x7ff1f019d550 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800b970 tx=0x7ff1e800bc80 comp rx=0 tx=0).stop 2023-11-16T16:37:18.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.285+0000 7ff1f7827700 1 -- 172.21.15.1:0/3093891486 shutdown_connections 2023-11-16T16:37:18.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.285+0000 7ff1f7827700 1 --2- 172.21.15.1:0/3093891486 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7ff1dc0a11d0 0x7ff1dc0a3680 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:18.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.285+0000 7ff1f7827700 1 --2- 172.21.15.1:0/3093891486 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1f0100db0 0x7ff1f019d550 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:18.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.285+0000 7ff1f7827700 1 --2- 172.21.15.1:0/3093891486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1f00ff170 0x7ff1f0195fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:18.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.285+0000 7ff1f7827700 1 --2- 172.21.15.1:0/3093891486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1f0100400 0x7ff1f01964e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:18.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.285+0000 7ff1f7827700 1 -- 172.21.15.1:0/3093891486 >> 172.21.15.1:0/3093891486 conn(0x7ff1f00fa750 msgr2=0x7ff1f0067e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:18.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.285+0000 7ff1f7827700 1 -- 172.21.15.1:0/3093891486 shutdown_connections 2023-11-16T16:37:18.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.285+0000 7ff1f7827700 1 -- 172.21.15.1:0/3093891486 wait complete. 2023-11-16T16:37:18.305 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:17 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:37:18.305 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:17 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:37:18.305 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:17 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:37:18.305 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:17 smithi001 ceph-mon[98800]: Service mds.all has no related daemon already placed. 2023-11-16T16:37:18.305 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:17 smithi001 ceph-mon[98800]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-11-16T16:37:18.305 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:17 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1258227699' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-11-16T16:37:18.306 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:17 smithi001 ceph-mon[98800]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-11-16T16:37:18.306 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:17 smithi001 ceph-mon[98800]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-16T16:37:18.306 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:17 smithi001 ceph-mon[98800]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-11-16T16:37:18.306 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:17 smithi001 ceph-mon[98800]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-11-16T16:37:18.306 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:17 smithi001 ceph-mon[98800]: pgmap v262: 97 pgs: 81 active+clean, 6 creating+peering, 3 creating+activating, 7 unknown; 579 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 8 op/s 2023-11-16T16:37:18.306 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:17 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3093891486' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-11-16T16:37:18.307 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-11-16T16:37:18.330 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:17 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:37:18.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:17 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:37:18.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:17 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:37:18.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:17 smithi053 ceph-mon[100638]: Service mds.all has no related daemon already placed. 2023-11-16T16:37:18.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:17 smithi053 ceph-mon[100638]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-11-16T16:37:18.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:17 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1258227699' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-11-16T16:37:18.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:17 smithi053 ceph-mon[100638]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-11-16T16:37:18.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:17 smithi053 ceph-mon[100638]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-16T16:37:18.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:17 smithi053 ceph-mon[100638]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-11-16T16:37:18.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:17 smithi053 ceph-mon[100638]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-11-16T16:37:18.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:17 smithi053 ceph-mon[100638]: pgmap v262: 97 pgs: 81 active+clean, 6 creating+peering, 3 creating+activating, 7 unknown; 579 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 8 op/s 2023-11-16T16:37:18.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:17 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/3093891486' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-11-16T16:37:18.440 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.437+0000 7f6167359700 1 -- 172.21.15.1:0/3699565758 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6160100380 msgr2=0x7f61601007d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:18.440 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.437+0000 7f6167359700 1 --2- 172.21.15.1:0/3699565758 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6160100380 0x7f61601007d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6150009990 tx=0x7f6150009ca0 comp rx=0 tx=0).stop 2023-11-16T16:37:18.440 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.437+0000 7f6167359700 1 -- 172.21.15.1:0/3699565758 shutdown_connections 2023-11-16T16:37:18.440 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.437+0000 7f6167359700 1 --2- 172.21.15.1:0/3699565758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6160100d10 0x7f61601075d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:18.440 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.437+0000 7f6167359700 1 --2- 172.21.15.1:0/3699565758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61600ff190 0x7f61600ff5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:18.440 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.437+0000 7f6167359700 1 --2- 172.21.15.1:0/3699565758 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6160100380 0x7f61601007d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:18.440 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.437+0000 7f6167359700 1 -- 172.21.15.1:0/3699565758 >> 172.21.15.1:0/3699565758 conn(0x7f61600fa770 msgr2=0x7f61600fcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:18.441 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.437+0000 7f6167359700 1 -- 172.21.15.1:0/3699565758 shutdown_connections 2023-11-16T16:37:18.441 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.437+0000 7f6167359700 1 -- 172.21.15.1:0/3699565758 wait complete. 2023-11-16T16:37:18.441 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.438+0000 7f6167359700 1 Processor -- start 2023-11-16T16:37:18.441 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.438+0000 7f6167359700 1 -- start start 2023-11-16T16:37:18.441 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.438+0000 7f6167359700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61600ff190 0x7f6160194f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:18.441 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.438+0000 7f6167359700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6160100380 0x7f6160195450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:18.441 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.438+0000 7f6167359700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6160100d10 0x7f616019c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:18.441 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.438+0000 7f6167359700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616019ca90 con 0x7f6160100d10 2023-11-16T16:37:18.441 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.438+0000 7f6167359700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616019cbd0 con 0x7f61600ff190 2023-11-16T16:37:18.441 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.438+0000 7f6167359700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616019cd60 con 0x7f6160100380 2023-11-16T16:37:18.441 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.438+0000 7f61658f6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6160100d10 0x7f616019c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:18.442 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.438+0000 7f61658f6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6160100d10 0x7f616019c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41512/0 (socket says 172.21.15.1:41512) 2023-11-16T16:37:18.442 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.438+0000 7f61658f6700 1 -- 172.21.15.1:0/1048054619 learned_addr learned my addr 172.21.15.1:0/1048054619 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:18.442 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.438+0000 7f61648f4700 1 --2- 172.21.15.1:0/1048054619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6160100380 0x7f6160195450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:18.442 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.438+0000 7f61650f5700 1 --2- 172.21.15.1:0/1048054619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61600ff190 0x7f6160194f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:18.442 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.438+0000 7f61658f6700 1 -- 172.21.15.1:0/1048054619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6160100380 msgr2=0x7f6160195450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:18.442 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.438+0000 7f61658f6700 1 --2- 172.21.15.1:0/1048054619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6160100380 0x7f6160195450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:18.442 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.438+0000 7f61658f6700 1 -- 172.21.15.1:0/1048054619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61600ff190 msgr2=0x7f6160194f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:18.442 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.438+0000 7f61658f6700 1 --2- 172.21.15.1:0/1048054619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61600ff190 0x7f6160194f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:18.442 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.438+0000 7f61658f6700 1 -- 172.21.15.1:0/1048054619 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61500095f0 con 0x7f6160100d10 2023-11-16T16:37:18.442 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.439+0000 7f61658f6700 1 --2- 172.21.15.1:0/1048054619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6160100d10 0x7f616019c4c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f615800ada0 tx=0x7f615800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:18.442 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.439+0000 7f61567fc700 1 -- 172.21.15.1:0/1048054619 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615800f070 con 0x7f6160100d10 2023-11-16T16:37:18.442 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.439+0000 7f61567fc700 1 -- 172.21.15.1:0/1048054619 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6158007b30 con 0x7f6160100d10 2023-11-16T16:37:18.443 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.439+0000 7f6167359700 1 -- 172.21.15.1:0/1048054619 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f616010ab50 con 0x7f6160100d10 2023-11-16T16:37:18.443 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.439+0000 7f61567fc700 1 -- 172.21.15.1:0/1048054619 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6158005280 con 0x7f6160100d10 2023-11-16T16:37:18.443 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.439+0000 7f6167359700 1 -- 172.21.15.1:0/1048054619 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f616010a9c0 con 0x7f6160100d10 2023-11-16T16:37:18.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.440+0000 7f6167359700 1 -- 172.21.15.1:0/1048054619 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6160106170 con 0x7f6160100d10 2023-11-16T16:37:18.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.441+0000 7f61567fc700 1 -- 172.21.15.1:0/1048054619 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f615800a090 con 0x7f6160100d10 2023-11-16T16:37:18.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.441+0000 7f61567fc700 1 --2- 172.21.15.1:0/1048054619 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f614c0a1150 0x7f614c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:18.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.442+0000 7f61650f5700 1 --2- 172.21.15.1:0/1048054619 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f614c0a1150 0x7f614c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:18.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.442+0000 7f61567fc700 1 -- 172.21.15.1:0/1048054619 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f61580cc260 con 0x7f6160100d10 2023-11-16T16:37:18.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.442+0000 7f61650f5700 1 --2- 172.21.15.1:0/1048054619 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f614c0a1150 0x7f614c0a3600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f61601001e0 tx=0x7f615c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:18.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.445+0000 7f61567fc700 1 -- 172.21.15.1:0/1048054619 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6158099620 con 0x7f6160100d10 2023-11-16T16:37:18.693 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:18.692+0000 7f6167359700 1 -- 172.21.15.1:0/1048054619 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f61600619b0 con 0x7f6160100d10 2023-11-16T16:37:19.293 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.292+0000 7f61567fc700 1 -- 172.21.15.1:0/1048054619 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7f61580991b0 con 0x7f6160100d10 2023-11-16T16:37:19.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.296+0000 7f6167359700 1 -- 172.21.15.1:0/1048054619 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f614c0a1150 msgr2=0x7f614c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:19.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.296+0000 7f6167359700 1 --2- 172.21.15.1:0/1048054619 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f614c0a1150 0x7f614c0a3600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f61601001e0 tx=0x7f615c00a380 comp rx=0 tx=0).stop 2023-11-16T16:37:19.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.296+0000 7f6167359700 1 -- 172.21.15.1:0/1048054619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6160100d10 msgr2=0x7f616019c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:19.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.296+0000 7f6167359700 1 --2- 172.21.15.1:0/1048054619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6160100d10 0x7f616019c4c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f615800ada0 tx=0x7f615800c330 comp rx=0 tx=0).stop 2023-11-16T16:37:19.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.296+0000 7f6167359700 1 -- 172.21.15.1:0/1048054619 shutdown_connections 2023-11-16T16:37:19.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.296+0000 7f6167359700 1 --2- 172.21.15.1:0/1048054619 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f614c0a1150 0x7f614c0a3600 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:19.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.296+0000 7f6167359700 1 --2- 172.21.15.1:0/1048054619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6160100d10 0x7f616019c4c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:19.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.296+0000 7f6167359700 1 --2- 172.21.15.1:0/1048054619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61600ff190 0x7f6160194f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:19.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.296+0000 7f6167359700 1 --2- 172.21.15.1:0/1048054619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6160100380 0x7f6160195450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:19.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.296+0000 7f6167359700 1 -- 172.21.15.1:0/1048054619 >> 172.21.15.1:0/1048054619 conn(0x7f61600fa770 msgr2=0x7f61600fcac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:19.298 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.296+0000 7f6167359700 1 -- 172.21.15.1:0/1048054619 shutdown_connections 2023-11-16T16:37:19.298 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.296+0000 7f6167359700 1 -- 172.21.15.1:0/1048054619 wait complete. 2023-11-16T16:37:19.312 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2023-11-16T16:37:19.312 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-11-16T16:37:19.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.447+0000 7fd45ce15700 1 -- 172.21.15.1:0/506274425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4580ff190 msgr2=0x7fd4580ff5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:19.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.447+0000 7fd45ce15700 1 --2- 172.21.15.1:0/506274425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4580ff190 0x7fd4580ff5a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd448009990 tx=0x7fd448009ca0 comp rx=0 tx=0).stop 2023-11-16T16:37:19.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.448+0000 7fd45ce15700 1 -- 172.21.15.1:0/506274425 shutdown_connections 2023-11-16T16:37:19.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.448+0000 7fd45ce15700 1 --2- 172.21.15.1:0/506274425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd458100d10 0x7fd4581075d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:19.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.448+0000 7fd45ce15700 1 --2- 172.21.15.1:0/506274425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4580ff190 0x7fd4580ff5a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:19.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.448+0000 7fd45ce15700 1 --2- 172.21.15.1:0/506274425 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd458100380 0x7fd4581007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:19.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.448+0000 7fd45ce15700 1 -- 172.21.15.1:0/506274425 >> 172.21.15.1:0/506274425 conn(0x7fd4580fa770 msgr2=0x7fd4580fcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:19.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.448+0000 7fd45ce15700 1 -- 172.21.15.1:0/506274425 shutdown_connections 2023-11-16T16:37:19.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.448+0000 7fd45ce15700 1 -- 172.21.15.1:0/506274425 wait complete. 2023-11-16T16:37:19.451 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.448+0000 7fd45ce15700 1 Processor -- start 2023-11-16T16:37:19.451 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.448+0000 7fd45ce15700 1 -- start start 2023-11-16T16:37:19.451 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.449+0000 7fd45ce15700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4580ff190 0x7fd458069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:19.451 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.449+0000 7fd45ce15700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd458100380 0x7fd45819acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:19.451 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.449+0000 7fd45ce15700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd458100d10 0x7fd45819d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:19.451 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.449+0000 7fd45ce15700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45819db20 con 0x7fd4580ff190 2023-11-16T16:37:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.449+0000 7fd45ce15700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45819dc90 con 0x7fd458100380 2023-11-16T16:37:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.449+0000 7fd45ce15700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45819de50 con 0x7fd458100d10 2023-11-16T16:37:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.449+0000 7fd45659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4580ff190 0x7fd458069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.449+0000 7fd45659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4580ff190 0x7fd458069a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41522/0 (socket says 172.21.15.1:41522) 2023-11-16T16:37:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.449+0000 7fd45659c700 1 -- 172.21.15.1:0/2669759351 learned_addr learned my addr 172.21.15.1:0/2669759351 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.449+0000 7fd456d9d700 1 --2- 172.21.15.1:0/2669759351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd458100d10 0x7fd45819d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.449+0000 7fd455d9b700 1 --2- 172.21.15.1:0/2669759351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd458100380 0x7fd45819acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.449+0000 7fd45659c700 1 -- 172.21.15.1:0/2669759351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd458100d10 msgr2=0x7fd45819d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.449+0000 7fd45659c700 1 --2- 172.21.15.1:0/2669759351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd458100d10 0x7fd45819d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.449+0000 7fd45659c700 1 -- 172.21.15.1:0/2669759351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd458100380 msgr2=0x7fd45819acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.449+0000 7fd45659c700 1 --2- 172.21.15.1:0/2669759351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd458100380 0x7fd45819acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.449+0000 7fd45659c700 1 -- 172.21.15.1:0/2669759351 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4480095f0 con 0x7fd4580ff190 2023-11-16T16:37:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.449+0000 7fd456d9d700 1 --2- 172.21.15.1:0/2669759351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd458100d10 0x7fd45819d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:37:19.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.449+0000 7fd45659c700 1 --2- 172.21.15.1:0/2669759351 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4580ff190 0x7fd458069a10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd448000c00 tx=0x7fd448009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:19.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.449+0000 7fd4477fe700 1 -- 172.21.15.1:0/2669759351 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44801d070 con 0x7fd4580ff190 2023-11-16T16:37:19.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.449+0000 7fd4477fe700 1 -- 172.21.15.1:0/2669759351 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd448004180 con 0x7fd4580ff190 2023-11-16T16:37:19.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.449+0000 7fd45ce15700 1 -- 172.21.15.1:0/2669759351 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd45810aaf0 con 0x7fd4580ff190 2023-11-16T16:37:19.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.449+0000 7fd4477fe700 1 -- 172.21.15.1:0/2669759351 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd448022e00 con 0x7fd4580ff190 2023-11-16T16:37:19.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.449+0000 7fd45ce15700 1 -- 172.21.15.1:0/2669759351 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd45810a960 con 0x7fd4580ff190 2023-11-16T16:37:19.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.450+0000 7fd45ce15700 1 -- 172.21.15.1:0/2669759351 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd45819e4d0 con 0x7fd4580ff190 2023-11-16T16:37:19.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.451+0000 7fd4477fe700 1 -- 172.21.15.1:0/2669759351 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fd448022500 con 0x7fd4580ff190 2023-11-16T16:37:19.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.452+0000 7fd4477fe700 1 --2- 172.21.15.1:0/2669759351 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd4400a11d0 0x7fd4400a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:19.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.452+0000 7fd455d9b700 1 --2- 172.21.15.1:0/2669759351 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd4400a11d0 0x7fd4400a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:19.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.452+0000 7fd4477fe700 1 -- 172.21.15.1:0/2669759351 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd4480cde30 con 0x7fd4580ff190 2023-11-16T16:37:19.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.452+0000 7fd455d9b700 1 --2- 172.21.15.1:0/2669759351 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd4400a11d0 0x7fd4400a3680 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd44c005950 tx=0x7fd44c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:19.456 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.455+0000 7fd4477fe700 1 -- 172.21.15.1:0/2669759351 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd4480976c0 con 0x7fd4580ff190 2023-11-16T16:37:19.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:19 smithi026 ceph-mon[100112]: mds.? [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] up:active 2023-11-16T16:37:19.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:19 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/3093891486' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-11-16T16:37:19.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:19 smithi026 ceph-mon[100112]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-11-16T16:37:19.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:19 smithi026 ceph-mon[100112]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-11-16T16:37:19.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:19 smithi026 ceph-mon[100112]: Cluster is now healthy 2023-11-16T16:37:19.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:19 smithi026 ceph-mon[100112]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-11-16T16:37:19.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:19 smithi026 ceph-mon[100112]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2023-11-16T16:37:19.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:19 smithi026 ceph-mon[100112]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-11-16T16:37:19.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:19 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1048054619' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-11-16T16:37:19.509 INFO:journalctl@ceph.mds.b.smithi026.stdout:Nov 16 16:37:19 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-b[125397]: 2023-11-16T16:37:19.251+0000 7f1f86e0f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-16T16:37:19.509 INFO:journalctl@ceph.mds.b.smithi026.stdout:Nov 16 16:37:19 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-b[125397]: 2023-11-16T16:37:19.251+0000 7f1f86e0f700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-16T16:37:19.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:19 smithi053 ceph-mon[100638]: mds.? [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] up:active 2023-11-16T16:37:19.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:19 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/3093891486' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-11-16T16:37:19.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:19 smithi053 ceph-mon[100638]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-11-16T16:37:19.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:19 smithi053 ceph-mon[100638]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-11-16T16:37:19.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:19 smithi053 ceph-mon[100638]: Cluster is now healthy 2023-11-16T16:37:19.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:19 smithi053 ceph-mon[100638]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-11-16T16:37:19.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:19 smithi053 ceph-mon[100638]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2023-11-16T16:37:19.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:19 smithi053 ceph-mon[100638]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-11-16T16:37:19.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:19 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1048054619' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-11-16T16:37:19.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:19 smithi001 ceph-mon[98800]: mds.? [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] up:active 2023-11-16T16:37:19.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:19 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3093891486' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-11-16T16:37:19.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:19 smithi001 ceph-mon[98800]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-11-16T16:37:19.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:19 smithi001 ceph-mon[98800]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-11-16T16:37:19.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:19 smithi001 ceph-mon[98800]: Cluster is now healthy 2023-11-16T16:37:19.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:19 smithi001 ceph-mon[98800]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-11-16T16:37:19.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:19 smithi001 ceph-mon[98800]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2023-11-16T16:37:19.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:19 smithi001 ceph-mon[98800]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-11-16T16:37:19.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:19 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1048054619' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-11-16T16:37:19.691 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.690+0000 7fd45ce15700 1 -- 172.21.15.1:0/2669759351 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7fd458060f30 con 0x7fd4400a11d0 2023-11-16T16:37:19.738 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.737+0000 7fd4477fe700 1 -- 172.21.15.1:0/2669759351 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd458060f30 con 0x7fd4400a11d0 2023-11-16T16:37:19.740 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.739+0000 7fd45ce15700 1 -- 172.21.15.1:0/2669759351 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd4400a11d0 msgr2=0x7fd4400a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:19.740 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.739+0000 7fd45ce15700 1 --2- 172.21.15.1:0/2669759351 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd4400a11d0 0x7fd4400a3680 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd44c005950 tx=0x7fd44c0058e0 comp rx=0 tx=0).stop 2023-11-16T16:37:19.740 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.739+0000 7fd45ce15700 1 -- 172.21.15.1:0/2669759351 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4580ff190 msgr2=0x7fd458069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:19.740 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.739+0000 7fd45ce15700 1 --2- 172.21.15.1:0/2669759351 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4580ff190 0x7fd458069a10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd448000c00 tx=0x7fd448009f90 comp rx=0 tx=0).stop 2023-11-16T16:37:19.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.739+0000 7fd45ce15700 1 -- 172.21.15.1:0/2669759351 shutdown_connections 2023-11-16T16:37:19.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.739+0000 7fd45ce15700 1 --2- 172.21.15.1:0/2669759351 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd4400a11d0 0x7fd4400a3680 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:19.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.739+0000 7fd45ce15700 1 --2- 172.21.15.1:0/2669759351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd458100d10 0x7fd45819d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:19.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.739+0000 7fd45ce15700 1 --2- 172.21.15.1:0/2669759351 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4580ff190 0x7fd458069a10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:19.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.739+0000 7fd45ce15700 1 --2- 172.21.15.1:0/2669759351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd458100380 0x7fd45819acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:19.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.739+0000 7fd45ce15700 1 -- 172.21.15.1:0/2669759351 >> 172.21.15.1:0/2669759351 conn(0x7fd4580fa770 msgr2=0x7fd4580fcaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:19.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.739+0000 7fd45ce15700 1 -- 172.21.15.1:0/2669759351 shutdown_connections 2023-11-16T16:37:19.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.739+0000 7fd45ce15700 1 -- 172.21.15.1:0/2669759351 wait complete. 2023-11-16T16:37:19.752 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-11-16T16:37:19.881 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.880+0000 7f713f5f7700 1 -- 172.21.15.1:0/251951057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7138105e50 msgr2=0x7f7138108300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:19.881 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.880+0000 7f713f5f7700 1 --2- 172.21.15.1:0/251951057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7138105e50 0x7f7138108300 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f713000b0a0 tx=0x7f713000b3b0 comp rx=0 tx=0).stop 2023-11-16T16:37:19.881 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.880+0000 7f713f5f7700 1 -- 172.21.15.1:0/251951057 shutdown_connections 2023-11-16T16:37:19.881 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.880+0000 7f713f5f7700 1 --2- 172.21.15.1:0/251951057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7138105e50 0x7f7138108300 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:19.881 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.880+0000 7f713f5f7700 1 --2- 172.21.15.1:0/251951057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7138068b00 0x7f7138102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:19.881 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.880+0000 7f713f5f7700 1 --2- 172.21.15.1:0/251951057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7138103340 0x7f71381057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:19.881 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.880+0000 7f713f5f7700 1 -- 172.21.15.1:0/251951057 >> 172.21.15.1:0/251951057 conn(0x7f71380fa750 msgr2=0x7f71380fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:19.881 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.880+0000 7f713f5f7700 1 -- 172.21.15.1:0/251951057 shutdown_connections 2023-11-16T16:37:19.881 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.880+0000 7f713f5f7700 1 -- 172.21.15.1:0/251951057 wait complete. 2023-11-16T16:37:19.884 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.881+0000 7f713f5f7700 1 Processor -- start 2023-11-16T16:37:19.884 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.881+0000 7f713f5f7700 1 -- start start 2023-11-16T16:37:19.884 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.881+0000 7f713f5f7700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7138068b00 0x7f7138196090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:19.884 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.881+0000 7f713f5f7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7138103340 0x7f71381965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:19.884 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.881+0000 7f713f5f7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7138105e50 0x7f713819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:19.884 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.881+0000 7f713f5f7700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713819dc10 con 0x7f7138068b00 2023-11-16T16:37:19.884 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.881+0000 7f713f5f7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713819dd50 con 0x7f7138105e50 2023-11-16T16:37:19.884 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.881+0000 7f713f5f7700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713819dee0 con 0x7f7138103340 2023-11-16T16:37:19.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.881+0000 7f713d393700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7138068b00 0x7f7138196090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:19.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.881+0000 7f713d393700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7138068b00 0x7f7138196090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41554/0 (socket says 172.21.15.1:41554) 2023-11-16T16:37:19.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.881+0000 7f713d393700 1 -- 172.21.15.1:0/3868512023 learned_addr learned my addr 172.21.15.1:0/3868512023 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:19.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.881+0000 7f713db94700 1 --2- 172.21.15.1:0/3868512023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7138105e50 0x7f713819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:19.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.881+0000 7f713cb92700 1 --2- 172.21.15.1:0/3868512023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7138103340 0x7f71381965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:19.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.881+0000 7f713d393700 1 -- 172.21.15.1:0/3868512023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7138103340 msgr2=0x7f71381965d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:19.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.881+0000 7f713d393700 1 --2- 172.21.15.1:0/3868512023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7138103340 0x7f71381965d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:19.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.881+0000 7f713d393700 1 -- 172.21.15.1:0/3868512023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7138105e50 msgr2=0x7f713819d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:19.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.881+0000 7f713d393700 1 --2- 172.21.15.1:0/3868512023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7138105e50 0x7f713819d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:19.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.881+0000 7f713d393700 1 -- 172.21.15.1:0/3868512023 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7130009d00 con 0x7f7138068b00 2023-11-16T16:37:19.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.881+0000 7f713cb92700 1 --2- 172.21.15.1:0/3868512023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7138103340 0x7f71381965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:37:19.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.881+0000 7f713db94700 1 --2- 172.21.15.1:0/3868512023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7138105e50 0x7f713819d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:37:19.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.881+0000 7f713d393700 1 --2- 172.21.15.1:0/3868512023 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7138068b00 0x7f7138196090 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f712800e970 tx=0x7f712800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:19.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.882+0000 7f712e7fc700 1 -- 172.21.15.1:0/3868512023 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7128009790 con 0x7f7138068b00 2023-11-16T16:37:19.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.882+0000 7f713f5f7700 1 -- 172.21.15.1:0/3868512023 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f713810bbf0 con 0x7f7138068b00 2023-11-16T16:37:19.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.882+0000 7f712e7fc700 1 -- 172.21.15.1:0/3868512023 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7128004cf0 con 0x7f7138068b00 2023-11-16T16:37:19.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.882+0000 7f712e7fc700 1 -- 172.21.15.1:0/3868512023 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7128005680 con 0x7f7138068b00 2023-11-16T16:37:19.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.882+0000 7f713f5f7700 1 -- 172.21.15.1:0/3868512023 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f713810ba60 con 0x7f7138068b00 2023-11-16T16:37:19.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.883+0000 7f713f5f7700 1 -- 172.21.15.1:0/3868512023 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71380fc2c0 con 0x7f7138068b00 2023-11-16T16:37:19.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.884+0000 7f712e7fc700 1 -- 172.21.15.1:0/3868512023 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f7128004750 con 0x7f7138068b00 2023-11-16T16:37:19.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.884+0000 7f712e7fc700 1 --2- 172.21.15.1:0/3868512023 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f71240a1150 0x7f71240a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:19.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.884+0000 7f712e7fc700 1 -- 172.21.15.1:0/3868512023 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7128014070 con 0x7f7138068b00 2023-11-16T16:37:19.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.884+0000 7f713cb92700 1 --2- 172.21.15.1:0/3868512023 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f71240a1150 0x7f71240a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:19.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.885+0000 7f713cb92700 1 --2- 172.21.15.1:0/3868512023 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f71240a1150 0x7f71240a3600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7134005d90 tx=0x7f7134005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:19.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:19.888+0000 7f712e7fc700 1 -- 172.21.15.1:0/3868512023 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7128096550 con 0x7f7138068b00 2023-11-16T16:37:20.121 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.120+0000 7f713f5f7700 1 -- 172.21.15.1:0/3868512023 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f7138061c30 con 0x7f71240a1150 2023-11-16T16:37:20.126 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.126+0000 7f712e7fc700 1 -- 172.21.15.1:0/3868512023 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f7138061c30 con 0x7f71240a1150 2023-11-16T16:37:20.126 INFO:teuthology.orchestra.run.smithi001.stdout:/volumes/_nogroup/sv_0/a0675134-da9d-4657-a177-0d59647594e3 2023-11-16T16:37:20.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.128+0000 7f713f5f7700 1 -- 172.21.15.1:0/3868512023 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f71240a1150 msgr2=0x7f71240a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:20.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.128+0000 7f713f5f7700 1 --2- 172.21.15.1:0/3868512023 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f71240a1150 0x7f71240a3600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7134005d90 tx=0x7f7134005d00 comp rx=0 tx=0).stop 2023-11-16T16:37:20.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.128+0000 7f713f5f7700 1 -- 172.21.15.1:0/3868512023 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7138068b00 msgr2=0x7f7138196090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:20.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.128+0000 7f713f5f7700 1 --2- 172.21.15.1:0/3868512023 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7138068b00 0x7f7138196090 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f712800e970 tx=0x7f712800ed30 comp rx=0 tx=0).stop 2023-11-16T16:37:20.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.128+0000 7f713f5f7700 1 -- 172.21.15.1:0/3868512023 shutdown_connections 2023-11-16T16:37:20.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.128+0000 7f713f5f7700 1 --2- 172.21.15.1:0/3868512023 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f71240a1150 0x7f71240a3600 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:20.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.128+0000 7f713f5f7700 1 --2- 172.21.15.1:0/3868512023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7138105e50 0x7f713819d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:20.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.128+0000 7f713f5f7700 1 --2- 172.21.15.1:0/3868512023 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7138068b00 0x7f7138196090 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:20.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.128+0000 7f713f5f7700 1 --2- 172.21.15.1:0/3868512023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7138103340 0x7f71381965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:20.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.128+0000 7f713f5f7700 1 -- 172.21.15.1:0/3868512023 >> 172.21.15.1:0/3868512023 conn(0x7f71380fa750 msgr2=0x7f7138101680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:20.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.128+0000 7f713f5f7700 1 -- 172.21.15.1:0/3868512023 shutdown_connections 2023-11-16T16:37:20.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.128+0000 7f713f5f7700 1 -- 172.21.15.1:0/3868512023 wait complete. 2023-11-16T16:37:20.140 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:37:19 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:37:19.695+0000 7f936b448700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-16T16:37:20.140 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:37:19 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:37:19.695+0000 7f936b448700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-16T16:37:20.140 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:37:19 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:37:19.695+0000 7f936b448700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-16T16:37:20.140 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:37:19 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:37:19.695+0000 7f936b448700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-16T16:37:20.140 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:37:19 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:37:19.695+0000 7f936b448700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-16T16:37:20.141 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-11-16T16:37:20.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.270+0000 7f463518e700 1 -- 172.21.15.1:0/3289009600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46300ff170 msgr2=0x7f46300ff580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:20.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.270+0000 7f463518e700 1 --2- 172.21.15.1:0/3289009600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46300ff170 0x7f46300ff580 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4618009990 tx=0x7f4618009ca0 comp rx=0 tx=0).stop 2023-11-16T16:37:20.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.270+0000 7f463518e700 1 -- 172.21.15.1:0/3289009600 shutdown_connections 2023-11-16T16:37:20.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.270+0000 7f463518e700 1 --2- 172.21.15.1:0/3289009600 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4630100db0 0x7f4630107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:20.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.270+0000 7f463518e700 1 --2- 172.21.15.1:0/3289009600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46300ff170 0x7f46300ff580 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:20.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.270+0000 7f463518e700 1 --2- 172.21.15.1:0/3289009600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4630100400 0x7f4630100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:20.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.270+0000 7f463518e700 1 -- 172.21.15.1:0/3289009600 >> 172.21.15.1:0/3289009600 conn(0x7f46300fa750 msgr2=0x7f46300fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:20.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.270+0000 7f463518e700 1 -- 172.21.15.1:0/3289009600 shutdown_connections 2023-11-16T16:37:20.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.270+0000 7f463518e700 1 -- 172.21.15.1:0/3289009600 wait complete. 2023-11-16T16:37:20.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.271+0000 7f463518e700 1 Processor -- start 2023-11-16T16:37:20.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.271+0000 7f463518e700 1 -- start start 2023-11-16T16:37:20.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.272+0000 7f463518e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46300ff170 0x7f4630068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:20.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.272+0000 7f463518e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4630100400 0x7f463019aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:20.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.272+0000 7f463518e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4630100db0 0x7f463019d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:20.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.272+0000 7f463518e700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463019da80 con 0x7f46300ff170 2023-11-16T16:37:20.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.272+0000 7f463518e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463019dbf0 con 0x7f4630100400 2023-11-16T16:37:20.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.272+0000 7f463518e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463019ddb0 con 0x7f4630100db0 2023-11-16T16:37:20.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.272+0000 7f462ed9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46300ff170 0x7f4630068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:20.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.272+0000 7f462ed9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46300ff170 0x7f4630068930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41568/0 (socket says 172.21.15.1:41568) 2023-11-16T16:37:20.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.272+0000 7f462ed9d700 1 -- 172.21.15.1:0/523430664 learned_addr learned my addr 172.21.15.1:0/523430664 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:20.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.272+0000 7f462e59c700 1 --2- 172.21.15.1:0/523430664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4630100400 0x7f463019aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:20.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.272+0000 7f462f59e700 1 --2- 172.21.15.1:0/523430664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4630100db0 0x7f463019d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:20.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.272+0000 7f462ed9d700 1 -- 172.21.15.1:0/523430664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4630100db0 msgr2=0x7f463019d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:20.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.272+0000 7f462ed9d700 1 --2- 172.21.15.1:0/523430664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4630100db0 0x7f463019d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:20.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.272+0000 7f462ed9d700 1 -- 172.21.15.1:0/523430664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4630100400 msgr2=0x7f463019aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:20.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.272+0000 7f462ed9d700 1 --2- 172.21.15.1:0/523430664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4630100400 0x7f463019aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:20.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.272+0000 7f462ed9d700 1 -- 172.21.15.1:0/523430664 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46180095f0 con 0x7f46300ff170 2023-11-16T16:37:20.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.272+0000 7f462e59c700 1 --2- 172.21.15.1:0/523430664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4630100400 0x7f463019aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:37:20.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.272+0000 7f462ed9d700 1 --2- 172.21.15.1:0/523430664 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46300ff170 0x7f4630068930 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4618005910 tx=0x7f461800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:20.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.272+0000 7f4627fff700 1 -- 172.21.15.1:0/523430664 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461801d070 con 0x7f46300ff170 2023-11-16T16:37:20.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.273+0000 7f463518e700 1 -- 172.21.15.1:0/523430664 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f463010ab40 con 0x7f46300ff170 2023-11-16T16:37:20.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.273+0000 7f4627fff700 1 -- 172.21.15.1:0/523430664 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f461800fc30 con 0x7f46300ff170 2023-11-16T16:37:20.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.273+0000 7f4627fff700 1 -- 172.21.15.1:0/523430664 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4618017650 con 0x7f46300ff170 2023-11-16T16:37:20.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.273+0000 7f463518e700 1 -- 172.21.15.1:0/523430664 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f463010a9b0 con 0x7f46300ff170 2023-11-16T16:37:20.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.273+0000 7f463518e700 1 -- 172.21.15.1:0/523430664 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f463019e350 con 0x7f46300ff170 2023-11-16T16:37:20.277 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.275+0000 7f4627fff700 1 -- 172.21.15.1:0/523430664 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f4618022470 con 0x7f46300ff170 2023-11-16T16:37:20.277 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.276+0000 7f4627fff700 1 --2- 172.21.15.1:0/523430664 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f461c0a1150 0x7f461c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:20.277 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.276+0000 7f462e59c700 1 --2- 172.21.15.1:0/523430664 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f461c0a1150 0x7f461c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:20.277 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.276+0000 7f4627fff700 1 -- 172.21.15.1:0/523430664 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f46180ce050 con 0x7f46300ff170 2023-11-16T16:37:20.277 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.276+0000 7f462e59c700 1 --2- 172.21.15.1:0/523430664 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f461c0a1150 0x7f461c0a3600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4620005fd0 tx=0x7f4620005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:20.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.279+0000 7f4627fff700 1 -- 172.21.15.1:0/523430664 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4618097a10 con 0x7f46300ff170 2023-11-16T16:37:20.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:20 smithi001 ceph-mon[98800]: mds.? [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] up:active 2023-11-16T16:37:20.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:20 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1048054619' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-11-16T16:37:20.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:20 smithi001 ceph-mon[98800]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2023-11-16T16:37:20.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:20 smithi001 ceph-mon[98800]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:37:20.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:20 smithi001 ceph-mon[98800]: pgmap v263: 97 pgs: 97 active+clean; 579 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s wr, 12 op/s 2023-11-16T16:37:20.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.511+0000 7f463518e700 1 -- 172.21.15.1:0/523430664 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f463019e990 con 0x7f461c0a1150 2023-11-16T16:37:20.542 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.542+0000 7f4627fff700 1 -- 172.21.15.1:0/523430664 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f463019e990 con 0x7f461c0a1150 2023-11-16T16:37:20.545 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.544+0000 7f463518e700 1 -- 172.21.15.1:0/523430664 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f461c0a1150 msgr2=0x7f461c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:20.545 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.544+0000 7f463518e700 1 --2- 172.21.15.1:0/523430664 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f461c0a1150 0x7f461c0a3600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4620005fd0 tx=0x7f4620005e20 comp rx=0 tx=0).stop 2023-11-16T16:37:20.545 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.544+0000 7f463518e700 1 -- 172.21.15.1:0/523430664 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46300ff170 msgr2=0x7f4630068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:20.545 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.544+0000 7f463518e700 1 --2- 172.21.15.1:0/523430664 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46300ff170 0x7f4630068930 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4618005910 tx=0x7f461800f740 comp rx=0 tx=0).stop 2023-11-16T16:37:20.545 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.544+0000 7f463518e700 1 -- 172.21.15.1:0/523430664 shutdown_connections 2023-11-16T16:37:20.545 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.544+0000 7f463518e700 1 --2- 172.21.15.1:0/523430664 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f461c0a1150 0x7f461c0a3600 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:20.545 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.544+0000 7f463518e700 1 --2- 172.21.15.1:0/523430664 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4630100db0 0x7f463019d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:20.545 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.544+0000 7f463518e700 1 --2- 172.21.15.1:0/523430664 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46300ff170 0x7f4630068930 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:20.545 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.544+0000 7f463518e700 1 --2- 172.21.15.1:0/523430664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4630100400 0x7f463019aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:20.545 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.544+0000 7f463518e700 1 -- 172.21.15.1:0/523430664 >> 172.21.15.1:0/523430664 conn(0x7f46300fa750 msgr2=0x7f46300fca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:20.546 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.544+0000 7f463518e700 1 -- 172.21.15.1:0/523430664 shutdown_connections 2023-11-16T16:37:20.546 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.544+0000 7f463518e700 1 -- 172.21.15.1:0/523430664 wait complete. 2023-11-16T16:37:20.557 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-11-16T16:37:20.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:20 smithi053 ceph-mon[100638]: mds.? [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] up:active 2023-11-16T16:37:20.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:20 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1048054619' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-11-16T16:37:20.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:20 smithi053 ceph-mon[100638]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2023-11-16T16:37:20.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:20 smithi053 ceph-mon[100638]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:37:20.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:20 smithi053 ceph-mon[100638]: pgmap v263: 97 pgs: 97 active+clean; 579 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s wr, 12 op/s 2023-11-16T16:37:20.685 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.684+0000 7f21a59fa700 1 -- 172.21.15.1:0/2285883805 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21a00ff1a0 msgr2=0x7f21a00ff5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:20.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.684+0000 7f21a59fa700 1 --2- 172.21.15.1:0/2285883805 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21a00ff1a0 0x7f21a00ff5b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2194009990 tx=0x7f2194009ca0 comp rx=0 tx=0).stop 2023-11-16T16:37:20.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.685+0000 7f21a59fa700 1 -- 172.21.15.1:0/2285883805 shutdown_connections 2023-11-16T16:37:20.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.685+0000 7f21a59fa700 1 --2- 172.21.15.1:0/2285883805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21a0100d20 0x7f21a01075e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:20.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.685+0000 7f21a59fa700 1 --2- 172.21.15.1:0/2285883805 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21a00ff1a0 0x7f21a00ff5b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:20.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.685+0000 7f21a59fa700 1 --2- 172.21.15.1:0/2285883805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21a0100390 0x7f21a01007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:20.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.685+0000 7f21a59fa700 1 -- 172.21.15.1:0/2285883805 >> 172.21.15.1:0/2285883805 conn(0x7f21a00fa7c0 msgr2=0x7f21a00fcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:20.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.685+0000 7f21a59fa700 1 -- 172.21.15.1:0/2285883805 shutdown_connections 2023-11-16T16:37:20.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.685+0000 7f21a59fa700 1 -- 172.21.15.1:0/2285883805 wait complete. 2023-11-16T16:37:20.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.685+0000 7f21a59fa700 1 Processor -- start 2023-11-16T16:37:20.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.685+0000 7f21a59fa700 1 -- start start 2023-11-16T16:37:20.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.686+0000 7f21a59fa700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21a00ff1a0 0x7f21a0194ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:20.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.686+0000 7f21a59fa700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21a0100390 0x7f21a0195410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:20.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.686+0000 7f21a59fa700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21a0100d20 0x7f21a019c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:20.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.686+0000 7f21a59fa700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a019ca50 con 0x7f21a0100390 2023-11-16T16:37:20.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.686+0000 7f21a59fa700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a019cb90 con 0x7f21a00ff1a0 2023-11-16T16:37:20.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.686+0000 7f21a59fa700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a019cd20 con 0x7f21a0100d20 2023-11-16T16:37:20.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.686+0000 7f219e7fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21a0100390 0x7f21a0195410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:20.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.686+0000 7f219e7fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21a0100390 0x7f21a0195410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41586/0 (socket says 172.21.15.1:41586) 2023-11-16T16:37:20.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.686+0000 7f219e7fc700 1 -- 172.21.15.1:0/4240481612 learned_addr learned my addr 172.21.15.1:0/4240481612 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:20.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.686+0000 7f219f7fe700 1 --2- 172.21.15.1:0/4240481612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21a0100d20 0x7f21a019c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:20.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.686+0000 7f219effd700 1 --2- 172.21.15.1:0/4240481612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21a00ff1a0 0x7f21a0194ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:20.690 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.686+0000 7f219e7fc700 1 -- 172.21.15.1:0/4240481612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21a0100d20 msgr2=0x7f21a019c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:20.690 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.686+0000 7f219e7fc700 1 --2- 172.21.15.1:0/4240481612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21a0100d20 0x7f21a019c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:20.690 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.686+0000 7f219e7fc700 1 -- 172.21.15.1:0/4240481612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21a00ff1a0 msgr2=0x7f21a0194ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:20.690 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.686+0000 7f219e7fc700 1 --2- 172.21.15.1:0/4240481612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21a00ff1a0 0x7f21a0194ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:20.690 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.686+0000 7f219e7fc700 1 -- 172.21.15.1:0/4240481612 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21940095f0 con 0x7f21a0100390 2023-11-16T16:37:20.690 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.686+0000 7f219f7fe700 1 --2- 172.21.15.1:0/4240481612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21a0100d20 0x7f21a019c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:37:20.690 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.686+0000 7f219e7fc700 1 --2- 172.21.15.1:0/4240481612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21a0100390 0x7f21a0195410 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f219000b530 tx=0x7f219000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:20.690 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.686+0000 7f21a49f8700 1 -- 172.21.15.1:0/4240481612 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2190011830 con 0x7f21a0100390 2023-11-16T16:37:20.690 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.686+0000 7f21a49f8700 1 -- 172.21.15.1:0/4240481612 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2190011e50 con 0x7f21a0100390 2023-11-16T16:37:20.690 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.686+0000 7f21a59fa700 1 -- 172.21.15.1:0/4240481612 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21a010abd0 con 0x7f21a0100390 2023-11-16T16:37:20.690 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.686+0000 7f21a49f8700 1 -- 172.21.15.1:0/4240481612 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219000f2e0 con 0x7f21a0100390 2023-11-16T16:37:20.691 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.686+0000 7f21a59fa700 1 -- 172.21.15.1:0/4240481612 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21a010aa40 con 0x7f21a0100390 2023-11-16T16:37:20.691 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.688+0000 7f21a59fa700 1 -- 172.21.15.1:0/4240481612 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21a01066e0 con 0x7f21a0100390 2023-11-16T16:37:20.691 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.688+0000 7f21a49f8700 1 -- 172.21.15.1:0/4240481612 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f2190010580 con 0x7f21a0100390 2023-11-16T16:37:20.691 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.689+0000 7f21a49f8700 1 --2- 172.21.15.1:0/4240481612 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f21880a1150 0x7f21880a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:20.692 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.689+0000 7f219effd700 1 --2- 172.21.15.1:0/4240481612 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f21880a1150 0x7f21880a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:20.692 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.689+0000 7f21a49f8700 1 -- 172.21.15.1:0/4240481612 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f21900cc2c0 con 0x7f21a0100390 2023-11-16T16:37:20.692 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.690+0000 7f219effd700 1 --2- 172.21.15.1:0/4240481612 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f21880a1150 0x7f21880a3600 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f219400b5c0 tx=0x7f2194005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:20.694 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.694+0000 7f21a49f8700 1 -- 172.21.15.1:0/4240481612 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2190095b60 con 0x7f21a0100390 2023-11-16T16:37:20.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:20 smithi026 ceph-mon[100112]: mds.? [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] up:active 2023-11-16T16:37:20.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:20 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1048054619' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-11-16T16:37:20.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:20 smithi026 ceph-mon[100112]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2023-11-16T16:37:20.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:20 smithi026 ceph-mon[100112]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:37:20.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:20 smithi026 ceph-mon[100112]: pgmap v263: 97 pgs: 97 active+clean; 579 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s wr, 12 op/s 2023-11-16T16:37:20.931 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.931+0000 7f21a59fa700 1 -- 172.21.15.1:0/4240481612 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f21a0061c70 con 0x7f21880a1150 2023-11-16T16:37:20.936 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.935+0000 7f21a49f8700 1 -- 172.21.15.1:0/4240481612 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f21a0061c70 con 0x7f21880a1150 2023-11-16T16:37:20.936 INFO:teuthology.orchestra.run.smithi001.stdout:/volumes/_nogroup/sv_1/bdad94b8-b08e-45e7-b7ae-3497461cc4bb 2023-11-16T16:37:20.938 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.937+0000 7f21a59fa700 1 -- 172.21.15.1:0/4240481612 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f21880a1150 msgr2=0x7f21880a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:20.938 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.937+0000 7f21a59fa700 1 --2- 172.21.15.1:0/4240481612 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f21880a1150 0x7f21880a3600 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f219400b5c0 tx=0x7f2194005fb0 comp rx=0 tx=0).stop 2023-11-16T16:37:20.939 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.937+0000 7f21a59fa700 1 -- 172.21.15.1:0/4240481612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21a0100390 msgr2=0x7f21a0195410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:20.939 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.937+0000 7f21a59fa700 1 --2- 172.21.15.1:0/4240481612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21a0100390 0x7f21a0195410 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f219000b530 tx=0x7f219000b8f0 comp rx=0 tx=0).stop 2023-11-16T16:37:20.939 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.937+0000 7f21a59fa700 1 -- 172.21.15.1:0/4240481612 shutdown_connections 2023-11-16T16:37:20.939 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.937+0000 7f21a59fa700 1 --2- 172.21.15.1:0/4240481612 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f21880a1150 0x7f21880a3600 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:20.939 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.937+0000 7f21a59fa700 1 --2- 172.21.15.1:0/4240481612 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f21a0100d20 0x7f21a019c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:20.939 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.937+0000 7f21a59fa700 1 --2- 172.21.15.1:0/4240481612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21a00ff1a0 0x7f21a0194ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:20.939 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.937+0000 7f21a59fa700 1 --2- 172.21.15.1:0/4240481612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f21a0100390 0x7f21a0195410 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:20.939 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.937+0000 7f21a59fa700 1 -- 172.21.15.1:0/4240481612 >> 172.21.15.1:0/4240481612 conn(0x7f21a00fa7c0 msgr2=0x7f21a0103820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:20.939 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.937+0000 7f21a59fa700 1 -- 172.21.15.1:0/4240481612 shutdown_connections 2023-11-16T16:37:20.939 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:20.937+0000 7f21a59fa700 1 -- 172.21.15.1:0/4240481612 wait complete. 2023-11-16T16:37:20.950 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:37:21.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.075+0000 7f8f00d87700 1 -- 172.21.15.1:0/445942775 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8efc100d40 msgr2=0x7f8efc107600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:21.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.075+0000 7f8f00d87700 1 --2- 172.21.15.1:0/445942775 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8efc100d40 0x7f8efc107600 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4009990 tx=0x7f8ef4009ca0 comp rx=0 tx=0).stop 2023-11-16T16:37:21.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.075+0000 7f8f00d87700 1 -- 172.21.15.1:0/445942775 shutdown_connections 2023-11-16T16:37:21.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.075+0000 7f8f00d87700 1 --2- 172.21.15.1:0/445942775 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8efc100d40 0x7f8efc107600 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:21.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.075+0000 7f8f00d87700 1 --2- 172.21.15.1:0/445942775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8efc0ff1c0 0x7f8efc0ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:21.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.075+0000 7f8f00d87700 1 --2- 172.21.15.1:0/445942775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8efc1003b0 0x7f8efc100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:21.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.075+0000 7f8f00d87700 1 -- 172.21.15.1:0/445942775 >> 172.21.15.1:0/445942775 conn(0x7f8efc0fa7c0 msgr2=0x7f8efc0fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:21.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.075+0000 7f8f00d87700 1 -- 172.21.15.1:0/445942775 shutdown_connections 2023-11-16T16:37:21.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.075+0000 7f8f00d87700 1 -- 172.21.15.1:0/445942775 wait complete. 2023-11-16T16:37:21.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.076+0000 7f8f00d87700 1 Processor -- start 2023-11-16T16:37:21.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.076+0000 7f8f00d87700 1 -- start start 2023-11-16T16:37:21.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.076+0000 7f8f00d87700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8efc0ff1c0 0x7f8efc196060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:21.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.076+0000 7f8f00d87700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8efc1003b0 0x7f8efc1965a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:21.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.076+0000 7f8f00d87700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8efc100d40 0x7f8efc19d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:21.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.076+0000 7f8f00d87700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc19dbe0 con 0x7f8efc1003b0 2023-11-16T16:37:21.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.076+0000 7f8f00d87700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc19dd20 con 0x7f8efc100d40 2023-11-16T16:37:21.080 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.076+0000 7f8f00d87700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc19deb0 con 0x7f8efc0ff1c0 2023-11-16T16:37:21.080 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.076+0000 7f8ef9d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8efc1003b0 0x7f8efc1965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:21.080 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.076+0000 7f8ef9d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8efc1003b0 0x7f8efc1965a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41600/0 (socket says 172.21.15.1:41600) 2023-11-16T16:37:21.080 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.076+0000 7f8ef9d9b700 1 -- 172.21.15.1:0/2923912856 learned_addr learned my addr 172.21.15.1:0/2923912856 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:21.080 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.076+0000 7f8efa59c700 1 --2- 172.21.15.1:0/2923912856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8efc0ff1c0 0x7f8efc196060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:21.080 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.076+0000 7f8efad9d700 1 --2- 172.21.15.1:0/2923912856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8efc100d40 0x7f8efc19d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:21.080 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.077+0000 7f8ef9d9b700 1 -- 172.21.15.1:0/2923912856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8efc0ff1c0 msgr2=0x7f8efc196060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:21.080 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.077+0000 7f8ef9d9b700 1 --2- 172.21.15.1:0/2923912856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8efc0ff1c0 0x7f8efc196060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:21.080 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.077+0000 7f8ef9d9b700 1 -- 172.21.15.1:0/2923912856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8efc100d40 msgr2=0x7f8efc19d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:21.080 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.077+0000 7f8ef9d9b700 1 --2- 172.21.15.1:0/2923912856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8efc100d40 0x7f8efc19d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:21.080 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.077+0000 7f8ef9d9b700 1 -- 172.21.15.1:0/2923912856 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef40095f0 con 0x7f8efc1003b0 2023-11-16T16:37:21.080 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.077+0000 7f8efa59c700 1 --2- 172.21.15.1:0/2923912856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8efc0ff1c0 0x7f8efc196060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:37:21.080 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.077+0000 7f8ef9d9b700 1 --2- 172.21.15.1:0/2923912856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8efc1003b0 0x7f8efc1965a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8ee400c920 tx=0x7f8ee400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:21.081 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.077+0000 7f8ef37fe700 1 -- 172.21.15.1:0/2923912856 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee4011b60 con 0x7f8efc1003b0 2023-11-16T16:37:21.081 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.077+0000 7f8ef37fe700 1 -- 172.21.15.1:0/2923912856 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ee400ce70 con 0x7f8efc1003b0 2023-11-16T16:37:21.081 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.077+0000 7f8f00d87700 1 -- 172.21.15.1:0/2923912856 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8efc10ab80 con 0x7f8efc1003b0 2023-11-16T16:37:21.081 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.077+0000 7f8ef37fe700 1 -- 172.21.15.1:0/2923912856 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee40054b0 con 0x7f8efc1003b0 2023-11-16T16:37:21.081 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.077+0000 7f8f00d87700 1 -- 172.21.15.1:0/2923912856 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8efc10a9f0 con 0x7f8efc1003b0 2023-11-16T16:37:21.081 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.078+0000 7f8f00d87700 1 -- 172.21.15.1:0/2923912856 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8efc101a90 con 0x7f8efc1003b0 2023-11-16T16:37:21.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.079+0000 7f8ef37fe700 1 -- 172.21.15.1:0/2923912856 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f8ee4004750 con 0x7f8efc1003b0 2023-11-16T16:37:21.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.080+0000 7f8ef37fe700 1 --2- 172.21.15.1:0/2923912856 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8ee80a11d0 0x7f8ee80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:21.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.080+0000 7f8ef37fe700 1 -- 172.21.15.1:0/2923912856 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8ee40d0450 con 0x7f8efc1003b0 2023-11-16T16:37:21.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.080+0000 7f8efa59c700 1 --2- 172.21.15.1:0/2923912856 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8ee80a11d0 0x7f8ee80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:21.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.080+0000 7f8efa59c700 1 --2- 172.21.15.1:0/2923912856 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8ee80a11d0 0x7f8ee80a3680 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8efc100210 tx=0x7f8eec00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:21.085 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.084+0000 7f8ef37fe700 1 -- 172.21.15.1:0/2923912856 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8ee4052730 con 0x7f8efc1003b0 2023-11-16T16:37:21.331 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.330+0000 7f8f00d87700 1 -- 172.21.15.1:0/2923912856 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8efc102040 con 0x7f8efc1003b0 2023-11-16T16:37:21.331 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.331+0000 7f8ef37fe700 1 -- 172.21.15.1:0/2923912856 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10774 (secure 0 0 0) 0x7f8ee4099c50 con 0x7f8efc1003b0 2023-11-16T16:37:21.332 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:37:21.332 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2587563684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2587563684},{"type":"v1","addr":"172.21.15.1:6835","nonce":2587563684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/277936168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":277936168},{"type":"v1","addr":"172.21.15.1:6837","nonce":277936168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/1172575045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1172575045},{"type":"v1","addr":"172.21.15.1:6839","nonce":1172575045}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/2631175899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2631175899},{"type":"v1","addr":"172.21.15.1:6841","nonce":2631175899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3693905267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3693905267},{"type":"v1","addr":"172.21.15.53:6833","nonce":3693905267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/164668033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":164668033},{"type":"v1","addr":"172.21.15.26:6833","nonce":164668033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-16T16:37:13.918943+0000","modified":"2023-11-16T16:37:19.292898+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2140542192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2140542192},{"type":"v1","addr":"172.21.15.53:6835","nonce":2140542192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1010760275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1010760275},{"type":"v1","addr":"172.21.15.26:6835","nonce":1010760275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3197060302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3197060302},{"type":"v1","addr":"172.21.15.53:6837","nonce":3197060302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6837/1453504863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1453504863},{"type":"v1","addr":"172.21.15.26:6837","nonce":1453504863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.53:6839/1044753024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1044753024},{"type":"v1","addr":"172.21.15.53:6839","nonce":1044753024}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6839/2192205949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2192205949},{"type":"v1","addr":"172.21.15.26:6839","nonce":2192205949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-16T16:37:21.334 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.333+0000 7f8f00d87700 1 -- 172.21.15.1:0/2923912856 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8ee80a11d0 msgr2=0x7f8ee80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:21.334 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.333+0000 7f8f00d87700 1 --2- 172.21.15.1:0/2923912856 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8ee80a11d0 0x7f8ee80a3680 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8efc100210 tx=0x7f8eec00b040 comp rx=0 tx=0).stop 2023-11-16T16:37:21.334 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.333+0000 7f8f00d87700 1 -- 172.21.15.1:0/2923912856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8efc1003b0 msgr2=0x7f8efc1965a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:21.335 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.333+0000 7f8f00d87700 1 --2- 172.21.15.1:0/2923912856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8efc1003b0 0x7f8efc1965a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8ee400c920 tx=0x7f8ee400cc30 comp rx=0 tx=0).stop 2023-11-16T16:37:21.335 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.333+0000 7f8f00d87700 1 -- 172.21.15.1:0/2923912856 shutdown_connections 2023-11-16T16:37:21.335 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.333+0000 7f8f00d87700 1 --2- 172.21.15.1:0/2923912856 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8ee80a11d0 0x7f8ee80a3680 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:21.335 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.333+0000 7f8f00d87700 1 --2- 172.21.15.1:0/2923912856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8efc100d40 0x7f8efc19d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:21.335 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.333+0000 7f8f00d87700 1 --2- 172.21.15.1:0/2923912856 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8efc0ff1c0 0x7f8efc196060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:21.335 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.333+0000 7f8f00d87700 1 --2- 172.21.15.1:0/2923912856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8efc1003b0 0x7f8efc1965a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:21.335 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.333+0000 7f8f00d87700 1 -- 172.21.15.1:0/2923912856 >> 172.21.15.1:0/2923912856 conn(0x7f8efc0fa7c0 msgr2=0x7f8efc067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:21.335 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.333+0000 7f8f00d87700 1 -- 172.21.15.1:0/2923912856 shutdown_connections 2023-11-16T16:37:21.335 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.333+0000 7f8f00d87700 1 -- 172.21.15.1:0/2923912856 wait complete. 2023-11-16T16:37:21.336 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 23 2023-11-16T16:37:21.346 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-16T16:37:21.473 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.472+0000 7f9ce7b66700 1 -- 172.21.15.1:0/166727320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ce00ff1c0 msgr2=0x7f9ce00ff5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:21.473 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.472+0000 7f9ce7b66700 1 --2- 172.21.15.1:0/166727320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ce00ff1c0 0x7f9ce00ff5d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc009990 tx=0x7f9cdc009ca0 comp rx=0 tx=0).stop 2023-11-16T16:37:21.473 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.472+0000 7f9ce7b66700 1 -- 172.21.15.1:0/166727320 shutdown_connections 2023-11-16T16:37:21.473 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.472+0000 7f9ce7b66700 1 --2- 172.21.15.1:0/166727320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ce0100d40 0x7f9ce0107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:21.473 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.472+0000 7f9ce7b66700 1 --2- 172.21.15.1:0/166727320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ce00ff1c0 0x7f9ce00ff5d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:21.473 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.472+0000 7f9ce7b66700 1 --2- 172.21.15.1:0/166727320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ce01003b0 0x7f9ce0100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:21.473 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.472+0000 7f9ce7b66700 1 -- 172.21.15.1:0/166727320 >> 172.21.15.1:0/166727320 conn(0x7f9ce00fa7c0 msgr2=0x7f9ce00fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:21.473 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.472+0000 7f9ce7b66700 1 -- 172.21.15.1:0/166727320 shutdown_connections 2023-11-16T16:37:21.473 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.472+0000 7f9ce7b66700 1 -- 172.21.15.1:0/166727320 wait complete. 2023-11-16T16:37:21.475 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.472+0000 7f9ce7b66700 1 Processor -- start 2023-11-16T16:37:21.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.472+0000 7f9ce7b66700 1 -- start start 2023-11-16T16:37:21.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.473+0000 7f9ce7b66700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ce00ff1c0 0x7f9ce0068950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:21.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.473+0000 7f9ce7b66700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ce01003b0 0x7f9ce019ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:21.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.473+0000 7f9ce7b66700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ce0100d40 0x7f9ce019d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:21.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.473+0000 7f9ce7b66700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce019db80 con 0x7f9ce00ff1c0 2023-11-16T16:37:21.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.473+0000 7f9ce7b66700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce019dcf0 con 0x7f9ce0100d40 2023-11-16T16:37:21.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.473+0000 7f9ce7b66700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce019deb0 con 0x7f9ce01003b0 2023-11-16T16:37:21.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.473+0000 7f9ce5902700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ce00ff1c0 0x7f9ce0068950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:21.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.473+0000 7f9ce5902700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ce00ff1c0 0x7f9ce0068950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41612/0 (socket says 172.21.15.1:41612) 2023-11-16T16:37:21.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.473+0000 7f9ce5902700 1 -- 172.21.15.1:0/3902182019 learned_addr learned my addr 172.21.15.1:0/3902182019 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:21.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.473+0000 7f9ce6103700 1 --2- 172.21.15.1:0/3902182019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ce0100d40 0x7f9ce019d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:21.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.473+0000 7f9ce5902700 1 -- 172.21.15.1:0/3902182019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ce01003b0 msgr2=0x7f9ce019ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:21.477 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.473+0000 7f9ce5902700 1 --2- 172.21.15.1:0/3902182019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ce01003b0 0x7f9ce019ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:21.477 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.473+0000 7f9ce5902700 1 -- 172.21.15.1:0/3902182019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ce0100d40 msgr2=0x7f9ce019d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:21.477 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.473+0000 7f9ce5902700 1 --2- 172.21.15.1:0/3902182019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ce0100d40 0x7f9ce019d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:21.477 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.473+0000 7f9ce5902700 1 -- 172.21.15.1:0/3902182019 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cdc0095f0 con 0x7f9ce00ff1c0 2023-11-16T16:37:21.477 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.473+0000 7f9ce5902700 1 --2- 172.21.15.1:0/3902182019 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ce00ff1c0 0x7f9ce0068950 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc0036c0 tx=0x7f9cdc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:21.477 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.473+0000 7f9cd2ffd700 1 -- 172.21.15.1:0/3902182019 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc01d070 con 0x7f9ce00ff1c0 2023-11-16T16:37:21.477 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.473+0000 7f9cd2ffd700 1 -- 172.21.15.1:0/3902182019 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cdc003fe0 con 0x7f9ce00ff1c0 2023-11-16T16:37:21.477 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.473+0000 7f9ce7b66700 1 -- 172.21.15.1:0/3902182019 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ce010ab20 con 0x7f9ce00ff1c0 2023-11-16T16:37:21.477 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.473+0000 7f9cd2ffd700 1 -- 172.21.15.1:0/3902182019 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc022dc0 con 0x7f9ce00ff1c0 2023-11-16T16:37:21.477 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.474+0000 7f9ce7b66700 1 -- 172.21.15.1:0/3902182019 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ce010a990 con 0x7f9ce00ff1c0 2023-11-16T16:37:21.477 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.474+0000 7f9ce7b66700 1 -- 172.21.15.1:0/3902182019 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ce0106140 con 0x7f9ce00ff1c0 2023-11-16T16:37:21.478 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.476+0000 7f9cd2ffd700 1 -- 172.21.15.1:0/3902182019 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f9cdc003b20 con 0x7f9ce00ff1c0 2023-11-16T16:37:21.478 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.476+0000 7f9cd2ffd700 1 --2- 172.21.15.1:0/3902182019 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f9ccc0a1150 0x7f9ccc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:21.478 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.476+0000 7f9ce5101700 1 --2- 172.21.15.1:0/3902182019 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f9ccc0a1150 0x7f9ccc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:21.478 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.477+0000 7f9cd2ffd700 1 -- 172.21.15.1:0/3902182019 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9cdc0ce410 con 0x7f9ce00ff1c0 2023-11-16T16:37:21.478 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.477+0000 7f9ce5101700 1 --2- 172.21.15.1:0/3902182019 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f9ccc0a1150 0x7f9ccc0a3600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4005fd0 tx=0x7f9cd4005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:21.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.480+0000 7f9cd2ffd700 1 -- 172.21.15.1:0/3902182019 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9cdc097d40 con 0x7f9ce00ff1c0 2023-11-16T16:37:21.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:21 smithi053 ceph-mon[100638]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:37:21.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:21 smithi053 ceph-mon[100638]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:37:21.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:21 smithi001 ceph-mon[98800]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:37:21.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:21 smithi001 ceph-mon[98800]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:37:21.684 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.684+0000 7f9ce7b66700 1 -- 172.21.15.1:0/3902182019 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ce0067af0 con 0x7f9ce00ff1c0 2023-11-16T16:37:21.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.685+0000 7f9cd2ffd700 1 -- 172.21.15.1:0/3902182019 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+21988 (secure 0 0 0) 0x7f9cdc09b360 con 0x7f9ce00ff1c0 2023-11-16T16:37:21.686 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:37:21.686 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":84,"fsid":"35d9b010-849d-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:30:06.217778+0000","modified":"2023-11-16T16:37:14.920337+0000","last_up_change":"2023-11-16T16:36:31.735403+0000","last_in_change":"2023-11-16T16:36:14.083021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:33:15.916723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-16T16:37:12.158031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-16T16:37:13.272343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a8480507-5275-4e52-953b-626110beb436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6803","nonce":2469671609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6805","nonce":2469671609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6809","nonce":2469671609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6807","nonce":2469671609}]},"public_addr":"172.21.15.1:6803/2469671609","cluster_addr":"172.21.15.1:6805/2469671609","heartbeat_back_addr":"172.21.15.1:6809/2469671609","heartbeat_front_addr":"172.21.15.1:6807/2469671609","state":["exists","up"]},{"osd":1,"uuid":"1d011db2-99d1-4575-bdc5-51f88a308fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6801","nonce":247994796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6803","nonce":247994796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6807","nonce":247994796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6805","nonce":247994796}]},"public_addr":"172.21.15.26:6801/247994796","cluster_addr":"172.21.15.26:6803/247994796","heartbeat_back_addr":"172.21.15.26:6807/247994796","heartbeat_front_addr":"172.21.15.26:6805/247994796","state":["exists","up"]},{"osd":2,"uuid":"4d7a2bd1-3c8e-4546-a243-f0354473b031","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6801","nonce":1659316290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6803","nonce":1659316290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6807","nonce":1659316290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6805","nonce":1659316290}]},"public_addr":"172.21.15.53:6801/1659316290","cluster_addr":"172.21.15.53:6803/1659316290","heartbeat_back_addr":"172.21.15.53:6807/1659316290","heartbeat_front_addr":"172.21.15.53:6805/1659316290","state":["exists","up"]},{"osd":3,"uuid":"9c290fd9-e121-4a81-9687-b0eb93debd86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6811","nonce":2728987281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6813","nonce":2728987281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6817","nonce":2728987281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6815","nonce":2728987281}]},"public_addr":"172.21.15.1:6811/2728987281","cluster_addr":"172.21.15.1:6813/2728987281","heartbeat_back_addr":"172.21.15.1:6817/2728987281","heartbeat_front_addr":"172.21.15.1:6815/2728987281","state":["exists","up"]},{"osd":4,"uuid":"533226bc-cc57-4f15-9e29-820294d57e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6809","nonce":2954740024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6811","nonce":2954740024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6815","nonce":2954740024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6813","nonce":2954740024}]},"public_addr":"172.21.15.26:6809/2954740024","cluster_addr":"172.21.15.26:6811/2954740024","heartbeat_back_addr":"172.21.15.26:6815/2954740024","heartbeat_front_addr":"172.21.15.26:6813/2954740024","state":["exists","up"]},{"osd":5,"uuid":"d3ab25eb-e1d4-4557-bc89-efc9072401ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6809","nonce":3966666318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6811","nonce":3966666318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6815","nonce":3966666318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6813","nonce":3966666318}]},"public_addr":"172.21.15.53:6809/3966666318","cluster_addr":"172.21.15.53:6811/3966666318","heartbeat_back_addr":"172.21.15.53:6815/3966666318","heartbeat_front_addr":"172.21.15.53:6813/3966666318","state":["exists","up"]},{"osd":6,"uuid":"180f506e-afb8-4043-b0b4-2dc86f6d4bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6819","nonce":3364032646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6821","nonce":3364032646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6825","nonce":3364032646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6823","nonce":3364032646}]},"public_addr":"172.21.15.1:6819/3364032646","cluster_addr":"172.21.15.1:6821/3364032646","heartbeat_back_addr":"172.21.15.1:6825/3364032646","heartbeat_front_addr":"172.21.15.1:6823/3364032646","state":["exists","up"]},{"osd":7,"uuid":"0d837bac-194f-498d-b7fb-b3147e992329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6817","nonce":3766901391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6819","nonce":3766901391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6823","nonce":3766901391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6821","nonce":3766901391}]},"public_addr":"172.21.15.26:6817/3766901391","cluster_addr":"172.21.15.26:6819/3766901391","heartbeat_back_addr":"172.21.15.26:6823/3766901391","heartbeat_front_addr":"172.21.15.26:6821/3766901391","state":["exists","up"]},{"osd":8,"uuid":"933f231a-7454-47b0-9291-f1fa64f0baec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6817","nonce":3224335871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6819","nonce":3224335871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6823","nonce":3224335871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6821","nonce":3224335871}]},"public_addr":"172.21.15.53:6817/3224335871","cluster_addr":"172.21.15.53:6819/3224335871","heartbeat_back_addr":"172.21.15.53:6823/3224335871","heartbeat_front_addr":"172.21.15.53:6821/3224335871","state":["exists","up"]},{"osd":9,"uuid":"245fcbb3-c651-4703-ab80-309dc98bd983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6827","nonce":1391199844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6829","nonce":1391199844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6833","nonce":1391199844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6831","nonce":1391199844}]},"public_addr":"172.21.15.1:6827/1391199844","cluster_addr":"172.21.15.1:6829/1391199844","heartbeat_back_addr":"172.21.15.1:6833/1391199844","heartbeat_front_addr":"172.21.15.1:6831/1391199844","state":["exists","up"]},{"osd":10,"uuid":"2c090e7a-5240-4d52-ae4f-e94fe05383d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6825","nonce":1773545447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6827","nonce":1773545447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6831","nonce":1773545447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6829","nonce":1773545447}]},"public_addr":"172.21.15.26:6825/1773545447","cluster_addr":"172.21.15.26:6827/1773545447","heartbeat_back_addr":"172.21.15.26:6831/1773545447","heartbeat_front_addr":"172.21.15.26:6829/1773545447","state":["exists","up"]},{"osd":11,"uuid":"4511e32d-e469-45e1-8310-0ff400cb7b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6825","nonce":2564669546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6827","nonce":2564669546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6831","nonce":2564669546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6829","nonce":2564669546}]},"public_addr":"172.21.15.53:6825/2564669546","cluster_addr":"172.21.15.53:6827/2564669546","heartbeat_back_addr":"172.21.15.53:6831/2564669546","heartbeat_front_addr":"172.21.15.53:6829/2564669546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:28.140250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:49.885461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:12.521487+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:34.926197+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:56.815594+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:18.387751+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:40.587106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:02.319370+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:23.832298+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:46.338040+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:08.730414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:30.050070+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1494336948":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6801/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6800/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6800/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/1641976669":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/2895923638":"2023-11-17T16:30:35.064289+0000","172.21.15.1:6801/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/78811711":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/4131957708":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/344403698":"2023-11-17T16:30:35.064289+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:37:21.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.687+0000 7f9ce7b66700 1 -- 172.21.15.1:0/3902182019 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f9ccc0a1150 msgr2=0x7f9ccc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:21.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.687+0000 7f9ce7b66700 1 --2- 172.21.15.1:0/3902182019 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f9ccc0a1150 0x7f9ccc0a3600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4005fd0 tx=0x7f9cd4005d00 comp rx=0 tx=0).stop 2023-11-16T16:37:21.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.687+0000 7f9ce7b66700 1 -- 172.21.15.1:0/3902182019 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ce00ff1c0 msgr2=0x7f9ce0068950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:21.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.687+0000 7f9ce7b66700 1 --2- 172.21.15.1:0/3902182019 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ce00ff1c0 0x7f9ce0068950 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc0036c0 tx=0x7f9cdc0057f0 comp rx=0 tx=0).stop 2023-11-16T16:37:21.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.687+0000 7f9ce7b66700 1 -- 172.21.15.1:0/3902182019 shutdown_connections 2023-11-16T16:37:21.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.687+0000 7f9ce7b66700 1 --2- 172.21.15.1:0/3902182019 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f9ccc0a1150 0x7f9ccc0a3600 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:21.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.687+0000 7f9ce7b66700 1 --2- 172.21.15.1:0/3902182019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ce0100d40 0x7f9ce019d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:21.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.687+0000 7f9ce7b66700 1 --2- 172.21.15.1:0/3902182019 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ce00ff1c0 0x7f9ce0068950 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:21.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.687+0000 7f9ce7b66700 1 --2- 172.21.15.1:0/3902182019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9ce01003b0 0x7f9ce019ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:21.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.687+0000 7f9ce7b66700 1 -- 172.21.15.1:0/3902182019 >> 172.21.15.1:0/3902182019 conn(0x7f9ce00fa7c0 msgr2=0x7f9ce00fcaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:21.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.688+0000 7f9ce7b66700 1 -- 172.21.15.1:0/3902182019 shutdown_connections 2023-11-16T16:37:21.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.688+0000 7f9ce7b66700 1 -- 172.21.15.1:0/3902182019 wait complete. 2023-11-16T16:37:21.701 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-16T16:37:21.701 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-16T16:37:21.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:21 smithi026 ceph-mon[100112]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:37:21.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:21 smithi026 ceph-mon[100112]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:37:21.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.826+0000 7f02b4b4a700 1 -- 172.21.15.1:0/1961664967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02b01003b0 msgr2=0x7f02b0100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:21.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.826+0000 7f02b4b4a700 1 --2- 172.21.15.1:0/1961664967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02b01003b0 0x7f02b0100800 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f02a0009990 tx=0x7f02a0009ca0 comp rx=0 tx=0).stop 2023-11-16T16:37:21.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.826+0000 7f02b4b4a700 1 -- 172.21.15.1:0/1961664967 shutdown_connections 2023-11-16T16:37:21.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.826+0000 7f02b4b4a700 1 --2- 172.21.15.1:0/1961664967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02b0100d40 0x7f02b0107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:21.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.826+0000 7f02b4b4a700 1 --2- 172.21.15.1:0/1961664967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02b00ff1c0 0x7f02b00ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:21.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.826+0000 7f02b4b4a700 1 --2- 172.21.15.1:0/1961664967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02b01003b0 0x7f02b0100800 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:21.828 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.826+0000 7f02b4b4a700 1 -- 172.21.15.1:0/1961664967 >> 172.21.15.1:0/1961664967 conn(0x7f02b00fa7c0 msgr2=0x7f02b00fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:21.828 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.826+0000 7f02b4b4a700 1 -- 172.21.15.1:0/1961664967 shutdown_connections 2023-11-16T16:37:21.828 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.826+0000 7f02b4b4a700 1 -- 172.21.15.1:0/1961664967 wait complete. 2023-11-16T16:37:21.830 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.827+0000 7f02b4b4a700 1 Processor -- start 2023-11-16T16:37:21.830 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.827+0000 7f02b4b4a700 1 -- start start 2023-11-16T16:37:21.830 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.827+0000 7f02b4b4a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02b00ff1c0 0x7f02b0068950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:21.831 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.827+0000 7f02b4b4a700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02b01003b0 0x7f02b019ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:21.831 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.827+0000 7f02b4b4a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02b0100d40 0x7f02b019d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:21.831 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.827+0000 7f02b4b4a700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b019dab0 con 0x7f02b01003b0 2023-11-16T16:37:21.831 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.827+0000 7f02b4b4a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b019dc20 con 0x7f02b0100d40 2023-11-16T16:37:21.831 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.827+0000 7f02b4b4a700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b019dde0 con 0x7f02b00ff1c0 2023-11-16T16:37:21.831 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.827+0000 7f02add9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02b01003b0 0x7f02b019ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:21.831 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.827+0000 7f02add9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02b01003b0 0x7f02b019ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41628/0 (socket says 172.21.15.1:41628) 2023-11-16T16:37:21.831 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.827+0000 7f02add9b700 1 -- 172.21.15.1:0/1585020704 learned_addr learned my addr 172.21.15.1:0/1585020704 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:21.831 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.827+0000 7f02aed9d700 1 --2- 172.21.15.1:0/1585020704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02b0100d40 0x7f02b019d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:21.831 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.827+0000 7f02ae59c700 1 --2- 172.21.15.1:0/1585020704 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02b00ff1c0 0x7f02b0068950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:21.831 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.827+0000 7f02add9b700 1 -- 172.21.15.1:0/1585020704 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02b00ff1c0 msgr2=0x7f02b0068950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:21.831 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.827+0000 7f02add9b700 1 --2- 172.21.15.1:0/1585020704 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02b00ff1c0 0x7f02b0068950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:21.831 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.827+0000 7f02add9b700 1 -- 172.21.15.1:0/1585020704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02b0100d40 msgr2=0x7f02b019d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:21.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.827+0000 7f02add9b700 1 --2- 172.21.15.1:0/1585020704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02b0100d40 0x7f02b019d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:21.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.827+0000 7f02add9b700 1 -- 172.21.15.1:0/1585020704 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a00095f0 con 0x7f02b01003b0 2023-11-16T16:37:21.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.827+0000 7f02aed9d700 1 --2- 172.21.15.1:0/1585020704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02b0100d40 0x7f02b019d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:37:21.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.827+0000 7f02add9b700 1 --2- 172.21.15.1:0/1585020704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02b01003b0 0x7f02b019ac90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f02a00059d0 tx=0x7f02a0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:21.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.828+0000 7f029f7fe700 1 -- 172.21.15.1:0/1585020704 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a001d070 con 0x7f02b01003b0 2023-11-16T16:37:21.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.828+0000 7f02b4b4a700 1 -- 172.21.15.1:0/1585020704 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02b010ab30 con 0x7f02b01003b0 2023-11-16T16:37:21.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.828+0000 7f029f7fe700 1 -- 172.21.15.1:0/1585020704 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02a00040e0 con 0x7f02b01003b0 2023-11-16T16:37:21.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.828+0000 7f029f7fe700 1 -- 172.21.15.1:0/1585020704 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a000f460 con 0x7f02b01003b0 2023-11-16T16:37:21.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.828+0000 7f02b4b4a700 1 -- 172.21.15.1:0/1585020704 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b010a9a0 con 0x7f02b01003b0 2023-11-16T16:37:21.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.829+0000 7f02b4b4a700 1 -- 172.21.15.1:0/1585020704 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02b0106070 con 0x7f02b01003b0 2023-11-16T16:37:21.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.830+0000 7f029f7fe700 1 -- 172.21.15.1:0/1585020704 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f02a0003c20 con 0x7f02b01003b0 2023-11-16T16:37:21.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.830+0000 7f029f7fe700 1 --2- 172.21.15.1:0/1585020704 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f02980a1150 0x7f02980a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:21.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.830+0000 7f029f7fe700 1 -- 172.21.15.1:0/1585020704 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f02a00cdfa0 con 0x7f02b01003b0 2023-11-16T16:37:21.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.830+0000 7f02ae59c700 1 --2- 172.21.15.1:0/1585020704 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f02980a1150 0x7f02980a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:21.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.831+0000 7f02ae59c700 1 --2- 172.21.15.1:0/1585020704 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f02980a1150 0x7f02980a3600 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f02b0100210 tx=0x7f02a4005d70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:21.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:21.834+0000 7f029f7fe700 1 -- 172.21.15.1:0/1585020704 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f02a00978d0 con 0x7f02b01003b0 2023-11-16T16:37:22.034 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.033+0000 7f02b4b4a700 1 -- 172.21.15.1:0/1585020704 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f02b0061b70 con 0x7f02980a1150 2023-11-16T16:37:22.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.046+0000 7f029f7fe700 1 -- 172.21.15.1:0/1585020704 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+335786 (secure 0 0 0) 0x7f02b0061b70 con 0x7f02980a1150 2023-11-16T16:37:22.051 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:37:22.054 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.050+0000 7f02b4b4a700 1 -- 172.21.15.1:0/1585020704 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f02980a1150 msgr2=0x7f02980a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:22.054 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.050+0000 7f02b4b4a700 1 --2- 172.21.15.1:0/1585020704 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f02980a1150 0x7f02980a3600 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f02b0100210 tx=0x7f02a4005d70 comp rx=0 tx=0).stop 2023-11-16T16:37:22.054 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.050+0000 7f02b4b4a700 1 -- 172.21.15.1:0/1585020704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02b01003b0 msgr2=0x7f02b019ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:22.054 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.050+0000 7f02b4b4a700 1 --2- 172.21.15.1:0/1585020704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02b01003b0 0x7f02b019ac90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f02a00059d0 tx=0x7f02a0005740 comp rx=0 tx=0).stop 2023-11-16T16:37:22.054 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.050+0000 7f02b4b4a700 1 -- 172.21.15.1:0/1585020704 shutdown_connections 2023-11-16T16:37:22.054 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.050+0000 7f02b4b4a700 1 --2- 172.21.15.1:0/1585020704 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f02980a1150 0x7f02980a3600 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:22.054 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.050+0000 7f02b4b4a700 1 --2- 172.21.15.1:0/1585020704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02b0100d40 0x7f02b019d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:22.054 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.050+0000 7f02b4b4a700 1 --2- 172.21.15.1:0/1585020704 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f02b00ff1c0 0x7f02b0068950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:22.054 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.050+0000 7f02b4b4a700 1 --2- 172.21.15.1:0/1585020704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02b01003b0 0x7f02b019ac90 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:22.055 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.050+0000 7f02b4b4a700 1 -- 172.21.15.1:0/1585020704 >> 172.21.15.1:0/1585020704 conn(0x7f02b00fa7c0 msgr2=0x7f02b00fc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:22.055 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.050+0000 7f02b4b4a700 1 -- 172.21.15.1:0/1585020704 shutdown_connections 2023-11-16T16:37:22.055 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.050+0000 7f02b4b4a700 1 -- 172.21.15.1:0/1585020704 wait complete. 2023-11-16T16:37:22.055 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2023-11-16T16:37:22.063 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":264,"stamp":"2023-11-16T16:37:21.467153+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":613234,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":159,"num_read_kb":154,"num_write":308,"num_write_kb":2995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":248,"ondisk_log_size":248,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":78156,"kb_used_data":10924,"kb_used_omap":0,"kb_used_meta":67136,"kb_avail":1124863668,"statfs":{"total":1151940427776,"available":1151860396032,"internally_reserved":0,"allocated":11186176,"data_stored":6983803,"data_compressed":106695,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":68747264},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22866,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":23,"num_read_kb":35,"num_write":127,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.986938"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157701+0000","last_change":"2023-11-16T16:37:13.941551+0000","last_active":"2023-11-16T16:37:15.157701+0000","last_peered":"2023-11-16T16:37:15.157701+0000","last_clean":"2023-11-16T16:37:15.157701+0000","last_became_active":"2023-11-16T16:37:13.941190+0000","last_became_peered":"2023-11-16T16:37:13.941190+0000","last_unstale":"2023-11-16T16:37:15.157701+0000","last_undegraded":"2023-11-16T16:37:15.157701+0000","last_fullsized":"2023-11-16T16:37:15.157701+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:26:51.970409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158170+0000","last_change":"2023-11-16T16:37:13.942300+0000","last_active":"2023-11-16T16:37:15.158170+0000","last_peered":"2023-11-16T16:37:15.158170+0000","last_clean":"2023-11-16T16:37:15.158170+0000","last_became_active":"2023-11-16T16:37:13.941966+0000","last_became_peered":"2023-11-16T16:37:13.941966+0000","last_unstale":"2023-11-16T16:37:15.158170+0000","last_undegraded":"2023-11-16T16:37:15.158170+0000","last_fullsized":"2023-11-16T16:37:15.158170+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:12:01.659205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158388+0000","last_change":"2023-11-16T16:37:13.943461+0000","last_active":"2023-11-16T16:37:15.158388+0000","last_peered":"2023-11-16T16:37:15.158388+0000","last_clean":"2023-11-16T16:37:15.158388+0000","last_became_active":"2023-11-16T16:37:13.943165+0000","last_became_peered":"2023-11-16T16:37:13.943165+0000","last_unstale":"2023-11-16T16:37:15.158388+0000","last_undegraded":"2023-11-16T16:37:15.158388+0000","last_fullsized":"2023-11-16T16:37:15.158388+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:09:30.792766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.122028+0000","last_change":"2023-11-16T16:37:13.940205+0000","last_active":"2023-11-16T16:37:15.122028+0000","last_peered":"2023-11-16T16:37:15.122028+0000","last_clean":"2023-11-16T16:37:15.122028+0000","last_became_active":"2023-11-16T16:37:13.939819+0000","last_became_peered":"2023-11-16T16:37:13.939819+0000","last_unstale":"2023-11-16T16:37:15.122028+0000","last_undegraded":"2023-11-16T16:37:15.122028+0000","last_fullsized":"2023-11-16T16:37:15.122028+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:40:08.471596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.923975+0000","last_change":"2023-11-16T16:37:13.939819+0000","last_active":"2023-11-16T16:37:14.923975+0000","last_peered":"2023-11-16T16:37:14.923975+0000","last_clean":"2023-11-16T16:37:14.923975+0000","last_became_active":"2023-11-16T16:37:13.939367+0000","last_became_peered":"2023-11-16T16:37:13.939367+0000","last_unstale":"2023-11-16T16:37:14.923975+0000","last_undegraded":"2023-11-16T16:37:14.923975+0000","last_fullsized":"2023-11-16T16:37:14.923975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:27:49.657269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.924379+0000","last_change":"2023-11-16T16:37:13.946613+0000","last_active":"2023-11-16T16:37:14.924379+0000","last_peered":"2023-11-16T16:37:14.924379+0000","last_clean":"2023-11-16T16:37:14.924379+0000","last_became_active":"2023-11-16T16:37:13.946281+0000","last_became_peered":"2023-11-16T16:37:13.946281+0000","last_unstale":"2023-11-16T16:37:14.924379+0000","last_undegraded":"2023-11-16T16:37:14.924379+0000","last_fullsized":"2023-11-16T16:37:14.924379+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:26:07.902412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.390884+0000","last_change":"2023-11-16T16:37:13.936899+0000","last_active":"2023-11-16T16:37:15.390884+0000","last_peered":"2023-11-16T16:37:15.390884+0000","last_clean":"2023-11-16T16:37:15.390884+0000","last_became_active":"2023-11-16T16:37:13.936543+0000","last_became_peered":"2023-11-16T16:37:13.936543+0000","last_unstale":"2023-11-16T16:37:15.390884+0000","last_undegraded":"2023-11-16T16:37:15.390884+0000","last_fullsized":"2023-11-16T16:37:15.390884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:55:14.512583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157287+0000","last_change":"2023-11-16T16:37:13.941463+0000","last_active":"2023-11-16T16:37:15.157287+0000","last_peered":"2023-11-16T16:37:15.157287+0000","last_clean":"2023-11-16T16:37:15.157287+0000","last_became_active":"2023-11-16T16:37:13.941125+0000","last_became_peered":"2023-11-16T16:37:13.941125+0000","last_unstale":"2023-11-16T16:37:15.157287+0000","last_undegraded":"2023-11-16T16:37:15.157287+0000","last_fullsized":"2023-11-16T16:37:15.157287+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:29:10.391228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157152+0000","last_change":"2023-11-16T16:37:13.934105+0000","last_active":"2023-11-16T16:37:15.157152+0000","last_peered":"2023-11-16T16:37:15.157152+0000","last_clean":"2023-11-16T16:37:15.157152+0000","last_became_active":"2023-11-16T16:37:13.933887+0000","last_became_peered":"2023-11-16T16:37:13.933887+0000","last_unstale":"2023-11-16T16:37:15.157152+0000","last_undegraded":"2023-11-16T16:37:15.157152+0000","last_fullsized":"2023-11-16T16:37:15.157152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:37:28.260833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.925511+0000","last_change":"2023-11-16T16:37:13.947266+0000","last_active":"2023-11-16T16:37:14.925511+0000","last_peered":"2023-11-16T16:37:14.925511+0000","last_clean":"2023-11-16T16:37:14.925511+0000","last_became_active":"2023-11-16T16:37:13.946919+0000","last_became_peered":"2023-11-16T16:37:13.946919+0000","last_unstale":"2023-11-16T16:37:14.925511+0000","last_undegraded":"2023-11-16T16:37:14.925511+0000","last_fullsized":"2023-11-16T16:37:14.925511+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:07:33.154784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.391182+0000","last_change":"2023-11-16T16:37:13.941597+0000","last_active":"2023-11-16T16:37:15.391182+0000","last_peered":"2023-11-16T16:37:15.391182+0000","last_clean":"2023-11-16T16:37:15.391182+0000","last_became_active":"2023-11-16T16:37:13.941253+0000","last_became_peered":"2023-11-16T16:37:13.941253+0000","last_unstale":"2023-11-16T16:37:15.391182+0000","last_undegraded":"2023-11-16T16:37:15.391182+0000","last_fullsized":"2023-11-16T16:37:15.391182+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:29:31.101716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.121996+0000","last_change":"2023-11-16T16:37:13.941673+0000","last_active":"2023-11-16T16:37:15.121996+0000","last_peered":"2023-11-16T16:37:15.121996+0000","last_clean":"2023-11-16T16:37:15.121996+0000","last_became_active":"2023-11-16T16:37:13.941345+0000","last_became_peered":"2023-11-16T16:37:13.941345+0000","last_unstale":"2023-11-16T16:37:15.121996+0000","last_undegraded":"2023-11-16T16:37:15.121996+0000","last_fullsized":"2023-11-16T16:37:15.121996+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:01:29.436252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157470+0000","last_change":"2023-11-16T16:37:13.947991+0000","last_active":"2023-11-16T16:37:15.157470+0000","last_peered":"2023-11-16T16:37:15.157470+0000","last_clean":"2023-11-16T16:37:15.157470+0000","last_became_active":"2023-11-16T16:37:13.947454+0000","last_became_peered":"2023-11-16T16:37:13.947454+0000","last_unstale":"2023-11-16T16:37:15.157470+0000","last_undegraded":"2023-11-16T16:37:15.157470+0000","last_fullsized":"2023-11-16T16:37:15.157470+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:35:41.400206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157374+0000","last_change":"2023-11-16T16:37:13.941502+0000","last_active":"2023-11-16T16:37:15.157374+0000","last_peered":"2023-11-16T16:37:15.157374+0000","last_clean":"2023-11-16T16:37:15.157374+0000","last_became_active":"2023-11-16T16:37:13.941191+0000","last_became_peered":"2023-11-16T16:37:13.941191+0000","last_unstale":"2023-11-16T16:37:15.157374+0000","last_undegraded":"2023-11-16T16:37:15.157374+0000","last_fullsized":"2023-11-16T16:37:15.157374+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:59:03.243696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.156825+0000","last_change":"2023-11-16T16:37:13.949198+0000","last_active":"2023-11-16T16:37:15.156825+0000","last_peered":"2023-11-16T16:37:15.156825+0000","last_clean":"2023-11-16T16:37:15.156825+0000","last_became_active":"2023-11-16T16:37:13.947147+0000","last_became_peered":"2023-11-16T16:37:13.947147+0000","last_unstale":"2023-11-16T16:37:15.156825+0000","last_undegraded":"2023-11-16T16:37:15.156825+0000","last_fullsized":"2023-11-16T16:37:15.156825+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:01:56.205870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.156530+0000","last_change":"2023-11-16T16:37:13.943915+0000","last_active":"2023-11-16T16:37:15.156530+0000","last_peered":"2023-11-16T16:37:15.156530+0000","last_clean":"2023-11-16T16:37:15.156530+0000","last_became_active":"2023-11-16T16:37:13.942995+0000","last_became_peered":"2023-11-16T16:37:13.942995+0000","last_unstale":"2023-11-16T16:37:15.156530+0000","last_undegraded":"2023-11-16T16:37:15.156530+0000","last_fullsized":"2023-11-16T16:37:15.156530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:35:55.774014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157841+0000","last_change":"2023-11-16T16:37:13.941816+0000","last_active":"2023-11-16T16:37:15.157841+0000","last_peered":"2023-11-16T16:37:15.157841+0000","last_clean":"2023-11-16T16:37:15.157841+0000","last_became_active":"2023-11-16T16:37:13.941541+0000","last_became_peered":"2023-11-16T16:37:13.941541+0000","last_unstale":"2023-11-16T16:37:15.157841+0000","last_undegraded":"2023-11-16T16:37:15.157841+0000","last_fullsized":"2023-11-16T16:37:15.157841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:18:40.904015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158134+0000","last_change":"2023-11-16T16:37:13.940639+0000","last_active":"2023-11-16T16:37:15.158134+0000","last_peered":"2023-11-16T16:37:15.158134+0000","last_clean":"2023-11-16T16:37:15.158134+0000","last_became_active":"2023-11-16T16:37:13.940323+0000","last_became_peered":"2023-11-16T16:37:13.940323+0000","last_unstale":"2023-11-16T16:37:15.158134+0000","last_undegraded":"2023-11-16T16:37:15.158134+0000","last_fullsized":"2023-11-16T16:37:15.158134+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:54:11.977928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158148+0000","last_change":"2023-11-16T16:37:13.934802+0000","last_active":"2023-11-16T16:37:15.158148+0000","last_peered":"2023-11-16T16:37:15.158148+0000","last_clean":"2023-11-16T16:37:15.158148+0000","last_became_active":"2023-11-16T16:37:13.934496+0000","last_became_peered":"2023-11-16T16:37:13.934496+0000","last_unstale":"2023-11-16T16:37:15.158148+0000","last_undegraded":"2023-11-16T16:37:15.158148+0000","last_fullsized":"2023-11-16T16:37:15.158148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:50:10.271792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'9","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:18.297486+0000","last_change":"2023-11-16T16:37:13.499284+0000","last_active":"2023-11-16T16:37:18.297486+0000","last_peered":"2023-11-16T16:37:18.297486+0000","last_clean":"2023-11-16T16:37:18.297486+0000","last_became_active":"2023-11-16T16:37:13.498931+0000","last_became_peered":"2023-11-16T16:37:13.498931+0000","last_unstale":"2023-11-16T16:37:18.297486+0000","last_undegraded":"2023-11-16T16:37:18.297486+0000","last_fullsized":"2023-11-16T16:37:18.297486+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:52:32.314115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157923+0000","last_change":"2023-11-16T16:37:13.939631+0000","last_active":"2023-11-16T16:37:15.157923+0000","last_peered":"2023-11-16T16:37:15.157923+0000","last_clean":"2023-11-16T16:37:15.157923+0000","last_became_active":"2023-11-16T16:37:13.939318+0000","last_became_peered":"2023-11-16T16:37:13.939318+0000","last_unstale":"2023-11-16T16:37:15.157923+0000","last_undegraded":"2023-11-16T16:37:15.157923+0000","last_fullsized":"2023-11-16T16:37:15.157923+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:51:05.730477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157928+0000","last_change":"2023-11-16T16:37:13.495972+0000","last_active":"2023-11-16T16:37:15.157928+0000","last_peered":"2023-11-16T16:37:15.157928+0000","last_clean":"2023-11-16T16:37:15.157928+0000","last_became_active":"2023-11-16T16:37:13.495610+0000","last_became_peered":"2023-11-16T16:37:13.495610+0000","last_unstale":"2023-11-16T16:37:15.157928+0000","last_undegraded":"2023-11-16T16:37:15.157928+0000","last_fullsized":"2023-11-16T16:37:15.157928+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:58:27.825452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158624+0000","last_change":"2023-11-16T16:37:13.935236+0000","last_active":"2023-11-16T16:37:15.158624+0000","last_peered":"2023-11-16T16:37:15.158624+0000","last_clean":"2023-11-16T16:37:15.158624+0000","last_became_active":"2023-11-16T16:37:13.934920+0000","last_became_peered":"2023-11-16T16:37:13.934920+0000","last_unstale":"2023-11-16T16:37:15.158624+0000","last_undegraded":"2023-11-16T16:37:15.158624+0000","last_fullsized":"2023-11-16T16:37:15.158624+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:58:27.305887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"84'4","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:18.296421+0000","last_change":"2023-11-16T16:37:13.509828+0000","last_active":"2023-11-16T16:37:18.296421+0000","last_peered":"2023-11-16T16:37:18.296421+0000","last_clean":"2023-11-16T16:37:18.296421+0000","last_became_active":"2023-11-16T16:37:13.509431+0000","last_became_peered":"2023-11-16T16:37:13.509431+0000","last_unstale":"2023-11-16T16:37:18.296421+0000","last_undegraded":"2023-11-16T16:37:18.296421+0000","last_fullsized":"2023-11-16T16:37:18.296421+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:49:39.240865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.156807+0000","last_change":"2023-11-16T16:37:13.940838+0000","last_active":"2023-11-16T16:37:15.156807+0000","last_peered":"2023-11-16T16:37:15.156807+0000","last_clean":"2023-11-16T16:37:15.156807+0000","last_became_active":"2023-11-16T16:37:13.940538+0000","last_became_peered":"2023-11-16T16:37:13.940538+0000","last_unstale":"2023-11-16T16:37:15.156807+0000","last_undegraded":"2023-11-16T16:37:15.156807+0000","last_fullsized":"2023-11-16T16:37:15.156807+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:24:41.595556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.925256+0000","last_change":"2023-11-16T16:37:13.495114+0000","last_active":"2023-11-16T16:37:14.925256+0000","last_peered":"2023-11-16T16:37:14.925256+0000","last_clean":"2023-11-16T16:37:14.925256+0000","last_became_active":"2023-11-16T16:37:13.494807+0000","last_became_peered":"2023-11-16T16:37:13.494807+0000","last_unstale":"2023-11-16T16:37:14.925256+0000","last_undegraded":"2023-11-16T16:37:14.925256+0000","last_fullsized":"2023-11-16T16:37:14.925256+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:24:04.736603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.924173+0000","last_change":"2023-11-16T16:37:13.943652+0000","last_active":"2023-11-16T16:37:14.924173+0000","last_peered":"2023-11-16T16:37:14.924173+0000","last_clean":"2023-11-16T16:37:14.924173+0000","last_became_active":"2023-11-16T16:37:13.943298+0000","last_became_peered":"2023-11-16T16:37:13.943298+0000","last_unstale":"2023-11-16T16:37:14.924173+0000","last_undegraded":"2023-11-16T16:37:14.924173+0000","last_fullsized":"2023-11-16T16:37:14.924173+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:34:34.876458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"84'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:20.303742+0000","last_change":"2023-11-16T16:37:13.509801+0000","last_active":"2023-11-16T16:37:20.303742+0000","last_peered":"2023-11-16T16:37:20.303742+0000","last_clean":"2023-11-16T16:37:20.303742+0000","last_became_active":"2023-11-16T16:37:13.509389+0000","last_became_peered":"2023-11-16T16:37:13.509389+0000","last_unstale":"2023-11-16T16:37:20.303742+0000","last_undegraded":"2023-11-16T16:37:20.303742+0000","last_fullsized":"2023-11-16T16:37:20.303742+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:55:02.922591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.924573+0000","last_change":"2023-11-16T16:37:13.933871+0000","last_active":"2023-11-16T16:37:14.924573+0000","last_peered":"2023-11-16T16:37:14.924573+0000","last_clean":"2023-11-16T16:37:14.924573+0000","last_became_active":"2023-11-16T16:37:13.933549+0000","last_became_peered":"2023-11-16T16:37:13.933549+0000","last_unstale":"2023-11-16T16:37:14.924573+0000","last_undegraded":"2023-11-16T16:37:14.924573+0000","last_fullsized":"2023-11-16T16:37:14.924573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:16:26.022577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"84'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:18.294818+0000","last_change":"2023-11-16T16:37:13.501339+0000","last_active":"2023-11-16T16:37:18.294818+0000","last_peered":"2023-11-16T16:37:18.294818+0000","last_clean":"2023-11-16T16:37:18.294818+0000","last_became_active":"2023-11-16T16:37:13.500949+0000","last_became_peered":"2023-11-16T16:37:13.500949+0000","last_unstale":"2023-11-16T16:37:18.294818+0000","last_undegraded":"2023-11-16T16:37:18.294818+0000","last_fullsized":"2023-11-16T16:37:18.294818+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:31:08.744759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.923893+0000","last_change":"2023-11-16T16:37:13.944158+0000","last_active":"2023-11-16T16:37:14.923893+0000","last_peered":"2023-11-16T16:37:14.923893+0000","last_clean":"2023-11-16T16:37:14.923893+0000","last_became_active":"2023-11-16T16:37:13.943833+0000","last_became_peered":"2023-11-16T16:37:13.943833+0000","last_unstale":"2023-11-16T16:37:14.923893+0000","last_undegraded":"2023-11-16T16:37:14.923893+0000","last_fullsized":"2023-11-16T16:37:14.923893+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:36:01.615006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"84'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:18.295129+0000","last_change":"2023-11-16T16:37:13.505481+0000","last_active":"2023-11-16T16:37:18.295129+0000","last_peered":"2023-11-16T16:37:18.295129+0000","last_clean":"2023-11-16T16:37:18.295129+0000","last_became_active":"2023-11-16T16:37:13.505085+0000","last_became_peered":"2023-11-16T16:37:13.505085+0000","last_unstale":"2023-11-16T16:37:18.295129+0000","last_undegraded":"2023-11-16T16:37:18.295129+0000","last_fullsized":"2023-11-16T16:37:18.295129+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:11:35.628249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157157+0000","last_change":"2023-11-16T16:37:13.943985+0000","last_active":"2023-11-16T16:37:15.157157+0000","last_peered":"2023-11-16T16:37:15.157157+0000","last_clean":"2023-11-16T16:37:15.157157+0000","last_became_active":"2023-11-16T16:37:13.943628+0000","last_became_peered":"2023-11-16T16:37:13.943628+0000","last_unstale":"2023-11-16T16:37:15.157157+0000","last_undegraded":"2023-11-16T16:37:15.157157+0000","last_fullsized":"2023-11-16T16:37:15.157157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:19:10.518280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.462357+0000","last_change":"2023-11-16T16:37:13.498648+0000","last_active":"2023-11-16T16:37:15.462357+0000","last_peered":"2023-11-16T16:37:15.462357+0000","last_clean":"2023-11-16T16:37:15.462357+0000","last_became_active":"2023-11-16T16:37:13.497390+0000","last_became_peered":"2023-11-16T16:37:13.497390+0000","last_unstale":"2023-11-16T16:37:15.462357+0000","last_undegraded":"2023-11-16T16:37:15.462357+0000","last_fullsized":"2023-11-16T16:37:15.462357+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:26:59.991663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157618+0000","last_change":"2023-11-16T16:37:13.942337+0000","last_active":"2023-11-16T16:37:15.157618+0000","last_peered":"2023-11-16T16:37:15.157618+0000","last_clean":"2023-11-16T16:37:15.157618+0000","last_became_active":"2023-11-16T16:37:13.941982+0000","last_became_peered":"2023-11-16T16:37:13.941982+0000","last_unstale":"2023-11-16T16:37:15.157618+0000","last_undegraded":"2023-11-16T16:37:15.157618+0000","last_fullsized":"2023-11-16T16:37:15.157618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:38:42.000956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:19.300306+0000","last_change":"2023-11-16T16:37:13.503690+0000","last_active":"2023-11-16T16:37:19.300306+0000","last_peered":"2023-11-16T16:37:19.300306+0000","last_clean":"2023-11-16T16:37:19.300306+0000","last_became_active":"2023-11-16T16:37:13.503203+0000","last_became_peered":"2023-11-16T16:37:13.503203+0000","last_unstale":"2023-11-16T16:37:19.300306+0000","last_undegraded":"2023-11-16T16:37:19.300306+0000","last_fullsized":"2023-11-16T16:37:19.300306+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:27:52.629218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.156964+0000","last_change":"2023-11-16T16:37:13.943975+0000","last_active":"2023-11-16T16:37:15.156964+0000","last_peered":"2023-11-16T16:37:15.156964+0000","last_clean":"2023-11-16T16:37:15.156964+0000","last_became_active":"2023-11-16T16:37:13.943611+0000","last_became_peered":"2023-11-16T16:37:13.943611+0000","last_unstale":"2023-11-16T16:37:15.156964+0000","last_undegraded":"2023-11-16T16:37:15.156964+0000","last_fullsized":"2023-11-16T16:37:15.156964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:21:29.078427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:17.285557+0000","last_change":"2023-11-16T16:37:13.506076+0000","last_active":"2023-11-16T16:37:17.285557+0000","last_peered":"2023-11-16T16:37:17.285557+0000","last_clean":"2023-11-16T16:37:17.285557+0000","last_became_active":"2023-11-16T16:37:13.505715+0000","last_became_peered":"2023-11-16T16:37:13.505715+0000","last_unstale":"2023-11-16T16:37:17.285557+0000","last_undegraded":"2023-11-16T16:37:17.285557+0000","last_fullsized":"2023-11-16T16:37:17.285557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:03:10.081004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158690+0000","last_change":"2023-11-16T16:37:13.949031+0000","last_active":"2023-11-16T16:37:15.158690+0000","last_peered":"2023-11-16T16:37:15.158690+0000","last_clean":"2023-11-16T16:37:15.158690+0000","last_became_active":"2023-11-16T16:37:13.946923+0000","last_became_peered":"2023-11-16T16:37:13.946923+0000","last_unstale":"2023-11-16T16:37:15.158690+0000","last_undegraded":"2023-11-16T16:37:15.158690+0000","last_fullsized":"2023-11-16T16:37:15.158690+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:00:35.683602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:17.286359+0000","last_change":"2023-11-16T16:37:13.495003+0000","last_active":"2023-11-16T16:37:17.286359+0000","last_peered":"2023-11-16T16:37:17.286359+0000","last_clean":"2023-11-16T16:37:17.286359+0000","last_became_active":"2023-11-16T16:37:13.494563+0000","last_became_peered":"2023-11-16T16:37:13.494563+0000","last_unstale":"2023-11-16T16:37:17.286359+0000","last_undegraded":"2023-11-16T16:37:17.286359+0000","last_fullsized":"2023-11-16T16:37:17.286359+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:12:53.840907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.156822+0000","last_change":"2023-11-16T16:37:13.943736+0000","last_active":"2023-11-16T16:37:15.156822+0000","last_peered":"2023-11-16T16:37:15.156822+0000","last_clean":"2023-11-16T16:37:15.156822+0000","last_became_active":"2023-11-16T16:37:13.943391+0000","last_became_peered":"2023-11-16T16:37:13.943391+0000","last_unstale":"2023-11-16T16:37:15.156822+0000","last_undegraded":"2023-11-16T16:37:15.156822+0000","last_fullsized":"2023-11-16T16:37:15.156822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:03:21.402516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'14","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:20.305738+0000","last_change":"2023-11-16T16:37:13.506508+0000","last_active":"2023-11-16T16:37:20.305738+0000","last_peered":"2023-11-16T16:37:20.305738+0000","last_clean":"2023-11-16T16:37:20.305738+0000","last_became_active":"2023-11-16T16:37:13.506045+0000","last_became_peered":"2023-11-16T16:37:13.506045+0000","last_unstale":"2023-11-16T16:37:20.305738+0000","last_undegraded":"2023-11-16T16:37:20.305738+0000","last_fullsized":"2023-11-16T16:37:20.305738+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:26:04.265699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18929,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":21,"num_write":15,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.924203+0000","last_change":"2023-11-16T16:37:13.939717+0000","last_active":"2023-11-16T16:37:14.924203+0000","last_peered":"2023-11-16T16:37:14.924203+0000","last_clean":"2023-11-16T16:37:14.924203+0000","last_became_active":"2023-11-16T16:37:13.939369+0000","last_became_peered":"2023-11-16T16:37:13.939369+0000","last_unstale":"2023-11-16T16:37:14.924203+0000","last_undegraded":"2023-11-16T16:37:14.924203+0000","last_fullsized":"2023-11-16T16:37:14.924203+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:15:46.898377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'6","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:18.296464+0000","last_change":"2023-11-16T16:37:13.506056+0000","last_active":"2023-11-16T16:37:18.296464+0000","last_peered":"2023-11-16T16:37:18.296464+0000","last_clean":"2023-11-16T16:37:18.296464+0000","last_became_active":"2023-11-16T16:37:13.505707+0000","last_became_peered":"2023-11-16T16:37:13.505707+0000","last_unstale":"2023-11-16T16:37:18.296464+0000","last_undegraded":"2023-11-16T16:37:18.296464+0000","last_fullsized":"2023-11-16T16:37:18.296464+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:36:08.667708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157120+0000","last_change":"2023-11-16T16:37:13.947764+0000","last_active":"2023-11-16T16:37:15.157120+0000","last_peered":"2023-11-16T16:37:15.157120+0000","last_clean":"2023-11-16T16:37:15.157120+0000","last_became_active":"2023-11-16T16:37:13.947510+0000","last_became_peered":"2023-11-16T16:37:13.947510+0000","last_unstale":"2023-11-16T16:37:15.157120+0000","last_undegraded":"2023-11-16T16:37:15.157120+0000","last_fullsized":"2023-11-16T16:37:15.157120+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:08:15.784489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:17.287513+0000","last_change":"2023-11-16T16:37:13.498178+0000","last_active":"2023-11-16T16:37:17.287513+0000","last_peered":"2023-11-16T16:37:17.287513+0000","last_clean":"2023-11-16T16:37:17.287513+0000","last_became_active":"2023-11-16T16:37:13.497312+0000","last_became_peered":"2023-11-16T16:37:13.497312+0000","last_unstale":"2023-11-16T16:37:17.287513+0000","last_undegraded":"2023-11-16T16:37:17.287513+0000","last_fullsized":"2023-11-16T16:37:17.287513+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:01:58.840690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.391132+0000","last_change":"2023-11-16T16:37:13.947854+0000","last_active":"2023-11-16T16:37:15.391132+0000","last_peered":"2023-11-16T16:37:15.391132+0000","last_clean":"2023-11-16T16:37:15.391132+0000","last_became_active":"2023-11-16T16:37:13.947530+0000","last_became_peered":"2023-11-16T16:37:13.947530+0000","last_unstale":"2023-11-16T16:37:15.391132+0000","last_undegraded":"2023-11-16T16:37:15.391132+0000","last_fullsized":"2023-11-16T16:37:15.391132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:25:25.919855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:18.296418+0000","last_change":"2023-11-16T16:37:13.500127+0000","last_active":"2023-11-16T16:37:18.296418+0000","last_peered":"2023-11-16T16:37:18.296418+0000","last_clean":"2023-11-16T16:37:18.296418+0000","last_became_active":"2023-11-16T16:37:13.499758+0000","last_became_peered":"2023-11-16T16:37:13.499758+0000","last_unstale":"2023-11-16T16:37:18.296418+0000","last_undegraded":"2023-11-16T16:37:18.296418+0000","last_fullsized":"2023-11-16T16:37:18.296418+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:12:24.610215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.390796+0000","last_change":"2023-11-16T16:37:13.940962+0000","last_active":"2023-11-16T16:37:15.390796+0000","last_peered":"2023-11-16T16:37:15.390796+0000","last_clean":"2023-11-16T16:37:15.390796+0000","last_became_active":"2023-11-16T16:37:13.940450+0000","last_became_peered":"2023-11-16T16:37:13.940450+0000","last_unstale":"2023-11-16T16:37:15.390796+0000","last_undegraded":"2023-11-16T16:37:15.390796+0000","last_fullsized":"2023-11-16T16:37:15.390796+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:20:04.879494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'15","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:18.297513+0000","last_change":"2023-11-16T16:37:13.493017+0000","last_active":"2023-11-16T16:37:18.297513+0000","last_peered":"2023-11-16T16:37:18.297513+0000","last_clean":"2023-11-16T16:37:18.297513+0000","last_became_active":"2023-11-16T16:37:13.492599+0000","last_became_peered":"2023-11-16T16:37:13.492599+0000","last_unstale":"2023-11-16T16:37:18.297513+0000","last_undegraded":"2023-11-16T16:37:18.297513+0000","last_fullsized":"2023-11-16T16:37:18.297513+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:25:52.252764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.924063+0000","last_change":"2023-11-16T16:37:13.944878+0000","last_active":"2023-11-16T16:37:14.924063+0000","last_peered":"2023-11-16T16:37:14.924063+0000","last_clean":"2023-11-16T16:37:14.924063+0000","last_became_active":"2023-11-16T16:37:13.944580+0000","last_became_peered":"2023-11-16T16:37:13.944580+0000","last_unstale":"2023-11-16T16:37:14.924063+0000","last_undegraded":"2023-11-16T16:37:14.924063+0000","last_fullsized":"2023-11-16T16:37:14.924063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:01:52.805052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:19.300067+0000","last_change":"2023-11-16T16:37:13.499139+0000","last_active":"2023-11-16T16:37:19.300067+0000","last_peered":"2023-11-16T16:37:19.300067+0000","last_clean":"2023-11-16T16:37:19.300067+0000","last_became_active":"2023-11-16T16:37:13.498755+0000","last_became_peered":"2023-11-16T16:37:13.498755+0000","last_unstale":"2023-11-16T16:37:19.300067+0000","last_undegraded":"2023-11-16T16:37:19.300067+0000","last_fullsized":"2023-11-16T16:37:19.300067+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:07:40.535306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157785+0000","last_change":"2023-11-16T16:37:13.946187+0000","last_active":"2023-11-16T16:37:15.157785+0000","last_peered":"2023-11-16T16:37:15.157785+0000","last_clean":"2023-11-16T16:37:15.157785+0000","last_became_active":"2023-11-16T16:37:13.945887+0000","last_became_peered":"2023-11-16T16:37:13.945887+0000","last_unstale":"2023-11-16T16:37:15.157785+0000","last_undegraded":"2023-11-16T16:37:15.157785+0000","last_fullsized":"2023-11-16T16:37:15.157785+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:29:41.439670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157191+0000","last_change":"2023-11-16T16:37:13.507150+0000","last_active":"2023-11-16T16:37:15.157191+0000","last_peered":"2023-11-16T16:37:15.157191+0000","last_clean":"2023-11-16T16:37:15.157191+0000","last_became_active":"2023-11-16T16:37:13.506814+0000","last_became_peered":"2023-11-16T16:37:13.506814+0000","last_unstale":"2023-11-16T16:37:15.157191+0000","last_undegraded":"2023-11-16T16:37:15.157191+0000","last_fullsized":"2023-11-16T16:37:15.157191+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:58:14.692816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.462555+0000","last_change":"2023-11-16T16:37:13.947866+0000","last_active":"2023-11-16T16:37:15.462555+0000","last_peered":"2023-11-16T16:37:15.462555+0000","last_clean":"2023-11-16T16:37:15.462555+0000","last_became_active":"2023-11-16T16:37:13.947544+0000","last_became_peered":"2023-11-16T16:37:13.947544+0000","last_unstale":"2023-11-16T16:37:15.462555+0000","last_undegraded":"2023-11-16T16:37:15.462555+0000","last_fullsized":"2023-11-16T16:37:15.462555+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:58:42.237793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"84'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.391037+0000","last_change":"2023-11-16T16:37:13.497800+0000","last_active":"2023-11-16T16:37:15.391037+0000","last_peered":"2023-11-16T16:37:15.391037+0000","last_clean":"2023-11-16T16:37:15.391037+0000","last_became_active":"2023-11-16T16:37:13.497414+0000","last_became_peered":"2023-11-16T16:37:13.497414+0000","last_unstale":"2023-11-16T16:37:15.391037+0000","last_undegraded":"2023-11-16T16:37:15.391037+0000","last_fullsized":"2023-11-16T16:37:15.391037+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:57:26.591481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.924384+0000","last_change":"2023-11-16T16:37:13.946780+0000","last_active":"2023-11-16T16:37:14.924384+0000","last_peered":"2023-11-16T16:37:14.924384+0000","last_clean":"2023-11-16T16:37:14.924384+0000","last_became_active":"2023-11-16T16:37:13.946441+0000","last_became_peered":"2023-11-16T16:37:13.946441+0000","last_unstale":"2023-11-16T16:37:14.924384+0000","last_undegraded":"2023-11-16T16:37:14.924384+0000","last_fullsized":"2023-11-16T16:37:14.924384+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:20:09.943373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.923519+0000","last_change":"2023-11-16T16:37:13.937898+0000","last_active":"2023-11-16T16:37:14.923519+0000","last_peered":"2023-11-16T16:37:14.923519+0000","last_clean":"2023-11-16T16:37:14.923519+0000","last_became_active":"2023-11-16T16:37:13.937598+0000","last_became_peered":"2023-11-16T16:37:13.937598+0000","last_unstale":"2023-11-16T16:37:14.923519+0000","last_undegraded":"2023-11-16T16:37:14.923519+0000","last_fullsized":"2023-11-16T16:37:14.923519+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:57:03.646617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"84'4","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:18.297366+0000","last_change":"2023-11-16T16:37:13.498129+0000","last_active":"2023-11-16T16:37:18.297366+0000","last_peered":"2023-11-16T16:37:18.297366+0000","last_clean":"2023-11-16T16:37:18.297366+0000","last_became_active":"2023-11-16T16:37:13.497765+0000","last_became_peered":"2023-11-16T16:37:13.497765+0000","last_unstale":"2023-11-16T16:37:18.297366+0000","last_undegraded":"2023-11-16T16:37:18.297366+0000","last_fullsized":"2023-11-16T16:37:18.297366+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:13:33.579950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158156+0000","last_change":"2023-11-16T16:37:13.938417+0000","last_active":"2023-11-16T16:37:15.158156+0000","last_peered":"2023-11-16T16:37:15.158156+0000","last_clean":"2023-11-16T16:37:15.158156+0000","last_became_active":"2023-11-16T16:37:13.938114+0000","last_became_peered":"2023-11-16T16:37:13.938114+0000","last_unstale":"2023-11-16T16:37:15.158156+0000","last_undegraded":"2023-11-16T16:37:15.158156+0000","last_fullsized":"2023-11-16T16:37:15.158156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:35:24.220360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"84'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:17.288003+0000","last_change":"2023-11-16T16:37:13.499045+0000","last_active":"2023-11-16T16:37:17.288003+0000","last_peered":"2023-11-16T16:37:17.288003+0000","last_clean":"2023-11-16T16:37:17.288003+0000","last_became_active":"2023-11-16T16:37:13.498598+0000","last_became_peered":"2023-11-16T16:37:13.498598+0000","last_unstale":"2023-11-16T16:37:17.288003+0000","last_undegraded":"2023-11-16T16:37:17.288003+0000","last_fullsized":"2023-11-16T16:37:17.288003+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:25:26.520931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":345,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.390796+0000","last_change":"2023-11-16T16:36:14.090213+0000","last_active":"2023-11-16T16:37:15.390796+0000","last_peered":"2023-11-16T16:37:15.390796+0000","last_clean":"2023-11-16T16:37:15.390796+0000","last_became_active":"2023-11-16T16:36:14.089742+0000","last_became_peered":"2023-11-16T16:36:14.089742+0000","last_unstale":"2023-11-16T16:37:15.390796+0000","last_undegraded":"2023-11-16T16:37:15.390796+0000","last_fullsized":"2023-11-16T16:37:15.390796+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:33:16.178716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:33:16.178716+0000","last_clean_scrub_stamp":"2023-11-16T16:33:16.178716+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:11:37.401838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157103+0000","last_change":"2023-11-16T16:37:13.945380+0000","last_active":"2023-11-16T16:37:15.157103+0000","last_peered":"2023-11-16T16:37:15.157103+0000","last_clean":"2023-11-16T16:37:15.157103+0000","last_became_active":"2023-11-16T16:37:13.945038+0000","last_became_peered":"2023-11-16T16:37:13.945038+0000","last_unstale":"2023-11-16T16:37:15.157103+0000","last_undegraded":"2023-11-16T16:37:15.157103+0000","last_fullsized":"2023-11-16T16:37:15.157103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:35:29.662867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"84'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:18.296049+0000","last_change":"2023-11-16T16:37:13.497586+0000","last_active":"2023-11-16T16:37:18.296049+0000","last_peered":"2023-11-16T16:37:18.296049+0000","last_clean":"2023-11-16T16:37:18.296049+0000","last_became_active":"2023-11-16T16:37:13.497233+0000","last_became_peered":"2023-11-16T16:37:13.497233+0000","last_unstale":"2023-11-16T16:37:18.296049+0000","last_undegraded":"2023-11-16T16:37:18.296049+0000","last_fullsized":"2023-11-16T16:37:18.296049+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:24:30.086496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"84'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:17.285613+0000","last_change":"2023-11-16T16:37:13.500621+0000","last_active":"2023-11-16T16:37:17.285613+0000","last_peered":"2023-11-16T16:37:17.285613+0000","last_clean":"2023-11-16T16:37:17.285613+0000","last_became_active":"2023-11-16T16:37:13.500265+0000","last_became_peered":"2023-11-16T16:37:13.500265+0000","last_unstale":"2023-11-16T16:37:17.285613+0000","last_undegraded":"2023-11-16T16:37:17.285613+0000","last_fullsized":"2023-11-16T16:37:17.285613+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:51:28.450861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157560+0000","last_change":"2023-11-16T16:37:13.930992+0000","last_active":"2023-11-16T16:37:15.157560+0000","last_peered":"2023-11-16T16:37:15.157560+0000","last_clean":"2023-11-16T16:37:15.157560+0000","last_became_active":"2023-11-16T16:37:13.930641+0000","last_became_peered":"2023-11-16T16:37:13.930641+0000","last_unstale":"2023-11-16T16:37:15.157560+0000","last_undegraded":"2023-11-16T16:37:15.157560+0000","last_fullsized":"2023-11-16T16:37:15.157560+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:58:37.743753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:18.297061+0000","last_change":"2023-11-16T16:37:13.498637+0000","last_active":"2023-11-16T16:37:18.297061+0000","last_peered":"2023-11-16T16:37:18.297061+0000","last_clean":"2023-11-16T16:37:18.297061+0000","last_became_active":"2023-11-16T16:37:13.498341+0000","last_became_peered":"2023-11-16T16:37:13.498341+0000","last_unstale":"2023-11-16T16:37:18.297061+0000","last_undegraded":"2023-11-16T16:37:18.297061+0000","last_fullsized":"2023-11-16T16:37:18.297061+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:00:31.333304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158333+0000","last_change":"2023-11-16T16:37:13.944142+0000","last_active":"2023-11-16T16:37:15.158333+0000","last_peered":"2023-11-16T16:37:15.158333+0000","last_clean":"2023-11-16T16:37:15.158333+0000","last_became_active":"2023-11-16T16:37:13.943758+0000","last_became_peered":"2023-11-16T16:37:13.943758+0000","last_unstale":"2023-11-16T16:37:15.158333+0000","last_undegraded":"2023-11-16T16:37:15.158333+0000","last_fullsized":"2023-11-16T16:37:15.158333+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:55:52.343549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'4","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:17.287097+0000","last_change":"2023-11-16T16:37:13.496900+0000","last_active":"2023-11-16T16:37:17.287097+0000","last_peered":"2023-11-16T16:37:17.287097+0000","last_clean":"2023-11-16T16:37:17.287097+0000","last_became_active":"2023-11-16T16:37:13.496528+0000","last_became_peered":"2023-11-16T16:37:13.496528+0000","last_unstale":"2023-11-16T16:37:17.287097+0000","last_undegraded":"2023-11-16T16:37:17.287097+0000","last_fullsized":"2023-11-16T16:37:17.287097+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:37:22.646222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158258+0000","last_change":"2023-11-16T16:37:13.943104+0000","last_active":"2023-11-16T16:37:15.158258+0000","last_peered":"2023-11-16T16:37:15.158258+0000","last_clean":"2023-11-16T16:37:15.158258+0000","last_became_active":"2023-11-16T16:37:13.942774+0000","last_became_peered":"2023-11-16T16:37:13.942774+0000","last_unstale":"2023-11-16T16:37:15.158258+0000","last_undegraded":"2023-11-16T16:37:15.158258+0000","last_fullsized":"2023-11-16T16:37:15.158258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:00:01.892620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:17.287386+0000","last_change":"2023-11-16T16:37:13.505994+0000","last_active":"2023-11-16T16:37:17.287386+0000","last_peered":"2023-11-16T16:37:17.287386+0000","last_clean":"2023-11-16T16:37:17.287386+0000","last_became_active":"2023-11-16T16:37:13.505663+0000","last_became_peered":"2023-11-16T16:37:13.505663+0000","last_unstale":"2023-11-16T16:37:17.287386+0000","last_undegraded":"2023-11-16T16:37:17.287386+0000","last_fullsized":"2023-11-16T16:37:17.287386+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:11:39.436119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.923847+0000","last_change":"2023-11-16T16:37:13.941970+0000","last_active":"2023-11-16T16:37:14.923847+0000","last_peered":"2023-11-16T16:37:14.923847+0000","last_clean":"2023-11-16T16:37:14.923847+0000","last_became_active":"2023-11-16T16:37:13.941649+0000","last_became_peered":"2023-11-16T16:37:13.941649+0000","last_unstale":"2023-11-16T16:37:14.923847+0000","last_undegraded":"2023-11-16T16:37:14.923847+0000","last_fullsized":"2023-11-16T16:37:14.923847+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:00:21.646221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157550+0000","last_change":"2023-11-16T16:37:13.503429+0000","last_active":"2023-11-16T16:37:15.157550+0000","last_peered":"2023-11-16T16:37:15.157550+0000","last_clean":"2023-11-16T16:37:15.157550+0000","last_became_active":"2023-11-16T16:37:13.503114+0000","last_became_peered":"2023-11-16T16:37:13.503114+0000","last_unstale":"2023-11-16T16:37:15.157550+0000","last_undegraded":"2023-11-16T16:37:15.157550+0000","last_fullsized":"2023-11-16T16:37:15.157550+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:05:42.643043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157164+0000","last_change":"2023-11-16T16:37:13.941225+0000","last_active":"2023-11-16T16:37:15.157164+0000","last_peered":"2023-11-16T16:37:15.157164+0000","last_clean":"2023-11-16T16:37:15.157164+0000","last_became_active":"2023-11-16T16:37:13.940964+0000","last_became_peered":"2023-11-16T16:37:13.940964+0000","last_unstale":"2023-11-16T16:37:15.157164+0000","last_undegraded":"2023-11-16T16:37:15.157164+0000","last_fullsized":"2023-11-16T16:37:15.157164+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:16:06.854930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157641+0000","last_change":"2023-11-16T16:37:13.510154+0000","last_active":"2023-11-16T16:37:15.157641+0000","last_peered":"2023-11-16T16:37:15.157641+0000","last_clean":"2023-11-16T16:37:15.157641+0000","last_became_active":"2023-11-16T16:37:13.509790+0000","last_became_peered":"2023-11-16T16:37:13.509790+0000","last_unstale":"2023-11-16T16:37:15.157641+0000","last_undegraded":"2023-11-16T16:37:15.157641+0000","last_fullsized":"2023-11-16T16:37:15.157641+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T16:54:36.581541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158172+0000","last_change":"2023-11-16T16:37:13.942203+0000","last_active":"2023-11-16T16:37:15.158172+0000","last_peered":"2023-11-16T16:37:15.158172+0000","last_clean":"2023-11-16T16:37:15.158172+0000","last_became_active":"2023-11-16T16:37:13.941766+0000","last_became_peered":"2023-11-16T16:37:13.941766+0000","last_unstale":"2023-11-16T16:37:15.158172+0000","last_undegraded":"2023-11-16T16:37:15.158172+0000","last_fullsized":"2023-11-16T16:37:15.158172+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:12:25.582734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:17.286155+0000","last_change":"2023-11-16T16:37:13.509854+0000","last_active":"2023-11-16T16:37:17.286155+0000","last_peered":"2023-11-16T16:37:17.286155+0000","last_clean":"2023-11-16T16:37:17.286155+0000","last_became_active":"2023-11-16T16:37:13.509466+0000","last_became_peered":"2023-11-16T16:37:13.509466+0000","last_unstale":"2023-11-16T16:37:17.286155+0000","last_undegraded":"2023-11-16T16:37:17.286155+0000","last_fullsized":"2023-11-16T16:37:17.286155+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:05:41.806694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.391285+0000","last_change":"2023-11-16T16:37:13.947906+0000","last_active":"2023-11-16T16:37:15.391285+0000","last_peered":"2023-11-16T16:37:15.391285+0000","last_clean":"2023-11-16T16:37:15.391285+0000","last_became_active":"2023-11-16T16:37:13.947602+0000","last_became_peered":"2023-11-16T16:37:13.947602+0000","last_unstale":"2023-11-16T16:37:15.391285+0000","last_undegraded":"2023-11-16T16:37:15.391285+0000","last_fullsized":"2023-11-16T16:37:15.391285+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:16:47.335757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:18.297466+0000","last_change":"2023-11-16T16:37:13.506108+0000","last_active":"2023-11-16T16:37:18.297466+0000","last_peered":"2023-11-16T16:37:18.297466+0000","last_clean":"2023-11-16T16:37:18.297466+0000","last_became_active":"2023-11-16T16:37:13.505785+0000","last_became_peered":"2023-11-16T16:37:13.505785+0000","last_unstale":"2023-11-16T16:37:18.297466+0000","last_undegraded":"2023-11-16T16:37:18.297466+0000","last_fullsized":"2023-11-16T16:37:18.297466+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:01:15.544970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.391265+0000","last_change":"2023-11-16T16:37:13.935863+0000","last_active":"2023-11-16T16:37:15.391265+0000","last_peered":"2023-11-16T16:37:15.391265+0000","last_clean":"2023-11-16T16:37:15.391265+0000","last_became_active":"2023-11-16T16:37:13.935399+0000","last_became_peered":"2023-11-16T16:37:13.935399+0000","last_unstale":"2023-11-16T16:37:15.391265+0000","last_undegraded":"2023-11-16T16:37:15.391265+0000","last_fullsized":"2023-11-16T16:37:15.391265+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:38:54.382348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.462427+0000","last_change":"2023-11-16T16:37:13.498367+0000","last_active":"2023-11-16T16:37:15.462427+0000","last_peered":"2023-11-16T16:37:15.462427+0000","last_clean":"2023-11-16T16:37:15.462427+0000","last_became_active":"2023-11-16T16:37:13.497039+0000","last_became_peered":"2023-11-16T16:37:13.497039+0000","last_unstale":"2023-11-16T16:37:15.462427+0000","last_undegraded":"2023-11-16T16:37:15.462427+0000","last_fullsized":"2023-11-16T16:37:15.462427+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:07:52.215240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.391119+0000","last_change":"2023-11-16T16:37:13.940617+0000","last_active":"2023-11-16T16:37:15.391119+0000","last_peered":"2023-11-16T16:37:15.391119+0000","last_clean":"2023-11-16T16:37:15.391119+0000","last_became_active":"2023-11-16T16:37:13.940264+0000","last_became_peered":"2023-11-16T16:37:13.940264+0000","last_unstale":"2023-11-16T16:37:15.391119+0000","last_undegraded":"2023-11-16T16:37:15.391119+0000","last_fullsized":"2023-11-16T16:37:15.391119+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:22:54.941976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157351+0000","last_change":"2023-11-16T16:37:13.505950+0000","last_active":"2023-11-16T16:37:15.157351+0000","last_peered":"2023-11-16T16:37:15.157351+0000","last_clean":"2023-11-16T16:37:15.157351+0000","last_became_active":"2023-11-16T16:37:13.505628+0000","last_became_peered":"2023-11-16T16:37:13.505628+0000","last_unstale":"2023-11-16T16:37:15.157351+0000","last_undegraded":"2023-11-16T16:37:15.157351+0000","last_fullsized":"2023-11-16T16:37:15.157351+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:11:17.082705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.391279+0000","last_change":"2023-11-16T16:37:13.943096+0000","last_active":"2023-11-16T16:37:15.391279+0000","last_peered":"2023-11-16T16:37:15.391279+0000","last_clean":"2023-11-16T16:37:15.391279+0000","last_became_active":"2023-11-16T16:37:13.942745+0000","last_became_peered":"2023-11-16T16:37:13.942745+0000","last_unstale":"2023-11-16T16:37:15.391279+0000","last_undegraded":"2023-11-16T16:37:15.391279+0000","last_fullsized":"2023-11-16T16:37:15.391279+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:35:21.092899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157245+0000","last_change":"2023-11-16T16:37:13.938087+0000","last_active":"2023-11-16T16:37:15.157245+0000","last_peered":"2023-11-16T16:37:15.157245+0000","last_clean":"2023-11-16T16:37:15.157245+0000","last_became_active":"2023-11-16T16:37:13.937780+0000","last_became_peered":"2023-11-16T16:37:13.937780+0000","last_unstale":"2023-11-16T16:37:15.157245+0000","last_undegraded":"2023-11-16T16:37:15.157245+0000","last_fullsized":"2023-11-16T16:37:15.157245+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:41:00.785196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157978+0000","last_change":"2023-11-16T16:37:13.939000+0000","last_active":"2023-11-16T16:37:15.157978+0000","last_peered":"2023-11-16T16:37:15.157978+0000","last_clean":"2023-11-16T16:37:15.157978+0000","last_became_active":"2023-11-16T16:37:13.938585+0000","last_became_peered":"2023-11-16T16:37:13.938585+0000","last_unstale":"2023-11-16T16:37:15.157978+0000","last_undegraded":"2023-11-16T16:37:15.157978+0000","last_fullsized":"2023-11-16T16:37:15.157978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:17:47.012171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157516+0000","last_change":"2023-11-16T16:37:13.937696+0000","last_active":"2023-11-16T16:37:15.157516+0000","last_peered":"2023-11-16T16:37:15.157516+0000","last_clean":"2023-11-16T16:37:15.157516+0000","last_became_active":"2023-11-16T16:37:13.936920+0000","last_became_peered":"2023-11-16T16:37:13.936920+0000","last_unstale":"2023-11-16T16:37:15.157516+0000","last_undegraded":"2023-11-16T16:37:15.157516+0000","last_fullsized":"2023-11-16T16:37:15.157516+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:40:10.578228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157738+0000","last_change":"2023-11-16T16:37:13.942977+0000","last_active":"2023-11-16T16:37:15.157738+0000","last_peered":"2023-11-16T16:37:15.157738+0000","last_clean":"2023-11-16T16:37:15.157738+0000","last_became_active":"2023-11-16T16:37:13.942632+0000","last_became_peered":"2023-11-16T16:37:13.942632+0000","last_unstale":"2023-11-16T16:37:15.157738+0000","last_undegraded":"2023-11-16T16:37:15.157738+0000","last_fullsized":"2023-11-16T16:37:15.157738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:44:02.416862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157651+0000","last_change":"2023-11-16T16:37:13.949940+0000","last_active":"2023-11-16T16:37:15.157651+0000","last_peered":"2023-11-16T16:37:15.157651+0000","last_clean":"2023-11-16T16:37:15.157651+0000","last_became_active":"2023-11-16T16:37:13.949464+0000","last_became_peered":"2023-11-16T16:37:13.949464+0000","last_unstale":"2023-11-16T16:37:15.157651+0000","last_undegraded":"2023-11-16T16:37:15.157651+0000","last_fullsized":"2023-11-16T16:37:15.157651+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:52:47.439482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157535+0000","last_change":"2023-11-16T16:37:13.940534+0000","last_active":"2023-11-16T16:37:15.157535+0000","last_peered":"2023-11-16T16:37:15.157535+0000","last_clean":"2023-11-16T16:37:15.157535+0000","last_became_active":"2023-11-16T16:37:13.940174+0000","last_became_peered":"2023-11-16T16:37:13.940174+0000","last_unstale":"2023-11-16T16:37:15.157535+0000","last_undegraded":"2023-11-16T16:37:15.157535+0000","last_fullsized":"2023-11-16T16:37:15.157535+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:25:01.730261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157969+0000","last_change":"2023-11-16T16:37:13.937519+0000","last_active":"2023-11-16T16:37:15.157969+0000","last_peered":"2023-11-16T16:37:15.157969+0000","last_clean":"2023-11-16T16:37:15.157969+0000","last_became_active":"2023-11-16T16:37:13.937175+0000","last_became_peered":"2023-11-16T16:37:13.937175+0000","last_unstale":"2023-11-16T16:37:15.157969+0000","last_undegraded":"2023-11-16T16:37:15.157969+0000","last_fullsized":"2023-11-16T16:37:15.157969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T16:47:03.319128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158262+0000","last_change":"2023-11-16T16:37:13.951506+0000","last_active":"2023-11-16T16:37:15.158262+0000","last_peered":"2023-11-16T16:37:15.158262+0000","last_clean":"2023-11-16T16:37:15.158262+0000","last_became_active":"2023-11-16T16:37:13.951192+0000","last_became_peered":"2023-11-16T16:37:13.951192+0000","last_unstale":"2023-11-16T16:37:15.158262+0000","last_undegraded":"2023-11-16T16:37:15.158262+0000","last_fullsized":"2023-11-16T16:37:15.158262+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:18:48.061220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.923594+0000","last_change":"2023-11-16T16:37:13.935614+0000","last_active":"2023-11-16T16:37:14.923594+0000","last_peered":"2023-11-16T16:37:14.923594+0000","last_clean":"2023-11-16T16:37:14.923594+0000","last_became_active":"2023-11-16T16:37:13.935237+0000","last_became_peered":"2023-11-16T16:37:13.935237+0000","last_unstale":"2023-11-16T16:37:14.923594+0000","last_undegraded":"2023-11-16T16:37:14.923594+0000","last_fullsized":"2023-11-16T16:37:14.923594+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:02:10.333347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157636+0000","last_change":"2023-11-16T16:37:13.939387+0000","last_active":"2023-11-16T16:37:15.157636+0000","last_peered":"2023-11-16T16:37:15.157636+0000","last_clean":"2023-11-16T16:37:15.157636+0000","last_became_active":"2023-11-16T16:37:13.939082+0000","last_became_peered":"2023-11-16T16:37:13.939082+0000","last_unstale":"2023-11-16T16:37:15.157636+0000","last_undegraded":"2023-11-16T16:37:15.157636+0000","last_fullsized":"2023-11-16T16:37:15.157636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:42:24.707911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.156764+0000","last_change":"2023-11-16T16:37:13.936381+0000","last_active":"2023-11-16T16:37:15.156764+0000","last_peered":"2023-11-16T16:37:15.156764+0000","last_clean":"2023-11-16T16:37:15.156764+0000","last_became_active":"2023-11-16T16:37:13.936077+0000","last_became_peered":"2023-11-16T16:37:13.936077+0000","last_unstale":"2023-11-16T16:37:15.156764+0000","last_undegraded":"2023-11-16T16:37:15.156764+0000","last_fullsized":"2023-11-16T16:37:15.156764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:04:48.256913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157497+0000","last_change":"2023-11-16T16:37:13.934005+0000","last_active":"2023-11-16T16:37:15.157497+0000","last_peered":"2023-11-16T16:37:15.157497+0000","last_clean":"2023-11-16T16:37:15.157497+0000","last_became_active":"2023-11-16T16:37:13.933633+0000","last_became_peered":"2023-11-16T16:37:13.933633+0000","last_unstale":"2023-11-16T16:37:15.157497+0000","last_undegraded":"2023-11-16T16:37:15.157497+0000","last_fullsized":"2023-11-16T16:37:15.157497+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:45:12.620887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158013+0000","last_change":"2023-11-16T16:37:13.946495+0000","last_active":"2023-11-16T16:37:15.158013+0000","last_peered":"2023-11-16T16:37:15.158013+0000","last_clean":"2023-11-16T16:37:15.158013+0000","last_became_active":"2023-11-16T16:37:13.946148+0000","last_became_peered":"2023-11-16T16:37:13.946148+0000","last_unstale":"2023-11-16T16:37:15.158013+0000","last_undegraded":"2023-11-16T16:37:15.158013+0000","last_fullsized":"2023-11-16T16:37:15.158013+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:17:05.329636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":3542208,"data_compressed":211753,"data_compressed_allocated":270336,"data_compressed_original":3522560,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":22743,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":23,"num_read_kb":35,"num_write":126,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":299008,"data_stored":84655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416396,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5940,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":93739212,"statfs":{"total":95995035648,"available":95988953088,"internally_reserved":0,"allocated":897024,"data_stored":428869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]}]},{"osd":10,"up_from":72,"seq":309237645328,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6304,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":93738848,"statfs":{"total":95995035648,"available":95988580352,"internally_reserved":0,"allocated":1007616,"data_stored":1039488,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":1,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.34200000000000003}]},{"osd":2,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.72599999999999998}]},{"osd":3,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.56200000000000006}]},{"osd":4,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.5}]},{"osd":5,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.58399999999999996}]},{"osd":6,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.65100000000000002}]},{"osd":7,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.505}]},{"osd":8,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.71099999999999997}]},{"osd":9,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.53000000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]}]},{"osd":9,"up_from":67,"seq":287762808851,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6088,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93739064,"statfs":{"total":95995035648,"available":95988801536,"internally_reserved":0,"allocated":917504,"data_stored":429687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:36:50 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.29399999999999998}]},{"osd":1,"last update":"Thu Nov 16 16:36:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.56399999999999995}]},{"osd":2,"last update":"Thu Nov 16 16:36:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.66000000000000003}]},{"osd":3,"last update":"Thu Nov 16 16:36:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.36099999999999999}]},{"osd":4,"last update":"Thu Nov 16 16:36:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.502},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.48399999999999999}]},{"osd":5,"last update":"Thu Nov 16 16:36:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.627}]},{"osd":6,"last update":"Thu Nov 16 16:36:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.38200000000000001}]},{"osd":7,"last update":"Thu Nov 16 16:36:50 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.67200000000000004}]},{"osd":8,"last update":"Thu Nov 16 16:36:50 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.70999999999999996}]},{"osd":10,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.68100000000000005}]}]},{"osd":8,"up_from":59,"seq":253403070490,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6280,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738872,"statfs":{"total":95995035648,"available":95988604928,"internally_reserved":0,"allocated":917504,"data_stored":429607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:36:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.61099999999999999}]},{"osd":1,"last update":"Thu Nov 16 16:36:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.875},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.748}]},{"osd":2,"last update":"Thu Nov 16 16:36:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.56399999999999995}]},{"osd":3,"last update":"Thu Nov 16 16:36:27 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.66600000000000004}]},{"osd":4,"last update":"Thu Nov 16 16:36:27 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.92400000000000004}]},{"osd":5,"last update":"Thu Nov 16 16:36:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.68999999999999995}]},{"osd":6,"last update":"Thu Nov 16 16:36:27 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.65100000000000002}]},{"osd":7,"last update":"Thu Nov 16 16:36:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.83199999999999996}]},{"osd":9,"last update":"Thu Nov 16 16:36:50 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":1.004}]},{"osd":10,"last update":"Thu Nov 16 16:37:13 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":3.5720000000000001,"5min":3.5720000000000001,"15min":3.5720000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":3.5920000000000001,"5min":3.5920000000000001,"15min":3.5920000000000001},"last":0.96599999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]}]},{"osd":7,"up_from":52,"seq":223338299421,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6388,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738764,"statfs":{"total":95995035648,"available":95988494336,"internally_reserved":0,"allocated":962560,"data_stored":1020933,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:36:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55800000000000005}]},{"osd":1,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.41799999999999998,"15min":0.41099999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.434,"15min":0.42599999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.314}]},{"osd":2,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.35799999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.58099999999999996}]},{"osd":3,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57499999999999996,"15min":0.57099999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.625,"15min":0.621},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.40000000000000002}]},{"osd":4,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.52400000000000002,"15min":0.51400000000000001},"min":{"1min":0.34799999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.498,"15min":0.48199999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.33200000000000002}]},{"osd":5,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.65200000000000002,"15min":0.64700000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91300000000000003,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.63800000000000001,"15min":0.63200000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.53000000000000003}]},{"osd":6,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.70299999999999996,"15min":0.69499999999999995},"min":{"1min":0.52500000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.68600000000000005,"15min":0.67600000000000005},"min":{"1min":0.47299999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.54400000000000004}]},{"osd":8,"last update":"Thu Nov 16 16:36:25 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.505},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.49299999999999999}]},{"osd":9,"last update":"Thu Nov 16 16:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.55600000000000005}]},{"osd":10,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.62},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.67100000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]}]},{"osd":6,"up_from":47,"seq":201863462945,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6196,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":93738956,"statfs":{"total":95995035648,"available":95988690944,"internally_reserved":0,"allocated":897024,"data_stored":428869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.41599999999999998,"15min":0.41199999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.437,"15min":0.433},"min":{"1min":0.30199999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.36599999999999999}]},{"osd":1,"last update":"Thu Nov 16 16:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.41499999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.80300000000000005,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.41099999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79300000000000004,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.52200000000000002}]},{"osd":2,"last update":"Thu Nov 16 16:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.625,"15min":0.629},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.77300000000000002,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.624,"15min":0.621},"min":{"1min":0.46200000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.48299999999999998}]},{"osd":3,"last update":"Thu Nov 16 16:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.48599999999999999,"15min":0.47699999999999998},"min":{"1min":0.36399999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.53600000000000003,"15min":0.53400000000000003},"min":{"1min":0.35299999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.40699999999999997}]},{"osd":4,"last update":"Thu Nov 16 16:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.58799999999999997,"15min":0.58099999999999996},"min":{"1min":0.442,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.61399999999999999,"15min":0.60199999999999998},"min":{"1min":0.47599999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.86299999999999999,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.58399999999999996}]},{"osd":5,"last update":"Thu Nov 16 16:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.64200000000000002}]},{"osd":7,"last update":"Thu Nov 16 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.70499999999999996,"15min":0.70299999999999996},"min":{"1min":0.48999999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.878,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.71499999999999997,"15min":0.70899999999999996},"min":{"1min":0.55300000000000005,"5min":0.436,"15min":0.436},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.67000000000000004}]},{"osd":8,"last update":"Thu Nov 16 16:36:27 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.59899999999999998}]},{"osd":9,"last update":"Thu Nov 16 16:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.74199999999999999}]},{"osd":10,"last update":"Thu Nov 16 16:37:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.625}]}]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7200,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":6272,"kb_avail":93737952,"statfs":{"total":95995035648,"available":95987662848,"internally_reserved":0,"allocated":942080,"data_stored":465106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6422528},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.48499999999999999,"15min":0.45800000000000002},"min":{"1min":0.33100000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.51200000000000001,"15min":0.48499999999999999},"min":{"1min":0.32000000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.47499999999999998}]},{"osd":2,"last update":"Thu Nov 16 16:37:19 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.54300000000000004,"15min":0.498},"min":{"1min":0.36699999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.84799999999999998,"5min":0.876,"15min":0.876},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.54000000000000004,"15min":0.503},"min":{"1min":0.40500000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.91000000000000003,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.59199999999999997}]},{"osd":3,"last update":"Thu Nov 16 16:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.57399999999999995,"15min":0.55400000000000005},"min":{"1min":0.45900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.59199999999999997,"15min":0.57299999999999995},"min":{"1min":0.42499999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.57699999999999996}]},{"osd":4,"last update":"Thu Nov 16 16:37:03 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.441,"15min":0.42299999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.374},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.52200000000000002,"15min":0.47599999999999998},"min":{"1min":0.34599999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.53000000000000003}]},{"osd":5,"last update":"Thu Nov 16 16:36:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.61399999999999999,"15min":0.60299999999999998},"min":{"1min":0.49099999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.59699999999999998,"15min":0.58899999999999997},"min":{"1min":0.443,"5min":0.437,"15min":0.437},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.72999999999999998}]},{"osd":6,"last update":"Thu Nov 16 16:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.68600000000000005,"15min":0.66900000000000004},"min":{"1min":0.48699999999999999,"5min":0.376,"15min":0.376},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.67900000000000005,"15min":0.66200000000000003},"min":{"1min":0.47499999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1,"5min":1,"15min":1},"last":0.54800000000000004}]},{"osd":7,"last update":"Thu Nov 16 16:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.64500000000000002,"15min":0.63200000000000001},"min":{"1min":0.35199999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.69299999999999995,"15min":0.67700000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.68799999999999994}]},{"osd":8,"last update":"Thu Nov 16 16:36:27 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.78200000000000003}]},{"osd":9,"last update":"Thu Nov 16 16:36:53 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.76100000000000001}]},{"osd":10,"last update":"Thu Nov 16 16:37:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.65200000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738429,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7184,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":6272,"kb_avail":93737968,"statfs":{"total":95995035648,"available":95987679232,"internally_reserved":0,"allocated":925696,"data_stored":430347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6422528},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Nov 16 16:37:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.503,"15min":0.46700000000000003},"min":{"1min":0.32500000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.504},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.499,"15min":0.46100000000000002},"min":{"1min":0.36099999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78800000000000003,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.47399999999999998}]},{"osd":2,"last update":"Thu Nov 16 16:36:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.53400000000000003,"15min":0.50600000000000001},"min":{"1min":0.443,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.54600000000000004,"15min":0.52600000000000002},"min":{"1min":0.502,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.72599999999999998}]},{"osd":3,"last update":"Thu Nov 16 16:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.46500000000000002,"15min":0.45600000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.45700000000000002,"15min":0.44700000000000001},"min":{"1min":0.33700000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.40000000000000002}]},{"osd":4,"last update":"Thu Nov 16 16:37:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.57899999999999996,"15min":0.55100000000000005},"min":{"1min":0.38900000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.58199999999999996,"15min":0.55000000000000004},"min":{"1min":0.40999999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.51800000000000002}]},{"osd":5,"last update":"Thu Nov 16 16:36:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.56399999999999995,"15min":0.55200000000000005},"min":{"1min":0.45200000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.63900000000000001,"15min":0.61899999999999999},"min":{"1min":0.51200000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.79100000000000004}]},{"osd":6,"last update":"Thu Nov 16 16:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.60599999999999998,"15min":0.58299999999999996},"min":{"1min":0.35999999999999999,"5min":0.316,"15min":0.316},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.56899999999999995,"15min":0.55500000000000005},"min":{"1min":0.38800000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.438}]},{"osd":7,"last update":"Thu Nov 16 16:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.68400000000000005,"15min":0.67600000000000005},"min":{"1min":0.495,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.74199999999999999,"15min":0.73199999999999998},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.53400000000000003}]},{"osd":8,"last update":"Thu Nov 16 16:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.77000000000000002}]},{"osd":9,"last update":"Thu Nov 16 16:36:51 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.48799999999999999}]},{"osd":10,"last update":"Thu Nov 16 16:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.55400000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]}]},{"osd":2,"up_from":18,"seq":77309411380,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7192,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":6272,"kb_avail":93737960,"statfs":{"total":95995035648,"available":95987671040,"internally_reserved":0,"allocated":933888,"data_stored":432125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6422528},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:36:21 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.51300000000000001,"15min":0.51900000000000002},"min":{"1min":0.311,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.65400000000000003,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.495,"5min":0.48699999999999999,"15min":0.48499999999999999},"min":{"1min":0.33200000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.65800000000000003,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.49399999999999999}]},{"osd":1,"last update":"Thu Nov 16 16:36:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51500000000000001,"15min":0.51000000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55300000000000005,"15min":0.54900000000000004},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.443}]},{"osd":3,"last update":"Thu Nov 16 16:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.44700000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56200000000000006,"15min":0.55200000000000005},"min":{"1min":0.47299999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.60499999999999998}]},{"osd":4,"last update":"Thu Nov 16 16:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.55300000000000005,"15min":0.53000000000000003},"min":{"1min":0.376,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59099999999999997,"15min":0.58199999999999996},"min":{"1min":0.40000000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83799999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.62}]},{"osd":5,"last update":"Thu Nov 16 16:36:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.32300000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.46000000000000002,"15min":0.45500000000000002},"min":{"1min":0.307,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.66300000000000003}]},{"osd":6,"last update":"Thu Nov 16 16:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64500000000000002,"15min":0.64300000000000002},"min":{"1min":0.47099999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.60699999999999998,"15min":0.60099999999999998},"min":{"1min":0.35699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83899999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.71799999999999997}]},{"osd":7,"last update":"Thu Nov 16 16:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.64300000000000002,"15min":0.64400000000000002},"min":{"1min":0.41599999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70199999999999996,"15min":0.70399999999999996},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.92700000000000005,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.57699999999999996}]},{"osd":8,"last update":"Thu Nov 16 16:36:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.70599999999999996}]},{"osd":9,"last update":"Thu Nov 16 16:36:51 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":4.524,"5min":4.524,"15min":4.524},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.81699999999999995}]},{"osd":10,"last update":"Thu Nov 16 16:37:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.80100000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]}]},{"osd":3,"up_from":26,"seq":111669149741,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6520,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738632,"statfs":{"total":95995035648,"available":95988359168,"internally_reserved":0,"allocated":901120,"data_stored":428642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.35899999999999999,"15min":0.34699999999999998},"min":{"1min":0.28999999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.378,"15min":0.35699999999999998},"min":{"1min":0.253,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.28499999999999998}]},{"osd":1,"last update":"Thu Nov 16 16:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.55800000000000005,"15min":0.55000000000000004},"min":{"1min":0.374,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.437},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55900000000000005,"15min":0.55700000000000005},"min":{"1min":0.33800000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.55200000000000005}]},{"osd":2,"last update":"Thu Nov 16 16:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.57499999999999996,"15min":0.55500000000000005},"min":{"1min":0.38700000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.56499999999999995,"15min":0.54700000000000004},"min":{"1min":0.48299999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.40600000000000003}]},{"osd":4,"last update":"Thu Nov 16 16:37:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55700000000000005,"15min":0.54700000000000004},"min":{"1min":0.376,"5min":0.37,"15min":0.37},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.45500000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.47399999999999998}]},{"osd":5,"last update":"Thu Nov 16 16:36:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.59199999999999997,"15min":0.57799999999999996},"min":{"1min":0.39500000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.60499999999999998,"15min":0.59099999999999997},"min":{"1min":0.44600000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.46300000000000002}]},{"osd":6,"last update":"Thu Nov 16 16:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.51600000000000001,"15min":0.497},"min":{"1min":0.35399999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.53000000000000003,"15min":0.51300000000000001},"min":{"1min":0.35799999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.377}]},{"osd":7,"last update":"Thu Nov 16 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.65600000000000003,"15min":0.65000000000000002},"min":{"1min":0.44,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.69299999999999995,"15min":0.68700000000000006},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.42699999999999999}]},{"osd":8,"last update":"Thu Nov 16 16:36:27 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.63200000000000001}]},{"osd":9,"last update":"Thu Nov 16 16:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.41799999999999998}]},{"osd":10,"last update":"Thu Nov 16 16:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.54000000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]}]},{"osd":4,"up_from":34,"seq":146028888106,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6336,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738816,"statfs":{"total":95995035648,"available":95988547584,"internally_reserved":0,"allocated":909312,"data_stored":429465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.52400000000000002,"15min":0.502},"min":{"1min":0.38200000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.52900000000000003,"15min":0.51500000000000001},"min":{"1min":0.40899999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.378}]},{"osd":1,"last update":"Thu Nov 16 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.39400000000000002,"15min":0.38700000000000001},"min":{"1min":0.27900000000000003,"5min":0.253,"15min":0.253},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.41199999999999998,"15min":0.40300000000000002},"min":{"1min":0.29699999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.378}]},{"osd":2,"last update":"Thu Nov 16 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.442,"5min":0.371,"15min":0.371},"max":{"1min":0.79200000000000004,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.60499999999999998,"15min":0.58699999999999997},"min":{"1min":0.44600000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.90200000000000002,"5min":1.056,"15min":1.056},"last":0.53000000000000003}]},{"osd":3,"last update":"Thu Nov 16 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.56299999999999994,"15min":0.54000000000000004},"min":{"1min":0.39000000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.56100000000000005,"15min":0.54900000000000004},"min":{"1min":0.40500000000000003,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.40100000000000002}]},{"osd":5,"last update":"Thu Nov 16 16:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.622,"15min":0.627},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.70399999999999996,"5min":1.075,"15min":1.075},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.621,"15min":0.62},"min":{"1min":0.51500000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.92900000000000005,"5min":1.093,"15min":1.093},"last":0.69199999999999995}]},{"osd":6,"last update":"Thu Nov 16 16:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.60799999999999998,"15min":0.59499999999999997},"min":{"1min":0.434,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.41899999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.46600000000000003}]},{"osd":7,"last update":"Thu Nov 16 16:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.57199999999999995,"15min":0.54800000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.63200000000000001,"15min":0.60799999999999998},"min":{"1min":0.41199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.51600000000000001}]},{"osd":8,"last update":"Thu Nov 16 16:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.70799999999999996}]},{"osd":9,"last update":"Thu Nov 16 16:36:51 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.501}]},{"osd":10,"last update":"Thu Nov 16 16:37:15 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.629}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]}]},{"osd":5,"up_from":39,"seq":167503724583,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6528,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738624,"statfs":{"total":95995035648,"available":95988350976,"internally_reserved":0,"allocated":974848,"data_stored":1020665,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:36:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.50900000000000001,"15min":0.497},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.53000000000000003,"15min":0.52500000000000002},"min":{"1min":0.40699999999999997,"5min":0.311,"15min":0.311},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.312}]},{"osd":1,"last update":"Thu Nov 16 16:36:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58899999999999997,"15min":0.58599999999999997},"min":{"1min":0.40200000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81899999999999995,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.41699999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85599999999999998,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.52300000000000002}]},{"osd":2,"last update":"Thu Nov 16 16:36:21 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.42499999999999999,"15min":0.42299999999999999},"min":{"1min":0.33000000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.63200000000000001,"5min":1.417,"15min":1.417},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.439,"15min":0.432},"min":{"1min":0.35499999999999998,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.628,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.34899999999999998}]},{"osd":3,"last update":"Thu Nov 16 16:36:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.46400000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.377},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.60899999999999999,"15min":0.59999999999999998},"min":{"1min":0.45100000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.84299999999999997,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.43099999999999999}]},{"osd":4,"last update":"Thu Nov 16 16:36:21 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.60899999999999999,"15min":0.60599999999999998},"min":{"1min":0.51900000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82999999999999996,"5min":1.514,"15min":1.514},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.442,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.40400000000000003}]},{"osd":6,"last update":"Thu Nov 16 16:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.627,"15min":0.61899999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.91400000000000003,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.502},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.63600000000000001,"15min":0.63100000000000001},"min":{"1min":0.46400000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85599999999999998,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.41399999999999998}]},{"osd":7,"last update":"Thu Nov 16 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.70999999999999996,"15min":0.71199999999999997},"min":{"1min":0.503,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.83799999999999997,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.72499999999999998,"15min":0.72799999999999998},"min":{"1min":0.56599999999999995,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.83699999999999997,"5min":1.544,"15min":1.544},"last":0.48099999999999998}]},{"osd":8,"last update":"Thu Nov 16 16:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.58499999999999996}]},{"osd":9,"last update":"Thu Nov 16 16:36:51 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.56200000000000006}]},{"osd":10,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.57899999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33928,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":36237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":20621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":20459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-16T16:37:22.070 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-16T16:37:22.232 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.229+0000 7efea8315700 1 -- 172.21.15.1:0/3448113953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efea01016d0 msgr2=0x7efea0101b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:22.232 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.229+0000 7efea8315700 1 --2- 172.21.15.1:0/3448113953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efea01016d0 0x7efea0101b20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7efe9c009990 tx=0x7efe9c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:37:22.232 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.229+0000 7efea8315700 1 -- 172.21.15.1:0/3448113953 shutdown_connections 2023-11-16T16:37:22.232 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.229+0000 7efea8315700 1 --2- 172.21.15.1:0/3448113953 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efea0102060 0x7efea0108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:22.232 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.229+0000 7efea8315700 1 --2- 172.21.15.1:0/3448113953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea01004e0 0x7efea01008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:22.232 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.229+0000 7efea8315700 1 --2- 172.21.15.1:0/3448113953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efea01016d0 0x7efea0101b20 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:22.233 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.229+0000 7efea8315700 1 -- 172.21.15.1:0/3448113953 >> 172.21.15.1:0/3448113953 conn(0x7efea00fbac0 msgr2=0x7efea00fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:22.233 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.229+0000 7efea8315700 1 -- 172.21.15.1:0/3448113953 shutdown_connections 2023-11-16T16:37:22.233 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.229+0000 7efea8315700 1 -- 172.21.15.1:0/3448113953 wait complete. 2023-11-16T16:37:22.233 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.230+0000 7efea8315700 1 Processor -- start 2023-11-16T16:37:22.233 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.230+0000 7efea8315700 1 -- start start 2023-11-16T16:37:22.233 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.230+0000 7efea8315700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efea01004e0 0x7efea0196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:22.233 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.230+0000 7efea8315700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efea01016d0 0x7efea01967a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:22.233 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.230+0000 7efea8315700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea0102060 0x7efea019d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:22.233 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.230+0000 7efea8315700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea019dde0 con 0x7efea01004e0 2023-11-16T16:37:22.233 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.230+0000 7efea8315700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea019df20 con 0x7efea0102060 2023-11-16T16:37:22.233 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.230+0000 7efea8315700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea019e0b0 con 0x7efea01016d0 2023-11-16T16:37:22.233 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.230+0000 7efea60b1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efea01004e0 0x7efea0196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:22.233 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.230+0000 7efea60b1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efea01004e0 0x7efea0196260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41640/0 (socket says 172.21.15.1:41640) 2023-11-16T16:37:22.234 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.230+0000 7efea60b1700 1 -- 172.21.15.1:0/284578965 learned_addr learned my addr 172.21.15.1:0/284578965 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:22.234 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.230+0000 7efea58b0700 1 --2- 172.21.15.1:0/284578965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efea01016d0 0x7efea01967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:22.234 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.230+0000 7efea60b1700 1 -- 172.21.15.1:0/284578965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efea01016d0 msgr2=0x7efea01967a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:22.234 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.230+0000 7efea60b1700 1 --2- 172.21.15.1:0/284578965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efea01016d0 0x7efea01967a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:22.234 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.230+0000 7efea60b1700 1 -- 172.21.15.1:0/284578965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea0102060 msgr2=0x7efea019d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:22.234 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.230+0000 7efea68b2700 1 --2- 172.21.15.1:0/284578965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea0102060 0x7efea019d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:22.234 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.230+0000 7efea60b1700 1 --2- 172.21.15.1:0/284578965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea0102060 0x7efea019d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:22.234 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.230+0000 7efea60b1700 1 -- 172.21.15.1:0/284578965 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe9c0095f0 con 0x7efea01004e0 2023-11-16T16:37:22.234 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.230+0000 7efea68b2700 1 --2- 172.21.15.1:0/284578965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea0102060 0x7efea019d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:37:22.234 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.230+0000 7efea60b1700 1 --2- 172.21.15.1:0/284578965 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efea01004e0 0x7efea0196260 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efe9400d980 tx=0x7efe9400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:22.234 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.231+0000 7efe937fe700 1 -- 172.21.15.1:0/284578965 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9400de60 con 0x7efea01004e0 2023-11-16T16:37:22.235 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.231+0000 7efe937fe700 1 -- 172.21.15.1:0/284578965 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe940044e0 con 0x7efea01004e0 2023-11-16T16:37:22.235 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.231+0000 7efea8315700 1 -- 172.21.15.1:0/284578965 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efea010bec0 con 0x7efea01004e0 2023-11-16T16:37:22.235 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.231+0000 7efe937fe700 1 -- 172.21.15.1:0/284578965 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe94003be0 con 0x7efea01004e0 2023-11-16T16:37:22.235 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.231+0000 7efea8315700 1 -- 172.21.15.1:0/284578965 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea010bd30 con 0x7efea01004e0 2023-11-16T16:37:22.235 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.232+0000 7efea8315700 1 -- 172.21.15.1:0/284578965 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efea01079d0 con 0x7efea01004e0 2023-11-16T16:37:22.236 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.233+0000 7efe937fe700 1 -- 172.21.15.1:0/284578965 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7efe94009bb0 con 0x7efea01004e0 2023-11-16T16:37:22.236 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.233+0000 7efe937fe700 1 --2- 172.21.15.1:0/284578965 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7efe8c0a11d0 0x7efe8c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:22.236 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.233+0000 7efe937fe700 1 -- 172.21.15.1:0/284578965 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7efe940cc860 con 0x7efea01004e0 2023-11-16T16:37:22.236 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.233+0000 7efea58b0700 1 --2- 172.21.15.1:0/284578965 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7efe8c0a11d0 0x7efe8c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:22.236 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.234+0000 7efea58b0700 1 --2- 172.21.15.1:0/284578965 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7efe8c0a11d0 0x7efe8c0a3680 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efe9c006010 tx=0x7efe9c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:22.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.237+0000 7efe937fe700 1 -- 172.21.15.1:0/284578965 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7efe94096180 con 0x7efea01004e0 2023-11-16T16:37:22.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.311+0000 7efe937fe700 1 -- 172.21.15.1:0/284578965 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7efe94096550 con 0x7efea01004e0 2023-11-16T16:37:22.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:22 smithi001 ceph-mon[98800]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:37:22.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:22 smithi001 ceph-mon[98800]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:37:22.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:22 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/2923912856' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:37:22.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:22 smithi001 ceph-mon[98800]: pgmap v264: 97 pgs: 97 active+clean; 599 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7.1 KiB/s wr, 18 op/s 2023-11-16T16:37:22.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:22 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3902182019' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:37:22.438 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.438+0000 7efea8315700 1 -- 172.21.15.1:0/284578965 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efea0060f30 con 0x7efe8c0a11d0 2023-11-16T16:37:22.450 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.449+0000 7efe937fe700 1 -- 172.21.15.1:0/284578965 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+335786 (secure 0 0 0) 0x7efea0060f30 con 0x7efe8c0a11d0 2023-11-16T16:37:22.454 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:37:22.456 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.453+0000 7efea8315700 1 -- 172.21.15.1:0/284578965 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7efe8c0a11d0 msgr2=0x7efe8c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:22.456 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.453+0000 7efea8315700 1 --2- 172.21.15.1:0/284578965 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7efe8c0a11d0 0x7efe8c0a3680 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efe9c006010 tx=0x7efe9c00b540 comp rx=0 tx=0).stop 2023-11-16T16:37:22.456 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.453+0000 7efea8315700 1 -- 172.21.15.1:0/284578965 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efea01004e0 msgr2=0x7efea0196260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:22.456 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.453+0000 7efea8315700 1 --2- 172.21.15.1:0/284578965 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efea01004e0 0x7efea0196260 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efe9400d980 tx=0x7efe9400dd40 comp rx=0 tx=0).stop 2023-11-16T16:37:22.456 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.453+0000 7efea8315700 1 -- 172.21.15.1:0/284578965 shutdown_connections 2023-11-16T16:37:22.457 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.453+0000 7efea8315700 1 --2- 172.21.15.1:0/284578965 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7efe8c0a11d0 0x7efe8c0a3680 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:22.457 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.453+0000 7efea8315700 1 --2- 172.21.15.1:0/284578965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea0102060 0x7efea019d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:22.457 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.453+0000 7efea8315700 1 --2- 172.21.15.1:0/284578965 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efea01004e0 0x7efea0196260 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:22.457 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.453+0000 7efea8315700 1 --2- 172.21.15.1:0/284578965 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efea01016d0 0x7efea01967a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:22.457 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.453+0000 7efea8315700 1 -- 172.21.15.1:0/284578965 >> 172.21.15.1:0/284578965 conn(0x7efea00fbac0 msgr2=0x7efea0104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:22.457 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.454+0000 7efea8315700 1 -- 172.21.15.1:0/284578965 shutdown_connections 2023-11-16T16:37:22.457 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:22.454+0000 7efea8315700 1 -- 172.21.15.1:0/284578965 wait complete. 2023-11-16T16:37:22.457 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2023-11-16T16:37:22.466 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":264,"stamp":"2023-11-16T16:37:21.467153+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":613234,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":159,"num_read_kb":154,"num_write":308,"num_write_kb":2995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":248,"ondisk_log_size":248,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":78156,"kb_used_data":10924,"kb_used_omap":0,"kb_used_meta":67136,"kb_avail":1124863668,"statfs":{"total":1151940427776,"available":1151860396032,"internally_reserved":0,"allocated":11186176,"data_stored":6983803,"data_compressed":106695,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":68747264},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22866,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":23,"num_read_kb":35,"num_write":127,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.986938"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157701+0000","last_change":"2023-11-16T16:37:13.941551+0000","last_active":"2023-11-16T16:37:15.157701+0000","last_peered":"2023-11-16T16:37:15.157701+0000","last_clean":"2023-11-16T16:37:15.157701+0000","last_became_active":"2023-11-16T16:37:13.941190+0000","last_became_peered":"2023-11-16T16:37:13.941190+0000","last_unstale":"2023-11-16T16:37:15.157701+0000","last_undegraded":"2023-11-16T16:37:15.157701+0000","last_fullsized":"2023-11-16T16:37:15.157701+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:26:51.970409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158170+0000","last_change":"2023-11-16T16:37:13.942300+0000","last_active":"2023-11-16T16:37:15.158170+0000","last_peered":"2023-11-16T16:37:15.158170+0000","last_clean":"2023-11-16T16:37:15.158170+0000","last_became_active":"2023-11-16T16:37:13.941966+0000","last_became_peered":"2023-11-16T16:37:13.941966+0000","last_unstale":"2023-11-16T16:37:15.158170+0000","last_undegraded":"2023-11-16T16:37:15.158170+0000","last_fullsized":"2023-11-16T16:37:15.158170+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:12:01.659205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158388+0000","last_change":"2023-11-16T16:37:13.943461+0000","last_active":"2023-11-16T16:37:15.158388+0000","last_peered":"2023-11-16T16:37:15.158388+0000","last_clean":"2023-11-16T16:37:15.158388+0000","last_became_active":"2023-11-16T16:37:13.943165+0000","last_became_peered":"2023-11-16T16:37:13.943165+0000","last_unstale":"2023-11-16T16:37:15.158388+0000","last_undegraded":"2023-11-16T16:37:15.158388+0000","last_fullsized":"2023-11-16T16:37:15.158388+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:09:30.792766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.122028+0000","last_change":"2023-11-16T16:37:13.940205+0000","last_active":"2023-11-16T16:37:15.122028+0000","last_peered":"2023-11-16T16:37:15.122028+0000","last_clean":"2023-11-16T16:37:15.122028+0000","last_became_active":"2023-11-16T16:37:13.939819+0000","last_became_peered":"2023-11-16T16:37:13.939819+0000","last_unstale":"2023-11-16T16:37:15.122028+0000","last_undegraded":"2023-11-16T16:37:15.122028+0000","last_fullsized":"2023-11-16T16:37:15.122028+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:40:08.471596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.923975+0000","last_change":"2023-11-16T16:37:13.939819+0000","last_active":"2023-11-16T16:37:14.923975+0000","last_peered":"2023-11-16T16:37:14.923975+0000","last_clean":"2023-11-16T16:37:14.923975+0000","last_became_active":"2023-11-16T16:37:13.939367+0000","last_became_peered":"2023-11-16T16:37:13.939367+0000","last_unstale":"2023-11-16T16:37:14.923975+0000","last_undegraded":"2023-11-16T16:37:14.923975+0000","last_fullsized":"2023-11-16T16:37:14.923975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:27:49.657269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.924379+0000","last_change":"2023-11-16T16:37:13.946613+0000","last_active":"2023-11-16T16:37:14.924379+0000","last_peered":"2023-11-16T16:37:14.924379+0000","last_clean":"2023-11-16T16:37:14.924379+0000","last_became_active":"2023-11-16T16:37:13.946281+0000","last_became_peered":"2023-11-16T16:37:13.946281+0000","last_unstale":"2023-11-16T16:37:14.924379+0000","last_undegraded":"2023-11-16T16:37:14.924379+0000","last_fullsized":"2023-11-16T16:37:14.924379+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:26:07.902412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.390884+0000","last_change":"2023-11-16T16:37:13.936899+0000","last_active":"2023-11-16T16:37:15.390884+0000","last_peered":"2023-11-16T16:37:15.390884+0000","last_clean":"2023-11-16T16:37:15.390884+0000","last_became_active":"2023-11-16T16:37:13.936543+0000","last_became_peered":"2023-11-16T16:37:13.936543+0000","last_unstale":"2023-11-16T16:37:15.390884+0000","last_undegraded":"2023-11-16T16:37:15.390884+0000","last_fullsized":"2023-11-16T16:37:15.390884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:55:14.512583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157287+0000","last_change":"2023-11-16T16:37:13.941463+0000","last_active":"2023-11-16T16:37:15.157287+0000","last_peered":"2023-11-16T16:37:15.157287+0000","last_clean":"2023-11-16T16:37:15.157287+0000","last_became_active":"2023-11-16T16:37:13.941125+0000","last_became_peered":"2023-11-16T16:37:13.941125+0000","last_unstale":"2023-11-16T16:37:15.157287+0000","last_undegraded":"2023-11-16T16:37:15.157287+0000","last_fullsized":"2023-11-16T16:37:15.157287+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:29:10.391228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157152+0000","last_change":"2023-11-16T16:37:13.934105+0000","last_active":"2023-11-16T16:37:15.157152+0000","last_peered":"2023-11-16T16:37:15.157152+0000","last_clean":"2023-11-16T16:37:15.157152+0000","last_became_active":"2023-11-16T16:37:13.933887+0000","last_became_peered":"2023-11-16T16:37:13.933887+0000","last_unstale":"2023-11-16T16:37:15.157152+0000","last_undegraded":"2023-11-16T16:37:15.157152+0000","last_fullsized":"2023-11-16T16:37:15.157152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:37:28.260833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.925511+0000","last_change":"2023-11-16T16:37:13.947266+0000","last_active":"2023-11-16T16:37:14.925511+0000","last_peered":"2023-11-16T16:37:14.925511+0000","last_clean":"2023-11-16T16:37:14.925511+0000","last_became_active":"2023-11-16T16:37:13.946919+0000","last_became_peered":"2023-11-16T16:37:13.946919+0000","last_unstale":"2023-11-16T16:37:14.925511+0000","last_undegraded":"2023-11-16T16:37:14.925511+0000","last_fullsized":"2023-11-16T16:37:14.925511+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:07:33.154784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.391182+0000","last_change":"2023-11-16T16:37:13.941597+0000","last_active":"2023-11-16T16:37:15.391182+0000","last_peered":"2023-11-16T16:37:15.391182+0000","last_clean":"2023-11-16T16:37:15.391182+0000","last_became_active":"2023-11-16T16:37:13.941253+0000","last_became_peered":"2023-11-16T16:37:13.941253+0000","last_unstale":"2023-11-16T16:37:15.391182+0000","last_undegraded":"2023-11-16T16:37:15.391182+0000","last_fullsized":"2023-11-16T16:37:15.391182+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:29:31.101716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.121996+0000","last_change":"2023-11-16T16:37:13.941673+0000","last_active":"2023-11-16T16:37:15.121996+0000","last_peered":"2023-11-16T16:37:15.121996+0000","last_clean":"2023-11-16T16:37:15.121996+0000","last_became_active":"2023-11-16T16:37:13.941345+0000","last_became_peered":"2023-11-16T16:37:13.941345+0000","last_unstale":"2023-11-16T16:37:15.121996+0000","last_undegraded":"2023-11-16T16:37:15.121996+0000","last_fullsized":"2023-11-16T16:37:15.121996+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:01:29.436252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157470+0000","last_change":"2023-11-16T16:37:13.947991+0000","last_active":"2023-11-16T16:37:15.157470+0000","last_peered":"2023-11-16T16:37:15.157470+0000","last_clean":"2023-11-16T16:37:15.157470+0000","last_became_active":"2023-11-16T16:37:13.947454+0000","last_became_peered":"2023-11-16T16:37:13.947454+0000","last_unstale":"2023-11-16T16:37:15.157470+0000","last_undegraded":"2023-11-16T16:37:15.157470+0000","last_fullsized":"2023-11-16T16:37:15.157470+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:35:41.400206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157374+0000","last_change":"2023-11-16T16:37:13.941502+0000","last_active":"2023-11-16T16:37:15.157374+0000","last_peered":"2023-11-16T16:37:15.157374+0000","last_clean":"2023-11-16T16:37:15.157374+0000","last_became_active":"2023-11-16T16:37:13.941191+0000","last_became_peered":"2023-11-16T16:37:13.941191+0000","last_unstale":"2023-11-16T16:37:15.157374+0000","last_undegraded":"2023-11-16T16:37:15.157374+0000","last_fullsized":"2023-11-16T16:37:15.157374+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:59:03.243696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.156825+0000","last_change":"2023-11-16T16:37:13.949198+0000","last_active":"2023-11-16T16:37:15.156825+0000","last_peered":"2023-11-16T16:37:15.156825+0000","last_clean":"2023-11-16T16:37:15.156825+0000","last_became_active":"2023-11-16T16:37:13.947147+0000","last_became_peered":"2023-11-16T16:37:13.947147+0000","last_unstale":"2023-11-16T16:37:15.156825+0000","last_undegraded":"2023-11-16T16:37:15.156825+0000","last_fullsized":"2023-11-16T16:37:15.156825+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:01:56.205870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.156530+0000","last_change":"2023-11-16T16:37:13.943915+0000","last_active":"2023-11-16T16:37:15.156530+0000","last_peered":"2023-11-16T16:37:15.156530+0000","last_clean":"2023-11-16T16:37:15.156530+0000","last_became_active":"2023-11-16T16:37:13.942995+0000","last_became_peered":"2023-11-16T16:37:13.942995+0000","last_unstale":"2023-11-16T16:37:15.156530+0000","last_undegraded":"2023-11-16T16:37:15.156530+0000","last_fullsized":"2023-11-16T16:37:15.156530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:35:55.774014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157841+0000","last_change":"2023-11-16T16:37:13.941816+0000","last_active":"2023-11-16T16:37:15.157841+0000","last_peered":"2023-11-16T16:37:15.157841+0000","last_clean":"2023-11-16T16:37:15.157841+0000","last_became_active":"2023-11-16T16:37:13.941541+0000","last_became_peered":"2023-11-16T16:37:13.941541+0000","last_unstale":"2023-11-16T16:37:15.157841+0000","last_undegraded":"2023-11-16T16:37:15.157841+0000","last_fullsized":"2023-11-16T16:37:15.157841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:18:40.904015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158134+0000","last_change":"2023-11-16T16:37:13.940639+0000","last_active":"2023-11-16T16:37:15.158134+0000","last_peered":"2023-11-16T16:37:15.158134+0000","last_clean":"2023-11-16T16:37:15.158134+0000","last_became_active":"2023-11-16T16:37:13.940323+0000","last_became_peered":"2023-11-16T16:37:13.940323+0000","last_unstale":"2023-11-16T16:37:15.158134+0000","last_undegraded":"2023-11-16T16:37:15.158134+0000","last_fullsized":"2023-11-16T16:37:15.158134+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:54:11.977928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158148+0000","last_change":"2023-11-16T16:37:13.934802+0000","last_active":"2023-11-16T16:37:15.158148+0000","last_peered":"2023-11-16T16:37:15.158148+0000","last_clean":"2023-11-16T16:37:15.158148+0000","last_became_active":"2023-11-16T16:37:13.934496+0000","last_became_peered":"2023-11-16T16:37:13.934496+0000","last_unstale":"2023-11-16T16:37:15.158148+0000","last_undegraded":"2023-11-16T16:37:15.158148+0000","last_fullsized":"2023-11-16T16:37:15.158148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:50:10.271792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'9","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:18.297486+0000","last_change":"2023-11-16T16:37:13.499284+0000","last_active":"2023-11-16T16:37:18.297486+0000","last_peered":"2023-11-16T16:37:18.297486+0000","last_clean":"2023-11-16T16:37:18.297486+0000","last_became_active":"2023-11-16T16:37:13.498931+0000","last_became_peered":"2023-11-16T16:37:13.498931+0000","last_unstale":"2023-11-16T16:37:18.297486+0000","last_undegraded":"2023-11-16T16:37:18.297486+0000","last_fullsized":"2023-11-16T16:37:18.297486+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:52:32.314115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157923+0000","last_change":"2023-11-16T16:37:13.939631+0000","last_active":"2023-11-16T16:37:15.157923+0000","last_peered":"2023-11-16T16:37:15.157923+0000","last_clean":"2023-11-16T16:37:15.157923+0000","last_became_active":"2023-11-16T16:37:13.939318+0000","last_became_peered":"2023-11-16T16:37:13.939318+0000","last_unstale":"2023-11-16T16:37:15.157923+0000","last_undegraded":"2023-11-16T16:37:15.157923+0000","last_fullsized":"2023-11-16T16:37:15.157923+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:51:05.730477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157928+0000","last_change":"2023-11-16T16:37:13.495972+0000","last_active":"2023-11-16T16:37:15.157928+0000","last_peered":"2023-11-16T16:37:15.157928+0000","last_clean":"2023-11-16T16:37:15.157928+0000","last_became_active":"2023-11-16T16:37:13.495610+0000","last_became_peered":"2023-11-16T16:37:13.495610+0000","last_unstale":"2023-11-16T16:37:15.157928+0000","last_undegraded":"2023-11-16T16:37:15.157928+0000","last_fullsized":"2023-11-16T16:37:15.157928+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:58:27.825452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158624+0000","last_change":"2023-11-16T16:37:13.935236+0000","last_active":"2023-11-16T16:37:15.158624+0000","last_peered":"2023-11-16T16:37:15.158624+0000","last_clean":"2023-11-16T16:37:15.158624+0000","last_became_active":"2023-11-16T16:37:13.934920+0000","last_became_peered":"2023-11-16T16:37:13.934920+0000","last_unstale":"2023-11-16T16:37:15.158624+0000","last_undegraded":"2023-11-16T16:37:15.158624+0000","last_fullsized":"2023-11-16T16:37:15.158624+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:58:27.305887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"84'4","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:18.296421+0000","last_change":"2023-11-16T16:37:13.509828+0000","last_active":"2023-11-16T16:37:18.296421+0000","last_peered":"2023-11-16T16:37:18.296421+0000","last_clean":"2023-11-16T16:37:18.296421+0000","last_became_active":"2023-11-16T16:37:13.509431+0000","last_became_peered":"2023-11-16T16:37:13.509431+0000","last_unstale":"2023-11-16T16:37:18.296421+0000","last_undegraded":"2023-11-16T16:37:18.296421+0000","last_fullsized":"2023-11-16T16:37:18.296421+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:49:39.240865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.156807+0000","last_change":"2023-11-16T16:37:13.940838+0000","last_active":"2023-11-16T16:37:15.156807+0000","last_peered":"2023-11-16T16:37:15.156807+0000","last_clean":"2023-11-16T16:37:15.156807+0000","last_became_active":"2023-11-16T16:37:13.940538+0000","last_became_peered":"2023-11-16T16:37:13.940538+0000","last_unstale":"2023-11-16T16:37:15.156807+0000","last_undegraded":"2023-11-16T16:37:15.156807+0000","last_fullsized":"2023-11-16T16:37:15.156807+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:24:41.595556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.925256+0000","last_change":"2023-11-16T16:37:13.495114+0000","last_active":"2023-11-16T16:37:14.925256+0000","last_peered":"2023-11-16T16:37:14.925256+0000","last_clean":"2023-11-16T16:37:14.925256+0000","last_became_active":"2023-11-16T16:37:13.494807+0000","last_became_peered":"2023-11-16T16:37:13.494807+0000","last_unstale":"2023-11-16T16:37:14.925256+0000","last_undegraded":"2023-11-16T16:37:14.925256+0000","last_fullsized":"2023-11-16T16:37:14.925256+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:24:04.736603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.924173+0000","last_change":"2023-11-16T16:37:13.943652+0000","last_active":"2023-11-16T16:37:14.924173+0000","last_peered":"2023-11-16T16:37:14.924173+0000","last_clean":"2023-11-16T16:37:14.924173+0000","last_became_active":"2023-11-16T16:37:13.943298+0000","last_became_peered":"2023-11-16T16:37:13.943298+0000","last_unstale":"2023-11-16T16:37:14.924173+0000","last_undegraded":"2023-11-16T16:37:14.924173+0000","last_fullsized":"2023-11-16T16:37:14.924173+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:34:34.876458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"84'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:20.303742+0000","last_change":"2023-11-16T16:37:13.509801+0000","last_active":"2023-11-16T16:37:20.303742+0000","last_peered":"2023-11-16T16:37:20.303742+0000","last_clean":"2023-11-16T16:37:20.303742+0000","last_became_active":"2023-11-16T16:37:13.509389+0000","last_became_peered":"2023-11-16T16:37:13.509389+0000","last_unstale":"2023-11-16T16:37:20.303742+0000","last_undegraded":"2023-11-16T16:37:20.303742+0000","last_fullsized":"2023-11-16T16:37:20.303742+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:55:02.922591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.924573+0000","last_change":"2023-11-16T16:37:13.933871+0000","last_active":"2023-11-16T16:37:14.924573+0000","last_peered":"2023-11-16T16:37:14.924573+0000","last_clean":"2023-11-16T16:37:14.924573+0000","last_became_active":"2023-11-16T16:37:13.933549+0000","last_became_peered":"2023-11-16T16:37:13.933549+0000","last_unstale":"2023-11-16T16:37:14.924573+0000","last_undegraded":"2023-11-16T16:37:14.924573+0000","last_fullsized":"2023-11-16T16:37:14.924573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:16:26.022577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"84'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:18.294818+0000","last_change":"2023-11-16T16:37:13.501339+0000","last_active":"2023-11-16T16:37:18.294818+0000","last_peered":"2023-11-16T16:37:18.294818+0000","last_clean":"2023-11-16T16:37:18.294818+0000","last_became_active":"2023-11-16T16:37:13.500949+0000","last_became_peered":"2023-11-16T16:37:13.500949+0000","last_unstale":"2023-11-16T16:37:18.294818+0000","last_undegraded":"2023-11-16T16:37:18.294818+0000","last_fullsized":"2023-11-16T16:37:18.294818+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:31:08.744759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.923893+0000","last_change":"2023-11-16T16:37:13.944158+0000","last_active":"2023-11-16T16:37:14.923893+0000","last_peered":"2023-11-16T16:37:14.923893+0000","last_clean":"2023-11-16T16:37:14.923893+0000","last_became_active":"2023-11-16T16:37:13.943833+0000","last_became_peered":"2023-11-16T16:37:13.943833+0000","last_unstale":"2023-11-16T16:37:14.923893+0000","last_undegraded":"2023-11-16T16:37:14.923893+0000","last_fullsized":"2023-11-16T16:37:14.923893+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:36:01.615006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"84'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:18.295129+0000","last_change":"2023-11-16T16:37:13.505481+0000","last_active":"2023-11-16T16:37:18.295129+0000","last_peered":"2023-11-16T16:37:18.295129+0000","last_clean":"2023-11-16T16:37:18.295129+0000","last_became_active":"2023-11-16T16:37:13.505085+0000","last_became_peered":"2023-11-16T16:37:13.505085+0000","last_unstale":"2023-11-16T16:37:18.295129+0000","last_undegraded":"2023-11-16T16:37:18.295129+0000","last_fullsized":"2023-11-16T16:37:18.295129+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:11:35.628249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157157+0000","last_change":"2023-11-16T16:37:13.943985+0000","last_active":"2023-11-16T16:37:15.157157+0000","last_peered":"2023-11-16T16:37:15.157157+0000","last_clean":"2023-11-16T16:37:15.157157+0000","last_became_active":"2023-11-16T16:37:13.943628+0000","last_became_peered":"2023-11-16T16:37:13.943628+0000","last_unstale":"2023-11-16T16:37:15.157157+0000","last_undegraded":"2023-11-16T16:37:15.157157+0000","last_fullsized":"2023-11-16T16:37:15.157157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:19:10.518280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.462357+0000","last_change":"2023-11-16T16:37:13.498648+0000","last_active":"2023-11-16T16:37:15.462357+0000","last_peered":"2023-11-16T16:37:15.462357+0000","last_clean":"2023-11-16T16:37:15.462357+0000","last_became_active":"2023-11-16T16:37:13.497390+0000","last_became_peered":"2023-11-16T16:37:13.497390+0000","last_unstale":"2023-11-16T16:37:15.462357+0000","last_undegraded":"2023-11-16T16:37:15.462357+0000","last_fullsized":"2023-11-16T16:37:15.462357+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:26:59.991663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157618+0000","last_change":"2023-11-16T16:37:13.942337+0000","last_active":"2023-11-16T16:37:15.157618+0000","last_peered":"2023-11-16T16:37:15.157618+0000","last_clean":"2023-11-16T16:37:15.157618+0000","last_became_active":"2023-11-16T16:37:13.941982+0000","last_became_peered":"2023-11-16T16:37:13.941982+0000","last_unstale":"2023-11-16T16:37:15.157618+0000","last_undegraded":"2023-11-16T16:37:15.157618+0000","last_fullsized":"2023-11-16T16:37:15.157618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:38:42.000956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:19.300306+0000","last_change":"2023-11-16T16:37:13.503690+0000","last_active":"2023-11-16T16:37:19.300306+0000","last_peered":"2023-11-16T16:37:19.300306+0000","last_clean":"2023-11-16T16:37:19.300306+0000","last_became_active":"2023-11-16T16:37:13.503203+0000","last_became_peered":"2023-11-16T16:37:13.503203+0000","last_unstale":"2023-11-16T16:37:19.300306+0000","last_undegraded":"2023-11-16T16:37:19.300306+0000","last_fullsized":"2023-11-16T16:37:19.300306+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:27:52.629218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.156964+0000","last_change":"2023-11-16T16:37:13.943975+0000","last_active":"2023-11-16T16:37:15.156964+0000","last_peered":"2023-11-16T16:37:15.156964+0000","last_clean":"2023-11-16T16:37:15.156964+0000","last_became_active":"2023-11-16T16:37:13.943611+0000","last_became_peered":"2023-11-16T16:37:13.943611+0000","last_unstale":"2023-11-16T16:37:15.156964+0000","last_undegraded":"2023-11-16T16:37:15.156964+0000","last_fullsized":"2023-11-16T16:37:15.156964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:21:29.078427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:17.285557+0000","last_change":"2023-11-16T16:37:13.506076+0000","last_active":"2023-11-16T16:37:17.285557+0000","last_peered":"2023-11-16T16:37:17.285557+0000","last_clean":"2023-11-16T16:37:17.285557+0000","last_became_active":"2023-11-16T16:37:13.505715+0000","last_became_peered":"2023-11-16T16:37:13.505715+0000","last_unstale":"2023-11-16T16:37:17.285557+0000","last_undegraded":"2023-11-16T16:37:17.285557+0000","last_fullsized":"2023-11-16T16:37:17.285557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:03:10.081004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158690+0000","last_change":"2023-11-16T16:37:13.949031+0000","last_active":"2023-11-16T16:37:15.158690+0000","last_peered":"2023-11-16T16:37:15.158690+0000","last_clean":"2023-11-16T16:37:15.158690+0000","last_became_active":"2023-11-16T16:37:13.946923+0000","last_became_peered":"2023-11-16T16:37:13.946923+0000","last_unstale":"2023-11-16T16:37:15.158690+0000","last_undegraded":"2023-11-16T16:37:15.158690+0000","last_fullsized":"2023-11-16T16:37:15.158690+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:00:35.683602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:17.286359+0000","last_change":"2023-11-16T16:37:13.495003+0000","last_active":"2023-11-16T16:37:17.286359+0000","last_peered":"2023-11-16T16:37:17.286359+0000","last_clean":"2023-11-16T16:37:17.286359+0000","last_became_active":"2023-11-16T16:37:13.494563+0000","last_became_peered":"2023-11-16T16:37:13.494563+0000","last_unstale":"2023-11-16T16:37:17.286359+0000","last_undegraded":"2023-11-16T16:37:17.286359+0000","last_fullsized":"2023-11-16T16:37:17.286359+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:12:53.840907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.156822+0000","last_change":"2023-11-16T16:37:13.943736+0000","last_active":"2023-11-16T16:37:15.156822+0000","last_peered":"2023-11-16T16:37:15.156822+0000","last_clean":"2023-11-16T16:37:15.156822+0000","last_became_active":"2023-11-16T16:37:13.943391+0000","last_became_peered":"2023-11-16T16:37:13.943391+0000","last_unstale":"2023-11-16T16:37:15.156822+0000","last_undegraded":"2023-11-16T16:37:15.156822+0000","last_fullsized":"2023-11-16T16:37:15.156822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:03:21.402516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'14","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:20.305738+0000","last_change":"2023-11-16T16:37:13.506508+0000","last_active":"2023-11-16T16:37:20.305738+0000","last_peered":"2023-11-16T16:37:20.305738+0000","last_clean":"2023-11-16T16:37:20.305738+0000","last_became_active":"2023-11-16T16:37:13.506045+0000","last_became_peered":"2023-11-16T16:37:13.506045+0000","last_unstale":"2023-11-16T16:37:20.305738+0000","last_undegraded":"2023-11-16T16:37:20.305738+0000","last_fullsized":"2023-11-16T16:37:20.305738+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:26:04.265699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18929,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":21,"num_write":15,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.924203+0000","last_change":"2023-11-16T16:37:13.939717+0000","last_active":"2023-11-16T16:37:14.924203+0000","last_peered":"2023-11-16T16:37:14.924203+0000","last_clean":"2023-11-16T16:37:14.924203+0000","last_became_active":"2023-11-16T16:37:13.939369+0000","last_became_peered":"2023-11-16T16:37:13.939369+0000","last_unstale":"2023-11-16T16:37:14.924203+0000","last_undegraded":"2023-11-16T16:37:14.924203+0000","last_fullsized":"2023-11-16T16:37:14.924203+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:15:46.898377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'6","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:18.296464+0000","last_change":"2023-11-16T16:37:13.506056+0000","last_active":"2023-11-16T16:37:18.296464+0000","last_peered":"2023-11-16T16:37:18.296464+0000","last_clean":"2023-11-16T16:37:18.296464+0000","last_became_active":"2023-11-16T16:37:13.505707+0000","last_became_peered":"2023-11-16T16:37:13.505707+0000","last_unstale":"2023-11-16T16:37:18.296464+0000","last_undegraded":"2023-11-16T16:37:18.296464+0000","last_fullsized":"2023-11-16T16:37:18.296464+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:36:08.667708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157120+0000","last_change":"2023-11-16T16:37:13.947764+0000","last_active":"2023-11-16T16:37:15.157120+0000","last_peered":"2023-11-16T16:37:15.157120+0000","last_clean":"2023-11-16T16:37:15.157120+0000","last_became_active":"2023-11-16T16:37:13.947510+0000","last_became_peered":"2023-11-16T16:37:13.947510+0000","last_unstale":"2023-11-16T16:37:15.157120+0000","last_undegraded":"2023-11-16T16:37:15.157120+0000","last_fullsized":"2023-11-16T16:37:15.157120+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:08:15.784489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:17.287513+0000","last_change":"2023-11-16T16:37:13.498178+0000","last_active":"2023-11-16T16:37:17.287513+0000","last_peered":"2023-11-16T16:37:17.287513+0000","last_clean":"2023-11-16T16:37:17.287513+0000","last_became_active":"2023-11-16T16:37:13.497312+0000","last_became_peered":"2023-11-16T16:37:13.497312+0000","last_unstale":"2023-11-16T16:37:17.287513+0000","last_undegraded":"2023-11-16T16:37:17.287513+0000","last_fullsized":"2023-11-16T16:37:17.287513+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:01:58.840690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.391132+0000","last_change":"2023-11-16T16:37:13.947854+0000","last_active":"2023-11-16T16:37:15.391132+0000","last_peered":"2023-11-16T16:37:15.391132+0000","last_clean":"2023-11-16T16:37:15.391132+0000","last_became_active":"2023-11-16T16:37:13.947530+0000","last_became_peered":"2023-11-16T16:37:13.947530+0000","last_unstale":"2023-11-16T16:37:15.391132+0000","last_undegraded":"2023-11-16T16:37:15.391132+0000","last_fullsized":"2023-11-16T16:37:15.391132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:25:25.919855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:18.296418+0000","last_change":"2023-11-16T16:37:13.500127+0000","last_active":"2023-11-16T16:37:18.296418+0000","last_peered":"2023-11-16T16:37:18.296418+0000","last_clean":"2023-11-16T16:37:18.296418+0000","last_became_active":"2023-11-16T16:37:13.499758+0000","last_became_peered":"2023-11-16T16:37:13.499758+0000","last_unstale":"2023-11-16T16:37:18.296418+0000","last_undegraded":"2023-11-16T16:37:18.296418+0000","last_fullsized":"2023-11-16T16:37:18.296418+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:12:24.610215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.390796+0000","last_change":"2023-11-16T16:37:13.940962+0000","last_active":"2023-11-16T16:37:15.390796+0000","last_peered":"2023-11-16T16:37:15.390796+0000","last_clean":"2023-11-16T16:37:15.390796+0000","last_became_active":"2023-11-16T16:37:13.940450+0000","last_became_peered":"2023-11-16T16:37:13.940450+0000","last_unstale":"2023-11-16T16:37:15.390796+0000","last_undegraded":"2023-11-16T16:37:15.390796+0000","last_fullsized":"2023-11-16T16:37:15.390796+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:20:04.879494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'15","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:18.297513+0000","last_change":"2023-11-16T16:37:13.493017+0000","last_active":"2023-11-16T16:37:18.297513+0000","last_peered":"2023-11-16T16:37:18.297513+0000","last_clean":"2023-11-16T16:37:18.297513+0000","last_became_active":"2023-11-16T16:37:13.492599+0000","last_became_peered":"2023-11-16T16:37:13.492599+0000","last_unstale":"2023-11-16T16:37:18.297513+0000","last_undegraded":"2023-11-16T16:37:18.297513+0000","last_fullsized":"2023-11-16T16:37:18.297513+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:25:52.252764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.924063+0000","last_change":"2023-11-16T16:37:13.944878+0000","last_active":"2023-11-16T16:37:14.924063+0000","last_peered":"2023-11-16T16:37:14.924063+0000","last_clean":"2023-11-16T16:37:14.924063+0000","last_became_active":"2023-11-16T16:37:13.944580+0000","last_became_peered":"2023-11-16T16:37:13.944580+0000","last_unstale":"2023-11-16T16:37:14.924063+0000","last_undegraded":"2023-11-16T16:37:14.924063+0000","last_fullsized":"2023-11-16T16:37:14.924063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:01:52.805052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:19.300067+0000","last_change":"2023-11-16T16:37:13.499139+0000","last_active":"2023-11-16T16:37:19.300067+0000","last_peered":"2023-11-16T16:37:19.300067+0000","last_clean":"2023-11-16T16:37:19.300067+0000","last_became_active":"2023-11-16T16:37:13.498755+0000","last_became_peered":"2023-11-16T16:37:13.498755+0000","last_unstale":"2023-11-16T16:37:19.300067+0000","last_undegraded":"2023-11-16T16:37:19.300067+0000","last_fullsized":"2023-11-16T16:37:19.300067+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:07:40.535306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157785+0000","last_change":"2023-11-16T16:37:13.946187+0000","last_active":"2023-11-16T16:37:15.157785+0000","last_peered":"2023-11-16T16:37:15.157785+0000","last_clean":"2023-11-16T16:37:15.157785+0000","last_became_active":"2023-11-16T16:37:13.945887+0000","last_became_peered":"2023-11-16T16:37:13.945887+0000","last_unstale":"2023-11-16T16:37:15.157785+0000","last_undegraded":"2023-11-16T16:37:15.157785+0000","last_fullsized":"2023-11-16T16:37:15.157785+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:29:41.439670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157191+0000","last_change":"2023-11-16T16:37:13.507150+0000","last_active":"2023-11-16T16:37:15.157191+0000","last_peered":"2023-11-16T16:37:15.157191+0000","last_clean":"2023-11-16T16:37:15.157191+0000","last_became_active":"2023-11-16T16:37:13.506814+0000","last_became_peered":"2023-11-16T16:37:13.506814+0000","last_unstale":"2023-11-16T16:37:15.157191+0000","last_undegraded":"2023-11-16T16:37:15.157191+0000","last_fullsized":"2023-11-16T16:37:15.157191+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:58:14.692816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.462555+0000","last_change":"2023-11-16T16:37:13.947866+0000","last_active":"2023-11-16T16:37:15.462555+0000","last_peered":"2023-11-16T16:37:15.462555+0000","last_clean":"2023-11-16T16:37:15.462555+0000","last_became_active":"2023-11-16T16:37:13.947544+0000","last_became_peered":"2023-11-16T16:37:13.947544+0000","last_unstale":"2023-11-16T16:37:15.462555+0000","last_undegraded":"2023-11-16T16:37:15.462555+0000","last_fullsized":"2023-11-16T16:37:15.462555+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:58:42.237793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"84'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.391037+0000","last_change":"2023-11-16T16:37:13.497800+0000","last_active":"2023-11-16T16:37:15.391037+0000","last_peered":"2023-11-16T16:37:15.391037+0000","last_clean":"2023-11-16T16:37:15.391037+0000","last_became_active":"2023-11-16T16:37:13.497414+0000","last_became_peered":"2023-11-16T16:37:13.497414+0000","last_unstale":"2023-11-16T16:37:15.391037+0000","last_undegraded":"2023-11-16T16:37:15.391037+0000","last_fullsized":"2023-11-16T16:37:15.391037+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:57:26.591481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.924384+0000","last_change":"2023-11-16T16:37:13.946780+0000","last_active":"2023-11-16T16:37:14.924384+0000","last_peered":"2023-11-16T16:37:14.924384+0000","last_clean":"2023-11-16T16:37:14.924384+0000","last_became_active":"2023-11-16T16:37:13.946441+0000","last_became_peered":"2023-11-16T16:37:13.946441+0000","last_unstale":"2023-11-16T16:37:14.924384+0000","last_undegraded":"2023-11-16T16:37:14.924384+0000","last_fullsized":"2023-11-16T16:37:14.924384+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:20:09.943373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.923519+0000","last_change":"2023-11-16T16:37:13.937898+0000","last_active":"2023-11-16T16:37:14.923519+0000","last_peered":"2023-11-16T16:37:14.923519+0000","last_clean":"2023-11-16T16:37:14.923519+0000","last_became_active":"2023-11-16T16:37:13.937598+0000","last_became_peered":"2023-11-16T16:37:13.937598+0000","last_unstale":"2023-11-16T16:37:14.923519+0000","last_undegraded":"2023-11-16T16:37:14.923519+0000","last_fullsized":"2023-11-16T16:37:14.923519+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:57:03.646617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"84'4","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:18.297366+0000","last_change":"2023-11-16T16:37:13.498129+0000","last_active":"2023-11-16T16:37:18.297366+0000","last_peered":"2023-11-16T16:37:18.297366+0000","last_clean":"2023-11-16T16:37:18.297366+0000","last_became_active":"2023-11-16T16:37:13.497765+0000","last_became_peered":"2023-11-16T16:37:13.497765+0000","last_unstale":"2023-11-16T16:37:18.297366+0000","last_undegraded":"2023-11-16T16:37:18.297366+0000","last_fullsized":"2023-11-16T16:37:18.297366+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:13:33.579950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158156+0000","last_change":"2023-11-16T16:37:13.938417+0000","last_active":"2023-11-16T16:37:15.158156+0000","last_peered":"2023-11-16T16:37:15.158156+0000","last_clean":"2023-11-16T16:37:15.158156+0000","last_became_active":"2023-11-16T16:37:13.938114+0000","last_became_peered":"2023-11-16T16:37:13.938114+0000","last_unstale":"2023-11-16T16:37:15.158156+0000","last_undegraded":"2023-11-16T16:37:15.158156+0000","last_fullsized":"2023-11-16T16:37:15.158156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:35:24.220360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"84'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:17.288003+0000","last_change":"2023-11-16T16:37:13.499045+0000","last_active":"2023-11-16T16:37:17.288003+0000","last_peered":"2023-11-16T16:37:17.288003+0000","last_clean":"2023-11-16T16:37:17.288003+0000","last_became_active":"2023-11-16T16:37:13.498598+0000","last_became_peered":"2023-11-16T16:37:13.498598+0000","last_unstale":"2023-11-16T16:37:17.288003+0000","last_undegraded":"2023-11-16T16:37:17.288003+0000","last_fullsized":"2023-11-16T16:37:17.288003+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:25:26.520931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":345,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.390796+0000","last_change":"2023-11-16T16:36:14.090213+0000","last_active":"2023-11-16T16:37:15.390796+0000","last_peered":"2023-11-16T16:37:15.390796+0000","last_clean":"2023-11-16T16:37:15.390796+0000","last_became_active":"2023-11-16T16:36:14.089742+0000","last_became_peered":"2023-11-16T16:36:14.089742+0000","last_unstale":"2023-11-16T16:37:15.390796+0000","last_undegraded":"2023-11-16T16:37:15.390796+0000","last_fullsized":"2023-11-16T16:37:15.390796+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:33:16.178716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:33:16.178716+0000","last_clean_scrub_stamp":"2023-11-16T16:33:16.178716+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:11:37.401838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157103+0000","last_change":"2023-11-16T16:37:13.945380+0000","last_active":"2023-11-16T16:37:15.157103+0000","last_peered":"2023-11-16T16:37:15.157103+0000","last_clean":"2023-11-16T16:37:15.157103+0000","last_became_active":"2023-11-16T16:37:13.945038+0000","last_became_peered":"2023-11-16T16:37:13.945038+0000","last_unstale":"2023-11-16T16:37:15.157103+0000","last_undegraded":"2023-11-16T16:37:15.157103+0000","last_fullsized":"2023-11-16T16:37:15.157103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:35:29.662867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"84'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:18.296049+0000","last_change":"2023-11-16T16:37:13.497586+0000","last_active":"2023-11-16T16:37:18.296049+0000","last_peered":"2023-11-16T16:37:18.296049+0000","last_clean":"2023-11-16T16:37:18.296049+0000","last_became_active":"2023-11-16T16:37:13.497233+0000","last_became_peered":"2023-11-16T16:37:13.497233+0000","last_unstale":"2023-11-16T16:37:18.296049+0000","last_undegraded":"2023-11-16T16:37:18.296049+0000","last_fullsized":"2023-11-16T16:37:18.296049+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:24:30.086496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"84'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:17.285613+0000","last_change":"2023-11-16T16:37:13.500621+0000","last_active":"2023-11-16T16:37:17.285613+0000","last_peered":"2023-11-16T16:37:17.285613+0000","last_clean":"2023-11-16T16:37:17.285613+0000","last_became_active":"2023-11-16T16:37:13.500265+0000","last_became_peered":"2023-11-16T16:37:13.500265+0000","last_unstale":"2023-11-16T16:37:17.285613+0000","last_undegraded":"2023-11-16T16:37:17.285613+0000","last_fullsized":"2023-11-16T16:37:17.285613+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:51:28.450861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157560+0000","last_change":"2023-11-16T16:37:13.930992+0000","last_active":"2023-11-16T16:37:15.157560+0000","last_peered":"2023-11-16T16:37:15.157560+0000","last_clean":"2023-11-16T16:37:15.157560+0000","last_became_active":"2023-11-16T16:37:13.930641+0000","last_became_peered":"2023-11-16T16:37:13.930641+0000","last_unstale":"2023-11-16T16:37:15.157560+0000","last_undegraded":"2023-11-16T16:37:15.157560+0000","last_fullsized":"2023-11-16T16:37:15.157560+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:58:37.743753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:18.297061+0000","last_change":"2023-11-16T16:37:13.498637+0000","last_active":"2023-11-16T16:37:18.297061+0000","last_peered":"2023-11-16T16:37:18.297061+0000","last_clean":"2023-11-16T16:37:18.297061+0000","last_became_active":"2023-11-16T16:37:13.498341+0000","last_became_peered":"2023-11-16T16:37:13.498341+0000","last_unstale":"2023-11-16T16:37:18.297061+0000","last_undegraded":"2023-11-16T16:37:18.297061+0000","last_fullsized":"2023-11-16T16:37:18.297061+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:00:31.333304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158333+0000","last_change":"2023-11-16T16:37:13.944142+0000","last_active":"2023-11-16T16:37:15.158333+0000","last_peered":"2023-11-16T16:37:15.158333+0000","last_clean":"2023-11-16T16:37:15.158333+0000","last_became_active":"2023-11-16T16:37:13.943758+0000","last_became_peered":"2023-11-16T16:37:13.943758+0000","last_unstale":"2023-11-16T16:37:15.158333+0000","last_undegraded":"2023-11-16T16:37:15.158333+0000","last_fullsized":"2023-11-16T16:37:15.158333+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:55:52.343549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'4","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:17.287097+0000","last_change":"2023-11-16T16:37:13.496900+0000","last_active":"2023-11-16T16:37:17.287097+0000","last_peered":"2023-11-16T16:37:17.287097+0000","last_clean":"2023-11-16T16:37:17.287097+0000","last_became_active":"2023-11-16T16:37:13.496528+0000","last_became_peered":"2023-11-16T16:37:13.496528+0000","last_unstale":"2023-11-16T16:37:17.287097+0000","last_undegraded":"2023-11-16T16:37:17.287097+0000","last_fullsized":"2023-11-16T16:37:17.287097+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:37:22.646222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158258+0000","last_change":"2023-11-16T16:37:13.943104+0000","last_active":"2023-11-16T16:37:15.158258+0000","last_peered":"2023-11-16T16:37:15.158258+0000","last_clean":"2023-11-16T16:37:15.158258+0000","last_became_active":"2023-11-16T16:37:13.942774+0000","last_became_peered":"2023-11-16T16:37:13.942774+0000","last_unstale":"2023-11-16T16:37:15.158258+0000","last_undegraded":"2023-11-16T16:37:15.158258+0000","last_fullsized":"2023-11-16T16:37:15.158258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:00:01.892620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:17.287386+0000","last_change":"2023-11-16T16:37:13.505994+0000","last_active":"2023-11-16T16:37:17.287386+0000","last_peered":"2023-11-16T16:37:17.287386+0000","last_clean":"2023-11-16T16:37:17.287386+0000","last_became_active":"2023-11-16T16:37:13.505663+0000","last_became_peered":"2023-11-16T16:37:13.505663+0000","last_unstale":"2023-11-16T16:37:17.287386+0000","last_undegraded":"2023-11-16T16:37:17.287386+0000","last_fullsized":"2023-11-16T16:37:17.287386+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:11:39.436119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.923847+0000","last_change":"2023-11-16T16:37:13.941970+0000","last_active":"2023-11-16T16:37:14.923847+0000","last_peered":"2023-11-16T16:37:14.923847+0000","last_clean":"2023-11-16T16:37:14.923847+0000","last_became_active":"2023-11-16T16:37:13.941649+0000","last_became_peered":"2023-11-16T16:37:13.941649+0000","last_unstale":"2023-11-16T16:37:14.923847+0000","last_undegraded":"2023-11-16T16:37:14.923847+0000","last_fullsized":"2023-11-16T16:37:14.923847+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:00:21.646221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157550+0000","last_change":"2023-11-16T16:37:13.503429+0000","last_active":"2023-11-16T16:37:15.157550+0000","last_peered":"2023-11-16T16:37:15.157550+0000","last_clean":"2023-11-16T16:37:15.157550+0000","last_became_active":"2023-11-16T16:37:13.503114+0000","last_became_peered":"2023-11-16T16:37:13.503114+0000","last_unstale":"2023-11-16T16:37:15.157550+0000","last_undegraded":"2023-11-16T16:37:15.157550+0000","last_fullsized":"2023-11-16T16:37:15.157550+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:05:42.643043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157164+0000","last_change":"2023-11-16T16:37:13.941225+0000","last_active":"2023-11-16T16:37:15.157164+0000","last_peered":"2023-11-16T16:37:15.157164+0000","last_clean":"2023-11-16T16:37:15.157164+0000","last_became_active":"2023-11-16T16:37:13.940964+0000","last_became_peered":"2023-11-16T16:37:13.940964+0000","last_unstale":"2023-11-16T16:37:15.157164+0000","last_undegraded":"2023-11-16T16:37:15.157164+0000","last_fullsized":"2023-11-16T16:37:15.157164+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:16:06.854930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157641+0000","last_change":"2023-11-16T16:37:13.510154+0000","last_active":"2023-11-16T16:37:15.157641+0000","last_peered":"2023-11-16T16:37:15.157641+0000","last_clean":"2023-11-16T16:37:15.157641+0000","last_became_active":"2023-11-16T16:37:13.509790+0000","last_became_peered":"2023-11-16T16:37:13.509790+0000","last_unstale":"2023-11-16T16:37:15.157641+0000","last_undegraded":"2023-11-16T16:37:15.157641+0000","last_fullsized":"2023-11-16T16:37:15.157641+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T16:54:36.581541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158172+0000","last_change":"2023-11-16T16:37:13.942203+0000","last_active":"2023-11-16T16:37:15.158172+0000","last_peered":"2023-11-16T16:37:15.158172+0000","last_clean":"2023-11-16T16:37:15.158172+0000","last_became_active":"2023-11-16T16:37:13.941766+0000","last_became_peered":"2023-11-16T16:37:13.941766+0000","last_unstale":"2023-11-16T16:37:15.158172+0000","last_undegraded":"2023-11-16T16:37:15.158172+0000","last_fullsized":"2023-11-16T16:37:15.158172+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:12:25.582734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:17.286155+0000","last_change":"2023-11-16T16:37:13.509854+0000","last_active":"2023-11-16T16:37:17.286155+0000","last_peered":"2023-11-16T16:37:17.286155+0000","last_clean":"2023-11-16T16:37:17.286155+0000","last_became_active":"2023-11-16T16:37:13.509466+0000","last_became_peered":"2023-11-16T16:37:13.509466+0000","last_unstale":"2023-11-16T16:37:17.286155+0000","last_undegraded":"2023-11-16T16:37:17.286155+0000","last_fullsized":"2023-11-16T16:37:17.286155+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:05:41.806694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.391285+0000","last_change":"2023-11-16T16:37:13.947906+0000","last_active":"2023-11-16T16:37:15.391285+0000","last_peered":"2023-11-16T16:37:15.391285+0000","last_clean":"2023-11-16T16:37:15.391285+0000","last_became_active":"2023-11-16T16:37:13.947602+0000","last_became_peered":"2023-11-16T16:37:13.947602+0000","last_unstale":"2023-11-16T16:37:15.391285+0000","last_undegraded":"2023-11-16T16:37:15.391285+0000","last_fullsized":"2023-11-16T16:37:15.391285+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:16:47.335757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:18.297466+0000","last_change":"2023-11-16T16:37:13.506108+0000","last_active":"2023-11-16T16:37:18.297466+0000","last_peered":"2023-11-16T16:37:18.297466+0000","last_clean":"2023-11-16T16:37:18.297466+0000","last_became_active":"2023-11-16T16:37:13.505785+0000","last_became_peered":"2023-11-16T16:37:13.505785+0000","last_unstale":"2023-11-16T16:37:18.297466+0000","last_undegraded":"2023-11-16T16:37:18.297466+0000","last_fullsized":"2023-11-16T16:37:18.297466+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:01:15.544970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.391265+0000","last_change":"2023-11-16T16:37:13.935863+0000","last_active":"2023-11-16T16:37:15.391265+0000","last_peered":"2023-11-16T16:37:15.391265+0000","last_clean":"2023-11-16T16:37:15.391265+0000","last_became_active":"2023-11-16T16:37:13.935399+0000","last_became_peered":"2023-11-16T16:37:13.935399+0000","last_unstale":"2023-11-16T16:37:15.391265+0000","last_undegraded":"2023-11-16T16:37:15.391265+0000","last_fullsized":"2023-11-16T16:37:15.391265+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:38:54.382348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.462427+0000","last_change":"2023-11-16T16:37:13.498367+0000","last_active":"2023-11-16T16:37:15.462427+0000","last_peered":"2023-11-16T16:37:15.462427+0000","last_clean":"2023-11-16T16:37:15.462427+0000","last_became_active":"2023-11-16T16:37:13.497039+0000","last_became_peered":"2023-11-16T16:37:13.497039+0000","last_unstale":"2023-11-16T16:37:15.462427+0000","last_undegraded":"2023-11-16T16:37:15.462427+0000","last_fullsized":"2023-11-16T16:37:15.462427+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:07:52.215240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.391119+0000","last_change":"2023-11-16T16:37:13.940617+0000","last_active":"2023-11-16T16:37:15.391119+0000","last_peered":"2023-11-16T16:37:15.391119+0000","last_clean":"2023-11-16T16:37:15.391119+0000","last_became_active":"2023-11-16T16:37:13.940264+0000","last_became_peered":"2023-11-16T16:37:13.940264+0000","last_unstale":"2023-11-16T16:37:15.391119+0000","last_undegraded":"2023-11-16T16:37:15.391119+0000","last_fullsized":"2023-11-16T16:37:15.391119+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:22:54.941976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157351+0000","last_change":"2023-11-16T16:37:13.505950+0000","last_active":"2023-11-16T16:37:15.157351+0000","last_peered":"2023-11-16T16:37:15.157351+0000","last_clean":"2023-11-16T16:37:15.157351+0000","last_became_active":"2023-11-16T16:37:13.505628+0000","last_became_peered":"2023-11-16T16:37:13.505628+0000","last_unstale":"2023-11-16T16:37:15.157351+0000","last_undegraded":"2023-11-16T16:37:15.157351+0000","last_fullsized":"2023-11-16T16:37:15.157351+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:11:17.082705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.391279+0000","last_change":"2023-11-16T16:37:13.943096+0000","last_active":"2023-11-16T16:37:15.391279+0000","last_peered":"2023-11-16T16:37:15.391279+0000","last_clean":"2023-11-16T16:37:15.391279+0000","last_became_active":"2023-11-16T16:37:13.942745+0000","last_became_peered":"2023-11-16T16:37:13.942745+0000","last_unstale":"2023-11-16T16:37:15.391279+0000","last_undegraded":"2023-11-16T16:37:15.391279+0000","last_fullsized":"2023-11-16T16:37:15.391279+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:35:21.092899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157245+0000","last_change":"2023-11-16T16:37:13.938087+0000","last_active":"2023-11-16T16:37:15.157245+0000","last_peered":"2023-11-16T16:37:15.157245+0000","last_clean":"2023-11-16T16:37:15.157245+0000","last_became_active":"2023-11-16T16:37:13.937780+0000","last_became_peered":"2023-11-16T16:37:13.937780+0000","last_unstale":"2023-11-16T16:37:15.157245+0000","last_undegraded":"2023-11-16T16:37:15.157245+0000","last_fullsized":"2023-11-16T16:37:15.157245+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:41:00.785196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157978+0000","last_change":"2023-11-16T16:37:13.939000+0000","last_active":"2023-11-16T16:37:15.157978+0000","last_peered":"2023-11-16T16:37:15.157978+0000","last_clean":"2023-11-16T16:37:15.157978+0000","last_became_active":"2023-11-16T16:37:13.938585+0000","last_became_peered":"2023-11-16T16:37:13.938585+0000","last_unstale":"2023-11-16T16:37:15.157978+0000","last_undegraded":"2023-11-16T16:37:15.157978+0000","last_fullsized":"2023-11-16T16:37:15.157978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:17:47.012171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157516+0000","last_change":"2023-11-16T16:37:13.937696+0000","last_active":"2023-11-16T16:37:15.157516+0000","last_peered":"2023-11-16T16:37:15.157516+0000","last_clean":"2023-11-16T16:37:15.157516+0000","last_became_active":"2023-11-16T16:37:13.936920+0000","last_became_peered":"2023-11-16T16:37:13.936920+0000","last_unstale":"2023-11-16T16:37:15.157516+0000","last_undegraded":"2023-11-16T16:37:15.157516+0000","last_fullsized":"2023-11-16T16:37:15.157516+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:40:10.578228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157738+0000","last_change":"2023-11-16T16:37:13.942977+0000","last_active":"2023-11-16T16:37:15.157738+0000","last_peered":"2023-11-16T16:37:15.157738+0000","last_clean":"2023-11-16T16:37:15.157738+0000","last_became_active":"2023-11-16T16:37:13.942632+0000","last_became_peered":"2023-11-16T16:37:13.942632+0000","last_unstale":"2023-11-16T16:37:15.157738+0000","last_undegraded":"2023-11-16T16:37:15.157738+0000","last_fullsized":"2023-11-16T16:37:15.157738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:44:02.416862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157651+0000","last_change":"2023-11-16T16:37:13.949940+0000","last_active":"2023-11-16T16:37:15.157651+0000","last_peered":"2023-11-16T16:37:15.157651+0000","last_clean":"2023-11-16T16:37:15.157651+0000","last_became_active":"2023-11-16T16:37:13.949464+0000","last_became_peered":"2023-11-16T16:37:13.949464+0000","last_unstale":"2023-11-16T16:37:15.157651+0000","last_undegraded":"2023-11-16T16:37:15.157651+0000","last_fullsized":"2023-11-16T16:37:15.157651+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:52:47.439482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157535+0000","last_change":"2023-11-16T16:37:13.940534+0000","last_active":"2023-11-16T16:37:15.157535+0000","last_peered":"2023-11-16T16:37:15.157535+0000","last_clean":"2023-11-16T16:37:15.157535+0000","last_became_active":"2023-11-16T16:37:13.940174+0000","last_became_peered":"2023-11-16T16:37:13.940174+0000","last_unstale":"2023-11-16T16:37:15.157535+0000","last_undegraded":"2023-11-16T16:37:15.157535+0000","last_fullsized":"2023-11-16T16:37:15.157535+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:25:01.730261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157969+0000","last_change":"2023-11-16T16:37:13.937519+0000","last_active":"2023-11-16T16:37:15.157969+0000","last_peered":"2023-11-16T16:37:15.157969+0000","last_clean":"2023-11-16T16:37:15.157969+0000","last_became_active":"2023-11-16T16:37:13.937175+0000","last_became_peered":"2023-11-16T16:37:13.937175+0000","last_unstale":"2023-11-16T16:37:15.157969+0000","last_undegraded":"2023-11-16T16:37:15.157969+0000","last_fullsized":"2023-11-16T16:37:15.157969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T16:47:03.319128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158262+0000","last_change":"2023-11-16T16:37:13.951506+0000","last_active":"2023-11-16T16:37:15.158262+0000","last_peered":"2023-11-16T16:37:15.158262+0000","last_clean":"2023-11-16T16:37:15.158262+0000","last_became_active":"2023-11-16T16:37:13.951192+0000","last_became_peered":"2023-11-16T16:37:13.951192+0000","last_unstale":"2023-11-16T16:37:15.158262+0000","last_undegraded":"2023-11-16T16:37:15.158262+0000","last_fullsized":"2023-11-16T16:37:15.158262+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:18:48.061220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.923594+0000","last_change":"2023-11-16T16:37:13.935614+0000","last_active":"2023-11-16T16:37:14.923594+0000","last_peered":"2023-11-16T16:37:14.923594+0000","last_clean":"2023-11-16T16:37:14.923594+0000","last_became_active":"2023-11-16T16:37:13.935237+0000","last_became_peered":"2023-11-16T16:37:13.935237+0000","last_unstale":"2023-11-16T16:37:14.923594+0000","last_undegraded":"2023-11-16T16:37:14.923594+0000","last_fullsized":"2023-11-16T16:37:14.923594+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:02:10.333347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157636+0000","last_change":"2023-11-16T16:37:13.939387+0000","last_active":"2023-11-16T16:37:15.157636+0000","last_peered":"2023-11-16T16:37:15.157636+0000","last_clean":"2023-11-16T16:37:15.157636+0000","last_became_active":"2023-11-16T16:37:13.939082+0000","last_became_peered":"2023-11-16T16:37:13.939082+0000","last_unstale":"2023-11-16T16:37:15.157636+0000","last_undegraded":"2023-11-16T16:37:15.157636+0000","last_fullsized":"2023-11-16T16:37:15.157636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:42:24.707911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.156764+0000","last_change":"2023-11-16T16:37:13.936381+0000","last_active":"2023-11-16T16:37:15.156764+0000","last_peered":"2023-11-16T16:37:15.156764+0000","last_clean":"2023-11-16T16:37:15.156764+0000","last_became_active":"2023-11-16T16:37:13.936077+0000","last_became_peered":"2023-11-16T16:37:13.936077+0000","last_unstale":"2023-11-16T16:37:15.156764+0000","last_undegraded":"2023-11-16T16:37:15.156764+0000","last_fullsized":"2023-11-16T16:37:15.156764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:04:48.256913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157497+0000","last_change":"2023-11-16T16:37:13.934005+0000","last_active":"2023-11-16T16:37:15.157497+0000","last_peered":"2023-11-16T16:37:15.157497+0000","last_clean":"2023-11-16T16:37:15.157497+0000","last_became_active":"2023-11-16T16:37:13.933633+0000","last_became_peered":"2023-11-16T16:37:13.933633+0000","last_unstale":"2023-11-16T16:37:15.157497+0000","last_undegraded":"2023-11-16T16:37:15.157497+0000","last_fullsized":"2023-11-16T16:37:15.157497+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:45:12.620887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158013+0000","last_change":"2023-11-16T16:37:13.946495+0000","last_active":"2023-11-16T16:37:15.158013+0000","last_peered":"2023-11-16T16:37:15.158013+0000","last_clean":"2023-11-16T16:37:15.158013+0000","last_became_active":"2023-11-16T16:37:13.946148+0000","last_became_peered":"2023-11-16T16:37:13.946148+0000","last_unstale":"2023-11-16T16:37:15.158013+0000","last_undegraded":"2023-11-16T16:37:15.158013+0000","last_fullsized":"2023-11-16T16:37:15.158013+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:17:05.329636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":3542208,"data_compressed":211753,"data_compressed_allocated":270336,"data_compressed_original":3522560,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":22743,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":23,"num_read_kb":35,"num_write":126,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":299008,"data_stored":84655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416396,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5940,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":93739212,"statfs":{"total":95995035648,"available":95988953088,"internally_reserved":0,"allocated":897024,"data_stored":428869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]}]},{"osd":10,"up_from":72,"seq":309237645328,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6304,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":93738848,"statfs":{"total":95995035648,"available":95988580352,"internally_reserved":0,"allocated":1007616,"data_stored":1039488,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":1,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.34200000000000003}]},{"osd":2,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.72599999999999998}]},{"osd":3,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.56200000000000006}]},{"osd":4,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.5}]},{"osd":5,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.58399999999999996}]},{"osd":6,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.65100000000000002}]},{"osd":7,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.505}]},{"osd":8,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.71099999999999997}]},{"osd":9,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.53000000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]}]},{"osd":9,"up_from":67,"seq":287762808851,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6088,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93739064,"statfs":{"total":95995035648,"available":95988801536,"internally_reserved":0,"allocated":917504,"data_stored":429687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:36:50 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.29399999999999998}]},{"osd":1,"last update":"Thu Nov 16 16:36:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.56399999999999995}]},{"osd":2,"last update":"Thu Nov 16 16:36:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.66000000000000003}]},{"osd":3,"last update":"Thu Nov 16 16:36:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.36099999999999999}]},{"osd":4,"last update":"Thu Nov 16 16:36:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.502},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.48399999999999999}]},{"osd":5,"last update":"Thu Nov 16 16:36:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.627}]},{"osd":6,"last update":"Thu Nov 16 16:36:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.38200000000000001}]},{"osd":7,"last update":"Thu Nov 16 16:36:50 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.67200000000000004}]},{"osd":8,"last update":"Thu Nov 16 16:36:50 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.70999999999999996}]},{"osd":10,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.68100000000000005}]}]},{"osd":8,"up_from":59,"seq":253403070490,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6280,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738872,"statfs":{"total":95995035648,"available":95988604928,"internally_reserved":0,"allocated":917504,"data_stored":429607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:36:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.61099999999999999}]},{"osd":1,"last update":"Thu Nov 16 16:36:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.875},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.748}]},{"osd":2,"last update":"Thu Nov 16 16:36:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.56399999999999995}]},{"osd":3,"last update":"Thu Nov 16 16:36:27 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.66600000000000004}]},{"osd":4,"last update":"Thu Nov 16 16:36:27 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.92400000000000004}]},{"osd":5,"last update":"Thu Nov 16 16:36:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.68999999999999995}]},{"osd":6,"last update":"Thu Nov 16 16:36:27 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.65100000000000002}]},{"osd":7,"last update":"Thu Nov 16 16:36:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.83199999999999996}]},{"osd":9,"last update":"Thu Nov 16 16:36:50 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":1.004}]},{"osd":10,"last update":"Thu Nov 16 16:37:13 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":3.5720000000000001,"5min":3.5720000000000001,"15min":3.5720000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":3.5920000000000001,"5min":3.5920000000000001,"15min":3.5920000000000001},"last":0.96599999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]}]},{"osd":7,"up_from":52,"seq":223338299421,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6388,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738764,"statfs":{"total":95995035648,"available":95988494336,"internally_reserved":0,"allocated":962560,"data_stored":1020933,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:36:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55800000000000005}]},{"osd":1,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.41799999999999998,"15min":0.41099999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.434,"15min":0.42599999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.314}]},{"osd":2,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.35799999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.58099999999999996}]},{"osd":3,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57499999999999996,"15min":0.57099999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.625,"15min":0.621},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.40000000000000002}]},{"osd":4,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.52400000000000002,"15min":0.51400000000000001},"min":{"1min":0.34799999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.498,"15min":0.48199999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.33200000000000002}]},{"osd":5,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.65200000000000002,"15min":0.64700000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91300000000000003,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.63800000000000001,"15min":0.63200000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.53000000000000003}]},{"osd":6,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.70299999999999996,"15min":0.69499999999999995},"min":{"1min":0.52500000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.68600000000000005,"15min":0.67600000000000005},"min":{"1min":0.47299999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.54400000000000004}]},{"osd":8,"last update":"Thu Nov 16 16:36:25 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.505},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.49299999999999999}]},{"osd":9,"last update":"Thu Nov 16 16:36:49 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.55600000000000005}]},{"osd":10,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.62},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.67100000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]}]},{"osd":6,"up_from":47,"seq":201863462945,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6196,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":93738956,"statfs":{"total":95995035648,"available":95988690944,"internally_reserved":0,"allocated":897024,"data_stored":428869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.41599999999999998,"15min":0.41199999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.437,"15min":0.433},"min":{"1min":0.30199999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.36599999999999999}]},{"osd":1,"last update":"Thu Nov 16 16:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.41499999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.80300000000000005,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.41099999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79300000000000004,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.52200000000000002}]},{"osd":2,"last update":"Thu Nov 16 16:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.625,"15min":0.629},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.77300000000000002,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.624,"15min":0.621},"min":{"1min":0.46200000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.48299999999999998}]},{"osd":3,"last update":"Thu Nov 16 16:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.48599999999999999,"15min":0.47699999999999998},"min":{"1min":0.36399999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.53600000000000003,"15min":0.53400000000000003},"min":{"1min":0.35299999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.40699999999999997}]},{"osd":4,"last update":"Thu Nov 16 16:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.58799999999999997,"15min":0.58099999999999996},"min":{"1min":0.442,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.61399999999999999,"15min":0.60199999999999998},"min":{"1min":0.47599999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.86299999999999999,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.58399999999999996}]},{"osd":5,"last update":"Thu Nov 16 16:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.64200000000000002}]},{"osd":7,"last update":"Thu Nov 16 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.70499999999999996,"15min":0.70299999999999996},"min":{"1min":0.48999999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.878,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.71499999999999997,"15min":0.70899999999999996},"min":{"1min":0.55300000000000005,"5min":0.436,"15min":0.436},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.67000000000000004}]},{"osd":8,"last update":"Thu Nov 16 16:36:27 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.59899999999999998}]},{"osd":9,"last update":"Thu Nov 16 16:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.74199999999999999}]},{"osd":10,"last update":"Thu Nov 16 16:37:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.625}]}]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7200,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":6272,"kb_avail":93737952,"statfs":{"total":95995035648,"available":95987662848,"internally_reserved":0,"allocated":942080,"data_stored":465106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6422528},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.48499999999999999,"15min":0.45800000000000002},"min":{"1min":0.33100000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.51200000000000001,"15min":0.48499999999999999},"min":{"1min":0.32000000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.47499999999999998}]},{"osd":2,"last update":"Thu Nov 16 16:37:19 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.54300000000000004,"15min":0.498},"min":{"1min":0.36699999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.84799999999999998,"5min":0.876,"15min":0.876},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.54000000000000004,"15min":0.503},"min":{"1min":0.40500000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.91000000000000003,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.59199999999999997}]},{"osd":3,"last update":"Thu Nov 16 16:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.57399999999999995,"15min":0.55400000000000005},"min":{"1min":0.45900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.59199999999999997,"15min":0.57299999999999995},"min":{"1min":0.42499999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.57699999999999996}]},{"osd":4,"last update":"Thu Nov 16 16:37:03 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.441,"15min":0.42299999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.374},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.52200000000000002,"15min":0.47599999999999998},"min":{"1min":0.34599999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.53000000000000003}]},{"osd":5,"last update":"Thu Nov 16 16:36:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.61399999999999999,"15min":0.60299999999999998},"min":{"1min":0.49099999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.59699999999999998,"15min":0.58899999999999997},"min":{"1min":0.443,"5min":0.437,"15min":0.437},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.72999999999999998}]},{"osd":6,"last update":"Thu Nov 16 16:36:47 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.68600000000000005,"15min":0.66900000000000004},"min":{"1min":0.48699999999999999,"5min":0.376,"15min":0.376},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.67900000000000005,"15min":0.66200000000000003},"min":{"1min":0.47499999999999998,"5min":0.379,"15min":0.379},"max":{"1min":1,"5min":1,"15min":1},"last":0.54800000000000004}]},{"osd":7,"last update":"Thu Nov 16 16:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.64500000000000002,"15min":0.63200000000000001},"min":{"1min":0.35199999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.69299999999999995,"15min":0.67700000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.68799999999999994}]},{"osd":8,"last update":"Thu Nov 16 16:36:27 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.78200000000000003}]},{"osd":9,"last update":"Thu Nov 16 16:36:53 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.76100000000000001}]},{"osd":10,"last update":"Thu Nov 16 16:37:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.65200000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738429,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7184,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":6272,"kb_avail":93737968,"statfs":{"total":95995035648,"available":95987679232,"internally_reserved":0,"allocated":925696,"data_stored":430347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6422528},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Nov 16 16:37:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.503,"15min":0.46700000000000003},"min":{"1min":0.32500000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.504},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.499,"15min":0.46100000000000002},"min":{"1min":0.36099999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78800000000000003,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.47399999999999998}]},{"osd":2,"last update":"Thu Nov 16 16:36:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.53400000000000003,"15min":0.50600000000000001},"min":{"1min":0.443,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.54600000000000004,"15min":0.52600000000000002},"min":{"1min":0.502,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.72599999999999998}]},{"osd":3,"last update":"Thu Nov 16 16:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.46500000000000002,"15min":0.45600000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.45700000000000002,"15min":0.44700000000000001},"min":{"1min":0.33700000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.40000000000000002}]},{"osd":4,"last update":"Thu Nov 16 16:37:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.57899999999999996,"15min":0.55100000000000005},"min":{"1min":0.38900000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.58199999999999996,"15min":0.55000000000000004},"min":{"1min":0.40999999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.51800000000000002}]},{"osd":5,"last update":"Thu Nov 16 16:36:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.56399999999999995,"15min":0.55200000000000005},"min":{"1min":0.45200000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.63900000000000001,"15min":0.61899999999999999},"min":{"1min":0.51200000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.79100000000000004}]},{"osd":6,"last update":"Thu Nov 16 16:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.60599999999999998,"15min":0.58299999999999996},"min":{"1min":0.35999999999999999,"5min":0.316,"15min":0.316},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.56899999999999995,"15min":0.55500000000000005},"min":{"1min":0.38800000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.438}]},{"osd":7,"last update":"Thu Nov 16 16:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.68400000000000005,"15min":0.67600000000000005},"min":{"1min":0.495,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.74199999999999999,"15min":0.73199999999999998},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.53400000000000003}]},{"osd":8,"last update":"Thu Nov 16 16:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.77000000000000002}]},{"osd":9,"last update":"Thu Nov 16 16:36:51 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.48799999999999999}]},{"osd":10,"last update":"Thu Nov 16 16:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.55400000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]}]},{"osd":2,"up_from":18,"seq":77309411380,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7192,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":6272,"kb_avail":93737960,"statfs":{"total":95995035648,"available":95987671040,"internally_reserved":0,"allocated":933888,"data_stored":432125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6422528},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:36:21 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.51300000000000001,"15min":0.51900000000000002},"min":{"1min":0.311,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.65400000000000003,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.495,"5min":0.48699999999999999,"15min":0.48499999999999999},"min":{"1min":0.33200000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.65800000000000003,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.49399999999999999}]},{"osd":1,"last update":"Thu Nov 16 16:36:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51500000000000001,"15min":0.51000000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55300000000000005,"15min":0.54900000000000004},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.443}]},{"osd":3,"last update":"Thu Nov 16 16:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.44700000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56200000000000006,"15min":0.55200000000000005},"min":{"1min":0.47299999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.60499999999999998}]},{"osd":4,"last update":"Thu Nov 16 16:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.55300000000000005,"15min":0.53000000000000003},"min":{"1min":0.376,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59099999999999997,"15min":0.58199999999999996},"min":{"1min":0.40000000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83799999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.62}]},{"osd":5,"last update":"Thu Nov 16 16:36:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.32300000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.46000000000000002,"15min":0.45500000000000002},"min":{"1min":0.307,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.66300000000000003}]},{"osd":6,"last update":"Thu Nov 16 16:36:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64500000000000002,"15min":0.64300000000000002},"min":{"1min":0.47099999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.60699999999999998,"15min":0.60099999999999998},"min":{"1min":0.35699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83899999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.71799999999999997}]},{"osd":7,"last update":"Thu Nov 16 16:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.64300000000000002,"15min":0.64400000000000002},"min":{"1min":0.41599999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70199999999999996,"15min":0.70399999999999996},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.92700000000000005,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.57699999999999996}]},{"osd":8,"last update":"Thu Nov 16 16:36:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.70599999999999996}]},{"osd":9,"last update":"Thu Nov 16 16:36:51 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":4.524,"5min":4.524,"15min":4.524},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.81699999999999995}]},{"osd":10,"last update":"Thu Nov 16 16:37:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.80100000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]}]},{"osd":3,"up_from":26,"seq":111669149741,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6520,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738632,"statfs":{"total":95995035648,"available":95988359168,"internally_reserved":0,"allocated":901120,"data_stored":428642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.35899999999999999,"15min":0.34699999999999998},"min":{"1min":0.28999999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.378,"15min":0.35699999999999998},"min":{"1min":0.253,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.28499999999999998}]},{"osd":1,"last update":"Thu Nov 16 16:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.55800000000000005,"15min":0.55000000000000004},"min":{"1min":0.374,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.437},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55900000000000005,"15min":0.55700000000000005},"min":{"1min":0.33800000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.55200000000000005}]},{"osd":2,"last update":"Thu Nov 16 16:36:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.57499999999999996,"15min":0.55500000000000005},"min":{"1min":0.38700000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.56499999999999995,"15min":0.54700000000000004},"min":{"1min":0.48299999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.40600000000000003}]},{"osd":4,"last update":"Thu Nov 16 16:37:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55700000000000005,"15min":0.54700000000000004},"min":{"1min":0.376,"5min":0.37,"15min":0.37},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.45500000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.47399999999999998}]},{"osd":5,"last update":"Thu Nov 16 16:36:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.59199999999999997,"15min":0.57799999999999996},"min":{"1min":0.39500000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.60499999999999998,"15min":0.59099999999999997},"min":{"1min":0.44600000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.46300000000000002}]},{"osd":6,"last update":"Thu Nov 16 16:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.51600000000000001,"15min":0.497},"min":{"1min":0.35399999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.53000000000000003,"15min":0.51300000000000001},"min":{"1min":0.35799999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.377}]},{"osd":7,"last update":"Thu Nov 16 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.65600000000000003,"15min":0.65000000000000002},"min":{"1min":0.44,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.69299999999999995,"15min":0.68700000000000006},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.42699999999999999}]},{"osd":8,"last update":"Thu Nov 16 16:36:27 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.63200000000000001}]},{"osd":9,"last update":"Thu Nov 16 16:36:48 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.41799999999999998}]},{"osd":10,"last update":"Thu Nov 16 16:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.54000000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]}]},{"osd":4,"up_from":34,"seq":146028888106,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6336,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738816,"statfs":{"total":95995035648,"available":95988547584,"internally_reserved":0,"allocated":909312,"data_stored":429465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.52400000000000002,"15min":0.502},"min":{"1min":0.38200000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.52900000000000003,"15min":0.51500000000000001},"min":{"1min":0.40899999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.378}]},{"osd":1,"last update":"Thu Nov 16 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.39400000000000002,"15min":0.38700000000000001},"min":{"1min":0.27900000000000003,"5min":0.253,"15min":0.253},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.41199999999999998,"15min":0.40300000000000002},"min":{"1min":0.29699999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.378}]},{"osd":2,"last update":"Thu Nov 16 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.442,"5min":0.371,"15min":0.371},"max":{"1min":0.79200000000000004,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.60499999999999998,"15min":0.58699999999999997},"min":{"1min":0.44600000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.90200000000000002,"5min":1.056,"15min":1.056},"last":0.53000000000000003}]},{"osd":3,"last update":"Thu Nov 16 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.56299999999999994,"15min":0.54000000000000004},"min":{"1min":0.39000000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.56100000000000005,"15min":0.54900000000000004},"min":{"1min":0.40500000000000003,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.40100000000000002}]},{"osd":5,"last update":"Thu Nov 16 16:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.622,"15min":0.627},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.70399999999999996,"5min":1.075,"15min":1.075},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.621,"15min":0.62},"min":{"1min":0.51500000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.92900000000000005,"5min":1.093,"15min":1.093},"last":0.69199999999999995}]},{"osd":6,"last update":"Thu Nov 16 16:36:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.60799999999999998,"15min":0.59499999999999997},"min":{"1min":0.434,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.41899999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.46600000000000003}]},{"osd":7,"last update":"Thu Nov 16 16:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.57199999999999995,"15min":0.54800000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.63200000000000001,"15min":0.60799999999999998},"min":{"1min":0.41199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.51600000000000001}]},{"osd":8,"last update":"Thu Nov 16 16:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.70799999999999996}]},{"osd":9,"last update":"Thu Nov 16 16:36:51 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.501}]},{"osd":10,"last update":"Thu Nov 16 16:37:15 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.629}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]}]},{"osd":5,"up_from":39,"seq":167503724583,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6528,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738624,"statfs":{"total":95995035648,"available":95988350976,"internally_reserved":0,"allocated":974848,"data_stored":1020665,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:36:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.50900000000000001,"15min":0.497},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.53000000000000003,"15min":0.52500000000000002},"min":{"1min":0.40699999999999997,"5min":0.311,"15min":0.311},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.312}]},{"osd":1,"last update":"Thu Nov 16 16:36:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58899999999999997,"15min":0.58599999999999997},"min":{"1min":0.40200000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81899999999999995,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.41699999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85599999999999998,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.52300000000000002}]},{"osd":2,"last update":"Thu Nov 16 16:36:21 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.42499999999999999,"15min":0.42299999999999999},"min":{"1min":0.33000000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.63200000000000001,"5min":1.417,"15min":1.417},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.439,"15min":0.432},"min":{"1min":0.35499999999999998,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.628,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.34899999999999998}]},{"osd":3,"last update":"Thu Nov 16 16:36:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.46400000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.377},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.60899999999999999,"15min":0.59999999999999998},"min":{"1min":0.45100000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.84299999999999997,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.43099999999999999}]},{"osd":4,"last update":"Thu Nov 16 16:36:21 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.60899999999999999,"15min":0.60599999999999998},"min":{"1min":0.51900000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82999999999999996,"5min":1.514,"15min":1.514},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.442,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.40400000000000003}]},{"osd":6,"last update":"Thu Nov 16 16:36:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.627,"15min":0.61899999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.91400000000000003,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.502},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.63600000000000001,"15min":0.63100000000000001},"min":{"1min":0.46400000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85599999999999998,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.41399999999999998}]},{"osd":7,"last update":"Thu Nov 16 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.70999999999999996,"15min":0.71199999999999997},"min":{"1min":0.503,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.83799999999999997,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.72499999999999998,"15min":0.72799999999999998},"min":{"1min":0.56599999999999995,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.83699999999999997,"5min":1.544,"15min":1.544},"last":0.48099999999999998}]},{"osd":8,"last update":"Thu Nov 16 16:36:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.58499999999999996}]},{"osd":9,"last update":"Thu Nov 16 16:36:51 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.56200000000000006}]},{"osd":10,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.57899999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33928,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":36237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":20621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":20459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-16T16:37:22.472 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-16T16:37:22.473 INFO:tasks.cephadm:Setting up client nodes... 2023-11-16T16:37:22.473 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-11-16T16:37:22.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:22 smithi053 ceph-mon[100638]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:37:22.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:22 smithi053 ceph-mon[100638]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:37:22.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:22 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/2923912856' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:37:22.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:22 smithi053 ceph-mon[100638]: pgmap v264: 97 pgs: 97 active+clean; 599 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7.1 KiB/s wr, 18 op/s 2023-11-16T16:37:22.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:22 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/3902182019' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:37:22.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:22 smithi026 ceph-mon[100112]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:37:22.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:22 smithi026 ceph-mon[100112]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:37:22.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:22 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/2923912856' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:37:22.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:22 smithi026 ceph-mon[100112]: pgmap v264: 97 pgs: 97 active+clean; 599 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7.1 KiB/s wr, 18 op/s 2023-11-16T16:37:22.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:22 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/3902182019' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:37:23.153 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:37:23.556 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:23 smithi001 ceph-mon[98800]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:37:23.556 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:23 smithi001 ceph-mon[98800]: mgrmap e17: x(active, since 6m), standbys: y, z 2023-11-16T16:37:23.556 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:23 smithi001 ceph-mon[98800]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:37:23.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:23 smithi053 ceph-mon[100638]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:37:23.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:23 smithi053 ceph-mon[100638]: mgrmap e17: x(active, since 6m), standbys: y, z 2023-11-16T16:37:23.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:23 smithi053 ceph-mon[100638]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:37:23.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:23 smithi026 ceph-mon[100112]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:37:23.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:23 smithi026 ceph-mon[100112]: mgrmap e17: x(active, since 6m), standbys: y, z 2023-11-16T16:37:23.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:23 smithi026 ceph-mon[100112]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:37:24.509 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.506+0000 7f2c145d9700 1 -- 172.21.15.1:0/765529443 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c0c101e80 msgr2=0x7f2c0c108740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:24.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.506+0000 7f2c145d9700 1 --2- 172.21.15.1:0/765529443 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c0c101e80 0x7f2c0c108740 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2c0400b0a0 tx=0x7f2c0400b3b0 comp rx=0 tx=0).stop 2023-11-16T16:37:24.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.506+0000 7f2c145d9700 1 -- 172.21.15.1:0/765529443 shutdown_connections 2023-11-16T16:37:24.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.506+0000 7f2c145d9700 1 --2- 172.21.15.1:0/765529443 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c0c101e80 0x7f2c0c108740 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:24.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.506+0000 7f2c145d9700 1 --2- 172.21.15.1:0/765529443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c0c1002f0 0x7f2c0c100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:24.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.506+0000 7f2c145d9700 1 --2- 172.21.15.1:0/765529443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c0c1014f0 0x7f2c0c101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:24.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.506+0000 7f2c145d9700 1 -- 172.21.15.1:0/765529443 >> 172.21.15.1:0/765529443 conn(0x7f2c0c0fb900 msgr2=0x7f2c0c0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:24.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.506+0000 7f2c145d9700 1 -- 172.21.15.1:0/765529443 shutdown_connections 2023-11-16T16:37:24.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.506+0000 7f2c145d9700 1 -- 172.21.15.1:0/765529443 wait complete. 2023-11-16T16:37:24.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.506+0000 7f2c145d9700 1 Processor -- start 2023-11-16T16:37:24.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.506+0000 7f2c145d9700 1 -- start start 2023-11-16T16:37:24.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.507+0000 7f2c145d9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c0c1002f0 0x7f2c0c074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:24.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.507+0000 7f2c145d9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c0c1014f0 0x7f2c0c072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:24.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.507+0000 7f2c145d9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c0c101e80 0x7f2c0c073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:24.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.507+0000 7f2c145d9700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c073ba0 con 0x7f2c0c101e80 2023-11-16T16:37:24.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.507+0000 7f2c145d9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c073d10 con 0x7f2c0c1002f0 2023-11-16T16:37:24.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.507+0000 7f2c145d9700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c1a00f0 con 0x7f2c0c1014f0 2023-11-16T16:37:24.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.507+0000 7f2c12b76700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c0c101e80 0x7f2c0c073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:24.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.507+0000 7f2c12b76700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c0c101e80 0x7f2c0c073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41658/0 (socket says 172.21.15.1:41658) 2023-11-16T16:37:24.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.507+0000 7f2c12b76700 1 -- 172.21.15.1:0/1968359215 learned_addr learned my addr 172.21.15.1:0/1968359215 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:24.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.507+0000 7f2c12375700 1 --2- 172.21.15.1:0/1968359215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c0c1002f0 0x7f2c0c074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:24.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.507+0000 7f2c11b74700 1 --2- 172.21.15.1:0/1968359215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c0c1014f0 0x7f2c0c072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:24.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.507+0000 7f2c12b76700 1 -- 172.21.15.1:0/1968359215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c0c1014f0 msgr2=0x7f2c0c072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:24.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.507+0000 7f2c12b76700 1 --2- 172.21.15.1:0/1968359215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c0c1014f0 0x7f2c0c072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:24.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.507+0000 7f2c12b76700 1 -- 172.21.15.1:0/1968359215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c0c1002f0 msgr2=0x7f2c0c074890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:24.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.507+0000 7f2c12b76700 1 --2- 172.21.15.1:0/1968359215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c0c1002f0 0x7f2c0c074890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:24.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.507+0000 7f2c12b76700 1 -- 172.21.15.1:0/1968359215 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c04009d00 con 0x7f2c0c101e80 2023-11-16T16:37:24.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.507+0000 7f2c11b74700 1 --2- 172.21.15.1:0/1968359215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c0c1014f0 0x7f2c0c072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:37:24.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.507+0000 7f2c12b76700 1 --2- 172.21.15.1:0/1968359215 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c0c101e80 0x7f2c0c073420 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2c0400b070 tx=0x7f2c04004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:24.513 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.507+0000 7f2c037fe700 1 -- 172.21.15.1:0/1968359215 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0400e040 con 0x7f2c0c101e80 2023-11-16T16:37:24.513 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.507+0000 7f2c145d9700 1 -- 172.21.15.1:0/1968359215 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c0c10bc50 con 0x7f2c0c101e80 2023-11-16T16:37:24.513 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.507+0000 7f2c037fe700 1 -- 172.21.15.1:0/1968359215 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c0401bb20 con 0x7f2c0c101e80 2023-11-16T16:37:24.513 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.507+0000 7f2c037fe700 1 -- 172.21.15.1:0/1968359215 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c04005800 con 0x7f2c0c101e80 2023-11-16T16:37:24.513 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.508+0000 7f2c145d9700 1 -- 172.21.15.1:0/1968359215 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c0c10bac0 con 0x7f2c0c101e80 2023-11-16T16:37:24.514 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.508+0000 7f2c145d9700 1 -- 172.21.15.1:0/1968359215 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c0c066ea0 con 0x7f2c0c101e80 2023-11-16T16:37:24.514 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.509+0000 7f2c037fe700 1 -- 172.21.15.1:0/1968359215 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f2c0401bc90 con 0x7f2c0c101e80 2023-11-16T16:37:24.514 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.510+0000 7f2c037fe700 1 --2- 172.21.15.1:0/1968359215 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f2bf80a1260 0x7f2bf80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:24.514 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.510+0000 7f2c12375700 1 --2- 172.21.15.1:0/1968359215 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f2bf80a1260 0x7f2bf80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:24.514 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.510+0000 7f2c037fe700 1 -- 172.21.15.1:0/1968359215 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2c040ce730 con 0x7f2c0c101e80 2023-11-16T16:37:24.514 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.510+0000 7f2c12375700 1 --2- 172.21.15.1:0/1968359215 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f2bf80a1260 0x7f2bf80a3710 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c101350 tx=0x7f2c0800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:24.516 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.515+0000 7f2c037fe700 1 -- 172.21.15.1:0/1968359215 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2c04097f60 con 0x7f2c0c101e80 2023-11-16T16:37:24.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:24 smithi053 ceph-mon[100638]: pgmap v265: 97 pgs: 97 active+clean; 599 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6.1 KiB/s wr, 16 op/s 2023-11-16T16:37:24.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:24 smithi001 ceph-mon[98800]: pgmap v265: 97 pgs: 97 active+clean; 599 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6.1 KiB/s wr, 16 op/s 2023-11-16T16:37:24.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:24 smithi026 ceph-mon[100112]: pgmap v265: 97 pgs: 97 active+clean; 599 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6.1 KiB/s wr, 16 op/s 2023-11-16T16:37:24.793 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.791+0000 7f2c145d9700 1 -- 172.21.15.1:0/1968359215 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f2c0c107150 con 0x7f2c0c101e80 2023-11-16T16:37:24.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.795+0000 7f2c037fe700 1 -- 172.21.15.1:0/1968359215 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7f2c0409b580 con 0x7f2c0c101e80 2023-11-16T16:37:24.797 INFO:teuthology.orchestra.run.smithi001.stdout:[client.0] 2023-11-16T16:37:24.797 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQBERVZlKPhELxAAlaXb5qojwyBSa6uQ/of5BA== 2023-11-16T16:37:24.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.797+0000 7f2c145d9700 1 -- 172.21.15.1:0/1968359215 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f2bf80a1260 msgr2=0x7f2bf80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:24.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.797+0000 7f2c145d9700 1 --2- 172.21.15.1:0/1968359215 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f2bf80a1260 0x7f2bf80a3710 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c101350 tx=0x7f2c0800a380 comp rx=0 tx=0).stop 2023-11-16T16:37:24.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.797+0000 7f2c145d9700 1 -- 172.21.15.1:0/1968359215 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c0c101e80 msgr2=0x7f2c0c073420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:24.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.797+0000 7f2c145d9700 1 --2- 172.21.15.1:0/1968359215 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c0c101e80 0x7f2c0c073420 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2c0400b070 tx=0x7f2c04004d00 comp rx=0 tx=0).stop 2023-11-16T16:37:24.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.797+0000 7f2c145d9700 1 -- 172.21.15.1:0/1968359215 shutdown_connections 2023-11-16T16:37:24.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.797+0000 7f2c145d9700 1 --2- 172.21.15.1:0/1968359215 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f2bf80a1260 0x7f2bf80a3710 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:24.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.797+0000 7f2c145d9700 1 --2- 172.21.15.1:0/1968359215 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c0c101e80 0x7f2c0c073420 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:24.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.797+0000 7f2c145d9700 1 --2- 172.21.15.1:0/1968359215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c0c1002f0 0x7f2c0c074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:24.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.797+0000 7f2c145d9700 1 --2- 172.21.15.1:0/1968359215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c0c1014f0 0x7f2c0c072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:24.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.797+0000 7f2c145d9700 1 -- 172.21.15.1:0/1968359215 >> 172.21.15.1:0/1968359215 conn(0x7f2c0c0fb900 msgr2=0x7f2c0c0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:24.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.798+0000 7f2c145d9700 1 -- 172.21.15.1:0/1968359215 shutdown_connections 2023-11-16T16:37:24.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:24.798+0000 7f2c145d9700 1 -- 172.21.15.1:0/1968359215 wait complete. 2023-11-16T16:37:25.214 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2023-11-16T16:37:25.214 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-11-16T16:37:25.214 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-16T16:37:25.256 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-16T16:37:25.256 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-11-16T16:37:25.256 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph mgr dump --format=json 2023-11-16T16:37:25.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:25 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1968359215' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-11-16T16:37:25.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:25 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1968359215' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-11-16T16:37:25.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:25 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1968359215' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-11-16T16:37:25.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:25 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1968359215' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-11-16T16:37:25.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:25 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1968359215' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-11-16T16:37:25.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:25 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1968359215' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-11-16T16:37:25.949 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:37:26.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:26 smithi001 ceph-mon[98800]: pgmap v266: 97 pgs: 97 active+clean; 620 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 7.6 KiB/s wr, 18 op/s 2023-11-16T16:37:26.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:26 smithi053 ceph-mon[100638]: pgmap v266: 97 pgs: 97 active+clean; 620 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 7.6 KiB/s wr, 18 op/s 2023-11-16T16:37:26.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:26 smithi026 ceph-mon[100112]: pgmap v266: 97 pgs: 97 active+clean; 620 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 7.6 KiB/s wr, 18 op/s 2023-11-16T16:37:27.147 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.145+0000 7f11c8d29700 1 -- 172.21.15.1:0/1343749334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11c40ff460 msgr2=0x7f11c40ff830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:27.147 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.145+0000 7f11c8d29700 1 --2- 172.21.15.1:0/1343749334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11c40ff460 0x7f11c40ff830 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f11b40099c0 tx=0x7f11b4009cd0 comp rx=0 tx=0).stop 2023-11-16T16:37:27.147 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.145+0000 7f11c8d29700 1 -- 172.21.15.1:0/1343749334 shutdown_connections 2023-11-16T16:37:27.147 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.145+0000 7f11c8d29700 1 --2- 172.21.15.1:0/1343749334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11c4104500 0x7f11c410c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:27.147 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.145+0000 7f11c8d29700 1 --2- 172.21.15.1:0/1343749334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11c40ff460 0x7f11c40ff830 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:27.147 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.145+0000 7f11c8d29700 1 --2- 172.21.15.1:0/1343749334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11c40ffe00 0x7f11c4103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:27.147 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.145+0000 7f11c8d29700 1 -- 172.21.15.1:0/1343749334 >> 172.21.15.1:0/1343749334 conn(0x7f11c40770a0 msgr2=0x7f11c40774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:27.147 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.145+0000 7f11c8d29700 1 -- 172.21.15.1:0/1343749334 shutdown_connections 2023-11-16T16:37:27.148 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.145+0000 7f11c8d29700 1 -- 172.21.15.1:0/1343749334 wait complete. 2023-11-16T16:37:27.150 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.146+0000 7f11c8d29700 1 Processor -- start 2023-11-16T16:37:27.150 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.146+0000 7f11c8d29700 1 -- start start 2023-11-16T16:37:27.150 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.146+0000 7f11c8d29700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11c40ff460 0x7f11c4195360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:27.150 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.146+0000 7f11c8d29700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11c40ffe00 0x7f11c41958a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:27.150 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.146+0000 7f11c8d29700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11c4104500 0x7f11c4199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:27.150 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.146+0000 7f11c8d29700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c419a320 con 0x7f11c40ff460 2023-11-16T16:37:27.150 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.146+0000 7f11c8d29700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c419a490 con 0x7f11c40ffe00 2023-11-16T16:37:27.150 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.146+0000 7f11c8d29700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c419a650 con 0x7f11c4104500 2023-11-16T16:37:27.150 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.146+0000 7f11c259c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11c40ff460 0x7f11c4195360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:27.150 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.146+0000 7f11c259c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11c40ff460 0x7f11c4195360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41668/0 (socket says 172.21.15.1:41668) 2023-11-16T16:37:27.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.146+0000 7f11c259c700 1 -- 172.21.15.1:0/2003171438 learned_addr learned my addr 172.21.15.1:0/2003171438 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:27.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.146+0000 7f11c1d9b700 1 --2- 172.21.15.1:0/2003171438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11c40ffe00 0x7f11c41958a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:27.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.146+0000 7f11c2d9d700 1 --2- 172.21.15.1:0/2003171438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11c4104500 0x7f11c4199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:27.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.146+0000 7f11c259c700 1 -- 172.21.15.1:0/2003171438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11c4104500 msgr2=0x7f11c4199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:27.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.146+0000 7f11c259c700 1 --2- 172.21.15.1:0/2003171438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11c4104500 0x7f11c4199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:27.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.146+0000 7f11c259c700 1 -- 172.21.15.1:0/2003171438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11c40ffe00 msgr2=0x7f11c41958a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:27.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.146+0000 7f11c259c700 1 --2- 172.21.15.1:0/2003171438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11c40ffe00 0x7f11c41958a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:27.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.146+0000 7f11c259c700 1 -- 172.21.15.1:0/2003171438 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b40095f0 con 0x7f11c40ff460 2023-11-16T16:37:27.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.147+0000 7f11c259c700 1 --2- 172.21.15.1:0/2003171438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11c40ff460 0x7f11c4195360 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f11b4009990 tx=0x7f11b4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:27.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.147+0000 7f11b37fe700 1 -- 172.21.15.1:0/2003171438 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b401d070 con 0x7f11c40ff460 2023-11-16T16:37:27.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.147+0000 7f11b37fe700 1 -- 172.21.15.1:0/2003171438 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11b4003f90 con 0x7f11c40ff460 2023-11-16T16:37:27.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.147+0000 7f11c8d29700 1 -- 172.21.15.1:0/2003171438 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c410f2b0 con 0x7f11c40ff460 2023-11-16T16:37:27.152 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.147+0000 7f11b37fe700 1 -- 172.21.15.1:0/2003171438 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b4022ce0 con 0x7f11c40ff460 2023-11-16T16:37:27.152 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.147+0000 7f11c8d29700 1 -- 172.21.15.1:0/2003171438 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c410f120 con 0x7f11c40ff460 2023-11-16T16:37:27.152 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.148+0000 7f11c8d29700 1 -- 172.21.15.1:0/2003171438 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c410f440 con 0x7f11c40ff460 2023-11-16T16:37:27.152 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.149+0000 7f11b37fe700 1 -- 172.21.15.1:0/2003171438 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f11b4022470 con 0x7f11c40ff460 2023-11-16T16:37:27.152 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.150+0000 7f11b37fe700 1 --2- 172.21.15.1:0/2003171438 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f11ac0a1260 0x7f11ac0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:27.153 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.150+0000 7f11c1d9b700 1 --2- 172.21.15.1:0/2003171438 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f11ac0a1260 0x7f11ac0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:27.153 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.150+0000 7f11b37fe700 1 -- 172.21.15.1:0/2003171438 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f11b40cda70 con 0x7f11c40ff460 2023-11-16T16:37:27.153 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.150+0000 7f11c1d9b700 1 --2- 172.21.15.1:0/2003171438 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f11ac0a1260 0x7f11ac0a3710 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f11c4196980 tx=0x7f11b8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:27.156 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.154+0000 7f11b37fe700 1 -- 172.21.15.1:0/2003171438 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f11b40972a0 con 0x7f11c40ff460 2023-11-16T16:37:27.409 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.407+0000 7f11c8d29700 1 -- 172.21.15.1:0/2003171438 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f11c4066ea0 con 0x7f11c40ff460 2023-11-16T16:37:27.414 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.412+0000 7f11b37fe700 1 -- 172.21.15.1:0/2003171438 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260117 (secure 0 0 0) 0x7f11b409a8c0 con 0x7f11c40ff460 2023-11-16T16:37:27.417 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:37:27.419 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.415+0000 7f11c8d29700 1 -- 172.21.15.1:0/2003171438 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f11ac0a1260 msgr2=0x7f11ac0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:27.419 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.415+0000 7f11c8d29700 1 --2- 172.21.15.1:0/2003171438 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f11ac0a1260 0x7f11ac0a3710 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f11c4196980 tx=0x7f11b8009190 comp rx=0 tx=0).stop 2023-11-16T16:37:27.419 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.415+0000 7f11c8d29700 1 -- 172.21.15.1:0/2003171438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11c40ff460 msgr2=0x7f11c4195360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:27.419 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.415+0000 7f11c8d29700 1 --2- 172.21.15.1:0/2003171438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11c40ff460 0x7f11c4195360 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f11b4009990 tx=0x7f11b4009580 comp rx=0 tx=0).stop 2023-11-16T16:37:27.419 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.415+0000 7f11c8d29700 1 -- 172.21.15.1:0/2003171438 shutdown_connections 2023-11-16T16:37:27.419 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.415+0000 7f11c8d29700 1 --2- 172.21.15.1:0/2003171438 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f11ac0a1260 0x7f11ac0a3710 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:27.419 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.415+0000 7f11c8d29700 1 --2- 172.21.15.1:0/2003171438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f11c4104500 0x7f11c4199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:27.419 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.415+0000 7f11c8d29700 1 --2- 172.21.15.1:0/2003171438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11c40ff460 0x7f11c4195360 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:27.420 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.415+0000 7f11c8d29700 1 --2- 172.21.15.1:0/2003171438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11c40ffe00 0x7f11c41958a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:27.420 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.415+0000 7f11c8d29700 1 -- 172.21.15.1:0/2003171438 >> 172.21.15.1:0/2003171438 conn(0x7f11c40770a0 msgr2=0x7f11c4102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:27.420 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.415+0000 7f11c8d29700 1 -- 172.21.15.1:0/2003171438 shutdown_connections 2023-11-16T16:37:27.420 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:27.415+0000 7f11c8d29700 1 -- 172.21.15.1:0/2003171438 wait complete. 2023-11-16T16:37:27.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:27 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/2003171438' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-11-16T16:37:27.795 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:27 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/2003171438' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-11-16T16:37:27.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:27 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/2003171438' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-11-16T16:37:27.895 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":17,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":2490729488},{"type":"v1","addr":"172.21.15.1:6801","nonce":2490729488}]},"active_addr":"172.21.15.1:6801/2490729488","active_change":"2023-11-16T16:31:01.391306+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24107,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.1:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":2308375878}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":3270932790}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":1768923877}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":1210074029}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":3855931722}]}]} 2023-11-16T16:37:27.900 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-11-16T16:37:27.900 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-11-16T16:37:27.900 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph osd dump --format=json 2023-11-16T16:37:28.557 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:37:28.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:28 smithi026 ceph-mon[100112]: pgmap v267: 97 pgs: 97 active+clean; 620 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.7 KiB/s wr, 14 op/s 2023-11-16T16:37:28.765 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:28 smithi001 ceph-mon[98800]: pgmap v267: 97 pgs: 97 active+clean; 620 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.7 KiB/s wr, 14 op/s 2023-11-16T16:37:28.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:28 smithi053 ceph-mon[100638]: pgmap v267: 97 pgs: 97 active+clean; 620 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.7 KiB/s wr, 14 op/s 2023-11-16T16:37:29.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.794+0000 7f3952f0b700 1 -- 172.21.15.1:0/4248382207 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f394c0ff470 msgr2=0x7f394c0ff840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:29.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.794+0000 7f3952f0b700 1 --2- 172.21.15.1:0/4248382207 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f394c0ff470 0x7f394c0ff840 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f39400099c0 tx=0x7f3940009cd0 comp rx=0 tx=0).stop 2023-11-16T16:37:29.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.794+0000 7f3952f0b700 1 -- 172.21.15.1:0/4248382207 shutdown_connections 2023-11-16T16:37:29.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.794+0000 7f3952f0b700 1 --2- 172.21.15.1:0/4248382207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f394c104510 0x7f394c10c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:29.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.794+0000 7f3952f0b700 1 --2- 172.21.15.1:0/4248382207 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f394c0ff470 0x7f394c0ff840 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:29.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.794+0000 7f3952f0b700 1 --2- 172.21.15.1:0/4248382207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f394c0ffe10 0x7f394c103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:29.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.794+0000 7f3952f0b700 1 -- 172.21.15.1:0/4248382207 >> 172.21.15.1:0/4248382207 conn(0x7f394c0770d0 msgr2=0x7f394c0774d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:29.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.794+0000 7f3952f0b700 1 -- 172.21.15.1:0/4248382207 shutdown_connections 2023-11-16T16:37:29.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.794+0000 7f3952f0b700 1 -- 172.21.15.1:0/4248382207 wait complete. 2023-11-16T16:37:29.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.795+0000 7f3952f0b700 1 Processor -- start 2023-11-16T16:37:29.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.795+0000 7f3952f0b700 1 -- start start 2023-11-16T16:37:29.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.795+0000 7f3952f0b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f394c0ff470 0x7f394c1953a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:29.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.795+0000 7f3952f0b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f394c0ffe10 0x7f394c1958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:29.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.795+0000 7f3952f0b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f394c104510 0x7f394c199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:29.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.795+0000 7f3952f0b700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394c19a360 con 0x7f394c0ffe10 2023-11-16T16:37:29.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.795+0000 7f3952f0b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394c19a4d0 con 0x7f394c104510 2023-11-16T16:37:29.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.795+0000 7f3952f0b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394c19a690 con 0x7f394c0ff470 2023-11-16T16:37:29.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.795+0000 7f394bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f394c0ffe10 0x7f394c1958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:29.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.795+0000 7f394bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f394c0ffe10 0x7f394c1958e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41682/0 (socket says 172.21.15.1:41682) 2023-11-16T16:37:29.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.795+0000 7f394bfff700 1 -- 172.21.15.1:0/2707155438 learned_addr learned my addr 172.21.15.1:0/2707155438 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:29.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.795+0000 7f39514a8700 1 --2- 172.21.15.1:0/2707155438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f394c104510 0x7f394c199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:29.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.795+0000 7f3950ca7700 1 --2- 172.21.15.1:0/2707155438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f394c0ff470 0x7f394c1953a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:29.802 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.796+0000 7f394bfff700 1 -- 172.21.15.1:0/2707155438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f394c0ff470 msgr2=0x7f394c1953a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:29.802 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.796+0000 7f394bfff700 1 --2- 172.21.15.1:0/2707155438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f394c0ff470 0x7f394c1953a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:29.802 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.796+0000 7f394bfff700 1 -- 172.21.15.1:0/2707155438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f394c104510 msgr2=0x7f394c199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:29.802 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.796+0000 7f394bfff700 1 --2- 172.21.15.1:0/2707155438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f394c104510 0x7f394c199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:29.802 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.796+0000 7f394bfff700 1 -- 172.21.15.1:0/2707155438 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39400095f0 con 0x7f394c0ffe10 2023-11-16T16:37:29.802 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.796+0000 7f3950ca7700 1 --2- 172.21.15.1:0/2707155438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f394c0ff470 0x7f394c1953a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:37:29.802 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.796+0000 7f39514a8700 1 --2- 172.21.15.1:0/2707155438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f394c104510 0x7f394c199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:37:29.802 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.796+0000 7f394bfff700 1 --2- 172.21.15.1:0/2707155438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f394c0ffe10 0x7f394c1958e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f393800b500 tx=0x7f393800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:29.802 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.796+0000 7f3949ffb700 1 -- 172.21.15.1:0/2707155438 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938011830 con 0x7f394c0ffe10 2023-11-16T16:37:29.802 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.796+0000 7f3949ffb700 1 -- 172.21.15.1:0/2707155438 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3938011e50 con 0x7f394c0ffe10 2023-11-16T16:37:29.803 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.796+0000 7f3952f0b700 1 -- 172.21.15.1:0/2707155438 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f394c10f340 con 0x7f394c0ffe10 2023-11-16T16:37:29.803 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.796+0000 7f3949ffb700 1 -- 172.21.15.1:0/2707155438 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393800f2e0 con 0x7f394c0ffe10 2023-11-16T16:37:29.803 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.796+0000 7f3952f0b700 1 -- 172.21.15.1:0/2707155438 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f394c10f1b0 con 0x7f394c0ffe10 2023-11-16T16:37:29.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.797+0000 7f3952f0b700 1 -- 172.21.15.1:0/2707155438 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f394c10f4d0 con 0x7f394c0ffe10 2023-11-16T16:37:29.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.798+0000 7f3949ffb700 1 -- 172.21.15.1:0/2707155438 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f39380105a0 con 0x7f394c0ffe10 2023-11-16T16:37:29.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.799+0000 7f3949ffb700 1 --2- 172.21.15.1:0/2707155438 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f393c0a11e0 0x7f393c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:29.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.799+0000 7f3950ca7700 1 --2- 172.21.15.1:0/2707155438 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f393c0a11e0 0x7f393c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:29.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.799+0000 7f3949ffb700 1 -- 172.21.15.1:0/2707155438 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f39380cc2b0 con 0x7f394c0ffe10 2023-11-16T16:37:29.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.799+0000 7f3950ca7700 1 --2- 172.21.15.1:0/2707155438 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f393c0a11e0 0x7f393c0a3690 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3940000c00 tx=0x7f3940005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:29.805 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:29.803+0000 7f3949ffb700 1 -- 172.21.15.1:0/2707155438 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3938095b60 con 0x7f394c0ffe10 2023-11-16T16:37:29.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:29 smithi053 ceph-mon[100638]: pgmap v268: 97 pgs: 97 active+clean; 620 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.7 KiB/s wr, 14 op/s 2023-11-16T16:37:29.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:29 smithi001 ceph-mon[98800]: pgmap v268: 97 pgs: 97 active+clean; 620 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.7 KiB/s wr, 14 op/s 2023-11-16T16:37:30.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:30.005+0000 7f3952f0b700 1 -- 172.21.15.1:0/2707155438 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f394c1a24e0 con 0x7f394c0ffe10 2023-11-16T16:37:30.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:29 smithi026 ceph-mon[100112]: pgmap v268: 97 pgs: 97 active+clean; 620 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.7 KiB/s wr, 14 op/s 2023-11-16T16:37:30.008 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:30.007+0000 7f3949ffb700 1 -- 172.21.15.1:0/2707155438 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+21988 (secure 0 0 0) 0x7f3938099180 con 0x7f394c0ffe10 2023-11-16T16:37:30.009 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:37:30.009 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":84,"fsid":"35d9b010-849d-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:30:06.217778+0000","modified":"2023-11-16T16:37:14.920337+0000","last_up_change":"2023-11-16T16:36:31.735403+0000","last_in_change":"2023-11-16T16:36:14.083021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:33:15.916723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-16T16:37:12.158031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-16T16:37:13.272343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a8480507-5275-4e52-953b-626110beb436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6803","nonce":2469671609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6805","nonce":2469671609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6809","nonce":2469671609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6807","nonce":2469671609}]},"public_addr":"172.21.15.1:6803/2469671609","cluster_addr":"172.21.15.1:6805/2469671609","heartbeat_back_addr":"172.21.15.1:6809/2469671609","heartbeat_front_addr":"172.21.15.1:6807/2469671609","state":["exists","up"]},{"osd":1,"uuid":"1d011db2-99d1-4575-bdc5-51f88a308fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6801","nonce":247994796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6803","nonce":247994796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6807","nonce":247994796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6805","nonce":247994796}]},"public_addr":"172.21.15.26:6801/247994796","cluster_addr":"172.21.15.26:6803/247994796","heartbeat_back_addr":"172.21.15.26:6807/247994796","heartbeat_front_addr":"172.21.15.26:6805/247994796","state":["exists","up"]},{"osd":2,"uuid":"4d7a2bd1-3c8e-4546-a243-f0354473b031","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6801","nonce":1659316290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6803","nonce":1659316290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6807","nonce":1659316290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6805","nonce":1659316290}]},"public_addr":"172.21.15.53:6801/1659316290","cluster_addr":"172.21.15.53:6803/1659316290","heartbeat_back_addr":"172.21.15.53:6807/1659316290","heartbeat_front_addr":"172.21.15.53:6805/1659316290","state":["exists","up"]},{"osd":3,"uuid":"9c290fd9-e121-4a81-9687-b0eb93debd86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6811","nonce":2728987281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6813","nonce":2728987281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6817","nonce":2728987281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6815","nonce":2728987281}]},"public_addr":"172.21.15.1:6811/2728987281","cluster_addr":"172.21.15.1:6813/2728987281","heartbeat_back_addr":"172.21.15.1:6817/2728987281","heartbeat_front_addr":"172.21.15.1:6815/2728987281","state":["exists","up"]},{"osd":4,"uuid":"533226bc-cc57-4f15-9e29-820294d57e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6809","nonce":2954740024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6811","nonce":2954740024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6815","nonce":2954740024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6813","nonce":2954740024}]},"public_addr":"172.21.15.26:6809/2954740024","cluster_addr":"172.21.15.26:6811/2954740024","heartbeat_back_addr":"172.21.15.26:6815/2954740024","heartbeat_front_addr":"172.21.15.26:6813/2954740024","state":["exists","up"]},{"osd":5,"uuid":"d3ab25eb-e1d4-4557-bc89-efc9072401ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6809","nonce":3966666318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6811","nonce":3966666318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6815","nonce":3966666318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6813","nonce":3966666318}]},"public_addr":"172.21.15.53:6809/3966666318","cluster_addr":"172.21.15.53:6811/3966666318","heartbeat_back_addr":"172.21.15.53:6815/3966666318","heartbeat_front_addr":"172.21.15.53:6813/3966666318","state":["exists","up"]},{"osd":6,"uuid":"180f506e-afb8-4043-b0b4-2dc86f6d4bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6819","nonce":3364032646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6821","nonce":3364032646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6825","nonce":3364032646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6823","nonce":3364032646}]},"public_addr":"172.21.15.1:6819/3364032646","cluster_addr":"172.21.15.1:6821/3364032646","heartbeat_back_addr":"172.21.15.1:6825/3364032646","heartbeat_front_addr":"172.21.15.1:6823/3364032646","state":["exists","up"]},{"osd":7,"uuid":"0d837bac-194f-498d-b7fb-b3147e992329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6817","nonce":3766901391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6819","nonce":3766901391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6823","nonce":3766901391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6821","nonce":3766901391}]},"public_addr":"172.21.15.26:6817/3766901391","cluster_addr":"172.21.15.26:6819/3766901391","heartbeat_back_addr":"172.21.15.26:6823/3766901391","heartbeat_front_addr":"172.21.15.26:6821/3766901391","state":["exists","up"]},{"osd":8,"uuid":"933f231a-7454-47b0-9291-f1fa64f0baec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6817","nonce":3224335871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6819","nonce":3224335871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6823","nonce":3224335871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6821","nonce":3224335871}]},"public_addr":"172.21.15.53:6817/3224335871","cluster_addr":"172.21.15.53:6819/3224335871","heartbeat_back_addr":"172.21.15.53:6823/3224335871","heartbeat_front_addr":"172.21.15.53:6821/3224335871","state":["exists","up"]},{"osd":9,"uuid":"245fcbb3-c651-4703-ab80-309dc98bd983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6827","nonce":1391199844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6829","nonce":1391199844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6833","nonce":1391199844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6831","nonce":1391199844}]},"public_addr":"172.21.15.1:6827/1391199844","cluster_addr":"172.21.15.1:6829/1391199844","heartbeat_back_addr":"172.21.15.1:6833/1391199844","heartbeat_front_addr":"172.21.15.1:6831/1391199844","state":["exists","up"]},{"osd":10,"uuid":"2c090e7a-5240-4d52-ae4f-e94fe05383d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6825","nonce":1773545447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6827","nonce":1773545447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6831","nonce":1773545447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6829","nonce":1773545447}]},"public_addr":"172.21.15.26:6825/1773545447","cluster_addr":"172.21.15.26:6827/1773545447","heartbeat_back_addr":"172.21.15.26:6831/1773545447","heartbeat_front_addr":"172.21.15.26:6829/1773545447","state":["exists","up"]},{"osd":11,"uuid":"4511e32d-e469-45e1-8310-0ff400cb7b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6825","nonce":2564669546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6827","nonce":2564669546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6831","nonce":2564669546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6829","nonce":2564669546}]},"public_addr":"172.21.15.53:6825/2564669546","cluster_addr":"172.21.15.53:6827/2564669546","heartbeat_back_addr":"172.21.15.53:6831/2564669546","heartbeat_front_addr":"172.21.15.53:6829/2564669546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:28.140250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:49.885461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:12.521487+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:34.926197+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:56.815594+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:18.387751+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:40.587106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:02.319370+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:23.832298+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:46.338040+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:08.730414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:30.050070+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1494336948":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6801/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6800/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6800/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/1641976669":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/2895923638":"2023-11-17T16:30:35.064289+0000","172.21.15.1:6801/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/78811711":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/4131957708":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/344403698":"2023-11-17T16:30:35.064289+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:37:30.011 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:30.009+0000 7f3952f0b700 1 -- 172.21.15.1:0/2707155438 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f393c0a11e0 msgr2=0x7f393c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:30.011 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:30.009+0000 7f3952f0b700 1 --2- 172.21.15.1:0/2707155438 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f393c0a11e0 0x7f393c0a3690 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3940000c00 tx=0x7f3940005fb0 comp rx=0 tx=0).stop 2023-11-16T16:37:30.011 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:30.009+0000 7f3952f0b700 1 -- 172.21.15.1:0/2707155438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f394c0ffe10 msgr2=0x7f394c1958e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:30.011 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:30.009+0000 7f3952f0b700 1 --2- 172.21.15.1:0/2707155438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f394c0ffe10 0x7f394c1958e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f393800b500 tx=0x7f393800b8c0 comp rx=0 tx=0).stop 2023-11-16T16:37:30.011 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:30.009+0000 7f3952f0b700 1 -- 172.21.15.1:0/2707155438 shutdown_connections 2023-11-16T16:37:30.011 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:30.009+0000 7f3952f0b700 1 --2- 172.21.15.1:0/2707155438 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f393c0a11e0 0x7f393c0a3690 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:30.011 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:30.009+0000 7f3952f0b700 1 --2- 172.21.15.1:0/2707155438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f394c104510 0x7f394c199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:30.011 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:30.009+0000 7f3952f0b700 1 --2- 172.21.15.1:0/2707155438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f394c0ff470 0x7f394c1953a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:30.012 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:30.009+0000 7f3952f0b700 1 --2- 172.21.15.1:0/2707155438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f394c0ffe10 0x7f394c1958e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:30.012 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:30.009+0000 7f3952f0b700 1 -- 172.21.15.1:0/2707155438 >> 172.21.15.1:0/2707155438 conn(0x7f394c0770d0 msgr2=0x7f394c0fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:30.012 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:30.009+0000 7f3952f0b700 1 -- 172.21.15.1:0/2707155438 shutdown_connections 2023-11-16T16:37:30.012 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:30.009+0000 7f3952f0b700 1 -- 172.21.15.1:0/2707155438 wait complete. 2023-11-16T16:37:30.417 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-11-16T16:37:30.418 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph osd dump --format=json 2023-11-16T16:37:30.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:30 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/2707155438' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:37:30.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:30 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/2707155438' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:37:31.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:30 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/2707155438' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:37:31.072 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:37:32.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.377+0000 7f6e93148700 1 -- 172.21.15.1:0/732151986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e8c0ff460 msgr2=0x7f6e8c0ff830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:32.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.377+0000 7f6e93148700 1 --2- 172.21.15.1:0/732151986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e8c0ff460 0x7f6e8c0ff830 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6e800099c0 tx=0x7f6e80009cd0 comp rx=0 tx=0).stop 2023-11-16T16:37:32.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.377+0000 7f6e93148700 1 -- 172.21.15.1:0/732151986 shutdown_connections 2023-11-16T16:37:32.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.377+0000 7f6e93148700 1 --2- 172.21.15.1:0/732151986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e8c104500 0x7f6e8c10c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:32.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.377+0000 7f6e93148700 1 --2- 172.21.15.1:0/732151986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e8c0ff460 0x7f6e8c0ff830 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:32.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.377+0000 7f6e93148700 1 --2- 172.21.15.1:0/732151986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e8c0ffe00 0x7f6e8c103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:32.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.377+0000 7f6e93148700 1 -- 172.21.15.1:0/732151986 >> 172.21.15.1:0/732151986 conn(0x7f6e8c0770a0 msgr2=0x7f6e8c0774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:32.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.377+0000 7f6e93148700 1 -- 172.21.15.1:0/732151986 shutdown_connections 2023-11-16T16:37:32.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.377+0000 7f6e93148700 1 -- 172.21.15.1:0/732151986 wait complete. 2023-11-16T16:37:32.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.378+0000 7f6e93148700 1 Processor -- start 2023-11-16T16:37:32.384 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.378+0000 7f6e93148700 1 -- start start 2023-11-16T16:37:32.384 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.378+0000 7f6e93148700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e8c0ff460 0x7f6e8c19e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:32.384 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.378+0000 7f6e93148700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e8c0ffe00 0x7f6e8c19ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:32.384 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.378+0000 7f6e93148700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e8c104500 0x7f6e8c198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:32.384 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.378+0000 7f6e93148700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c198be0 con 0x7f6e8c0ff460 2023-11-16T16:37:32.384 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.378+0000 7f6e93148700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c198d50 con 0x7f6e8c0ffe00 2023-11-16T16:37:32.384 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.378+0000 7f6e93148700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c198f10 con 0x7f6e8c104500 2023-11-16T16:37:32.384 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.378+0000 7f6e90ee4700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e8c0ff460 0x7f6e8c19e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:32.385 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.378+0000 7f6e90ee4700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e8c0ff460 0x7f6e8c19e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41694/0 (socket says 172.21.15.1:41694) 2023-11-16T16:37:32.385 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.378+0000 7f6e90ee4700 1 -- 172.21.15.1:0/1596936545 learned_addr learned my addr 172.21.15.1:0/1596936545 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:32.385 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.379+0000 7f6e916e5700 1 --2- 172.21.15.1:0/1596936545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e8c104500 0x7f6e8c198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:32.385 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.379+0000 7f6e8bfff700 1 --2- 172.21.15.1:0/1596936545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e8c0ffe00 0x7f6e8c19ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:32.385 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.379+0000 7f6e90ee4700 1 -- 172.21.15.1:0/1596936545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e8c104500 msgr2=0x7f6e8c198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:32.385 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.379+0000 7f6e90ee4700 1 --2- 172.21.15.1:0/1596936545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e8c104500 0x7f6e8c198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:32.385 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.379+0000 7f6e90ee4700 1 -- 172.21.15.1:0/1596936545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e8c0ffe00 msgr2=0x7f6e8c19ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:32.385 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.379+0000 7f6e90ee4700 1 --2- 172.21.15.1:0/1596936545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e8c0ffe00 0x7f6e8c19ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:32.385 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.379+0000 7f6e90ee4700 1 -- 172.21.15.1:0/1596936545 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e800095f0 con 0x7f6e8c0ff460 2023-11-16T16:37:32.386 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.379+0000 7f6e916e5700 1 --2- 172.21.15.1:0/1596936545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e8c104500 0x7f6e8c198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:37:32.386 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.379+0000 7f6e90ee4700 1 --2- 172.21.15.1:0/1596936545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e8c0ff460 0x7f6e8c19e4f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6e80009990 tx=0x7f6e80009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:32.386 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.379+0000 7f6e89ffb700 1 -- 172.21.15.1:0/1596936545 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8001d070 con 0x7f6e8c0ff460 2023-11-16T16:37:32.386 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.379+0000 7f6e89ffb700 1 -- 172.21.15.1:0/1596936545 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e80004010 con 0x7f6e8c0ff460 2023-11-16T16:37:32.386 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.379+0000 7f6e89ffb700 1 -- 172.21.15.1:0/1596936545 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e80022d40 con 0x7f6e8c0ff460 2023-11-16T16:37:32.386 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.379+0000 7f6e93148700 1 -- 172.21.15.1:0/1596936545 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e8c10f2b0 con 0x7f6e8c0ff460 2023-11-16T16:37:32.386 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.379+0000 7f6e93148700 1 -- 172.21.15.1:0/1596936545 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e8c10f120 con 0x7f6e8c0ff460 2023-11-16T16:37:32.386 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.380+0000 7f6e93148700 1 -- 172.21.15.1:0/1596936545 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e8c10f440 con 0x7f6e8c0ff460 2023-11-16T16:37:32.387 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.382+0000 7f6e89ffb700 1 -- 172.21.15.1:0/1596936545 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f6e80022470 con 0x7f6e8c0ff460 2023-11-16T16:37:32.387 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.382+0000 7f6e89ffb700 1 --2- 172.21.15.1:0/1596936545 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f6e7c0a1260 0x7f6e7c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:32.388 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.382+0000 7f6e89ffb700 1 -- 172.21.15.1:0/1596936545 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6e800cdff0 con 0x7f6e8c0ff460 2023-11-16T16:37:32.388 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.382+0000 7f6e8bfff700 1 --2- 172.21.15.1:0/1596936545 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f6e7c0a1260 0x7f6e7c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:32.388 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.383+0000 7f6e8bfff700 1 --2- 172.21.15.1:0/1596936545 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f6e7c0a1260 0x7f6e7c0a3710 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6e78006fd0 tx=0x7f6e78008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:32.388 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.386+0000 7f6e89ffb700 1 -- 172.21.15.1:0/1596936545 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6e80097820 con 0x7f6e8c0ff460 2023-11-16T16:37:32.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:32 smithi001 ceph-mon[98800]: pgmap v269: 97 pgs: 97 active+clean; 633 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2023-11-16T16:37:32.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:32 smithi026 ceph-mon[100112]: pgmap v269: 97 pgs: 97 active+clean; 633 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2023-11-16T16:37:32.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:32 smithi053 ceph-mon[100638]: pgmap v269: 97 pgs: 97 active+clean; 633 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2023-11-16T16:37:32.585 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.583+0000 7f6e93148700 1 -- 172.21.15.1:0/1596936545 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e8c066ea0 con 0x7f6e8c0ff460 2023-11-16T16:37:32.586 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.584+0000 7f6e89ffb700 1 -- 172.21.15.1:0/1596936545 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+21988 (secure 0 0 0) 0x7f6e8009ae40 con 0x7f6e8c0ff460 2023-11-16T16:37:32.587 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:37:32.587 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":84,"fsid":"35d9b010-849d-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:30:06.217778+0000","modified":"2023-11-16T16:37:14.920337+0000","last_up_change":"2023-11-16T16:36:31.735403+0000","last_in_change":"2023-11-16T16:36:14.083021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:33:15.916723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-16T16:37:12.158031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-16T16:37:13.272343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a8480507-5275-4e52-953b-626110beb436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6803","nonce":2469671609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6805","nonce":2469671609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6809","nonce":2469671609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6807","nonce":2469671609}]},"public_addr":"172.21.15.1:6803/2469671609","cluster_addr":"172.21.15.1:6805/2469671609","heartbeat_back_addr":"172.21.15.1:6809/2469671609","heartbeat_front_addr":"172.21.15.1:6807/2469671609","state":["exists","up"]},{"osd":1,"uuid":"1d011db2-99d1-4575-bdc5-51f88a308fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6801","nonce":247994796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6803","nonce":247994796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6807","nonce":247994796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6805","nonce":247994796}]},"public_addr":"172.21.15.26:6801/247994796","cluster_addr":"172.21.15.26:6803/247994796","heartbeat_back_addr":"172.21.15.26:6807/247994796","heartbeat_front_addr":"172.21.15.26:6805/247994796","state":["exists","up"]},{"osd":2,"uuid":"4d7a2bd1-3c8e-4546-a243-f0354473b031","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6801","nonce":1659316290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6803","nonce":1659316290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6807","nonce":1659316290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6805","nonce":1659316290}]},"public_addr":"172.21.15.53:6801/1659316290","cluster_addr":"172.21.15.53:6803/1659316290","heartbeat_back_addr":"172.21.15.53:6807/1659316290","heartbeat_front_addr":"172.21.15.53:6805/1659316290","state":["exists","up"]},{"osd":3,"uuid":"9c290fd9-e121-4a81-9687-b0eb93debd86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6811","nonce":2728987281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6813","nonce":2728987281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6817","nonce":2728987281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6815","nonce":2728987281}]},"public_addr":"172.21.15.1:6811/2728987281","cluster_addr":"172.21.15.1:6813/2728987281","heartbeat_back_addr":"172.21.15.1:6817/2728987281","heartbeat_front_addr":"172.21.15.1:6815/2728987281","state":["exists","up"]},{"osd":4,"uuid":"533226bc-cc57-4f15-9e29-820294d57e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6809","nonce":2954740024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6811","nonce":2954740024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6815","nonce":2954740024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6813","nonce":2954740024}]},"public_addr":"172.21.15.26:6809/2954740024","cluster_addr":"172.21.15.26:6811/2954740024","heartbeat_back_addr":"172.21.15.26:6815/2954740024","heartbeat_front_addr":"172.21.15.26:6813/2954740024","state":["exists","up"]},{"osd":5,"uuid":"d3ab25eb-e1d4-4557-bc89-efc9072401ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6809","nonce":3966666318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6811","nonce":3966666318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6815","nonce":3966666318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6813","nonce":3966666318}]},"public_addr":"172.21.15.53:6809/3966666318","cluster_addr":"172.21.15.53:6811/3966666318","heartbeat_back_addr":"172.21.15.53:6815/3966666318","heartbeat_front_addr":"172.21.15.53:6813/3966666318","state":["exists","up"]},{"osd":6,"uuid":"180f506e-afb8-4043-b0b4-2dc86f6d4bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6819","nonce":3364032646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6821","nonce":3364032646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6825","nonce":3364032646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6823","nonce":3364032646}]},"public_addr":"172.21.15.1:6819/3364032646","cluster_addr":"172.21.15.1:6821/3364032646","heartbeat_back_addr":"172.21.15.1:6825/3364032646","heartbeat_front_addr":"172.21.15.1:6823/3364032646","state":["exists","up"]},{"osd":7,"uuid":"0d837bac-194f-498d-b7fb-b3147e992329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6817","nonce":3766901391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6819","nonce":3766901391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6823","nonce":3766901391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6821","nonce":3766901391}]},"public_addr":"172.21.15.26:6817/3766901391","cluster_addr":"172.21.15.26:6819/3766901391","heartbeat_back_addr":"172.21.15.26:6823/3766901391","heartbeat_front_addr":"172.21.15.26:6821/3766901391","state":["exists","up"]},{"osd":8,"uuid":"933f231a-7454-47b0-9291-f1fa64f0baec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6817","nonce":3224335871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6819","nonce":3224335871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6823","nonce":3224335871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6821","nonce":3224335871}]},"public_addr":"172.21.15.53:6817/3224335871","cluster_addr":"172.21.15.53:6819/3224335871","heartbeat_back_addr":"172.21.15.53:6823/3224335871","heartbeat_front_addr":"172.21.15.53:6821/3224335871","state":["exists","up"]},{"osd":9,"uuid":"245fcbb3-c651-4703-ab80-309dc98bd983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6827","nonce":1391199844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6829","nonce":1391199844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6833","nonce":1391199844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6831","nonce":1391199844}]},"public_addr":"172.21.15.1:6827/1391199844","cluster_addr":"172.21.15.1:6829/1391199844","heartbeat_back_addr":"172.21.15.1:6833/1391199844","heartbeat_front_addr":"172.21.15.1:6831/1391199844","state":["exists","up"]},{"osd":10,"uuid":"2c090e7a-5240-4d52-ae4f-e94fe05383d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6825","nonce":1773545447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6827","nonce":1773545447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6831","nonce":1773545447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6829","nonce":1773545447}]},"public_addr":"172.21.15.26:6825/1773545447","cluster_addr":"172.21.15.26:6827/1773545447","heartbeat_back_addr":"172.21.15.26:6831/1773545447","heartbeat_front_addr":"172.21.15.26:6829/1773545447","state":["exists","up"]},{"osd":11,"uuid":"4511e32d-e469-45e1-8310-0ff400cb7b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6825","nonce":2564669546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6827","nonce":2564669546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6831","nonce":2564669546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6829","nonce":2564669546}]},"public_addr":"172.21.15.53:6825/2564669546","cluster_addr":"172.21.15.53:6827/2564669546","heartbeat_back_addr":"172.21.15.53:6831/2564669546","heartbeat_front_addr":"172.21.15.53:6829/2564669546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:28.140250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:49.885461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:12.521487+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:34.926197+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:56.815594+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:18.387751+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:40.587106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:02.319370+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:23.832298+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:46.338040+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:08.730414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:30.050070+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1494336948":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6801/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6800/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6800/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/1641976669":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/2895923638":"2023-11-17T16:30:35.064289+0000","172.21.15.1:6801/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/78811711":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/4131957708":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/344403698":"2023-11-17T16:30:35.064289+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:37:32.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.586+0000 7f6e93148700 1 -- 172.21.15.1:0/1596936545 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f6e7c0a1260 msgr2=0x7f6e7c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:32.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.586+0000 7f6e93148700 1 --2- 172.21.15.1:0/1596936545 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f6e7c0a1260 0x7f6e7c0a3710 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6e78006fd0 tx=0x7f6e78008040 comp rx=0 tx=0).stop 2023-11-16T16:37:32.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.586+0000 7f6e93148700 1 -- 172.21.15.1:0/1596936545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e8c0ff460 msgr2=0x7f6e8c19e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:32.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.586+0000 7f6e93148700 1 --2- 172.21.15.1:0/1596936545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e8c0ff460 0x7f6e8c19e4f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6e80009990 tx=0x7f6e80009580 comp rx=0 tx=0).stop 2023-11-16T16:37:32.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.587+0000 7f6e93148700 1 -- 172.21.15.1:0/1596936545 shutdown_connections 2023-11-16T16:37:32.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.587+0000 7f6e93148700 1 --2- 172.21.15.1:0/1596936545 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f6e7c0a1260 0x7f6e7c0a3710 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:32.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.587+0000 7f6e93148700 1 --2- 172.21.15.1:0/1596936545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6e8c104500 0x7f6e8c198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:32.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.587+0000 7f6e93148700 1 --2- 172.21.15.1:0/1596936545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6e8c0ff460 0x7f6e8c19e4f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:32.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.587+0000 7f6e93148700 1 --2- 172.21.15.1:0/1596936545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e8c0ffe00 0x7f6e8c19ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:32.590 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.587+0000 7f6e93148700 1 -- 172.21.15.1:0/1596936545 >> 172.21.15.1:0/1596936545 conn(0x7f6e8c0770a0 msgr2=0x7f6e8c0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:32.590 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.587+0000 7f6e93148700 1 -- 172.21.15.1:0/1596936545 shutdown_connections 2023-11-16T16:37:32.590 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:32.587+0000 7f6e93148700 1 -- 172.21.15.1:0/1596936545 wait complete. 2023-11-16T16:37:33.033 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph tell osd.0 flush_pg_stats 2023-11-16T16:37:33.033 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph tell osd.1 flush_pg_stats 2023-11-16T16:37:33.034 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph tell osd.2 flush_pg_stats 2023-11-16T16:37:33.034 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph tell osd.3 flush_pg_stats 2023-11-16T16:37:33.035 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph tell osd.4 flush_pg_stats 2023-11-16T16:37:33.035 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph tell osd.5 flush_pg_stats 2023-11-16T16:37:33.036 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph tell osd.6 flush_pg_stats 2023-11-16T16:37:33.036 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph tell osd.7 flush_pg_stats 2023-11-16T16:37:33.037 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph tell osd.8 flush_pg_stats 2023-11-16T16:37:33.037 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph tell osd.9 flush_pg_stats 2023-11-16T16:37:33.037 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph tell osd.10 flush_pg_stats 2023-11-16T16:37:33.038 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph tell osd.11 flush_pg_stats 2023-11-16T16:37:33.435 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:33 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1596936545' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:37:33.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:33 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1596936545' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:37:33.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:33 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1596936545' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:37:34.050 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:37:34.224 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:37:34.455 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:37:34.486 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:34 smithi001 ceph-mon[98800]: pgmap v270: 97 pgs: 97 active+clean; 633 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.4 KiB/s wr, 7 op/s 2023-11-16T16:37:34.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:34 smithi026 ceph-mon[100112]: pgmap v270: 97 pgs: 97 active+clean; 633 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.4 KiB/s wr, 7 op/s 2023-11-16T16:37:34.519 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:37:34.519 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:37:34.520 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:37:34.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:34 smithi053 ceph-mon[100638]: pgmap v270: 97 pgs: 97 active+clean; 633 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.4 KiB/s wr, 7 op/s 2023-11-16T16:37:34.643 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:37:35.022 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:37:35.233 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:37:35.234 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:37:35.410 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:37:35.412 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:37:35.764 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:35 smithi001 ceph-mon[98800]: pgmap v271: 97 pgs: 97 active+clean; 633 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.3 KiB/s wr, 9 op/s 2023-11-16T16:37:35.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:35 smithi053 ceph-mon[100638]: pgmap v271: 97 pgs: 97 active+clean; 633 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.3 KiB/s wr, 9 op/s 2023-11-16T16:37:36.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:35 smithi026 ceph-mon[100112]: pgmap v271: 97 pgs: 97 active+clean; 633 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.3 KiB/s wr, 9 op/s 2023-11-16T16:37:37.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.989+0000 7f9c4eab0700 1 -- 172.21.15.1:0/2802825198 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c480ff9e0 msgr2=0x7f9c4810c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:37.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.989+0000 7f9c4eab0700 1 --2- 172.21.15.1:0/2802825198 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c480ff9e0 0x7f9c4810c740 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9c34009990 tx=0x7f9c34009ca0 comp rx=0 tx=0).stop 2023-11-16T16:37:37.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.989+0000 7f9c4eab0700 1 -- 172.21.15.1:0/2802825198 shutdown_connections 2023-11-16T16:37:37.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.989+0000 7f9c4eab0700 1 --2- 172.21.15.1:0/2802825198 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c4810cc80 0x7f9c4810f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:37.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.989+0000 7f9c4eab0700 1 --2- 172.21.15.1:0/2802825198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c480ff0d0 0x7f9c480ff4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:37.168 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.989+0000 7f9c4eab0700 1 --2- 172.21.15.1:0/2802825198 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c480ff9e0 0x7f9c4810c740 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:37.168 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.989+0000 7f9c4eab0700 1 -- 172.21.15.1:0/2802825198 >> 172.21.15.1:0/2802825198 conn(0x7f9c480770a0 msgr2=0x7f9c480774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:37.168 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.989+0000 7f9c4eab0700 1 -- 172.21.15.1:0/2802825198 shutdown_connections 2023-11-16T16:37:37.168 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.989+0000 7f9c4eab0700 1 -- 172.21.15.1:0/2802825198 wait complete. 2023-11-16T16:37:37.168 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.990+0000 7f9c4eab0700 1 Processor -- start 2023-11-16T16:37:37.168 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.990+0000 7f9c4eab0700 1 -- start start 2023-11-16T16:37:37.168 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.990+0000 7f9c4eab0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c480ff0d0 0x7f9c4819e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:37.168 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.990+0000 7f9c4eab0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c480ff9e0 0x7f9c4819ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:37.168 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.990+0000 7f9c4eab0700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c4810cc80 0x7f9c481985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:37.169 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.990+0000 7f9c4eab0700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c48198c20 con 0x7f9c480ff9e0 2023-11-16T16:37:37.169 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.990+0000 7f9c4eab0700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c48198d90 con 0x7f9c480ff0d0 2023-11-16T16:37:37.169 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.990+0000 7f9c4eab0700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c48198f50 con 0x7f9c4810cc80 2023-11-16T16:37:37.169 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.990+0000 7f9c47fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c480ff9e0 0x7f9c4819ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:37.169 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.990+0000 7f9c47fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c480ff9e0 0x7f9c4819ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41710/0 (socket says 172.21.15.1:41710) 2023-11-16T16:37:37.169 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.990+0000 7f9c47fff700 1 -- 172.21.15.1:0/191930271 learned_addr learned my addr 172.21.15.1:0/191930271 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:37.169 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.990+0000 7f9c4c84c700 1 --2- 172.21.15.1:0/191930271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c480ff0d0 0x7f9c4819e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:37.169 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.990+0000 7f9c4d04d700 1 --2- 172.21.15.1:0/191930271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c4810cc80 0x7f9c481985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:37.169 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.991+0000 7f9c47fff700 1 -- 172.21.15.1:0/191930271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c4810cc80 msgr2=0x7f9c481985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:37.170 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.991+0000 7f9c47fff700 1 --2- 172.21.15.1:0/191930271 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9c4810cc80 0x7f9c481985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:37.170 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.991+0000 7f9c47fff700 1 -- 172.21.15.1:0/191930271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c480ff0d0 msgr2=0x7f9c4819e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:37.170 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.991+0000 7f9c47fff700 1 --2- 172.21.15.1:0/191930271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c480ff0d0 0x7f9c4819e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:37.170 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.991+0000 7f9c47fff700 1 -- 172.21.15.1:0/191930271 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c340095f0 con 0x7f9c480ff9e0 2023-11-16T16:37:37.170 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.991+0000 7f9c4c84c700 1 --2- 172.21.15.1:0/191930271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c480ff0d0 0x7f9c4819e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:37:37.170 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.991+0000 7f9c47fff700 1 --2- 172.21.15.1:0/191930271 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c480ff9e0 0x7f9c4819ea70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9c3400b680 tx=0x7f9c34004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:37.170 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.991+0000 7f9c45ffb700 1 -- 172.21.15.1:0/191930271 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3401d070 con 0x7f9c480ff9e0 2023-11-16T16:37:37.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.991+0000 7f9c4eab0700 1 -- 172.21.15.1:0/191930271 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c48112570 con 0x7f9c480ff9e0 2023-11-16T16:37:37.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.991+0000 7f9c45ffb700 1 -- 172.21.15.1:0/191930271 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c34005030 con 0x7f9c480ff9e0 2023-11-16T16:37:37.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.991+0000 7f9c45ffb700 1 -- 172.21.15.1:0/191930271 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c34004c40 con 0x7f9c480ff9e0 2023-11-16T16:37:37.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.991+0000 7f9c4eab0700 1 -- 172.21.15.1:0/191930271 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c481123e0 con 0x7f9c480ff9e0 2023-11-16T16:37:37.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.992+0000 7f9c4eab0700 1 -- 172.21.15.1:0/191930271 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9c4810ea40 con 0x7f9c480ff9e0 2023-11-16T16:37:37.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.994+0000 7f9c45ffb700 1 -- 172.21.15.1:0/191930271 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f9c340054a0 con 0x7f9c480ff9e0 2023-11-16T16:37:37.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.994+0000 7f9c45ffb700 1 --2- 172.21.15.1:0/191930271 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f9c380a11e0 0x7f9c380a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:37.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.995+0000 7f9c45ffb700 1 -- 172.21.15.1:0/191930271 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9c340ce010 con 0x7f9c480ff9e0 2023-11-16T16:37:37.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.995+0000 7f9c4c84c700 1 --2- 172.21.15.1:0/191930271 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f9c380a11e0 0x7f9c380a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:37.172 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.995+0000 7f9c45ffb700 1 --2- 172.21.15.1:0/191930271 >> [v2:172.21.15.26:6808/2954740024,v1:172.21.15.26:6809/2954740024] conn(0x7f9c380aa9b0 0x7f9c380acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:37.172 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.995+0000 7f9c45ffb700 1 -- 172.21.15.1:0/191930271 --> [v2:172.21.15.26:6808/2954740024,v1:172.21.15.26:6809/2954740024] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9c380ad470 con 0x7f9c380aa9b0 2023-11-16T16:37:37.172 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.995+0000 7f9c45ffb700 1 -- 172.21.15.1:0/191930271 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c340ce4b0 con 0x7f9c480ff9e0 2023-11-16T16:37:37.172 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.995+0000 7f9c4d04d700 1 --2- 172.21.15.1:0/191930271 >> [v2:172.21.15.26:6808/2954740024,v1:172.21.15.26:6809/2954740024] conn(0x7f9c380aa9b0 0x7f9c380acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:37.172 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.995+0000 7f9c4c84c700 1 --2- 172.21.15.1:0/191930271 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f9c380a11e0 0x7f9c380a3690 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c011490 tx=0x7f9c3c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:37.172 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.995+0000 7f9c4d04d700 1 --2- 172.21.15.1:0/191930271 >> [v2:172.21.15.26:6808/2954740024,v1:172.21.15.26:6809/2954740024] conn(0x7f9c380aa9b0 0x7f9c380acdc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:37.172 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:36.996+0000 7f9c45ffb700 1 -- 172.21.15.1:0/191930271 <== osd.4 v2:172.21.15.26:6808/2954740024 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9c380ad470 con 0x7f9c380aa9b0 2023-11-16T16:37:37.172 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:37.018+0000 7f9c4eab0700 1 -- 172.21.15.1:0/191930271 --> [v2:172.21.15.26:6808/2954740024,v1:172.21.15.26:6809/2954740024] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9c48112700 con 0x7f9c380aa9b0 2023-11-16T16:37:37.173 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:37.019+0000 7f9c45ffb700 1 -- 172.21.15.1:0/191930271 <== osd.4 v2:172.21.15.26:6808/2954740024 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9c48112700 con 0x7f9c380aa9b0 2023-11-16T16:37:37.173 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:37.019+0000 7f9c4eab0700 1 -- 172.21.15.1:0/191930271 >> [v2:172.21.15.26:6808/2954740024,v1:172.21.15.26:6809/2954740024] conn(0x7f9c380aa9b0 msgr2=0x7f9c380acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:37.173 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:37.019+0000 7f9c4eab0700 1 --2- 172.21.15.1:0/191930271 >> [v2:172.21.15.26:6808/2954740024,v1:172.21.15.26:6809/2954740024] conn(0x7f9c380aa9b0 0x7f9c380acdc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:37.173 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:37.020+0000 7f9c4eab0700 1 -- 172.21.15.1:0/191930271 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f9c380a11e0 msgr2=0x7f9c380a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:37.173 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:37.020+0000 7f9c4eab0700 1 --2- 172.21.15.1:0/191930271 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f9c380a11e0 0x7f9c380a3690 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c011490 tx=0x7f9c3c009260 comp rx=0 tx=0).stop 2023-11-16T16:37:37.173 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:37.020+0000 7f9c4eab0700 1 -- 172.21.15.1:0/191930271 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c480ff9e0 msgr2=0x7f9c4819ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:37.173 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:37.020+0000 7f9c4eab0700 1 --2- 172.21.15.1:0/191930271 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c480ff9e0 0x7f9c4819ea70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9c3400b680 tx=0x7f9c34004730 comp rx=0 tx=0).stop 2023-11-16T16:37:37.173 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:37.020+0000 7f9c4d04d700 1 -- 172.21.15.1:0/191930271 reap_dead start 2023-11-16T16:37:37.174 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:37.020+0000 7f9c4eab0700 1 -- 172.21.15.1:0/191930271 shutdown_connections 2023-11-16T16:37:37.174 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:37.020+0000 7f9c4eab0700 1 -- 172.21.15.1:0/191930271 >> 172.21.15.1:0/191930271 conn(0x7f9c480770a0 msgr2=0x7f9c4810e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:37.174 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:37.020+0000 7f9c4eab0700 1 -- 172.21.15.1:0/191930271 shutdown_connections 2023-11-16T16:37:37.174 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:37.020+0000 7f9c4eab0700 1 -- 172.21.15.1:0/191930271 wait complete. 2023-11-16T16:37:37.667 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:37 smithi001 ceph-mon[98800]: pgmap v272: 97 pgs: 97 active+clean; 633 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2023-11-16T16:37:37.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:37 smithi053 ceph-mon[100638]: pgmap v272: 97 pgs: 97 active+clean; 633 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2023-11-16T16:37:38.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:37 smithi026 ceph-mon[100112]: pgmap v272: 97 pgs: 97 active+clean; 633 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2023-11-16T16:37:39.066 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.063+0000 7fd35c71d700 1 -- 172.21.15.1:0/1397236759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd35410ce40 msgr2=0x7fd35410f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:39.066 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.063+0000 7fd35c71d700 1 --2- 172.21.15.1:0/1397236759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd35410ce40 0x7fd35410f220 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00b0a0 tx=0x7fd34c00b3b0 comp rx=0 tx=0).stop 2023-11-16T16:37:39.066 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.063+0000 7fd35c71d700 1 -- 172.21.15.1:0/1397236759 shutdown_connections 2023-11-16T16:37:39.066 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.063+0000 7fd35c71d700 1 --2- 172.21.15.1:0/1397236759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd35410ce40 0x7fd35410f220 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:39.066 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.063+0000 7fd35c71d700 1 --2- 172.21.15.1:0/1397236759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd354072d20 0x7fd3540730f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:39.067 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.063+0000 7fd35c71d700 1 --2- 172.21.15.1:0/1397236759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd354073630 0x7fd35410c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:39.067 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.063+0000 7fd35c71d700 1 -- 172.21.15.1:0/1397236759 >> 172.21.15.1:0/1397236759 conn(0x7fd354078430 msgr2=0x7fd354078830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:39.067 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.063+0000 7fd35c71d700 1 -- 172.21.15.1:0/1397236759 shutdown_connections 2023-11-16T16:37:39.067 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.063+0000 7fd35c71d700 1 -- 172.21.15.1:0/1397236759 wait complete. 2023-11-16T16:37:39.071 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.064+0000 7fd35c71d700 1 Processor -- start 2023-11-16T16:37:39.071 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.064+0000 7fd35c71d700 1 -- start start 2023-11-16T16:37:39.071 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.065+0000 7fd35c71d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd354072d20 0x7fd3541986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:39.071 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.065+0000 7fd35c71d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd354073630 0x7fd354198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:39.071 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.065+0000 7fd35c71d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd35410ce40 0x7fd35419bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:39.071 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.065+0000 7fd35c71d700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35419c5d0 con 0x7fd354073630 2023-11-16T16:37:39.071 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.065+0000 7fd35c71d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35419c740 con 0x7fd354072d20 2023-11-16T16:37:39.072 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.065+0000 7fd35c71d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35419c900 con 0x7fd35410ce40 2023-11-16T16:37:39.072 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.065+0000 7fd359cb8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd354073630 0x7fd354198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:39.072 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.065+0000 7fd359cb8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd354073630 0x7fd354198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41724/0 (socket says 172.21.15.1:41724) 2023-11-16T16:37:39.072 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.065+0000 7fd359cb8700 1 -- 172.21.15.1:0/429760930 learned_addr learned my addr 172.21.15.1:0/429760930 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:39.072 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.065+0000 7fd35a4b9700 1 --2- 172.21.15.1:0/429760930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd354072d20 0x7fd3541986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:39.072 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.065+0000 7fd35acba700 1 --2- 172.21.15.1:0/429760930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd35410ce40 0x7fd35419bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:39.072 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.065+0000 7fd359cb8700 1 -- 172.21.15.1:0/429760930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd35410ce40 msgr2=0x7fd35419bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:39.072 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.065+0000 7fd359cb8700 1 --2- 172.21.15.1:0/429760930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd35410ce40 0x7fd35419bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:39.073 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.065+0000 7fd359cb8700 1 -- 172.21.15.1:0/429760930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd354072d20 msgr2=0x7fd3541986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:39.073 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.065+0000 7fd359cb8700 1 --2- 172.21.15.1:0/429760930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd354072d20 0x7fd3541986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:39.073 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.065+0000 7fd359cb8700 1 -- 172.21.15.1:0/429760930 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd34c009d00 con 0x7fd354073630 2023-11-16T16:37:39.073 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.065+0000 7fd35acba700 1 --2- 172.21.15.1:0/429760930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd35410ce40 0x7fd35419bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:37:39.073 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.065+0000 7fd35a4b9700 1 --2- 172.21.15.1:0/429760930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd354072d20 0x7fd3541986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:37:39.073 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.065+0000 7fd359cb8700 1 --2- 172.21.15.1:0/429760930 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd354073630 0x7fd354198c10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd34400d950 tx=0x7fd34400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:39.073 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.065+0000 7fd34b7fe700 1 -- 172.21.15.1:0/429760930 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd344009940 con 0x7fd354073630 2023-11-16T16:37:39.073 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.066+0000 7fd34b7fe700 1 -- 172.21.15.1:0/429760930 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd344010460 con 0x7fd354073630 2023-11-16T16:37:39.073 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.066+0000 7fd34b7fe700 1 -- 172.21.15.1:0/429760930 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34400f620 con 0x7fd354073630 2023-11-16T16:37:39.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.066+0000 7fd35c71d700 1 -- 172.21.15.1:0/429760930 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd354112520 con 0x7fd354073630 2023-11-16T16:37:39.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.066+0000 7fd35c71d700 1 -- 172.21.15.1:0/429760930 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd354112390 con 0x7fd354073630 2023-11-16T16:37:39.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.066+0000 7fd35c71d700 1 -- 172.21.15.1:0/429760930 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd3541072b0 con 0x7fd354073630 2023-11-16T16:37:39.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.068+0000 7fd34b7fe700 1 -- 172.21.15.1:0/429760930 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fd3440108d0 con 0x7fd354073630 2023-11-16T16:37:39.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.068+0000 7fd34b7fe700 1 --2- 172.21.15.1:0/429760930 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd3400a1260 0x7fd3400a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:39.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.068+0000 7fd35a4b9700 1 --2- 172.21.15.1:0/429760930 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd3400a1260 0x7fd3400a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:39.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.068+0000 7fd34b7fe700 1 -- 172.21.15.1:0/429760930 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd3440ccb80 con 0x7fd354073630 2023-11-16T16:37:39.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.068+0000 7fd34b7fe700 1 --2- 172.21.15.1:0/429760930 >> [v2:172.21.15.1:6818/3364032646,v1:172.21.15.1:6819/3364032646] conn(0x7fd3400aaa30 0x7fd3400ace40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:39.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.068+0000 7fd34b7fe700 1 -- 172.21.15.1:0/429760930 --> [v2:172.21.15.1:6818/3364032646,v1:172.21.15.1:6819/3364032646] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd3400ad4f0 con 0x7fd3400aaa30 2023-11-16T16:37:39.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.068+0000 7fd34b7fe700 1 -- 172.21.15.1:0/429760930 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3440cd020 con 0x7fd354073630 2023-11-16T16:37:39.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.069+0000 7fd35a4b9700 1 --2- 172.21.15.1:0/429760930 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd3400a1260 0x7fd3400a3710 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd350011490 tx=0x7fd350009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:39.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.069+0000 7fd35acba700 1 --2- 172.21.15.1:0/429760930 >> [v2:172.21.15.1:6818/3364032646,v1:172.21.15.1:6819/3364032646] conn(0x7fd3400aaa30 0x7fd3400ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:39.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.069+0000 7fd35acba700 1 --2- 172.21.15.1:0/429760930 >> [v2:172.21.15.1:6818/3364032646,v1:172.21.15.1:6819/3364032646] conn(0x7fd3400aaa30 0x7fd3400ace40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:39.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.070+0000 7fd34b7fe700 1 -- 172.21.15.1:0/429760930 <== osd.6 v2:172.21.15.1:6818/3364032646 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fd3400ad4f0 con 0x7fd3400aaa30 2023-11-16T16:37:39.088 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.085+0000 7fd35c71d700 1 -- 172.21.15.1:0/429760930 --> [v2:172.21.15.1:6818/3364032646,v1:172.21.15.1:6819/3364032646] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd3541126b0 con 0x7fd3400aaa30 2023-11-16T16:37:39.088 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.086+0000 7fd34b7fe700 1 -- 172.21.15.1:0/429760930 <== osd.6 v2:172.21.15.1:6818/3364032646 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fd3541126b0 con 0x7fd3400aaa30 2023-11-16T16:37:39.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.086+0000 7fd35c71d700 1 -- 172.21.15.1:0/429760930 >> [v2:172.21.15.1:6818/3364032646,v1:172.21.15.1:6819/3364032646] conn(0x7fd3400aaa30 msgr2=0x7fd3400ace40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:39.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.086+0000 7fd35c71d700 1 --2- 172.21.15.1:0/429760930 >> [v2:172.21.15.1:6818/3364032646,v1:172.21.15.1:6819/3364032646] conn(0x7fd3400aaa30 0x7fd3400ace40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:39.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.086+0000 7fd35c71d700 1 -- 172.21.15.1:0/429760930 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd3400a1260 msgr2=0x7fd3400a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:39.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.086+0000 7fd35c71d700 1 --2- 172.21.15.1:0/429760930 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd3400a1260 0x7fd3400a3710 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd350011490 tx=0x7fd350009260 comp rx=0 tx=0).stop 2023-11-16T16:37:39.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.086+0000 7fd35c71d700 1 -- 172.21.15.1:0/429760930 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd354073630 msgr2=0x7fd354198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:39.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.086+0000 7fd35c71d700 1 --2- 172.21.15.1:0/429760930 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd354073630 0x7fd354198c10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd34400d950 tx=0x7fd34400dc60 comp rx=0 tx=0).stop 2023-11-16T16:37:39.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.086+0000 7fd35acba700 1 -- 172.21.15.1:0/429760930 reap_dead start 2023-11-16T16:37:39.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.086+0000 7fd35c71d700 1 -- 172.21.15.1:0/429760930 shutdown_connections 2023-11-16T16:37:39.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.086+0000 7fd35c71d700 1 -- 172.21.15.1:0/429760930 >> 172.21.15.1:0/429760930 conn(0x7fd354078430 msgr2=0x7fd3541068e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:39.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.086+0000 7fd35c71d700 1 -- 172.21.15.1:0/429760930 shutdown_connections 2023-11-16T16:37:39.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:39.086+0000 7fd35c71d700 1 -- 172.21.15.1:0/429760930 wait complete. 2023-11-16T16:37:39.250 INFO:teuthology.orchestra.run.smithi001.stdout:146028888110 2023-11-16T16:37:39.251 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph osd last-stat-seq osd.4 2023-11-16T16:37:39.792 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:39 smithi001 ceph-mon[98800]: pgmap v273: 97 pgs: 97 active+clean; 633 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2023-11-16T16:37:39.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:39 smithi053 ceph-mon[100638]: pgmap v273: 97 pgs: 97 active+clean; 633 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2023-11-16T16:37:40.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:39 smithi026 ceph-mon[100112]: pgmap v273: 97 pgs: 97 active+clean; 633 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2023-11-16T16:37:40.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.564+0000 7f6712dc1700 1 -- 172.21.15.1:0/959938479 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f670c0ffe00 msgr2=0x7f670c103dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:40.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.564+0000 7f6712dc1700 1 --2- 172.21.15.1:0/959938479 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f670c0ffe00 0x7f670c103dd0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f66f80099c0 tx=0x7f66f8009cd0 comp rx=0 tx=0).stop 2023-11-16T16:37:40.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.564+0000 7f6712dc1700 1 -- 172.21.15.1:0/959938479 shutdown_connections 2023-11-16T16:37:40.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.564+0000 7f6712dc1700 1 --2- 172.21.15.1:0/959938479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f670c104500 0x7f670c10c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:40.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.564+0000 7f6712dc1700 1 --2- 172.21.15.1:0/959938479 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f670c0ff460 0x7f670c0ff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:40.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.564+0000 7f6712dc1700 1 --2- 172.21.15.1:0/959938479 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f670c0ffe00 0x7f670c103dd0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:40.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.564+0000 7f6712dc1700 1 -- 172.21.15.1:0/959938479 >> 172.21.15.1:0/959938479 conn(0x7f670c0770a0 msgr2=0x7f670c0774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:40.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.564+0000 7f6712dc1700 1 -- 172.21.15.1:0/959938479 shutdown_connections 2023-11-16T16:37:40.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.564+0000 7f6712dc1700 1 -- 172.21.15.1:0/959938479 wait complete. 2023-11-16T16:37:40.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.565+0000 7f6712dc1700 1 Processor -- start 2023-11-16T16:37:40.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.565+0000 7f6712dc1700 1 -- start start 2023-11-16T16:37:40.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.565+0000 7f6712dc1700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f670c0ff460 0x7f670c106120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:40.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.565+0000 7f6712dc1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f670c0ffe00 0x7f670c106660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:40.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.565+0000 7f6712dc1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f670c104500 0x7f670c106be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:40.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.565+0000 7f6712dc1700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670c1a4910 con 0x7f670c0ffe00 2023-11-16T16:37:40.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.565+0000 7f6712dc1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670c1a4a80 con 0x7f670c104500 2023-11-16T16:37:40.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.565+0000 7f6712dc1700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670c1a4c40 con 0x7f670c0ff460 2023-11-16T16:37:40.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.565+0000 7f670bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f670c0ffe00 0x7f670c106660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:40.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.565+0000 7f670bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f670c0ffe00 0x7f670c106660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41738/0 (socket says 172.21.15.1:41738) 2023-11-16T16:37:40.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.565+0000 7f670bfff700 1 -- 172.21.15.1:0/3055800260 learned_addr learned my addr 172.21.15.1:0/3055800260 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:40.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.565+0000 7f671135e700 1 --2- 172.21.15.1:0/3055800260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f670c104500 0x7f670c106be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:40.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.565+0000 7f6710b5d700 1 --2- 172.21.15.1:0/3055800260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f670c0ff460 0x7f670c106120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:40.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.566+0000 7f670bfff700 1 -- 172.21.15.1:0/3055800260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f670c0ff460 msgr2=0x7f670c106120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:40.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.566+0000 7f670bfff700 1 --2- 172.21.15.1:0/3055800260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f670c0ff460 0x7f670c106120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:40.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.566+0000 7f670bfff700 1 -- 172.21.15.1:0/3055800260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f670c104500 msgr2=0x7f670c106be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:40.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.566+0000 7f670bfff700 1 --2- 172.21.15.1:0/3055800260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f670c104500 0x7f670c106be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:40.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.566+0000 7f670bfff700 1 -- 172.21.15.1:0/3055800260 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f80095f0 con 0x7f670c0ffe00 2023-11-16T16:37:40.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.566+0000 7f6710b5d700 1 --2- 172.21.15.1:0/3055800260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f670c0ff460 0x7f670c106120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:37:40.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.566+0000 7f670bfff700 1 --2- 172.21.15.1:0/3055800260 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f670c0ffe00 0x7f670c106660 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f66f8009990 tx=0x7f66f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:40.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.566+0000 7f6709ffb700 1 -- 172.21.15.1:0/3055800260 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f801d070 con 0x7f670c0ffe00 2023-11-16T16:37:40.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.566+0000 7f6709ffb700 1 -- 172.21.15.1:0/3055800260 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66f80054f0 con 0x7f670c0ffe00 2023-11-16T16:37:40.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.566+0000 7f6712dc1700 1 -- 172.21.15.1:0/3055800260 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f670c10f230 con 0x7f670c0ffe00 2023-11-16T16:37:40.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.566+0000 7f6709ffb700 1 -- 172.21.15.1:0/3055800260 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f8003ad0 con 0x7f670c0ffe00 2023-11-16T16:37:40.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.566+0000 7f6712dc1700 1 -- 172.21.15.1:0/3055800260 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f670c10f0a0 con 0x7f670c0ffe00 2023-11-16T16:37:40.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.567+0000 7f6712dc1700 1 -- 172.21.15.1:0/3055800260 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f670c103700 con 0x7f670c0ffe00 2023-11-16T16:37:40.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.568+0000 7f6709ffb700 1 -- 172.21.15.1:0/3055800260 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f66f80047a0 con 0x7f670c0ffe00 2023-11-16T16:37:40.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.569+0000 7f6709ffb700 1 --2- 172.21.15.1:0/3055800260 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f66fc0a1260 0x7f66fc0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:40.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.569+0000 7f6710b5d700 1 --2- 172.21.15.1:0/3055800260 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f66fc0a1260 0x7f66fc0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:40.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.569+0000 7f6709ffb700 1 -- 172.21.15.1:0/3055800260 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f66f80cde90 con 0x7f670c0ffe00 2023-11-16T16:37:40.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.569+0000 7f6709ffb700 1 --2- 172.21.15.1:0/3055800260 >> [v2:172.21.15.26:6816/3766901391,v1:172.21.15.26:6817/3766901391] conn(0x7f66fc0aaa30 0x7f66fc0ace40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:40.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.569+0000 7f6709ffb700 1 -- 172.21.15.1:0/3055800260 --> [v2:172.21.15.26:6816/3766901391,v1:172.21.15.26:6817/3766901391] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f66fc0ad4f0 con 0x7f66fc0aaa30 2023-11-16T16:37:40.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.569+0000 7f6709ffb700 1 -- 172.21.15.1:0/3055800260 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66f80ce200 con 0x7f670c0ffe00 2023-11-16T16:37:40.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.569+0000 7f6710b5d700 1 --2- 172.21.15.1:0/3055800260 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f66fc0a1260 0x7f66fc0a3710 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6700005950 tx=0x7f67000058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:40.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.569+0000 7f671135e700 1 --2- 172.21.15.1:0/3055800260 >> [v2:172.21.15.26:6816/3766901391,v1:172.21.15.26:6817/3766901391] conn(0x7f66fc0aaa30 0x7f66fc0ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:40.576 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.570+0000 7f671135e700 1 --2- 172.21.15.1:0/3055800260 >> [v2:172.21.15.26:6816/3766901391,v1:172.21.15.26:6817/3766901391] conn(0x7f66fc0aaa30 0x7f66fc0ace40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:40.576 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.571+0000 7f6709ffb700 1 -- 172.21.15.1:0/3055800260 <== osd.7 v2:172.21.15.26:6816/3766901391 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f66fc0ad4f0 con 0x7f66fc0aaa30 2023-11-16T16:37:40.588 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.586+0000 7f6712dc1700 1 -- 172.21.15.1:0/3055800260 --> [v2:172.21.15.26:6816/3766901391,v1:172.21.15.26:6817/3766901391] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f670c10f3c0 con 0x7f66fc0aaa30 2023-11-16T16:37:40.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.587+0000 7f6709ffb700 1 -- 172.21.15.1:0/3055800260 <== osd.7 v2:172.21.15.26:6816/3766901391 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f670c10f3c0 con 0x7f66fc0aaa30 2023-11-16T16:37:40.591 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.587+0000 7f6712dc1700 1 -- 172.21.15.1:0/3055800260 >> [v2:172.21.15.26:6816/3766901391,v1:172.21.15.26:6817/3766901391] conn(0x7f66fc0aaa30 msgr2=0x7f66fc0ace40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:40.591 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.587+0000 7f6712dc1700 1 --2- 172.21.15.1:0/3055800260 >> [v2:172.21.15.26:6816/3766901391,v1:172.21.15.26:6817/3766901391] conn(0x7f66fc0aaa30 0x7f66fc0ace40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:40.591 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.588+0000 7f6712dc1700 1 -- 172.21.15.1:0/3055800260 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f66fc0a1260 msgr2=0x7f66fc0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:40.591 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.588+0000 7f6712dc1700 1 --2- 172.21.15.1:0/3055800260 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f66fc0a1260 0x7f66fc0a3710 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6700005950 tx=0x7f67000058e0 comp rx=0 tx=0).stop 2023-11-16T16:37:40.592 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.588+0000 7f6712dc1700 1 -- 172.21.15.1:0/3055800260 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f670c0ffe00 msgr2=0x7f670c106660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:40.592 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.588+0000 7f6712dc1700 1 --2- 172.21.15.1:0/3055800260 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f670c0ffe00 0x7f670c106660 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f66f8009990 tx=0x7f66f8004730 comp rx=0 tx=0).stop 2023-11-16T16:37:40.592 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.588+0000 7f671135e700 1 -- 172.21.15.1:0/3055800260 reap_dead start 2023-11-16T16:37:40.592 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.588+0000 7f6712dc1700 1 -- 172.21.15.1:0/3055800260 shutdown_connections 2023-11-16T16:37:40.592 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.588+0000 7f6712dc1700 1 -- 172.21.15.1:0/3055800260 >> 172.21.15.1:0/3055800260 conn(0x7f670c0770a0 msgr2=0x7f670c102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:40.592 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.588+0000 7f6712dc1700 1 -- 172.21.15.1:0/3055800260 shutdown_connections 2023-11-16T16:37:40.592 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:40.588+0000 7f6712dc1700 1 -- 172.21.15.1:0/3055800260 wait complete. 2023-11-16T16:37:40.737 INFO:teuthology.orchestra.run.smithi001.stdout:201863462950 2023-11-16T16:37:40.738 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph osd last-stat-seq osd.6 2023-11-16T16:37:41.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.211+0000 7fc6af928700 1 -- 172.21.15.1:0/3134372322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc6a810a890 msgr2=0x7fc6a810ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:41.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.211+0000 7fc6af928700 1 --2- 172.21.15.1:0/3134372322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc6a810a890 0x7fc6a810ccd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc6a000b0a0 tx=0x7fc6a000b3b0 comp rx=0 tx=0).stop 2023-11-16T16:37:41.217 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.212+0000 7fc6af928700 1 -- 172.21.15.1:0/3134372322 shutdown_connections 2023-11-16T16:37:41.217 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.212+0000 7fc6af928700 1 --2- 172.21.15.1:0/3134372322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc6a810a890 0x7fc6a810ccd0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:41.217 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.212+0000 7fc6af928700 1 --2- 172.21.15.1:0/3134372322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6a8101540 0x7fc6a8101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:41.217 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.212+0000 7fc6af928700 1 --2- 172.21.15.1:0/3134372322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6a8101e50 0x7fc6a810a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:41.217 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.212+0000 7fc6af928700 1 -- 172.21.15.1:0/3134372322 >> 172.21.15.1:0/3134372322 conn(0x7fc6a80770a0 msgr2=0x7fc6a80774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:41.217 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.212+0000 7fc6af928700 1 -- 172.21.15.1:0/3134372322 shutdown_connections 2023-11-16T16:37:41.217 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.212+0000 7fc6af928700 1 -- 172.21.15.1:0/3134372322 wait complete. 2023-11-16T16:37:41.218 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.212+0000 7fc6af928700 1 Processor -- start 2023-11-16T16:37:41.218 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.212+0000 7fc6af928700 1 -- start start 2023-11-16T16:37:41.218 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.213+0000 7fc6af928700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc6a8101540 0x7fc6a81964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:41.218 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.213+0000 7fc6af928700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6a8101e50 0x7fc6a8196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:41.218 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.213+0000 7fc6af928700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6a810a890 0x7fc6a819ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:41.218 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.213+0000 7fc6af928700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a819b490 con 0x7fc6a8101540 2023-11-16T16:37:41.218 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.213+0000 7fc6af928700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a819b600 con 0x7fc6a810a890 2023-11-16T16:37:41.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.213+0000 7fc6af928700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a819b7c0 con 0x7fc6a8101e50 2023-11-16T16:37:41.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.213+0000 7fc6ad6c4700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc6a8101540 0x7fc6a81964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:41.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.213+0000 7fc6ad6c4700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc6a8101540 0x7fc6a81964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41752/0 (socket says 172.21.15.1:41752) 2023-11-16T16:37:41.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.213+0000 7fc6ad6c4700 1 -- 172.21.15.1:0/4197830552 learned_addr learned my addr 172.21.15.1:0/4197830552 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:41.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.213+0000 7fc6ad6c4700 1 -- 172.21.15.1:0/4197830552 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6a8101e50 msgr2=0x7fc6a8196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:41.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.213+0000 7fc6ad6c4700 1 --2- 172.21.15.1:0/4197830552 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc6a8101e50 0x7fc6a8196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:41.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.213+0000 7fc6ad6c4700 1 -- 172.21.15.1:0/4197830552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6a810a890 msgr2=0x7fc6a819ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:41.220 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.213+0000 7fc6ad6c4700 1 --2- 172.21.15.1:0/4197830552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6a810a890 0x7fc6a819ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:41.220 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.213+0000 7fc6ad6c4700 1 -- 172.21.15.1:0/4197830552 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a0009d00 con 0x7fc6a8101540 2023-11-16T16:37:41.220 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.213+0000 7fc6ad6c4700 1 --2- 172.21.15.1:0/4197830552 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc6a8101540 0x7fc6a81964d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc6a400c920 tx=0x7fc6a400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:41.220 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.213+0000 7fc69e7fc700 1 -- 172.21.15.1:0/4197830552 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4007b00 con 0x7fc6a8101540 2023-11-16T16:37:41.220 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.213+0000 7fc6af928700 1 -- 172.21.15.1:0/4197830552 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6a8110300 con 0x7fc6a8101540 2023-11-16T16:37:41.220 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.213+0000 7fc69e7fc700 1 -- 172.21.15.1:0/4197830552 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6a4004cf0 con 0x7fc6a8101540 2023-11-16T16:37:41.220 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.213+0000 7fc69e7fc700 1 -- 172.21.15.1:0/4197830552 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4005520 con 0x7fc6a8101540 2023-11-16T16:37:41.220 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.213+0000 7fc6af928700 1 -- 172.21.15.1:0/4197830552 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6a8110170 con 0x7fc6a8101540 2023-11-16T16:37:41.221 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.214+0000 7fc6af928700 1 -- 172.21.15.1:0/4197830552 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc6a80fc760 con 0x7fc6a8101540 2023-11-16T16:37:41.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.215+0000 7fc69e7fc700 1 -- 172.21.15.1:0/4197830552 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fc6a4004750 con 0x7fc6a8101540 2023-11-16T16:37:41.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.216+0000 7fc69e7fc700 1 --2- 172.21.15.1:0/4197830552 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc6940a11e0 0x7fc6940a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:41.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.216+0000 7fc6acec3700 1 --2- 172.21.15.1:0/4197830552 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc6940a11e0 0x7fc6940a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:41.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.216+0000 7fc69e7fc700 1 -- 172.21.15.1:0/4197830552 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc6a40cc970 con 0x7fc6a8101540 2023-11-16T16:37:41.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.216+0000 7fc69e7fc700 1 --2- 172.21.15.1:0/4197830552 >> [v2:172.21.15.53:6808/3966666318,v1:172.21.15.53:6809/3966666318] conn(0x7fc6940aa9b0 0x7fc6940acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:41.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.216+0000 7fc69e7fc700 1 -- 172.21.15.1:0/4197830552 --> [v2:172.21.15.53:6808/3966666318,v1:172.21.15.53:6809/3966666318] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc6940ad470 con 0x7fc6940aa9b0 2023-11-16T16:37:41.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.216+0000 7fc69e7fc700 1 -- 172.21.15.1:0/4197830552 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6a40cce10 con 0x7fc6a8101540 2023-11-16T16:37:41.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.216+0000 7fc6acec3700 1 --2- 172.21.15.1:0/4197830552 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc6940a11e0 0x7fc6940a3690 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc698005fd0 tx=0x7fc698005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:41.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.217+0000 7fc6adec5700 1 --2- 172.21.15.1:0/4197830552 >> [v2:172.21.15.53:6808/3966666318,v1:172.21.15.53:6809/3966666318] conn(0x7fc6940aa9b0 0x7fc6940acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:41.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.217+0000 7fc6adec5700 1 --2- 172.21.15.1:0/4197830552 >> [v2:172.21.15.53:6808/3966666318,v1:172.21.15.53:6809/3966666318] conn(0x7fc6940aa9b0 0x7fc6940acdc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:41.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.218+0000 7fc69e7fc700 1 -- 172.21.15.1:0/4197830552 <== osd.5 v2:172.21.15.53:6808/3966666318 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fc6940ad470 con 0x7fc6940aa9b0 2023-11-16T16:37:41.235 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.233+0000 7fc6af928700 1 -- 172.21.15.1:0/4197830552 --> [v2:172.21.15.53:6808/3966666318,v1:172.21.15.53:6809/3966666318] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc6a8110490 con 0x7fc6940aa9b0 2023-11-16T16:37:41.236 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.234+0000 7fc69e7fc700 1 -- 172.21.15.1:0/4197830552 <== osd.5 v2:172.21.15.53:6808/3966666318 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc6a8110490 con 0x7fc6940aa9b0 2023-11-16T16:37:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.234+0000 7fc6af928700 1 -- 172.21.15.1:0/4197830552 >> [v2:172.21.15.53:6808/3966666318,v1:172.21.15.53:6809/3966666318] conn(0x7fc6940aa9b0 msgr2=0x7fc6940acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.234+0000 7fc6af928700 1 --2- 172.21.15.1:0/4197830552 >> [v2:172.21.15.53:6808/3966666318,v1:172.21.15.53:6809/3966666318] conn(0x7fc6940aa9b0 0x7fc6940acdc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.234+0000 7fc6af928700 1 -- 172.21.15.1:0/4197830552 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc6940a11e0 msgr2=0x7fc6940a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.234+0000 7fc6af928700 1 --2- 172.21.15.1:0/4197830552 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc6940a11e0 0x7fc6940a3690 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc698005fd0 tx=0x7fc698005ee0 comp rx=0 tx=0).stop 2023-11-16T16:37:41.238 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.235+0000 7fc6af928700 1 -- 172.21.15.1:0/4197830552 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc6a8101540 msgr2=0x7fc6a81964d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:41.238 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.235+0000 7fc6af928700 1 --2- 172.21.15.1:0/4197830552 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc6a8101540 0x7fc6a81964d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc6a400c920 tx=0x7fc6a400cce0 comp rx=0 tx=0).stop 2023-11-16T16:37:41.238 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.235+0000 7fc6adec5700 1 -- 172.21.15.1:0/4197830552 reap_dead start 2023-11-16T16:37:41.238 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.235+0000 7fc6af928700 1 -- 172.21.15.1:0/4197830552 shutdown_connections 2023-11-16T16:37:41.238 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.235+0000 7fc6af928700 1 -- 172.21.15.1:0/4197830552 >> 172.21.15.1:0/4197830552 conn(0x7fc6a80770a0 msgr2=0x7fc6a80fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:41.238 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.235+0000 7fc6af928700 1 -- 172.21.15.1:0/4197830552 shutdown_connections 2023-11-16T16:37:41.238 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:41.235+0000 7fc6af928700 1 -- 172.21.15.1:0/4197830552 wait complete. 2023-11-16T16:37:41.943 INFO:teuthology.orchestra.run.smithi001.stdout:223338299426 2023-11-16T16:37:41.943 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph osd last-stat-seq osd.7 2023-11-16T16:37:42.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:42 smithi001 ceph-mon[98800]: pgmap v274: 97 pgs: 97 active+clean; 636 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-11-16T16:37:42.185 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.182+0000 7fc4bb704700 1 -- 172.21.15.1:0/1239746111 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4b40ff470 msgr2=0x7fc4b40ff840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:42.185 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.182+0000 7fc4bb704700 1 --2- 172.21.15.1:0/1239746111 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4b40ff470 0x7fc4b40ff840 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc4b00099c0 tx=0x7fc4b0009cd0 comp rx=0 tx=0).stop 2023-11-16T16:37:42.185 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.182+0000 7fc4bb704700 1 -- 172.21.15.1:0/1239746111 shutdown_connections 2023-11-16T16:37:42.185 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.182+0000 7fc4bb704700 1 --2- 172.21.15.1:0/1239746111 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4b4104510 0x7fc4b410c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:42.185 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.182+0000 7fc4bb704700 1 --2- 172.21.15.1:0/1239746111 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4b40ff470 0x7fc4b40ff840 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:42.186 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.182+0000 7fc4bb704700 1 --2- 172.21.15.1:0/1239746111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4b40ffe10 0x7fc4b4103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:42.186 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.182+0000 7fc4bb704700 1 -- 172.21.15.1:0/1239746111 >> 172.21.15.1:0/1239746111 conn(0x7fc4b40770d0 msgr2=0x7fc4b40774d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:42.186 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.182+0000 7fc4bb704700 1 -- 172.21.15.1:0/1239746111 shutdown_connections 2023-11-16T16:37:42.187 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.182+0000 7fc4bb704700 1 -- 172.21.15.1:0/1239746111 wait complete. 2023-11-16T16:37:42.188 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.183+0000 7fc4bb704700 1 Processor -- start 2023-11-16T16:37:42.189 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.183+0000 7fc4bb704700 1 -- start start 2023-11-16T16:37:42.189 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.184+0000 7fc4bb704700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4b40ff470 0x7fc4b41972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:42.189 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.184+0000 7fc4bb704700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4b40ffe10 0x7fc4b41977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:42.190 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.184+0000 7fc4bb704700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4b4104510 0x7fc4b41a2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:42.190 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.184+0000 7fc4bb704700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b41980b0 con 0x7fc4b40ff470 2023-11-16T16:37:42.190 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.184+0000 7fc4bb704700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b41a2870 con 0x7fc4b40ffe10 2023-11-16T16:37:42.190 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.184+0000 7fc4bb704700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b41a29b0 con 0x7fc4b4104510 2023-11-16T16:37:42.191 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.184+0000 7fc4b94a0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4b40ff470 0x7fc4b41972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:42.191 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.184+0000 7fc4b94a0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4b40ff470 0x7fc4b41972b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41768/0 (socket says 172.21.15.1:41768) 2023-11-16T16:37:42.191 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.184+0000 7fc4b94a0700 1 -- 172.21.15.1:0/2380619830 learned_addr learned my addr 172.21.15.1:0/2380619830 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:42.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.184+0000 7fc4b8c9f700 1 --2- 172.21.15.1:0/2380619830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4b40ffe10 0x7fc4b41977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:42.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.184+0000 7fc4b9ca1700 1 --2- 172.21.15.1:0/2380619830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4b4104510 0x7fc4b41a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:42.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.184+0000 7fc4b94a0700 1 -- 172.21.15.1:0/2380619830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4b4104510 msgr2=0x7fc4b41a2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:42.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.184+0000 7fc4b94a0700 1 --2- 172.21.15.1:0/2380619830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4b4104510 0x7fc4b41a2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:42.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.184+0000 7fc4b94a0700 1 -- 172.21.15.1:0/2380619830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4b40ffe10 msgr2=0x7fc4b41977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:42.193 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.184+0000 7fc4b94a0700 1 --2- 172.21.15.1:0/2380619830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4b40ffe10 0x7fc4b41977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:42.193 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.184+0000 7fc4b94a0700 1 -- 172.21.15.1:0/2380619830 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b00095f0 con 0x7fc4b40ff470 2023-11-16T16:37:42.193 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.184+0000 7fc4b8c9f700 1 --2- 172.21.15.1:0/2380619830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4b40ffe10 0x7fc4b41977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:37:42.193 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.184+0000 7fc4b9ca1700 1 --2- 172.21.15.1:0/2380619830 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc4b4104510 0x7fc4b41a2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:37:42.194 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.184+0000 7fc4b94a0700 1 --2- 172.21.15.1:0/2380619830 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4b40ff470 0x7fc4b41972b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0009990 tx=0x7fc4b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:42.194 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.185+0000 7fc4aa7fc700 1 -- 172.21.15.1:0/2380619830 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b001d070 con 0x7fc4b40ff470 2023-11-16T16:37:42.194 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.185+0000 7fc4aa7fc700 1 -- 172.21.15.1:0/2380619830 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4b0004010 con 0x7fc4b40ff470 2023-11-16T16:37:42.194 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.185+0000 7fc4bb704700 1 -- 172.21.15.1:0/2380619830 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4b410f2a0 con 0x7fc4b40ff470 2023-11-16T16:37:42.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.185+0000 7fc4aa7fc700 1 -- 172.21.15.1:0/2380619830 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b000fd30 con 0x7fc4b40ff470 2023-11-16T16:37:42.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.185+0000 7fc4bb704700 1 -- 172.21.15.1:0/2380619830 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4b410f110 con 0x7fc4b40ff470 2023-11-16T16:37:42.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.186+0000 7fc4bb704700 1 -- 172.21.15.1:0/2380619830 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc4b40fd7f0 con 0x7fc4b40ff470 2023-11-16T16:37:42.197 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.187+0000 7fc4aa7fc700 1 -- 172.21.15.1:0/2380619830 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fc4b000f460 con 0x7fc4b40ff470 2023-11-16T16:37:42.197 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.188+0000 7fc4aa7fc700 1 --2- 172.21.15.1:0/2380619830 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc4a00a11e0 0x7fc4a00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:42.197 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.188+0000 7fc4b8c9f700 1 --2- 172.21.15.1:0/2380619830 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc4a00a11e0 0x7fc4a00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:42.198 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.188+0000 7fc4aa7fc700 1 -- 172.21.15.1:0/2380619830 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc4b00ce040 con 0x7fc4b40ff470 2023-11-16T16:37:42.198 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.189+0000 7fc4aa7fc700 1 --2- 172.21.15.1:0/2380619830 >> [v2:172.21.15.53:6800/1659316290,v1:172.21.15.53:6801/1659316290] conn(0x7fc4a00aa9b0 0x7fc4a00acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:42.198 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.189+0000 7fc4aa7fc700 1 -- 172.21.15.1:0/2380619830 --> [v2:172.21.15.53:6800/1659316290,v1:172.21.15.53:6801/1659316290] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc4a00ad470 con 0x7fc4a00aa9b0 2023-11-16T16:37:42.199 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.189+0000 7fc4aa7fc700 1 -- 172.21.15.1:0/2380619830 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4b00ce4e0 con 0x7fc4b40ff470 2023-11-16T16:37:42.199 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.189+0000 7fc4b8c9f700 1 --2- 172.21.15.1:0/2380619830 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc4a00a11e0 0x7fc4a00a3690 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4005fd0 tx=0x7fc4a4005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:42.199 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.189+0000 7fc4b9ca1700 1 --2- 172.21.15.1:0/2380619830 >> [v2:172.21.15.53:6800/1659316290,v1:172.21.15.53:6801/1659316290] conn(0x7fc4a00aa9b0 0x7fc4a00acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:42.199 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.189+0000 7fc4b9ca1700 1 --2- 172.21.15.1:0/2380619830 >> [v2:172.21.15.53:6800/1659316290,v1:172.21.15.53:6801/1659316290] conn(0x7fc4a00aa9b0 0x7fc4a00acdc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:42.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.190+0000 7fc4aa7fc700 1 -- 172.21.15.1:0/2380619830 <== osd.2 v2:172.21.15.53:6800/1659316290 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fc4a00ad470 con 0x7fc4a00aa9b0 2023-11-16T16:37:42.209 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.207+0000 7fc4bb704700 1 -- 172.21.15.1:0/2380619830 --> [v2:172.21.15.53:6800/1659316290,v1:172.21.15.53:6801/1659316290] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc4b410f430 con 0x7fc4a00aa9b0 2023-11-16T16:37:42.209 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.207+0000 7fc4aa7fc700 1 -- 172.21.15.1:0/2380619830 <== osd.2 v2:172.21.15.53:6800/1659316290 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc4b410f430 con 0x7fc4a00aa9b0 2023-11-16T16:37:42.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.208+0000 7fc4bb704700 1 -- 172.21.15.1:0/2380619830 >> [v2:172.21.15.53:6800/1659316290,v1:172.21.15.53:6801/1659316290] conn(0x7fc4a00aa9b0 msgr2=0x7fc4a00acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:42.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.208+0000 7fc4bb704700 1 --2- 172.21.15.1:0/2380619830 >> [v2:172.21.15.53:6800/1659316290,v1:172.21.15.53:6801/1659316290] conn(0x7fc4a00aa9b0 0x7fc4a00acdc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:42.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.208+0000 7fc4bb704700 1 -- 172.21.15.1:0/2380619830 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc4a00a11e0 msgr2=0x7fc4a00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:42.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.208+0000 7fc4bb704700 1 --2- 172.21.15.1:0/2380619830 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc4a00a11e0 0x7fc4a00a3690 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4005fd0 tx=0x7fc4a4005dc0 comp rx=0 tx=0).stop 2023-11-16T16:37:42.212 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.208+0000 7fc4bb704700 1 -- 172.21.15.1:0/2380619830 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4b40ff470 msgr2=0x7fc4b41972b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:42.212 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.208+0000 7fc4bb704700 1 --2- 172.21.15.1:0/2380619830 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4b40ff470 0x7fc4b41972b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0009990 tx=0x7fc4b00057f0 comp rx=0 tx=0).stop 2023-11-16T16:37:42.212 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.208+0000 7fc4b9ca1700 1 -- 172.21.15.1:0/2380619830 reap_dead start 2023-11-16T16:37:42.212 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.208+0000 7fc4bb704700 1 -- 172.21.15.1:0/2380619830 shutdown_connections 2023-11-16T16:37:42.212 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.208+0000 7fc4bb704700 1 -- 172.21.15.1:0/2380619830 >> 172.21.15.1:0/2380619830 conn(0x7fc4b40770d0 msgr2=0x7fc4b40fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:42.213 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.208+0000 7fc4bb704700 1 -- 172.21.15.1:0/2380619830 shutdown_connections 2023-11-16T16:37:42.213 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.208+0000 7fc4bb704700 1 -- 172.21.15.1:0/2380619830 wait complete. 2023-11-16T16:37:42.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:42 smithi026 ceph-mon[100112]: pgmap v274: 97 pgs: 97 active+clean; 636 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-11-16T16:37:42.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:42 smithi053 ceph-mon[100638]: pgmap v274: 97 pgs: 97 active+clean; 636 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-11-16T16:37:42.808 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.804+0000 7fcef6391700 1 -- 172.21.15.1:0/3068786032 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef00ffe00 msgr2=0x7fcef0103dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:42.809 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.804+0000 7fcef6391700 1 --2- 172.21.15.1:0/3068786032 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef00ffe00 0x7fcef0103dd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcee40099c0 tx=0x7fcee4009cd0 comp rx=0 tx=0).stop 2023-11-16T16:37:42.809 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.804+0000 7fcef6391700 1 -- 172.21.15.1:0/3068786032 shutdown_connections 2023-11-16T16:37:42.809 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.804+0000 7fcef6391700 1 --2- 172.21.15.1:0/3068786032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcef0104500 0x7fcef010c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:42.809 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.804+0000 7fcef6391700 1 --2- 172.21.15.1:0/3068786032 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcef00ff460 0x7fcef00ff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:42.809 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.804+0000 7fcef6391700 1 --2- 172.21.15.1:0/3068786032 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef00ffe00 0x7fcef0103dd0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:42.809 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.804+0000 7fcef6391700 1 -- 172.21.15.1:0/3068786032 >> 172.21.15.1:0/3068786032 conn(0x7fcef00770a0 msgr2=0x7fcef00774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:42.809 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.804+0000 7fcef6391700 1 -- 172.21.15.1:0/3068786032 shutdown_connections 2023-11-16T16:37:42.809 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.804+0000 7fcef6391700 1 -- 172.21.15.1:0/3068786032 wait complete. 2023-11-16T16:37:42.810 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.805+0000 7fcef6391700 1 Processor -- start 2023-11-16T16:37:42.810 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.805+0000 7fcef6391700 1 -- start start 2023-11-16T16:37:42.810 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.805+0000 7fcef6391700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcef00ff460 0x7fcef010c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:42.810 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.805+0000 7fcef6391700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcef00ffe00 0x7fcef010ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:42.810 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.805+0000 7fcef6391700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef0104500 0x7fcef010d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:42.810 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.805+0000 7fcef6391700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef01a4830 con 0x7fcef0104500 2023-11-16T16:37:42.810 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.805+0000 7fcef6391700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef01a49a0 con 0x7fcef00ffe00 2023-11-16T16:37:42.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.805+0000 7fcef6391700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef01a4b60 con 0x7fcef00ff460 2023-11-16T16:37:42.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.805+0000 7fcef492e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef0104500 0x7fcef010d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:42.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.806+0000 7fcef492e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef0104500 0x7fcef010d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41788/0 (socket says 172.21.15.1:41788) 2023-11-16T16:37:42.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.806+0000 7fcef492e700 1 -- 172.21.15.1:0/40697850 learned_addr learned my addr 172.21.15.1:0/40697850 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:42.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.806+0000 7fceeffff700 1 --2- 172.21.15.1:0/40697850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcef00ff460 0x7fcef010c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:42.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.806+0000 7fceef7fe700 1 --2- 172.21.15.1:0/40697850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcef00ffe00 0x7fcef010ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:42.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.806+0000 7fcef492e700 1 -- 172.21.15.1:0/40697850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcef00ff460 msgr2=0x7fcef010c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:42.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.806+0000 7fcef492e700 1 --2- 172.21.15.1:0/40697850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcef00ff460 0x7fcef010c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:42.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.806+0000 7fcef492e700 1 -- 172.21.15.1:0/40697850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcef00ffe00 msgr2=0x7fcef010ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:42.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.806+0000 7fcef492e700 1 --2- 172.21.15.1:0/40697850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcef00ffe00 0x7fcef010ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:42.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.806+0000 7fcef492e700 1 -- 172.21.15.1:0/40697850 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee40095f0 con 0x7fcef0104500 2023-11-16T16:37:42.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.806+0000 7fceeffff700 1 --2- 172.21.15.1:0/40697850 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcef00ff460 0x7fcef010c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:37:42.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.806+0000 7fcef492e700 1 --2- 172.21.15.1:0/40697850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef0104500 0x7fcef010d450 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcee800ef30 tx=0x7fcee800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:42.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.806+0000 7fceed7fa700 1 -- 172.21.15.1:0/40697850 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee8018070 con 0x7fcef0104500 2023-11-16T16:37:42.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.806+0000 7fcef6391700 1 -- 172.21.15.1:0/40697850 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef01103e0 con 0x7fcef0104500 2023-11-16T16:37:42.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.806+0000 7fceed7fa700 1 -- 172.21.15.1:0/40697850 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcee800f040 con 0x7fcef0104500 2023-11-16T16:37:42.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.806+0000 7fceed7fa700 1 -- 172.21.15.1:0/40697850 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee800cd00 con 0x7fcef0104500 2023-11-16T16:37:42.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.806+0000 7fcef6391700 1 -- 172.21.15.1:0/40697850 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef0110250 con 0x7fcef0104500 2023-11-16T16:37:42.814 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.807+0000 7fcef6391700 1 -- 172.21.15.1:0/40697850 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcef00fd6e0 con 0x7fcef0104500 2023-11-16T16:37:42.814 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.809+0000 7fceed7fa700 1 -- 172.21.15.1:0/40697850 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fcee8003a00 con 0x7fcef0104500 2023-11-16T16:37:42.815 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.809+0000 7fceed7fa700 1 --2- 172.21.15.1:0/40697850 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fced80a11e0 0x7fced80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:42.815 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.809+0000 7fceeffff700 1 --2- 172.21.15.1:0/40697850 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fced80a11e0 0x7fced80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:42.815 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.809+0000 7fceed7fa700 1 -- 172.21.15.1:0/40697850 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fcee80ccd30 con 0x7fcef0104500 2023-11-16T16:37:42.815 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.810+0000 7fceed7fa700 1 --2- 172.21.15.1:0/40697850 >> [v2:172.21.15.26:6824/1773545447,v1:172.21.15.26:6825/1773545447] conn(0x7fced80aa9b0 0x7fced80acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:42.815 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.810+0000 7fceed7fa700 1 -- 172.21.15.1:0/40697850 --> [v2:172.21.15.26:6824/1773545447,v1:172.21.15.26:6825/1773545447] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fced80ad470 con 0x7fced80aa9b0 2023-11-16T16:37:42.815 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.810+0000 7fceed7fa700 1 -- 172.21.15.1:0/40697850 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcee80cd1d0 con 0x7fcef0104500 2023-11-16T16:37:42.815 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.810+0000 7fceeffff700 1 --2- 172.21.15.1:0/40697850 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fced80a11e0 0x7fced80a3690 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcee0011490 tx=0x7fcee0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:42.815 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.810+0000 7fceef7fe700 1 --2- 172.21.15.1:0/40697850 >> [v2:172.21.15.26:6824/1773545447,v1:172.21.15.26:6825/1773545447] conn(0x7fced80aa9b0 0x7fced80acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:42.815 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.810+0000 7fceef7fe700 1 --2- 172.21.15.1:0/40697850 >> [v2:172.21.15.26:6824/1773545447,v1:172.21.15.26:6825/1773545447] conn(0x7fced80aa9b0 0x7fced80acdc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:42.815 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.811+0000 7fceed7fa700 1 -- 172.21.15.1:0/40697850 <== osd.10 v2:172.21.15.26:6824/1773545447 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fced80ad470 con 0x7fced80aa9b0 2023-11-16T16:37:42.830 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.828+0000 7fcef6391700 1 -- 172.21.15.1:0/40697850 --> [v2:172.21.15.26:6824/1773545447,v1:172.21.15.26:6825/1773545447] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcef0110570 con 0x7fced80aa9b0 2023-11-16T16:37:42.830 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.828+0000 7fceed7fa700 1 -- 172.21.15.1:0/40697850 <== osd.10 v2:172.21.15.26:6824/1773545447 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fcef0110570 con 0x7fced80aa9b0 2023-11-16T16:37:42.831 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.829+0000 7fcef6391700 1 -- 172.21.15.1:0/40697850 >> [v2:172.21.15.26:6824/1773545447,v1:172.21.15.26:6825/1773545447] conn(0x7fced80aa9b0 msgr2=0x7fced80acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:42.831 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.829+0000 7fcef6391700 1 --2- 172.21.15.1:0/40697850 >> [v2:172.21.15.26:6824/1773545447,v1:172.21.15.26:6825/1773545447] conn(0x7fced80aa9b0 0x7fced80acdc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:42.831 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.829+0000 7fcef6391700 1 -- 172.21.15.1:0/40697850 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fced80a11e0 msgr2=0x7fced80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:42.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.829+0000 7fcef6391700 1 --2- 172.21.15.1:0/40697850 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fced80a11e0 0x7fced80a3690 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcee0011490 tx=0x7fcee0009260 comp rx=0 tx=0).stop 2023-11-16T16:37:42.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.829+0000 7fcef6391700 1 -- 172.21.15.1:0/40697850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef0104500 msgr2=0x7fcef010d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:42.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.829+0000 7fcef6391700 1 --2- 172.21.15.1:0/40697850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef0104500 0x7fcef010d450 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcee800ef30 tx=0x7fcee800c3a0 comp rx=0 tx=0).stop 2023-11-16T16:37:42.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.829+0000 7fcef492e700 1 -- 172.21.15.1:0/40697850 reap_dead start 2023-11-16T16:37:42.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.829+0000 7fcef6391700 1 -- 172.21.15.1:0/40697850 shutdown_connections 2023-11-16T16:37:42.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.829+0000 7fcef6391700 1 -- 172.21.15.1:0/40697850 >> 172.21.15.1:0/40697850 conn(0x7fcef00770a0 msgr2=0x7fcef00fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:42.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.829+0000 7fcef6391700 1 -- 172.21.15.1:0/40697850 shutdown_connections 2023-11-16T16:37:42.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:42.829+0000 7fcef6391700 1 -- 172.21.15.1:0/40697850 wait complete. 2023-11-16T16:37:43.046 INFO:teuthology.orchestra.run.smithi001.stdout:167503724588 2023-11-16T16:37:43.047 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph osd last-stat-seq osd.5 2023-11-16T16:37:43.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.024+0000 7fa0ec34c700 1 -- 172.21.15.1:0/1293671290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0e4101cf0 msgr2=0x7fa0e410a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:43.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.024+0000 7fa0ec34c700 1 --2- 172.21.15.1:0/1293671290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0e4101cf0 0x7fa0e410a3c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4009990 tx=0x7fa0d4009ca0 comp rx=0 tx=0).stop 2023-11-16T16:37:43.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.025+0000 7fa0ec34c700 1 -- 172.21.15.1:0/1293671290 shutdown_connections 2023-11-16T16:37:43.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.025+0000 7fa0ec34c700 1 --2- 172.21.15.1:0/1293671290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0e410a900 0x7fa0e410cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:43.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.025+0000 7fa0ec34c700 1 --2- 172.21.15.1:0/1293671290 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0e41013e0 0x7fa0e41017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:43.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.025+0000 7fa0ec34c700 1 --2- 172.21.15.1:0/1293671290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0e4101cf0 0x7fa0e410a3c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:43.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.025+0000 7fa0ec34c700 1 -- 172.21.15.1:0/1293671290 >> 172.21.15.1:0/1293671290 conn(0x7fa0e40770d0 msgr2=0x7fa0e40774d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:43.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.025+0000 7fa0ec34c700 1 -- 172.21.15.1:0/1293671290 shutdown_connections 2023-11-16T16:37:43.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.025+0000 7fa0ec34c700 1 -- 172.21.15.1:0/1293671290 wait complete. 2023-11-16T16:37:43.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.025+0000 7fa0ec34c700 1 Processor -- start 2023-11-16T16:37:43.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.025+0000 7fa0ec34c700 1 -- start start 2023-11-16T16:37:43.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.026+0000 7fa0ec34c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0e41013e0 0x7fa0e4198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:43.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.026+0000 7fa0ec34c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0e4101cf0 0x7fa0e4198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:43.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.026+0000 7fa0ec34c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0e410a900 0x7fa0e419cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:43.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.026+0000 7fa0ec34c700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e419d5c0 con 0x7fa0e41013e0 2023-11-16T16:37:43.557 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.026+0000 7fa0ec34c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e419d730 con 0x7fa0e410a900 2023-11-16T16:37:43.557 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.026+0000 7fa0ec34c700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e419d8f0 con 0x7fa0e4101cf0 2023-11-16T16:37:43.557 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.026+0000 7fa0ea0e8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0e41013e0 0x7fa0e4198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:43.557 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.026+0000 7fa0ea0e8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0e41013e0 0x7fa0e4198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41800/0 (socket says 172.21.15.1:41800) 2023-11-16T16:37:43.557 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.026+0000 7fa0ea0e8700 1 -- 172.21.15.1:0/2398497991 learned_addr learned my addr 172.21.15.1:0/2398497991 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:43.557 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.026+0000 7fa0e98e7700 1 --2- 172.21.15.1:0/2398497991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0e4101cf0 0x7fa0e4198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:43.558 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.026+0000 7fa0ea8e9700 1 --2- 172.21.15.1:0/2398497991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0e410a900 0x7fa0e419cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:43.558 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.026+0000 7fa0ea0e8700 1 -- 172.21.15.1:0/2398497991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0e4101cf0 msgr2=0x7fa0e4198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:43.558 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.026+0000 7fa0ea0e8700 1 --2- 172.21.15.1:0/2398497991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0e4101cf0 0x7fa0e4198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:43.558 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.026+0000 7fa0ea0e8700 1 -- 172.21.15.1:0/2398497991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0e410a900 msgr2=0x7fa0e419cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:43.558 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.026+0000 7fa0ea0e8700 1 --2- 172.21.15.1:0/2398497991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0e410a900 0x7fa0e419cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:43.558 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.026+0000 7fa0ea0e8700 1 -- 172.21.15.1:0/2398497991 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d40095f0 con 0x7fa0e41013e0 2023-11-16T16:37:43.558 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.026+0000 7fa0e98e7700 1 --2- 172.21.15.1:0/2398497991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa0e4101cf0 0x7fa0e4198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:37:43.559 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.026+0000 7fa0ea0e8700 1 --2- 172.21.15.1:0/2398497991 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0e41013e0 0x7fa0e4198600 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa0e000d950 tx=0x7fa0e000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:43.559 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.026+0000 7fa0db7fe700 1 -- 172.21.15.1:0/2398497991 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e0009b40 con 0x7fa0e41013e0 2023-11-16T16:37:43.559 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.027+0000 7fa0ec34c700 1 -- 172.21.15.1:0/2398497991 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0e4100910 con 0x7fa0e41013e0 2023-11-16T16:37:43.559 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.027+0000 7fa0db7fe700 1 -- 172.21.15.1:0/2398497991 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0e0009ca0 con 0x7fa0e41013e0 2023-11-16T16:37:43.559 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.027+0000 7fa0db7fe700 1 -- 172.21.15.1:0/2398497991 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e000f620 con 0x7fa0e41013e0 2023-11-16T16:37:43.559 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.027+0000 7fa0ec34c700 1 -- 172.21.15.1:0/2398497991 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0e4100780 con 0x7fa0e41013e0 2023-11-16T16:37:43.559 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.027+0000 7fa0ec34c700 1 -- 172.21.15.1:0/2398497991 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa0e4061270 con 0x7fa0e41013e0 2023-11-16T16:37:43.559 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.029+0000 7fa0db7fe700 1 -- 172.21.15.1:0/2398497991 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fa0e0010460 con 0x7fa0e41013e0 2023-11-16T16:37:43.559 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.029+0000 7fa0db7fe700 1 --2- 172.21.15.1:0/2398497991 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fa0d00a11e0 0x7fa0d00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:43.560 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.030+0000 7fa0db7fe700 1 -- 172.21.15.1:0/2398497991 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa0e00ccfb0 con 0x7fa0e41013e0 2023-11-16T16:37:43.560 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.030+0000 7fa0e98e7700 1 --2- 172.21.15.1:0/2398497991 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fa0d00a11e0 0x7fa0d00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:43.560 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.030+0000 7fa0db7fe700 1 --2- 172.21.15.1:0/2398497991 >> [v2:172.21.15.53:6816/3224335871,v1:172.21.15.53:6817/3224335871] conn(0x7fa0d00aa9b0 0x7fa0d00acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:43.560 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.030+0000 7fa0db7fe700 1 -- 172.21.15.1:0/2398497991 --> [v2:172.21.15.53:6816/3224335871,v1:172.21.15.53:6817/3224335871] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa0d00ad470 con 0x7fa0d00aa9b0 2023-11-16T16:37:43.560 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.030+0000 7fa0db7fe700 1 -- 172.21.15.1:0/2398497991 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0e00cd320 con 0x7fa0e41013e0 2023-11-16T16:37:43.560 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.030+0000 7fa0e98e7700 1 --2- 172.21.15.1:0/2398497991 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fa0d00a11e0 0x7fa0d00a3690 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4006010 tx=0x7fa0d400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:43.560 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.030+0000 7fa0ea8e9700 1 --2- 172.21.15.1:0/2398497991 >> [v2:172.21.15.53:6816/3224335871,v1:172.21.15.53:6817/3224335871] conn(0x7fa0d00aa9b0 0x7fa0d00acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:43.560 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.031+0000 7fa0ea8e9700 1 --2- 172.21.15.1:0/2398497991 >> [v2:172.21.15.53:6816/3224335871,v1:172.21.15.53:6817/3224335871] conn(0x7fa0d00aa9b0 0x7fa0d00acdc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:43.560 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.031+0000 7fa0db7fe700 1 -- 172.21.15.1:0/2398497991 <== osd.8 v2:172.21.15.53:6816/3224335871 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa0d00ad470 con 0x7fa0d00aa9b0 2023-11-16T16:37:43.560 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.048+0000 7fa0ec34c700 1 -- 172.21.15.1:0/2398497991 --> [v2:172.21.15.53:6816/3224335871,v1:172.21.15.53:6817/3224335871] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa0e4100aa0 con 0x7fa0d00aa9b0 2023-11-16T16:37:43.561 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.048+0000 7fa0db7fe700 1 -- 172.21.15.1:0/2398497991 <== osd.8 v2:172.21.15.53:6816/3224335871 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa0e4100aa0 con 0x7fa0d00aa9b0 2023-11-16T16:37:43.561 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.049+0000 7fa0ec34c700 1 -- 172.21.15.1:0/2398497991 >> [v2:172.21.15.53:6816/3224335871,v1:172.21.15.53:6817/3224335871] conn(0x7fa0d00aa9b0 msgr2=0x7fa0d00acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:43.561 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.049+0000 7fa0ec34c700 1 --2- 172.21.15.1:0/2398497991 >> [v2:172.21.15.53:6816/3224335871,v1:172.21.15.53:6817/3224335871] conn(0x7fa0d00aa9b0 0x7fa0d00acdc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:43.561 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.049+0000 7fa0ec34c700 1 -- 172.21.15.1:0/2398497991 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fa0d00a11e0 msgr2=0x7fa0d00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:43.561 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.049+0000 7fa0ec34c700 1 --2- 172.21.15.1:0/2398497991 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fa0d00a11e0 0x7fa0d00a3690 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4006010 tx=0x7fa0d400b540 comp rx=0 tx=0).stop 2023-11-16T16:37:43.561 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.049+0000 7fa0ec34c700 1 -- 172.21.15.1:0/2398497991 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0e41013e0 msgr2=0x7fa0e4198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:43.561 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.049+0000 7fa0ec34c700 1 --2- 172.21.15.1:0/2398497991 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0e41013e0 0x7fa0e4198600 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa0e000d950 tx=0x7fa0e000dd10 comp rx=0 tx=0).stop 2023-11-16T16:37:43.561 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.049+0000 7fa0ea8e9700 1 -- 172.21.15.1:0/2398497991 reap_dead start 2023-11-16T16:37:43.561 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.049+0000 7fa0ec34c700 1 -- 172.21.15.1:0/2398497991 shutdown_connections 2023-11-16T16:37:43.561 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.049+0000 7fa0ec34c700 1 -- 172.21.15.1:0/2398497991 >> 172.21.15.1:0/2398497991 conn(0x7fa0e40770d0 msgr2=0x7fa0e4102360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:43.562 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.049+0000 7fa0ec34c700 1 -- 172.21.15.1:0/2398497991 shutdown_connections 2023-11-16T16:37:43.562 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.049+0000 7fa0ec34c700 1 -- 172.21.15.1:0/2398497991 wait complete. 2023-11-16T16:37:43.565 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.563+0000 7f1e1c3fc700 1 -- 172.21.15.1:0/2089241834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e14104500 msgr2=0x7f1e1410c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:43.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.563+0000 7f1e1c3fc700 1 --2- 172.21.15.1:0/2089241834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e14104500 0x7f1e1410c080 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c00b0d0 tx=0x7f1e0c00b3e0 comp rx=0 tx=0).stop 2023-11-16T16:37:43.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.563+0000 7f1e1c3fc700 1 -- 172.21.15.1:0/2089241834 shutdown_connections 2023-11-16T16:37:43.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.563+0000 7f1e1c3fc700 1 --2- 172.21.15.1:0/2089241834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e14104500 0x7f1e1410c080 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:43.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.563+0000 7f1e1c3fc700 1 --2- 172.21.15.1:0/2089241834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e140ff460 0x7f1e140ff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:43.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.563+0000 7f1e1c3fc700 1 --2- 172.21.15.1:0/2089241834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e140ffe00 0x7f1e14103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:43.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.563+0000 7f1e1c3fc700 1 -- 172.21.15.1:0/2089241834 >> 172.21.15.1:0/2089241834 conn(0x7f1e140770a0 msgr2=0x7f1e140774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:43.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.564+0000 7f1e1c3fc700 1 -- 172.21.15.1:0/2089241834 shutdown_connections 2023-11-16T16:37:43.567 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.564+0000 7f1e1c3fc700 1 -- 172.21.15.1:0/2089241834 wait complete. 2023-11-16T16:37:43.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.564+0000 7f1e1c3fc700 1 Processor -- start 2023-11-16T16:37:43.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.564+0000 7f1e1c3fc700 1 -- start start 2023-11-16T16:37:43.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.564+0000 7f1e1c3fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e140ff460 0x7f1e141964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:43.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.564+0000 7f1e1c3fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e140ffe00 0x7f1e14196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:43.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.564+0000 7f1e1c3fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e14104500 0x7f1e1419ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:43.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.564+0000 7f1e1c3fc700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1419b490 con 0x7f1e140ffe00 2023-11-16T16:37:43.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.565+0000 7f1e1c3fc700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1419b600 con 0x7f1e14104500 2023-11-16T16:37:43.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.565+0000 7f1e1c3fc700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1419b7c0 con 0x7f1e140ff460 2023-11-16T16:37:43.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.565+0000 7f1e19997700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e140ffe00 0x7f1e14196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:43.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.565+0000 7f1e19997700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e140ffe00 0x7f1e14196a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41818/0 (socket says 172.21.15.1:41818) 2023-11-16T16:37:43.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.565+0000 7f1e19997700 1 -- 172.21.15.1:0/3985936152 learned_addr learned my addr 172.21.15.1:0/3985936152 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:43.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.565+0000 7f1e1a198700 1 --2- 172.21.15.1:0/3985936152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e140ff460 0x7f1e141964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:43.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.565+0000 7f1e1a999700 1 --2- 172.21.15.1:0/3985936152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e14104500 0x7f1e1419ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:43.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.565+0000 7f1e19997700 1 -- 172.21.15.1:0/3985936152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e140ff460 msgr2=0x7f1e141964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:43.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.565+0000 7f1e19997700 1 --2- 172.21.15.1:0/3985936152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1e140ff460 0x7f1e141964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:43.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.565+0000 7f1e19997700 1 -- 172.21.15.1:0/3985936152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e14104500 msgr2=0x7f1e1419ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:43.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.565+0000 7f1e19997700 1 --2- 172.21.15.1:0/3985936152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e14104500 0x7f1e1419ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:43.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.565+0000 7f1e19997700 1 -- 172.21.15.1:0/3985936152 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0c009d00 con 0x7f1e140ffe00 2023-11-16T16:37:43.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.565+0000 7f1e19997700 1 --2- 172.21.15.1:0/3985936152 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e140ffe00 0x7f1e14196a10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1e0400c8f0 tx=0x7f1e0400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:43.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.565+0000 7f1e0b7fe700 1 -- 172.21.15.1:0/3985936152 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e04007940 con 0x7f1e140ffe00 2023-11-16T16:37:43.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.565+0000 7f1e0b7fe700 1 -- 172.21.15.1:0/3985936152 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e0400f450 con 0x7f1e140ffe00 2023-11-16T16:37:43.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.565+0000 7f1e1c3fc700 1 -- 172.21.15.1:0/3985936152 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e141103b0 con 0x7f1e140ffe00 2023-11-16T16:37:43.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.565+0000 7f1e0b7fe700 1 -- 172.21.15.1:0/3985936152 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e04018660 con 0x7f1e140ffe00 2023-11-16T16:37:43.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.565+0000 7f1e1c3fc700 1 -- 172.21.15.1:0/3985936152 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e14110220 con 0x7f1e140ffe00 2023-11-16T16:37:43.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.566+0000 7f1e1c3fc700 1 -- 172.21.15.1:0/3985936152 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1e140fd630 con 0x7f1e140ffe00 2023-11-16T16:37:43.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.567+0000 7f1e0b7fe700 1 -- 172.21.15.1:0/3985936152 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f1e0400f5c0 con 0x7f1e140ffe00 2023-11-16T16:37:43.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.568+0000 7f1e0b7fe700 1 --2- 172.21.15.1:0/3985936152 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f1e000a11e0 0x7f1e000a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:43.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.568+0000 7f1e1a198700 1 --2- 172.21.15.1:0/3985936152 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f1e000a11e0 0x7f1e000a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:43.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.568+0000 7f1e0b7fe700 1 -- 172.21.15.1:0/3985936152 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1e040ccaf0 con 0x7f1e140ffe00 2023-11-16T16:37:43.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.568+0000 7f1e0b7fe700 1 --2- 172.21.15.1:0/3985936152 >> [v2:172.21.15.1:6826/1391199844,v1:172.21.15.1:6827/1391199844] conn(0x7f1e000aa9b0 0x7f1e000acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:43.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.568+0000 7f1e0b7fe700 1 -- 172.21.15.1:0/3985936152 --> [v2:172.21.15.1:6826/1391199844,v1:172.21.15.1:6827/1391199844] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1e000ad470 con 0x7f1e000aa9b0 2023-11-16T16:37:43.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.568+0000 7f1e0b7fe700 1 -- 172.21.15.1:0/3985936152 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e040cce60 con 0x7f1e140ffe00 2023-11-16T16:37:43.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.568+0000 7f1e1a999700 1 --2- 172.21.15.1:0/3985936152 >> [v2:172.21.15.1:6826/1391199844,v1:172.21.15.1:6827/1391199844] conn(0x7f1e000aa9b0 0x7f1e000acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:43.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.568+0000 7f1e1a198700 1 --2- 172.21.15.1:0/3985936152 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f1e000a11e0 0x7f1e000a3690 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1e10005950 tx=0x7f1e1000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:43.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.569+0000 7f1e1a999700 1 --2- 172.21.15.1:0/3985936152 >> [v2:172.21.15.1:6826/1391199844,v1:172.21.15.1:6827/1391199844] conn(0x7f1e000aa9b0 0x7f1e000acdc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:43.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.569+0000 7f1e0b7fe700 1 -- 172.21.15.1:0/3985936152 <== osd.9 v2:172.21.15.1:6826/1391199844 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f1e000ad470 con 0x7f1e000aa9b0 2023-11-16T16:37:43.590 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.588+0000 7f1e1c3fc700 1 -- 172.21.15.1:0/3985936152 --> [v2:172.21.15.1:6826/1391199844,v1:172.21.15.1:6827/1391199844] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1e14110540 con 0x7f1e000aa9b0 2023-11-16T16:37:43.590 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.588+0000 7f1e0b7fe700 1 -- 172.21.15.1:0/3985936152 <== osd.9 v2:172.21.15.1:6826/1391199844 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1e14110540 con 0x7f1e000aa9b0 2023-11-16T16:37:43.590 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.588+0000 7f1e1c3fc700 1 -- 172.21.15.1:0/3985936152 >> [v2:172.21.15.1:6826/1391199844,v1:172.21.15.1:6827/1391199844] conn(0x7f1e000aa9b0 msgr2=0x7f1e000acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:43.591 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.588+0000 7f1e1c3fc700 1 --2- 172.21.15.1:0/3985936152 >> [v2:172.21.15.1:6826/1391199844,v1:172.21.15.1:6827/1391199844] conn(0x7f1e000aa9b0 0x7f1e000acdc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:43.591 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.589+0000 7f1e1c3fc700 1 -- 172.21.15.1:0/3985936152 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f1e000a11e0 msgr2=0x7f1e000a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:43.592 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.589+0000 7f1e1c3fc700 1 --2- 172.21.15.1:0/3985936152 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f1e000a11e0 0x7f1e000a3690 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1e10005950 tx=0x7f1e1000b410 comp rx=0 tx=0).stop 2023-11-16T16:37:43.592 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.589+0000 7f1e1c3fc700 1 -- 172.21.15.1:0/3985936152 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e140ffe00 msgr2=0x7f1e14196a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:43.592 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.589+0000 7f1e1c3fc700 1 --2- 172.21.15.1:0/3985936152 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1e140ffe00 0x7f1e14196a10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1e0400c8f0 tx=0x7f1e0400cc00 comp rx=0 tx=0).stop 2023-11-16T16:37:43.592 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.589+0000 7f1e1a999700 1 -- 172.21.15.1:0/3985936152 reap_dead start 2023-11-16T16:37:43.592 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.589+0000 7f1e1c3fc700 1 -- 172.21.15.1:0/3985936152 shutdown_connections 2023-11-16T16:37:43.592 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.589+0000 7f1e1c3fc700 1 -- 172.21.15.1:0/3985936152 >> 172.21.15.1:0/3985936152 conn(0x7f1e140770a0 msgr2=0x7f1e140fcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:43.592 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.589+0000 7f1e1c3fc700 1 -- 172.21.15.1:0/3985936152 shutdown_connections 2023-11-16T16:37:43.592 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.589+0000 7f1e1c3fc700 1 -- 172.21.15.1:0/3985936152 wait complete. 2023-11-16T16:37:43.726 INFO:teuthology.orchestra.run.smithi001.stdout:77309411385 2023-11-16T16:37:43.726 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph osd last-stat-seq osd.2 2023-11-16T16:37:43.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:43 smithi053 ceph-mon[100638]: pgmap v275: 97 pgs: 97 active+clean; 636 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 5 op/s 2023-11-16T16:37:43.873 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:43 smithi001 ceph-mon[98800]: pgmap v275: 97 pgs: 97 active+clean; 636 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 5 op/s 2023-11-16T16:37:44.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:43 smithi026 ceph-mon[100112]: pgmap v275: 97 pgs: 97 active+clean; 636 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 5 op/s 2023-11-16T16:37:44.018 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.016+0000 7f555dba8700 1 -- 172.21.15.1:0/3348667458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55580ff470 msgr2=0x7f55580ff840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:44.019 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.016+0000 7f555dba8700 1 --2- 172.21.15.1:0/3348667458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55580ff470 0x7f55580ff840 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f55480099c0 tx=0x7f5548009cd0 comp rx=0 tx=0).stop 2023-11-16T16:37:44.019 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.016+0000 7f555dba8700 1 -- 172.21.15.1:0/3348667458 shutdown_connections 2023-11-16T16:37:44.019 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.016+0000 7f555dba8700 1 --2- 172.21.15.1:0/3348667458 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5558104510 0x7f555810c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:44.019 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.016+0000 7f555dba8700 1 --2- 172.21.15.1:0/3348667458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55580ff470 0x7f55580ff840 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:44.019 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.016+0000 7f555dba8700 1 --2- 172.21.15.1:0/3348667458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55580ffe10 0x7f5558103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:44.019 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.016+0000 7f555dba8700 1 -- 172.21.15.1:0/3348667458 >> 172.21.15.1:0/3348667458 conn(0x7f55580770d0 msgr2=0x7f55580774d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:44.019 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.016+0000 7f555dba8700 1 -- 172.21.15.1:0/3348667458 shutdown_connections 2023-11-16T16:37:44.019 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.016+0000 7f555dba8700 1 -- 172.21.15.1:0/3348667458 wait complete. 2023-11-16T16:37:44.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.017+0000 7f555dba8700 1 Processor -- start 2023-11-16T16:37:44.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.017+0000 7f555dba8700 1 -- start start 2023-11-16T16:37:44.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.018+0000 7f555dba8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55580ff470 0x7f5558194160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:44.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.018+0000 7f555dba8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55580ffe10 0x7f55581946a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:44.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.018+0000 7f555dba8700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5558104510 0x7f5558198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:44.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.018+0000 7f555dba8700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5558199120 con 0x7f55580ffe10 2023-11-16T16:37:44.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.018+0000 7f555dba8700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5558199290 con 0x7f55580ff470 2023-11-16T16:37:44.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.018+0000 7f555dba8700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5558199450 con 0x7f5558104510 2023-11-16T16:37:44.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.018+0000 7f5556ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55580ffe10 0x7f55581946a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:44.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.018+0000 7f5556ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55580ffe10 0x7f55581946a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41850/0 (socket says 172.21.15.1:41850) 2023-11-16T16:37:44.023 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.018+0000 7f5556ffd700 1 -- 172.21.15.1:0/3519371328 learned_addr learned my addr 172.21.15.1:0/3519371328 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:44.023 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.018+0000 7f55577fe700 1 --2- 172.21.15.1:0/3519371328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55580ff470 0x7f5558194160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:44.023 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.018+0000 7f5557fff700 1 --2- 172.21.15.1:0/3519371328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5558104510 0x7f5558198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:44.023 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.018+0000 7f5556ffd700 1 -- 172.21.15.1:0/3519371328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5558104510 msgr2=0x7f5558198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:44.023 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.018+0000 7f5556ffd700 1 --2- 172.21.15.1:0/3519371328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5558104510 0x7f5558198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:44.023 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.018+0000 7f5556ffd700 1 -- 172.21.15.1:0/3519371328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55580ff470 msgr2=0x7f5558194160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:44.023 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.018+0000 7f5556ffd700 1 --2- 172.21.15.1:0/3519371328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55580ff470 0x7f5558194160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:44.023 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.018+0000 7f5556ffd700 1 -- 172.21.15.1:0/3519371328 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55480095f0 con 0x7f55580ffe10 2023-11-16T16:37:44.023 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.018+0000 7f5557fff700 1 --2- 172.21.15.1:0/3519371328 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5558104510 0x7f5558198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:37:44.023 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.019+0000 7f5556ffd700 1 --2- 172.21.15.1:0/3519371328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55580ffe10 0x7f55581946a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f554c00e970 tx=0x7f554c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:44.024 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.019+0000 7f5554ff9700 1 -- 172.21.15.1:0/3519371328 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554c009790 con 0x7f55580ffe10 2023-11-16T16:37:44.024 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.019+0000 7f5554ff9700 1 -- 172.21.15.1:0/3519371328 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f554c00eec0 con 0x7f55580ffe10 2023-11-16T16:37:44.024 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.019+0000 7f5554ff9700 1 -- 172.21.15.1:0/3519371328 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554c005590 con 0x7f55580ffe10 2023-11-16T16:37:44.024 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.019+0000 7f555dba8700 1 -- 172.21.15.1:0/3519371328 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f555810e200 con 0x7f55580ffe10 2023-11-16T16:37:44.024 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.019+0000 7f555dba8700 1 -- 172.21.15.1:0/3519371328 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f555810e070 con 0x7f55580ffe10 2023-11-16T16:37:44.024 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.020+0000 7f555dba8700 1 -- 172.21.15.1:0/3519371328 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f55580fd6c0 con 0x7f55580ffe10 2023-11-16T16:37:44.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.021+0000 7f5554ff9700 1 -- 172.21.15.1:0/3519371328 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f554c00cbd0 con 0x7f55580ffe10 2023-11-16T16:37:44.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.022+0000 7f5554ff9700 1 --2- 172.21.15.1:0/3519371328 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f55400a34f0 0x7f55400a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:44.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.022+0000 7f5554ff9700 1 -- 172.21.15.1:0/3519371328 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f554c014070 con 0x7f55580ffe10 2023-11-16T16:37:44.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.022+0000 7f55577fe700 1 --2- 172.21.15.1:0/3519371328 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f55400a34f0 0x7f55400a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:44.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.022+0000 7f5554ff9700 1 --2- 172.21.15.1:0/3519371328 >> [v2:172.21.15.1:6810/2728987281,v1:172.21.15.1:6811/2728987281] conn(0x7f55400accc0 0x7f55400af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:44.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.022+0000 7f5554ff9700 1 -- 172.21.15.1:0/3519371328 --> [v2:172.21.15.1:6810/2728987281,v1:172.21.15.1:6811/2728987281] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f55400af780 con 0x7f55400accc0 2023-11-16T16:37:44.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.022+0000 7f5554ff9700 1 -- 172.21.15.1:0/3519371328 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f554c0d12a0 con 0x7f55580ffe10 2023-11-16T16:37:44.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.022+0000 7f5557fff700 1 --2- 172.21.15.1:0/3519371328 >> [v2:172.21.15.1:6810/2728987281,v1:172.21.15.1:6811/2728987281] conn(0x7f55400accc0 0x7f55400af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:44.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.023+0000 7f55577fe700 1 --2- 172.21.15.1:0/3519371328 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f55400a34f0 0x7f55400a59a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5548009990 tx=0x7f5548005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:44.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.023+0000 7f5557fff700 1 --2- 172.21.15.1:0/3519371328 >> [v2:172.21.15.1:6810/2728987281,v1:172.21.15.1:6811/2728987281] conn(0x7f55400accc0 0x7f55400af0d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:44.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.024+0000 7f5554ff9700 1 -- 172.21.15.1:0/3519371328 <== osd.3 v2:172.21.15.1:6810/2728987281 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f55400af780 con 0x7f55400accc0 2023-11-16T16:37:44.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.043+0000 7f555dba8700 1 -- 172.21.15.1:0/3519371328 --> [v2:172.21.15.1:6810/2728987281,v1:172.21.15.1:6811/2728987281] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f555810e390 con 0x7f55400accc0 2023-11-16T16:37:44.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.043+0000 7f5554ff9700 1 -- 172.21.15.1:0/3519371328 <== osd.3 v2:172.21.15.1:6810/2728987281 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f555810e390 con 0x7f55400accc0 2023-11-16T16:37:44.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.044+0000 7f555dba8700 1 -- 172.21.15.1:0/3519371328 >> [v2:172.21.15.1:6810/2728987281,v1:172.21.15.1:6811/2728987281] conn(0x7f55400accc0 msgr2=0x7f55400af0d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:44.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.044+0000 7f555dba8700 1 --2- 172.21.15.1:0/3519371328 >> [v2:172.21.15.1:6810/2728987281,v1:172.21.15.1:6811/2728987281] conn(0x7f55400accc0 0x7f55400af0d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:44.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.044+0000 7f555dba8700 1 -- 172.21.15.1:0/3519371328 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f55400a34f0 msgr2=0x7f55400a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:44.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.044+0000 7f555dba8700 1 --2- 172.21.15.1:0/3519371328 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f55400a34f0 0x7f55400a59a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5548009990 tx=0x7f5548005fb0 comp rx=0 tx=0).stop 2023-11-16T16:37:44.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.044+0000 7f555dba8700 1 -- 172.21.15.1:0/3519371328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55580ffe10 msgr2=0x7f55581946a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:44.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.044+0000 7f555dba8700 1 --2- 172.21.15.1:0/3519371328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55580ffe10 0x7f55581946a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f554c00e970 tx=0x7f554c00ec80 comp rx=0 tx=0).stop 2023-11-16T16:37:44.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.044+0000 7f5557fff700 1 -- 172.21.15.1:0/3519371328 reap_dead start 2023-11-16T16:37:44.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.044+0000 7f555dba8700 1 -- 172.21.15.1:0/3519371328 shutdown_connections 2023-11-16T16:37:44.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.044+0000 7f555dba8700 1 -- 172.21.15.1:0/3519371328 >> 172.21.15.1:0/3519371328 conn(0x7f55580770d0 msgr2=0x7f55580fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:44.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.044+0000 7f555dba8700 1 -- 172.21.15.1:0/3519371328 shutdown_connections 2023-11-16T16:37:44.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:44.044+0000 7f555dba8700 1 -- 172.21.15.1:0/3519371328 wait complete. 2023-11-16T16:37:44.062 INFO:teuthology.orchestra.run.smithi001.stdout:253403070495 2023-11-16T16:37:44.063 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph osd last-stat-seq osd.8 2023-11-16T16:37:44.103 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.820+0000 7fe48471a700 1 -- 172.21.15.1:0/2742531230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe47c0ff140 msgr2=0x7fe47c0ff510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:44.103 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.820+0000 7fe48471a700 1 --2- 172.21.15.1:0/2742531230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe47c0ff140 0x7fe47c0ff510 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe478009990 tx=0x7fe478009ca0 comp rx=0 tx=0).stop 2023-11-16T16:37:44.103 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.820+0000 7fe48471a700 1 -- 172.21.15.1:0/2742531230 shutdown_connections 2023-11-16T16:37:44.103 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.820+0000 7fe48471a700 1 --2- 172.21.15.1:0/2742531230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe47c10ccf0 0x7fe47c10f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:44.103 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.820+0000 7fe48471a700 1 --2- 172.21.15.1:0/2742531230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe47c0ff140 0x7fe47c0ff510 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:44.103 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.820+0000 7fe48471a700 1 --2- 172.21.15.1:0/2742531230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe47c0ffa50 0x7fe47c10c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:44.103 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.820+0000 7fe48471a700 1 -- 172.21.15.1:0/2742531230 >> 172.21.15.1:0/2742531230 conn(0x7fe47c0770d0 msgr2=0x7fe47c0774d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:44.104 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.820+0000 7fe48471a700 1 -- 172.21.15.1:0/2742531230 shutdown_connections 2023-11-16T16:37:44.104 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.820+0000 7fe48471a700 1 -- 172.21.15.1:0/2742531230 wait complete. 2023-11-16T16:37:44.104 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.821+0000 7fe48471a700 1 Processor -- start 2023-11-16T16:37:44.104 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.821+0000 7fe48471a700 1 -- start start 2023-11-16T16:37:44.104 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.821+0000 7fe48471a700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe47c0ff140 0x7fe47c1985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:44.104 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.821+0000 7fe48471a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe47c0ffa50 0x7fe47c198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:44.104 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.821+0000 7fe48471a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe47c10ccf0 0x7fe47c19cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:44.104 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.821+0000 7fe48471a700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47c19d560 con 0x7fe47c0ff140 2023-11-16T16:37:44.104 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.821+0000 7fe48471a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47c19d6d0 con 0x7fe47c0ffa50 2023-11-16T16:37:44.104 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.821+0000 7fe48471a700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47c19d890 con 0x7fe47c10ccf0 2023-11-16T16:37:44.104 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.821+0000 7fe4824b6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe47c0ff140 0x7fe47c1985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:44.104 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.821+0000 7fe4824b6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe47c0ff140 0x7fe47c1985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41832/0 (socket says 172.21.15.1:41832) 2023-11-16T16:37:44.104 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.821+0000 7fe4824b6700 1 -- 172.21.15.1:0/2973950302 learned_addr learned my addr 172.21.15.1:0/2973950302 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:44.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.821+0000 7fe481cb5700 1 --2- 172.21.15.1:0/2973950302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe47c0ffa50 0x7fe47c198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:44.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.821+0000 7fe482cb7700 1 --2- 172.21.15.1:0/2973950302 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe47c10ccf0 0x7fe47c19cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:44.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.821+0000 7fe4824b6700 1 -- 172.21.15.1:0/2973950302 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe47c10ccf0 msgr2=0x7fe47c19cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:44.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.821+0000 7fe4824b6700 1 --2- 172.21.15.1:0/2973950302 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe47c10ccf0 0x7fe47c19cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:44.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.821+0000 7fe4824b6700 1 -- 172.21.15.1:0/2973950302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe47c0ffa50 msgr2=0x7fe47c198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:44.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.821+0000 7fe4824b6700 1 --2- 172.21.15.1:0/2973950302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe47c0ffa50 0x7fe47c198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:44.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.821+0000 7fe4824b6700 1 -- 172.21.15.1:0/2973950302 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4780095f0 con 0x7fe47c0ff140 2023-11-16T16:37:44.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.822+0000 7fe4824b6700 1 --2- 172.21.15.1:0/2973950302 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe47c0ff140 0x7fe47c1985a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe4780095c0 tx=0x7fe47800f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:44.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.822+0000 7fe4737fe700 1 -- 172.21.15.1:0/2973950302 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47801d070 con 0x7fe47c0ff140 2023-11-16T16:37:44.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.822+0000 7fe48471a700 1 -- 172.21.15.1:0/2973950302 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe47c112540 con 0x7fe47c0ff140 2023-11-16T16:37:44.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.822+0000 7fe4737fe700 1 -- 172.21.15.1:0/2973950302 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4780054f0 con 0x7fe47c0ff140 2023-11-16T16:37:44.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.822+0000 7fe4737fe700 1 -- 172.21.15.1:0/2973950302 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe478004a70 con 0x7fe47c0ff140 2023-11-16T16:37:44.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.822+0000 7fe48471a700 1 -- 172.21.15.1:0/2973950302 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe47c1123b0 con 0x7fe47c0ff140 2023-11-16T16:37:44.106 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.823+0000 7fe48471a700 1 -- 172.21.15.1:0/2973950302 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe47c10e920 con 0x7fe47c0ff140 2023-11-16T16:37:44.106 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.824+0000 7fe4737fe700 1 -- 172.21.15.1:0/2973950302 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fe478004bd0 con 0x7fe47c0ff140 2023-11-16T16:37:44.106 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.825+0000 7fe4737fe700 1 --2- 172.21.15.1:0/2973950302 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fe4680a11e0 0x7fe4680a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:44.106 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.825+0000 7fe481cb5700 1 --2- 172.21.15.1:0/2973950302 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fe4680a11e0 0x7fe4680a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:44.106 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.825+0000 7fe4737fe700 1 -- 172.21.15.1:0/2973950302 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe4780cdd50 con 0x7fe47c0ff140 2023-11-16T16:37:44.106 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.825+0000 7fe4737fe700 1 --2- 172.21.15.1:0/2973950302 >> [v2:172.21.15.1:6802/2469671609,v1:172.21.15.1:6803/2469671609] conn(0x7fe4680aa9b0 0x7fe4680acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:44.106 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.825+0000 7fe4737fe700 1 -- 172.21.15.1:0/2973950302 --> [v2:172.21.15.1:6802/2469671609,v1:172.21.15.1:6803/2469671609] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe4680ad470 con 0x7fe4680aa9b0 2023-11-16T16:37:44.106 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.825+0000 7fe4737fe700 1 -- 172.21.15.1:0/2973950302 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4780ce1f0 con 0x7fe47c0ff140 2023-11-16T16:37:44.106 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.825+0000 7fe482cb7700 1 --2- 172.21.15.1:0/2973950302 >> [v2:172.21.15.1:6802/2469671609,v1:172.21.15.1:6803/2469671609] conn(0x7fe4680aa9b0 0x7fe4680acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:44.106 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.825+0000 7fe481cb5700 1 --2- 172.21.15.1:0/2973950302 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fe4680a11e0 0x7fe4680a3690 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe47c199bc0 tx=0x7fe46c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.825+0000 7fe482cb7700 1 --2- 172.21.15.1:0/2973950302 >> [v2:172.21.15.1:6802/2469671609,v1:172.21.15.1:6803/2469671609] conn(0x7fe4680aa9b0 0x7fe4680acdc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.826+0000 7fe4737fe700 1 -- 172.21.15.1:0/2973950302 <== osd.0 v2:172.21.15.1:6802/2469671609 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fe4680ad470 con 0x7fe4680aa9b0 2023-11-16T16:37:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.843+0000 7fe48471a700 1 -- 172.21.15.1:0/2973950302 --> [v2:172.21.15.1:6802/2469671609,v1:172.21.15.1:6803/2469671609] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe47c1126d0 con 0x7fe4680aa9b0 2023-11-16T16:37:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.843+0000 7fe4737fe700 1 -- 172.21.15.1:0/2973950302 <== osd.0 v2:172.21.15.1:6802/2469671609 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe47c1126d0 con 0x7fe4680aa9b0 2023-11-16T16:37:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.843+0000 7fe48471a700 1 -- 172.21.15.1:0/2973950302 >> [v2:172.21.15.1:6802/2469671609,v1:172.21.15.1:6803/2469671609] conn(0x7fe4680aa9b0 msgr2=0x7fe4680acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.843+0000 7fe48471a700 1 --2- 172.21.15.1:0/2973950302 >> [v2:172.21.15.1:6802/2469671609,v1:172.21.15.1:6803/2469671609] conn(0x7fe4680aa9b0 0x7fe4680acdc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.843+0000 7fe48471a700 1 -- 172.21.15.1:0/2973950302 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fe4680a11e0 msgr2=0x7fe4680a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.843+0000 7fe48471a700 1 --2- 172.21.15.1:0/2973950302 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fe4680a11e0 0x7fe4680a3690 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe47c199bc0 tx=0x7fe46c00b410 comp rx=0 tx=0).stop 2023-11-16T16:37:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.843+0000 7fe48471a700 1 -- 172.21.15.1:0/2973950302 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe47c0ff140 msgr2=0x7fe47c1985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.843+0000 7fe48471a700 1 --2- 172.21.15.1:0/2973950302 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe47c0ff140 0x7fe47c1985a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe4780095c0 tx=0x7fe47800f710 comp rx=0 tx=0).stop 2023-11-16T16:37:44.108 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.843+0000 7fe482cb7700 1 -- 172.21.15.1:0/2973950302 reap_dead start 2023-11-16T16:37:44.108 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.843+0000 7fe48471a700 1 -- 172.21.15.1:0/2973950302 shutdown_connections 2023-11-16T16:37:44.108 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.843+0000 7fe48471a700 1 -- 172.21.15.1:0/2973950302 >> 172.21.15.1:0/2973950302 conn(0x7fe47c0770d0 msgr2=0x7fe47c10df50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:44.108 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.844+0000 7fe48471a700 1 -- 172.21.15.1:0/2973950302 shutdown_connections 2023-11-16T16:37:44.108 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:43.844+0000 7fe48471a700 1 -- 172.21.15.1:0/2973950302 wait complete. 2023-11-16T16:37:44.754 INFO:teuthology.orchestra.run.smithi001.stdout:287762808856 2023-11-16T16:37:44.755 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph osd last-stat-seq osd.9 2023-11-16T16:37:45.007 INFO:teuthology.orchestra.run.smithi001.stdout:111669149747 2023-11-16T16:37:45.007 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph osd last-stat-seq osd.3 2023-11-16T16:37:45.223 INFO:teuthology.orchestra.run.smithi001.stdout:309237645333 2023-11-16T16:37:45.223 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph osd last-stat-seq osd.10 2023-11-16T16:37:45.479 INFO:teuthology.orchestra.run.smithi001.stdout:34359738434 2023-11-16T16:37:45.480 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph osd last-stat-seq osd.0 2023-11-16T16:37:45.555 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:45 smithi001 ceph-mon[98800]: pgmap v276: 97 pgs: 97 active+clean; 636 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 341 B/s wr, 7 op/s 2023-11-16T16:37:45.733 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.731+0000 7fd5d5069700 1 -- 172.21.15.1:0/3506753049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5d00ff470 msgr2=0x7fd5d00ff840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:45.733 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.731+0000 7fd5d5069700 1 --2- 172.21.15.1:0/3506753049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5d00ff470 0x7fd5d00ff840 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd5c00099c0 tx=0x7fd5c0009cd0 comp rx=0 tx=0).stop 2023-11-16T16:37:45.733 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.731+0000 7fd5d5069700 1 -- 172.21.15.1:0/3506753049 shutdown_connections 2023-11-16T16:37:45.733 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.731+0000 7fd5d5069700 1 --2- 172.21.15.1:0/3506753049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5d0104510 0x7fd5d010c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:45.733 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.731+0000 7fd5d5069700 1 --2- 172.21.15.1:0/3506753049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5d00ff470 0x7fd5d00ff840 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:45.733 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.731+0000 7fd5d5069700 1 --2- 172.21.15.1:0/3506753049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5d00ffe10 0x7fd5d0103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:45.734 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.731+0000 7fd5d5069700 1 -- 172.21.15.1:0/3506753049 >> 172.21.15.1:0/3506753049 conn(0x7fd5d00770d0 msgr2=0x7fd5d00774d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:45.734 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.731+0000 7fd5d5069700 1 -- 172.21.15.1:0/3506753049 shutdown_connections 2023-11-16T16:37:45.734 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.731+0000 7fd5d5069700 1 -- 172.21.15.1:0/3506753049 wait complete. 2023-11-16T16:37:45.736 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.732+0000 7fd5d5069700 1 Processor -- start 2023-11-16T16:37:45.736 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.732+0000 7fd5d5069700 1 -- start start 2023-11-16T16:37:45.736 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.732+0000 7fd5d5069700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5d00ff470 0x7fd5d01971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:45.737 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.732+0000 7fd5d5069700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5d00ffe10 0x7fd5d01976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:45.737 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.732+0000 7fd5d5069700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5d0104510 0x7fd5d01a2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:45.737 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.732+0000 7fd5d5069700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d0198030 con 0x7fd5d00ff470 2023-11-16T16:37:45.737 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.732+0000 7fd5d5069700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d01a2770 con 0x7fd5d00ffe10 2023-11-16T16:37:45.737 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.732+0000 7fd5d5069700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d01a2930 con 0x7fd5d0104510 2023-11-16T16:37:45.737 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.732+0000 7fd5ced9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5d00ff470 0x7fd5d01971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:45.737 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.732+0000 7fd5ced9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5d00ff470 0x7fd5d01971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41864/0 (socket says 172.21.15.1:41864) 2023-11-16T16:37:45.737 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.732+0000 7fd5ced9d700 1 -- 172.21.15.1:0/2011868991 learned_addr learned my addr 172.21.15.1:0/2011868991 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:45.737 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.732+0000 7fd5ce59c700 1 --2- 172.21.15.1:0/2011868991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5d00ffe10 0x7fd5d01976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:45.738 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.732+0000 7fd5cf59e700 1 --2- 172.21.15.1:0/2011868991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5d0104510 0x7fd5d01a2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:45.738 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.733+0000 7fd5ced9d700 1 -- 172.21.15.1:0/2011868991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5d0104510 msgr2=0x7fd5d01a2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:45.738 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.733+0000 7fd5ced9d700 1 --2- 172.21.15.1:0/2011868991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5d0104510 0x7fd5d01a2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:45.738 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.733+0000 7fd5ced9d700 1 -- 172.21.15.1:0/2011868991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5d00ffe10 msgr2=0x7fd5d01976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:45.738 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.733+0000 7fd5ced9d700 1 --2- 172.21.15.1:0/2011868991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5d00ffe10 0x7fd5d01976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:45.738 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.733+0000 7fd5ced9d700 1 -- 172.21.15.1:0/2011868991 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c00095f0 con 0x7fd5d00ff470 2023-11-16T16:37:45.738 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.733+0000 7fd5cf59e700 1 --2- 172.21.15.1:0/2011868991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5d0104510 0x7fd5d01a2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:37:45.739 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.733+0000 7fd5ced9d700 1 --2- 172.21.15.1:0/2011868991 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5d00ff470 0x7fd5d01971b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd5c000b580 tx=0x7fd5c0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:45.739 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.733+0000 7fd5b7fff700 1 -- 172.21.15.1:0/2011868991 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c001d070 con 0x7fd5d00ff470 2023-11-16T16:37:45.739 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.733+0000 7fd5b7fff700 1 -- 172.21.15.1:0/2011868991 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5c0003f90 con 0x7fd5d00ff470 2023-11-16T16:37:45.739 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.733+0000 7fd5d5069700 1 -- 172.21.15.1:0/2011868991 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5d010f250 con 0x7fd5d00ff470 2023-11-16T16:37:45.739 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.733+0000 7fd5b7fff700 1 -- 172.21.15.1:0/2011868991 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0022d20 con 0x7fd5d00ff470 2023-11-16T16:37:45.740 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.733+0000 7fd5d5069700 1 -- 172.21.15.1:0/2011868991 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5d010f0c0 con 0x7fd5d00ff470 2023-11-16T16:37:45.740 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.734+0000 7fd5d5069700 1 -- 172.21.15.1:0/2011868991 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd5d00fd700 con 0x7fd5d00ff470 2023-11-16T16:37:45.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.736+0000 7fd5b7fff700 1 -- 172.21.15.1:0/2011868991 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fd5c0022470 con 0x7fd5d00ff470 2023-11-16T16:37:45.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.737+0000 7fd5b7fff700 1 --2- 172.21.15.1:0/2011868991 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd5b80a11e0 0x7fd5b80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:45.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.737+0000 7fd5ce59c700 1 --2- 172.21.15.1:0/2011868991 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd5b80a11e0 0x7fd5b80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:45.742 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.737+0000 7fd5b7fff700 1 -- 172.21.15.1:0/2011868991 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd5c00ce640 con 0x7fd5d00ff470 2023-11-16T16:37:45.742 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.737+0000 7fd5b7fff700 1 --2- 172.21.15.1:0/2011868991 >> [v2:172.21.15.53:6824/2564669546,v1:172.21.15.53:6825/2564669546] conn(0x7fd5b80aa9b0 0x7fd5b80acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:45.742 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.737+0000 7fd5b7fff700 1 -- 172.21.15.1:0/2011868991 --> [v2:172.21.15.53:6824/2564669546,v1:172.21.15.53:6825/2564669546] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd5b80ad470 con 0x7fd5b80aa9b0 2023-11-16T16:37:45.742 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.737+0000 7fd5b7fff700 1 -- 172.21.15.1:0/2011868991 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5c00ceae0 con 0x7fd5d00ff470 2023-11-16T16:37:45.742 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.737+0000 7fd5ce59c700 1 --2- 172.21.15.1:0/2011868991 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd5b80a11e0 0x7fd5b80a3690 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0069710 tx=0x7fd5c40073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:45.742 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.737+0000 7fd5cf59e700 1 --2- 172.21.15.1:0/2011868991 >> [v2:172.21.15.53:6824/2564669546,v1:172.21.15.53:6825/2564669546] conn(0x7fd5b80aa9b0 0x7fd5b80acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:45.742 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.738+0000 7fd5cf59e700 1 --2- 172.21.15.1:0/2011868991 >> [v2:172.21.15.53:6824/2564669546,v1:172.21.15.53:6825/2564669546] conn(0x7fd5b80aa9b0 0x7fd5b80acdc0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:45.742 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.739+0000 7fd5b7fff700 1 -- 172.21.15.1:0/2011868991 <== osd.11 v2:172.21.15.53:6824/2564669546 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fd5b80ad470 con 0x7fd5b80aa9b0 2023-11-16T16:37:45.760 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.758+0000 7fd5d5069700 1 -- 172.21.15.1:0/2011868991 --> [v2:172.21.15.53:6824/2564669546,v1:172.21.15.53:6825/2564669546] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd5d010f3e0 con 0x7fd5b80aa9b0 2023-11-16T16:37:45.761 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.758+0000 7fd5b7fff700 1 -- 172.21.15.1:0/2011868991 <== osd.11 v2:172.21.15.53:6824/2564669546 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fd5d010f3e0 con 0x7fd5b80aa9b0 2023-11-16T16:37:45.762 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.759+0000 7fd5d5069700 1 -- 172.21.15.1:0/2011868991 >> [v2:172.21.15.53:6824/2564669546,v1:172.21.15.53:6825/2564669546] conn(0x7fd5b80aa9b0 msgr2=0x7fd5b80acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:45.762 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.759+0000 7fd5d5069700 1 --2- 172.21.15.1:0/2011868991 >> [v2:172.21.15.53:6824/2564669546,v1:172.21.15.53:6825/2564669546] conn(0x7fd5b80aa9b0 0x7fd5b80acdc0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:45.762 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.759+0000 7fd5d5069700 1 -- 172.21.15.1:0/2011868991 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd5b80a11e0 msgr2=0x7fd5b80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:45.763 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.759+0000 7fd5d5069700 1 --2- 172.21.15.1:0/2011868991 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd5b80a11e0 0x7fd5b80a3690 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0069710 tx=0x7fd5c40073d0 comp rx=0 tx=0).stop 2023-11-16T16:37:45.763 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.759+0000 7fd5d5069700 1 -- 172.21.15.1:0/2011868991 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5d00ff470 msgr2=0x7fd5d01971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:45.763 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.759+0000 7fd5d5069700 1 --2- 172.21.15.1:0/2011868991 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5d00ff470 0x7fd5d01971b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd5c000b580 tx=0x7fd5c0005740 comp rx=0 tx=0).stop 2023-11-16T16:37:45.763 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.759+0000 7fd5cf59e700 1 -- 172.21.15.1:0/2011868991 reap_dead start 2023-11-16T16:37:45.763 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.759+0000 7fd5d5069700 1 -- 172.21.15.1:0/2011868991 shutdown_connections 2023-11-16T16:37:45.763 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.759+0000 7fd5d5069700 1 -- 172.21.15.1:0/2011868991 >> 172.21.15.1:0/2011868991 conn(0x7fd5d00770d0 msgr2=0x7fd5d00fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:45.763 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.759+0000 7fd5d5069700 1 -- 172.21.15.1:0/2011868991 shutdown_connections 2023-11-16T16:37:45.764 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.759+0000 7fd5d5069700 1 -- 172.21.15.1:0/2011868991 wait complete. 2023-11-16T16:37:45.822 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:37:45.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:45 smithi053 ceph-mon[100638]: pgmap v276: 97 pgs: 97 active+clean; 636 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 341 B/s wr, 7 op/s 2023-11-16T16:37:45.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.838+0000 7fd083d3f700 1 -- 172.21.15.1:0/378886986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd07c0ffe20 msgr2=0x7fd07c103df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:45.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.838+0000 7fd083d3f700 1 --2- 172.21.15.1:0/378886986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd07c0ffe20 0x7fd07c103df0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd06c0099c0 tx=0x7fd06c009cd0 comp rx=0 tx=0).stop 2023-11-16T16:37:45.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.838+0000 7fd083d3f700 1 -- 172.21.15.1:0/378886986 shutdown_connections 2023-11-16T16:37:45.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.838+0000 7fd083d3f700 1 --2- 172.21.15.1:0/378886986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd07c104520 0x7fd07c10c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:45.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.838+0000 7fd083d3f700 1 --2- 172.21.15.1:0/378886986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd07c0ff480 0x7fd07c0ff850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:45.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.838+0000 7fd083d3f700 1 --2- 172.21.15.1:0/378886986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd07c0ffe20 0x7fd07c103df0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:45.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.838+0000 7fd083d3f700 1 -- 172.21.15.1:0/378886986 >> 172.21.15.1:0/378886986 conn(0x7fd07c0770d0 msgr2=0x7fd07c0774d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:45.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.838+0000 7fd083d3f700 1 -- 172.21.15.1:0/378886986 shutdown_connections 2023-11-16T16:37:45.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.838+0000 7fd083d3f700 1 -- 172.21.15.1:0/378886986 wait complete. 2023-11-16T16:37:45.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.839+0000 7fd083d3f700 1 Processor -- start 2023-11-16T16:37:45.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.839+0000 7fd083d3f700 1 -- start start 2023-11-16T16:37:45.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.839+0000 7fd083d3f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd07c0ff480 0x7fd07c1963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:45.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.839+0000 7fd083d3f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd07c0ffe20 0x7fd07c196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:45.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.839+0000 7fd083d3f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd07c104520 0x7fd07c19ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:45.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.839+0000 7fd083d3f700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07c19b3a0 con 0x7fd07c104520 2023-11-16T16:37:45.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.839+0000 7fd083d3f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07c19b510 con 0x7fd07c0ff480 2023-11-16T16:37:45.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.839+0000 7fd083d3f700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07c19b6d0 con 0x7fd07c0ffe20 2023-11-16T16:37:45.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.839+0000 7fd0822dc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd07c104520 0x7fd07c19ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:45.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.839+0000 7fd0822dc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd07c104520 0x7fd07c19ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41884/0 (socket says 172.21.15.1:41884) 2023-11-16T16:37:45.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.839+0000 7fd0822dc700 1 -- 172.21.15.1:0/3336245847 learned_addr learned my addr 172.21.15.1:0/3336245847 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:45.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.839+0000 7fd0812da700 1 --2- 172.21.15.1:0/3336245847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd07c0ffe20 0x7fd07c196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:45.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.839+0000 7fd081adb700 1 --2- 172.21.15.1:0/3336245847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd07c0ff480 0x7fd07c1963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:45.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.840+0000 7fd0822dc700 1 -- 172.21.15.1:0/3336245847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd07c0ffe20 msgr2=0x7fd07c196920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:45.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.840+0000 7fd0822dc700 1 --2- 172.21.15.1:0/3336245847 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd07c0ffe20 0x7fd07c196920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:45.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.840+0000 7fd0822dc700 1 -- 172.21.15.1:0/3336245847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd07c0ff480 msgr2=0x7fd07c1963e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:45.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.840+0000 7fd0822dc700 1 --2- 172.21.15.1:0/3336245847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd07c0ff480 0x7fd07c1963e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:45.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.840+0000 7fd0822dc700 1 -- 172.21.15.1:0/3336245847 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd06c0095f0 con 0x7fd07c104520 2023-11-16T16:37:45.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.840+0000 7fd0822dc700 1 --2- 172.21.15.1:0/3336245847 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd07c104520 0x7fd07c19ad90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd074007ee0 tx=0x7fd07400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:45.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.840+0000 7fd072ffd700 1 -- 172.21.15.1:0/3336245847 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd074017070 con 0x7fd07c104520 2023-11-16T16:37:45.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.840+0000 7fd072ffd700 1 -- 172.21.15.1:0/3336245847 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd07400f040 con 0x7fd07c104520 2023-11-16T16:37:45.847 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.840+0000 7fd072ffd700 1 -- 172.21.15.1:0/3336245847 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd074005580 con 0x7fd07c104520 2023-11-16T16:37:45.847 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.841+0000 7fd083d3f700 1 -- 172.21.15.1:0/3336245847 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd07c10f250 con 0x7fd07c104520 2023-11-16T16:37:45.847 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.841+0000 7fd083d3f700 1 -- 172.21.15.1:0/3336245847 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd07c10f0c0 con 0x7fd07c104520 2023-11-16T16:37:45.847 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.842+0000 7fd083d3f700 1 -- 172.21.15.1:0/3336245847 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd07c069420 con 0x7fd07c104520 2023-11-16T16:37:45.847 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.844+0000 7fd072ffd700 1 -- 172.21.15.1:0/3336245847 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fd07400db20 con 0x7fd07c104520 2023-11-16T16:37:45.848 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.845+0000 7fd072ffd700 1 --2- 172.21.15.1:0/3336245847 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd0680a5670 0x7fd0680a7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:45.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.845+0000 7fd081adb700 1 --2- 172.21.15.1:0/3336245847 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd0680a5670 0x7fd0680a7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:45.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.845+0000 7fd072ffd700 1 -- 172.21.15.1:0/3336245847 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd0740cc050 con 0x7fd07c104520 2023-11-16T16:37:45.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.845+0000 7fd072ffd700 1 --2- 172.21.15.1:0/3336245847 >> [v2:172.21.15.26:6800/247994796,v1:172.21.15.26:6801/247994796] conn(0x7fd0680aee40 0x7fd0680b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:45.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.845+0000 7fd072ffd700 1 -- 172.21.15.1:0/3336245847 --> [v2:172.21.15.26:6800/247994796,v1:172.21.15.26:6801/247994796] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd0680b1900 con 0x7fd0680aee40 2023-11-16T16:37:45.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.845+0000 7fd072ffd700 1 -- 172.21.15.1:0/3336245847 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0740cc4f0 con 0x7fd07c104520 2023-11-16T16:37:45.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.846+0000 7fd081adb700 1 --2- 172.21.15.1:0/3336245847 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd0680a5670 0x7fd0680a7b20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd078009c60 tx=0x7fd078009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:45.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.846+0000 7fd0812da700 1 --2- 172.21.15.1:0/3336245847 >> [v2:172.21.15.26:6800/247994796,v1:172.21.15.26:6801/247994796] conn(0x7fd0680aee40 0x7fd0680b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:45.850 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.846+0000 7fd0812da700 1 --2- 172.21.15.1:0/3336245847 >> [v2:172.21.15.26:6800/247994796,v1:172.21.15.26:6801/247994796] conn(0x7fd0680aee40 0x7fd0680b1250 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:45.850 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.847+0000 7fd072ffd700 1 -- 172.21.15.1:0/3336245847 <== osd.1 v2:172.21.15.26:6800/247994796 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fd0680b1900 con 0x7fd0680aee40 2023-11-16T16:37:45.868 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.866+0000 7fd083d3f700 1 -- 172.21.15.1:0/3336245847 --> [v2:172.21.15.26:6800/247994796,v1:172.21.15.26:6801/247994796] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd07c10f3e0 con 0x7fd0680aee40 2023-11-16T16:37:45.869 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.867+0000 7fd072ffd700 1 -- 172.21.15.1:0/3336245847 <== osd.1 v2:172.21.15.26:6800/247994796 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd07c10f3e0 con 0x7fd0680aee40 2023-11-16T16:37:45.870 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.867+0000 7fd083d3f700 1 -- 172.21.15.1:0/3336245847 >> [v2:172.21.15.26:6800/247994796,v1:172.21.15.26:6801/247994796] conn(0x7fd0680aee40 msgr2=0x7fd0680b1250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:45.870 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.867+0000 7fd083d3f700 1 --2- 172.21.15.1:0/3336245847 >> [v2:172.21.15.26:6800/247994796,v1:172.21.15.26:6801/247994796] conn(0x7fd0680aee40 0x7fd0680b1250 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:45.871 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.868+0000 7fd083d3f700 1 -- 172.21.15.1:0/3336245847 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd0680a5670 msgr2=0x7fd0680a7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:45.871 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.868+0000 7fd083d3f700 1 --2- 172.21.15.1:0/3336245847 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd0680a5670 0x7fd0680a7b20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd078009c60 tx=0x7fd078009260 comp rx=0 tx=0).stop 2023-11-16T16:37:45.871 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.868+0000 7fd083d3f700 1 -- 172.21.15.1:0/3336245847 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd07c104520 msgr2=0x7fd07c19ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:45.871 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.868+0000 7fd083d3f700 1 --2- 172.21.15.1:0/3336245847 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd07c104520 0x7fd07c19ad90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd074007ee0 tx=0x7fd07400d3b0 comp rx=0 tx=0).stop 2023-11-16T16:37:45.871 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.868+0000 7fd0822dc700 1 -- 172.21.15.1:0/3336245847 reap_dead start 2023-11-16T16:37:45.871 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.868+0000 7fd083d3f700 1 -- 172.21.15.1:0/3336245847 shutdown_connections 2023-11-16T16:37:45.871 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.868+0000 7fd083d3f700 1 -- 172.21.15.1:0/3336245847 >> 172.21.15.1:0/3336245847 conn(0x7fd07c0770d0 msgr2=0x7fd07c102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:45.871 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.868+0000 7fd083d3f700 1 -- 172.21.15.1:0/3336245847 shutdown_connections 2023-11-16T16:37:45.872 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:45.868+0000 7fd083d3f700 1 -- 172.21.15.1:0/3336245847 wait complete. 2023-11-16T16:37:45.997 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:37:46.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:45 smithi026 ceph-mon[100112]: pgmap v276: 97 pgs: 97 active+clean; 636 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 341 B/s wr, 7 op/s 2023-11-16T16:37:46.346 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:37:46.372 INFO:teuthology.orchestra.run.smithi001.stdout:55834574908 2023-11-16T16:37:46.372 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph osd last-stat-seq osd.1 2023-11-16T16:37:46.376 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:37:46.485 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:37:46.835 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:37:47.425 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:37:47.778 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:47 smithi001 ceph-mon[98800]: pgmap v277: 97 pgs: 97 active+clean; 636 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2023-11-16T16:37:47.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:47 smithi053 ceph-mon[100638]: pgmap v277: 97 pgs: 97 active+clean; 636 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2023-11-16T16:37:47.979 INFO:teuthology.orchestra.run.smithi001.stdout:339302416402 2023-11-16T16:37:47.980 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph osd last-stat-seq osd.11 2023-11-16T16:37:48.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:47 smithi026 ceph-mon[100112]: pgmap v277: 97 pgs: 97 active+clean; 636 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2023-11-16T16:37:48.442 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:37:49.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.441+0000 7f19443b3700 1 -- 172.21.15.1:0/1718095100 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f193c1055e0 msgr2=0x7f193c10d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:49.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.441+0000 7f19443b3700 1 --2- 172.21.15.1:0/1718095100 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f193c1055e0 0x7f193c10d160 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f193400b0d0 tx=0x7f193400b3e0 comp rx=0 tx=0).stop 2023-11-16T16:37:49.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.441+0000 7f19443b3700 1 -- 172.21.15.1:0/1718095100 shutdown_connections 2023-11-16T16:37:49.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.441+0000 7f19443b3700 1 --2- 172.21.15.1:0/1718095100 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f193c1055e0 0x7f193c10d160 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:49.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.441+0000 7f19443b3700 1 --2- 172.21.15.1:0/1718095100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f193c100540 0x7f193c100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:49.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.441+0000 7f19443b3700 1 --2- 172.21.15.1:0/1718095100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f193c100ee0 0x7f193c104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:49.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.441+0000 7f19443b3700 1 -- 172.21.15.1:0/1718095100 >> 172.21.15.1:0/1718095100 conn(0x7f193c0fbde0 msgr2=0x7f193c0fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:49.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.441+0000 7f19443b3700 1 -- 172.21.15.1:0/1718095100 shutdown_connections 2023-11-16T16:37:49.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.441+0000 7f19443b3700 1 -- 172.21.15.1:0/1718095100 wait complete. 2023-11-16T16:37:49.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.442+0000 7f19443b3700 1 Processor -- start 2023-11-16T16:37:49.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.442+0000 7f19443b3700 1 -- start start 2023-11-16T16:37:49.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.442+0000 7f19443b3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f193c100540 0x7f193c072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:49.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.442+0000 7f19443b3700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f193c100ee0 0x7f193c073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:49.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.442+0000 7f19443b3700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f193c1055e0 0x7f193c0738b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:49.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.442+0000 7f19443b3700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193c1a4900 con 0x7f193c100ee0 2023-11-16T16:37:49.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.442+0000 7f19443b3700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193c1a4a70 con 0x7f193c100540 2023-11-16T16:37:49.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.442+0000 7f19443b3700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193c1a4c30 con 0x7f193c1055e0 2023-11-16T16:37:49.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.442+0000 7f194194e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f193c100ee0 0x7f193c073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:49.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.442+0000 7f194194e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f193c100ee0 0x7f193c073330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41898/0 (socket says 172.21.15.1:41898) 2023-11-16T16:37:49.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.442+0000 7f194194e700 1 -- 172.21.15.1:0/136528205 learned_addr learned my addr 172.21.15.1:0/136528205 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:49.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.442+0000 7f1942950700 1 --2- 172.21.15.1:0/136528205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f193c1055e0 0x7f193c0738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:49.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.442+0000 7f194214f700 1 --2- 172.21.15.1:0/136528205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f193c100540 0x7f193c072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:49.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.443+0000 7f194194e700 1 -- 172.21.15.1:0/136528205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f193c1055e0 msgr2=0x7f193c0738b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:49.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.443+0000 7f194194e700 1 --2- 172.21.15.1:0/136528205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f193c1055e0 0x7f193c0738b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:49.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.443+0000 7f194194e700 1 -- 172.21.15.1:0/136528205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f193c100540 msgr2=0x7f193c072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:49.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.443+0000 7f194194e700 1 --2- 172.21.15.1:0/136528205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f193c100540 0x7f193c072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:49.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.443+0000 7f194194e700 1 -- 172.21.15.1:0/136528205 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1934009d00 con 0x7f193c100ee0 2023-11-16T16:37:49.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.443+0000 7f194194e700 1 --2- 172.21.15.1:0/136528205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f193c100ee0 0x7f193c073330 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f192c00d980 tx=0x7f192c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:49.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.443+0000 7f19337fe700 1 -- 172.21.15.1:0/136528205 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c009930 con 0x7f193c100ee0 2023-11-16T16:37:49.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.443+0000 7f19337fe700 1 -- 172.21.15.1:0/136528205 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f192c010460 con 0x7f193c100ee0 2023-11-16T16:37:49.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.443+0000 7f19443b3700 1 -- 172.21.15.1:0/136528205 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f193c110370 con 0x7f193c100ee0 2023-11-16T16:37:49.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.443+0000 7f19337fe700 1 -- 172.21.15.1:0/136528205 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c00f620 con 0x7f193c100ee0 2023-11-16T16:37:49.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.443+0000 7f19443b3700 1 -- 172.21.15.1:0/136528205 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f193c1101e0 con 0x7f193c100ee0 2023-11-16T16:37:49.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.444+0000 7f19443b3700 1 -- 172.21.15.1:0/136528205 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f193c110500 con 0x7f193c100ee0 2023-11-16T16:37:49.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.445+0000 7f19337fe700 1 -- 172.21.15.1:0/136528205 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f192c0108d0 con 0x7f193c100ee0 2023-11-16T16:37:49.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.446+0000 7f19337fe700 1 --2- 172.21.15.1:0/136528205 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f19280a1260 0x7f19280a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:49.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.446+0000 7f194214f700 1 --2- 172.21.15.1:0/136528205 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f19280a1260 0x7f19280a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:49.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.446+0000 7f19337fe700 1 -- 172.21.15.1:0/136528205 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f192c0cc630 con 0x7f193c100ee0 2023-11-16T16:37:49.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.446+0000 7f194214f700 1 --2- 172.21.15.1:0/136528205 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f19280a1260 0x7f19280a3710 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1938005950 tx=0x7f19380058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:49.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.450+0000 7f19337fe700 1 -- 172.21.15.1:0/136528205 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f192c095dd0 con 0x7f193c100ee0 2023-11-16T16:37:49.525 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:37:49.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.671+0000 7f19443b3700 1 -- 172.21.15.1:0/136528205 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f193c066ea0 con 0x7f193c100ee0 2023-11-16T16:37:49.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.671+0000 7f19337fe700 1 -- 172.21.15.1:0/136528205 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f192c0993f0 con 0x7f193c100ee0 2023-11-16T16:37:49.673 INFO:teuthology.orchestra.run.smithi001.stdout:201863462952 2023-11-16T16:37:49.676 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.674+0000 7f19443b3700 1 -- 172.21.15.1:0/136528205 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f19280a1260 msgr2=0x7f19280a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:49.676 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.674+0000 7f19443b3700 1 --2- 172.21.15.1:0/136528205 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f19280a1260 0x7f19280a3710 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1938005950 tx=0x7f19380058e0 comp rx=0 tx=0).stop 2023-11-16T16:37:49.676 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.674+0000 7f19443b3700 1 -- 172.21.15.1:0/136528205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f193c100ee0 msgr2=0x7f193c073330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:49.676 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.674+0000 7f19443b3700 1 --2- 172.21.15.1:0/136528205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f193c100ee0 0x7f193c073330 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f192c00d980 tx=0x7f192c00dc90 comp rx=0 tx=0).stop 2023-11-16T16:37:49.676 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.674+0000 7f19443b3700 1 -- 172.21.15.1:0/136528205 shutdown_connections 2023-11-16T16:37:49.676 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.674+0000 7f19443b3700 1 --2- 172.21.15.1:0/136528205 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f19280a1260 0x7f19280a3710 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:49.676 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.674+0000 7f19443b3700 1 --2- 172.21.15.1:0/136528205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f193c1055e0 0x7f193c0738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:49.676 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.674+0000 7f19443b3700 1 --2- 172.21.15.1:0/136528205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f193c100540 0x7f193c072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:49.676 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.674+0000 7f19443b3700 1 --2- 172.21.15.1:0/136528205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f193c100ee0 0x7f193c073330 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:49.676 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.674+0000 7f19443b3700 1 -- 172.21.15.1:0/136528205 >> 172.21.15.1:0/136528205 conn(0x7f193c0fbde0 msgr2=0x7f193c0fc730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:49.676 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.674+0000 7f19443b3700 1 -- 172.21.15.1:0/136528205 shutdown_connections 2023-11-16T16:37:49.676 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:49.674+0000 7f19443b3700 1 -- 172.21.15.1:0/136528205 wait complete. 2023-11-16T16:37:49.802 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:49 smithi001 ceph-mon[98800]: pgmap v278: 97 pgs: 97 active+clean; 636 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 5 op/s 2023-11-16T16:37:49.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:49 smithi053 ceph-mon[100638]: pgmap v278: 97 pgs: 97 active+clean; 636 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 5 op/s 2023-11-16T16:37:50.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:49 smithi026 ceph-mon[100112]: pgmap v278: 97 pgs: 97 active+clean; 636 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 5 op/s 2023-11-16T16:37:50.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:50 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/136528205' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-11-16T16:37:50.712 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.561+0000 7fa500e39700 1 -- 172.21.15.1:0/3574690993 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4fc109c30 msgr2=0x7fa4fc10c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:50.712 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.561+0000 7fa500e39700 1 --2- 172.21.15.1:0/3574690993 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4fc109c30 0x7fa4fc10c010 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa4f400b0a0 tx=0x7fa4f400b3b0 comp rx=0 tx=0).stop 2023-11-16T16:37:50.712 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.561+0000 7fa500e39700 1 -- 172.21.15.1:0/3574690993 shutdown_connections 2023-11-16T16:37:50.713 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.561+0000 7fa500e39700 1 --2- 172.21.15.1:0/3574690993 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4fc109c30 0x7fa4fc10c010 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:50.713 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.561+0000 7fa500e39700 1 --2- 172.21.15.1:0/3574690993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4fc100680 0x7fa4fc100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:50.713 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.561+0000 7fa500e39700 1 --2- 172.21.15.1:0/3574690993 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4fc100f90 0x7fa4fc1096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:50.713 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.561+0000 7fa500e39700 1 -- 172.21.15.1:0/3574690993 >> 172.21.15.1:0/3574690993 conn(0x7fa4fc074b30 msgr2=0x7fa4fc074f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:50.713 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.561+0000 7fa500e39700 1 -- 172.21.15.1:0/3574690993 shutdown_connections 2023-11-16T16:37:50.713 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.561+0000 7fa500e39700 1 -- 172.21.15.1:0/3574690993 wait complete. 2023-11-16T16:37:50.713 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.562+0000 7fa500e39700 1 Processor -- start 2023-11-16T16:37:50.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.562+0000 7fa500e39700 1 -- start start 2023-11-16T16:37:50.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.562+0000 7fa500e39700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4fc100680 0x7fa4fc1964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:50.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.562+0000 7fa500e39700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4fc100f90 0x7fa4fc196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:50.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.562+0000 7fa500e39700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4fc109c30 0x7fa4fc19ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:50.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.562+0000 7fa500e39700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4fc19b4a0 con 0x7fa4fc109c30 2023-11-16T16:37:50.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.562+0000 7fa500e39700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4fc19b610 con 0x7fa4fc100f90 2023-11-16T16:37:50.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.562+0000 7fa500e39700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4fc19b7d0 con 0x7fa4fc100680 2023-11-16T16:37:50.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.562+0000 7fa4fad9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4fc109c30 0x7fa4fc19ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:50.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.562+0000 7fa4fad9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4fc109c30 0x7fa4fc19ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41914/0 (socket says 172.21.15.1:41914) 2023-11-16T16:37:50.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.562+0000 7fa4fad9d700 1 -- 172.21.15.1:0/4155324510 learned_addr learned my addr 172.21.15.1:0/4155324510 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:50.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.562+0000 7fa4fa59c700 1 --2- 172.21.15.1:0/4155324510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4fc100680 0x7fa4fc1964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:50.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.562+0000 7fa4f9d9b700 1 --2- 172.21.15.1:0/4155324510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4fc100f90 0x7fa4fc196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:50.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.562+0000 7fa4fad9d700 1 -- 172.21.15.1:0/4155324510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4fc100680 msgr2=0x7fa4fc1964e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:50.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.562+0000 7fa4fad9d700 1 --2- 172.21.15.1:0/4155324510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4fc100680 0x7fa4fc1964e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:50.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.562+0000 7fa4fad9d700 1 -- 172.21.15.1:0/4155324510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4fc100f90 msgr2=0x7fa4fc196a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:50.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.562+0000 7fa4fad9d700 1 --2- 172.21.15.1:0/4155324510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4fc100f90 0x7fa4fc196a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:50.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.562+0000 7fa4fad9d700 1 -- 172.21.15.1:0/4155324510 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f4009d00 con 0x7fa4fc109c30 2023-11-16T16:37:50.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.563+0000 7fa4fad9d700 1 --2- 172.21.15.1:0/4155324510 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4fc109c30 0x7fa4fc19ae90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa4f400b070 tx=0x7fa4f40078e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:50.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.563+0000 7fa4eb7fe700 1 -- 172.21.15.1:0/4155324510 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f400e040 con 0x7fa4fc109c30 2023-11-16T16:37:50.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.563+0000 7fa500e39700 1 -- 172.21.15.1:0/4155324510 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4fc10f300 con 0x7fa4fc109c30 2023-11-16T16:37:50.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.563+0000 7fa4eb7fe700 1 -- 172.21.15.1:0/4155324510 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4f401d470 con 0x7fa4fc109c30 2023-11-16T16:37:50.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.563+0000 7fa4eb7fe700 1 -- 172.21.15.1:0/4155324510 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f4005280 con 0x7fa4fc109c30 2023-11-16T16:37:50.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.563+0000 7fa500e39700 1 -- 172.21.15.1:0/4155324510 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4fc10f170 con 0x7fa4fc109c30 2023-11-16T16:37:50.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.564+0000 7fa500e39700 1 -- 172.21.15.1:0/4155324510 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4fc10f490 con 0x7fa4fc109c30 2023-11-16T16:37:50.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.565+0000 7fa4eb7fe700 1 -- 172.21.15.1:0/4155324510 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fa4f401d5e0 con 0x7fa4fc109c30 2023-11-16T16:37:50.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.565+0000 7fa4eb7fe700 1 --2- 172.21.15.1:0/4155324510 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fa4e40a11e0 0x7fa4e40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:50.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.565+0000 7fa4eb7fe700 1 -- 172.21.15.1:0/4155324510 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa4f40cdc10 con 0x7fa4fc109c30 2023-11-16T16:37:50.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.565+0000 7fa4fa59c700 1 --2- 172.21.15.1:0/4155324510 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fa4e40a11e0 0x7fa4e40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:50.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.566+0000 7fa4fa59c700 1 --2- 172.21.15.1:0/4155324510 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fa4e40a11e0 0x7fa4e40a3690 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec009c60 tx=0x7fa4ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:50.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.570+0000 7fa4eb7fe700 1 -- 172.21.15.1:0/4155324510 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa4f4097440 con 0x7fa4fc109c30 2023-11-16T16:37:50.774 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.772+0000 7fa500e39700 1 -- 172.21.15.1:0/4155324510 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fa4fc1a3560 con 0x7fa4fc109c30 2023-11-16T16:37:50.774 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.773+0000 7fa4eb7fe700 1 -- 172.21.15.1:0/4155324510 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa4f409aa60 con 0x7fa4fc109c30 2023-11-16T16:37:50.775 INFO:teuthology.orchestra.run.smithi001.stdout:146028888113 2023-11-16T16:37:50.776 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.774+0000 7fa500e39700 1 -- 172.21.15.1:0/4155324510 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fa4e40a11e0 msgr2=0x7fa4e40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:50.777 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.774+0000 7fa500e39700 1 --2- 172.21.15.1:0/4155324510 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fa4e40a11e0 0x7fa4e40a3690 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec009c60 tx=0x7fa4ec009260 comp rx=0 tx=0).stop 2023-11-16T16:37:50.777 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.775+0000 7fa500e39700 1 -- 172.21.15.1:0/4155324510 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4fc109c30 msgr2=0x7fa4fc19ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:50.777 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.775+0000 7fa500e39700 1 --2- 172.21.15.1:0/4155324510 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4fc109c30 0x7fa4fc19ae90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa4f400b070 tx=0x7fa4f40078e0 comp rx=0 tx=0).stop 2023-11-16T16:37:50.777 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.775+0000 7fa500e39700 1 -- 172.21.15.1:0/4155324510 shutdown_connections 2023-11-16T16:37:50.777 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.775+0000 7fa500e39700 1 --2- 172.21.15.1:0/4155324510 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fa4e40a11e0 0x7fa4e40a3690 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:50.777 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.775+0000 7fa500e39700 1 --2- 172.21.15.1:0/4155324510 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa4fc109c30 0x7fa4fc19ae90 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:50.778 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.775+0000 7fa500e39700 1 --2- 172.21.15.1:0/4155324510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa4fc100680 0x7fa4fc1964e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:50.778 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.775+0000 7fa500e39700 1 --2- 172.21.15.1:0/4155324510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4fc100f90 0x7fa4fc196a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:50.778 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.775+0000 7fa500e39700 1 -- 172.21.15.1:0/4155324510 >> 172.21.15.1:0/4155324510 conn(0x7fa4fc074b30 msgr2=0x7fa4fc0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:50.778 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.775+0000 7fa500e39700 1 -- 172.21.15.1:0/4155324510 shutdown_connections 2023-11-16T16:37:50.778 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:50.775+0000 7fa500e39700 1 -- 172.21.15.1:0/4155324510 wait complete. 2023-11-16T16:37:50.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:50 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/136528205' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-11-16T16:37:51.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:50 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/136528205' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-11-16T16:37:51.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.496+0000 7f2e2b6dc700 1 -- 172.21.15.1:0/3925250252 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e240fe830 msgr2=0x7f2e240feca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:51.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.496+0000 7f2e2b6dc700 1 --2- 172.21.15.1:0/3925250252 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e240fe830 0x7f2e240feca0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2e14009990 tx=0x7f2e14009ca0 comp rx=0 tx=0).stop 2023-11-16T16:37:51.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.496+0000 7f2e2b6dc700 1 -- 172.21.15.1:0/3925250252 shutdown_connections 2023-11-16T16:37:51.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.496+0000 7f2e2b6dc700 1 --2- 172.21.15.1:0/3925250252 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e240ff1e0 0x7f2e2410d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:51.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.496+0000 7f2e2b6dc700 1 --2- 172.21.15.1:0/3925250252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e24105290 0x7f2e24105660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:51.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.496+0000 7f2e2b6dc700 1 --2- 172.21.15.1:0/3925250252 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e240fe830 0x7f2e240feca0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:51.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.496+0000 7f2e2b6dc700 1 -- 172.21.15.1:0/3925250252 >> 172.21.15.1:0/3925250252 conn(0x7f2e240758a0 msgr2=0x7f2e24075ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:51.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.496+0000 7f2e2b6dc700 1 -- 172.21.15.1:0/3925250252 shutdown_connections 2023-11-16T16:37:51.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.496+0000 7f2e2b6dc700 1 -- 172.21.15.1:0/3925250252 wait complete. 2023-11-16T16:37:51.502 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.497+0000 7f2e2b6dc700 1 Processor -- start 2023-11-16T16:37:51.502 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.497+0000 7f2e2b6dc700 1 -- start start 2023-11-16T16:37:51.502 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.497+0000 7f2e2b6dc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e240fe830 0x7f2e240ffa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:51.502 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.497+0000 7f2e2b6dc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e240ff1e0 0x7f2e240fff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:51.503 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.497+0000 7f2e2b6dc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e24105290 0x7f2e241a4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:51.503 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.497+0000 7f2e2b6dc700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e24100840 con 0x7f2e240ff1e0 2023-11-16T16:37:51.503 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.497+0000 7f2e2b6dc700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e241a4a90 con 0x7f2e24105290 2023-11-16T16:37:51.503 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.497+0000 7f2e2b6dc700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e241a4c00 con 0x7f2e240fe830 2023-11-16T16:37:51.503 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.497+0000 7f2e28c77700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e240ff1e0 0x7f2e240fff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:51.503 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.497+0000 7f2e28c77700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e240ff1e0 0x7f2e240fff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41926/0 (socket says 172.21.15.1:41926) 2023-11-16T16:37:51.503 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.497+0000 7f2e28c77700 1 -- 172.21.15.1:0/827179054 learned_addr learned my addr 172.21.15.1:0/827179054 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:51.503 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.497+0000 7f2e29478700 1 --2- 172.21.15.1:0/827179054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e240fe830 0x7f2e240ffa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:51.503 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.498+0000 7f2e28c77700 1 -- 172.21.15.1:0/827179054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e240fe830 msgr2=0x7f2e240ffa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:51.504 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.498+0000 7f2e28c77700 1 --2- 172.21.15.1:0/827179054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e240fe830 0x7f2e240ffa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:51.504 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.498+0000 7f2e28c77700 1 -- 172.21.15.1:0/827179054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e24105290 msgr2=0x7f2e241a4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:51.504 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.498+0000 7f2e28c77700 1 --2- 172.21.15.1:0/827179054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e24105290 0x7f2e241a4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:51.504 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.498+0000 7f2e28c77700 1 -- 172.21.15.1:0/827179054 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e140095f0 con 0x7f2e240ff1e0 2023-11-16T16:37:51.504 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.498+0000 7f2e28c77700 1 --2- 172.21.15.1:0/827179054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e240ff1e0 0x7f2e240fff50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2e140095c0 tx=0x7f2e1400f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:51.504 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.498+0000 7f2e1a7fc700 1 -- 172.21.15.1:0/827179054 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1401d070 con 0x7f2e240ff1e0 2023-11-16T16:37:51.504 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.498+0000 7f2e1a7fc700 1 -- 172.21.15.1:0/827179054 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e1400fb50 con 0x7f2e240ff1e0 2023-11-16T16:37:51.504 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.498+0000 7f2e1a7fc700 1 -- 172.21.15.1:0/827179054 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e140173f0 con 0x7f2e240ff1e0 2023-11-16T16:37:51.504 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.498+0000 7f2e2b6dc700 1 -- 172.21.15.1:0/827179054 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e24110410 con 0x7f2e240ff1e0 2023-11-16T16:37:51.505 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.498+0000 7f2e2b6dc700 1 -- 172.21.15.1:0/827179054 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e24110280 con 0x7f2e240ff1e0 2023-11-16T16:37:51.505 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.499+0000 7f2e2b6dc700 1 -- 172.21.15.1:0/827179054 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e241105a0 con 0x7f2e240ff1e0 2023-11-16T16:37:51.505 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.500+0000 7f2e1a7fc700 1 -- 172.21.15.1:0/827179054 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f2e1400fcc0 con 0x7f2e240ff1e0 2023-11-16T16:37:51.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.501+0000 7f2e1a7fc700 1 --2- 172.21.15.1:0/827179054 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f2e100a1260 0x7f2e100a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:51.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.501+0000 7f2e29478700 1 --2- 172.21.15.1:0/827179054 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f2e100a1260 0x7f2e100a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:51.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.501+0000 7f2e1a7fc700 1 -- 172.21.15.1:0/827179054 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2e140cd8f0 con 0x7f2e240ff1e0 2023-11-16T16:37:51.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.502+0000 7f2e29478700 1 --2- 172.21.15.1:0/827179054 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f2e100a1260 0x7f2e100a3710 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2e20007b60 tx=0x7f2e200073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:51.507 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.505+0000 7f2e1a7fc700 1 -- 172.21.15.1:0/827179054 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2e1405c730 con 0x7f2e240ff1e0 2023-11-16T16:37:51.727 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.724+0000 7f2e2b6dc700 1 -- 172.21.15.1:0/827179054 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f2e241a5550 con 0x7f2e240ff1e0 2023-11-16T16:37:51.727 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.725+0000 7f2e1a7fc700 1 -- 172.21.15.1:0/827179054 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2e14027090 con 0x7f2e240ff1e0 2023-11-16T16:37:51.727 INFO:teuthology.orchestra.run.smithi001.stdout:223338299428 2023-11-16T16:37:51.730 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.728+0000 7f2e2b6dc700 1 -- 172.21.15.1:0/827179054 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f2e100a1260 msgr2=0x7f2e100a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:51.731 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.728+0000 7f2e2b6dc700 1 --2- 172.21.15.1:0/827179054 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f2e100a1260 0x7f2e100a3710 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2e20007b60 tx=0x7f2e200073d0 comp rx=0 tx=0).stop 2023-11-16T16:37:51.731 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.728+0000 7f2e2b6dc700 1 -- 172.21.15.1:0/827179054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e240ff1e0 msgr2=0x7f2e240fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:51.731 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.728+0000 7f2e2b6dc700 1 --2- 172.21.15.1:0/827179054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e240ff1e0 0x7f2e240fff50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2e140095c0 tx=0x7f2e1400f710 comp rx=0 tx=0).stop 2023-11-16T16:37:51.731 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.729+0000 7f2e2b6dc700 1 -- 172.21.15.1:0/827179054 shutdown_connections 2023-11-16T16:37:51.732 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.729+0000 7f2e2b6dc700 1 --2- 172.21.15.1:0/827179054 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f2e100a1260 0x7f2e100a3710 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:51.732 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.729+0000 7f2e2b6dc700 1 --2- 172.21.15.1:0/827179054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e24105290 0x7f2e241a4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:51.732 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.729+0000 7f2e2b6dc700 1 --2- 172.21.15.1:0/827179054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2e240fe830 0x7f2e240ffa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:51.732 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.729+0000 7f2e2b6dc700 1 --2- 172.21.15.1:0/827179054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e240ff1e0 0x7f2e240fff50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:51.732 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.729+0000 7f2e2b6dc700 1 -- 172.21.15.1:0/827179054 >> 172.21.15.1:0/827179054 conn(0x7f2e240758a0 msgr2=0x7f2e240fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:51.732 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.729+0000 7f2e2b6dc700 1 -- 172.21.15.1:0/827179054 shutdown_connections 2023-11-16T16:37:51.732 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:51.729+0000 7f2e2b6dc700 1 -- 172.21.15.1:0/827179054 wait complete. 2023-11-16T16:37:51.760 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:51 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/4155324510' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-11-16T16:37:51.760 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:51 smithi001 ceph-mon[98800]: pgmap v279: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2023-11-16T16:37:51.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:51 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/4155324510' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-11-16T16:37:51.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:51 smithi053 ceph-mon[100638]: pgmap v279: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2023-11-16T16:37:52.004 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462950 got 201863462952 for osd.6 2023-11-16T16:37:52.005 DEBUG:teuthology.parallel:result is None 2023-11-16T16:37:52.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:51 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/4155324510' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-11-16T16:37:52.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:51 smithi026 ceph-mon[100112]: pgmap v279: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2023-11-16T16:37:52.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.213+0000 7f4495ce9700 1 -- 172.21.15.1:0/2903705530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4490101540 msgr2=0x7f4490101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:52.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.213+0000 7f4495ce9700 1 --2- 172.21.15.1:0/2903705530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4490101540 0x7f4490101910 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4480009990 tx=0x7f4480009ca0 comp rx=0 tx=0).stop 2023-11-16T16:37:52.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.213+0000 7f4495ce9700 1 -- 172.21.15.1:0/2903705530 shutdown_connections 2023-11-16T16:37:52.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.213+0000 7f4495ce9700 1 --2- 172.21.15.1:0/2903705530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f449010a890 0x7f449010ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:52.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.213+0000 7f4495ce9700 1 --2- 172.21.15.1:0/2903705530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4490101540 0x7f4490101910 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:52.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.213+0000 7f4495ce9700 1 --2- 172.21.15.1:0/2903705530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4490101e50 0x7f449010a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:52.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.213+0000 7f4495ce9700 1 -- 172.21.15.1:0/2903705530 >> 172.21.15.1:0/2903705530 conn(0x7f44900770a0 msgr2=0x7f44900774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:52.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.213+0000 7f4495ce9700 1 -- 172.21.15.1:0/2903705530 shutdown_connections 2023-11-16T16:37:52.217 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.213+0000 7f4495ce9700 1 -- 172.21.15.1:0/2903705530 wait complete. 2023-11-16T16:37:52.218 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.214+0000 7f4495ce9700 1 Processor -- start 2023-11-16T16:37:52.218 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.214+0000 7f4495ce9700 1 -- start start 2023-11-16T16:37:52.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.214+0000 7f4495ce9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4490101540 0x7f44901964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:52.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.214+0000 7f4495ce9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4490101e50 0x7f44901969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:52.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.214+0000 7f4495ce9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f449010a890 0x7f449019ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:52.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.214+0000 7f4495ce9700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449019b460 con 0x7f4490101e50 2023-11-16T16:37:52.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.214+0000 7f4495ce9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449019b5d0 con 0x7f449010a890 2023-11-16T16:37:52.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.214+0000 7f4495ce9700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449019b790 con 0x7f4490101540 2023-11-16T16:37:52.220 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.215+0000 7f448effd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4490101e50 0x7f44901969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:52.220 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.215+0000 7f448effd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4490101e50 0x7f44901969e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41940/0 (socket says 172.21.15.1:41940) 2023-11-16T16:37:52.220 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.215+0000 7f448effd700 1 -- 172.21.15.1:0/2766675674 learned_addr learned my addr 172.21.15.1:0/2766675674 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:52.220 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.215+0000 7f448f7fe700 1 --2- 172.21.15.1:0/2766675674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4490101540 0x7f44901964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:52.220 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.215+0000 7f448ffff700 1 --2- 172.21.15.1:0/2766675674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f449010a890 0x7f449019ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:52.220 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.215+0000 7f448effd700 1 -- 172.21.15.1:0/2766675674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4490101540 msgr2=0x7f44901964a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:52.220 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.215+0000 7f448effd700 1 --2- 172.21.15.1:0/2766675674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4490101540 0x7f44901964a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:52.221 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.215+0000 7f448effd700 1 -- 172.21.15.1:0/2766675674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f449010a890 msgr2=0x7f449019ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:52.221 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.215+0000 7f448effd700 1 --2- 172.21.15.1:0/2766675674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f449010a890 0x7f449019ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:52.221 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.215+0000 7f448effd700 1 -- 172.21.15.1:0/2766675674 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44800095f0 con 0x7f4490101e50 2023-11-16T16:37:52.221 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.215+0000 7f448effd700 1 --2- 172.21.15.1:0/2766675674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4490101e50 0x7f44901969e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f448400c970 tx=0x7f448400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:52.221 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.215+0000 7f448cff9700 1 -- 172.21.15.1:0/2766675674 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4484007aa0 con 0x7f4490101e50 2023-11-16T16:37:52.221 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.215+0000 7f448cff9700 1 -- 172.21.15.1:0/2766675674 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4484004cf0 con 0x7f4490101e50 2023-11-16T16:37:52.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.215+0000 7f448cff9700 1 -- 172.21.15.1:0/2766675674 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4484005600 con 0x7f4490101e50 2023-11-16T16:37:52.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.215+0000 7f4495ce9700 1 -- 172.21.15.1:0/2766675674 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4490110330 con 0x7f4490101e50 2023-11-16T16:37:52.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.215+0000 7f4495ce9700 1 -- 172.21.15.1:0/2766675674 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44901101a0 con 0x7f4490101e50 2023-11-16T16:37:52.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.216+0000 7f4495ce9700 1 -- 172.21.15.1:0/2766675674 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44901104c0 con 0x7f4490101e50 2023-11-16T16:37:52.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.217+0000 7f448cff9700 1 -- 172.21.15.1:0/2766675674 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f4484004750 con 0x7f4490101e50 2023-11-16T16:37:52.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.218+0000 7f448cff9700 1 --2- 172.21.15.1:0/2766675674 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f44780a1260 0x7f44780a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:52.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.218+0000 7f448f7fe700 1 --2- 172.21.15.1:0/2766675674 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f44780a1260 0x7f44780a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:52.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.218+0000 7f448cff9700 1 -- 172.21.15.1:0/2766675674 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4484013070 con 0x7f4490101e50 2023-11-16T16:37:52.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.219+0000 7f448f7fe700 1 --2- 172.21.15.1:0/2766675674 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f44780a1260 0x7f44780a3710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4480006010 tx=0x7f4480003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:52.224 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.223+0000 7f448cff9700 1 -- 172.21.15.1:0/2766675674 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4484095ea0 con 0x7f4490101e50 2023-11-16T16:37:52.264 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888110 got 146028888113 for osd.4 2023-11-16T16:37:52.264 DEBUG:teuthology.parallel:result is None 2023-11-16T16:37:52.426 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.424+0000 7f4495ce9700 1 -- 172.21.15.1:0/2766675674 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4490066ea0 con 0x7f4490101e50 2023-11-16T16:37:52.426 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.424+0000 7f448cff9700 1 -- 172.21.15.1:0/2766675674 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f44840994c0 con 0x7f4490101e50 2023-11-16T16:37:52.426 INFO:teuthology.orchestra.run.smithi001.stdout:77309411387 2023-11-16T16:37:52.428 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.426+0000 7f4495ce9700 1 -- 172.21.15.1:0/2766675674 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f44780a1260 msgr2=0x7f44780a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:52.428 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.426+0000 7f4495ce9700 1 --2- 172.21.15.1:0/2766675674 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f44780a1260 0x7f44780a3710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4480006010 tx=0x7f4480003680 comp rx=0 tx=0).stop 2023-11-16T16:37:52.429 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.426+0000 7f4495ce9700 1 -- 172.21.15.1:0/2766675674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4490101e50 msgr2=0x7f44901969e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:52.429 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.426+0000 7f4495ce9700 1 --2- 172.21.15.1:0/2766675674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4490101e50 0x7f44901969e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f448400c970 tx=0x7f448400cd30 comp rx=0 tx=0).stop 2023-11-16T16:37:52.429 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.426+0000 7f4495ce9700 1 -- 172.21.15.1:0/2766675674 shutdown_connections 2023-11-16T16:37:52.429 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.426+0000 7f4495ce9700 1 --2- 172.21.15.1:0/2766675674 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f44780a1260 0x7f44780a3710 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:52.429 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.426+0000 7f4495ce9700 1 --2- 172.21.15.1:0/2766675674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f449010a890 0x7f449019ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:52.429 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.426+0000 7f4495ce9700 1 --2- 172.21.15.1:0/2766675674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4490101540 0x7f44901964a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:52.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.426+0000 7f4495ce9700 1 --2- 172.21.15.1:0/2766675674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4490101e50 0x7f44901969e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:52.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.426+0000 7f4495ce9700 1 -- 172.21.15.1:0/2766675674 >> 172.21.15.1:0/2766675674 conn(0x7f44900770a0 msgr2=0x7f44900fbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:52.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.426+0000 7f4495ce9700 1 -- 172.21.15.1:0/2766675674 shutdown_connections 2023-11-16T16:37:52.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:52.426+0000 7f4495ce9700 1 -- 172.21.15.1:0/2766675674 wait complete. 2023-11-16T16:37:52.760 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:52 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/827179054' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-11-16T16:37:52.760 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:52 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/2766675674' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-11-16T16:37:52.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:52 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/827179054' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-11-16T16:37:52.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:52 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/2766675674' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-11-16T16:37:52.942 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299426 got 223338299428 for osd.7 2023-11-16T16:37:52.942 DEBUG:teuthology.parallel:result is None 2023-11-16T16:37:53.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:52 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/827179054' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-11-16T16:37:53.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:52 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/2766675674' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-11-16T16:37:53.221 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:37:53.253 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:37:53.596 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.592+0000 7efc09a59700 1 -- 172.21.15.1:0/3983743460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc04105650 msgr2=0x7efc0410d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:53.597 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.592+0000 7efc09a59700 1 --2- 172.21.15.1:0/3983743460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc04105650 0x7efc0410d1d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efbfc00b0d0 tx=0x7efbfc00b3e0 comp rx=0 tx=0).stop 2023-11-16T16:37:53.597 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.593+0000 7efc09a59700 1 -- 172.21.15.1:0/3983743460 shutdown_connections 2023-11-16T16:37:53.597 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.593+0000 7efc09a59700 1 --2- 172.21.15.1:0/3983743460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc04105650 0x7efc0410d1d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:53.597 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.593+0000 7efc09a59700 1 --2- 172.21.15.1:0/3983743460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc041006f0 0x7efc04100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:53.597 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.593+0000 7efc09a59700 1 --2- 172.21.15.1:0/3983743460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc04101090 0x7efc04104f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:53.597 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.593+0000 7efc09a59700 1 -- 172.21.15.1:0/3983743460 >> 172.21.15.1:0/3983743460 conn(0x7efc04078460 msgr2=0x7efc04078860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:53.597 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.593+0000 7efc09a59700 1 -- 172.21.15.1:0/3983743460 shutdown_connections 2023-11-16T16:37:53.597 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.593+0000 7efc09a59700 1 -- 172.21.15.1:0/3983743460 wait complete. 2023-11-16T16:37:53.597 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.593+0000 7efc09a59700 1 Processor -- start 2023-11-16T16:37:53.598 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.593+0000 7efc09a59700 1 -- start start 2023-11-16T16:37:53.598 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.594+0000 7efc09a59700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc041006f0 0x7efc04072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:53.598 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.594+0000 7efc09a59700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc04101090 0x7efc04073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:53.598 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.594+0000 7efc09a59700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc04105650 0x7efc040738b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:53.598 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.594+0000 7efc09a59700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc041a4960 con 0x7efc041006f0 2023-11-16T16:37:53.598 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.594+0000 7efc09a59700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc041a4ad0 con 0x7efc04101090 2023-11-16T16:37:53.599 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.594+0000 7efc09a59700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc041a4c90 con 0x7efc04105650 2023-11-16T16:37:53.599 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.594+0000 7efc02ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc041006f0 0x7efc04072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:53.599 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.594+0000 7efc02ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc041006f0 0x7efc04072df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41952/0 (socket says 172.21.15.1:41952) 2023-11-16T16:37:53.599 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.594+0000 7efc02ffd700 1 -- 172.21.15.1:0/3319948110 learned_addr learned my addr 172.21.15.1:0/3319948110 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:53.599 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.594+0000 7efc027fc700 1 --2- 172.21.15.1:0/3319948110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc04101090 0x7efc04073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:53.599 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.594+0000 7efc037fe700 1 --2- 172.21.15.1:0/3319948110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc04105650 0x7efc040738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:53.599 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.594+0000 7efc02ffd700 1 -- 172.21.15.1:0/3319948110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc04105650 msgr2=0x7efc040738b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:53.599 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.594+0000 7efc02ffd700 1 --2- 172.21.15.1:0/3319948110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc04105650 0x7efc040738b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:53.599 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.594+0000 7efc02ffd700 1 -- 172.21.15.1:0/3319948110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc04101090 msgr2=0x7efc04073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:53.599 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.594+0000 7efc02ffd700 1 --2- 172.21.15.1:0/3319948110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc04101090 0x7efc04073330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:53.600 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.594+0000 7efc02ffd700 1 -- 172.21.15.1:0/3319948110 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbfc009d00 con 0x7efc041006f0 2023-11-16T16:37:53.600 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.594+0000 7efc037fe700 1 --2- 172.21.15.1:0/3319948110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc04105650 0x7efc040738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:37:53.600 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.594+0000 7efc02ffd700 1 --2- 172.21.15.1:0/3319948110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc041006f0 0x7efc04072df0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7efbf800d980 tx=0x7efbf800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:53.600 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.594+0000 7efc08a57700 1 -- 172.21.15.1:0/3319948110 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf8004a30 con 0x7efc041006f0 2023-11-16T16:37:53.600 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.594+0000 7efc08a57700 1 -- 172.21.15.1:0/3319948110 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efbf80054e0 con 0x7efc041006f0 2023-11-16T16:37:53.600 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.594+0000 7efc09a59700 1 -- 172.21.15.1:0/3319948110 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc041103c0 con 0x7efc041006f0 2023-11-16T16:37:53.600 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.594+0000 7efc08a57700 1 -- 172.21.15.1:0/3319948110 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf8009c90 con 0x7efc041006f0 2023-11-16T16:37:53.600 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.594+0000 7efc09a59700 1 -- 172.21.15.1:0/3319948110 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc04110230 con 0x7efc041006f0 2023-11-16T16:37:53.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.596+0000 7efc09a59700 1 -- 172.21.15.1:0/3319948110 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc04110550 con 0x7efc041006f0 2023-11-16T16:37:53.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.597+0000 7efc08a57700 1 -- 172.21.15.1:0/3319948110 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7efbf8004c20 con 0x7efc041006f0 2023-11-16T16:37:53.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.598+0000 7efc08a57700 1 --2- 172.21.15.1:0/3319948110 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7efbec0a1260 0x7efbec0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:53.602 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.598+0000 7efc027fc700 1 --2- 172.21.15.1:0/3319948110 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7efbec0a1260 0x7efbec0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:53.602 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.598+0000 7efc08a57700 1 -- 172.21.15.1:0/3319948110 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7efbf80ccab0 con 0x7efc041006f0 2023-11-16T16:37:53.602 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.598+0000 7efc027fc700 1 --2- 172.21.15.1:0/3319948110 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7efbec0a1260 0x7efbec0a3710 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efc040745e0 tx=0x7efbf40073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:53.603 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.602+0000 7efc08a57700 1 -- 172.21.15.1:0/3319948110 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7efbf80962d0 con 0x7efc041006f0 2023-11-16T16:37:53.808 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.806+0000 7efc09a59700 1 -- 172.21.15.1:0/3319948110 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7efc04074320 con 0x7efc041006f0 2023-11-16T16:37:53.808 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.806+0000 7efc08a57700 1 -- 172.21.15.1:0/3319948110 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efc04074320 con 0x7efc041006f0 2023-11-16T16:37:53.808 INFO:teuthology.orchestra.run.smithi001.stdout:253403070497 2023-11-16T16:37:53.810 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.808+0000 7efc09a59700 1 -- 172.21.15.1:0/3319948110 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7efbec0a1260 msgr2=0x7efbec0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:53.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.808+0000 7efc09a59700 1 --2- 172.21.15.1:0/3319948110 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7efbec0a1260 0x7efbec0a3710 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efc040745e0 tx=0x7efbf40073d0 comp rx=0 tx=0).stop 2023-11-16T16:37:53.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.808+0000 7efc09a59700 1 -- 172.21.15.1:0/3319948110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc041006f0 msgr2=0x7efc04072df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:53.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.808+0000 7efc09a59700 1 --2- 172.21.15.1:0/3319948110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc041006f0 0x7efc04072df0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7efbf800d980 tx=0x7efbf800dc90 comp rx=0 tx=0).stop 2023-11-16T16:37:53.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.809+0000 7efc09a59700 1 -- 172.21.15.1:0/3319948110 shutdown_connections 2023-11-16T16:37:53.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.809+0000 7efc09a59700 1 --2- 172.21.15.1:0/3319948110 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7efbec0a1260 0x7efbec0a3710 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:53.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.809+0000 7efc09a59700 1 --2- 172.21.15.1:0/3319948110 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7efc04105650 0x7efc040738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:53.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.809+0000 7efc09a59700 1 --2- 172.21.15.1:0/3319948110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc041006f0 0x7efc04072df0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:53.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.809+0000 7efc09a59700 1 --2- 172.21.15.1:0/3319948110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc04101090 0x7efc04073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:53.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.809+0000 7efc09a59700 1 -- 172.21.15.1:0/3319948110 >> 172.21.15.1:0/3319948110 conn(0x7efc04078460 msgr2=0x7efc040fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:53.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.809+0000 7efc09a59700 1 -- 172.21.15.1:0/3319948110 shutdown_connections 2023-11-16T16:37:53.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.809+0000 7efc09a59700 1 -- 172.21.15.1:0/3319948110 wait complete. 2023-11-16T16:37:53.827 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:53 smithi001 ceph-mon[98800]: pgmap v280: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2023-11-16T16:37:53.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:53 smithi053 ceph-mon[100638]: pgmap v280: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2023-11-16T16:37:53.936 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411385 got 77309411387 for osd.2 2023-11-16T16:37:53.936 DEBUG:teuthology.parallel:result is None 2023-11-16T16:37:54.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:53 smithi026 ceph-mon[100112]: pgmap v280: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2023-11-16T16:37:54.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.701+0000 7f996897c700 1 -- 172.21.15.1:0/3895107158 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99640ff9e0 msgr2=0x7f996410c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:54.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.701+0000 7f996897c700 1 --2- 172.21.15.1:0/3895107158 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99640ff9e0 0x7f996410c740 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9958009990 tx=0x7f9958009ca0 comp rx=0 tx=0).stop 2023-11-16T16:37:54.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.701+0000 7f996897c700 1 -- 172.21.15.1:0/3895107158 shutdown_connections 2023-11-16T16:37:54.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.701+0000 7f996897c700 1 --2- 172.21.15.1:0/3895107158 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f996410cc80 0x7f996410f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:54.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.701+0000 7f996897c700 1 --2- 172.21.15.1:0/3895107158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99640ff0d0 0x7f99640ff4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:54.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.701+0000 7f996897c700 1 --2- 172.21.15.1:0/3895107158 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99640ff9e0 0x7f996410c740 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:54.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.701+0000 7f996897c700 1 -- 172.21.15.1:0/3895107158 >> 172.21.15.1:0/3895107158 conn(0x7f99640770a0 msgr2=0x7f99640774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:54.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.701+0000 7f996897c700 1 -- 172.21.15.1:0/3895107158 shutdown_connections 2023-11-16T16:37:54.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.701+0000 7f996897c700 1 -- 172.21.15.1:0/3895107158 wait complete. 2023-11-16T16:37:54.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.702+0000 7f996897c700 1 Processor -- start 2023-11-16T16:37:54.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.702+0000 7f996897c700 1 -- start start 2023-11-16T16:37:54.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.702+0000 7f996897c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99640ff0d0 0x7f996419e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:54.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.702+0000 7f996897c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99640ff9e0 0x7f996419ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:54.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.702+0000 7f996897c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f996410cc80 0x7f99641985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:54.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.702+0000 7f996897c700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9964198c20 con 0x7f99640ff0d0 2023-11-16T16:37:54.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.702+0000 7f996897c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9964198d90 con 0x7f996410cc80 2023-11-16T16:37:54.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.703+0000 7f996897c700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9964198f50 con 0x7f99640ff9e0 2023-11-16T16:37:54.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.703+0000 7f996259c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99640ff0d0 0x7f996419e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:54.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.703+0000 7f996259c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99640ff0d0 0x7f996419e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41966/0 (socket says 172.21.15.1:41966) 2023-11-16T16:37:54.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.703+0000 7f996259c700 1 -- 172.21.15.1:0/1751343654 learned_addr learned my addr 172.21.15.1:0/1751343654 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:54.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.703+0000 7f9962d9d700 1 --2- 172.21.15.1:0/1751343654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f996410cc80 0x7f99641985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:54.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.703+0000 7f9961d9b700 1 --2- 172.21.15.1:0/1751343654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99640ff9e0 0x7f996419ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:54.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.703+0000 7f996259c700 1 -- 172.21.15.1:0/1751343654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99640ff9e0 msgr2=0x7f996419ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:54.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.703+0000 7f996259c700 1 --2- 172.21.15.1:0/1751343654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99640ff9e0 0x7f996419ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:54.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.703+0000 7f996259c700 1 -- 172.21.15.1:0/1751343654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f996410cc80 msgr2=0x7f99641985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:54.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.703+0000 7f996259c700 1 --2- 172.21.15.1:0/1751343654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f996410cc80 0x7f99641985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:54.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.703+0000 7f996259c700 1 -- 172.21.15.1:0/1751343654 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99580095f0 con 0x7f99640ff0d0 2023-11-16T16:37:54.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.703+0000 7f9961d9b700 1 --2- 172.21.15.1:0/1751343654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99640ff9e0 0x7f996419ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:37:54.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.703+0000 7f996259c700 1 --2- 172.21.15.1:0/1751343654 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99640ff0d0 0x7f996419e530 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f995400e9a0 tx=0x7f995400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:54.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.703+0000 7f99537fe700 1 -- 172.21.15.1:0/1751343654 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9954009790 con 0x7f99640ff0d0 2023-11-16T16:37:54.049 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.703+0000 7f996897c700 1 -- 172.21.15.1:0/1751343654 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9964112570 con 0x7f99640ff0d0 2023-11-16T16:37:54.049 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.703+0000 7f99537fe700 1 -- 172.21.15.1:0/1751343654 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f995400cbd0 con 0x7f99640ff0d0 2023-11-16T16:37:54.049 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.703+0000 7f99537fe700 1 -- 172.21.15.1:0/1751343654 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9954018680 con 0x7f99640ff0d0 2023-11-16T16:37:54.049 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.703+0000 7f996897c700 1 -- 172.21.15.1:0/1751343654 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99641123e0 con 0x7f99640ff0d0 2023-11-16T16:37:54.049 INFO:teuthology.orchestra.run.smithi001.stdout:111669149749 2023-11-16T16:37:54.049 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.705+0000 7f996897c700 1 -- 172.21.15.1:0/1751343654 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9964112700 con 0x7f99640ff0d0 2023-11-16T16:37:54.049 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.706+0000 7f99537fe700 1 -- 172.21.15.1:0/1751343654 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f995400cd40 con 0x7f99640ff0d0 2023-11-16T16:37:54.049 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.706+0000 7f99537fe700 1 --2- 172.21.15.1:0/1751343654 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f994c0a11e0 0x7f994c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:54.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.706+0000 7f9961d9b700 1 --2- 172.21.15.1:0/1751343654 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f994c0a11e0 0x7f994c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:54.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.706+0000 7f99537fe700 1 -- 172.21.15.1:0/1751343654 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9954014070 con 0x7f99640ff0d0 2023-11-16T16:37:54.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.707+0000 7f9961d9b700 1 --2- 172.21.15.1:0/1751343654 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f994c0a11e0 0x7f994c0a3690 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f995800b2e0 tx=0x7f9958009530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:54.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.711+0000 7f99537fe700 1 -- 172.21.15.1:0/1751343654 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9954051730 con 0x7f99640ff0d0 2023-11-16T16:37:54.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.933+0000 7f996897c700 1 -- 172.21.15.1:0/1751343654 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9964199ac0 con 0x7f99640ff0d0 2023-11-16T16:37:54.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.941+0000 7f99537fe700 1 -- 172.21.15.1:0/1751343654 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f99540962e0 con 0x7f99640ff0d0 2023-11-16T16:37:54.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.944+0000 7f996897c700 1 -- 172.21.15.1:0/1751343654 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f994c0a11e0 msgr2=0x7f994c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:54.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.944+0000 7f996897c700 1 --2- 172.21.15.1:0/1751343654 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f994c0a11e0 0x7f994c0a3690 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f995800b2e0 tx=0x7f9958009530 comp rx=0 tx=0).stop 2023-11-16T16:37:54.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.944+0000 7f996897c700 1 -- 172.21.15.1:0/1751343654 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99640ff0d0 msgr2=0x7f996419e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:54.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.944+0000 7f996897c700 1 --2- 172.21.15.1:0/1751343654 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99640ff0d0 0x7f996419e530 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f995400e9a0 tx=0x7f995400ecb0 comp rx=0 tx=0).stop 2023-11-16T16:37:54.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.944+0000 7f996897c700 1 -- 172.21.15.1:0/1751343654 shutdown_connections 2023-11-16T16:37:54.051 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.944+0000 7f996897c700 1 --2- 172.21.15.1:0/1751343654 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f994c0a11e0 0x7f994c0a3690 secure :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f995800b2e0 tx=0x7f9958009530 comp rx=0 tx=0).stop 2023-11-16T16:37:54.051 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.944+0000 7f996897c700 1 --2- 172.21.15.1:0/1751343654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f996410cc80 0x7f99641985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:54.051 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.944+0000 7f996897c700 1 --2- 172.21.15.1:0/1751343654 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99640ff0d0 0x7f996419e530 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:54.051 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.944+0000 7f996897c700 1 --2- 172.21.15.1:0/1751343654 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99640ff9e0 0x7f996419ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:54.051 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.944+0000 7f996897c700 1 -- 172.21.15.1:0/1751343654 >> 172.21.15.1:0/1751343654 conn(0x7f99640770a0 msgr2=0x7f996410e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:54.051 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.944+0000 7f996897c700 1 -- 172.21.15.1:0/1751343654 shutdown_connections 2023-11-16T16:37:54.051 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.944+0000 7f996897c700 1 -- 172.21.15.1:0/1751343654 wait complete. 2023-11-16T16:37:54.128 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.919+0000 7fc765a0f700 1 -- 172.21.15.1:0/3339798585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc760100490 msgr2=0x7fc760100900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:54.128 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.919+0000 7fc765a0f700 1 --2- 172.21.15.1:0/3339798585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc760100490 0x7fc760100900 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc750009990 tx=0x7fc750009ca0 comp rx=0 tx=0).stop 2023-11-16T16:37:54.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.921+0000 7fc765a0f700 1 -- 172.21.15.1:0/3339798585 shutdown_connections 2023-11-16T16:37:54.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.921+0000 7fc765a0f700 1 --2- 172.21.15.1:0/3339798585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc760100e40 0x7fc76010d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:54.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.921+0000 7fc765a0f700 1 --2- 172.21.15.1:0/3339798585 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc760106370 0x7fc760106740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:54.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.921+0000 7fc765a0f700 1 --2- 172.21.15.1:0/3339798585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc760100490 0x7fc760100900 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:54.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.921+0000 7fc765a0f700 1 -- 172.21.15.1:0/3339798585 >> 172.21.15.1:0/3339798585 conn(0x7fc760078430 msgr2=0x7fc760078830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:54.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.926+0000 7fc765a0f700 1 -- 172.21.15.1:0/3339798585 shutdown_connections 2023-11-16T16:37:54.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.926+0000 7fc765a0f700 1 -- 172.21.15.1:0/3339798585 wait complete. 2023-11-16T16:37:54.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.927+0000 7fc765a0f700 1 Processor -- start 2023-11-16T16:37:54.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.927+0000 7fc765a0f700 1 -- start start 2023-11-16T16:37:54.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.928+0000 7fc765a0f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc760100490 0x7fc7601940f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:54.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.928+0000 7fc765a0f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc760100e40 0x7fc760194630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:54.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.928+0000 7fc765a0f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc760106370 0x7fc760198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:54.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.928+0000 7fc765a0f700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7601990b0 con 0x7fc760100e40 2023-11-16T16:37:54.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.928+0000 7fc765a0f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc760199220 con 0x7fc760100490 2023-11-16T16:37:54.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.928+0000 7fc765a0f700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7601993e0 con 0x7fc760106370 2023-11-16T16:37:54.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.928+0000 7fc75e7fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc760100e40 0x7fc760194630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:54.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.928+0000 7fc75e7fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc760100e40 0x7fc760194630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41980/0 (socket says 172.21.15.1:41980) 2023-11-16T16:37:54.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.928+0000 7fc75e7fc700 1 -- 172.21.15.1:0/3904505815 learned_addr learned my addr 172.21.15.1:0/3904505815 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:54.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.928+0000 7fc75f7fe700 1 --2- 172.21.15.1:0/3904505815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc760106370 0x7fc760198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:54.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.928+0000 7fc75e7fc700 1 -- 172.21.15.1:0/3904505815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc760106370 msgr2=0x7fc760198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:54.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.928+0000 7fc75e7fc700 1 --2- 172.21.15.1:0/3904505815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc760106370 0x7fc760198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:54.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.928+0000 7fc75e7fc700 1 -- 172.21.15.1:0/3904505815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc760100490 msgr2=0x7fc7601940f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-16T16:37:54.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.928+0000 7fc75e7fc700 1 --2- 172.21.15.1:0/3904505815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc760100490 0x7fc7601940f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:54.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.929+0000 7fc75e7fc700 1 -- 172.21.15.1:0/3904505815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7500095f0 con 0x7fc760100e40 2023-11-16T16:37:54.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.929+0000 7fc75e7fc700 1 --2- 172.21.15.1:0/3904505815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc760100e40 0x7fc760194630 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc75000b650 tx=0x7fc750004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:54.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.929+0000 7fc764a0d700 1 -- 172.21.15.1:0/3904505815 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75001d070 con 0x7fc760100e40 2023-11-16T16:37:54.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.929+0000 7fc764a0d700 1 -- 172.21.15.1:0/3904505815 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc750022470 con 0x7fc760100e40 2023-11-16T16:37:54.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.929+0000 7fc765a0f700 1 -- 172.21.15.1:0/3904505815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc76010e200 con 0x7fc760100e40 2023-11-16T16:37:54.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.929+0000 7fc765a0f700 1 -- 172.21.15.1:0/3904505815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc76010e070 con 0x7fc760100e40 2023-11-16T16:37:54.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.929+0000 7fc764a0d700 1 -- 172.21.15.1:0/3904505815 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc750017650 con 0x7fc760100e40 2023-11-16T16:37:54.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.932+0000 7fc765a0f700 1 -- 172.21.15.1:0/3904505815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc74c005320 con 0x7fc760100e40 2023-11-16T16:37:54.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.933+0000 7fc764a0d700 1 -- 172.21.15.1:0/3904505815 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fc75000fb10 con 0x7fc760100e40 2023-11-16T16:37:54.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.933+0000 7fc764a0d700 1 --2- 172.21.15.1:0/3904505815 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc7480a1260 0x7fc7480a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:54.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.934+0000 7fc75effd700 1 --2- 172.21.15.1:0/3904505815 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc7480a1260 0x7fc7480a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:54.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.934+0000 7fc764a0d700 1 -- 172.21.15.1:0/3904505815 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc7500cdc70 con 0x7fc760100e40 2023-11-16T16:37:54.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.934+0000 7fc75effd700 1 --2- 172.21.15.1:0/3904505815 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc7480a1260 0x7fc7480a3710 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc758009240 tx=0x7fc758009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:54.132 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:53.941+0000 7fc764a0d700 1 -- 172.21.15.1:0/3904505815 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc7500974a0 con 0x7fc760100e40 2023-11-16T16:37:54.166 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.164+0000 7fc765a0f700 1 -- 172.21.15.1:0/3904505815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fc74c005cc0 con 0x7fc760100e40 2023-11-16T16:37:54.166 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.165+0000 7fc764a0d700 1 -- 172.21.15.1:0/3904505815 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fc75009aac0 con 0x7fc760100e40 2023-11-16T16:37:54.167 INFO:teuthology.orchestra.run.smithi001.stdout:309237645335 2023-11-16T16:37:54.168 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.167+0000 7fc765a0f700 1 -- 172.21.15.1:0/3904505815 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc7480a1260 msgr2=0x7fc7480a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:54.169 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.167+0000 7fc765a0f700 1 --2- 172.21.15.1:0/3904505815 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc7480a1260 0x7fc7480a3710 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc758009240 tx=0x7fc758009920 comp rx=0 tx=0).stop 2023-11-16T16:37:54.169 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.167+0000 7fc765a0f700 1 -- 172.21.15.1:0/3904505815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc760100e40 msgr2=0x7fc760194630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:54.169 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.167+0000 7fc765a0f700 1 --2- 172.21.15.1:0/3904505815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc760100e40 0x7fc760194630 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc75000b650 tx=0x7fc750004730 comp rx=0 tx=0).stop 2023-11-16T16:37:54.169 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.167+0000 7fc765a0f700 1 -- 172.21.15.1:0/3904505815 shutdown_connections 2023-11-16T16:37:54.169 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.167+0000 7fc765a0f700 1 --2- 172.21.15.1:0/3904505815 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc7480a1260 0x7fc7480a3710 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:54.169 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.167+0000 7fc765a0f700 1 --2- 172.21.15.1:0/3904505815 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc760106370 0x7fc760198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:54.169 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.167+0000 7fc765a0f700 1 --2- 172.21.15.1:0/3904505815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc760100490 0x7fc7601940f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:54.169 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.167+0000 7fc765a0f700 1 --2- 172.21.15.1:0/3904505815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc760100e40 0x7fc760194630 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:54.169 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.167+0000 7fc765a0f700 1 -- 172.21.15.1:0/3904505815 >> 172.21.15.1:0/3904505815 conn(0x7fc760078430 msgr2=0x7fc7600fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:54.169 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.167+0000 7fc765a0f700 1 -- 172.21.15.1:0/3904505815 shutdown_connections 2023-11-16T16:37:54.169 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.167+0000 7fc765a0f700 1 -- 172.21.15.1:0/3904505815 wait complete. 2023-11-16T16:37:54.552 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:54 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3319948110' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-11-16T16:37:54.552 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:54 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1751343654' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-11-16T16:37:54.552 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:54 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3904505815' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-11-16T16:37:54.586 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.493+0000 7ff10d619700 1 -- 172.21.15.1:0/3934496945 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff10810a900 msgr2=0x7ff10810cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:54.586 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.493+0000 7ff10d619700 1 --2- 172.21.15.1:0/3934496945 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff10810a900 0x7ff10810cd00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff10000b0a0 tx=0x7ff10000b3b0 comp rx=0 tx=0).stop 2023-11-16T16:37:54.586 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.493+0000 7ff10d619700 1 -- 172.21.15.1:0/3934496945 shutdown_connections 2023-11-16T16:37:54.586 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.493+0000 7ff10d619700 1 --2- 172.21.15.1:0/3934496945 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff10810a900 0x7ff10810cd00 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:54.586 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.493+0000 7ff10d619700 1 --2- 172.21.15.1:0/3934496945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff1081013e0 0x7ff1081017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:54.587 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.493+0000 7ff10d619700 1 --2- 172.21.15.1:0/3934496945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff108101cf0 0x7ff10810a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:54.587 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.493+0000 7ff10d619700 1 -- 172.21.15.1:0/3934496945 >> 172.21.15.1:0/3934496945 conn(0x7ff1080770d0 msgr2=0x7ff1080774d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:54.587 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.493+0000 7ff10d619700 1 -- 172.21.15.1:0/3934496945 shutdown_connections 2023-11-16T16:37:54.587 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.493+0000 7ff10d619700 1 -- 172.21.15.1:0/3934496945 wait complete. 2023-11-16T16:37:54.587 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.494+0000 7ff10d619700 1 Processor -- start 2023-11-16T16:37:54.587 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.494+0000 7ff10d619700 1 -- start start 2023-11-16T16:37:54.587 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.494+0000 7ff10d619700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1081013e0 0x7ff108196510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:54.587 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.494+0000 7ff10d619700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff108101cf0 0x7ff108196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:54.587 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.494+0000 7ff10d619700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff10810a900 0x7ff10819aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:54.587 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.494+0000 7ff10d619700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10819b4d0 con 0x7ff1081013e0 2023-11-16T16:37:54.587 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.494+0000 7ff10d619700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10819b640 con 0x7ff10810a900 2023-11-16T16:37:54.587 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.494+0000 7ff10d619700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10819b800 con 0x7ff108101cf0 2023-11-16T16:37:54.588 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.494+0000 7ff106ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1081013e0 0x7ff108196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:54.588 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.494+0000 7ff106ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1081013e0 0x7ff108196510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41992/0 (socket says 172.21.15.1:41992) 2023-11-16T16:37:54.588 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.494+0000 7ff106ffd700 1 -- 172.21.15.1:0/1290772226 learned_addr learned my addr 172.21.15.1:0/1290772226 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:54.588 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.495+0000 7ff106ffd700 1 -- 172.21.15.1:0/1290772226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff108101cf0 msgr2=0x7ff108196a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:54.588 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.495+0000 7ff1067fc700 1 --2- 172.21.15.1:0/1290772226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff108101cf0 0x7ff108196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:54.588 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.495+0000 7ff1077fe700 1 --2- 172.21.15.1:0/1290772226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff10810a900 0x7ff10819aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:54.588 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.495+0000 7ff106ffd700 1 --2- 172.21.15.1:0/1290772226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff108101cf0 0x7ff108196a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:54.588 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.495+0000 7ff106ffd700 1 -- 172.21.15.1:0/1290772226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff10810a900 msgr2=0x7ff10819aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:54.588 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.495+0000 7ff106ffd700 1 --2- 172.21.15.1:0/1290772226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff10810a900 0x7ff10819aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:54.588 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.495+0000 7ff106ffd700 1 -- 172.21.15.1:0/1290772226 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff100009d00 con 0x7ff1081013e0 2023-11-16T16:37:54.588 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.495+0000 7ff1067fc700 1 --2- 172.21.15.1:0/1290772226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff108101cf0 0x7ff108196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:37:54.588 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.495+0000 7ff106ffd700 1 --2- 172.21.15.1:0/1290772226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1081013e0 0x7ff108196510 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00e9c0 tx=0x7ff0fc00ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:54.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.495+0000 7ff0effff700 1 -- 172.21.15.1:0/1290772226 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc0097b0 con 0x7ff1081013e0 2023-11-16T16:37:54.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.495+0000 7ff0effff700 1 -- 172.21.15.1:0/1290772226 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0fc004cf0 con 0x7ff1081013e0 2023-11-16T16:37:54.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.495+0000 7ff10d619700 1 -- 172.21.15.1:0/1290772226 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff108110350 con 0x7ff1081013e0 2023-11-16T16:37:54.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.495+0000 7ff0effff700 1 -- 172.21.15.1:0/1290772226 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc005700 con 0x7ff1081013e0 2023-11-16T16:37:54.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.495+0000 7ff10d619700 1 -- 172.21.15.1:0/1290772226 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1081101c0 con 0x7ff1081013e0 2023-11-16T16:37:54.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.496+0000 7ff10d619700 1 -- 172.21.15.1:0/1290772226 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1081104e0 con 0x7ff1081013e0 2023-11-16T16:37:54.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.498+0000 7ff0effff700 1 -- 172.21.15.1:0/1290772226 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7ff0fc00cbd0 con 0x7ff1081013e0 2023-11-16T16:37:54.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.498+0000 7ff0effff700 1 --2- 172.21.15.1:0/1290772226 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7ff0f00a11e0 0x7ff0f00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:54.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.498+0000 7ff1067fc700 1 --2- 172.21.15.1:0/1290772226 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7ff0f00a11e0 0x7ff0f00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:54.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.498+0000 7ff0effff700 1 -- 172.21.15.1:0/1290772226 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff0fc014070 con 0x7ff1081013e0 2023-11-16T16:37:54.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.499+0000 7ff1067fc700 1 --2- 172.21.15.1:0/1290772226 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7ff0f00a11e0 0x7ff0f00a3690 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff108197b30 tx=0x7ff0f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:54.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.502+0000 7ff0effff700 1 -- 172.21.15.1:0/1290772226 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff0fc09a100 con 0x7ff1081013e0 2023-11-16T16:37:54.686 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070495 got 253403070497 for osd.8 2023-11-16T16:37:54.687 DEBUG:teuthology.parallel:result is None 2023-11-16T16:37:54.687 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:37:54.712 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.710+0000 7ff10d619700 1 -- 172.21.15.1:0/1290772226 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7ff108105220 con 0x7ff1081013e0 2023-11-16T16:37:54.713 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.711+0000 7ff0effff700 1 -- 172.21.15.1:0/1290772226 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff0fc099c90 con 0x7ff1081013e0 2023-11-16T16:37:54.713 INFO:teuthology.orchestra.run.smithi001.stdout:167503724590 2023-11-16T16:37:54.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.713+0000 7ff10d619700 1 -- 172.21.15.1:0/1290772226 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7ff0f00a11e0 msgr2=0x7ff0f00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:54.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.713+0000 7ff10d619700 1 --2- 172.21.15.1:0/1290772226 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7ff0f00a11e0 0x7ff0f00a3690 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff108197b30 tx=0x7ff0f8009260 comp rx=0 tx=0).stop 2023-11-16T16:37:54.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.713+0000 7ff10d619700 1 -- 172.21.15.1:0/1290772226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1081013e0 msgr2=0x7ff108196510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:54.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.713+0000 7ff10d619700 1 --2- 172.21.15.1:0/1290772226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1081013e0 0x7ff108196510 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00e9c0 tx=0x7ff0fc00ed80 comp rx=0 tx=0).stop 2023-11-16T16:37:54.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.715+0000 7ff10d619700 1 -- 172.21.15.1:0/1290772226 shutdown_connections 2023-11-16T16:37:54.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.715+0000 7ff10d619700 1 --2- 172.21.15.1:0/1290772226 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7ff0f00a11e0 0x7ff0f00a3690 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:54.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.715+0000 7ff10d619700 1 --2- 172.21.15.1:0/1290772226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff10810a900 0x7ff10819aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:54.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.715+0000 7ff10d619700 1 --2- 172.21.15.1:0/1290772226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff1081013e0 0x7ff108196510 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:54.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.715+0000 7ff10d619700 1 --2- 172.21.15.1:0/1290772226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff108101cf0 0x7ff108196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:54.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.715+0000 7ff10d619700 1 -- 172.21.15.1:0/1290772226 >> 172.21.15.1:0/1290772226 conn(0x7ff1080770d0 msgr2=0x7ff108102360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:54.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.715+0000 7ff10d619700 1 -- 172.21.15.1:0/1290772226 shutdown_connections 2023-11-16T16:37:54.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:54.715+0000 7ff10d619700 1 -- 172.21.15.1:0/1290772226 wait complete. 2023-11-16T16:37:54.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:54 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/3319948110' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-11-16T16:37:54.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:54 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1751343654' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-11-16T16:37:54.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:54 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/3904505815' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-11-16T16:37:55.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:54 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/3319948110' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-11-16T16:37:55.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:54 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1751343654' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-11-16T16:37:55.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:54 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/3904505815' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-11-16T16:37:55.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.073+0000 7fe524969700 1 -- 172.21.15.1:0/327135714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe5200ff460 msgr2=0x7fe5200ff830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:55.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.073+0000 7fe524969700 1 --2- 172.21.15.1:0/327135714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe5200ff460 0x7fe5200ff830 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe5100099c0 tx=0x7fe510009cd0 comp rx=0 tx=0).stop 2023-11-16T16:37:55.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.073+0000 7fe524969700 1 -- 172.21.15.1:0/327135714 shutdown_connections 2023-11-16T16:37:55.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.073+0000 7fe524969700 1 --2- 172.21.15.1:0/327135714 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe520104500 0x7fe52010c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:55.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.073+0000 7fe524969700 1 --2- 172.21.15.1:0/327135714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe5200ff460 0x7fe5200ff830 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:55.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.073+0000 7fe524969700 1 --2- 172.21.15.1:0/327135714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5200ffe00 0x7fe520103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:55.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.073+0000 7fe524969700 1 -- 172.21.15.1:0/327135714 >> 172.21.15.1:0/327135714 conn(0x7fe5200770a0 msgr2=0x7fe5200774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:55.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.073+0000 7fe524969700 1 -- 172.21.15.1:0/327135714 shutdown_connections 2023-11-16T16:37:55.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.073+0000 7fe524969700 1 -- 172.21.15.1:0/327135714 wait complete. 2023-11-16T16:37:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.074+0000 7fe524969700 1 Processor -- start 2023-11-16T16:37:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.074+0000 7fe524969700 1 -- start start 2023-11-16T16:37:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.074+0000 7fe524969700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5200ff460 0x7fe520198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.074+0000 7fe524969700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe5200ffe00 0x7fe520198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.074+0000 7fe524969700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe520104500 0x7fe52019d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.074+0000 7fe524969700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52019d630 con 0x7fe5200ffe00 2023-11-16T16:37:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.074+0000 7fe524969700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52019d7a0 con 0x7fe5200ff460 2023-11-16T16:37:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.074+0000 7fe524969700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52019d960 con 0x7fe520104500 2023-11-16T16:37:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.074+0000 7fe51dd9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe5200ffe00 0x7fe520198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.074+0000 7fe51dd9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe5200ffe00 0x7fe520198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42008/0 (socket says 172.21.15.1:42008) 2023-11-16T16:37:55.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.074+0000 7fe51dd9b700 1 -- 172.21.15.1:0/48144484 learned_addr learned my addr 172.21.15.1:0/48144484 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:55.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.075+0000 7fe51ed9d700 1 --2- 172.21.15.1:0/48144484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe520104500 0x7fe52019d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:55.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.075+0000 7fe51e59c700 1 --2- 172.21.15.1:0/48144484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5200ff460 0x7fe520198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:55.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.075+0000 7fe51dd9b700 1 -- 172.21.15.1:0/48144484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe520104500 msgr2=0x7fe52019d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:55.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.075+0000 7fe51dd9b700 1 --2- 172.21.15.1:0/48144484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe520104500 0x7fe52019d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:55.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.075+0000 7fe51dd9b700 1 -- 172.21.15.1:0/48144484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5200ff460 msgr2=0x7fe520198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:55.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.075+0000 7fe51dd9b700 1 --2- 172.21.15.1:0/48144484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5200ff460 0x7fe520198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:55.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.075+0000 7fe51dd9b700 1 -- 172.21.15.1:0/48144484 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5100095f0 con 0x7fe5200ffe00 2023-11-16T16:37:55.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.075+0000 7fe51dd9b700 1 --2- 172.21.15.1:0/48144484 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe5200ffe00 0x7fe520198bb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe51400e970 tx=0x7fe51400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:55.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.075+0000 7fe50f7fe700 1 -- 172.21.15.1:0/48144484 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe514009790 con 0x7fe5200ffe00 2023-11-16T16:37:55.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.075+0000 7fe50f7fe700 1 -- 172.21.15.1:0/48144484 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe51400eec0 con 0x7fe5200ffe00 2023-11-16T16:37:55.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.075+0000 7fe524969700 1 -- 172.21.15.1:0/48144484 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe52010f2b0 con 0x7fe5200ffe00 2023-11-16T16:37:55.080 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.075+0000 7fe50f7fe700 1 -- 172.21.15.1:0/48144484 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe514005590 con 0x7fe5200ffe00 2023-11-16T16:37:55.080 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.076+0000 7fe524969700 1 -- 172.21.15.1:0/48144484 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe52010f120 con 0x7fe5200ffe00 2023-11-16T16:37:55.080 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.077+0000 7fe524969700 1 -- 172.21.15.1:0/48144484 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe52010f440 con 0x7fe5200ffe00 2023-11-16T16:37:55.080 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.077+0000 7fe50f7fe700 1 -- 172.21.15.1:0/48144484 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fe51400cbd0 con 0x7fe5200ffe00 2023-11-16T16:37:55.081 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.078+0000 7fe50f7fe700 1 --2- 172.21.15.1:0/48144484 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fe5080a11e0 0x7fe5080a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:55.081 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.078+0000 7fe50f7fe700 1 -- 172.21.15.1:0/48144484 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe514014070 con 0x7fe5200ffe00 2023-11-16T16:37:55.081 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.079+0000 7fe51e59c700 1 --2- 172.21.15.1:0/48144484 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fe5080a11e0 0x7fe5080a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:55.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.080+0000 7fe51e59c700 1 --2- 172.21.15.1:0/48144484 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fe5080a11e0 0x7fe5080a3690 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe510009990 tx=0x7fe51001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:55.085 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.083+0000 7fe50f7fe700 1 -- 172.21.15.1:0/48144484 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe5140962d0 con 0x7fe5200ffe00 2023-11-16T16:37:55.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.283+0000 7fe524969700 1 -- 172.21.15.1:0/48144484 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7fe5201999d0 con 0x7fe5200ffe00 2023-11-16T16:37:55.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.283+0000 7fe50f7fe700 1 -- 172.21.15.1:0/48144484 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe514004ef0 con 0x7fe5200ffe00 2023-11-16T16:37:55.285 INFO:teuthology.orchestra.run.smithi001.stdout:287762808858 2023-11-16T16:37:55.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.285+0000 7fe524969700 1 -- 172.21.15.1:0/48144484 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fe5080a11e0 msgr2=0x7fe5080a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:55.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.285+0000 7fe524969700 1 --2- 172.21.15.1:0/48144484 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fe5080a11e0 0x7fe5080a3690 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe510009990 tx=0x7fe51001a040 comp rx=0 tx=0).stop 2023-11-16T16:37:55.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.285+0000 7fe524969700 1 -- 172.21.15.1:0/48144484 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe5200ffe00 msgr2=0x7fe520198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:55.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.285+0000 7fe524969700 1 --2- 172.21.15.1:0/48144484 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe5200ffe00 0x7fe520198bb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe51400e970 tx=0x7fe51400ec80 comp rx=0 tx=0).stop 2023-11-16T16:37:55.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.285+0000 7fe524969700 1 -- 172.21.15.1:0/48144484 shutdown_connections 2023-11-16T16:37:55.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.285+0000 7fe524969700 1 --2- 172.21.15.1:0/48144484 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fe5080a11e0 0x7fe5080a3690 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:55.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.285+0000 7fe524969700 1 --2- 172.21.15.1:0/48144484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe520104500 0x7fe52019d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:55.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.285+0000 7fe524969700 1 --2- 172.21.15.1:0/48144484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5200ff460 0x7fe520198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:55.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.285+0000 7fe524969700 1 --2- 172.21.15.1:0/48144484 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe5200ffe00 0x7fe520198bb0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:55.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.285+0000 7fe524969700 1 -- 172.21.15.1:0/48144484 >> 172.21.15.1:0/48144484 conn(0x7fe5200770a0 msgr2=0x7fe5200fcd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:55.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.286+0000 7fe524969700 1 -- 172.21.15.1:0/48144484 shutdown_connections 2023-11-16T16:37:55.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:55.286+0000 7fe524969700 1 -- 172.21.15.1:0/48144484 wait complete. 2023-11-16T16:37:55.617 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:55 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1290772226' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-11-16T16:37:55.617 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:55 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/48144484' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-11-16T16:37:55.617 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:55 smithi001 ceph-mon[98800]: pgmap v281: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 7 op/s 2023-11-16T16:37:55.763 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149747 got 111669149749 for osd.3 2023-11-16T16:37:55.763 DEBUG:teuthology.parallel:result is None 2023-11-16T16:37:55.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:55 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1290772226' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-11-16T16:37:55.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:55 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/48144484' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-11-16T16:37:55.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:55 smithi053 ceph-mon[100638]: pgmap v281: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 7 op/s 2023-11-16T16:37:55.948 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808856 got 287762808858 for osd.9 2023-11-16T16:37:55.948 DEBUG:teuthology.parallel:result is None 2023-11-16T16:37:56.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:55 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1290772226' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-11-16T16:37:56.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:55 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/48144484' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-11-16T16:37:56.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:55 smithi026 ceph-mon[100112]: pgmap v281: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 7 op/s 2023-11-16T16:37:56.770 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645333 got 309237645335 for osd.10 2023-11-16T16:37:56.770 DEBUG:teuthology.parallel:result is None 2023-11-16T16:37:56.977 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724588 got 167503724590 for osd.5 2023-11-16T16:37:56.977 DEBUG:teuthology.parallel:result is None 2023-11-16T16:37:57.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.197+0000 7f0e5a530700 1 -- 172.21.15.1:0/3925934364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e54104500 msgr2=0x7f0e5410c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:57.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.197+0000 7f0e5a530700 1 --2- 172.21.15.1:0/3925934364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e54104500 0x7f0e5410c080 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c00b0d0 tx=0x7f0e4c00b3e0 comp rx=0 tx=0).stop 2023-11-16T16:37:57.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.197+0000 7f0e5a530700 1 -- 172.21.15.1:0/3925934364 shutdown_connections 2023-11-16T16:37:57.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.197+0000 7f0e5a530700 1 --2- 172.21.15.1:0/3925934364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e54104500 0x7f0e5410c080 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:57.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.197+0000 7f0e5a530700 1 --2- 172.21.15.1:0/3925934364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e540ff460 0x7f0e540ff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:57.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.197+0000 7f0e5a530700 1 --2- 172.21.15.1:0/3925934364 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e540ffe00 0x7f0e54103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:57.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.197+0000 7f0e5a530700 1 -- 172.21.15.1:0/3925934364 >> 172.21.15.1:0/3925934364 conn(0x7f0e540770a0 msgr2=0x7f0e540774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:57.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.197+0000 7f0e5a530700 1 -- 172.21.15.1:0/3925934364 shutdown_connections 2023-11-16T16:37:57.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.197+0000 7f0e5a530700 1 -- 172.21.15.1:0/3925934364 wait complete. 2023-11-16T16:37:57.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.198+0000 7f0e5a530700 1 Processor -- start 2023-11-16T16:37:57.201 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.198+0000 7f0e5a530700 1 -- start start 2023-11-16T16:37:57.202 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.198+0000 7f0e5a530700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e540ff460 0x7f0e54197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:57.202 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.198+0000 7f0e5a530700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e540ffe00 0x7f0e54197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:57.202 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.198+0000 7f0e5a530700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e54104500 0x7f0e541a24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:57.202 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.198+0000 7f0e5a530700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e54198280 con 0x7f0e540ffe00 2023-11-16T16:37:57.202 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.198+0000 7f0e5a530700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e541a29f0 con 0x7f0e540ff460 2023-11-16T16:37:57.203 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.198+0000 7f0e5a530700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e541a2b60 con 0x7f0e54104500 2023-11-16T16:37:57.203 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.198+0000 7f0e537fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e540ffe00 0x7f0e54197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:57.203 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.199+0000 7f0e537fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e540ffe00 0x7f0e54197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42022/0 (socket says 172.21.15.1:42022) 2023-11-16T16:37:57.203 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.199+0000 7f0e537fe700 1 -- 172.21.15.1:0/4159864166 learned_addr learned my addr 172.21.15.1:0/4159864166 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:57.203 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.199+0000 7f0e53fff700 1 --2- 172.21.15.1:0/4159864166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e540ff460 0x7f0e54197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:57.203 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.199+0000 7f0e58acd700 1 --2- 172.21.15.1:0/4159864166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e54104500 0x7f0e541a24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:57.203 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.199+0000 7f0e537fe700 1 -- 172.21.15.1:0/4159864166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e54104500 msgr2=0x7f0e541a24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:57.203 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.199+0000 7f0e537fe700 1 --2- 172.21.15.1:0/4159864166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e54104500 0x7f0e541a24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:57.203 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.199+0000 7f0e537fe700 1 -- 172.21.15.1:0/4159864166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e540ff460 msgr2=0x7f0e54197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:57.203 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.199+0000 7f0e537fe700 1 --2- 172.21.15.1:0/4159864166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e540ff460 0x7f0e54197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:57.203 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.199+0000 7f0e537fe700 1 -- 172.21.15.1:0/4159864166 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e4c009d00 con 0x7f0e540ffe00 2023-11-16T16:37:57.203 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.199+0000 7f0e58acd700 1 --2- 172.21.15.1:0/4159864166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e54104500 0x7f0e541a24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:37:57.204 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.199+0000 7f0e537fe700 1 --2- 172.21.15.1:0/4159864166 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e540ffe00 0x7f0e54197970 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c00c970 tx=0x7f0e3c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:57.204 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.199+0000 7f0e517fa700 1 -- 172.21.15.1:0/4159864166 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c007b00 con 0x7f0e540ffe00 2023-11-16T16:37:57.204 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.199+0000 7f0e517fa700 1 -- 172.21.15.1:0/4159864166 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e3c00ce60 con 0x7f0e540ffe00 2023-11-16T16:37:57.204 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.199+0000 7f0e5a530700 1 -- 172.21.15.1:0/4159864166 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e5410f320 con 0x7f0e540ffe00 2023-11-16T16:37:57.204 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.199+0000 7f0e517fa700 1 -- 172.21.15.1:0/4159864166 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c0054e0 con 0x7f0e540ffe00 2023-11-16T16:37:57.204 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.199+0000 7f0e5a530700 1 -- 172.21.15.1:0/4159864166 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e5410f190 con 0x7f0e540ffe00 2023-11-16T16:37:57.204 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.200+0000 7f0e5a530700 1 -- 172.21.15.1:0/4159864166 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e5410f4b0 con 0x7f0e540ffe00 2023-11-16T16:37:57.205 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.201+0000 7f0e517fa700 1 -- 172.21.15.1:0/4159864166 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f0e3c004750 con 0x7f0e540ffe00 2023-11-16T16:37:57.205 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.202+0000 7f0e517fa700 1 --2- 172.21.15.1:0/4159864166 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f0e400a11e0 0x7f0e400a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:57.205 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.202+0000 7f0e517fa700 1 -- 172.21.15.1:0/4159864166 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0e3c0cc510 con 0x7f0e540ffe00 2023-11-16T16:37:57.205 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.202+0000 7f0e53fff700 1 --2- 172.21.15.1:0/4159864166 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f0e400a11e0 0x7f0e400a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:57.205 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.202+0000 7f0e53fff700 1 --2- 172.21.15.1:0/4159864166 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f0e400a11e0 0x7f0e400a3690 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009a10 tx=0x7f0e44009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:57.208 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.206+0000 7f0e517fa700 1 -- 172.21.15.1:0/4159864166 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0e3c095cb0 con 0x7f0e540ffe00 2023-11-16T16:37:57.421 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.419+0000 7f0e5a530700 1 -- 172.21.15.1:0/4159864166 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0e541a3580 con 0x7f0e540ffe00 2023-11-16T16:37:57.421 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.420+0000 7f0e517fa700 1 -- 172.21.15.1:0/4159864166 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0e3c0992d0 con 0x7f0e540ffe00 2023-11-16T16:37:57.422 INFO:teuthology.orchestra.run.smithi001.stdout:34359738436 2023-11-16T16:37:57.424 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.422+0000 7f0e5a530700 1 -- 172.21.15.1:0/4159864166 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f0e400a11e0 msgr2=0x7f0e400a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:57.424 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.422+0000 7f0e5a530700 1 --2- 172.21.15.1:0/4159864166 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f0e400a11e0 0x7f0e400a3690 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009a10 tx=0x7f0e44009190 comp rx=0 tx=0).stop 2023-11-16T16:37:57.424 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.422+0000 7f0e5a530700 1 -- 172.21.15.1:0/4159864166 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e540ffe00 msgr2=0x7f0e54197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:57.424 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.422+0000 7f0e5a530700 1 --2- 172.21.15.1:0/4159864166 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e540ffe00 0x7f0e54197970 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c00c970 tx=0x7f0e3c00cc80 comp rx=0 tx=0).stop 2023-11-16T16:37:57.424 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.422+0000 7f0e5a530700 1 -- 172.21.15.1:0/4159864166 shutdown_connections 2023-11-16T16:37:57.425 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.422+0000 7f0e5a530700 1 --2- 172.21.15.1:0/4159864166 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f0e400a11e0 0x7f0e400a3690 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:57.425 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.422+0000 7f0e5a530700 1 --2- 172.21.15.1:0/4159864166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0e54104500 0x7f0e541a24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:57.425 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.422+0000 7f0e5a530700 1 --2- 172.21.15.1:0/4159864166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e540ff460 0x7f0e54197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:57.425 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.422+0000 7f0e5a530700 1 --2- 172.21.15.1:0/4159864166 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e540ffe00 0x7f0e54197970 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:57.425 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.422+0000 7f0e5a530700 1 -- 172.21.15.1:0/4159864166 >> 172.21.15.1:0/4159864166 conn(0x7f0e540770a0 msgr2=0x7f0e540fcc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:57.425 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.422+0000 7f0e5a530700 1 -- 172.21.15.1:0/4159864166 shutdown_connections 2023-11-16T16:37:57.425 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:57.422+0000 7f0e5a530700 1 -- 172.21.15.1:0/4159864166 wait complete. 2023-11-16T16:37:57.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:57 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/4159864166' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-11-16T16:37:57.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:57 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/4159864166' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-11-16T16:37:57.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:57 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/4159864166' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-11-16T16:37:58.030 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.027+0000 7f87a283d700 1 -- 172.21.15.1:0/2082156776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f879c1013e0 msgr2=0x7f879c1017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:58.030 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.027+0000 7f87a283d700 1 --2- 172.21.15.1:0/2082156776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f879c1013e0 0x7f879c1017b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f878c009990 tx=0x7f878c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:37:58.030 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.027+0000 7f87a283d700 1 -- 172.21.15.1:0/2082156776 shutdown_connections 2023-11-16T16:37:58.030 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.027+0000 7f87a283d700 1 --2- 172.21.15.1:0/2082156776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f879c10a900 0x7f879c10cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:58.030 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.027+0000 7f87a283d700 1 --2- 172.21.15.1:0/2082156776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f879c1013e0 0x7f879c1017b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:58.030 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.027+0000 7f87a283d700 1 --2- 172.21.15.1:0/2082156776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f879c101cf0 0x7f879c10a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:58.030 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.027+0000 7f87a283d700 1 -- 172.21.15.1:0/2082156776 >> 172.21.15.1:0/2082156776 conn(0x7f879c0770d0 msgr2=0x7f879c0774d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:58.030 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.027+0000 7f87a283d700 1 -- 172.21.15.1:0/2082156776 shutdown_connections 2023-11-16T16:37:58.031 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.027+0000 7f87a283d700 1 -- 172.21.15.1:0/2082156776 wait complete. 2023-11-16T16:37:58.031 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.028+0000 7f87a283d700 1 Processor -- start 2023-11-16T16:37:58.031 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.028+0000 7f87a283d700 1 -- start start 2023-11-16T16:37:58.031 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.028+0000 7f87a283d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f879c1013e0 0x7f879c0ff210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:58.031 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.028+0000 7f87a283d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f879c101cf0 0x7f879c0ff750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:58.031 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.028+0000 7f87a283d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f879c10a900 0x7f879c0ffcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:58.031 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.028+0000 7f87a283d700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879c1a4810 con 0x7f879c101cf0 2023-11-16T16:37:58.031 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.028+0000 7f87a283d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879c1a4980 con 0x7f879c10a900 2023-11-16T16:37:58.031 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.028+0000 7f87a283d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879c1a4b40 con 0x7f879c1013e0 2023-11-16T16:37:58.032 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.028+0000 7f879b7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f879c101cf0 0x7f879c0ff750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:58.032 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.028+0000 7f879b7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f879c101cf0 0x7f879c0ff750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42034/0 (socket says 172.21.15.1:42034) 2023-11-16T16:37:58.033 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.028+0000 7f879b7fe700 1 -- 172.21.15.1:0/169309724 learned_addr learned my addr 172.21.15.1:0/169309724 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:58.033 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.028+0000 7f87a0dda700 1 --2- 172.21.15.1:0/169309724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f879c10a900 0x7f879c0ffcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:58.033 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.029+0000 7f879bfff700 1 --2- 172.21.15.1:0/169309724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f879c1013e0 0x7f879c0ff210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:58.033 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.029+0000 7f879b7fe700 1 -- 172.21.15.1:0/169309724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f879c1013e0 msgr2=0x7f879c0ff210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:58.033 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.029+0000 7f879b7fe700 1 --2- 172.21.15.1:0/169309724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f879c1013e0 0x7f879c0ff210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:58.033 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.029+0000 7f879b7fe700 1 -- 172.21.15.1:0/169309724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f879c10a900 msgr2=0x7f879c0ffcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:58.033 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.029+0000 7f879b7fe700 1 --2- 172.21.15.1:0/169309724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f879c10a900 0x7f879c0ffcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:58.033 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.029+0000 7f879b7fe700 1 -- 172.21.15.1:0/169309724 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f878c0095f0 con 0x7f879c101cf0 2023-11-16T16:37:58.033 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.029+0000 7f879bfff700 1 --2- 172.21.15.1:0/169309724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f879c1013e0 0x7f879c0ff210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:37:58.033 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.029+0000 7f879b7fe700 1 --2- 172.21.15.1:0/169309724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f879c101cf0 0x7f879c0ff750 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f879000c920 tx=0x7f879000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:58.033 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.029+0000 7f87997fa700 1 -- 172.21.15.1:0/169309724 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8790007b00 con 0x7f879c101cf0 2023-11-16T16:37:58.033 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.029+0000 7f87a283d700 1 -- 172.21.15.1:0/169309724 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f879c110370 con 0x7f879c101cf0 2023-11-16T16:37:58.034 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.029+0000 7f87997fa700 1 -- 172.21.15.1:0/169309724 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f879000ce70 con 0x7f879c101cf0 2023-11-16T16:37:58.034 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.029+0000 7f87997fa700 1 -- 172.21.15.1:0/169309724 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8790005460 con 0x7f879c101cf0 2023-11-16T16:37:58.034 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.030+0000 7f87a283d700 1 -- 172.21.15.1:0/169309724 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f879c1101e0 con 0x7f879c101cf0 2023-11-16T16:37:58.034 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.031+0000 7f87a283d700 1 -- 172.21.15.1:0/169309724 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f879c110500 con 0x7f879c101cf0 2023-11-16T16:37:58.035 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.032+0000 7f87997fa700 1 -- 172.21.15.1:0/169309724 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f8790004750 con 0x7f879c101cf0 2023-11-16T16:37:58.035 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.032+0000 7f87997fa700 1 --2- 172.21.15.1:0/169309724 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f87840a11e0 0x7f87840a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:58.035 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.033+0000 7f879bfff700 1 --2- 172.21.15.1:0/169309724 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f87840a11e0 0x7f87840a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:58.035 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.033+0000 7f87997fa700 1 -- 172.21.15.1:0/169309724 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f87900cc1a0 con 0x7f879c101cf0 2023-11-16T16:37:58.035 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.033+0000 7f879bfff700 1 --2- 172.21.15.1:0/169309724 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f87840a11e0 0x7f87840a3690 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f878c006010 tx=0x7f878c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:58.040 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.038+0000 7f87997fa700 1 -- 172.21.15.1:0/169309724 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8790095940 con 0x7f879c101cf0 2023-11-16T16:37:58.047 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738434 got 34359738436 for osd.0 2023-11-16T16:37:58.047 DEBUG:teuthology.parallel:result is None 2023-11-16T16:37:58.247 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.245+0000 7f87a283d700 1 -- 172.21.15.1:0/169309724 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f879c1a5670 con 0x7f879c101cf0 2023-11-16T16:37:58.247 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.246+0000 7f87997fa700 1 -- 172.21.15.1:0/169309724 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8790098f60 con 0x7f879c101cf0 2023-11-16T16:37:58.248 INFO:teuthology.orchestra.run.smithi001.stdout:55834574910 2023-11-16T16:37:58.250 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.248+0000 7f87a283d700 1 -- 172.21.15.1:0/169309724 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f87840a11e0 msgr2=0x7f87840a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:58.250 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.248+0000 7f87a283d700 1 --2- 172.21.15.1:0/169309724 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f87840a11e0 0x7f87840a3690 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f878c006010 tx=0x7f878c00b540 comp rx=0 tx=0).stop 2023-11-16T16:37:58.250 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.248+0000 7f87a283d700 1 -- 172.21.15.1:0/169309724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f879c101cf0 msgr2=0x7f879c0ff750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:58.250 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.248+0000 7f87a283d700 1 --2- 172.21.15.1:0/169309724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f879c101cf0 0x7f879c0ff750 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f879000c920 tx=0x7f879000cc30 comp rx=0 tx=0).stop 2023-11-16T16:37:58.250 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.248+0000 7f87a283d700 1 -- 172.21.15.1:0/169309724 shutdown_connections 2023-11-16T16:37:58.250 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.248+0000 7f87a283d700 1 --2- 172.21.15.1:0/169309724 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f87840a11e0 0x7f87840a3690 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:58.250 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.248+0000 7f87a283d700 1 --2- 172.21.15.1:0/169309724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f879c10a900 0x7f879c0ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:58.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.248+0000 7f87a283d700 1 --2- 172.21.15.1:0/169309724 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f879c1013e0 0x7f879c0ff210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:58.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.248+0000 7f87a283d700 1 --2- 172.21.15.1:0/169309724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f879c101cf0 0x7f879c0ff750 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:58.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.248+0000 7f87a283d700 1 -- 172.21.15.1:0/169309724 >> 172.21.15.1:0/169309724 conn(0x7f879c0770d0 msgr2=0x7f879c0fbd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:58.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.248+0000 7f87a283d700 1 -- 172.21.15.1:0/169309724 shutdown_connections 2023-11-16T16:37:58.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.248+0000 7f87a283d700 1 -- 172.21.15.1:0/169309724 wait complete. 2023-11-16T16:37:58.479 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:58 smithi001 ceph-mon[98800]: pgmap v282: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2023-11-16T16:37:58.479 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:58 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/169309724' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-11-16T16:37:58.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.422+0000 7f9324e20700 1 -- 172.21.15.1:0/1566106609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9320104500 msgr2=0x7f932010c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:58.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.422+0000 7f9324e20700 1 --2- 172.21.15.1:0/1566106609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9320104500 0x7f932010c080 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f93180099c0 tx=0x7f9318009cd0 comp rx=0 tx=0).stop 2023-11-16T16:37:58.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.422+0000 7f9324e20700 1 -- 172.21.15.1:0/1566106609 shutdown_connections 2023-11-16T16:37:58.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.422+0000 7f9324e20700 1 --2- 172.21.15.1:0/1566106609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9320104500 0x7f932010c080 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:58.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.422+0000 7f9324e20700 1 --2- 172.21.15.1:0/1566106609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93200ff460 0x7f93200ff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:58.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.422+0000 7f9324e20700 1 --2- 172.21.15.1:0/1566106609 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93200ffe00 0x7f9320103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:58.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.422+0000 7f9324e20700 1 -- 172.21.15.1:0/1566106609 >> 172.21.15.1:0/1566106609 conn(0x7f93200770a0 msgr2=0x7f93200774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:58.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.422+0000 7f9324e20700 1 -- 172.21.15.1:0/1566106609 shutdown_connections 2023-11-16T16:37:58.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.422+0000 7f9324e20700 1 -- 172.21.15.1:0/1566106609 wait complete. 2023-11-16T16:37:58.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.423+0000 7f9324e20700 1 Processor -- start 2023-11-16T16:37:58.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.423+0000 7f9324e20700 1 -- start start 2023-11-16T16:37:58.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.423+0000 7f9324e20700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93200ff460 0x7f93201971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:58.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.423+0000 7f9324e20700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93200ffe00 0x7f93201976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:58.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.423+0000 7f9324e20700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9320104500 0x7f93201a2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:58.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.423+0000 7f9324e20700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9320198020 con 0x7f9320104500 2023-11-16T16:37:58.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.423+0000 7f9324e20700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93201a2760 con 0x7f93200ff460 2023-11-16T16:37:58.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.423+0000 7f9324e20700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93201a2920 con 0x7f93200ffe00 2023-11-16T16:37:58.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.423+0000 7f931ed9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9320104500 0x7f93201a2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:58.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.423+0000 7f931ed9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9320104500 0x7f93201a2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42052/0 (socket says 172.21.15.1:42052) 2023-11-16T16:37:58.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.423+0000 7f931ed9d700 1 -- 172.21.15.1:0/2902504534 learned_addr learned my addr 172.21.15.1:0/2902504534 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:37:58.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.423+0000 7f931e59c700 1 --2- 172.21.15.1:0/2902504534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93200ff460 0x7f93201971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:58.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.423+0000 7f931dd9b700 1 --2- 172.21.15.1:0/2902504534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93200ffe00 0x7f93201976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:58.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.423+0000 7f931ed9d700 1 -- 172.21.15.1:0/2902504534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93200ffe00 msgr2=0x7f93201976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:58.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.423+0000 7f931ed9d700 1 --2- 172.21.15.1:0/2902504534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93200ffe00 0x7f93201976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:58.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.423+0000 7f931ed9d700 1 -- 172.21.15.1:0/2902504534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93200ff460 msgr2=0x7f93201971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:58.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.423+0000 7f931ed9d700 1 --2- 172.21.15.1:0/2902504534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93200ff460 0x7f93201971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:58.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.423+0000 7f931ed9d700 1 -- 172.21.15.1:0/2902504534 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93180095f0 con 0x7f9320104500 2023-11-16T16:37:58.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.424+0000 7f931ed9d700 1 --2- 172.21.15.1:0/2902504534 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9320104500 0x7f93201a2220 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9318009990 tx=0x7f9318009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:58.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.424+0000 7f93177fe700 1 -- 172.21.15.1:0/2902504534 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931801d070 con 0x7f9320104500 2023-11-16T16:37:58.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.424+0000 7f9324e20700 1 -- 172.21.15.1:0/2902504534 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f932010f240 con 0x7f9320104500 2023-11-16T16:37:58.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.424+0000 7f93177fe700 1 -- 172.21.15.1:0/2902504534 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9318003f90 con 0x7f9320104500 2023-11-16T16:37:58.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.424+0000 7f93177fe700 1 -- 172.21.15.1:0/2902504534 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9318022c90 con 0x7f9320104500 2023-11-16T16:37:58.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.424+0000 7f9324e20700 1 -- 172.21.15.1:0/2902504534 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f932010f0b0 con 0x7f9320104500 2023-11-16T16:37:58.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.425+0000 7f9324e20700 1 -- 172.21.15.1:0/2902504534 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f932010f3d0 con 0x7f9320104500 2023-11-16T16:37:58.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.426+0000 7f93177fe700 1 -- 172.21.15.1:0/2902504534 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f9318022470 con 0x7f9320104500 2023-11-16T16:37:58.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.426+0000 7f93177fe700 1 --2- 172.21.15.1:0/2902504534 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f930c0a11e0 0x7f930c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:37:58.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.427+0000 7f93177fe700 1 -- 172.21.15.1:0/2902504534 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f93180cddb0 con 0x7f9320104500 2023-11-16T16:37:58.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.427+0000 7f931e59c700 1 --2- 172.21.15.1:0/2902504534 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f930c0a11e0 0x7f930c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:37:58.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.427+0000 7f931e59c700 1 --2- 172.21.15.1:0/2902504534 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f930c0a11e0 0x7f930c0a3690 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9310009550 tx=0x7f93100091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:37:58.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.431+0000 7f93177fe700 1 -- 172.21.15.1:0/2902504534 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f93180975e0 con 0x7f9320104500 2023-11-16T16:37:58.630 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.629+0000 7f9324e20700 1 -- 172.21.15.1:0/2902504534 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f9320066ea0 con 0x7f9320104500 2023-11-16T16:37:58.631 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.629+0000 7f93177fe700 1 -- 172.21.15.1:0/2902504534 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f931809ac00 con 0x7f9320104500 2023-11-16T16:37:58.631 INFO:teuthology.orchestra.run.smithi001.stdout:339302416404 2023-11-16T16:37:58.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.631+0000 7f9324e20700 1 -- 172.21.15.1:0/2902504534 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f930c0a11e0 msgr2=0x7f930c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:58.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.631+0000 7f9324e20700 1 --2- 172.21.15.1:0/2902504534 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f930c0a11e0 0x7f930c0a3690 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9310009550 tx=0x7f93100091b0 comp rx=0 tx=0).stop 2023-11-16T16:37:58.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.631+0000 7f9324e20700 1 -- 172.21.15.1:0/2902504534 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9320104500 msgr2=0x7f93201a2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:37:58.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.631+0000 7f9324e20700 1 --2- 172.21.15.1:0/2902504534 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9320104500 0x7f93201a2220 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9318009990 tx=0x7f9318009580 comp rx=0 tx=0).stop 2023-11-16T16:37:58.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.631+0000 7f9324e20700 1 -- 172.21.15.1:0/2902504534 shutdown_connections 2023-11-16T16:37:58.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.631+0000 7f9324e20700 1 --2- 172.21.15.1:0/2902504534 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f930c0a11e0 0x7f930c0a3690 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:58.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.631+0000 7f9324e20700 1 --2- 172.21.15.1:0/2902504534 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9320104500 0x7f93201a2220 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:58.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.631+0000 7f9324e20700 1 --2- 172.21.15.1:0/2902504534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93200ff460 0x7f93201971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:58.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.631+0000 7f9324e20700 1 --2- 172.21.15.1:0/2902504534 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93200ffe00 0x7f93201976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:37:58.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.631+0000 7f9324e20700 1 -- 172.21.15.1:0/2902504534 >> 172.21.15.1:0/2902504534 conn(0x7f93200770a0 msgr2=0x7f93200fcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:37:58.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.631+0000 7f9324e20700 1 -- 172.21.15.1:0/2902504534 shutdown_connections 2023-11-16T16:37:58.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:37:58.631+0000 7f9324e20700 1 -- 172.21.15.1:0/2902504534 wait complete. 2023-11-16T16:37:58.705 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574908 got 55834574910 for osd.1 2023-11-16T16:37:58.705 DEBUG:teuthology.parallel:result is None 2023-11-16T16:37:58.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:58 smithi026 ceph-mon[100112]: pgmap v282: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2023-11-16T16:37:58.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:58 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/169309724' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-11-16T16:37:58.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:58 smithi053 ceph-mon[100638]: pgmap v282: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2023-11-16T16:37:58.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:58 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/169309724' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-11-16T16:37:59.103 INFO:tasks.cephadm.ceph_manager.ceph:need seq 339302416402 got 339302416404 for osd.11 2023-11-16T16:37:59.103 DEBUG:teuthology.parallel:result is None 2023-11-16T16:37:59.103 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-11-16T16:37:59.103 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph pg dump --format=json 2023-11-16T16:37:59.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:37:59 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/2902504534' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-11-16T16:37:59.759 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:37:59.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:37:59 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/2902504534' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-11-16T16:37:59.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:37:59 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/2902504534' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-11-16T16:38:00.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:00 smithi001 ceph-mon[98800]: pgmap v283: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2023-11-16T16:38:00.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:00 smithi026 ceph-mon[100112]: pgmap v283: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2023-11-16T16:38:00.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:00 smithi053 ceph-mon[100638]: pgmap v283: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2023-11-16T16:38:01.030 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.028+0000 7f07fa813700 1 -- 172.21.15.1:0/2205894286 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07f40ff460 msgr2=0x7f07f40ff830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:01.030 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.028+0000 7f07fa813700 1 --2- 172.21.15.1:0/2205894286 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07f40ff460 0x7f07f40ff830 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f07e40099c0 tx=0x7f07e4009cd0 comp rx=0 tx=0).stop 2023-11-16T16:38:01.030 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.028+0000 7f07fa813700 1 -- 172.21.15.1:0/2205894286 shutdown_connections 2023-11-16T16:38:01.031 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.028+0000 7f07fa813700 1 --2- 172.21.15.1:0/2205894286 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07f4104500 0x7f07f410c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:01.031 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.028+0000 7f07fa813700 1 --2- 172.21.15.1:0/2205894286 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07f40ff460 0x7f07f40ff830 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:01.031 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.028+0000 7f07fa813700 1 --2- 172.21.15.1:0/2205894286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f40ffe00 0x7f07f4103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:01.031 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.028+0000 7f07fa813700 1 -- 172.21.15.1:0/2205894286 >> 172.21.15.1:0/2205894286 conn(0x7f07f40770a0 msgr2=0x7f07f40774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:01.031 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.028+0000 7f07fa813700 1 -- 172.21.15.1:0/2205894286 shutdown_connections 2023-11-16T16:38:01.031 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.028+0000 7f07fa813700 1 -- 172.21.15.1:0/2205894286 wait complete. 2023-11-16T16:38:01.035 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.028+0000 7f07fa813700 1 Processor -- start 2023-11-16T16:38:01.035 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.029+0000 7f07fa813700 1 -- start start 2023-11-16T16:38:01.035 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.029+0000 7f07fa813700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07f40ff460 0x7f07f41971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:01.035 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.029+0000 7f07fa813700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f40ffe00 0x7f07f4197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:01.035 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.029+0000 7f07fa813700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07f4104500 0x7f07f41a2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:01.035 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.029+0000 7f07fa813700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f4198040 con 0x7f07f40ff460 2023-11-16T16:38:01.036 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.029+0000 7f07fa813700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f41a27b0 con 0x7f07f40ffe00 2023-11-16T16:38:01.036 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.029+0000 7f07fa813700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f41a2920 con 0x7f07f4104500 2023-11-16T16:38:01.036 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.029+0000 7f07f3fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07f40ff460 0x7f07f41971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:01.036 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.029+0000 7f07f3fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07f40ff460 0x7f07f41971f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42060/0 (socket says 172.21.15.1:42060) 2023-11-16T16:38:01.036 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.029+0000 7f07f3fff700 1 -- 172.21.15.1:0/2246566072 learned_addr learned my addr 172.21.15.1:0/2246566072 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:01.036 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.029+0000 7f07f8db0700 1 --2- 172.21.15.1:0/2246566072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07f4104500 0x7f07f41a2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:01.036 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.029+0000 7f07f37fe700 1 --2- 172.21.15.1:0/2246566072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f40ffe00 0x7f07f4197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:01.036 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.029+0000 7f07f3fff700 1 -- 172.21.15.1:0/2246566072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07f4104500 msgr2=0x7f07f41a2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:01.036 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.029+0000 7f07f3fff700 1 --2- 172.21.15.1:0/2246566072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07f4104500 0x7f07f41a2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:01.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.029+0000 7f07f3fff700 1 -- 172.21.15.1:0/2246566072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f40ffe00 msgr2=0x7f07f4197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:01.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.029+0000 7f07f3fff700 1 --2- 172.21.15.1:0/2246566072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f40ffe00 0x7f07f4197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:01.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.029+0000 7f07f3fff700 1 -- 172.21.15.1:0/2246566072 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e40095f0 con 0x7f07f40ff460 2023-11-16T16:38:01.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.029+0000 7f07f37fe700 1 --2- 172.21.15.1:0/2246566072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f40ffe00 0x7f07f4197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:38:01.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.030+0000 7f07f3fff700 1 --2- 172.21.15.1:0/2246566072 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07f40ff460 0x7f07f41971f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f07e4009990 tx=0x7f07e4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:01.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.030+0000 7f07f17fa700 1 -- 172.21.15.1:0/2246566072 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e401d070 con 0x7f07f40ff460 2023-11-16T16:38:01.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.030+0000 7f07f17fa700 1 -- 172.21.15.1:0/2246566072 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07e4003f90 con 0x7f07f40ff460 2023-11-16T16:38:01.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.030+0000 7f07fa813700 1 -- 172.21.15.1:0/2246566072 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07f410f290 con 0x7f07f40ff460 2023-11-16T16:38:01.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.030+0000 7f07f17fa700 1 -- 172.21.15.1:0/2246566072 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e4022cb0 con 0x7f07f40ff460 2023-11-16T16:38:01.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.030+0000 7f07fa813700 1 -- 172.21.15.1:0/2246566072 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07f410f100 con 0x7f07f40ff460 2023-11-16T16:38:01.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.031+0000 7f07fa813700 1 -- 172.21.15.1:0/2246566072 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07f410f420 con 0x7f07f40ff460 2023-11-16T16:38:01.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.032+0000 7f07f17fa700 1 -- 172.21.15.1:0/2246566072 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f07e4022470 con 0x7f07f40ff460 2023-11-16T16:38:01.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.032+0000 7f07f17fa700 1 --2- 172.21.15.1:0/2246566072 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f07dc0a1260 0x7f07dc0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:01.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.033+0000 7f07f17fa700 1 -- 172.21.15.1:0/2246566072 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f07e40ce4f0 con 0x7f07f40ff460 2023-11-16T16:38:01.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.033+0000 7f07f37fe700 1 --2- 172.21.15.1:0/2246566072 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f07dc0a1260 0x7f07dc0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:01.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.033+0000 7f07f37fe700 1 --2- 172.21.15.1:0/2246566072 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f07dc0a1260 0x7f07dc0a3710 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f07f4068e60 tx=0x7f07e800a680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:01.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.036+0000 7f07f17fa700 1 -- 172.21.15.1:0/2246566072 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f07e4097c20 con 0x7f07f40ff460 2023-11-16T16:38:01.230 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.228+0000 7f07fa813700 1 -- 172.21.15.1:0/2246566072 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f07f40fd330 con 0x7f07dc0a1260 2023-11-16T16:38:01.242 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.240+0000 7f07f17fa700 1 -- 172.21.15.1:0/2246566072 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340067 (secure 0 0 0) 0x7f07f40fd330 con 0x7f07dc0a1260 2023-11-16T16:38:01.247 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:38:01.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.243+0000 7f07fa813700 1 -- 172.21.15.1:0/2246566072 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f07dc0a1260 msgr2=0x7f07dc0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:01.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.243+0000 7f07fa813700 1 --2- 172.21.15.1:0/2246566072 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f07dc0a1260 0x7f07dc0a3710 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f07f4068e60 tx=0x7f07e800a680 comp rx=0 tx=0).stop 2023-11-16T16:38:01.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.243+0000 7f07fa813700 1 -- 172.21.15.1:0/2246566072 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07f40ff460 msgr2=0x7f07f41971f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:01.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.243+0000 7f07fa813700 1 --2- 172.21.15.1:0/2246566072 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07f40ff460 0x7f07f41971f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f07e4009990 tx=0x7f07e4005740 comp rx=0 tx=0).stop 2023-11-16T16:38:01.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.243+0000 7f07fa813700 1 -- 172.21.15.1:0/2246566072 shutdown_connections 2023-11-16T16:38:01.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.243+0000 7f07fa813700 1 --2- 172.21.15.1:0/2246566072 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f07dc0a1260 0x7f07dc0a3710 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:01.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.243+0000 7f07fa813700 1 --2- 172.21.15.1:0/2246566072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07f4104500 0x7f07f41a2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:01.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.243+0000 7f07fa813700 1 --2- 172.21.15.1:0/2246566072 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07f40ff460 0x7f07f41971f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:01.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.243+0000 7f07fa813700 1 --2- 172.21.15.1:0/2246566072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f40ffe00 0x7f07f4197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:01.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.243+0000 7f07fa813700 1 -- 172.21.15.1:0/2246566072 >> 172.21.15.1:0/2246566072 conn(0x7f07f40770a0 msgr2=0x7f07f40fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:01.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.243+0000 7f07fa813700 1 -- 172.21.15.1:0/2246566072 shutdown_connections 2023-11-16T16:38:01.253 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:01.243+0000 7f07fa813700 1 -- 172.21.15.1:0/2246566072 wait complete. 2023-11-16T16:38:01.253 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2023-11-16T16:38:01.721 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":283,"stamp":"2023-11-16T16:37:59.482508+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653045,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":391,"num_read_kb":312,"num_write":333,"num_write_kb":3043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":78568,"kb_used_data":11080,"kb_used_omap":0,"kb_used_meta":67392,"kb_avail":1124863256,"statfs":{"total":1151940427776,"available":1151859974144,"internally_reserved":0,"allocated":11345920,"data_stored":7104901,"data_compressed":106695,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":69009408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1652,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":71,"num_read_kb":37,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003095"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157701+0000","last_change":"2023-11-16T16:37:13.941551+0000","last_active":"2023-11-16T16:37:15.157701+0000","last_peered":"2023-11-16T16:37:15.157701+0000","last_clean":"2023-11-16T16:37:15.157701+0000","last_became_active":"2023-11-16T16:37:13.941190+0000","last_became_peered":"2023-11-16T16:37:13.941190+0000","last_unstale":"2023-11-16T16:37:15.157701+0000","last_undegraded":"2023-11-16T16:37:15.157701+0000","last_fullsized":"2023-11-16T16:37:15.157701+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:26:51.970409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158170+0000","last_change":"2023-11-16T16:37:13.942300+0000","last_active":"2023-11-16T16:37:15.158170+0000","last_peered":"2023-11-16T16:37:15.158170+0000","last_clean":"2023-11-16T16:37:15.158170+0000","last_became_active":"2023-11-16T16:37:13.941966+0000","last_became_peered":"2023-11-16T16:37:13.941966+0000","last_unstale":"2023-11-16T16:37:15.158170+0000","last_undegraded":"2023-11-16T16:37:15.158170+0000","last_fullsized":"2023-11-16T16:37:15.158170+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:12:01.659205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158388+0000","last_change":"2023-11-16T16:37:13.943461+0000","last_active":"2023-11-16T16:37:15.158388+0000","last_peered":"2023-11-16T16:37:15.158388+0000","last_clean":"2023-11-16T16:37:15.158388+0000","last_became_active":"2023-11-16T16:37:13.943165+0000","last_became_peered":"2023-11-16T16:37:13.943165+0000","last_unstale":"2023-11-16T16:37:15.158388+0000","last_undegraded":"2023-11-16T16:37:15.158388+0000","last_fullsized":"2023-11-16T16:37:15.158388+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:09:30.792766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.122028+0000","last_change":"2023-11-16T16:37:13.940205+0000","last_active":"2023-11-16T16:37:15.122028+0000","last_peered":"2023-11-16T16:37:15.122028+0000","last_clean":"2023-11-16T16:37:15.122028+0000","last_became_active":"2023-11-16T16:37:13.939819+0000","last_became_peered":"2023-11-16T16:37:13.939819+0000","last_unstale":"2023-11-16T16:37:15.122028+0000","last_undegraded":"2023-11-16T16:37:15.122028+0000","last_fullsized":"2023-11-16T16:37:15.122028+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:40:08.471596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.923975+0000","last_change":"2023-11-16T16:37:13.939819+0000","last_active":"2023-11-16T16:37:14.923975+0000","last_peered":"2023-11-16T16:37:14.923975+0000","last_clean":"2023-11-16T16:37:14.923975+0000","last_became_active":"2023-11-16T16:37:13.939367+0000","last_became_peered":"2023-11-16T16:37:13.939367+0000","last_unstale":"2023-11-16T16:37:14.923975+0000","last_undegraded":"2023-11-16T16:37:14.923975+0000","last_fullsized":"2023-11-16T16:37:14.923975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:27:49.657269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.924379+0000","last_change":"2023-11-16T16:37:13.946613+0000","last_active":"2023-11-16T16:37:14.924379+0000","last_peered":"2023-11-16T16:37:14.924379+0000","last_clean":"2023-11-16T16:37:14.924379+0000","last_became_active":"2023-11-16T16:37:13.946281+0000","last_became_peered":"2023-11-16T16:37:13.946281+0000","last_unstale":"2023-11-16T16:37:14.924379+0000","last_undegraded":"2023-11-16T16:37:14.924379+0000","last_fullsized":"2023-11-16T16:37:14.924379+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:26:07.902412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.390884+0000","last_change":"2023-11-16T16:37:13.936899+0000","last_active":"2023-11-16T16:37:15.390884+0000","last_peered":"2023-11-16T16:37:15.390884+0000","last_clean":"2023-11-16T16:37:15.390884+0000","last_became_active":"2023-11-16T16:37:13.936543+0000","last_became_peered":"2023-11-16T16:37:13.936543+0000","last_unstale":"2023-11-16T16:37:15.390884+0000","last_undegraded":"2023-11-16T16:37:15.390884+0000","last_fullsized":"2023-11-16T16:37:15.390884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:55:14.512583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157287+0000","last_change":"2023-11-16T16:37:13.941463+0000","last_active":"2023-11-16T16:37:15.157287+0000","last_peered":"2023-11-16T16:37:15.157287+0000","last_clean":"2023-11-16T16:37:15.157287+0000","last_became_active":"2023-11-16T16:37:13.941125+0000","last_became_peered":"2023-11-16T16:37:13.941125+0000","last_unstale":"2023-11-16T16:37:15.157287+0000","last_undegraded":"2023-11-16T16:37:15.157287+0000","last_fullsized":"2023-11-16T16:37:15.157287+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:29:10.391228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157152+0000","last_change":"2023-11-16T16:37:13.934105+0000","last_active":"2023-11-16T16:37:15.157152+0000","last_peered":"2023-11-16T16:37:15.157152+0000","last_clean":"2023-11-16T16:37:15.157152+0000","last_became_active":"2023-11-16T16:37:13.933887+0000","last_became_peered":"2023-11-16T16:37:13.933887+0000","last_unstale":"2023-11-16T16:37:15.157152+0000","last_undegraded":"2023-11-16T16:37:15.157152+0000","last_fullsized":"2023-11-16T16:37:15.157152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:37:28.260833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.925511+0000","last_change":"2023-11-16T16:37:13.947266+0000","last_active":"2023-11-16T16:37:14.925511+0000","last_peered":"2023-11-16T16:37:14.925511+0000","last_clean":"2023-11-16T16:37:14.925511+0000","last_became_active":"2023-11-16T16:37:13.946919+0000","last_became_peered":"2023-11-16T16:37:13.946919+0000","last_unstale":"2023-11-16T16:37:14.925511+0000","last_undegraded":"2023-11-16T16:37:14.925511+0000","last_fullsized":"2023-11-16T16:37:14.925511+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:07:33.154784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.391182+0000","last_change":"2023-11-16T16:37:13.941597+0000","last_active":"2023-11-16T16:37:15.391182+0000","last_peered":"2023-11-16T16:37:15.391182+0000","last_clean":"2023-11-16T16:37:15.391182+0000","last_became_active":"2023-11-16T16:37:13.941253+0000","last_became_peered":"2023-11-16T16:37:13.941253+0000","last_unstale":"2023-11-16T16:37:15.391182+0000","last_undegraded":"2023-11-16T16:37:15.391182+0000","last_fullsized":"2023-11-16T16:37:15.391182+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:29:31.101716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.121996+0000","last_change":"2023-11-16T16:37:13.941673+0000","last_active":"2023-11-16T16:37:15.121996+0000","last_peered":"2023-11-16T16:37:15.121996+0000","last_clean":"2023-11-16T16:37:15.121996+0000","last_became_active":"2023-11-16T16:37:13.941345+0000","last_became_peered":"2023-11-16T16:37:13.941345+0000","last_unstale":"2023-11-16T16:37:15.121996+0000","last_undegraded":"2023-11-16T16:37:15.121996+0000","last_fullsized":"2023-11-16T16:37:15.121996+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:01:29.436252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157470+0000","last_change":"2023-11-16T16:37:13.947991+0000","last_active":"2023-11-16T16:37:15.157470+0000","last_peered":"2023-11-16T16:37:15.157470+0000","last_clean":"2023-11-16T16:37:15.157470+0000","last_became_active":"2023-11-16T16:37:13.947454+0000","last_became_peered":"2023-11-16T16:37:13.947454+0000","last_unstale":"2023-11-16T16:37:15.157470+0000","last_undegraded":"2023-11-16T16:37:15.157470+0000","last_fullsized":"2023-11-16T16:37:15.157470+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:35:41.400206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157374+0000","last_change":"2023-11-16T16:37:13.941502+0000","last_active":"2023-11-16T16:37:15.157374+0000","last_peered":"2023-11-16T16:37:15.157374+0000","last_clean":"2023-11-16T16:37:15.157374+0000","last_became_active":"2023-11-16T16:37:13.941191+0000","last_became_peered":"2023-11-16T16:37:13.941191+0000","last_unstale":"2023-11-16T16:37:15.157374+0000","last_undegraded":"2023-11-16T16:37:15.157374+0000","last_fullsized":"2023-11-16T16:37:15.157374+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:59:03.243696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.156825+0000","last_change":"2023-11-16T16:37:13.949198+0000","last_active":"2023-11-16T16:37:15.156825+0000","last_peered":"2023-11-16T16:37:15.156825+0000","last_clean":"2023-11-16T16:37:15.156825+0000","last_became_active":"2023-11-16T16:37:13.947147+0000","last_became_peered":"2023-11-16T16:37:13.947147+0000","last_unstale":"2023-11-16T16:37:15.156825+0000","last_undegraded":"2023-11-16T16:37:15.156825+0000","last_fullsized":"2023-11-16T16:37:15.156825+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:01:56.205870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.156530+0000","last_change":"2023-11-16T16:37:13.943915+0000","last_active":"2023-11-16T16:37:15.156530+0000","last_peered":"2023-11-16T16:37:15.156530+0000","last_clean":"2023-11-16T16:37:15.156530+0000","last_became_active":"2023-11-16T16:37:13.942995+0000","last_became_peered":"2023-11-16T16:37:13.942995+0000","last_unstale":"2023-11-16T16:37:15.156530+0000","last_undegraded":"2023-11-16T16:37:15.156530+0000","last_fullsized":"2023-11-16T16:37:15.156530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:35:55.774014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157841+0000","last_change":"2023-11-16T16:37:13.941816+0000","last_active":"2023-11-16T16:37:15.157841+0000","last_peered":"2023-11-16T16:37:15.157841+0000","last_clean":"2023-11-16T16:37:15.157841+0000","last_became_active":"2023-11-16T16:37:13.941541+0000","last_became_peered":"2023-11-16T16:37:13.941541+0000","last_unstale":"2023-11-16T16:37:15.157841+0000","last_undegraded":"2023-11-16T16:37:15.157841+0000","last_fullsized":"2023-11-16T16:37:15.157841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:18:40.904015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158134+0000","last_change":"2023-11-16T16:37:13.940639+0000","last_active":"2023-11-16T16:37:15.158134+0000","last_peered":"2023-11-16T16:37:15.158134+0000","last_clean":"2023-11-16T16:37:15.158134+0000","last_became_active":"2023-11-16T16:37:13.940323+0000","last_became_peered":"2023-11-16T16:37:13.940323+0000","last_unstale":"2023-11-16T16:37:15.158134+0000","last_undegraded":"2023-11-16T16:37:15.158134+0000","last_fullsized":"2023-11-16T16:37:15.158134+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:54:11.977928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158148+0000","last_change":"2023-11-16T16:37:13.934802+0000","last_active":"2023-11-16T16:37:15.158148+0000","last_peered":"2023-11-16T16:37:15.158148+0000","last_clean":"2023-11-16T16:37:15.158148+0000","last_became_active":"2023-11-16T16:37:13.934496+0000","last_became_peered":"2023-11-16T16:37:13.934496+0000","last_unstale":"2023-11-16T16:37:15.158148+0000","last_undegraded":"2023-11-16T16:37:15.158148+0000","last_fullsized":"2023-11-16T16:37:15.158148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:50:10.271792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'9","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:18.297674+0000","last_change":"2023-11-16T16:37:13.499284+0000","last_active":"2023-11-16T16:37:18.297674+0000","last_peered":"2023-11-16T16:37:18.297674+0000","last_clean":"2023-11-16T16:37:18.297674+0000","last_became_active":"2023-11-16T16:37:13.498931+0000","last_became_peered":"2023-11-16T16:37:13.498931+0000","last_unstale":"2023-11-16T16:37:18.297674+0000","last_undegraded":"2023-11-16T16:37:18.297674+0000","last_fullsized":"2023-11-16T16:37:18.297674+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:52:32.314115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157923+0000","last_change":"2023-11-16T16:37:13.939631+0000","last_active":"2023-11-16T16:37:15.157923+0000","last_peered":"2023-11-16T16:37:15.157923+0000","last_clean":"2023-11-16T16:37:15.157923+0000","last_became_active":"2023-11-16T16:37:13.939318+0000","last_became_peered":"2023-11-16T16:37:13.939318+0000","last_unstale":"2023-11-16T16:37:15.157923+0000","last_undegraded":"2023-11-16T16:37:15.157923+0000","last_fullsized":"2023-11-16T16:37:15.157923+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:51:05.730477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:16.528248+0000","last_change":"2023-11-16T16:37:13.495972+0000","last_active":"2023-11-16T16:37:16.528248+0000","last_peered":"2023-11-16T16:37:16.528248+0000","last_clean":"2023-11-16T16:37:16.528248+0000","last_became_active":"2023-11-16T16:37:13.495610+0000","last_became_peered":"2023-11-16T16:37:13.495610+0000","last_unstale":"2023-11-16T16:37:16.528248+0000","last_undegraded":"2023-11-16T16:37:16.528248+0000","last_fullsized":"2023-11-16T16:37:16.528248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:58:27.825452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158624+0000","last_change":"2023-11-16T16:37:13.935236+0000","last_active":"2023-11-16T16:37:15.158624+0000","last_peered":"2023-11-16T16:37:15.158624+0000","last_clean":"2023-11-16T16:37:15.158624+0000","last_became_active":"2023-11-16T16:37:13.934920+0000","last_became_peered":"2023-11-16T16:37:13.934920+0000","last_unstale":"2023-11-16T16:37:15.158624+0000","last_undegraded":"2023-11-16T16:37:15.158624+0000","last_fullsized":"2023-11-16T16:37:15.158624+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:58:27.305887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"84'4","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:18.296421+0000","last_change":"2023-11-16T16:37:13.509828+0000","last_active":"2023-11-16T16:37:18.296421+0000","last_peered":"2023-11-16T16:37:18.296421+0000","last_clean":"2023-11-16T16:37:18.296421+0000","last_became_active":"2023-11-16T16:37:13.509431+0000","last_became_peered":"2023-11-16T16:37:13.509431+0000","last_unstale":"2023-11-16T16:37:18.296421+0000","last_undegraded":"2023-11-16T16:37:18.296421+0000","last_fullsized":"2023-11-16T16:37:18.296421+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:49:39.240865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.156807+0000","last_change":"2023-11-16T16:37:13.940838+0000","last_active":"2023-11-16T16:37:15.156807+0000","last_peered":"2023-11-16T16:37:15.156807+0000","last_clean":"2023-11-16T16:37:15.156807+0000","last_became_active":"2023-11-16T16:37:13.940538+0000","last_became_peered":"2023-11-16T16:37:13.940538+0000","last_unstale":"2023-11-16T16:37:15.156807+0000","last_undegraded":"2023-11-16T16:37:15.156807+0000","last_fullsized":"2023-11-16T16:37:15.156807+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:24:41.595556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.925256+0000","last_change":"2023-11-16T16:37:13.495114+0000","last_active":"2023-11-16T16:37:14.925256+0000","last_peered":"2023-11-16T16:37:14.925256+0000","last_clean":"2023-11-16T16:37:14.925256+0000","last_became_active":"2023-11-16T16:37:13.494807+0000","last_became_peered":"2023-11-16T16:37:13.494807+0000","last_unstale":"2023-11-16T16:37:14.925256+0000","last_undegraded":"2023-11-16T16:37:14.925256+0000","last_fullsized":"2023-11-16T16:37:14.925256+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:24:04.736603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.924173+0000","last_change":"2023-11-16T16:37:13.943652+0000","last_active":"2023-11-16T16:37:14.924173+0000","last_peered":"2023-11-16T16:37:14.924173+0000","last_clean":"2023-11-16T16:37:14.924173+0000","last_became_active":"2023-11-16T16:37:13.943298+0000","last_became_peered":"2023-11-16T16:37:13.943298+0000","last_unstale":"2023-11-16T16:37:14.924173+0000","last_undegraded":"2023-11-16T16:37:14.924173+0000","last_fullsized":"2023-11-16T16:37:14.924173+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:34:34.876458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"84'6","reported_seq":60,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:54.357108+0000","last_change":"2023-11-16T16:37:13.509801+0000","last_active":"2023-11-16T16:37:54.357108+0000","last_peered":"2023-11-16T16:37:54.357108+0000","last_clean":"2023-11-16T16:37:54.357108+0000","last_became_active":"2023-11-16T16:37:13.509389+0000","last_became_peered":"2023-11-16T16:37:13.509389+0000","last_unstale":"2023-11-16T16:37:54.357108+0000","last_undegraded":"2023-11-16T16:37:54.357108+0000","last_fullsized":"2023-11-16T16:37:54.357108+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:55:02.922591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.924573+0000","last_change":"2023-11-16T16:37:13.933871+0000","last_active":"2023-11-16T16:37:14.924573+0000","last_peered":"2023-11-16T16:37:14.924573+0000","last_clean":"2023-11-16T16:37:14.924573+0000","last_became_active":"2023-11-16T16:37:13.933549+0000","last_became_peered":"2023-11-16T16:37:13.933549+0000","last_unstale":"2023-11-16T16:37:14.924573+0000","last_undegraded":"2023-11-16T16:37:14.924573+0000","last_fullsized":"2023-11-16T16:37:14.924573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:16:26.022577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"84'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:18.294818+0000","last_change":"2023-11-16T16:37:13.501339+0000","last_active":"2023-11-16T16:37:18.294818+0000","last_peered":"2023-11-16T16:37:18.294818+0000","last_clean":"2023-11-16T16:37:18.294818+0000","last_became_active":"2023-11-16T16:37:13.500949+0000","last_became_peered":"2023-11-16T16:37:13.500949+0000","last_unstale":"2023-11-16T16:37:18.294818+0000","last_undegraded":"2023-11-16T16:37:18.294818+0000","last_fullsized":"2023-11-16T16:37:18.294818+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:31:08.744759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.923893+0000","last_change":"2023-11-16T16:37:13.944158+0000","last_active":"2023-11-16T16:37:14.923893+0000","last_peered":"2023-11-16T16:37:14.923893+0000","last_clean":"2023-11-16T16:37:14.923893+0000","last_became_active":"2023-11-16T16:37:13.943833+0000","last_became_peered":"2023-11-16T16:37:13.943833+0000","last_unstale":"2023-11-16T16:37:14.923893+0000","last_undegraded":"2023-11-16T16:37:14.923893+0000","last_fullsized":"2023-11-16T16:37:14.923893+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:36:01.615006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"84'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:18.295129+0000","last_change":"2023-11-16T16:37:13.505481+0000","last_active":"2023-11-16T16:37:18.295129+0000","last_peered":"2023-11-16T16:37:18.295129+0000","last_clean":"2023-11-16T16:37:18.295129+0000","last_became_active":"2023-11-16T16:37:13.505085+0000","last_became_peered":"2023-11-16T16:37:13.505085+0000","last_unstale":"2023-11-16T16:37:18.295129+0000","last_undegraded":"2023-11-16T16:37:18.295129+0000","last_fullsized":"2023-11-16T16:37:18.295129+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:11:35.628249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157157+0000","last_change":"2023-11-16T16:37:13.943985+0000","last_active":"2023-11-16T16:37:15.157157+0000","last_peered":"2023-11-16T16:37:15.157157+0000","last_clean":"2023-11-16T16:37:15.157157+0000","last_became_active":"2023-11-16T16:37:13.943628+0000","last_became_peered":"2023-11-16T16:37:13.943628+0000","last_unstale":"2023-11-16T16:37:15.157157+0000","last_undegraded":"2023-11-16T16:37:15.157157+0000","last_fullsized":"2023-11-16T16:37:15.157157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:19:10.518280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.462357+0000","last_change":"2023-11-16T16:37:13.498648+0000","last_active":"2023-11-16T16:37:15.462357+0000","last_peered":"2023-11-16T16:37:15.462357+0000","last_clean":"2023-11-16T16:37:15.462357+0000","last_became_active":"2023-11-16T16:37:13.497390+0000","last_became_peered":"2023-11-16T16:37:13.497390+0000","last_unstale":"2023-11-16T16:37:15.462357+0000","last_undegraded":"2023-11-16T16:37:15.462357+0000","last_fullsized":"2023-11-16T16:37:15.462357+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:26:59.991663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157618+0000","last_change":"2023-11-16T16:37:13.942337+0000","last_active":"2023-11-16T16:37:15.157618+0000","last_peered":"2023-11-16T16:37:15.157618+0000","last_clean":"2023-11-16T16:37:15.157618+0000","last_became_active":"2023-11-16T16:37:13.941982+0000","last_became_peered":"2023-11-16T16:37:13.941982+0000","last_unstale":"2023-11-16T16:37:15.157618+0000","last_undegraded":"2023-11-16T16:37:15.157618+0000","last_fullsized":"2023-11-16T16:37:15.157618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:38:42.000956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'3","reported_seq":54,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:54.356512+0000","last_change":"2023-11-16T16:37:13.503690+0000","last_active":"2023-11-16T16:37:54.356512+0000","last_peered":"2023-11-16T16:37:54.356512+0000","last_clean":"2023-11-16T16:37:54.356512+0000","last_became_active":"2023-11-16T16:37:13.503203+0000","last_became_peered":"2023-11-16T16:37:13.503203+0000","last_unstale":"2023-11-16T16:37:54.356512+0000","last_undegraded":"2023-11-16T16:37:54.356512+0000","last_fullsized":"2023-11-16T16:37:54.356512+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:27:52.629218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.156964+0000","last_change":"2023-11-16T16:37:13.943975+0000","last_active":"2023-11-16T16:37:15.156964+0000","last_peered":"2023-11-16T16:37:15.156964+0000","last_clean":"2023-11-16T16:37:15.156964+0000","last_became_active":"2023-11-16T16:37:13.943611+0000","last_became_peered":"2023-11-16T16:37:13.943611+0000","last_unstale":"2023-11-16T16:37:15.156964+0000","last_undegraded":"2023-11-16T16:37:15.156964+0000","last_fullsized":"2023-11-16T16:37:15.156964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:21:29.078427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:17.285557+0000","last_change":"2023-11-16T16:37:13.506076+0000","last_active":"2023-11-16T16:37:17.285557+0000","last_peered":"2023-11-16T16:37:17.285557+0000","last_clean":"2023-11-16T16:37:17.285557+0000","last_became_active":"2023-11-16T16:37:13.505715+0000","last_became_peered":"2023-11-16T16:37:13.505715+0000","last_unstale":"2023-11-16T16:37:17.285557+0000","last_undegraded":"2023-11-16T16:37:17.285557+0000","last_fullsized":"2023-11-16T16:37:17.285557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:03:10.081004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158690+0000","last_change":"2023-11-16T16:37:13.949031+0000","last_active":"2023-11-16T16:37:15.158690+0000","last_peered":"2023-11-16T16:37:15.158690+0000","last_clean":"2023-11-16T16:37:15.158690+0000","last_became_active":"2023-11-16T16:37:13.946923+0000","last_became_peered":"2023-11-16T16:37:13.946923+0000","last_unstale":"2023-11-16T16:37:15.158690+0000","last_undegraded":"2023-11-16T16:37:15.158690+0000","last_fullsized":"2023-11-16T16:37:15.158690+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:00:35.683602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:17.286359+0000","last_change":"2023-11-16T16:37:13.495003+0000","last_active":"2023-11-16T16:37:17.286359+0000","last_peered":"2023-11-16T16:37:17.286359+0000","last_clean":"2023-11-16T16:37:17.286359+0000","last_became_active":"2023-11-16T16:37:13.494563+0000","last_became_peered":"2023-11-16T16:37:13.494563+0000","last_unstale":"2023-11-16T16:37:17.286359+0000","last_undegraded":"2023-11-16T16:37:17.286359+0000","last_fullsized":"2023-11-16T16:37:17.286359+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:12:53.840907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.156822+0000","last_change":"2023-11-16T16:37:13.943736+0000","last_active":"2023-11-16T16:37:15.156822+0000","last_peered":"2023-11-16T16:37:15.156822+0000","last_clean":"2023-11-16T16:37:15.156822+0000","last_became_active":"2023-11-16T16:37:13.943391+0000","last_became_peered":"2023-11-16T16:37:13.943391+0000","last_unstale":"2023-11-16T16:37:15.156822+0000","last_undegraded":"2023-11-16T16:37:15.156822+0000","last_fullsized":"2023-11-16T16:37:15.156822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:03:21.402516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'28","reported_seq":124,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:55.367854+0000","last_change":"2023-11-16T16:37:13.506508+0000","last_active":"2023-11-16T16:37:55.367854+0000","last_peered":"2023-11-16T16:37:55.367854+0000","last_clean":"2023-11-16T16:37:55.367854+0000","last_became_active":"2023-11-16T16:37:13.506045+0000","last_became_peered":"2023-11-16T16:37:13.506045+0000","last_unstale":"2023-11-16T16:37:55.367854+0000","last_undegraded":"2023-11-16T16:37:55.367854+0000","last_fullsized":"2023-11-16T16:37:55.367854+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:26:04.265699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58527,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":97,"num_write":29,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.924203+0000","last_change":"2023-11-16T16:37:13.939717+0000","last_active":"2023-11-16T16:37:14.924203+0000","last_peered":"2023-11-16T16:37:14.924203+0000","last_clean":"2023-11-16T16:37:14.924203+0000","last_became_active":"2023-11-16T16:37:13.939369+0000","last_became_peered":"2023-11-16T16:37:13.939369+0000","last_unstale":"2023-11-16T16:37:14.924203+0000","last_undegraded":"2023-11-16T16:37:14.924203+0000","last_fullsized":"2023-11-16T16:37:14.924203+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:15:46.898377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'6","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:18.296464+0000","last_change":"2023-11-16T16:37:13.506056+0000","last_active":"2023-11-16T16:37:18.296464+0000","last_peered":"2023-11-16T16:37:18.296464+0000","last_clean":"2023-11-16T16:37:18.296464+0000","last_became_active":"2023-11-16T16:37:13.505707+0000","last_became_peered":"2023-11-16T16:37:13.505707+0000","last_unstale":"2023-11-16T16:37:18.296464+0000","last_undegraded":"2023-11-16T16:37:18.296464+0000","last_fullsized":"2023-11-16T16:37:18.296464+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:36:08.667708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157120+0000","last_change":"2023-11-16T16:37:13.947764+0000","last_active":"2023-11-16T16:37:15.157120+0000","last_peered":"2023-11-16T16:37:15.157120+0000","last_clean":"2023-11-16T16:37:15.157120+0000","last_became_active":"2023-11-16T16:37:13.947510+0000","last_became_peered":"2023-11-16T16:37:13.947510+0000","last_unstale":"2023-11-16T16:37:15.157120+0000","last_undegraded":"2023-11-16T16:37:15.157120+0000","last_fullsized":"2023-11-16T16:37:15.157120+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:08:15.784489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:17.287513+0000","last_change":"2023-11-16T16:37:13.498178+0000","last_active":"2023-11-16T16:37:17.287513+0000","last_peered":"2023-11-16T16:37:17.287513+0000","last_clean":"2023-11-16T16:37:17.287513+0000","last_became_active":"2023-11-16T16:37:13.497312+0000","last_became_peered":"2023-11-16T16:37:13.497312+0000","last_unstale":"2023-11-16T16:37:17.287513+0000","last_undegraded":"2023-11-16T16:37:17.287513+0000","last_fullsized":"2023-11-16T16:37:17.287513+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:01:58.840690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.391132+0000","last_change":"2023-11-16T16:37:13.947854+0000","last_active":"2023-11-16T16:37:15.391132+0000","last_peered":"2023-11-16T16:37:15.391132+0000","last_clean":"2023-11-16T16:37:15.391132+0000","last_became_active":"2023-11-16T16:37:13.947530+0000","last_became_peered":"2023-11-16T16:37:13.947530+0000","last_unstale":"2023-11-16T16:37:15.391132+0000","last_undegraded":"2023-11-16T16:37:15.391132+0000","last_fullsized":"2023-11-16T16:37:15.391132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:25:25.919855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'3","reported_seq":59,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:57.360983+0000","last_change":"2023-11-16T16:37:13.500127+0000","last_active":"2023-11-16T16:37:57.360983+0000","last_peered":"2023-11-16T16:37:57.360983+0000","last_clean":"2023-11-16T16:37:57.360983+0000","last_became_active":"2023-11-16T16:37:13.499758+0000","last_became_peered":"2023-11-16T16:37:13.499758+0000","last_unstale":"2023-11-16T16:37:57.360983+0000","last_undegraded":"2023-11-16T16:37:57.360983+0000","last_fullsized":"2023-11-16T16:37:57.360983+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:12:24.610215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.390796+0000","last_change":"2023-11-16T16:37:13.940962+0000","last_active":"2023-11-16T16:37:15.390796+0000","last_peered":"2023-11-16T16:37:15.390796+0000","last_clean":"2023-11-16T16:37:15.390796+0000","last_became_active":"2023-11-16T16:37:13.940450+0000","last_became_peered":"2023-11-16T16:37:13.940450+0000","last_unstale":"2023-11-16T16:37:15.390796+0000","last_undegraded":"2023-11-16T16:37:15.390796+0000","last_fullsized":"2023-11-16T16:37:15.390796+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:20:04.879494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'15","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:18.297513+0000","last_change":"2023-11-16T16:37:13.493017+0000","last_active":"2023-11-16T16:37:18.297513+0000","last_peered":"2023-11-16T16:37:18.297513+0000","last_clean":"2023-11-16T16:37:18.297513+0000","last_became_active":"2023-11-16T16:37:13.492599+0000","last_became_peered":"2023-11-16T16:37:13.492599+0000","last_unstale":"2023-11-16T16:37:18.297513+0000","last_undegraded":"2023-11-16T16:37:18.297513+0000","last_fullsized":"2023-11-16T16:37:18.297513+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:25:52.252764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.924063+0000","last_change":"2023-11-16T16:37:13.944878+0000","last_active":"2023-11-16T16:37:14.924063+0000","last_peered":"2023-11-16T16:37:14.924063+0000","last_clean":"2023-11-16T16:37:14.924063+0000","last_became_active":"2023-11-16T16:37:13.944580+0000","last_became_peered":"2023-11-16T16:37:13.944580+0000","last_unstale":"2023-11-16T16:37:14.924063+0000","last_undegraded":"2023-11-16T16:37:14.924063+0000","last_fullsized":"2023-11-16T16:37:14.924063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:01:52.805052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:19.300067+0000","last_change":"2023-11-16T16:37:13.499139+0000","last_active":"2023-11-16T16:37:19.300067+0000","last_peered":"2023-11-16T16:37:19.300067+0000","last_clean":"2023-11-16T16:37:19.300067+0000","last_became_active":"2023-11-16T16:37:13.498755+0000","last_became_peered":"2023-11-16T16:37:13.498755+0000","last_unstale":"2023-11-16T16:37:19.300067+0000","last_undegraded":"2023-11-16T16:37:19.300067+0000","last_fullsized":"2023-11-16T16:37:19.300067+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:07:40.535306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157785+0000","last_change":"2023-11-16T16:37:13.946187+0000","last_active":"2023-11-16T16:37:15.157785+0000","last_peered":"2023-11-16T16:37:15.157785+0000","last_clean":"2023-11-16T16:37:15.157785+0000","last_became_active":"2023-11-16T16:37:13.945887+0000","last_became_peered":"2023-11-16T16:37:13.945887+0000","last_unstale":"2023-11-16T16:37:15.157785+0000","last_undegraded":"2023-11-16T16:37:15.157785+0000","last_fullsized":"2023-11-16T16:37:15.157785+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:29:41.439670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157191+0000","last_change":"2023-11-16T16:37:13.507150+0000","last_active":"2023-11-16T16:37:15.157191+0000","last_peered":"2023-11-16T16:37:15.157191+0000","last_clean":"2023-11-16T16:37:15.157191+0000","last_became_active":"2023-11-16T16:37:13.506814+0000","last_became_peered":"2023-11-16T16:37:13.506814+0000","last_unstale":"2023-11-16T16:37:15.157191+0000","last_undegraded":"2023-11-16T16:37:15.157191+0000","last_fullsized":"2023-11-16T16:37:15.157191+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:58:14.692816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.462555+0000","last_change":"2023-11-16T16:37:13.947866+0000","last_active":"2023-11-16T16:37:15.462555+0000","last_peered":"2023-11-16T16:37:15.462555+0000","last_clean":"2023-11-16T16:37:15.462555+0000","last_became_active":"2023-11-16T16:37:13.947544+0000","last_became_peered":"2023-11-16T16:37:13.947544+0000","last_unstale":"2023-11-16T16:37:15.462555+0000","last_undegraded":"2023-11-16T16:37:15.462555+0000","last_fullsized":"2023-11-16T16:37:15.462555+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:58:42.237793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"84'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.391037+0000","last_change":"2023-11-16T16:37:13.497800+0000","last_active":"2023-11-16T16:37:15.391037+0000","last_peered":"2023-11-16T16:37:15.391037+0000","last_clean":"2023-11-16T16:37:15.391037+0000","last_became_active":"2023-11-16T16:37:13.497414+0000","last_became_peered":"2023-11-16T16:37:13.497414+0000","last_unstale":"2023-11-16T16:37:15.391037+0000","last_undegraded":"2023-11-16T16:37:15.391037+0000","last_fullsized":"2023-11-16T16:37:15.391037+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:57:26.591481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.924384+0000","last_change":"2023-11-16T16:37:13.946780+0000","last_active":"2023-11-16T16:37:14.924384+0000","last_peered":"2023-11-16T16:37:14.924384+0000","last_clean":"2023-11-16T16:37:14.924384+0000","last_became_active":"2023-11-16T16:37:13.946441+0000","last_became_peered":"2023-11-16T16:37:13.946441+0000","last_unstale":"2023-11-16T16:37:14.924384+0000","last_undegraded":"2023-11-16T16:37:14.924384+0000","last_fullsized":"2023-11-16T16:37:14.924384+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:20:09.943373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.923519+0000","last_change":"2023-11-16T16:37:13.937898+0000","last_active":"2023-11-16T16:37:14.923519+0000","last_peered":"2023-11-16T16:37:14.923519+0000","last_clean":"2023-11-16T16:37:14.923519+0000","last_became_active":"2023-11-16T16:37:13.937598+0000","last_became_peered":"2023-11-16T16:37:13.937598+0000","last_unstale":"2023-11-16T16:37:14.923519+0000","last_undegraded":"2023-11-16T16:37:14.923519+0000","last_fullsized":"2023-11-16T16:37:14.923519+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:57:03.646617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"84'4","reported_seq":58,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:57.360346+0000","last_change":"2023-11-16T16:37:13.498129+0000","last_active":"2023-11-16T16:37:57.360346+0000","last_peered":"2023-11-16T16:37:57.360346+0000","last_clean":"2023-11-16T16:37:57.360346+0000","last_became_active":"2023-11-16T16:37:13.497765+0000","last_became_peered":"2023-11-16T16:37:13.497765+0000","last_unstale":"2023-11-16T16:37:57.360346+0000","last_undegraded":"2023-11-16T16:37:57.360346+0000","last_fullsized":"2023-11-16T16:37:57.360346+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:13:33.579950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158156+0000","last_change":"2023-11-16T16:37:13.938417+0000","last_active":"2023-11-16T16:37:15.158156+0000","last_peered":"2023-11-16T16:37:15.158156+0000","last_clean":"2023-11-16T16:37:15.158156+0000","last_became_active":"2023-11-16T16:37:13.938114+0000","last_became_peered":"2023-11-16T16:37:13.938114+0000","last_unstale":"2023-11-16T16:37:15.158156+0000","last_undegraded":"2023-11-16T16:37:15.158156+0000","last_fullsized":"2023-11-16T16:37:15.158156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:35:24.220360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"84'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:19.298484+0000","last_change":"2023-11-16T16:37:13.499045+0000","last_active":"2023-11-16T16:37:19.298484+0000","last_peered":"2023-11-16T16:37:19.298484+0000","last_clean":"2023-11-16T16:37:19.298484+0000","last_became_active":"2023-11-16T16:37:13.498598+0000","last_became_peered":"2023-11-16T16:37:13.498598+0000","last_unstale":"2023-11-16T16:37:19.298484+0000","last_undegraded":"2023-11-16T16:37:19.298484+0000","last_fullsized":"2023-11-16T16:37:19.298484+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:25:26.520931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":345,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.390796+0000","last_change":"2023-11-16T16:36:14.090213+0000","last_active":"2023-11-16T16:37:15.390796+0000","last_peered":"2023-11-16T16:37:15.390796+0000","last_clean":"2023-11-16T16:37:15.390796+0000","last_became_active":"2023-11-16T16:36:14.089742+0000","last_became_peered":"2023-11-16T16:36:14.089742+0000","last_unstale":"2023-11-16T16:37:15.390796+0000","last_undegraded":"2023-11-16T16:37:15.390796+0000","last_fullsized":"2023-11-16T16:37:15.390796+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:33:16.178716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:33:16.178716+0000","last_clean_scrub_stamp":"2023-11-16T16:33:16.178716+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:11:37.401838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157103+0000","last_change":"2023-11-16T16:37:13.945380+0000","last_active":"2023-11-16T16:37:15.157103+0000","last_peered":"2023-11-16T16:37:15.157103+0000","last_clean":"2023-11-16T16:37:15.157103+0000","last_became_active":"2023-11-16T16:37:13.945038+0000","last_became_peered":"2023-11-16T16:37:13.945038+0000","last_unstale":"2023-11-16T16:37:15.157103+0000","last_undegraded":"2023-11-16T16:37:15.157103+0000","last_fullsized":"2023-11-16T16:37:15.157103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:35:29.662867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"84'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:18.296049+0000","last_change":"2023-11-16T16:37:13.497586+0000","last_active":"2023-11-16T16:37:18.296049+0000","last_peered":"2023-11-16T16:37:18.296049+0000","last_clean":"2023-11-16T16:37:18.296049+0000","last_became_active":"2023-11-16T16:37:13.497233+0000","last_became_peered":"2023-11-16T16:37:13.497233+0000","last_unstale":"2023-11-16T16:37:18.296049+0000","last_undegraded":"2023-11-16T16:37:18.296049+0000","last_fullsized":"2023-11-16T16:37:18.296049+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:24:30.086496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"84'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:17.285613+0000","last_change":"2023-11-16T16:37:13.500621+0000","last_active":"2023-11-16T16:37:17.285613+0000","last_peered":"2023-11-16T16:37:17.285613+0000","last_clean":"2023-11-16T16:37:17.285613+0000","last_became_active":"2023-11-16T16:37:13.500265+0000","last_became_peered":"2023-11-16T16:37:13.500265+0000","last_unstale":"2023-11-16T16:37:17.285613+0000","last_undegraded":"2023-11-16T16:37:17.285613+0000","last_fullsized":"2023-11-16T16:37:17.285613+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:51:28.450861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157560+0000","last_change":"2023-11-16T16:37:13.930992+0000","last_active":"2023-11-16T16:37:15.157560+0000","last_peered":"2023-11-16T16:37:15.157560+0000","last_clean":"2023-11-16T16:37:15.157560+0000","last_became_active":"2023-11-16T16:37:13.930641+0000","last_became_peered":"2023-11-16T16:37:13.930641+0000","last_unstale":"2023-11-16T16:37:15.157560+0000","last_undegraded":"2023-11-16T16:37:15.157560+0000","last_fullsized":"2023-11-16T16:37:15.157560+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:58:37.743753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'5","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:24.183823+0000","last_change":"2023-11-16T16:37:13.498637+0000","last_active":"2023-11-16T16:37:24.183823+0000","last_peered":"2023-11-16T16:37:24.183823+0000","last_clean":"2023-11-16T16:37:24.183823+0000","last_became_active":"2023-11-16T16:37:13.498341+0000","last_became_peered":"2023-11-16T16:37:13.498341+0000","last_unstale":"2023-11-16T16:37:24.183823+0000","last_undegraded":"2023-11-16T16:37:24.183823+0000","last_fullsized":"2023-11-16T16:37:24.183823+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:00:31.333304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158333+0000","last_change":"2023-11-16T16:37:13.944142+0000","last_active":"2023-11-16T16:37:15.158333+0000","last_peered":"2023-11-16T16:37:15.158333+0000","last_clean":"2023-11-16T16:37:15.158333+0000","last_became_active":"2023-11-16T16:37:13.943758+0000","last_became_peered":"2023-11-16T16:37:13.943758+0000","last_unstale":"2023-11-16T16:37:15.158333+0000","last_undegraded":"2023-11-16T16:37:15.158333+0000","last_fullsized":"2023-11-16T16:37:15.158333+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:55:52.343549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'4","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:17.287097+0000","last_change":"2023-11-16T16:37:13.496900+0000","last_active":"2023-11-16T16:37:17.287097+0000","last_peered":"2023-11-16T16:37:17.287097+0000","last_clean":"2023-11-16T16:37:17.287097+0000","last_became_active":"2023-11-16T16:37:13.496528+0000","last_became_peered":"2023-11-16T16:37:13.496528+0000","last_unstale":"2023-11-16T16:37:17.287097+0000","last_undegraded":"2023-11-16T16:37:17.287097+0000","last_fullsized":"2023-11-16T16:37:17.287097+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:37:22.646222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158258+0000","last_change":"2023-11-16T16:37:13.943104+0000","last_active":"2023-11-16T16:37:15.158258+0000","last_peered":"2023-11-16T16:37:15.158258+0000","last_clean":"2023-11-16T16:37:15.158258+0000","last_became_active":"2023-11-16T16:37:13.942774+0000","last_became_peered":"2023-11-16T16:37:13.942774+0000","last_unstale":"2023-11-16T16:37:15.158258+0000","last_undegraded":"2023-11-16T16:37:15.158258+0000","last_fullsized":"2023-11-16T16:37:15.158258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:00:01.892620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:17.287386+0000","last_change":"2023-11-16T16:37:13.505994+0000","last_active":"2023-11-16T16:37:17.287386+0000","last_peered":"2023-11-16T16:37:17.287386+0000","last_clean":"2023-11-16T16:37:17.287386+0000","last_became_active":"2023-11-16T16:37:13.505663+0000","last_became_peered":"2023-11-16T16:37:13.505663+0000","last_unstale":"2023-11-16T16:37:17.287386+0000","last_undegraded":"2023-11-16T16:37:17.287386+0000","last_fullsized":"2023-11-16T16:37:17.287386+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:11:39.436119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.923847+0000","last_change":"2023-11-16T16:37:13.941970+0000","last_active":"2023-11-16T16:37:14.923847+0000","last_peered":"2023-11-16T16:37:14.923847+0000","last_clean":"2023-11-16T16:37:14.923847+0000","last_became_active":"2023-11-16T16:37:13.941649+0000","last_became_peered":"2023-11-16T16:37:13.941649+0000","last_unstale":"2023-11-16T16:37:14.923847+0000","last_undegraded":"2023-11-16T16:37:14.923847+0000","last_fullsized":"2023-11-16T16:37:14.923847+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:00:21.646221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:17.287982+0000","last_change":"2023-11-16T16:37:13.503429+0000","last_active":"2023-11-16T16:37:17.287982+0000","last_peered":"2023-11-16T16:37:17.287982+0000","last_clean":"2023-11-16T16:37:17.287982+0000","last_became_active":"2023-11-16T16:37:13.503114+0000","last_became_peered":"2023-11-16T16:37:13.503114+0000","last_unstale":"2023-11-16T16:37:17.287982+0000","last_undegraded":"2023-11-16T16:37:17.287982+0000","last_fullsized":"2023-11-16T16:37:17.287982+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:05:42.643043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157164+0000","last_change":"2023-11-16T16:37:13.941225+0000","last_active":"2023-11-16T16:37:15.157164+0000","last_peered":"2023-11-16T16:37:15.157164+0000","last_clean":"2023-11-16T16:37:15.157164+0000","last_became_active":"2023-11-16T16:37:13.940964+0000","last_became_peered":"2023-11-16T16:37:13.940964+0000","last_unstale":"2023-11-16T16:37:15.157164+0000","last_undegraded":"2023-11-16T16:37:15.157164+0000","last_fullsized":"2023-11-16T16:37:15.157164+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:16:06.854930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157663+0000","last_change":"2023-11-16T16:37:13.510154+0000","last_active":"2023-11-16T16:37:15.157663+0000","last_peered":"2023-11-16T16:37:15.157663+0000","last_clean":"2023-11-16T16:37:15.157663+0000","last_became_active":"2023-11-16T16:37:13.509790+0000","last_became_peered":"2023-11-16T16:37:13.509790+0000","last_unstale":"2023-11-16T16:37:15.157663+0000","last_undegraded":"2023-11-16T16:37:15.157663+0000","last_fullsized":"2023-11-16T16:37:15.157663+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T16:54:36.581541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158172+0000","last_change":"2023-11-16T16:37:13.942203+0000","last_active":"2023-11-16T16:37:15.158172+0000","last_peered":"2023-11-16T16:37:15.158172+0000","last_clean":"2023-11-16T16:37:15.158172+0000","last_became_active":"2023-11-16T16:37:13.941766+0000","last_became_peered":"2023-11-16T16:37:13.941766+0000","last_unstale":"2023-11-16T16:37:15.158172+0000","last_undegraded":"2023-11-16T16:37:15.158172+0000","last_fullsized":"2023-11-16T16:37:15.158172+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:12:25.582734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:17.286155+0000","last_change":"2023-11-16T16:37:13.509854+0000","last_active":"2023-11-16T16:37:17.286155+0000","last_peered":"2023-11-16T16:37:17.286155+0000","last_clean":"2023-11-16T16:37:17.286155+0000","last_became_active":"2023-11-16T16:37:13.509466+0000","last_became_peered":"2023-11-16T16:37:13.509466+0000","last_unstale":"2023-11-16T16:37:17.286155+0000","last_undegraded":"2023-11-16T16:37:17.286155+0000","last_fullsized":"2023-11-16T16:37:17.286155+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:05:41.806694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.391285+0000","last_change":"2023-11-16T16:37:13.947906+0000","last_active":"2023-11-16T16:37:15.391285+0000","last_peered":"2023-11-16T16:37:15.391285+0000","last_clean":"2023-11-16T16:37:15.391285+0000","last_became_active":"2023-11-16T16:37:13.947602+0000","last_became_peered":"2023-11-16T16:37:13.947602+0000","last_unstale":"2023-11-16T16:37:15.391285+0000","last_undegraded":"2023-11-16T16:37:15.391285+0000","last_fullsized":"2023-11-16T16:37:15.391285+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:16:47.335757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:18.297466+0000","last_change":"2023-11-16T16:37:13.506108+0000","last_active":"2023-11-16T16:37:18.297466+0000","last_peered":"2023-11-16T16:37:18.297466+0000","last_clean":"2023-11-16T16:37:18.297466+0000","last_became_active":"2023-11-16T16:37:13.505785+0000","last_became_peered":"2023-11-16T16:37:13.505785+0000","last_unstale":"2023-11-16T16:37:18.297466+0000","last_undegraded":"2023-11-16T16:37:18.297466+0000","last_fullsized":"2023-11-16T16:37:18.297466+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:01:15.544970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.391265+0000","last_change":"2023-11-16T16:37:13.935863+0000","last_active":"2023-11-16T16:37:15.391265+0000","last_peered":"2023-11-16T16:37:15.391265+0000","last_clean":"2023-11-16T16:37:15.391265+0000","last_became_active":"2023-11-16T16:37:13.935399+0000","last_became_peered":"2023-11-16T16:37:13.935399+0000","last_unstale":"2023-11-16T16:37:15.391265+0000","last_undegraded":"2023-11-16T16:37:15.391265+0000","last_fullsized":"2023-11-16T16:37:15.391265+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:38:54.382348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.462427+0000","last_change":"2023-11-16T16:37:13.498367+0000","last_active":"2023-11-16T16:37:15.462427+0000","last_peered":"2023-11-16T16:37:15.462427+0000","last_clean":"2023-11-16T16:37:15.462427+0000","last_became_active":"2023-11-16T16:37:13.497039+0000","last_became_peered":"2023-11-16T16:37:13.497039+0000","last_unstale":"2023-11-16T16:37:15.462427+0000","last_undegraded":"2023-11-16T16:37:15.462427+0000","last_fullsized":"2023-11-16T16:37:15.462427+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:07:52.215240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.391224+0000","last_change":"2023-11-16T16:37:13.940617+0000","last_active":"2023-11-16T16:37:15.391224+0000","last_peered":"2023-11-16T16:37:15.391224+0000","last_clean":"2023-11-16T16:37:15.391224+0000","last_became_active":"2023-11-16T16:37:13.940264+0000","last_became_peered":"2023-11-16T16:37:13.940264+0000","last_unstale":"2023-11-16T16:37:15.391224+0000","last_undegraded":"2023-11-16T16:37:15.391224+0000","last_fullsized":"2023-11-16T16:37:15.391224+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:22:54.941976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157351+0000","last_change":"2023-11-16T16:37:13.505950+0000","last_active":"2023-11-16T16:37:15.157351+0000","last_peered":"2023-11-16T16:37:15.157351+0000","last_clean":"2023-11-16T16:37:15.157351+0000","last_became_active":"2023-11-16T16:37:13.505628+0000","last_became_peered":"2023-11-16T16:37:13.505628+0000","last_unstale":"2023-11-16T16:37:15.157351+0000","last_undegraded":"2023-11-16T16:37:15.157351+0000","last_fullsized":"2023-11-16T16:37:15.157351+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:11:17.082705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.391279+0000","last_change":"2023-11-16T16:37:13.943096+0000","last_active":"2023-11-16T16:37:15.391279+0000","last_peered":"2023-11-16T16:37:15.391279+0000","last_clean":"2023-11-16T16:37:15.391279+0000","last_became_active":"2023-11-16T16:37:13.942745+0000","last_became_peered":"2023-11-16T16:37:13.942745+0000","last_unstale":"2023-11-16T16:37:15.391279+0000","last_undegraded":"2023-11-16T16:37:15.391279+0000","last_fullsized":"2023-11-16T16:37:15.391279+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:35:21.092899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157245+0000","last_change":"2023-11-16T16:37:13.938087+0000","last_active":"2023-11-16T16:37:15.157245+0000","last_peered":"2023-11-16T16:37:15.157245+0000","last_clean":"2023-11-16T16:37:15.157245+0000","last_became_active":"2023-11-16T16:37:13.937780+0000","last_became_peered":"2023-11-16T16:37:13.937780+0000","last_unstale":"2023-11-16T16:37:15.157245+0000","last_undegraded":"2023-11-16T16:37:15.157245+0000","last_fullsized":"2023-11-16T16:37:15.157245+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:41:00.785196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157978+0000","last_change":"2023-11-16T16:37:13.939000+0000","last_active":"2023-11-16T16:37:15.157978+0000","last_peered":"2023-11-16T16:37:15.157978+0000","last_clean":"2023-11-16T16:37:15.157978+0000","last_became_active":"2023-11-16T16:37:13.938585+0000","last_became_peered":"2023-11-16T16:37:13.938585+0000","last_unstale":"2023-11-16T16:37:15.157978+0000","last_undegraded":"2023-11-16T16:37:15.157978+0000","last_fullsized":"2023-11-16T16:37:15.157978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:17:47.012171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157516+0000","last_change":"2023-11-16T16:37:13.937696+0000","last_active":"2023-11-16T16:37:15.157516+0000","last_peered":"2023-11-16T16:37:15.157516+0000","last_clean":"2023-11-16T16:37:15.157516+0000","last_became_active":"2023-11-16T16:37:13.936920+0000","last_became_peered":"2023-11-16T16:37:13.936920+0000","last_unstale":"2023-11-16T16:37:15.157516+0000","last_undegraded":"2023-11-16T16:37:15.157516+0000","last_fullsized":"2023-11-16T16:37:15.157516+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:40:10.578228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157738+0000","last_change":"2023-11-16T16:37:13.942977+0000","last_active":"2023-11-16T16:37:15.157738+0000","last_peered":"2023-11-16T16:37:15.157738+0000","last_clean":"2023-11-16T16:37:15.157738+0000","last_became_active":"2023-11-16T16:37:13.942632+0000","last_became_peered":"2023-11-16T16:37:13.942632+0000","last_unstale":"2023-11-16T16:37:15.157738+0000","last_undegraded":"2023-11-16T16:37:15.157738+0000","last_fullsized":"2023-11-16T16:37:15.157738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:44:02.416862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157651+0000","last_change":"2023-11-16T16:37:13.949940+0000","last_active":"2023-11-16T16:37:15.157651+0000","last_peered":"2023-11-16T16:37:15.157651+0000","last_clean":"2023-11-16T16:37:15.157651+0000","last_became_active":"2023-11-16T16:37:13.949464+0000","last_became_peered":"2023-11-16T16:37:13.949464+0000","last_unstale":"2023-11-16T16:37:15.157651+0000","last_undegraded":"2023-11-16T16:37:15.157651+0000","last_fullsized":"2023-11-16T16:37:15.157651+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:52:47.439482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157535+0000","last_change":"2023-11-16T16:37:13.940534+0000","last_active":"2023-11-16T16:37:15.157535+0000","last_peered":"2023-11-16T16:37:15.157535+0000","last_clean":"2023-11-16T16:37:15.157535+0000","last_became_active":"2023-11-16T16:37:13.940174+0000","last_became_peered":"2023-11-16T16:37:13.940174+0000","last_unstale":"2023-11-16T16:37:15.157535+0000","last_undegraded":"2023-11-16T16:37:15.157535+0000","last_fullsized":"2023-11-16T16:37:15.157535+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:25:01.730261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157969+0000","last_change":"2023-11-16T16:37:13.937519+0000","last_active":"2023-11-16T16:37:15.157969+0000","last_peered":"2023-11-16T16:37:15.157969+0000","last_clean":"2023-11-16T16:37:15.157969+0000","last_became_active":"2023-11-16T16:37:13.937175+0000","last_became_peered":"2023-11-16T16:37:13.937175+0000","last_unstale":"2023-11-16T16:37:15.157969+0000","last_undegraded":"2023-11-16T16:37:15.157969+0000","last_fullsized":"2023-11-16T16:37:15.157969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T16:47:03.319128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158262+0000","last_change":"2023-11-16T16:37:13.951506+0000","last_active":"2023-11-16T16:37:15.158262+0000","last_peered":"2023-11-16T16:37:15.158262+0000","last_clean":"2023-11-16T16:37:15.158262+0000","last_became_active":"2023-11-16T16:37:13.951192+0000","last_became_peered":"2023-11-16T16:37:13.951192+0000","last_unstale":"2023-11-16T16:37:15.158262+0000","last_undegraded":"2023-11-16T16:37:15.158262+0000","last_fullsized":"2023-11-16T16:37:15.158262+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:18:48.061220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.923594+0000","last_change":"2023-11-16T16:37:13.935614+0000","last_active":"2023-11-16T16:37:14.923594+0000","last_peered":"2023-11-16T16:37:14.923594+0000","last_clean":"2023-11-16T16:37:14.923594+0000","last_became_active":"2023-11-16T16:37:13.935237+0000","last_became_peered":"2023-11-16T16:37:13.935237+0000","last_unstale":"2023-11-16T16:37:14.923594+0000","last_undegraded":"2023-11-16T16:37:14.923594+0000","last_fullsized":"2023-11-16T16:37:14.923594+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:02:10.333347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157636+0000","last_change":"2023-11-16T16:37:13.939387+0000","last_active":"2023-11-16T16:37:15.157636+0000","last_peered":"2023-11-16T16:37:15.157636+0000","last_clean":"2023-11-16T16:37:15.157636+0000","last_became_active":"2023-11-16T16:37:13.939082+0000","last_became_peered":"2023-11-16T16:37:13.939082+0000","last_unstale":"2023-11-16T16:37:15.157636+0000","last_undegraded":"2023-11-16T16:37:15.157636+0000","last_fullsized":"2023-11-16T16:37:15.157636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:42:24.707911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.156764+0000","last_change":"2023-11-16T16:37:13.936381+0000","last_active":"2023-11-16T16:37:15.156764+0000","last_peered":"2023-11-16T16:37:15.156764+0000","last_clean":"2023-11-16T16:37:15.156764+0000","last_became_active":"2023-11-16T16:37:13.936077+0000","last_became_peered":"2023-11-16T16:37:13.936077+0000","last_unstale":"2023-11-16T16:37:15.156764+0000","last_undegraded":"2023-11-16T16:37:15.156764+0000","last_fullsized":"2023-11-16T16:37:15.156764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:04:48.256913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157497+0000","last_change":"2023-11-16T16:37:13.934005+0000","last_active":"2023-11-16T16:37:15.157497+0000","last_peered":"2023-11-16T16:37:15.157497+0000","last_clean":"2023-11-16T16:37:15.157497+0000","last_became_active":"2023-11-16T16:37:13.933633+0000","last_became_peered":"2023-11-16T16:37:13.933633+0000","last_unstale":"2023-11-16T16:37:15.157497+0000","last_undegraded":"2023-11-16T16:37:15.157497+0000","last_fullsized":"2023-11-16T16:37:15.157497+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:45:12.620887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158013+0000","last_change":"2023-11-16T16:37:13.946495+0000","last_active":"2023-11-16T16:37:15.158013+0000","last_peered":"2023-11-16T16:37:15.158013+0000","last_clean":"2023-11-16T16:37:15.158013+0000","last_became_active":"2023-11-16T16:37:13.946148+0000","last_became_peered":"2023-11-16T16:37:13.946148+0000","last_unstale":"2023-11-16T16:37:15.158013+0000","last_undegraded":"2023-11-16T16:37:15.158013+0000","last_fullsized":"2023-11-16T16:37:15.158013+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:17:05.329636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":3542208,"data_compressed":211753,"data_compressed_allocated":270336,"data_compressed_original":3522560,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62431,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":255,"num_read_kb":193,"num_write":150,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":187293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132,"ondisk_log_size":132,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416405,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5944,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":93739208,"statfs":{"total":95995035648,"available":95988948992,"internally_reserved":0,"allocated":901120,"data_stored":429451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.56399999999999995}]},{"osd":1,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.46200000000000002}]},{"osd":2,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.438},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.48599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":4,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.64500000000000002}]},{"osd":5,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.81200000000000006}]},{"osd":6,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":0.69699999999999995}]},{"osd":7,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.82999999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":9,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.95299999999999996}]},{"osd":10,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.863,"5min":1.863,"15min":1.863},"last":0.79900000000000004}]}]},{"osd":10,"up_from":72,"seq":309237645336,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6408,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738744,"statfs":{"total":95995035648,"available":95988473856,"internally_reserved":0,"allocated":1048576,"data_stored":1079086,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":1,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.65800000000000003}]},{"osd":2,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.77200000000000002}]},{"osd":3,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.754}]},{"osd":4,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.84599999999999997}]},{"osd":5,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.93500000000000005}]},{"osd":6,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.97099999999999997}]},{"osd":7,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.505}]},{"osd":8,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":1.0760000000000001},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":1.0169999999999999}]},{"osd":9,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.98499999999999999}]},{"osd":11,"last update":"Thu Nov 16 16:37:33 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":1.0900000000000001},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":1.1180000000000001}]}]},{"osd":9,"up_from":67,"seq":287762808860,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6088,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93739064,"statfs":{"total":95995035648,"available":95988801536,"internally_reserved":0,"allocated":917504,"data_stored":429687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.45100000000000001,"15min":0.46300000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.59699999999999998,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.47199999999999998,"15min":0.48199999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.28199999999999997}]},{"osd":1,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.59199999999999997,"15min":0.60599999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.61799999999999999,"15min":0.63300000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76100000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.433}]},{"osd":2,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.63200000000000001,"15min":0.64600000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.78900000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.63500000000000001,"15min":0.64300000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.46400000000000002}]},{"osd":3,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.54800000000000004,"15min":0.56399999999999995},"min":{"1min":0.33200000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73299999999999998,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.55100000000000005,"15min":0.57299999999999995},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.70899999999999996,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.39600000000000002}]},{"osd":4,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.66200000000000003,"15min":0.68100000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.873,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.67300000000000004,"15min":0.68999999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.83599999999999997,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.45500000000000002}]},{"osd":5,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.69599999999999995,"15min":0.70999999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.878,"5min":1.056,"15min":1.056},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.72099999999999997,"15min":0.73899999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85999999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.48299999999999998}]},{"osd":6,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.64900000000000002,"15min":0.67200000000000004},"min":{"1min":0.38400000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.81699999999999995,"5min":1.069,"15min":1.069},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.70399999999999996,"15min":0.72799999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85099999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.503}]},{"osd":7,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.755,"15min":0.77300000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92200000000000004,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.77100000000000002,"15min":0.79500000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.89600000000000002,"5min":1.111,"15min":1.111},"last":0.55200000000000005}]},{"osd":8,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.80400000000000005,"15min":0.82499999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.95899999999999996,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.80100000000000005,"15min":0.81599999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.97399999999999998,"5min":1.089,"15min":1.089},"last":0.58199999999999996}]},{"osd":10,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.53700000000000003}]}]},{"osd":8,"up_from":59,"seq":253403070498,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6280,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738872,"statfs":{"total":95995035648,"available":95988604928,"internally_reserved":0,"allocated":917504,"data_stored":429607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.77000000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.44400000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.47399999999999998}]},{"osd":1,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.627,"15min":0.61699999999999999},"min":{"1min":0.46100000000000002,"5min":0.318,"15min":0.318},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.46200000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.63800000000000001}]},{"osd":2,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.50600000000000001,"15min":0.51100000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71299999999999997,"5min":1.056,"15min":1.056},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.51500000000000001,"15min":0.52000000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71499999999999997,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.57599999999999996}]},{"osd":3,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.624,"15min":0.625},"min":{"1min":0.40999999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.75800000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.433,"5min":0.376,"15min":0.376},"max":{"1min":0.877,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.68600000000000005}]},{"osd":4,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.65800000000000003,"15min":0.63300000000000001},"min":{"1min":0.47099999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":4.0599999999999996,"5min":4.0599999999999996,"15min":4.0599999999999996},"last":0.754},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69899999999999995,"15min":0.69799999999999995},"min":{"1min":0.42899999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.77400000000000002}]},{"osd":5,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60999999999999999,"15min":0.61199999999999999},"min":{"1min":0.33500000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.89000000000000001,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.624,"15min":0.63},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.84599999999999997,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.59299999999999997}]},{"osd":6,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.747,"15min":0.745},"min":{"1min":0.45800000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94599999999999995,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.745,"15min":0.75},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94299999999999995,"5min":1.173,"15min":1.173},"last":0.73299999999999998}]},{"osd":7,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.76800000000000002,"15min":0.76400000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.75700000000000001,"15min":0.753},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.80200000000000005}]},{"osd":9,"last update":"Thu Nov 16 16:37:55 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.73799999999999999,"15min":0.72999999999999998},"min":{"1min":0.54000000000000004,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.018,"5min":1.036,"15min":1.036},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.875,"5min":0.76900000000000002,"15min":0.751},"min":{"1min":0.58699999999999997,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.874}]},{"osd":10,"last update":"Thu Nov 16 16:37:13 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":3.5720000000000001,"5min":3.5720000000000001,"15min":3.5720000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":3.5920000000000001,"5min":3.5920000000000001,"15min":3.5920000000000001},"last":0.89900000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]}]},{"osd":7,"up_from":52,"seq":223338299430,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6392,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738760,"statfs":{"total":95995035648,"available":95988490240,"internally_reserved":0,"allocated":966656,"data_stored":1021446,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:36:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55800000000000005}]},{"osd":1,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.41799999999999998,"15min":0.41099999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.434,"15min":0.42599999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.48299999999999998}]},{"osd":2,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.35799999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.80800000000000005}]},{"osd":3,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57499999999999996,"15min":0.57099999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.625,"15min":0.621},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.41699999999999998}]},{"osd":4,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.52400000000000002,"15min":0.51400000000000001},"min":{"1min":0.34799999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.498,"15min":0.48199999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.379}]},{"osd":5,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.65200000000000002,"15min":0.64700000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91300000000000003,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.63800000000000001,"15min":0.63200000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.90100000000000002}]},{"osd":6,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.70299999999999996,"15min":0.69499999999999995},"min":{"1min":0.52500000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.75,"5min":0.68600000000000005,"15min":0.67600000000000005},"min":{"1min":0.47299999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.51100000000000001}]},{"osd":8,"last update":"Thu Nov 16 16:37:25 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.748,"15min":0.747},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0580000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.752,"5min":0.76100000000000001,"15min":0.76300000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.72499999999999998}]},{"osd":9,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.80300000000000005,"15min":0.78200000000000003},"min":{"1min":0.57999999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":4.7370000000000001,"5min":4.7370000000000001,"15min":4.7370000000000001},"last":0.75},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.84399999999999997,"15min":0.82199999999999995},"min":{"1min":0.55600000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":4.7809999999999997,"5min":4.7809999999999997,"15min":4.7809999999999997},"last":0.76700000000000002}]},{"osd":10,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.84699999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]}]},{"osd":6,"up_from":47,"seq":201863462954,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6204,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":93738948,"statfs":{"total":95995035648,"available":95988682752,"internally_reserved":0,"allocated":905216,"data_stored":429082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.439,"15min":0.41999999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":2.3140000000000001,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.47399999999999998,"15min":0.44600000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":3.472,"5min":3.472,"15min":3.472},"last":0.56499999999999995}]},{"osd":1,"last update":"Thu Nov 16 16:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.59699999999999998,"15min":0.58299999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":3.3290000000000002,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.64600000000000002,"15min":0.59999999999999998},"min":{"1min":0.38200000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":4.4669999999999996,"5min":4.4669999999999996,"15min":4.4669999999999996},"last":0.65000000000000002}]},{"osd":2,"last update":"Thu Nov 16 16:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.65400000000000003,"15min":0.63800000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":3.5369999999999999,"5min":3.5369999999999999,"15min":3.5369999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.66200000000000003,"15min":0.63400000000000001},"min":{"1min":0.46400000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.4049999999999998,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.48999999999999999}]},{"osd":3,"last update":"Thu Nov 16 16:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.52400000000000002,"15min":0.48999999999999999},"min":{"1min":0.30499999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":3.3700000000000001,"5min":3.3700000000000001,"15min":3.3700000000000001},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.55600000000000005,"15min":0.54100000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.35099999999999998}]},{"osd":4,"last update":"Thu Nov 16 16:37:50 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.67600000000000005,"15min":0.60999999999999999},"min":{"1min":0.45300000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":4.4900000000000002,"5min":4.4900000000000002,"15min":4.4900000000000002},"last":0.58599999999999997},{"interface":"front","average":{"1min":1.026,"5min":0.69899999999999995,"15min":0.63100000000000001},"min":{"1min":0.45200000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":4.4509999999999996,"5min":4.4509999999999996,"15min":4.4509999999999996},"last":0.63100000000000001}]},{"osd":5,"last update":"Thu Nov 16 16:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.68899999999999995,"15min":0.68100000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":2.4209999999999998,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":0.443},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.72999999999999998,"15min":0.70999999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":3.5019999999999998,"5min":3.5019999999999998,"15min":3.5019999999999998},"last":0.53600000000000003}]},{"osd":7,"last update":"Thu Nov 16 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.70499999999999996,"15min":0.70299999999999996},"min":{"1min":0.48999999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.878,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":1.1200000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.71499999999999997,"15min":0.70899999999999996},"min":{"1min":0.55300000000000005,"5min":0.436,"15min":0.436},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":1.111}]},{"osd":8,"last update":"Thu Nov 16 16:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.76100000000000001,"15min":0.77200000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.91200000000000003,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.77000000000000002,"15min":0.78000000000000003},"min":{"1min":0.54000000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.016,"5min":1.091,"15min":1.091},"last":0.503}]},{"osd":9,"last update":"Thu Nov 16 16:37:50 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":0.81799999999999995,"15min":0.78200000000000003},"min":{"1min":0.50800000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":6.1539999999999999,"5min":6.1539999999999999,"15min":6.1539999999999999},"last":0.40500000000000003},{"interface":"front","average":{"1min":1.1910000000000001,"5min":0.83999999999999997,"15min":0.78200000000000003},"min":{"1min":0.55100000000000005,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":8.2460000000000004,"5min":8.2460000000000004,"15min":8.2460000000000004},"last":0.42499999999999999}]},{"osd":10,"last update":"Thu Nov 16 16:37:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.68500000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7288,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":6336,"kb_avail":93737864,"statfs":{"total":95995035648,"available":95987572736,"internally_reserved":0,"allocated":966656,"data_stored":487396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6488064},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.48499999999999999,"15min":0.45800000000000002},"min":{"1min":0.33100000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.442},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.51200000000000001,"15min":0.48499999999999999},"min":{"1min":0.32000000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.46600000000000003}]},{"osd":2,"last update":"Thu Nov 16 16:37:19 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.54300000000000004,"15min":0.498},"min":{"1min":0.36699999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.84799999999999998,"5min":0.876,"15min":0.876},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.54000000000000004,"15min":0.503},"min":{"1min":0.40500000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.91000000000000003,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.84699999999999998}]},{"osd":3,"last update":"Thu Nov 16 16:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57699999999999996,"15min":0.55500000000000005},"min":{"1min":0.372,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85599999999999998,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59899999999999998,"15min":0.57499999999999996},"min":{"1min":0.41399999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.58499999999999996}]},{"osd":4,"last update":"Thu Nov 16 16:37:03 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.441,"15min":0.42299999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.52200000000000002,"15min":0.47599999999999998},"min":{"1min":0.34599999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.82899999999999996}]},{"osd":5,"last update":"Thu Nov 16 16:37:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.63800000000000001,"15min":0.61099999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.877},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.621,"15min":0.59699999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97399999999999998,"5min":1.004,"15min":1.004},"last":0.70599999999999996}]},{"osd":6,"last update":"Thu Nov 16 16:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.68400000000000005,"15min":0.66800000000000004},"min":{"1min":0.44700000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.95099999999999996,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68400000000000005,"15min":0.66400000000000003},"min":{"1min":0.45500000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.95299999999999996,"5min":1,"15min":1},"last":0.69399999999999995}]},{"osd":7,"last update":"Thu Nov 16 16:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.64500000000000002,"15min":0.63200000000000001},"min":{"1min":0.35199999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.752},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.69299999999999995,"15min":0.67700000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.93000000000000005}]},{"osd":8,"last update":"Thu Nov 16 16:37:27 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.75900000000000001,"15min":0.755},"min":{"1min":0.54800000000000004,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.74099999999999999,"15min":0.73699999999999999},"min":{"1min":0.47399999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.89500000000000002}]},{"osd":9,"last update":"Thu Nov 16 16:37:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.77900000000000003,"15min":0.79900000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.87,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.84599999999999997,"15min":0.86399999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.97299999999999998,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.97299999999999998}]},{"osd":10,"last update":"Thu Nov 16 16:37:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.94399999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]}]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7184,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":6272,"kb_avail":93737968,"statfs":{"total":95995035648,"available":95987679232,"internally_reserved":0,"allocated":925696,"data_stored":430347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6422528},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Nov 16 16:37:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.503,"15min":0.46700000000000003},"min":{"1min":0.32500000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.499,"15min":0.46100000000000002},"min":{"1min":0.36099999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78800000000000003,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.64200000000000002}]},{"osd":2,"last update":"Thu Nov 16 16:37:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.55600000000000005,"15min":0.51400000000000001},"min":{"1min":0.41999999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.80600000000000005,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.498},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56299999999999994,"15min":0.53100000000000003},"min":{"1min":0.372,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.42599999999999999}]},{"osd":3,"last update":"Thu Nov 16 16:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.46500000000000002,"15min":0.45600000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.45700000000000002,"15min":0.44700000000000001},"min":{"1min":0.33700000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.35699999999999998}]},{"osd":4,"last update":"Thu Nov 16 16:37:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.57899999999999996,"15min":0.55100000000000005},"min":{"1min":0.38900000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.58199999999999996,"15min":0.55000000000000004},"min":{"1min":0.40999999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.57699999999999996}]},{"osd":5,"last update":"Thu Nov 16 16:37:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.56200000000000006,"15min":0.55100000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.66100000000000003,"5min":1.012,"15min":1.012},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64600000000000002,"15min":0.621},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.82699999999999996,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.38800000000000001}]},{"osd":6,"last update":"Thu Nov 16 16:37:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.64800000000000002,"15min":0.59699999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":6.5190000000000001,"5min":6.5190000000000001,"15min":6.5190000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.63100000000000001,"15min":0.57499999999999996},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":9.141,"5min":9.141,"15min":9.141},"last":0.51800000000000002}]},{"osd":7,"last update":"Thu Nov 16 16:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.68400000000000005,"15min":0.67600000000000005},"min":{"1min":0.495,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.74199999999999999,"15min":0.73199999999999998},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.53400000000000003}]},{"osd":8,"last update":"Thu Nov 16 16:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.78800000000000003,"15min":0.80900000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.91900000000000004,"5min":1.163,"15min":1.163},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.83499999999999996,"15min":0.85299999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.90800000000000003,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.60699999999999998}]},{"osd":9,"last update":"Thu Nov 16 16:36:51 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.58799999999999997}]},{"osd":10,"last update":"Thu Nov 16 16:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.67900000000000005}]},{"osd":11,"last update":"Thu Nov 16 16:37:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":9.1039999999999992,"5min":9.1039999999999992,"15min":9.1039999999999992},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":3.7530000000000001,"5min":3.7530000000000001,"15min":3.7530000000000001},"last":0.61899999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7376,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":93737776,"statfs":{"total":95995035648,"available":95987482624,"internally_reserved":0,"allocated":991232,"data_stored":489088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.29999999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.497,"15min":0.48799999999999999},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.56399999999999995}]},{"osd":1,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52600000000000002,"15min":0.51400000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.40500000000000003,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.89900000000000002,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.69599999999999995}]},{"osd":3,"last update":"Thu Nov 16 16:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.66300000000000003,"15min":0.60699999999999998},"min":{"1min":0.40699999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":6.9740000000000002,"5min":6.9740000000000002,"15min":6.9740000000000002},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.57499999999999996,"15min":0.55700000000000005},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.46600000000000003}]},{"osd":4,"last update":"Thu Nov 16 16:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.55300000000000005,"15min":0.53000000000000003},"min":{"1min":0.376,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59099999999999997,"15min":0.58199999999999996},"min":{"1min":0.40000000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83799999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.53300000000000003}]},{"osd":5,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.49399999999999999,"15min":0.48399999999999999},"min":{"1min":0.315,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.47199999999999998,"15min":0.45900000000000002},"min":{"1min":0.32200000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.62}]},{"osd":6,"last update":"Thu Nov 16 16:37:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.68000000000000005,"15min":0.65400000000000003},"min":{"1min":0.40899999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.3969999999999998,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.64400000000000002,"15min":0.61299999999999999},"min":{"1min":0.39400000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":2.3460000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.64700000000000002}]},{"osd":7,"last update":"Thu Nov 16 16:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.64300000000000002,"15min":0.64400000000000002},"min":{"1min":0.41599999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70199999999999996,"15min":0.70399999999999996},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.92700000000000005,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.57699999999999996}]},{"osd":8,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.57499999999999996,"15min":0.56899999999999995},"min":{"1min":0.38500000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.85199999999999998,"5min":1.008,"15min":1.008},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.628,"15min":0.625},"min":{"1min":0.38400000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.753}]},{"osd":9,"last update":"Thu Nov 16 16:37:54 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.86799999999999999,"15min":0.86699999999999999},"min":{"1min":0.47499999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.6080000000000001,"5min":4.524,"15min":4.524},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.751,"15min":0.73299999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.6200000000000001,"5min":2.6200000000000001,"15min":2.6200000000000001},"last":0.72399999999999998}]},{"osd":10,"last update":"Thu Nov 16 16:37:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.78700000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]}]},{"osd":3,"up_from":26,"seq":111669149750,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6536,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738616,"statfs":{"total":95995035648,"available":95988342784,"internally_reserved":0,"allocated":917504,"data_stored":429068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36499999999999999,"15min":0.34899999999999998},"min":{"1min":0.26500000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.52400000000000002,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.38,"15min":0.35799999999999998},"min":{"1min":0.27800000000000002,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.55100000000000005,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.311}]},{"osd":1,"last update":"Thu Nov 16 16:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.55400000000000005,"15min":0.54900000000000004},"min":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55900000000000005,"15min":0.55700000000000005},"min":{"1min":0.36099999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.83399999999999996,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.55100000000000005}]},{"osd":2,"last update":"Thu Nov 16 16:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57899999999999996,"15min":0.55700000000000005},"min":{"1min":0.42599999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.83299999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.439},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56699999999999995,"15min":0.54700000000000004},"min":{"1min":0.40600000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.69699999999999995,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.377}]},{"osd":4,"last update":"Thu Nov 16 16:37:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55700000000000005,"15min":0.54700000000000004},"min":{"1min":0.376,"5min":0.37,"15min":0.37},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.45500000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.53300000000000003}]},{"osd":5,"last update":"Thu Nov 16 16:37:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.59799999999999998,"15min":0.57999999999999996},"min":{"1min":0.438,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60499999999999998,"15min":0.59099999999999997},"min":{"1min":0.38600000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.87,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.48099999999999998}]},{"osd":6,"last update":"Thu Nov 16 16:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51800000000000002,"15min":0.498},"min":{"1min":0.35599999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.72399999999999998,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53200000000000003,"15min":0.51400000000000001},"min":{"1min":0.377,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.75800000000000001,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.35299999999999998}]},{"osd":7,"last update":"Thu Nov 16 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.65600000000000003,"15min":0.65000000000000002},"min":{"1min":0.44,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.69299999999999995,"15min":0.68700000000000006},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.68300000000000005}]},{"osd":8,"last update":"Thu Nov 16 16:36:27 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.63200000000000001}]},{"osd":9,"last update":"Thu Nov 16 16:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.73799999999999999,"15min":0.76300000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80500000000000005,"5min":1.024,"15min":1.024},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.68999999999999995,"15min":0.70899999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90300000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.46700000000000003}]},{"osd":10,"last update":"Thu Nov 16 16:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.66100000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]}]},{"osd":4,"up_from":34,"seq":146028888115,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6336,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738816,"statfs":{"total":95995035648,"available":95988547584,"internally_reserved":0,"allocated":909312,"data_stored":429465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.52400000000000002,"15min":0.502},"min":{"1min":0.38200000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.52900000000000003,"15min":0.51500000000000001},"min":{"1min":0.40899999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.378}]},{"osd":1,"last update":"Thu Nov 16 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.39400000000000002,"15min":0.38700000000000001},"min":{"1min":0.27900000000000003,"5min":0.253,"15min":0.253},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.41199999999999998,"15min":0.40300000000000002},"min":{"1min":0.29699999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.442}]},{"osd":2,"last update":"Thu Nov 16 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.442,"5min":0.371,"15min":0.371},"max":{"1min":0.79200000000000004,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.60499999999999998,"15min":0.58699999999999997},"min":{"1min":0.44600000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.90200000000000002,"5min":1.056,"15min":1.056},"last":0.58499999999999996}]},{"osd":3,"last update":"Thu Nov 16 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.56299999999999994,"15min":0.54000000000000004},"min":{"1min":0.39000000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.56100000000000005,"15min":0.54900000000000004},"min":{"1min":0.40500000000000003,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.55000000000000004}]},{"osd":5,"last update":"Thu Nov 16 16:37:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.623,"15min":0.627},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.92100000000000004,"5min":1.075,"15min":1.075},"last":0.624},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.63800000000000001,"15min":0.626},"min":{"1min":0.52200000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.94499999999999995,"5min":1.093,"15min":1.093},"last":0.64800000000000002}]},{"osd":6,"last update":"Thu Nov 16 16:37:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.63,"15min":0.60199999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84799999999999998,"5min":1.004,"15min":1.004},"last":0.749},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.60899999999999999,"15min":0.59499999999999997},"min":{"1min":0.42399999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.66900000000000004}]},{"osd":7,"last update":"Thu Nov 16 16:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.57199999999999995,"15min":0.54800000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.63200000000000001,"15min":0.60799999999999998},"min":{"1min":0.41199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.86299999999999999}]},{"osd":8,"last update":"Thu Nov 16 16:37:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.73999999999999999,"15min":0.72399999999999998},"min":{"1min":0.58999999999999997,"5min":0.495,"15min":0.495},"max":{"1min":1.0189999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.747,"15min":0.73099999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.80100000000000005}]},{"osd":9,"last update":"Thu Nov 16 16:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.79200000000000004,"15min":0.78900000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.94099999999999995,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.72099999999999997,"15min":0.71799999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94199999999999995,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.71499999999999997}]},{"osd":10,"last update":"Thu Nov 16 16:37:15 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.83599999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]}]},{"osd":5,"up_from":39,"seq":167503724591,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6532,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738620,"statfs":{"total":95995035648,"available":95988346880,"internally_reserved":0,"allocated":978944,"data_stored":1021178,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.499,"15min":0.49399999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.64600000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52800000000000002,"15min":0.52400000000000002},"min":{"1min":0.312,"5min":0.311,"15min":0.311},"max":{"1min":0.75700000000000001,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.65200000000000002}]},{"osd":1,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.372,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.86399999999999999,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58799999999999997,"15min":0.58499999999999996},"min":{"1min":0.43099999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.94699999999999995,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.78500000000000003}]},{"osd":2,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.42799999999999999,"15min":0.42399999999999999},"min":{"1min":0.29599999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.67300000000000004,"5min":1.417,"15min":1.417},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.443,"15min":0.433},"min":{"1min":0.32300000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.69899999999999995,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.56999999999999995}]},{"osd":3,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55700000000000005,"15min":0.54700000000000004},"min":{"1min":0.36199999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.81100000000000005,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.75600000000000001}]},{"osd":4,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.67900000000000005,"15min":0.629},"min":{"1min":0.40100000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":5.5789999999999997,"5min":5.5789999999999997,"15min":5.5789999999999997},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59699999999999998,"15min":0.59099999999999997},"min":{"1min":0.40400000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.80300000000000005}]},{"osd":6,"last update":"Thu Nov 16 16:37:47 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.67800000000000005,"15min":0.63600000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":3.274,"5min":3.274,"15min":3.274},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.68100000000000005,"15min":0.64600000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":3.472,"5min":3.472,"15min":3.472},"last":0.69099999999999995}]},{"osd":7,"last update":"Thu Nov 16 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.70999999999999996,"15min":0.71199999999999997},"min":{"1min":0.503,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.83799999999999997,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.72499999999999998,"15min":0.72799999999999998},"min":{"1min":0.56599999999999995,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.83699999999999997,"5min":1.544,"15min":1.544},"last":0.89400000000000002}]},{"osd":8,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.64600000000000002,"15min":0.64500000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72199999999999998,"15min":0.72399999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":1.03}]},{"osd":9,"last update":"Thu Nov 16 16:37:52 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":0.90100000000000002,"15min":0.82699999999999996},"min":{"1min":0.58699999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":5.5679999999999996,"5min":5.5679999999999996,"15min":5.5679999999999996},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.099,"5min":0.81599999999999995,"15min":0.76900000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":5.3230000000000004,"5min":5.3230000000000004,"15min":5.3230000000000004},"last":0.93999999999999995}]},{"osd":10,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":1.0549999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.98099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33928,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":58527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-16T16:38:01.728 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph pg dump --format=json 2023-11-16T16:38:02.385 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:38:02.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:02 smithi001 ceph-mon[98800]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:38:02.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:02 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2023-11-16T16:38:02.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:02 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2023-11-16T16:38:02.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:02 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2023-11-16T16:38:02.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:02 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2023-11-16T16:38:02.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:02 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2023-11-16T16:38:02.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:02 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2023-11-16T16:38:02.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:02 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2023-11-16T16:38:02.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:02 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2023-11-16T16:38:02.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:02 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2023-11-16T16:38:02.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:02 smithi001 ceph-mon[98800]: pgmap v284: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2023-11-16T16:38:02.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:02 smithi026 ceph-mon[100112]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:38:02.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:02 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2023-11-16T16:38:02.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:02 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2023-11-16T16:38:02.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:02 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2023-11-16T16:38:02.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:02 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2023-11-16T16:38:02.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:02 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2023-11-16T16:38:02.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:02 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2023-11-16T16:38:02.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:02 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2023-11-16T16:38:02.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:02 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2023-11-16T16:38:02.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:02 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2023-11-16T16:38:02.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:02 smithi026 ceph-mon[100112]: pgmap v284: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2023-11-16T16:38:02.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:02 smithi053 ceph-mon[100638]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:38:02.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:02 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2023-11-16T16:38:02.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:02 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2023-11-16T16:38:02.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:02 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2023-11-16T16:38:02.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:02 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2023-11-16T16:38:02.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:02 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2023-11-16T16:38:02.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:02 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2023-11-16T16:38:02.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:02 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2023-11-16T16:38:02.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:02 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2023-11-16T16:38:02.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:02 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2023-11-16T16:38:02.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:02 smithi053 ceph-mon[100638]: pgmap v284: 97 pgs: 97 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2023-11-16T16:38:03.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:03 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2023-11-16T16:38:03.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:03 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2023-11-16T16:38:03.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:03 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2023-11-16T16:38:03.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:03 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2023-11-16T16:38:03.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:03 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2023-11-16T16:38:03.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:03 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2023-11-16T16:38:03.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:03 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2023-11-16T16:38:03.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:03 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2023-11-16T16:38:03.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:03 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2023-11-16T16:38:03.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:03 smithi001 ceph-mon[98800]: osdmap e85: 12 total, 12 up, 12 in 2023-11-16T16:38:03.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:03 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2023-11-16T16:38:03.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:03 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2023-11-16T16:38:03.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:03 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2023-11-16T16:38:03.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:03 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2023-11-16T16:38:03.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:03 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2023-11-16T16:38:03.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:03 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2023-11-16T16:38:03.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:03 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2023-11-16T16:38:03.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:03 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2023-11-16T16:38:03.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:03 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2023-11-16T16:38:03.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:03 smithi026 ceph-mon[100112]: osdmap e85: 12 total, 12 up, 12 in 2023-11-16T16:38:03.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:03 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2023-11-16T16:38:03.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:03 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2023-11-16T16:38:03.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:03 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2023-11-16T16:38:03.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:03 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2023-11-16T16:38:03.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:03 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2023-11-16T16:38:03.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:03 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2023-11-16T16:38:03.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:03 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2023-11-16T16:38:03.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:03 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2023-11-16T16:38:03.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:03 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2023-11-16T16:38:03.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:03 smithi053 ceph-mon[100638]: osdmap e85: 12 total, 12 up, 12 in 2023-11-16T16:38:03.709 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.707+0000 7f2f10ebb700 1 -- 172.21.15.1:0/2409760221 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f0c101540 msgr2=0x7f2f0c101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:03.709 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.707+0000 7f2f10ebb700 1 --2- 172.21.15.1:0/2409760221 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f0c101540 0x7f2f0c101910 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2efc009990 tx=0x7f2efc009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:03.710 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.707+0000 7f2f10ebb700 1 -- 172.21.15.1:0/2409760221 shutdown_connections 2023-11-16T16:38:03.710 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.707+0000 7f2f10ebb700 1 --2- 172.21.15.1:0/2409760221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f0c10a890 0x7f2f0c10ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:03.710 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.707+0000 7f2f10ebb700 1 --2- 172.21.15.1:0/2409760221 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f0c101540 0x7f2f0c101910 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:03.710 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.707+0000 7f2f10ebb700 1 --2- 172.21.15.1:0/2409760221 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f0c101e50 0x7f2f0c10a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:03.710 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.707+0000 7f2f10ebb700 1 -- 172.21.15.1:0/2409760221 >> 172.21.15.1:0/2409760221 conn(0x7f2f0c0770a0 msgr2=0x7f2f0c0774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:03.710 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.707+0000 7f2f10ebb700 1 -- 172.21.15.1:0/2409760221 shutdown_connections 2023-11-16T16:38:03.711 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.707+0000 7f2f10ebb700 1 -- 172.21.15.1:0/2409760221 wait complete. 2023-11-16T16:38:03.713 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.708+0000 7f2f10ebb700 1 Processor -- start 2023-11-16T16:38:03.713 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.708+0000 7f2f10ebb700 1 -- start start 2023-11-16T16:38:03.713 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.708+0000 7f2f10ebb700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f0c101540 0x7f2f0c1964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:03.713 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.708+0000 7f2f10ebb700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f0c101e50 0x7f2f0c196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:03.713 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.708+0000 7f2f10ebb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f0c10a890 0x7f2f0c19ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:03.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.708+0000 7f2f10ebb700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0c19b4a0 con 0x7f2f0c101540 2023-11-16T16:38:03.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.708+0000 7f2f10ebb700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0c19b610 con 0x7f2f0c10a890 2023-11-16T16:38:03.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.708+0000 7f2f10ebb700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0c19b7d0 con 0x7f2f0c101e50 2023-11-16T16:38:03.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.708+0000 7f2f0a59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f0c101540 0x7f2f0c1964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:03.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.708+0000 7f2f0a59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f0c101540 0x7f2f0c1964e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42076/0 (socket says 172.21.15.1:42076) 2023-11-16T16:38:03.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.708+0000 7f2f0a59c700 1 -- 172.21.15.1:0/1434191684 learned_addr learned my addr 172.21.15.1:0/1434191684 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:03.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.709+0000 7f2f09d9b700 1 --2- 172.21.15.1:0/1434191684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f0c101e50 0x7f2f0c196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:03.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.709+0000 7f2f0ad9d700 1 --2- 172.21.15.1:0/1434191684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f0c10a890 0x7f2f0c19ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:03.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.709+0000 7f2f0a59c700 1 -- 172.21.15.1:0/1434191684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f0c101e50 msgr2=0x7f2f0c196a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:03.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.709+0000 7f2f0a59c700 1 --2- 172.21.15.1:0/1434191684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f0c101e50 0x7f2f0c196a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:03.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.709+0000 7f2f0a59c700 1 -- 172.21.15.1:0/1434191684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f0c10a890 msgr2=0x7f2f0c19ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:03.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.709+0000 7f2f0a59c700 1 --2- 172.21.15.1:0/1434191684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f0c10a890 0x7f2f0c19ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:03.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.709+0000 7f2f0a59c700 1 -- 172.21.15.1:0/1434191684 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2efc0095f0 con 0x7f2f0c101540 2023-11-16T16:38:03.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.709+0000 7f2f09d9b700 1 --2- 172.21.15.1:0/1434191684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f0c101e50 0x7f2f0c196a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:38:03.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.709+0000 7f2f0a59c700 1 --2- 172.21.15.1:0/1434191684 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f0c101540 0x7f2f0c1964e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2efc000c00 tx=0x7f2efc00f830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:03.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.709+0000 7f2efb7fe700 1 -- 172.21.15.1:0/1434191684 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2efc01d070 con 0x7f2f0c101540 2023-11-16T16:38:03.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.709+0000 7f2f10ebb700 1 -- 172.21.15.1:0/1434191684 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f0c110400 con 0x7f2f0c101540 2023-11-16T16:38:03.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.709+0000 7f2efb7fe700 1 -- 172.21.15.1:0/1434191684 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2efc00fce0 con 0x7f2f0c101540 2023-11-16T16:38:03.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.709+0000 7f2efb7fe700 1 -- 172.21.15.1:0/1434191684 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2efc017770 con 0x7f2f0c101540 2023-11-16T16:38:03.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.709+0000 7f2f10ebb700 1 -- 172.21.15.1:0/1434191684 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f0c110270 con 0x7f2f0c101540 2023-11-16T16:38:03.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.710+0000 7f2f10ebb700 1 -- 172.21.15.1:0/1434191684 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f0c110590 con 0x7f2f0c101540 2023-11-16T16:38:03.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.711+0000 7f2efb7fe700 1 -- 172.21.15.1:0/1434191684 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f2efc0178d0 con 0x7f2f0c101540 2023-11-16T16:38:03.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.712+0000 7f2efb7fe700 1 --2- 172.21.15.1:0/1434191684 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f2ef40a11e0 0x7f2ef40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:03.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.712+0000 7f2f09d9b700 1 --2- 172.21.15.1:0/1434191684 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f2ef40a11e0 0x7f2ef40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:03.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.712+0000 7f2efb7fe700 1 -- 172.21.15.1:0/1434191684 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2efc0cdfb0 con 0x7f2f0c101540 2023-11-16T16:38:03.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.712+0000 7f2f09d9b700 1 --2- 172.21.15.1:0/1434191684 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f2ef40a11e0 0x7f2ef40a3690 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c197b00 tx=0x7f2f00008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:03.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.716+0000 7f2efb7fe700 1 -- 172.21.15.1:0/1434191684 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2efc05c730 con 0x7f2f0c101540 2023-11-16T16:38:03.909 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.907+0000 7f2f10ebb700 1 -- 172.21.15.1:0/1434191684 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2f0c197950 con 0x7f2ef40a11e0 2023-11-16T16:38:03.921 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.919+0000 7f2efb7fe700 1 -- 172.21.15.1:0/1434191684 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340079 (secure 0 0 0) 0x7f2f0c197950 con 0x7f2ef40a11e0 2023-11-16T16:38:03.926 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:38:03.930 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.922+0000 7f2f10ebb700 1 -- 172.21.15.1:0/1434191684 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f2ef40a11e0 msgr2=0x7f2ef40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:03.931 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.922+0000 7f2f10ebb700 1 --2- 172.21.15.1:0/1434191684 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f2ef40a11e0 0x7f2ef40a3690 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c197b00 tx=0x7f2f00008040 comp rx=0 tx=0).stop 2023-11-16T16:38:03.931 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.922+0000 7f2f10ebb700 1 -- 172.21.15.1:0/1434191684 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f0c101540 msgr2=0x7f2f0c1964e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:03.931 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.922+0000 7f2f10ebb700 1 --2- 172.21.15.1:0/1434191684 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f0c101540 0x7f2f0c1964e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2efc000c00 tx=0x7f2efc00f830 comp rx=0 tx=0).stop 2023-11-16T16:38:03.931 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.922+0000 7f2f10ebb700 1 -- 172.21.15.1:0/1434191684 shutdown_connections 2023-11-16T16:38:03.931 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.922+0000 7f2f10ebb700 1 --2- 172.21.15.1:0/1434191684 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f2ef40a11e0 0x7f2ef40a3690 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:03.931 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.922+0000 7f2f10ebb700 1 --2- 172.21.15.1:0/1434191684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f0c10a890 0x7f2f0c19ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:03.931 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.922+0000 7f2f10ebb700 1 --2- 172.21.15.1:0/1434191684 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f0c101540 0x7f2f0c1964e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:03.931 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.922+0000 7f2f10ebb700 1 --2- 172.21.15.1:0/1434191684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2f0c101e50 0x7f2f0c196a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:03.931 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.922+0000 7f2f10ebb700 1 -- 172.21.15.1:0/1434191684 >> 172.21.15.1:0/1434191684 conn(0x7f2f0c0770a0 msgr2=0x7f2f0c0fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:03.932 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.922+0000 7f2f10ebb700 1 -- 172.21.15.1:0/1434191684 shutdown_connections 2023-11-16T16:38:03.932 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:03.922+0000 7f2f10ebb700 1 -- 172.21.15.1:0/1434191684 wait complete. 2023-11-16T16:38:03.932 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2023-11-16T16:38:04.409 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:04 smithi001 ceph-mon[98800]: osdmap e86: 12 total, 12 up, 12 in 2023-11-16T16:38:04.409 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:04 smithi001 ceph-mon[98800]: pgmap v287: 97 pgs: 2 peering, 95 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2023-11-16T16:38:04.443 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":287,"stamp":"2023-11-16T16:38:03.483408+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653045,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":10,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":410,"num_read_kb":321,"num_write":333,"num_write_kb":3043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":78568,"kb_used_data":11080,"kb_used_omap":0,"kb_used_meta":67392,"kb_avail":1124863256,"statfs":{"total":1151940427776,"available":1151859974144,"internally_reserved":0,"allocated":11345920,"data_stored":7104901,"data_compressed":106695,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":69009408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":10,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":14,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001898"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-16T16:38:02.188094+0000","last_change":"2023-11-16T16:37:13.941551+0000","last_active":"2023-11-16T16:38:02.188094+0000","last_peered":"2023-11-16T16:38:02.188094+0000","last_clean":"2023-11-16T16:38:02.188094+0000","last_became_active":"2023-11-16T16:37:13.941190+0000","last_became_peered":"2023-11-16T16:37:13.941190+0000","last_unstale":"2023-11-16T16:38:02.188094+0000","last_undegraded":"2023-11-16T16:38:02.188094+0000","last_fullsized":"2023-11-16T16:38:02.188094+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T16:38:21.389028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158170+0000","last_change":"2023-11-16T16:37:13.942300+0000","last_active":"2023-11-16T16:37:15.158170+0000","last_peered":"2023-11-16T16:37:15.158170+0000","last_clean":"2023-11-16T16:37:15.158170+0000","last_became_active":"2023-11-16T16:37:13.941966+0000","last_became_peered":"2023-11-16T16:37:13.941966+0000","last_unstale":"2023-11-16T16:37:15.158170+0000","last_undegraded":"2023-11-16T16:37:15.158170+0000","last_fullsized":"2023-11-16T16:37:15.158170+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:12:01.659205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158388+0000","last_change":"2023-11-16T16:37:13.943461+0000","last_active":"2023-11-16T16:37:15.158388+0000","last_peered":"2023-11-16T16:37:15.158388+0000","last_clean":"2023-11-16T16:37:15.158388+0000","last_became_active":"2023-11-16T16:37:13.943165+0000","last_became_peered":"2023-11-16T16:37:13.943165+0000","last_unstale":"2023-11-16T16:37:15.158388+0000","last_undegraded":"2023-11-16T16:37:15.158388+0000","last_fullsized":"2023-11-16T16:37:15.158388+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:09:30.792766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.122028+0000","last_change":"2023-11-16T16:37:13.940205+0000","last_active":"2023-11-16T16:37:15.122028+0000","last_peered":"2023-11-16T16:37:15.122028+0000","last_clean":"2023-11-16T16:37:15.122028+0000","last_became_active":"2023-11-16T16:37:13.939819+0000","last_became_peered":"2023-11-16T16:37:13.939819+0000","last_unstale":"2023-11-16T16:37:15.122028+0000","last_undegraded":"2023-11-16T16:37:15.122028+0000","last_fullsized":"2023-11-16T16:37:15.122028+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:40:08.471596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.923975+0000","last_change":"2023-11-16T16:37:13.939819+0000","last_active":"2023-11-16T16:37:14.923975+0000","last_peered":"2023-11-16T16:37:14.923975+0000","last_clean":"2023-11-16T16:37:14.923975+0000","last_became_active":"2023-11-16T16:37:13.939367+0000","last_became_peered":"2023-11-16T16:37:13.939367+0000","last_unstale":"2023-11-16T16:37:14.923975+0000","last_undegraded":"2023-11-16T16:37:14.923975+0000","last_fullsized":"2023-11-16T16:37:14.923975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:27:49.657269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.924379+0000","last_change":"2023-11-16T16:37:13.946613+0000","last_active":"2023-11-16T16:37:14.924379+0000","last_peered":"2023-11-16T16:37:14.924379+0000","last_clean":"2023-11-16T16:37:14.924379+0000","last_became_active":"2023-11-16T16:37:13.946281+0000","last_became_peered":"2023-11-16T16:37:13.946281+0000","last_unstale":"2023-11-16T16:37:14.924379+0000","last_undegraded":"2023-11-16T16:37:14.924379+0000","last_fullsized":"2023-11-16T16:37:14.924379+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:26:07.902412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.390884+0000","last_change":"2023-11-16T16:37:13.936899+0000","last_active":"2023-11-16T16:37:15.390884+0000","last_peered":"2023-11-16T16:37:15.390884+0000","last_clean":"2023-11-16T16:37:15.390884+0000","last_became_active":"2023-11-16T16:37:13.936543+0000","last_became_peered":"2023-11-16T16:37:13.936543+0000","last_unstale":"2023-11-16T16:37:15.390884+0000","last_undegraded":"2023-11-16T16:37:15.390884+0000","last_fullsized":"2023-11-16T16:37:15.390884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:55:14.512583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157287+0000","last_change":"2023-11-16T16:37:13.941463+0000","last_active":"2023-11-16T16:37:15.157287+0000","last_peered":"2023-11-16T16:37:15.157287+0000","last_clean":"2023-11-16T16:37:15.157287+0000","last_became_active":"2023-11-16T16:37:13.941125+0000","last_became_peered":"2023-11-16T16:37:13.941125+0000","last_unstale":"2023-11-16T16:37:15.157287+0000","last_undegraded":"2023-11-16T16:37:15.157287+0000","last_fullsized":"2023-11-16T16:37:15.157287+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:29:10.391228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157152+0000","last_change":"2023-11-16T16:37:13.934105+0000","last_active":"2023-11-16T16:37:15.157152+0000","last_peered":"2023-11-16T16:37:15.157152+0000","last_clean":"2023-11-16T16:37:15.157152+0000","last_became_active":"2023-11-16T16:37:13.933887+0000","last_became_peered":"2023-11-16T16:37:13.933887+0000","last_unstale":"2023-11-16T16:37:15.157152+0000","last_undegraded":"2023-11-16T16:37:15.157152+0000","last_fullsized":"2023-11-16T16:37:15.157152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:37:28.260833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.925511+0000","last_change":"2023-11-16T16:37:13.947266+0000","last_active":"2023-11-16T16:37:14.925511+0000","last_peered":"2023-11-16T16:37:14.925511+0000","last_clean":"2023-11-16T16:37:14.925511+0000","last_became_active":"2023-11-16T16:37:13.946919+0000","last_became_peered":"2023-11-16T16:37:13.946919+0000","last_unstale":"2023-11-16T16:37:14.925511+0000","last_undegraded":"2023-11-16T16:37:14.925511+0000","last_fullsized":"2023-11-16T16:37:14.925511+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:07:33.154784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.391182+0000","last_change":"2023-11-16T16:37:13.941597+0000","last_active":"2023-11-16T16:37:15.391182+0000","last_peered":"2023-11-16T16:37:15.391182+0000","last_clean":"2023-11-16T16:37:15.391182+0000","last_became_active":"2023-11-16T16:37:13.941253+0000","last_became_peered":"2023-11-16T16:37:13.941253+0000","last_unstale":"2023-11-16T16:37:15.391182+0000","last_undegraded":"2023-11-16T16:37:15.391182+0000","last_fullsized":"2023-11-16T16:37:15.391182+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:29:31.101716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.121996+0000","last_change":"2023-11-16T16:37:13.941673+0000","last_active":"2023-11-16T16:37:15.121996+0000","last_peered":"2023-11-16T16:37:15.121996+0000","last_clean":"2023-11-16T16:37:15.121996+0000","last_became_active":"2023-11-16T16:37:13.941345+0000","last_became_peered":"2023-11-16T16:37:13.941345+0000","last_unstale":"2023-11-16T16:37:15.121996+0000","last_undegraded":"2023-11-16T16:37:15.121996+0000","last_fullsized":"2023-11-16T16:37:15.121996+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:01:29.436252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157470+0000","last_change":"2023-11-16T16:37:13.947991+0000","last_active":"2023-11-16T16:37:15.157470+0000","last_peered":"2023-11-16T16:37:15.157470+0000","last_clean":"2023-11-16T16:37:15.157470+0000","last_became_active":"2023-11-16T16:37:13.947454+0000","last_became_peered":"2023-11-16T16:37:13.947454+0000","last_unstale":"2023-11-16T16:37:15.157470+0000","last_undegraded":"2023-11-16T16:37:15.157470+0000","last_fullsized":"2023-11-16T16:37:15.157470+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:35:41.400206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157374+0000","last_change":"2023-11-16T16:37:13.941502+0000","last_active":"2023-11-16T16:37:15.157374+0000","last_peered":"2023-11-16T16:37:15.157374+0000","last_clean":"2023-11-16T16:37:15.157374+0000","last_became_active":"2023-11-16T16:37:13.941191+0000","last_became_peered":"2023-11-16T16:37:13.941191+0000","last_unstale":"2023-11-16T16:37:15.157374+0000","last_undegraded":"2023-11-16T16:37:15.157374+0000","last_fullsized":"2023-11-16T16:37:15.157374+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:59:03.243696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.156825+0000","last_change":"2023-11-16T16:37:13.949198+0000","last_active":"2023-11-16T16:37:15.156825+0000","last_peered":"2023-11-16T16:37:15.156825+0000","last_clean":"2023-11-16T16:37:15.156825+0000","last_became_active":"2023-11-16T16:37:13.947147+0000","last_became_peered":"2023-11-16T16:37:13.947147+0000","last_unstale":"2023-11-16T16:37:15.156825+0000","last_undegraded":"2023-11-16T16:37:15.156825+0000","last_fullsized":"2023-11-16T16:37:15.156825+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:01:56.205870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.156530+0000","last_change":"2023-11-16T16:37:13.943915+0000","last_active":"2023-11-16T16:37:15.156530+0000","last_peered":"2023-11-16T16:37:15.156530+0000","last_clean":"2023-11-16T16:37:15.156530+0000","last_became_active":"2023-11-16T16:37:13.942995+0000","last_became_peered":"2023-11-16T16:37:13.942995+0000","last_unstale":"2023-11-16T16:37:15.156530+0000","last_undegraded":"2023-11-16T16:37:15.156530+0000","last_fullsized":"2023-11-16T16:37:15.156530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:35:55.774014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-16T16:38:02.188250+0000","last_change":"2023-11-16T16:37:13.941816+0000","last_active":"2023-11-16T16:38:02.188250+0000","last_peered":"2023-11-16T16:38:02.188250+0000","last_clean":"2023-11-16T16:38:02.188250+0000","last_became_active":"2023-11-16T16:37:13.941541+0000","last_became_peered":"2023-11-16T16:37:13.941541+0000","last_unstale":"2023-11-16T16:38:02.188250+0000","last_undegraded":"2023-11-16T16:38:02.188250+0000","last_fullsized":"2023-11-16T16:38:02.188250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:53:58.693324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158134+0000","last_change":"2023-11-16T16:37:13.940639+0000","last_active":"2023-11-16T16:37:15.158134+0000","last_peered":"2023-11-16T16:37:15.158134+0000","last_clean":"2023-11-16T16:37:15.158134+0000","last_became_active":"2023-11-16T16:37:13.940323+0000","last_became_peered":"2023-11-16T16:37:13.940323+0000","last_unstale":"2023-11-16T16:37:15.158134+0000","last_undegraded":"2023-11-16T16:37:15.158134+0000","last_fullsized":"2023-11-16T16:37:15.158134+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:54:11.977928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158148+0000","last_change":"2023-11-16T16:37:13.934802+0000","last_active":"2023-11-16T16:37:15.158148+0000","last_peered":"2023-11-16T16:37:15.158148+0000","last_clean":"2023-11-16T16:37:15.158148+0000","last_became_active":"2023-11-16T16:37:13.934496+0000","last_became_peered":"2023-11-16T16:37:13.934496+0000","last_unstale":"2023-11-16T16:37:15.158148+0000","last_undegraded":"2023-11-16T16:37:15.158148+0000","last_fullsized":"2023-11-16T16:37:15.158148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:50:10.271792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'9","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:18.297674+0000","last_change":"2023-11-16T16:37:13.499284+0000","last_active":"2023-11-16T16:37:18.297674+0000","last_peered":"2023-11-16T16:37:18.297674+0000","last_clean":"2023-11-16T16:37:18.297674+0000","last_became_active":"2023-11-16T16:37:13.498931+0000","last_became_peered":"2023-11-16T16:37:13.498931+0000","last_unstale":"2023-11-16T16:37:18.297674+0000","last_undegraded":"2023-11-16T16:37:18.297674+0000","last_fullsized":"2023-11-16T16:37:18.297674+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:52:32.314115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157923+0000","last_change":"2023-11-16T16:37:13.939631+0000","last_active":"2023-11-16T16:37:15.157923+0000","last_peered":"2023-11-16T16:37:15.157923+0000","last_clean":"2023-11-16T16:37:15.157923+0000","last_became_active":"2023-11-16T16:37:13.939318+0000","last_became_peered":"2023-11-16T16:37:13.939318+0000","last_unstale":"2023-11-16T16:37:15.157923+0000","last_undegraded":"2023-11-16T16:37:15.157923+0000","last_fullsized":"2023-11-16T16:37:15.157923+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:51:05.730477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:16.528248+0000","last_change":"2023-11-16T16:37:13.495972+0000","last_active":"2023-11-16T16:37:16.528248+0000","last_peered":"2023-11-16T16:37:16.528248+0000","last_clean":"2023-11-16T16:37:16.528248+0000","last_became_active":"2023-11-16T16:37:13.495610+0000","last_became_peered":"2023-11-16T16:37:13.495610+0000","last_unstale":"2023-11-16T16:37:16.528248+0000","last_undegraded":"2023-11-16T16:37:16.528248+0000","last_fullsized":"2023-11-16T16:37:16.528248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:58:27.825452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158624+0000","last_change":"2023-11-16T16:37:13.935236+0000","last_active":"2023-11-16T16:37:15.158624+0000","last_peered":"2023-11-16T16:37:15.158624+0000","last_clean":"2023-11-16T16:37:15.158624+0000","last_became_active":"2023-11-16T16:37:13.934920+0000","last_became_peered":"2023-11-16T16:37:13.934920+0000","last_unstale":"2023-11-16T16:37:15.158624+0000","last_undegraded":"2023-11-16T16:37:15.158624+0000","last_fullsized":"2023-11-16T16:37:15.158624+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:58:27.305887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"84'4","reported_seq":2,"reported_epoch":85,"state":"peering","last_fresh":"2023-11-16T16:37:16.431881+0000","last_change":"2023-11-16T16:38:02.736871+0000","last_active":"2023-11-16T16:38:02.736871+0000","last_peered":"2023-11-16T16:37:16.431881+0000","last_clean":"2023-11-16T16:37:16.431881+0000","last_became_active":"2023-11-16T16:37:13.509431+0000","last_became_peered":"2023-11-16T16:37:13.509431+0000","last_unstale":"2023-11-16T16:37:16.431881+0000","last_undegraded":"2023-11-16T16:37:16.431881+0000","last_fullsized":"2023-11-16T16:37:16.431881+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.156807+0000","last_change":"2023-11-16T16:37:13.940838+0000","last_active":"2023-11-16T16:37:15.156807+0000","last_peered":"2023-11-16T16:37:15.156807+0000","last_clean":"2023-11-16T16:37:15.156807+0000","last_became_active":"2023-11-16T16:37:13.940538+0000","last_became_peered":"2023-11-16T16:37:13.940538+0000","last_unstale":"2023-11-16T16:37:15.156807+0000","last_undegraded":"2023-11-16T16:37:15.156807+0000","last_fullsized":"2023-11-16T16:37:15.156807+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:24:41.595556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.925256+0000","last_change":"2023-11-16T16:37:13.495114+0000","last_active":"2023-11-16T16:37:14.925256+0000","last_peered":"2023-11-16T16:37:14.925256+0000","last_clean":"2023-11-16T16:37:14.925256+0000","last_became_active":"2023-11-16T16:37:13.494807+0000","last_became_peered":"2023-11-16T16:37:13.494807+0000","last_unstale":"2023-11-16T16:37:14.925256+0000","last_undegraded":"2023-11-16T16:37:14.925256+0000","last_fullsized":"2023-11-16T16:37:14.925256+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:24:04.736603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.924173+0000","last_change":"2023-11-16T16:37:13.943652+0000","last_active":"2023-11-16T16:37:14.924173+0000","last_peered":"2023-11-16T16:37:14.924173+0000","last_clean":"2023-11-16T16:37:14.924173+0000","last_became_active":"2023-11-16T16:37:13.943298+0000","last_became_peered":"2023-11-16T16:37:13.943298+0000","last_unstale":"2023-11-16T16:37:14.924173+0000","last_undegraded":"2023-11-16T16:37:14.924173+0000","last_fullsized":"2023-11-16T16:37:14.924173+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:34:34.876458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"84'6","reported_seq":65,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:59.363830+0000","last_change":"2023-11-16T16:37:13.509801+0000","last_active":"2023-11-16T16:37:59.363830+0000","last_peered":"2023-11-16T16:37:59.363830+0000","last_clean":"2023-11-16T16:37:59.363830+0000","last_became_active":"2023-11-16T16:37:13.509389+0000","last_became_peered":"2023-11-16T16:37:13.509389+0000","last_unstale":"2023-11-16T16:37:59.363830+0000","last_undegraded":"2023-11-16T16:37:59.363830+0000","last_fullsized":"2023-11-16T16:37:59.363830+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:55:02.922591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.924573+0000","last_change":"2023-11-16T16:37:13.933871+0000","last_active":"2023-11-16T16:37:14.924573+0000","last_peered":"2023-11-16T16:37:14.924573+0000","last_clean":"2023-11-16T16:37:14.924573+0000","last_became_active":"2023-11-16T16:37:13.933549+0000","last_became_peered":"2023-11-16T16:37:13.933549+0000","last_unstale":"2023-11-16T16:37:14.924573+0000","last_undegraded":"2023-11-16T16:37:14.924573+0000","last_fullsized":"2023-11-16T16:37:14.924573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:16:26.022577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"84'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:18.294818+0000","last_change":"2023-11-16T16:37:13.501339+0000","last_active":"2023-11-16T16:37:18.294818+0000","last_peered":"2023-11-16T16:37:18.294818+0000","last_clean":"2023-11-16T16:37:18.294818+0000","last_became_active":"2023-11-16T16:37:13.500949+0000","last_became_peered":"2023-11-16T16:37:13.500949+0000","last_unstale":"2023-11-16T16:37:18.294818+0000","last_undegraded":"2023-11-16T16:37:18.294818+0000","last_fullsized":"2023-11-16T16:37:18.294818+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:31:08.744759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.923893+0000","last_change":"2023-11-16T16:37:13.944158+0000","last_active":"2023-11-16T16:37:14.923893+0000","last_peered":"2023-11-16T16:37:14.923893+0000","last_clean":"2023-11-16T16:37:14.923893+0000","last_became_active":"2023-11-16T16:37:13.943833+0000","last_became_peered":"2023-11-16T16:37:13.943833+0000","last_unstale":"2023-11-16T16:37:14.923893+0000","last_undegraded":"2023-11-16T16:37:14.923893+0000","last_fullsized":"2023-11-16T16:37:14.923893+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:36:01.615006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"84'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:18.295129+0000","last_change":"2023-11-16T16:37:13.505481+0000","last_active":"2023-11-16T16:37:18.295129+0000","last_peered":"2023-11-16T16:37:18.295129+0000","last_clean":"2023-11-16T16:37:18.295129+0000","last_became_active":"2023-11-16T16:37:13.505085+0000","last_became_peered":"2023-11-16T16:37:13.505085+0000","last_unstale":"2023-11-16T16:37:18.295129+0000","last_undegraded":"2023-11-16T16:37:18.295129+0000","last_fullsized":"2023-11-16T16:37:18.295129+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:11:35.628249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157157+0000","last_change":"2023-11-16T16:37:13.943985+0000","last_active":"2023-11-16T16:37:15.157157+0000","last_peered":"2023-11-16T16:37:15.157157+0000","last_clean":"2023-11-16T16:37:15.157157+0000","last_became_active":"2023-11-16T16:37:13.943628+0000","last_became_peered":"2023-11-16T16:37:13.943628+0000","last_unstale":"2023-11-16T16:37:15.157157+0000","last_undegraded":"2023-11-16T16:37:15.157157+0000","last_fullsized":"2023-11-16T16:37:15.157157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:19:10.518280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.462357+0000","last_change":"2023-11-16T16:37:13.498648+0000","last_active":"2023-11-16T16:37:15.462357+0000","last_peered":"2023-11-16T16:37:15.462357+0000","last_clean":"2023-11-16T16:37:15.462357+0000","last_became_active":"2023-11-16T16:37:13.497390+0000","last_became_peered":"2023-11-16T16:37:13.497390+0000","last_unstale":"2023-11-16T16:37:15.462357+0000","last_undegraded":"2023-11-16T16:37:15.462357+0000","last_fullsized":"2023-11-16T16:37:15.462357+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:26:59.991663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157618+0000","last_change":"2023-11-16T16:37:13.942337+0000","last_active":"2023-11-16T16:37:15.157618+0000","last_peered":"2023-11-16T16:37:15.157618+0000","last_clean":"2023-11-16T16:37:15.157618+0000","last_became_active":"2023-11-16T16:37:13.941982+0000","last_became_peered":"2023-11-16T16:37:13.941982+0000","last_unstale":"2023-11-16T16:37:15.157618+0000","last_undegraded":"2023-11-16T16:37:15.157618+0000","last_fullsized":"2023-11-16T16:37:15.157618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:38:42.000956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'3","reported_seq":59,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:59.363291+0000","last_change":"2023-11-16T16:37:13.503690+0000","last_active":"2023-11-16T16:37:59.363291+0000","last_peered":"2023-11-16T16:37:59.363291+0000","last_clean":"2023-11-16T16:37:59.363291+0000","last_became_active":"2023-11-16T16:37:13.503203+0000","last_became_peered":"2023-11-16T16:37:13.503203+0000","last_unstale":"2023-11-16T16:37:59.363291+0000","last_undegraded":"2023-11-16T16:37:59.363291+0000","last_fullsized":"2023-11-16T16:37:59.363291+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:27:52.629218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.156964+0000","last_change":"2023-11-16T16:37:13.943975+0000","last_active":"2023-11-16T16:37:15.156964+0000","last_peered":"2023-11-16T16:37:15.156964+0000","last_clean":"2023-11-16T16:37:15.156964+0000","last_became_active":"2023-11-16T16:37:13.943611+0000","last_became_peered":"2023-11-16T16:37:13.943611+0000","last_unstale":"2023-11-16T16:37:15.156964+0000","last_undegraded":"2023-11-16T16:37:15.156964+0000","last_fullsized":"2023-11-16T16:37:15.156964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:21:29.078427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:17.285557+0000","last_change":"2023-11-16T16:37:13.506076+0000","last_active":"2023-11-16T16:37:17.285557+0000","last_peered":"2023-11-16T16:37:17.285557+0000","last_clean":"2023-11-16T16:37:17.285557+0000","last_became_active":"2023-11-16T16:37:13.505715+0000","last_became_peered":"2023-11-16T16:37:13.505715+0000","last_unstale":"2023-11-16T16:37:17.285557+0000","last_undegraded":"2023-11-16T16:37:17.285557+0000","last_fullsized":"2023-11-16T16:37:17.285557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:03:10.081004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158690+0000","last_change":"2023-11-16T16:37:13.949031+0000","last_active":"2023-11-16T16:37:15.158690+0000","last_peered":"2023-11-16T16:37:15.158690+0000","last_clean":"2023-11-16T16:37:15.158690+0000","last_became_active":"2023-11-16T16:37:13.946923+0000","last_became_peered":"2023-11-16T16:37:13.946923+0000","last_unstale":"2023-11-16T16:37:15.158690+0000","last_undegraded":"2023-11-16T16:37:15.158690+0000","last_fullsized":"2023-11-16T16:37:15.158690+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:00:35.683602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:17.286359+0000","last_change":"2023-11-16T16:37:13.495003+0000","last_active":"2023-11-16T16:37:17.286359+0000","last_peered":"2023-11-16T16:37:17.286359+0000","last_clean":"2023-11-16T16:37:17.286359+0000","last_became_active":"2023-11-16T16:37:13.494563+0000","last_became_peered":"2023-11-16T16:37:13.494563+0000","last_unstale":"2023-11-16T16:37:17.286359+0000","last_undegraded":"2023-11-16T16:37:17.286359+0000","last_fullsized":"2023-11-16T16:37:17.286359+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:12:53.840907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.156822+0000","last_change":"2023-11-16T16:37:13.943736+0000","last_active":"2023-11-16T16:37:15.156822+0000","last_peered":"2023-11-16T16:37:15.156822+0000","last_clean":"2023-11-16T16:37:15.156822+0000","last_became_active":"2023-11-16T16:37:13.943391+0000","last_became_peered":"2023-11-16T16:37:13.943391+0000","last_unstale":"2023-11-16T16:37:15.156822+0000","last_undegraded":"2023-11-16T16:37:15.156822+0000","last_fullsized":"2023-11-16T16:37:15.156822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:03:21.402516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'28","reported_seq":134,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:38:00.375703+0000","last_change":"2023-11-16T16:37:13.506508+0000","last_active":"2023-11-16T16:38:00.375703+0000","last_peered":"2023-11-16T16:38:00.375703+0000","last_clean":"2023-11-16T16:38:00.375703+0000","last_became_active":"2023-11-16T16:37:13.506045+0000","last_became_peered":"2023-11-16T16:37:13.506045+0000","last_unstale":"2023-11-16T16:38:00.375703+0000","last_undegraded":"2023-11-16T16:38:00.375703+0000","last_fullsized":"2023-11-16T16:38:00.375703+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:26:04.265699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58527,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":92,"num_read_kb":102,"num_write":29,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.924203+0000","last_change":"2023-11-16T16:37:13.939717+0000","last_active":"2023-11-16T16:37:14.924203+0000","last_peered":"2023-11-16T16:37:14.924203+0000","last_clean":"2023-11-16T16:37:14.924203+0000","last_became_active":"2023-11-16T16:37:13.939369+0000","last_became_peered":"2023-11-16T16:37:13.939369+0000","last_unstale":"2023-11-16T16:37:14.924203+0000","last_undegraded":"2023-11-16T16:37:14.924203+0000","last_fullsized":"2023-11-16T16:37:14.924203+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:15:46.898377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'6","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:18.296464+0000","last_change":"2023-11-16T16:37:13.506056+0000","last_active":"2023-11-16T16:37:18.296464+0000","last_peered":"2023-11-16T16:37:18.296464+0000","last_clean":"2023-11-16T16:37:18.296464+0000","last_became_active":"2023-11-16T16:37:13.505707+0000","last_became_peered":"2023-11-16T16:37:13.505707+0000","last_unstale":"2023-11-16T16:37:18.296464+0000","last_undegraded":"2023-11-16T16:37:18.296464+0000","last_fullsized":"2023-11-16T16:37:18.296464+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:36:08.667708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157120+0000","last_change":"2023-11-16T16:37:13.947764+0000","last_active":"2023-11-16T16:37:15.157120+0000","last_peered":"2023-11-16T16:37:15.157120+0000","last_clean":"2023-11-16T16:37:15.157120+0000","last_became_active":"2023-11-16T16:37:13.947510+0000","last_became_peered":"2023-11-16T16:37:13.947510+0000","last_unstale":"2023-11-16T16:37:15.157120+0000","last_undegraded":"2023-11-16T16:37:15.157120+0000","last_fullsized":"2023-11-16T16:37:15.157120+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:08:15.784489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:17.287513+0000","last_change":"2023-11-16T16:37:13.498178+0000","last_active":"2023-11-16T16:37:17.287513+0000","last_peered":"2023-11-16T16:37:17.287513+0000","last_clean":"2023-11-16T16:37:17.287513+0000","last_became_active":"2023-11-16T16:37:13.497312+0000","last_became_peered":"2023-11-16T16:37:13.497312+0000","last_unstale":"2023-11-16T16:37:17.287513+0000","last_undegraded":"2023-11-16T16:37:17.287513+0000","last_fullsized":"2023-11-16T16:37:17.287513+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:01:58.840690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.391132+0000","last_change":"2023-11-16T16:37:13.947854+0000","last_active":"2023-11-16T16:37:15.391132+0000","last_peered":"2023-11-16T16:37:15.391132+0000","last_clean":"2023-11-16T16:37:15.391132+0000","last_became_active":"2023-11-16T16:37:13.947530+0000","last_became_peered":"2023-11-16T16:37:13.947530+0000","last_unstale":"2023-11-16T16:37:15.391132+0000","last_undegraded":"2023-11-16T16:37:15.391132+0000","last_fullsized":"2023-11-16T16:37:15.391132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:25:25.919855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'3","reported_seq":59,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:57.360983+0000","last_change":"2023-11-16T16:37:13.500127+0000","last_active":"2023-11-16T16:37:57.360983+0000","last_peered":"2023-11-16T16:37:57.360983+0000","last_clean":"2023-11-16T16:37:57.360983+0000","last_became_active":"2023-11-16T16:37:13.499758+0000","last_became_peered":"2023-11-16T16:37:13.499758+0000","last_unstale":"2023-11-16T16:37:57.360983+0000","last_undegraded":"2023-11-16T16:37:57.360983+0000","last_fullsized":"2023-11-16T16:37:57.360983+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:12:24.610215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.390796+0000","last_change":"2023-11-16T16:37:13.940962+0000","last_active":"2023-11-16T16:37:15.390796+0000","last_peered":"2023-11-16T16:37:15.390796+0000","last_clean":"2023-11-16T16:37:15.390796+0000","last_became_active":"2023-11-16T16:37:13.940450+0000","last_became_peered":"2023-11-16T16:37:13.940450+0000","last_unstale":"2023-11-16T16:37:15.390796+0000","last_undegraded":"2023-11-16T16:37:15.390796+0000","last_fullsized":"2023-11-16T16:37:15.390796+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:20:04.879494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'15","reported_seq":2,"reported_epoch":85,"state":"peering","last_fresh":"2023-11-16T16:37:17.285577+0000","last_change":"2023-11-16T16:38:02.736820+0000","last_active":"2023-11-16T16:38:02.736820+0000","last_peered":"2023-11-16T16:37:17.285577+0000","last_clean":"2023-11-16T16:37:17.285577+0000","last_became_active":"2023-11-16T16:37:13.492599+0000","last_became_peered":"2023-11-16T16:37:13.492599+0000","last_unstale":"2023-11-16T16:37:17.285577+0000","last_undegraded":"2023-11-16T16:37:17.285577+0000","last_fullsized":"2023-11-16T16:37:17.285577+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":8,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.924063+0000","last_change":"2023-11-16T16:37:13.944878+0000","last_active":"2023-11-16T16:37:14.924063+0000","last_peered":"2023-11-16T16:37:14.924063+0000","last_clean":"2023-11-16T16:37:14.924063+0000","last_became_active":"2023-11-16T16:37:13.944580+0000","last_became_peered":"2023-11-16T16:37:13.944580+0000","last_unstale":"2023-11-16T16:37:14.924063+0000","last_undegraded":"2023-11-16T16:37:14.924063+0000","last_fullsized":"2023-11-16T16:37:14.924063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:01:52.805052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:19.300067+0000","last_change":"2023-11-16T16:37:13.499139+0000","last_active":"2023-11-16T16:37:19.300067+0000","last_peered":"2023-11-16T16:37:19.300067+0000","last_clean":"2023-11-16T16:37:19.300067+0000","last_became_active":"2023-11-16T16:37:13.498755+0000","last_became_peered":"2023-11-16T16:37:13.498755+0000","last_unstale":"2023-11-16T16:37:19.300067+0000","last_undegraded":"2023-11-16T16:37:19.300067+0000","last_fullsized":"2023-11-16T16:37:19.300067+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:07:40.535306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-16T16:38:02.187750+0000","last_change":"2023-11-16T16:37:13.946187+0000","last_active":"2023-11-16T16:38:02.187750+0000","last_peered":"2023-11-16T16:38:02.187750+0000","last_clean":"2023-11-16T16:38:02.187750+0000","last_became_active":"2023-11-16T16:37:13.945887+0000","last_became_peered":"2023-11-16T16:37:13.945887+0000","last_unstale":"2023-11-16T16:38:02.187750+0000","last_undegraded":"2023-11-16T16:38:02.187750+0000","last_fullsized":"2023-11-16T16:38:02.187750+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:53:33.133780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157191+0000","last_change":"2023-11-16T16:37:13.507150+0000","last_active":"2023-11-16T16:37:15.157191+0000","last_peered":"2023-11-16T16:37:15.157191+0000","last_clean":"2023-11-16T16:37:15.157191+0000","last_became_active":"2023-11-16T16:37:13.506814+0000","last_became_peered":"2023-11-16T16:37:13.506814+0000","last_unstale":"2023-11-16T16:37:15.157191+0000","last_undegraded":"2023-11-16T16:37:15.157191+0000","last_fullsized":"2023-11-16T16:37:15.157191+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:58:14.692816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.462555+0000","last_change":"2023-11-16T16:37:13.947866+0000","last_active":"2023-11-16T16:37:15.462555+0000","last_peered":"2023-11-16T16:37:15.462555+0000","last_clean":"2023-11-16T16:37:15.462555+0000","last_became_active":"2023-11-16T16:37:13.947544+0000","last_became_peered":"2023-11-16T16:37:13.947544+0000","last_unstale":"2023-11-16T16:37:15.462555+0000","last_undegraded":"2023-11-16T16:37:15.462555+0000","last_fullsized":"2023-11-16T16:37:15.462555+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:58:42.237793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"84'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.391037+0000","last_change":"2023-11-16T16:37:13.497800+0000","last_active":"2023-11-16T16:37:15.391037+0000","last_peered":"2023-11-16T16:37:15.391037+0000","last_clean":"2023-11-16T16:37:15.391037+0000","last_became_active":"2023-11-16T16:37:13.497414+0000","last_became_peered":"2023-11-16T16:37:13.497414+0000","last_unstale":"2023-11-16T16:37:15.391037+0000","last_undegraded":"2023-11-16T16:37:15.391037+0000","last_fullsized":"2023-11-16T16:37:15.391037+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:57:26.591481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.924384+0000","last_change":"2023-11-16T16:37:13.946780+0000","last_active":"2023-11-16T16:37:14.924384+0000","last_peered":"2023-11-16T16:37:14.924384+0000","last_clean":"2023-11-16T16:37:14.924384+0000","last_became_active":"2023-11-16T16:37:13.946441+0000","last_became_peered":"2023-11-16T16:37:13.946441+0000","last_unstale":"2023-11-16T16:37:14.924384+0000","last_undegraded":"2023-11-16T16:37:14.924384+0000","last_fullsized":"2023-11-16T16:37:14.924384+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:20:09.943373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.923519+0000","last_change":"2023-11-16T16:37:13.937898+0000","last_active":"2023-11-16T16:37:14.923519+0000","last_peered":"2023-11-16T16:37:14.923519+0000","last_clean":"2023-11-16T16:37:14.923519+0000","last_became_active":"2023-11-16T16:37:13.937598+0000","last_became_peered":"2023-11-16T16:37:13.937598+0000","last_unstale":"2023-11-16T16:37:14.923519+0000","last_undegraded":"2023-11-16T16:37:14.923519+0000","last_fullsized":"2023-11-16T16:37:14.923519+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:57:03.646617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"84'4","reported_seq":58,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:57.360346+0000","last_change":"2023-11-16T16:37:13.498129+0000","last_active":"2023-11-16T16:37:57.360346+0000","last_peered":"2023-11-16T16:37:57.360346+0000","last_clean":"2023-11-16T16:37:57.360346+0000","last_became_active":"2023-11-16T16:37:13.497765+0000","last_became_peered":"2023-11-16T16:37:13.497765+0000","last_unstale":"2023-11-16T16:37:57.360346+0000","last_undegraded":"2023-11-16T16:37:57.360346+0000","last_fullsized":"2023-11-16T16:37:57.360346+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:13:33.579950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158156+0000","last_change":"2023-11-16T16:37:13.938417+0000","last_active":"2023-11-16T16:37:15.158156+0000","last_peered":"2023-11-16T16:37:15.158156+0000","last_clean":"2023-11-16T16:37:15.158156+0000","last_became_active":"2023-11-16T16:37:13.938114+0000","last_became_peered":"2023-11-16T16:37:13.938114+0000","last_unstale":"2023-11-16T16:37:15.158156+0000","last_undegraded":"2023-11-16T16:37:15.158156+0000","last_fullsized":"2023-11-16T16:37:15.158156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:35:24.220360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"84'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:19.298484+0000","last_change":"2023-11-16T16:37:13.499045+0000","last_active":"2023-11-16T16:37:19.298484+0000","last_peered":"2023-11-16T16:37:19.298484+0000","last_clean":"2023-11-16T16:37:19.298484+0000","last_became_active":"2023-11-16T16:37:13.498598+0000","last_became_peered":"2023-11-16T16:37:13.498598+0000","last_unstale":"2023-11-16T16:37:19.298484+0000","last_undegraded":"2023-11-16T16:37:19.298484+0000","last_fullsized":"2023-11-16T16:37:19.298484+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:25:26.520931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":345,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.390796+0000","last_change":"2023-11-16T16:36:14.090213+0000","last_active":"2023-11-16T16:37:15.390796+0000","last_peered":"2023-11-16T16:37:15.390796+0000","last_clean":"2023-11-16T16:37:15.390796+0000","last_became_active":"2023-11-16T16:36:14.089742+0000","last_became_peered":"2023-11-16T16:36:14.089742+0000","last_unstale":"2023-11-16T16:37:15.390796+0000","last_undegraded":"2023-11-16T16:37:15.390796+0000","last_fullsized":"2023-11-16T16:37:15.390796+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:33:16.178716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:33:16.178716+0000","last_clean_scrub_stamp":"2023-11-16T16:33:16.178716+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:11:37.401838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157103+0000","last_change":"2023-11-16T16:37:13.945380+0000","last_active":"2023-11-16T16:37:15.157103+0000","last_peered":"2023-11-16T16:37:15.157103+0000","last_clean":"2023-11-16T16:37:15.157103+0000","last_became_active":"2023-11-16T16:37:13.945038+0000","last_became_peered":"2023-11-16T16:37:13.945038+0000","last_unstale":"2023-11-16T16:37:15.157103+0000","last_undegraded":"2023-11-16T16:37:15.157103+0000","last_fullsized":"2023-11-16T16:37:15.157103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:35:29.662867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"84'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:18.296049+0000","last_change":"2023-11-16T16:37:13.497586+0000","last_active":"2023-11-16T16:37:18.296049+0000","last_peered":"2023-11-16T16:37:18.296049+0000","last_clean":"2023-11-16T16:37:18.296049+0000","last_became_active":"2023-11-16T16:37:13.497233+0000","last_became_peered":"2023-11-16T16:37:13.497233+0000","last_unstale":"2023-11-16T16:37:18.296049+0000","last_undegraded":"2023-11-16T16:37:18.296049+0000","last_fullsized":"2023-11-16T16:37:18.296049+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:24:30.086496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"84'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:17.285613+0000","last_change":"2023-11-16T16:37:13.500621+0000","last_active":"2023-11-16T16:37:17.285613+0000","last_peered":"2023-11-16T16:37:17.285613+0000","last_clean":"2023-11-16T16:37:17.285613+0000","last_became_active":"2023-11-16T16:37:13.500265+0000","last_became_peered":"2023-11-16T16:37:13.500265+0000","last_unstale":"2023-11-16T16:37:17.285613+0000","last_undegraded":"2023-11-16T16:37:17.285613+0000","last_fullsized":"2023-11-16T16:37:17.285613+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:51:28.450861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157560+0000","last_change":"2023-11-16T16:37:13.930992+0000","last_active":"2023-11-16T16:37:15.157560+0000","last_peered":"2023-11-16T16:37:15.157560+0000","last_clean":"2023-11-16T16:37:15.157560+0000","last_became_active":"2023-11-16T16:37:13.930641+0000","last_became_peered":"2023-11-16T16:37:13.930641+0000","last_unstale":"2023-11-16T16:37:15.157560+0000","last_undegraded":"2023-11-16T16:37:15.157560+0000","last_fullsized":"2023-11-16T16:37:15.157560+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:58:37.743753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'5","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:24.183823+0000","last_change":"2023-11-16T16:37:13.498637+0000","last_active":"2023-11-16T16:37:24.183823+0000","last_peered":"2023-11-16T16:37:24.183823+0000","last_clean":"2023-11-16T16:37:24.183823+0000","last_became_active":"2023-11-16T16:37:13.498341+0000","last_became_peered":"2023-11-16T16:37:13.498341+0000","last_unstale":"2023-11-16T16:37:24.183823+0000","last_undegraded":"2023-11-16T16:37:24.183823+0000","last_fullsized":"2023-11-16T16:37:24.183823+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:00:31.333304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158333+0000","last_change":"2023-11-16T16:37:13.944142+0000","last_active":"2023-11-16T16:37:15.158333+0000","last_peered":"2023-11-16T16:37:15.158333+0000","last_clean":"2023-11-16T16:37:15.158333+0000","last_became_active":"2023-11-16T16:37:13.943758+0000","last_became_peered":"2023-11-16T16:37:13.943758+0000","last_unstale":"2023-11-16T16:37:15.158333+0000","last_undegraded":"2023-11-16T16:37:15.158333+0000","last_fullsized":"2023-11-16T16:37:15.158333+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:55:52.343549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'4","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:17.287097+0000","last_change":"2023-11-16T16:37:13.496900+0000","last_active":"2023-11-16T16:37:17.287097+0000","last_peered":"2023-11-16T16:37:17.287097+0000","last_clean":"2023-11-16T16:37:17.287097+0000","last_became_active":"2023-11-16T16:37:13.496528+0000","last_became_peered":"2023-11-16T16:37:13.496528+0000","last_unstale":"2023-11-16T16:37:17.287097+0000","last_undegraded":"2023-11-16T16:37:17.287097+0000","last_fullsized":"2023-11-16T16:37:17.287097+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:37:22.646222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158258+0000","last_change":"2023-11-16T16:37:13.943104+0000","last_active":"2023-11-16T16:37:15.158258+0000","last_peered":"2023-11-16T16:37:15.158258+0000","last_clean":"2023-11-16T16:37:15.158258+0000","last_became_active":"2023-11-16T16:37:13.942774+0000","last_became_peered":"2023-11-16T16:37:13.942774+0000","last_unstale":"2023-11-16T16:37:15.158258+0000","last_undegraded":"2023-11-16T16:37:15.158258+0000","last_fullsized":"2023-11-16T16:37:15.158258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:00:01.892620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:17.287386+0000","last_change":"2023-11-16T16:37:13.505994+0000","last_active":"2023-11-16T16:37:17.287386+0000","last_peered":"2023-11-16T16:37:17.287386+0000","last_clean":"2023-11-16T16:37:17.287386+0000","last_became_active":"2023-11-16T16:37:13.505663+0000","last_became_peered":"2023-11-16T16:37:13.505663+0000","last_unstale":"2023-11-16T16:37:17.287386+0000","last_undegraded":"2023-11-16T16:37:17.287386+0000","last_fullsized":"2023-11-16T16:37:17.287386+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:11:39.436119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.923847+0000","last_change":"2023-11-16T16:37:13.941970+0000","last_active":"2023-11-16T16:37:14.923847+0000","last_peered":"2023-11-16T16:37:14.923847+0000","last_clean":"2023-11-16T16:37:14.923847+0000","last_became_active":"2023-11-16T16:37:13.941649+0000","last_became_peered":"2023-11-16T16:37:13.941649+0000","last_unstale":"2023-11-16T16:37:14.923847+0000","last_undegraded":"2023-11-16T16:37:14.923847+0000","last_fullsized":"2023-11-16T16:37:14.923847+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:00:21.646221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:17.287982+0000","last_change":"2023-11-16T16:37:13.503429+0000","last_active":"2023-11-16T16:37:17.287982+0000","last_peered":"2023-11-16T16:37:17.287982+0000","last_clean":"2023-11-16T16:37:17.287982+0000","last_became_active":"2023-11-16T16:37:13.503114+0000","last_became_peered":"2023-11-16T16:37:13.503114+0000","last_unstale":"2023-11-16T16:37:17.287982+0000","last_undegraded":"2023-11-16T16:37:17.287982+0000","last_fullsized":"2023-11-16T16:37:17.287982+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:05:42.643043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-16T16:38:02.187143+0000","last_change":"2023-11-16T16:37:13.941225+0000","last_active":"2023-11-16T16:38:02.187143+0000","last_peered":"2023-11-16T16:38:02.187143+0000","last_clean":"2023-11-16T16:38:02.187143+0000","last_became_active":"2023-11-16T16:37:13.940964+0000","last_became_peered":"2023-11-16T16:37:13.940964+0000","last_unstale":"2023-11-16T16:38:02.187143+0000","last_undegraded":"2023-11-16T16:38:02.187143+0000","last_fullsized":"2023-11-16T16:38:02.187143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:43:22.130214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157663+0000","last_change":"2023-11-16T16:37:13.510154+0000","last_active":"2023-11-16T16:37:15.157663+0000","last_peered":"2023-11-16T16:37:15.157663+0000","last_clean":"2023-11-16T16:37:15.157663+0000","last_became_active":"2023-11-16T16:37:13.509790+0000","last_became_peered":"2023-11-16T16:37:13.509790+0000","last_unstale":"2023-11-16T16:37:15.157663+0000","last_undegraded":"2023-11-16T16:37:15.157663+0000","last_fullsized":"2023-11-16T16:37:15.157663+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T16:54:36.581541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158172+0000","last_change":"2023-11-16T16:37:13.942203+0000","last_active":"2023-11-16T16:37:15.158172+0000","last_peered":"2023-11-16T16:37:15.158172+0000","last_clean":"2023-11-16T16:37:15.158172+0000","last_became_active":"2023-11-16T16:37:13.941766+0000","last_became_peered":"2023-11-16T16:37:13.941766+0000","last_unstale":"2023-11-16T16:37:15.158172+0000","last_undegraded":"2023-11-16T16:37:15.158172+0000","last_fullsized":"2023-11-16T16:37:15.158172+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:12:25.582734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:17.286155+0000","last_change":"2023-11-16T16:37:13.509854+0000","last_active":"2023-11-16T16:37:17.286155+0000","last_peered":"2023-11-16T16:37:17.286155+0000","last_clean":"2023-11-16T16:37:17.286155+0000","last_became_active":"2023-11-16T16:37:13.509466+0000","last_became_peered":"2023-11-16T16:37:13.509466+0000","last_unstale":"2023-11-16T16:37:17.286155+0000","last_undegraded":"2023-11-16T16:37:17.286155+0000","last_fullsized":"2023-11-16T16:37:17.286155+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:05:41.806694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.391285+0000","last_change":"2023-11-16T16:37:13.947906+0000","last_active":"2023-11-16T16:37:15.391285+0000","last_peered":"2023-11-16T16:37:15.391285+0000","last_clean":"2023-11-16T16:37:15.391285+0000","last_became_active":"2023-11-16T16:37:13.947602+0000","last_became_peered":"2023-11-16T16:37:13.947602+0000","last_unstale":"2023-11-16T16:37:15.391285+0000","last_undegraded":"2023-11-16T16:37:15.391285+0000","last_fullsized":"2023-11-16T16:37:15.391285+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:16:47.335757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'3","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:18.297466+0000","last_change":"2023-11-16T16:37:13.506108+0000","last_active":"2023-11-16T16:37:18.297466+0000","last_peered":"2023-11-16T16:37:18.297466+0000","last_clean":"2023-11-16T16:37:18.297466+0000","last_became_active":"2023-11-16T16:37:13.505785+0000","last_became_peered":"2023-11-16T16:37:13.505785+0000","last_unstale":"2023-11-16T16:37:18.297466+0000","last_undegraded":"2023-11-16T16:37:18.297466+0000","last_fullsized":"2023-11-16T16:37:18.297466+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:01:15.544970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.391265+0000","last_change":"2023-11-16T16:37:13.935863+0000","last_active":"2023-11-16T16:37:15.391265+0000","last_peered":"2023-11-16T16:37:15.391265+0000","last_clean":"2023-11-16T16:37:15.391265+0000","last_became_active":"2023-11-16T16:37:13.935399+0000","last_became_peered":"2023-11-16T16:37:13.935399+0000","last_unstale":"2023-11-16T16:37:15.391265+0000","last_undegraded":"2023-11-16T16:37:15.391265+0000","last_fullsized":"2023-11-16T16:37:15.391265+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:38:54.382348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.462427+0000","last_change":"2023-11-16T16:37:13.498367+0000","last_active":"2023-11-16T16:37:15.462427+0000","last_peered":"2023-11-16T16:37:15.462427+0000","last_clean":"2023-11-16T16:37:15.462427+0000","last_became_active":"2023-11-16T16:37:13.497039+0000","last_became_peered":"2023-11-16T16:37:13.497039+0000","last_unstale":"2023-11-16T16:37:15.462427+0000","last_undegraded":"2023-11-16T16:37:15.462427+0000","last_fullsized":"2023-11-16T16:37:15.462427+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:07:52.215240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.391224+0000","last_change":"2023-11-16T16:37:13.940617+0000","last_active":"2023-11-16T16:37:15.391224+0000","last_peered":"2023-11-16T16:37:15.391224+0000","last_clean":"2023-11-16T16:37:15.391224+0000","last_became_active":"2023-11-16T16:37:13.940264+0000","last_became_peered":"2023-11-16T16:37:13.940264+0000","last_unstale":"2023-11-16T16:37:15.391224+0000","last_undegraded":"2023-11-16T16:37:15.391224+0000","last_fullsized":"2023-11-16T16:37:15.391224+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:22:54.941976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157351+0000","last_change":"2023-11-16T16:37:13.505950+0000","last_active":"2023-11-16T16:37:15.157351+0000","last_peered":"2023-11-16T16:37:15.157351+0000","last_clean":"2023-11-16T16:37:15.157351+0000","last_became_active":"2023-11-16T16:37:13.505628+0000","last_became_peered":"2023-11-16T16:37:13.505628+0000","last_unstale":"2023-11-16T16:37:15.157351+0000","last_undegraded":"2023-11-16T16:37:15.157351+0000","last_fullsized":"2023-11-16T16:37:15.157351+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:11:17.082705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.391279+0000","last_change":"2023-11-16T16:37:13.943096+0000","last_active":"2023-11-16T16:37:15.391279+0000","last_peered":"2023-11-16T16:37:15.391279+0000","last_clean":"2023-11-16T16:37:15.391279+0000","last_became_active":"2023-11-16T16:37:13.942745+0000","last_became_peered":"2023-11-16T16:37:13.942745+0000","last_unstale":"2023-11-16T16:37:15.391279+0000","last_undegraded":"2023-11-16T16:37:15.391279+0000","last_fullsized":"2023-11-16T16:37:15.391279+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:35:21.092899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-16T16:38:02.187324+0000","last_change":"2023-11-16T16:37:13.938087+0000","last_active":"2023-11-16T16:38:02.187324+0000","last_peered":"2023-11-16T16:38:02.187324+0000","last_clean":"2023-11-16T16:38:02.187324+0000","last_became_active":"2023-11-16T16:37:13.937780+0000","last_became_peered":"2023-11-16T16:37:13.937780+0000","last_unstale":"2023-11-16T16:38:02.187324+0000","last_undegraded":"2023-11-16T16:38:02.187324+0000","last_fullsized":"2023-11-16T16:38:02.187324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:08:08.645514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157978+0000","last_change":"2023-11-16T16:37:13.939000+0000","last_active":"2023-11-16T16:37:15.157978+0000","last_peered":"2023-11-16T16:37:15.157978+0000","last_clean":"2023-11-16T16:37:15.157978+0000","last_became_active":"2023-11-16T16:37:13.938585+0000","last_became_peered":"2023-11-16T16:37:13.938585+0000","last_unstale":"2023-11-16T16:37:15.157978+0000","last_undegraded":"2023-11-16T16:37:15.157978+0000","last_fullsized":"2023-11-16T16:37:15.157978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:17:47.012171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157516+0000","last_change":"2023-11-16T16:37:13.937696+0000","last_active":"2023-11-16T16:37:15.157516+0000","last_peered":"2023-11-16T16:37:15.157516+0000","last_clean":"2023-11-16T16:37:15.157516+0000","last_became_active":"2023-11-16T16:37:13.936920+0000","last_became_peered":"2023-11-16T16:37:13.936920+0000","last_unstale":"2023-11-16T16:37:15.157516+0000","last_undegraded":"2023-11-16T16:37:15.157516+0000","last_fullsized":"2023-11-16T16:37:15.157516+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:40:10.578228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157738+0000","last_change":"2023-11-16T16:37:13.942977+0000","last_active":"2023-11-16T16:37:15.157738+0000","last_peered":"2023-11-16T16:37:15.157738+0000","last_clean":"2023-11-16T16:37:15.157738+0000","last_became_active":"2023-11-16T16:37:13.942632+0000","last_became_peered":"2023-11-16T16:37:13.942632+0000","last_unstale":"2023-11-16T16:37:15.157738+0000","last_undegraded":"2023-11-16T16:37:15.157738+0000","last_fullsized":"2023-11-16T16:37:15.157738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:44:02.416862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157651+0000","last_change":"2023-11-16T16:37:13.949940+0000","last_active":"2023-11-16T16:37:15.157651+0000","last_peered":"2023-11-16T16:37:15.157651+0000","last_clean":"2023-11-16T16:37:15.157651+0000","last_became_active":"2023-11-16T16:37:13.949464+0000","last_became_peered":"2023-11-16T16:37:13.949464+0000","last_unstale":"2023-11-16T16:37:15.157651+0000","last_undegraded":"2023-11-16T16:37:15.157651+0000","last_fullsized":"2023-11-16T16:37:15.157651+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:52:47.439482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-16T16:38:02.187885+0000","last_change":"2023-11-16T16:37:13.940534+0000","last_active":"2023-11-16T16:38:02.187885+0000","last_peered":"2023-11-16T16:38:02.187885+0000","last_clean":"2023-11-16T16:38:02.187885+0000","last_became_active":"2023-11-16T16:37:13.940174+0000","last_became_peered":"2023-11-16T16:37:13.940174+0000","last_unstale":"2023-11-16T16:38:02.187885+0000","last_undegraded":"2023-11-16T16:38:02.187885+0000","last_fullsized":"2023-11-16T16:38:02.187885+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:20:45.829821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157969+0000","last_change":"2023-11-16T16:37:13.937519+0000","last_active":"2023-11-16T16:37:15.157969+0000","last_peered":"2023-11-16T16:37:15.157969+0000","last_clean":"2023-11-16T16:37:15.157969+0000","last_became_active":"2023-11-16T16:37:13.937175+0000","last_became_peered":"2023-11-16T16:37:13.937175+0000","last_unstale":"2023-11-16T16:37:15.157969+0000","last_undegraded":"2023-11-16T16:37:15.157969+0000","last_fullsized":"2023-11-16T16:37:15.157969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T16:47:03.319128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158262+0000","last_change":"2023-11-16T16:37:13.951506+0000","last_active":"2023-11-16T16:37:15.158262+0000","last_peered":"2023-11-16T16:37:15.158262+0000","last_clean":"2023-11-16T16:37:15.158262+0000","last_became_active":"2023-11-16T16:37:13.951192+0000","last_became_peered":"2023-11-16T16:37:13.951192+0000","last_unstale":"2023-11-16T16:37:15.158262+0000","last_undegraded":"2023-11-16T16:37:15.158262+0000","last_fullsized":"2023-11-16T16:37:15.158262+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:18:48.061220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.923594+0000","last_change":"2023-11-16T16:37:13.935614+0000","last_active":"2023-11-16T16:37:14.923594+0000","last_peered":"2023-11-16T16:37:14.923594+0000","last_clean":"2023-11-16T16:37:14.923594+0000","last_became_active":"2023-11-16T16:37:13.935237+0000","last_became_peered":"2023-11-16T16:37:13.935237+0000","last_unstale":"2023-11-16T16:37:14.923594+0000","last_undegraded":"2023-11-16T16:37:14.923594+0000","last_fullsized":"2023-11-16T16:37:14.923594+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:02:10.333347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157636+0000","last_change":"2023-11-16T16:37:13.939387+0000","last_active":"2023-11-16T16:37:15.157636+0000","last_peered":"2023-11-16T16:37:15.157636+0000","last_clean":"2023-11-16T16:37:15.157636+0000","last_became_active":"2023-11-16T16:37:13.939082+0000","last_became_peered":"2023-11-16T16:37:13.939082+0000","last_unstale":"2023-11-16T16:37:15.157636+0000","last_undegraded":"2023-11-16T16:37:15.157636+0000","last_fullsized":"2023-11-16T16:37:15.157636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:42:24.707911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.156764+0000","last_change":"2023-11-16T16:37:13.936381+0000","last_active":"2023-11-16T16:37:15.156764+0000","last_peered":"2023-11-16T16:37:15.156764+0000","last_clean":"2023-11-16T16:37:15.156764+0000","last_became_active":"2023-11-16T16:37:13.936077+0000","last_became_peered":"2023-11-16T16:37:13.936077+0000","last_unstale":"2023-11-16T16:37:15.156764+0000","last_undegraded":"2023-11-16T16:37:15.156764+0000","last_fullsized":"2023-11-16T16:37:15.156764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:04:48.256913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157497+0000","last_change":"2023-11-16T16:37:13.934005+0000","last_active":"2023-11-16T16:37:15.157497+0000","last_peered":"2023-11-16T16:37:15.157497+0000","last_clean":"2023-11-16T16:37:15.157497+0000","last_became_active":"2023-11-16T16:37:13.933633+0000","last_became_peered":"2023-11-16T16:37:13.933633+0000","last_unstale":"2023-11-16T16:37:15.157497+0000","last_undegraded":"2023-11-16T16:37:15.157497+0000","last_fullsized":"2023-11-16T16:37:15.157497+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:45:12.620887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-16T16:38:02.187300+0000","last_change":"2023-11-16T16:37:13.946495+0000","last_active":"2023-11-16T16:38:02.187300+0000","last_peered":"2023-11-16T16:38:02.187300+0000","last_clean":"2023-11-16T16:38:02.187300+0000","last_became_active":"2023-11-16T16:37:13.946148+0000","last_became_peered":"2023-11-16T16:37:13.946148+0000","last_unstale":"2023-11-16T16:38:02.187300+0000","last_undegraded":"2023-11-16T16:38:02.187300+0000","last_fullsized":"2023-11-16T16:38:02.187300+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:15:23.875543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":3542208,"data_compressed":211753,"data_compressed_allocated":270336,"data_compressed_original":3522560,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62431,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":10,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":274,"num_read_kb":202,"num_write":150,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":187293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132,"ondisk_log_size":132,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416406,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5944,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":93739208,"statfs":{"total":95995035648,"available":95988948992,"internally_reserved":0,"allocated":901120,"data_stored":429451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.68700000000000006}]},{"osd":1,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.79700000000000004}]},{"osd":2,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.48599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":4,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.70899999999999996}]},{"osd":5,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.61699999999999999}]},{"osd":6,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":0.85699999999999998}]},{"osd":7,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.84399999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":9,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.88}]},{"osd":10,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.863,"5min":1.863,"15min":1.863},"last":0.92200000000000004}]}]},{"osd":10,"up_from":72,"seq":309237645337,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6408,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738744,"statfs":{"total":95995035648,"available":95988473856,"internally_reserved":0,"allocated":1048576,"data_stored":1079086,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":1,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.70299999999999996}]},{"osd":2,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.63300000000000001}]},{"osd":3,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.51100000000000001}]},{"osd":4,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.753}]},{"osd":5,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.78500000000000003}]},{"osd":6,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.76900000000000002}]},{"osd":7,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.505}]},{"osd":8,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.80300000000000005}]},{"osd":9,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.85699999999999998}]},{"osd":11,"last update":"Thu Nov 16 16:37:33 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.91400000000000003}]}]},{"osd":9,"up_from":67,"seq":287762808860,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6088,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93739064,"statfs":{"total":95995035648,"available":95988801536,"internally_reserved":0,"allocated":917504,"data_stored":429687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.45100000000000001,"15min":0.46300000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.59699999999999998,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.47199999999999998,"15min":0.48199999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.28199999999999997}]},{"osd":1,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.59199999999999997,"15min":0.60599999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.61799999999999999,"15min":0.63300000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76100000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.433}]},{"osd":2,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.63200000000000001,"15min":0.64600000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.78900000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.63500000000000001,"15min":0.64300000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.46400000000000002}]},{"osd":3,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.54800000000000004,"15min":0.56399999999999995},"min":{"1min":0.33200000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73299999999999998,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.55100000000000005,"15min":0.57299999999999995},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.70899999999999996,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.39600000000000002}]},{"osd":4,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.66200000000000003,"15min":0.68100000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.873,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.67300000000000004,"15min":0.68999999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.83599999999999997,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.45500000000000002}]},{"osd":5,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.69599999999999995,"15min":0.70999999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.878,"5min":1.056,"15min":1.056},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.72099999999999997,"15min":0.73899999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85999999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.48299999999999998}]},{"osd":6,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.64900000000000002,"15min":0.67200000000000004},"min":{"1min":0.38400000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.81699999999999995,"5min":1.069,"15min":1.069},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.70399999999999996,"15min":0.72799999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85099999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.503}]},{"osd":7,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.755,"15min":0.77300000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92200000000000004,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.77100000000000002,"15min":0.79500000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.89600000000000002,"5min":1.111,"15min":1.111},"last":0.55200000000000005}]},{"osd":8,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.80400000000000005,"15min":0.82499999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.95899999999999996,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.80100000000000005,"15min":0.81599999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.97399999999999998,"5min":1.089,"15min":1.089},"last":0.58199999999999996}]},{"osd":10,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.53700000000000003}]}]},{"osd":8,"up_from":59,"seq":253403070499,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6280,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738872,"statfs":{"total":95995035648,"available":95988604928,"internally_reserved":0,"allocated":917504,"data_stored":429607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.77000000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.439},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.44400000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.46999999999999997}]},{"osd":1,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.627,"15min":0.61699999999999999},"min":{"1min":0.46100000000000002,"5min":0.318,"15min":0.318},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.749},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.46200000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.76700000000000002}]},{"osd":2,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.50600000000000001,"15min":0.51100000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71299999999999997,"5min":1.056,"15min":1.056},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.51500000000000001,"15min":0.52000000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71499999999999997,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.70499999999999996}]},{"osd":3,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.624,"15min":0.625},"min":{"1min":0.40999999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.75800000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.433,"5min":0.376,"15min":0.376},"max":{"1min":0.877,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.60599999999999998}]},{"osd":4,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.65800000000000003,"15min":0.63300000000000001},"min":{"1min":0.47099999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":4.0599999999999996,"5min":4.0599999999999996,"15min":4.0599999999999996},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69899999999999995,"15min":0.69799999999999995},"min":{"1min":0.42899999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.79400000000000004}]},{"osd":5,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60999999999999999,"15min":0.61199999999999999},"min":{"1min":0.33500000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.89000000000000001,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.624,"15min":0.63},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.84599999999999997,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.65100000000000002}]},{"osd":6,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.747,"15min":0.745},"min":{"1min":0.45800000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94599999999999995,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.745,"15min":0.75},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94299999999999995,"5min":1.173,"15min":1.173},"last":0.61899999999999999}]},{"osd":7,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.76800000000000002,"15min":0.76400000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.75700000000000001,"15min":0.753},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.81299999999999994}]},{"osd":9,"last update":"Thu Nov 16 16:37:55 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.73799999999999999,"15min":0.72999999999999998},"min":{"1min":0.54000000000000004,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.018,"5min":1.036,"15min":1.036},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.875,"5min":0.76900000000000002,"15min":0.751},"min":{"1min":0.58699999999999997,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.84599999999999997}]},{"osd":10,"last update":"Thu Nov 16 16:37:13 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":3.5720000000000001,"5min":3.5720000000000001,"15min":3.5720000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":3.5920000000000001,"5min":3.5920000000000001,"15min":3.5920000000000001},"last":0.94599999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]}]},{"osd":7,"up_from":52,"seq":223338299430,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6392,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738760,"statfs":{"total":95995035648,"available":95988490240,"internally_reserved":0,"allocated":966656,"data_stored":1021446,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:36:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55800000000000005}]},{"osd":1,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.41799999999999998,"15min":0.41099999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.434,"15min":0.42599999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.48299999999999998}]},{"osd":2,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.35799999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.80800000000000005}]},{"osd":3,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57499999999999996,"15min":0.57099999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.625,"15min":0.621},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.41699999999999998}]},{"osd":4,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.52400000000000002,"15min":0.51400000000000001},"min":{"1min":0.34799999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.498,"15min":0.48199999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.379}]},{"osd":5,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.65200000000000002,"15min":0.64700000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91300000000000003,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.63800000000000001,"15min":0.63200000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.90100000000000002}]},{"osd":6,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.70299999999999996,"15min":0.69499999999999995},"min":{"1min":0.52500000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.75,"5min":0.68600000000000005,"15min":0.67600000000000005},"min":{"1min":0.47299999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.51100000000000001}]},{"osd":8,"last update":"Thu Nov 16 16:37:25 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.748,"15min":0.747},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0580000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.752,"5min":0.76100000000000001,"15min":0.76300000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.72499999999999998}]},{"osd":9,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.80300000000000005,"15min":0.78200000000000003},"min":{"1min":0.57999999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":4.7370000000000001,"5min":4.7370000000000001,"15min":4.7370000000000001},"last":0.75},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.84399999999999997,"15min":0.82199999999999995},"min":{"1min":0.55600000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":4.7809999999999997,"5min":4.7809999999999997,"15min":4.7809999999999997},"last":0.76700000000000002}]},{"osd":10,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.84699999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]}]},{"osd":6,"up_from":47,"seq":201863462954,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6204,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":93738948,"statfs":{"total":95995035648,"available":95988682752,"internally_reserved":0,"allocated":905216,"data_stored":429082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.439,"15min":0.41999999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":2.3140000000000001,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.47399999999999998,"15min":0.44600000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":3.472,"5min":3.472,"15min":3.472},"last":0.56499999999999995}]},{"osd":1,"last update":"Thu Nov 16 16:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.59699999999999998,"15min":0.58299999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":3.3290000000000002,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.64600000000000002,"15min":0.59999999999999998},"min":{"1min":0.38200000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":4.4669999999999996,"5min":4.4669999999999996,"15min":4.4669999999999996},"last":0.65000000000000002}]},{"osd":2,"last update":"Thu Nov 16 16:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.65400000000000003,"15min":0.63800000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":3.5369999999999999,"5min":3.5369999999999999,"15min":3.5369999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.66200000000000003,"15min":0.63400000000000001},"min":{"1min":0.46400000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.4049999999999998,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.48999999999999999}]},{"osd":3,"last update":"Thu Nov 16 16:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.52400000000000002,"15min":0.48999999999999999},"min":{"1min":0.30499999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":3.3700000000000001,"5min":3.3700000000000001,"15min":3.3700000000000001},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.55600000000000005,"15min":0.54100000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.35099999999999998}]},{"osd":4,"last update":"Thu Nov 16 16:37:50 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.67600000000000005,"15min":0.60999999999999999},"min":{"1min":0.45300000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":4.4900000000000002,"5min":4.4900000000000002,"15min":4.4900000000000002},"last":0.58599999999999997},{"interface":"front","average":{"1min":1.026,"5min":0.69899999999999995,"15min":0.63100000000000001},"min":{"1min":0.45200000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":4.4509999999999996,"5min":4.4509999999999996,"15min":4.4509999999999996},"last":0.63100000000000001}]},{"osd":5,"last update":"Thu Nov 16 16:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.68899999999999995,"15min":0.68100000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":2.4209999999999998,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":0.443},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.72999999999999998,"15min":0.70999999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":3.5019999999999998,"5min":3.5019999999999998,"15min":3.5019999999999998},"last":0.53600000000000003}]},{"osd":7,"last update":"Thu Nov 16 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.70499999999999996,"15min":0.70299999999999996},"min":{"1min":0.48999999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.878,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":1.1200000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.71499999999999997,"15min":0.70899999999999996},"min":{"1min":0.55300000000000005,"5min":0.436,"15min":0.436},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":1.111}]},{"osd":8,"last update":"Thu Nov 16 16:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.76100000000000001,"15min":0.77200000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.91200000000000003,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.77000000000000002,"15min":0.78000000000000003},"min":{"1min":0.54000000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.016,"5min":1.091,"15min":1.091},"last":0.503}]},{"osd":9,"last update":"Thu Nov 16 16:37:50 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":0.81799999999999995,"15min":0.78200000000000003},"min":{"1min":0.50800000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":6.1539999999999999,"5min":6.1539999999999999,"15min":6.1539999999999999},"last":0.40500000000000003},{"interface":"front","average":{"1min":1.1910000000000001,"5min":0.83999999999999997,"15min":0.78200000000000003},"min":{"1min":0.55100000000000005,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":8.2460000000000004,"5min":8.2460000000000004,"15min":8.2460000000000004},"last":0.42499999999999999}]},{"osd":10,"last update":"Thu Nov 16 16:37:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.68500000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7288,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":6336,"kb_avail":93737864,"statfs":{"total":95995035648,"available":95987572736,"internally_reserved":0,"allocated":966656,"data_stored":487396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6488064},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.48499999999999999,"15min":0.45800000000000002},"min":{"1min":0.33100000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.51200000000000001,"15min":0.48499999999999999},"min":{"1min":0.32000000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.42699999999999999}]},{"osd":2,"last update":"Thu Nov 16 16:37:19 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.54300000000000004,"15min":0.498},"min":{"1min":0.36699999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.84799999999999998,"5min":0.876,"15min":0.876},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.54000000000000004,"15min":0.503},"min":{"1min":0.40500000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.91000000000000003,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.59799999999999998}]},{"osd":3,"last update":"Thu Nov 16 16:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57699999999999996,"15min":0.55500000000000005},"min":{"1min":0.372,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85599999999999998,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59899999999999998,"15min":0.57499999999999996},"min":{"1min":0.41399999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Nov 16 16:37:03 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.441,"15min":0.42299999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.52200000000000002,"15min":0.47599999999999998},"min":{"1min":0.34599999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.81699999999999995}]},{"osd":5,"last update":"Thu Nov 16 16:37:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.63800000000000001,"15min":0.61099999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.621,"15min":0.59699999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97399999999999998,"5min":1.004,"15min":1.004},"last":0.85899999999999999}]},{"osd":6,"last update":"Thu Nov 16 16:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.68400000000000005,"15min":0.66800000000000004},"min":{"1min":0.44700000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.95099999999999996,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68400000000000005,"15min":0.66400000000000003},"min":{"1min":0.45500000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.95299999999999996,"5min":1,"15min":1},"last":0.627}]},{"osd":7,"last update":"Thu Nov 16 16:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.64500000000000002,"15min":0.63200000000000001},"min":{"1min":0.35199999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.69299999999999995,"15min":0.67700000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.64000000000000001}]},{"osd":8,"last update":"Thu Nov 16 16:37:27 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.75900000000000001,"15min":0.755},"min":{"1min":0.54800000000000004,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.74099999999999999,"15min":0.73699999999999999},"min":{"1min":0.47399999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.754}]},{"osd":9,"last update":"Thu Nov 16 16:37:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.77900000000000003,"15min":0.79900000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.87,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.84599999999999997,"15min":0.86399999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.97299999999999998,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.68600000000000005}]},{"osd":10,"last update":"Thu Nov 16 16:37:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.66100000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]}]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7184,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":6272,"kb_avail":93737968,"statfs":{"total":95995035648,"available":95987679232,"internally_reserved":0,"allocated":925696,"data_stored":430347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6422528},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Nov 16 16:37:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.503,"15min":0.46700000000000003},"min":{"1min":0.32500000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.499,"15min":0.46100000000000002},"min":{"1min":0.36099999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78800000000000003,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.66000000000000003}]},{"osd":2,"last update":"Thu Nov 16 16:37:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.55600000000000005,"15min":0.51400000000000001},"min":{"1min":0.41999999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.80600000000000005,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.496},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56299999999999994,"15min":0.53100000000000003},"min":{"1min":0.372,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.70399999999999996}]},{"osd":3,"last update":"Thu Nov 16 16:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.46500000000000002,"15min":0.45600000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.45700000000000002,"15min":0.44700000000000001},"min":{"1min":0.33700000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.35599999999999998}]},{"osd":4,"last update":"Thu Nov 16 16:37:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.57899999999999996,"15min":0.55100000000000005},"min":{"1min":0.38900000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.58199999999999996,"15min":0.55000000000000004},"min":{"1min":0.40999999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.61699999999999999}]},{"osd":5,"last update":"Thu Nov 16 16:37:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.56200000000000006,"15min":0.55100000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.66100000000000003,"5min":1.012,"15min":1.012},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64600000000000002,"15min":0.621},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.82699999999999996,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.437}]},{"osd":6,"last update":"Thu Nov 16 16:37:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.64800000000000002,"15min":0.59699999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":6.5190000000000001,"5min":6.5190000000000001,"15min":6.5190000000000001},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.63100000000000001,"15min":0.57499999999999996},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":9.141,"5min":9.141,"15min":9.141},"last":0.46000000000000002}]},{"osd":7,"last update":"Thu Nov 16 16:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.68400000000000005,"15min":0.67600000000000005},"min":{"1min":0.495,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.74199999999999999,"15min":0.73199999999999998},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.53400000000000003}]},{"osd":8,"last update":"Thu Nov 16 16:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.78800000000000003,"15min":0.80900000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.91900000000000004,"5min":1.163,"15min":1.163},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.83499999999999996,"15min":0.85299999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.90800000000000003,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.57099999999999995}]},{"osd":9,"last update":"Thu Nov 16 16:37:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.85499999999999998,"15min":0.86199999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":6.6349999999999998,"5min":6.6349999999999998,"15min":6.6349999999999998},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.79000000000000004,"15min":0.76600000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":9.1690000000000005,"5min":9.1690000000000005,"15min":9.1690000000000005},"last":0.51600000000000001}]},{"osd":10,"last update":"Thu Nov 16 16:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.69399999999999995}]},{"osd":11,"last update":"Thu Nov 16 16:37:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":9.1039999999999992,"5min":9.1039999999999992,"15min":9.1039999999999992},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":3.7530000000000001,"5min":3.7530000000000001,"15min":3.7530000000000001},"last":0.64500000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7376,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":93737776,"statfs":{"total":95995035648,"available":95987482624,"internally_reserved":0,"allocated":991232,"data_stored":489088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.29999999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.497,"15min":0.48799999999999999},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.56399999999999995}]},{"osd":1,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52600000000000002,"15min":0.51400000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.40500000000000003,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.89900000000000002,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.69599999999999995}]},{"osd":3,"last update":"Thu Nov 16 16:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.66300000000000003,"15min":0.60699999999999998},"min":{"1min":0.40699999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":6.9740000000000002,"5min":6.9740000000000002,"15min":6.9740000000000002},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.57499999999999996,"15min":0.55700000000000005},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.46600000000000003}]},{"osd":4,"last update":"Thu Nov 16 16:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.55300000000000005,"15min":0.53000000000000003},"min":{"1min":0.376,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59099999999999997,"15min":0.58199999999999996},"min":{"1min":0.40000000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83799999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.53300000000000003}]},{"osd":5,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.49399999999999999,"15min":0.48399999999999999},"min":{"1min":0.315,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.47199999999999998,"15min":0.45900000000000002},"min":{"1min":0.32200000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.62}]},{"osd":6,"last update":"Thu Nov 16 16:37:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.68000000000000005,"15min":0.65400000000000003},"min":{"1min":0.40899999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.3969999999999998,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.64400000000000002,"15min":0.61299999999999999},"min":{"1min":0.39400000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":2.3460000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.64700000000000002}]},{"osd":7,"last update":"Thu Nov 16 16:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.64300000000000002,"15min":0.64400000000000002},"min":{"1min":0.41599999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70199999999999996,"15min":0.70399999999999996},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.92700000000000005,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.57699999999999996}]},{"osd":8,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.57499999999999996,"15min":0.56899999999999995},"min":{"1min":0.38500000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.85199999999999998,"5min":1.008,"15min":1.008},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.628,"15min":0.625},"min":{"1min":0.38400000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.753}]},{"osd":9,"last update":"Thu Nov 16 16:37:54 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.86799999999999999,"15min":0.86699999999999999},"min":{"1min":0.47499999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.6080000000000001,"5min":4.524,"15min":4.524},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.751,"15min":0.73299999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.6200000000000001,"5min":2.6200000000000001,"15min":2.6200000000000001},"last":0.72399999999999998}]},{"osd":10,"last update":"Thu Nov 16 16:37:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.78700000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]}]},{"osd":3,"up_from":26,"seq":111669149751,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6536,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738616,"statfs":{"total":95995035648,"available":95988342784,"internally_reserved":0,"allocated":917504,"data_stored":429068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36499999999999999,"15min":0.34899999999999998},"min":{"1min":0.26500000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.52400000000000002,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.38,"15min":0.35799999999999998},"min":{"1min":0.27800000000000002,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.55100000000000005,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.39600000000000002}]},{"osd":1,"last update":"Thu Nov 16 16:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.55400000000000005,"15min":0.54900000000000004},"min":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55900000000000005,"15min":0.55700000000000005},"min":{"1min":0.36099999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.83399999999999996,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.63200000000000001}]},{"osd":2,"last update":"Thu Nov 16 16:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57899999999999996,"15min":0.55700000000000005},"min":{"1min":0.42599999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.83299999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56699999999999995,"15min":0.54700000000000004},"min":{"1min":0.40600000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.69699999999999995,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.65100000000000002}]},{"osd":4,"last update":"Thu Nov 16 16:37:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55700000000000005,"15min":0.54700000000000004},"min":{"1min":0.376,"5min":0.37,"15min":0.37},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.45500000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.61399999999999999}]},{"osd":5,"last update":"Thu Nov 16 16:37:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.59799999999999998,"15min":0.57999999999999996},"min":{"1min":0.438,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60499999999999998,"15min":0.59099999999999997},"min":{"1min":0.38600000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.87,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.67300000000000004}]},{"osd":6,"last update":"Thu Nov 16 16:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51800000000000002,"15min":0.498},"min":{"1min":0.35599999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.72399999999999998,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.496},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53200000000000003,"15min":0.51400000000000001},"min":{"1min":0.377,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.75800000000000001,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.50600000000000001}]},{"osd":7,"last update":"Thu Nov 16 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.65600000000000003,"15min":0.65000000000000002},"min":{"1min":0.44,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.69299999999999995,"15min":0.68700000000000006},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.72899999999999998}]},{"osd":8,"last update":"Thu Nov 16 16:36:27 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.63200000000000001}]},{"osd":9,"last update":"Thu Nov 16 16:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.73799999999999999,"15min":0.76300000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80500000000000005,"5min":1.024,"15min":1.024},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.68999999999999995,"15min":0.70899999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90300000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.54000000000000004}]},{"osd":10,"last update":"Thu Nov 16 16:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.70099999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]}]},{"osd":4,"up_from":34,"seq":146028888115,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6336,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738816,"statfs":{"total":95995035648,"available":95988547584,"internally_reserved":0,"allocated":909312,"data_stored":429465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.52400000000000002,"15min":0.502},"min":{"1min":0.38200000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.52900000000000003,"15min":0.51500000000000001},"min":{"1min":0.40899999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.378}]},{"osd":1,"last update":"Thu Nov 16 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.39400000000000002,"15min":0.38700000000000001},"min":{"1min":0.27900000000000003,"5min":0.253,"15min":0.253},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.41199999999999998,"15min":0.40300000000000002},"min":{"1min":0.29699999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.442}]},{"osd":2,"last update":"Thu Nov 16 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.442,"5min":0.371,"15min":0.371},"max":{"1min":0.79200000000000004,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.60499999999999998,"15min":0.58699999999999997},"min":{"1min":0.44600000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.90200000000000002,"5min":1.056,"15min":1.056},"last":0.58499999999999996}]},{"osd":3,"last update":"Thu Nov 16 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.56299999999999994,"15min":0.54000000000000004},"min":{"1min":0.39000000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.56100000000000005,"15min":0.54900000000000004},"min":{"1min":0.40500000000000003,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.55000000000000004}]},{"osd":5,"last update":"Thu Nov 16 16:37:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.623,"15min":0.627},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.92100000000000004,"5min":1.075,"15min":1.075},"last":0.624},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.63800000000000001,"15min":0.626},"min":{"1min":0.52200000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.94499999999999995,"5min":1.093,"15min":1.093},"last":0.64800000000000002}]},{"osd":6,"last update":"Thu Nov 16 16:37:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.63,"15min":0.60199999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84799999999999998,"5min":1.004,"15min":1.004},"last":0.749},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.60899999999999999,"15min":0.59499999999999997},"min":{"1min":0.42399999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.66900000000000004}]},{"osd":7,"last update":"Thu Nov 16 16:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.57199999999999995,"15min":0.54800000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.63200000000000001,"15min":0.60799999999999998},"min":{"1min":0.41199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.86299999999999999}]},{"osd":8,"last update":"Thu Nov 16 16:37:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.73999999999999999,"15min":0.72399999999999998},"min":{"1min":0.58999999999999997,"5min":0.495,"15min":0.495},"max":{"1min":1.0189999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.747,"15min":0.73099999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.80100000000000005}]},{"osd":9,"last update":"Thu Nov 16 16:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.79200000000000004,"15min":0.78900000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.94099999999999995,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.72099999999999997,"15min":0.71799999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94199999999999995,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.71499999999999997}]},{"osd":10,"last update":"Thu Nov 16 16:37:15 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.83599999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]}]},{"osd":5,"up_from":39,"seq":167503724592,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6532,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738620,"statfs":{"total":95995035648,"available":95988346880,"internally_reserved":0,"allocated":978944,"data_stored":1021178,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.499,"15min":0.49399999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.64600000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52800000000000002,"15min":0.52400000000000002},"min":{"1min":0.312,"5min":0.311,"15min":0.311},"max":{"1min":0.75700000000000001,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.61099999999999999}]},{"osd":1,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.372,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.86399999999999999,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58799999999999997,"15min":0.58499999999999996},"min":{"1min":0.43099999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.94699999999999995,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.53800000000000003}]},{"osd":2,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.42799999999999999,"15min":0.42399999999999999},"min":{"1min":0.29599999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.67300000000000004,"5min":1.417,"15min":1.417},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.443,"15min":0.433},"min":{"1min":0.32300000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.69899999999999995,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.443}]},{"osd":3,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55700000000000005,"15min":0.54700000000000004},"min":{"1min":0.36199999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.81100000000000005,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.66100000000000003}]},{"osd":4,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.67900000000000005,"15min":0.629},"min":{"1min":0.40100000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":5.5789999999999997,"5min":5.5789999999999997,"15min":5.5789999999999997},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59699999999999998,"15min":0.59099999999999997},"min":{"1min":0.40400000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.55700000000000005}]},{"osd":6,"last update":"Thu Nov 16 16:37:47 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.67800000000000005,"15min":0.63600000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":3.274,"5min":3.274,"15min":3.274},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.68100000000000005,"15min":0.64600000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":3.472,"5min":3.472,"15min":3.472},"last":0.40100000000000002}]},{"osd":7,"last update":"Thu Nov 16 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.70999999999999996,"15min":0.71199999999999997},"min":{"1min":0.503,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.83799999999999997,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.72499999999999998,"15min":0.72799999999999998},"min":{"1min":0.56599999999999995,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.83699999999999997,"5min":1.544,"15min":1.544},"last":0.68899999999999995}]},{"osd":8,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.64600000000000002,"15min":0.64500000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72199999999999998,"15min":0.72399999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.76300000000000001}]},{"osd":9,"last update":"Thu Nov 16 16:37:52 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":0.90100000000000002,"15min":0.82699999999999996},"min":{"1min":0.58699999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":5.5679999999999996,"5min":5.5679999999999996,"15min":5.5679999999999996},"last":0.52400000000000002},{"interface":"front","average":{"1min":1.099,"5min":0.81599999999999995,"15min":0.76900000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":5.3230000000000004,"5min":5.3230000000000004,"15min":5.3230000000000004},"last":0.57599999999999996}]},{"osd":10,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.748}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33928,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":58527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-16T16:38:04.450 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph pg dump --format=json 2023-11-16T16:38:04.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:04 smithi026 ceph-mon[100112]: osdmap e86: 12 total, 12 up, 12 in 2023-11-16T16:38:04.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:04 smithi026 ceph-mon[100112]: pgmap v287: 97 pgs: 2 peering, 95 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2023-11-16T16:38:04.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:04 smithi053 ceph-mon[100638]: osdmap e86: 12 total, 12 up, 12 in 2023-11-16T16:38:04.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:04 smithi053 ceph-mon[100638]: pgmap v287: 97 pgs: 2 peering, 95 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2023-11-16T16:38:05.106 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:38:05.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:05 smithi026 ceph-mon[100112]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:38:05.512 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:05 smithi001 ceph-mon[98800]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:38:05.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:05 smithi053 ceph-mon[100638]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:38:06.432 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.427+0000 7f0bf3c27700 1 -- 172.21.15.1:0/2064710699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bec100f00 msgr2=0x7f0bec104ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:06.432 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.427+0000 7f0bf3c27700 1 --2- 172.21.15.1:0/2064710699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bec100f00 0x7f0bec104ed0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0be80099c0 tx=0x7f0be8009cd0 comp rx=0 tx=0).stop 2023-11-16T16:38:06.432 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.427+0000 7f0bf3c27700 1 -- 172.21.15.1:0/2064710699 shutdown_connections 2023-11-16T16:38:06.432 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.427+0000 7f0bf3c27700 1 --2- 172.21.15.1:0/2064710699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bec105600 0x7f0bec10d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:06.432 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.427+0000 7f0bf3c27700 1 --2- 172.21.15.1:0/2064710699 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bec100560 0x7f0bec100930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:06.432 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.427+0000 7f0bf3c27700 1 --2- 172.21.15.1:0/2064710699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bec100f00 0x7f0bec104ed0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:06.432 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.427+0000 7f0bf3c27700 1 -- 172.21.15.1:0/2064710699 >> 172.21.15.1:0/2064710699 conn(0x7f0bec0fbde0 msgr2=0x7f0bec0fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:06.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.427+0000 7f0bf3c27700 1 -- 172.21.15.1:0/2064710699 shutdown_connections 2023-11-16T16:38:06.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.427+0000 7f0bf3c27700 1 -- 172.21.15.1:0/2064710699 wait complete. 2023-11-16T16:38:06.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.428+0000 7f0bf3c27700 1 Processor -- start 2023-11-16T16:38:06.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.428+0000 7f0bf3c27700 1 -- start start 2023-11-16T16:38:06.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.428+0000 7f0bf3c27700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bec100560 0x7f0bec196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:06.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.428+0000 7f0bf3c27700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bec100f00 0x7f0bec196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:06.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.428+0000 7f0bf3c27700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bec105600 0x7f0bec19ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:06.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.428+0000 7f0bf3c27700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec19b410 con 0x7f0bec105600 2023-11-16T16:38:06.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.428+0000 7f0bf3c27700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec19b580 con 0x7f0bec100f00 2023-11-16T16:38:06.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.428+0000 7f0bf3c27700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec19b740 con 0x7f0bec100560 2023-11-16T16:38:06.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.428+0000 7f0bf21c4700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bec105600 0x7f0bec19ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:06.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.428+0000 7f0bf21c4700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bec105600 0x7f0bec19ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42090/0 (socket says 172.21.15.1:42090) 2023-11-16T16:38:06.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.428+0000 7f0bf21c4700 1 -- 172.21.15.1:0/2512706841 learned_addr learned my addr 172.21.15.1:0/2512706841 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:06.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.428+0000 7f0bf11c2700 1 --2- 172.21.15.1:0/2512706841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bec100f00 0x7f0bec196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:06.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.428+0000 7f0bf19c3700 1 --2- 172.21.15.1:0/2512706841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bec100560 0x7f0bec196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:06.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.429+0000 7f0bf21c4700 1 -- 172.21.15.1:0/2512706841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bec100560 msgr2=0x7f0bec196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:06.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.429+0000 7f0bf21c4700 1 --2- 172.21.15.1:0/2512706841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bec100560 0x7f0bec196450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:06.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.429+0000 7f0bf21c4700 1 -- 172.21.15.1:0/2512706841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bec100f00 msgr2=0x7f0bec196990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:06.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.429+0000 7f0bf21c4700 1 --2- 172.21.15.1:0/2512706841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bec100f00 0x7f0bec196990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:06.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.429+0000 7f0bf21c4700 1 -- 172.21.15.1:0/2512706841 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be80095f0 con 0x7f0bec105600 2023-11-16T16:38:06.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.429+0000 7f0bf21c4700 1 --2- 172.21.15.1:0/2512706841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bec105600 0x7f0bec19ae00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0be400ada0 tx=0x7f0be400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:06.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.429+0000 7f0be2ffd700 1 -- 172.21.15.1:0/2512706841 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be400f070 con 0x7f0bec105600 2023-11-16T16:38:06.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.429+0000 7f0be2ffd700 1 -- 172.21.15.1:0/2512706841 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0be4007b30 con 0x7f0bec105600 2023-11-16T16:38:06.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.429+0000 7f0bf3c27700 1 -- 172.21.15.1:0/2512706841 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bec110330 con 0x7f0bec105600 2023-11-16T16:38:06.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.429+0000 7f0be2ffd700 1 -- 172.21.15.1:0/2512706841 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be40053e0 con 0x7f0bec105600 2023-11-16T16:38:06.436 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.429+0000 7f0bf3c27700 1 -- 172.21.15.1:0/2512706841 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bec1101a0 con 0x7f0bec105600 2023-11-16T16:38:06.436 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.430+0000 7f0bf3c27700 1 -- 172.21.15.1:0/2512706841 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bec1104c0 con 0x7f0bec105600 2023-11-16T16:38:06.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.431+0000 7f0be2ffd700 1 -- 172.21.15.1:0/2512706841 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f0be400c980 con 0x7f0bec105600 2023-11-16T16:38:06.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.432+0000 7f0be2ffd700 1 --2- 172.21.15.1:0/2512706841 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f0bd80a11e0 0x7f0bd80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:06.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.432+0000 7f0bf19c3700 1 --2- 172.21.15.1:0/2512706841 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f0bd80a11e0 0x7f0bd80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:06.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.432+0000 7f0be2ffd700 1 -- 172.21.15.1:0/2512706841 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0be40cc5b0 con 0x7f0bec105600 2023-11-16T16:38:06.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.432+0000 7f0bf19c3700 1 --2- 172.21.15.1:0/2512706841 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f0bd80a11e0 0x7f0bd80a3690 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc005fd0 tx=0x7f0bdc005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:06.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.435+0000 7f0be2ffd700 1 -- 172.21.15.1:0/2512706841 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0be4095c30 con 0x7f0bec105600 2023-11-16T16:38:06.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:06 smithi026 ceph-mon[100112]: pgmap v288: 97 pgs: 2 peering, 95 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2023-11-16T16:38:06.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:06 smithi053 ceph-mon[100638]: pgmap v288: 97 pgs: 2 peering, 95 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2023-11-16T16:38:06.627 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.624+0000 7f0bf3c27700 1 -- 172.21.15.1:0/2512706841 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0bec0fcf10 con 0x7f0bd80a11e0 2023-11-16T16:38:06.638 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.636+0000 7f0be2ffd700 1 -- 172.21.15.1:0/2512706841 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340117 (secure 0 0 0) 0x7f0bec0fcf10 con 0x7f0bd80a11e0 2023-11-16T16:38:06.644 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:38:06.647 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.639+0000 7f0bf3c27700 1 -- 172.21.15.1:0/2512706841 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f0bd80a11e0 msgr2=0x7f0bd80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:06.648 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.639+0000 7f0bf3c27700 1 --2- 172.21.15.1:0/2512706841 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f0bd80a11e0 0x7f0bd80a3690 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc005fd0 tx=0x7f0bdc005f60 comp rx=0 tx=0).stop 2023-11-16T16:38:06.648 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.639+0000 7f0bf3c27700 1 -- 172.21.15.1:0/2512706841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bec105600 msgr2=0x7f0bec19ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:06.648 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.639+0000 7f0bf3c27700 1 --2- 172.21.15.1:0/2512706841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bec105600 0x7f0bec19ae00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0be400ada0 tx=0x7f0be400c330 comp rx=0 tx=0).stop 2023-11-16T16:38:06.648 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.639+0000 7f0bf3c27700 1 -- 172.21.15.1:0/2512706841 shutdown_connections 2023-11-16T16:38:06.648 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.639+0000 7f0bf3c27700 1 --2- 172.21.15.1:0/2512706841 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f0bd80a11e0 0x7f0bd80a3690 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:06.648 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.639+0000 7f0bf3c27700 1 --2- 172.21.15.1:0/2512706841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bec105600 0x7f0bec19ae00 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:06.648 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.639+0000 7f0bf3c27700 1 --2- 172.21.15.1:0/2512706841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bec100560 0x7f0bec196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:06.648 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.639+0000 7f0bf3c27700 1 --2- 172.21.15.1:0/2512706841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bec100f00 0x7f0bec196990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:06.649 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.639+0000 7f0bf3c27700 1 -- 172.21.15.1:0/2512706841 >> 172.21.15.1:0/2512706841 conn(0x7f0bec0fbde0 msgr2=0x7f0bec0fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:06.649 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.640+0000 7f0bf3c27700 1 -- 172.21.15.1:0/2512706841 shutdown_connections 2023-11-16T16:38:06.649 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:06.640+0000 7f0bf3c27700 1 -- 172.21.15.1:0/2512706841 wait complete. 2023-11-16T16:38:06.649 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2023-11-16T16:38:06.664 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:06 smithi001 ceph-mon[98800]: pgmap v288: 97 pgs: 2 peering, 95 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2023-11-16T16:38:07.052 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2023-11-16T16:38:05.483781+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653045,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":10,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":440,"num_read_kb":336,"num_write":333,"num_write_kb":3043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":286,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":79044,"kb_used_data":11236,"kb_used_omap":0,"kb_used_meta":67712,"kb_avail":1124862780,"statfs":{"total":1151940427776,"available":1151859486720,"internally_reserved":0,"allocated":11505664,"data_stored":7218350,"data_compressed":144476,"data_compressed_allocated":188416,"data_compressed_original":1875968,"omap_allocated":0,"internal_metadata":69337088},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":10,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":29,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001724"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-16T16:38:02.188094+0000","last_change":"2023-11-16T16:37:13.941551+0000","last_active":"2023-11-16T16:38:02.188094+0000","last_peered":"2023-11-16T16:38:02.188094+0000","last_clean":"2023-11-16T16:38:02.188094+0000","last_became_active":"2023-11-16T16:37:13.941190+0000","last_became_peered":"2023-11-16T16:37:13.941190+0000","last_unstale":"2023-11-16T16:38:02.188094+0000","last_undegraded":"2023-11-16T16:38:02.188094+0000","last_fullsized":"2023-11-16T16:38:02.188094+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T16:38:21.389028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158170+0000","last_change":"2023-11-16T16:37:13.942300+0000","last_active":"2023-11-16T16:37:15.158170+0000","last_peered":"2023-11-16T16:37:15.158170+0000","last_clean":"2023-11-16T16:37:15.158170+0000","last_became_active":"2023-11-16T16:37:13.941966+0000","last_became_peered":"2023-11-16T16:37:13.941966+0000","last_unstale":"2023-11-16T16:37:15.158170+0000","last_undegraded":"2023-11-16T16:37:15.158170+0000","last_fullsized":"2023-11-16T16:37:15.158170+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:12:01.659205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158388+0000","last_change":"2023-11-16T16:37:13.943461+0000","last_active":"2023-11-16T16:37:15.158388+0000","last_peered":"2023-11-16T16:37:15.158388+0000","last_clean":"2023-11-16T16:37:15.158388+0000","last_became_active":"2023-11-16T16:37:13.943165+0000","last_became_peered":"2023-11-16T16:37:13.943165+0000","last_unstale":"2023-11-16T16:37:15.158388+0000","last_undegraded":"2023-11-16T16:37:15.158388+0000","last_fullsized":"2023-11-16T16:37:15.158388+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:09:30.792766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184244+0000","last_change":"2023-11-16T16:37:13.940205+0000","last_active":"2023-11-16T16:38:03.184244+0000","last_peered":"2023-11-16T16:38:03.184244+0000","last_clean":"2023-11-16T16:38:03.184244+0000","last_became_active":"2023-11-16T16:37:13.939819+0000","last_became_peered":"2023-11-16T16:37:13.939819+0000","last_unstale":"2023-11-16T16:38:03.184244+0000","last_undegraded":"2023-11-16T16:38:03.184244+0000","last_fullsized":"2023-11-16T16:38:03.184244+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:52:42.183240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.923975+0000","last_change":"2023-11-16T16:37:13.939819+0000","last_active":"2023-11-16T16:37:14.923975+0000","last_peered":"2023-11-16T16:37:14.923975+0000","last_clean":"2023-11-16T16:37:14.923975+0000","last_became_active":"2023-11-16T16:37:13.939367+0000","last_became_peered":"2023-11-16T16:37:13.939367+0000","last_unstale":"2023-11-16T16:37:14.923975+0000","last_undegraded":"2023-11-16T16:37:14.923975+0000","last_fullsized":"2023-11-16T16:37:14.923975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:27:49.657269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.187402+0000","last_change":"2023-11-16T16:37:13.946613+0000","last_active":"2023-11-16T16:38:03.187402+0000","last_peered":"2023-11-16T16:38:03.187402+0000","last_clean":"2023-11-16T16:38:03.187402+0000","last_became_active":"2023-11-16T16:37:13.946281+0000","last_became_peered":"2023-11-16T16:37:13.946281+0000","last_unstale":"2023-11-16T16:38:03.187402+0000","last_undegraded":"2023-11-16T16:38:03.187402+0000","last_fullsized":"2023-11-16T16:38:03.187402+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:29:42.599960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184741+0000","last_change":"2023-11-16T16:37:13.936899+0000","last_active":"2023-11-16T16:38:03.184741+0000","last_peered":"2023-11-16T16:38:03.184741+0000","last_clean":"2023-11-16T16:38:03.184741+0000","last_became_active":"2023-11-16T16:37:13.936543+0000","last_became_peered":"2023-11-16T16:37:13.936543+0000","last_unstale":"2023-11-16T16:38:03.184741+0000","last_undegraded":"2023-11-16T16:38:03.184741+0000","last_fullsized":"2023-11-16T16:38:03.184741+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:43:46.820689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.189989+0000","last_change":"2023-11-16T16:38:03.189467+0000","last_active":"2023-11-16T16:38:03.189989+0000","last_peered":"2023-11-16T16:38:03.189989+0000","last_clean":"2023-11-16T16:38:03.189989+0000","last_became_active":"2023-11-16T16:38:03.189017+0000","last_became_peered":"2023-11-16T16:38:03.189017+0000","last_unstale":"2023-11-16T16:38:03.189989+0000","last_undegraded":"2023-11-16T16:38:03.189989+0000","last_fullsized":"2023-11-16T16:38:03.189989+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:47:50.088436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157152+0000","last_change":"2023-11-16T16:37:13.934105+0000","last_active":"2023-11-16T16:37:15.157152+0000","last_peered":"2023-11-16T16:37:15.157152+0000","last_clean":"2023-11-16T16:37:15.157152+0000","last_became_active":"2023-11-16T16:37:13.933887+0000","last_became_peered":"2023-11-16T16:37:13.933887+0000","last_unstale":"2023-11-16T16:37:15.157152+0000","last_undegraded":"2023-11-16T16:37:15.157152+0000","last_fullsized":"2023-11-16T16:37:15.157152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:37:28.260833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.191855+0000","last_change":"2023-11-16T16:38:03.191855+0000","last_active":"2023-11-16T16:38:03.191855+0000","last_peered":"2023-11-16T16:38:03.191855+0000","last_clean":"2023-11-16T16:38:03.191855+0000","last_became_active":"2023-11-16T16:38:03.191398+0000","last_became_peered":"2023-11-16T16:38:03.191398+0000","last_unstale":"2023-11-16T16:38:03.191855+0000","last_undegraded":"2023-11-16T16:38:03.191855+0000","last_fullsized":"2023-11-16T16:38:03.191855+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:09:44.431927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184854+0000","last_change":"2023-11-16T16:37:13.941597+0000","last_active":"2023-11-16T16:38:03.184854+0000","last_peered":"2023-11-16T16:38:03.184854+0000","last_clean":"2023-11-16T16:38:03.184854+0000","last_became_active":"2023-11-16T16:37:13.941253+0000","last_became_peered":"2023-11-16T16:37:13.941253+0000","last_unstale":"2023-11-16T16:38:03.184854+0000","last_undegraded":"2023-11-16T16:38:03.184854+0000","last_fullsized":"2023-11-16T16:38:03.184854+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:13:35.256952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184130+0000","last_change":"2023-11-16T16:37:13.941673+0000","last_active":"2023-11-16T16:38:03.184130+0000","last_peered":"2023-11-16T16:38:03.184130+0000","last_clean":"2023-11-16T16:38:03.184130+0000","last_became_active":"2023-11-16T16:37:13.941345+0000","last_became_peered":"2023-11-16T16:37:13.941345+0000","last_unstale":"2023-11-16T16:38:03.184130+0000","last_undegraded":"2023-11-16T16:38:03.184130+0000","last_fullsized":"2023-11-16T16:38:03.184130+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:17:51.018141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184481+0000","last_change":"2023-11-16T16:37:13.947991+0000","last_active":"2023-11-16T16:38:03.184481+0000","last_peered":"2023-11-16T16:38:03.184481+0000","last_clean":"2023-11-16T16:38:03.184481+0000","last_became_active":"2023-11-16T16:37:13.947454+0000","last_became_peered":"2023-11-16T16:37:13.947454+0000","last_unstale":"2023-11-16T16:38:03.184481+0000","last_undegraded":"2023-11-16T16:38:03.184481+0000","last_fullsized":"2023-11-16T16:38:03.184481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:19:19.825458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157374+0000","last_change":"2023-11-16T16:37:13.941502+0000","last_active":"2023-11-16T16:37:15.157374+0000","last_peered":"2023-11-16T16:37:15.157374+0000","last_clean":"2023-11-16T16:37:15.157374+0000","last_became_active":"2023-11-16T16:37:13.941191+0000","last_became_peered":"2023-11-16T16:37:13.941191+0000","last_unstale":"2023-11-16T16:37:15.157374+0000","last_undegraded":"2023-11-16T16:37:15.157374+0000","last_fullsized":"2023-11-16T16:37:15.157374+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:59:03.243696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.156825+0000","last_change":"2023-11-16T16:37:13.949198+0000","last_active":"2023-11-16T16:37:15.156825+0000","last_peered":"2023-11-16T16:37:15.156825+0000","last_clean":"2023-11-16T16:37:15.156825+0000","last_became_active":"2023-11-16T16:37:13.947147+0000","last_became_peered":"2023-11-16T16:37:13.947147+0000","last_unstale":"2023-11-16T16:37:15.156825+0000","last_undegraded":"2023-11-16T16:37:15.156825+0000","last_fullsized":"2023-11-16T16:37:15.156825+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:01:56.205870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.156530+0000","last_change":"2023-11-16T16:37:13.943915+0000","last_active":"2023-11-16T16:37:15.156530+0000","last_peered":"2023-11-16T16:37:15.156530+0000","last_clean":"2023-11-16T16:37:15.156530+0000","last_became_active":"2023-11-16T16:37:13.942995+0000","last_became_peered":"2023-11-16T16:37:13.942995+0000","last_unstale":"2023-11-16T16:37:15.156530+0000","last_undegraded":"2023-11-16T16:37:15.156530+0000","last_fullsized":"2023-11-16T16:37:15.156530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:35:55.774014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-16T16:38:02.188250+0000","last_change":"2023-11-16T16:37:13.941816+0000","last_active":"2023-11-16T16:38:02.188250+0000","last_peered":"2023-11-16T16:38:02.188250+0000","last_clean":"2023-11-16T16:38:02.188250+0000","last_became_active":"2023-11-16T16:37:13.941541+0000","last_became_peered":"2023-11-16T16:37:13.941541+0000","last_unstale":"2023-11-16T16:38:02.188250+0000","last_undegraded":"2023-11-16T16:38:02.188250+0000","last_fullsized":"2023-11-16T16:38:02.188250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:53:58.693324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185550+0000","last_change":"2023-11-16T16:37:13.940639+0000","last_active":"2023-11-16T16:38:03.185550+0000","last_peered":"2023-11-16T16:38:03.185550+0000","last_clean":"2023-11-16T16:38:03.185550+0000","last_became_active":"2023-11-16T16:37:13.940323+0000","last_became_peered":"2023-11-16T16:37:13.940323+0000","last_unstale":"2023-11-16T16:38:03.185550+0000","last_undegraded":"2023-11-16T16:38:03.185550+0000","last_fullsized":"2023-11-16T16:38:03.185550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:32:03.882320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.189374+0000","last_change":"2023-11-16T16:37:13.934802+0000","last_active":"2023-11-16T16:38:03.189374+0000","last_peered":"2023-11-16T16:38:03.189374+0000","last_clean":"2023-11-16T16:38:03.189374+0000","last_became_active":"2023-11-16T16:37:13.934496+0000","last_became_peered":"2023-11-16T16:37:13.934496+0000","last_unstale":"2023-11-16T16:38:03.189374+0000","last_undegraded":"2023-11-16T16:38:03.189374+0000","last_fullsized":"2023-11-16T16:38:03.189374+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:15:58.249424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185585+0000","last_change":"2023-11-16T16:37:13.499284+0000","last_active":"2023-11-16T16:38:03.185585+0000","last_peered":"2023-11-16T16:38:03.185585+0000","last_clean":"2023-11-16T16:38:03.185585+0000","last_became_active":"2023-11-16T16:37:13.498931+0000","last_became_peered":"2023-11-16T16:37:13.498931+0000","last_unstale":"2023-11-16T16:38:03.185585+0000","last_undegraded":"2023-11-16T16:38:03.185585+0000","last_fullsized":"2023-11-16T16:38:03.185585+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:53:33.061352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157923+0000","last_change":"2023-11-16T16:37:13.939631+0000","last_active":"2023-11-16T16:37:15.157923+0000","last_peered":"2023-11-16T16:37:15.157923+0000","last_clean":"2023-11-16T16:37:15.157923+0000","last_became_active":"2023-11-16T16:37:13.939318+0000","last_became_peered":"2023-11-16T16:37:13.939318+0000","last_unstale":"2023-11-16T16:37:15.157923+0000","last_undegraded":"2023-11-16T16:37:15.157923+0000","last_fullsized":"2023-11-16T16:37:15.157923+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:51:05.730477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:16.528248+0000","last_change":"2023-11-16T16:37:13.495972+0000","last_active":"2023-11-16T16:37:16.528248+0000","last_peered":"2023-11-16T16:37:16.528248+0000","last_clean":"2023-11-16T16:37:16.528248+0000","last_became_active":"2023-11-16T16:37:13.495610+0000","last_became_peered":"2023-11-16T16:37:13.495610+0000","last_unstale":"2023-11-16T16:37:16.528248+0000","last_undegraded":"2023-11-16T16:37:16.528248+0000","last_fullsized":"2023-11-16T16:37:16.528248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:58:27.825452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158624+0000","last_change":"2023-11-16T16:37:13.935236+0000","last_active":"2023-11-16T16:37:15.158624+0000","last_peered":"2023-11-16T16:37:15.158624+0000","last_clean":"2023-11-16T16:37:15.158624+0000","last_became_active":"2023-11-16T16:37:13.934920+0000","last_became_peered":"2023-11-16T16:37:13.934920+0000","last_unstale":"2023-11-16T16:37:15.158624+0000","last_undegraded":"2023-11-16T16:37:15.158624+0000","last_fullsized":"2023-11-16T16:37:15.158624+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:58:27.305887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"84'4","reported_seq":2,"reported_epoch":85,"state":"peering","last_fresh":"2023-11-16T16:37:16.431881+0000","last_change":"2023-11-16T16:38:02.736871+0000","last_active":"2023-11-16T16:38:02.736871+0000","last_peered":"2023-11-16T16:37:16.431881+0000","last_clean":"2023-11-16T16:37:16.431881+0000","last_became_active":"2023-11-16T16:37:13.509431+0000","last_became_peered":"2023-11-16T16:37:13.509431+0000","last_unstale":"2023-11-16T16:37:16.431881+0000","last_undegraded":"2023-11-16T16:37:16.431881+0000","last_fullsized":"2023-11-16T16:37:16.431881+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.156807+0000","last_change":"2023-11-16T16:37:13.940838+0000","last_active":"2023-11-16T16:37:15.156807+0000","last_peered":"2023-11-16T16:37:15.156807+0000","last_clean":"2023-11-16T16:37:15.156807+0000","last_became_active":"2023-11-16T16:37:13.940538+0000","last_became_peered":"2023-11-16T16:37:13.940538+0000","last_unstale":"2023-11-16T16:37:15.156807+0000","last_undegraded":"2023-11-16T16:37:15.156807+0000","last_fullsized":"2023-11-16T16:37:15.156807+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:24:41.595556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.925256+0000","last_change":"2023-11-16T16:37:13.495114+0000","last_active":"2023-11-16T16:37:14.925256+0000","last_peered":"2023-11-16T16:37:14.925256+0000","last_clean":"2023-11-16T16:37:14.925256+0000","last_became_active":"2023-11-16T16:37:13.494807+0000","last_became_peered":"2023-11-16T16:37:13.494807+0000","last_unstale":"2023-11-16T16:37:14.925256+0000","last_undegraded":"2023-11-16T16:37:14.925256+0000","last_fullsized":"2023-11-16T16:37:14.925256+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:24:04.736603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185195+0000","last_change":"2023-11-16T16:37:13.943652+0000","last_active":"2023-11-16T16:38:03.185195+0000","last_peered":"2023-11-16T16:38:03.185195+0000","last_clean":"2023-11-16T16:38:03.185195+0000","last_became_active":"2023-11-16T16:37:13.943298+0000","last_became_peered":"2023-11-16T16:37:13.943298+0000","last_unstale":"2023-11-16T16:38:03.185195+0000","last_undegraded":"2023-11-16T16:38:03.185195+0000","last_fullsized":"2023-11-16T16:38:03.185195+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:17:55.154731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"84'6","reported_seq":74,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:04.371227+0000","last_change":"2023-11-16T16:37:13.509801+0000","last_active":"2023-11-16T16:38:04.371227+0000","last_peered":"2023-11-16T16:38:04.371227+0000","last_clean":"2023-11-16T16:38:04.371227+0000","last_became_active":"2023-11-16T16:37:13.509389+0000","last_became_peered":"2023-11-16T16:37:13.509389+0000","last_unstale":"2023-11-16T16:38:04.371227+0000","last_undegraded":"2023-11-16T16:38:04.371227+0000","last_fullsized":"2023-11-16T16:38:04.371227+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:46:55.751281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.924573+0000","last_change":"2023-11-16T16:37:13.933871+0000","last_active":"2023-11-16T16:37:14.924573+0000","last_peered":"2023-11-16T16:37:14.924573+0000","last_clean":"2023-11-16T16:37:14.924573+0000","last_became_active":"2023-11-16T16:37:13.933549+0000","last_became_peered":"2023-11-16T16:37:13.933549+0000","last_unstale":"2023-11-16T16:37:14.924573+0000","last_undegraded":"2023-11-16T16:37:14.924573+0000","last_fullsized":"2023-11-16T16:37:14.924573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:16:26.022577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"84'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.187212+0000","last_change":"2023-11-16T16:37:13.501339+0000","last_active":"2023-11-16T16:38:03.187212+0000","last_peered":"2023-11-16T16:38:03.187212+0000","last_clean":"2023-11-16T16:38:03.187212+0000","last_became_active":"2023-11-16T16:37:13.500949+0000","last_became_peered":"2023-11-16T16:37:13.500949+0000","last_unstale":"2023-11-16T16:38:03.187212+0000","last_undegraded":"2023-11-16T16:38:03.187212+0000","last_fullsized":"2023-11-16T16:38:03.187212+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:56:34.503681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.192595+0000","last_change":"2023-11-16T16:38:03.192595+0000","last_active":"2023-11-16T16:38:03.192595+0000","last_peered":"2023-11-16T16:38:03.192595+0000","last_clean":"2023-11-16T16:38:03.192595+0000","last_became_active":"2023-11-16T16:38:03.192208+0000","last_became_peered":"2023-11-16T16:38:03.192208+0000","last_unstale":"2023-11-16T16:38:03.192595+0000","last_undegraded":"2023-11-16T16:38:03.192595+0000","last_fullsized":"2023-11-16T16:38:03.192595+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:30:24.627049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"84'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184789+0000","last_change":"2023-11-16T16:37:13.505481+0000","last_active":"2023-11-16T16:38:03.184789+0000","last_peered":"2023-11-16T16:38:03.184789+0000","last_clean":"2023-11-16T16:38:03.184789+0000","last_became_active":"2023-11-16T16:37:13.505085+0000","last_became_peered":"2023-11-16T16:37:13.505085+0000","last_unstale":"2023-11-16T16:38:03.184789+0000","last_undegraded":"2023-11-16T16:38:03.184789+0000","last_fullsized":"2023-11-16T16:38:03.184789+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:21:13.673154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157157+0000","last_change":"2023-11-16T16:37:13.943985+0000","last_active":"2023-11-16T16:37:15.157157+0000","last_peered":"2023-11-16T16:37:15.157157+0000","last_clean":"2023-11-16T16:37:15.157157+0000","last_became_active":"2023-11-16T16:37:13.943628+0000","last_became_peered":"2023-11-16T16:37:13.943628+0000","last_unstale":"2023-11-16T16:37:15.157157+0000","last_undegraded":"2023-11-16T16:37:15.157157+0000","last_fullsized":"2023-11-16T16:37:15.157157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:19:10.518280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184704+0000","last_change":"2023-11-16T16:37:13.498648+0000","last_active":"2023-11-16T16:38:03.184704+0000","last_peered":"2023-11-16T16:38:03.184704+0000","last_clean":"2023-11-16T16:38:03.184704+0000","last_became_active":"2023-11-16T16:37:13.497390+0000","last_became_peered":"2023-11-16T16:37:13.497390+0000","last_unstale":"2023-11-16T16:38:03.184704+0000","last_undegraded":"2023-11-16T16:38:03.184704+0000","last_fullsized":"2023-11-16T16:38:03.184704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:20:50.541938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185282+0000","last_change":"2023-11-16T16:37:13.942337+0000","last_active":"2023-11-16T16:38:03.185282+0000","last_peered":"2023-11-16T16:38:03.185282+0000","last_clean":"2023-11-16T16:38:03.185282+0000","last_became_active":"2023-11-16T16:37:13.941982+0000","last_became_peered":"2023-11-16T16:37:13.941982+0000","last_unstale":"2023-11-16T16:38:03.185282+0000","last_undegraded":"2023-11-16T16:38:03.185282+0000","last_fullsized":"2023-11-16T16:38:03.185282+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:09:19.949618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'3","reported_seq":68,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:04.370630+0000","last_change":"2023-11-16T16:37:13.503690+0000","last_active":"2023-11-16T16:38:04.370630+0000","last_peered":"2023-11-16T16:38:04.370630+0000","last_clean":"2023-11-16T16:38:04.370630+0000","last_became_active":"2023-11-16T16:37:13.503203+0000","last_became_peered":"2023-11-16T16:37:13.503203+0000","last_unstale":"2023-11-16T16:38:04.370630+0000","last_undegraded":"2023-11-16T16:38:04.370630+0000","last_fullsized":"2023-11-16T16:38:04.370630+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:47:38.415047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":47,"num_read_kb":47,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.156964+0000","last_change":"2023-11-16T16:37:13.943975+0000","last_active":"2023-11-16T16:37:15.156964+0000","last_peered":"2023-11-16T16:37:15.156964+0000","last_clean":"2023-11-16T16:37:15.156964+0000","last_became_active":"2023-11-16T16:37:13.943611+0000","last_became_peered":"2023-11-16T16:37:13.943611+0000","last_unstale":"2023-11-16T16:37:15.156964+0000","last_undegraded":"2023-11-16T16:37:15.156964+0000","last_fullsized":"2023-11-16T16:37:15.156964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:21:29.078427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'2","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.344191+0000","last_change":"2023-11-16T16:38:03.344191+0000","last_active":"2023-11-16T16:38:03.344191+0000","last_peered":"2023-11-16T16:38:03.344191+0000","last_clean":"2023-11-16T16:38:03.344191+0000","last_became_active":"2023-11-16T16:38:03.189976+0000","last_became_peered":"2023-11-16T16:38:03.189976+0000","last_unstale":"2023-11-16T16:38:03.344191+0000","last_undegraded":"2023-11-16T16:38:03.344191+0000","last_fullsized":"2023-11-16T16:38:03.344191+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:04:56.889493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158690+0000","last_change":"2023-11-16T16:37:13.949031+0000","last_active":"2023-11-16T16:37:15.158690+0000","last_peered":"2023-11-16T16:37:15.158690+0000","last_clean":"2023-11-16T16:37:15.158690+0000","last_became_active":"2023-11-16T16:37:13.946923+0000","last_became_peered":"2023-11-16T16:37:13.946923+0000","last_unstale":"2023-11-16T16:37:15.158690+0000","last_undegraded":"2023-11-16T16:37:15.158690+0000","last_fullsized":"2023-11-16T16:37:15.158690+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:00:35.683602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185398+0000","last_change":"2023-11-16T16:37:13.495003+0000","last_active":"2023-11-16T16:38:03.185398+0000","last_peered":"2023-11-16T16:38:03.185398+0000","last_clean":"2023-11-16T16:38:03.185398+0000","last_became_active":"2023-11-16T16:37:13.494563+0000","last_became_peered":"2023-11-16T16:37:13.494563+0000","last_unstale":"2023-11-16T16:38:03.185398+0000","last_undegraded":"2023-11-16T16:38:03.185398+0000","last_fullsized":"2023-11-16T16:38:03.185398+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:36:36.403721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.156822+0000","last_change":"2023-11-16T16:37:13.943736+0000","last_active":"2023-11-16T16:37:15.156822+0000","last_peered":"2023-11-16T16:37:15.156822+0000","last_clean":"2023-11-16T16:37:15.156822+0000","last_became_active":"2023-11-16T16:37:13.943391+0000","last_became_peered":"2023-11-16T16:37:13.943391+0000","last_unstale":"2023-11-16T16:37:15.156822+0000","last_undegraded":"2023-11-16T16:37:15.156822+0000","last_fullsized":"2023-11-16T16:37:15.156822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:03:21.402516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'28","reported_seq":148,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:05.383759+0000","last_change":"2023-11-16T16:37:13.506508+0000","last_active":"2023-11-16T16:38:05.383759+0000","last_peered":"2023-11-16T16:38:05.383759+0000","last_clean":"2023-11-16T16:38:05.383759+0000","last_became_active":"2023-11-16T16:37:13.506045+0000","last_became_peered":"2023-11-16T16:37:13.506045+0000","last_unstale":"2023-11-16T16:38:05.383759+0000","last_undegraded":"2023-11-16T16:38:05.383759+0000","last_fullsized":"2023-11-16T16:38:05.383759+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:57:44.753422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58527,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":102,"num_read_kb":107,"num_write":29,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.924203+0000","last_change":"2023-11-16T16:37:13.939717+0000","last_active":"2023-11-16T16:37:14.924203+0000","last_peered":"2023-11-16T16:37:14.924203+0000","last_clean":"2023-11-16T16:37:14.924203+0000","last_became_active":"2023-11-16T16:37:13.939369+0000","last_became_peered":"2023-11-16T16:37:13.939369+0000","last_unstale":"2023-11-16T16:37:14.924203+0000","last_undegraded":"2023-11-16T16:37:14.924203+0000","last_fullsized":"2023-11-16T16:37:14.924203+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:15:46.898377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.188492+0000","last_change":"2023-11-16T16:37:13.506056+0000","last_active":"2023-11-16T16:38:03.188492+0000","last_peered":"2023-11-16T16:38:03.188492+0000","last_clean":"2023-11-16T16:38:03.188492+0000","last_became_active":"2023-11-16T16:37:13.505707+0000","last_became_peered":"2023-11-16T16:37:13.505707+0000","last_unstale":"2023-11-16T16:38:03.188492+0000","last_undegraded":"2023-11-16T16:38:03.188492+0000","last_fullsized":"2023-11-16T16:38:03.188492+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:29:47.242470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184551+0000","last_change":"2023-11-16T16:37:13.947764+0000","last_active":"2023-11-16T16:38:03.184551+0000","last_peered":"2023-11-16T16:38:03.184551+0000","last_clean":"2023-11-16T16:38:03.184551+0000","last_became_active":"2023-11-16T16:37:13.947510+0000","last_became_peered":"2023-11-16T16:37:13.947510+0000","last_unstale":"2023-11-16T16:38:03.184551+0000","last_undegraded":"2023-11-16T16:38:03.184551+0000","last_fullsized":"2023-11-16T16:38:03.184551+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:41:59.672529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184939+0000","last_change":"2023-11-16T16:37:13.498178+0000","last_active":"2023-11-16T16:38:03.184939+0000","last_peered":"2023-11-16T16:38:03.184939+0000","last_clean":"2023-11-16T16:38:03.184939+0000","last_became_active":"2023-11-16T16:37:13.497312+0000","last_became_peered":"2023-11-16T16:37:13.497312+0000","last_unstale":"2023-11-16T16:38:03.184939+0000","last_undegraded":"2023-11-16T16:38:03.184939+0000","last_fullsized":"2023-11-16T16:38:03.184939+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:54:40.375590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185267+0000","last_change":"2023-11-16T16:37:13.947854+0000","last_active":"2023-11-16T16:38:03.185267+0000","last_peered":"2023-11-16T16:38:03.185267+0000","last_clean":"2023-11-16T16:38:03.185267+0000","last_became_active":"2023-11-16T16:37:13.947530+0000","last_became_peered":"2023-11-16T16:37:13.947530+0000","last_unstale":"2023-11-16T16:38:03.185267+0000","last_undegraded":"2023-11-16T16:38:03.185267+0000","last_fullsized":"2023-11-16T16:38:03.185267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:41:00.205641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'3","reported_seq":68,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.183671+0000","last_change":"2023-11-16T16:37:13.500127+0000","last_active":"2023-11-16T16:38:03.183671+0000","last_peered":"2023-11-16T16:38:03.183671+0000","last_clean":"2023-11-16T16:38:03.183671+0000","last_became_active":"2023-11-16T16:37:13.499758+0000","last_became_peered":"2023-11-16T16:37:13.499758+0000","last_unstale":"2023-11-16T16:38:03.183671+0000","last_undegraded":"2023-11-16T16:38:03.183671+0000","last_fullsized":"2023-11-16T16:38:03.183671+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:39:00.268144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":48,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.183305+0000","last_change":"2023-11-16T16:37:13.940962+0000","last_active":"2023-11-16T16:38:03.183305+0000","last_peered":"2023-11-16T16:38:03.183305+0000","last_clean":"2023-11-16T16:38:03.183305+0000","last_became_active":"2023-11-16T16:37:13.940450+0000","last_became_peered":"2023-11-16T16:37:13.940450+0000","last_unstale":"2023-11-16T16:38:03.183305+0000","last_undegraded":"2023-11-16T16:38:03.183305+0000","last_fullsized":"2023-11-16T16:38:03.183305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:28:45.946454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'15","reported_seq":2,"reported_epoch":85,"state":"peering","last_fresh":"2023-11-16T16:37:17.285577+0000","last_change":"2023-11-16T16:38:02.736820+0000","last_active":"2023-11-16T16:38:02.736820+0000","last_peered":"2023-11-16T16:37:17.285577+0000","last_clean":"2023-11-16T16:37:17.285577+0000","last_became_active":"2023-11-16T16:37:13.492599+0000","last_became_peered":"2023-11-16T16:37:13.492599+0000","last_unstale":"2023-11-16T16:37:17.285577+0000","last_undegraded":"2023-11-16T16:37:17.285577+0000","last_fullsized":"2023-11-16T16:37:17.285577+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":8,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184704+0000","last_change":"2023-11-16T16:37:13.944878+0000","last_active":"2023-11-16T16:38:03.184704+0000","last_peered":"2023-11-16T16:38:03.184704+0000","last_clean":"2023-11-16T16:38:03.184704+0000","last_became_active":"2023-11-16T16:37:13.944580+0000","last_became_peered":"2023-11-16T16:37:13.944580+0000","last_unstale":"2023-11-16T16:38:03.184704+0000","last_undegraded":"2023-11-16T16:38:03.184704+0000","last_fullsized":"2023-11-16T16:38:03.184704+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:00:01.106243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:19.300067+0000","last_change":"2023-11-16T16:37:13.499139+0000","last_active":"2023-11-16T16:37:19.300067+0000","last_peered":"2023-11-16T16:37:19.300067+0000","last_clean":"2023-11-16T16:37:19.300067+0000","last_became_active":"2023-11-16T16:37:13.498755+0000","last_became_peered":"2023-11-16T16:37:13.498755+0000","last_unstale":"2023-11-16T16:37:19.300067+0000","last_undegraded":"2023-11-16T16:37:19.300067+0000","last_fullsized":"2023-11-16T16:37:19.300067+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:07:40.535306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-16T16:38:02.187750+0000","last_change":"2023-11-16T16:37:13.946187+0000","last_active":"2023-11-16T16:38:02.187750+0000","last_peered":"2023-11-16T16:38:02.187750+0000","last_clean":"2023-11-16T16:38:02.187750+0000","last_became_active":"2023-11-16T16:37:13.945887+0000","last_became_peered":"2023-11-16T16:37:13.945887+0000","last_unstale":"2023-11-16T16:38:02.187750+0000","last_undegraded":"2023-11-16T16:38:02.187750+0000","last_fullsized":"2023-11-16T16:38:02.187750+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:53:33.133780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185150+0000","last_change":"2023-11-16T16:37:13.507150+0000","last_active":"2023-11-16T16:38:03.185150+0000","last_peered":"2023-11-16T16:38:03.185150+0000","last_clean":"2023-11-16T16:38:03.185150+0000","last_became_active":"2023-11-16T16:37:13.506814+0000","last_became_peered":"2023-11-16T16:37:13.506814+0000","last_unstale":"2023-11-16T16:38:03.185150+0000","last_undegraded":"2023-11-16T16:38:03.185150+0000","last_fullsized":"2023-11-16T16:38:03.185150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:54:45.031046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.187711+0000","last_change":"2023-11-16T16:37:13.947866+0000","last_active":"2023-11-16T16:38:03.187711+0000","last_peered":"2023-11-16T16:38:03.187711+0000","last_clean":"2023-11-16T16:38:03.187711+0000","last_became_active":"2023-11-16T16:37:13.947544+0000","last_became_peered":"2023-11-16T16:37:13.947544+0000","last_unstale":"2023-11-16T16:38:03.187711+0000","last_undegraded":"2023-11-16T16:38:03.187711+0000","last_fullsized":"2023-11-16T16:38:03.187711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:23:27.220130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"84'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184212+0000","last_change":"2023-11-16T16:37:13.497800+0000","last_active":"2023-11-16T16:38:03.184212+0000","last_peered":"2023-11-16T16:38:03.184212+0000","last_clean":"2023-11-16T16:38:03.184212+0000","last_became_active":"2023-11-16T16:37:13.497414+0000","last_became_peered":"2023-11-16T16:37:13.497414+0000","last_unstale":"2023-11-16T16:38:03.184212+0000","last_undegraded":"2023-11-16T16:38:03.184212+0000","last_fullsized":"2023-11-16T16:38:03.184212+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:17:42.179250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.187179+0000","last_change":"2023-11-16T16:37:13.946780+0000","last_active":"2023-11-16T16:38:03.187179+0000","last_peered":"2023-11-16T16:38:03.187179+0000","last_clean":"2023-11-16T16:38:03.187179+0000","last_became_active":"2023-11-16T16:37:13.946441+0000","last_became_peered":"2023-11-16T16:37:13.946441+0000","last_unstale":"2023-11-16T16:38:03.187179+0000","last_undegraded":"2023-11-16T16:38:03.187179+0000","last_fullsized":"2023-11-16T16:38:03.187179+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:46:10.856707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.192571+0000","last_change":"2023-11-16T16:38:03.191887+0000","last_active":"2023-11-16T16:38:03.192571+0000","last_peered":"2023-11-16T16:38:03.192571+0000","last_clean":"2023-11-16T16:38:03.192571+0000","last_became_active":"2023-11-16T16:38:03.191402+0000","last_became_peered":"2023-11-16T16:38:03.191402+0000","last_unstale":"2023-11-16T16:38:03.192571+0000","last_undegraded":"2023-11-16T16:38:03.192571+0000","last_fullsized":"2023-11-16T16:38:03.192571+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:47:26.769740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"84'4","reported_seq":67,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184801+0000","last_change":"2023-11-16T16:37:13.498129+0000","last_active":"2023-11-16T16:38:03.184801+0000","last_peered":"2023-11-16T16:38:03.184801+0000","last_clean":"2023-11-16T16:38:03.184801+0000","last_became_active":"2023-11-16T16:37:13.497765+0000","last_became_peered":"2023-11-16T16:37:13.497765+0000","last_unstale":"2023-11-16T16:38:03.184801+0000","last_undegraded":"2023-11-16T16:38:03.184801+0000","last_fullsized":"2023-11-16T16:38:03.184801+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:15:51.118817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":45,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158156+0000","last_change":"2023-11-16T16:37:13.938417+0000","last_active":"2023-11-16T16:37:15.158156+0000","last_peered":"2023-11-16T16:37:15.158156+0000","last_clean":"2023-11-16T16:37:15.158156+0000","last_became_active":"2023-11-16T16:37:13.938114+0000","last_became_peered":"2023-11-16T16:37:13.938114+0000","last_unstale":"2023-11-16T16:37:15.158156+0000","last_undegraded":"2023-11-16T16:37:15.158156+0000","last_fullsized":"2023-11-16T16:37:15.158156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:35:24.220360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"84'3","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.183990+0000","last_change":"2023-11-16T16:37:13.499045+0000","last_active":"2023-11-16T16:38:03.183990+0000","last_peered":"2023-11-16T16:38:03.183990+0000","last_clean":"2023-11-16T16:38:03.183990+0000","last_became_active":"2023-11-16T16:37:13.498598+0000","last_became_peered":"2023-11-16T16:37:13.498598+0000","last_unstale":"2023-11-16T16:38:03.183990+0000","last_undegraded":"2023-11-16T16:38:03.183990+0000","last_fullsized":"2023-11-16T16:38:03.183990+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:29:46.898930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":349,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.183816+0000","last_change":"2023-11-16T16:36:14.090213+0000","last_active":"2023-11-16T16:38:03.183816+0000","last_peered":"2023-11-16T16:38:03.183816+0000","last_clean":"2023-11-16T16:38:03.183816+0000","last_became_active":"2023-11-16T16:36:14.089742+0000","last_became_peered":"2023-11-16T16:36:14.089742+0000","last_unstale":"2023-11-16T16:38:03.183816+0000","last_undegraded":"2023-11-16T16:38:03.183816+0000","last_fullsized":"2023-11-16T16:38:03.183816+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:33:16.178716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:33:16.178716+0000","last_clean_scrub_stamp":"2023-11-16T16:33:16.178716+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:34:52.910521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184225+0000","last_change":"2023-11-16T16:37:13.945380+0000","last_active":"2023-11-16T16:38:03.184225+0000","last_peered":"2023-11-16T16:38:03.184225+0000","last_clean":"2023-11-16T16:38:03.184225+0000","last_became_active":"2023-11-16T16:37:13.945038+0000","last_became_peered":"2023-11-16T16:37:13.945038+0000","last_unstale":"2023-11-16T16:38:03.184225+0000","last_undegraded":"2023-11-16T16:38:03.184225+0000","last_fullsized":"2023-11-16T16:38:03.184225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:29:11.812243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"84'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184055+0000","last_change":"2023-11-16T16:37:13.497586+0000","last_active":"2023-11-16T16:38:03.184055+0000","last_peered":"2023-11-16T16:38:03.184055+0000","last_clean":"2023-11-16T16:38:03.184055+0000","last_became_active":"2023-11-16T16:37:13.497233+0000","last_became_peered":"2023-11-16T16:37:13.497233+0000","last_unstale":"2023-11-16T16:38:03.184055+0000","last_undegraded":"2023-11-16T16:38:03.184055+0000","last_fullsized":"2023-11-16T16:38:03.184055+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:47:21.097628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"84'1","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:17.285613+0000","last_change":"2023-11-16T16:37:13.500621+0000","last_active":"2023-11-16T16:37:17.285613+0000","last_peered":"2023-11-16T16:37:17.285613+0000","last_clean":"2023-11-16T16:37:17.285613+0000","last_became_active":"2023-11-16T16:37:13.500265+0000","last_became_peered":"2023-11-16T16:37:13.500265+0000","last_unstale":"2023-11-16T16:37:17.285613+0000","last_undegraded":"2023-11-16T16:37:17.285613+0000","last_fullsized":"2023-11-16T16:37:17.285613+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:51:28.450861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184391+0000","last_change":"2023-11-16T16:37:13.930992+0000","last_active":"2023-11-16T16:38:03.184391+0000","last_peered":"2023-11-16T16:38:03.184391+0000","last_clean":"2023-11-16T16:38:03.184391+0000","last_became_active":"2023-11-16T16:37:13.930641+0000","last_became_peered":"2023-11-16T16:37:13.930641+0000","last_unstale":"2023-11-16T16:38:03.184391+0000","last_undegraded":"2023-11-16T16:38:03.184391+0000","last_fullsized":"2023-11-16T16:38:03.184391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:24:49.314642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'5","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185381+0000","last_change":"2023-11-16T16:37:13.498637+0000","last_active":"2023-11-16T16:38:03.185381+0000","last_peered":"2023-11-16T16:38:03.185381+0000","last_clean":"2023-11-16T16:38:03.185381+0000","last_became_active":"2023-11-16T16:37:13.498341+0000","last_became_peered":"2023-11-16T16:37:13.498341+0000","last_unstale":"2023-11-16T16:38:03.185381+0000","last_undegraded":"2023-11-16T16:38:03.185381+0000","last_fullsized":"2023-11-16T16:38:03.185381+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:30:04.558815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158333+0000","last_change":"2023-11-16T16:37:13.944142+0000","last_active":"2023-11-16T16:37:15.158333+0000","last_peered":"2023-11-16T16:37:15.158333+0000","last_clean":"2023-11-16T16:37:15.158333+0000","last_became_active":"2023-11-16T16:37:13.943758+0000","last_became_peered":"2023-11-16T16:37:13.943758+0000","last_unstale":"2023-11-16T16:37:15.158333+0000","last_undegraded":"2023-11-16T16:37:15.158333+0000","last_fullsized":"2023-11-16T16:37:15.158333+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:55:52.343549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184558+0000","last_change":"2023-11-16T16:37:13.496900+0000","last_active":"2023-11-16T16:38:03.184558+0000","last_peered":"2023-11-16T16:38:03.184558+0000","last_clean":"2023-11-16T16:38:03.184558+0000","last_became_active":"2023-11-16T16:37:13.496528+0000","last_became_peered":"2023-11-16T16:37:13.496528+0000","last_unstale":"2023-11-16T16:38:03.184558+0000","last_undegraded":"2023-11-16T16:38:03.184558+0000","last_fullsized":"2023-11-16T16:38:03.184558+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:56:42.304030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158258+0000","last_change":"2023-11-16T16:37:13.943104+0000","last_active":"2023-11-16T16:37:15.158258+0000","last_peered":"2023-11-16T16:37:15.158258+0000","last_clean":"2023-11-16T16:37:15.158258+0000","last_became_active":"2023-11-16T16:37:13.942774+0000","last_became_peered":"2023-11-16T16:37:13.942774+0000","last_unstale":"2023-11-16T16:37:15.158258+0000","last_undegraded":"2023-11-16T16:37:15.158258+0000","last_fullsized":"2023-11-16T16:37:15.158258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:00:01.892620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:17.287386+0000","last_change":"2023-11-16T16:37:13.505994+0000","last_active":"2023-11-16T16:37:17.287386+0000","last_peered":"2023-11-16T16:37:17.287386+0000","last_clean":"2023-11-16T16:37:17.287386+0000","last_became_active":"2023-11-16T16:37:13.505663+0000","last_became_peered":"2023-11-16T16:37:13.505663+0000","last_unstale":"2023-11-16T16:37:17.287386+0000","last_undegraded":"2023-11-16T16:37:17.287386+0000","last_fullsized":"2023-11-16T16:37:17.287386+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:11:39.436119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:14.923847+0000","last_change":"2023-11-16T16:37:13.941970+0000","last_active":"2023-11-16T16:37:14.923847+0000","last_peered":"2023-11-16T16:37:14.923847+0000","last_clean":"2023-11-16T16:37:14.923847+0000","last_became_active":"2023-11-16T16:37:13.941649+0000","last_became_peered":"2023-11-16T16:37:13.941649+0000","last_unstale":"2023-11-16T16:37:14.923847+0000","last_undegraded":"2023-11-16T16:37:14.923847+0000","last_fullsized":"2023-11-16T16:37:14.923847+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:00:21.646221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:17.287982+0000","last_change":"2023-11-16T16:37:13.503429+0000","last_active":"2023-11-16T16:37:17.287982+0000","last_peered":"2023-11-16T16:37:17.287982+0000","last_clean":"2023-11-16T16:37:17.287982+0000","last_became_active":"2023-11-16T16:37:13.503114+0000","last_became_peered":"2023-11-16T16:37:13.503114+0000","last_unstale":"2023-11-16T16:37:17.287982+0000","last_undegraded":"2023-11-16T16:37:17.287982+0000","last_fullsized":"2023-11-16T16:37:17.287982+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:05:42.643043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-16T16:38:02.187143+0000","last_change":"2023-11-16T16:37:13.941225+0000","last_active":"2023-11-16T16:38:02.187143+0000","last_peered":"2023-11-16T16:38:02.187143+0000","last_clean":"2023-11-16T16:38:02.187143+0000","last_became_active":"2023-11-16T16:37:13.940964+0000","last_became_peered":"2023-11-16T16:37:13.940964+0000","last_unstale":"2023-11-16T16:38:02.187143+0000","last_undegraded":"2023-11-16T16:38:02.187143+0000","last_fullsized":"2023-11-16T16:38:02.187143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:43:22.130214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.188151+0000","last_change":"2023-11-16T16:37:13.510154+0000","last_active":"2023-11-16T16:38:03.188151+0000","last_peered":"2023-11-16T16:38:03.188151+0000","last_clean":"2023-11-16T16:38:03.188151+0000","last_became_active":"2023-11-16T16:37:13.509790+0000","last_became_peered":"2023-11-16T16:37:13.509790+0000","last_unstale":"2023-11-16T16:38:03.188151+0000","last_undegraded":"2023-11-16T16:38:03.188151+0000","last_fullsized":"2023-11-16T16:38:03.188151+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:09:32.999016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158172+0000","last_change":"2023-11-16T16:37:13.942203+0000","last_active":"2023-11-16T16:37:15.158172+0000","last_peered":"2023-11-16T16:37:15.158172+0000","last_clean":"2023-11-16T16:37:15.158172+0000","last_became_active":"2023-11-16T16:37:13.941766+0000","last_became_peered":"2023-11-16T16:37:13.941766+0000","last_unstale":"2023-11-16T16:37:15.158172+0000","last_undegraded":"2023-11-16T16:37:15.158172+0000","last_fullsized":"2023-11-16T16:37:15.158172+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:12:25.582734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:17.286155+0000","last_change":"2023-11-16T16:37:13.509854+0000","last_active":"2023-11-16T16:37:17.286155+0000","last_peered":"2023-11-16T16:37:17.286155+0000","last_clean":"2023-11-16T16:37:17.286155+0000","last_became_active":"2023-11-16T16:37:13.509466+0000","last_became_peered":"2023-11-16T16:37:13.509466+0000","last_unstale":"2023-11-16T16:37:17.286155+0000","last_undegraded":"2023-11-16T16:37:17.286155+0000","last_fullsized":"2023-11-16T16:37:17.286155+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:05:41.806694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184479+0000","last_change":"2023-11-16T16:37:13.947906+0000","last_active":"2023-11-16T16:38:03.184479+0000","last_peered":"2023-11-16T16:38:03.184479+0000","last_clean":"2023-11-16T16:38:03.184479+0000","last_became_active":"2023-11-16T16:37:13.947602+0000","last_became_peered":"2023-11-16T16:37:13.947602+0000","last_unstale":"2023-11-16T16:38:03.184479+0000","last_undegraded":"2023-11-16T16:38:03.184479+0000","last_fullsized":"2023-11-16T16:38:03.184479+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:16:45.611613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.188948+0000","last_change":"2023-11-16T16:37:13.506108+0000","last_active":"2023-11-16T16:38:03.188948+0000","last_peered":"2023-11-16T16:38:03.188948+0000","last_clean":"2023-11-16T16:38:03.188948+0000","last_became_active":"2023-11-16T16:37:13.505785+0000","last_became_peered":"2023-11-16T16:37:13.505785+0000","last_unstale":"2023-11-16T16:38:03.188948+0000","last_undegraded":"2023-11-16T16:38:03.188948+0000","last_fullsized":"2023-11-16T16:38:03.188948+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:11:02.793557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185008+0000","last_change":"2023-11-16T16:37:13.935863+0000","last_active":"2023-11-16T16:38:03.185008+0000","last_peered":"2023-11-16T16:38:03.185008+0000","last_clean":"2023-11-16T16:38:03.185008+0000","last_became_active":"2023-11-16T16:37:13.935399+0000","last_became_peered":"2023-11-16T16:37:13.935399+0000","last_unstale":"2023-11-16T16:38:03.185008+0000","last_undegraded":"2023-11-16T16:38:03.185008+0000","last_fullsized":"2023-11-16T16:38:03.185008+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:45:56.613743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184907+0000","last_change":"2023-11-16T16:37:13.498367+0000","last_active":"2023-11-16T16:38:03.184907+0000","last_peered":"2023-11-16T16:38:03.184907+0000","last_clean":"2023-11-16T16:38:03.184907+0000","last_became_active":"2023-11-16T16:37:13.497039+0000","last_became_peered":"2023-11-16T16:37:13.497039+0000","last_unstale":"2023-11-16T16:38:03.184907+0000","last_undegraded":"2023-11-16T16:38:03.184907+0000","last_fullsized":"2023-11-16T16:38:03.184907+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:57:46.571219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184415+0000","last_change":"2023-11-16T16:37:13.940617+0000","last_active":"2023-11-16T16:38:03.184415+0000","last_peered":"2023-11-16T16:38:03.184415+0000","last_clean":"2023-11-16T16:38:03.184415+0000","last_became_active":"2023-11-16T16:37:13.940264+0000","last_became_peered":"2023-11-16T16:37:13.940264+0000","last_unstale":"2023-11-16T16:38:03.184415+0000","last_undegraded":"2023-11-16T16:38:03.184415+0000","last_fullsized":"2023-11-16T16:38:03.184415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:59:36.630848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.193922+0000","last_change":"2023-11-16T16:38:03.193922+0000","last_active":"2023-11-16T16:38:03.193922+0000","last_peered":"2023-11-16T16:38:03.193922+0000","last_clean":"2023-11-16T16:38:03.193922+0000","last_became_active":"2023-11-16T16:38:03.190364+0000","last_became_peered":"2023-11-16T16:38:03.190364+0000","last_unstale":"2023-11-16T16:38:03.193922+0000","last_undegraded":"2023-11-16T16:38:03.193922+0000","last_fullsized":"2023-11-16T16:38:03.193922+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:59:04.681132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185280+0000","last_change":"2023-11-16T16:37:13.943096+0000","last_active":"2023-11-16T16:38:03.185280+0000","last_peered":"2023-11-16T16:38:03.185280+0000","last_clean":"2023-11-16T16:38:03.185280+0000","last_became_active":"2023-11-16T16:37:13.942745+0000","last_became_peered":"2023-11-16T16:37:13.942745+0000","last_unstale":"2023-11-16T16:38:03.185280+0000","last_undegraded":"2023-11-16T16:38:03.185280+0000","last_fullsized":"2023-11-16T16:38:03.185280+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:02:12.721195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-16T16:38:02.187324+0000","last_change":"2023-11-16T16:37:13.938087+0000","last_active":"2023-11-16T16:38:02.187324+0000","last_peered":"2023-11-16T16:38:02.187324+0000","last_clean":"2023-11-16T16:38:02.187324+0000","last_became_active":"2023-11-16T16:37:13.937780+0000","last_became_peered":"2023-11-16T16:37:13.937780+0000","last_unstale":"2023-11-16T16:38:02.187324+0000","last_undegraded":"2023-11-16T16:38:02.187324+0000","last_fullsized":"2023-11-16T16:38:02.187324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:08:08.645514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.188244+0000","last_change":"2023-11-16T16:37:13.939000+0000","last_active":"2023-11-16T16:38:03.188244+0000","last_peered":"2023-11-16T16:38:03.188244+0000","last_clean":"2023-11-16T16:38:03.188244+0000","last_became_active":"2023-11-16T16:37:13.938585+0000","last_became_peered":"2023-11-16T16:37:13.938585+0000","last_unstale":"2023-11-16T16:38:03.188244+0000","last_undegraded":"2023-11-16T16:38:03.188244+0000","last_fullsized":"2023-11-16T16:38:03.188244+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:13:17.753666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.187970+0000","last_change":"2023-11-16T16:37:13.937696+0000","last_active":"2023-11-16T16:38:03.187970+0000","last_peered":"2023-11-16T16:38:03.187970+0000","last_clean":"2023-11-16T16:38:03.187970+0000","last_became_active":"2023-11-16T16:37:13.936920+0000","last_became_peered":"2023-11-16T16:37:13.936920+0000","last_unstale":"2023-11-16T16:38:03.187970+0000","last_undegraded":"2023-11-16T16:38:03.187970+0000","last_fullsized":"2023-11-16T16:38:03.187970+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:55:00.163322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157738+0000","last_change":"2023-11-16T16:37:13.942977+0000","last_active":"2023-11-16T16:37:15.157738+0000","last_peered":"2023-11-16T16:37:15.157738+0000","last_clean":"2023-11-16T16:37:15.157738+0000","last_became_active":"2023-11-16T16:37:13.942632+0000","last_became_peered":"2023-11-16T16:37:13.942632+0000","last_unstale":"2023-11-16T16:37:15.157738+0000","last_undegraded":"2023-11-16T16:37:15.157738+0000","last_fullsized":"2023-11-16T16:37:15.157738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:44:02.416862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184448+0000","last_change":"2023-11-16T16:37:13.949940+0000","last_active":"2023-11-16T16:38:03.184448+0000","last_peered":"2023-11-16T16:38:03.184448+0000","last_clean":"2023-11-16T16:38:03.184448+0000","last_became_active":"2023-11-16T16:37:13.949464+0000","last_became_peered":"2023-11-16T16:37:13.949464+0000","last_unstale":"2023-11-16T16:38:03.184448+0000","last_undegraded":"2023-11-16T16:38:03.184448+0000","last_fullsized":"2023-11-16T16:38:03.184448+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:05:15.768098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-16T16:38:02.187885+0000","last_change":"2023-11-16T16:37:13.940534+0000","last_active":"2023-11-16T16:38:02.187885+0000","last_peered":"2023-11-16T16:38:02.187885+0000","last_clean":"2023-11-16T16:38:02.187885+0000","last_became_active":"2023-11-16T16:37:13.940174+0000","last_became_peered":"2023-11-16T16:37:13.940174+0000","last_unstale":"2023-11-16T16:38:02.187885+0000","last_undegraded":"2023-11-16T16:38:02.187885+0000","last_fullsized":"2023-11-16T16:38:02.187885+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:20:45.829821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.190148+0000","last_change":"2023-11-16T16:37:13.937519+0000","last_active":"2023-11-16T16:38:03.190148+0000","last_peered":"2023-11-16T16:38:03.190148+0000","last_clean":"2023-11-16T16:38:03.190148+0000","last_became_active":"2023-11-16T16:37:13.937175+0000","last_became_peered":"2023-11-16T16:37:13.937175+0000","last_unstale":"2023-11-16T16:38:03.190148+0000","last_undegraded":"2023-11-16T16:38:03.190148+0000","last_fullsized":"2023-11-16T16:38:03.190148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:25:23.849158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.158262+0000","last_change":"2023-11-16T16:37:13.951506+0000","last_active":"2023-11-16T16:37:15.158262+0000","last_peered":"2023-11-16T16:37:15.158262+0000","last_clean":"2023-11-16T16:37:15.158262+0000","last_became_active":"2023-11-16T16:37:13.951192+0000","last_became_peered":"2023-11-16T16:37:13.951192+0000","last_unstale":"2023-11-16T16:37:15.158262+0000","last_undegraded":"2023-11-16T16:37:15.158262+0000","last_fullsized":"2023-11-16T16:37:15.158262+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:18:48.061220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184841+0000","last_change":"2023-11-16T16:37:13.935614+0000","last_active":"2023-11-16T16:38:03.184841+0000","last_peered":"2023-11-16T16:38:03.184841+0000","last_clean":"2023-11-16T16:38:03.184841+0000","last_became_active":"2023-11-16T16:37:13.935237+0000","last_became_peered":"2023-11-16T16:37:13.935237+0000","last_unstale":"2023-11-16T16:38:03.184841+0000","last_undegraded":"2023-11-16T16:38:03.184841+0000","last_fullsized":"2023-11-16T16:38:03.184841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:30:01.772328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.157636+0000","last_change":"2023-11-16T16:37:13.939387+0000","last_active":"2023-11-16T16:37:15.157636+0000","last_peered":"2023-11-16T16:37:15.157636+0000","last_clean":"2023-11-16T16:37:15.157636+0000","last_became_active":"2023-11-16T16:37:13.939082+0000","last_became_peered":"2023-11-16T16:37:13.939082+0000","last_unstale":"2023-11-16T16:37:15.157636+0000","last_undegraded":"2023-11-16T16:37:15.157636+0000","last_fullsized":"2023-11-16T16:37:15.157636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:42:24.707911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-16T16:37:15.156764+0000","last_change":"2023-11-16T16:37:13.936381+0000","last_active":"2023-11-16T16:37:15.156764+0000","last_peered":"2023-11-16T16:37:15.156764+0000","last_clean":"2023-11-16T16:37:15.156764+0000","last_became_active":"2023-11-16T16:37:13.936077+0000","last_became_peered":"2023-11-16T16:37:13.936077+0000","last_unstale":"2023-11-16T16:37:15.156764+0000","last_undegraded":"2023-11-16T16:37:15.156764+0000","last_fullsized":"2023-11-16T16:37:15.156764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:04:48.256913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184590+0000","last_change":"2023-11-16T16:37:13.934005+0000","last_active":"2023-11-16T16:38:03.184590+0000","last_peered":"2023-11-16T16:38:03.184590+0000","last_clean":"2023-11-16T16:38:03.184590+0000","last_became_active":"2023-11-16T16:37:13.933633+0000","last_became_peered":"2023-11-16T16:37:13.933633+0000","last_unstale":"2023-11-16T16:38:03.184590+0000","last_undegraded":"2023-11-16T16:38:03.184590+0000","last_fullsized":"2023-11-16T16:38:03.184590+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:41:36.990242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-16T16:38:02.187300+0000","last_change":"2023-11-16T16:37:13.946495+0000","last_active":"2023-11-16T16:38:02.187300+0000","last_peered":"2023-11-16T16:38:02.187300+0000","last_clean":"2023-11-16T16:38:02.187300+0000","last_became_active":"2023-11-16T16:37:13.946148+0000","last_became_peered":"2023-11-16T16:37:13.946148+0000","last_unstale":"2023-11-16T16:38:02.187300+0000","last_undegraded":"2023-11-16T16:38:02.187300+0000","last_fullsized":"2023-11-16T16:38:02.187300+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:15:23.875543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":3542208,"data_compressed":211753,"data_compressed_allocated":270336,"data_compressed_original":3522560,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62431,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":10,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":304,"num_read_kb":217,"num_write":150,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":187293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132,"ondisk_log_size":132,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416406,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5944,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":93739208,"statfs":{"total":95995035648,"available":95988948992,"internally_reserved":0,"allocated":901120,"data_stored":429451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.68700000000000006}]},{"osd":1,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.79700000000000004}]},{"osd":2,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.48599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":4,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.70899999999999996}]},{"osd":5,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.61699999999999999}]},{"osd":6,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":0.85699999999999998}]},{"osd":7,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.84399999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":9,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.88}]},{"osd":10,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.863,"5min":1.863,"15min":1.863},"last":0.92200000000000004}]}]},{"osd":10,"up_from":72,"seq":309237645338,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6432,"kb_used_data":1048,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738720,"statfs":{"total":95995035648,"available":95988449280,"internally_reserved":0,"allocated":1073152,"data_stored":1096544,"data_compressed":41378,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":1,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.61499999999999999}]},{"osd":2,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.745}]},{"osd":3,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.72299999999999998}]},{"osd":4,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.64700000000000002}]},{"osd":5,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.80300000000000005}]},{"osd":6,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.77400000000000002}]},{"osd":7,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.505}]},{"osd":8,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.86099999999999999}]},{"osd":9,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.83199999999999996}]},{"osd":11,"last update":"Thu Nov 16 16:37:33 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.877},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.97799999999999998}]}]},{"osd":9,"up_from":67,"seq":287762808861,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6176,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":93738976,"statfs":{"total":95995035648,"available":95988711424,"internally_reserved":0,"allocated":942080,"data_stored":447145,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.45100000000000001,"15min":0.46300000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.59699999999999998,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.47199999999999998,"15min":0.48199999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.314}]},{"osd":1,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.59199999999999997,"15min":0.60599999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.61799999999999999,"15min":0.63300000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76100000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.41899999999999998}]},{"osd":2,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.63200000000000001,"15min":0.64600000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.78900000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.63500000000000001,"15min":0.64300000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.46600000000000003}]},{"osd":3,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.54800000000000004,"15min":0.56399999999999995},"min":{"1min":0.33200000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73299999999999998,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.55100000000000005,"15min":0.57299999999999995},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.70899999999999996,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.36799999999999999}]},{"osd":4,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.66200000000000003,"15min":0.68100000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.873,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.441},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.67300000000000004,"15min":0.68999999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.83599999999999997,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.47299999999999998}]},{"osd":5,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.69599999999999995,"15min":0.70999999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.878,"5min":1.056,"15min":1.056},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.72099999999999997,"15min":0.73899999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85999999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.70799999999999996}]},{"osd":6,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.64900000000000002,"15min":0.67200000000000004},"min":{"1min":0.38400000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.81699999999999995,"5min":1.069,"15min":1.069},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.70399999999999996,"15min":0.72799999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85099999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.48399999999999999}]},{"osd":7,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.755,"15min":0.77300000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92200000000000004,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.77100000000000002,"15min":0.79500000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.89600000000000002,"5min":1.111,"15min":1.111},"last":0.60499999999999998}]},{"osd":8,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.80400000000000005,"15min":0.82499999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.95899999999999996,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.80100000000000005,"15min":0.81599999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.97399999999999998,"5min":1.089,"15min":1.089},"last":0.66100000000000003}]},{"osd":10,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.55600000000000005}]}]},{"osd":8,"up_from":59,"seq":253403070499,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6280,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738872,"statfs":{"total":95995035648,"available":95988604928,"internally_reserved":0,"allocated":917504,"data_stored":429607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.77000000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.439},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.44400000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.46999999999999997}]},{"osd":1,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.627,"15min":0.61699999999999999},"min":{"1min":0.46100000000000002,"5min":0.318,"15min":0.318},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.749},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.46200000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.76700000000000002}]},{"osd":2,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.50600000000000001,"15min":0.51100000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71299999999999997,"5min":1.056,"15min":1.056},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.51500000000000001,"15min":0.52000000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71499999999999997,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.70499999999999996}]},{"osd":3,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.624,"15min":0.625},"min":{"1min":0.40999999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.75800000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.433,"5min":0.376,"15min":0.376},"max":{"1min":0.877,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.60599999999999998}]},{"osd":4,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.65800000000000003,"15min":0.63300000000000001},"min":{"1min":0.47099999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":4.0599999999999996,"5min":4.0599999999999996,"15min":4.0599999999999996},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69899999999999995,"15min":0.69799999999999995},"min":{"1min":0.42899999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.79400000000000004}]},{"osd":5,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60999999999999999,"15min":0.61199999999999999},"min":{"1min":0.33500000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.89000000000000001,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.624,"15min":0.63},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.84599999999999997,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.65100000000000002}]},{"osd":6,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.747,"15min":0.745},"min":{"1min":0.45800000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94599999999999995,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.745,"15min":0.75},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94299999999999995,"5min":1.173,"15min":1.173},"last":0.61899999999999999}]},{"osd":7,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.76800000000000002,"15min":0.76400000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.75700000000000001,"15min":0.753},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.81299999999999994}]},{"osd":9,"last update":"Thu Nov 16 16:37:55 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.73799999999999999,"15min":0.72999999999999998},"min":{"1min":0.54000000000000004,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.018,"5min":1.036,"15min":1.036},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.875,"5min":0.76900000000000002,"15min":0.751},"min":{"1min":0.58699999999999997,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.84599999999999997}]},{"osd":10,"last update":"Thu Nov 16 16:37:13 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":3.5720000000000001,"5min":3.5720000000000001,"15min":3.5720000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":3.5920000000000001,"5min":3.5920000000000001,"15min":3.5920000000000001},"last":0.94599999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]}]},{"osd":7,"up_from":52,"seq":223338299431,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6480,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738672,"statfs":{"total":95995035648,"available":95988400128,"internally_reserved":0,"allocated":991232,"data_stored":1038904,"data_compressed":41378,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:36:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55800000000000005}]},{"osd":1,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.41799999999999998,"15min":0.41099999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.434,"15min":0.42599999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.56699999999999995}]},{"osd":2,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.35799999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.84499999999999997}]},{"osd":3,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57499999999999996,"15min":0.57099999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.625,"15min":0.621},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.81699999999999995}]},{"osd":4,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.52400000000000002,"15min":0.51400000000000001},"min":{"1min":0.34799999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.498,"15min":0.48199999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.63100000000000001}]},{"osd":5,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.65200000000000002,"15min":0.64700000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91300000000000003,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.63800000000000001,"15min":0.63200000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":1.0349999999999999}]},{"osd":6,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.70299999999999996,"15min":0.69499999999999995},"min":{"1min":0.52500000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.75,"5min":0.68600000000000005,"15min":0.67600000000000005},"min":{"1min":0.47299999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.69099999999999995}]},{"osd":8,"last update":"Thu Nov 16 16:37:25 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.748,"15min":0.747},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0580000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":1},{"interface":"front","average":{"1min":0.752,"5min":0.76100000000000001,"15min":0.76300000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":1.012}]},{"osd":9,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.80300000000000005,"15min":0.78200000000000003},"min":{"1min":0.57999999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":4.7370000000000001,"5min":4.7370000000000001,"15min":4.7370000000000001},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.84399999999999997,"15min":0.82199999999999995},"min":{"1min":0.55600000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":4.7809999999999997,"5min":4.7809999999999997,"15min":4.7809999999999997},"last":0.96799999999999997}]},{"osd":10,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.92900000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]}]},{"osd":6,"up_from":47,"seq":201863462955,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6280,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738872,"statfs":{"total":95995035648,"available":95988604928,"internally_reserved":0,"allocated":917504,"data_stored":437783,"data_compressed":2903,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.439,"15min":0.41999999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":2.3140000000000001,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.47399999999999998,"15min":0.44600000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":3.472,"5min":3.472,"15min":3.472},"last":0.42299999999999999}]},{"osd":1,"last update":"Thu Nov 16 16:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.59699999999999998,"15min":0.58299999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":3.3290000000000002,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.64600000000000002,"15min":0.59999999999999998},"min":{"1min":0.38200000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":4.4669999999999996,"5min":4.4669999999999996,"15min":4.4669999999999996},"last":0.63300000000000001}]},{"osd":2,"last update":"Thu Nov 16 16:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.65400000000000003,"15min":0.63800000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":3.5369999999999999,"5min":3.5369999999999999,"15min":3.5369999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.66200000000000003,"15min":0.63400000000000001},"min":{"1min":0.46400000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.4049999999999998,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.58799999999999997}]},{"osd":3,"last update":"Thu Nov 16 16:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.52400000000000002,"15min":0.48999999999999999},"min":{"1min":0.30499999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":3.3700000000000001,"5min":3.3700000000000001,"15min":3.3700000000000001},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.55600000000000005,"15min":0.54100000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.495}]},{"osd":4,"last update":"Thu Nov 16 16:37:50 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.67600000000000005,"15min":0.60999999999999999},"min":{"1min":0.45300000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":4.4900000000000002,"5min":4.4900000000000002,"15min":4.4900000000000002},"last":0.56299999999999994},{"interface":"front","average":{"1min":1.026,"5min":0.69899999999999995,"15min":0.63100000000000001},"min":{"1min":0.45200000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":4.4509999999999996,"5min":4.4509999999999996,"15min":4.4509999999999996},"last":0.55200000000000005}]},{"osd":5,"last update":"Thu Nov 16 16:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.68899999999999995,"15min":0.68100000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":2.4209999999999998,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.72999999999999998,"15min":0.70999999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":3.5019999999999998,"5min":3.5019999999999998,"15min":3.5019999999999998},"last":0.68500000000000005}]},{"osd":7,"last update":"Thu Nov 16 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.70499999999999996,"15min":0.70299999999999996},"min":{"1min":0.48999999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.878,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.71499999999999997,"15min":0.70899999999999996},"min":{"1min":0.55300000000000005,"5min":0.436,"15min":0.436},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.621}]},{"osd":8,"last update":"Thu Nov 16 16:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.76100000000000001,"15min":0.77200000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.91200000000000003,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.77000000000000002,"15min":0.78000000000000003},"min":{"1min":0.54000000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.016,"5min":1.091,"15min":1.091},"last":0.64900000000000002}]},{"osd":9,"last update":"Thu Nov 16 16:37:50 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":0.81799999999999995,"15min":0.78200000000000003},"min":{"1min":0.50800000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":6.1539999999999999,"5min":6.1539999999999999,"15min":6.1539999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":1.1910000000000001,"5min":0.83999999999999997,"15min":0.78200000000000003},"min":{"1min":0.55100000000000005,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":8.2460000000000004,"5min":8.2460000000000004,"15min":8.2460000000000004},"last":0.50900000000000001}]},{"osd":10,"last update":"Thu Nov 16 16:37:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.66100000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7288,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":6336,"kb_avail":93737864,"statfs":{"total":95995035648,"available":95987572736,"internally_reserved":0,"allocated":966656,"data_stored":487396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6488064},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.48499999999999999,"15min":0.45800000000000002},"min":{"1min":0.33100000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.51200000000000001,"15min":0.48499999999999999},"min":{"1min":0.32000000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.42699999999999999}]},{"osd":2,"last update":"Thu Nov 16 16:37:19 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.54300000000000004,"15min":0.498},"min":{"1min":0.36699999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.84799999999999998,"5min":0.876,"15min":0.876},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.54000000000000004,"15min":0.503},"min":{"1min":0.40500000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.91000000000000003,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.59799999999999998}]},{"osd":3,"last update":"Thu Nov 16 16:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57699999999999996,"15min":0.55500000000000005},"min":{"1min":0.372,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85599999999999998,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59899999999999998,"15min":0.57499999999999996},"min":{"1min":0.41399999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Nov 16 16:37:03 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.441,"15min":0.42299999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.52200000000000002,"15min":0.47599999999999998},"min":{"1min":0.34599999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.81699999999999995}]},{"osd":5,"last update":"Thu Nov 16 16:37:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.63800000000000001,"15min":0.61099999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.621,"15min":0.59699999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97399999999999998,"5min":1.004,"15min":1.004},"last":0.85899999999999999}]},{"osd":6,"last update":"Thu Nov 16 16:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.68400000000000005,"15min":0.66800000000000004},"min":{"1min":0.44700000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.95099999999999996,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68400000000000005,"15min":0.66400000000000003},"min":{"1min":0.45500000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.95299999999999996,"5min":1,"15min":1},"last":0.627}]},{"osd":7,"last update":"Thu Nov 16 16:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.64500000000000002,"15min":0.63200000000000001},"min":{"1min":0.35199999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.69299999999999995,"15min":0.67700000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.64000000000000001}]},{"osd":8,"last update":"Thu Nov 16 16:37:27 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.75900000000000001,"15min":0.755},"min":{"1min":0.54800000000000004,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.74099999999999999,"15min":0.73699999999999999},"min":{"1min":0.47399999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.754}]},{"osd":9,"last update":"Thu Nov 16 16:37:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.77900000000000003,"15min":0.79900000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.87,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.84599999999999997,"15min":0.86399999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.97299999999999998,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.68600000000000005}]},{"osd":10,"last update":"Thu Nov 16 16:37:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.66100000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]}]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7184,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":6272,"kb_avail":93737968,"statfs":{"total":95995035648,"available":95987679232,"internally_reserved":0,"allocated":925696,"data_stored":430347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6422528},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Nov 16 16:37:04 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.503,"15min":0.46700000000000003},"min":{"1min":0.32500000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.499,"15min":0.46100000000000002},"min":{"1min":0.36099999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78800000000000003,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.66000000000000003}]},{"osd":2,"last update":"Thu Nov 16 16:37:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.55600000000000005,"15min":0.51400000000000001},"min":{"1min":0.41999999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.80600000000000005,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.496},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56299999999999994,"15min":0.53100000000000003},"min":{"1min":0.372,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.70399999999999996}]},{"osd":3,"last update":"Thu Nov 16 16:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.46500000000000002,"15min":0.45600000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.45700000000000002,"15min":0.44700000000000001},"min":{"1min":0.33700000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.35599999999999998}]},{"osd":4,"last update":"Thu Nov 16 16:37:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.57899999999999996,"15min":0.55100000000000005},"min":{"1min":0.38900000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.58199999999999996,"15min":0.55000000000000004},"min":{"1min":0.40999999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.61699999999999999}]},{"osd":5,"last update":"Thu Nov 16 16:37:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.56200000000000006,"15min":0.55100000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.66100000000000003,"5min":1.012,"15min":1.012},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64600000000000002,"15min":0.621},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.82699999999999996,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.437}]},{"osd":6,"last update":"Thu Nov 16 16:37:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.64800000000000002,"15min":0.59699999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":6.5190000000000001,"5min":6.5190000000000001,"15min":6.5190000000000001},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.63100000000000001,"15min":0.57499999999999996},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":9.141,"5min":9.141,"15min":9.141},"last":0.46000000000000002}]},{"osd":7,"last update":"Thu Nov 16 16:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.68400000000000005,"15min":0.67600000000000005},"min":{"1min":0.495,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.74199999999999999,"15min":0.73199999999999998},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.53400000000000003}]},{"osd":8,"last update":"Thu Nov 16 16:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.78800000000000003,"15min":0.80900000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.91900000000000004,"5min":1.163,"15min":1.163},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.83499999999999996,"15min":0.85299999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.90800000000000003,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.57099999999999995}]},{"osd":9,"last update":"Thu Nov 16 16:37:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.85499999999999998,"15min":0.86199999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":6.6349999999999998,"5min":6.6349999999999998,"15min":6.6349999999999998},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.79000000000000004,"15min":0.76600000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":9.1690000000000005,"5min":9.1690000000000005,"15min":9.1690000000000005},"last":0.51600000000000001}]},{"osd":10,"last update":"Thu Nov 16 16:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.69399999999999995}]},{"osd":11,"last update":"Thu Nov 16 16:37:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":9.1039999999999992,"5min":9.1039999999999992,"15min":9.1039999999999992},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":3.7530000000000001,"5min":3.7530000000000001,"15min":3.7530000000000001},"last":0.64500000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411390,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7464,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":93737688,"statfs":{"total":95995035648,"available":95987392512,"internally_reserved":0,"allocated":1015808,"data_stored":506546,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.29999999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.497,"15min":0.48799999999999999},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.56899999999999995}]},{"osd":1,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52600000000000002,"15min":0.51400000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.40500000000000003,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.89900000000000002,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.73399999999999999}]},{"osd":3,"last update":"Thu Nov 16 16:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.66300000000000003,"15min":0.60699999999999998},"min":{"1min":0.40699999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":6.9740000000000002,"5min":6.9740000000000002,"15min":6.9740000000000002},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.57499999999999996,"15min":0.55700000000000005},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.67300000000000004}]},{"osd":4,"last update":"Thu Nov 16 16:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.55300000000000005,"15min":0.53000000000000003},"min":{"1min":0.376,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59099999999999997,"15min":0.58199999999999996},"min":{"1min":0.40000000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83799999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.64800000000000002}]},{"osd":5,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.49399999999999999,"15min":0.48399999999999999},"min":{"1min":0.315,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.47199999999999998,"15min":0.45900000000000002},"min":{"1min":0.32200000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.54600000000000004}]},{"osd":6,"last update":"Thu Nov 16 16:37:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.68000000000000005,"15min":0.65400000000000003},"min":{"1min":0.40899999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.3969999999999998,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.64400000000000002,"15min":0.61299999999999999},"min":{"1min":0.39400000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":2.3460000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.70499999999999996}]},{"osd":7,"last update":"Thu Nov 16 16:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.64300000000000002,"15min":0.64400000000000002},"min":{"1min":0.41599999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70199999999999996,"15min":0.70399999999999996},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.92700000000000005,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.57699999999999996}]},{"osd":8,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.57499999999999996,"15min":0.56899999999999995},"min":{"1min":0.38500000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.85199999999999998,"5min":1.008,"15min":1.008},"last":0.627},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.628,"15min":0.625},"min":{"1min":0.38400000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.71299999999999997}]},{"osd":9,"last update":"Thu Nov 16 16:37:54 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.86799999999999999,"15min":0.86699999999999999},"min":{"1min":0.47499999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.6080000000000001,"5min":4.524,"15min":4.524},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.751,"15min":0.73299999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.6200000000000001,"5min":2.6200000000000001,"15min":2.6200000000000001},"last":0.72799999999999998}]},{"osd":10,"last update":"Thu Nov 16 16:37:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.746},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.76000000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]}]},{"osd":3,"up_from":26,"seq":111669149751,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6536,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738616,"statfs":{"total":95995035648,"available":95988342784,"internally_reserved":0,"allocated":917504,"data_stored":429068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36499999999999999,"15min":0.34899999999999998},"min":{"1min":0.26500000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.52400000000000002,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.38,"15min":0.35799999999999998},"min":{"1min":0.27800000000000002,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.55100000000000005,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.39600000000000002}]},{"osd":1,"last update":"Thu Nov 16 16:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.55400000000000005,"15min":0.54900000000000004},"min":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55900000000000005,"15min":0.55700000000000005},"min":{"1min":0.36099999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.83399999999999996,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.63200000000000001}]},{"osd":2,"last update":"Thu Nov 16 16:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57899999999999996,"15min":0.55700000000000005},"min":{"1min":0.42599999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.83299999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56699999999999995,"15min":0.54700000000000004},"min":{"1min":0.40600000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.69699999999999995,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.65100000000000002}]},{"osd":4,"last update":"Thu Nov 16 16:37:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55700000000000005,"15min":0.54700000000000004},"min":{"1min":0.376,"5min":0.37,"15min":0.37},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.45500000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.61399999999999999}]},{"osd":5,"last update":"Thu Nov 16 16:37:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.59799999999999998,"15min":0.57999999999999996},"min":{"1min":0.438,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60499999999999998,"15min":0.59099999999999997},"min":{"1min":0.38600000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.87,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.67300000000000004}]},{"osd":6,"last update":"Thu Nov 16 16:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51800000000000002,"15min":0.498},"min":{"1min":0.35599999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.72399999999999998,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.496},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53200000000000003,"15min":0.51400000000000001},"min":{"1min":0.377,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.75800000000000001,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.50600000000000001}]},{"osd":7,"last update":"Thu Nov 16 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.65600000000000003,"15min":0.65000000000000002},"min":{"1min":0.44,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.69299999999999995,"15min":0.68700000000000006},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.72899999999999998}]},{"osd":8,"last update":"Thu Nov 16 16:36:27 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.63200000000000001}]},{"osd":9,"last update":"Thu Nov 16 16:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.73799999999999999,"15min":0.76300000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80500000000000005,"5min":1.024,"15min":1.024},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.68999999999999995,"15min":0.70899999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90300000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.54000000000000004}]},{"osd":10,"last update":"Thu Nov 16 16:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.70099999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]}]},{"osd":4,"up_from":34,"seq":146028888116,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6424,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738728,"statfs":{"total":95995035648,"available":95988457472,"internally_reserved":0,"allocated":933888,"data_stored":446923,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.52400000000000002,"15min":0.502},"min":{"1min":0.38200000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.52900000000000003,"15min":0.51500000000000001},"min":{"1min":0.40899999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.378}]},{"osd":1,"last update":"Thu Nov 16 16:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.41099999999999998,"15min":0.39300000000000002},"min":{"1min":0.33700000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.67900000000000005,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.434,"15min":0.40999999999999998},"min":{"1min":0.378,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.748,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.42999999999999999}]},{"osd":2,"last update":"Thu Nov 16 16:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.77300000000000002,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.623,"15min":0.59299999999999997},"min":{"1min":0.53000000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.81999999999999995,"5min":1.056,"15min":1.056},"last":0.54700000000000004}]},{"osd":3,"last update":"Thu Nov 16 16:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.59199999999999997,"15min":0.55000000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57299999999999995,"15min":0.55300000000000005},"min":{"1min":0.40100000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.68899999999999995}]},{"osd":5,"last update":"Thu Nov 16 16:37:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.623,"15min":0.627},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.92100000000000004,"5min":1.075,"15min":1.075},"last":0.495},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.63800000000000001,"15min":0.626},"min":{"1min":0.52200000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.94499999999999995,"5min":1.093,"15min":1.093},"last":0.64800000000000002}]},{"osd":6,"last update":"Thu Nov 16 16:37:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.63,"15min":0.60199999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84799999999999998,"5min":1.004,"15min":1.004},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.60899999999999999,"15min":0.59499999999999997},"min":{"1min":0.42399999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.73799999999999999}]},{"osd":7,"last update":"Thu Nov 16 16:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.57199999999999995,"15min":0.54800000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.63200000000000001,"15min":0.60799999999999998},"min":{"1min":0.41199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.60199999999999998}]},{"osd":8,"last update":"Thu Nov 16 16:37:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.73999999999999999,"15min":0.72399999999999998},"min":{"1min":0.58999999999999997,"5min":0.495,"15min":0.495},"max":{"1min":1.0189999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.747,"15min":0.73099999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.77200000000000002}]},{"osd":9,"last update":"Thu Nov 16 16:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.79200000000000004,"15min":0.78900000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.94099999999999995,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.72099999999999997,"15min":0.71799999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94199999999999995,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.78300000000000003}]},{"osd":10,"last update":"Thu Nov 16 16:37:15 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.89900000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]}]},{"osd":5,"up_from":39,"seq":167503724593,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6556,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738596,"statfs":{"total":95995035648,"available":95988322304,"internally_reserved":0,"allocated":1003520,"data_stored":1038636,"data_compressed":41378,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.499,"15min":0.49399999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.64600000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52800000000000002,"15min":0.52400000000000002},"min":{"1min":0.312,"5min":0.311,"15min":0.311},"max":{"1min":0.75700000000000001,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.51400000000000001}]},{"osd":1,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.372,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.86399999999999999,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58799999999999997,"15min":0.58499999999999996},"min":{"1min":0.43099999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.94699999999999995,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.68400000000000005}]},{"osd":2,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.42799999999999999,"15min":0.42399999999999999},"min":{"1min":0.29599999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.67300000000000004,"5min":1.417,"15min":1.417},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.443,"15min":0.433},"min":{"1min":0.32300000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.69899999999999995,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.47299999999999998}]},{"osd":3,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55700000000000005,"15min":0.54700000000000004},"min":{"1min":0.36199999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.81100000000000005,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.60899999999999999}]},{"osd":4,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.67900000000000005,"15min":0.629},"min":{"1min":0.40100000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":5.5789999999999997,"5min":5.5789999999999997,"15min":5.5789999999999997},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59699999999999998,"15min":0.59099999999999997},"min":{"1min":0.40400000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.66000000000000003}]},{"osd":6,"last update":"Thu Nov 16 16:37:47 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.67800000000000005,"15min":0.63600000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":3.274,"5min":3.274,"15min":3.274},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.68100000000000005,"15min":0.64600000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":3.472,"5min":3.472,"15min":3.472},"last":0.58499999999999996}]},{"osd":7,"last update":"Thu Nov 16 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.70999999999999996,"15min":0.71199999999999997},"min":{"1min":0.503,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.83799999999999997,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.72499999999999998,"15min":0.72799999999999998},"min":{"1min":0.56599999999999995,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.83699999999999997,"5min":1.544,"15min":1.544},"last":0.77800000000000002}]},{"osd":8,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.64600000000000002,"15min":0.64500000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.751},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72199999999999998,"15min":0.72399999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.88400000000000001}]},{"osd":9,"last update":"Thu Nov 16 16:37:52 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":0.90100000000000002,"15min":0.82699999999999996},"min":{"1min":0.58699999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":5.5679999999999996,"5min":5.5679999999999996,"15min":5.5679999999999996},"last":0.85899999999999999},{"interface":"front","average":{"1min":1.099,"5min":0.81599999999999995,"15min":0.76900000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":5.3230000000000004,"5min":5.3230000000000004,"15min":5.3230000000000004},"last":0.81000000000000005}]},{"osd":10,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.84599999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33928,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":58527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-16T16:38:07.058 INFO:tasks.cephadm.ceph_manager.ceph:PG 2.1d is not active+clean 2023-11-16T16:38:07.058 INFO:tasks.cephadm.ceph_manager.ceph:{'pgid': '2.1d', 'version': "84'4", 'reported_seq': 2, 'reported_epoch': 85, 'state': 'peering', 'last_fresh': '2023-11-16T16:37:16.431881+0000', 'last_change': '2023-11-16T16:38:02.736871+0000', 'last_active': '2023-11-16T16:38:02.736871+0000', 'last_peered': '2023-11-16T16:37:16.431881+0000', 'last_clean': '2023-11-16T16:37:16.431881+0000', 'last_became_active': '2023-11-16T16:37:13.509431+0000', 'last_became_peered': '2023-11-16T16:37:13.509431+0000', 'last_unstale': '2023-11-16T16:37:16.431881+0000', 'last_undegraded': '2023-11-16T16:37:16.431881+0000', 'last_fullsized': '2023-11-16T16:37:16.431881+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 82, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-16T16:37:12.473444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-16T16:37:12.473444+0000', 'last_clean_scrub_stamp': '2023-11-16T16:37:12.473444+0000', 'objects_scrubbed': 0, 'log_size': 4, 'log_dups_size': 0, 'ondisk_log_size': 4, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 6, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 2, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5, 'num_write_kb': 6, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 2, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [11, 2, 7], 'acting': [11, 2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 11, 'acting_primary': 11, 'purged_snaps': []} 2023-11-16T16:38:07.058 INFO:tasks.cephadm.ceph_manager.ceph:PG 2.10 is not active+clean 2023-11-16T16:38:07.059 INFO:tasks.cephadm.ceph_manager.ceph:{'pgid': '2.10', 'version': "84'15", 'reported_seq': 2, 'reported_epoch': 85, 'state': 'peering', 'last_fresh': '2023-11-16T16:37:17.285577+0000', 'last_change': '2023-11-16T16:38:02.736820+0000', 'last_active': '2023-11-16T16:38:02.736820+0000', 'last_peered': '2023-11-16T16:37:17.285577+0000', 'last_clean': '2023-11-16T16:37:17.285577+0000', 'last_became_active': '2023-11-16T16:37:13.492599+0000', 'last_became_peered': '2023-11-16T16:37:13.492599+0000', 'last_unstale': '2023-11-16T16:37:17.285577+0000', 'last_undegraded': '2023-11-16T16:37:17.285577+0000', 'last_fullsized': '2023-11-16T16:37:17.285577+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 82, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-16T16:37:12.473444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-16T16:37:12.473444+0000', 'last_clean_scrub_stamp': '2023-11-16T16:37:12.473444+0000', 'objects_scrubbed': 0, 'log_size': 15, 'log_dups_size': 0, 'ondisk_log_size': 15, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 112, 'num_objects': 8, 'num_object_clones': 0, 'num_object_copies': 24, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 8, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 6, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [11, 0, 2], 'acting': [11, 0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 11, 'acting_primary': 11, 'purged_snaps': []} 2023-11-16T16:38:07.059 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph status --format=json 2023-11-16T16:38:07.747 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:38:07.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:07 smithi053 ceph-mon[100638]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:38:07.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:07 smithi053 ceph-mon[100638]: pgmap v289: 97 pgs: 2 peering, 95 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2023-11-16T16:38:07.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:07 smithi001 ceph-mon[98800]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:38:07.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:07 smithi001 ceph-mon[98800]: pgmap v289: 97 pgs: 2 peering, 95 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2023-11-16T16:38:08.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:07 smithi026 ceph-mon[100112]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:38:08.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:07 smithi026 ceph-mon[100112]: pgmap v289: 97 pgs: 2 peering, 95 active+clean; 638 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2023-11-16T16:38:08.934 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.930+0000 7f78b4d4d700 1 -- 172.21.15.1:0/4191871825 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f78b00ff460 msgr2=0x7f78b00ff830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:08.934 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.930+0000 7f78b4d4d700 1 --2- 172.21.15.1:0/4191871825 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f78b00ff460 0x7f78b00ff830 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f78a000b0d0 tx=0x7f78a000b3e0 comp rx=0 tx=0).stop 2023-11-16T16:38:08.934 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.930+0000 7f78b4d4d700 1 -- 172.21.15.1:0/4191871825 shutdown_connections 2023-11-16T16:38:08.935 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.930+0000 7f78b4d4d700 1 --2- 172.21.15.1:0/4191871825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78b0104500 0x7f78b010c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:08.935 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.930+0000 7f78b4d4d700 1 --2- 172.21.15.1:0/4191871825 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f78b00ff460 0x7f78b00ff830 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:08.935 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.930+0000 7f78b4d4d700 1 --2- 172.21.15.1:0/4191871825 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78b00ffe00 0x7f78b0103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:08.935 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.930+0000 7f78b4d4d700 1 -- 172.21.15.1:0/4191871825 >> 172.21.15.1:0/4191871825 conn(0x7f78b00770a0 msgr2=0x7f78b00774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:08.935 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.930+0000 7f78b4d4d700 1 -- 172.21.15.1:0/4191871825 shutdown_connections 2023-11-16T16:38:08.935 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.931+0000 7f78b4d4d700 1 -- 172.21.15.1:0/4191871825 wait complete. 2023-11-16T16:38:08.935 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.931+0000 7f78b4d4d700 1 Processor -- start 2023-11-16T16:38:08.935 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.931+0000 7f78b4d4d700 1 -- start start 2023-11-16T16:38:08.935 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.931+0000 7f78b4d4d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78b00ff460 0x7f78b01973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:08.935 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.931+0000 7f78b4d4d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78b00ffe00 0x7f78b01978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:08.935 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.931+0000 7f78b4d4d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f78b0104500 0x7f78b01a2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:08.935 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.932+0000 7f78b4d4d700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b01a2960 con 0x7f78b0104500 2023-11-16T16:38:08.936 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.932+0000 7f78b4d4d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b01a2aa0 con 0x7f78b00ffe00 2023-11-16T16:38:08.936 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.932+0000 7f78b4d4d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b01a2be0 con 0x7f78b00ff460 2023-11-16T16:38:08.936 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.932+0000 7f78aed9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f78b0104500 0x7f78b01a2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:08.936 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.932+0000 7f78aed9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f78b0104500 0x7f78b01a2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42104/0 (socket says 172.21.15.1:42104) 2023-11-16T16:38:08.936 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.932+0000 7f78aed9d700 1 -- 172.21.15.1:0/1136062171 learned_addr learned my addr 172.21.15.1:0/1136062171 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:08.936 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.932+0000 7f78ae59c700 1 --2- 172.21.15.1:0/1136062171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78b00ff460 0x7f78b01973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:08.936 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.932+0000 7f78add9b700 1 --2- 172.21.15.1:0/1136062171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78b00ffe00 0x7f78b01978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:08.936 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.932+0000 7f78aed9d700 1 -- 172.21.15.1:0/1136062171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78b00ff460 msgr2=0x7f78b01973a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:08.936 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.932+0000 7f78aed9d700 1 --2- 172.21.15.1:0/1136062171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78b00ff460 0x7f78b01973a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:08.936 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.932+0000 7f78aed9d700 1 -- 172.21.15.1:0/1136062171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78b00ffe00 msgr2=0x7f78b01978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:08.936 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.932+0000 7f78aed9d700 1 --2- 172.21.15.1:0/1136062171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78b00ffe00 0x7f78b01978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:08.937 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.932+0000 7f78aed9d700 1 -- 172.21.15.1:0/1136062171 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a0009d00 con 0x7f78b0104500 2023-11-16T16:38:08.937 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.932+0000 7f78ae59c700 1 --2- 172.21.15.1:0/1136062171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78b00ff460 0x7f78b01973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:38:08.937 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.932+0000 7f78aed9d700 1 --2- 172.21.15.1:0/1136062171 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f78b0104500 0x7f78b01a2420 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f78a800b550 tx=0x7f78a800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:08.937 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.932+0000 7f789f7fe700 1 -- 172.21.15.1:0/1136062171 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a8011830 con 0x7f78b0104500 2023-11-16T16:38:08.937 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.932+0000 7f78b4d4d700 1 -- 172.21.15.1:0/1136062171 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78b010f2c0 con 0x7f78b0104500 2023-11-16T16:38:08.937 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.932+0000 7f789f7fe700 1 -- 172.21.15.1:0/1136062171 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78a8011e50 con 0x7f78b0104500 2023-11-16T16:38:08.937 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.932+0000 7f789f7fe700 1 -- 172.21.15.1:0/1136062171 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a8010e70 con 0x7f78b0104500 2023-11-16T16:38:08.938 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.932+0000 7f78b4d4d700 1 -- 172.21.15.1:0/1136062171 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78b010f130 con 0x7f78b0104500 2023-11-16T16:38:08.938 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.933+0000 7f78b4d4d700 1 -- 172.21.15.1:0/1136062171 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78b010f450 con 0x7f78b0104500 2023-11-16T16:38:08.938 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.935+0000 7f789f7fe700 1 -- 172.21.15.1:0/1136062171 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f78a8010510 con 0x7f78b0104500 2023-11-16T16:38:08.939 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.935+0000 7f789f7fe700 1 --2- 172.21.15.1:0/1136062171 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f78980a11e0 0x7f78980a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:08.939 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.935+0000 7f789f7fe700 1 -- 172.21.15.1:0/1136062171 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f78a80cc9b0 con 0x7f78b0104500 2023-11-16T16:38:08.939 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.935+0000 7f78ae59c700 1 --2- 172.21.15.1:0/1136062171 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f78980a11e0 0x7f78980a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:08.939 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.936+0000 7f78ae59c700 1 --2- 172.21.15.1:0/1136062171 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f78980a11e0 0x7f78980a3690 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f78a0009fd0 tx=0x7f78a000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:08.940 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:08.939+0000 7f789f7fe700 1 -- 172.21.15.1:0/1136062171 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f78a80960b0 con 0x7f78b0104500 2023-11-16T16:38:09.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:09.217+0000 7f78b4d4d700 1 -- 172.21.15.1:0/1136062171 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f78b0066ea0 con 0x7f78b0104500 2023-11-16T16:38:09.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:09.217+0000 7f789f7fe700 1 -- 172.21.15.1:0/1136062171 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+2512 (secure 0 0 0) 0x7f78a80996d0 con 0x7f78b0104500 2023-11-16T16:38:09.220 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:38:09.220 INFO:teuthology.orchestra.run.smithi001.stdout:{"fsid":"35d9b010-849d-11ee-8dbe-212e2dc638e7","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":371,"monmap":{"epoch":3,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":86,"num_osds":12,"num_up_osds":12,"osd_up_since":1700152591,"num_in_osds":12,"osd_in_since":1700152574,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":95},{"state_name":"peering","count":2}],"num_pgs":97,"num_pools":3,"num_objects":64,"data_bytes":653045,"bytes_used":81227776,"bytes_avail":1151859200000,"bytes_total":1151940427776,"inactive_pgs_ratio":0.020618556067347527,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"read_bytes_sec":3071,"read_op_per_sec":6,"write_op_per_sec":0},"fsmap":{"epoch":23,"id":1,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":1,"rank":2,"name":"f","status":"up:standby-replay","gid":24463},{"filesystem_id":1,"rank":1,"name":"e","status":"up:standby-replay","gid":24467},{"filesystem_id":1,"rank":0,"name":"c","status":"up:standby-replay","gid":24472},{"filesystem_id":1,"rank":2,"name":"h","status":"up:active","gid":24482},{"filesystem_id":1,"rank":1,"name":"i","status":"up:active","gid":24493},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24521}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.1:8443/"}},"servicemap":{"epoch":24,"modified":"2023-11-16T16:38:07.484480+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"e":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"f":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-16T16:38:09.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:09.220+0000 7f78b4d4d700 1 -- 172.21.15.1:0/1136062171 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f78980a11e0 msgr2=0x7f78980a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:09.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:09.220+0000 7f78b4d4d700 1 --2- 172.21.15.1:0/1136062171 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f78980a11e0 0x7f78980a3690 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f78a0009fd0 tx=0x7f78a000b040 comp rx=0 tx=0).stop 2023-11-16T16:38:09.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:09.220+0000 7f78b4d4d700 1 -- 172.21.15.1:0/1136062171 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f78b0104500 msgr2=0x7f78b01a2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:09.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:09.220+0000 7f78b4d4d700 1 --2- 172.21.15.1:0/1136062171 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f78b0104500 0x7f78b01a2420 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f78a800b550 tx=0x7f78a800b910 comp rx=0 tx=0).stop 2023-11-16T16:38:09.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:09.220+0000 7f78b4d4d700 1 -- 172.21.15.1:0/1136062171 shutdown_connections 2023-11-16T16:38:09.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:09.220+0000 7f78b4d4d700 1 --2- 172.21.15.1:0/1136062171 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f78980a11e0 0x7f78980a3690 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:09.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:09.220+0000 7f78b4d4d700 1 --2- 172.21.15.1:0/1136062171 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f78b0104500 0x7f78b01a2420 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:09.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:09.220+0000 7f78b4d4d700 1 --2- 172.21.15.1:0/1136062171 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f78b00ff460 0x7f78b01973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:09.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:09.220+0000 7f78b4d4d700 1 --2- 172.21.15.1:0/1136062171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78b00ffe00 0x7f78b01978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:09.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:09.220+0000 7f78b4d4d700 1 -- 172.21.15.1:0/1136062171 >> 172.21.15.1:0/1136062171 conn(0x7f78b00770a0 msgr2=0x7f78b00fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:09.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:09.220+0000 7f78b4d4d700 1 -- 172.21.15.1:0/1136062171 shutdown_connections 2023-11-16T16:38:09.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:09.220+0000 7f78b4d4d700 1 -- 172.21.15.1:0/1136062171 wait complete. 2023-11-16T16:38:09.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:09 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1136062171' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-11-16T16:38:09.593 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:09 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1136062171' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-11-16T16:38:09.595 INFO:tasks.cephadm.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-11-16T16:38:09.595 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph pg dump --format=json 2023-11-16T16:38:09.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:09 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1136062171' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-11-16T16:38:10.251 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:38:10.529 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:10 smithi001 ceph-mon[98800]: pgmap v290: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 13 B/s, 1 keys/s, 1 objects/s recovering 2023-11-16T16:38:10.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:10 smithi053 ceph-mon[100638]: pgmap v290: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 13 B/s, 1 keys/s, 1 objects/s recovering 2023-11-16T16:38:10.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:10 smithi026 ceph-mon[100112]: pgmap v290: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 13 B/s, 1 keys/s, 1 objects/s recovering 2023-11-16T16:38:11.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.565+0000 7f2427ab9700 1 -- 172.21.15.1:0/1219321467 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24200ffe10 msgr2=0x7f2420103de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:11.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.565+0000 7f2427ab9700 1 --2- 172.21.15.1:0/1219321467 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24200ffe10 0x7f2420103de0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f24100099c0 tx=0x7f2410009cd0 comp rx=0 tx=0).stop 2023-11-16T16:38:11.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.565+0000 7f2427ab9700 1 -- 172.21.15.1:0/1219321467 shutdown_connections 2023-11-16T16:38:11.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.565+0000 7f2427ab9700 1 --2- 172.21.15.1:0/1219321467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2420104510 0x7f242010c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:11.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.565+0000 7f2427ab9700 1 --2- 172.21.15.1:0/1219321467 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24200ff470 0x7f24200ff840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:11.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.565+0000 7f2427ab9700 1 --2- 172.21.15.1:0/1219321467 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24200ffe10 0x7f2420103de0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:11.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.565+0000 7f2427ab9700 1 -- 172.21.15.1:0/1219321467 >> 172.21.15.1:0/1219321467 conn(0x7f24200770d0 msgr2=0x7f24200774d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:11.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.565+0000 7f2427ab9700 1 -- 172.21.15.1:0/1219321467 shutdown_connections 2023-11-16T16:38:11.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.565+0000 7f2427ab9700 1 -- 172.21.15.1:0/1219321467 wait complete. 2023-11-16T16:38:11.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.565+0000 7f2427ab9700 1 Processor -- start 2023-11-16T16:38:11.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.566+0000 7f2427ab9700 1 -- start start 2023-11-16T16:38:11.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.566+0000 7f2427ab9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24200ff470 0x7f24201963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:11.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.566+0000 7f2427ab9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24200ffe10 0x7f2420196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:11.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.566+0000 7f2427ab9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2420104510 0x7f242019ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:11.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.566+0000 7f2427ab9700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242019b3a0 con 0x7f24200ff470 2023-11-16T16:38:11.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.566+0000 7f2427ab9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242019b510 con 0x7f2420104510 2023-11-16T16:38:11.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.566+0000 7f2427ab9700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242019b6d0 con 0x7f24200ffe10 2023-11-16T16:38:11.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.566+0000 7f2425855700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24200ff470 0x7f24201963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:11.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.566+0000 7f2425855700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24200ff470 0x7f24201963e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42114/0 (socket says 172.21.15.1:42114) 2023-11-16T16:38:11.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.566+0000 7f2425855700 1 -- 172.21.15.1:0/3002662412 learned_addr learned my addr 172.21.15.1:0/3002662412 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:11.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.566+0000 7f2425855700 1 -- 172.21.15.1:0/3002662412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24200ffe10 msgr2=0x7f2420196920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:11.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.566+0000 7f2425855700 1 --2- 172.21.15.1:0/3002662412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24200ffe10 0x7f2420196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:11.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.566+0000 7f2425855700 1 -- 172.21.15.1:0/3002662412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2420104510 msgr2=0x7f242019ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:11.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.566+0000 7f2426056700 1 --2- 172.21.15.1:0/3002662412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2420104510 0x7f242019ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:11.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.566+0000 7f2425855700 1 --2- 172.21.15.1:0/3002662412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2420104510 0x7f242019ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:11.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.566+0000 7f2425855700 1 -- 172.21.15.1:0/3002662412 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24100095f0 con 0x7f24200ff470 2023-11-16T16:38:11.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.566+0000 7f2426056700 1 --2- 172.21.15.1:0/3002662412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2420104510 0x7f242019ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:38:11.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.566+0000 7f2425855700 1 --2- 172.21.15.1:0/3002662412 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24200ff470 0x7f24201963e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f241c00d980 tx=0x7f241c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:11.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.566+0000 7f2416ffd700 1 -- 172.21.15.1:0/3002662412 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c009b40 con 0x7f24200ff470 2023-11-16T16:38:11.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.567+0000 7f2427ab9700 1 -- 172.21.15.1:0/3002662412 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f242010f240 con 0x7f24200ff470 2023-11-16T16:38:11.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.567+0000 7f2416ffd700 1 -- 172.21.15.1:0/3002662412 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f241c00de60 con 0x7f24200ff470 2023-11-16T16:38:11.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.567+0000 7f2416ffd700 1 -- 172.21.15.1:0/3002662412 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c005580 con 0x7f24200ff470 2023-11-16T16:38:11.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.567+0000 7f2427ab9700 1 -- 172.21.15.1:0/3002662412 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f242010f0b0 con 0x7f24200ff470 2023-11-16T16:38:11.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.568+0000 7f2427ab9700 1 -- 172.21.15.1:0/3002662412 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f242010f3d0 con 0x7f24200ff470 2023-11-16T16:38:11.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.569+0000 7f2416ffd700 1 -- 172.21.15.1:0/3002662412 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f241c004750 con 0x7f24200ff470 2023-11-16T16:38:11.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.569+0000 7f2416ffd700 1 --2- 172.21.15.1:0/3002662412 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f240c0a11e0 0x7f240c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:11.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.570+0000 7f2425054700 1 --2- 172.21.15.1:0/3002662412 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f240c0a11e0 0x7f240c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:11.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.570+0000 7f2416ffd700 1 -- 172.21.15.1:0/3002662412 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f241c0cce90 con 0x7f24200ff470 2023-11-16T16:38:11.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.570+0000 7f2425054700 1 --2- 172.21.15.1:0/3002662412 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f240c0a11e0 0x7f240c0a3690 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2410009990 tx=0x7f2410000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:11.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.574+0000 7f2416ffd700 1 -- 172.21.15.1:0/3002662412 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f241c096480 con 0x7f24200ff470 2023-11-16T16:38:11.769 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.767+0000 7f2427ab9700 1 -- 172.21.15.1:0/3002662412 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f24200696d0 con 0x7f240c0a11e0 2023-11-16T16:38:11.780 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.779+0000 7f2416ffd700 1 -- 172.21.15.1:0/3002662412 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340282 (secure 0 0 0) 0x7f24200696d0 con 0x7f240c0a11e0 2023-11-16T16:38:11.785 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:38:11.788 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.782+0000 7f2427ab9700 1 -- 172.21.15.1:0/3002662412 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f240c0a11e0 msgr2=0x7f240c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:11.788 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.782+0000 7f2427ab9700 1 --2- 172.21.15.1:0/3002662412 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f240c0a11e0 0x7f240c0a3690 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2410009990 tx=0x7f2410000bc0 comp rx=0 tx=0).stop 2023-11-16T16:38:11.788 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.782+0000 7f2427ab9700 1 -- 172.21.15.1:0/3002662412 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24200ff470 msgr2=0x7f24201963e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:11.788 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.782+0000 7f2427ab9700 1 --2- 172.21.15.1:0/3002662412 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24200ff470 0x7f24201963e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f241c00d980 tx=0x7f241c00dd40 comp rx=0 tx=0).stop 2023-11-16T16:38:11.788 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.782+0000 7f2427ab9700 1 -- 172.21.15.1:0/3002662412 shutdown_connections 2023-11-16T16:38:11.788 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.782+0000 7f2427ab9700 1 --2- 172.21.15.1:0/3002662412 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f240c0a11e0 0x7f240c0a3690 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:11.788 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.782+0000 7f2427ab9700 1 --2- 172.21.15.1:0/3002662412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2420104510 0x7f242019ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:11.788 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.782+0000 7f2427ab9700 1 --2- 172.21.15.1:0/3002662412 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24200ff470 0x7f24201963e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:11.788 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.782+0000 7f2427ab9700 1 --2- 172.21.15.1:0/3002662412 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f24200ffe10 0x7f2420196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:11.789 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.782+0000 7f2427ab9700 1 -- 172.21.15.1:0/3002662412 >> 172.21.15.1:0/3002662412 conn(0x7f24200770d0 msgr2=0x7f2420068fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:11.789 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.782+0000 7f2427ab9700 1 -- 172.21.15.1:0/3002662412 shutdown_connections 2023-11-16T16:38:11.789 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:11.782+0000 7f2427ab9700 1 -- 172.21.15.1:0/3002662412 wait complete. 2023-11-16T16:38:11.789 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2023-11-16T16:38:12.215 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2023-11-16T16:38:11.485400+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653045,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":470,"num_read_kb":351,"num_write":333,"num_write_kb":3043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180848,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":292,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":79432,"kb_used_data":11368,"kb_used_omap":0,"kb_used_meta":67968,"kb_avail":1124862392,"statfs":{"total":1151940427776,"available":1151859089408,"internally_reserved":0,"allocated":11640832,"data_stored":7314397,"data_compressed":176451,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":0,"internal_metadata":69599232},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":29,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.305903"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.187022+0000","last_change":"2023-11-16T16:37:13.941551+0000","last_active":"2023-11-16T16:38:03.187022+0000","last_peered":"2023-11-16T16:38:03.187022+0000","last_clean":"2023-11-16T16:38:03.187022+0000","last_became_active":"2023-11-16T16:37:13.941190+0000","last_became_peered":"2023-11-16T16:37:13.941190+0000","last_unstale":"2023-11-16T16:38:03.187022+0000","last_undegraded":"2023-11-16T16:38:03.187022+0000","last_fullsized":"2023-11-16T16:38:03.187022+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:42:03.561127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.186116+0000","last_change":"2023-11-16T16:37:13.942300+0000","last_active":"2023-11-16T16:38:03.186116+0000","last_peered":"2023-11-16T16:38:03.186116+0000","last_clean":"2023-11-16T16:38:03.186116+0000","last_became_active":"2023-11-16T16:37:13.941966+0000","last_became_peered":"2023-11-16T16:37:13.941966+0000","last_unstale":"2023-11-16T16:38:03.186116+0000","last_undegraded":"2023-11-16T16:38:03.186116+0000","last_fullsized":"2023-11-16T16:38:03.186116+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:12:41.529896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.186042+0000","last_change":"2023-11-16T16:37:13.943461+0000","last_active":"2023-11-16T16:38:03.186042+0000","last_peered":"2023-11-16T16:38:03.186042+0000","last_clean":"2023-11-16T16:38:03.186042+0000","last_became_active":"2023-11-16T16:37:13.943165+0000","last_became_peered":"2023-11-16T16:37:13.943165+0000","last_unstale":"2023-11-16T16:38:03.186042+0000","last_undegraded":"2023-11-16T16:38:03.186042+0000","last_fullsized":"2023-11-16T16:38:03.186042+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:17:44.567499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184244+0000","last_change":"2023-11-16T16:37:13.940205+0000","last_active":"2023-11-16T16:38:03.184244+0000","last_peered":"2023-11-16T16:38:03.184244+0000","last_clean":"2023-11-16T16:38:03.184244+0000","last_became_active":"2023-11-16T16:37:13.939819+0000","last_became_peered":"2023-11-16T16:37:13.939819+0000","last_unstale":"2023-11-16T16:38:03.184244+0000","last_undegraded":"2023-11-16T16:38:03.184244+0000","last_fullsized":"2023-11-16T16:38:03.184244+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:52:42.183240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.190889+0000","last_change":"2023-11-16T16:37:13.939819+0000","last_active":"2023-11-16T16:38:03.190889+0000","last_peered":"2023-11-16T16:38:03.190889+0000","last_clean":"2023-11-16T16:38:03.190889+0000","last_became_active":"2023-11-16T16:37:13.939367+0000","last_became_peered":"2023-11-16T16:37:13.939367+0000","last_unstale":"2023-11-16T16:38:03.190889+0000","last_undegraded":"2023-11-16T16:38:03.190889+0000","last_fullsized":"2023-11-16T16:38:03.190889+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:17:30.563282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.187402+0000","last_change":"2023-11-16T16:37:13.946613+0000","last_active":"2023-11-16T16:38:03.187402+0000","last_peered":"2023-11-16T16:38:03.187402+0000","last_clean":"2023-11-16T16:38:03.187402+0000","last_became_active":"2023-11-16T16:37:13.946281+0000","last_became_peered":"2023-11-16T16:37:13.946281+0000","last_unstale":"2023-11-16T16:38:03.187402+0000","last_undegraded":"2023-11-16T16:38:03.187402+0000","last_fullsized":"2023-11-16T16:38:03.187402+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:29:42.599960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184741+0000","last_change":"2023-11-16T16:37:13.936899+0000","last_active":"2023-11-16T16:38:03.184741+0000","last_peered":"2023-11-16T16:38:03.184741+0000","last_clean":"2023-11-16T16:38:03.184741+0000","last_became_active":"2023-11-16T16:37:13.936543+0000","last_became_peered":"2023-11-16T16:37:13.936543+0000","last_unstale":"2023-11-16T16:38:03.184741+0000","last_undegraded":"2023-11-16T16:38:03.184741+0000","last_fullsized":"2023-11-16T16:38:03.184741+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:43:46.820689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.189989+0000","last_change":"2023-11-16T16:38:03.189467+0000","last_active":"2023-11-16T16:38:03.189989+0000","last_peered":"2023-11-16T16:38:03.189989+0000","last_clean":"2023-11-16T16:38:03.189989+0000","last_became_active":"2023-11-16T16:38:03.189017+0000","last_became_peered":"2023-11-16T16:38:03.189017+0000","last_unstale":"2023-11-16T16:38:03.189989+0000","last_undegraded":"2023-11-16T16:38:03.189989+0000","last_fullsized":"2023-11-16T16:38:03.189989+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:47:50.088436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.188864+0000","last_change":"2023-11-16T16:37:13.934105+0000","last_active":"2023-11-16T16:38:03.188864+0000","last_peered":"2023-11-16T16:38:03.188864+0000","last_clean":"2023-11-16T16:38:03.188864+0000","last_became_active":"2023-11-16T16:37:13.933887+0000","last_became_peered":"2023-11-16T16:37:13.933887+0000","last_unstale":"2023-11-16T16:38:03.188864+0000","last_undegraded":"2023-11-16T16:38:03.188864+0000","last_fullsized":"2023-11-16T16:38:03.188864+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:53:26.890511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.191855+0000","last_change":"2023-11-16T16:38:03.191855+0000","last_active":"2023-11-16T16:38:03.191855+0000","last_peered":"2023-11-16T16:38:03.191855+0000","last_clean":"2023-11-16T16:38:03.191855+0000","last_became_active":"2023-11-16T16:38:03.191398+0000","last_became_peered":"2023-11-16T16:38:03.191398+0000","last_unstale":"2023-11-16T16:38:03.191855+0000","last_undegraded":"2023-11-16T16:38:03.191855+0000","last_fullsized":"2023-11-16T16:38:03.191855+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:09:44.431927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184854+0000","last_change":"2023-11-16T16:37:13.941597+0000","last_active":"2023-11-16T16:38:03.184854+0000","last_peered":"2023-11-16T16:38:03.184854+0000","last_clean":"2023-11-16T16:38:03.184854+0000","last_became_active":"2023-11-16T16:37:13.941253+0000","last_became_peered":"2023-11-16T16:37:13.941253+0000","last_unstale":"2023-11-16T16:38:03.184854+0000","last_undegraded":"2023-11-16T16:38:03.184854+0000","last_fullsized":"2023-11-16T16:38:03.184854+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:13:35.256952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184130+0000","last_change":"2023-11-16T16:37:13.941673+0000","last_active":"2023-11-16T16:38:03.184130+0000","last_peered":"2023-11-16T16:38:03.184130+0000","last_clean":"2023-11-16T16:38:03.184130+0000","last_became_active":"2023-11-16T16:37:13.941345+0000","last_became_peered":"2023-11-16T16:37:13.941345+0000","last_unstale":"2023-11-16T16:38:03.184130+0000","last_undegraded":"2023-11-16T16:38:03.184130+0000","last_fullsized":"2023-11-16T16:38:03.184130+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:17:51.018141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184481+0000","last_change":"2023-11-16T16:37:13.947991+0000","last_active":"2023-11-16T16:38:03.184481+0000","last_peered":"2023-11-16T16:38:03.184481+0000","last_clean":"2023-11-16T16:38:03.184481+0000","last_became_active":"2023-11-16T16:37:13.947454+0000","last_became_peered":"2023-11-16T16:37:13.947454+0000","last_unstale":"2023-11-16T16:38:03.184481+0000","last_undegraded":"2023-11-16T16:38:03.184481+0000","last_fullsized":"2023-11-16T16:38:03.184481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:19:19.825458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.186271+0000","last_change":"2023-11-16T16:37:13.941502+0000","last_active":"2023-11-16T16:38:03.186271+0000","last_peered":"2023-11-16T16:38:03.186271+0000","last_clean":"2023-11-16T16:38:03.186271+0000","last_became_active":"2023-11-16T16:37:13.941191+0000","last_became_peered":"2023-11-16T16:37:13.941191+0000","last_unstale":"2023-11-16T16:38:03.186271+0000","last_undegraded":"2023-11-16T16:38:03.186271+0000","last_fullsized":"2023-11-16T16:38:03.186271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:19:35.239602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185898+0000","last_change":"2023-11-16T16:37:13.949198+0000","last_active":"2023-11-16T16:38:03.185898+0000","last_peered":"2023-11-16T16:38:03.185898+0000","last_clean":"2023-11-16T16:38:03.185898+0000","last_became_active":"2023-11-16T16:37:13.947147+0000","last_became_peered":"2023-11-16T16:37:13.947147+0000","last_unstale":"2023-11-16T16:38:03.185898+0000","last_undegraded":"2023-11-16T16:38:03.185898+0000","last_fullsized":"2023-11-16T16:38:03.185898+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:55:04.616796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.188109+0000","last_change":"2023-11-16T16:37:13.943915+0000","last_active":"2023-11-16T16:38:03.188109+0000","last_peered":"2023-11-16T16:38:03.188109+0000","last_clean":"2023-11-16T16:38:03.188109+0000","last_became_active":"2023-11-16T16:37:13.942995+0000","last_became_peered":"2023-11-16T16:37:13.942995+0000","last_unstale":"2023-11-16T16:38:03.188109+0000","last_undegraded":"2023-11-16T16:38:03.188109+0000","last_fullsized":"2023-11-16T16:38:03.188109+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:23:05.411045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.187167+0000","last_change":"2023-11-16T16:37:13.941816+0000","last_active":"2023-11-16T16:38:03.187167+0000","last_peered":"2023-11-16T16:38:03.187167+0000","last_clean":"2023-11-16T16:38:03.187167+0000","last_became_active":"2023-11-16T16:37:13.941541+0000","last_became_peered":"2023-11-16T16:37:13.941541+0000","last_unstale":"2023-11-16T16:38:03.187167+0000","last_undegraded":"2023-11-16T16:38:03.187167+0000","last_fullsized":"2023-11-16T16:38:03.187167+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:53:11.142308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185550+0000","last_change":"2023-11-16T16:37:13.940639+0000","last_active":"2023-11-16T16:38:03.185550+0000","last_peered":"2023-11-16T16:38:03.185550+0000","last_clean":"2023-11-16T16:38:03.185550+0000","last_became_active":"2023-11-16T16:37:13.940323+0000","last_became_peered":"2023-11-16T16:37:13.940323+0000","last_unstale":"2023-11-16T16:38:03.185550+0000","last_undegraded":"2023-11-16T16:38:03.185550+0000","last_fullsized":"2023-11-16T16:38:03.185550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:32:03.882320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.189374+0000","last_change":"2023-11-16T16:37:13.934802+0000","last_active":"2023-11-16T16:38:03.189374+0000","last_peered":"2023-11-16T16:38:03.189374+0000","last_clean":"2023-11-16T16:38:03.189374+0000","last_became_active":"2023-11-16T16:37:13.934496+0000","last_became_peered":"2023-11-16T16:37:13.934496+0000","last_unstale":"2023-11-16T16:38:03.189374+0000","last_undegraded":"2023-11-16T16:38:03.189374+0000","last_fullsized":"2023-11-16T16:38:03.189374+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:15:58.249424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185585+0000","last_change":"2023-11-16T16:37:13.499284+0000","last_active":"2023-11-16T16:38:03.185585+0000","last_peered":"2023-11-16T16:38:03.185585+0000","last_clean":"2023-11-16T16:38:03.185585+0000","last_became_active":"2023-11-16T16:37:13.498931+0000","last_became_peered":"2023-11-16T16:37:13.498931+0000","last_unstale":"2023-11-16T16:38:03.185585+0000","last_undegraded":"2023-11-16T16:38:03.185585+0000","last_fullsized":"2023-11-16T16:38:03.185585+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:53:33.061352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.182899+0000","last_change":"2023-11-16T16:37:13.939631+0000","last_active":"2023-11-16T16:38:03.182899+0000","last_peered":"2023-11-16T16:38:03.182899+0000","last_clean":"2023-11-16T16:38:03.182899+0000","last_became_active":"2023-11-16T16:37:13.939318+0000","last_became_peered":"2023-11-16T16:37:13.939318+0000","last_unstale":"2023-11-16T16:38:03.182899+0000","last_undegraded":"2023-11-16T16:38:03.182899+0000","last_fullsized":"2023-11-16T16:38:03.182899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:11:33.421798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.183008+0000","last_change":"2023-11-16T16:37:13.495972+0000","last_active":"2023-11-16T16:38:03.183008+0000","last_peered":"2023-11-16T16:38:03.183008+0000","last_clean":"2023-11-16T16:38:03.183008+0000","last_became_active":"2023-11-16T16:37:13.495610+0000","last_became_peered":"2023-11-16T16:37:13.495610+0000","last_unstale":"2023-11-16T16:38:03.183008+0000","last_undegraded":"2023-11-16T16:38:03.183008+0000","last_fullsized":"2023-11-16T16:38:03.183008+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:47:20.115622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.186338+0000","last_change":"2023-11-16T16:37:13.935236+0000","last_active":"2023-11-16T16:38:03.186338+0000","last_peered":"2023-11-16T16:38:03.186338+0000","last_clean":"2023-11-16T16:38:03.186338+0000","last_became_active":"2023-11-16T16:37:13.934920+0000","last_became_peered":"2023-11-16T16:37:13.934920+0000","last_unstale":"2023-11-16T16:38:03.186338+0000","last_undegraded":"2023-11-16T16:38:03.186338+0000","last_fullsized":"2023-11-16T16:38:03.186338+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:04:59.207169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"84'4","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.197657+0000","last_change":"2023-11-16T16:38:03.197657+0000","last_active":"2023-11-16T16:38:03.197657+0000","last_peered":"2023-11-16T16:38:03.197657+0000","last_clean":"2023-11-16T16:38:03.197657+0000","last_became_active":"2023-11-16T16:38:03.190952+0000","last_became_peered":"2023-11-16T16:38:03.190952+0000","last_unstale":"2023-11-16T16:38:03.197657+0000","last_undegraded":"2023-11-16T16:38:03.197657+0000","last_fullsized":"2023-11-16T16:38:03.197657+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:04:50.256412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.188922+0000","last_change":"2023-11-16T16:37:13.940838+0000","last_active":"2023-11-16T16:38:03.188922+0000","last_peered":"2023-11-16T16:38:03.188922+0000","last_clean":"2023-11-16T16:38:03.188922+0000","last_became_active":"2023-11-16T16:37:13.940538+0000","last_became_peered":"2023-11-16T16:37:13.940538+0000","last_unstale":"2023-11-16T16:38:03.188922+0000","last_undegraded":"2023-11-16T16:38:03.188922+0000","last_fullsized":"2023-11-16T16:38:03.188922+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:47:36.031628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.190408+0000","last_change":"2023-11-16T16:37:13.495114+0000","last_active":"2023-11-16T16:38:03.190408+0000","last_peered":"2023-11-16T16:38:03.190408+0000","last_clean":"2023-11-16T16:38:03.190408+0000","last_became_active":"2023-11-16T16:37:13.494807+0000","last_became_peered":"2023-11-16T16:37:13.494807+0000","last_unstale":"2023-11-16T16:38:03.190408+0000","last_undegraded":"2023-11-16T16:38:03.190408+0000","last_fullsized":"2023-11-16T16:38:03.190408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:35:31.040933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185195+0000","last_change":"2023-11-16T16:37:13.943652+0000","last_active":"2023-11-16T16:38:03.185195+0000","last_peered":"2023-11-16T16:38:03.185195+0000","last_clean":"2023-11-16T16:38:03.185195+0000","last_became_active":"2023-11-16T16:37:13.943298+0000","last_became_peered":"2023-11-16T16:37:13.943298+0000","last_unstale":"2023-11-16T16:38:03.185195+0000","last_undegraded":"2023-11-16T16:38:03.185195+0000","last_fullsized":"2023-11-16T16:38:03.185195+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:17:55.154731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"84'6","reported_seq":79,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:09.379655+0000","last_change":"2023-11-16T16:37:13.509801+0000","last_active":"2023-11-16T16:38:09.379655+0000","last_peered":"2023-11-16T16:38:09.379655+0000","last_clean":"2023-11-16T16:38:09.379655+0000","last_became_active":"2023-11-16T16:37:13.509389+0000","last_became_peered":"2023-11-16T16:37:13.509389+0000","last_unstale":"2023-11-16T16:38:09.379655+0000","last_undegraded":"2023-11-16T16:38:09.379655+0000","last_fullsized":"2023-11-16T16:38:09.379655+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:46:55.751281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.189967+0000","last_change":"2023-11-16T16:37:13.933871+0000","last_active":"2023-11-16T16:38:03.189967+0000","last_peered":"2023-11-16T16:38:03.189967+0000","last_clean":"2023-11-16T16:38:03.189967+0000","last_became_active":"2023-11-16T16:37:13.933549+0000","last_became_peered":"2023-11-16T16:37:13.933549+0000","last_unstale":"2023-11-16T16:38:03.189967+0000","last_undegraded":"2023-11-16T16:38:03.189967+0000","last_fullsized":"2023-11-16T16:38:03.189967+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:35:18.854524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"84'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.187212+0000","last_change":"2023-11-16T16:37:13.501339+0000","last_active":"2023-11-16T16:38:03.187212+0000","last_peered":"2023-11-16T16:38:03.187212+0000","last_clean":"2023-11-16T16:38:03.187212+0000","last_became_active":"2023-11-16T16:37:13.500949+0000","last_became_peered":"2023-11-16T16:37:13.500949+0000","last_unstale":"2023-11-16T16:38:03.187212+0000","last_undegraded":"2023-11-16T16:38:03.187212+0000","last_fullsized":"2023-11-16T16:38:03.187212+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:56:34.503681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.192595+0000","last_change":"2023-11-16T16:38:03.192595+0000","last_active":"2023-11-16T16:38:03.192595+0000","last_peered":"2023-11-16T16:38:03.192595+0000","last_clean":"2023-11-16T16:38:03.192595+0000","last_became_active":"2023-11-16T16:38:03.192208+0000","last_became_peered":"2023-11-16T16:38:03.192208+0000","last_unstale":"2023-11-16T16:38:03.192595+0000","last_undegraded":"2023-11-16T16:38:03.192595+0000","last_fullsized":"2023-11-16T16:38:03.192595+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:30:24.627049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"84'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184789+0000","last_change":"2023-11-16T16:37:13.505481+0000","last_active":"2023-11-16T16:38:03.184789+0000","last_peered":"2023-11-16T16:38:03.184789+0000","last_clean":"2023-11-16T16:38:03.184789+0000","last_became_active":"2023-11-16T16:37:13.505085+0000","last_became_peered":"2023-11-16T16:37:13.505085+0000","last_unstale":"2023-11-16T16:38:03.184789+0000","last_undegraded":"2023-11-16T16:38:03.184789+0000","last_fullsized":"2023-11-16T16:38:03.184789+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:21:13.673154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.182224+0000","last_change":"2023-11-16T16:37:13.943985+0000","last_active":"2023-11-16T16:38:03.182224+0000","last_peered":"2023-11-16T16:38:03.182224+0000","last_clean":"2023-11-16T16:38:03.182224+0000","last_became_active":"2023-11-16T16:37:13.943628+0000","last_became_peered":"2023-11-16T16:37:13.943628+0000","last_unstale":"2023-11-16T16:38:03.182224+0000","last_undegraded":"2023-11-16T16:38:03.182224+0000","last_fullsized":"2023-11-16T16:38:03.182224+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:01:16.596531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184704+0000","last_change":"2023-11-16T16:37:13.498648+0000","last_active":"2023-11-16T16:38:03.184704+0000","last_peered":"2023-11-16T16:38:03.184704+0000","last_clean":"2023-11-16T16:38:03.184704+0000","last_became_active":"2023-11-16T16:37:13.497390+0000","last_became_peered":"2023-11-16T16:37:13.497390+0000","last_unstale":"2023-11-16T16:38:03.184704+0000","last_undegraded":"2023-11-16T16:38:03.184704+0000","last_fullsized":"2023-11-16T16:38:03.184704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:20:50.541938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185282+0000","last_change":"2023-11-16T16:37:13.942337+0000","last_active":"2023-11-16T16:38:03.185282+0000","last_peered":"2023-11-16T16:38:03.185282+0000","last_clean":"2023-11-16T16:38:03.185282+0000","last_became_active":"2023-11-16T16:37:13.941982+0000","last_became_peered":"2023-11-16T16:37:13.941982+0000","last_unstale":"2023-11-16T16:38:03.185282+0000","last_undegraded":"2023-11-16T16:38:03.185282+0000","last_fullsized":"2023-11-16T16:38:03.185282+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:09:19.949618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'3","reported_seq":73,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:09.378858+0000","last_change":"2023-11-16T16:37:13.503690+0000","last_active":"2023-11-16T16:38:09.378858+0000","last_peered":"2023-11-16T16:38:09.378858+0000","last_clean":"2023-11-16T16:38:09.378858+0000","last_became_active":"2023-11-16T16:37:13.503203+0000","last_became_peered":"2023-11-16T16:37:13.503203+0000","last_unstale":"2023-11-16T16:38:09.378858+0000","last_undegraded":"2023-11-16T16:38:09.378858+0000","last_fullsized":"2023-11-16T16:38:09.378858+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:47:38.415047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.189257+0000","last_change":"2023-11-16T16:37:13.943975+0000","last_active":"2023-11-16T16:38:03.189257+0000","last_peered":"2023-11-16T16:38:03.189257+0000","last_clean":"2023-11-16T16:38:03.189257+0000","last_became_active":"2023-11-16T16:37:13.943611+0000","last_became_peered":"2023-11-16T16:37:13.943611+0000","last_unstale":"2023-11-16T16:38:03.189257+0000","last_undegraded":"2023-11-16T16:38:03.189257+0000","last_fullsized":"2023-11-16T16:38:03.189257+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:37:02.660396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'2","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.344191+0000","last_change":"2023-11-16T16:38:03.344191+0000","last_active":"2023-11-16T16:38:03.344191+0000","last_peered":"2023-11-16T16:38:03.344191+0000","last_clean":"2023-11-16T16:38:03.344191+0000","last_became_active":"2023-11-16T16:38:03.189976+0000","last_became_peered":"2023-11-16T16:38:03.189976+0000","last_unstale":"2023-11-16T16:38:03.344191+0000","last_undegraded":"2023-11-16T16:38:03.344191+0000","last_fullsized":"2023-11-16T16:38:03.344191+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:04:56.889493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.186497+0000","last_change":"2023-11-16T16:37:13.949031+0000","last_active":"2023-11-16T16:38:03.186497+0000","last_peered":"2023-11-16T16:38:03.186497+0000","last_clean":"2023-11-16T16:38:03.186497+0000","last_became_active":"2023-11-16T16:37:13.946923+0000","last_became_peered":"2023-11-16T16:37:13.946923+0000","last_unstale":"2023-11-16T16:38:03.186497+0000","last_undegraded":"2023-11-16T16:38:03.186497+0000","last_fullsized":"2023-11-16T16:38:03.186497+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:04:43.521722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185398+0000","last_change":"2023-11-16T16:37:13.495003+0000","last_active":"2023-11-16T16:38:03.185398+0000","last_peered":"2023-11-16T16:38:03.185398+0000","last_clean":"2023-11-16T16:38:03.185398+0000","last_became_active":"2023-11-16T16:37:13.494563+0000","last_became_peered":"2023-11-16T16:37:13.494563+0000","last_unstale":"2023-11-16T16:38:03.185398+0000","last_undegraded":"2023-11-16T16:38:03.185398+0000","last_fullsized":"2023-11-16T16:38:03.185398+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:36:36.403721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.189090+0000","last_change":"2023-11-16T16:37:13.943736+0000","last_active":"2023-11-16T16:38:03.189090+0000","last_peered":"2023-11-16T16:38:03.189090+0000","last_clean":"2023-11-16T16:38:03.189090+0000","last_became_active":"2023-11-16T16:37:13.943391+0000","last_became_peered":"2023-11-16T16:37:13.943391+0000","last_unstale":"2023-11-16T16:38:03.189090+0000","last_undegraded":"2023-11-16T16:38:03.189090+0000","last_fullsized":"2023-11-16T16:38:03.189090+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:27:01.809877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'28","reported_seq":158,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:10.391449+0000","last_change":"2023-11-16T16:37:13.506508+0000","last_active":"2023-11-16T16:38:10.391449+0000","last_peered":"2023-11-16T16:38:10.391449+0000","last_clean":"2023-11-16T16:38:10.391449+0000","last_became_active":"2023-11-16T16:37:13.506045+0000","last_became_peered":"2023-11-16T16:37:13.506045+0000","last_unstale":"2023-11-16T16:38:10.391449+0000","last_undegraded":"2023-11-16T16:38:10.391449+0000","last_fullsized":"2023-11-16T16:38:10.391449+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:57:44.753422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58527,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":29,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.191240+0000","last_change":"2023-11-16T16:37:13.939717+0000","last_active":"2023-11-16T16:38:03.191240+0000","last_peered":"2023-11-16T16:38:03.191240+0000","last_clean":"2023-11-16T16:38:03.191240+0000","last_became_active":"2023-11-16T16:37:13.939369+0000","last_became_peered":"2023-11-16T16:37:13.939369+0000","last_unstale":"2023-11-16T16:38:03.191240+0000","last_undegraded":"2023-11-16T16:38:03.191240+0000","last_fullsized":"2023-11-16T16:38:03.191240+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:10:31.454459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.188492+0000","last_change":"2023-11-16T16:37:13.506056+0000","last_active":"2023-11-16T16:38:03.188492+0000","last_peered":"2023-11-16T16:38:03.188492+0000","last_clean":"2023-11-16T16:38:03.188492+0000","last_became_active":"2023-11-16T16:37:13.505707+0000","last_became_peered":"2023-11-16T16:37:13.505707+0000","last_unstale":"2023-11-16T16:38:03.188492+0000","last_undegraded":"2023-11-16T16:38:03.188492+0000","last_fullsized":"2023-11-16T16:38:03.188492+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:29:47.242470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184551+0000","last_change":"2023-11-16T16:37:13.947764+0000","last_active":"2023-11-16T16:38:03.184551+0000","last_peered":"2023-11-16T16:38:03.184551+0000","last_clean":"2023-11-16T16:38:03.184551+0000","last_became_active":"2023-11-16T16:37:13.947510+0000","last_became_peered":"2023-11-16T16:37:13.947510+0000","last_unstale":"2023-11-16T16:38:03.184551+0000","last_undegraded":"2023-11-16T16:38:03.184551+0000","last_fullsized":"2023-11-16T16:38:03.184551+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:41:59.672529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184939+0000","last_change":"2023-11-16T16:37:13.498178+0000","last_active":"2023-11-16T16:38:03.184939+0000","last_peered":"2023-11-16T16:38:03.184939+0000","last_clean":"2023-11-16T16:38:03.184939+0000","last_became_active":"2023-11-16T16:37:13.497312+0000","last_became_peered":"2023-11-16T16:37:13.497312+0000","last_unstale":"2023-11-16T16:38:03.184939+0000","last_undegraded":"2023-11-16T16:38:03.184939+0000","last_fullsized":"2023-11-16T16:38:03.184939+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:54:40.375590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185267+0000","last_change":"2023-11-16T16:37:13.947854+0000","last_active":"2023-11-16T16:38:03.185267+0000","last_peered":"2023-11-16T16:38:03.185267+0000","last_clean":"2023-11-16T16:38:03.185267+0000","last_became_active":"2023-11-16T16:37:13.947530+0000","last_became_peered":"2023-11-16T16:37:13.947530+0000","last_unstale":"2023-11-16T16:38:03.185267+0000","last_undegraded":"2023-11-16T16:38:03.185267+0000","last_fullsized":"2023-11-16T16:38:03.185267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:41:00.205641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'3","reported_seq":73,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:07.376386+0000","last_change":"2023-11-16T16:37:13.500127+0000","last_active":"2023-11-16T16:38:07.376386+0000","last_peered":"2023-11-16T16:38:07.376386+0000","last_clean":"2023-11-16T16:38:07.376386+0000","last_became_active":"2023-11-16T16:37:13.499758+0000","last_became_peered":"2023-11-16T16:37:13.499758+0000","last_unstale":"2023-11-16T16:38:07.376386+0000","last_undegraded":"2023-11-16T16:38:07.376386+0000","last_fullsized":"2023-11-16T16:38:07.376386+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:39:00.268144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.183305+0000","last_change":"2023-11-16T16:37:13.940962+0000","last_active":"2023-11-16T16:38:03.183305+0000","last_peered":"2023-11-16T16:38:03.183305+0000","last_clean":"2023-11-16T16:38:03.183305+0000","last_became_active":"2023-11-16T16:37:13.940450+0000","last_became_peered":"2023-11-16T16:37:13.940450+0000","last_unstale":"2023-11-16T16:38:03.183305+0000","last_undegraded":"2023-11-16T16:38:03.183305+0000","last_fullsized":"2023-11-16T16:38:03.183305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:28:45.946454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'15","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.228174+0000","last_change":"2023-11-16T16:38:03.228174+0000","last_active":"2023-11-16T16:38:03.228174+0000","last_peered":"2023-11-16T16:38:03.228174+0000","last_clean":"2023-11-16T16:38:03.228174+0000","last_became_active":"2023-11-16T16:38:03.192020+0000","last_became_peered":"2023-11-16T16:38:03.192020+0000","last_unstale":"2023-11-16T16:38:03.228174+0000","last_undegraded":"2023-11-16T16:38:03.228174+0000","last_fullsized":"2023-11-16T16:38:03.228174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:54:01.231005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184704+0000","last_change":"2023-11-16T16:37:13.944878+0000","last_active":"2023-11-16T16:38:03.184704+0000","last_peered":"2023-11-16T16:38:03.184704+0000","last_clean":"2023-11-16T16:38:03.184704+0000","last_became_active":"2023-11-16T16:37:13.944580+0000","last_became_peered":"2023-11-16T16:37:13.944580+0000","last_unstale":"2023-11-16T16:38:03.184704+0000","last_undegraded":"2023-11-16T16:38:03.184704+0000","last_fullsized":"2023-11-16T16:38:03.184704+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:00:01.106243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.190628+0000","last_change":"2023-11-16T16:37:13.499139+0000","last_active":"2023-11-16T16:38:03.190628+0000","last_peered":"2023-11-16T16:38:03.190628+0000","last_clean":"2023-11-16T16:38:03.190628+0000","last_became_active":"2023-11-16T16:37:13.498755+0000","last_became_peered":"2023-11-16T16:37:13.498755+0000","last_unstale":"2023-11-16T16:38:03.190628+0000","last_undegraded":"2023-11-16T16:38:03.190628+0000","last_fullsized":"2023-11-16T16:38:03.190628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T16:55:06.065509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.186856+0000","last_change":"2023-11-16T16:37:13.946187+0000","last_active":"2023-11-16T16:38:03.186856+0000","last_peered":"2023-11-16T16:38:03.186856+0000","last_clean":"2023-11-16T16:38:03.186856+0000","last_became_active":"2023-11-16T16:37:13.945887+0000","last_became_peered":"2023-11-16T16:37:13.945887+0000","last_unstale":"2023-11-16T16:38:03.186856+0000","last_undegraded":"2023-11-16T16:38:03.186856+0000","last_fullsized":"2023-11-16T16:38:03.186856+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:41:31.726749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185150+0000","last_change":"2023-11-16T16:37:13.507150+0000","last_active":"2023-11-16T16:38:03.185150+0000","last_peered":"2023-11-16T16:38:03.185150+0000","last_clean":"2023-11-16T16:38:03.185150+0000","last_became_active":"2023-11-16T16:37:13.506814+0000","last_became_peered":"2023-11-16T16:37:13.506814+0000","last_unstale":"2023-11-16T16:38:03.185150+0000","last_undegraded":"2023-11-16T16:38:03.185150+0000","last_fullsized":"2023-11-16T16:38:03.185150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:54:45.031046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.187711+0000","last_change":"2023-11-16T16:37:13.947866+0000","last_active":"2023-11-16T16:38:03.187711+0000","last_peered":"2023-11-16T16:38:03.187711+0000","last_clean":"2023-11-16T16:38:03.187711+0000","last_became_active":"2023-11-16T16:37:13.947544+0000","last_became_peered":"2023-11-16T16:37:13.947544+0000","last_unstale":"2023-11-16T16:38:03.187711+0000","last_undegraded":"2023-11-16T16:38:03.187711+0000","last_fullsized":"2023-11-16T16:38:03.187711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:23:27.220130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"84'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184212+0000","last_change":"2023-11-16T16:37:13.497800+0000","last_active":"2023-11-16T16:38:03.184212+0000","last_peered":"2023-11-16T16:38:03.184212+0000","last_clean":"2023-11-16T16:38:03.184212+0000","last_became_active":"2023-11-16T16:37:13.497414+0000","last_became_peered":"2023-11-16T16:37:13.497414+0000","last_unstale":"2023-11-16T16:38:03.184212+0000","last_undegraded":"2023-11-16T16:38:03.184212+0000","last_fullsized":"2023-11-16T16:38:03.184212+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:17:42.179250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.187179+0000","last_change":"2023-11-16T16:37:13.946780+0000","last_active":"2023-11-16T16:38:03.187179+0000","last_peered":"2023-11-16T16:38:03.187179+0000","last_clean":"2023-11-16T16:38:03.187179+0000","last_became_active":"2023-11-16T16:37:13.946441+0000","last_became_peered":"2023-11-16T16:37:13.946441+0000","last_unstale":"2023-11-16T16:38:03.187179+0000","last_undegraded":"2023-11-16T16:38:03.187179+0000","last_fullsized":"2023-11-16T16:38:03.187179+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:46:10.856707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.192571+0000","last_change":"2023-11-16T16:38:03.191887+0000","last_active":"2023-11-16T16:38:03.192571+0000","last_peered":"2023-11-16T16:38:03.192571+0000","last_clean":"2023-11-16T16:38:03.192571+0000","last_became_active":"2023-11-16T16:38:03.191402+0000","last_became_peered":"2023-11-16T16:38:03.191402+0000","last_unstale":"2023-11-16T16:38:03.192571+0000","last_undegraded":"2023-11-16T16:38:03.192571+0000","last_fullsized":"2023-11-16T16:38:03.192571+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:47:26.769740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"84'4","reported_seq":72,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:07.375694+0000","last_change":"2023-11-16T16:37:13.498129+0000","last_active":"2023-11-16T16:38:07.375694+0000","last_peered":"2023-11-16T16:38:07.375694+0000","last_clean":"2023-11-16T16:38:07.375694+0000","last_became_active":"2023-11-16T16:37:13.497765+0000","last_became_peered":"2023-11-16T16:37:13.497765+0000","last_unstale":"2023-11-16T16:38:07.375694+0000","last_undegraded":"2023-11-16T16:38:07.375694+0000","last_fullsized":"2023-11-16T16:38:07.375694+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:15:51.118817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.182609+0000","last_change":"2023-11-16T16:37:13.938417+0000","last_active":"2023-11-16T16:38:03.182609+0000","last_peered":"2023-11-16T16:38:03.182609+0000","last_clean":"2023-11-16T16:38:03.182609+0000","last_became_active":"2023-11-16T16:37:13.938114+0000","last_became_peered":"2023-11-16T16:37:13.938114+0000","last_unstale":"2023-11-16T16:38:03.182609+0000","last_undegraded":"2023-11-16T16:38:03.182609+0000","last_fullsized":"2023-11-16T16:38:03.182609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:07:00.909431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"84'3","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.183990+0000","last_change":"2023-11-16T16:37:13.499045+0000","last_active":"2023-11-16T16:38:03.183990+0000","last_peered":"2023-11-16T16:38:03.183990+0000","last_clean":"2023-11-16T16:38:03.183990+0000","last_became_active":"2023-11-16T16:37:13.498598+0000","last_became_peered":"2023-11-16T16:37:13.498598+0000","last_unstale":"2023-11-16T16:38:03.183990+0000","last_undegraded":"2023-11-16T16:38:03.183990+0000","last_fullsized":"2023-11-16T16:38:03.183990+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:29:46.898930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":349,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.183816+0000","last_change":"2023-11-16T16:36:14.090213+0000","last_active":"2023-11-16T16:38:03.183816+0000","last_peered":"2023-11-16T16:38:03.183816+0000","last_clean":"2023-11-16T16:38:03.183816+0000","last_became_active":"2023-11-16T16:36:14.089742+0000","last_became_peered":"2023-11-16T16:36:14.089742+0000","last_unstale":"2023-11-16T16:38:03.183816+0000","last_undegraded":"2023-11-16T16:38:03.183816+0000","last_fullsized":"2023-11-16T16:38:03.183816+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:33:16.178716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:33:16.178716+0000","last_clean_scrub_stamp":"2023-11-16T16:33:16.178716+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:34:52.910521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184225+0000","last_change":"2023-11-16T16:37:13.945380+0000","last_active":"2023-11-16T16:38:03.184225+0000","last_peered":"2023-11-16T16:38:03.184225+0000","last_clean":"2023-11-16T16:38:03.184225+0000","last_became_active":"2023-11-16T16:37:13.945038+0000","last_became_peered":"2023-11-16T16:37:13.945038+0000","last_unstale":"2023-11-16T16:38:03.184225+0000","last_undegraded":"2023-11-16T16:38:03.184225+0000","last_fullsized":"2023-11-16T16:38:03.184225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:29:11.812243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"84'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184055+0000","last_change":"2023-11-16T16:37:13.497586+0000","last_active":"2023-11-16T16:38:03.184055+0000","last_peered":"2023-11-16T16:38:03.184055+0000","last_clean":"2023-11-16T16:38:03.184055+0000","last_became_active":"2023-11-16T16:37:13.497233+0000","last_became_peered":"2023-11-16T16:37:13.497233+0000","last_unstale":"2023-11-16T16:38:03.184055+0000","last_undegraded":"2023-11-16T16:38:03.184055+0000","last_fullsized":"2023-11-16T16:38:03.184055+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:47:21.097628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"84'1","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.191390+0000","last_change":"2023-11-16T16:37:13.500621+0000","last_active":"2023-11-16T16:38:03.191390+0000","last_peered":"2023-11-16T16:38:03.191390+0000","last_clean":"2023-11-16T16:38:03.191390+0000","last_became_active":"2023-11-16T16:37:13.500265+0000","last_became_peered":"2023-11-16T16:37:13.500265+0000","last_unstale":"2023-11-16T16:38:03.191390+0000","last_undegraded":"2023-11-16T16:38:03.191390+0000","last_fullsized":"2023-11-16T16:38:03.191390+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:22:59.748224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184391+0000","last_change":"2023-11-16T16:37:13.930992+0000","last_active":"2023-11-16T16:38:03.184391+0000","last_peered":"2023-11-16T16:38:03.184391+0000","last_clean":"2023-11-16T16:38:03.184391+0000","last_became_active":"2023-11-16T16:37:13.930641+0000","last_became_peered":"2023-11-16T16:37:13.930641+0000","last_unstale":"2023-11-16T16:38:03.184391+0000","last_undegraded":"2023-11-16T16:38:03.184391+0000","last_fullsized":"2023-11-16T16:38:03.184391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:24:49.314642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'5","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185381+0000","last_change":"2023-11-16T16:37:13.498637+0000","last_active":"2023-11-16T16:38:03.185381+0000","last_peered":"2023-11-16T16:38:03.185381+0000","last_clean":"2023-11-16T16:38:03.185381+0000","last_became_active":"2023-11-16T16:37:13.498341+0000","last_became_peered":"2023-11-16T16:37:13.498341+0000","last_unstale":"2023-11-16T16:38:03.185381+0000","last_undegraded":"2023-11-16T16:38:03.185381+0000","last_fullsized":"2023-11-16T16:38:03.185381+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:30:04.558815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.182831+0000","last_change":"2023-11-16T16:37:13.944142+0000","last_active":"2023-11-16T16:38:03.182831+0000","last_peered":"2023-11-16T16:38:03.182831+0000","last_clean":"2023-11-16T16:38:03.182831+0000","last_became_active":"2023-11-16T16:37:13.943758+0000","last_became_peered":"2023-11-16T16:37:13.943758+0000","last_unstale":"2023-11-16T16:38:03.182831+0000","last_undegraded":"2023-11-16T16:38:03.182831+0000","last_fullsized":"2023-11-16T16:38:03.182831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:57:58.601449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184558+0000","last_change":"2023-11-16T16:37:13.496900+0000","last_active":"2023-11-16T16:38:03.184558+0000","last_peered":"2023-11-16T16:38:03.184558+0000","last_clean":"2023-11-16T16:38:03.184558+0000","last_became_active":"2023-11-16T16:37:13.496528+0000","last_became_peered":"2023-11-16T16:37:13.496528+0000","last_unstale":"2023-11-16T16:38:03.184558+0000","last_undegraded":"2023-11-16T16:38:03.184558+0000","last_fullsized":"2023-11-16T16:38:03.184558+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:56:42.304030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.186630+0000","last_change":"2023-11-16T16:37:13.943104+0000","last_active":"2023-11-16T16:38:03.186630+0000","last_peered":"2023-11-16T16:38:03.186630+0000","last_clean":"2023-11-16T16:38:03.186630+0000","last_became_active":"2023-11-16T16:37:13.942774+0000","last_became_peered":"2023-11-16T16:37:13.942774+0000","last_unstale":"2023-11-16T16:38:03.186630+0000","last_undegraded":"2023-11-16T16:38:03.186630+0000","last_fullsized":"2023-11-16T16:38:03.186630+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:24:11.162929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.188270+0000","last_change":"2023-11-16T16:37:13.505994+0000","last_active":"2023-11-16T16:38:03.188270+0000","last_peered":"2023-11-16T16:38:03.188270+0000","last_clean":"2023-11-16T16:38:03.188270+0000","last_became_active":"2023-11-16T16:37:13.505663+0000","last_became_peered":"2023-11-16T16:37:13.505663+0000","last_unstale":"2023-11-16T16:38:03.188270+0000","last_undegraded":"2023-11-16T16:38:03.188270+0000","last_fullsized":"2023-11-16T16:38:03.188270+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:06:13.680491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.189650+0000","last_change":"2023-11-16T16:37:13.941970+0000","last_active":"2023-11-16T16:38:03.189650+0000","last_peered":"2023-11-16T16:38:03.189650+0000","last_clean":"2023-11-16T16:38:03.189650+0000","last_became_active":"2023-11-16T16:37:13.941649+0000","last_became_peered":"2023-11-16T16:37:13.941649+0000","last_unstale":"2023-11-16T16:38:03.189650+0000","last_undegraded":"2023-11-16T16:38:03.189650+0000","last_fullsized":"2023-11-16T16:38:03.189650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:23:17.814240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.182332+0000","last_change":"2023-11-16T16:37:13.503429+0000","last_active":"2023-11-16T16:38:03.182332+0000","last_peered":"2023-11-16T16:38:03.182332+0000","last_clean":"2023-11-16T16:38:03.182332+0000","last_became_active":"2023-11-16T16:37:13.503114+0000","last_became_peered":"2023-11-16T16:37:13.503114+0000","last_unstale":"2023-11-16T16:38:03.182332+0000","last_undegraded":"2023-11-16T16:38:03.182332+0000","last_fullsized":"2023-11-16T16:38:03.182332+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:57:19.910282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.186389+0000","last_change":"2023-11-16T16:37:13.941225+0000","last_active":"2023-11-16T16:38:03.186389+0000","last_peered":"2023-11-16T16:38:03.186389+0000","last_clean":"2023-11-16T16:38:03.186389+0000","last_became_active":"2023-11-16T16:37:13.940964+0000","last_became_peered":"2023-11-16T16:37:13.940964+0000","last_unstale":"2023-11-16T16:38:03.186389+0000","last_undegraded":"2023-11-16T16:38:03.186389+0000","last_fullsized":"2023-11-16T16:38:03.186389+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:21:58.992812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.188151+0000","last_change":"2023-11-16T16:37:13.510154+0000","last_active":"2023-11-16T16:38:03.188151+0000","last_peered":"2023-11-16T16:38:03.188151+0000","last_clean":"2023-11-16T16:38:03.188151+0000","last_became_active":"2023-11-16T16:37:13.509790+0000","last_became_peered":"2023-11-16T16:37:13.509790+0000","last_unstale":"2023-11-16T16:38:03.188151+0000","last_undegraded":"2023-11-16T16:38:03.188151+0000","last_fullsized":"2023-11-16T16:38:03.188151+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:09:32.999016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.182648+0000","last_change":"2023-11-16T16:37:13.942203+0000","last_active":"2023-11-16T16:38:03.182648+0000","last_peered":"2023-11-16T16:38:03.182648+0000","last_clean":"2023-11-16T16:38:03.182648+0000","last_became_active":"2023-11-16T16:37:13.941766+0000","last_became_peered":"2023-11-16T16:37:13.941766+0000","last_unstale":"2023-11-16T16:38:03.182648+0000","last_undegraded":"2023-11-16T16:38:03.182648+0000","last_fullsized":"2023-11-16T16:38:03.182648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:50:04.530728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.190859+0000","last_change":"2023-11-16T16:37:13.509854+0000","last_active":"2023-11-16T16:38:03.190859+0000","last_peered":"2023-11-16T16:38:03.190859+0000","last_clean":"2023-11-16T16:38:03.190859+0000","last_became_active":"2023-11-16T16:37:13.509466+0000","last_became_peered":"2023-11-16T16:37:13.509466+0000","last_unstale":"2023-11-16T16:38:03.190859+0000","last_undegraded":"2023-11-16T16:38:03.190859+0000","last_fullsized":"2023-11-16T16:38:03.190859+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:38:13.537860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184479+0000","last_change":"2023-11-16T16:37:13.947906+0000","last_active":"2023-11-16T16:38:03.184479+0000","last_peered":"2023-11-16T16:38:03.184479+0000","last_clean":"2023-11-16T16:38:03.184479+0000","last_became_active":"2023-11-16T16:37:13.947602+0000","last_became_peered":"2023-11-16T16:37:13.947602+0000","last_unstale":"2023-11-16T16:38:03.184479+0000","last_undegraded":"2023-11-16T16:38:03.184479+0000","last_fullsized":"2023-11-16T16:38:03.184479+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:16:45.611613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.188948+0000","last_change":"2023-11-16T16:37:13.506108+0000","last_active":"2023-11-16T16:38:03.188948+0000","last_peered":"2023-11-16T16:38:03.188948+0000","last_clean":"2023-11-16T16:38:03.188948+0000","last_became_active":"2023-11-16T16:37:13.505785+0000","last_became_peered":"2023-11-16T16:37:13.505785+0000","last_unstale":"2023-11-16T16:38:03.188948+0000","last_undegraded":"2023-11-16T16:38:03.188948+0000","last_fullsized":"2023-11-16T16:38:03.188948+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:11:02.793557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185008+0000","last_change":"2023-11-16T16:37:13.935863+0000","last_active":"2023-11-16T16:38:03.185008+0000","last_peered":"2023-11-16T16:38:03.185008+0000","last_clean":"2023-11-16T16:38:03.185008+0000","last_became_active":"2023-11-16T16:37:13.935399+0000","last_became_peered":"2023-11-16T16:37:13.935399+0000","last_unstale":"2023-11-16T16:38:03.185008+0000","last_undegraded":"2023-11-16T16:38:03.185008+0000","last_fullsized":"2023-11-16T16:38:03.185008+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:45:56.613743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184907+0000","last_change":"2023-11-16T16:37:13.498367+0000","last_active":"2023-11-16T16:38:03.184907+0000","last_peered":"2023-11-16T16:38:03.184907+0000","last_clean":"2023-11-16T16:38:03.184907+0000","last_became_active":"2023-11-16T16:37:13.497039+0000","last_became_peered":"2023-11-16T16:37:13.497039+0000","last_unstale":"2023-11-16T16:38:03.184907+0000","last_undegraded":"2023-11-16T16:38:03.184907+0000","last_fullsized":"2023-11-16T16:38:03.184907+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:57:46.571219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184415+0000","last_change":"2023-11-16T16:37:13.940617+0000","last_active":"2023-11-16T16:38:03.184415+0000","last_peered":"2023-11-16T16:38:03.184415+0000","last_clean":"2023-11-16T16:38:03.184415+0000","last_became_active":"2023-11-16T16:37:13.940264+0000","last_became_peered":"2023-11-16T16:37:13.940264+0000","last_unstale":"2023-11-16T16:38:03.184415+0000","last_undegraded":"2023-11-16T16:38:03.184415+0000","last_fullsized":"2023-11-16T16:38:03.184415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:59:36.630848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.193922+0000","last_change":"2023-11-16T16:38:03.193922+0000","last_active":"2023-11-16T16:38:03.193922+0000","last_peered":"2023-11-16T16:38:03.193922+0000","last_clean":"2023-11-16T16:38:03.193922+0000","last_became_active":"2023-11-16T16:38:03.190364+0000","last_became_peered":"2023-11-16T16:38:03.190364+0000","last_unstale":"2023-11-16T16:38:03.193922+0000","last_undegraded":"2023-11-16T16:38:03.193922+0000","last_fullsized":"2023-11-16T16:38:03.193922+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:59:04.681132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185280+0000","last_change":"2023-11-16T16:37:13.943096+0000","last_active":"2023-11-16T16:38:03.185280+0000","last_peered":"2023-11-16T16:38:03.185280+0000","last_clean":"2023-11-16T16:38:03.185280+0000","last_became_active":"2023-11-16T16:37:13.942745+0000","last_became_peered":"2023-11-16T16:37:13.942745+0000","last_unstale":"2023-11-16T16:38:03.185280+0000","last_undegraded":"2023-11-16T16:38:03.185280+0000","last_fullsized":"2023-11-16T16:38:03.185280+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:02:12.721195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.186978+0000","last_change":"2023-11-16T16:37:13.938087+0000","last_active":"2023-11-16T16:38:03.186978+0000","last_peered":"2023-11-16T16:38:03.186978+0000","last_clean":"2023-11-16T16:38:03.186978+0000","last_became_active":"2023-11-16T16:37:13.937780+0000","last_became_peered":"2023-11-16T16:37:13.937780+0000","last_unstale":"2023-11-16T16:38:03.186978+0000","last_undegraded":"2023-11-16T16:38:03.186978+0000","last_fullsized":"2023-11-16T16:38:03.186978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:05:14.399850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.188244+0000","last_change":"2023-11-16T16:37:13.939000+0000","last_active":"2023-11-16T16:38:03.188244+0000","last_peered":"2023-11-16T16:38:03.188244+0000","last_clean":"2023-11-16T16:38:03.188244+0000","last_became_active":"2023-11-16T16:37:13.938585+0000","last_became_peered":"2023-11-16T16:37:13.938585+0000","last_unstale":"2023-11-16T16:38:03.188244+0000","last_undegraded":"2023-11-16T16:38:03.188244+0000","last_fullsized":"2023-11-16T16:38:03.188244+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:13:17.753666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.187970+0000","last_change":"2023-11-16T16:37:13.937696+0000","last_active":"2023-11-16T16:38:03.187970+0000","last_peered":"2023-11-16T16:38:03.187970+0000","last_clean":"2023-11-16T16:38:03.187970+0000","last_became_active":"2023-11-16T16:37:13.936920+0000","last_became_peered":"2023-11-16T16:37:13.936920+0000","last_unstale":"2023-11-16T16:38:03.187970+0000","last_undegraded":"2023-11-16T16:38:03.187970+0000","last_fullsized":"2023-11-16T16:38:03.187970+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:55:00.163322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.189679+0000","last_change":"2023-11-16T16:38:03.189679+0000","last_active":"2023-11-16T16:38:03.189679+0000","last_peered":"2023-11-16T16:38:03.189679+0000","last_clean":"2023-11-16T16:38:03.189679+0000","last_became_active":"2023-11-16T16:38:03.189292+0000","last_became_peered":"2023-11-16T16:38:03.189292+0000","last_unstale":"2023-11-16T16:38:03.189679+0000","last_undegraded":"2023-11-16T16:38:03.189679+0000","last_fullsized":"2023-11-16T16:38:03.189679+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:24:48.303402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184448+0000","last_change":"2023-11-16T16:37:13.949940+0000","last_active":"2023-11-16T16:38:03.184448+0000","last_peered":"2023-11-16T16:38:03.184448+0000","last_clean":"2023-11-16T16:38:03.184448+0000","last_became_active":"2023-11-16T16:37:13.949464+0000","last_became_peered":"2023-11-16T16:37:13.949464+0000","last_unstale":"2023-11-16T16:38:03.184448+0000","last_undegraded":"2023-11-16T16:38:03.184448+0000","last_fullsized":"2023-11-16T16:38:03.184448+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:05:15.768098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.187084+0000","last_change":"2023-11-16T16:37:13.940534+0000","last_active":"2023-11-16T16:38:03.187084+0000","last_peered":"2023-11-16T16:38:03.187084+0000","last_clean":"2023-11-16T16:38:03.187084+0000","last_became_active":"2023-11-16T16:37:13.940174+0000","last_became_peered":"2023-11-16T16:37:13.940174+0000","last_unstale":"2023-11-16T16:38:03.187084+0000","last_undegraded":"2023-11-16T16:38:03.187084+0000","last_fullsized":"2023-11-16T16:38:03.187084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:29:35.708635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.190148+0000","last_change":"2023-11-16T16:37:13.937519+0000","last_active":"2023-11-16T16:38:03.190148+0000","last_peered":"2023-11-16T16:38:03.190148+0000","last_clean":"2023-11-16T16:38:03.190148+0000","last_became_active":"2023-11-16T16:37:13.937175+0000","last_became_peered":"2023-11-16T16:37:13.937175+0000","last_unstale":"2023-11-16T16:38:03.190148+0000","last_undegraded":"2023-11-16T16:38:03.190148+0000","last_fullsized":"2023-11-16T16:38:03.190148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:25:23.849158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.182804+0000","last_change":"2023-11-16T16:37:13.951506+0000","last_active":"2023-11-16T16:38:03.182804+0000","last_peered":"2023-11-16T16:38:03.182804+0000","last_clean":"2023-11-16T16:38:03.182804+0000","last_became_active":"2023-11-16T16:37:13.951192+0000","last_became_peered":"2023-11-16T16:37:13.951192+0000","last_unstale":"2023-11-16T16:38:03.182804+0000","last_undegraded":"2023-11-16T16:38:03.182804+0000","last_fullsized":"2023-11-16T16:38:03.182804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:17:38.223843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184841+0000","last_change":"2023-11-16T16:37:13.935614+0000","last_active":"2023-11-16T16:38:03.184841+0000","last_peered":"2023-11-16T16:38:03.184841+0000","last_clean":"2023-11-16T16:38:03.184841+0000","last_became_active":"2023-11-16T16:37:13.935237+0000","last_became_peered":"2023-11-16T16:37:13.935237+0000","last_unstale":"2023-11-16T16:38:03.184841+0000","last_undegraded":"2023-11-16T16:38:03.184841+0000","last_fullsized":"2023-11-16T16:38:03.184841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:30:01.772328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.182369+0000","last_change":"2023-11-16T16:37:13.939387+0000","last_active":"2023-11-16T16:38:03.182369+0000","last_peered":"2023-11-16T16:38:03.182369+0000","last_clean":"2023-11-16T16:38:03.182369+0000","last_became_active":"2023-11-16T16:37:13.939082+0000","last_became_peered":"2023-11-16T16:37:13.939082+0000","last_unstale":"2023-11-16T16:38:03.182369+0000","last_undegraded":"2023-11-16T16:38:03.182369+0000","last_fullsized":"2023-11-16T16:38:03.182369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:28:19.026881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.188347+0000","last_change":"2023-11-16T16:37:13.936381+0000","last_active":"2023-11-16T16:38:03.188347+0000","last_peered":"2023-11-16T16:38:03.188347+0000","last_clean":"2023-11-16T16:38:03.188347+0000","last_became_active":"2023-11-16T16:37:13.936077+0000","last_became_peered":"2023-11-16T16:37:13.936077+0000","last_unstale":"2023-11-16T16:38:03.188347+0000","last_undegraded":"2023-11-16T16:38:03.188347+0000","last_fullsized":"2023-11-16T16:38:03.188347+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:20:31.015344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184590+0000","last_change":"2023-11-16T16:37:13.934005+0000","last_active":"2023-11-16T16:38:03.184590+0000","last_peered":"2023-11-16T16:38:03.184590+0000","last_clean":"2023-11-16T16:38:03.184590+0000","last_became_active":"2023-11-16T16:37:13.933633+0000","last_became_peered":"2023-11-16T16:37:13.933633+0000","last_unstale":"2023-11-16T16:38:03.184590+0000","last_undegraded":"2023-11-16T16:38:03.184590+0000","last_fullsized":"2023-11-16T16:38:03.184590+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:41:36.990242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.186787+0000","last_change":"2023-11-16T16:37:13.946495+0000","last_active":"2023-11-16T16:38:03.186787+0000","last_peered":"2023-11-16T16:38:03.186787+0000","last_clean":"2023-11-16T16:38:03.186787+0000","last_became_active":"2023-11-16T16:37:13.946148+0000","last_became_peered":"2023-11-16T16:37:13.946148+0000","last_unstale":"2023-11-16T16:38:03.186787+0000","last_undegraded":"2023-11-16T16:38:03.186787+0000","last_fullsized":"2023-11-16T16:38:03.186787+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:25:08.609723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":3542208,"data_compressed":211753,"data_compressed_allocated":270336,"data_compressed_original":3522560,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62431,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":334,"num_read_kb":232,"num_write":150,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":187293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132,"ondisk_log_size":132,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416407,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6040,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":93739112,"statfs":{"total":95995035648,"available":95988850688,"internally_reserved":0,"allocated":933888,"data_stored":447021,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.58099999999999996}]},{"osd":1,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.72099999999999997}]},{"osd":2,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.624}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":4,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.97199999999999998}]},{"osd":5,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.752}]},{"osd":6,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":0.96199999999999997}]},{"osd":7,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.92400000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":9,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":1.0309999999999999}]},{"osd":10,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":1.018},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.863,"5min":1.863,"15min":1.863},"last":1.0049999999999999}]}]},{"osd":10,"up_from":72,"seq":309237645339,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6432,"kb_used_data":1048,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738720,"statfs":{"total":95995035648,"available":95988449280,"internally_reserved":0,"allocated":1073152,"data_stored":1096544,"data_compressed":41378,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":1,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.56200000000000006}]},{"osd":2,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.58799999999999997}]},{"osd":3,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.66500000000000004}]},{"osd":4,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.501},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.68100000000000005}]},{"osd":5,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.71399999999999997}]},{"osd":6,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.78100000000000003}]},{"osd":7,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.505}]},{"osd":8,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.875}]},{"osd":9,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.85099999999999998}]},{"osd":11,"last update":"Thu Nov 16 16:37:33 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.755},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.86299999999999999}]}]},{"osd":9,"up_from":67,"seq":287762808862,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6176,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":93738976,"statfs":{"total":95995035648,"available":95988711424,"internally_reserved":0,"allocated":942080,"data_stored":447145,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.45100000000000001,"15min":0.46300000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.59699999999999998,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.47199999999999998,"15min":0.48199999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.64500000000000002}]},{"osd":1,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.59199999999999997,"15min":0.60599999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.61799999999999999,"15min":0.63300000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76100000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.53200000000000003}]},{"osd":2,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.63200000000000001,"15min":0.64600000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.78900000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.63500000000000001,"15min":0.64300000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.65900000000000003}]},{"osd":3,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.54800000000000004,"15min":0.56399999999999995},"min":{"1min":0.33200000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73299999999999998,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.55100000000000005,"15min":0.57299999999999995},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.70899999999999996,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.51500000000000001}]},{"osd":4,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.66200000000000003,"15min":0.68100000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.873,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.624},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.67300000000000004,"15min":0.68999999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.83599999999999997,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.57599999999999996}]},{"osd":5,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.69599999999999995,"15min":0.70999999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.878,"5min":1.056,"15min":1.056},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.72099999999999997,"15min":0.73899999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85999999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.67400000000000004}]},{"osd":6,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.64900000000000002,"15min":0.67200000000000004},"min":{"1min":0.38400000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.81699999999999995,"5min":1.069,"15min":1.069},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.70399999999999996,"15min":0.72799999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85099999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.69299999999999995}]},{"osd":7,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.755,"15min":0.77300000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92200000000000004,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.77100000000000002,"15min":0.79500000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.89600000000000002,"5min":1.111,"15min":1.111},"last":0.63800000000000001}]},{"osd":8,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.80400000000000005,"15min":0.82499999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.95899999999999996,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.80100000000000005,"15min":0.81599999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.97399999999999998,"5min":1.089,"15min":1.089},"last":0.78800000000000003}]},{"osd":10,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.76000000000000001}]}]},{"osd":8,"up_from":59,"seq":253403070501,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6360,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738792,"statfs":{"total":95995035648,"available":95988523008,"internally_reserved":0,"allocated":933888,"data_stored":446953,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.77000000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.44400000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.59399999999999997}]},{"osd":1,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.627,"15min":0.61699999999999999},"min":{"1min":0.46100000000000002,"5min":0.318,"15min":0.318},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.46200000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.621}]},{"osd":2,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.50600000000000001,"15min":0.51100000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71299999999999997,"5min":1.056,"15min":1.056},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.51500000000000001,"15min":0.52000000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71499999999999997,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.80200000000000005}]},{"osd":3,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.624,"15min":0.625},"min":{"1min":0.40999999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.75800000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.433,"5min":0.376,"15min":0.376},"max":{"1min":0.877,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.85699999999999998}]},{"osd":4,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.65800000000000003,"15min":0.63300000000000001},"min":{"1min":0.47099999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":4.0599999999999996,"5min":4.0599999999999996,"15min":4.0599999999999996},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69899999999999995,"15min":0.69799999999999995},"min":{"1min":0.42899999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":1.036}]},{"osd":5,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60999999999999999,"15min":0.61199999999999999},"min":{"1min":0.33500000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.89000000000000001,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.624,"15min":0.63},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.84599999999999997,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.76200000000000001}]},{"osd":6,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.747,"15min":0.745},"min":{"1min":0.45800000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94599999999999995,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.745,"15min":0.75},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94299999999999995,"5min":1.173,"15min":1.173},"last":0.96399999999999997}]},{"osd":7,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.76800000000000002,"15min":0.76400000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.0089999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.75700000000000001,"15min":0.753},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.88}]},{"osd":9,"last update":"Thu Nov 16 16:37:55 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.73799999999999999,"15min":0.72999999999999998},"min":{"1min":0.54000000000000004,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.018,"5min":1.036,"15min":1.036},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.875,"5min":0.76900000000000002,"15min":0.751},"min":{"1min":0.58699999999999997,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.94799999999999995}]},{"osd":10,"last update":"Thu Nov 16 16:37:13 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":3.5720000000000001,"5min":3.5720000000000001,"15min":3.5720000000000001},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":3.5920000000000001,"5min":3.5920000000000001,"15min":3.5920000000000001},"last":1.0820000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]}]},{"osd":7,"up_from":52,"seq":223338299432,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6480,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738672,"statfs":{"total":95995035648,"available":95988400128,"internally_reserved":0,"allocated":991232,"data_stored":1038904,"data_compressed":41378,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:36:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55800000000000005}]},{"osd":1,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.41799999999999998,"15min":0.41099999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.313},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.434,"15min":0.42599999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.41499999999999998}]},{"osd":2,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.35799999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.53100000000000003}]},{"osd":3,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57499999999999996,"15min":0.57099999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.625,"15min":0.621},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.36399999999999999}]},{"osd":4,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.52400000000000002,"15min":0.51400000000000001},"min":{"1min":0.34799999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.62},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.498,"15min":0.48199999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.443}]},{"osd":5,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.65200000000000002,"15min":0.64700000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91300000000000003,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.745},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.63800000000000001,"15min":0.63200000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.69899999999999995}]},{"osd":6,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.70299999999999996,"15min":0.69499999999999995},"min":{"1min":0.52500000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.75,"5min":0.68600000000000005,"15min":0.67600000000000005},"min":{"1min":0.47299999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.57199999999999995}]},{"osd":8,"last update":"Thu Nov 16 16:37:25 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.748,"15min":0.747},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0580000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.76100000000000001,"15min":0.76300000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.68899999999999995}]},{"osd":9,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.80300000000000005,"15min":0.78200000000000003},"min":{"1min":0.57999999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":4.7370000000000001,"5min":4.7370000000000001,"15min":4.7370000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.84399999999999997,"15min":0.82199999999999995},"min":{"1min":0.55600000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":4.7809999999999997,"5min":4.7809999999999997,"15min":4.7809999999999997},"last":0.54500000000000004}]},{"osd":10,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.64400000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]}]},{"osd":6,"up_from":47,"seq":201863462956,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6292,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738860,"statfs":{"total":95995035648,"available":95988592640,"internally_reserved":0,"allocated":929792,"data_stored":446540,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.439,"15min":0.41999999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":2.3140000000000001,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.47399999999999998,"15min":0.44600000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":3.472,"5min":3.472,"15min":3.472},"last":0.29699999999999999}]},{"osd":1,"last update":"Thu Nov 16 16:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.59699999999999998,"15min":0.58299999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":3.3290000000000002,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.64600000000000002,"15min":0.59999999999999998},"min":{"1min":0.38200000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":4.4669999999999996,"5min":4.4669999999999996,"15min":4.4669999999999996},"last":0.48999999999999999}]},{"osd":2,"last update":"Thu Nov 16 16:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.65400000000000003,"15min":0.63800000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":3.5369999999999999,"5min":3.5369999999999999,"15min":3.5369999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.66200000000000003,"15min":0.63400000000000001},"min":{"1min":0.46400000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.4049999999999998,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.61799999999999999}]},{"osd":3,"last update":"Thu Nov 16 16:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.52400000000000002,"15min":0.48999999999999999},"min":{"1min":0.30499999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":3.3700000000000001,"5min":3.3700000000000001,"15min":3.3700000000000001},"last":0.314},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.55600000000000005,"15min":0.54100000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.41099999999999998}]},{"osd":4,"last update":"Thu Nov 16 16:37:50 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.67600000000000005,"15min":0.60999999999999999},"min":{"1min":0.45300000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":4.4900000000000002,"5min":4.4900000000000002,"15min":4.4900000000000002},"last":0.56200000000000006},{"interface":"front","average":{"1min":1.026,"5min":0.69899999999999995,"15min":0.63100000000000001},"min":{"1min":0.45200000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":4.4509999999999996,"5min":4.4509999999999996,"15min":4.4509999999999996},"last":0.47299999999999998}]},{"osd":5,"last update":"Thu Nov 16 16:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.68899999999999995,"15min":0.68100000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":2.4209999999999998,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.72999999999999998,"15min":0.70999999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":3.5019999999999998,"5min":3.5019999999999998,"15min":3.5019999999999998},"last":0.629}]},{"osd":7,"last update":"Thu Nov 16 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.70499999999999996,"15min":0.70299999999999996},"min":{"1min":0.48999999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.878,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.747,"5min":0.71499999999999997,"15min":0.70899999999999996},"min":{"1min":0.55300000000000005,"5min":0.436,"15min":0.436},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.59699999999999998}]},{"osd":8,"last update":"Thu Nov 16 16:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.76100000000000001,"15min":0.77200000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.91200000000000003,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.77000000000000002,"15min":0.78000000000000003},"min":{"1min":0.54000000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.016,"5min":1.091,"15min":1.091},"last":0.66400000000000003}]},{"osd":9,"last update":"Thu Nov 16 16:37:50 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":0.81799999999999995,"15min":0.78200000000000003},"min":{"1min":0.50800000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":6.1539999999999999,"5min":6.1539999999999999,"15min":6.1539999999999999},"last":0.503},{"interface":"front","average":{"1min":1.1910000000000001,"5min":0.83999999999999997,"15min":0.78200000000000003},"min":{"1min":0.55100000000000005,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":8.2460000000000004,"5min":8.2460000000000004,"15min":8.2460000000000004},"last":0.57699999999999996}]},{"osd":10,"last update":"Thu Nov 16 16:37:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.63800000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7376,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":93737776,"statfs":{"total":95995035648,"available":95987482624,"internally_reserved":0,"allocated":991232,"data_stored":504854,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:38:06 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.48499999999999999,"15min":0.45800000000000002},"min":{"1min":0.34899999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.67000000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52100000000000002,"15min":0.48799999999999999},"min":{"1min":0.311,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.83499999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.57899999999999996}]},{"osd":2,"last update":"Thu Nov 16 16:37:19 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.54300000000000004,"15min":0.498},"min":{"1min":0.36699999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.84799999999999998,"5min":0.876,"15min":0.876},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.54000000000000004,"15min":0.503},"min":{"1min":0.40500000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.91000000000000003,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.70799999999999996}]},{"osd":3,"last update":"Thu Nov 16 16:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57699999999999996,"15min":0.55500000000000005},"min":{"1min":0.372,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85599999999999998,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59899999999999998,"15min":0.57499999999999996},"min":{"1min":0.41399999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.63400000000000001}]},{"osd":4,"last update":"Thu Nov 16 16:38:06 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.46300000000000002,"15min":0.43099999999999999},"min":{"1min":0.33700000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.79500000000000004,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55000000000000004,"15min":0.48499999999999999},"min":{"1min":0.35499999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.82899999999999996,"5min":1.242,"15min":1.242},"last":0.78700000000000003}]},{"osd":5,"last update":"Thu Nov 16 16:37:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.63800000000000001,"15min":0.61099999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.621,"15min":0.59699999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97399999999999998,"5min":1.004,"15min":1.004},"last":0.82799999999999996}]},{"osd":6,"last update":"Thu Nov 16 16:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.68400000000000005,"15min":0.66800000000000004},"min":{"1min":0.44700000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.95099999999999996,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68400000000000005,"15min":0.66400000000000003},"min":{"1min":0.45500000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.95299999999999996,"5min":1,"15min":1},"last":0.76000000000000001}]},{"osd":7,"last update":"Thu Nov 16 16:38:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65300000000000002,"15min":0.63500000000000001},"min":{"1min":0.39500000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.0569999999999999,"5min":1.077,"15min":1.077},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.67900000000000005},"min":{"1min":0.42599999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.018,"5min":1.143,"15min":1.143},"last":0.84899999999999998}]},{"osd":8,"last update":"Thu Nov 16 16:37:27 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.75900000000000001,"15min":0.755},"min":{"1min":0.54800000000000004,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.74099999999999999,"15min":0.73699999999999999},"min":{"1min":0.47399999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.97099999999999997}]},{"osd":9,"last update":"Thu Nov 16 16:37:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.77900000000000003,"15min":0.79900000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.87,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.84599999999999997,"15min":0.86399999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.97299999999999998,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.95299999999999996}]},{"osd":10,"last update":"Thu Nov 16 16:37:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.873},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.88400000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]}]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7208,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":6272,"kb_avail":93737944,"statfs":{"total":95995035648,"available":95987654656,"internally_reserved":0,"allocated":950272,"data_stored":447805,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":6422528},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Nov 16 16:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.54700000000000004,"15min":0.48199999999999998},"min":{"1min":0.38400000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":3.6869999999999998,"5min":3.6869999999999998,"15min":3.6869999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.57999999999999996,"15min":0.48799999999999999},"min":{"1min":0.40600000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":6.6059999999999999,"5min":6.6059999999999999,"15min":6.6059999999999999},"last":0.57099999999999995}]},{"osd":2,"last update":"Thu Nov 16 16:37:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.55600000000000005,"15min":0.51400000000000001},"min":{"1min":0.41999999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.80600000000000005,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56299999999999994,"15min":0.53100000000000003},"min":{"1min":0.372,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.46200000000000002}]},{"osd":3,"last update":"Thu Nov 16 16:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.46500000000000002,"15min":0.45600000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.45700000000000002,"15min":0.44700000000000001},"min":{"1min":0.33700000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.35199999999999998}]},{"osd":4,"last update":"Thu Nov 16 16:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.61499999999999999,"15min":0.56299999999999994},"min":{"1min":0.44,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":3.8109999999999999,"5min":3.8109999999999999,"15min":3.8109999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.871,"5min":0.64900000000000002,"15min":0.57199999999999995},"min":{"1min":0.42099999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":6.577,"5min":6.577,"15min":6.577},"last":0.49299999999999999}]},{"osd":5,"last update":"Thu Nov 16 16:37:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.56200000000000006,"15min":0.55100000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.66100000000000003,"5min":1.012,"15min":1.012},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64600000000000002,"15min":0.621},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.82699999999999996,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.53200000000000003}]},{"osd":6,"last update":"Thu Nov 16 16:37:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.64800000000000002,"15min":0.59699999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":6.5190000000000001,"5min":6.5190000000000001,"15min":6.5190000000000001},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.63100000000000001,"15min":0.57499999999999996},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":9.141,"5min":9.141,"15min":9.141},"last":0.40600000000000003}]},{"osd":7,"last update":"Thu Nov 16 16:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.68400000000000005,"15min":0.67600000000000005},"min":{"1min":0.495,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.441},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.74199999999999999,"15min":0.73199999999999998},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.59299999999999997}]},{"osd":8,"last update":"Thu Nov 16 16:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.78800000000000003,"15min":0.80900000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.91900000000000004,"5min":1.163,"15min":1.163},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.83499999999999996,"15min":0.85299999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.90800000000000003,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.60699999999999998}]},{"osd":9,"last update":"Thu Nov 16 16:37:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.85499999999999998,"15min":0.86199999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":6.6349999999999998,"5min":6.6349999999999998,"15min":6.6349999999999998},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.79000000000000004,"15min":0.76600000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":9.1690000000000005,"5min":9.1690000000000005,"15min":9.1690000000000005},"last":0.52000000000000002}]},{"osd":10,"last update":"Thu Nov 16 16:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.67200000000000004}]},{"osd":11,"last update":"Thu Nov 16 16:37:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":9.1039999999999992,"5min":9.1039999999999992,"15min":9.1039999999999992},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":3.7530000000000001,"5min":3.7530000000000001,"15min":3.7530000000000001},"last":0.626}]}]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7464,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":93737688,"statfs":{"total":95995035648,"available":95987392512,"internally_reserved":0,"allocated":1015808,"data_stored":506546,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.29999999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.497,"15min":0.48799999999999999},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.35999999999999999}]},{"osd":1,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52600000000000002,"15min":0.51400000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.40500000000000003,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.89900000000000002,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.83299999999999996}]},{"osd":3,"last update":"Thu Nov 16 16:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.66300000000000003,"15min":0.60699999999999998},"min":{"1min":0.40699999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":6.9740000000000002,"5min":6.9740000000000002,"15min":6.9740000000000002},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.57499999999999996,"15min":0.55700000000000005},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.55100000000000005}]},{"osd":4,"last update":"Thu Nov 16 16:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.55300000000000005,"15min":0.53000000000000003},"min":{"1min":0.376,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59099999999999997,"15min":0.58199999999999996},"min":{"1min":0.40000000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83799999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.67900000000000005}]},{"osd":5,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.49399999999999999,"15min":0.48399999999999999},"min":{"1min":0.315,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.47199999999999998,"15min":0.45900000000000002},"min":{"1min":0.32200000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.41999999999999998}]},{"osd":6,"last update":"Thu Nov 16 16:37:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.68000000000000005,"15min":0.65400000000000003},"min":{"1min":0.40899999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.3969999999999998,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.64400000000000002,"15min":0.61299999999999999},"min":{"1min":0.39400000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":2.3460000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.59299999999999997}]},{"osd":7,"last update":"Thu Nov 16 16:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.64300000000000002,"15min":0.64400000000000002},"min":{"1min":0.41599999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70199999999999996,"15min":0.70399999999999996},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.92700000000000005,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.57699999999999996}]},{"osd":8,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.57499999999999996,"15min":0.56899999999999995},"min":{"1min":0.38500000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.85199999999999998,"5min":1.008,"15min":1.008},"last":0.502},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.628,"15min":0.625},"min":{"1min":0.38400000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.61499999999999999}]},{"osd":9,"last update":"Thu Nov 16 16:37:54 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.86799999999999999,"15min":0.86699999999999999},"min":{"1min":0.47499999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.6080000000000001,"5min":4.524,"15min":4.524},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.751,"15min":0.73299999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.6200000000000001,"5min":2.6200000000000001,"15min":2.6200000000000001},"last":0.66200000000000003}]},{"osd":10,"last update":"Thu Nov 16 16:37:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.747}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149752,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6624,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738528,"statfs":{"total":95995035648,"available":95988252672,"internally_reserved":0,"allocated":942080,"data_stored":446526,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36499999999999999,"15min":0.34899999999999998},"min":{"1min":0.26500000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.52400000000000002,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.38,"15min":0.35799999999999998},"min":{"1min":0.27800000000000002,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.55100000000000005,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.34699999999999998}]},{"osd":1,"last update":"Thu Nov 16 16:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.55400000000000005,"15min":0.54900000000000004},"min":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55900000000000005,"15min":0.55700000000000005},"min":{"1min":0.36099999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.83399999999999996,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.63800000000000001}]},{"osd":2,"last update":"Thu Nov 16 16:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57899999999999996,"15min":0.55700000000000005},"min":{"1min":0.42599999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.83299999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56699999999999995,"15min":0.54700000000000004},"min":{"1min":0.40600000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.69699999999999995,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.628}]},{"osd":4,"last update":"Thu Nov 16 16:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56299999999999994,"15min":0.54900000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79900000000000004,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60599999999999998,"15min":0.59599999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.60799999999999998}]},{"osd":5,"last update":"Thu Nov 16 16:37:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.59799999999999998,"15min":0.57999999999999996},"min":{"1min":0.438,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60499999999999998,"15min":0.59099999999999997},"min":{"1min":0.38600000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.87,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.56999999999999995}]},{"osd":6,"last update":"Thu Nov 16 16:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51800000000000002,"15min":0.498},"min":{"1min":0.35599999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.72399999999999998,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.495},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53200000000000003,"15min":0.51400000000000001},"min":{"1min":0.377,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.75800000000000001,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.374}]},{"osd":7,"last update":"Thu Nov 16 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.65600000000000003,"15min":0.65000000000000002},"min":{"1min":0.44,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.69299999999999995,"15min":0.68700000000000006},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.747}]},{"osd":8,"last update":"Thu Nov 16 16:36:27 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.63200000000000001}]},{"osd":9,"last update":"Thu Nov 16 16:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.73799999999999999,"15min":0.76300000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80500000000000005,"5min":1.024,"15min":1.024},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.68999999999999995,"15min":0.70899999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90300000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.44600000000000001}]},{"osd":10,"last update":"Thu Nov 16 16:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.71599999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]}]},{"osd":4,"up_from":34,"seq":146028888117,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6424,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738728,"statfs":{"total":95995035648,"available":95988457472,"internally_reserved":0,"allocated":933888,"data_stored":446923,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.52400000000000002,"15min":0.502},"min":{"1min":0.38200000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.52900000000000003,"15min":0.51500000000000001},"min":{"1min":0.40899999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.378}]},{"osd":1,"last update":"Thu Nov 16 16:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.41099999999999998,"15min":0.39300000000000002},"min":{"1min":0.33700000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.67900000000000005,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.434,"15min":0.40999999999999998},"min":{"1min":0.378,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.748,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.36699999999999999}]},{"osd":2,"last update":"Thu Nov 16 16:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.77300000000000002,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.623,"15min":0.59299999999999997},"min":{"1min":0.53000000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.81999999999999995,"5min":1.056,"15min":1.056},"last":0.59699999999999998}]},{"osd":3,"last update":"Thu Nov 16 16:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.59199999999999997,"15min":0.55000000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57299999999999995,"15min":0.55300000000000005},"min":{"1min":0.40100000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.54000000000000004}]},{"osd":5,"last update":"Thu Nov 16 16:37:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.623,"15min":0.627},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.92100000000000004,"5min":1.075,"15min":1.075},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.63800000000000001,"15min":0.626},"min":{"1min":0.52200000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.94499999999999995,"5min":1.093,"15min":1.093},"last":0.67800000000000005}]},{"osd":6,"last update":"Thu Nov 16 16:37:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.63,"15min":0.60199999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84799999999999998,"5min":1.004,"15min":1.004},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.60899999999999999,"15min":0.59499999999999997},"min":{"1min":0.42399999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.496}]},{"osd":7,"last update":"Thu Nov 16 16:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.57199999999999995,"15min":0.54800000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.63200000000000001,"15min":0.60799999999999998},"min":{"1min":0.41199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.66500000000000004}]},{"osd":8,"last update":"Thu Nov 16 16:37:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.73999999999999999,"15min":0.72399999999999998},"min":{"1min":0.58999999999999997,"5min":0.495,"15min":0.495},"max":{"1min":1.0189999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.747,"15min":0.73099999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.70199999999999996}]},{"osd":9,"last update":"Thu Nov 16 16:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.79200000000000004,"15min":0.78900000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.94099999999999995,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.72099999999999997,"15min":0.71799999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94199999999999995,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.64500000000000002}]},{"osd":10,"last update":"Thu Nov 16 16:37:15 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.76000000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]}]},{"osd":5,"up_from":39,"seq":167503724594,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6556,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738596,"statfs":{"total":95995035648,"available":95988322304,"internally_reserved":0,"allocated":1003520,"data_stored":1038636,"data_compressed":41378,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.499,"15min":0.49399999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.64600000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52800000000000002,"15min":0.52400000000000002},"min":{"1min":0.312,"5min":0.311,"15min":0.311},"max":{"1min":0.75700000000000001,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.46000000000000002}]},{"osd":1,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.372,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.86399999999999999,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58799999999999997,"15min":0.58499999999999996},"min":{"1min":0.43099999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.94699999999999995,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.65000000000000002}]},{"osd":2,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.42799999999999999,"15min":0.42399999999999999},"min":{"1min":0.29599999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.67300000000000004,"5min":1.417,"15min":1.417},"last":0.438},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.443,"15min":0.433},"min":{"1min":0.32300000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.69899999999999995,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.48199999999999998}]},{"osd":3,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55700000000000005,"15min":0.54700000000000004},"min":{"1min":0.36199999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.81100000000000005,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.55100000000000005}]},{"osd":4,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.67900000000000005,"15min":0.629},"min":{"1min":0.40100000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":5.5789999999999997,"5min":5.5789999999999997,"15min":5.5789999999999997},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59699999999999998,"15min":0.59099999999999997},"min":{"1min":0.40400000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.68000000000000005}]},{"osd":6,"last update":"Thu Nov 16 16:37:47 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.67800000000000005,"15min":0.63600000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":3.274,"5min":3.274,"15min":3.274},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.68100000000000005,"15min":0.64600000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":3.472,"5min":3.472,"15min":3.472},"last":0.59399999999999997}]},{"osd":7,"last update":"Thu Nov 16 16:38:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.72199999999999998,"15min":0.71599999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0449999999999999,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.73799999999999999,"15min":0.73199999999999998},"min":{"1min":0.48099999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0580000000000001,"5min":1.544,"15min":1.544},"last":0.81499999999999995}]},{"osd":8,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.64600000000000002,"15min":0.64500000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72199999999999998,"15min":0.72399999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.79500000000000004}]},{"osd":9,"last update":"Thu Nov 16 16:37:52 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":0.90100000000000002,"15min":0.82699999999999996},"min":{"1min":0.58699999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":5.5679999999999996,"5min":5.5679999999999996,"15min":5.5679999999999996},"last":0.69699999999999995},{"interface":"front","average":{"1min":1.099,"5min":0.81599999999999995,"15min":0.76900000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":5.3230000000000004,"5min":5.3230000000000004,"15min":5.3230000000000004},"last":0.85599999999999998}]},{"osd":10,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.89400000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33928,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":58527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-16T16:38:12.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:12 smithi001 ceph-mon[98800]: pgmap v291: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 12 B/s, 1 keys/s, 1 objects/s recovering 2023-11-16T16:38:12.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:12 smithi026 ceph-mon[100112]: pgmap v291: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 12 B/s, 1 keys/s, 1 objects/s recovering 2023-11-16T16:38:12.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:12 smithi053 ceph-mon[100638]: pgmap v291: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 12 B/s, 1 keys/s, 1 objects/s recovering 2023-11-16T16:38:13.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:13 smithi026 ceph-mon[100112]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:38:13.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:13 smithi053 ceph-mon[100638]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:38:13.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:13 smithi001 ceph-mon[98800]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:38:14.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:14 smithi026 ceph-mon[100112]: pgmap v292: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 10 B/s, 0 keys/s, 1 objects/s recovering 2023-11-16T16:38:14.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:14 smithi053 ceph-mon[100638]: pgmap v292: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 10 B/s, 0 keys/s, 1 objects/s recovering 2023-11-16T16:38:14.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:14 smithi001 ceph-mon[98800]: pgmap v292: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 10 B/s, 0 keys/s, 1 objects/s recovering 2023-11-16T16:38:15.221 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph pg dump --format=json 2023-11-16T16:38:15.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:15 smithi053 ceph-mon[100638]: pgmap v293: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2023-11-16T16:38:15.890 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:38:15.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:15 smithi001 ceph-mon[98800]: pgmap v293: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2023-11-16T16:38:16.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:15 smithi026 ceph-mon[100112]: pgmap v293: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2023-11-16T16:38:17.104 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.102+0000 7ff926469700 1 -- 172.21.15.1:0/571267925 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff92010a900 msgr2=0x7ff92010cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:17.104 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.102+0000 7ff926469700 1 --2- 172.21.15.1:0/571267925 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff92010a900 0x7ff92010cd00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff91800b0a0 tx=0x7ff91800b3b0 comp rx=0 tx=0).stop 2023-11-16T16:38:17.104 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.102+0000 7ff926469700 1 -- 172.21.15.1:0/571267925 shutdown_connections 2023-11-16T16:38:17.104 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.102+0000 7ff926469700 1 --2- 172.21.15.1:0/571267925 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff92010a900 0x7ff92010cd00 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:17.104 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.102+0000 7ff926469700 1 --2- 172.21.15.1:0/571267925 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff9201013e0 0x7ff9201017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:17.104 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.102+0000 7ff926469700 1 --2- 172.21.15.1:0/571267925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff920101cf0 0x7ff92010a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:17.104 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.102+0000 7ff926469700 1 -- 172.21.15.1:0/571267925 >> 172.21.15.1:0/571267925 conn(0x7ff9200770d0 msgr2=0x7ff9200774d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:17.104 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.102+0000 7ff926469700 1 -- 172.21.15.1:0/571267925 shutdown_connections 2023-11-16T16:38:17.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.102+0000 7ff926469700 1 -- 172.21.15.1:0/571267925 wait complete. 2023-11-16T16:38:17.106 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.103+0000 7ff926469700 1 Processor -- start 2023-11-16T16:38:17.106 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.103+0000 7ff926469700 1 -- start start 2023-11-16T16:38:17.106 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.103+0000 7ff926469700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9201013e0 0x7ff9200ff210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:17.106 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.103+0000 7ff926469700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff920101cf0 0x7ff9200ff750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:17.107 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.103+0000 7ff926469700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff92010a900 0x7ff9200ffcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:17.107 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.103+0000 7ff926469700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9201a4a10 con 0x7ff92010a900 2023-11-16T16:38:17.107 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.103+0000 7ff926469700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9201a4b80 con 0x7ff9201013e0 2023-11-16T16:38:17.107 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.103+0000 7ff926469700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9201a4d40 con 0x7ff920101cf0 2023-11-16T16:38:17.107 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.103+0000 7ff924a06700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff92010a900 0x7ff9200ffcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:17.107 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.103+0000 7ff924a06700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff92010a900 0x7ff9200ffcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42132/0 (socket says 172.21.15.1:42132) 2023-11-16T16:38:17.107 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.103+0000 7ff91ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9201013e0 0x7ff9200ff210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:17.107 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.103+0000 7ff924a06700 1 -- 172.21.15.1:0/38874684 learned_addr learned my addr 172.21.15.1:0/38874684 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:17.107 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.103+0000 7ff91f7fe700 1 --2- 172.21.15.1:0/38874684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff920101cf0 0x7ff9200ff750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:17.107 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.103+0000 7ff924a06700 1 -- 172.21.15.1:0/38874684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff920101cf0 msgr2=0x7ff9200ff750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:17.107 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.103+0000 7ff924a06700 1 --2- 172.21.15.1:0/38874684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff920101cf0 0x7ff9200ff750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:17.107 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.104+0000 7ff924a06700 1 -- 172.21.15.1:0/38874684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9201013e0 msgr2=0x7ff9200ff210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:17.108 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.104+0000 7ff924a06700 1 --2- 172.21.15.1:0/38874684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9201013e0 0x7ff9200ff210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:17.108 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.104+0000 7ff924a06700 1 -- 172.21.15.1:0/38874684 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff918009d00 con 0x7ff92010a900 2023-11-16T16:38:17.108 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.104+0000 7ff91ffff700 1 --2- 172.21.15.1:0/38874684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9201013e0 0x7ff9200ff210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:38:17.108 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.104+0000 7ff924a06700 1 --2- 172.21.15.1:0/38874684 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff92010a900 0x7ff9200ffcd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff918007990 tx=0x7ff9180079c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:17.108 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.104+0000 7ff91d7fa700 1 -- 172.21.15.1:0/38874684 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91800e040 con 0x7ff92010a900 2023-11-16T16:38:17.108 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.104+0000 7ff91d7fa700 1 -- 172.21.15.1:0/38874684 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff91800be80 con 0x7ff92010a900 2023-11-16T16:38:17.108 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.104+0000 7ff926469700 1 -- 172.21.15.1:0/38874684 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff920110330 con 0x7ff92010a900 2023-11-16T16:38:17.108 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.104+0000 7ff91d7fa700 1 -- 172.21.15.1:0/38874684 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9180052a0 con 0x7ff92010a900 2023-11-16T16:38:17.108 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.104+0000 7ff926469700 1 -- 172.21.15.1:0/38874684 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9201101a0 con 0x7ff92010a900 2023-11-16T16:38:17.109 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.105+0000 7ff926469700 1 -- 172.21.15.1:0/38874684 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9201104c0 con 0x7ff92010a900 2023-11-16T16:38:17.109 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.106+0000 7ff91d7fa700 1 -- 172.21.15.1:0/38874684 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7ff91801d470 con 0x7ff92010a900 2023-11-16T16:38:17.109 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.107+0000 7ff91d7fa700 1 --2- 172.21.15.1:0/38874684 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7ff9080a1260 0x7ff9080a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:17.109 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.107+0000 7ff91ffff700 1 --2- 172.21.15.1:0/38874684 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7ff9080a1260 0x7ff9080a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:17.110 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.107+0000 7ff91d7fa700 1 -- 172.21.15.1:0/38874684 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff9180ce4b0 con 0x7ff92010a900 2023-11-16T16:38:17.110 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.107+0000 7ff91ffff700 1 --2- 172.21.15.1:0/38874684 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7ff9080a1260 0x7ff9080a3710 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff910011490 tx=0x7ff910009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:17.112 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.110+0000 7ff91d7fa700 1 -- 172.21.15.1:0/38874684 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff918097b30 con 0x7ff92010a900 2023-11-16T16:38:17.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.313+0000 7ff926469700 1 -- 172.21.15.1:0/38874684 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff920100740 con 0x7ff9080a1260 2023-11-16T16:38:17.327 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.325+0000 7ff91d7fa700 1 -- 172.21.15.1:0/38874684 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+341579 (secure 0 0 0) 0x7ff920100740 con 0x7ff9080a1260 2023-11-16T16:38:17.331 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:38:17.335 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.328+0000 7ff926469700 1 -- 172.21.15.1:0/38874684 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7ff9080a1260 msgr2=0x7ff9080a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:17.335 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.328+0000 7ff926469700 1 --2- 172.21.15.1:0/38874684 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7ff9080a1260 0x7ff9080a3710 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff910011490 tx=0x7ff910009260 comp rx=0 tx=0).stop 2023-11-16T16:38:17.335 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.328+0000 7ff926469700 1 -- 172.21.15.1:0/38874684 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff92010a900 msgr2=0x7ff9200ffcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:17.335 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.328+0000 7ff926469700 1 --2- 172.21.15.1:0/38874684 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff92010a900 0x7ff9200ffcd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff918007990 tx=0x7ff9180079c0 comp rx=0 tx=0).stop 2023-11-16T16:38:17.335 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.328+0000 7ff926469700 1 -- 172.21.15.1:0/38874684 shutdown_connections 2023-11-16T16:38:17.335 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.328+0000 7ff926469700 1 --2- 172.21.15.1:0/38874684 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7ff9080a1260 0x7ff9080a3710 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:17.335 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.328+0000 7ff926469700 1 --2- 172.21.15.1:0/38874684 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff92010a900 0x7ff9200ffcd0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:17.336 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.328+0000 7ff926469700 1 --2- 172.21.15.1:0/38874684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9201013e0 0x7ff9200ff210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:17.336 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.328+0000 7ff926469700 1 --2- 172.21.15.1:0/38874684 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff920101cf0 0x7ff9200ff750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:17.336 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.328+0000 7ff926469700 1 -- 172.21.15.1:0/38874684 >> 172.21.15.1:0/38874684 conn(0x7ff9200770d0 msgr2=0x7ff920102360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:17.336 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.328+0000 7ff926469700 1 -- 172.21.15.1:0/38874684 shutdown_connections 2023-11-16T16:38:17.336 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:17.328+0000 7ff926469700 1 -- 172.21.15.1:0/38874684 wait complete. 2023-11-16T16:38:17.336 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2023-11-16T16:38:17.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:17 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:38:17.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:17 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:38:17.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:17 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:38:17.961 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2023-11-16T16:38:15.486375+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653045,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":500,"num_read_kb":366,"num_write":333,"num_write_kb":3043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180848,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":79432,"kb_used_data":11368,"kb_used_omap":0,"kb_used_meta":67968,"kb_avail":1124862392,"statfs":{"total":1151940427776,"available":1151859089408,"internally_reserved":0,"allocated":11640832,"data_stored":7314397,"data_compressed":176451,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":0,"internal_metadata":69599232},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":-10,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002967"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.187022+0000","last_change":"2023-11-16T16:37:13.941551+0000","last_active":"2023-11-16T16:38:03.187022+0000","last_peered":"2023-11-16T16:38:03.187022+0000","last_clean":"2023-11-16T16:38:03.187022+0000","last_became_active":"2023-11-16T16:37:13.941190+0000","last_became_peered":"2023-11-16T16:37:13.941190+0000","last_unstale":"2023-11-16T16:38:03.187022+0000","last_undegraded":"2023-11-16T16:38:03.187022+0000","last_fullsized":"2023-11-16T16:38:03.187022+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:42:03.561127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.186116+0000","last_change":"2023-11-16T16:37:13.942300+0000","last_active":"2023-11-16T16:38:03.186116+0000","last_peered":"2023-11-16T16:38:03.186116+0000","last_clean":"2023-11-16T16:38:03.186116+0000","last_became_active":"2023-11-16T16:37:13.941966+0000","last_became_peered":"2023-11-16T16:37:13.941966+0000","last_unstale":"2023-11-16T16:38:03.186116+0000","last_undegraded":"2023-11-16T16:38:03.186116+0000","last_fullsized":"2023-11-16T16:38:03.186116+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:12:41.529896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.186042+0000","last_change":"2023-11-16T16:37:13.943461+0000","last_active":"2023-11-16T16:38:03.186042+0000","last_peered":"2023-11-16T16:38:03.186042+0000","last_clean":"2023-11-16T16:38:03.186042+0000","last_became_active":"2023-11-16T16:37:13.943165+0000","last_became_peered":"2023-11-16T16:37:13.943165+0000","last_unstale":"2023-11-16T16:38:03.186042+0000","last_undegraded":"2023-11-16T16:38:03.186042+0000","last_fullsized":"2023-11-16T16:38:03.186042+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:17:44.567499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184244+0000","last_change":"2023-11-16T16:37:13.940205+0000","last_active":"2023-11-16T16:38:03.184244+0000","last_peered":"2023-11-16T16:38:03.184244+0000","last_clean":"2023-11-16T16:38:03.184244+0000","last_became_active":"2023-11-16T16:37:13.939819+0000","last_became_peered":"2023-11-16T16:37:13.939819+0000","last_unstale":"2023-11-16T16:38:03.184244+0000","last_undegraded":"2023-11-16T16:38:03.184244+0000","last_fullsized":"2023-11-16T16:38:03.184244+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:52:42.183240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.190889+0000","last_change":"2023-11-16T16:37:13.939819+0000","last_active":"2023-11-16T16:38:03.190889+0000","last_peered":"2023-11-16T16:38:03.190889+0000","last_clean":"2023-11-16T16:38:03.190889+0000","last_became_active":"2023-11-16T16:37:13.939367+0000","last_became_peered":"2023-11-16T16:37:13.939367+0000","last_unstale":"2023-11-16T16:38:03.190889+0000","last_undegraded":"2023-11-16T16:38:03.190889+0000","last_fullsized":"2023-11-16T16:38:03.190889+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:17:30.563282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.187402+0000","last_change":"2023-11-16T16:37:13.946613+0000","last_active":"2023-11-16T16:38:03.187402+0000","last_peered":"2023-11-16T16:38:03.187402+0000","last_clean":"2023-11-16T16:38:03.187402+0000","last_became_active":"2023-11-16T16:37:13.946281+0000","last_became_peered":"2023-11-16T16:37:13.946281+0000","last_unstale":"2023-11-16T16:38:03.187402+0000","last_undegraded":"2023-11-16T16:38:03.187402+0000","last_fullsized":"2023-11-16T16:38:03.187402+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:29:42.599960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184741+0000","last_change":"2023-11-16T16:37:13.936899+0000","last_active":"2023-11-16T16:38:03.184741+0000","last_peered":"2023-11-16T16:38:03.184741+0000","last_clean":"2023-11-16T16:38:03.184741+0000","last_became_active":"2023-11-16T16:37:13.936543+0000","last_became_peered":"2023-11-16T16:37:13.936543+0000","last_unstale":"2023-11-16T16:38:03.184741+0000","last_undegraded":"2023-11-16T16:38:03.184741+0000","last_fullsized":"2023-11-16T16:38:03.184741+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:43:46.820689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.189989+0000","last_change":"2023-11-16T16:38:03.189467+0000","last_active":"2023-11-16T16:38:03.189989+0000","last_peered":"2023-11-16T16:38:03.189989+0000","last_clean":"2023-11-16T16:38:03.189989+0000","last_became_active":"2023-11-16T16:38:03.189017+0000","last_became_peered":"2023-11-16T16:38:03.189017+0000","last_unstale":"2023-11-16T16:38:03.189989+0000","last_undegraded":"2023-11-16T16:38:03.189989+0000","last_fullsized":"2023-11-16T16:38:03.189989+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:47:50.088436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.188864+0000","last_change":"2023-11-16T16:37:13.934105+0000","last_active":"2023-11-16T16:38:03.188864+0000","last_peered":"2023-11-16T16:38:03.188864+0000","last_clean":"2023-11-16T16:38:03.188864+0000","last_became_active":"2023-11-16T16:37:13.933887+0000","last_became_peered":"2023-11-16T16:37:13.933887+0000","last_unstale":"2023-11-16T16:38:03.188864+0000","last_undegraded":"2023-11-16T16:38:03.188864+0000","last_fullsized":"2023-11-16T16:38:03.188864+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:53:26.890511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.191855+0000","last_change":"2023-11-16T16:38:03.191855+0000","last_active":"2023-11-16T16:38:03.191855+0000","last_peered":"2023-11-16T16:38:03.191855+0000","last_clean":"2023-11-16T16:38:03.191855+0000","last_became_active":"2023-11-16T16:38:03.191398+0000","last_became_peered":"2023-11-16T16:38:03.191398+0000","last_unstale":"2023-11-16T16:38:03.191855+0000","last_undegraded":"2023-11-16T16:38:03.191855+0000","last_fullsized":"2023-11-16T16:38:03.191855+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:09:44.431927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184854+0000","last_change":"2023-11-16T16:37:13.941597+0000","last_active":"2023-11-16T16:38:03.184854+0000","last_peered":"2023-11-16T16:38:03.184854+0000","last_clean":"2023-11-16T16:38:03.184854+0000","last_became_active":"2023-11-16T16:37:13.941253+0000","last_became_peered":"2023-11-16T16:37:13.941253+0000","last_unstale":"2023-11-16T16:38:03.184854+0000","last_undegraded":"2023-11-16T16:38:03.184854+0000","last_fullsized":"2023-11-16T16:38:03.184854+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:13:35.256952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184130+0000","last_change":"2023-11-16T16:37:13.941673+0000","last_active":"2023-11-16T16:38:03.184130+0000","last_peered":"2023-11-16T16:38:03.184130+0000","last_clean":"2023-11-16T16:38:03.184130+0000","last_became_active":"2023-11-16T16:37:13.941345+0000","last_became_peered":"2023-11-16T16:37:13.941345+0000","last_unstale":"2023-11-16T16:38:03.184130+0000","last_undegraded":"2023-11-16T16:38:03.184130+0000","last_fullsized":"2023-11-16T16:38:03.184130+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:17:51.018141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184481+0000","last_change":"2023-11-16T16:37:13.947991+0000","last_active":"2023-11-16T16:38:03.184481+0000","last_peered":"2023-11-16T16:38:03.184481+0000","last_clean":"2023-11-16T16:38:03.184481+0000","last_became_active":"2023-11-16T16:37:13.947454+0000","last_became_peered":"2023-11-16T16:37:13.947454+0000","last_unstale":"2023-11-16T16:38:03.184481+0000","last_undegraded":"2023-11-16T16:38:03.184481+0000","last_fullsized":"2023-11-16T16:38:03.184481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:19:19.825458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.186271+0000","last_change":"2023-11-16T16:37:13.941502+0000","last_active":"2023-11-16T16:38:03.186271+0000","last_peered":"2023-11-16T16:38:03.186271+0000","last_clean":"2023-11-16T16:38:03.186271+0000","last_became_active":"2023-11-16T16:37:13.941191+0000","last_became_peered":"2023-11-16T16:37:13.941191+0000","last_unstale":"2023-11-16T16:38:03.186271+0000","last_undegraded":"2023-11-16T16:38:03.186271+0000","last_fullsized":"2023-11-16T16:38:03.186271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:19:35.239602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185898+0000","last_change":"2023-11-16T16:37:13.949198+0000","last_active":"2023-11-16T16:38:03.185898+0000","last_peered":"2023-11-16T16:38:03.185898+0000","last_clean":"2023-11-16T16:38:03.185898+0000","last_became_active":"2023-11-16T16:37:13.947147+0000","last_became_peered":"2023-11-16T16:37:13.947147+0000","last_unstale":"2023-11-16T16:38:03.185898+0000","last_undegraded":"2023-11-16T16:38:03.185898+0000","last_fullsized":"2023-11-16T16:38:03.185898+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:55:04.616796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.188109+0000","last_change":"2023-11-16T16:37:13.943915+0000","last_active":"2023-11-16T16:38:03.188109+0000","last_peered":"2023-11-16T16:38:03.188109+0000","last_clean":"2023-11-16T16:38:03.188109+0000","last_became_active":"2023-11-16T16:37:13.942995+0000","last_became_peered":"2023-11-16T16:37:13.942995+0000","last_unstale":"2023-11-16T16:38:03.188109+0000","last_undegraded":"2023-11-16T16:38:03.188109+0000","last_fullsized":"2023-11-16T16:38:03.188109+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:23:05.411045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.187167+0000","last_change":"2023-11-16T16:37:13.941816+0000","last_active":"2023-11-16T16:38:03.187167+0000","last_peered":"2023-11-16T16:38:03.187167+0000","last_clean":"2023-11-16T16:38:03.187167+0000","last_became_active":"2023-11-16T16:37:13.941541+0000","last_became_peered":"2023-11-16T16:37:13.941541+0000","last_unstale":"2023-11-16T16:38:03.187167+0000","last_undegraded":"2023-11-16T16:38:03.187167+0000","last_fullsized":"2023-11-16T16:38:03.187167+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:53:11.142308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185550+0000","last_change":"2023-11-16T16:37:13.940639+0000","last_active":"2023-11-16T16:38:03.185550+0000","last_peered":"2023-11-16T16:38:03.185550+0000","last_clean":"2023-11-16T16:38:03.185550+0000","last_became_active":"2023-11-16T16:37:13.940323+0000","last_became_peered":"2023-11-16T16:37:13.940323+0000","last_unstale":"2023-11-16T16:38:03.185550+0000","last_undegraded":"2023-11-16T16:38:03.185550+0000","last_fullsized":"2023-11-16T16:38:03.185550+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:32:03.882320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.189374+0000","last_change":"2023-11-16T16:37:13.934802+0000","last_active":"2023-11-16T16:38:03.189374+0000","last_peered":"2023-11-16T16:38:03.189374+0000","last_clean":"2023-11-16T16:38:03.189374+0000","last_became_active":"2023-11-16T16:37:13.934496+0000","last_became_peered":"2023-11-16T16:37:13.934496+0000","last_unstale":"2023-11-16T16:38:03.189374+0000","last_undegraded":"2023-11-16T16:38:03.189374+0000","last_fullsized":"2023-11-16T16:38:03.189374+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:15:58.249424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185585+0000","last_change":"2023-11-16T16:37:13.499284+0000","last_active":"2023-11-16T16:38:03.185585+0000","last_peered":"2023-11-16T16:38:03.185585+0000","last_clean":"2023-11-16T16:38:03.185585+0000","last_became_active":"2023-11-16T16:37:13.498931+0000","last_became_peered":"2023-11-16T16:37:13.498931+0000","last_unstale":"2023-11-16T16:38:03.185585+0000","last_undegraded":"2023-11-16T16:38:03.185585+0000","last_fullsized":"2023-11-16T16:38:03.185585+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:53:33.061352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.182899+0000","last_change":"2023-11-16T16:37:13.939631+0000","last_active":"2023-11-16T16:38:03.182899+0000","last_peered":"2023-11-16T16:38:03.182899+0000","last_clean":"2023-11-16T16:38:03.182899+0000","last_became_active":"2023-11-16T16:37:13.939318+0000","last_became_peered":"2023-11-16T16:37:13.939318+0000","last_unstale":"2023-11-16T16:38:03.182899+0000","last_undegraded":"2023-11-16T16:38:03.182899+0000","last_fullsized":"2023-11-16T16:38:03.182899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:11:33.421798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.183008+0000","last_change":"2023-11-16T16:37:13.495972+0000","last_active":"2023-11-16T16:38:03.183008+0000","last_peered":"2023-11-16T16:38:03.183008+0000","last_clean":"2023-11-16T16:38:03.183008+0000","last_became_active":"2023-11-16T16:37:13.495610+0000","last_became_peered":"2023-11-16T16:37:13.495610+0000","last_unstale":"2023-11-16T16:38:03.183008+0000","last_undegraded":"2023-11-16T16:38:03.183008+0000","last_fullsized":"2023-11-16T16:38:03.183008+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:47:20.115622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.186338+0000","last_change":"2023-11-16T16:37:13.935236+0000","last_active":"2023-11-16T16:38:03.186338+0000","last_peered":"2023-11-16T16:38:03.186338+0000","last_clean":"2023-11-16T16:38:03.186338+0000","last_became_active":"2023-11-16T16:37:13.934920+0000","last_became_peered":"2023-11-16T16:37:13.934920+0000","last_unstale":"2023-11-16T16:38:03.186338+0000","last_undegraded":"2023-11-16T16:38:03.186338+0000","last_fullsized":"2023-11-16T16:38:03.186338+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:04:59.207169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"84'4","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.197657+0000","last_change":"2023-11-16T16:38:03.197657+0000","last_active":"2023-11-16T16:38:03.197657+0000","last_peered":"2023-11-16T16:38:03.197657+0000","last_clean":"2023-11-16T16:38:03.197657+0000","last_became_active":"2023-11-16T16:38:03.190952+0000","last_became_peered":"2023-11-16T16:38:03.190952+0000","last_unstale":"2023-11-16T16:38:03.197657+0000","last_undegraded":"2023-11-16T16:38:03.197657+0000","last_fullsized":"2023-11-16T16:38:03.197657+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:04:50.256412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.188922+0000","last_change":"2023-11-16T16:37:13.940838+0000","last_active":"2023-11-16T16:38:03.188922+0000","last_peered":"2023-11-16T16:38:03.188922+0000","last_clean":"2023-11-16T16:38:03.188922+0000","last_became_active":"2023-11-16T16:37:13.940538+0000","last_became_peered":"2023-11-16T16:37:13.940538+0000","last_unstale":"2023-11-16T16:38:03.188922+0000","last_undegraded":"2023-11-16T16:38:03.188922+0000","last_fullsized":"2023-11-16T16:38:03.188922+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:47:36.031628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.190408+0000","last_change":"2023-11-16T16:37:13.495114+0000","last_active":"2023-11-16T16:38:03.190408+0000","last_peered":"2023-11-16T16:38:03.190408+0000","last_clean":"2023-11-16T16:38:03.190408+0000","last_became_active":"2023-11-16T16:37:13.494807+0000","last_became_peered":"2023-11-16T16:37:13.494807+0000","last_unstale":"2023-11-16T16:38:03.190408+0000","last_undegraded":"2023-11-16T16:38:03.190408+0000","last_fullsized":"2023-11-16T16:38:03.190408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:35:31.040933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185195+0000","last_change":"2023-11-16T16:37:13.943652+0000","last_active":"2023-11-16T16:38:03.185195+0000","last_peered":"2023-11-16T16:38:03.185195+0000","last_clean":"2023-11-16T16:38:03.185195+0000","last_became_active":"2023-11-16T16:37:13.943298+0000","last_became_peered":"2023-11-16T16:37:13.943298+0000","last_unstale":"2023-11-16T16:38:03.185195+0000","last_undegraded":"2023-11-16T16:38:03.185195+0000","last_fullsized":"2023-11-16T16:38:03.185195+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:17:55.154731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"84'6","reported_seq":84,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:14.387650+0000","last_change":"2023-11-16T16:37:13.509801+0000","last_active":"2023-11-16T16:38:14.387650+0000","last_peered":"2023-11-16T16:38:14.387650+0000","last_clean":"2023-11-16T16:38:14.387650+0000","last_became_active":"2023-11-16T16:37:13.509389+0000","last_became_peered":"2023-11-16T16:37:13.509389+0000","last_unstale":"2023-11-16T16:38:14.387650+0000","last_undegraded":"2023-11-16T16:38:14.387650+0000","last_fullsized":"2023-11-16T16:38:14.387650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:46:55.751281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":60,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.189967+0000","last_change":"2023-11-16T16:37:13.933871+0000","last_active":"2023-11-16T16:38:03.189967+0000","last_peered":"2023-11-16T16:38:03.189967+0000","last_clean":"2023-11-16T16:38:03.189967+0000","last_became_active":"2023-11-16T16:37:13.933549+0000","last_became_peered":"2023-11-16T16:37:13.933549+0000","last_unstale":"2023-11-16T16:38:03.189967+0000","last_undegraded":"2023-11-16T16:38:03.189967+0000","last_fullsized":"2023-11-16T16:38:03.189967+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:35:18.854524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"84'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.187212+0000","last_change":"2023-11-16T16:37:13.501339+0000","last_active":"2023-11-16T16:38:03.187212+0000","last_peered":"2023-11-16T16:38:03.187212+0000","last_clean":"2023-11-16T16:38:03.187212+0000","last_became_active":"2023-11-16T16:37:13.500949+0000","last_became_peered":"2023-11-16T16:37:13.500949+0000","last_unstale":"2023-11-16T16:38:03.187212+0000","last_undegraded":"2023-11-16T16:38:03.187212+0000","last_fullsized":"2023-11-16T16:38:03.187212+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:56:34.503681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.192595+0000","last_change":"2023-11-16T16:38:03.192595+0000","last_active":"2023-11-16T16:38:03.192595+0000","last_peered":"2023-11-16T16:38:03.192595+0000","last_clean":"2023-11-16T16:38:03.192595+0000","last_became_active":"2023-11-16T16:38:03.192208+0000","last_became_peered":"2023-11-16T16:38:03.192208+0000","last_unstale":"2023-11-16T16:38:03.192595+0000","last_undegraded":"2023-11-16T16:38:03.192595+0000","last_fullsized":"2023-11-16T16:38:03.192595+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:30:24.627049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"84'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184789+0000","last_change":"2023-11-16T16:37:13.505481+0000","last_active":"2023-11-16T16:38:03.184789+0000","last_peered":"2023-11-16T16:38:03.184789+0000","last_clean":"2023-11-16T16:38:03.184789+0000","last_became_active":"2023-11-16T16:37:13.505085+0000","last_became_peered":"2023-11-16T16:37:13.505085+0000","last_unstale":"2023-11-16T16:38:03.184789+0000","last_undegraded":"2023-11-16T16:38:03.184789+0000","last_fullsized":"2023-11-16T16:38:03.184789+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:21:13.673154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.182224+0000","last_change":"2023-11-16T16:37:13.943985+0000","last_active":"2023-11-16T16:38:03.182224+0000","last_peered":"2023-11-16T16:38:03.182224+0000","last_clean":"2023-11-16T16:38:03.182224+0000","last_became_active":"2023-11-16T16:37:13.943628+0000","last_became_peered":"2023-11-16T16:37:13.943628+0000","last_unstale":"2023-11-16T16:38:03.182224+0000","last_undegraded":"2023-11-16T16:38:03.182224+0000","last_fullsized":"2023-11-16T16:38:03.182224+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:01:16.596531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184704+0000","last_change":"2023-11-16T16:37:13.498648+0000","last_active":"2023-11-16T16:38:03.184704+0000","last_peered":"2023-11-16T16:38:03.184704+0000","last_clean":"2023-11-16T16:38:03.184704+0000","last_became_active":"2023-11-16T16:37:13.497390+0000","last_became_peered":"2023-11-16T16:37:13.497390+0000","last_unstale":"2023-11-16T16:38:03.184704+0000","last_undegraded":"2023-11-16T16:38:03.184704+0000","last_fullsized":"2023-11-16T16:38:03.184704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:20:50.541938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185282+0000","last_change":"2023-11-16T16:37:13.942337+0000","last_active":"2023-11-16T16:38:03.185282+0000","last_peered":"2023-11-16T16:38:03.185282+0000","last_clean":"2023-11-16T16:38:03.185282+0000","last_became_active":"2023-11-16T16:37:13.941982+0000","last_became_peered":"2023-11-16T16:37:13.941982+0000","last_unstale":"2023-11-16T16:38:03.185282+0000","last_undegraded":"2023-11-16T16:38:03.185282+0000","last_fullsized":"2023-11-16T16:38:03.185282+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:09:19.949618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'3","reported_seq":78,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:14.386891+0000","last_change":"2023-11-16T16:37:13.503690+0000","last_active":"2023-11-16T16:38:14.386891+0000","last_peered":"2023-11-16T16:38:14.386891+0000","last_clean":"2023-11-16T16:38:14.386891+0000","last_became_active":"2023-11-16T16:37:13.503203+0000","last_became_peered":"2023-11-16T16:37:13.503203+0000","last_unstale":"2023-11-16T16:38:14.386891+0000","last_undegraded":"2023-11-16T16:38:14.386891+0000","last_fullsized":"2023-11-16T16:38:14.386891+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:47:38.415047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":57,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.189257+0000","last_change":"2023-11-16T16:37:13.943975+0000","last_active":"2023-11-16T16:38:03.189257+0000","last_peered":"2023-11-16T16:38:03.189257+0000","last_clean":"2023-11-16T16:38:03.189257+0000","last_became_active":"2023-11-16T16:37:13.943611+0000","last_became_peered":"2023-11-16T16:37:13.943611+0000","last_unstale":"2023-11-16T16:38:03.189257+0000","last_undegraded":"2023-11-16T16:38:03.189257+0000","last_fullsized":"2023-11-16T16:38:03.189257+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:37:02.660396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'2","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.344191+0000","last_change":"2023-11-16T16:38:03.344191+0000","last_active":"2023-11-16T16:38:03.344191+0000","last_peered":"2023-11-16T16:38:03.344191+0000","last_clean":"2023-11-16T16:38:03.344191+0000","last_became_active":"2023-11-16T16:38:03.189976+0000","last_became_peered":"2023-11-16T16:38:03.189976+0000","last_unstale":"2023-11-16T16:38:03.344191+0000","last_undegraded":"2023-11-16T16:38:03.344191+0000","last_fullsized":"2023-11-16T16:38:03.344191+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:04:56.889493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.186497+0000","last_change":"2023-11-16T16:37:13.949031+0000","last_active":"2023-11-16T16:38:03.186497+0000","last_peered":"2023-11-16T16:38:03.186497+0000","last_clean":"2023-11-16T16:38:03.186497+0000","last_became_active":"2023-11-16T16:37:13.946923+0000","last_became_peered":"2023-11-16T16:37:13.946923+0000","last_unstale":"2023-11-16T16:38:03.186497+0000","last_undegraded":"2023-11-16T16:38:03.186497+0000","last_fullsized":"2023-11-16T16:38:03.186497+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:04:43.521722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185398+0000","last_change":"2023-11-16T16:37:13.495003+0000","last_active":"2023-11-16T16:38:03.185398+0000","last_peered":"2023-11-16T16:38:03.185398+0000","last_clean":"2023-11-16T16:38:03.185398+0000","last_became_active":"2023-11-16T16:37:13.494563+0000","last_became_peered":"2023-11-16T16:37:13.494563+0000","last_unstale":"2023-11-16T16:38:03.185398+0000","last_undegraded":"2023-11-16T16:38:03.185398+0000","last_fullsized":"2023-11-16T16:38:03.185398+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:36:36.403721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.189090+0000","last_change":"2023-11-16T16:37:13.943736+0000","last_active":"2023-11-16T16:38:03.189090+0000","last_peered":"2023-11-16T16:38:03.189090+0000","last_clean":"2023-11-16T16:38:03.189090+0000","last_became_active":"2023-11-16T16:37:13.943391+0000","last_became_peered":"2023-11-16T16:37:13.943391+0000","last_unstale":"2023-11-16T16:38:03.189090+0000","last_undegraded":"2023-11-16T16:38:03.189090+0000","last_fullsized":"2023-11-16T16:38:03.189090+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:27:01.809877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'28","reported_seq":168,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:15.399607+0000","last_change":"2023-11-16T16:37:13.506508+0000","last_active":"2023-11-16T16:38:15.399607+0000","last_peered":"2023-11-16T16:38:15.399607+0000","last_clean":"2023-11-16T16:38:15.399607+0000","last_became_active":"2023-11-16T16:37:13.506045+0000","last_became_peered":"2023-11-16T16:37:13.506045+0000","last_unstale":"2023-11-16T16:38:15.399607+0000","last_undegraded":"2023-11-16T16:38:15.399607+0000","last_fullsized":"2023-11-16T16:38:15.399607+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:57:44.753422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58527,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":122,"num_read_kb":117,"num_write":29,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.191240+0000","last_change":"2023-11-16T16:37:13.939717+0000","last_active":"2023-11-16T16:38:03.191240+0000","last_peered":"2023-11-16T16:38:03.191240+0000","last_clean":"2023-11-16T16:38:03.191240+0000","last_became_active":"2023-11-16T16:37:13.939369+0000","last_became_peered":"2023-11-16T16:37:13.939369+0000","last_unstale":"2023-11-16T16:38:03.191240+0000","last_undegraded":"2023-11-16T16:38:03.191240+0000","last_fullsized":"2023-11-16T16:38:03.191240+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:10:31.454459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.188492+0000","last_change":"2023-11-16T16:37:13.506056+0000","last_active":"2023-11-16T16:38:03.188492+0000","last_peered":"2023-11-16T16:38:03.188492+0000","last_clean":"2023-11-16T16:38:03.188492+0000","last_became_active":"2023-11-16T16:37:13.505707+0000","last_became_peered":"2023-11-16T16:37:13.505707+0000","last_unstale":"2023-11-16T16:38:03.188492+0000","last_undegraded":"2023-11-16T16:38:03.188492+0000","last_fullsized":"2023-11-16T16:38:03.188492+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:29:47.242470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184551+0000","last_change":"2023-11-16T16:37:13.947764+0000","last_active":"2023-11-16T16:38:03.184551+0000","last_peered":"2023-11-16T16:38:03.184551+0000","last_clean":"2023-11-16T16:38:03.184551+0000","last_became_active":"2023-11-16T16:37:13.947510+0000","last_became_peered":"2023-11-16T16:37:13.947510+0000","last_unstale":"2023-11-16T16:38:03.184551+0000","last_undegraded":"2023-11-16T16:38:03.184551+0000","last_fullsized":"2023-11-16T16:38:03.184551+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:41:59.672529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184939+0000","last_change":"2023-11-16T16:37:13.498178+0000","last_active":"2023-11-16T16:38:03.184939+0000","last_peered":"2023-11-16T16:38:03.184939+0000","last_clean":"2023-11-16T16:38:03.184939+0000","last_became_active":"2023-11-16T16:37:13.497312+0000","last_became_peered":"2023-11-16T16:37:13.497312+0000","last_unstale":"2023-11-16T16:38:03.184939+0000","last_undegraded":"2023-11-16T16:38:03.184939+0000","last_fullsized":"2023-11-16T16:38:03.184939+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:54:40.375590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185267+0000","last_change":"2023-11-16T16:37:13.947854+0000","last_active":"2023-11-16T16:38:03.185267+0000","last_peered":"2023-11-16T16:38:03.185267+0000","last_clean":"2023-11-16T16:38:03.185267+0000","last_became_active":"2023-11-16T16:37:13.947530+0000","last_became_peered":"2023-11-16T16:37:13.947530+0000","last_unstale":"2023-11-16T16:38:03.185267+0000","last_undegraded":"2023-11-16T16:38:03.185267+0000","last_fullsized":"2023-11-16T16:38:03.185267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:41:00.205641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'3","reported_seq":78,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:12.384667+0000","last_change":"2023-11-16T16:37:13.500127+0000","last_active":"2023-11-16T16:38:12.384667+0000","last_peered":"2023-11-16T16:38:12.384667+0000","last_clean":"2023-11-16T16:38:12.384667+0000","last_became_active":"2023-11-16T16:37:13.499758+0000","last_became_peered":"2023-11-16T16:37:13.499758+0000","last_unstale":"2023-11-16T16:38:12.384667+0000","last_undegraded":"2023-11-16T16:38:12.384667+0000","last_fullsized":"2023-11-16T16:38:12.384667+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:39:00.268144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.183305+0000","last_change":"2023-11-16T16:37:13.940962+0000","last_active":"2023-11-16T16:38:03.183305+0000","last_peered":"2023-11-16T16:38:03.183305+0000","last_clean":"2023-11-16T16:38:03.183305+0000","last_became_active":"2023-11-16T16:37:13.940450+0000","last_became_peered":"2023-11-16T16:37:13.940450+0000","last_unstale":"2023-11-16T16:38:03.183305+0000","last_undegraded":"2023-11-16T16:38:03.183305+0000","last_fullsized":"2023-11-16T16:38:03.183305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:28:45.946454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'15","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.228174+0000","last_change":"2023-11-16T16:38:03.228174+0000","last_active":"2023-11-16T16:38:03.228174+0000","last_peered":"2023-11-16T16:38:03.228174+0000","last_clean":"2023-11-16T16:38:03.228174+0000","last_became_active":"2023-11-16T16:38:03.192020+0000","last_became_peered":"2023-11-16T16:38:03.192020+0000","last_unstale":"2023-11-16T16:38:03.228174+0000","last_undegraded":"2023-11-16T16:38:03.228174+0000","last_fullsized":"2023-11-16T16:38:03.228174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:54:01.231005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184704+0000","last_change":"2023-11-16T16:37:13.944878+0000","last_active":"2023-11-16T16:38:03.184704+0000","last_peered":"2023-11-16T16:38:03.184704+0000","last_clean":"2023-11-16T16:38:03.184704+0000","last_became_active":"2023-11-16T16:37:13.944580+0000","last_became_peered":"2023-11-16T16:37:13.944580+0000","last_unstale":"2023-11-16T16:38:03.184704+0000","last_undegraded":"2023-11-16T16:38:03.184704+0000","last_fullsized":"2023-11-16T16:38:03.184704+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:00:01.106243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.190628+0000","last_change":"2023-11-16T16:37:13.499139+0000","last_active":"2023-11-16T16:38:03.190628+0000","last_peered":"2023-11-16T16:38:03.190628+0000","last_clean":"2023-11-16T16:38:03.190628+0000","last_became_active":"2023-11-16T16:37:13.498755+0000","last_became_peered":"2023-11-16T16:37:13.498755+0000","last_unstale":"2023-11-16T16:38:03.190628+0000","last_undegraded":"2023-11-16T16:38:03.190628+0000","last_fullsized":"2023-11-16T16:38:03.190628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T16:55:06.065509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.186856+0000","last_change":"2023-11-16T16:37:13.946187+0000","last_active":"2023-11-16T16:38:03.186856+0000","last_peered":"2023-11-16T16:38:03.186856+0000","last_clean":"2023-11-16T16:38:03.186856+0000","last_became_active":"2023-11-16T16:37:13.945887+0000","last_became_peered":"2023-11-16T16:37:13.945887+0000","last_unstale":"2023-11-16T16:38:03.186856+0000","last_undegraded":"2023-11-16T16:38:03.186856+0000","last_fullsized":"2023-11-16T16:38:03.186856+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:41:31.726749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185150+0000","last_change":"2023-11-16T16:37:13.507150+0000","last_active":"2023-11-16T16:38:03.185150+0000","last_peered":"2023-11-16T16:38:03.185150+0000","last_clean":"2023-11-16T16:38:03.185150+0000","last_became_active":"2023-11-16T16:37:13.506814+0000","last_became_peered":"2023-11-16T16:37:13.506814+0000","last_unstale":"2023-11-16T16:38:03.185150+0000","last_undegraded":"2023-11-16T16:38:03.185150+0000","last_fullsized":"2023-11-16T16:38:03.185150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:54:45.031046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.187711+0000","last_change":"2023-11-16T16:37:13.947866+0000","last_active":"2023-11-16T16:38:03.187711+0000","last_peered":"2023-11-16T16:38:03.187711+0000","last_clean":"2023-11-16T16:38:03.187711+0000","last_became_active":"2023-11-16T16:37:13.947544+0000","last_became_peered":"2023-11-16T16:37:13.947544+0000","last_unstale":"2023-11-16T16:38:03.187711+0000","last_undegraded":"2023-11-16T16:38:03.187711+0000","last_fullsized":"2023-11-16T16:38:03.187711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:23:27.220130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"84'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184212+0000","last_change":"2023-11-16T16:37:13.497800+0000","last_active":"2023-11-16T16:38:03.184212+0000","last_peered":"2023-11-16T16:38:03.184212+0000","last_clean":"2023-11-16T16:38:03.184212+0000","last_became_active":"2023-11-16T16:37:13.497414+0000","last_became_peered":"2023-11-16T16:37:13.497414+0000","last_unstale":"2023-11-16T16:38:03.184212+0000","last_undegraded":"2023-11-16T16:38:03.184212+0000","last_fullsized":"2023-11-16T16:38:03.184212+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:17:42.179250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.187179+0000","last_change":"2023-11-16T16:37:13.946780+0000","last_active":"2023-11-16T16:38:03.187179+0000","last_peered":"2023-11-16T16:38:03.187179+0000","last_clean":"2023-11-16T16:38:03.187179+0000","last_became_active":"2023-11-16T16:37:13.946441+0000","last_became_peered":"2023-11-16T16:37:13.946441+0000","last_unstale":"2023-11-16T16:38:03.187179+0000","last_undegraded":"2023-11-16T16:38:03.187179+0000","last_fullsized":"2023-11-16T16:38:03.187179+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:46:10.856707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.192571+0000","last_change":"2023-11-16T16:38:03.191887+0000","last_active":"2023-11-16T16:38:03.192571+0000","last_peered":"2023-11-16T16:38:03.192571+0000","last_clean":"2023-11-16T16:38:03.192571+0000","last_became_active":"2023-11-16T16:38:03.191402+0000","last_became_peered":"2023-11-16T16:38:03.191402+0000","last_unstale":"2023-11-16T16:38:03.192571+0000","last_undegraded":"2023-11-16T16:38:03.192571+0000","last_fullsized":"2023-11-16T16:38:03.192571+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:47:26.769740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"84'4","reported_seq":77,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:12.384108+0000","last_change":"2023-11-16T16:37:13.498129+0000","last_active":"2023-11-16T16:38:12.384108+0000","last_peered":"2023-11-16T16:38:12.384108+0000","last_clean":"2023-11-16T16:38:12.384108+0000","last_became_active":"2023-11-16T16:37:13.497765+0000","last_became_peered":"2023-11-16T16:37:13.497765+0000","last_unstale":"2023-11-16T16:38:12.384108+0000","last_undegraded":"2023-11-16T16:38:12.384108+0000","last_fullsized":"2023-11-16T16:38:12.384108+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:15:51.118817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.182609+0000","last_change":"2023-11-16T16:37:13.938417+0000","last_active":"2023-11-16T16:38:03.182609+0000","last_peered":"2023-11-16T16:38:03.182609+0000","last_clean":"2023-11-16T16:38:03.182609+0000","last_became_active":"2023-11-16T16:37:13.938114+0000","last_became_peered":"2023-11-16T16:37:13.938114+0000","last_unstale":"2023-11-16T16:38:03.182609+0000","last_undegraded":"2023-11-16T16:38:03.182609+0000","last_fullsized":"2023-11-16T16:38:03.182609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:07:00.909431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"84'3","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.183990+0000","last_change":"2023-11-16T16:37:13.499045+0000","last_active":"2023-11-16T16:38:03.183990+0000","last_peered":"2023-11-16T16:38:03.183990+0000","last_clean":"2023-11-16T16:38:03.183990+0000","last_became_active":"2023-11-16T16:37:13.498598+0000","last_became_peered":"2023-11-16T16:37:13.498598+0000","last_unstale":"2023-11-16T16:38:03.183990+0000","last_undegraded":"2023-11-16T16:38:03.183990+0000","last_fullsized":"2023-11-16T16:38:03.183990+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:29:46.898930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":349,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.183816+0000","last_change":"2023-11-16T16:36:14.090213+0000","last_active":"2023-11-16T16:38:03.183816+0000","last_peered":"2023-11-16T16:38:03.183816+0000","last_clean":"2023-11-16T16:38:03.183816+0000","last_became_active":"2023-11-16T16:36:14.089742+0000","last_became_peered":"2023-11-16T16:36:14.089742+0000","last_unstale":"2023-11-16T16:38:03.183816+0000","last_undegraded":"2023-11-16T16:38:03.183816+0000","last_fullsized":"2023-11-16T16:38:03.183816+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:33:16.178716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:33:16.178716+0000","last_clean_scrub_stamp":"2023-11-16T16:33:16.178716+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:34:52.910521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184225+0000","last_change":"2023-11-16T16:37:13.945380+0000","last_active":"2023-11-16T16:38:03.184225+0000","last_peered":"2023-11-16T16:38:03.184225+0000","last_clean":"2023-11-16T16:38:03.184225+0000","last_became_active":"2023-11-16T16:37:13.945038+0000","last_became_peered":"2023-11-16T16:37:13.945038+0000","last_unstale":"2023-11-16T16:38:03.184225+0000","last_undegraded":"2023-11-16T16:38:03.184225+0000","last_fullsized":"2023-11-16T16:38:03.184225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:29:11.812243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"84'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184055+0000","last_change":"2023-11-16T16:37:13.497586+0000","last_active":"2023-11-16T16:38:03.184055+0000","last_peered":"2023-11-16T16:38:03.184055+0000","last_clean":"2023-11-16T16:38:03.184055+0000","last_became_active":"2023-11-16T16:37:13.497233+0000","last_became_peered":"2023-11-16T16:37:13.497233+0000","last_unstale":"2023-11-16T16:38:03.184055+0000","last_undegraded":"2023-11-16T16:38:03.184055+0000","last_fullsized":"2023-11-16T16:38:03.184055+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:47:21.097628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"84'1","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.191390+0000","last_change":"2023-11-16T16:37:13.500621+0000","last_active":"2023-11-16T16:38:03.191390+0000","last_peered":"2023-11-16T16:38:03.191390+0000","last_clean":"2023-11-16T16:38:03.191390+0000","last_became_active":"2023-11-16T16:37:13.500265+0000","last_became_peered":"2023-11-16T16:37:13.500265+0000","last_unstale":"2023-11-16T16:38:03.191390+0000","last_undegraded":"2023-11-16T16:38:03.191390+0000","last_fullsized":"2023-11-16T16:38:03.191390+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:22:59.748224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184391+0000","last_change":"2023-11-16T16:37:13.930992+0000","last_active":"2023-11-16T16:38:03.184391+0000","last_peered":"2023-11-16T16:38:03.184391+0000","last_clean":"2023-11-16T16:38:03.184391+0000","last_became_active":"2023-11-16T16:37:13.930641+0000","last_became_peered":"2023-11-16T16:37:13.930641+0000","last_unstale":"2023-11-16T16:38:03.184391+0000","last_undegraded":"2023-11-16T16:38:03.184391+0000","last_fullsized":"2023-11-16T16:38:03.184391+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:24:49.314642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'5","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185381+0000","last_change":"2023-11-16T16:37:13.498637+0000","last_active":"2023-11-16T16:38:03.185381+0000","last_peered":"2023-11-16T16:38:03.185381+0000","last_clean":"2023-11-16T16:38:03.185381+0000","last_became_active":"2023-11-16T16:37:13.498341+0000","last_became_peered":"2023-11-16T16:37:13.498341+0000","last_unstale":"2023-11-16T16:38:03.185381+0000","last_undegraded":"2023-11-16T16:38:03.185381+0000","last_fullsized":"2023-11-16T16:38:03.185381+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:30:04.558815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.182831+0000","last_change":"2023-11-16T16:37:13.944142+0000","last_active":"2023-11-16T16:38:03.182831+0000","last_peered":"2023-11-16T16:38:03.182831+0000","last_clean":"2023-11-16T16:38:03.182831+0000","last_became_active":"2023-11-16T16:37:13.943758+0000","last_became_peered":"2023-11-16T16:37:13.943758+0000","last_unstale":"2023-11-16T16:38:03.182831+0000","last_undegraded":"2023-11-16T16:38:03.182831+0000","last_fullsized":"2023-11-16T16:38:03.182831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:57:58.601449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184558+0000","last_change":"2023-11-16T16:37:13.496900+0000","last_active":"2023-11-16T16:38:03.184558+0000","last_peered":"2023-11-16T16:38:03.184558+0000","last_clean":"2023-11-16T16:38:03.184558+0000","last_became_active":"2023-11-16T16:37:13.496528+0000","last_became_peered":"2023-11-16T16:37:13.496528+0000","last_unstale":"2023-11-16T16:38:03.184558+0000","last_undegraded":"2023-11-16T16:38:03.184558+0000","last_fullsized":"2023-11-16T16:38:03.184558+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:56:42.304030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.186630+0000","last_change":"2023-11-16T16:37:13.943104+0000","last_active":"2023-11-16T16:38:03.186630+0000","last_peered":"2023-11-16T16:38:03.186630+0000","last_clean":"2023-11-16T16:38:03.186630+0000","last_became_active":"2023-11-16T16:37:13.942774+0000","last_became_peered":"2023-11-16T16:37:13.942774+0000","last_unstale":"2023-11-16T16:38:03.186630+0000","last_undegraded":"2023-11-16T16:38:03.186630+0000","last_fullsized":"2023-11-16T16:38:03.186630+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:24:11.162929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.188270+0000","last_change":"2023-11-16T16:37:13.505994+0000","last_active":"2023-11-16T16:38:03.188270+0000","last_peered":"2023-11-16T16:38:03.188270+0000","last_clean":"2023-11-16T16:38:03.188270+0000","last_became_active":"2023-11-16T16:37:13.505663+0000","last_became_peered":"2023-11-16T16:37:13.505663+0000","last_unstale":"2023-11-16T16:38:03.188270+0000","last_undegraded":"2023-11-16T16:38:03.188270+0000","last_fullsized":"2023-11-16T16:38:03.188270+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:06:13.680491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.189650+0000","last_change":"2023-11-16T16:37:13.941970+0000","last_active":"2023-11-16T16:38:03.189650+0000","last_peered":"2023-11-16T16:38:03.189650+0000","last_clean":"2023-11-16T16:38:03.189650+0000","last_became_active":"2023-11-16T16:37:13.941649+0000","last_became_peered":"2023-11-16T16:37:13.941649+0000","last_unstale":"2023-11-16T16:38:03.189650+0000","last_undegraded":"2023-11-16T16:38:03.189650+0000","last_fullsized":"2023-11-16T16:38:03.189650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:23:17.814240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.182332+0000","last_change":"2023-11-16T16:37:13.503429+0000","last_active":"2023-11-16T16:38:03.182332+0000","last_peered":"2023-11-16T16:38:03.182332+0000","last_clean":"2023-11-16T16:38:03.182332+0000","last_became_active":"2023-11-16T16:37:13.503114+0000","last_became_peered":"2023-11-16T16:37:13.503114+0000","last_unstale":"2023-11-16T16:38:03.182332+0000","last_undegraded":"2023-11-16T16:38:03.182332+0000","last_fullsized":"2023-11-16T16:38:03.182332+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:57:19.910282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.186389+0000","last_change":"2023-11-16T16:37:13.941225+0000","last_active":"2023-11-16T16:38:03.186389+0000","last_peered":"2023-11-16T16:38:03.186389+0000","last_clean":"2023-11-16T16:38:03.186389+0000","last_became_active":"2023-11-16T16:37:13.940964+0000","last_became_peered":"2023-11-16T16:37:13.940964+0000","last_unstale":"2023-11-16T16:38:03.186389+0000","last_undegraded":"2023-11-16T16:38:03.186389+0000","last_fullsized":"2023-11-16T16:38:03.186389+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:21:58.992812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.188151+0000","last_change":"2023-11-16T16:37:13.510154+0000","last_active":"2023-11-16T16:38:03.188151+0000","last_peered":"2023-11-16T16:38:03.188151+0000","last_clean":"2023-11-16T16:38:03.188151+0000","last_became_active":"2023-11-16T16:37:13.509790+0000","last_became_peered":"2023-11-16T16:37:13.509790+0000","last_unstale":"2023-11-16T16:38:03.188151+0000","last_undegraded":"2023-11-16T16:38:03.188151+0000","last_fullsized":"2023-11-16T16:38:03.188151+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:09:32.999016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.182648+0000","last_change":"2023-11-16T16:37:13.942203+0000","last_active":"2023-11-16T16:38:03.182648+0000","last_peered":"2023-11-16T16:38:03.182648+0000","last_clean":"2023-11-16T16:38:03.182648+0000","last_became_active":"2023-11-16T16:37:13.941766+0000","last_became_peered":"2023-11-16T16:37:13.941766+0000","last_unstale":"2023-11-16T16:38:03.182648+0000","last_undegraded":"2023-11-16T16:38:03.182648+0000","last_fullsized":"2023-11-16T16:38:03.182648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:50:04.530728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.190859+0000","last_change":"2023-11-16T16:37:13.509854+0000","last_active":"2023-11-16T16:38:03.190859+0000","last_peered":"2023-11-16T16:38:03.190859+0000","last_clean":"2023-11-16T16:38:03.190859+0000","last_became_active":"2023-11-16T16:37:13.509466+0000","last_became_peered":"2023-11-16T16:37:13.509466+0000","last_unstale":"2023-11-16T16:38:03.190859+0000","last_undegraded":"2023-11-16T16:38:03.190859+0000","last_fullsized":"2023-11-16T16:38:03.190859+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:38:13.537860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184479+0000","last_change":"2023-11-16T16:37:13.947906+0000","last_active":"2023-11-16T16:38:03.184479+0000","last_peered":"2023-11-16T16:38:03.184479+0000","last_clean":"2023-11-16T16:38:03.184479+0000","last_became_active":"2023-11-16T16:37:13.947602+0000","last_became_peered":"2023-11-16T16:37:13.947602+0000","last_unstale":"2023-11-16T16:38:03.184479+0000","last_undegraded":"2023-11-16T16:38:03.184479+0000","last_fullsized":"2023-11-16T16:38:03.184479+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:16:45.611613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.188948+0000","last_change":"2023-11-16T16:37:13.506108+0000","last_active":"2023-11-16T16:38:03.188948+0000","last_peered":"2023-11-16T16:38:03.188948+0000","last_clean":"2023-11-16T16:38:03.188948+0000","last_became_active":"2023-11-16T16:37:13.505785+0000","last_became_peered":"2023-11-16T16:37:13.505785+0000","last_unstale":"2023-11-16T16:38:03.188948+0000","last_undegraded":"2023-11-16T16:38:03.188948+0000","last_fullsized":"2023-11-16T16:38:03.188948+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:11:02.793557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185008+0000","last_change":"2023-11-16T16:37:13.935863+0000","last_active":"2023-11-16T16:38:03.185008+0000","last_peered":"2023-11-16T16:38:03.185008+0000","last_clean":"2023-11-16T16:38:03.185008+0000","last_became_active":"2023-11-16T16:37:13.935399+0000","last_became_peered":"2023-11-16T16:37:13.935399+0000","last_unstale":"2023-11-16T16:38:03.185008+0000","last_undegraded":"2023-11-16T16:38:03.185008+0000","last_fullsized":"2023-11-16T16:38:03.185008+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:45:56.613743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184907+0000","last_change":"2023-11-16T16:37:13.498367+0000","last_active":"2023-11-16T16:38:03.184907+0000","last_peered":"2023-11-16T16:38:03.184907+0000","last_clean":"2023-11-16T16:38:03.184907+0000","last_became_active":"2023-11-16T16:37:13.497039+0000","last_became_peered":"2023-11-16T16:37:13.497039+0000","last_unstale":"2023-11-16T16:38:03.184907+0000","last_undegraded":"2023-11-16T16:38:03.184907+0000","last_fullsized":"2023-11-16T16:38:03.184907+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:57:46.571219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184415+0000","last_change":"2023-11-16T16:37:13.940617+0000","last_active":"2023-11-16T16:38:03.184415+0000","last_peered":"2023-11-16T16:38:03.184415+0000","last_clean":"2023-11-16T16:38:03.184415+0000","last_became_active":"2023-11-16T16:37:13.940264+0000","last_became_peered":"2023-11-16T16:37:13.940264+0000","last_unstale":"2023-11-16T16:38:03.184415+0000","last_undegraded":"2023-11-16T16:38:03.184415+0000","last_fullsized":"2023-11-16T16:38:03.184415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:59:36.630848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.193922+0000","last_change":"2023-11-16T16:38:03.193922+0000","last_active":"2023-11-16T16:38:03.193922+0000","last_peered":"2023-11-16T16:38:03.193922+0000","last_clean":"2023-11-16T16:38:03.193922+0000","last_became_active":"2023-11-16T16:38:03.190364+0000","last_became_peered":"2023-11-16T16:38:03.190364+0000","last_unstale":"2023-11-16T16:38:03.193922+0000","last_undegraded":"2023-11-16T16:38:03.193922+0000","last_fullsized":"2023-11-16T16:38:03.193922+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:59:04.681132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185280+0000","last_change":"2023-11-16T16:37:13.943096+0000","last_active":"2023-11-16T16:38:03.185280+0000","last_peered":"2023-11-16T16:38:03.185280+0000","last_clean":"2023-11-16T16:38:03.185280+0000","last_became_active":"2023-11-16T16:37:13.942745+0000","last_became_peered":"2023-11-16T16:37:13.942745+0000","last_unstale":"2023-11-16T16:38:03.185280+0000","last_undegraded":"2023-11-16T16:38:03.185280+0000","last_fullsized":"2023-11-16T16:38:03.185280+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:02:12.721195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.186978+0000","last_change":"2023-11-16T16:37:13.938087+0000","last_active":"2023-11-16T16:38:03.186978+0000","last_peered":"2023-11-16T16:38:03.186978+0000","last_clean":"2023-11-16T16:38:03.186978+0000","last_became_active":"2023-11-16T16:37:13.937780+0000","last_became_peered":"2023-11-16T16:37:13.937780+0000","last_unstale":"2023-11-16T16:38:03.186978+0000","last_undegraded":"2023-11-16T16:38:03.186978+0000","last_fullsized":"2023-11-16T16:38:03.186978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:05:14.399850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.188244+0000","last_change":"2023-11-16T16:37:13.939000+0000","last_active":"2023-11-16T16:38:03.188244+0000","last_peered":"2023-11-16T16:38:03.188244+0000","last_clean":"2023-11-16T16:38:03.188244+0000","last_became_active":"2023-11-16T16:37:13.938585+0000","last_became_peered":"2023-11-16T16:37:13.938585+0000","last_unstale":"2023-11-16T16:38:03.188244+0000","last_undegraded":"2023-11-16T16:38:03.188244+0000","last_fullsized":"2023-11-16T16:38:03.188244+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:13:17.753666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.187970+0000","last_change":"2023-11-16T16:37:13.937696+0000","last_active":"2023-11-16T16:38:03.187970+0000","last_peered":"2023-11-16T16:38:03.187970+0000","last_clean":"2023-11-16T16:38:03.187970+0000","last_became_active":"2023-11-16T16:37:13.936920+0000","last_became_peered":"2023-11-16T16:37:13.936920+0000","last_unstale":"2023-11-16T16:38:03.187970+0000","last_undegraded":"2023-11-16T16:38:03.187970+0000","last_fullsized":"2023-11-16T16:38:03.187970+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:55:00.163322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.189679+0000","last_change":"2023-11-16T16:38:03.189679+0000","last_active":"2023-11-16T16:38:03.189679+0000","last_peered":"2023-11-16T16:38:03.189679+0000","last_clean":"2023-11-16T16:38:03.189679+0000","last_became_active":"2023-11-16T16:38:03.189292+0000","last_became_peered":"2023-11-16T16:38:03.189292+0000","last_unstale":"2023-11-16T16:38:03.189679+0000","last_undegraded":"2023-11-16T16:38:03.189679+0000","last_fullsized":"2023-11-16T16:38:03.189679+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:24:48.303402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184448+0000","last_change":"2023-11-16T16:37:13.949940+0000","last_active":"2023-11-16T16:38:03.184448+0000","last_peered":"2023-11-16T16:38:03.184448+0000","last_clean":"2023-11-16T16:38:03.184448+0000","last_became_active":"2023-11-16T16:37:13.949464+0000","last_became_peered":"2023-11-16T16:37:13.949464+0000","last_unstale":"2023-11-16T16:38:03.184448+0000","last_undegraded":"2023-11-16T16:38:03.184448+0000","last_fullsized":"2023-11-16T16:38:03.184448+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:05:15.768098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.187084+0000","last_change":"2023-11-16T16:37:13.940534+0000","last_active":"2023-11-16T16:38:03.187084+0000","last_peered":"2023-11-16T16:38:03.187084+0000","last_clean":"2023-11-16T16:38:03.187084+0000","last_became_active":"2023-11-16T16:37:13.940174+0000","last_became_peered":"2023-11-16T16:37:13.940174+0000","last_unstale":"2023-11-16T16:38:03.187084+0000","last_undegraded":"2023-11-16T16:38:03.187084+0000","last_fullsized":"2023-11-16T16:38:03.187084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:29:35.708635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.190148+0000","last_change":"2023-11-16T16:37:13.937519+0000","last_active":"2023-11-16T16:38:03.190148+0000","last_peered":"2023-11-16T16:38:03.190148+0000","last_clean":"2023-11-16T16:38:03.190148+0000","last_became_active":"2023-11-16T16:37:13.937175+0000","last_became_peered":"2023-11-16T16:37:13.937175+0000","last_unstale":"2023-11-16T16:38:03.190148+0000","last_undegraded":"2023-11-16T16:38:03.190148+0000","last_fullsized":"2023-11-16T16:38:03.190148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:25:23.849158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.182804+0000","last_change":"2023-11-16T16:37:13.951506+0000","last_active":"2023-11-16T16:38:03.182804+0000","last_peered":"2023-11-16T16:38:03.182804+0000","last_clean":"2023-11-16T16:38:03.182804+0000","last_became_active":"2023-11-16T16:37:13.951192+0000","last_became_peered":"2023-11-16T16:37:13.951192+0000","last_unstale":"2023-11-16T16:38:03.182804+0000","last_undegraded":"2023-11-16T16:38:03.182804+0000","last_fullsized":"2023-11-16T16:38:03.182804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:17:38.223843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184841+0000","last_change":"2023-11-16T16:37:13.935614+0000","last_active":"2023-11-16T16:38:03.184841+0000","last_peered":"2023-11-16T16:38:03.184841+0000","last_clean":"2023-11-16T16:38:03.184841+0000","last_became_active":"2023-11-16T16:37:13.935237+0000","last_became_peered":"2023-11-16T16:37:13.935237+0000","last_unstale":"2023-11-16T16:38:03.184841+0000","last_undegraded":"2023-11-16T16:38:03.184841+0000","last_fullsized":"2023-11-16T16:38:03.184841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:30:01.772328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.182369+0000","last_change":"2023-11-16T16:37:13.939387+0000","last_active":"2023-11-16T16:38:03.182369+0000","last_peered":"2023-11-16T16:38:03.182369+0000","last_clean":"2023-11-16T16:38:03.182369+0000","last_became_active":"2023-11-16T16:37:13.939082+0000","last_became_peered":"2023-11-16T16:37:13.939082+0000","last_unstale":"2023-11-16T16:38:03.182369+0000","last_undegraded":"2023-11-16T16:38:03.182369+0000","last_fullsized":"2023-11-16T16:38:03.182369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:28:19.026881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.188347+0000","last_change":"2023-11-16T16:37:13.936381+0000","last_active":"2023-11-16T16:38:03.188347+0000","last_peered":"2023-11-16T16:38:03.188347+0000","last_clean":"2023-11-16T16:38:03.188347+0000","last_became_active":"2023-11-16T16:37:13.936077+0000","last_became_peered":"2023-11-16T16:37:13.936077+0000","last_unstale":"2023-11-16T16:38:03.188347+0000","last_undegraded":"2023-11-16T16:38:03.188347+0000","last_fullsized":"2023-11-16T16:38:03.188347+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:20:31.015344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184590+0000","last_change":"2023-11-16T16:37:13.934005+0000","last_active":"2023-11-16T16:38:03.184590+0000","last_peered":"2023-11-16T16:38:03.184590+0000","last_clean":"2023-11-16T16:38:03.184590+0000","last_became_active":"2023-11-16T16:37:13.933633+0000","last_became_peered":"2023-11-16T16:37:13.933633+0000","last_unstale":"2023-11-16T16:38:03.184590+0000","last_undegraded":"2023-11-16T16:38:03.184590+0000","last_fullsized":"2023-11-16T16:38:03.184590+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:41:36.990242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.186787+0000","last_change":"2023-11-16T16:37:13.946495+0000","last_active":"2023-11-16T16:38:03.186787+0000","last_peered":"2023-11-16T16:38:03.186787+0000","last_clean":"2023-11-16T16:38:03.186787+0000","last_became_active":"2023-11-16T16:37:13.946148+0000","last_became_peered":"2023-11-16T16:37:13.946148+0000","last_unstale":"2023-11-16T16:38:03.186787+0000","last_undegraded":"2023-11-16T16:38:03.186787+0000","last_fullsized":"2023-11-16T16:38:03.186787+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:25:08.609723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":3542208,"data_compressed":211753,"data_compressed_allocated":270336,"data_compressed_original":3522560,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62431,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":364,"num_read_kb":247,"num_write":150,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":187293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132,"ondisk_log_size":132,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416408,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6040,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":93739112,"statfs":{"total":95995035648,"available":95988850688,"internally_reserved":0,"allocated":933888,"data_stored":447021,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.56000000000000005}]},{"osd":1,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.626},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.44700000000000001}]},{"osd":2,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.41099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":4,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.72199999999999998}]},{"osd":5,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.48099999999999998}]},{"osd":6,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":0.60799999999999998}]},{"osd":7,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.82799999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":9,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.749},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.53200000000000003}]},{"osd":10,"last update":"Thu Nov 16 16:37:32 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.863,"5min":1.863,"15min":1.863},"last":0.80000000000000004}]}]},{"osd":10,"up_from":72,"seq":309237645340,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6432,"kb_used_data":1048,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738720,"statfs":{"total":95995035648,"available":95988449280,"internally_reserved":0,"allocated":1073152,"data_stored":1096544,"data_compressed":41378,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:38:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.84299999999999997}]},{"osd":1,"last update":"Thu Nov 16 16:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.49199999999999999,"15min":0.49299999999999999},"min":{"1min":0.34399999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.67900000000000005,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.52800000000000002,"15min":0.52700000000000002},"min":{"1min":0.33300000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.86499999999999999,"5min":0.879,"15min":0.879},"last":0.44600000000000001}]},{"osd":2,"last update":"Thu Nov 16 16:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64300000000000002,"15min":0.64100000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89300000000000002,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.70899999999999996,"15min":0.71599999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.80400000000000005,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.58199999999999996}]},{"osd":3,"last update":"Thu Nov 16 16:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65900000000000003,"15min":0.66500000000000004},"min":{"1min":0.44,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82399999999999995,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.629},{"interface":"front","average":{"1min":0.625,"5min":0.67300000000000004,"15min":0.68100000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.85699999999999998,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.74299999999999999}]},{"osd":4,"last update":"Thu Nov 16 16:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63700000000000001,"15min":0.63500000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.94399999999999995,"5min":1.141,"15min":1.141},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.58299999999999996,"15min":0.57199999999999995},"min":{"1min":0.48199999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.91600000000000004,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.47599999999999998}]},{"osd":5,"last update":"Thu Nov 16 16:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.752,"15min":0.754},"min":{"1min":0.57699999999999996,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.95499999999999996,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.754,"15min":0.75900000000000001},"min":{"1min":0.58399999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.94099999999999995,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.80500000000000005}]},{"osd":6,"last update":"Thu Nov 16 16:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.76200000000000001,"15min":0.76800000000000002},"min":{"1min":0.498,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0029999999999999,"5min":1.282,"15min":1.282},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.77700000000000002,"15min":0.77600000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0149999999999999,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.61399999999999999}]},{"osd":7,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.505}]},{"osd":8,"last update":"Thu Nov 16 16:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.83699999999999997,"15min":0.83499999999999996},"min":{"1min":0.60199999999999998,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0760000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.61599999999999999,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0169999999999999,"5min":1.5,"15min":1.5},"last":0.77400000000000002}]},{"osd":9,"last update":"Thu Nov 16 16:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0289999999999999,"5min":1.337,"15min":1.337},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81799999999999995,"15min":0.81899999999999995},"min":{"1min":0.53000000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0369999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.88700000000000001}]},{"osd":11,"last update":"Thu Nov 16 16:37:33 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.82299999999999995}]}]},{"osd":9,"up_from":67,"seq":287762808863,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6176,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":93738976,"statfs":{"total":95995035648,"available":95988711424,"internally_reserved":0,"allocated":942080,"data_stored":447145,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.45100000000000001,"15min":0.46300000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.59699999999999998,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.47199999999999998,"15min":0.48199999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.41099999999999998}]},{"osd":1,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.59199999999999997,"15min":0.60599999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.61799999999999999,"15min":0.63300000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76100000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.47799999999999998}]},{"osd":2,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.63200000000000001,"15min":0.64600000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.78900000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.63500000000000001,"15min":0.64300000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.54800000000000004,"15min":0.56399999999999995},"min":{"1min":0.33200000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73299999999999998,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.55100000000000005,"15min":0.57299999999999995},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.70899999999999996,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.45100000000000001}]},{"osd":4,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.66200000000000003,"15min":0.68100000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.873,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.67300000000000004,"15min":0.68999999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.83599999999999997,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.55000000000000004}]},{"osd":5,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.69599999999999995,"15min":0.70999999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.878,"5min":1.056,"15min":1.056},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.72099999999999997,"15min":0.73899999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85999999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.69699999999999995}]},{"osd":6,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.64900000000000002,"15min":0.67200000000000004},"min":{"1min":0.38400000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.81699999999999995,"5min":1.069,"15min":1.069},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.70399999999999996,"15min":0.72799999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85099999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.64000000000000001}]},{"osd":7,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.755,"15min":0.77300000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92200000000000004,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.622},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.77100000000000002,"15min":0.79500000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.89600000000000002,"5min":1.111,"15min":1.111},"last":0.73299999999999998}]},{"osd":8,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.80400000000000005,"15min":0.82499999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.95899999999999996,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.80100000000000005,"15min":0.81599999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.97399999999999998,"5min":1.089,"15min":1.089},"last":0.67300000000000004}]},{"osd":10,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.70899999999999996}]}]},{"osd":8,"up_from":59,"seq":253403070501,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6360,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738792,"statfs":{"total":95995035648,"available":95988523008,"internally_reserved":0,"allocated":933888,"data_stored":446953,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.77000000000000002,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.44400000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.59399999999999997}]},{"osd":1,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.627,"15min":0.61699999999999999},"min":{"1min":0.46100000000000002,"5min":0.318,"15min":0.318},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.46200000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.621}]},{"osd":2,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.50600000000000001,"15min":0.51100000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71299999999999997,"5min":1.056,"15min":1.056},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.51500000000000001,"15min":0.52000000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71499999999999997,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.80200000000000005}]},{"osd":3,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.624,"15min":0.625},"min":{"1min":0.40999999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.75800000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.433,"5min":0.376,"15min":0.376},"max":{"1min":0.877,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.85699999999999998}]},{"osd":4,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.65800000000000003,"15min":0.63300000000000001},"min":{"1min":0.47099999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":4.0599999999999996,"5min":4.0599999999999996,"15min":4.0599999999999996},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69899999999999995,"15min":0.69799999999999995},"min":{"1min":0.42899999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":1.036}]},{"osd":5,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60999999999999999,"15min":0.61199999999999999},"min":{"1min":0.33500000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.89000000000000001,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.624,"15min":0.63},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.84599999999999997,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.76200000000000001}]},{"osd":6,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.747,"15min":0.745},"min":{"1min":0.45800000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94599999999999995,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.745,"15min":0.75},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94299999999999995,"5min":1.173,"15min":1.173},"last":0.96399999999999997}]},{"osd":7,"last update":"Thu Nov 16 16:37:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.76800000000000002,"15min":0.76400000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.0089999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.75700000000000001,"15min":0.753},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.88}]},{"osd":9,"last update":"Thu Nov 16 16:37:55 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.73799999999999999,"15min":0.72999999999999998},"min":{"1min":0.54000000000000004,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.018,"5min":1.036,"15min":1.036},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.875,"5min":0.76900000000000002,"15min":0.751},"min":{"1min":0.58699999999999997,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.94799999999999995}]},{"osd":10,"last update":"Thu Nov 16 16:37:13 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":3.5720000000000001,"5min":3.5720000000000001,"15min":3.5720000000000001},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":3.5920000000000001,"5min":3.5920000000000001,"15min":3.5920000000000001},"last":1.0820000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]}]},{"osd":7,"up_from":52,"seq":223338299433,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6480,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738672,"statfs":{"total":95995035648,"available":95988400128,"internally_reserved":0,"allocated":991232,"data_stored":1038904,"data_compressed":41378,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:36:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55800000000000005}]},{"osd":1,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.41799999999999998,"15min":0.41099999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.434,"15min":0.42599999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.47899999999999998}]},{"osd":2,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.35799999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.62}]},{"osd":3,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.57499999999999996,"15min":0.57099999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.625,"15min":0.621},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.64900000000000002}]},{"osd":4,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.52400000000000002,"15min":0.51400000000000001},"min":{"1min":0.34799999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.498,"15min":0.48199999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.60499999999999998}]},{"osd":5,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.65200000000000002,"15min":0.64700000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91300000000000003,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.63800000000000001,"15min":0.63200000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.71499999999999997}]},{"osd":6,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.70299999999999996,"15min":0.69499999999999995},"min":{"1min":0.52500000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.75,"5min":0.68600000000000005,"15min":0.67600000000000005},"min":{"1min":0.47299999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.78500000000000003}]},{"osd":8,"last update":"Thu Nov 16 16:37:25 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.748,"15min":0.747},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0580000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.752,"5min":0.76100000000000001,"15min":0.76300000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.754}]},{"osd":9,"last update":"Thu Nov 16 16:37:51 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.80300000000000005,"15min":0.78200000000000003},"min":{"1min":0.57999999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":4.7370000000000001,"5min":4.7370000000000001,"15min":4.7370000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.84399999999999997,"15min":0.82199999999999995},"min":{"1min":0.55600000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":4.7809999999999997,"5min":4.7809999999999997,"15min":4.7809999999999997},"last":0.90000000000000002}]},{"osd":10,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.81599999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]}]},{"osd":6,"up_from":47,"seq":201863462957,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6292,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738860,"statfs":{"total":95995035648,"available":95988592640,"internally_reserved":0,"allocated":929792,"data_stored":446540,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.439,"15min":0.41999999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":2.3140000000000001,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.47399999999999998,"15min":0.44600000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":3.472,"5min":3.472,"15min":3.472},"last":0.34999999999999998}]},{"osd":1,"last update":"Thu Nov 16 16:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.59699999999999998,"15min":0.58299999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":3.3290000000000002,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.64600000000000002,"15min":0.59999999999999998},"min":{"1min":0.38200000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":4.4669999999999996,"5min":4.4669999999999996,"15min":4.4669999999999996},"last":0.64500000000000002}]},{"osd":2,"last update":"Thu Nov 16 16:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.65400000000000003,"15min":0.63800000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":3.5369999999999999,"5min":3.5369999999999999,"15min":3.5369999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.66200000000000003,"15min":0.63400000000000001},"min":{"1min":0.46400000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.4049999999999998,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.621}]},{"osd":3,"last update":"Thu Nov 16 16:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.52400000000000002,"15min":0.48999999999999999},"min":{"1min":0.30499999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":3.3700000000000001,"5min":3.3700000000000001,"15min":3.3700000000000001},"last":0.375},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.55600000000000005,"15min":0.54100000000000004},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.39200000000000002}]},{"osd":4,"last update":"Thu Nov 16 16:37:50 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.67600000000000005,"15min":0.60999999999999999},"min":{"1min":0.45300000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":4.4900000000000002,"5min":4.4900000000000002,"15min":4.4900000000000002},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.026,"5min":0.69899999999999995,"15min":0.63100000000000001},"min":{"1min":0.45200000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":4.4509999999999996,"5min":4.4509999999999996,"15min":4.4509999999999996},"last":0.69399999999999995}]},{"osd":5,"last update":"Thu Nov 16 16:37:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.68899999999999995,"15min":0.68100000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":2.4209999999999998,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.72999999999999998,"15min":0.70999999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":3.5019999999999998,"5min":3.5019999999999998,"15min":3.5019999999999998},"last":0.59799999999999998}]},{"osd":7,"last update":"Thu Nov 16 16:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.70899999999999996,"15min":0.70499999999999996},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":2.452,"5min":2.452,"15min":2.452},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.73999999999999999,"15min":0.71799999999999997},"min":{"1min":0.54400000000000004,"5min":0.436,"15min":0.436},"max":{"1min":3.4449999999999998,"5min":3.4449999999999998,"15min":3.4449999999999998},"last":0.73599999999999999}]},{"osd":8,"last update":"Thu Nov 16 16:37:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.76100000000000001,"15min":0.77200000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.91200000000000003,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.77000000000000002,"15min":0.78000000000000003},"min":{"1min":0.54000000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.016,"5min":1.091,"15min":1.091},"last":0.66100000000000003}]},{"osd":9,"last update":"Thu Nov 16 16:37:50 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":0.81799999999999995,"15min":0.78200000000000003},"min":{"1min":0.50800000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":6.1539999999999999,"5min":6.1539999999999999,"15min":6.1539999999999999},"last":0.432},{"interface":"front","average":{"1min":1.1910000000000001,"5min":0.83999999999999997,"15min":0.78200000000000003},"min":{"1min":0.55100000000000005,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":8.2460000000000004,"5min":8.2460000000000004,"15min":8.2460000000000004},"last":0.50900000000000001}]},{"osd":10,"last update":"Thu Nov 16 16:37:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.77400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7376,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":93737776,"statfs":{"total":95995035648,"available":95987482624,"internally_reserved":0,"allocated":991232,"data_stored":504854,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:38:06 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.48499999999999999,"15min":0.45800000000000002},"min":{"1min":0.34899999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.67000000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52100000000000002,"15min":0.48799999999999999},"min":{"1min":0.311,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.83499999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.57899999999999996}]},{"osd":2,"last update":"Thu Nov 16 16:37:19 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.54300000000000004,"15min":0.498},"min":{"1min":0.36699999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.84799999999999998,"5min":0.876,"15min":0.876},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.54000000000000004,"15min":0.503},"min":{"1min":0.40500000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.91000000000000003,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.70799999999999996}]},{"osd":3,"last update":"Thu Nov 16 16:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57699999999999996,"15min":0.55500000000000005},"min":{"1min":0.372,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85599999999999998,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59899999999999998,"15min":0.57499999999999996},"min":{"1min":0.41399999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.63400000000000001}]},{"osd":4,"last update":"Thu Nov 16 16:38:06 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.46300000000000002,"15min":0.43099999999999999},"min":{"1min":0.33700000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.79500000000000004,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55000000000000004,"15min":0.48499999999999999},"min":{"1min":0.35499999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.82899999999999996,"5min":1.242,"15min":1.242},"last":0.78700000000000003}]},{"osd":5,"last update":"Thu Nov 16 16:37:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.63800000000000001,"15min":0.61099999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.621,"15min":0.59699999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97399999999999998,"5min":1.004,"15min":1.004},"last":0.82799999999999996}]},{"osd":6,"last update":"Thu Nov 16 16:37:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.68400000000000005,"15min":0.66800000000000004},"min":{"1min":0.44700000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.95099999999999996,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68400000000000005,"15min":0.66400000000000003},"min":{"1min":0.45500000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.95299999999999996,"5min":1,"15min":1},"last":0.76000000000000001}]},{"osd":7,"last update":"Thu Nov 16 16:38:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65300000000000002,"15min":0.63500000000000001},"min":{"1min":0.39500000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.0569999999999999,"5min":1.077,"15min":1.077},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.67900000000000005},"min":{"1min":0.42599999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.018,"5min":1.143,"15min":1.143},"last":0.84899999999999998}]},{"osd":8,"last update":"Thu Nov 16 16:37:27 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.75900000000000001,"15min":0.755},"min":{"1min":0.54800000000000004,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.74099999999999999,"15min":0.73699999999999999},"min":{"1min":0.47399999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.97099999999999997}]},{"osd":9,"last update":"Thu Nov 16 16:37:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.77900000000000003,"15min":0.79900000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.87,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.84599999999999997,"15min":0.86399999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.97299999999999998,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.95299999999999996}]},{"osd":10,"last update":"Thu Nov 16 16:37:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.873},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.88400000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]}]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7208,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":6272,"kb_avail":93737944,"statfs":{"total":95995035648,"available":95987654656,"internally_reserved":0,"allocated":950272,"data_stored":447805,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":6422528},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Nov 16 16:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.54700000000000004,"15min":0.48199999999999998},"min":{"1min":0.38400000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":3.6869999999999998,"5min":3.6869999999999998,"15min":3.6869999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.57999999999999996,"15min":0.48799999999999999},"min":{"1min":0.40600000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":6.6059999999999999,"5min":6.6059999999999999,"15min":6.6059999999999999},"last":0.57099999999999995}]},{"osd":2,"last update":"Thu Nov 16 16:37:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.55600000000000005,"15min":0.51400000000000001},"min":{"1min":0.41999999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.80600000000000005,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56299999999999994,"15min":0.53100000000000003},"min":{"1min":0.372,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.46200000000000002}]},{"osd":3,"last update":"Thu Nov 16 16:35:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.46500000000000002,"15min":0.45600000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.45700000000000002,"15min":0.44700000000000001},"min":{"1min":0.33700000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.35199999999999998}]},{"osd":4,"last update":"Thu Nov 16 16:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.61499999999999999,"15min":0.56299999999999994},"min":{"1min":0.44,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":3.8109999999999999,"5min":3.8109999999999999,"15min":3.8109999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.871,"5min":0.64900000000000002,"15min":0.57199999999999995},"min":{"1min":0.42099999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":6.577,"5min":6.577,"15min":6.577},"last":0.49299999999999999}]},{"osd":5,"last update":"Thu Nov 16 16:37:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.56200000000000006,"15min":0.55100000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.66100000000000003,"5min":1.012,"15min":1.012},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64600000000000002,"15min":0.621},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.82699999999999996,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.53200000000000003}]},{"osd":6,"last update":"Thu Nov 16 16:37:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.64800000000000002,"15min":0.59699999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":6.5190000000000001,"5min":6.5190000000000001,"15min":6.5190000000000001},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.63100000000000001,"15min":0.57499999999999996},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":9.141,"5min":9.141,"15min":9.141},"last":0.40600000000000003}]},{"osd":7,"last update":"Thu Nov 16 16:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.68400000000000005,"15min":0.67600000000000005},"min":{"1min":0.495,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.441},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.74199999999999999,"15min":0.73199999999999998},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.59299999999999997}]},{"osd":8,"last update":"Thu Nov 16 16:37:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.78800000000000003,"15min":0.80900000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.91900000000000004,"5min":1.163,"15min":1.163},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.83499999999999996,"15min":0.85299999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.90800000000000003,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.60699999999999998}]},{"osd":9,"last update":"Thu Nov 16 16:37:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.85499999999999998,"15min":0.86199999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":6.6349999999999998,"5min":6.6349999999999998,"15min":6.6349999999999998},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.79000000000000004,"15min":0.76600000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":9.1690000000000005,"5min":9.1690000000000005,"15min":9.1690000000000005},"last":0.52000000000000002}]},{"osd":10,"last update":"Thu Nov 16 16:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.67200000000000004}]},{"osd":11,"last update":"Thu Nov 16 16:37:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":9.1039999999999992,"5min":9.1039999999999992,"15min":9.1039999999999992},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":3.7530000000000001,"5min":3.7530000000000001,"15min":3.7530000000000001},"last":0.626}]}]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7464,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":93737688,"statfs":{"total":95995035648,"available":95987392512,"internally_reserved":0,"allocated":1015808,"data_stored":506546,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.29999999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.372},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.497,"15min":0.48799999999999999},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.70899999999999996}]},{"osd":1,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52600000000000002,"15min":0.51400000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.40500000000000003,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.89900000000000002,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.77800000000000002}]},{"osd":3,"last update":"Thu Nov 16 16:37:42 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.66300000000000003,"15min":0.60699999999999998},"min":{"1min":0.40699999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":6.9740000000000002,"5min":6.9740000000000002,"15min":6.9740000000000002},"last":0.623},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.57499999999999996,"15min":0.55700000000000005},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.503}]},{"osd":4,"last update":"Thu Nov 16 16:38:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.75}]},{"osd":5,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.49399999999999999,"15min":0.48399999999999999},"min":{"1min":0.315,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.432},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.47199999999999998,"15min":0.45900000000000002},"min":{"1min":0.32200000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.39300000000000002}]},{"osd":6,"last update":"Thu Nov 16 16:37:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.68000000000000005,"15min":0.65400000000000003},"min":{"1min":0.40899999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.3969999999999998,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.64400000000000002,"15min":0.61299999999999999},"min":{"1min":0.39400000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":2.3460000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.45400000000000001}]},{"osd":7,"last update":"Thu Nov 16 16:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.64300000000000002,"15min":0.64400000000000002},"min":{"1min":0.41599999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70199999999999996,"15min":0.70399999999999996},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.92700000000000005,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.57699999999999996}]},{"osd":8,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.57499999999999996,"15min":0.56899999999999995},"min":{"1min":0.38500000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.85199999999999998,"5min":1.008,"15min":1.008},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.628,"15min":0.625},"min":{"1min":0.38400000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.47499999999999998}]},{"osd":9,"last update":"Thu Nov 16 16:37:54 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.86799999999999999,"15min":0.86699999999999999},"min":{"1min":0.47499999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.6080000000000001,"5min":4.524,"15min":4.524},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.751,"15min":0.73299999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.6200000000000001,"5min":2.6200000000000001,"15min":2.6200000000000001},"last":0.76300000000000001}]},{"osd":10,"last update":"Thu Nov 16 16:37:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.68500000000000005}]},{"osd":11,"last update":"Thu Nov 16 16:38:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.56299999999999994}]}]},{"osd":3,"up_from":26,"seq":111669149753,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6624,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738528,"statfs":{"total":95995035648,"available":95988252672,"internally_reserved":0,"allocated":942080,"data_stored":446526,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36499999999999999,"15min":0.34899999999999998},"min":{"1min":0.26500000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.52400000000000002,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.38,"15min":0.35799999999999998},"min":{"1min":0.27800000000000002,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.55100000000000005,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.54800000000000004}]},{"osd":1,"last update":"Thu Nov 16 16:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.55400000000000005,"15min":0.54900000000000004},"min":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55900000000000005,"15min":0.55700000000000005},"min":{"1min":0.36099999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.83399999999999996,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.53200000000000003}]},{"osd":2,"last update":"Thu Nov 16 16:37:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.57899999999999996,"15min":0.55700000000000005},"min":{"1min":0.42599999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.83299999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56699999999999995,"15min":0.54700000000000004},"min":{"1min":0.40600000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.69699999999999995,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.65600000000000003}]},{"osd":4,"last update":"Thu Nov 16 16:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56299999999999994,"15min":0.54900000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79900000000000004,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60599999999999998,"15min":0.59599999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.62}]},{"osd":5,"last update":"Thu Nov 16 16:37:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.59799999999999998,"15min":0.57999999999999996},"min":{"1min":0.438,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60499999999999998,"15min":0.59099999999999997},"min":{"1min":0.38600000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.87,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.77700000000000002}]},{"osd":6,"last update":"Thu Nov 16 16:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51800000000000002,"15min":0.498},"min":{"1min":0.35599999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.72399999999999998,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53200000000000003,"15min":0.51400000000000001},"min":{"1min":0.377,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.75800000000000001,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.67400000000000004}]},{"osd":7,"last update":"Thu Nov 16 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.65600000000000003,"15min":0.65000000000000002},"min":{"1min":0.44,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.69299999999999995,"15min":0.68700000000000006},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.72799999999999998}]},{"osd":8,"last update":"Thu Nov 16 16:36:27 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.63200000000000001}]},{"osd":9,"last update":"Thu Nov 16 16:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.73799999999999999,"15min":0.76300000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80500000000000005,"5min":1.024,"15min":1.024},"last":0.748},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.68999999999999995,"15min":0.70899999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90300000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.64400000000000002}]},{"osd":10,"last update":"Thu Nov 16 16:37:12 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.84399999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]}]},{"osd":4,"up_from":34,"seq":146028888118,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6424,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738728,"statfs":{"total":95995035648,"available":95988457472,"internally_reserved":0,"allocated":933888,"data_stored":446923,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.52400000000000002,"15min":0.502},"min":{"1min":0.38200000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.52900000000000003,"15min":0.51500000000000001},"min":{"1min":0.40899999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.378}]},{"osd":1,"last update":"Thu Nov 16 16:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.41099999999999998,"15min":0.39300000000000002},"min":{"1min":0.33700000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.67900000000000005,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.434,"15min":0.40999999999999998},"min":{"1min":0.378,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.748,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.38700000000000001}]},{"osd":2,"last update":"Thu Nov 16 16:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.77300000000000002,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.623,"15min":0.59299999999999997},"min":{"1min":0.53000000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.81999999999999995,"5min":1.056,"15min":1.056},"last":0.73599999999999999}]},{"osd":3,"last update":"Thu Nov 16 16:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.59199999999999997,"15min":0.55000000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57299999999999995,"15min":0.55300000000000005},"min":{"1min":0.40100000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.65400000000000003}]},{"osd":5,"last update":"Thu Nov 16 16:37:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.623,"15min":0.627},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.92100000000000004,"5min":1.075,"15min":1.075},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.63800000000000001,"15min":0.626},"min":{"1min":0.52200000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.94499999999999995,"5min":1.093,"15min":1.093},"last":0.78900000000000003}]},{"osd":6,"last update":"Thu Nov 16 16:37:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.63,"15min":0.60199999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84799999999999998,"5min":1.004,"15min":1.004},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.60899999999999999,"15min":0.59499999999999997},"min":{"1min":0.42399999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.57199999999999995}]},{"osd":7,"last update":"Thu Nov 16 16:38:13 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.61499999999999999,"15min":0.56299999999999994},"min":{"1min":0.45400000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.99399999999999999,"5min":1.123,"15min":1.123},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.67300000000000004,"15min":0.621},"min":{"1min":0.51600000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.99199999999999999,"5min":1.069,"15min":1.069},"last":0.627}]},{"osd":8,"last update":"Thu Nov 16 16:37:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.73999999999999999,"15min":0.72399999999999998},"min":{"1min":0.58999999999999997,"5min":0.495,"15min":0.495},"max":{"1min":1.0189999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.747,"15min":0.73099999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.83499999999999996}]},{"osd":9,"last update":"Thu Nov 16 16:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.79200000000000004,"15min":0.78900000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.94099999999999995,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.72099999999999997,"15min":0.71799999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94199999999999995,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.70899999999999996}]},{"osd":10,"last update":"Thu Nov 16 16:37:15 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.76300000000000001}]},{"osd":11,"last update":"Thu Nov 16 16:38:13 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.871},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.85999999999999999}]}]},{"osd":5,"up_from":39,"seq":167503724595,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6556,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738596,"statfs":{"total":95995035648,"available":95988322304,"internally_reserved":0,"allocated":1003520,"data_stored":1038636,"data_compressed":41378,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.499,"15min":0.49399999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.64600000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52800000000000002,"15min":0.52400000000000002},"min":{"1min":0.312,"5min":0.311,"15min":0.311},"max":{"1min":0.75700000000000001,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.754}]},{"osd":1,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.372,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.86399999999999999,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58799999999999997,"15min":0.58499999999999996},"min":{"1min":0.43099999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.94699999999999995,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.77600000000000002}]},{"osd":2,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.42799999999999999,"15min":0.42399999999999999},"min":{"1min":0.29599999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.67300000000000004,"5min":1.417,"15min":1.417},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.443,"15min":0.433},"min":{"1min":0.32300000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.69899999999999995,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.68200000000000005}]},{"osd":3,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55700000000000005,"15min":0.54700000000000004},"min":{"1min":0.36199999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.81100000000000005,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.86899999999999999}]},{"osd":4,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.67900000000000005,"15min":0.629},"min":{"1min":0.40100000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":5.5789999999999997,"5min":5.5789999999999997,"15min":5.5789999999999997},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59699999999999998,"15min":0.59099999999999997},"min":{"1min":0.40400000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.84499999999999997}]},{"osd":6,"last update":"Thu Nov 16 16:37:47 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.67800000000000005,"15min":0.63600000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":3.274,"5min":3.274,"15min":3.274},"last":0.996},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.68100000000000005,"15min":0.64600000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":3.472,"5min":3.472,"15min":3.472},"last":1.0109999999999999}]},{"osd":7,"last update":"Thu Nov 16 16:38:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.72199999999999998,"15min":0.71599999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0449999999999999,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.73799999999999999,"15min":0.73199999999999998},"min":{"1min":0.48099999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0580000000000001,"5min":1.544,"15min":1.544},"last":0.98199999999999998}]},{"osd":8,"last update":"Thu Nov 16 16:37:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.64600000000000002,"15min":0.64500000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72199999999999998,"15min":0.72399999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":1.091}]},{"osd":9,"last update":"Thu Nov 16 16:37:52 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":0.90100000000000002,"15min":0.82699999999999996},"min":{"1min":0.58699999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":5.5679999999999996,"5min":5.5679999999999996,"15min":5.5679999999999996},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.099,"5min":0.81599999999999995,"15min":0.76900000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":5.3230000000000004,"5min":5.3230000000000004,"15min":5.3230000000000004},"last":1.0640000000000001}]},{"osd":10,"last update":"Thu Nov 16 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":1.1299999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.95399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33928,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":58527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-16T16:38:17.968 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-11-16T16:38:17.969 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-16T16:38:17.969 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-11-16T16:38:17.969 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph health --format=json 2023-11-16T16:38:18.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:18 smithi001 ceph-mon[98800]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:38:18.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:18 smithi001 ceph-mon[98800]: pgmap v294: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2023-11-16T16:38:18.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:18 smithi026 ceph-mon[100112]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:38:18.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:18 smithi026 ceph-mon[100112]: pgmap v294: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2023-11-16T16:38:18.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:18 smithi053 ceph-mon[100638]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:38:18.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:18 smithi053 ceph-mon[100638]: pgmap v294: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2023-11-16T16:38:18.677 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:38:19.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:19 smithi001 ceph-mon[98800]: pgmap v295: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2023-11-16T16:38:19.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:19 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:38:19.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:19 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:38:19.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:19 smithi053 ceph-mon[100638]: pgmap v295: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2023-11-16T16:38:19.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:19 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:38:19.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:19 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:38:20.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:19 smithi026 ceph-mon[100112]: pgmap v295: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 9 B/s, 0 keys/s, 0 objects/s recovering 2023-11-16T16:38:20.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:19 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:38:20.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:19 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:38:20.070 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.067+0000 7f8df8aea700 1 -- 172.21.15.1:0/1999173497 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8df4101540 msgr2=0x7f8df4101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:20.070 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.067+0000 7f8df8aea700 1 --2- 172.21.15.1:0/1999173497 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8df4101540 0x7f8df4101910 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8de4009990 tx=0x7f8de4009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:20.070 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.068+0000 7f8df8aea700 1 -- 172.21.15.1:0/1999173497 shutdown_connections 2023-11-16T16:38:20.070 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.068+0000 7f8df8aea700 1 --2- 172.21.15.1:0/1999173497 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8df410a890 0x7f8df410ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:20.071 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.068+0000 7f8df8aea700 1 --2- 172.21.15.1:0/1999173497 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8df4101540 0x7f8df4101910 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:20.071 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.068+0000 7f8df8aea700 1 --2- 172.21.15.1:0/1999173497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8df4101e50 0x7f8df410a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:20.071 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.068+0000 7f8df8aea700 1 -- 172.21.15.1:0/1999173497 >> 172.21.15.1:0/1999173497 conn(0x7f8df40770a0 msgr2=0x7f8df40774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:20.071 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.068+0000 7f8df8aea700 1 -- 172.21.15.1:0/1999173497 shutdown_connections 2023-11-16T16:38:20.071 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.068+0000 7f8df8aea700 1 -- 172.21.15.1:0/1999173497 wait complete. 2023-11-16T16:38:20.073 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.068+0000 7f8df8aea700 1 Processor -- start 2023-11-16T16:38:20.073 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.068+0000 7f8df8aea700 1 -- start start 2023-11-16T16:38:20.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.069+0000 7f8df8aea700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8df4101540 0x7f8df40ff1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:20.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.069+0000 7f8df8aea700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8df4101e50 0x7f8df40ff6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:20.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.069+0000 7f8df8aea700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8df410a890 0x7f8df40ffc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:20.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.069+0000 7f8df8aea700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df41a4880 con 0x7f8df4101540 2023-11-16T16:38:20.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.069+0000 7f8df8aea700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df41a49f0 con 0x7f8df410a890 2023-11-16T16:38:20.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.069+0000 7f8df8aea700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df41a4bb0 con 0x7f8df4101e50 2023-11-16T16:38:20.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.069+0000 7f8df259c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8df4101540 0x7f8df40ff1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:20.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.069+0000 7f8df259c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8df4101540 0x7f8df40ff1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42140/0 (socket says 172.21.15.1:42140) 2023-11-16T16:38:20.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.069+0000 7f8df259c700 1 -- 172.21.15.1:0/3893915389 learned_addr learned my addr 172.21.15.1:0/3893915389 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:20.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.069+0000 7f8df1d9b700 1 --2- 172.21.15.1:0/3893915389 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8df4101e50 0x7f8df40ff6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:20.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.069+0000 7f8df2d9d700 1 --2- 172.21.15.1:0/3893915389 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8df410a890 0x7f8df40ffc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:20.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.069+0000 7f8df259c700 1 -- 172.21.15.1:0/3893915389 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8df4101e50 msgr2=0x7f8df40ff6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:20.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.069+0000 7f8df259c700 1 --2- 172.21.15.1:0/3893915389 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8df4101e50 0x7f8df40ff6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:20.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.069+0000 7f8df259c700 1 -- 172.21.15.1:0/3893915389 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8df410a890 msgr2=0x7f8df40ffc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:20.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.069+0000 7f8df259c700 1 --2- 172.21.15.1:0/3893915389 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8df410a890 0x7f8df40ffc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:20.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.069+0000 7f8df259c700 1 -- 172.21.15.1:0/3893915389 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de40095f0 con 0x7f8df4101540 2023-11-16T16:38:20.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.069+0000 7f8df2d9d700 1 --2- 172.21.15.1:0/3893915389 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8df410a890 0x7f8df40ffc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:38:20.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.069+0000 7f8df259c700 1 --2- 172.21.15.1:0/3893915389 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8df4101540 0x7f8df40ff1a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8de4000c00 tx=0x7f8de4004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:20.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.070+0000 7f8de37fe700 1 -- 172.21.15.1:0/3893915389 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de401d070 con 0x7f8df4101540 2023-11-16T16:38:20.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.070+0000 7f8de37fe700 1 -- 172.21.15.1:0/3893915389 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8de4021410 con 0x7f8df4101540 2023-11-16T16:38:20.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.070+0000 7f8de37fe700 1 -- 172.21.15.1:0/3893915389 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de4017650 con 0x7f8df4101540 2023-11-16T16:38:20.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.070+0000 7f8df8aea700 1 -- 172.21.15.1:0/3893915389 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df4110310 con 0x7f8df4101540 2023-11-16T16:38:20.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.070+0000 7f8df8aea700 1 -- 172.21.15.1:0/3893915389 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df4110180 con 0x7f8df4101540 2023-11-16T16:38:20.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.071+0000 7f8df8aea700 1 -- 172.21.15.1:0/3893915389 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8df41104a0 con 0x7f8df4101540 2023-11-16T16:38:20.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.073+0000 7f8de37fe700 1 -- 172.21.15.1:0/3893915389 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f8de4021580 con 0x7f8df4101540 2023-11-16T16:38:20.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.073+0000 7f8de37fe700 1 --2- 172.21.15.1:0/3893915389 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8ddc0a11e0 0x7f8ddc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:20.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.073+0000 7f8df1d9b700 1 --2- 172.21.15.1:0/3893915389 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8ddc0a11e0 0x7f8ddc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:20.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.073+0000 7f8de37fe700 1 -- 172.21.15.1:0/3893915389 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8de40ce170 con 0x7f8df4101540 2023-11-16T16:38:20.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.074+0000 7f8df1d9b700 1 --2- 172.21.15.1:0/3893915389 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8ddc0a11e0 0x7f8ddc0a3690 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8df4100990 tx=0x7f8de8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:20.080 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.076+0000 7f8de37fe700 1 -- 172.21.15.1:0/3893915389 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8de40977f0 con 0x7f8df4101540 2023-11-16T16:38:20.332 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.330+0000 7f8df8aea700 1 -- 172.21.15.1:0/3893915389 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f8df41006d0 con 0x7f8df4101540 2023-11-16T16:38:20.332 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.330+0000 7f8de37fe700 1 -- 172.21.15.1:0/3893915389 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f8de409ae10 con 0x7f8df4101540 2023-11-16T16:38:20.332 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:38:20.332 INFO:teuthology.orchestra.run.smithi001.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-16T16:38:20.334 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.332+0000 7f8df8aea700 1 -- 172.21.15.1:0/3893915389 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8ddc0a11e0 msgr2=0x7f8ddc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:20.334 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.332+0000 7f8df8aea700 1 --2- 172.21.15.1:0/3893915389 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8ddc0a11e0 0x7f8ddc0a3690 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8df4100990 tx=0x7f8de8009260 comp rx=0 tx=0).stop 2023-11-16T16:38:20.335 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.332+0000 7f8df8aea700 1 -- 172.21.15.1:0/3893915389 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8df4101540 msgr2=0x7f8df40ff1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:20.335 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.332+0000 7f8df8aea700 1 --2- 172.21.15.1:0/3893915389 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8df4101540 0x7f8df40ff1a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8de4000c00 tx=0x7f8de4004790 comp rx=0 tx=0).stop 2023-11-16T16:38:20.335 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.333+0000 7f8df8aea700 1 -- 172.21.15.1:0/3893915389 shutdown_connections 2023-11-16T16:38:20.335 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.333+0000 7f8df8aea700 1 --2- 172.21.15.1:0/3893915389 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8ddc0a11e0 0x7f8ddc0a3690 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:20.335 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.333+0000 7f8df8aea700 1 --2- 172.21.15.1:0/3893915389 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8df410a890 0x7f8df40ffc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:20.335 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.333+0000 7f8df8aea700 1 --2- 172.21.15.1:0/3893915389 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8df4101540 0x7f8df40ff1a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:20.335 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.333+0000 7f8df8aea700 1 --2- 172.21.15.1:0/3893915389 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8df4101e50 0x7f8df40ff6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:20.335 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.333+0000 7f8df8aea700 1 -- 172.21.15.1:0/3893915389 >> 172.21.15.1:0/3893915389 conn(0x7f8df40770a0 msgr2=0x7f8df40fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:20.335 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.333+0000 7f8df8aea700 1 -- 172.21.15.1:0/3893915389 shutdown_connections 2023-11-16T16:38:20.335 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.333+0000 7f8df8aea700 1 -- 172.21.15.1:0/3893915389 wait complete. 2023-11-16T16:38:20.840 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-11-16T16:38:20.840 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-16T16:38:20.866 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:20 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:38:20.866 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:20 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:38:20.866 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:20 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:38:20.866 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:20 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:38:20.866 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:20 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3893915389' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-11-16T16:38:20.974 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.970+0000 7f967f140700 1 -- 172.21.15.1:0/19287666 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96781018e0 msgr2=0x7f9678101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:20.974 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.970+0000 7f967f140700 1 --2- 172.21.15.1:0/19287666 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96781018e0 0x7f9678101cf0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f967400b0a0 tx=0x7f967400b3b0 comp rx=0 tx=0).stop 2023-11-16T16:38:20.974 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.970+0000 7f967f140700 1 -- 172.21.15.1:0/19287666 shutdown_connections 2023-11-16T16:38:20.974 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.970+0000 7f967f140700 1 --2- 172.21.15.1:0/19287666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96780fece0 0x7f96781088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:20.974 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.970+0000 7f967f140700 1 --2- 172.21.15.1:0/19287666 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96781018e0 0x7f9678101cf0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:20.974 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.970+0000 7f967f140700 1 --2- 172.21.15.1:0/19287666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96780fe350 0x7f96780fe7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:20.975 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.970+0000 7f967f140700 1 -- 172.21.15.1:0/19287666 >> 172.21.15.1:0/19287666 conn(0x7f96780f9f80 msgr2=0x7f96780fc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:20.975 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.970+0000 7f967f140700 1 -- 172.21.15.1:0/19287666 shutdown_connections 2023-11-16T16:38:20.975 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.970+0000 7f967f140700 1 -- 172.21.15.1:0/19287666 wait complete. 2023-11-16T16:38:20.975 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.970+0000 7f967f140700 1 Processor -- start 2023-11-16T16:38:20.975 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.971+0000 7f967f140700 1 -- start start 2023-11-16T16:38:20.975 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.971+0000 7f967f140700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96780fe350 0x7f96781961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:20.975 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.971+0000 7f967f140700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96780fece0 0x7f9678196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:20.975 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.971+0000 7f967f140700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96781018e0 0x7f967819d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:20.975 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.971+0000 7f967f140700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967819dd70 con 0x7f96781018e0 2023-11-16T16:38:20.975 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.971+0000 7f967f140700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967819deb0 con 0x7f96780fe350 2023-11-16T16:38:20.975 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.971+0000 7f967f140700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967819e040 con 0x7f96780fece0 2023-11-16T16:38:20.975 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.971+0000 7f967d6dd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96781018e0 0x7f967819d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:20.975 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.971+0000 7f967d6dd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96781018e0 0x7f967819d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42156/0 (socket says 172.21.15.1:42156) 2023-11-16T16:38:20.976 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.971+0000 7f967d6dd700 1 -- 172.21.15.1:0/1178299063 learned_addr learned my addr 172.21.15.1:0/1178299063 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:20.976 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.971+0000 7f967cedc700 1 --2- 172.21.15.1:0/1178299063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96780fe350 0x7f96781961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:20.976 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.971+0000 7f966ffff700 1 --2- 172.21.15.1:0/1178299063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96780fece0 0x7f9678196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:20.976 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.971+0000 7f967d6dd700 1 -- 172.21.15.1:0/1178299063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96780fece0 msgr2=0x7f9678196730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:20.976 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.971+0000 7f967d6dd700 1 --2- 172.21.15.1:0/1178299063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96780fece0 0x7f9678196730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:20.976 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.971+0000 7f967d6dd700 1 -- 172.21.15.1:0/1178299063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96780fe350 msgr2=0x7f96781961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:20.976 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.971+0000 7f967d6dd700 1 --2- 172.21.15.1:0/1178299063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96780fe350 0x7f96781961f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:20.976 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.971+0000 7f967d6dd700 1 -- 172.21.15.1:0/1178299063 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9674009d00 con 0x7f96781018e0 2023-11-16T16:38:20.976 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.971+0000 7f966ffff700 1 --2- 172.21.15.1:0/1178299063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96780fece0 0x7f9678196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:38:20.976 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.971+0000 7f967d6dd700 1 --2- 172.21.15.1:0/1178299063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96781018e0 0x7f967819d7a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f967000d980 tx=0x7f967000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:20.976 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.971+0000 7f966dffb700 1 -- 172.21.15.1:0/1178299063 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9670004220 con 0x7f96781018e0 2023-11-16T16:38:20.976 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.972+0000 7f966dffb700 1 -- 172.21.15.1:0/1178299063 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96700075e0 con 0x7f96781018e0 2023-11-16T16:38:20.976 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.972+0000 7f967f140700 1 -- 172.21.15.1:0/1178299063 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f967810bcc0 con 0x7f96781018e0 2023-11-16T16:38:20.977 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.972+0000 7f966dffb700 1 -- 172.21.15.1:0/1178299063 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9670005650 con 0x7f96781018e0 2023-11-16T16:38:20.977 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.972+0000 7f967f140700 1 -- 172.21.15.1:0/1178299063 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f967810bb30 con 0x7f96781018e0 2023-11-16T16:38:20.977 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.973+0000 7f967f140700 1 -- 172.21.15.1:0/1178299063 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96781073b0 con 0x7f96781018e0 2023-11-16T16:38:20.977 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.974+0000 7f966dffb700 1 -- 172.21.15.1:0/1178299063 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f9670004750 con 0x7f96781018e0 2023-11-16T16:38:20.978 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.974+0000 7f966dffb700 1 --2- 172.21.15.1:0/1178299063 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f96600a1260 0x7f96600a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:20.978 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.975+0000 7f966dffb700 1 -- 172.21.15.1:0/1178299063 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f96700cc8a0 con 0x7f96781018e0 2023-11-16T16:38:20.978 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.975+0000 7f967cedc700 1 --2- 172.21.15.1:0/1178299063 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f96600a1260 0x7f96600a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:20.978 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.975+0000 7f967cedc700 1 --2- 172.21.15.1:0/1178299063 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f96600a1260 0x7f96600a3710 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9674009fd0 tx=0x7f967400b7f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:20.980 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:20.979+0000 7f966dffb700 1 -- 172.21.15.1:0/1178299063 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9670095e50 con 0x7f96781018e0 2023-11-16T16:38:21.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:20 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:38:21.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:20 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:38:21.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:20 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:38:21.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:20 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:38:21.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:20 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/3893915389' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-11-16T16:38:21.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:20 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:38:21.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:20 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:38:21.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:20 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:38:21.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:20 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:38:21.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:20 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/3893915389' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-11-16T16:38:21.185 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.184+0000 7f967f140700 1 -- 172.21.15.1:0/1178299063 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f967802cfe0 con 0x7f96781018e0 2023-11-16T16:38:21.189 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.187+0000 7f966dffb700 1 -- 172.21.15.1:0/1178299063 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22469 (secure 0 0 0) 0x7f9670016020 con 0x7f96781018e0 2023-11-16T16:38:21.190 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:38:21.190 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":86,"fsid":"35d9b010-849d-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:30:06.217778+0000","modified":"2023-11-16T16:38:03.179244+0000","last_up_change":"2023-11-16T16:36:31.735403+0000","last_in_change":"2023-11-16T16:36:14.083021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:33:15.916723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-16T16:37:12.158031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-16T16:37:13.272343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a8480507-5275-4e52-953b-626110beb436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6803","nonce":2469671609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6805","nonce":2469671609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6809","nonce":2469671609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6807","nonce":2469671609}]},"public_addr":"172.21.15.1:6803/2469671609","cluster_addr":"172.21.15.1:6805/2469671609","heartbeat_back_addr":"172.21.15.1:6809/2469671609","heartbeat_front_addr":"172.21.15.1:6807/2469671609","state":["exists","up"]},{"osd":1,"uuid":"1d011db2-99d1-4575-bdc5-51f88a308fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6801","nonce":247994796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6803","nonce":247994796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6807","nonce":247994796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6805","nonce":247994796}]},"public_addr":"172.21.15.26:6801/247994796","cluster_addr":"172.21.15.26:6803/247994796","heartbeat_back_addr":"172.21.15.26:6807/247994796","heartbeat_front_addr":"172.21.15.26:6805/247994796","state":["exists","up"]},{"osd":2,"uuid":"4d7a2bd1-3c8e-4546-a243-f0354473b031","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6801","nonce":1659316290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6803","nonce":1659316290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6807","nonce":1659316290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6805","nonce":1659316290}]},"public_addr":"172.21.15.53:6801/1659316290","cluster_addr":"172.21.15.53:6803/1659316290","heartbeat_back_addr":"172.21.15.53:6807/1659316290","heartbeat_front_addr":"172.21.15.53:6805/1659316290","state":["exists","up"]},{"osd":3,"uuid":"9c290fd9-e121-4a81-9687-b0eb93debd86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6811","nonce":2728987281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6813","nonce":2728987281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6817","nonce":2728987281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6815","nonce":2728987281}]},"public_addr":"172.21.15.1:6811/2728987281","cluster_addr":"172.21.15.1:6813/2728987281","heartbeat_back_addr":"172.21.15.1:6817/2728987281","heartbeat_front_addr":"172.21.15.1:6815/2728987281","state":["exists","up"]},{"osd":4,"uuid":"533226bc-cc57-4f15-9e29-820294d57e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6809","nonce":2954740024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6811","nonce":2954740024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6815","nonce":2954740024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6813","nonce":2954740024}]},"public_addr":"172.21.15.26:6809/2954740024","cluster_addr":"172.21.15.26:6811/2954740024","heartbeat_back_addr":"172.21.15.26:6815/2954740024","heartbeat_front_addr":"172.21.15.26:6813/2954740024","state":["exists","up"]},{"osd":5,"uuid":"d3ab25eb-e1d4-4557-bc89-efc9072401ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6809","nonce":3966666318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6811","nonce":3966666318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6815","nonce":3966666318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6813","nonce":3966666318}]},"public_addr":"172.21.15.53:6809/3966666318","cluster_addr":"172.21.15.53:6811/3966666318","heartbeat_back_addr":"172.21.15.53:6815/3966666318","heartbeat_front_addr":"172.21.15.53:6813/3966666318","state":["exists","up"]},{"osd":6,"uuid":"180f506e-afb8-4043-b0b4-2dc86f6d4bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6819","nonce":3364032646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6821","nonce":3364032646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6825","nonce":3364032646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6823","nonce":3364032646}]},"public_addr":"172.21.15.1:6819/3364032646","cluster_addr":"172.21.15.1:6821/3364032646","heartbeat_back_addr":"172.21.15.1:6825/3364032646","heartbeat_front_addr":"172.21.15.1:6823/3364032646","state":["exists","up"]},{"osd":7,"uuid":"0d837bac-194f-498d-b7fb-b3147e992329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6817","nonce":3766901391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6819","nonce":3766901391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6823","nonce":3766901391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6821","nonce":3766901391}]},"public_addr":"172.21.15.26:6817/3766901391","cluster_addr":"172.21.15.26:6819/3766901391","heartbeat_back_addr":"172.21.15.26:6823/3766901391","heartbeat_front_addr":"172.21.15.26:6821/3766901391","state":["exists","up"]},{"osd":8,"uuid":"933f231a-7454-47b0-9291-f1fa64f0baec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6817","nonce":3224335871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6819","nonce":3224335871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6823","nonce":3224335871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6821","nonce":3224335871}]},"public_addr":"172.21.15.53:6817/3224335871","cluster_addr":"172.21.15.53:6819/3224335871","heartbeat_back_addr":"172.21.15.53:6823/3224335871","heartbeat_front_addr":"172.21.15.53:6821/3224335871","state":["exists","up"]},{"osd":9,"uuid":"245fcbb3-c651-4703-ab80-309dc98bd983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6827","nonce":1391199844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6829","nonce":1391199844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6833","nonce":1391199844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6831","nonce":1391199844}]},"public_addr":"172.21.15.1:6827/1391199844","cluster_addr":"172.21.15.1:6829/1391199844","heartbeat_back_addr":"172.21.15.1:6833/1391199844","heartbeat_front_addr":"172.21.15.1:6831/1391199844","state":["exists","up"]},{"osd":10,"uuid":"2c090e7a-5240-4d52-ae4f-e94fe05383d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6825","nonce":1773545447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6827","nonce":1773545447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6831","nonce":1773545447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6829","nonce":1773545447}]},"public_addr":"172.21.15.26:6825/1773545447","cluster_addr":"172.21.15.26:6827/1773545447","heartbeat_back_addr":"172.21.15.26:6831/1773545447","heartbeat_front_addr":"172.21.15.26:6829/1773545447","state":["exists","up"]},{"osd":11,"uuid":"4511e32d-e469-45e1-8310-0ff400cb7b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6825","nonce":2564669546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6827","nonce":2564669546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6831","nonce":2564669546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6829","nonce":2564669546}]},"public_addr":"172.21.15.53:6825/2564669546","cluster_addr":"172.21.15.53:6827/2564669546","heartbeat_back_addr":"172.21.15.53:6831/2564669546","heartbeat_front_addr":"172.21.15.53:6829/2564669546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:28.140250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:49.885461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:12.521487+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:34.926197+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:56.815594+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:18.387751+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:40.587106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:02.319370+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:23.832298+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:46.338040+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:08.730414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:30.050070+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1494336948":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6801/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6800/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6800/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/1641976669":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/2895923638":"2023-11-17T16:30:35.064289+0000","172.21.15.1:6801/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/78811711":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/4131957708":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/344403698":"2023-11-17T16:30:35.064289+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:38:21.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.190+0000 7f967f140700 1 -- 172.21.15.1:0/1178299063 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f96600a1260 msgr2=0x7f96600a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:21.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.190+0000 7f967f140700 1 --2- 172.21.15.1:0/1178299063 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f96600a1260 0x7f96600a3710 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9674009fd0 tx=0x7f967400b7f0 comp rx=0 tx=0).stop 2023-11-16T16:38:21.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.190+0000 7f967f140700 1 -- 172.21.15.1:0/1178299063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96781018e0 msgr2=0x7f967819d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:21.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.190+0000 7f967f140700 1 --2- 172.21.15.1:0/1178299063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96781018e0 0x7f967819d7a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f967000d980 tx=0x7f967000dc90 comp rx=0 tx=0).stop 2023-11-16T16:38:21.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.190+0000 7f967f140700 1 -- 172.21.15.1:0/1178299063 shutdown_connections 2023-11-16T16:38:21.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.190+0000 7f967f140700 1 --2- 172.21.15.1:0/1178299063 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f96600a1260 0x7f96600a3710 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:21.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.190+0000 7f967f140700 1 --2- 172.21.15.1:0/1178299063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96781018e0 0x7f967819d7a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:21.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.190+0000 7f967f140700 1 --2- 172.21.15.1:0/1178299063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96780fe350 0x7f96781961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:21.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.190+0000 7f967f140700 1 --2- 172.21.15.1:0/1178299063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f96780fece0 0x7f9678196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:21.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.190+0000 7f967f140700 1 -- 172.21.15.1:0/1178299063 >> 172.21.15.1:0/1178299063 conn(0x7f96780f9f80 msgr2=0x7f96780fc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:21.193 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.190+0000 7f967f140700 1 -- 172.21.15.1:0/1178299063 shutdown_connections 2023-11-16T16:38:21.193 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.190+0000 7f967f140700 1 -- 172.21.15.1:0/1178299063 wait complete. 2023-11-16T16:38:21.205 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-16T16:33:15.916723+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-16T16:37:12.158031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-16T16:37:13.272343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-16T16:38:21.205 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-16T16:38:21.376 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.372+0000 7f54ef3f6700 1 -- 172.21.15.1:0/2125236177 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54e8100db0 msgr2=0x7f54e8107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:21.376 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.372+0000 7f54ef3f6700 1 --2- 172.21.15.1:0/2125236177 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54e8100db0 0x7f54e8107650 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f54e000b0a0 tx=0x7f54e000b3b0 comp rx=0 tx=0).stop 2023-11-16T16:38:21.376 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.372+0000 7f54ef3f6700 1 -- 172.21.15.1:0/2125236177 shutdown_connections 2023-11-16T16:38:21.377 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.372+0000 7f54ef3f6700 1 --2- 172.21.15.1:0/2125236177 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54e8100db0 0x7f54e8107650 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:21.377 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.372+0000 7f54ef3f6700 1 --2- 172.21.15.1:0/2125236177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54e80ff170 0x7f54e80ff580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:21.377 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.372+0000 7f54ef3f6700 1 --2- 172.21.15.1:0/2125236177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54e8100400 0x7f54e8100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:21.377 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.372+0000 7f54ef3f6700 1 -- 172.21.15.1:0/2125236177 >> 172.21.15.1:0/2125236177 conn(0x7f54e80fa750 msgr2=0x7f54e80fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:21.377 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.373+0000 7f54ef3f6700 1 -- 172.21.15.1:0/2125236177 shutdown_connections 2023-11-16T16:38:21.377 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.373+0000 7f54ef3f6700 1 -- 172.21.15.1:0/2125236177 wait complete. 2023-11-16T16:38:21.377 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.373+0000 7f54ef3f6700 1 Processor -- start 2023-11-16T16:38:21.377 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.373+0000 7f54ef3f6700 1 -- start start 2023-11-16T16:38:21.377 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.373+0000 7f54ef3f6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54e80ff170 0x7f54e8194ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:21.377 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.373+0000 7f54ef3f6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54e8100400 0x7f54e8195430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:21.377 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.373+0000 7f54ef3f6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54e8100db0 0x7f54e819c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:21.377 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.373+0000 7f54ef3f6700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e819ca70 con 0x7f54e80ff170 2023-11-16T16:38:21.378 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.373+0000 7f54ef3f6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e819cbb0 con 0x7f54e8100db0 2023-11-16T16:38:21.378 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.373+0000 7f54ef3f6700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e819cd40 con 0x7f54e8100400 2023-11-16T16:38:21.378 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.373+0000 7f54ed192700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54e80ff170 0x7f54e8194ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:21.378 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.373+0000 7f54ed192700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54e80ff170 0x7f54e8194ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42166/0 (socket says 172.21.15.1:42166) 2023-11-16T16:38:21.378 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.373+0000 7f54ed192700 1 -- 172.21.15.1:0/1469203424 learned_addr learned my addr 172.21.15.1:0/1469203424 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:21.378 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.374+0000 7f54ec991700 1 --2- 172.21.15.1:0/1469203424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54e8100400 0x7f54e8195430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:21.378 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.374+0000 7f54ed993700 1 --2- 172.21.15.1:0/1469203424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54e8100db0 0x7f54e819c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:21.378 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.374+0000 7f54ed192700 1 -- 172.21.15.1:0/1469203424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54e8100400 msgr2=0x7f54e8195430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:21.378 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.374+0000 7f54ed192700 1 --2- 172.21.15.1:0/1469203424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54e8100400 0x7f54e8195430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:21.378 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.374+0000 7f54ed192700 1 -- 172.21.15.1:0/1469203424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54e8100db0 msgr2=0x7f54e819c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:21.378 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.374+0000 7f54ed192700 1 --2- 172.21.15.1:0/1469203424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54e8100db0 0x7f54e819c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:21.378 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.374+0000 7f54ed192700 1 -- 172.21.15.1:0/1469203424 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e0009d00 con 0x7f54e80ff170 2023-11-16T16:38:21.379 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.374+0000 7f54ed192700 1 --2- 172.21.15.1:0/1469203424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54e80ff170 0x7f54e8194ef0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f54e4007b50 tx=0x7f54e4007f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:21.379 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.374+0000 7f54da7fc700 1 -- 172.21.15.1:0/1469203424 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e4011070 con 0x7f54e80ff170 2023-11-16T16:38:21.379 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.374+0000 7f54da7fc700 1 -- 172.21.15.1:0/1469203424 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54e400c2e0 con 0x7f54e80ff170 2023-11-16T16:38:21.379 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.374+0000 7f54da7fc700 1 -- 172.21.15.1:0/1469203424 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e400e5c0 con 0x7f54e80ff170 2023-11-16T16:38:21.379 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.374+0000 7f54ef3f6700 1 -- 172.21.15.1:0/1469203424 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54e810ab70 con 0x7f54e80ff170 2023-11-16T16:38:21.379 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.374+0000 7f54ef3f6700 1 -- 172.21.15.1:0/1469203424 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54e810a9e0 con 0x7f54e80ff170 2023-11-16T16:38:21.380 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.375+0000 7f54ef3f6700 1 -- 172.21.15.1:0/1469203424 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54e8068a00 con 0x7f54e80ff170 2023-11-16T16:38:21.380 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.376+0000 7f54da7fc700 1 -- 172.21.15.1:0/1469203424 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f54e4004750 con 0x7f54e80ff170 2023-11-16T16:38:21.380 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.377+0000 7f54da7fc700 1 --2- 172.21.15.1:0/1469203424 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f54d40a1260 0x7f54d40a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:21.380 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.377+0000 7f54ec991700 1 --2- 172.21.15.1:0/1469203424 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f54d40a1260 0x7f54d40a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:21.380 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.377+0000 7f54da7fc700 1 -- 172.21.15.1:0/1469203424 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f54e40ce3a0 con 0x7f54e80ff170 2023-11-16T16:38:21.380 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.377+0000 7f54ec991700 1 --2- 172.21.15.1:0/1469203424 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f54d40a1260 0x7f54d40a3710 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f54dc005ea0 tx=0x7f54dc005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:21.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.380+0000 7f54da7fc700 1 -- 172.21.15.1:0/1469203424 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f54e40979e0 con 0x7f54e80ff170 2023-11-16T16:38:21.581 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.579+0000 7f54ef3f6700 1 -- 172.21.15.1:0/1469203424 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f54e8068d20 con 0x7f54e80ff170 2023-11-16T16:38:21.581 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.579+0000 7f54da7fc700 1 -- 172.21.15.1:0/1469203424 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f54e409b000 con 0x7f54e80ff170 2023-11-16T16:38:21.581 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2023-11-16T16:38:21.583 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.581+0000 7f54ef3f6700 1 -- 172.21.15.1:0/1469203424 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f54d40a1260 msgr2=0x7f54d40a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:21.583 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.581+0000 7f54ef3f6700 1 --2- 172.21.15.1:0/1469203424 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f54d40a1260 0x7f54d40a3710 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f54dc005ea0 tx=0x7f54dc005e30 comp rx=0 tx=0).stop 2023-11-16T16:38:21.583 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.581+0000 7f54ef3f6700 1 -- 172.21.15.1:0/1469203424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54e80ff170 msgr2=0x7f54e8194ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:21.583 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.581+0000 7f54ef3f6700 1 --2- 172.21.15.1:0/1469203424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54e80ff170 0x7f54e8194ef0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f54e4007b50 tx=0x7f54e4007f10 comp rx=0 tx=0).stop 2023-11-16T16:38:21.584 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.581+0000 7f54ef3f6700 1 -- 172.21.15.1:0/1469203424 shutdown_connections 2023-11-16T16:38:21.584 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.581+0000 7f54ef3f6700 1 --2- 172.21.15.1:0/1469203424 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f54d40a1260 0x7f54d40a3710 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:21.584 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.582+0000 7f54ef3f6700 1 --2- 172.21.15.1:0/1469203424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54e8100db0 0x7f54e819c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:21.584 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.582+0000 7f54ef3f6700 1 --2- 172.21.15.1:0/1469203424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54e80ff170 0x7f54e8194ef0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:21.584 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.582+0000 7f54ef3f6700 1 --2- 172.21.15.1:0/1469203424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f54e8100400 0x7f54e8195430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:21.584 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.582+0000 7f54ef3f6700 1 -- 172.21.15.1:0/1469203424 >> 172.21.15.1:0/1469203424 conn(0x7f54e80fa750 msgr2=0x7f54e80fc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:21.584 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.582+0000 7f54ef3f6700 1 -- 172.21.15.1:0/1469203424 shutdown_connections 2023-11-16T16:38:21.584 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.582+0000 7f54ef3f6700 1 -- 172.21.15.1:0/1469203424 wait complete. 2023-11-16T16:38:21.596 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-16T16:38:21.726 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.724+0000 7fbc845d9700 1 -- 172.21.15.1:0/3043665606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc7c100db0 msgr2=0x7fbc7c107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:21.726 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.724+0000 7fbc845d9700 1 --2- 172.21.15.1:0/3043665606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc7c100db0 0x7fbc7c107650 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbc7400b0a0 tx=0x7fbc7400b3b0 comp rx=0 tx=0).stop 2023-11-16T16:38:21.726 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.724+0000 7fbc845d9700 1 -- 172.21.15.1:0/3043665606 shutdown_connections 2023-11-16T16:38:21.727 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.724+0000 7fbc845d9700 1 --2- 172.21.15.1:0/3043665606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc7c100db0 0x7fbc7c107650 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:21.727 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.724+0000 7fbc845d9700 1 --2- 172.21.15.1:0/3043665606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc7c0ff170 0x7fbc7c0ff580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:21.727 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.724+0000 7fbc845d9700 1 --2- 172.21.15.1:0/3043665606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc7c100400 0x7fbc7c100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:21.727 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.724+0000 7fbc845d9700 1 -- 172.21.15.1:0/3043665606 >> 172.21.15.1:0/3043665606 conn(0x7fbc7c0fa750 msgr2=0x7fbc7c0fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:21.727 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.724+0000 7fbc845d9700 1 -- 172.21.15.1:0/3043665606 shutdown_connections 2023-11-16T16:38:21.727 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.724+0000 7fbc845d9700 1 -- 172.21.15.1:0/3043665606 wait complete. 2023-11-16T16:38:21.729 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.725+0000 7fbc845d9700 1 Processor -- start 2023-11-16T16:38:21.729 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.725+0000 7fbc845d9700 1 -- start start 2023-11-16T16:38:21.729 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.725+0000 7fbc845d9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc7c0ff170 0x7fbc7c194ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:21.730 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.725+0000 7fbc845d9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc7c100400 0x7fbc7c195430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:21.730 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.725+0000 7fbc845d9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc7c100db0 0x7fbc7c19c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:21.730 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.725+0000 7fbc845d9700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7c19ca70 con 0x7fbc7c100400 2023-11-16T16:38:21.730 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.725+0000 7fbc845d9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7c19cbb0 con 0x7fbc7c0ff170 2023-11-16T16:38:21.730 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.725+0000 7fbc845d9700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7c19cd40 con 0x7fbc7c100db0 2023-11-16T16:38:21.730 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.726+0000 7fbc81b74700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc7c100400 0x7fbc7c195430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:21.730 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.726+0000 7fbc81b74700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc7c100400 0x7fbc7c195430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42182/0 (socket says 172.21.15.1:42182) 2023-11-16T16:38:21.730 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.726+0000 7fbc81b74700 1 -- 172.21.15.1:0/3454775261 learned_addr learned my addr 172.21.15.1:0/3454775261 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:21.730 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.726+0000 7fbc82375700 1 --2- 172.21.15.1:0/3454775261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc7c0ff170 0x7fbc7c194ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:21.730 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.726+0000 7fbc82b76700 1 --2- 172.21.15.1:0/3454775261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc7c100db0 0x7fbc7c19c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:21.730 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.726+0000 7fbc81b74700 1 -- 172.21.15.1:0/3454775261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc7c100db0 msgr2=0x7fbc7c19c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:21.730 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.726+0000 7fbc81b74700 1 --2- 172.21.15.1:0/3454775261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc7c100db0 0x7fbc7c19c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:21.731 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.726+0000 7fbc81b74700 1 -- 172.21.15.1:0/3454775261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc7c0ff170 msgr2=0x7fbc7c194ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:21.731 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.726+0000 7fbc81b74700 1 --2- 172.21.15.1:0/3454775261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc7c0ff170 0x7fbc7c194ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:21.731 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.726+0000 7fbc81b74700 1 -- 172.21.15.1:0/3454775261 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc74009d00 con 0x7fbc7c100400 2023-11-16T16:38:21.731 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.726+0000 7fbc82375700 1 --2- 172.21.15.1:0/3454775261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc7c0ff170 0x7fbc7c194ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:38:21.731 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.726+0000 7fbc82b76700 1 --2- 172.21.15.1:0/3454775261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc7c100db0 0x7fbc7c19c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:38:21.731 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.726+0000 7fbc81b74700 1 --2- 172.21.15.1:0/3454775261 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc7c100400 0x7fbc7c195430 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbc7000d950 tx=0x7fbc7000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:21.731 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.726+0000 7fbc6f7fe700 1 -- 172.21.15.1:0/3454775261 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc70009b40 con 0x7fbc7c100400 2023-11-16T16:38:21.731 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.726+0000 7fbc6f7fe700 1 -- 172.21.15.1:0/3454775261 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc70009ca0 con 0x7fbc7c100400 2023-11-16T16:38:21.731 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.726+0000 7fbc845d9700 1 -- 172.21.15.1:0/3454775261 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc7c10ab10 con 0x7fbc7c100400 2023-11-16T16:38:21.731 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.726+0000 7fbc6f7fe700 1 -- 172.21.15.1:0/3454775261 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7000f620 con 0x7fbc7c100400 2023-11-16T16:38:21.731 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.726+0000 7fbc845d9700 1 -- 172.21.15.1:0/3454775261 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc7c10a980 con 0x7fbc7c100400 2023-11-16T16:38:21.731 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.727+0000 7fbc845d9700 1 -- 172.21.15.1:0/3454775261 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc7c1060c0 con 0x7fbc7c100400 2023-11-16T16:38:21.732 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.728+0000 7fbc6f7fe700 1 -- 172.21.15.1:0/3454775261 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fbc70010460 con 0x7fbc7c100400 2023-11-16T16:38:21.732 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.729+0000 7fbc6f7fe700 1 --2- 172.21.15.1:0/3454775261 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbc680a11e0 0x7fbc680a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:21.732 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.729+0000 7fbc6f7fe700 1 -- 172.21.15.1:0/3454775261 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbc700ccdf0 con 0x7fbc7c100400 2023-11-16T16:38:21.732 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.729+0000 7fbc82375700 1 --2- 172.21.15.1:0/3454775261 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbc680a11e0 0x7fbc680a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:21.732 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.729+0000 7fbc82375700 1 --2- 172.21.15.1:0/3454775261 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbc680a11e0 0x7fbc680a3690 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbc78005fd0 tx=0x7fbc78005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:21.735 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.733+0000 7fbc6f7fe700 1 -- 172.21.15.1:0/3454775261 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbc700963e0 con 0x7fbc7c100400 2023-11-16T16:38:21.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:21 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:38:21.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:21 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:38:21.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:21 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:38:21.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:21 smithi001 ceph-mon[98800]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:38:21.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:21 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:38:21.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:21 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:38:21.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:21 smithi001 ceph-mon[98800]: Service mds.all has no related daemon already placed. 2023-11-16T16:38:21.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:21 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1178299063' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:38:21.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:21 smithi001 ceph-mon[98800]: pgmap v296: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-16T16:38:21.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:21 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1469203424' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:38:21.936 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.935+0000 7fbc845d9700 1 -- 172.21.15.1:0/3454775261 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbc7c0619b0 con 0x7fbc7c100400 2023-11-16T16:38:21.937 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.935+0000 7fbc6f7fe700 1 -- 172.21.15.1:0/3454775261 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbc70020020 con 0x7fbc7c100400 2023-11-16T16:38:21.937 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2023-11-16T16:38:21.939 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.937+0000 7fbc845d9700 1 -- 172.21.15.1:0/3454775261 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbc680a11e0 msgr2=0x7fbc680a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:21.939 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.937+0000 7fbc845d9700 1 --2- 172.21.15.1:0/3454775261 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbc680a11e0 0x7fbc680a3690 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbc78005fd0 tx=0x7fbc78005e30 comp rx=0 tx=0).stop 2023-11-16T16:38:21.939 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.937+0000 7fbc845d9700 1 -- 172.21.15.1:0/3454775261 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc7c100400 msgr2=0x7fbc7c195430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:21.939 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.937+0000 7fbc845d9700 1 --2- 172.21.15.1:0/3454775261 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc7c100400 0x7fbc7c195430 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbc7000d950 tx=0x7fbc7000dd10 comp rx=0 tx=0).stop 2023-11-16T16:38:21.940 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.937+0000 7fbc845d9700 1 -- 172.21.15.1:0/3454775261 shutdown_connections 2023-11-16T16:38:21.940 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.937+0000 7fbc845d9700 1 --2- 172.21.15.1:0/3454775261 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbc680a11e0 0x7fbc680a3690 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:21.940 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.937+0000 7fbc845d9700 1 --2- 172.21.15.1:0/3454775261 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc7c100db0 0x7fbc7c19c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:21.940 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.937+0000 7fbc845d9700 1 --2- 172.21.15.1:0/3454775261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc7c0ff170 0x7fbc7c194ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:21.940 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.937+0000 7fbc845d9700 1 --2- 172.21.15.1:0/3454775261 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc7c100400 0x7fbc7c195430 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:21.940 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.937+0000 7fbc845d9700 1 -- 172.21.15.1:0/3454775261 >> 172.21.15.1:0/3454775261 conn(0x7fbc7c0fa750 msgr2=0x7fbc7c0fc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:21.940 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.937+0000 7fbc845d9700 1 -- 172.21.15.1:0/3454775261 shutdown_connections 2023-11-16T16:38:21.940 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:21.937+0000 7fbc845d9700 1 -- 172.21.15.1:0/3454775261 wait complete. 2023-11-16T16:38:21.951 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-16T16:38:22.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:21 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:38:22.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:21 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:38:22.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:21 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:38:22.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:21 smithi026 ceph-mon[100112]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:38:22.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:21 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:38:22.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:21 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:38:22.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:21 smithi026 ceph-mon[100112]: Service mds.all has no related daemon already placed. 2023-11-16T16:38:22.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:21 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1178299063' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:38:22.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:21 smithi026 ceph-mon[100112]: pgmap v296: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-16T16:38:22.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:21 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1469203424' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:38:22.080 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.078+0000 7fdcc8bd5700 1 -- 172.21.15.1:0/4269557266 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdcc40fe900 msgr2=0x7fdcc40fed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:22.081 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.078+0000 7fdcc8bd5700 1 --2- 172.21.15.1:0/4269557266 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdcc40fe900 0x7fdcc40fed10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4009990 tx=0x7fdcb4009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:22.081 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.078+0000 7fdcc8bd5700 1 -- 172.21.15.1:0/4269557266 shutdown_connections 2023-11-16T16:38:22.081 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.078+0000 7fdcc8bd5700 1 --2- 172.21.15.1:0/4269557266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcc4100540 0x7fdcc4106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:22.081 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.078+0000 7fdcc8bd5700 1 --2- 172.21.15.1:0/4269557266 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdcc40fe900 0x7fdcc40fed10 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:22.081 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.078+0000 7fdcc8bd5700 1 --2- 172.21.15.1:0/4269557266 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdcc40ffb90 0x7fdcc4100000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:22.081 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.078+0000 7fdcc8bd5700 1 -- 172.21.15.1:0/4269557266 >> 172.21.15.1:0/4269557266 conn(0x7fdcc40f9f00 msgr2=0x7fdcc40fc350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:22.081 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.078+0000 7fdcc8bd5700 1 -- 172.21.15.1:0/4269557266 shutdown_connections 2023-11-16T16:38:22.081 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.078+0000 7fdcc8bd5700 1 -- 172.21.15.1:0/4269557266 wait complete. 2023-11-16T16:38:22.082 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:21 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:38:22.082 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:21 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:38:22.082 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:21 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:38:22.082 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:21 smithi053 ceph-mon[100638]: from='mgr.14152 172.21.15.1:0/1400803141' entity='mgr.x' 2023-11-16T16:38:22.082 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:21 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:38:22.082 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:21 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:38:22.082 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:21 smithi053 ceph-mon[100638]: Service mds.all has no related daemon already placed. 2023-11-16T16:38:22.083 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:21 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1178299063' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:38:22.083 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:21 smithi053 ceph-mon[100638]: pgmap v296: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-16T16:38:22.083 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:21 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1469203424' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:38:22.085 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.079+0000 7fdcc8bd5700 1 Processor -- start 2023-11-16T16:38:22.085 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.079+0000 7fdcc8bd5700 1 -- start start 2023-11-16T16:38:22.085 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.080+0000 7fdcc8bd5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcc40fe900 0x7fdcc4194eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:22.085 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.080+0000 7fdcc8bd5700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdcc40ffb90 0x7fdcc41953f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:22.085 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.080+0000 7fdcc8bd5700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdcc4100540 0x7fdcc419c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:22.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.080+0000 7fdcc8bd5700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc419ca30 con 0x7fdcc40ffb90 2023-11-16T16:38:22.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.080+0000 7fdcc8bd5700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc419cb70 con 0x7fdcc40fe900 2023-11-16T16:38:22.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.080+0000 7fdcc8bd5700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc419cd00 con 0x7fdcc4100540 2023-11-16T16:38:22.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.080+0000 7fdcc1d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdcc40ffb90 0x7fdcc41953f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:22.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.080+0000 7fdcc1d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdcc40ffb90 0x7fdcc41953f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42196/0 (socket says 172.21.15.1:42196) 2023-11-16T16:38:22.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.080+0000 7fdcc1d9b700 1 -- 172.21.15.1:0/3400870854 learned_addr learned my addr 172.21.15.1:0/3400870854 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:22.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.080+0000 7fdcc2d9d700 1 --2- 172.21.15.1:0/3400870854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdcc4100540 0x7fdcc419c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:22.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.080+0000 7fdcc1d9b700 1 -- 172.21.15.1:0/3400870854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdcc4100540 msgr2=0x7fdcc419c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:22.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.080+0000 7fdcc1d9b700 1 --2- 172.21.15.1:0/3400870854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdcc4100540 0x7fdcc419c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:22.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.080+0000 7fdcc1d9b700 1 -- 172.21.15.1:0/3400870854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcc40fe900 msgr2=0x7fdcc4194eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:22.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.080+0000 7fdcc259c700 1 --2- 172.21.15.1:0/3400870854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcc40fe900 0x7fdcc4194eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:22.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.080+0000 7fdcc1d9b700 1 --2- 172.21.15.1:0/3400870854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcc40fe900 0x7fdcc4194eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:22.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.080+0000 7fdcc1d9b700 1 -- 172.21.15.1:0/3400870854 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb40095f0 con 0x7fdcc40ffb90 2023-11-16T16:38:22.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.080+0000 7fdcc2d9d700 1 --2- 172.21.15.1:0/3400870854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdcc4100540 0x7fdcc419c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:38:22.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.080+0000 7fdcc259c700 1 --2- 172.21.15.1:0/3400870854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcc40fe900 0x7fdcc4194eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:38:22.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.080+0000 7fdcc1d9b700 1 --2- 172.21.15.1:0/3400870854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdcc40ffb90 0x7fdcc41953f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800c920 tx=0x7fdcb800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:22.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.080+0000 7fdcb37fe700 1 -- 172.21.15.1:0/3400870854 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8007b00 con 0x7fdcc40ffb90 2023-11-16T16:38:22.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.080+0000 7fdcb37fe700 1 -- 172.21.15.1:0/3400870854 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdcb8004cf0 con 0x7fdcc40ffb90 2023-11-16T16:38:22.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.080+0000 7fdcc8bd5700 1 -- 172.21.15.1:0/3400870854 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcc410ab20 con 0x7fdcc40ffb90 2023-11-16T16:38:22.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.081+0000 7fdcb37fe700 1 -- 172.21.15.1:0/3400870854 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8005610 con 0x7fdcc40ffb90 2023-11-16T16:38:22.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.081+0000 7fdcc8bd5700 1 -- 172.21.15.1:0/3400870854 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc410a990 con 0x7fdcc40ffb90 2023-11-16T16:38:22.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.082+0000 7fdcc8bd5700 1 -- 172.21.15.1:0/3400870854 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcc40671c0 con 0x7fdcc40ffb90 2023-11-16T16:38:22.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.083+0000 7fdcb37fe700 1 -- 172.21.15.1:0/3400870854 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fdcb8004750 con 0x7fdcc40ffb90 2023-11-16T16:38:22.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.083+0000 7fdcb37fe700 1 --2- 172.21.15.1:0/3400870854 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fdcac0a11e0 0x7fdcac0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:22.088 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.083+0000 7fdcb37fe700 1 -- 172.21.15.1:0/3400870854 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdcb80cca20 con 0x7fdcc40ffb90 2023-11-16T16:38:22.088 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.083+0000 7fdcc259c700 1 --2- 172.21.15.1:0/3400870854 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fdcac0a11e0 0x7fdcac0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:22.088 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.084+0000 7fdcc259c700 1 --2- 172.21.15.1:0/3400870854 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fdcac0a11e0 0x7fdcac0a3690 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400b5c0 tx=0x7fdcb4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:22.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.087+0000 7fdcb37fe700 1 -- 172.21.15.1:0/3400870854 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fdcb8096010 con 0x7fdcc40ffb90 2023-11-16T16:38:22.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.287+0000 7fdcc8bd5700 1 -- 172.21.15.1:0/3400870854 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdcc419d7b0 con 0x7fdcc40ffb90 2023-11-16T16:38:22.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.287+0000 7fdcb37fe700 1 -- 172.21.15.1:0/3400870854 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdcc419d7b0 con 0x7fdcc40ffb90 2023-11-16T16:38:22.289 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2023-11-16T16:38:22.291 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.289+0000 7fdcc8bd5700 1 -- 172.21.15.1:0/3400870854 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fdcac0a11e0 msgr2=0x7fdcac0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:22.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.289+0000 7fdcc8bd5700 1 --2- 172.21.15.1:0/3400870854 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fdcac0a11e0 0x7fdcac0a3690 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400b5c0 tx=0x7fdcb4005fb0 comp rx=0 tx=0).stop 2023-11-16T16:38:22.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.289+0000 7fdcc8bd5700 1 -- 172.21.15.1:0/3400870854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdcc40ffb90 msgr2=0x7fdcc41953f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:22.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.289+0000 7fdcc8bd5700 1 --2- 172.21.15.1:0/3400870854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdcc40ffb90 0x7fdcc41953f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800c920 tx=0x7fdcb800cce0 comp rx=0 tx=0).stop 2023-11-16T16:38:22.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.289+0000 7fdcc8bd5700 1 -- 172.21.15.1:0/3400870854 shutdown_connections 2023-11-16T16:38:22.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.289+0000 7fdcc8bd5700 1 --2- 172.21.15.1:0/3400870854 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fdcac0a11e0 0x7fdcac0a3690 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:22.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.289+0000 7fdcc8bd5700 1 --2- 172.21.15.1:0/3400870854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdcc4100540 0x7fdcc419c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:22.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.289+0000 7fdcc8bd5700 1 --2- 172.21.15.1:0/3400870854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcc40fe900 0x7fdcc4194eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:22.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.289+0000 7fdcc8bd5700 1 --2- 172.21.15.1:0/3400870854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdcc40ffb90 0x7fdcc41953f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:22.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.289+0000 7fdcc8bd5700 1 -- 172.21.15.1:0/3400870854 >> 172.21.15.1:0/3400870854 conn(0x7fdcc40f9f00 msgr2=0x7fdcc40fc270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:22.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.290+0000 7fdcc8bd5700 1 -- 172.21.15.1:0/3400870854 shutdown_connections 2023-11-16T16:38:22.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.290+0000 7fdcc8bd5700 1 -- 172.21.15.1:0/3400870854 wait complete. 2023-11-16T16:38:22.304 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:38:22.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.428+0000 7fd4c8e70700 1 -- 172.21.15.1:0/3575783419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4c40ff200 msgr2=0x7fd4c40ff610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:22.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.428+0000 7fd4c8e70700 1 --2- 172.21.15.1:0/3575783419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4c40ff200 0x7fd4c40ff610 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd4b4009990 tx=0x7fd4b4009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:22.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.428+0000 7fd4c8e70700 1 -- 172.21.15.1:0/3575783419 shutdown_connections 2023-11-16T16:38:22.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.428+0000 7fd4c8e70700 1 --2- 172.21.15.1:0/3575783419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4c4100d80 0x7fd4c4107640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:22.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.428+0000 7fd4c8e70700 1 --2- 172.21.15.1:0/3575783419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4c40ff200 0x7fd4c40ff610 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:22.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.428+0000 7fd4c8e70700 1 --2- 172.21.15.1:0/3575783419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4c41003f0 0x7fd4c4100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:22.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.428+0000 7fd4c8e70700 1 -- 172.21.15.1:0/3575783419 >> 172.21.15.1:0/3575783419 conn(0x7fd4c40fa820 msgr2=0x7fd4c40fcc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:22.431 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.428+0000 7fd4c8e70700 1 -- 172.21.15.1:0/3575783419 shutdown_connections 2023-11-16T16:38:22.431 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.428+0000 7fd4c8e70700 1 -- 172.21.15.1:0/3575783419 wait complete. 2023-11-16T16:38:22.432 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.429+0000 7fd4c8e70700 1 Processor -- start 2023-11-16T16:38:22.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.429+0000 7fd4c8e70700 1 -- start start 2023-11-16T16:38:22.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.429+0000 7fd4c8e70700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4c40ff200 0x7fd4c4103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:22.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.429+0000 7fd4c8e70700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4c41003f0 0x7fd4c41017d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:22.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.429+0000 7fd4c8e70700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4c4100d80 0x7fd4c4101d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:22.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.429+0000 7fd4c8e70700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c41024b0 con 0x7fd4c40ff200 2023-11-16T16:38:22.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.429+0000 7fd4c8e70700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c4102620 con 0x7fd4c4100d80 2023-11-16T16:38:22.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.429+0000 7fd4c8e70700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c41a0100 con 0x7fd4c41003f0 2023-11-16T16:38:22.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.429+0000 7fd4c259c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4c40ff200 0x7fd4c4103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:22.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.429+0000 7fd4c259c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4c40ff200 0x7fd4c4103190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42208/0 (socket says 172.21.15.1:42208) 2023-11-16T16:38:22.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.429+0000 7fd4c259c700 1 -- 172.21.15.1:0/2890273834 learned_addr learned my addr 172.21.15.1:0/2890273834 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:22.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.429+0000 7fd4c2d9d700 1 --2- 172.21.15.1:0/2890273834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4c4100d80 0x7fd4c4101d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:22.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.429+0000 7fd4c1d9b700 1 --2- 172.21.15.1:0/2890273834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4c41003f0 0x7fd4c41017d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:22.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.429+0000 7fd4c259c700 1 -- 172.21.15.1:0/2890273834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4c41003f0 msgr2=0x7fd4c41017d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:22.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.430+0000 7fd4c259c700 1 --2- 172.21.15.1:0/2890273834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4c41003f0 0x7fd4c41017d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:22.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.430+0000 7fd4c259c700 1 -- 172.21.15.1:0/2890273834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4c4100d80 msgr2=0x7fd4c4101d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:22.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.430+0000 7fd4c259c700 1 --2- 172.21.15.1:0/2890273834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4c4100d80 0x7fd4c4101d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:22.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.430+0000 7fd4c259c700 1 -- 172.21.15.1:0/2890273834 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b40095f0 con 0x7fd4c40ff200 2023-11-16T16:38:22.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.430+0000 7fd4c259c700 1 --2- 172.21.15.1:0/2890273834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4c40ff200 0x7fd4c4103190 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd4b40059d0 tx=0x7fd4b4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:22.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.430+0000 7fd4bb7fe700 1 -- 172.21.15.1:0/2890273834 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b401d070 con 0x7fd4c40ff200 2023-11-16T16:38:22.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.430+0000 7fd4bb7fe700 1 -- 172.21.15.1:0/2890273834 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4b4004020 con 0x7fd4c40ff200 2023-11-16T16:38:22.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.430+0000 7fd4c8e70700 1 -- 172.21.15.1:0/2890273834 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4c410abf0 con 0x7fd4c40ff200 2023-11-16T16:38:22.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.430+0000 7fd4bb7fe700 1 -- 172.21.15.1:0/2890273834 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b4022c40 con 0x7fd4c40ff200 2023-11-16T16:38:22.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.430+0000 7fd4c8e70700 1 -- 172.21.15.1:0/2890273834 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4c410aa60 con 0x7fd4c40ff200 2023-11-16T16:38:22.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.431+0000 7fd4c8e70700 1 -- 172.21.15.1:0/2890273834 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4c41a0720 con 0x7fd4c40ff200 2023-11-16T16:38:22.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.432+0000 7fd4bb7fe700 1 -- 172.21.15.1:0/2890273834 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fd4b4022470 con 0x7fd4c40ff200 2023-11-16T16:38:22.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.432+0000 7fd4bb7fe700 1 --2- 172.21.15.1:0/2890273834 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd4b00a11e0 0x7fd4b00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:22.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.433+0000 7fd4c1d9b700 1 --2- 172.21.15.1:0/2890273834 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd4b00a11e0 0x7fd4b00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:22.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.433+0000 7fd4bb7fe700 1 -- 172.21.15.1:0/2890273834 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd4b40cdff0 con 0x7fd4c40ff200 2023-11-16T16:38:22.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.433+0000 7fd4c1d9b700 1 --2- 172.21.15.1:0/2890273834 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd4b00a11e0 0x7fd4b00a3690 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac005950 tx=0x7fd4ac00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:22.438 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.437+0000 7fd4bb7fe700 1 -- 172.21.15.1:0/2890273834 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd4b4097630 con 0x7fd4c40ff200 2023-11-16T16:38:22.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.682+0000 7fd4c8e70700 1 -- 172.21.15.1:0/2890273834 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4c41a1000 con 0x7fd4c40ff200 2023-11-16T16:38:22.684 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.682+0000 7fd4bb7fe700 1 -- 172.21.15.1:0/2890273834 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10774 (secure 0 0 0) 0x7fd4b409ac50 con 0x7fd4c40ff200 2023-11-16T16:38:22.684 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:38:22.684 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2587563684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2587563684},{"type":"v1","addr":"172.21.15.1:6835","nonce":2587563684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/277936168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":277936168},{"type":"v1","addr":"172.21.15.1:6837","nonce":277936168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/1172575045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1172575045},{"type":"v1","addr":"172.21.15.1:6839","nonce":1172575045}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/2631175899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2631175899},{"type":"v1","addr":"172.21.15.1:6841","nonce":2631175899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3693905267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3693905267},{"type":"v1","addr":"172.21.15.53:6833","nonce":3693905267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/164668033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":164668033},{"type":"v1","addr":"172.21.15.26:6833","nonce":164668033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-16T16:37:13.918943+0000","modified":"2023-11-16T16:37:19.292898+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2140542192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2140542192},{"type":"v1","addr":"172.21.15.53:6835","nonce":2140542192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1010760275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1010760275},{"type":"v1","addr":"172.21.15.26:6835","nonce":1010760275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3197060302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3197060302},{"type":"v1","addr":"172.21.15.53:6837","nonce":3197060302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6837/1453504863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1453504863},{"type":"v1","addr":"172.21.15.26:6837","nonce":1453504863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.53:6839/1044753024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1044753024},{"type":"v1","addr":"172.21.15.53:6839","nonce":1044753024}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6839/2192205949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2192205949},{"type":"v1","addr":"172.21.15.26:6839","nonce":2192205949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-16T16:38:22.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.685+0000 7fd4c8e70700 1 -- 172.21.15.1:0/2890273834 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd4b00a11e0 msgr2=0x7fd4b00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:22.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.685+0000 7fd4c8e70700 1 --2- 172.21.15.1:0/2890273834 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd4b00a11e0 0x7fd4b00a3690 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac005950 tx=0x7fd4ac00b410 comp rx=0 tx=0).stop 2023-11-16T16:38:22.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.685+0000 7fd4c8e70700 1 -- 172.21.15.1:0/2890273834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4c40ff200 msgr2=0x7fd4c4103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:22.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.685+0000 7fd4c8e70700 1 --2- 172.21.15.1:0/2890273834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4c40ff200 0x7fd4c4103190 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd4b40059d0 tx=0x7fd4b4005740 comp rx=0 tx=0).stop 2023-11-16T16:38:22.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.685+0000 7fd4c8e70700 1 -- 172.21.15.1:0/2890273834 shutdown_connections 2023-11-16T16:38:22.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.685+0000 7fd4c8e70700 1 --2- 172.21.15.1:0/2890273834 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd4b00a11e0 0x7fd4b00a3690 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:22.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.685+0000 7fd4c8e70700 1 --2- 172.21.15.1:0/2890273834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4c4100d80 0x7fd4c4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:22.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.685+0000 7fd4c8e70700 1 --2- 172.21.15.1:0/2890273834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4c40ff200 0x7fd4c4103190 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:22.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.685+0000 7fd4c8e70700 1 --2- 172.21.15.1:0/2890273834 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd4c41003f0 0x7fd4c41017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:22.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.685+0000 7fd4c8e70700 1 -- 172.21.15.1:0/2890273834 >> 172.21.15.1:0/2890273834 conn(0x7fd4c40fa820 msgr2=0x7fd4c4069100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:22.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.685+0000 7fd4c8e70700 1 -- 172.21.15.1:0/2890273834 shutdown_connections 2023-11-16T16:38:22.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.685+0000 7fd4c8e70700 1 -- 172.21.15.1:0/2890273834 wait complete. 2023-11-16T16:38:22.688 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 23 2023-11-16T16:38:22.699 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-16T16:38:22.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.823+0000 7f6843113700 1 -- 172.21.15.1:0/711335053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f683c0ffd80 msgr2=0x7f683c107630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:22.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.823+0000 7f6843113700 1 --2- 172.21.15.1:0/711335053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f683c0ffd80 0x7f683c107630 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f683400b0a0 tx=0x7f683400b3b0 comp rx=0 tx=0).stop 2023-11-16T16:38:22.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.823+0000 7f6843113700 1 -- 172.21.15.1:0/711335053 shutdown_connections 2023-11-16T16:38:22.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.823+0000 7f6843113700 1 --2- 172.21.15.1:0/711335053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f683c0ffd80 0x7f683c107630 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:22.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.823+0000 7f6843113700 1 --2- 172.21.15.1:0/711335053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f683c100780 0x7f683c100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:22.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.823+0000 7f6843113700 1 --2- 172.21.15.1:0/711335053 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f683c0ff3d0 0x7f683c0ff840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:22.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.823+0000 7f6843113700 1 -- 172.21.15.1:0/711335053 >> 172.21.15.1:0/711335053 conn(0x7f683c0f9f50 msgr2=0x7f683c0fc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:22.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.823+0000 7f6843113700 1 -- 172.21.15.1:0/711335053 shutdown_connections 2023-11-16T16:38:22.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.823+0000 7f6843113700 1 -- 172.21.15.1:0/711335053 wait complete. 2023-11-16T16:38:22.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.823+0000 7f6843113700 1 Processor -- start 2023-11-16T16:38:22.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.823+0000 7f6843113700 1 -- start start 2023-11-16T16:38:22.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.823+0000 7f6843113700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f683c0ff3d0 0x7f683c196100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:22.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.824+0000 7f6843113700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f683c0ffd80 0x7f683c196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:22.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.824+0000 7f6843113700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f683c100780 0x7f683c19d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:22.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.824+0000 7f6843113700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683c19dc80 con 0x7f683c100780 2023-11-16T16:38:22.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.824+0000 7f6843113700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683c19ddc0 con 0x7f683c0ff3d0 2023-11-16T16:38:22.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.824+0000 7f6843113700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683c19df50 con 0x7f683c0ffd80 2023-11-16T16:38:22.828 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.824+0000 7f68416b0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f683c100780 0x7f683c19d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:22.828 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.824+0000 7f68416b0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f683c100780 0x7f683c19d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42226/0 (socket says 172.21.15.1:42226) 2023-11-16T16:38:22.828 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.824+0000 7f68416b0700 1 -- 172.21.15.1:0/809717535 learned_addr learned my addr 172.21.15.1:0/809717535 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:22.828 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.824+0000 7f6840eaf700 1 --2- 172.21.15.1:0/809717535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f683c0ff3d0 0x7f683c196100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:22.828 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.824+0000 7f683bfff700 1 --2- 172.21.15.1:0/809717535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f683c0ffd80 0x7f683c196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:22.828 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.824+0000 7f68416b0700 1 -- 172.21.15.1:0/809717535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f683c0ffd80 msgr2=0x7f683c196640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:22.828 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.824+0000 7f68416b0700 1 --2- 172.21.15.1:0/809717535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f683c0ffd80 0x7f683c196640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:22.828 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.824+0000 7f68416b0700 1 -- 172.21.15.1:0/809717535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f683c0ff3d0 msgr2=0x7f683c196100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:22.828 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.824+0000 7f68416b0700 1 --2- 172.21.15.1:0/809717535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f683c0ff3d0 0x7f683c196100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:22.828 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.824+0000 7f68416b0700 1 -- 172.21.15.1:0/809717535 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6834009d00 con 0x7f683c100780 2023-11-16T16:38:22.828 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.824+0000 7f68416b0700 1 --2- 172.21.15.1:0/809717535 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f683c100780 0x7f683c19d6b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f683400b780 tx=0x7f683400bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:22.828 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.824+0000 7f6839ffb700 1 -- 172.21.15.1:0/809717535 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683400e040 con 0x7f683c100780 2023-11-16T16:38:22.828 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.824+0000 7f6843113700 1 -- 172.21.15.1:0/809717535 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f683c10ab20 con 0x7f683c100780 2023-11-16T16:38:22.828 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.824+0000 7f6839ffb700 1 -- 172.21.15.1:0/809717535 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68340094a0 con 0x7f683c100780 2023-11-16T16:38:22.829 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.824+0000 7f6839ffb700 1 -- 172.21.15.1:0/809717535 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6834005330 con 0x7f683c100780 2023-11-16T16:38:22.829 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.824+0000 7f6843113700 1 -- 172.21.15.1:0/809717535 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f683c10a990 con 0x7f683c100780 2023-11-16T16:38:22.829 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.825+0000 7f6843113700 1 -- 172.21.15.1:0/809717535 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f683c067260 con 0x7f683c100780 2023-11-16T16:38:22.829 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.827+0000 7f6839ffb700 1 -- 172.21.15.1:0/809717535 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f683401da30 con 0x7f683c100780 2023-11-16T16:38:22.830 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.828+0000 7f6839ffb700 1 --2- 172.21.15.1:0/809717535 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f682c0a11e0 0x7f682c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:22.830 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.828+0000 7f6840eaf700 1 --2- 172.21.15.1:0/809717535 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f682c0a11e0 0x7f682c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:22.830 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.828+0000 7f6839ffb700 1 -- 172.21.15.1:0/809717535 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f68340ce500 con 0x7f683c100780 2023-11-16T16:38:22.830 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.828+0000 7f6840eaf700 1 --2- 172.21.15.1:0/809717535 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f682c0a11e0 0x7f682c0a3690 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6830009c60 tx=0x7f6830009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:22.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:22.831+0000 7f6839ffb700 1 -- 172.21.15.1:0/809717535 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6834097b40 con 0x7f683c100780 2023-11-16T16:38:22.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:22 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3454775261' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:38:22.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:22 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3400870854' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:38:23.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:22 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/3454775261' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:38:23.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:22 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/3400870854' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:38:23.033 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.032+0000 7f6843113700 1 -- 172.21.15.1:0/809717535 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f683c19eab0 con 0x7f683c100780 2023-11-16T16:38:23.035 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.033+0000 7f6839ffb700 1 -- 172.21.15.1:0/809717535 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22469 (secure 0 0 0) 0x7f683409b160 con 0x7f683c100780 2023-11-16T16:38:23.035 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:38:23.035 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":86,"fsid":"35d9b010-849d-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:30:06.217778+0000","modified":"2023-11-16T16:38:03.179244+0000","last_up_change":"2023-11-16T16:36:31.735403+0000","last_in_change":"2023-11-16T16:36:14.083021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:33:15.916723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-16T16:37:12.158031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-16T16:37:13.272343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a8480507-5275-4e52-953b-626110beb436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6803","nonce":2469671609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6805","nonce":2469671609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6809","nonce":2469671609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6807","nonce":2469671609}]},"public_addr":"172.21.15.1:6803/2469671609","cluster_addr":"172.21.15.1:6805/2469671609","heartbeat_back_addr":"172.21.15.1:6809/2469671609","heartbeat_front_addr":"172.21.15.1:6807/2469671609","state":["exists","up"]},{"osd":1,"uuid":"1d011db2-99d1-4575-bdc5-51f88a308fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6801","nonce":247994796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6803","nonce":247994796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6807","nonce":247994796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6805","nonce":247994796}]},"public_addr":"172.21.15.26:6801/247994796","cluster_addr":"172.21.15.26:6803/247994796","heartbeat_back_addr":"172.21.15.26:6807/247994796","heartbeat_front_addr":"172.21.15.26:6805/247994796","state":["exists","up"]},{"osd":2,"uuid":"4d7a2bd1-3c8e-4546-a243-f0354473b031","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6801","nonce":1659316290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6803","nonce":1659316290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6807","nonce":1659316290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6805","nonce":1659316290}]},"public_addr":"172.21.15.53:6801/1659316290","cluster_addr":"172.21.15.53:6803/1659316290","heartbeat_back_addr":"172.21.15.53:6807/1659316290","heartbeat_front_addr":"172.21.15.53:6805/1659316290","state":["exists","up"]},{"osd":3,"uuid":"9c290fd9-e121-4a81-9687-b0eb93debd86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6811","nonce":2728987281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6813","nonce":2728987281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6817","nonce":2728987281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6815","nonce":2728987281}]},"public_addr":"172.21.15.1:6811/2728987281","cluster_addr":"172.21.15.1:6813/2728987281","heartbeat_back_addr":"172.21.15.1:6817/2728987281","heartbeat_front_addr":"172.21.15.1:6815/2728987281","state":["exists","up"]},{"osd":4,"uuid":"533226bc-cc57-4f15-9e29-820294d57e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6809","nonce":2954740024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6811","nonce":2954740024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6815","nonce":2954740024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6813","nonce":2954740024}]},"public_addr":"172.21.15.26:6809/2954740024","cluster_addr":"172.21.15.26:6811/2954740024","heartbeat_back_addr":"172.21.15.26:6815/2954740024","heartbeat_front_addr":"172.21.15.26:6813/2954740024","state":["exists","up"]},{"osd":5,"uuid":"d3ab25eb-e1d4-4557-bc89-efc9072401ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6809","nonce":3966666318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6811","nonce":3966666318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6815","nonce":3966666318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6813","nonce":3966666318}]},"public_addr":"172.21.15.53:6809/3966666318","cluster_addr":"172.21.15.53:6811/3966666318","heartbeat_back_addr":"172.21.15.53:6815/3966666318","heartbeat_front_addr":"172.21.15.53:6813/3966666318","state":["exists","up"]},{"osd":6,"uuid":"180f506e-afb8-4043-b0b4-2dc86f6d4bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6819","nonce":3364032646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6821","nonce":3364032646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6825","nonce":3364032646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6823","nonce":3364032646}]},"public_addr":"172.21.15.1:6819/3364032646","cluster_addr":"172.21.15.1:6821/3364032646","heartbeat_back_addr":"172.21.15.1:6825/3364032646","heartbeat_front_addr":"172.21.15.1:6823/3364032646","state":["exists","up"]},{"osd":7,"uuid":"0d837bac-194f-498d-b7fb-b3147e992329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6817","nonce":3766901391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6819","nonce":3766901391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6823","nonce":3766901391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6821","nonce":3766901391}]},"public_addr":"172.21.15.26:6817/3766901391","cluster_addr":"172.21.15.26:6819/3766901391","heartbeat_back_addr":"172.21.15.26:6823/3766901391","heartbeat_front_addr":"172.21.15.26:6821/3766901391","state":["exists","up"]},{"osd":8,"uuid":"933f231a-7454-47b0-9291-f1fa64f0baec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6817","nonce":3224335871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6819","nonce":3224335871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6823","nonce":3224335871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6821","nonce":3224335871}]},"public_addr":"172.21.15.53:6817/3224335871","cluster_addr":"172.21.15.53:6819/3224335871","heartbeat_back_addr":"172.21.15.53:6823/3224335871","heartbeat_front_addr":"172.21.15.53:6821/3224335871","state":["exists","up"]},{"osd":9,"uuid":"245fcbb3-c651-4703-ab80-309dc98bd983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6827","nonce":1391199844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6829","nonce":1391199844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6833","nonce":1391199844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6831","nonce":1391199844}]},"public_addr":"172.21.15.1:6827/1391199844","cluster_addr":"172.21.15.1:6829/1391199844","heartbeat_back_addr":"172.21.15.1:6833/1391199844","heartbeat_front_addr":"172.21.15.1:6831/1391199844","state":["exists","up"]},{"osd":10,"uuid":"2c090e7a-5240-4d52-ae4f-e94fe05383d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6825","nonce":1773545447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6827","nonce":1773545447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6831","nonce":1773545447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6829","nonce":1773545447}]},"public_addr":"172.21.15.26:6825/1773545447","cluster_addr":"172.21.15.26:6827/1773545447","heartbeat_back_addr":"172.21.15.26:6831/1773545447","heartbeat_front_addr":"172.21.15.26:6829/1773545447","state":["exists","up"]},{"osd":11,"uuid":"4511e32d-e469-45e1-8310-0ff400cb7b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6825","nonce":2564669546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6827","nonce":2564669546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6831","nonce":2564669546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6829","nonce":2564669546}]},"public_addr":"172.21.15.53:6825/2564669546","cluster_addr":"172.21.15.53:6827/2564669546","heartbeat_back_addr":"172.21.15.53:6831/2564669546","heartbeat_front_addr":"172.21.15.53:6829/2564669546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:28.140250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:49.885461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:12.521487+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:34.926197+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:56.815594+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:18.387751+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:40.587106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:02.319370+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:23.832298+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:46.338040+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:08.730414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:30.050070+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1494336948":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6801/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6800/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6800/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/1641976669":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/2895923638":"2023-11-17T16:30:35.064289+0000","172.21.15.1:6801/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/78811711":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/4131957708":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/344403698":"2023-11-17T16:30:35.064289+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:38:23.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.035+0000 7f6843113700 1 -- 172.21.15.1:0/809717535 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f682c0a11e0 msgr2=0x7f682c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:23.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.035+0000 7f6843113700 1 --2- 172.21.15.1:0/809717535 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f682c0a11e0 0x7f682c0a3690 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6830009c60 tx=0x7f6830009260 comp rx=0 tx=0).stop 2023-11-16T16:38:23.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.035+0000 7f6843113700 1 -- 172.21.15.1:0/809717535 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f683c100780 msgr2=0x7f683c19d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:23.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.035+0000 7f6843113700 1 --2- 172.21.15.1:0/809717535 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f683c100780 0x7f683c19d6b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f683400b780 tx=0x7f683400bd60 comp rx=0 tx=0).stop 2023-11-16T16:38:23.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.035+0000 7f6843113700 1 -- 172.21.15.1:0/809717535 shutdown_connections 2023-11-16T16:38:23.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.035+0000 7f6843113700 1 --2- 172.21.15.1:0/809717535 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f682c0a11e0 0x7f682c0a3690 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:23.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.035+0000 7f6843113700 1 --2- 172.21.15.1:0/809717535 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f683c100780 0x7f683c19d6b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:23.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.035+0000 7f6843113700 1 --2- 172.21.15.1:0/809717535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f683c0ff3d0 0x7f683c196100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:23.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.035+0000 7f6843113700 1 --2- 172.21.15.1:0/809717535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f683c0ffd80 0x7f683c196640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:23.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.035+0000 7f6843113700 1 -- 172.21.15.1:0/809717535 >> 172.21.15.1:0/809717535 conn(0x7f683c0f9f50 msgr2=0x7f683c0fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:23.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.035+0000 7f6843113700 1 -- 172.21.15.1:0/809717535 shutdown_connections 2023-11-16T16:38:23.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.035+0000 7f6843113700 1 -- 172.21.15.1:0/809717535 wait complete. 2023-11-16T16:38:23.050 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-16T16:33:15.916723+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-16T16:37:12.158031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-16T16:37:13.272343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-16T16:38:23.050 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-16T16:38:23.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:22 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/3454775261' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:38:23.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:22 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/3400870854' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:38:23.179 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.177+0000 7f51f659f700 1 -- 172.21.15.1:0/1099088333 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51f0101e10 msgr2=0x7f51f01086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:23.179 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.177+0000 7f51f659f700 1 --2- 172.21.15.1:0/1099088333 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51f0101e10 0x7f51f01086d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f51e800b0a0 tx=0x7f51e800b3b0 comp rx=0 tx=0).stop 2023-11-16T16:38:23.179 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.177+0000 7f51f659f700 1 -- 172.21.15.1:0/1099088333 shutdown_connections 2023-11-16T16:38:23.179 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.177+0000 7f51f659f700 1 --2- 172.21.15.1:0/1099088333 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51f0101e10 0x7f51f01086d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:23.180 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.177+0000 7f51f659f700 1 --2- 172.21.15.1:0/1099088333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51f0100290 0x7f51f01006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:23.180 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.177+0000 7f51f659f700 1 --2- 172.21.15.1:0/1099088333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f0101480 0x7f51f01018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:23.180 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.177+0000 7f51f659f700 1 -- 172.21.15.1:0/1099088333 >> 172.21.15.1:0/1099088333 conn(0x7f51f00fb8b0 msgr2=0x7f51f00fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:23.180 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.177+0000 7f51f659f700 1 -- 172.21.15.1:0/1099088333 shutdown_connections 2023-11-16T16:38:23.180 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.177+0000 7f51f659f700 1 -- 172.21.15.1:0/1099088333 wait complete. 2023-11-16T16:38:23.180 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.178+0000 7f51f659f700 1 Processor -- start 2023-11-16T16:38:23.180 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.178+0000 7f51f659f700 1 -- start start 2023-11-16T16:38:23.182 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.178+0000 7f51f659f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51f0100290 0x7f51f0196040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:23.182 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.178+0000 7f51f659f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51f0101480 0x7f51f0196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:23.182 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.178+0000 7f51f659f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f0101e10 0x7f51f019d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:23.182 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.178+0000 7f51f659f700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f019dbc0 con 0x7f51f0101480 2023-11-16T16:38:23.182 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.178+0000 7f51f659f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f019dd00 con 0x7f51f0100290 2023-11-16T16:38:23.182 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.178+0000 7f51f659f700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f019de90 con 0x7f51f0101e10 2023-11-16T16:38:23.182 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.178+0000 7f51ef7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51f0101480 0x7f51f0196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:23.182 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.178+0000 7f51ef7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51f0101480 0x7f51f0196580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42238/0 (socket says 172.21.15.1:42238) 2023-11-16T16:38:23.182 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.178+0000 7f51ef7fe700 1 -- 172.21.15.1:0/3052033299 learned_addr learned my addr 172.21.15.1:0/3052033299 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:23.182 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.178+0000 7f51f4b3c700 1 --2- 172.21.15.1:0/3052033299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f0101e10 0x7f51f019d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:23.182 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.178+0000 7f51effff700 1 --2- 172.21.15.1:0/3052033299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51f0100290 0x7f51f0196040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:23.183 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.178+0000 7f51ef7fe700 1 -- 172.21.15.1:0/3052033299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f0101e10 msgr2=0x7f51f019d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:23.183 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.178+0000 7f51ef7fe700 1 --2- 172.21.15.1:0/3052033299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f0101e10 0x7f51f019d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:23.183 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.178+0000 7f51ef7fe700 1 -- 172.21.15.1:0/3052033299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51f0100290 msgr2=0x7f51f0196040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:23.183 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.178+0000 7f51ef7fe700 1 --2- 172.21.15.1:0/3052033299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51f0100290 0x7f51f0196040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:23.183 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.178+0000 7f51ef7fe700 1 -- 172.21.15.1:0/3052033299 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e8009d00 con 0x7f51f0101480 2023-11-16T16:38:23.183 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.179+0000 7f51ef7fe700 1 --2- 172.21.15.1:0/3052033299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51f0101480 0x7f51f0196580 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f51d800d980 tx=0x7f51d800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:23.183 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.179+0000 7f51ed7fa700 1 -- 172.21.15.1:0/3052033299 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d8004220 con 0x7f51f0101480 2023-11-16T16:38:23.183 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.179+0000 7f51ed7fa700 1 -- 172.21.15.1:0/3052033299 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51d8007640 con 0x7f51f0101480 2023-11-16T16:38:23.183 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.179+0000 7f51f659f700 1 -- 172.21.15.1:0/3052033299 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51f010bc70 con 0x7f51f0101480 2023-11-16T16:38:23.183 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.179+0000 7f51ed7fa700 1 -- 172.21.15.1:0/3052033299 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d8005610 con 0x7f51f0101480 2023-11-16T16:38:23.183 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.179+0000 7f51f659f700 1 -- 172.21.15.1:0/3052033299 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f010bae0 con 0x7f51f0101480 2023-11-16T16:38:23.183 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.180+0000 7f51f659f700 1 -- 172.21.15.1:0/3052033299 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51f0107730 con 0x7f51f0101480 2023-11-16T16:38:23.184 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.181+0000 7f51ed7fa700 1 -- 172.21.15.1:0/3052033299 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f51d8004750 con 0x7f51f0101480 2023-11-16T16:38:23.184 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.182+0000 7f51ed7fa700 1 --2- 172.21.15.1:0/3052033299 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f51dc0a1260 0x7f51dc0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:23.184 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.182+0000 7f51effff700 1 --2- 172.21.15.1:0/3052033299 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f51dc0a1260 0x7f51dc0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:23.184 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.182+0000 7f51ed7fa700 1 -- 172.21.15.1:0/3052033299 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f51d80cc850 con 0x7f51f0101480 2023-11-16T16:38:23.184 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.182+0000 7f51effff700 1 --2- 172.21.15.1:0/3052033299 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f51dc0a1260 0x7f51dc0a3710 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f51e0005fd0 tx=0x7f51e0005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:23.188 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.186+0000 7f51ed7fa700 1 -- 172.21.15.1:0/3052033299 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f51d8095e40 con 0x7f51f0101480 2023-11-16T16:38:23.386 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.385+0000 7f51f659f700 1 -- 172.21.15.1:0/3052033299 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f51f0061be0 con 0x7f51f0101480 2023-11-16T16:38:23.387 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.385+0000 7f51ed7fa700 1 -- 172.21.15.1:0/3052033299 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f51d8016020 con 0x7f51f0101480 2023-11-16T16:38:23.387 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2023-11-16T16:38:23.389 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.387+0000 7f51f659f700 1 -- 172.21.15.1:0/3052033299 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f51dc0a1260 msgr2=0x7f51dc0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:23.389 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.387+0000 7f51f659f700 1 --2- 172.21.15.1:0/3052033299 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f51dc0a1260 0x7f51dc0a3710 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f51e0005fd0 tx=0x7f51e0005e50 comp rx=0 tx=0).stop 2023-11-16T16:38:23.389 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.387+0000 7f51f659f700 1 -- 172.21.15.1:0/3052033299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51f0101480 msgr2=0x7f51f0196580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:23.389 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.387+0000 7f51f659f700 1 --2- 172.21.15.1:0/3052033299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51f0101480 0x7f51f0196580 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f51d800d980 tx=0x7f51d800dc90 comp rx=0 tx=0).stop 2023-11-16T16:38:23.389 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.387+0000 7f51f659f700 1 -- 172.21.15.1:0/3052033299 shutdown_connections 2023-11-16T16:38:23.390 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.387+0000 7f51f659f700 1 --2- 172.21.15.1:0/3052033299 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f51dc0a1260 0x7f51dc0a3710 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:23.390 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.387+0000 7f51f659f700 1 --2- 172.21.15.1:0/3052033299 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f51f0101e10 0x7f51f019d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:23.390 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.387+0000 7f51f659f700 1 --2- 172.21.15.1:0/3052033299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51f0100290 0x7f51f0196040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:23.390 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.387+0000 7f51f659f700 1 --2- 172.21.15.1:0/3052033299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51f0101480 0x7f51f0196580 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:23.390 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.387+0000 7f51f659f700 1 -- 172.21.15.1:0/3052033299 >> 172.21.15.1:0/3052033299 conn(0x7f51f00fb8b0 msgr2=0x7f51f0104870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:23.390 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.387+0000 7f51f659f700 1 -- 172.21.15.1:0/3052033299 shutdown_connections 2023-11-16T16:38:23.390 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.387+0000 7f51f659f700 1 -- 172.21.15.1:0/3052033299 wait complete. 2023-11-16T16:38:23.401 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-16T16:38:23.529 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.527+0000 7f8f0de40700 1 -- 172.21.15.1:0/3047175166 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f081003b0 msgr2=0x7f8f08100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:23.530 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.527+0000 7f8f0de40700 1 --2- 172.21.15.1:0/3047175166 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f081003b0 0x7f8f08100800 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8009990 tx=0x7f8ef8009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:23.530 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.527+0000 7f8f0de40700 1 -- 172.21.15.1:0/3047175166 shutdown_connections 2023-11-16T16:38:23.530 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.527+0000 7f8f0de40700 1 --2- 172.21.15.1:0/3047175166 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f08100d40 0x7f8f08107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:23.530 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.527+0000 7f8f0de40700 1 --2- 172.21.15.1:0/3047175166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f080ff1c0 0x7f8f080ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:23.530 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.527+0000 7f8f0de40700 1 --2- 172.21.15.1:0/3047175166 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f081003b0 0x7f8f08100800 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:23.530 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.527+0000 7f8f0de40700 1 -- 172.21.15.1:0/3047175166 >> 172.21.15.1:0/3047175166 conn(0x7f8f080fa800 msgr2=0x7f8f080fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:23.530 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.527+0000 7f8f0de40700 1 -- 172.21.15.1:0/3047175166 shutdown_connections 2023-11-16T16:38:23.530 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.527+0000 7f8f0de40700 1 -- 172.21.15.1:0/3047175166 wait complete. 2023-11-16T16:38:23.530 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.528+0000 7f8f0de40700 1 Processor -- start 2023-11-16T16:38:23.530 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.528+0000 7f8f0de40700 1 -- start start 2023-11-16T16:38:23.532 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.528+0000 7f8f0de40700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f080ff1c0 0x7f8f08069970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:23.532 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.528+0000 7f8f0de40700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f081003b0 0x7f8f0819ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:23.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.528+0000 7f8f0de40700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f08100d40 0x7f8f0819d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:23.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.528+0000 7f8f0de40700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0819db30 con 0x7f8f080ff1c0 2023-11-16T16:38:23.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.528+0000 7f8f0de40700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0819dca0 con 0x7f8f081003b0 2023-11-16T16:38:23.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.528+0000 7f8f0de40700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0819de60 con 0x7f8f08100d40 2023-11-16T16:38:23.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.528+0000 7f8f077fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f080ff1c0 0x7f8f08069970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:23.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.528+0000 7f8f077fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f080ff1c0 0x7f8f08069970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42250/0 (socket says 172.21.15.1:42250) 2023-11-16T16:38:23.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.528+0000 7f8f077fe700 1 -- 172.21.15.1:0/3771097311 learned_addr learned my addr 172.21.15.1:0/3771097311 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:23.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.528+0000 7f8f07fff700 1 --2- 172.21.15.1:0/3771097311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f08100d40 0x7f8f0819d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:23.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.529+0000 7f8f06ffd700 1 --2- 172.21.15.1:0/3771097311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f081003b0 0x7f8f0819ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:23.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.529+0000 7f8f077fe700 1 -- 172.21.15.1:0/3771097311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f08100d40 msgr2=0x7f8f0819d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:23.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.529+0000 7f8f077fe700 1 --2- 172.21.15.1:0/3771097311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f08100d40 0x7f8f0819d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:23.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.529+0000 7f8f077fe700 1 -- 172.21.15.1:0/3771097311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f081003b0 msgr2=0x7f8f0819ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:23.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.529+0000 7f8f077fe700 1 --2- 172.21.15.1:0/3771097311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f081003b0 0x7f8f0819ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:23.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.529+0000 7f8f077fe700 1 -- 172.21.15.1:0/3771097311 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef80095f0 con 0x7f8f080ff1c0 2023-11-16T16:38:23.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.529+0000 7f8f07fff700 1 --2- 172.21.15.1:0/3771097311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f08100d40 0x7f8f0819d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:38:23.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.529+0000 7f8f06ffd700 1 --2- 172.21.15.1:0/3771097311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f081003b0 0x7f8f0819ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:38:23.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.529+0000 7f8f077fe700 1 --2- 172.21.15.1:0/3771097311 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f080ff1c0 0x7f8f08069970 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8efc00e970 tx=0x7f8efc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:23.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.529+0000 7f8f04ff9700 1 -- 172.21.15.1:0/3771097311 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc009790 con 0x7f8f080ff1c0 2023-11-16T16:38:23.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.529+0000 7f8f04ff9700 1 -- 172.21.15.1:0/3771097311 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8efc0044e0 con 0x7f8f080ff1c0 2023-11-16T16:38:23.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.529+0000 7f8f04ff9700 1 -- 172.21.15.1:0/3771097311 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc003e30 con 0x7f8f080ff1c0 2023-11-16T16:38:23.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.529+0000 7f8f0de40700 1 -- 172.21.15.1:0/3771097311 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f0810ab70 con 0x7f8f080ff1c0 2023-11-16T16:38:23.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.529+0000 7f8f0de40700 1 -- 172.21.15.1:0/3771097311 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f0810a9e0 con 0x7f8f080ff1c0 2023-11-16T16:38:23.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.530+0000 7f8f0de40700 1 -- 172.21.15.1:0/3771097311 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f081067b0 con 0x7f8f080ff1c0 2023-11-16T16:38:23.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.531+0000 7f8f04ff9700 1 -- 172.21.15.1:0/3771097311 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f8efc003f90 con 0x7f8f080ff1c0 2023-11-16T16:38:23.535 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.532+0000 7f8f04ff9700 1 --2- 172.21.15.1:0/3771097311 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8ef00a1260 0x7f8ef00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:23.535 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.532+0000 7f8f06ffd700 1 --2- 172.21.15.1:0/3771097311 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8ef00a1260 0x7f8ef00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:23.535 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.532+0000 7f8f04ff9700 1 -- 172.21.15.1:0/3771097311 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8efc014070 con 0x7f8f080ff1c0 2023-11-16T16:38:23.535 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.532+0000 7f8f06ffd700 1 --2- 172.21.15.1:0/3771097311 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8ef00a1260 0x7f8ef00a3710 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8006010 tx=0x7f8ef8005c20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:23.538 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.536+0000 7f8f04ff9700 1 -- 172.21.15.1:0/3771097311 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8efc096310 con 0x7f8f080ff1c0 2023-11-16T16:38:23.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:23 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/2890273834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:38:23.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:23 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/809717535' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:38:23.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:23 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3052033299' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:38:23.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:23 smithi001 ceph-mon[98800]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:38:23.738 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.736+0000 7f8f0de40700 1 -- 172.21.15.1:0/3771097311 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8f08061b00 con 0x7f8f080ff1c0 2023-11-16T16:38:23.739 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.737+0000 7f8f04ff9700 1 -- 172.21.15.1:0/3771097311 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8efc099930 con 0x7f8f080ff1c0 2023-11-16T16:38:23.739 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2023-11-16T16:38:23.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.739+0000 7f8f0de40700 1 -- 172.21.15.1:0/3771097311 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8ef00a1260 msgr2=0x7f8ef00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:23.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.739+0000 7f8f0de40700 1 --2- 172.21.15.1:0/3771097311 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8ef00a1260 0x7f8ef00a3710 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8006010 tx=0x7f8ef8005c20 comp rx=0 tx=0).stop 2023-11-16T16:38:23.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.739+0000 7f8f0de40700 1 -- 172.21.15.1:0/3771097311 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f080ff1c0 msgr2=0x7f8f08069970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:23.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.739+0000 7f8f0de40700 1 --2- 172.21.15.1:0/3771097311 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f080ff1c0 0x7f8f08069970 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8efc00e970 tx=0x7f8efc00ed30 comp rx=0 tx=0).stop 2023-11-16T16:38:23.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.739+0000 7f8f0de40700 1 -- 172.21.15.1:0/3771097311 shutdown_connections 2023-11-16T16:38:23.742 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.739+0000 7f8f0de40700 1 --2- 172.21.15.1:0/3771097311 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8ef00a1260 0x7f8ef00a3710 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:23.742 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.739+0000 7f8f0de40700 1 --2- 172.21.15.1:0/3771097311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8f08100d40 0x7f8f0819d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:23.742 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.739+0000 7f8f0de40700 1 --2- 172.21.15.1:0/3771097311 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f080ff1c0 0x7f8f08069970 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:23.742 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.739+0000 7f8f0de40700 1 --2- 172.21.15.1:0/3771097311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f081003b0 0x7f8f0819ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:23.742 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.739+0000 7f8f0de40700 1 -- 172.21.15.1:0/3771097311 >> 172.21.15.1:0/3771097311 conn(0x7f8f080fa800 msgr2=0x7f8f081038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:23.742 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.739+0000 7f8f0de40700 1 -- 172.21.15.1:0/3771097311 shutdown_connections 2023-11-16T16:38:23.742 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.739+0000 7f8f0de40700 1 -- 172.21.15.1:0/3771097311 wait complete. 2023-11-16T16:38:23.753 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-16T16:38:23.883 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.880+0000 7fc679ac9700 1 -- 172.21.15.1:0/2221180351 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc674100310 msgr2=0x7fc674100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:23.883 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.880+0000 7fc679ac9700 1 --2- 172.21.15.1:0/2221180351 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc674100310 0x7fc674100720 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc664009990 tx=0x7fc664009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:23.883 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.880+0000 7fc679ac9700 1 -- 172.21.15.1:0/2221180351 shutdown_connections 2023-11-16T16:38:23.883 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.880+0000 7fc679ac9700 1 --2- 172.21.15.1:0/2221180351 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc674101e90 0x7fc674108750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:23.883 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.880+0000 7fc679ac9700 1 --2- 172.21.15.1:0/2221180351 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc674100310 0x7fc674100720 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:23.883 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.880+0000 7fc679ac9700 1 --2- 172.21.15.1:0/2221180351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc674101500 0x7fc674101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:23.883 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.880+0000 7fc679ac9700 1 -- 172.21.15.1:0/2221180351 >> 172.21.15.1:0/2221180351 conn(0x7fc6740fb910 msgr2=0x7fc6740fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:23.884 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.880+0000 7fc679ac9700 1 -- 172.21.15.1:0/2221180351 shutdown_connections 2023-11-16T16:38:23.884 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.880+0000 7fc679ac9700 1 -- 172.21.15.1:0/2221180351 wait complete. 2023-11-16T16:38:23.884 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.881+0000 7fc679ac9700 1 Processor -- start 2023-11-16T16:38:23.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.881+0000 7fc679ac9700 1 -- start start 2023-11-16T16:38:23.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.882+0000 7fc679ac9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc674100310 0x7fc674196030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:23.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.882+0000 7fc679ac9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc674101500 0x7fc674196570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:23.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.882+0000 7fc679ac9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc674101e90 0x7fc67419d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:23.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.882+0000 7fc679ac9700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc67419dbb0 con 0x7fc674101500 2023-11-16T16:38:23.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.882+0000 7fc679ac9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc67419dcf0 con 0x7fc674101e90 2023-11-16T16:38:23.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.882+0000 7fc679ac9700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc67419de80 con 0x7fc674100310 2023-11-16T16:38:23.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.882+0000 7fc672ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc674101500 0x7fc674196570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:23.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.882+0000 7fc672ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc674101500 0x7fc674196570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42266/0 (socket says 172.21.15.1:42266) 2023-11-16T16:38:23.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.882+0000 7fc672ffd700 1 -- 172.21.15.1:0/2656437413 learned_addr learned my addr 172.21.15.1:0/2656437413 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:23.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.882+0000 7fc6737fe700 1 --2- 172.21.15.1:0/2656437413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc674100310 0x7fc674196030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:23.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.882+0000 7fc672ffd700 1 -- 172.21.15.1:0/2656437413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc674100310 msgr2=0x7fc674196030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:23.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.882+0000 7fc672ffd700 1 --2- 172.21.15.1:0/2656437413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc674100310 0x7fc674196030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:23.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.882+0000 7fc672ffd700 1 -- 172.21.15.1:0/2656437413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc674101e90 msgr2=0x7fc67419d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:23.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.882+0000 7fc672ffd700 1 --2- 172.21.15.1:0/2656437413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc674101e90 0x7fc67419d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:23.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.882+0000 7fc672ffd700 1 -- 172.21.15.1:0/2656437413 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6640095f0 con 0x7fc674101500 2023-11-16T16:38:23.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.882+0000 7fc672ffd700 1 --2- 172.21.15.1:0/2656437413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc674101500 0x7fc674196570 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc66800d980 tx=0x7fc66800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:23.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.882+0000 7fc670ff9700 1 -- 172.21.15.1:0/2656437413 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc668004220 con 0x7fc674101500 2023-11-16T16:38:23.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.882+0000 7fc670ff9700 1 -- 172.21.15.1:0/2656437413 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc668007640 con 0x7fc674101500 2023-11-16T16:38:23.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.882+0000 7fc679ac9700 1 -- 172.21.15.1:0/2656437413 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc67410bc70 con 0x7fc674101500 2023-11-16T16:38:23.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.882+0000 7fc670ff9700 1 -- 172.21.15.1:0/2656437413 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc668005610 con 0x7fc674101500 2023-11-16T16:38:23.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.882+0000 7fc679ac9700 1 -- 172.21.15.1:0/2656437413 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc67410bae0 con 0x7fc674101500 2023-11-16T16:38:23.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.883+0000 7fc679ac9700 1 -- 172.21.15.1:0/2656437413 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc674107920 con 0x7fc674101500 2023-11-16T16:38:23.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.884+0000 7fc670ff9700 1 -- 172.21.15.1:0/2656437413 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fc668004750 con 0x7fc674101500 2023-11-16T16:38:23.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.885+0000 7fc670ff9700 1 --2- 172.21.15.1:0/2656437413 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc65c0a1260 0x7fc65c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:23.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.885+0000 7fc670ff9700 1 -- 172.21.15.1:0/2656437413 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc6680cc850 con 0x7fc674101500 2023-11-16T16:38:23.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.885+0000 7fc6737fe700 1 --2- 172.21.15.1:0/2656437413 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc65c0a1260 0x7fc65c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:23.891 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.886+0000 7fc6737fe700 1 --2- 172.21.15.1:0/2656437413 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc65c0a1260 0x7fc65c0a3710 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc66400b5c0 tx=0x7fc664005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:23.891 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:23.889+0000 7fc670ff9700 1 -- 172.21.15.1:0/2656437413 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc668095e40 con 0x7fc674101500 2023-11-16T16:38:24.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:23 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/2890273834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:38:24.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:23 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/809717535' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:38:24.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:23 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/3052033299' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:38:24.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:23 smithi026 ceph-mon[100112]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:38:24.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:23 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/2890273834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:38:24.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:23 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/809717535' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:38:24.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:23 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/3052033299' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:38:24.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:23 smithi053 ceph-mon[100638]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:38:24.093 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.091+0000 7fc679ac9700 1 -- 172.21.15.1:0/2656437413 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc674061700 con 0x7fc674101500 2023-11-16T16:38:24.093 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.092+0000 7fc670ff9700 1 -- 172.21.15.1:0/2656437413 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc668016020 con 0x7fc674101500 2023-11-16T16:38:24.094 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2023-11-16T16:38:24.096 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.094+0000 7fc679ac9700 1 -- 172.21.15.1:0/2656437413 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc65c0a1260 msgr2=0x7fc65c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:24.096 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.094+0000 7fc679ac9700 1 --2- 172.21.15.1:0/2656437413 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc65c0a1260 0x7fc65c0a3710 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc66400b5c0 tx=0x7fc664005fd0 comp rx=0 tx=0).stop 2023-11-16T16:38:24.096 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.094+0000 7fc679ac9700 1 -- 172.21.15.1:0/2656437413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc674101500 msgr2=0x7fc674196570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:24.096 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.094+0000 7fc679ac9700 1 --2- 172.21.15.1:0/2656437413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc674101500 0x7fc674196570 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc66800d980 tx=0x7fc66800dc90 comp rx=0 tx=0).stop 2023-11-16T16:38:24.096 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.094+0000 7fc679ac9700 1 -- 172.21.15.1:0/2656437413 shutdown_connections 2023-11-16T16:38:24.096 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.094+0000 7fc679ac9700 1 --2- 172.21.15.1:0/2656437413 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc65c0a1260 0x7fc65c0a3710 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:24.097 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.094+0000 7fc679ac9700 1 --2- 172.21.15.1:0/2656437413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc674101e90 0x7fc67419d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:24.097 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.094+0000 7fc679ac9700 1 --2- 172.21.15.1:0/2656437413 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc674100310 0x7fc674196030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:24.097 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.094+0000 7fc679ac9700 1 --2- 172.21.15.1:0/2656437413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc674101500 0x7fc674196570 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:24.097 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.094+0000 7fc679ac9700 1 -- 172.21.15.1:0/2656437413 >> 172.21.15.1:0/2656437413 conn(0x7fc6740fb910 msgr2=0x7fc674104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:24.097 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.094+0000 7fc679ac9700 1 -- 172.21.15.1:0/2656437413 shutdown_connections 2023-11-16T16:38:24.097 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.094+0000 7fc679ac9700 1 -- 172.21.15.1:0/2656437413 wait complete. 2023-11-16T16:38:24.108 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:38:24.235 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.232+0000 7f742c1e2700 1 -- 172.21.15.1:0/2908144293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7424100d70 msgr2=0x7f7424103150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:24.236 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.232+0000 7f742c1e2700 1 --2- 172.21.15.1:0/2908144293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7424100d70 0x7f7424103150 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7414009990 tx=0x7f7414009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:24.236 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.232+0000 7f742c1e2700 1 -- 172.21.15.1:0/2908144293 shutdown_connections 2023-11-16T16:38:24.236 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.232+0000 7f742c1e2700 1 --2- 172.21.15.1:0/2908144293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7424105fb0 0x7f7424108410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:24.236 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.232+0000 7f742c1e2700 1 --2- 172.21.15.1:0/2908144293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7424100d70 0x7f7424103150 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:24.236 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.232+0000 7f742c1e2700 1 --2- 172.21.15.1:0/2908144293 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7424103690 0x7f7424105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:24.236 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.232+0000 7f742c1e2700 1 -- 172.21.15.1:0/2908144293 >> 172.21.15.1:0/2908144293 conn(0x7f74240fa7d0 msgr2=0x7f74240fcc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:24.236 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.232+0000 7f742c1e2700 1 -- 172.21.15.1:0/2908144293 shutdown_connections 2023-11-16T16:38:24.236 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.232+0000 7f742c1e2700 1 -- 172.21.15.1:0/2908144293 wait complete. 2023-11-16T16:38:24.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.232+0000 7f742c1e2700 1 Processor -- start 2023-11-16T16:38:24.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.232+0000 7f742c1e2700 1 -- start start 2023-11-16T16:38:24.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.233+0000 7f742c1e2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7424100d70 0x7f7424196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:24.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.233+0000 7f742c1e2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7424103690 0x7f7424196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:24.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.233+0000 7f742c1e2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7424105fb0 0x7f742419d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:24.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.233+0000 7f742c1e2700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742419db80 con 0x7f7424100d70 2023-11-16T16:38:24.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.233+0000 7f742c1e2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742419dcc0 con 0x7f7424105fb0 2023-11-16T16:38:24.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.233+0000 7f742c1e2700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742419de50 con 0x7f7424103690 2023-11-16T16:38:24.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.233+0000 7f7429f7e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7424100d70 0x7f7424196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:24.238 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.233+0000 7f7429f7e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7424100d70 0x7f7424196000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42280/0 (socket says 172.21.15.1:42280) 2023-11-16T16:38:24.238 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.233+0000 7f7429f7e700 1 -- 172.21.15.1:0/456715779 learned_addr learned my addr 172.21.15.1:0/456715779 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:24.238 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.233+0000 7f742a77f700 1 --2- 172.21.15.1:0/456715779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7424105fb0 0x7f742419d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:24.238 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.233+0000 7f7429f7e700 1 -- 172.21.15.1:0/456715779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7424103690 msgr2=0x7f7424196540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:24.238 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.233+0000 7f7429f7e700 1 --2- 172.21.15.1:0/456715779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7424103690 0x7f7424196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:24.238 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.233+0000 7f7429f7e700 1 -- 172.21.15.1:0/456715779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7424105fb0 msgr2=0x7f742419d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:24.238 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.233+0000 7f7429f7e700 1 --2- 172.21.15.1:0/456715779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7424105fb0 0x7f742419d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:24.238 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.233+0000 7f7429f7e700 1 -- 172.21.15.1:0/456715779 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74140095f0 con 0x7f7424100d70 2023-11-16T16:38:24.238 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.233+0000 7f7429f7e700 1 --2- 172.21.15.1:0/456715779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7424100d70 0x7f7424196000 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7414006010 tx=0x7f741400f7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:24.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.233+0000 7f741affd700 1 -- 172.21.15.1:0/456715779 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741401d070 con 0x7f7424100d70 2023-11-16T16:38:24.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.233+0000 7f741affd700 1 -- 172.21.15.1:0/456715779 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f741400fca0 con 0x7f7424100d70 2023-11-16T16:38:24.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.233+0000 7f742c1e2700 1 -- 172.21.15.1:0/456715779 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f742410bca0 con 0x7f7424100d70 2023-11-16T16:38:24.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.233+0000 7f741affd700 1 -- 172.21.15.1:0/456715779 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74140176e0 con 0x7f7424100d70 2023-11-16T16:38:24.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.233+0000 7f742c1e2700 1 -- 172.21.15.1:0/456715779 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f742410bb10 con 0x7f7424100d70 2023-11-16T16:38:24.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.234+0000 7f742c1e2700 1 -- 172.21.15.1:0/456715779 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74240fc340 con 0x7f7424100d70 2023-11-16T16:38:24.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.235+0000 7f741affd700 1 -- 172.21.15.1:0/456715779 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f7414017840 con 0x7f7424100d70 2023-11-16T16:38:24.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.236+0000 7f741affd700 1 --2- 172.21.15.1:0/456715779 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f74100a11e0 0x7f74100a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:24.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.236+0000 7f742977d700 1 --2- 172.21.15.1:0/456715779 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f74100a11e0 0x7f74100a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:24.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.237+0000 7f741affd700 1 -- 172.21.15.1:0/456715779 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f74140ce180 con 0x7f7424100d70 2023-11-16T16:38:24.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.237+0000 7f742977d700 1 --2- 172.21.15.1:0/456715779 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f74100a11e0 0x7f74100a3690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f742000a990 tx=0x7f7420005c80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:24.242 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.240+0000 7f741affd700 1 -- 172.21.15.1:0/456715779 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7414097800 con 0x7f7424100d70 2023-11-16T16:38:24.487 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.485+0000 7f742c1e2700 1 -- 172.21.15.1:0/456715779 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7424061700 con 0x7f7424100d70 2023-11-16T16:38:24.487 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.486+0000 7f741affd700 1 -- 172.21.15.1:0/456715779 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10774 (secure 0 0 0) 0x7f741409ae20 con 0x7f7424100d70 2023-11-16T16:38:24.488 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:38:24.488 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2587563684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2587563684},{"type":"v1","addr":"172.21.15.1:6835","nonce":2587563684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/277936168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":277936168},{"type":"v1","addr":"172.21.15.1:6837","nonce":277936168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/1172575045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1172575045},{"type":"v1","addr":"172.21.15.1:6839","nonce":1172575045}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/2631175899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2631175899},{"type":"v1","addr":"172.21.15.1:6841","nonce":2631175899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3693905267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3693905267},{"type":"v1","addr":"172.21.15.53:6833","nonce":3693905267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/164668033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":164668033},{"type":"v1","addr":"172.21.15.26:6833","nonce":164668033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-16T16:37:13.918943+0000","modified":"2023-11-16T16:37:19.292898+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2140542192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2140542192},{"type":"v1","addr":"172.21.15.53:6835","nonce":2140542192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1010760275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1010760275},{"type":"v1","addr":"172.21.15.26:6835","nonce":1010760275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3197060302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3197060302},{"type":"v1","addr":"172.21.15.53:6837","nonce":3197060302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6837/1453504863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1453504863},{"type":"v1","addr":"172.21.15.26:6837","nonce":1453504863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.53:6839/1044753024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1044753024},{"type":"v1","addr":"172.21.15.53:6839","nonce":1044753024}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6839/2192205949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2192205949},{"type":"v1","addr":"172.21.15.26:6839","nonce":2192205949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-16T16:38:24.490 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.488+0000 7f742c1e2700 1 -- 172.21.15.1:0/456715779 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f74100a11e0 msgr2=0x7f74100a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:24.490 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.488+0000 7f742c1e2700 1 --2- 172.21.15.1:0/456715779 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f74100a11e0 0x7f74100a3690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f742000a990 tx=0x7f7420005c80 comp rx=0 tx=0).stop 2023-11-16T16:38:24.490 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.488+0000 7f742c1e2700 1 -- 172.21.15.1:0/456715779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7424100d70 msgr2=0x7f7424196000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:24.490 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.488+0000 7f742c1e2700 1 --2- 172.21.15.1:0/456715779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7424100d70 0x7f7424196000 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7414006010 tx=0x7f741400f7b0 comp rx=0 tx=0).stop 2023-11-16T16:38:24.490 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.488+0000 7f742c1e2700 1 -- 172.21.15.1:0/456715779 shutdown_connections 2023-11-16T16:38:24.490 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.488+0000 7f742c1e2700 1 --2- 172.21.15.1:0/456715779 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f74100a11e0 0x7f74100a3690 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:24.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.488+0000 7f742c1e2700 1 --2- 172.21.15.1:0/456715779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7424105fb0 0x7f742419d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:24.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.488+0000 7f742c1e2700 1 --2- 172.21.15.1:0/456715779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7424100d70 0x7f7424196000 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:24.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.488+0000 7f742c1e2700 1 --2- 172.21.15.1:0/456715779 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7424103690 0x7f7424196540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:24.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.488+0000 7f742c1e2700 1 -- 172.21.15.1:0/456715779 >> 172.21.15.1:0/456715779 conn(0x7f74240fa7d0 msgr2=0x7f74241019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:24.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.488+0000 7f742c1e2700 1 -- 172.21.15.1:0/456715779 shutdown_connections 2023-11-16T16:38:24.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.488+0000 7f742c1e2700 1 -- 172.21.15.1:0/456715779 wait complete. 2023-11-16T16:38:24.492 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 23 2023-11-16T16:38:24.502 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-16T16:38:24.631 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.627+0000 7f84b885e700 1 -- 172.21.15.1:0/1647074546 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84b41003b0 msgr2=0x7f84b4100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:24.631 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.627+0000 7f84b885e700 1 --2- 172.21.15.1:0/1647074546 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84b41003b0 0x7f84b4100800 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f84a8009990 tx=0x7f84a8009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:24.631 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.627+0000 7f84b885e700 1 -- 172.21.15.1:0/1647074546 shutdown_connections 2023-11-16T16:38:24.632 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.627+0000 7f84b885e700 1 --2- 172.21.15.1:0/1647074546 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84b4100d40 0x7f84b4107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:24.632 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.627+0000 7f84b885e700 1 --2- 172.21.15.1:0/1647074546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84b40ff1c0 0x7f84b40ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:24.632 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.627+0000 7f84b885e700 1 --2- 172.21.15.1:0/1647074546 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84b41003b0 0x7f84b4100800 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:24.632 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.627+0000 7f84b885e700 1 -- 172.21.15.1:0/1647074546 >> 172.21.15.1:0/1647074546 conn(0x7f84b40fa7c0 msgr2=0x7f84b40fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:24.632 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.627+0000 7f84b885e700 1 -- 172.21.15.1:0/1647074546 shutdown_connections 2023-11-16T16:38:24.632 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.627+0000 7f84b885e700 1 -- 172.21.15.1:0/1647074546 wait complete. 2023-11-16T16:38:24.632 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.627+0000 7f84b885e700 1 Processor -- start 2023-11-16T16:38:24.632 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.627+0000 7f84b885e700 1 -- start start 2023-11-16T16:38:24.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.628+0000 7f84b885e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84b40ff1c0 0x7f84b41960d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:24.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.628+0000 7f84b885e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84b41003b0 0x7f84b4196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:24.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.628+0000 7f84b885e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84b4100d40 0x7f84b419d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:24.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.628+0000 7f84b885e700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b419dc50 con 0x7f84b4100d40 2023-11-16T16:38:24.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.628+0000 7f84b885e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b419dd90 con 0x7f84b40ff1c0 2023-11-16T16:38:24.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.628+0000 7f84b885e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b419df20 con 0x7f84b41003b0 2023-11-16T16:38:24.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.628+0000 7f84b2d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84b4100d40 0x7f84b419d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:24.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.628+0000 7f84b2d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84b4100d40 0x7f84b419d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42296/0 (socket says 172.21.15.1:42296) 2023-11-16T16:38:24.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.628+0000 7f84b2d9d700 1 -- 172.21.15.1:0/3694382309 learned_addr learned my addr 172.21.15.1:0/3694382309 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:24.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.628+0000 7f84b259c700 1 --2- 172.21.15.1:0/3694382309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84b40ff1c0 0x7f84b41960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:24.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.628+0000 7f84b1d9b700 1 --2- 172.21.15.1:0/3694382309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84b41003b0 0x7f84b4196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:24.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.628+0000 7f84b2d9d700 1 -- 172.21.15.1:0/3694382309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84b41003b0 msgr2=0x7f84b4196610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:24.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.628+0000 7f84b2d9d700 1 --2- 172.21.15.1:0/3694382309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84b41003b0 0x7f84b4196610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:24.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.628+0000 7f84b2d9d700 1 -- 172.21.15.1:0/3694382309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84b40ff1c0 msgr2=0x7f84b41960d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:24.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.628+0000 7f84b2d9d700 1 --2- 172.21.15.1:0/3694382309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84b40ff1c0 0x7f84b41960d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:24.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.628+0000 7f84b2d9d700 1 -- 172.21.15.1:0/3694382309 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a80095f0 con 0x7f84b4100d40 2023-11-16T16:38:24.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.628+0000 7f84b1d9b700 1 --2- 172.21.15.1:0/3694382309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84b41003b0 0x7f84b4196610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:38:24.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.628+0000 7f84b2d9d700 1 --2- 172.21.15.1:0/3694382309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84b4100d40 0x7f84b419d680 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f84ac007ee0 tx=0x7f84ac00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:24.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.628+0000 7f84a37fe700 1 -- 172.21.15.1:0/3694382309 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac017070 con 0x7f84b4100d40 2023-11-16T16:38:24.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.628+0000 7f84a37fe700 1 -- 172.21.15.1:0/3694382309 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84ac00f040 con 0x7f84b4100d40 2023-11-16T16:38:24.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.629+0000 7f84b885e700 1 -- 172.21.15.1:0/3694382309 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b410ab70 con 0x7f84b4100d40 2023-11-16T16:38:24.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.629+0000 7f84a37fe700 1 -- 172.21.15.1:0/3694382309 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac005740 con 0x7f84b4100d40 2023-11-16T16:38:24.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.629+0000 7f84b885e700 1 -- 172.21.15.1:0/3694382309 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84b410a9e0 con 0x7f84b4100d40 2023-11-16T16:38:24.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.629+0000 7f84b885e700 1 -- 172.21.15.1:0/3694382309 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84b4107340 con 0x7f84b4100d40 2023-11-16T16:38:24.636 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.630+0000 7f84a37fe700 1 -- 172.21.15.1:0/3694382309 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f84ac0058a0 con 0x7f84b4100d40 2023-11-16T16:38:24.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.631+0000 7f84a37fe700 1 --2- 172.21.15.1:0/3694382309 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f849c0a11e0 0x7f849c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:24.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.631+0000 7f84b259c700 1 --2- 172.21.15.1:0/3694382309 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f849c0a11e0 0x7f849c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:24.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.631+0000 7f84a37fe700 1 -- 172.21.15.1:0/3694382309 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f84ac0cc310 con 0x7f84b4100d40 2023-11-16T16:38:24.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.632+0000 7f84b259c700 1 --2- 172.21.15.1:0/3694382309 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f849c0a11e0 0x7f849c0a3690 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f84b4100210 tx=0x7f84a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:24.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.635+0000 7f84a37fe700 1 -- 172.21.15.1:0/3694382309 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f84ac095900 con 0x7f84b4100d40 2023-11-16T16:38:24.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.835+0000 7f84b885e700 1 -- 172.21.15.1:0/3694382309 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84b402cfe0 con 0x7f84b4100d40 2023-11-16T16:38:24.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.837+0000 7f84a37fe700 1 -- 172.21.15.1:0/3694382309 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22469 (secure 0 0 0) 0x7f84ac098f20 con 0x7f84b4100d40 2023-11-16T16:38:24.839 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:38:24.840 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":86,"fsid":"35d9b010-849d-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:30:06.217778+0000","modified":"2023-11-16T16:38:03.179244+0000","last_up_change":"2023-11-16T16:36:31.735403+0000","last_in_change":"2023-11-16T16:36:14.083021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:33:15.916723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-16T16:37:12.158031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-16T16:37:13.272343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a8480507-5275-4e52-953b-626110beb436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6803","nonce":2469671609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6805","nonce":2469671609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6809","nonce":2469671609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6807","nonce":2469671609}]},"public_addr":"172.21.15.1:6803/2469671609","cluster_addr":"172.21.15.1:6805/2469671609","heartbeat_back_addr":"172.21.15.1:6809/2469671609","heartbeat_front_addr":"172.21.15.1:6807/2469671609","state":["exists","up"]},{"osd":1,"uuid":"1d011db2-99d1-4575-bdc5-51f88a308fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6801","nonce":247994796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6803","nonce":247994796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6807","nonce":247994796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6805","nonce":247994796}]},"public_addr":"172.21.15.26:6801/247994796","cluster_addr":"172.21.15.26:6803/247994796","heartbeat_back_addr":"172.21.15.26:6807/247994796","heartbeat_front_addr":"172.21.15.26:6805/247994796","state":["exists","up"]},{"osd":2,"uuid":"4d7a2bd1-3c8e-4546-a243-f0354473b031","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6801","nonce":1659316290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6803","nonce":1659316290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6807","nonce":1659316290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6805","nonce":1659316290}]},"public_addr":"172.21.15.53:6801/1659316290","cluster_addr":"172.21.15.53:6803/1659316290","heartbeat_back_addr":"172.21.15.53:6807/1659316290","heartbeat_front_addr":"172.21.15.53:6805/1659316290","state":["exists","up"]},{"osd":3,"uuid":"9c290fd9-e121-4a81-9687-b0eb93debd86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6811","nonce":2728987281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6813","nonce":2728987281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6817","nonce":2728987281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6815","nonce":2728987281}]},"public_addr":"172.21.15.1:6811/2728987281","cluster_addr":"172.21.15.1:6813/2728987281","heartbeat_back_addr":"172.21.15.1:6817/2728987281","heartbeat_front_addr":"172.21.15.1:6815/2728987281","state":["exists","up"]},{"osd":4,"uuid":"533226bc-cc57-4f15-9e29-820294d57e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6809","nonce":2954740024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6811","nonce":2954740024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6815","nonce":2954740024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6813","nonce":2954740024}]},"public_addr":"172.21.15.26:6809/2954740024","cluster_addr":"172.21.15.26:6811/2954740024","heartbeat_back_addr":"172.21.15.26:6815/2954740024","heartbeat_front_addr":"172.21.15.26:6813/2954740024","state":["exists","up"]},{"osd":5,"uuid":"d3ab25eb-e1d4-4557-bc89-efc9072401ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6809","nonce":3966666318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6811","nonce":3966666318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6815","nonce":3966666318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6813","nonce":3966666318}]},"public_addr":"172.21.15.53:6809/3966666318","cluster_addr":"172.21.15.53:6811/3966666318","heartbeat_back_addr":"172.21.15.53:6815/3966666318","heartbeat_front_addr":"172.21.15.53:6813/3966666318","state":["exists","up"]},{"osd":6,"uuid":"180f506e-afb8-4043-b0b4-2dc86f6d4bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6819","nonce":3364032646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6821","nonce":3364032646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6825","nonce":3364032646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6823","nonce":3364032646}]},"public_addr":"172.21.15.1:6819/3364032646","cluster_addr":"172.21.15.1:6821/3364032646","heartbeat_back_addr":"172.21.15.1:6825/3364032646","heartbeat_front_addr":"172.21.15.1:6823/3364032646","state":["exists","up"]},{"osd":7,"uuid":"0d837bac-194f-498d-b7fb-b3147e992329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6817","nonce":3766901391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6819","nonce":3766901391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6823","nonce":3766901391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6821","nonce":3766901391}]},"public_addr":"172.21.15.26:6817/3766901391","cluster_addr":"172.21.15.26:6819/3766901391","heartbeat_back_addr":"172.21.15.26:6823/3766901391","heartbeat_front_addr":"172.21.15.26:6821/3766901391","state":["exists","up"]},{"osd":8,"uuid":"933f231a-7454-47b0-9291-f1fa64f0baec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6817","nonce":3224335871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6819","nonce":3224335871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6823","nonce":3224335871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6821","nonce":3224335871}]},"public_addr":"172.21.15.53:6817/3224335871","cluster_addr":"172.21.15.53:6819/3224335871","heartbeat_back_addr":"172.21.15.53:6823/3224335871","heartbeat_front_addr":"172.21.15.53:6821/3224335871","state":["exists","up"]},{"osd":9,"uuid":"245fcbb3-c651-4703-ab80-309dc98bd983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6827","nonce":1391199844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6829","nonce":1391199844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6833","nonce":1391199844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6831","nonce":1391199844}]},"public_addr":"172.21.15.1:6827/1391199844","cluster_addr":"172.21.15.1:6829/1391199844","heartbeat_back_addr":"172.21.15.1:6833/1391199844","heartbeat_front_addr":"172.21.15.1:6831/1391199844","state":["exists","up"]},{"osd":10,"uuid":"2c090e7a-5240-4d52-ae4f-e94fe05383d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6825","nonce":1773545447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6827","nonce":1773545447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6831","nonce":1773545447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6829","nonce":1773545447}]},"public_addr":"172.21.15.26:6825/1773545447","cluster_addr":"172.21.15.26:6827/1773545447","heartbeat_back_addr":"172.21.15.26:6831/1773545447","heartbeat_front_addr":"172.21.15.26:6829/1773545447","state":["exists","up"]},{"osd":11,"uuid":"4511e32d-e469-45e1-8310-0ff400cb7b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6825","nonce":2564669546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6827","nonce":2564669546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6831","nonce":2564669546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6829","nonce":2564669546}]},"public_addr":"172.21.15.53:6825/2564669546","cluster_addr":"172.21.15.53:6827/2564669546","heartbeat_back_addr":"172.21.15.53:6831/2564669546","heartbeat_front_addr":"172.21.15.53:6829/2564669546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:28.140250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:49.885461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:12.521487+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:34.926197+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:56.815594+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:18.387751+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:40.587106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:02.319370+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:23.832298+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:46.338040+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:08.730414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:30.050070+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1494336948":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6801/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6800/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6800/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/1641976669":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/2895923638":"2023-11-17T16:30:35.064289+0000","172.21.15.1:6801/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/78811711":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/4131957708":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/344403698":"2023-11-17T16:30:35.064289+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:38:24.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.839+0000 7f84b885e700 1 -- 172.21.15.1:0/3694382309 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f849c0a11e0 msgr2=0x7f849c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:24.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.839+0000 7f84b885e700 1 --2- 172.21.15.1:0/3694382309 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f849c0a11e0 0x7f849c0a3690 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f84b4100210 tx=0x7f84a4009260 comp rx=0 tx=0).stop 2023-11-16T16:38:24.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.839+0000 7f84b885e700 1 -- 172.21.15.1:0/3694382309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84b4100d40 msgr2=0x7f84b419d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:24.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.839+0000 7f84b885e700 1 --2- 172.21.15.1:0/3694382309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84b4100d40 0x7f84b419d680 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f84ac007ee0 tx=0x7f84ac00d3b0 comp rx=0 tx=0).stop 2023-11-16T16:38:24.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.839+0000 7f84b885e700 1 -- 172.21.15.1:0/3694382309 shutdown_connections 2023-11-16T16:38:24.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.839+0000 7f84b885e700 1 --2- 172.21.15.1:0/3694382309 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f849c0a11e0 0x7f849c0a3690 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:24.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.839+0000 7f84b885e700 1 --2- 172.21.15.1:0/3694382309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84b4100d40 0x7f84b419d680 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:24.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.839+0000 7f84b885e700 1 --2- 172.21.15.1:0/3694382309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84b40ff1c0 0x7f84b41960d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:24.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.839+0000 7f84b885e700 1 --2- 172.21.15.1:0/3694382309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f84b41003b0 0x7f84b4196610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:24.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.839+0000 7f84b885e700 1 -- 172.21.15.1:0/3694382309 >> 172.21.15.1:0/3694382309 conn(0x7f84b40fa7c0 msgr2=0x7f84b40fcab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:24.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.839+0000 7f84b885e700 1 -- 172.21.15.1:0/3694382309 shutdown_connections 2023-11-16T16:38:24.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.839+0000 7f84b885e700 1 -- 172.21.15.1:0/3694382309 wait complete. 2023-11-16T16:38:24.852 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:24 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3771097311' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:38:24.853 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:24 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/2656437413' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:38:24.853 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:24 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/456715779' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:38:24.854 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:38:24.980 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.978+0000 7fdeec89b700 1 -- 172.21.15.1:0/1486170213 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdee81003b0 msgr2=0x7fdee8100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:24.980 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.978+0000 7fdeec89b700 1 --2- 172.21.15.1:0/1486170213 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdee81003b0 0x7fdee8100800 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fded0009990 tx=0x7fded0009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:24.981 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.978+0000 7fdeec89b700 1 -- 172.21.15.1:0/1486170213 shutdown_connections 2023-11-16T16:38:24.981 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.978+0000 7fdeec89b700 1 --2- 172.21.15.1:0/1486170213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdee8100d40 0x7fdee8107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:24.981 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.978+0000 7fdeec89b700 1 --2- 172.21.15.1:0/1486170213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdee80ff1c0 0x7fdee80ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:24.981 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.978+0000 7fdeec89b700 1 --2- 172.21.15.1:0/1486170213 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdee81003b0 0x7fdee8100800 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:24.981 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.978+0000 7fdeec89b700 1 -- 172.21.15.1:0/1486170213 >> 172.21.15.1:0/1486170213 conn(0x7fdee80fa7c0 msgr2=0x7fdee80fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:24.981 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.978+0000 7fdeec89b700 1 -- 172.21.15.1:0/1486170213 shutdown_connections 2023-11-16T16:38:24.981 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.978+0000 7fdeec89b700 1 -- 172.21.15.1:0/1486170213 wait complete. 2023-11-16T16:38:24.984 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.979+0000 7fdeec89b700 1 Processor -- start 2023-11-16T16:38:24.984 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.979+0000 7fdeec89b700 1 -- start start 2023-11-16T16:38:24.984 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.979+0000 7fdeec89b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdee80ff1c0 0x7fdee8103150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:24.984 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.979+0000 7fdeec89b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdee81003b0 0x7fdee8101790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:24.984 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.979+0000 7fdeec89b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdee8100d40 0x7fdee8101cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:24.984 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.979+0000 7fdeec89b700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee8102470 con 0x7fdee8100d40 2023-11-16T16:38:24.984 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.979+0000 7fdeec89b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee81025e0 con 0x7fdee80ff1c0 2023-11-16T16:38:24.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.979+0000 7fdeec89b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee81a0120 con 0x7fdee81003b0 2023-11-16T16:38:24.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.979+0000 7fdee6d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdee8100d40 0x7fdee8101cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:24.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.979+0000 7fdee6d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdee8100d40 0x7fdee8101cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42310/0 (socket says 172.21.15.1:42310) 2023-11-16T16:38:24.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.979+0000 7fdee6d9d700 1 -- 172.21.15.1:0/2928545345 learned_addr learned my addr 172.21.15.1:0/2928545345 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:24.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.979+0000 7fdee659c700 1 --2- 172.21.15.1:0/2928545345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdee80ff1c0 0x7fdee8103150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:24.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.979+0000 7fdee5d9b700 1 --2- 172.21.15.1:0/2928545345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdee81003b0 0x7fdee8101790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:24.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.980+0000 7fdee6d9d700 1 -- 172.21.15.1:0/2928545345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdee81003b0 msgr2=0x7fdee8101790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:24.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.980+0000 7fdee6d9d700 1 --2- 172.21.15.1:0/2928545345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdee81003b0 0x7fdee8101790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:24.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.980+0000 7fdee6d9d700 1 -- 172.21.15.1:0/2928545345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdee80ff1c0 msgr2=0x7fdee8103150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:24.986 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.980+0000 7fdee6d9d700 1 --2- 172.21.15.1:0/2928545345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdee80ff1c0 0x7fdee8103150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:24.986 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.980+0000 7fdee6d9d700 1 -- 172.21.15.1:0/2928545345 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded00095f0 con 0x7fdee8100d40 2023-11-16T16:38:24.986 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.980+0000 7fdee6d9d700 1 --2- 172.21.15.1:0/2928545345 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdee8100d40 0x7fdee8101cd0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdee000ada0 tx=0x7fdee000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:24.986 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.980+0000 7fdedf7fe700 1 -- 172.21.15.1:0/2928545345 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee000f070 con 0x7fdee8100d40 2023-11-16T16:38:24.986 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.980+0000 7fdedf7fe700 1 -- 172.21.15.1:0/2928545345 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdee0007b30 con 0x7fdee8100d40 2023-11-16T16:38:24.986 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.980+0000 7fdeec89b700 1 -- 172.21.15.1:0/2928545345 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee810ab30 con 0x7fdee8100d40 2023-11-16T16:38:24.986 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.980+0000 7fdedf7fe700 1 -- 172.21.15.1:0/2928545345 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee0005280 con 0x7fdee8100d40 2023-11-16T16:38:24.986 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.980+0000 7fdeec89b700 1 -- 172.21.15.1:0/2928545345 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee810a9a0 con 0x7fdee8100d40 2023-11-16T16:38:24.986 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.981+0000 7fdeec89b700 1 -- 172.21.15.1:0/2928545345 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee8067af0 con 0x7fdee8100d40 2023-11-16T16:38:24.986 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.982+0000 7fdedf7fe700 1 -- 172.21.15.1:0/2928545345 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fdee000a090 con 0x7fdee8100d40 2023-11-16T16:38:24.987 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.983+0000 7fdedf7fe700 1 --2- 172.21.15.1:0/2928545345 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fded40a11e0 0x7fded40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:24.987 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.983+0000 7fdee659c700 1 --2- 172.21.15.1:0/2928545345 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fded40a11e0 0x7fded40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:24.988 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.983+0000 7fdedf7fe700 1 -- 172.21.15.1:0/2928545345 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdee00cc5b0 con 0x7fdee8100d40 2023-11-16T16:38:24.988 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.983+0000 7fdee659c700 1 --2- 172.21.15.1:0/2928545345 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fded40a11e0 0x7fded40a3690 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdee8100210 tx=0x7fded800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:24.988 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:24.986+0000 7fdedf7fe700 1 -- 172.21.15.1:0/2928545345 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fdee0099680 con 0x7fdee8100d40 2023-11-16T16:38:25.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:24 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/3771097311' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:38:25.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:24 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/2656437413' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:38:25.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:24 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/456715779' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:38:25.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:24 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/3771097311' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:38:25.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:24 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/2656437413' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:38:25.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:24 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/456715779' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:38:25.236 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.234+0000 7fdeec89b700 1 -- 172.21.15.1:0/2928545345 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdee81a0be0 con 0x7fdee8100d40 2023-11-16T16:38:25.236 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.235+0000 7fdedf7fe700 1 -- 172.21.15.1:0/2928545345 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10774 (secure 0 0 0) 0x7fdee0099210 con 0x7fdee8100d40 2023-11-16T16:38:25.237 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:38:25.237 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2587563684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2587563684},{"type":"v1","addr":"172.21.15.1:6835","nonce":2587563684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/277936168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":277936168},{"type":"v1","addr":"172.21.15.1:6837","nonce":277936168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/1172575045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1172575045},{"type":"v1","addr":"172.21.15.1:6839","nonce":1172575045}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/2631175899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2631175899},{"type":"v1","addr":"172.21.15.1:6841","nonce":2631175899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3693905267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3693905267},{"type":"v1","addr":"172.21.15.53:6833","nonce":3693905267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/164668033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":164668033},{"type":"v1","addr":"172.21.15.26:6833","nonce":164668033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-16T16:37:13.918943+0000","modified":"2023-11-16T16:37:19.292898+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2140542192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2140542192},{"type":"v1","addr":"172.21.15.53:6835","nonce":2140542192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1010760275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1010760275},{"type":"v1","addr":"172.21.15.26:6835","nonce":1010760275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3197060302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3197060302},{"type":"v1","addr":"172.21.15.53:6837","nonce":3197060302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6837/1453504863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1453504863},{"type":"v1","addr":"172.21.15.26:6837","nonce":1453504863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.53:6839/1044753024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1044753024},{"type":"v1","addr":"172.21.15.53:6839","nonce":1044753024}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6839/2192205949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2192205949},{"type":"v1","addr":"172.21.15.26:6839","nonce":2192205949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-16T16:38:25.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.237+0000 7fdeec89b700 1 -- 172.21.15.1:0/2928545345 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fded40a11e0 msgr2=0x7fded40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:25.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.237+0000 7fdeec89b700 1 --2- 172.21.15.1:0/2928545345 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fded40a11e0 0x7fded40a3690 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdee8100210 tx=0x7fded800b410 comp rx=0 tx=0).stop 2023-11-16T16:38:25.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.237+0000 7fdeec89b700 1 -- 172.21.15.1:0/2928545345 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdee8100d40 msgr2=0x7fdee8101cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:25.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.237+0000 7fdeec89b700 1 --2- 172.21.15.1:0/2928545345 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdee8100d40 0x7fdee8101cd0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdee000ada0 tx=0x7fdee000c330 comp rx=0 tx=0).stop 2023-11-16T16:38:25.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.237+0000 7fdeec89b700 1 -- 172.21.15.1:0/2928545345 shutdown_connections 2023-11-16T16:38:25.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.237+0000 7fdeec89b700 1 --2- 172.21.15.1:0/2928545345 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fded40a11e0 0x7fded40a3690 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.237+0000 7fdeec89b700 1 --2- 172.21.15.1:0/2928545345 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdee8100d40 0x7fdee8101cd0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.237+0000 7fdeec89b700 1 --2- 172.21.15.1:0/2928545345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdee80ff1c0 0x7fdee8103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.237+0000 7fdeec89b700 1 --2- 172.21.15.1:0/2928545345 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdee81003b0 0x7fdee8101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.237+0000 7fdeec89b700 1 -- 172.21.15.1:0/2928545345 >> 172.21.15.1:0/2928545345 conn(0x7fdee80fa7c0 msgr2=0x7fdee8067e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:25.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.237+0000 7fdeec89b700 1 -- 172.21.15.1:0/2928545345 shutdown_connections 2023-11-16T16:38:25.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.237+0000 7fdeec89b700 1 -- 172.21.15.1:0/2928545345 wait complete. 2023-11-16T16:38:25.240 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 23 2023-11-16T16:38:25.251 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-11-16T16:37:13.918943+0000', 'modified': '2023-11-16T16:37:19.292898+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24521, 'mds_1': 24493, 'mds_2': 24482}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24463': {'gid': 24463, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.53:6835/2140542192', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6834', 'nonce': 2140542192}, {'type': 'v1', 'addr': '172.21.15.53:6835', 'nonce': 2140542192}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24467': {'gid': 24467, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.26:6835/1010760275', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 1010760275}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 1010760275}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24472': {'gid': 24472, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.53:6837/3197060302', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6836', 'nonce': 3197060302}, {'type': 'v1', 'addr': '172.21.15.53:6837', 'nonce': 3197060302}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24482': {'gid': 24482, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.26:6837/1453504863', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6836', 'nonce': 1453504863}, {'type': 'v1', 'addr': '172.21.15.26:6837', 'nonce': 1453504863}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24493': {'gid': 24493, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.53:6839/1044753024', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6838', 'nonce': 1044753024}, {'type': 'v1', 'addr': '172.21.15.53:6839', 'nonce': 1044753024}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24521': {'gid': 24521, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6839/2192205949', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6838', 'nonce': 2192205949}, {'type': 'v1', 'addr': '172.21.15.26:6839', 'nonce': 2192205949}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-16T16:38:25.251 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-11-16T16:38:25.252 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-11-16T16:38:25.380 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.376+0000 7f672e6ed700 1 -- 172.21.15.1:0/2436055452 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67280ff110 msgr2=0x7f67280ff520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:25.380 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.376+0000 7f672e6ed700 1 --2- 172.21.15.1:0/2436055452 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67280ff110 0x7f67280ff520 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6718009990 tx=0x7f6718009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.380 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.376+0000 7f672e6ed700 1 -- 172.21.15.1:0/2436055452 shutdown_connections 2023-11-16T16:38:25.380 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.376+0000 7f672e6ed700 1 --2- 172.21.15.1:0/2436055452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6728100c90 0x7f6728107550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.381 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.376+0000 7f672e6ed700 1 --2- 172.21.15.1:0/2436055452 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67280ff110 0x7f67280ff520 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.381 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.376+0000 7f672e6ed700 1 --2- 172.21.15.1:0/2436055452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6728100300 0x7f6728100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.381 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.376+0000 7f672e6ed700 1 -- 172.21.15.1:0/2436055452 >> 172.21.15.1:0/2436055452 conn(0x7f67280fa6f0 msgr2=0x7f67280fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:25.381 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.376+0000 7f672e6ed700 1 -- 172.21.15.1:0/2436055452 shutdown_connections 2023-11-16T16:38:25.381 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.376+0000 7f672e6ed700 1 -- 172.21.15.1:0/2436055452 wait complete. 2023-11-16T16:38:25.381 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.377+0000 7f672e6ed700 1 Processor -- start 2023-11-16T16:38:25.381 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.377+0000 7f672e6ed700 1 -- start start 2023-11-16T16:38:25.381 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.377+0000 7f672e6ed700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67280ff110 0x7f6728198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.381 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.377+0000 7f672e6ed700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6728100300 0x7f6728198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.377+0000 7f672e6ed700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6728100c90 0x7f672819f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.377+0000 7f672e6ed700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672819fd80 con 0x7f6728100300 2023-11-16T16:38:25.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.377+0000 7f672e6ed700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672819fec0 con 0x7f6728100c90 2023-11-16T16:38:25.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.377+0000 7f672e6ed700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67281a0050 con 0x7f67280ff110 2023-11-16T16:38:25.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.377+0000 7f67277fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6728100300 0x7f6728198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.377+0000 7f67277fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6728100300 0x7f6728198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42322/0 (socket says 172.21.15.1:42322) 2023-11-16T16:38:25.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.377+0000 7f67277fe700 1 -- 172.21.15.1:0/318449880 learned_addr learned my addr 172.21.15.1:0/318449880 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:25.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.377+0000 7f6727fff700 1 --2- 172.21.15.1:0/318449880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67280ff110 0x7f6728198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.378+0000 7f672cc8a700 1 --2- 172.21.15.1:0/318449880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6728100c90 0x7f672819f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.378+0000 7f67277fe700 1 -- 172.21.15.1:0/318449880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67280ff110 msgr2=0x7f6728198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:25.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.378+0000 7f67277fe700 1 --2- 172.21.15.1:0/318449880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67280ff110 0x7f6728198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.378+0000 7f67277fe700 1 -- 172.21.15.1:0/318449880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6728100c90 msgr2=0x7f672819f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:25.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.378+0000 7f67277fe700 1 --2- 172.21.15.1:0/318449880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6728100c90 0x7f672819f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.378+0000 7f67277fe700 1 -- 172.21.15.1:0/318449880 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67180095f0 con 0x7f6728100300 2023-11-16T16:38:25.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.378+0000 7f67277fe700 1 --2- 172.21.15.1:0/318449880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6728100300 0x7f6728198740 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f671c00b530 tx=0x7f671c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:25.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.378+0000 7f67257fa700 1 -- 172.21.15.1:0/318449880 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c011830 con 0x7f6728100300 2023-11-16T16:38:25.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.378+0000 7f67257fa700 1 -- 172.21.15.1:0/318449880 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f671c011e50 con 0x7f6728100300 2023-11-16T16:38:25.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.378+0000 7f672e6ed700 1 -- 172.21.15.1:0/318449880 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f672810ab40 con 0x7f6728100300 2023-11-16T16:38:25.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.378+0000 7f67257fa700 1 -- 172.21.15.1:0/318449880 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c00f2e0 con 0x7f6728100300 2023-11-16T16:38:25.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.378+0000 7f672e6ed700 1 -- 172.21.15.1:0/318449880 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f672810a9b0 con 0x7f6728100300 2023-11-16T16:38:25.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.380+0000 7f67257fa700 1 -- 172.21.15.1:0/318449880 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f671c010380 con 0x7f6728100300 2023-11-16T16:38:25.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.381+0000 7f67257fa700 1 --2- 172.21.15.1:0/318449880 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f67100a11e0 0x7f67100a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.381+0000 7f67257fa700 1 -- 172.21.15.1:0/318449880 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f671c0cc530 con 0x7f6728100300 2023-11-16T16:38:25.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.381+0000 7f6727fff700 1 --2- 172.21.15.1:0/318449880 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f67100a11e0 0x7f67100a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.381+0000 7f6727fff700 1 --2- 172.21.15.1:0/318449880 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f67100a11e0 0x7f67100a3690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f671800b5c0 tx=0x7f6718005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:25.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.381+0000 7f673a879b80 1 Processor -- start 2023-11-16T16:38:25.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.382+0000 7f673a879b80 1 -- start start 2023-11-16T16:38:25.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.382+0000 7f673a879b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eeb5c6bb0 0x556eeb5c6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.382+0000 7f673a879b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556eeb5c7500 0x556eeb5c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.393 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.382+0000 7f673a879b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556eeb5c80c0 0x556eeb5c8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.393 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.382+0000 7f673a879b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eeb4cc010 con 0x556eeb5c80c0 2023-11-16T16:38:25.393 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.382+0000 7f673a879b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eeb51e850 con 0x556eeb5c7500 2023-11-16T16:38:25.393 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.382+0000 7f673a879b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eeb517ce0 con 0x556eeb5c6bb0 2023-11-16T16:38:25.393 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.382+0000 7f672cc8a700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556eeb5c80c0 0x556eeb5c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.393 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.382+0000 7f672cc8a700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556eeb5c80c0 0x556eeb5c8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42332/0 (socket says 172.21.15.1:42332) 2023-11-16T16:38:25.393 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.382+0000 7f672cc8a700 1 -- 172.21.15.1:0/1632823506 learned_addr learned my addr 172.21.15.1:0/1632823506 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:25.393 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.382+0000 7f6727fff700 1 --2- 172.21.15.1:0/1632823506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eeb5c6bb0 0x556eeb5c6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.393 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.382+0000 7f672cc8a700 1 -- 172.21.15.1:0/1632823506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eeb5c6bb0 msgr2=0x556eeb5c6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:25.393 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.382+0000 7f672cc8a700 1 --2- 172.21.15.1:0/1632823506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eeb5c6bb0 0x556eeb5c6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.382+0000 7f67277fe700 1 --2- 172.21.15.1:0/1632823506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556eeb5c7500 0x556eeb5c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.382+0000 7f672cc8a700 1 -- 172.21.15.1:0/1632823506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556eeb5c7500 msgr2=0x556eeb5c7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:25.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.382+0000 7f672cc8a700 1 --2- 172.21.15.1:0/1632823506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556eeb5c7500 0x556eeb5c7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.382+0000 7f672cc8a700 1 -- 172.21.15.1:0/1632823506 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556eeb3995e0 con 0x556eeb5c80c0 2023-11-16T16:38:25.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.382+0000 7f672cc8a700 1 --2- 172.21.15.1:0/1632823506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556eeb5c80c0 0x556eeb5c8530 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f672000e190 tx=0x7f672000e4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf8dfa1e260b0c0f server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:25.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.382+0000 7f670e7fc700 1 -- 172.21.15.1:0/1632823506 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6720018070 con 0x556eeb5c80c0 2023-11-16T16:38:25.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.383+0000 7f670e7fc700 1 -- 172.21.15.1:0/1632823506 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6720007c30 con 0x556eeb5c80c0 2023-11-16T16:38:25.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.383+0000 7f670e7fc700 1 -- 172.21.15.1:0/1632823506 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6720014ce0 con 0x556eeb5c80c0 2023-11-16T16:38:25.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.383+0000 7f673a879b80 1 -- 172.21.15.1:0/1632823506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556eeb5c80c0 msgr2=0x556eeb5c8530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:25.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.383+0000 7f673a879b80 1 --2- 172.21.15.1:0/1632823506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556eeb5c80c0 0x556eeb5c8530 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f672000e190 tx=0x7f672000e4a0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.383+0000 7f673a879b80 1 -- 172.21.15.1:0/1632823506 shutdown_connections 2023-11-16T16:38:25.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.383+0000 7f673a879b80 1 --2- 172.21.15.1:0/1632823506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556eeb5c80c0 0x556eeb5c8530 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.383+0000 7f673a879b80 1 --2- 172.21.15.1:0/1632823506 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eeb5c6bb0 0x556eeb5c6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.383+0000 7f673a879b80 1 --2- 172.21.15.1:0/1632823506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556eeb5c7500 0x556eeb5c7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.383+0000 7f673a879b80 1 -- 172.21.15.1:0/1632823506 >> 172.21.15.1:0/1632823506 conn(0x556eeb5bc310 msgr2=0x556eeb5c1620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:25.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.383+0000 7f673a879b80 1 -- 172.21.15.1:0/1632823506 shutdown_connections 2023-11-16T16:38:25.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.383+0000 7f673a879b80 1 -- 172.21.15.1:0/1632823506 wait complete. 2023-11-16T16:38:25.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.384+0000 7f673a879b80 1 -- start start 2023-11-16T16:38:25.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.384+0000 7f673a879b80 1 Processor -- start 2023-11-16T16:38:25.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.384+0000 7f673a879b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eeb5c6bb0 0x556eeb5d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.384+0000 7f673a879b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556eeb5c7500 0x556eeb5d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.384+0000 7f673a879b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556eeb5c80c0 0x556eeb5d7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.384+0000 7f673a879b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eeb4b9890 con 0x556eeb5c7500 2023-11-16T16:38:25.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.384+0000 7f673a879b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eeb4b9530 con 0x556eeb5c80c0 2023-11-16T16:38:25.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.384+0000 7f673a879b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eeb39b8d0 con 0x556eeb5c6bb0 2023-11-16T16:38:25.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.384+0000 7f673a879b80 10 client.0 _ll_register_callbacks cb 0x556eeb4d7730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:38:25.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.384+0000 7f67277fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556eeb5c7500 0x556eeb5d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.384+0000 7f67277fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556eeb5c7500 0x556eeb5d0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42336/0 (socket says 172.21.15.1:42336) 2023-11-16T16:38:25.397 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.384+0000 7f67277fe700 1 -- 172.21.15.1:0/1777135335 learned_addr learned my addr 172.21.15.1:0/1777135335 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:25.397 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.384+0000 7f6727fff700 1 --2- 172.21.15.1:0/1777135335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eeb5c6bb0 0x556eeb5d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.397 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.384+0000 7f672cc8a700 1 --2- 172.21.15.1:0/1777135335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556eeb5c80c0 0x556eeb5d7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.397 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.384+0000 7f67277fe700 1 -- 172.21.15.1:0/1777135335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eeb5c6bb0 msgr2=0x556eeb5d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:25.397 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.384+0000 7f67277fe700 1 --2- 172.21.15.1:0/1777135335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eeb5c6bb0 0x556eeb5d05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.397 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.384+0000 7f67277fe700 1 -- 172.21.15.1:0/1777135335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556eeb5c80c0 msgr2=0x556eeb5d7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:25.397 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.384+0000 7f67277fe700 1 --2- 172.21.15.1:0/1777135335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556eeb5c80c0 0x556eeb5d7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.397 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.384+0000 7f67277fe700 1 -- 172.21.15.1:0/1777135335 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f672000f040 con 0x556eeb5c7500 2023-11-16T16:38:25.397 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.384+0000 7f672cc8a700 1 --2- 172.21.15.1:0/1777135335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556eeb5c80c0 0x556eeb5d7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:38:25.397 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.385+0000 7f67277fe700 1 --2- 172.21.15.1:0/1777135335 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556eeb5c7500 0x556eeb5d0b30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f671c0995b0 tx=0x7f671c0999e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff35d5a451fb4ae server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:25.397 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.385+0000 7f67077fe700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:38:25.398 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.385+0000 7f67077fe700 1 -- 172.21.15.1:0/1777135335 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c098da0 con 0x556eeb5c7500 2023-11-16T16:38:25.398 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.385+0000 7f67077fe700 1 -- 172.21.15.1:0/1777135335 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f671c098f00 con 0x556eeb5c7500 2023-11-16T16:38:25.398 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.385+0000 7f673a879b80 1 -- 172.21.15.1:0/1777135335 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556eeb4da050 con 0x556eeb5c7500 2023-11-16T16:38:25.398 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.385+0000 7f67077fe700 1 -- 172.21.15.1:0/1777135335 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f671c0c6050 con 0x556eeb5c7500 2023-11-16T16:38:25.398 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.385+0000 7f673a879b80 10 client.15060 fetch_fsmap learned FSMap version 23 2023-11-16T16:38:25.398 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.385+0000 7f673a879b80 1 -- 172.21.15.1:0/1777135335 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x556eeb4de140 con 0x556eeb5c7500 2023-11-16T16:38:25.398 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.385+0000 7f67077fe700 1 -- 172.21.15.1:0/1777135335 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556eeb4da050 con 0x556eeb5c7500 2023-11-16T16:38:25.398 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.385+0000 7f67077fe700 1 -- 172.21.15.1:0/1777135335 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f671c0b9140 con 0x556eeb5c7500 2023-11-16T16:38:25.398 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.385+0000 7f673a879b80 10 client.15060 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:38:25.399 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.385+0000 7f673a879b80 10 client.15060 resolve_mds: resolved name 'h' to daemon mds.h 2023-11-16T16:38:25.399 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.385+0000 7f673a879b80 20 client.15060 populate_metadata read hostname 'smithi001' 2023-11-16T16:38:25.399 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.385+0000 7f673a879b80 1 --2- 172.21.15.1:0/1777135335 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x556eeb5db860 0x556eeb5dacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.399 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.385+0000 7f673a879b80 4 client.15060 mds_command: new command op to 24482 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:38:25.399 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.385+0000 7f673a879b80 1 -- 172.21.15.1:0/1777135335 --> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556eeb4e6610 con 0x556eeb5db860 2023-11-16T16:38:25.399 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.386+0000 7f6727fff700 1 --2- 172.21.15.1:0/1777135335 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x556eeb5db860 0x556eeb5dacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.399 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.386+0000 7f67077fe700 10 client.15060 ms_handle_connect on v2:172.21.15.26:6836/1453504863 2023-11-16T16:38:25.399 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.386+0000 7f6727fff700 1 --2- 172.21.15.1:0/1777135335 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x556eeb5db860 0x556eeb5dacd0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=12825bad0e985ba5 server_cookie=4d35864d35338f0d in_seq=0 out_seq=0 2023-11-16T16:38:25.399 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.387+0000 7f67077fe700 1 -- 172.21.15.1:0/1777135335 <== mds.2 v2:172.21.15.26:6836/1453504863 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556eeb4e6610 con 0x556eeb5db860 2023-11-16T16:38:25.399 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.387+0000 7f67077fe700 10 client.15060 handle_command_reply: tid=0 2023-11-16T16:38:25.400 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.387+0000 7f673a879b80 1 client.15060 shutdown 2023-11-16T16:38:25.400 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.387+0000 7f673a879b80 1 -- 172.21.15.1:0/1777135335 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556eeb5c7500 msgr2=0x556eeb5d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:25.400 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.387+0000 7f673a879b80 1 --2- 172.21.15.1:0/1777135335 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556eeb5c7500 0x556eeb5d0b30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f671c0995b0 tx=0x7f671c0999e0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.400 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.387+0000 7f673a879b80 1 -- 172.21.15.1:0/1777135335 shutdown_connections 2023-11-16T16:38:25.400 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.387+0000 7f673a879b80 1 --2- 172.21.15.1:0/1777135335 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x556eeb5db860 0x556eeb5dacd0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.400 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.387+0000 7f673a879b80 1 --2- 172.21.15.1:0/1777135335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556eeb5c80c0 0x556eeb5d7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.400 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.387+0000 7f673a879b80 1 --2- 172.21.15.1:0/1777135335 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eeb5c6bb0 0x556eeb5d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.400 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.387+0000 7f673a879b80 1 --2- 172.21.15.1:0/1777135335 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556eeb5c7500 0x556eeb5d0b30 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.400 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.387+0000 7f67077fe700 0 client.15060 ms_handle_reset on v2:172.21.15.26:6836/1453504863 2023-11-16T16:38:25.401 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.387+0000 7f673a879b80 1 -- 172.21.15.1:0/1777135335 >> 172.21.15.1:0/1777135335 conn(0x556eeb5bc310 msgr2=0x556eeb5c0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:25.401 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.387+0000 7f673a879b80 1 -- 172.21.15.1:0/1777135335 shutdown_connections 2023-11-16T16:38:25.401 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.387+0000 7f673a879b80 1 -- 172.21.15.1:0/1777135335 wait complete. 2023-11-16T16:38:25.401 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.387+0000 7f673a879b80 20 client.15060 trim_cache size 0 max 16384 2023-11-16T16:38:25.413 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.405+0000 7f673a879b80 1 Processor -- start 2023-11-16T16:38:25.413 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.405+0000 7f673a879b80 1 -- start start 2023-11-16T16:38:25.414 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.405+0000 7f673a879b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556eeb5c6bb0 0x556eeb5caf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.414 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.405+0000 7f673a879b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eeb5c7500 0x556eeb5cb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.414 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.405+0000 7f673a879b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556eeb5c80c0 0x556eeb5d7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.414 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.405+0000 7f673a879b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eeb4da050 con 0x556eeb5c6bb0 2023-11-16T16:38:25.414 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.405+0000 7f673a879b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eeb5bc120 con 0x556eeb5c80c0 2023-11-16T16:38:25.414 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.405+0000 7f673a879b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eeb4c1730 con 0x556eeb5c7500 2023-11-16T16:38:25.414 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.405+0000 7f6727fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556eeb5c6bb0 0x556eeb5caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.414 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.405+0000 7f6727fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556eeb5c6bb0 0x556eeb5caf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42342/0 (socket says 172.21.15.1:42342) 2023-11-16T16:38:25.414 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.405+0000 7f6727fff700 1 -- 172.21.15.1:0/1832278963 learned_addr learned my addr 172.21.15.1:0/1832278963 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:25.415 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.405+0000 7f67277fe700 1 --2- 172.21.15.1:0/1832278963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eeb5c7500 0x556eeb5cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.415 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.406+0000 7f6727fff700 1 -- 172.21.15.1:0/1832278963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eeb5c7500 msgr2=0x556eeb5cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:25.415 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.406+0000 7f6727fff700 1 --2- 172.21.15.1:0/1832278963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eeb5c7500 0x556eeb5cb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.415 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.406+0000 7f6727fff700 1 -- 172.21.15.1:0/1832278963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556eeb5c80c0 msgr2=0x556eeb5d7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:25.415 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.406+0000 7f6727fff700 1 --2- 172.21.15.1:0/1832278963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556eeb5c80c0 0x556eeb5d7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.415 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.406+0000 7f6727fff700 1 -- 172.21.15.1:0/1832278963 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556eeb5cf570 con 0x556eeb5c6bb0 2023-11-16T16:38:25.415 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.406+0000 7f6727fff700 1 --2- 172.21.15.1:0/1832278963 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556eeb5c6bb0 0x556eeb5caf30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6718005850 tx=0x7f6718020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c07e0582214817d server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:25.415 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.406+0000 7f670effd700 1 -- 172.21.15.1:0/1832278963 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6718029040 con 0x556eeb5c6bb0 2023-11-16T16:38:25.415 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.406+0000 7f670effd700 1 -- 172.21.15.1:0/1832278963 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f671802a040 con 0x556eeb5c6bb0 2023-11-16T16:38:25.415 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.406+0000 7f670effd700 1 -- 172.21.15.1:0/1832278963 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6718003af0 con 0x556eeb5c6bb0 2023-11-16T16:38:25.415 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.406+0000 7f673a879b80 1 -- 172.21.15.1:0/1832278963 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556eeb5c6bb0 msgr2=0x556eeb5caf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:25.416 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.406+0000 7f673a879b80 1 --2- 172.21.15.1:0/1832278963 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556eeb5c6bb0 0x556eeb5caf30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6718005850 tx=0x7f6718020000 comp rx=0 tx=0).stop 2023-11-16T16:38:25.416 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.406+0000 7f673a879b80 1 -- 172.21.15.1:0/1832278963 shutdown_connections 2023-11-16T16:38:25.416 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.406+0000 7f673a879b80 1 --2- 172.21.15.1:0/1832278963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556eeb5c80c0 0x556eeb5d7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.416 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.406+0000 7f673a879b80 1 --2- 172.21.15.1:0/1832278963 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556eeb5c6bb0 0x556eeb5caf30 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.416 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.407+0000 7f673a879b80 1 --2- 172.21.15.1:0/1832278963 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eeb5c7500 0x556eeb5cb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.416 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.407+0000 7f673a879b80 1 -- 172.21.15.1:0/1832278963 >> 172.21.15.1:0/1832278963 conn(0x556eeb5bc310 msgr2=0x556eeb5dacd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:25.416 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.407+0000 7f673a879b80 1 -- 172.21.15.1:0/1832278963 shutdown_connections 2023-11-16T16:38:25.416 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.407+0000 7f673a879b80 1 -- 172.21.15.1:0/1832278963 wait complete. 2023-11-16T16:38:25.416 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.407+0000 7f673a879b80 1 -- start start 2023-11-16T16:38:25.416 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.407+0000 7f673a879b80 1 Processor -- start 2023-11-16T16:38:25.417 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.407+0000 7f673a879b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556eeb5c6bb0 0x556eeb5d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.417 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.407+0000 7f673a879b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eeb5c7500 0x556eeb5d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.417 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.407+0000 7f673a879b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556eeb5c80c0 0x556eeb62f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.417 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.407+0000 7f673a879b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eeb5d7ca0 con 0x556eeb5c6bb0 2023-11-16T16:38:25.417 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.407+0000 7f673a879b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eeb5d7e10 con 0x556eeb5c80c0 2023-11-16T16:38:25.417 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.407+0000 7f673a879b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556eeb5d7fd0 con 0x556eeb5c7500 2023-11-16T16:38:25.417 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.407+0000 7f673a879b80 10 client.0 _ll_register_callbacks cb 0x7f671c099230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:38:25.417 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.407+0000 7f6727fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556eeb5c6bb0 0x556eeb5d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.417 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.407+0000 7f6727fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556eeb5c6bb0 0x556eeb5d2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42348/0 (socket says 172.21.15.1:42348) 2023-11-16T16:38:25.418 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.407+0000 7f6727fff700 1 -- 172.21.15.1:0/1358805563 learned_addr learned my addr 172.21.15.1:0/1358805563 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:25.418 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.408+0000 7f67277fe700 1 --2- 172.21.15.1:0/1358805563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eeb5c7500 0x556eeb5d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.418 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.408+0000 7f6727fff700 1 -- 172.21.15.1:0/1358805563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eeb5c7500 msgr2=0x556eeb5d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:25.418 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.408+0000 7f6727fff700 1 --2- 172.21.15.1:0/1358805563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eeb5c7500 0x556eeb5d2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.418 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.408+0000 7f6727fff700 1 -- 172.21.15.1:0/1358805563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556eeb5c80c0 msgr2=0x556eeb62f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:25.418 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.408+0000 7f6727fff700 1 --2- 172.21.15.1:0/1358805563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556eeb5c80c0 0x556eeb62f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.418 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.408+0000 7f6727fff700 1 -- 172.21.15.1:0/1358805563 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6728107e80 con 0x556eeb5c6bb0 2023-11-16T16:38:25.418 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.408+0000 7f67277fe700 1 --2- 172.21.15.1:0/1358805563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eeb5c7500 0x556eeb5d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:38:25.418 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.408+0000 7f6727fff700 1 --2- 172.21.15.1:0/1358805563 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556eeb5c6bb0 0x556eeb5d2730 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6718009990 tx=0x7f6718034310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f6c48e27e995adb server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:25.418 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.408+0000 7f67077fe700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:38:25.419 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.408+0000 7f67077fe700 1 -- 172.21.15.1:0/1358805563 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6718003a90 con 0x556eeb5c6bb0 2023-11-16T16:38:25.419 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.408+0000 7f67077fe700 1 -- 172.21.15.1:0/1358805563 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f671800be60 con 0x556eeb5c6bb0 2023-11-16T16:38:25.419 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.408+0000 7f673a879b80 1 -- 172.21.15.1:0/1358805563 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556eeb5b9ad0 con 0x556eeb5c6bb0 2023-11-16T16:38:25.419 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.408+0000 7f67077fe700 1 -- 172.21.15.1:0/1358805563 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6718003bf0 con 0x556eeb5c6bb0 2023-11-16T16:38:25.419 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.408+0000 7f673a879b80 10 client.15066 fetch_fsmap learned FSMap version 23 2023-11-16T16:38:25.419 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.408+0000 7f673a879b80 1 -- 172.21.15.1:0/1358805563 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x556eeb345d90 con 0x556eeb5c6bb0 2023-11-16T16:38:25.419 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.408+0000 7f67077fe700 1 -- 172.21.15.1:0/1358805563 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556eeb5b9ad0 con 0x556eeb5c6bb0 2023-11-16T16:38:25.419 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.408+0000 7f67077fe700 1 -- 172.21.15.1:0/1358805563 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6718004c40 con 0x556eeb5c6bb0 2023-11-16T16:38:25.420 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.409+0000 7f673a879b80 10 client.15066 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:38:25.420 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.409+0000 7f673a879b80 10 client.15066 resolve_mds: resolved name 'h' to daemon mds.h 2023-11-16T16:38:25.420 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.409+0000 7f673a879b80 20 client.15066 populate_metadata read hostname 'smithi001' 2023-11-16T16:38:25.420 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.409+0000 7f673a879b80 1 --2- 172.21.15.1:0/1358805563 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x556eeb6344c0 0x556eeb630e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.420 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.409+0000 7f673a879b80 4 client.15066 mds_command: new command op to 24482 tid=0[{"prefix": "status"}] 2023-11-16T16:38:25.420 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.409+0000 7f673a879b80 1 -- 172.21.15.1:0/1358805563 --> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556eeb366f20 con 0x556eeb6344c0 2023-11-16T16:38:25.420 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.409+0000 7f67277fe700 1 --2- 172.21.15.1:0/1358805563 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x556eeb6344c0 0x556eeb630e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.420 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.409+0000 7f67277fe700 1 --2- 172.21.15.1:0/1358805563 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x556eeb6344c0 0x556eeb630e10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=56cf15168fa7fba5 server_cookie=456bdc794300f3c8 in_seq=0 out_seq=0 2023-11-16T16:38:25.420 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.409+0000 7f67077fe700 10 client.15066 ms_handle_connect on v2:172.21.15.26:6836/1453504863 2023-11-16T16:38:25.420 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.409+0000 7f67077fe700 1 -- 172.21.15.1:0/1358805563 <== mds.2 v2:172.21.15.26:6836/1453504863 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x556eeb366f20 con 0x556eeb6344c0 2023-11-16T16:38:25.420 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.409+0000 7f67077fe700 10 client.15066 handle_command_reply: tid=0 2023-11-16T16:38:25.420 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.410+0000 7f673a879b80 1 client.15066 shutdown 2023-11-16T16:38:25.421 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.410+0000 7f673a879b80 1 -- 172.21.15.1:0/1358805563 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556eeb5c6bb0 msgr2=0x556eeb5d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:25.421 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.410+0000 7f673a879b80 1 --2- 172.21.15.1:0/1358805563 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556eeb5c6bb0 0x556eeb5d2730 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6718009990 tx=0x7f6718034310 comp rx=0 tx=0).stop 2023-11-16T16:38:25.421 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.410+0000 7f673a879b80 1 -- 172.21.15.1:0/1358805563 shutdown_connections 2023-11-16T16:38:25.421 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.410+0000 7f673a879b80 1 --2- 172.21.15.1:0/1358805563 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x556eeb6344c0 0x556eeb630e10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.421 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.410+0000 7f673a879b80 1 --2- 172.21.15.1:0/1358805563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556eeb5c80c0 0x556eeb62f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.421 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.410+0000 7f67077fe700 0 client.15066 ms_handle_reset on v2:172.21.15.26:6836/1453504863 2023-11-16T16:38:25.421 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.410+0000 7f673a879b80 1 --2- 172.21.15.1:0/1358805563 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556eeb5c6bb0 0x556eeb5d2730 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.421 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.410+0000 7f673a879b80 1 --2- 172.21.15.1:0/1358805563 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556eeb5c7500 0x556eeb5d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.421 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.410+0000 7f673a879b80 1 -- 172.21.15.1:0/1358805563 >> 172.21.15.1:0/1358805563 conn(0x556eeb5bc310 msgr2=0x556eeb5caf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:25.421 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.410+0000 7f673a879b80 1 -- 172.21.15.1:0/1358805563 shutdown_connections 2023-11-16T16:38:25.421 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.410+0000 7f673a879b80 1 -- 172.21.15.1:0/1358805563 wait complete. 2023-11-16T16:38:25.421 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.410+0000 7f673a879b80 20 client.15066 trim_cache size 0 max 16384 2023-11-16T16:38:25.422 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:38:25.422 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "35d9b010-849d-11ee-8dbe-212e2dc638e7", 2023-11-16T16:38:25.422 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 2, 2023-11-16T16:38:25.422 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24482, 2023-11-16T16:38:25.422 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2023-11-16T16:38:25.422 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2023-11-16T16:38:25.422 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2023-11-16T16:38:25.422 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 67.126063880000004, 2023-11-16T16:38:25.423 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 23, 2023-11-16T16:38:25.423 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 86, 2023-11-16T16:38:25.423 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 84, 2023-11-16T16:38:25.423 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 85.387703869000006 2023-11-16T16:38:25.423 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:38:25.424 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.410+0000 7f672e6ed700 1 -- 172.21.15.1:0/318449880 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f67100a11e0 msgr2=0x7f67100a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:25.424 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.410+0000 7f672e6ed700 1 --2- 172.21.15.1:0/318449880 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f67100a11e0 0x7f67100a3690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f671800b5c0 tx=0x7f6718005fd0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.424 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.410+0000 7f672e6ed700 1 -- 172.21.15.1:0/318449880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6728100300 msgr2=0x7f6728198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:25.425 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.410+0000 7f672e6ed700 1 --2- 172.21.15.1:0/318449880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6728100300 0x7f6728198740 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f671c00b530 tx=0x7f671c00b840 comp rx=0 tx=0).stop 2023-11-16T16:38:25.425 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.410+0000 7f672e6ed700 1 -- 172.21.15.1:0/318449880 shutdown_connections 2023-11-16T16:38:25.425 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.410+0000 7f672e6ed700 1 --2- 172.21.15.1:0/318449880 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f67100a11e0 0x7f67100a3690 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.425 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.410+0000 7f672e6ed700 1 --2- 172.21.15.1:0/318449880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6728100c90 0x7f672819f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.425 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.410+0000 7f672e6ed700 1 --2- 172.21.15.1:0/318449880 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67280ff110 0x7f6728198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.425 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.410+0000 7f672e6ed700 1 --2- 172.21.15.1:0/318449880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6728100300 0x7f6728198740 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.425 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.410+0000 7f672e6ed700 1 -- 172.21.15.1:0/318449880 >> 172.21.15.1:0/318449880 conn(0x7f67280fa6f0 msgr2=0x7f6728067de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:25.425 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.411+0000 7f672e6ed700 1 -- 172.21.15.1:0/318449880 shutdown_connections 2023-11-16T16:38:25.425 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.411+0000 7f672e6ed700 1 -- 172.21.15.1:0/318449880 wait complete. 2023-11-16T16:38:25.426 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-11-16T16:38:25.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.551+0000 7fb295688700 1 -- 172.21.15.1:0/3619849780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb290100240 msgr2=0x7fb290100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:25.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.551+0000 7fb295688700 1 --2- 172.21.15.1:0/3619849780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb290100240 0x7fb290100650 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb280009990 tx=0x7fb280009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.551+0000 7fb295688700 1 -- 172.21.15.1:0/3619849780 shutdown_connections 2023-11-16T16:38:25.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.551+0000 7fb295688700 1 --2- 172.21.15.1:0/3619849780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb290101dc0 0x7fb290108680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.557 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.551+0000 7fb295688700 1 --2- 172.21.15.1:0/3619849780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb290100240 0x7fb290100650 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.557 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.551+0000 7fb295688700 1 --2- 172.21.15.1:0/3619849780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb290101430 0x7fb290101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.557 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.551+0000 7fb295688700 1 -- 172.21.15.1:0/3619849780 >> 172.21.15.1:0/3619849780 conn(0x7fb2900fb820 msgr2=0x7fb2900fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:25.557 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.551+0000 7fb295688700 1 -- 172.21.15.1:0/3619849780 shutdown_connections 2023-11-16T16:38:25.557 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.551+0000 7fb295688700 1 -- 172.21.15.1:0/3619849780 wait complete. 2023-11-16T16:38:25.557 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.552+0000 7fb295688700 1 Processor -- start 2023-11-16T16:38:25.557 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.552+0000 7fb295688700 1 -- start start 2023-11-16T16:38:25.557 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.552+0000 7fb295688700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb290100240 0x7fb290195f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.557 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.552+0000 7fb295688700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb290101430 0x7fb290196440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.557 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.552+0000 7fb295688700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb290101dc0 0x7fb29019d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.557 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.552+0000 7fb295688700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29019da80 con 0x7fb290101430 2023-11-16T16:38:25.557 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.552+0000 7fb295688700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29019dbc0 con 0x7fb290100240 2023-11-16T16:38:25.558 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.552+0000 7fb295688700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29019dd50 con 0x7fb290101dc0 2023-11-16T16:38:25.558 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.552+0000 7fb28e7fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb290101430 0x7fb290196440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.558 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.553+0000 7fb28e7fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb290101430 0x7fb290196440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42364/0 (socket says 172.21.15.1:42364) 2023-11-16T16:38:25.558 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.553+0000 7fb28e7fc700 1 -- 172.21.15.1:0/2105479551 learned_addr learned my addr 172.21.15.1:0/2105479551 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:25.558 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.553+0000 7fb28effd700 1 --2- 172.21.15.1:0/2105479551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb290100240 0x7fb290195f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.558 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.553+0000 7fb28f7fe700 1 --2- 172.21.15.1:0/2105479551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb290101dc0 0x7fb29019d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.558 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.553+0000 7fb28e7fc700 1 -- 172.21.15.1:0/2105479551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb290101dc0 msgr2=0x7fb29019d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:25.558 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.553+0000 7fb28e7fc700 1 --2- 172.21.15.1:0/2105479551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb290101dc0 0x7fb29019d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.558 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.553+0000 7fb28e7fc700 1 -- 172.21.15.1:0/2105479551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb290100240 msgr2=0x7fb290195f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:25.558 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.553+0000 7fb28e7fc700 1 --2- 172.21.15.1:0/2105479551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb290100240 0x7fb290195f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.558 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.553+0000 7fb28e7fc700 1 -- 172.21.15.1:0/2105479551 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2800095f0 con 0x7fb290101430 2023-11-16T16:38:25.558 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.553+0000 7fb28f7fe700 1 --2- 172.21.15.1:0/2105479551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb290101dc0 0x7fb29019d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:38:25.559 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.553+0000 7fb28e7fc700 1 --2- 172.21.15.1:0/2105479551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb290101430 0x7fb290196440 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb28400c9b0 tx=0x7fb28400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:25.559 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.553+0000 7fb277fff700 1 -- 172.21.15.1:0/2105479551 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb284007b00 con 0x7fb290101430 2023-11-16T16:38:25.559 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.553+0000 7fb277fff700 1 -- 172.21.15.1:0/2105479551 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb284007c60 con 0x7fb290101430 2023-11-16T16:38:25.559 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.553+0000 7fb295688700 1 -- 172.21.15.1:0/2105479551 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb29010bbb0 con 0x7fb290101430 2023-11-16T16:38:25.559 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.553+0000 7fb277fff700 1 -- 172.21.15.1:0/2105479551 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb284018660 con 0x7fb290101430 2023-11-16T16:38:25.559 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.553+0000 7fb295688700 1 -- 172.21.15.1:0/2105479551 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb29010ba20 con 0x7fb290101430 2023-11-16T16:38:25.565 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.555+0000 7fb277fff700 1 -- 172.21.15.1:0/2105479551 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fb28400f450 con 0x7fb290101430 2023-11-16T16:38:25.565 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.556+0000 7fb277fff700 1 --2- 172.21.15.1:0/2105479551 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fb2780a11e0 0x7fb2780a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.565 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.556+0000 7fb277fff700 1 -- 172.21.15.1:0/2105479551 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb2840ccac0 con 0x7fb290101430 2023-11-16T16:38:25.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.556+0000 7fb28effd700 1 --2- 172.21.15.1:0/2105479551 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fb2780a11e0 0x7fb2780a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.557+0000 7fb28effd700 1 --2- 172.21.15.1:0/2105479551 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fb2780a11e0 0x7fb2780a3690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb28000b2e0 tx=0x7fb280005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:25.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.557+0000 7fb2a1814b80 1 Processor -- start 2023-11-16T16:38:25.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.557+0000 7fb2a1814b80 1 -- start start 2023-11-16T16:38:25.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.557+0000 7fb2a1814b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558b5f67bb0 0x5558b5f67fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.557+0000 7fb2a1814b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558b5f68500 0x5558b5f68ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.557+0000 7fb2a1814b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558b5f690c0 0x5558b5f69530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.557+0000 7fb2a1814b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558b5e6d010 con 0x5558b5f68500 2023-11-16T16:38:25.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.557+0000 7fb2a1814b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558b5ebf850 con 0x5558b5f67bb0 2023-11-16T16:38:25.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.557+0000 7fb2a1814b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558b5eb8ce0 con 0x5558b5f690c0 2023-11-16T16:38:25.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.557+0000 7fb28e7fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558b5f68500 0x5558b5f68ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.557+0000 7fb28e7fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558b5f68500 0x5558b5f68ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42372/0 (socket says 172.21.15.1:42372) 2023-11-16T16:38:25.567 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.557+0000 7fb28e7fc700 1 -- 172.21.15.1:0/2606998532 learned_addr learned my addr 172.21.15.1:0/2606998532 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:25.567 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.557+0000 7fb28effd700 1 --2- 172.21.15.1:0/2606998532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558b5f67bb0 0x5558b5f67fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.567 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.557+0000 7fb28f7fe700 1 --2- 172.21.15.1:0/2606998532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558b5f690c0 0x5558b5f69530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.567 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.557+0000 7fb28e7fc700 1 -- 172.21.15.1:0/2606998532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558b5f690c0 msgr2=0x5558b5f69530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:25.567 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.557+0000 7fb28e7fc700 1 --2- 172.21.15.1:0/2606998532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558b5f690c0 0x5558b5f69530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.567 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.557+0000 7fb28e7fc700 1 -- 172.21.15.1:0/2606998532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558b5f67bb0 msgr2=0x5558b5f67fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:25.567 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.557+0000 7fb28e7fc700 1 --2- 172.21.15.1:0/2606998532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558b5f67bb0 0x5558b5f67fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.567 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.557+0000 7fb28e7fc700 1 -- 172.21.15.1:0/2606998532 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558b5d3a5e0 con 0x5558b5f68500 2023-11-16T16:38:25.567 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.557+0000 7fb28effd700 1 --2- 172.21.15.1:0/2606998532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558b5f67bb0 0x5558b5f67fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:38:25.567 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.557+0000 7fb28e7fc700 1 --2- 172.21.15.1:0/2606998532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558b5f68500 0x5558b5f68ae0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb284096230 tx=0x7fb284099790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d91def59142529ec server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:25.567 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.558+0000 7fb2757fa700 1 -- 172.21.15.1:0/2606998532 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2840a2180 con 0x5558b5f68500 2023-11-16T16:38:25.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.558+0000 7fb2757fa700 1 -- 172.21.15.1:0/2606998532 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2840980d0 con 0x5558b5f68500 2023-11-16T16:38:25.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.558+0000 7fb2757fa700 1 -- 172.21.15.1:0/2606998532 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb284099f90 con 0x5558b5f68500 2023-11-16T16:38:25.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.558+0000 7fb2a1814b80 1 -- 172.21.15.1:0/2606998532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558b5f68500 msgr2=0x5558b5f68ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:25.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.558+0000 7fb2a1814b80 1 --2- 172.21.15.1:0/2606998532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558b5f68500 0x5558b5f68ae0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb284096230 tx=0x7fb284099790 comp rx=0 tx=0).stop 2023-11-16T16:38:25.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.558+0000 7fb2a1814b80 1 -- 172.21.15.1:0/2606998532 shutdown_connections 2023-11-16T16:38:25.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.558+0000 7fb2a1814b80 1 --2- 172.21.15.1:0/2606998532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558b5f690c0 0x5558b5f69530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.558+0000 7fb2a1814b80 1 --2- 172.21.15.1:0/2606998532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558b5f67bb0 0x5558b5f67fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.558+0000 7fb2a1814b80 1 --2- 172.21.15.1:0/2606998532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558b5f68500 0x5558b5f68ae0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.558+0000 7fb2a1814b80 1 -- 172.21.15.1:0/2606998532 >> 172.21.15.1:0/2606998532 conn(0x5558b5f5d310 msgr2=0x5558b5f62620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:25.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.558+0000 7fb2a1814b80 1 -- 172.21.15.1:0/2606998532 shutdown_connections 2023-11-16T16:38:25.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.558+0000 7fb2a1814b80 1 -- 172.21.15.1:0/2606998532 wait complete. 2023-11-16T16:38:25.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.559+0000 7fb2a1814b80 1 -- start start 2023-11-16T16:38:25.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.559+0000 7fb2a1814b80 1 Processor -- start 2023-11-16T16:38:25.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.559+0000 7fb2a1814b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558b5f67bb0 0x5558b5f71590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.559+0000 7fb2a1814b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558b5f68500 0x5558b5f71ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.559+0000 7fb2a1814b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558b5f690c0 0x5558b5f78b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.559+0000 7fb2a1814b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558b5e5a890 con 0x5558b5f68500 2023-11-16T16:38:25.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.559+0000 7fb2a1814b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558b5e5a530 con 0x5558b5f67bb0 2023-11-16T16:38:25.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.559+0000 7fb2a1814b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558b5d3c8d0 con 0x5558b5f690c0 2023-11-16T16:38:25.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.559+0000 7fb2a1814b80 10 client.0 _ll_register_callbacks cb 0x5558b5e78730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:38:25.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.559+0000 7fb28e7fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558b5f68500 0x5558b5f71ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.559+0000 7fb28e7fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558b5f68500 0x5558b5f71ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42378/0 (socket says 172.21.15.1:42378) 2023-11-16T16:38:25.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.559+0000 7fb28effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558b5f67bb0 0x5558b5f71590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.559+0000 7fb28e7fc700 1 -- 172.21.15.1:0/3510313745 learned_addr learned my addr 172.21.15.1:0/3510313745 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:25.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.559+0000 7fb28f7fe700 1 --2- 172.21.15.1:0/3510313745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558b5f690c0 0x5558b5f78b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.559+0000 7fb28e7fc700 1 -- 172.21.15.1:0/3510313745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558b5f690c0 msgr2=0x5558b5f78b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:25.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.559+0000 7fb28e7fc700 1 --2- 172.21.15.1:0/3510313745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558b5f690c0 0x5558b5f78b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.559+0000 7fb28e7fc700 1 -- 172.21.15.1:0/3510313745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558b5f67bb0 msgr2=0x5558b5f71590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:25.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.559+0000 7fb28e7fc700 1 --2- 172.21.15.1:0/3510313745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558b5f67bb0 0x5558b5f71590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.559+0000 7fb28e7fc700 1 -- 172.21.15.1:0/3510313745 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb29010ba20 con 0x5558b5f68500 2023-11-16T16:38:25.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.560+0000 7fb28e7fc700 1 --2- 172.21.15.1:0/3510313745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558b5f68500 0x5558b5f71ad0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb284000c00 tx=0x7fb2840ca3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0f73a492a4cf6d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:25.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.560+0000 7fb272ff5700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:38:25.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.560+0000 7fb272ff5700 1 -- 172.21.15.1:0/3510313745 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2840a2180 con 0x5558b5f68500 2023-11-16T16:38:25.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.560+0000 7fb272ff5700 1 -- 172.21.15.1:0/3510313745 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb28409c030 con 0x5558b5f68500 2023-11-16T16:38:25.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.560+0000 7fb2a1814b80 1 -- 172.21.15.1:0/3510313745 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5558b5f5aad0 con 0x5558b5f68500 2023-11-16T16:38:25.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.560+0000 7fb272ff5700 1 -- 172.21.15.1:0/3510313745 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2840ca0d0 con 0x5558b5f68500 2023-11-16T16:38:25.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.560+0000 7fb2a1814b80 10 client.15078 fetch_fsmap learned FSMap version 23 2023-11-16T16:38:25.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.560+0000 7fb2a1814b80 1 -- 172.21.15.1:0/3510313745 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5558b5e7f140 con 0x5558b5f68500 2023-11-16T16:38:25.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.560+0000 7fb272ff5700 1 -- 172.21.15.1:0/3510313745 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb28407c080 con 0x5558b5f68500 2023-11-16T16:38:25.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.560+0000 7fb272ff5700 1 -- 172.21.15.1:0/3510313745 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb2840b2220 con 0x5558b5f68500 2023-11-16T16:38:25.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.560+0000 7fb2a1814b80 10 client.15078 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:38:25.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.560+0000 7fb2a1814b80 10 client.15078 resolve_mds: resolved name 'i' to daemon mds.i 2023-11-16T16:38:25.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.560+0000 7fb2a1814b80 20 client.15078 populate_metadata read hostname 'smithi001' 2023-11-16T16:38:25.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.560+0000 7fb2a1814b80 1 --2- 172.21.15.1:0/3510313745 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x5558b5f7c940 0x5558b5f7ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.560+0000 7fb2a1814b80 4 client.15078 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:38:25.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.560+0000 7fb2a1814b80 1 -- 172.21.15.1:0/3510313745 --> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5558b5e7b050 con 0x5558b5f7c940 2023-11-16T16:38:25.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.561+0000 7fb28effd700 1 --2- 172.21.15.1:0/3510313745 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x5558b5f7c940 0x5558b5f7ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.561+0000 7fb272ff5700 10 client.15078 ms_handle_connect on v2:172.21.15.53:6838/1044753024 2023-11-16T16:38:25.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.561+0000 7fb28effd700 1 --2- 172.21.15.1:0/3510313745 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x5558b5f7c940 0x5558b5f7ba40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=67d863c53f9c58b5 server_cookie=6ba9da1809b5099e in_seq=0 out_seq=0 2023-11-16T16:38:25.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.562+0000 7fb272ff5700 1 -- 172.21.15.1:0/3510313745 <== mds.1 v2:172.21.15.53:6838/1044753024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5558b5e7b050 con 0x5558b5f7c940 2023-11-16T16:38:25.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.562+0000 7fb272ff5700 10 client.15078 handle_command_reply: tid=0 2023-11-16T16:38:25.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.562+0000 7fb2a1814b80 1 client.15078 shutdown 2023-11-16T16:38:25.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.562+0000 7fb2a1814b80 1 -- 172.21.15.1:0/3510313745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558b5f68500 msgr2=0x5558b5f71ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:25.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.562+0000 7fb2a1814b80 1 --2- 172.21.15.1:0/3510313745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558b5f68500 0x5558b5f71ad0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb284000c00 tx=0x7fb2840ca3a0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.562+0000 7fb2a1814b80 1 -- 172.21.15.1:0/3510313745 shutdown_connections 2023-11-16T16:38:25.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.562+0000 7fb2a1814b80 1 --2- 172.21.15.1:0/3510313745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558b5f690c0 0x5558b5f78b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.562+0000 7fb2a1814b80 1 --2- 172.21.15.1:0/3510313745 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x5558b5f7c940 0x5558b5f7ba40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.562+0000 7fb2a1814b80 1 --2- 172.21.15.1:0/3510313745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558b5f67bb0 0x5558b5f71590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.562+0000 7fb2a1814b80 1 --2- 172.21.15.1:0/3510313745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558b5f68500 0x5558b5f71ad0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.562+0000 7fb272ff5700 0 client.15078 ms_handle_reset on v2:172.21.15.53:6838/1044753024 2023-11-16T16:38:25.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.562+0000 7fb2a1814b80 1 -- 172.21.15.1:0/3510313745 >> 172.21.15.1:0/3510313745 conn(0x5558b5f5d310 msgr2=0x5558b5f61d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:25.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.562+0000 7fb2a1814b80 1 -- 172.21.15.1:0/3510313745 shutdown_connections 2023-11-16T16:38:25.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.562+0000 7fb2a1814b80 1 -- 172.21.15.1:0/3510313745 wait complete. 2023-11-16T16:38:25.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.562+0000 7fb2a1814b80 20 client.15078 trim_cache size 0 max 16384 2023-11-16T16:38:25.588 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.580+0000 7fb2a1814b80 1 Processor -- start 2023-11-16T16:38:25.588 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.581+0000 7fb2a1814b80 1 -- start start 2023-11-16T16:38:25.588 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.581+0000 7fb2a1814b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558b5f67bb0 0x5558b5f7ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.588 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.581+0000 7fb2a1814b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558b5f68500 0x5558b5f78ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.588 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.581+0000 7fb2a1814b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558b5f690c0 0x5558b5f6f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.588 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.581+0000 7fb2a1814b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558b5f699b0 con 0x5558b5f690c0 2023-11-16T16:38:25.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.581+0000 7fb2a1814b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558b5f72780 con 0x5558b5f67bb0 2023-11-16T16:38:25.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.581+0000 7fb2a1814b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558b5dc4ed0 con 0x5558b5f68500 2023-11-16T16:38:25.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.581+0000 7fb28f7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558b5f690c0 0x5558b5f6f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.581+0000 7fb28f7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558b5f690c0 0x5558b5f6f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42388/0 (socket says 172.21.15.1:42388) 2023-11-16T16:38:25.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.581+0000 7fb28f7fe700 1 -- 172.21.15.1:0/2639504030 learned_addr learned my addr 172.21.15.1:0/2639504030 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:25.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.581+0000 7fb28e7fc700 1 --2- 172.21.15.1:0/2639504030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558b5f68500 0x5558b5f78ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.581+0000 7fb28effd700 1 --2- 172.21.15.1:0/2639504030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558b5f67bb0 0x5558b5f7ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.581+0000 7fb28f7fe700 1 -- 172.21.15.1:0/2639504030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558b5f68500 msgr2=0x5558b5f78ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:25.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.581+0000 7fb28f7fe700 1 --2- 172.21.15.1:0/2639504030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558b5f68500 0x5558b5f78ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.581+0000 7fb28f7fe700 1 -- 172.21.15.1:0/2639504030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558b5f67bb0 msgr2=0x5558b5f7ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:25.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.581+0000 7fb28f7fe700 1 --2- 172.21.15.1:0/2639504030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558b5f67bb0 0x5558b5f7ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.590 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.581+0000 7fb28f7fe700 1 -- 172.21.15.1:0/2639504030 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558b5e7ee90 con 0x5558b5f690c0 2023-11-16T16:38:25.590 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.581+0000 7fb28effd700 1 --2- 172.21.15.1:0/2639504030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558b5f67bb0 0x5558b5f7ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:38:25.590 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.581+0000 7fb28f7fe700 1 --2- 172.21.15.1:0/2639504030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558b5f690c0 0x5558b5f6f430 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb288007630 tx=0x7fb28800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a034ef0565a4fc4 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:25.590 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.581+0000 7fb275ffb700 1 -- 172.21.15.1:0/2639504030 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2880091a0 con 0x5558b5f690c0 2023-11-16T16:38:25.590 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.581+0000 7fb275ffb700 1 -- 172.21.15.1:0/2639504030 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb28800f030 con 0x5558b5f690c0 2023-11-16T16:38:25.590 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.581+0000 7fb275ffb700 1 -- 172.21.15.1:0/2639504030 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb288016b40 con 0x5558b5f690c0 2023-11-16T16:38:25.590 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.582+0000 7fb2a1814b80 1 -- 172.21.15.1:0/2639504030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558b5f690c0 msgr2=0x5558b5f6f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:25.590 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.582+0000 7fb2a1814b80 1 --2- 172.21.15.1:0/2639504030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558b5f690c0 0x5558b5f6f430 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb288007630 tx=0x7fb28800b040 comp rx=0 tx=0).stop 2023-11-16T16:38:25.590 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.582+0000 7fb2a1814b80 1 -- 172.21.15.1:0/2639504030 shutdown_connections 2023-11-16T16:38:25.590 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.582+0000 7fb2a1814b80 1 --2- 172.21.15.1:0/2639504030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558b5f690c0 0x5558b5f6f430 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.590 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.582+0000 7fb2a1814b80 1 --2- 172.21.15.1:0/2639504030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558b5f67bb0 0x5558b5f7ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.591 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.582+0000 7fb2a1814b80 1 --2- 172.21.15.1:0/2639504030 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558b5f68500 0x5558b5f78ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.591 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.582+0000 7fb2a1814b80 1 -- 172.21.15.1:0/2639504030 >> 172.21.15.1:0/2639504030 conn(0x5558b5f5d310 msgr2=0x5558b5f702a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:25.591 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.582+0000 7fb2a1814b80 1 -- 172.21.15.1:0/2639504030 shutdown_connections 2023-11-16T16:38:25.591 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.582+0000 7fb2a1814b80 1 -- 172.21.15.1:0/2639504030 wait complete. 2023-11-16T16:38:25.591 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.582+0000 7fb2a1814b80 1 -- start start 2023-11-16T16:38:25.591 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.582+0000 7fb2a1814b80 1 Processor -- start 2023-11-16T16:38:25.591 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.583+0000 7fb2a1814b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558b5f67bb0 0x5558b5f733d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.591 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.583+0000 7fb2a1814b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558b5f68500 0x5558b5f78940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.591 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.583+0000 7fb2a1814b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558b5f690c0 0x5558b5f73910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.591 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.583+0000 7fb2a1814b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558b5f73f30 con 0x5558b5f67bb0 2023-11-16T16:38:25.591 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.583+0000 7fb2a1814b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558b5fd01d0 con 0x5558b5f690c0 2023-11-16T16:38:25.591 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.583+0000 7fb2a1814b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558b5fd0310 con 0x5558b5f68500 2023-11-16T16:38:25.591 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.583+0000 7fb2a1814b80 10 client.0 _ll_register_callbacks cb 0x5558b5e78730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:38:25.592 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.583+0000 7fb28effd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558b5f67bb0 0x5558b5f733d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.592 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.583+0000 7fb28effd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558b5f67bb0 0x5558b5f733d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42390/0 (socket says 172.21.15.1:42390) 2023-11-16T16:38:25.592 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.583+0000 7fb28effd700 1 -- 172.21.15.1:0/3723739652 learned_addr learned my addr 172.21.15.1:0/3723739652 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:25.592 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.583+0000 7fb28e7fc700 1 --2- 172.21.15.1:0/3723739652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558b5f68500 0x5558b5f78940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.592 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.583+0000 7fb28f7fe700 1 --2- 172.21.15.1:0/3723739652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558b5f690c0 0x5558b5f73910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.592 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.583+0000 7fb28effd700 1 -- 172.21.15.1:0/3723739652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558b5f68500 msgr2=0x5558b5f78940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:25.592 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.583+0000 7fb28effd700 1 --2- 172.21.15.1:0/3723739652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558b5f68500 0x5558b5f78940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.592 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.583+0000 7fb28effd700 1 -- 172.21.15.1:0/3723739652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558b5f690c0 msgr2=0x5558b5f73910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:25.592 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.583+0000 7fb28effd700 1 --2- 172.21.15.1:0/3723739652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558b5f690c0 0x5558b5f73910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.592 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.583+0000 7fb28effd700 1 -- 172.21.15.1:0/3723739652 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb288009300 con 0x5558b5f67bb0 2023-11-16T16:38:25.592 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.583+0000 7fb28f7fe700 1 --2- 172.21.15.1:0/3723739652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558b5f690c0 0x5558b5f73910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:38:25.593 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.583+0000 7fb272ff5700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:38:25.593 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.583+0000 7fb28effd700 1 --2- 172.21.15.1:0/3723739652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558b5f67bb0 0x5558b5f733d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb28000b620 tx=0x7fb280009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3129b7b80f12f8ea server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:25.593 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.583+0000 7fb272ff5700 1 -- 172.21.15.1:0/3723739652 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb280035ae0 con 0x5558b5f67bb0 2023-11-16T16:38:25.593 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.583+0000 7fb272ff5700 1 -- 172.21.15.1:0/3723739652 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb280035c40 con 0x5558b5f67bb0 2023-11-16T16:38:25.593 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.583+0000 7fb2a1814b80 1 -- 172.21.15.1:0/3723739652 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb2840980d0 con 0x5558b5f67bb0 2023-11-16T16:38:25.593 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.584+0000 7fb272ff5700 1 -- 172.21.15.1:0/3723739652 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb280035e00 con 0x5558b5f67bb0 2023-11-16T16:38:25.593 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.584+0000 7fb2a1814b80 10 client.15084 fetch_fsmap learned FSMap version 23 2023-11-16T16:38:25.593 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.584+0000 7fb2a1814b80 1 -- 172.21.15.1:0/3723739652 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5558b5ccfc10 con 0x5558b5f67bb0 2023-11-16T16:38:25.593 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.584+0000 7fb272ff5700 1 -- 172.21.15.1:0/3723739652 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2840980d0 con 0x5558b5f67bb0 2023-11-16T16:38:25.593 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.584+0000 7fb272ff5700 1 -- 172.21.15.1:0/3723739652 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb28003b020 con 0x5558b5f67bb0 2023-11-16T16:38:25.593 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.584+0000 7fb2a1814b80 10 client.15084 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:38:25.593 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.584+0000 7fb2a1814b80 10 client.15084 resolve_mds: resolved name 'i' to daemon mds.i 2023-11-16T16:38:25.594 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.584+0000 7fb2a1814b80 20 client.15084 populate_metadata read hostname 'smithi001' 2023-11-16T16:38:25.594 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.584+0000 7fb2a1814b80 1 --2- 172.21.15.1:0/3723739652 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x5558b5fd4ed0 0x5558b5fd1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.594 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.584+0000 7fb2a1814b80 4 client.15084 mds_command: new command op to 24493 tid=0[{"prefix": "status"}] 2023-11-16T16:38:25.594 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.584+0000 7fb2a1814b80 1 -- 172.21.15.1:0/3723739652 --> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5558b5e87610 con 0x5558b5fd4ed0 2023-11-16T16:38:25.594 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.584+0000 7fb28e7fc700 1 --2- 172.21.15.1:0/3723739652 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x5558b5fd4ed0 0x5558b5fd1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.594 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.585+0000 7fb28e7fc700 1 --2- 172.21.15.1:0/3723739652 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x5558b5fd4ed0 0x5558b5fd1800 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c35c50ab35fa2b4f server_cookie=8bcf103edf2d4a2a in_seq=0 out_seq=0 2023-11-16T16:38:25.594 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.585+0000 7fb272ff5700 10 client.15084 ms_handle_connect on v2:172.21.15.53:6838/1044753024 2023-11-16T16:38:25.594 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.585+0000 7fb272ff5700 1 -- 172.21.15.1:0/3723739652 <== mds.1 v2:172.21.15.53:6838/1044753024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5558b5e87610 con 0x5558b5fd4ed0 2023-11-16T16:38:25.594 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.585+0000 7fb272ff5700 10 client.15084 handle_command_reply: tid=0 2023-11-16T16:38:25.594 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.585+0000 7fb2a1814b80 1 client.15084 shutdown 2023-11-16T16:38:25.594 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.585+0000 7fb2a1814b80 1 -- 172.21.15.1:0/3723739652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558b5f67bb0 msgr2=0x5558b5f733d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:25.594 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.585+0000 7fb2a1814b80 1 --2- 172.21.15.1:0/3723739652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558b5f67bb0 0x5558b5f733d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb28000b620 tx=0x7fb280009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.595 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.585+0000 7fb2a1814b80 1 -- 172.21.15.1:0/3723739652 shutdown_connections 2023-11-16T16:38:25.595 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.585+0000 7fb2a1814b80 1 --2- 172.21.15.1:0/3723739652 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x5558b5fd4ed0 0x5558b5fd1800 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.595 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.585+0000 7fb2a1814b80 1 --2- 172.21.15.1:0/3723739652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5558b5f690c0 0x5558b5f73910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.595 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.585+0000 7fb272ff5700 0 client.15084 ms_handle_reset on v2:172.21.15.53:6838/1044753024 2023-11-16T16:38:25.595 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.585+0000 7fb2a1814b80 1 --2- 172.21.15.1:0/3723739652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558b5f67bb0 0x5558b5f733d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.595 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.585+0000 7fb2a1814b80 1 --2- 172.21.15.1:0/3723739652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5558b5f68500 0x5558b5f78940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.595 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.585+0000 7fb2a1814b80 1 -- 172.21.15.1:0/3723739652 >> 172.21.15.1:0/3723739652 conn(0x5558b5f5d310 msgr2=0x5558b5f7ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:25.595 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.585+0000 7fb2a1814b80 1 -- 172.21.15.1:0/3723739652 shutdown_connections 2023-11-16T16:38:25.595 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.585+0000 7fb2a1814b80 1 -- 172.21.15.1:0/3723739652 wait complete. 2023-11-16T16:38:25.595 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.585+0000 7fb2a1814b80 20 client.15084 trim_cache size 0 max 16384 2023-11-16T16:38:25.596 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:38:25.596 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "35d9b010-849d-11ee-8dbe-212e2dc638e7", 2023-11-16T16:38:25.596 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 1, 2023-11-16T16:38:25.596 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24493, 2023-11-16T16:38:25.596 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2023-11-16T16:38:25.596 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2023-11-16T16:38:25.596 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2023-11-16T16:38:25.596 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 68.309482876000004, 2023-11-16T16:38:25.596 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 23, 2023-11-16T16:38:25.596 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 86, 2023-11-16T16:38:25.596 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 84, 2023-11-16T16:38:25.597 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 83.286886928000001 2023-11-16T16:38:25.597 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:38:25.597 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.586+0000 7fb295688700 1 -- 172.21.15.1:0/2105479551 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fb2780a11e0 msgr2=0x7fb2780a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:25.597 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.586+0000 7fb295688700 1 --2- 172.21.15.1:0/2105479551 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fb2780a11e0 0x7fb2780a3690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb28000b2e0 tx=0x7fb280005960 comp rx=0 tx=0).stop 2023-11-16T16:38:25.597 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.586+0000 7fb295688700 1 -- 172.21.15.1:0/2105479551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb290101430 msgr2=0x7fb290196440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:25.598 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.586+0000 7fb295688700 1 --2- 172.21.15.1:0/2105479551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb290101430 0x7fb290196440 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb28400c9b0 tx=0x7fb28400ccc0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.598 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.586+0000 7fb295688700 1 -- 172.21.15.1:0/2105479551 shutdown_connections 2023-11-16T16:38:25.598 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.586+0000 7fb295688700 1 --2- 172.21.15.1:0/2105479551 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fb2780a11e0 0x7fb2780a3690 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.598 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.586+0000 7fb295688700 1 --2- 172.21.15.1:0/2105479551 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb290101dc0 0x7fb29019d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.598 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.586+0000 7fb295688700 1 --2- 172.21.15.1:0/2105479551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb290100240 0x7fb290195f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.598 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.586+0000 7fb295688700 1 --2- 172.21.15.1:0/2105479551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb290101430 0x7fb290196440 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.598 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.586+0000 7fb295688700 1 -- 172.21.15.1:0/2105479551 >> 172.21.15.1:0/2105479551 conn(0x7fb2900fb820 msgr2=0x7fb2900fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:25.598 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.586+0000 7fb295688700 1 -- 172.21.15.1:0/2105479551 shutdown_connections 2023-11-16T16:38:25.598 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.586+0000 7fb295688700 1 -- 172.21.15.1:0/2105479551 wait complete. 2023-11-16T16:38:25.600 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-16T16:38:25.728 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.726+0000 7ff80caf9700 1 -- 172.21.15.1:0/530828767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff8081017f0 msgr2=0x7ff808101c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:25.728 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.726+0000 7ff80caf9700 1 --2- 172.21.15.1:0/530828767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff8081017f0 0x7ff808101c00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8009990 tx=0x7ff7f8009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.728 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.726+0000 7ff80caf9700 1 -- 172.21.15.1:0/530828767 shutdown_connections 2023-11-16T16:38:25.729 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.726+0000 7ff80caf9700 1 --2- 172.21.15.1:0/530828767 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8080febf0 0x7ff8081087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.729 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.726+0000 7ff80caf9700 1 --2- 172.21.15.1:0/530828767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff8081017f0 0x7ff808101c00 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.729 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.726+0000 7ff80caf9700 1 --2- 172.21.15.1:0/530828767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8080fe260 0x7ff8080fe6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.729 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.726+0000 7ff80caf9700 1 -- 172.21.15.1:0/530828767 >> 172.21.15.1:0/530828767 conn(0x7ff8080f9eb0 msgr2=0x7ff8080fc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:25.729 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.726+0000 7ff80caf9700 1 -- 172.21.15.1:0/530828767 shutdown_connections 2023-11-16T16:38:25.729 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.726+0000 7ff80caf9700 1 -- 172.21.15.1:0/530828767 wait complete. 2023-11-16T16:38:25.731 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.727+0000 7ff80caf9700 1 Processor -- start 2023-11-16T16:38:25.731 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.727+0000 7ff80caf9700 1 -- start start 2023-11-16T16:38:25.731 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.727+0000 7ff80caf9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8080fe260 0x7ff808195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.731 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.727+0000 7ff80caf9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff8080febf0 0x7ff8081964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.731 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.727+0000 7ff80caf9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8081017f0 0x7ff80819d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.732 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.727+0000 7ff80caf9700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80819db30 con 0x7ff8080febf0 2023-11-16T16:38:25.732 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.727+0000 7ff80caf9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80819dc70 con 0x7ff8080fe260 2023-11-16T16:38:25.732 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.727+0000 7ff80caf9700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80819de00 con 0x7ff8081017f0 2023-11-16T16:38:25.732 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.727+0000 7ff805d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff8080febf0 0x7ff8081964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.732 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.727+0000 7ff805d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff8080febf0 0x7ff8081964f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42406/0 (socket says 172.21.15.1:42406) 2023-11-16T16:38:25.732 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.727+0000 7ff805d9b700 1 -- 172.21.15.1:0/2116979276 learned_addr learned my addr 172.21.15.1:0/2116979276 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:25.732 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.727+0000 7ff80659c700 1 --2- 172.21.15.1:0/2116979276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8080fe260 0x7ff808195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.732 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.728+0000 7ff806d9d700 1 --2- 172.21.15.1:0/2116979276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8081017f0 0x7ff80819d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.732 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.728+0000 7ff805d9b700 1 -- 172.21.15.1:0/2116979276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8081017f0 msgr2=0x7ff80819d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:25.732 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.728+0000 7ff805d9b700 1 --2- 172.21.15.1:0/2116979276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8081017f0 0x7ff80819d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.732 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.728+0000 7ff805d9b700 1 -- 172.21.15.1:0/2116979276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8080fe260 msgr2=0x7ff808195fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:25.733 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.728+0000 7ff805d9b700 1 --2- 172.21.15.1:0/2116979276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8080fe260 0x7ff808195fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.733 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.728+0000 7ff805d9b700 1 -- 172.21.15.1:0/2116979276 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f80095f0 con 0x7ff8080febf0 2023-11-16T16:38:25.733 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.728+0000 7ff806d9d700 1 --2- 172.21.15.1:0/2116979276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8081017f0 0x7ff80819d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:38:25.733 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.728+0000 7ff805d9b700 1 --2- 172.21.15.1:0/2116979276 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff8080febf0 0x7ff8081964f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff7f000e9a0 tx=0x7ff7f000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:25.733 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.728+0000 7ff7ff7fe700 1 -- 172.21.15.1:0/2116979276 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0009790 con 0x7ff8080febf0 2023-11-16T16:38:25.733 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.728+0000 7ff7ff7fe700 1 -- 172.21.15.1:0/2116979276 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7f000cbd0 con 0x7ff8080febf0 2023-11-16T16:38:25.733 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.728+0000 7ff80caf9700 1 -- 172.21.15.1:0/2116979276 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff80810bbb0 con 0x7ff8080febf0 2023-11-16T16:38:25.733 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.728+0000 7ff7ff7fe700 1 -- 172.21.15.1:0/2116979276 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0018680 con 0x7ff8080febf0 2023-11-16T16:38:25.733 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.728+0000 7ff80caf9700 1 -- 172.21.15.1:0/2116979276 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff80810ba20 con 0x7ff8080febf0 2023-11-16T16:38:25.740 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.730+0000 7ff7ff7fe700 1 -- 172.21.15.1:0/2116979276 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7ff7f000cd40 con 0x7ff8080febf0 2023-11-16T16:38:25.740 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.731+0000 7ff7ff7fe700 1 --2- 172.21.15.1:0/2116979276 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7ff7f40a1260 0x7ff7f40a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.740 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.731+0000 7ff80659c700 1 --2- 172.21.15.1:0/2116979276 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7ff7f40a1260 0x7ff7f40a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.740 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.731+0000 7ff7ff7fe700 1 -- 172.21.15.1:0/2116979276 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff7f0014070 con 0x7ff8080febf0 2023-11-16T16:38:25.740 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.731+0000 7ff818c85b80 1 Processor -- start 2023-11-16T16:38:25.740 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.732+0000 7ff80659c700 1 --2- 172.21.15.1:0/2116979276 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7ff7f40a1260 0x7ff7f40a3710 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff7f800b2e0 tx=0x7ff7f8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:25.740 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.732+0000 7ff818c85b80 1 -- start start 2023-11-16T16:38:25.740 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.732+0000 7ff818c85b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9468a7bb0 0x55d9468a7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.740 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.732+0000 7ff818c85b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9468a8500 0x55d9468a8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.740 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.732+0000 7ff818c85b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9468a90c0 0x55d9468a9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.732+0000 7ff818c85b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9467ad010 con 0x55d9468a7bb0 2023-11-16T16:38:25.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.732+0000 7ff818c85b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9467ff850 con 0x55d9468a8500 2023-11-16T16:38:25.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.732+0000 7ff818c85b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9467f8ce0 con 0x55d9468a90c0 2023-11-16T16:38:25.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.732+0000 7ff80659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9468a7bb0 0x55d9468a7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.732+0000 7ff80659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9468a7bb0 0x55d9468a7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42412/0 (socket says 172.21.15.1:42412) 2023-11-16T16:38:25.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.732+0000 7ff80659c700 1 -- 172.21.15.1:0/2698804013 learned_addr learned my addr 172.21.15.1:0/2698804013 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:25.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.732+0000 7ff806d9d700 1 --2- 172.21.15.1:0/2698804013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9468a90c0 0x55d9468a9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.732+0000 7ff805d9b700 1 --2- 172.21.15.1:0/2698804013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9468a8500 0x55d9468a8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.732+0000 7ff80659c700 1 -- 172.21.15.1:0/2698804013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9468a90c0 msgr2=0x55d9468a9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:25.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.732+0000 7ff80659c700 1 --2- 172.21.15.1:0/2698804013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9468a90c0 0x55d9468a9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.732+0000 7ff80659c700 1 -- 172.21.15.1:0/2698804013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9468a8500 msgr2=0x55d9468a8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:25.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.732+0000 7ff80659c700 1 --2- 172.21.15.1:0/2698804013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9468a8500 0x55d9468a8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.742 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.733+0000 7ff80659c700 1 -- 172.21.15.1:0/2698804013 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d94667a5e0 con 0x55d9468a7bb0 2023-11-16T16:38:25.742 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.733+0000 7ff80659c700 1 --2- 172.21.15.1:0/2698804013 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9468a7bb0 0x55d9468a7fc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8015000 tx=0x7ff7f8020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b694595ab0bf842 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:25.742 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.733+0000 7ff7fcff9700 1 -- 172.21.15.1:0/2698804013 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f802b070 con 0x55d9468a7bb0 2023-11-16T16:38:25.742 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.733+0000 7ff7fcff9700 1 -- 172.21.15.1:0/2698804013 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7f801e040 con 0x55d9468a7bb0 2023-11-16T16:38:25.742 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.733+0000 7ff7fcff9700 1 -- 172.21.15.1:0/2698804013 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f800f7c0 con 0x55d9468a7bb0 2023-11-16T16:38:25.742 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.734+0000 7ff818c85b80 1 -- 172.21.15.1:0/2698804013 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9468a7bb0 msgr2=0x55d9468a7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:25.742 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.734+0000 7ff818c85b80 1 --2- 172.21.15.1:0/2698804013 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9468a7bb0 0x55d9468a7fc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8015000 tx=0x7ff7f8020000 comp rx=0 tx=0).stop 2023-11-16T16:38:25.742 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.734+0000 7ff818c85b80 1 -- 172.21.15.1:0/2698804013 shutdown_connections 2023-11-16T16:38:25.742 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.734+0000 7ff818c85b80 1 --2- 172.21.15.1:0/2698804013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9468a90c0 0x55d9468a9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.742 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.734+0000 7ff818c85b80 1 --2- 172.21.15.1:0/2698804013 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9468a7bb0 0x55d9468a7fc0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.742 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.734+0000 7ff818c85b80 1 --2- 172.21.15.1:0/2698804013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9468a8500 0x55d9468a8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.742 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.734+0000 7ff818c85b80 1 -- 172.21.15.1:0/2698804013 >> 172.21.15.1:0/2698804013 conn(0x55d94689d310 msgr2=0x55d9468a2620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:25.743 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.734+0000 7ff818c85b80 1 -- 172.21.15.1:0/2698804013 shutdown_connections 2023-11-16T16:38:25.743 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.734+0000 7ff818c85b80 1 -- 172.21.15.1:0/2698804013 wait complete. 2023-11-16T16:38:25.743 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.734+0000 7ff818c85b80 1 -- start start 2023-11-16T16:38:25.743 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.734+0000 7ff818c85b80 1 Processor -- start 2023-11-16T16:38:25.743 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.734+0000 7ff818c85b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9468a7bb0 0x55d9468b14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.743 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.734+0000 7ff818c85b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9468a8500 0x55d9468b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.743 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.734+0000 7ff818c85b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9468a90c0 0x55d9468b8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.743 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.734+0000 7ff818c85b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d94679a890 con 0x55d9468a8500 2023-11-16T16:38:25.743 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.734+0000 7ff818c85b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d94679a530 con 0x55d9468a7bb0 2023-11-16T16:38:25.743 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.734+0000 7ff818c85b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d94667c8d0 con 0x55d9468a90c0 2023-11-16T16:38:25.743 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.734+0000 7ff818c85b80 10 client.0 _ll_register_callbacks cb 0x55d9467b8730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:38:25.743 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.735+0000 7ff805d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9468a8500 0x55d9468b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.744 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.735+0000 7ff805d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9468a8500 0x55d9468b19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42420/0 (socket says 172.21.15.1:42420) 2023-11-16T16:38:25.744 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.735+0000 7ff805d9b700 1 -- 172.21.15.1:0/2559407692 learned_addr learned my addr 172.21.15.1:0/2559407692 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:25.744 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.735+0000 7ff80659c700 1 --2- 172.21.15.1:0/2559407692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9468a7bb0 0x55d9468b14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.744 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.735+0000 7ff806d9d700 1 --2- 172.21.15.1:0/2559407692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9468a90c0 0x55d9468b8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.744 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.735+0000 7ff805d9b700 1 -- 172.21.15.1:0/2559407692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9468a90c0 msgr2=0x55d9468b8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:25.744 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.735+0000 7ff805d9b700 1 --2- 172.21.15.1:0/2559407692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9468a90c0 0x55d9468b8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.744 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.735+0000 7ff805d9b700 1 -- 172.21.15.1:0/2559407692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9468a7bb0 msgr2=0x55d9468b14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:25.744 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.735+0000 7ff805d9b700 1 --2- 172.21.15.1:0/2559407692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9468a7bb0 0x55d9468b14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.744 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.735+0000 7ff805d9b700 1 -- 172.21.15.1:0/2559407692 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff808108f90 con 0x55d9468a8500 2023-11-16T16:38:25.744 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.735+0000 7ff805d9b700 1 --2- 172.21.15.1:0/2559407692 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9468a8500 0x55d9468b19f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0099eb0 tx=0x7ff7f009a460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2251c690c43ffdc5 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:25.744 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.735+0000 7ff7edffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:38:25.744 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.735+0000 7ff7edffb700 1 -- 172.21.15.1:0/2559407692 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f00a20e0 con 0x55d9468a8500 2023-11-16T16:38:25.745 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.735+0000 7ff7edffb700 1 -- 172.21.15.1:0/2559407692 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff7f00a2370 con 0x55d9468a8500 2023-11-16T16:38:25.745 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.735+0000 7ff818c85b80 1 -- 172.21.15.1:0/2559407692 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d94689aad0 con 0x55d9468a8500 2023-11-16T16:38:25.745 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.735+0000 7ff7edffb700 1 -- 172.21.15.1:0/2559407692 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7f00b8170 con 0x55d9468a8500 2023-11-16T16:38:25.745 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.735+0000 7ff818c85b80 10 client.15096 fetch_fsmap learned FSMap version 23 2023-11-16T16:38:25.745 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.735+0000 7ff818c85b80 1 -- 172.21.15.1:0/2559407692 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d9467bf140 con 0x55d9468a8500 2023-11-16T16:38:25.745 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.735+0000 7ff7edffb700 1 -- 172.21.15.1:0/2559407692 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7f00c8080 con 0x55d9468a8500 2023-11-16T16:38:25.745 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.736+0000 7ff7edffb700 1 -- 172.21.15.1:0/2559407692 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff7f0098e80 con 0x55d9468a8500 2023-11-16T16:38:25.745 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.736+0000 7ff818c85b80 10 client.15096 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:38:25.745 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.736+0000 7ff818c85b80 10 client.15096 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-16T16:38:25.745 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.736+0000 7ff818c85b80 20 client.15096 populate_metadata read hostname 'smithi001' 2023-11-16T16:38:25.745 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.736+0000 7ff818c85b80 1 --2- 172.21.15.1:0/2559407692 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55d9468bc700 0x55d9468bbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.745 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.736+0000 7ff818c85b80 4 client.15096 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:38:25.746 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.736+0000 7ff818c85b80 1 -- 172.21.15.1:0/2559407692 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d9467bb050 con 0x55d9468bc700 2023-11-16T16:38:25.746 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.736+0000 7ff80659c700 1 --2- 172.21.15.1:0/2559407692 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55d9468bc700 0x55d9468bbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.746 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.736+0000 7ff7edffb700 10 client.15096 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:38:25.746 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.736+0000 7ff80659c700 1 --2- 172.21.15.1:0/2559407692 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55d9468bc700 0x55d9468bbb90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9f27f9cd768831 server_cookie=896829dd858acb04 in_seq=0 out_seq=0 2023-11-16T16:38:25.746 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.737+0000 7ff7edffb700 1 -- 172.21.15.1:0/2559407692 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d9467bb050 con 0x55d9468bc700 2023-11-16T16:38:25.746 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.737+0000 7ff7edffb700 10 client.15096 handle_command_reply: tid=0 2023-11-16T16:38:25.746 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.737+0000 7ff818c85b80 1 client.15096 shutdown 2023-11-16T16:38:25.746 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.738+0000 7ff818c85b80 1 -- 172.21.15.1:0/2559407692 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9468a8500 msgr2=0x55d9468b19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:25.746 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.738+0000 7ff818c85b80 1 --2- 172.21.15.1:0/2559407692 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9468a8500 0x55d9468b19f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0099eb0 tx=0x7ff7f009a460 comp rx=0 tx=0).stop 2023-11-16T16:38:25.746 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.738+0000 7ff818c85b80 1 -- 172.21.15.1:0/2559407692 shutdown_connections 2023-11-16T16:38:25.746 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.738+0000 7ff818c85b80 1 --2- 172.21.15.1:0/2559407692 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55d9468bc700 0x55d9468bbb90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.747 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.738+0000 7ff818c85b80 1 --2- 172.21.15.1:0/2559407692 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9468a90c0 0x55d9468b8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.747 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.738+0000 7ff818c85b80 1 --2- 172.21.15.1:0/2559407692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9468a7bb0 0x55d9468b14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.747 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.738+0000 7ff818c85b80 1 --2- 172.21.15.1:0/2559407692 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9468a8500 0x55d9468b19f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.747 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.738+0000 7ff7edffb700 0 client.15096 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:38:25.747 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.738+0000 7ff818c85b80 1 -- 172.21.15.1:0/2559407692 >> 172.21.15.1:0/2559407692 conn(0x55d94689d310 msgr2=0x55d9468a1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:25.747 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.738+0000 7ff818c85b80 1 -- 172.21.15.1:0/2559407692 shutdown_connections 2023-11-16T16:38:25.747 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.738+0000 7ff818c85b80 1 -- 172.21.15.1:0/2559407692 wait complete. 2023-11-16T16:38:25.747 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.738+0000 7ff818c85b80 20 client.15096 trim_cache size 0 max 16384 2023-11-16T16:38:25.759 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.756+0000 7ff818c85b80 1 Processor -- start 2023-11-16T16:38:25.759 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.756+0000 7ff818c85b80 1 -- start start 2023-11-16T16:38:25.759 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.756+0000 7ff818c85b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9468a7bb0 0x55d9468bbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.759 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.756+0000 7ff818c85b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9468a8500 0x55d9468b89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.759 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.756+0000 7ff818c85b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9468a90c0 0x55d9468abfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.759 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.756+0000 7ff818c85b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d94689cf00 con 0x55d9468a7bb0 2023-11-16T16:38:25.760 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.756+0000 7ff818c85b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9467bb050 con 0x55d9468a8500 2023-11-16T16:38:25.760 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.756+0000 7ff818c85b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d946704ed0 con 0x55d9468a90c0 2023-11-16T16:38:25.760 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.756+0000 7ff80659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9468a7bb0 0x55d9468bbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.760 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.756+0000 7ff80659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9468a7bb0 0x55d9468bbb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42426/0 (socket says 172.21.15.1:42426) 2023-11-16T16:38:25.760 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.756+0000 7ff80659c700 1 -- 172.21.15.1:0/972529020 learned_addr learned my addr 172.21.15.1:0/972529020 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:25.760 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.756+0000 7ff805d9b700 1 --2- 172.21.15.1:0/972529020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9468a8500 0x55d9468b89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.760 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.756+0000 7ff80659c700 1 -- 172.21.15.1:0/972529020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9468a90c0 msgr2=0x55d9468abfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:25.760 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.756+0000 7ff80659c700 1 --2- 172.21.15.1:0/972529020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9468a90c0 0x55d9468abfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.760 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.756+0000 7ff80659c700 1 -- 172.21.15.1:0/972529020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9468a8500 msgr2=0x55d9468b89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:25.760 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.756+0000 7ff80659c700 1 --2- 172.21.15.1:0/972529020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9468a8500 0x55d9468b89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.760 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.756+0000 7ff80659c700 1 -- 172.21.15.1:0/972529020 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d94689eab0 con 0x55d9468a7bb0 2023-11-16T16:38:25.761 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.756+0000 7ff805d9b700 1 --2- 172.21.15.1:0/972529020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9468a8500 0x55d9468b89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:38:25.761 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.756+0000 7ff80659c700 1 --2- 172.21.15.1:0/972529020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9468a7bb0 0x55d9468bbb90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8005010 tx=0x7ff7f80056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26c98f60daa27e7a server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:25.761 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.756+0000 7ff7fd7fa700 1 -- 172.21.15.1:0/972529020 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f802b040 con 0x55d9468a7bb0 2023-11-16T16:38:25.761 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.756+0000 7ff7fd7fa700 1 -- 172.21.15.1:0/972529020 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7f803b5d0 con 0x55d9468a7bb0 2023-11-16T16:38:25.761 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.757+0000 7ff7fd7fa700 1 -- 172.21.15.1:0/972529020 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f800faf0 con 0x55d9468a7bb0 2023-11-16T16:38:25.761 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.757+0000 7ff818c85b80 1 -- 172.21.15.1:0/972529020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9468a7bb0 msgr2=0x55d9468bbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:25.761 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.757+0000 7ff818c85b80 1 --2- 172.21.15.1:0/972529020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9468a7bb0 0x55d9468bbb90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8005010 tx=0x7ff7f80056e0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.761 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.757+0000 7ff818c85b80 1 -- 172.21.15.1:0/972529020 shutdown_connections 2023-11-16T16:38:25.761 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.757+0000 7ff818c85b80 1 --2- 172.21.15.1:0/972529020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9468a90c0 0x55d9468abfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.761 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.757+0000 7ff818c85b80 1 --2- 172.21.15.1:0/972529020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9468a7bb0 0x55d9468bbb90 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.761 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.757+0000 7ff818c85b80 1 --2- 172.21.15.1:0/972529020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9468a8500 0x55d9468b89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.761 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.757+0000 7ff818c85b80 1 -- 172.21.15.1:0/972529020 >> 172.21.15.1:0/972529020 conn(0x55d94689d310 msgr2=0x55d9468af490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:25.762 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.757+0000 7ff818c85b80 1 -- 172.21.15.1:0/972529020 shutdown_connections 2023-11-16T16:38:25.762 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.757+0000 7ff818c85b80 1 -- 172.21.15.1:0/972529020 wait complete. 2023-11-16T16:38:25.762 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.757+0000 7ff818c85b80 1 -- start start 2023-11-16T16:38:25.762 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.758+0000 7ff818c85b80 1 Processor -- start 2023-11-16T16:38:25.766 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.758+0000 7ff818c85b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9468a7bb0 0x55d9468b3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.766 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.758+0000 7ff818c85b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9468a8500 0x55d9468b8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.766 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.758+0000 7ff818c85b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9468a90c0 0x55d9468b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.766 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.758+0000 7ff818c85b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d946910580 con 0x55d9468a90c0 2023-11-16T16:38:25.766 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.758+0000 7ff818c85b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9469106c0 con 0x55d9468a7bb0 2023-11-16T16:38:25.766 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.758+0000 7ff818c85b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d946910800 con 0x55d9468a8500 2023-11-16T16:38:25.766 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.758+0000 7ff818c85b80 10 client.0 _ll_register_callbacks cb 0x7ff7f0099ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:38:25.766 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.758+0000 7ff806d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9468a90c0 0x55d9468b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.767 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.758+0000 7ff806d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9468a90c0 0x55d9468b3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42436/0 (socket says 172.21.15.1:42436) 2023-11-16T16:38:25.767 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.758+0000 7ff806d9d700 1 -- 172.21.15.1:0/406865205 learned_addr learned my addr 172.21.15.1:0/406865205 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:25.767 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.758+0000 7ff80659c700 1 --2- 172.21.15.1:0/406865205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9468a7bb0 0x55d9468b3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.767 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.758+0000 7ff805d9b700 1 --2- 172.21.15.1:0/406865205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9468a8500 0x55d9468b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.767 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.758+0000 7ff806d9d700 1 -- 172.21.15.1:0/406865205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9468a8500 msgr2=0x55d9468b8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:25.767 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.758+0000 7ff806d9d700 1 --2- 172.21.15.1:0/406865205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9468a8500 0x55d9468b8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.767 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.758+0000 7ff806d9d700 1 -- 172.21.15.1:0/406865205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9468a7bb0 msgr2=0x55d9468b3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:25.767 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.758+0000 7ff806d9d700 1 --2- 172.21.15.1:0/406865205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9468a7bb0 0x55d9468b3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.767 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.758+0000 7ff806d9d700 1 -- 172.21.15.1:0/406865205 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d94667a5e0 con 0x55d9468a90c0 2023-11-16T16:38:25.767 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.758+0000 7ff806d9d700 1 --2- 172.21.15.1:0/406865205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9468a90c0 0x55d9468b3a50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff800007b00 tx=0x7ff800007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5208d8483d67cc8 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:25.767 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.758+0000 7ff7edffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:38:25.767 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.759+0000 7ff7edffb700 1 -- 172.21.15.1:0/406865205 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff800012b00 con 0x55d9468a90c0 2023-11-16T16:38:25.768 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.759+0000 7ff7edffb700 1 -- 172.21.15.1:0/406865205 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff800012c60 con 0x55d9468a90c0 2023-11-16T16:38:25.768 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.759+0000 7ff818c85b80 1 -- 172.21.15.1:0/406865205 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff7f801e040 con 0x55d9468a90c0 2023-11-16T16:38:25.768 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.759+0000 7ff7edffb700 1 -- 172.21.15.1:0/406865205 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff800012e20 con 0x55d9468a90c0 2023-11-16T16:38:25.768 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.759+0000 7ff818c85b80 10 client.15102 fetch_fsmap learned FSMap version 23 2023-11-16T16:38:25.768 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.759+0000 7ff818c85b80 1 -- 172.21.15.1:0/406865205 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d946626d90 con 0x55d9468a90c0 2023-11-16T16:38:25.768 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.759+0000 7ff7edffb700 1 -- 172.21.15.1:0/406865205 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7f801e040 con 0x55d9468a90c0 2023-11-16T16:38:25.768 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.759+0000 7ff7edffb700 1 -- 172.21.15.1:0/406865205 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff800014040 con 0x55d9468a90c0 2023-11-16T16:38:25.768 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.759+0000 7ff818c85b80 10 client.15102 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:38:25.768 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.759+0000 7ff818c85b80 10 client.15102 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-16T16:38:25.768 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.759+0000 7ff818c85b80 20 client.15102 populate_metadata read hostname 'smithi001' 2023-11-16T16:38:25.768 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.759+0000 7ff818c85b80 1 --2- 172.21.15.1:0/406865205 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55d946915680 0x55d946911ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:25.768 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.759+0000 7ff818c85b80 4 client.15102 mds_command: new command op to 24521 tid=0[{"prefix": "status"}] 2023-11-16T16:38:25.769 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.759+0000 7ff818c85b80 1 -- 172.21.15.1:0/406865205 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d9467c7610 con 0x55d946915680 2023-11-16T16:38:25.769 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.759+0000 7ff80659c700 1 --2- 172.21.15.1:0/406865205 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55d946915680 0x55d946911ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:25.769 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.760+0000 7ff7edffb700 10 client.15102 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:38:25.769 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.760+0000 7ff80659c700 1 --2- 172.21.15.1:0/406865205 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55d946915680 0x55d946911ff0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af93042efba603df server_cookie=6d18f797cfec1217 in_seq=0 out_seq=0 2023-11-16T16:38:25.769 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.760+0000 7ff7edffb700 1 -- 172.21.15.1:0/406865205 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55d9467c7610 con 0x55d946915680 2023-11-16T16:38:25.769 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.760+0000 7ff7edffb700 10 client.15102 handle_command_reply: tid=0 2023-11-16T16:38:25.769 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.760+0000 7ff818c85b80 1 client.15102 shutdown 2023-11-16T16:38:25.769 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.760+0000 7ff818c85b80 1 -- 172.21.15.1:0/406865205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9468a90c0 msgr2=0x55d9468b3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:25.769 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.760+0000 7ff818c85b80 1 --2- 172.21.15.1:0/406865205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9468a90c0 0x55d9468b3a50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff800007b00 tx=0x7ff800007e10 comp rx=0 tx=0).stop 2023-11-16T16:38:25.769 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.760+0000 7ff818c85b80 1 -- 172.21.15.1:0/406865205 shutdown_connections 2023-11-16T16:38:25.769 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.760+0000 7ff818c85b80 1 --2- 172.21.15.1:0/406865205 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55d946915680 0x55d946911ff0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.769 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.760+0000 7ff818c85b80 1 --2- 172.21.15.1:0/406865205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9468a90c0 0x55d9468b3a50 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.769 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.760+0000 7ff818c85b80 1 --2- 172.21.15.1:0/406865205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d9468a7bb0 0x55d9468b3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.770 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.760+0000 7ff818c85b80 1 --2- 172.21.15.1:0/406865205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d9468a8500 0x55d9468b8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.770 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.760+0000 7ff818c85b80 1 -- 172.21.15.1:0/406865205 >> 172.21.15.1:0/406865205 conn(0x55d94689d310 msgr2=0x55d9468af490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:25.770 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.760+0000 7ff7edffb700 0 client.15102 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:38:25.770 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.760+0000 7ff818c85b80 1 -- 172.21.15.1:0/406865205 shutdown_connections 2023-11-16T16:38:25.770 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.760+0000 7ff818c85b80 1 -- 172.21.15.1:0/406865205 wait complete. 2023-11-16T16:38:25.770 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.760+0000 7ff818c85b80 20 client.15102 trim_cache size 0 max 16384 2023-11-16T16:38:25.770 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:38:25.770 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "35d9b010-849d-11ee-8dbe-212e2dc638e7", 2023-11-16T16:38:25.770 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 0, 2023-11-16T16:38:25.771 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24521, 2023-11-16T16:38:25.771 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2023-11-16T16:38:25.771 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2023-11-16T16:38:25.771 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2023-11-16T16:38:25.771 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 71.839168439000005, 2023-11-16T16:38:25.771 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 23, 2023-11-16T16:38:25.771 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 86, 2023-11-16T16:38:25.771 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 84, 2023-11-16T16:38:25.771 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 76.581318523999997 2023-11-16T16:38:25.771 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:38:25.772 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.761+0000 7ff80caf9700 1 -- 172.21.15.1:0/2116979276 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7ff7f40a1260 msgr2=0x7ff7f40a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:25.772 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.761+0000 7ff80caf9700 1 --2- 172.21.15.1:0/2116979276 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7ff7f40a1260 0x7ff7f40a3710 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff7f800b2e0 tx=0x7ff7f8005fb0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.772 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.761+0000 7ff80caf9700 1 -- 172.21.15.1:0/2116979276 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff8080febf0 msgr2=0x7ff8081964f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:25.772 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.761+0000 7ff80caf9700 1 --2- 172.21.15.1:0/2116979276 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff8080febf0 0x7ff8081964f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff7f000e9a0 tx=0x7ff7f000ecb0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.772 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.761+0000 7ff80caf9700 1 -- 172.21.15.1:0/2116979276 shutdown_connections 2023-11-16T16:38:25.773 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.761+0000 7ff80caf9700 1 --2- 172.21.15.1:0/2116979276 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7ff7f40a1260 0x7ff7f40a3710 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.773 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.761+0000 7ff80caf9700 1 --2- 172.21.15.1:0/2116979276 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff8081017f0 0x7ff80819d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.773 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.761+0000 7ff80caf9700 1 --2- 172.21.15.1:0/2116979276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8080fe260 0x7ff808195fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.773 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.761+0000 7ff80caf9700 1 --2- 172.21.15.1:0/2116979276 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff8080febf0 0x7ff8081964f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:25.773 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.761+0000 7ff80caf9700 1 -- 172.21.15.1:0/2116979276 >> 172.21.15.1:0/2116979276 conn(0x7ff8080f9eb0 msgr2=0x7ff808104a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:25.773 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.761+0000 7ff80caf9700 1 -- 172.21.15.1:0/2116979276 shutdown_connections 2023-11-16T16:38:25.773 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:25.761+0000 7ff80caf9700 1 -- 172.21.15.1:0/2116979276 wait complete. 2023-11-16T16:38:25.775 INFO:tasks.cephadm:Setup complete, yielding 2023-11-16T16:38:25.775 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-11-16T16:38:25.785 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi001.front.sepia.ceph.com 2023-11-16T16:38:25.785 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- bash -c 'ceph orch status' 2023-11-16T16:38:25.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:25 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3694382309' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:38:25.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:25 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/2928545345' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:38:25.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:25 smithi001 ceph-mon[98800]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:38:26.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:25 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/3694382309' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:38:26.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:25 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/2928545345' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:38:26.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:25 smithi026 ceph-mon[100112]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:38:26.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:25 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/3694382309' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:38:26.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:25 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/2928545345' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:38:26.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:25 smithi053 ceph-mon[100638]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:38:26.437 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:38:27.761 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.756+0000 7f5314eca700 1 -- 172.21.15.1:0/406538538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5310101ea0 msgr2=0x7f5310108760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:27.761 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.756+0000 7f5314eca700 1 --2- 172.21.15.1:0/406538538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5310101ea0 0x7f5310108760 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f530800b0a0 tx=0x7f530800b3b0 comp rx=0 tx=0).stop 2023-11-16T16:38:27.762 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.756+0000 7f5314eca700 1 -- 172.21.15.1:0/406538538 shutdown_connections 2023-11-16T16:38:27.762 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.756+0000 7f5314eca700 1 --2- 172.21.15.1:0/406538538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5310101ea0 0x7f5310108760 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:27.762 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.756+0000 7f5314eca700 1 --2- 172.21.15.1:0/406538538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5310100310 0x7f5310100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:27.762 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.756+0000 7f5314eca700 1 --2- 172.21.15.1:0/406538538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5310101510 0x7f5310101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:27.762 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.756+0000 7f5314eca700 1 -- 172.21.15.1:0/406538538 >> 172.21.15.1:0/406538538 conn(0x7f53100fb960 msgr2=0x7f53100fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:27.762 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.756+0000 7f5314eca700 1 -- 172.21.15.1:0/406538538 shutdown_connections 2023-11-16T16:38:27.762 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.756+0000 7f5314eca700 1 -- 172.21.15.1:0/406538538 wait complete. 2023-11-16T16:38:27.762 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.757+0000 7f5314eca700 1 Processor -- start 2023-11-16T16:38:27.763 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.757+0000 7f5314eca700 1 -- start start 2023-11-16T16:38:27.763 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.757+0000 7f5314eca700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5310100310 0x7f53101960a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:27.763 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.757+0000 7f5314eca700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5310101510 0x7f53101965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:27.763 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.757+0000 7f5314eca700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5310101ea0 0x7f531019d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:27.763 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.757+0000 7f5314eca700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531019dc30 con 0x7f5310100310 2023-11-16T16:38:27.763 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.757+0000 7f5314eca700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531019dd70 con 0x7f5310101ea0 2023-11-16T16:38:27.763 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.757+0000 7f5314eca700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531019df00 con 0x7f5310101510 2023-11-16T16:38:27.763 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.757+0000 7f530e59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5310100310 0x7f53101960a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:27.763 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.757+0000 7f530e59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5310100310 0x7f53101960a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42444/0 (socket says 172.21.15.1:42444) 2023-11-16T16:38:27.764 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.757+0000 7f530e59c700 1 -- 172.21.15.1:0/2651245608 learned_addr learned my addr 172.21.15.1:0/2651245608 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:27.764 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.757+0000 7f530ed9d700 1 --2- 172.21.15.1:0/2651245608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5310101ea0 0x7f531019d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:27.764 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.758+0000 7f530dd9b700 1 --2- 172.21.15.1:0/2651245608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5310101510 0x7f53101965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:27.764 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.758+0000 7f530e59c700 1 -- 172.21.15.1:0/2651245608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5310101510 msgr2=0x7f53101965e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:27.764 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.758+0000 7f530e59c700 1 --2- 172.21.15.1:0/2651245608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5310101510 0x7f53101965e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:27.764 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.758+0000 7f530e59c700 1 -- 172.21.15.1:0/2651245608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5310101ea0 msgr2=0x7f531019d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:27.764 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.758+0000 7f530e59c700 1 --2- 172.21.15.1:0/2651245608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5310101ea0 0x7f531019d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:27.764 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.758+0000 7f530e59c700 1 -- 172.21.15.1:0/2651245608 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5308009d00 con 0x7f5310100310 2023-11-16T16:38:27.765 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.758+0000 7f530dd9b700 1 --2- 172.21.15.1:0/2651245608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5310101510 0x7f53101965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:38:27.765 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.758+0000 7f530e59c700 1 --2- 172.21.15.1:0/2651245608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5310100310 0x7f53101960a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f530000d9b0 tx=0x7f530000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:27.765 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.758+0000 7f53077fe700 1 -- 172.21.15.1:0/2651245608 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5300009930 con 0x7f5310100310 2023-11-16T16:38:27.765 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.758+0000 7f53077fe700 1 -- 172.21.15.1:0/2651245608 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5300010460 con 0x7f5310100310 2023-11-16T16:38:27.765 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.758+0000 7f5314eca700 1 -- 172.21.15.1:0/2651245608 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f531010bc90 con 0x7f5310100310 2023-11-16T16:38:27.765 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.758+0000 7f53077fe700 1 -- 172.21.15.1:0/2651245608 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530000f620 con 0x7f5310100310 2023-11-16T16:38:27.765 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.758+0000 7f5314eca700 1 -- 172.21.15.1:0/2651245608 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f531010bb00 con 0x7f5310100310 2023-11-16T16:38:27.765 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.759+0000 7f5314eca700 1 -- 172.21.15.1:0/2651245608 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53101077f0 con 0x7f5310100310 2023-11-16T16:38:27.766 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.760+0000 7f53077fe700 1 -- 172.21.15.1:0/2651245608 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f5300010860 con 0x7f5310100310 2023-11-16T16:38:27.767 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.761+0000 7f53077fe700 1 --2- 172.21.15.1:0/2651245608 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f52fc0a11e0 0x7f52fc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:27.767 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.761+0000 7f530dd9b700 1 --2- 172.21.15.1:0/2651245608 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f52fc0a11e0 0x7f52fc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:27.767 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.761+0000 7f53077fe700 1 -- 172.21.15.1:0/2651245608 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f53000cc560 con 0x7f5310100310 2023-11-16T16:38:27.767 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.761+0000 7f530dd9b700 1 --2- 172.21.15.1:0/2651245608 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f52fc0a11e0 0x7f52fc0a3690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f52f8005fd0 tx=0x7f52f8005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:27.768 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.766+0000 7f53077fe700 1 -- 172.21.15.1:0/2651245608 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5300095b50 con 0x7f5310100310 2023-11-16T16:38:27.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:27 smithi053 ceph-mon[100638]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:38:27.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:27 smithi001 ceph-mon[98800]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:38:27.977 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.974+0000 7f5314eca700 1 -- 172.21.15.1:0/2651245608 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5310061a30 con 0x7f52fc0a11e0 2023-11-16T16:38:27.977 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.976+0000 7f53077fe700 1 -- 172.21.15.1:0/2651245608 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f5310061a30 con 0x7f52fc0a11e0 2023-11-16T16:38:27.978 INFO:teuthology.orchestra.run.smithi001.stdout:Backend: cephadm 2023-11-16T16:38:27.978 INFO:teuthology.orchestra.run.smithi001.stdout:Available: Yes 2023-11-16T16:38:27.978 INFO:teuthology.orchestra.run.smithi001.stdout:Paused: No 2023-11-16T16:38:27.980 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.977+0000 7f5314eca700 1 -- 172.21.15.1:0/2651245608 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f52fc0a11e0 msgr2=0x7f52fc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:27.980 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.977+0000 7f5314eca700 1 --2- 172.21.15.1:0/2651245608 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f52fc0a11e0 0x7f52fc0a3690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f52f8005fd0 tx=0x7f52f8005e30 comp rx=0 tx=0).stop 2023-11-16T16:38:27.980 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.977+0000 7f5314eca700 1 -- 172.21.15.1:0/2651245608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5310100310 msgr2=0x7f53101960a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:27.980 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.977+0000 7f5314eca700 1 --2- 172.21.15.1:0/2651245608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5310100310 0x7f53101960a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f530000d9b0 tx=0x7f530000dcc0 comp rx=0 tx=0).stop 2023-11-16T16:38:27.980 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.977+0000 7f5314eca700 1 -- 172.21.15.1:0/2651245608 shutdown_connections 2023-11-16T16:38:27.980 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.977+0000 7f5314eca700 1 --2- 172.21.15.1:0/2651245608 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f52fc0a11e0 0x7f52fc0a3690 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:27.981 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.977+0000 7f5314eca700 1 --2- 172.21.15.1:0/2651245608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5310101ea0 0x7f531019d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:27.981 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.977+0000 7f5314eca700 1 --2- 172.21.15.1:0/2651245608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5310100310 0x7f53101960a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:27.981 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.978+0000 7f5314eca700 1 --2- 172.21.15.1:0/2651245608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5310101510 0x7f53101965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:27.981 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.978+0000 7f5314eca700 1 -- 172.21.15.1:0/2651245608 >> 172.21.15.1:0/2651245608 conn(0x7f53100fb960 msgr2=0x7f5310104930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:27.981 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.978+0000 7f5314eca700 1 -- 172.21.15.1:0/2651245608 shutdown_connections 2023-11-16T16:38:27.981 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:27.978+0000 7f5314eca700 1 -- 172.21.15.1:0/2651245608 wait complete. 2023-11-16T16:38:28.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:27 smithi026 ceph-mon[100112]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:38:28.397 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- bash -c 'ceph orch ps' 2023-11-16T16:38:28.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:28 smithi001 ceph-mon[98800]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:38:28.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:28 smithi053 ceph-mon[100638]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:38:29.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:28 smithi026 ceph-mon[100112]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:38:29.055 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:38:29.815 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:29 smithi001 ceph-mon[98800]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-11-16T16:38:29.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:29 smithi053 ceph-mon[100638]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-11-16T16:38:30.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:29 smithi026 ceph-mon[100112]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-11-16T16:38:30.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.269+0000 7ff3aaed7700 1 -- 172.21.15.1:0/3405529720 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3a40ff1e0 msgr2=0x7ff3a40ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:30.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.269+0000 7ff3aaed7700 1 --2- 172.21.15.1:0/3405529720 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3a40ff1e0 0x7ff3a40ff5f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff398009990 tx=0x7ff398009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:30.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.269+0000 7ff3aaed7700 1 -- 172.21.15.1:0/3405529720 shutdown_connections 2023-11-16T16:38:30.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.269+0000 7ff3aaed7700 1 --2- 172.21.15.1:0/3405529720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3a4100d70 0x7ff3a4107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:30.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.269+0000 7ff3aaed7700 1 --2- 172.21.15.1:0/3405529720 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3a40ff1e0 0x7ff3a40ff5f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:30.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.269+0000 7ff3aaed7700 1 --2- 172.21.15.1:0/3405529720 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3a41003e0 0x7ff3a4100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:30.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.269+0000 7ff3aaed7700 1 -- 172.21.15.1:0/3405529720 >> 172.21.15.1:0/3405529720 conn(0x7ff3a40fa7f0 msgr2=0x7ff3a40fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:30.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.269+0000 7ff3aaed7700 1 -- 172.21.15.1:0/3405529720 shutdown_connections 2023-11-16T16:38:30.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.269+0000 7ff3aaed7700 1 -- 172.21.15.1:0/3405529720 wait complete. 2023-11-16T16:38:30.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.270+0000 7ff3aaed7700 1 Processor -- start 2023-11-16T16:38:30.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.270+0000 7ff3aaed7700 1 -- start start 2023-11-16T16:38:30.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.270+0000 7ff3aaed7700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3a40ff1e0 0x7ff3a4103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:30.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.270+0000 7ff3aaed7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3a41003e0 0x7ff3a41017c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:30.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.270+0000 7ff3aaed7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3a4100d70 0x7ff3a4101d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:30.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.270+0000 7ff3aaed7700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a4102480 con 0x7ff3a40ff1e0 2023-11-16T16:38:30.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.270+0000 7ff3aaed7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a41025f0 con 0x7ff3a41003e0 2023-11-16T16:38:30.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.270+0000 7ff3aaed7700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a41a01b0 con 0x7ff3a4100d70 2023-11-16T16:38:30.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.270+0000 7ff3a8c73700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3a40ff1e0 0x7ff3a4103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:30.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.270+0000 7ff3a8c73700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3a40ff1e0 0x7ff3a4103170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42458/0 (socket says 172.21.15.1:42458) 2023-11-16T16:38:30.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.270+0000 7ff3a8c73700 1 -- 172.21.15.1:0/1120483520 learned_addr learned my addr 172.21.15.1:0/1120483520 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:30.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.271+0000 7ff3a9474700 1 --2- 172.21.15.1:0/1120483520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3a4100d70 0x7ff3a4101d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:30.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.271+0000 7ff3a3fff700 1 --2- 172.21.15.1:0/1120483520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3a41003e0 0x7ff3a41017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:30.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.271+0000 7ff3a8c73700 1 -- 172.21.15.1:0/1120483520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3a4100d70 msgr2=0x7ff3a4101d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:30.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.271+0000 7ff3a8c73700 1 --2- 172.21.15.1:0/1120483520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3a4100d70 0x7ff3a4101d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:30.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.271+0000 7ff3a8c73700 1 -- 172.21.15.1:0/1120483520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3a41003e0 msgr2=0x7ff3a41017c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:30.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.271+0000 7ff3a8c73700 1 --2- 172.21.15.1:0/1120483520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3a41003e0 0x7ff3a41017c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:30.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.271+0000 7ff3a8c73700 1 -- 172.21.15.1:0/1120483520 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3980095f0 con 0x7ff3a40ff1e0 2023-11-16T16:38:30.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.271+0000 7ff3a8c73700 1 --2- 172.21.15.1:0/1120483520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3a40ff1e0 0x7ff3a4103170 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff3980095c0 tx=0x7ff39800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:30.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.271+0000 7ff3a1ffb700 1 -- 172.21.15.1:0/1120483520 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39801d070 con 0x7ff3a40ff1e0 2023-11-16T16:38:30.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.271+0000 7ff3a1ffb700 1 -- 172.21.15.1:0/1120483520 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff398005510 con 0x7ff3a40ff1e0 2023-11-16T16:38:30.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.271+0000 7ff3aaed7700 1 -- 172.21.15.1:0/1120483520 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3a410ac10 con 0x7ff3a40ff1e0 2023-11-16T16:38:30.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.271+0000 7ff3a1ffb700 1 -- 172.21.15.1:0/1120483520 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff398004ae0 con 0x7ff3a40ff1e0 2023-11-16T16:38:30.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.271+0000 7ff3aaed7700 1 -- 172.21.15.1:0/1120483520 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3a410aa80 con 0x7ff3a40ff1e0 2023-11-16T16:38:30.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.272+0000 7ff3aaed7700 1 -- 172.21.15.1:0/1120483520 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3a41a07a0 con 0x7ff3a40ff1e0 2023-11-16T16:38:30.277 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.274+0000 7ff3a1ffb700 1 -- 172.21.15.1:0/1120483520 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7ff398004ca0 con 0x7ff3a40ff1e0 2023-11-16T16:38:30.277 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.275+0000 7ff3a1ffb700 1 --2- 172.21.15.1:0/1120483520 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7ff3940a1260 0x7ff3940a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:30.277 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.275+0000 7ff3a3fff700 1 --2- 172.21.15.1:0/1120483520 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7ff3940a1260 0x7ff3940a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:30.277 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.275+0000 7ff3a1ffb700 1 -- 172.21.15.1:0/1120483520 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff3980cdec0 con 0x7ff3a40ff1e0 2023-11-16T16:38:30.277 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.275+0000 7ff3a3fff700 1 --2- 172.21.15.1:0/1120483520 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7ff3940a1260 0x7ff3940a3710 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff390005950 tx=0x7ff3900058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:30.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.278+0000 7ff3a1ffb700 1 -- 172.21.15.1:0/1120483520 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff398097500 con 0x7ff3a40ff1e0 2023-11-16T16:38:30.485 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.483+0000 7ff3aaed7700 1 -- 172.21.15.1:0/1120483520 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff3a41a0e10 con 0x7ff3940a1260 2023-11-16T16:38:30.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.504+0000 7ff3a1ffb700 1 -- 172.21.15.1:0/1120483520 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7ff3a41a0e10 con 0x7ff3940a1260 2023-11-16T16:38:30.506 INFO:teuthology.orchestra.run.smithi001.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-11-16T16:38:30.506 INFO:teuthology.orchestra.run.smithi001.stdout:mds.a smithi001 running (84s) 10s ago 83s 16.5M - 18.0.0-3905-g5e4a743a d5b1f852d682 e59f1e9a7cf3 2023-11-16T16:38:30.506 INFO:teuthology.orchestra.run.smithi001.stdout:mds.b smithi026 running (81s) 10s ago 81s 32.7M - 18.0.0-3905-g5e4a743a d5b1f852d682 d1169a3a3254 2023-11-16T16:38:30.506 INFO:teuthology.orchestra.run.smithi001.stdout:mds.c smithi053 running (93s) 10s ago 92s 19.2M - 18.0.0-3905-g5e4a743a d5b1f852d682 3ad3d4809900 2023-11-16T16:38:30.506 INFO:teuthology.orchestra.run.smithi001.stdout:mds.d smithi001 running (100s) 10s ago 99s 16.5M - 18.0.0-3905-g5e4a743a d5b1f852d682 c10f9f6ec765 2023-11-16T16:38:30.506 INFO:teuthology.orchestra.run.smithi001.stdout:mds.e smithi026 running (97s) 10s ago 97s 20.4M - 18.0.0-3905-g5e4a743a d5b1f852d682 288fc61e40ca 2023-11-16T16:38:30.506 INFO:teuthology.orchestra.run.smithi001.stdout:mds.f smithi053 running (95s) 10s ago 95s 17.9M - 18.0.0-3905-g5e4a743a d5b1f852d682 2cc821f71aaa 2023-11-16T16:38:30.507 INFO:teuthology.orchestra.run.smithi001.stdout:mds.g smithi001 running (86s) 10s ago 85s 16.4M - 18.0.0-3905-g5e4a743a d5b1f852d682 3b7913e030f8 2023-11-16T16:38:30.507 INFO:teuthology.orchestra.run.smithi001.stdout:mds.h smithi026 running (90s) 10s ago 90s 21.5M - 18.0.0-3905-g5e4a743a d5b1f852d682 0b2aed59634c 2023-11-16T16:38:30.507 INFO:teuthology.orchestra.run.smithi001.stdout:mds.i smithi053 running (88s) 10s ago 88s 19.3M - 18.0.0-3905-g5e4a743a d5b1f852d682 9e6d073b0887 2023-11-16T16:38:30.507 INFO:teuthology.orchestra.run.smithi001.stdout:mds.j smithi001 running (102s) 10s ago 101s 16.3M - 18.0.0-3905-g5e4a743a d5b1f852d682 d89b1b8e5201 2023-11-16T16:38:30.507 INFO:teuthology.orchestra.run.smithi001.stdout:mds.k smithi026 running (104s) 10s ago 104s 14.0M - 18.0.0-3905-g5e4a743a d5b1f852d682 b53348018d34 2023-11-16T16:38:30.507 INFO:teuthology.orchestra.run.smithi001.stdout:mds.l smithi053 running (106s) 10s ago 106s 15.5M - 18.0.0-3905-g5e4a743a d5b1f852d682 88cb45acd15c 2023-11-16T16:38:30.507 INFO:teuthology.orchestra.run.smithi001.stdout:mgr.x smithi001 *:9283,8765 running (8m) 10s ago 8m 516M - 18.0.0-3905-g5e4a743a d5b1f852d682 f1661433f611 2023-11-16T16:38:30.507 INFO:teuthology.orchestra.run.smithi001.stdout:mgr.y smithi026 *:8443,8765 running (6m) 10s ago 6m 446M - 18.0.0-3905-g5e4a743a d5b1f852d682 298db30201e1 2023-11-16T16:38:30.507 INFO:teuthology.orchestra.run.smithi001.stdout:mgr.z smithi053 *:8443,8765 running (6m) 10s ago 6m 448M - 18.0.0-3905-g5e4a743a d5b1f852d682 c996c0f1fc57 2023-11-16T16:38:30.507 INFO:teuthology.orchestra.run.smithi001.stdout:mon.a smithi001 running (8m) 10s ago 8m 95.6M 2048M 18.0.0-3905-g5e4a743a d5b1f852d682 86dd16c97ab9 2023-11-16T16:38:30.507 INFO:teuthology.orchestra.run.smithi001.stdout:mon.b smithi026 running (6m) 10s ago 6m 61.8M 2048M 18.0.0-3905-g5e4a743a d5b1f852d682 a9021255676c 2023-11-16T16:38:30.508 INFO:teuthology.orchestra.run.smithi001.stdout:mon.c smithi053 running (6m) 10s ago 6m 61.6M 2048M 18.0.0-3905-g5e4a743a d5b1f852d682 f3182d4a36b4 2023-11-16T16:38:30.508 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0 smithi001 running (6m) 10s ago 6m 66.9M 4293M 18.0.0-3905-g5e4a743a d5b1f852d682 4ca646e44d36 2023-11-16T16:38:30.508 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1 smithi026 running (5m) 10s ago 5m 74.5M 4293M 18.0.0-3905-g5e4a743a d5b1f852d682 62ad9cc34f87 2023-11-16T16:38:30.508 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2 smithi053 running (5m) 10s ago 5m 71.9M 4293M 18.0.0-3905-g5e4a743a d5b1f852d682 79df99ab9381 2023-11-16T16:38:30.508 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3 smithi001 running (5m) 10s ago 5m 63.1M 4293M 18.0.0-3905-g5e4a743a d5b1f852d682 f06347de187d 2023-11-16T16:38:30.508 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4 smithi026 running (4m) 10s ago 4m 58.9M 4293M 18.0.0-3905-g5e4a743a d5b1f852d682 b2f42eb22ae3 2023-11-16T16:38:30.508 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5 smithi053 running (4m) 10s ago 4m 63.1M 4293M 18.0.0-3905-g5e4a743a d5b1f852d682 be3db03d2026 2023-11-16T16:38:30.508 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6 smithi001 running (3m) 10s ago 3m 61.0M 4293M 18.0.0-3905-g5e4a743a d5b1f852d682 8948238b52b7 2023-11-16T16:38:30.508 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7 smithi026 running (3m) 10s ago 3m 61.3M 4293M 18.0.0-3905-g5e4a743a d5b1f852d682 f277ea3def7f 2023-11-16T16:38:30.508 INFO:teuthology.orchestra.run.smithi001.stdout:osd.8 smithi053 running (3m) 10s ago 3m 58.9M 4293M 18.0.0-3905-g5e4a743a d5b1f852d682 842be9d83789 2023-11-16T16:38:30.508 INFO:teuthology.orchestra.run.smithi001.stdout:osd.9 smithi001 running (2m) 10s ago 2m 55.0M 4293M 18.0.0-3905-g5e4a743a d5b1f852d682 fec0f97f16d0 2023-11-16T16:38:30.509 INFO:teuthology.orchestra.run.smithi001.stdout:osd.10 smithi026 running (2m) 10s ago 2m 60.6M 4293M 18.0.0-3905-g5e4a743a d5b1f852d682 0e3f8c81a82a 2023-11-16T16:38:30.509 INFO:teuthology.orchestra.run.smithi001.stdout:osd.11 smithi053 running (2m) 10s ago 2m 52.5M 4293M 18.0.0-3905-g5e4a743a d5b1f852d682 2fdc94fa40ea 2023-11-16T16:38:30.509 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.506+0000 7ff3aaed7700 1 -- 172.21.15.1:0/1120483520 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7ff3940a1260 msgr2=0x7ff3940a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:30.509 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.506+0000 7ff3aaed7700 1 --2- 172.21.15.1:0/1120483520 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7ff3940a1260 0x7ff3940a3710 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff390005950 tx=0x7ff3900058e0 comp rx=0 tx=0).stop 2023-11-16T16:38:30.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.506+0000 7ff3aaed7700 1 -- 172.21.15.1:0/1120483520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3a40ff1e0 msgr2=0x7ff3a4103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:30.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.506+0000 7ff3aaed7700 1 --2- 172.21.15.1:0/1120483520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3a40ff1e0 0x7ff3a4103170 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff3980095c0 tx=0x7ff39800f690 comp rx=0 tx=0).stop 2023-11-16T16:38:30.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.506+0000 7ff3aaed7700 1 -- 172.21.15.1:0/1120483520 shutdown_connections 2023-11-16T16:38:30.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.506+0000 7ff3aaed7700 1 --2- 172.21.15.1:0/1120483520 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7ff3940a1260 0x7ff3940a3710 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:30.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.506+0000 7ff3aaed7700 1 --2- 172.21.15.1:0/1120483520 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff3a4100d70 0x7ff3a4101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:30.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.506+0000 7ff3aaed7700 1 --2- 172.21.15.1:0/1120483520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff3a40ff1e0 0x7ff3a4103170 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:30.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.506+0000 7ff3aaed7700 1 --2- 172.21.15.1:0/1120483520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3a41003e0 0x7ff3a41017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:30.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.506+0000 7ff3aaed7700 1 -- 172.21.15.1:0/1120483520 >> 172.21.15.1:0/1120483520 conn(0x7ff3a40fa7f0 msgr2=0x7ff3a4068630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:30.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.506+0000 7ff3aaed7700 1 -- 172.21.15.1:0/1120483520 shutdown_connections 2023-11-16T16:38:30.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:30.506+0000 7ff3aaed7700 1 -- 172.21.15.1:0/1120483520 wait complete. 2023-11-16T16:38:30.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:30 smithi053 ceph-mon[100638]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:38:30.860 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:30 smithi001 ceph-mon[98800]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:38:30.906 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- bash -c 'ceph orch ls' 2023-11-16T16:38:31.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:30 smithi026 ceph-mon[100112]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:38:31.560 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:38:32.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.829+0000 7f35b5c7e700 1 -- 172.21.15.1:0/2459908034 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35b0106010 msgr2=0x7f35b0108410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:32.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.829+0000 7f35b5c7e700 1 --2- 172.21.15.1:0/2459908034 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35b0106010 0x7f35b0108410 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f35a800b0a0 tx=0x7f35a800b3b0 comp rx=0 tx=0).stop 2023-11-16T16:38:32.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.829+0000 7f35b5c7e700 1 -- 172.21.15.1:0/2459908034 shutdown_connections 2023-11-16T16:38:32.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.829+0000 7f35b5c7e700 1 --2- 172.21.15.1:0/2459908034 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35b0106010 0x7f35b0108410 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:32.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.829+0000 7f35b5c7e700 1 --2- 172.21.15.1:0/2459908034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35b0100dd0 0x7f35b01031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:32.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.829+0000 7f35b5c7e700 1 --2- 172.21.15.1:0/2459908034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35b01036f0 0x7f35b0105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:32.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.829+0000 7f35b5c7e700 1 -- 172.21.15.1:0/2459908034 >> 172.21.15.1:0/2459908034 conn(0x7f35b00fa7f0 msgr2=0x7f35b00fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:32.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.829+0000 7f35b5c7e700 1 -- 172.21.15.1:0/2459908034 shutdown_connections 2023-11-16T16:38:32.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.829+0000 7f35b5c7e700 1 -- 172.21.15.1:0/2459908034 wait complete. 2023-11-16T16:38:32.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.830+0000 7f35b5c7e700 1 Processor -- start 2023-11-16T16:38:32.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.830+0000 7f35b5c7e700 1 -- start start 2023-11-16T16:38:32.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.830+0000 7f35b5c7e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35b0100dd0 0x7f35b0196170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:32.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.830+0000 7f35b5c7e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35b01036f0 0x7f35b01966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:32.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.830+0000 7f35b5c7e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35b0106010 0x7f35b019d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:32.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.830+0000 7f35b5c7e700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b019dd00 con 0x7f35b0106010 2023-11-16T16:38:32.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.830+0000 7f35b5c7e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b019de40 con 0x7f35b01036f0 2023-11-16T16:38:32.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.830+0000 7f35b5c7e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b019dfd0 con 0x7f35b0100dd0 2023-11-16T16:38:32.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.830+0000 7f35affff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35b0106010 0x7f35b019d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:32.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.830+0000 7f35affff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35b0106010 0x7f35b019d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42476/0 (socket says 172.21.15.1:42476) 2023-11-16T16:38:32.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.830+0000 7f35affff700 1 -- 172.21.15.1:0/192221502 learned_addr learned my addr 172.21.15.1:0/192221502 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:32.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.830+0000 7f35aeffd700 1 --2- 172.21.15.1:0/192221502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35b01036f0 0x7f35b01966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:32.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.830+0000 7f35af7fe700 1 --2- 172.21.15.1:0/192221502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35b0100dd0 0x7f35b0196170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:32.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.831+0000 7f35affff700 1 -- 172.21.15.1:0/192221502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35b0100dd0 msgr2=0x7f35b0196170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:32.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.831+0000 7f35affff700 1 --2- 172.21.15.1:0/192221502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35b0100dd0 0x7f35b0196170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:32.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.831+0000 7f35affff700 1 -- 172.21.15.1:0/192221502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35b01036f0 msgr2=0x7f35b01966b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:32.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.831+0000 7f35affff700 1 --2- 172.21.15.1:0/192221502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35b01036f0 0x7f35b01966b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:32.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.831+0000 7f35affff700 1 -- 172.21.15.1:0/192221502 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a8009d00 con 0x7f35b0106010 2023-11-16T16:38:32.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.831+0000 7f35affff700 1 --2- 172.21.15.1:0/192221502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35b0106010 0x7f35b019d730 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f35a8006070 tx=0x7f35a80094e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:32.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.831+0000 7f35acff9700 1 -- 172.21.15.1:0/192221502 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a800e040 con 0x7f35b0106010 2023-11-16T16:38:32.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.831+0000 7f35b5c7e700 1 -- 172.21.15.1:0/192221502 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35b010bc50 con 0x7f35b0106010 2023-11-16T16:38:32.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.831+0000 7f35acff9700 1 -- 172.21.15.1:0/192221502 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35a8004de0 con 0x7f35b0106010 2023-11-16T16:38:32.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.831+0000 7f35acff9700 1 -- 172.21.15.1:0/192221502 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a801d480 con 0x7f35b0106010 2023-11-16T16:38:32.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.831+0000 7f35b5c7e700 1 -- 172.21.15.1:0/192221502 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b010bac0 con 0x7f35b0106010 2023-11-16T16:38:32.838 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.832+0000 7f35b5c7e700 1 -- 172.21.15.1:0/192221502 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35b00fc360 con 0x7f35b0106010 2023-11-16T16:38:32.838 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.833+0000 7f35acff9700 1 -- 172.21.15.1:0/192221502 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f35a8004020 con 0x7f35b0106010 2023-11-16T16:38:32.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.834+0000 7f35acff9700 1 --2- 172.21.15.1:0/192221502 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f359c0a1150 0x7f359c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:32.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.834+0000 7f35af7fe700 1 --2- 172.21.15.1:0/192221502 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f359c0a1150 0x7f359c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:32.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.834+0000 7f35acff9700 1 -- 172.21.15.1:0/192221502 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f35a80cdfa0 con 0x7f35b0106010 2023-11-16T16:38:32.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.834+0000 7f35af7fe700 1 --2- 172.21.15.1:0/192221502 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f359c0a1150 0x7f359c0a3600 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f35a0005950 tx=0x7f35a00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:32.840 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:32.839+0000 7f35acff9700 1 -- 172.21.15.1:0/192221502 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f35a8097820 con 0x7f35b0106010 2023-11-16T16:38:33.043 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:33.041+0000 7f35b5c7e700 1 -- 172.21.15.1:0/192221502 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f35b0061ab0 con 0x7f359c0a1150 2023-11-16T16:38:33.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:33.048+0000 7f35acff9700 1 -- 172.21.15.1:0/192221502 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+985 (secure 0 0 0) 0x7f35b0061ab0 con 0x7f359c0a1150 2023-11-16T16:38:33.051 INFO:teuthology.orchestra.run.smithi001.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-11-16T16:38:33.051 INFO:teuthology.orchestra.run.smithi001.stdout:mds.all 12/12 13s ago 111s smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi026=b;smithi026=e;smithi026=h;smithi026=k;smithi053=c;smithi053=f;smithi053=i;smithi053=l;count:12 2023-11-16T16:38:33.051 INFO:teuthology.orchestra.run.smithi001.stdout:mgr 3/3 13s ago 6m smithi001=x;smithi026=y;smithi053=z;count:3 2023-11-16T16:38:33.051 INFO:teuthology.orchestra.run.smithi001.stdout:mon 3/3 13s ago 6m smithi001:172.21.15.1=a;smithi026:172.21.15.26=b;smithi053:172.21.15.53=c;count:3 2023-11-16T16:38:33.051 INFO:teuthology.orchestra.run.smithi001.stdout:osd 12 13s ago - 2023-11-16T16:38:33.053 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:33.050+0000 7f35b5c7e700 1 -- 172.21.15.1:0/192221502 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f359c0a1150 msgr2=0x7f359c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:33.053 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:33.050+0000 7f35b5c7e700 1 --2- 172.21.15.1:0/192221502 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f359c0a1150 0x7f359c0a3600 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f35a0005950 tx=0x7f35a00058e0 comp rx=0 tx=0).stop 2023-11-16T16:38:33.053 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:33.050+0000 7f35b5c7e700 1 -- 172.21.15.1:0/192221502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35b0106010 msgr2=0x7f35b019d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:33.053 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:33.050+0000 7f35b5c7e700 1 --2- 172.21.15.1:0/192221502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35b0106010 0x7f35b019d730 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f35a8006070 tx=0x7f35a80094e0 comp rx=0 tx=0).stop 2023-11-16T16:38:33.053 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:33.050+0000 7f35b5c7e700 1 -- 172.21.15.1:0/192221502 shutdown_connections 2023-11-16T16:38:33.053 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:33.051+0000 7f35b5c7e700 1 --2- 172.21.15.1:0/192221502 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f359c0a1150 0x7f359c0a3600 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:33.053 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:33.051+0000 7f35b5c7e700 1 --2- 172.21.15.1:0/192221502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35b0106010 0x7f35b019d730 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:33.053 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:33.051+0000 7f35b5c7e700 1 --2- 172.21.15.1:0/192221502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f35b0100dd0 0x7f35b0196170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:33.053 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:33.051+0000 7f35b5c7e700 1 --2- 172.21.15.1:0/192221502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35b01036f0 0x7f35b01966b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:33.053 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:33.051+0000 7f35b5c7e700 1 -- 172.21.15.1:0/192221502 >> 172.21.15.1:0/192221502 conn(0x7f35b00fa7f0 msgr2=0x7f35b0101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:33.054 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:33.051+0000 7f35b5c7e700 1 -- 172.21.15.1:0/192221502 shutdown_connections 2023-11-16T16:38:33.054 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:33.051+0000 7f35b5c7e700 1 -- 172.21.15.1:0/192221502 wait complete. 2023-11-16T16:38:33.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:33 smithi001 ceph-mon[98800]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-16T16:38:33.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:33 smithi001 ceph-mon[98800]: mgrmap e18: x(active, since 7m), standbys: y, z 2023-11-16T16:38:33.508 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- bash -c 'ceph orch host ls' 2023-11-16T16:38:33.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:33 smithi026 ceph-mon[100112]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-16T16:38:33.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:33 smithi026 ceph-mon[100112]: mgrmap e18: x(active, since 7m), standbys: y, z 2023-11-16T16:38:33.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:33 smithi053 ceph-mon[100638]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-16T16:38:33.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:33 smithi053 ceph-mon[100638]: mgrmap e18: x(active, since 7m), standbys: y, z 2023-11-16T16:38:34.164 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:38:34.451 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:34 smithi001 ceph-mon[98800]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:38:34.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:34 smithi026 ceph-mon[100112]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:38:34.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:34 smithi053 ceph-mon[100638]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:38:35.399 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:35 smithi001 ceph-mon[98800]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:38:35.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:35 smithi026 ceph-mon[100112]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:38:35.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:35 smithi053 ceph-mon[100638]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:38:35.590 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.588+0000 7f2a9b860700 1 -- 172.21.15.1:0/1975900201 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a941002f0 msgr2=0x7f2a94100700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:35.591 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.588+0000 7f2a9b860700 1 --2- 172.21.15.1:0/1975900201 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a941002f0 0x7f2a94100700 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2a90009990 tx=0x7f2a90009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:35.591 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.588+0000 7f2a9b860700 1 -- 172.21.15.1:0/1975900201 shutdown_connections 2023-11-16T16:38:35.591 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.588+0000 7f2a9b860700 1 --2- 172.21.15.1:0/1975900201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a94101e80 0x7f2a94108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:35.591 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.588+0000 7f2a9b860700 1 --2- 172.21.15.1:0/1975900201 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a941002f0 0x7f2a94100700 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:35.591 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.588+0000 7f2a9b860700 1 --2- 172.21.15.1:0/1975900201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a941014f0 0x7f2a94101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:35.591 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.588+0000 7f2a9b860700 1 -- 172.21.15.1:0/1975900201 >> 172.21.15.1:0/1975900201 conn(0x7f2a940fb900 msgr2=0x7f2a940fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:35.591 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.589+0000 7f2a9b860700 1 -- 172.21.15.1:0/1975900201 shutdown_connections 2023-11-16T16:38:35.591 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.589+0000 7f2a9b860700 1 -- 172.21.15.1:0/1975900201 wait complete. 2023-11-16T16:38:35.593 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.589+0000 7f2a9b860700 1 Processor -- start 2023-11-16T16:38:35.594 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.589+0000 7f2a9b860700 1 -- start start 2023-11-16T16:38:35.594 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.589+0000 7f2a9b860700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a941002f0 0x7f2a94074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:35.594 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.589+0000 7f2a9b860700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a941014f0 0x7f2a94072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:35.594 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.589+0000 7f2a9b860700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a94101e80 0x7f2a94073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:35.594 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.589+0000 7f2a9b860700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a94073ba0 con 0x7f2a941014f0 2023-11-16T16:38:35.594 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.589+0000 7f2a9b860700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a94073d10 con 0x7f2a94101e80 2023-11-16T16:38:35.594 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.589+0000 7f2a9b860700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a941a00b0 con 0x7f2a941002f0 2023-11-16T16:38:35.595 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.590+0000 7f2a98dfb700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a941014f0 0x7f2a94072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:35.595 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.590+0000 7f2a98dfb700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a941014f0 0x7f2a94072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42488/0 (socket says 172.21.15.1:42488) 2023-11-16T16:38:35.595 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.590+0000 7f2a98dfb700 1 -- 172.21.15.1:0/909161702 learned_addr learned my addr 172.21.15.1:0/909161702 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:35.595 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.590+0000 7f2a995fc700 1 --2- 172.21.15.1:0/909161702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a941002f0 0x7f2a94074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:35.595 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.590+0000 7f2a99dfd700 1 --2- 172.21.15.1:0/909161702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a94101e80 0x7f2a94073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:35.595 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.590+0000 7f2a98dfb700 1 -- 172.21.15.1:0/909161702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a941002f0 msgr2=0x7f2a94074890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:35.595 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.590+0000 7f2a98dfb700 1 --2- 172.21.15.1:0/909161702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a941002f0 0x7f2a94074890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:35.595 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.590+0000 7f2a98dfb700 1 -- 172.21.15.1:0/909161702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a94101e80 msgr2=0x7f2a94073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:35.596 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.590+0000 7f2a98dfb700 1 --2- 172.21.15.1:0/909161702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a94101e80 0x7f2a94073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:35.596 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.590+0000 7f2a98dfb700 1 -- 172.21.15.1:0/909161702 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a900095f0 con 0x7f2a941014f0 2023-11-16T16:38:35.596 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.590+0000 7f2a995fc700 1 --2- 172.21.15.1:0/909161702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a941002f0 0x7f2a94074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:38:35.596 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.590+0000 7f2a99dfd700 1 --2- 172.21.15.1:0/909161702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a94101e80 0x7f2a94073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:38:35.596 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.590+0000 7f2a98dfb700 1 --2- 172.21.15.1:0/909161702 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a941014f0 0x7f2a94072ee0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2a8800d950 tx=0x7f2a8800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:35.596 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.590+0000 7f2a867fc700 1 -- 172.21.15.1:0/909161702 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a88009b40 con 0x7f2a941014f0 2023-11-16T16:38:35.596 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.590+0000 7f2a867fc700 1 -- 172.21.15.1:0/909161702 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a88009ca0 con 0x7f2a941014f0 2023-11-16T16:38:35.596 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.590+0000 7f2a9b860700 1 -- 172.21.15.1:0/909161702 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a9410bc40 con 0x7f2a941014f0 2023-11-16T16:38:35.596 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.590+0000 7f2a867fc700 1 -- 172.21.15.1:0/909161702 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8800f620 con 0x7f2a941014f0 2023-11-16T16:38:35.597 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.590+0000 7f2a9b860700 1 -- 172.21.15.1:0/909161702 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a9410bab0 con 0x7f2a941014f0 2023-11-16T16:38:35.597 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.591+0000 7f2a9b860700 1 -- 172.21.15.1:0/909161702 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a941a0790 con 0x7f2a941014f0 2023-11-16T16:38:35.597 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.592+0000 7f2a867fc700 1 -- 172.21.15.1:0/909161702 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f2a88010460 con 0x7f2a941014f0 2023-11-16T16:38:35.598 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.593+0000 7f2a867fc700 1 --2- 172.21.15.1:0/909161702 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f2a800a1150 0x7f2a800a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:35.598 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.593+0000 7f2a867fc700 1 -- 172.21.15.1:0/909161702 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2a880ccc90 con 0x7f2a941014f0 2023-11-16T16:38:35.598 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.593+0000 7f2a995fc700 1 --2- 172.21.15.1:0/909161702 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f2a800a1150 0x7f2a800a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:35.598 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.593+0000 7f2a995fc700 1 --2- 172.21.15.1:0/909161702 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f2a800a1150 0x7f2a800a3600 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2a9000b2e0 tx=0x7f2a90005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:35.599 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.596+0000 7f2a867fc700 1 -- 172.21.15.1:0/909161702 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2a880963e0 con 0x7f2a941014f0 2023-11-16T16:38:35.803 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.801+0000 7f2a9b860700 1 -- 172.21.15.1:0/909161702 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a94060f30 con 0x7f2a800a1150 2023-11-16T16:38:35.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.802+0000 7f2a867fc700 1 -- 172.21.15.1:0/909161702 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+187 (secure 0 0 0) 0x7f2a94060f30 con 0x7f2a800a1150 2023-11-16T16:38:35.804 INFO:teuthology.orchestra.run.smithi001.stdout:HOST ADDR LABELS STATUS 2023-11-16T16:38:35.805 INFO:teuthology.orchestra.run.smithi001.stdout:smithi001 172.21.15.1 2023-11-16T16:38:35.805 INFO:teuthology.orchestra.run.smithi001.stdout:smithi026 172.21.15.26 2023-11-16T16:38:35.805 INFO:teuthology.orchestra.run.smithi001.stdout:smithi053 172.21.15.53 2023-11-16T16:38:35.805 INFO:teuthology.orchestra.run.smithi001.stdout:3 hosts in cluster 2023-11-16T16:38:35.807 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.804+0000 7f2a9b860700 1 -- 172.21.15.1:0/909161702 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f2a800a1150 msgr2=0x7f2a800a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:35.807 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.804+0000 7f2a9b860700 1 --2- 172.21.15.1:0/909161702 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f2a800a1150 0x7f2a800a3600 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2a9000b2e0 tx=0x7f2a90005fb0 comp rx=0 tx=0).stop 2023-11-16T16:38:35.807 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.804+0000 7f2a9b860700 1 -- 172.21.15.1:0/909161702 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a941014f0 msgr2=0x7f2a94072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:35.807 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.804+0000 7f2a9b860700 1 --2- 172.21.15.1:0/909161702 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a941014f0 0x7f2a94072ee0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2a8800d950 tx=0x7f2a8800dd10 comp rx=0 tx=0).stop 2023-11-16T16:38:35.807 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.804+0000 7f2a9b860700 1 -- 172.21.15.1:0/909161702 shutdown_connections 2023-11-16T16:38:35.807 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.804+0000 7f2a9b860700 1 --2- 172.21.15.1:0/909161702 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f2a800a1150 0x7f2a800a3600 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:35.807 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.804+0000 7f2a9b860700 1 --2- 172.21.15.1:0/909161702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a94101e80 0x7f2a94073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:35.807 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.804+0000 7f2a9b860700 1 --2- 172.21.15.1:0/909161702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a941002f0 0x7f2a94074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:35.808 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.804+0000 7f2a9b860700 1 --2- 172.21.15.1:0/909161702 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a941014f0 0x7f2a94072ee0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:35.808 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.804+0000 7f2a9b860700 1 -- 172.21.15.1:0/909161702 >> 172.21.15.1:0/909161702 conn(0x7f2a940fb900 msgr2=0x7f2a940fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:35.808 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.805+0000 7f2a9b860700 1 -- 172.21.15.1:0/909161702 shutdown_connections 2023-11-16T16:38:35.808 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:35.805+0000 7f2a9b860700 1 -- 172.21.15.1:0/909161702 wait complete. 2023-11-16T16:38:36.218 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- bash -c 'ceph orch device ls' 2023-11-16T16:38:36.874 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:38:37.425 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:37 smithi001 ceph-mon[98800]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2023-11-16T16:38:37.425 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:37 smithi001 ceph-mon[98800]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:38:37.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:37 smithi026 ceph-mon[100112]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2023-11-16T16:38:37.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:37 smithi026 ceph-mon[100112]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:38:37.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:37 smithi053 ceph-mon[100638]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2023-11-16T16:38:37.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:37 smithi053 ceph-mon[100638]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:38:38.221 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.216+0000 7fe391d83700 1 -- 172.21.15.1:0/3092096519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe38c0ff1e0 msgr2=0x7fe38c0ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:38.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.216+0000 7fe391d83700 1 --2- 172.21.15.1:0/3092096519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe38c0ff1e0 0x7fe38c0ff5f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe37c009990 tx=0x7fe37c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:38.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.216+0000 7fe391d83700 1 -- 172.21.15.1:0/3092096519 shutdown_connections 2023-11-16T16:38:38.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.217+0000 7fe391d83700 1 --2- 172.21.15.1:0/3092096519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe38c100d70 0x7fe38c107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:38.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.217+0000 7fe391d83700 1 --2- 172.21.15.1:0/3092096519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe38c0ff1e0 0x7fe38c0ff5f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:38.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.217+0000 7fe391d83700 1 --2- 172.21.15.1:0/3092096519 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe38c1003e0 0x7fe38c100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:38.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.217+0000 7fe391d83700 1 -- 172.21.15.1:0/3092096519 >> 172.21.15.1:0/3092096519 conn(0x7fe38c0fa7f0 msgr2=0x7fe38c0fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:38.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.217+0000 7fe391d83700 1 -- 172.21.15.1:0/3092096519 shutdown_connections 2023-11-16T16:38:38.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.217+0000 7fe391d83700 1 -- 172.21.15.1:0/3092096519 wait complete. 2023-11-16T16:38:38.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.217+0000 7fe391d83700 1 Processor -- start 2023-11-16T16:38:38.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.217+0000 7fe391d83700 1 -- start start 2023-11-16T16:38:38.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.218+0000 7fe391d83700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe38c0ff1e0 0x7fe38c1982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:38.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.218+0000 7fe391d83700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe38c1003e0 0x7fe38c198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:38.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.218+0000 7fe391d83700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe38c100d70 0x7fe38c19f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:38.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.218+0000 7fe391d83700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38c19fe80 con 0x7fe38c100d70 2023-11-16T16:38:38.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.218+0000 7fe391d83700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38c19ffc0 con 0x7fe38c1003e0 2023-11-16T16:38:38.224 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.218+0000 7fe391d83700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38c1a0150 con 0x7fe38c0ff1e0 2023-11-16T16:38:38.224 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.218+0000 7fe38bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe38c100d70 0x7fe38c19f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:38.224 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.218+0000 7fe38bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe38c100d70 0x7fe38c19f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42502/0 (socket says 172.21.15.1:42502) 2023-11-16T16:38:38.224 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.218+0000 7fe38bfff700 1 -- 172.21.15.1:0/350159768 learned_addr learned my addr 172.21.15.1:0/350159768 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:38.224 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.218+0000 7fe38b7fe700 1 --2- 172.21.15.1:0/350159768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe38c0ff1e0 0x7fe38c1982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:38.224 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.218+0000 7fe38affd700 1 --2- 172.21.15.1:0/350159768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe38c1003e0 0x7fe38c198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:38.224 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.218+0000 7fe38bfff700 1 -- 172.21.15.1:0/350159768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe38c0ff1e0 msgr2=0x7fe38c1982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:38.224 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.218+0000 7fe38bfff700 1 --2- 172.21.15.1:0/350159768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe38c0ff1e0 0x7fe38c1982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:38.225 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.218+0000 7fe38bfff700 1 -- 172.21.15.1:0/350159768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe38c1003e0 msgr2=0x7fe38c198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:38.225 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.218+0000 7fe38bfff700 1 --2- 172.21.15.1:0/350159768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe38c1003e0 0x7fe38c198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:38.225 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.218+0000 7fe38bfff700 1 -- 172.21.15.1:0/350159768 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe37c0095f0 con 0x7fe38c100d70 2023-11-16T16:38:38.225 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.218+0000 7fe38bfff700 1 --2- 172.21.15.1:0/350159768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe38c100d70 0x7fe38c19f8b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe384007da0 tx=0x7fe38400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:38.225 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.218+0000 7fe388ff9700 1 -- 172.21.15.1:0/350159768 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38400ad00 con 0x7fe38c100d70 2023-11-16T16:38:38.225 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.218+0000 7fe391d83700 1 -- 172.21.15.1:0/350159768 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe38c10ac00 con 0x7fe38c100d70 2023-11-16T16:38:38.225 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.218+0000 7fe388ff9700 1 -- 172.21.15.1:0/350159768 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe38400ae60 con 0x7fe38c100d70 2023-11-16T16:38:38.226 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.218+0000 7fe388ff9700 1 -- 172.21.15.1:0/350159768 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe384011660 con 0x7fe38c100d70 2023-11-16T16:38:38.226 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.218+0000 7fe391d83700 1 -- 172.21.15.1:0/350159768 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe38c10aa70 con 0x7fe38c100d70 2023-11-16T16:38:38.226 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.219+0000 7fe391d83700 1 -- 172.21.15.1:0/350159768 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe38c1a0840 con 0x7fe38c100d70 2023-11-16T16:38:38.227 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.221+0000 7fe388ff9700 1 -- 172.21.15.1:0/350159768 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fe384004ad0 con 0x7fe38c100d70 2023-11-16T16:38:38.227 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.221+0000 7fe388ff9700 1 --2- 172.21.15.1:0/350159768 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fe3740a1150 0x7fe3740a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:38.227 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.221+0000 7fe388ff9700 1 -- 172.21.15.1:0/350159768 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe3840ccfa0 con 0x7fe38c100d70 2023-11-16T16:38:38.227 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.221+0000 7fe38b7fe700 1 --2- 172.21.15.1:0/350159768 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fe3740a1150 0x7fe3740a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:38.228 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.222+0000 7fe38b7fe700 1 --2- 172.21.15.1:0/350159768 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fe3740a1150 0x7fe3740a3600 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe37c0036c0 tx=0x7fe37c005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:38.228 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.225+0000 7fe388ff9700 1 -- 172.21.15.1:0/350159768 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe3840966f0 con 0x7fe38c100d70 2023-11-16T16:38:38.431 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.429+0000 7fe391d83700 1 -- 172.21.15.1:0/350159768 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fe38c1a0df0 con 0x7fe3740a1150 2023-11-16T16:38:38.434 INFO:teuthology.orchestra.run.smithi001.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-11-16T16:38:38.434 INFO:teuthology.orchestra.run.smithi001.stdout:smithi001 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513000D480DGN 447G No 2m ago LVM detected, locked 2023-11-16T16:38:38.434 INFO:teuthology.orchestra.run.smithi001.stdout:smithi026 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB74760020480DGN 447G No 2m ago LVM detected, locked 2023-11-16T16:38:38.435 INFO:teuthology.orchestra.run.smithi001.stdout:smithi053 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300G8480DGN 447G No 2m ago LVM detected, locked 2023-11-16T16:38:38.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.432+0000 7fe388ff9700 1 -- 172.21.15.1:0/350159768 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+492 (secure 0 0 0) 0x7fe38c1a0df0 con 0x7fe3740a1150 2023-11-16T16:38:38.436 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.434+0000 7fe391d83700 1 -- 172.21.15.1:0/350159768 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fe3740a1150 msgr2=0x7fe3740a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:38.436 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.434+0000 7fe391d83700 1 --2- 172.21.15.1:0/350159768 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fe3740a1150 0x7fe3740a3600 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe37c0036c0 tx=0x7fe37c005ee0 comp rx=0 tx=0).stop 2023-11-16T16:38:38.436 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.434+0000 7fe391d83700 1 -- 172.21.15.1:0/350159768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe38c100d70 msgr2=0x7fe38c19f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:38.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.434+0000 7fe391d83700 1 --2- 172.21.15.1:0/350159768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe38c100d70 0x7fe38c19f8b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe384007da0 tx=0x7fe38400a340 comp rx=0 tx=0).stop 2023-11-16T16:38:38.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.434+0000 7fe391d83700 1 -- 172.21.15.1:0/350159768 shutdown_connections 2023-11-16T16:38:38.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.434+0000 7fe391d83700 1 --2- 172.21.15.1:0/350159768 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fe3740a1150 0x7fe3740a3600 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:38.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.434+0000 7fe391d83700 1 --2- 172.21.15.1:0/350159768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe38c100d70 0x7fe38c19f8b0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:38.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.434+0000 7fe391d83700 1 --2- 172.21.15.1:0/350159768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe38c0ff1e0 0x7fe38c1982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:38.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.434+0000 7fe391d83700 1 --2- 172.21.15.1:0/350159768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe38c1003e0 0x7fe38c198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:38.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.434+0000 7fe391d83700 1 -- 172.21.15.1:0/350159768 >> 172.21.15.1:0/350159768 conn(0x7fe38c0fa7f0 msgr2=0x7fe38c068670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:38.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.434+0000 7fe391d83700 1 -- 172.21.15.1:0/350159768 shutdown_connections 2023-11-16T16:38:38.438 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:38.434+0000 7fe391d83700 1 -- 172.21.15.1:0/350159768 wait complete. 2023-11-16T16:38:38.933 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-11-16T16:38:38.943 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi001.front.sepia.ceph.com 2023-11-16T16:38:38.944 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- bash -c 'ceph fs dump' 2023-11-16T16:38:39.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:39 smithi026 ceph-mon[100112]: pgmap v304: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2023-11-16T16:38:39.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:39 smithi026 ceph-mon[100112]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:38:39.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:39 smithi053 ceph-mon[100638]: pgmap v304: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2023-11-16T16:38:39.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:39 smithi053 ceph-mon[100638]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:38:39.599 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:38:39.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:39 smithi001 ceph-mon[98800]: pgmap v304: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2023-11-16T16:38:39.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:39 smithi001 ceph-mon[98800]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:38:40.897 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.893+0000 7fa186c74700 1 -- 172.21.15.1:0/2841579400 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1800ff1e0 msgr2=0x7fa1800ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:40.897 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.893+0000 7fa186c74700 1 --2- 172.21.15.1:0/2841579400 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1800ff1e0 0x7fa1800ff5f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa174009990 tx=0x7fa174009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:40.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.893+0000 7fa186c74700 1 -- 172.21.15.1:0/2841579400 shutdown_connections 2023-11-16T16:38:40.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.893+0000 7fa186c74700 1 --2- 172.21.15.1:0/2841579400 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa180100d70 0x7fa180107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:40.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.893+0000 7fa186c74700 1 --2- 172.21.15.1:0/2841579400 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1800ff1e0 0x7fa1800ff5f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:40.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.893+0000 7fa186c74700 1 --2- 172.21.15.1:0/2841579400 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1801003e0 0x7fa180100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:40.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.893+0000 7fa186c74700 1 -- 172.21.15.1:0/2841579400 >> 172.21.15.1:0/2841579400 conn(0x7fa1800fa7f0 msgr2=0x7fa1800fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:40.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.893+0000 7fa186c74700 1 -- 172.21.15.1:0/2841579400 shutdown_connections 2023-11-16T16:38:40.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.893+0000 7fa186c74700 1 -- 172.21.15.1:0/2841579400 wait complete. 2023-11-16T16:38:40.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.894+0000 7fa186c74700 1 Processor -- start 2023-11-16T16:38:40.899 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.894+0000 7fa186c74700 1 -- start start 2023-11-16T16:38:40.899 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.894+0000 7fa186c74700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1800ff1e0 0x7fa180195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:40.899 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.894+0000 7fa186c74700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1801003e0 0x7fa180196530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:40.899 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.894+0000 7fa186c74700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa180100d70 0x7fa18019d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:40.899 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.894+0000 7fa186c74700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18019db80 con 0x7fa1800ff1e0 2023-11-16T16:38:40.899 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.894+0000 7fa186c74700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18019dcc0 con 0x7fa180100d70 2023-11-16T16:38:40.899 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.894+0000 7fa186c74700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18019de50 con 0x7fa1801003e0 2023-11-16T16:38:40.899 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.894+0000 7fa184a10700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1800ff1e0 0x7fa180195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:40.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.894+0000 7fa184a10700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1800ff1e0 0x7fa180195ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42508/0 (socket says 172.21.15.1:42508) 2023-11-16T16:38:40.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.894+0000 7fa184a10700 1 -- 172.21.15.1:0/3368453853 learned_addr learned my addr 172.21.15.1:0/3368453853 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:40.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.894+0000 7fa185211700 1 --2- 172.21.15.1:0/3368453853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa180100d70 0x7fa18019d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:40.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.894+0000 7fa17ffff700 1 --2- 172.21.15.1:0/3368453853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1801003e0 0x7fa180196530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:40.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.894+0000 7fa184a10700 1 -- 172.21.15.1:0/3368453853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1801003e0 msgr2=0x7fa180196530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:40.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.894+0000 7fa184a10700 1 --2- 172.21.15.1:0/3368453853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1801003e0 0x7fa180196530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:40.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.894+0000 7fa184a10700 1 -- 172.21.15.1:0/3368453853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa180100d70 msgr2=0x7fa18019d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:40.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.894+0000 7fa184a10700 1 --2- 172.21.15.1:0/3368453853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa180100d70 0x7fa18019d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:40.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.894+0000 7fa184a10700 1 -- 172.21.15.1:0/3368453853 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1740095f0 con 0x7fa1800ff1e0 2023-11-16T16:38:40.901 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.894+0000 7fa17ffff700 1 --2- 172.21.15.1:0/3368453853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1801003e0 0x7fa180196530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:38:40.901 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.894+0000 7fa185211700 1 --2- 172.21.15.1:0/3368453853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa180100d70 0x7fa18019d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:38:40.901 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.895+0000 7fa184a10700 1 --2- 172.21.15.1:0/3368453853 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1800ff1e0 0x7fa180195ff0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa17400c010 tx=0x7fa17400f7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:40.901 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.895+0000 7fa17dffb700 1 -- 172.21.15.1:0/3368453853 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17401d070 con 0x7fa1800ff1e0 2023-11-16T16:38:40.901 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.895+0000 7fa17dffb700 1 -- 172.21.15.1:0/3368453853 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa17400fc90 con 0x7fa1800ff1e0 2023-11-16T16:38:40.901 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.895+0000 7fa186c74700 1 -- 172.21.15.1:0/3368453853 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa18010abb0 con 0x7fa1800ff1e0 2023-11-16T16:38:40.901 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.895+0000 7fa17dffb700 1 -- 172.21.15.1:0/3368453853 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174017650 con 0x7fa1800ff1e0 2023-11-16T16:38:40.901 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.895+0000 7fa186c74700 1 -- 172.21.15.1:0/3368453853 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa18010aa20 con 0x7fa1800ff1e0 2023-11-16T16:38:40.902 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.896+0000 7fa186c74700 1 -- 172.21.15.1:0/3368453853 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa180107270 con 0x7fa1800ff1e0 2023-11-16T16:38:40.902 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.897+0000 7fa17dffb700 1 -- 172.21.15.1:0/3368453853 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fa174022470 con 0x7fa1800ff1e0 2023-11-16T16:38:40.903 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.898+0000 7fa17dffb700 1 --2- 172.21.15.1:0/3368453853 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fa1700a11d0 0x7fa1700a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:40.903 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.898+0000 7fa17ffff700 1 --2- 172.21.15.1:0/3368453853 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fa1700a11d0 0x7fa1700a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:40.903 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.898+0000 7fa17dffb700 1 -- 172.21.15.1:0/3368453853 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa1740cdde0 con 0x7fa1800ff1e0 2023-11-16T16:38:40.903 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.898+0000 7fa17ffff700 1 --2- 172.21.15.1:0/3368453853 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fa1700a11d0 0x7fa1700a3680 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa16c007900 tx=0x7fa16c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:40.903 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:40.902+0000 7fa17dffb700 1 -- 172.21.15.1:0/3368453853 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa17405c730 con 0x7fa1800ff1e0 2023-11-16T16:38:41.144 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:41.143+0000 7fa186c74700 1 -- 172.21.15.1:0/3368453853 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa18002cfe0 con 0x7fa1800ff1e0 2023-11-16T16:38:41.145 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:41.144+0000 7fa17dffb700 1 -- 172.21.15.1:0/3368453853 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+2877 (secure 0 0 0) 0x7fa174097580 con 0x7fa1800ff1e0 2023-11-16T16:38:41.145 INFO:teuthology.orchestra.run.smithi001.stdout:e23 2023-11-16T16:38:41.145 INFO:teuthology.orchestra.run.smithi001.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-11-16T16:38:41.146 INFO:teuthology.orchestra.run.smithi001.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-11-16T16:38:41.146 INFO:teuthology.orchestra.run.smithi001.stdout:legacy client fscid: 1 2023-11-16T16:38:41.146 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:38:41.146 INFO:teuthology.orchestra.run.smithi001.stdout:Filesystem 'cephfs' (1) 2023-11-16T16:38:41.146 INFO:teuthology.orchestra.run.smithi001.stdout:fs_name cephfs 2023-11-16T16:38:41.146 INFO:teuthology.orchestra.run.smithi001.stdout:epoch 23 2023-11-16T16:38:41.146 INFO:teuthology.orchestra.run.smithi001.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-11-16T16:38:41.146 INFO:teuthology.orchestra.run.smithi001.stdout:created 2023-11-16T16:37:13.918943+0000 2023-11-16T16:38:41.146 INFO:teuthology.orchestra.run.smithi001.stdout:modified 2023-11-16T16:37:19.292898+0000 2023-11-16T16:38:41.146 INFO:teuthology.orchestra.run.smithi001.stdout:tableserver 0 2023-11-16T16:38:41.147 INFO:teuthology.orchestra.run.smithi001.stdout:root 0 2023-11-16T16:38:41.147 INFO:teuthology.orchestra.run.smithi001.stdout:session_timeout 300 2023-11-16T16:38:41.147 INFO:teuthology.orchestra.run.smithi001.stdout:session_autoclose 300 2023-11-16T16:38:41.147 INFO:teuthology.orchestra.run.smithi001.stdout:max_file_size 1099511627776 2023-11-16T16:38:41.147 INFO:teuthology.orchestra.run.smithi001.stdout:max_xattr_size 65536 2023-11-16T16:38:41.147 INFO:teuthology.orchestra.run.smithi001.stdout:required_client_features {} 2023-11-16T16:38:41.147 INFO:teuthology.orchestra.run.smithi001.stdout:last_failure 0 2023-11-16T16:38:41.147 INFO:teuthology.orchestra.run.smithi001.stdout:last_failure_osd_epoch 0 2023-11-16T16:38:41.147 INFO:teuthology.orchestra.run.smithi001.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-11-16T16:38:41.147 INFO:teuthology.orchestra.run.smithi001.stdout:max_mds 3 2023-11-16T16:38:41.148 INFO:teuthology.orchestra.run.smithi001.stdout:in 0,1,2 2023-11-16T16:38:41.148 INFO:teuthology.orchestra.run.smithi001.stdout:up {0=24521,1=24493,2=24482} 2023-11-16T16:38:41.148 INFO:teuthology.orchestra.run.smithi001.stdout:failed 2023-11-16T16:38:41.148 INFO:teuthology.orchestra.run.smithi001.stdout:damaged 2023-11-16T16:38:41.148 INFO:teuthology.orchestra.run.smithi001.stdout:stopped 2023-11-16T16:38:41.148 INFO:teuthology.orchestra.run.smithi001.stdout:data_pools [3] 2023-11-16T16:38:41.148 INFO:teuthology.orchestra.run.smithi001.stdout:metadata_pool 2 2023-11-16T16:38:41.148 INFO:teuthology.orchestra.run.smithi001.stdout:inline_data disabled 2023-11-16T16:38:41.148 INFO:teuthology.orchestra.run.smithi001.stdout:balancer 2023-11-16T16:38:41.148 INFO:teuthology.orchestra.run.smithi001.stdout:bal_rank_mask -1 2023-11-16T16:38:41.149 INFO:teuthology.orchestra.run.smithi001.stdout:standby_count_wanted 0 2023-11-16T16:38:41.149 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.b{0:24521} state up:active seq 3 addr [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] compat {c=[1],r=[1],i=[7ff]}] 2023-11-16T16:38:41.149 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.c{0:24472} state up:standby-replay seq 1 addr [v2:172.21.15.53:6836/3197060302,v1:172.21.15.53:6837/3197060302] compat {c=[1],r=[1],i=[7ff]}] 2023-11-16T16:38:41.149 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.i{1:24493} state up:active seq 5 addr [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] compat {c=[1],r=[1],i=[7ff]}] 2023-11-16T16:38:41.149 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.e{1:24467} state up:standby-replay seq 1 addr [v2:172.21.15.26:6834/1010760275,v1:172.21.15.26:6835/1010760275] compat {c=[1],r=[1],i=[7ff]}] 2023-11-16T16:38:41.149 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.h{2:24482} state up:active seq 6 addr [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] compat {c=[1],r=[1],i=[7ff]}] 2023-11-16T16:38:41.149 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.f{2:24463} state up:standby-replay seq 1 addr [v2:172.21.15.53:6834/2140542192,v1:172.21.15.53:6835/2140542192] compat {c=[1],r=[1],i=[7ff]}] 2023-11-16T16:38:41.149 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:38:41.149 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:38:41.149 INFO:teuthology.orchestra.run.smithi001.stdout:Standby daemons: 2023-11-16T16:38:41.150 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:38:41.150 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.j{-1:14538} state up:standby seq 1 addr [v2:172.21.15.1:6834/2587563684,v1:172.21.15.1:6835/2587563684] compat {c=[1],r=[1],i=[7ff]}] 2023-11-16T16:38:41.150 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.d{-1:14550} state up:standby seq 1 addr [v2:172.21.15.1:6836/277936168,v1:172.21.15.1:6837/277936168] compat {c=[1],r=[1],i=[7ff]}] 2023-11-16T16:38:41.150 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.g{-1:14613} state up:standby seq 1 addr [v2:172.21.15.1:6838/1172575045,v1:172.21.15.1:6839/1172575045] compat {c=[1],r=[1],i=[7ff]}] 2023-11-16T16:38:41.150 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.a{-1:14625} state up:standby seq 1 addr [v2:172.21.15.1:6840/2631175899,v1:172.21.15.1:6841/2631175899] compat {c=[1],r=[1],i=[7ff]}] 2023-11-16T16:38:41.150 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.l{-1:24433} state up:standby seq 1 addr [v2:172.21.15.53:6832/3693905267,v1:172.21.15.53:6833/3693905267] compat {c=[1],r=[1],i=[7ff]}] 2023-11-16T16:38:41.150 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.k{-1:24446} state up:standby seq 1 addr [v2:172.21.15.26:6832/164668033,v1:172.21.15.26:6833/164668033] compat {c=[1],r=[1],i=[7ff]}] 2023-11-16T16:38:41.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:41.146+0000 7fa186c74700 1 -- 172.21.15.1:0/3368453853 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fa1700a11d0 msgr2=0x7fa1700a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:41.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:41.146+0000 7fa186c74700 1 --2- 172.21.15.1:0/3368453853 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fa1700a11d0 0x7fa1700a3680 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa16c007900 tx=0x7fa16c008040 comp rx=0 tx=0).stop 2023-11-16T16:38:41.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:41.146+0000 7fa186c74700 1 -- 172.21.15.1:0/3368453853 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1800ff1e0 msgr2=0x7fa180195ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:41.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:41.146+0000 7fa186c74700 1 --2- 172.21.15.1:0/3368453853 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1800ff1e0 0x7fa180195ff0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa17400c010 tx=0x7fa17400f7b0 comp rx=0 tx=0).stop 2023-11-16T16:38:41.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:41.146+0000 7fa186c74700 1 -- 172.21.15.1:0/3368453853 shutdown_connections 2023-11-16T16:38:41.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:41.146+0000 7fa186c74700 1 --2- 172.21.15.1:0/3368453853 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fa1700a11d0 0x7fa1700a3680 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:41.152 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:41.146+0000 7fa186c74700 1 --2- 172.21.15.1:0/3368453853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa180100d70 0x7fa18019d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:41.152 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:41.146+0000 7fa186c74700 1 --2- 172.21.15.1:0/3368453853 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1800ff1e0 0x7fa180195ff0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:41.152 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:41.146+0000 7fa186c74700 1 --2- 172.21.15.1:0/3368453853 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa1801003e0 0x7fa180196530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:41.152 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:41.146+0000 7fa186c74700 1 -- 172.21.15.1:0/3368453853 >> 172.21.15.1:0/3368453853 conn(0x7fa1800fa7f0 msgr2=0x7fa1800fc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:41.152 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:41.146+0000 7fa186c74700 1 -- 172.21.15.1:0/3368453853 shutdown_connections 2023-11-16T16:38:41.152 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:41.146+0000 7fa186c74700 1 -- 172.21.15.1:0/3368453853 wait complete. 2023-11-16T16:38:41.152 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 23 2023-11-16T16:38:41.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:41 smithi001 ceph-mon[98800]: pgmap v305: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2023-11-16T16:38:41.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:41 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3368453853' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-11-16T16:38:41.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:41 smithi026 ceph-mon[100112]: pgmap v305: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2023-11-16T16:38:41.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:41 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/3368453853' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-11-16T16:38:41.549 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- bash -c 'ceph osd dump' 2023-11-16T16:38:41.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:41 smithi053 ceph-mon[100638]: pgmap v305: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2023-11-16T16:38:41.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:41 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/3368453853' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-11-16T16:38:42.205 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:38:43.266 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:43 smithi001 ceph-mon[98800]: pgmap v306: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2023-11-16T16:38:43.412 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.408+0000 7f91acaf1700 1 -- 172.21.15.1:0/1369248130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91a81003e0 msgr2=0x7f91a8100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:43.412 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.408+0000 7f91acaf1700 1 --2- 172.21.15.1:0/1369248130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91a81003e0 0x7f91a8100830 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f919c009990 tx=0x7f919c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:43.412 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.408+0000 7f91acaf1700 1 -- 172.21.15.1:0/1369248130 shutdown_connections 2023-11-16T16:38:43.412 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.408+0000 7f91acaf1700 1 --2- 172.21.15.1:0/1369248130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91a8100d70 0x7f91a8107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:43.413 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.408+0000 7f91acaf1700 1 --2- 172.21.15.1:0/1369248130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91a80ff1e0 0x7f91a80ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:43.413 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.408+0000 7f91acaf1700 1 --2- 172.21.15.1:0/1369248130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91a81003e0 0x7f91a8100830 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:43.413 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.408+0000 7f91acaf1700 1 -- 172.21.15.1:0/1369248130 >> 172.21.15.1:0/1369248130 conn(0x7f91a80fa7f0 msgr2=0x7f91a80fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:43.413 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.408+0000 7f91acaf1700 1 -- 172.21.15.1:0/1369248130 shutdown_connections 2023-11-16T16:38:43.413 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.408+0000 7f91acaf1700 1 -- 172.21.15.1:0/1369248130 wait complete. 2023-11-16T16:38:43.413 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.409+0000 7f91acaf1700 1 Processor -- start 2023-11-16T16:38:43.413 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.409+0000 7f91acaf1700 1 -- start start 2023-11-16T16:38:43.413 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.409+0000 7f91acaf1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91a80ff1e0 0x7f91a8196050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:43.414 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.409+0000 7f91acaf1700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91a81003e0 0x7f91a8196590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:43.414 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.409+0000 7f91acaf1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91a8100d70 0x7f91a819d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:43.414 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.409+0000 7f91acaf1700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a819dbe0 con 0x7f91a8100d70 2023-11-16T16:38:43.414 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.409+0000 7f91acaf1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a819dd20 con 0x7f91a80ff1e0 2023-11-16T16:38:43.414 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.409+0000 7f91acaf1700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a819deb0 con 0x7f91a81003e0 2023-11-16T16:38:43.414 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.409+0000 7f91a6d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91a8100d70 0x7f91a819d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:43.414 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.409+0000 7f91a6d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91a8100d70 0x7f91a819d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42524/0 (socket says 172.21.15.1:42524) 2023-11-16T16:38:43.414 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.409+0000 7f91a6d9d700 1 -- 172.21.15.1:0/3491603671 learned_addr learned my addr 172.21.15.1:0/3491603671 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:43.414 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.409+0000 7f91a5d9b700 1 --2- 172.21.15.1:0/3491603671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91a81003e0 0x7f91a8196590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:43.415 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.409+0000 7f91a659c700 1 --2- 172.21.15.1:0/3491603671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91a80ff1e0 0x7f91a8196050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:43.415 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.409+0000 7f91a6d9d700 1 -- 172.21.15.1:0/3491603671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91a81003e0 msgr2=0x7f91a8196590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:43.415 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.409+0000 7f91a6d9d700 1 --2- 172.21.15.1:0/3491603671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91a81003e0 0x7f91a8196590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:43.415 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.409+0000 7f91a6d9d700 1 -- 172.21.15.1:0/3491603671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91a80ff1e0 msgr2=0x7f91a8196050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:43.415 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.409+0000 7f91a6d9d700 1 --2- 172.21.15.1:0/3491603671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91a80ff1e0 0x7f91a8196050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:43.415 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.409+0000 7f91a6d9d700 1 -- 172.21.15.1:0/3491603671 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f919c0095f0 con 0x7f91a8100d70 2023-11-16T16:38:43.415 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.409+0000 7f91a659c700 1 --2- 172.21.15.1:0/3491603671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91a80ff1e0 0x7f91a8196050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:38:43.415 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.409+0000 7f91a6d9d700 1 --2- 172.21.15.1:0/3491603671 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91a8100d70 0x7f91a819d610 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f91a0007f10 tx=0x7f91a000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:43.416 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.410+0000 7f91977fe700 1 -- 172.21.15.1:0/3491603671 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a0017070 con 0x7f91a8100d70 2023-11-16T16:38:43.416 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.410+0000 7f91acaf1700 1 -- 172.21.15.1:0/3491603671 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91a810ab40 con 0x7f91a8100d70 2023-11-16T16:38:43.416 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.410+0000 7f91977fe700 1 -- 172.21.15.1:0/3491603671 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91a000f040 con 0x7f91a8100d70 2023-11-16T16:38:43.416 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.410+0000 7f91977fe700 1 -- 172.21.15.1:0/3491603671 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a0005740 con 0x7f91a8100d70 2023-11-16T16:38:43.416 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.410+0000 7f91acaf1700 1 -- 172.21.15.1:0/3491603671 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91a810a9b0 con 0x7f91a8100d70 2023-11-16T16:38:43.416 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.411+0000 7f91acaf1700 1 -- 172.21.15.1:0/3491603671 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91a8066ea0 con 0x7f91a8100d70 2023-11-16T16:38:43.417 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.412+0000 7f91977fe700 1 -- 172.21.15.1:0/3491603671 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f91a00058a0 con 0x7f91a8100d70 2023-11-16T16:38:43.417 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.412+0000 7f91977fe700 1 --2- 172.21.15.1:0/3491603671 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f91900a1150 0x7f91900a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:43.417 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.412+0000 7f91a659c700 1 --2- 172.21.15.1:0/3491603671 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f91900a1150 0x7f91900a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:43.418 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.412+0000 7f91977fe700 1 -- 172.21.15.1:0/3491603671 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f91a00cc140 con 0x7f91a8100d70 2023-11-16T16:38:43.418 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.413+0000 7f91a659c700 1 --2- 172.21.15.1:0/3491603671 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f91900a1150 0x7f91900a3600 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9198005950 tx=0x7f91980058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:43.418 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.417+0000 7f91977fe700 1 -- 172.21.15.1:0/3491603671 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f91a0095890 con 0x7f91a8100d70 2023-11-16T16:38:43.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:43 smithi026 ceph-mon[100112]: pgmap v306: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2023-11-16T16:38:43.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:43 smithi053 ceph-mon[100638]: pgmap v306: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2023-11-16T16:38:43.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.614+0000 7f91acaf1700 1 -- 172.21.15.1:0/3491603671 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f91a819e960 con 0x7f91a8100d70 2023-11-16T16:38:43.616 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.615+0000 7f91977fe700 1 -- 172.21.15.1:0/3491603671 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v86) v1 ==== 56+0+5258 (secure 0 0 0) 0x7f91a0098eb0 con 0x7f91a8100d70 2023-11-16T16:38:43.616 INFO:teuthology.orchestra.run.smithi001.stdout:epoch 86 2023-11-16T16:38:43.617 INFO:teuthology.orchestra.run.smithi001.stdout:fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 2023-11-16T16:38:43.617 INFO:teuthology.orchestra.run.smithi001.stdout:created 2023-11-16T16:30:06.217778+0000 2023-11-16T16:38:43.617 INFO:teuthology.orchestra.run.smithi001.stdout:modified 2023-11-16T16:38:03.179244+0000 2023-11-16T16:38:43.617 INFO:teuthology.orchestra.run.smithi001.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-11-16T16:38:43.617 INFO:teuthology.orchestra.run.smithi001.stdout:crush_version 26 2023-11-16T16:38:43.617 INFO:teuthology.orchestra.run.smithi001.stdout:full_ratio 0.95 2023-11-16T16:38:43.617 INFO:teuthology.orchestra.run.smithi001.stdout:backfillfull_ratio 0.9 2023-11-16T16:38:43.617 INFO:teuthology.orchestra.run.smithi001.stdout:nearfull_ratio 0.85 2023-11-16T16:38:43.618 INFO:teuthology.orchestra.run.smithi001.stdout:require_min_compat_client luminous 2023-11-16T16:38:43.618 INFO:teuthology.orchestra.run.smithi001.stdout:min_compat_client luminous 2023-11-16T16:38:43.618 INFO:teuthology.orchestra.run.smithi001.stdout:require_osd_release reef 2023-11-16T16:38:43.618 INFO:teuthology.orchestra.run.smithi001.stdout:stretch_mode_enabled false 2023-11-16T16:38:43.618 INFO:teuthology.orchestra.run.smithi001.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-11-16T16:38:43.618 INFO:teuthology.orchestra.run.smithi001.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2023-11-16T16:38:43.618 INFO:teuthology.orchestra.run.smithi001.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2023-11-16T16:38:43.619 INFO:teuthology.orchestra.run.smithi001.stdout:max_osd 12 2023-11-16T16:38:43.619 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0 up in weight 1 up_from 8 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6802/2469671609,v1:172.21.15.1:6803/2469671609] [v2:172.21.15.1:6804/2469671609,v1:172.21.15.1:6805/2469671609] exists,up a8480507-5275-4e52-953b-626110beb436 2023-11-16T16:38:43.619 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1 up in weight 1 up_from 13 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6800/247994796,v1:172.21.15.26:6801/247994796] [v2:172.21.15.26:6802/247994796,v1:172.21.15.26:6803/247994796] exists,up 1d011db2-99d1-4575-bdc5-51f88a308fdc 2023-11-16T16:38:43.619 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2 up in weight 1 up_from 18 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6800/1659316290,v1:172.21.15.53:6801/1659316290] [v2:172.21.15.53:6802/1659316290,v1:172.21.15.53:6803/1659316290] exists,up 4d7a2bd1-3c8e-4546-a243-f0354473b031 2023-11-16T16:38:43.619 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3 up in weight 1 up_from 26 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6810/2728987281,v1:172.21.15.1:6811/2728987281] [v2:172.21.15.1:6812/2728987281,v1:172.21.15.1:6813/2728987281] exists,up 9c290fd9-e121-4a81-9687-b0eb93debd86 2023-11-16T16:38:43.619 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4 up in weight 1 up_from 34 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6808/2954740024,v1:172.21.15.26:6809/2954740024] [v2:172.21.15.26:6810/2954740024,v1:172.21.15.26:6811/2954740024] exists,up 533226bc-cc57-4f15-9e29-820294d57e25 2023-11-16T16:38:43.619 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5 up in weight 1 up_from 39 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6808/3966666318,v1:172.21.15.53:6809/3966666318] [v2:172.21.15.53:6810/3966666318,v1:172.21.15.53:6811/3966666318] exists,up d3ab25eb-e1d4-4557-bc89-efc9072401ce 2023-11-16T16:38:43.619 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6 up in weight 1 up_from 47 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6818/3364032646,v1:172.21.15.1:6819/3364032646] [v2:172.21.15.1:6820/3364032646,v1:172.21.15.1:6821/3364032646] exists,up 180f506e-afb8-4043-b0b4-2dc86f6d4bc3 2023-11-16T16:38:43.620 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7 up in weight 1 up_from 52 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6816/3766901391,v1:172.21.15.26:6817/3766901391] [v2:172.21.15.26:6818/3766901391,v1:172.21.15.26:6819/3766901391] exists,up 0d837bac-194f-498d-b7fb-b3147e992329 2023-11-16T16:38:43.620 INFO:teuthology.orchestra.run.smithi001.stdout:osd.8 up in weight 1 up_from 59 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6816/3224335871,v1:172.21.15.53:6817/3224335871] [v2:172.21.15.53:6818/3224335871,v1:172.21.15.53:6819/3224335871] exists,up 933f231a-7454-47b0-9291-f1fa64f0baec 2023-11-16T16:38:43.620 INFO:teuthology.orchestra.run.smithi001.stdout:osd.9 up in weight 1 up_from 67 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6826/1391199844,v1:172.21.15.1:6827/1391199844] [v2:172.21.15.1:6828/1391199844,v1:172.21.15.1:6829/1391199844] exists,up 245fcbb3-c651-4703-ab80-309dc98bd983 2023-11-16T16:38:43.620 INFO:teuthology.orchestra.run.smithi001.stdout:osd.10 up in weight 1 up_from 72 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6824/1773545447,v1:172.21.15.26:6825/1773545447] [v2:172.21.15.26:6826/1773545447,v1:172.21.15.26:6827/1773545447] exists,up 2c090e7a-5240-4d52-ae4f-e94fe05383d9 2023-11-16T16:38:43.620 INFO:teuthology.orchestra.run.smithi001.stdout:osd.11 up in weight 1 up_from 79 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6824/2564669546,v1:172.21.15.53:6825/2564669546] [v2:172.21.15.53:6826/2564669546,v1:172.21.15.53:6827/2564669546] exists,up 4511e32d-e469-45e1-8310-0ff400cb7b54 2023-11-16T16:38:43.620 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 2.e [2,1] 2023-11-16T16:38:43.620 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 2.10 [8,11] 2023-11-16T16:38:43.620 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 2.16 [2,4] 2023-11-16T16:38:43.621 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 2.1d [8,11] 2023-11-16T16:38:43.621 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.0 [5,9] 2023-11-16T16:38:43.621 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.1b [4,8] 2023-11-16T16:38:43.621 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.23 [4,7] 2023-11-16T16:38:43.621 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.36 [4,9] 2023-11-16T16:38:43.621 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.38 [3,9] 2023-11-16T16:38:43.621 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/1494336948 expires 2023-11-17T16:31:01.391197+0000 2023-11-16T16:38:43.621 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:6801/3137425292 expires 2023-11-17T16:31:01.391197+0000 2023-11-16T16:38:43.621 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:6800/3137425292 expires 2023-11-17T16:31:01.391197+0000 2023-11-16T16:38:43.622 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:6800/1726282374 expires 2023-11-17T16:30:35.064289+0000 2023-11-16T16:38:43.622 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/1641976669 expires 2023-11-17T16:30:35.064289+0000 2023-11-16T16:38:43.622 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/2895923638 expires 2023-11-17T16:30:35.064289+0000 2023-11-16T16:38:43.622 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:6801/1726282374 expires 2023-11-17T16:30:35.064289+0000 2023-11-16T16:38:43.622 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/78811711 expires 2023-11-17T16:31:01.391197+0000 2023-11-16T16:38:43.622 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/4131957708 expires 2023-11-17T16:31:01.391197+0000 2023-11-16T16:38:43.622 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/344403698 expires 2023-11-17T16:30:35.064289+0000 2023-11-16T16:38:43.623 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.617+0000 7f91acaf1700 1 -- 172.21.15.1:0/3491603671 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f91900a1150 msgr2=0x7f91900a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:43.623 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.617+0000 7f91acaf1700 1 --2- 172.21.15.1:0/3491603671 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f91900a1150 0x7f91900a3600 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9198005950 tx=0x7f91980058e0 comp rx=0 tx=0).stop 2023-11-16T16:38:43.623 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.617+0000 7f91acaf1700 1 -- 172.21.15.1:0/3491603671 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91a8100d70 msgr2=0x7f91a819d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:43.623 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.617+0000 7f91acaf1700 1 --2- 172.21.15.1:0/3491603671 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91a8100d70 0x7f91a819d610 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f91a0007f10 tx=0x7f91a000d3b0 comp rx=0 tx=0).stop 2023-11-16T16:38:43.624 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.617+0000 7f91acaf1700 1 -- 172.21.15.1:0/3491603671 shutdown_connections 2023-11-16T16:38:43.624 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.617+0000 7f91acaf1700 1 --2- 172.21.15.1:0/3491603671 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f91900a1150 0x7f91900a3600 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:43.624 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.617+0000 7f91acaf1700 1 --2- 172.21.15.1:0/3491603671 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91a8100d70 0x7f91a819d610 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:43.624 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.617+0000 7f91acaf1700 1 --2- 172.21.15.1:0/3491603671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91a80ff1e0 0x7f91a8196050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:43.624 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.617+0000 7f91acaf1700 1 --2- 172.21.15.1:0/3491603671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f91a81003e0 0x7f91a8196590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:43.624 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.617+0000 7f91acaf1700 1 -- 172.21.15.1:0/3491603671 >> 172.21.15.1:0/3491603671 conn(0x7f91a80fa7f0 msgr2=0x7f91a8068630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:43.624 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.617+0000 7f91acaf1700 1 -- 172.21.15.1:0/3491603671 shutdown_connections 2023-11-16T16:38:43.624 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:43.617+0000 7f91acaf1700 1 -- 172.21.15.1:0/3491603671 wait complete. 2023-11-16T16:38:44.064 INFO:teuthology.run_tasks:Running task fs.ready... 2023-11-16T16:38:44.075 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-16T16:38:44.208 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.204+0000 7fd01451e700 1 -- 172.21.15.1:0/1789519352 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd00c0734d0 msgr2=0x7fd00c073940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:44.208 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.204+0000 7fd01451e700 1 --2- 172.21.15.1:0/1789519352 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd00c0734d0 0x7fd00c073940 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd008009990 tx=0x7fd008009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:44.208 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.204+0000 7fd01451e700 1 -- 172.21.15.1:0/1789519352 shutdown_connections 2023-11-16T16:38:44.208 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.204+0000 7fd01451e700 1 --2- 172.21.15.1:0/1789519352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd00c108390 0x7fd00c10a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:44.208 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.204+0000 7fd01451e700 1 --2- 172.21.15.1:0/1789519352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd00c074ab0 0x7fd00c072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:44.208 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.204+0000 7fd01451e700 1 --2- 172.21.15.1:0/1789519352 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd00c0734d0 0x7fd00c073940 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:44.208 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.204+0000 7fd01451e700 1 -- 172.21.15.1:0/1789519352 >> 172.21.15.1:0/1789519352 conn(0x7fd00c0fbac0 msgr2=0x7fd00c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:44.208 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.204+0000 7fd01451e700 1 -- 172.21.15.1:0/1789519352 shutdown_connections 2023-11-16T16:38:44.209 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.204+0000 7fd01451e700 1 -- 172.21.15.1:0/1789519352 wait complete. 2023-11-16T16:38:44.209 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.205+0000 7fd01451e700 1 Processor -- start 2023-11-16T16:38:44.209 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.205+0000 7fd01451e700 1 -- start start 2023-11-16T16:38:44.209 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.205+0000 7fd01451e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd00c0734d0 0x7fd00c1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:44.209 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.205+0000 7fd01451e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd00c074ab0 0x7fd00c1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:44.209 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.205+0000 7fd01451e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd00c108390 0x7fd00c19fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:44.209 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.205+0000 7fd01451e700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c1a0030 con 0x7fd00c0734d0 2023-11-16T16:38:44.210 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.205+0000 7fd01451e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c1a0170 con 0x7fd00c108390 2023-11-16T16:38:44.210 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.205+0000 7fd01451e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c1a0300 con 0x7fd00c074ab0 2023-11-16T16:38:44.210 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.205+0000 7fd0122ba700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd00c0734d0 0x7fd00c1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:44.210 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.205+0000 7fd0122ba700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd00c0734d0 0x7fd00c1984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42536/0 (socket says 172.21.15.1:42536) 2023-11-16T16:38:44.210 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.205+0000 7fd0122ba700 1 -- 172.21.15.1:0/18414989 learned_addr learned my addr 172.21.15.1:0/18414989 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:44.210 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.205+0000 7fd012abb700 1 --2- 172.21.15.1:0/18414989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd00c108390 0x7fd00c19fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:44.210 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.205+0000 7fd011ab9700 1 --2- 172.21.15.1:0/18414989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd00c074ab0 0x7fd00c1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:44.210 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.205+0000 7fd0122ba700 1 -- 172.21.15.1:0/18414989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd00c074ab0 msgr2=0x7fd00c1989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:44.210 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.205+0000 7fd0122ba700 1 --2- 172.21.15.1:0/18414989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd00c074ab0 0x7fd00c1989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:44.210 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.205+0000 7fd0122ba700 1 -- 172.21.15.1:0/18414989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd00c108390 msgr2=0x7fd00c19fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:44.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.205+0000 7fd0122ba700 1 --2- 172.21.15.1:0/18414989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd00c108390 0x7fd00c19fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:44.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.205+0000 7fd0122ba700 1 -- 172.21.15.1:0/18414989 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0080095f0 con 0x7fd00c0734d0 2023-11-16T16:38:44.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.206+0000 7fd0122ba700 1 --2- 172.21.15.1:0/18414989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd00c0734d0 0x7fd00c1984b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcffc00b530 tx=0x7fcffc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:44.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.206+0000 7fd0037fe700 1 -- 172.21.15.1:0/18414989 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcffc011830 con 0x7fd00c0734d0 2023-11-16T16:38:44.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.206+0000 7fd0037fe700 1 -- 172.21.15.1:0/18414989 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcffc011e50 con 0x7fd00c0734d0 2023-11-16T16:38:44.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.206+0000 7fd01451e700 1 -- 172.21.15.1:0/18414989 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd00c10e0f0 con 0x7fd00c0734d0 2023-11-16T16:38:44.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.206+0000 7fd0037fe700 1 -- 172.21.15.1:0/18414989 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcffc00f2e0 con 0x7fd00c0734d0 2023-11-16T16:38:44.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.206+0000 7fd01451e700 1 -- 172.21.15.1:0/18414989 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd00c10df60 con 0x7fd00c0734d0 2023-11-16T16:38:44.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.207+0000 7fd01451e700 1 -- 172.21.15.1:0/18414989 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd00c106ad0 con 0x7fd00c0734d0 2023-11-16T16:38:44.212 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.208+0000 7fd0037fe700 1 -- 172.21.15.1:0/18414989 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fcffc0105a0 con 0x7fd00c0734d0 2023-11-16T16:38:44.212 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.208+0000 7fd0037fe700 1 --2- 172.21.15.1:0/18414989 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fcff80a11d0 0x7fcff80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:44.212 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.209+0000 7fd011ab9700 1 --2- 172.21.15.1:0/18414989 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fcff80a11d0 0x7fcff80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:44.212 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.209+0000 7fd0037fe700 1 -- 172.21.15.1:0/18414989 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcffc0cc240 con 0x7fd00c0734d0 2023-11-16T16:38:44.213 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.209+0000 7fd011ab9700 1 --2- 172.21.15.1:0/18414989 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fcff80a11d0 0x7fcff80a3680 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd00800b5c0 tx=0x7fd008005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:44.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.215+0000 7fd0037fe700 1 -- 172.21.15.1:0/18414989 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fcffc095990 con 0x7fd00c0734d0 2023-11-16T16:38:44.419 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.419+0000 7fd01451e700 1 -- 172.21.15.1:0/18414989 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd00c02cfe0 con 0x7fd00c0734d0 2023-11-16T16:38:44.421 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.420+0000 7fd0037fe700 1 -- 172.21.15.1:0/18414989 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22469 (secure 0 0 0) 0x7fcffc098fb0 con 0x7fd00c0734d0 2023-11-16T16:38:44.422 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:38:44.422 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":86,"fsid":"35d9b010-849d-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:30:06.217778+0000","modified":"2023-11-16T16:38:03.179244+0000","last_up_change":"2023-11-16T16:36:31.735403+0000","last_in_change":"2023-11-16T16:36:14.083021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:33:15.916723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-16T16:37:12.158031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-16T16:37:13.272343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a8480507-5275-4e52-953b-626110beb436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6803","nonce":2469671609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6805","nonce":2469671609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6809","nonce":2469671609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6807","nonce":2469671609}]},"public_addr":"172.21.15.1:6803/2469671609","cluster_addr":"172.21.15.1:6805/2469671609","heartbeat_back_addr":"172.21.15.1:6809/2469671609","heartbeat_front_addr":"172.21.15.1:6807/2469671609","state":["exists","up"]},{"osd":1,"uuid":"1d011db2-99d1-4575-bdc5-51f88a308fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6801","nonce":247994796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6803","nonce":247994796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6807","nonce":247994796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6805","nonce":247994796}]},"public_addr":"172.21.15.26:6801/247994796","cluster_addr":"172.21.15.26:6803/247994796","heartbeat_back_addr":"172.21.15.26:6807/247994796","heartbeat_front_addr":"172.21.15.26:6805/247994796","state":["exists","up"]},{"osd":2,"uuid":"4d7a2bd1-3c8e-4546-a243-f0354473b031","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6801","nonce":1659316290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6803","nonce":1659316290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6807","nonce":1659316290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6805","nonce":1659316290}]},"public_addr":"172.21.15.53:6801/1659316290","cluster_addr":"172.21.15.53:6803/1659316290","heartbeat_back_addr":"172.21.15.53:6807/1659316290","heartbeat_front_addr":"172.21.15.53:6805/1659316290","state":["exists","up"]},{"osd":3,"uuid":"9c290fd9-e121-4a81-9687-b0eb93debd86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6811","nonce":2728987281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6813","nonce":2728987281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6817","nonce":2728987281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6815","nonce":2728987281}]},"public_addr":"172.21.15.1:6811/2728987281","cluster_addr":"172.21.15.1:6813/2728987281","heartbeat_back_addr":"172.21.15.1:6817/2728987281","heartbeat_front_addr":"172.21.15.1:6815/2728987281","state":["exists","up"]},{"osd":4,"uuid":"533226bc-cc57-4f15-9e29-820294d57e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6809","nonce":2954740024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6811","nonce":2954740024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6815","nonce":2954740024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6813","nonce":2954740024}]},"public_addr":"172.21.15.26:6809/2954740024","cluster_addr":"172.21.15.26:6811/2954740024","heartbeat_back_addr":"172.21.15.26:6815/2954740024","heartbeat_front_addr":"172.21.15.26:6813/2954740024","state":["exists","up"]},{"osd":5,"uuid":"d3ab25eb-e1d4-4557-bc89-efc9072401ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6809","nonce":3966666318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6811","nonce":3966666318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6815","nonce":3966666318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6813","nonce":3966666318}]},"public_addr":"172.21.15.53:6809/3966666318","cluster_addr":"172.21.15.53:6811/3966666318","heartbeat_back_addr":"172.21.15.53:6815/3966666318","heartbeat_front_addr":"172.21.15.53:6813/3966666318","state":["exists","up"]},{"osd":6,"uuid":"180f506e-afb8-4043-b0b4-2dc86f6d4bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6819","nonce":3364032646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6821","nonce":3364032646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6825","nonce":3364032646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6823","nonce":3364032646}]},"public_addr":"172.21.15.1:6819/3364032646","cluster_addr":"172.21.15.1:6821/3364032646","heartbeat_back_addr":"172.21.15.1:6825/3364032646","heartbeat_front_addr":"172.21.15.1:6823/3364032646","state":["exists","up"]},{"osd":7,"uuid":"0d837bac-194f-498d-b7fb-b3147e992329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6817","nonce":3766901391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6819","nonce":3766901391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6823","nonce":3766901391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6821","nonce":3766901391}]},"public_addr":"172.21.15.26:6817/3766901391","cluster_addr":"172.21.15.26:6819/3766901391","heartbeat_back_addr":"172.21.15.26:6823/3766901391","heartbeat_front_addr":"172.21.15.26:6821/3766901391","state":["exists","up"]},{"osd":8,"uuid":"933f231a-7454-47b0-9291-f1fa64f0baec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6817","nonce":3224335871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6819","nonce":3224335871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6823","nonce":3224335871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6821","nonce":3224335871}]},"public_addr":"172.21.15.53:6817/3224335871","cluster_addr":"172.21.15.53:6819/3224335871","heartbeat_back_addr":"172.21.15.53:6823/3224335871","heartbeat_front_addr":"172.21.15.53:6821/3224335871","state":["exists","up"]},{"osd":9,"uuid":"245fcbb3-c651-4703-ab80-309dc98bd983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6827","nonce":1391199844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6829","nonce":1391199844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6833","nonce":1391199844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6831","nonce":1391199844}]},"public_addr":"172.21.15.1:6827/1391199844","cluster_addr":"172.21.15.1:6829/1391199844","heartbeat_back_addr":"172.21.15.1:6833/1391199844","heartbeat_front_addr":"172.21.15.1:6831/1391199844","state":["exists","up"]},{"osd":10,"uuid":"2c090e7a-5240-4d52-ae4f-e94fe05383d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6825","nonce":1773545447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6827","nonce":1773545447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6831","nonce":1773545447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6829","nonce":1773545447}]},"public_addr":"172.21.15.26:6825/1773545447","cluster_addr":"172.21.15.26:6827/1773545447","heartbeat_back_addr":"172.21.15.26:6831/1773545447","heartbeat_front_addr":"172.21.15.26:6829/1773545447","state":["exists","up"]},{"osd":11,"uuid":"4511e32d-e469-45e1-8310-0ff400cb7b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6825","nonce":2564669546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6827","nonce":2564669546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6831","nonce":2564669546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6829","nonce":2564669546}]},"public_addr":"172.21.15.53:6825/2564669546","cluster_addr":"172.21.15.53:6827/2564669546","heartbeat_back_addr":"172.21.15.53:6831/2564669546","heartbeat_front_addr":"172.21.15.53:6829/2564669546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:28.140250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:49.885461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:12.521487+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:34.926197+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:56.815594+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:18.387751+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:40.587106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:02.319370+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:23.832298+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:46.338040+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:08.730414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:30.050070+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1494336948":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6801/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6800/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6800/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/1641976669":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/2895923638":"2023-11-17T16:30:35.064289+0000","172.21.15.1:6801/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/78811711":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/4131957708":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/344403698":"2023-11-17T16:30:35.064289+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:38:44.423 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.422+0000 7fd01451e700 1 -- 172.21.15.1:0/18414989 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fcff80a11d0 msgr2=0x7fcff80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:44.424 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.422+0000 7fd01451e700 1 --2- 172.21.15.1:0/18414989 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fcff80a11d0 0x7fcff80a3680 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd00800b5c0 tx=0x7fd008005fb0 comp rx=0 tx=0).stop 2023-11-16T16:38:44.424 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.422+0000 7fd01451e700 1 -- 172.21.15.1:0/18414989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd00c0734d0 msgr2=0x7fd00c1984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:44.424 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.422+0000 7fd01451e700 1 --2- 172.21.15.1:0/18414989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd00c0734d0 0x7fd00c1984b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcffc00b530 tx=0x7fcffc00b840 comp rx=0 tx=0).stop 2023-11-16T16:38:44.424 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.422+0000 7fd01451e700 1 -- 172.21.15.1:0/18414989 shutdown_connections 2023-11-16T16:38:44.424 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.422+0000 7fd01451e700 1 --2- 172.21.15.1:0/18414989 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fcff80a11d0 0x7fcff80a3680 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:44.424 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.422+0000 7fd01451e700 1 --2- 172.21.15.1:0/18414989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd00c108390 0x7fd00c19fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:44.424 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.422+0000 7fd01451e700 1 --2- 172.21.15.1:0/18414989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd00c0734d0 0x7fd00c1984b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:44.424 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.422+0000 7fd01451e700 1 --2- 172.21.15.1:0/18414989 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd00c074ab0 0x7fd00c1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:44.424 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.422+0000 7fd01451e700 1 -- 172.21.15.1:0/18414989 >> 172.21.15.1:0/18414989 conn(0x7fd00c0fbac0 msgr2=0x7fd00c0fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:44.424 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.422+0000 7fd01451e700 1 -- 172.21.15.1:0/18414989 shutdown_connections 2023-11-16T16:38:44.424 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.422+0000 7fd01451e700 1 -- 172.21.15.1:0/18414989 wait complete. 2023-11-16T16:38:44.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:44 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3491603671' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-11-16T16:38:44.436 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-16T16:33:15.916723+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-16T16:37:12.158031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-16T16:37:13.272343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-16T16:38:44.436 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-16T16:38:44.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:44 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/3491603671' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-11-16T16:38:44.565 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.564+0000 7f082e919700 1 -- 172.21.15.1:0/1072845062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0828100d30 msgr2=0x7f08281075f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:44.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.564+0000 7f082e919700 1 --2- 172.21.15.1:0/1072845062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0828100d30 0x7f08281075f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f082000b0a0 tx=0x7f082000b3b0 comp rx=0 tx=0).stop 2023-11-16T16:38:44.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.564+0000 7f082e919700 1 -- 172.21.15.1:0/1072845062 shutdown_connections 2023-11-16T16:38:44.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.564+0000 7f082e919700 1 --2- 172.21.15.1:0/1072845062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0828100d30 0x7f08281075f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:44.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.564+0000 7f082e919700 1 --2- 172.21.15.1:0/1072845062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08280ff1b0 0x7f08280ff5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:44.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.564+0000 7f082e919700 1 --2- 172.21.15.1:0/1072845062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08281003a0 0x7f08281007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:44.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.564+0000 7f082e919700 1 -- 172.21.15.1:0/1072845062 >> 172.21.15.1:0/1072845062 conn(0x7f08280fa7b0 msgr2=0x7f08280fcc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:44.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.564+0000 7f082e919700 1 -- 172.21.15.1:0/1072845062 shutdown_connections 2023-11-16T16:38:44.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.564+0000 7f082e919700 1 -- 172.21.15.1:0/1072845062 wait complete. 2023-11-16T16:38:44.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.565+0000 7f082e919700 1 Processor -- start 2023-11-16T16:38:44.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.565+0000 7f082e919700 1 -- start start 2023-11-16T16:38:44.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.565+0000 7f082e919700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08280ff1b0 0x7f0828198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:44.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.565+0000 7f082e919700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08281003a0 0x7f0828198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:44.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.565+0000 7f082e919700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0828100d30 0x7f082819f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:44.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.565+0000 7f082e919700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082819fe80 con 0x7f0828100d30 2023-11-16T16:38:44.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.565+0000 7f082e919700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082819ffc0 con 0x7f08280ff1b0 2023-11-16T16:38:44.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.565+0000 7f082e919700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08281a0150 con 0x7f08281003a0 2023-11-16T16:38:44.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.565+0000 7f082ceb6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0828100d30 0x7f082819f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:44.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.565+0000 7f082ceb6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0828100d30 0x7f082819f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42554/0 (socket says 172.21.15.1:42554) 2023-11-16T16:38:44.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.565+0000 7f082ceb6700 1 -- 172.21.15.1:0/2537911593 learned_addr learned my addr 172.21.15.1:0/2537911593 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:44.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.565+0000 7f0827fff700 1 --2- 172.21.15.1:0/2537911593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08280ff1b0 0x7f0828198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:44.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.565+0000 7f08277fe700 1 --2- 172.21.15.1:0/2537911593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08281003a0 0x7f0828198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:44.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.565+0000 7f082ceb6700 1 -- 172.21.15.1:0/2537911593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08281003a0 msgr2=0x7f0828198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:44.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.565+0000 7f082ceb6700 1 --2- 172.21.15.1:0/2537911593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08281003a0 0x7f0828198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:44.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.565+0000 7f082ceb6700 1 -- 172.21.15.1:0/2537911593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08280ff1b0 msgr2=0x7f0828198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:44.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.565+0000 7f082ceb6700 1 --2- 172.21.15.1:0/2537911593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08280ff1b0 0x7f0828198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:44.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.565+0000 7f082ceb6700 1 -- 172.21.15.1:0/2537911593 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0820009d00 con 0x7f0828100d30 2023-11-16T16:38:44.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.566+0000 7f082ceb6700 1 --2- 172.21.15.1:0/2537911593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0828100d30 0x7f082819f8b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f082000b830 tx=0x7f0820004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:44.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.566+0000 7f08257fa700 1 -- 172.21.15.1:0/2537911593 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082000e040 con 0x7f0828100d30 2023-11-16T16:38:44.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.566+0000 7f082e919700 1 -- 172.21.15.1:0/2537911593 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f082810ab30 con 0x7f0828100d30 2023-11-16T16:38:44.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.566+0000 7f08257fa700 1 -- 172.21.15.1:0/2537911593 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08200094d0 con 0x7f0828100d30 2023-11-16T16:38:44.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.566+0000 7f08257fa700 1 -- 172.21.15.1:0/2537911593 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082001bd20 con 0x7f0828100d30 2023-11-16T16:38:44.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.566+0000 7f082e919700 1 -- 172.21.15.1:0/2537911593 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f082810a9a0 con 0x7f0828100d30 2023-11-16T16:38:44.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.567+0000 7f082e919700 1 -- 172.21.15.1:0/2537911593 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0828067ea0 con 0x7f0828100d30 2023-11-16T16:38:44.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.568+0000 7f08257fa700 1 -- 172.21.15.1:0/2537911593 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f082001b450 con 0x7f0828100d30 2023-11-16T16:38:44.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.568+0000 7f08257fa700 1 --2- 172.21.15.1:0/2537911593 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f08100a1150 0x7f08100a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:44.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.569+0000 7f0827fff700 1 --2- 172.21.15.1:0/2537911593 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f08100a1150 0x7f08100a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:44.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.569+0000 7f08257fa700 1 -- 172.21.15.1:0/2537911593 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f08200cec30 con 0x7f0828100d30 2023-11-16T16:38:44.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.569+0000 7f0827fff700 1 --2- 172.21.15.1:0/2537911593 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f08100a1150 0x7f08100a3600 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0828100200 tx=0x7f081800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:44.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.572+0000 7f08257fa700 1 -- 172.21.15.1:0/2537911593 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0820098410 con 0x7f0828100d30 2023-11-16T16:38:44.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:44 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/3491603671' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-11-16T16:38:44.774 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.771+0000 7f082e919700 1 -- 172.21.15.1:0/2537911593 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f08281a0cf0 con 0x7f0828100d30 2023-11-16T16:38:44.774 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.772+0000 7f08257fa700 1 -- 172.21.15.1:0/2537911593 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f08200210a0 con 0x7f0828100d30 2023-11-16T16:38:44.774 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2023-11-16T16:38:44.775 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.774+0000 7f082e919700 1 -- 172.21.15.1:0/2537911593 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f08100a1150 msgr2=0x7f08100a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:44.775 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.774+0000 7f082e919700 1 --2- 172.21.15.1:0/2537911593 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f08100a1150 0x7f08100a3600 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0828100200 tx=0x7f081800a380 comp rx=0 tx=0).stop 2023-11-16T16:38:44.775 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.774+0000 7f082e919700 1 -- 172.21.15.1:0/2537911593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0828100d30 msgr2=0x7f082819f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:44.775 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.774+0000 7f082e919700 1 --2- 172.21.15.1:0/2537911593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0828100d30 0x7f082819f8b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f082000b830 tx=0x7f0820004d00 comp rx=0 tx=0).stop 2023-11-16T16:38:44.775 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.774+0000 7f082e919700 1 -- 172.21.15.1:0/2537911593 shutdown_connections 2023-11-16T16:38:44.775 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.774+0000 7f082e919700 1 --2- 172.21.15.1:0/2537911593 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f08100a1150 0x7f08100a3600 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:44.776 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.774+0000 7f082e919700 1 --2- 172.21.15.1:0/2537911593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0828100d30 0x7f082819f8b0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:44.776 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.774+0000 7f082e919700 1 --2- 172.21.15.1:0/2537911593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08280ff1b0 0x7f0828198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:44.776 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.774+0000 7f082e919700 1 --2- 172.21.15.1:0/2537911593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f08281003a0 0x7f0828198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:44.776 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.774+0000 7f082e919700 1 -- 172.21.15.1:0/2537911593 >> 172.21.15.1:0/2537911593 conn(0x7f08280fa7b0 msgr2=0x7f08280690f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:44.776 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.774+0000 7f082e919700 1 -- 172.21.15.1:0/2537911593 shutdown_connections 2023-11-16T16:38:44.776 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.774+0000 7f082e919700 1 -- 172.21.15.1:0/2537911593 wait complete. 2023-11-16T16:38:44.787 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-16T16:38:44.918 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.915+0000 7f14321ec700 1 -- 172.21.15.1:0/4088169755 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f142c1003b0 msgr2=0x7f142c100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:44.919 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.915+0000 7f14321ec700 1 --2- 172.21.15.1:0/4088169755 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f142c1003b0 0x7f142c100800 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1420009990 tx=0x7f1420009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:44.919 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.915+0000 7f14321ec700 1 -- 172.21.15.1:0/4088169755 shutdown_connections 2023-11-16T16:38:44.919 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.915+0000 7f14321ec700 1 --2- 172.21.15.1:0/4088169755 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f142c100d40 0x7f142c107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:44.919 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.915+0000 7f14321ec700 1 --2- 172.21.15.1:0/4088169755 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f142c0ff1c0 0x7f142c0ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:44.919 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.915+0000 7f14321ec700 1 --2- 172.21.15.1:0/4088169755 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f142c1003b0 0x7f142c100800 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:44.919 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.915+0000 7f14321ec700 1 -- 172.21.15.1:0/4088169755 >> 172.21.15.1:0/4088169755 conn(0x7f142c0fa800 msgr2=0x7f142c0fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:44.919 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.915+0000 7f14321ec700 1 -- 172.21.15.1:0/4088169755 shutdown_connections 2023-11-16T16:38:44.919 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.915+0000 7f14321ec700 1 -- 172.21.15.1:0/4088169755 wait complete. 2023-11-16T16:38:44.919 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.916+0000 7f14321ec700 1 Processor -- start 2023-11-16T16:38:44.919 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.916+0000 7f14321ec700 1 -- start start 2023-11-16T16:38:44.920 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.916+0000 7f14321ec700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f142c0ff1c0 0x7f142c196050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:44.920 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.916+0000 7f14321ec700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f142c1003b0 0x7f142c196590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:44.920 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.916+0000 7f14321ec700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f142c100d40 0x7f142c19d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:44.920 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.916+0000 7f14321ec700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142c19dbd0 con 0x7f142c0ff1c0 2023-11-16T16:38:44.920 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.916+0000 7f14321ec700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142c19dd10 con 0x7f142c100d40 2023-11-16T16:38:44.920 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.916+0000 7f14321ec700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142c19dea0 con 0x7f142c1003b0 2023-11-16T16:38:44.920 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.916+0000 7f142b7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f142c0ff1c0 0x7f142c196050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:44.920 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.916+0000 7f142b7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f142c0ff1c0 0x7f142c196050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42564/0 (socket says 172.21.15.1:42564) 2023-11-16T16:38:44.921 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.916+0000 7f142b7fe700 1 -- 172.21.15.1:0/3492139125 learned_addr learned my addr 172.21.15.1:0/3492139125 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:44.921 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.916+0000 7f142affd700 1 --2- 172.21.15.1:0/3492139125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f142c1003b0 0x7f142c196590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:44.921 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.917+0000 7f142bfff700 1 --2- 172.21.15.1:0/3492139125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f142c100d40 0x7f142c19d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:44.921 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.917+0000 7f142b7fe700 1 -- 172.21.15.1:0/3492139125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f142c1003b0 msgr2=0x7f142c196590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:44.921 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.917+0000 7f142b7fe700 1 --2- 172.21.15.1:0/3492139125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f142c1003b0 0x7f142c196590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:44.921 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.917+0000 7f142b7fe700 1 -- 172.21.15.1:0/3492139125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f142c100d40 msgr2=0x7f142c19d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:44.921 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.917+0000 7f142b7fe700 1 --2- 172.21.15.1:0/3492139125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f142c100d40 0x7f142c19d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:44.921 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.917+0000 7f142b7fe700 1 -- 172.21.15.1:0/3492139125 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14200095f0 con 0x7f142c0ff1c0 2023-11-16T16:38:44.921 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.917+0000 7f142b7fe700 1 --2- 172.21.15.1:0/3492139125 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f142c0ff1c0 0x7f142c196050 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f141c00b500 tx=0x7f141c00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:44.921 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.917+0000 7f1428ff9700 1 -- 172.21.15.1:0/3492139125 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c011830 con 0x7f142c0ff1c0 2023-11-16T16:38:44.922 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.917+0000 7f1428ff9700 1 -- 172.21.15.1:0/3492139125 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f141c011e50 con 0x7f142c0ff1c0 2023-11-16T16:38:44.922 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.917+0000 7f14321ec700 1 -- 172.21.15.1:0/3492139125 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f142c10ac10 con 0x7f142c0ff1c0 2023-11-16T16:38:44.922 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.917+0000 7f1428ff9700 1 -- 172.21.15.1:0/3492139125 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c00f2e0 con 0x7f142c0ff1c0 2023-11-16T16:38:44.922 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.917+0000 7f14321ec700 1 -- 172.21.15.1:0/3492139125 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f142c10aa80 con 0x7f142c0ff1c0 2023-11-16T16:38:44.923 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.918+0000 7f14321ec700 1 -- 172.21.15.1:0/3492139125 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f142c19e6c0 con 0x7f142c0ff1c0 2023-11-16T16:38:44.923 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.919+0000 7f1428ff9700 1 -- 172.21.15.1:0/3492139125 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f141c010580 con 0x7f142c0ff1c0 2023-11-16T16:38:44.923 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.920+0000 7f1428ff9700 1 --2- 172.21.15.1:0/3492139125 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f14140a11d0 0x7f14140a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:44.923 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.920+0000 7f142affd700 1 --2- 172.21.15.1:0/3492139125 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f14140a11d0 0x7f14140a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:44.923 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.920+0000 7f1428ff9700 1 -- 172.21.15.1:0/3492139125 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f141c0cc2f0 con 0x7f142c0ff1c0 2023-11-16T16:38:44.923 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.920+0000 7f142affd700 1 --2- 172.21.15.1:0/3492139125 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f14140a11d0 0x7f14140a3680 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f142000b5c0 tx=0x7f1420005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:44.925 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:44.924+0000 7f1428ff9700 1 -- 172.21.15.1:0/3492139125 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f141c095a40 con 0x7f142c0ff1c0 2023-11-16T16:38:45.124 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.123+0000 7f14321ec700 1 -- 172.21.15.1:0/3492139125 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f142c19ec70 con 0x7f142c0ff1c0 2023-11-16T16:38:45.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.124+0000 7f1428ff9700 1 -- 172.21.15.1:0/3492139125 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7f141c099060 con 0x7f142c0ff1c0 2023-11-16T16:38:45.125 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2023-11-16T16:38:45.127 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.126+0000 7f14321ec700 1 -- 172.21.15.1:0/3492139125 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f14140a11d0 msgr2=0x7f14140a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:45.127 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.126+0000 7f14321ec700 1 --2- 172.21.15.1:0/3492139125 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f14140a11d0 0x7f14140a3680 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f142000b5c0 tx=0x7f1420005fb0 comp rx=0 tx=0).stop 2023-11-16T16:38:45.127 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.126+0000 7f14321ec700 1 -- 172.21.15.1:0/3492139125 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f142c0ff1c0 msgr2=0x7f142c196050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:45.128 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.126+0000 7f14321ec700 1 --2- 172.21.15.1:0/3492139125 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f142c0ff1c0 0x7f142c196050 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f141c00b500 tx=0x7f141c00b810 comp rx=0 tx=0).stop 2023-11-16T16:38:45.128 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.126+0000 7f14321ec700 1 -- 172.21.15.1:0/3492139125 shutdown_connections 2023-11-16T16:38:45.128 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.126+0000 7f14321ec700 1 --2- 172.21.15.1:0/3492139125 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f14140a11d0 0x7f14140a3680 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:45.128 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.126+0000 7f14321ec700 1 --2- 172.21.15.1:0/3492139125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f142c100d40 0x7f142c19d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:45.128 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.126+0000 7f14321ec700 1 --2- 172.21.15.1:0/3492139125 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f142c0ff1c0 0x7f142c196050 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:45.128 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.126+0000 7f14321ec700 1 --2- 172.21.15.1:0/3492139125 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f142c1003b0 0x7f142c196590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:45.128 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.126+0000 7f14321ec700 1 -- 172.21.15.1:0/3492139125 >> 172.21.15.1:0/3492139125 conn(0x7f142c0fa800 msgr2=0x7f142c068e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:45.128 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.126+0000 7f14321ec700 1 -- 172.21.15.1:0/3492139125 shutdown_connections 2023-11-16T16:38:45.128 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.126+0000 7f14321ec700 1 -- 172.21.15.1:0/3492139125 wait complete. 2023-11-16T16:38:45.139 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-16T16:38:45.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.269+0000 7fd98d658700 1 -- 172.21.15.1:0/4020159165 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd988100300 msgr2=0x7fd988100710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:45.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.269+0000 7fd98d658700 1 --2- 172.21.15.1:0/4020159165 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd988100300 0x7fd988100710 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd97c009990 tx=0x7fd97c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:45.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.269+0000 7fd98d658700 1 -- 172.21.15.1:0/4020159165 shutdown_connections 2023-11-16T16:38:45.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.269+0000 7fd98d658700 1 --2- 172.21.15.1:0/4020159165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd988101e80 0x7fd988108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:45.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.269+0000 7fd98d658700 1 --2- 172.21.15.1:0/4020159165 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd988100300 0x7fd988100710 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:45.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.269+0000 7fd98d658700 1 --2- 172.21.15.1:0/4020159165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd9881014f0 0x7fd988101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:45.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.269+0000 7fd98d658700 1 -- 172.21.15.1:0/4020159165 >> 172.21.15.1:0/4020159165 conn(0x7fd9880fb8e0 msgr2=0x7fd9880fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:45.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.269+0000 7fd98d658700 1 -- 172.21.15.1:0/4020159165 shutdown_connections 2023-11-16T16:38:45.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.269+0000 7fd98d658700 1 -- 172.21.15.1:0/4020159165 wait complete. 2023-11-16T16:38:45.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.269+0000 7fd98d658700 1 Processor -- start 2023-11-16T16:38:45.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.269+0000 7fd98d658700 1 -- start start 2023-11-16T16:38:45.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.270+0000 7fd98d658700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd988100300 0x7fd9880748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:45.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.270+0000 7fd98d658700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9881014f0 0x7fd988072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:45.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.270+0000 7fd98d658700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd988101e80 0x7fd988073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:45.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.270+0000 7fd98d658700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd988073bc0 con 0x7fd9881014f0 2023-11-16T16:38:45.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.270+0000 7fd98d658700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd988073d30 con 0x7fd988101e80 2023-11-16T16:38:45.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.270+0000 7fd98d658700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9881a0090 con 0x7fd988100300 2023-11-16T16:38:45.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.270+0000 7fd9867fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9881014f0 0x7fd988072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:45.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.270+0000 7fd9867fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9881014f0 0x7fd988072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42580/0 (socket says 172.21.15.1:42580) 2023-11-16T16:38:45.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.270+0000 7fd9867fc700 1 -- 172.21.15.1:0/2411021681 learned_addr learned my addr 172.21.15.1:0/2411021681 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:45.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.270+0000 7fd986ffd700 1 --2- 172.21.15.1:0/2411021681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd988100300 0x7fd9880748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:45.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.270+0000 7fd9877fe700 1 --2- 172.21.15.1:0/2411021681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd988101e80 0x7fd988073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:45.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.270+0000 7fd9867fc700 1 -- 172.21.15.1:0/2411021681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd988100300 msgr2=0x7fd9880748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:45.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.270+0000 7fd9867fc700 1 --2- 172.21.15.1:0/2411021681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd988100300 0x7fd9880748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:45.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.270+0000 7fd9867fc700 1 -- 172.21.15.1:0/2411021681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd988101e80 msgr2=0x7fd988073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:45.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.270+0000 7fd9867fc700 1 --2- 172.21.15.1:0/2411021681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd988101e80 0x7fd988073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:45.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.270+0000 7fd9867fc700 1 -- 172.21.15.1:0/2411021681 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd97c0095f0 con 0x7fd9881014f0 2023-11-16T16:38:45.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.270+0000 7fd9867fc700 1 --2- 172.21.15.1:0/2411021681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9881014f0 0x7fd988072ee0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd97800c9b0 tx=0x7fd97800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:45.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.270+0000 7fd96ffff700 1 -- 172.21.15.1:0/2411021681 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd978007b00 con 0x7fd9881014f0 2023-11-16T16:38:45.277 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.270+0000 7fd98d658700 1 -- 172.21.15.1:0/2411021681 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd98810bc40 con 0x7fd9881014f0 2023-11-16T16:38:45.277 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.270+0000 7fd96ffff700 1 -- 172.21.15.1:0/2411021681 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd978007c60 con 0x7fd9881014f0 2023-11-16T16:38:45.277 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.270+0000 7fd96ffff700 1 -- 172.21.15.1:0/2411021681 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd978018660 con 0x7fd9881014f0 2023-11-16T16:38:45.277 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.270+0000 7fd98d658700 1 -- 172.21.15.1:0/2411021681 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd98810bab0 con 0x7fd9881014f0 2023-11-16T16:38:45.277 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.271+0000 7fd98d658700 1 -- 172.21.15.1:0/2411021681 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9881078e0 con 0x7fd9881014f0 2023-11-16T16:38:45.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.272+0000 7fd96ffff700 1 -- 172.21.15.1:0/2411021681 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fd97800f450 con 0x7fd9881014f0 2023-11-16T16:38:45.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.273+0000 7fd96ffff700 1 --2- 172.21.15.1:0/2411021681 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd9700a1150 0x7fd9700a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:45.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.273+0000 7fd986ffd700 1 --2- 172.21.15.1:0/2411021681 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd9700a1150 0x7fd9700a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:45.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.273+0000 7fd96ffff700 1 -- 172.21.15.1:0/2411021681 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd9780ccba0 con 0x7fd9881014f0 2023-11-16T16:38:45.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.274+0000 7fd986ffd700 1 --2- 172.21.15.1:0/2411021681 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd9700a1150 0x7fd9700a3600 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd97c00b5c0 tx=0x7fd97c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:45.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.277+0000 7fd96ffff700 1 -- 172.21.15.1:0/2411021681 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd978096380 con 0x7fd9881014f0 2023-11-16T16:38:45.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:45 smithi001 ceph-mon[98800]: pgmap v307: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2023-11-16T16:38:45.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:45 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/18414989' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:38:45.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:45 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/2537911593' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:38:45.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:45 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3492139125' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:38:45.478 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.476+0000 7fd98d658700 1 -- 172.21.15.1:0/2411021681 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd988061700 con 0x7fd9881014f0 2023-11-16T16:38:45.479 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.476+0000 7fd96ffff700 1 -- 172.21.15.1:0/2411021681 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd9780999a0 con 0x7fd9881014f0 2023-11-16T16:38:45.479 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2023-11-16T16:38:45.480 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.478+0000 7fd98d658700 1 -- 172.21.15.1:0/2411021681 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd9700a1150 msgr2=0x7fd9700a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:45.480 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.478+0000 7fd98d658700 1 --2- 172.21.15.1:0/2411021681 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd9700a1150 0x7fd9700a3600 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd97c00b5c0 tx=0x7fd97c005fb0 comp rx=0 tx=0).stop 2023-11-16T16:38:45.480 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.478+0000 7fd98d658700 1 -- 172.21.15.1:0/2411021681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9881014f0 msgr2=0x7fd988072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:45.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.478+0000 7fd98d658700 1 --2- 172.21.15.1:0/2411021681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9881014f0 0x7fd988072ee0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd97800c9b0 tx=0x7fd97800ccc0 comp rx=0 tx=0).stop 2023-11-16T16:38:45.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.478+0000 7fd98d658700 1 -- 172.21.15.1:0/2411021681 shutdown_connections 2023-11-16T16:38:45.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.478+0000 7fd98d658700 1 --2- 172.21.15.1:0/2411021681 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd9700a1150 0x7fd9700a3600 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:45.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.478+0000 7fd98d658700 1 --2- 172.21.15.1:0/2411021681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd988101e80 0x7fd988073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:45.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.478+0000 7fd98d658700 1 --2- 172.21.15.1:0/2411021681 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd988100300 0x7fd9880748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:45.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.478+0000 7fd98d658700 1 --2- 172.21.15.1:0/2411021681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9881014f0 0x7fd988072ee0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:45.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.478+0000 7fd98d658700 1 -- 172.21.15.1:0/2411021681 >> 172.21.15.1:0/2411021681 conn(0x7fd9880fb8e0 msgr2=0x7fd988104a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:45.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.479+0000 7fd98d658700 1 -- 172.21.15.1:0/2411021681 shutdown_connections 2023-11-16T16:38:45.482 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.479+0000 7fd98d658700 1 -- 172.21.15.1:0/2411021681 wait complete. 2023-11-16T16:38:45.492 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:38:45.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:45 smithi026 ceph-mon[100112]: pgmap v307: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2023-11-16T16:38:45.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:45 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/18414989' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:38:45.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:45 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/2537911593' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:38:45.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:45 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/3492139125' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:38:45.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:45 smithi053 ceph-mon[100638]: pgmap v307: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2023-11-16T16:38:45.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:45 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/18414989' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:38:45.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:45 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/2537911593' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:38:45.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:45 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/3492139125' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:38:45.617 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.616+0000 7fa725857700 1 -- 172.21.15.1:0/2474091461 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7201036e0 msgr2=0x7fa720105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:45.617 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.616+0000 7fa725857700 1 --2- 172.21.15.1:0/2474091461 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7201036e0 0x7fa720105ac0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa710009990 tx=0x7fa710009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:45.617 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.616+0000 7fa725857700 1 -- 172.21.15.1:0/2474091461 shutdown_connections 2023-11-16T16:38:45.618 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.616+0000 7fa725857700 1 --2- 172.21.15.1:0/2474091461 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa720106000 0x7fa7201083e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:45.618 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.616+0000 7fa725857700 1 --2- 172.21.15.1:0/2474091461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa720100dc0 0x7fa7201031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:45.618 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.616+0000 7fa725857700 1 --2- 172.21.15.1:0/2474091461 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7201036e0 0x7fa720105ac0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:45.618 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.616+0000 7fa725857700 1 -- 172.21.15.1:0/2474091461 >> 172.21.15.1:0/2474091461 conn(0x7fa7200fa7c0 msgr2=0x7fa7200fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:45.618 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.616+0000 7fa725857700 1 -- 172.21.15.1:0/2474091461 shutdown_connections 2023-11-16T16:38:45.618 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.616+0000 7fa725857700 1 -- 172.21.15.1:0/2474091461 wait complete. 2023-11-16T16:38:45.620 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.617+0000 7fa725857700 1 Processor -- start 2023-11-16T16:38:45.620 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.617+0000 7fa725857700 1 -- start start 2023-11-16T16:38:45.620 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.617+0000 7fa725857700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa720100dc0 0x7fa720196010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:45.620 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.617+0000 7fa725857700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7201036e0 0x7fa720196550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:45.620 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.618+0000 7fa725857700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa720106000 0x7fa72019d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:45.620 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.618+0000 7fa725857700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72019db90 con 0x7fa720100dc0 2023-11-16T16:38:45.620 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.618+0000 7fa725857700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72019dcd0 con 0x7fa7201036e0 2023-11-16T16:38:45.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.618+0000 7fa725857700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72019de60 con 0x7fa720106000 2023-11-16T16:38:45.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.618+0000 7fa71effd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa720100dc0 0x7fa720196010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:45.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.618+0000 7fa71effd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa720100dc0 0x7fa720196010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42592/0 (socket says 172.21.15.1:42592) 2023-11-16T16:38:45.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.618+0000 7fa71effd700 1 -- 172.21.15.1:0/2185440086 learned_addr learned my addr 172.21.15.1:0/2185440086 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:45.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.618+0000 7fa71e7fc700 1 --2- 172.21.15.1:0/2185440086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7201036e0 0x7fa720196550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:45.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.618+0000 7fa71effd700 1 -- 172.21.15.1:0/2185440086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa720106000 msgr2=0x7fa72019d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:45.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.618+0000 7fa71effd700 1 --2- 172.21.15.1:0/2185440086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa720106000 0x7fa72019d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:45.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.618+0000 7fa71effd700 1 -- 172.21.15.1:0/2185440086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7201036e0 msgr2=0x7fa720196550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:45.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.618+0000 7fa71effd700 1 --2- 172.21.15.1:0/2185440086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7201036e0 0x7fa720196550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:45.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.618+0000 7fa71effd700 1 -- 172.21.15.1:0/2185440086 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7100095f0 con 0x7fa720100dc0 2023-11-16T16:38:45.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.618+0000 7fa71effd700 1 --2- 172.21.15.1:0/2185440086 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa720100dc0 0x7fa720196010 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa71400d950 tx=0x7fa71400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:45.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.618+0000 7fa724855700 1 -- 172.21.15.1:0/2185440086 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa714009b40 con 0x7fa720100dc0 2023-11-16T16:38:45.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.618+0000 7fa725857700 1 -- 172.21.15.1:0/2185440086 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa72010bcd0 con 0x7fa720100dc0 2023-11-16T16:38:45.622 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.618+0000 7fa724855700 1 -- 172.21.15.1:0/2185440086 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa714009ca0 con 0x7fa720100dc0 2023-11-16T16:38:45.622 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.618+0000 7fa724855700 1 -- 172.21.15.1:0/2185440086 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71400f620 con 0x7fa720100dc0 2023-11-16T16:38:45.622 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.618+0000 7fa725857700 1 -- 172.21.15.1:0/2185440086 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa72010bb40 con 0x7fa720100dc0 2023-11-16T16:38:45.622 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.619+0000 7fa725857700 1 -- 172.21.15.1:0/2185440086 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7200fc330 con 0x7fa720100dc0 2023-11-16T16:38:45.622 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.620+0000 7fa724855700 1 -- 172.21.15.1:0/2185440086 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fa714010460 con 0x7fa720100dc0 2023-11-16T16:38:45.623 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.621+0000 7fa724855700 1 --2- 172.21.15.1:0/2185440086 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fa7080a1150 0x7fa7080a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:45.623 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.621+0000 7fa71e7fc700 1 --2- 172.21.15.1:0/2185440086 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fa7080a1150 0x7fa7080a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:45.623 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.621+0000 7fa724855700 1 -- 172.21.15.1:0/2185440086 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa7140ccc90 con 0x7fa720100dc0 2023-11-16T16:38:45.623 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.621+0000 7fa71e7fc700 1 --2- 172.21.15.1:0/2185440086 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fa7080a1150 0x7fa7080a3600 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa71000b5c0 tx=0x7fa710005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:45.626 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.625+0000 7fa724855700 1 -- 172.21.15.1:0/2185440086 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa7140963e0 con 0x7fa720100dc0 2023-11-16T16:38:45.871 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.871+0000 7fa725857700 1 -- 172.21.15.1:0/2185440086 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa72002cfe0 con 0x7fa720100dc0 2023-11-16T16:38:45.872 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.871+0000 7fa724855700 1 -- 172.21.15.1:0/2185440086 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10774 (secure 0 0 0) 0x7fa714020090 con 0x7fa720100dc0 2023-11-16T16:38:45.872 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:38:45.872 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2587563684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2587563684},{"type":"v1","addr":"172.21.15.1:6835","nonce":2587563684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/277936168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":277936168},{"type":"v1","addr":"172.21.15.1:6837","nonce":277936168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/1172575045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1172575045},{"type":"v1","addr":"172.21.15.1:6839","nonce":1172575045}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/2631175899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2631175899},{"type":"v1","addr":"172.21.15.1:6841","nonce":2631175899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3693905267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3693905267},{"type":"v1","addr":"172.21.15.53:6833","nonce":3693905267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/164668033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":164668033},{"type":"v1","addr":"172.21.15.26:6833","nonce":164668033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-16T16:37:13.918943+0000","modified":"2023-11-16T16:37:19.292898+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2140542192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2140542192},{"type":"v1","addr":"172.21.15.53:6835","nonce":2140542192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1010760275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1010760275},{"type":"v1","addr":"172.21.15.26:6835","nonce":1010760275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3197060302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3197060302},{"type":"v1","addr":"172.21.15.53:6837","nonce":3197060302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6837/1453504863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1453504863},{"type":"v1","addr":"172.21.15.26:6837","nonce":1453504863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.53:6839/1044753024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1044753024},{"type":"v1","addr":"172.21.15.53:6839","nonce":1044753024}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6839/2192205949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2192205949},{"type":"v1","addr":"172.21.15.26:6839","nonce":2192205949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-16T16:38:45.874 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.873+0000 7fa725857700 1 -- 172.21.15.1:0/2185440086 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fa7080a1150 msgr2=0x7fa7080a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:45.875 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.873+0000 7fa725857700 1 --2- 172.21.15.1:0/2185440086 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fa7080a1150 0x7fa7080a3600 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa71000b5c0 tx=0x7fa710005fb0 comp rx=0 tx=0).stop 2023-11-16T16:38:45.875 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.874+0000 7fa725857700 1 -- 172.21.15.1:0/2185440086 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa720100dc0 msgr2=0x7fa720196010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:45.875 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.874+0000 7fa725857700 1 --2- 172.21.15.1:0/2185440086 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa720100dc0 0x7fa720196010 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa71400d950 tx=0x7fa71400dd10 comp rx=0 tx=0).stop 2023-11-16T16:38:45.875 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.874+0000 7fa725857700 1 -- 172.21.15.1:0/2185440086 shutdown_connections 2023-11-16T16:38:45.875 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.874+0000 7fa725857700 1 --2- 172.21.15.1:0/2185440086 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fa7080a1150 0x7fa7080a3600 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:45.875 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.874+0000 7fa725857700 1 --2- 172.21.15.1:0/2185440086 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa720106000 0x7fa72019d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:45.875 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.874+0000 7fa725857700 1 --2- 172.21.15.1:0/2185440086 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa720100dc0 0x7fa720196010 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:45.875 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.874+0000 7fa725857700 1 --2- 172.21.15.1:0/2185440086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7201036e0 0x7fa720196550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:45.875 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.874+0000 7fa725857700 1 -- 172.21.15.1:0/2185440086 >> 172.21.15.1:0/2185440086 conn(0x7fa7200fa7c0 msgr2=0x7fa720101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:45.875 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.874+0000 7fa725857700 1 -- 172.21.15.1:0/2185440086 shutdown_connections 2023-11-16T16:38:45.875 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:45.874+0000 7fa725857700 1 -- 172.21.15.1:0/2185440086 wait complete. 2023-11-16T16:38:45.876 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 23 2023-11-16T16:38:45.887 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-16T16:38:46.015 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.011+0000 7f6c24337700 1 -- 172.21.15.1:0/2111394781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c1c0ff1c0 msgr2=0x7f6c1c0ff5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:46.015 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.011+0000 7f6c24337700 1 --2- 172.21.15.1:0/2111394781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c1c0ff1c0 0x7f6c1c0ff5d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6c18009990 tx=0x7f6c18009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:46.015 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.011+0000 7f6c24337700 1 -- 172.21.15.1:0/2111394781 shutdown_connections 2023-11-16T16:38:46.015 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.011+0000 7f6c24337700 1 --2- 172.21.15.1:0/2111394781 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c1c100d40 0x7f6c1c107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:46.015 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.011+0000 7f6c24337700 1 --2- 172.21.15.1:0/2111394781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c1c0ff1c0 0x7f6c1c0ff5d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:46.015 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.011+0000 7f6c24337700 1 --2- 172.21.15.1:0/2111394781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c1c1003b0 0x7f6c1c100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:46.015 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.011+0000 7f6c24337700 1 -- 172.21.15.1:0/2111394781 >> 172.21.15.1:0/2111394781 conn(0x7f6c1c0fa7c0 msgr2=0x7f6c1c0fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:46.015 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.011+0000 7f6c24337700 1 -- 172.21.15.1:0/2111394781 shutdown_connections 2023-11-16T16:38:46.016 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.011+0000 7f6c24337700 1 -- 172.21.15.1:0/2111394781 wait complete. 2023-11-16T16:38:46.016 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.012+0000 7f6c24337700 1 Processor -- start 2023-11-16T16:38:46.016 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.012+0000 7f6c24337700 1 -- start start 2023-11-16T16:38:46.016 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.012+0000 7f6c24337700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c1c0ff1c0 0x7f6c1c194fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:46.016 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.012+0000 7f6c24337700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c1c1003b0 0x7f6c1c1954f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:46.016 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.012+0000 7f6c24337700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c1c100d40 0x7f6c1c19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:46.016 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.012+0000 7f6c24337700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c19cb30 con 0x7f6c1c1003b0 2023-11-16T16:38:46.016 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.012+0000 7f6c24337700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c19cc70 con 0x7f6c1c0ff1c0 2023-11-16T16:38:46.016 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.012+0000 7f6c24337700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c19ce00 con 0x7f6c1c100d40 2023-11-16T16:38:46.016 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.012+0000 7f6c218d2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c1c1003b0 0x7f6c1c1954f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:46.016 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.012+0000 7f6c218d2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c1c1003b0 0x7f6c1c1954f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42608/0 (socket says 172.21.15.1:42608) 2023-11-16T16:38:46.016 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.012+0000 7f6c218d2700 1 -- 172.21.15.1:0/1098654048 learned_addr learned my addr 172.21.15.1:0/1098654048 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:46.016 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.012+0000 7f6c220d3700 1 --2- 172.21.15.1:0/1098654048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c1c0ff1c0 0x7f6c1c194fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:46.017 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.012+0000 7f6c228d4700 1 --2- 172.21.15.1:0/1098654048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c1c100d40 0x7f6c1c19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:46.017 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.013+0000 7f6c218d2700 1 -- 172.21.15.1:0/1098654048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c1c100d40 msgr2=0x7f6c1c19c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:46.017 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.013+0000 7f6c218d2700 1 --2- 172.21.15.1:0/1098654048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c1c100d40 0x7f6c1c19c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:46.017 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.013+0000 7f6c218d2700 1 -- 172.21.15.1:0/1098654048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c1c0ff1c0 msgr2=0x7f6c1c194fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:46.017 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.013+0000 7f6c218d2700 1 --2- 172.21.15.1:0/1098654048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c1c0ff1c0 0x7f6c1c194fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:46.017 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.013+0000 7f6c218d2700 1 -- 172.21.15.1:0/1098654048 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c180095f0 con 0x7f6c1c1003b0 2023-11-16T16:38:46.017 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.013+0000 7f6c218d2700 1 --2- 172.21.15.1:0/1098654048 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c1c1003b0 0x7f6c1c1954f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6c1000c920 tx=0x7f6c1000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:46.017 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.013+0000 7f6c0f7fe700 1 -- 172.21.15.1:0/1098654048 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c10004a30 con 0x7f6c1c1003b0 2023-11-16T16:38:46.017 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.013+0000 7f6c0f7fe700 1 -- 172.21.15.1:0/1098654048 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c100054e0 con 0x7f6c1c1003b0 2023-11-16T16:38:46.017 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.013+0000 7f6c24337700 1 -- 172.21.15.1:0/1098654048 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c1c10ab20 con 0x7f6c1c1003b0 2023-11-16T16:38:46.017 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.013+0000 7f6c0f7fe700 1 -- 172.21.15.1:0/1098654048 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c10007c10 con 0x7f6c1c1003b0 2023-11-16T16:38:46.017 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.013+0000 7f6c24337700 1 -- 172.21.15.1:0/1098654048 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c1c10a990 con 0x7f6c1c1003b0 2023-11-16T16:38:46.017 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.014+0000 7f6c24337700 1 -- 172.21.15.1:0/1098654048 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c1c19d550 con 0x7f6c1c1003b0 2023-11-16T16:38:46.018 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.015+0000 7f6c0f7fe700 1 -- 172.21.15.1:0/1098654048 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f6c10005000 con 0x7f6c1c1003b0 2023-11-16T16:38:46.018 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.015+0000 7f6c0f7fe700 1 --2- 172.21.15.1:0/1098654048 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f6c080a1150 0x7f6c080a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:46.018 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.016+0000 7f6c220d3700 1 --2- 172.21.15.1:0/1098654048 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f6c080a1150 0x7f6c080a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:46.018 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.016+0000 7f6c0f7fe700 1 -- 172.21.15.1:0/1098654048 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6c100cc7c0 con 0x7f6c1c1003b0 2023-11-16T16:38:46.018 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.016+0000 7f6c220d3700 1 --2- 172.21.15.1:0/1098654048 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f6c080a1150 0x7f6c080a3600 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6c1800b5c0 tx=0x7f6c18005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:46.020 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.020+0000 7f6c0f7fe700 1 -- 172.21.15.1:0/1098654048 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6c10095f10 con 0x7f6c1c1003b0 2023-11-16T16:38:46.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.221+0000 7f6c24337700 1 -- 172.21.15.1:0/1098654048 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c1c19dbf0 con 0x7f6c1c1003b0 2023-11-16T16:38:46.225 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.224+0000 7f6c0f7fe700 1 -- 172.21.15.1:0/1098654048 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22469 (secure 0 0 0) 0x7f6c1001d090 con 0x7f6c1c1003b0 2023-11-16T16:38:46.225 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:38:46.225 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":86,"fsid":"35d9b010-849d-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:30:06.217778+0000","modified":"2023-11-16T16:38:03.179244+0000","last_up_change":"2023-11-16T16:36:31.735403+0000","last_in_change":"2023-11-16T16:36:14.083021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:33:15.916723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-16T16:37:12.158031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-16T16:37:13.272343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a8480507-5275-4e52-953b-626110beb436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6803","nonce":2469671609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6805","nonce":2469671609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6809","nonce":2469671609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6807","nonce":2469671609}]},"public_addr":"172.21.15.1:6803/2469671609","cluster_addr":"172.21.15.1:6805/2469671609","heartbeat_back_addr":"172.21.15.1:6809/2469671609","heartbeat_front_addr":"172.21.15.1:6807/2469671609","state":["exists","up"]},{"osd":1,"uuid":"1d011db2-99d1-4575-bdc5-51f88a308fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6801","nonce":247994796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6803","nonce":247994796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6807","nonce":247994796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6805","nonce":247994796}]},"public_addr":"172.21.15.26:6801/247994796","cluster_addr":"172.21.15.26:6803/247994796","heartbeat_back_addr":"172.21.15.26:6807/247994796","heartbeat_front_addr":"172.21.15.26:6805/247994796","state":["exists","up"]},{"osd":2,"uuid":"4d7a2bd1-3c8e-4546-a243-f0354473b031","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6801","nonce":1659316290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6803","nonce":1659316290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6807","nonce":1659316290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6805","nonce":1659316290}]},"public_addr":"172.21.15.53:6801/1659316290","cluster_addr":"172.21.15.53:6803/1659316290","heartbeat_back_addr":"172.21.15.53:6807/1659316290","heartbeat_front_addr":"172.21.15.53:6805/1659316290","state":["exists","up"]},{"osd":3,"uuid":"9c290fd9-e121-4a81-9687-b0eb93debd86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6811","nonce":2728987281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6813","nonce":2728987281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6817","nonce":2728987281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6815","nonce":2728987281}]},"public_addr":"172.21.15.1:6811/2728987281","cluster_addr":"172.21.15.1:6813/2728987281","heartbeat_back_addr":"172.21.15.1:6817/2728987281","heartbeat_front_addr":"172.21.15.1:6815/2728987281","state":["exists","up"]},{"osd":4,"uuid":"533226bc-cc57-4f15-9e29-820294d57e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6809","nonce":2954740024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6811","nonce":2954740024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6815","nonce":2954740024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6813","nonce":2954740024}]},"public_addr":"172.21.15.26:6809/2954740024","cluster_addr":"172.21.15.26:6811/2954740024","heartbeat_back_addr":"172.21.15.26:6815/2954740024","heartbeat_front_addr":"172.21.15.26:6813/2954740024","state":["exists","up"]},{"osd":5,"uuid":"d3ab25eb-e1d4-4557-bc89-efc9072401ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6809","nonce":3966666318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6811","nonce":3966666318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6815","nonce":3966666318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6813","nonce":3966666318}]},"public_addr":"172.21.15.53:6809/3966666318","cluster_addr":"172.21.15.53:6811/3966666318","heartbeat_back_addr":"172.21.15.53:6815/3966666318","heartbeat_front_addr":"172.21.15.53:6813/3966666318","state":["exists","up"]},{"osd":6,"uuid":"180f506e-afb8-4043-b0b4-2dc86f6d4bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6819","nonce":3364032646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6821","nonce":3364032646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6825","nonce":3364032646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6823","nonce":3364032646}]},"public_addr":"172.21.15.1:6819/3364032646","cluster_addr":"172.21.15.1:6821/3364032646","heartbeat_back_addr":"172.21.15.1:6825/3364032646","heartbeat_front_addr":"172.21.15.1:6823/3364032646","state":["exists","up"]},{"osd":7,"uuid":"0d837bac-194f-498d-b7fb-b3147e992329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6817","nonce":3766901391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6819","nonce":3766901391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6823","nonce":3766901391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6821","nonce":3766901391}]},"public_addr":"172.21.15.26:6817/3766901391","cluster_addr":"172.21.15.26:6819/3766901391","heartbeat_back_addr":"172.21.15.26:6823/3766901391","heartbeat_front_addr":"172.21.15.26:6821/3766901391","state":["exists","up"]},{"osd":8,"uuid":"933f231a-7454-47b0-9291-f1fa64f0baec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6817","nonce":3224335871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6819","nonce":3224335871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6823","nonce":3224335871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6821","nonce":3224335871}]},"public_addr":"172.21.15.53:6817/3224335871","cluster_addr":"172.21.15.53:6819/3224335871","heartbeat_back_addr":"172.21.15.53:6823/3224335871","heartbeat_front_addr":"172.21.15.53:6821/3224335871","state":["exists","up"]},{"osd":9,"uuid":"245fcbb3-c651-4703-ab80-309dc98bd983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6827","nonce":1391199844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6829","nonce":1391199844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6833","nonce":1391199844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6831","nonce":1391199844}]},"public_addr":"172.21.15.1:6827/1391199844","cluster_addr":"172.21.15.1:6829/1391199844","heartbeat_back_addr":"172.21.15.1:6833/1391199844","heartbeat_front_addr":"172.21.15.1:6831/1391199844","state":["exists","up"]},{"osd":10,"uuid":"2c090e7a-5240-4d52-ae4f-e94fe05383d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6825","nonce":1773545447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6827","nonce":1773545447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6831","nonce":1773545447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6829","nonce":1773545447}]},"public_addr":"172.21.15.26:6825/1773545447","cluster_addr":"172.21.15.26:6827/1773545447","heartbeat_back_addr":"172.21.15.26:6831/1773545447","heartbeat_front_addr":"172.21.15.26:6829/1773545447","state":["exists","up"]},{"osd":11,"uuid":"4511e32d-e469-45e1-8310-0ff400cb7b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6825","nonce":2564669546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6827","nonce":2564669546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6831","nonce":2564669546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6829","nonce":2564669546}]},"public_addr":"172.21.15.53:6825/2564669546","cluster_addr":"172.21.15.53:6827/2564669546","heartbeat_back_addr":"172.21.15.53:6831/2564669546","heartbeat_front_addr":"172.21.15.53:6829/2564669546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:28.140250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:49.885461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:12.521487+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:34.926197+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:56.815594+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:18.387751+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:40.587106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:02.319370+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:23.832298+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:46.338040+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:08.730414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:30.050070+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1494336948":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6801/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6800/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6800/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/1641976669":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/2895923638":"2023-11-17T16:30:35.064289+0000","172.21.15.1:6801/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/78811711":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/4131957708":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/344403698":"2023-11-17T16:30:35.064289+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:38:46.227 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.226+0000 7f6c24337700 1 -- 172.21.15.1:0/1098654048 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f6c080a1150 msgr2=0x7f6c080a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:46.227 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.226+0000 7f6c24337700 1 --2- 172.21.15.1:0/1098654048 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f6c080a1150 0x7f6c080a3600 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6c1800b5c0 tx=0x7f6c18005fb0 comp rx=0 tx=0).stop 2023-11-16T16:38:46.227 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.226+0000 7f6c24337700 1 -- 172.21.15.1:0/1098654048 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c1c1003b0 msgr2=0x7f6c1c1954f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:46.227 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.226+0000 7f6c24337700 1 --2- 172.21.15.1:0/1098654048 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c1c1003b0 0x7f6c1c1954f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6c1000c920 tx=0x7f6c1000cc30 comp rx=0 tx=0).stop 2023-11-16T16:38:46.227 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.226+0000 7f6c24337700 1 -- 172.21.15.1:0/1098654048 shutdown_connections 2023-11-16T16:38:46.227 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.226+0000 7f6c24337700 1 --2- 172.21.15.1:0/1098654048 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f6c080a1150 0x7f6c080a3600 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:46.228 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.226+0000 7f6c24337700 1 --2- 172.21.15.1:0/1098654048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6c1c100d40 0x7f6c1c19c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:46.228 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.226+0000 7f6c24337700 1 --2- 172.21.15.1:0/1098654048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c1c0ff1c0 0x7f6c1c194fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:46.228 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.226+0000 7f6c24337700 1 --2- 172.21.15.1:0/1098654048 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c1c1003b0 0x7f6c1c1954f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:46.228 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.226+0000 7f6c24337700 1 -- 172.21.15.1:0/1098654048 >> 172.21.15.1:0/1098654048 conn(0x7f6c1c0fa7c0 msgr2=0x7f6c1c0fcad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:46.228 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.226+0000 7f6c24337700 1 -- 172.21.15.1:0/1098654048 shutdown_connections 2023-11-16T16:38:46.228 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.226+0000 7f6c24337700 1 -- 172.21.15.1:0/1098654048 wait complete. 2023-11-16T16:38:46.239 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-16T16:33:15.916723+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-16T16:37:12.158031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-16T16:37:13.272343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-16T16:38:46.240 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-16T16:38:46.369 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.367+0000 7f9831e58700 1 -- 172.21.15.1:0/459563606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f982c100db0 msgr2=0x7f982c107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:46.369 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.367+0000 7f9831e58700 1 --2- 172.21.15.1:0/459563606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f982c100db0 0x7f982c107650 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f982400b0a0 tx=0x7f982400b3b0 comp rx=0 tx=0).stop 2023-11-16T16:38:46.369 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.368+0000 7f9831e58700 1 -- 172.21.15.1:0/459563606 shutdown_connections 2023-11-16T16:38:46.369 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.368+0000 7f9831e58700 1 --2- 172.21.15.1:0/459563606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f982c100db0 0x7f982c107650 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:46.369 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.368+0000 7f9831e58700 1 --2- 172.21.15.1:0/459563606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f982c0ff170 0x7f982c0ff580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:46.369 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.368+0000 7f9831e58700 1 --2- 172.21.15.1:0/459563606 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f982c100400 0x7f982c100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:46.369 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.368+0000 7f9831e58700 1 -- 172.21.15.1:0/459563606 >> 172.21.15.1:0/459563606 conn(0x7f982c0fa750 msgr2=0x7f982c0fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:46.369 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.368+0000 7f9831e58700 1 -- 172.21.15.1:0/459563606 shutdown_connections 2023-11-16T16:38:46.369 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.368+0000 7f9831e58700 1 -- 172.21.15.1:0/459563606 wait complete. 2023-11-16T16:38:46.369 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.368+0000 7f9831e58700 1 Processor -- start 2023-11-16T16:38:46.372 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.368+0000 7f9831e58700 1 -- start start 2023-11-16T16:38:46.372 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.369+0000 7f9831e58700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f982c0ff170 0x7f982c196080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:46.372 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.369+0000 7f9831e58700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f982c100400 0x7f982c1965c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:46.372 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.369+0000 7f9831e58700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f982c100db0 0x7f982c19d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:46.372 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.369+0000 7f9831e58700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f982c19dc00 con 0x7f982c0ff170 2023-11-16T16:38:46.372 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.369+0000 7f9831e58700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f982c19dd40 con 0x7f982c100db0 2023-11-16T16:38:46.372 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.369+0000 7f9831e58700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f982c19ded0 con 0x7f982c100400 2023-11-16T16:38:46.372 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.369+0000 7f982b7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f982c0ff170 0x7f982c196080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:46.372 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.369+0000 7f982b7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f982c0ff170 0x7f982c196080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42620/0 (socket says 172.21.15.1:42620) 2023-11-16T16:38:46.372 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.369+0000 7f982b7fe700 1 -- 172.21.15.1:0/3871300337 learned_addr learned my addr 172.21.15.1:0/3871300337 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:46.373 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.369+0000 7f982affd700 1 --2- 172.21.15.1:0/3871300337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f982c100400 0x7f982c1965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:46.373 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.369+0000 7f982bfff700 1 --2- 172.21.15.1:0/3871300337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f982c100db0 0x7f982c19d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:46.373 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.369+0000 7f982b7fe700 1 -- 172.21.15.1:0/3871300337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f982c100400 msgr2=0x7f982c1965c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:46.373 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.369+0000 7f982b7fe700 1 --2- 172.21.15.1:0/3871300337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f982c100400 0x7f982c1965c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:46.373 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.369+0000 7f982b7fe700 1 -- 172.21.15.1:0/3871300337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f982c100db0 msgr2=0x7f982c19d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:46.373 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.369+0000 7f982b7fe700 1 --2- 172.21.15.1:0/3871300337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f982c100db0 0x7f982c19d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:46.373 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.369+0000 7f982b7fe700 1 -- 172.21.15.1:0/3871300337 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9824009d00 con 0x7f982c0ff170 2023-11-16T16:38:46.373 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.369+0000 7f982bfff700 1 --2- 172.21.15.1:0/3871300337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f982c100db0 0x7f982c19d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:38:46.373 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.369+0000 7f982b7fe700 1 --2- 172.21.15.1:0/3871300337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f982c0ff170 0x7f982c196080 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f981c00b530 tx=0x7f981c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:46.373 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.369+0000 7f9828ff9700 1 -- 172.21.15.1:0/3871300337 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981c011830 con 0x7f982c0ff170 2023-11-16T16:38:46.373 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.369+0000 7f9831e58700 1 -- 172.21.15.1:0/3871300337 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f982c10ab70 con 0x7f982c0ff170 2023-11-16T16:38:46.374 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.369+0000 7f9828ff9700 1 -- 172.21.15.1:0/3871300337 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f981c011e50 con 0x7f982c0ff170 2023-11-16T16:38:46.374 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.369+0000 7f9828ff9700 1 -- 172.21.15.1:0/3871300337 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981c00f2e0 con 0x7f982c0ff170 2023-11-16T16:38:46.374 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.369+0000 7f9831e58700 1 -- 172.21.15.1:0/3871300337 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f982c10a9e0 con 0x7f982c0ff170 2023-11-16T16:38:46.374 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.370+0000 7f9831e58700 1 -- 172.21.15.1:0/3871300337 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f982c067a50 con 0x7f982c0ff170 2023-11-16T16:38:46.374 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.371+0000 7f9828ff9700 1 -- 172.21.15.1:0/3871300337 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f981c010380 con 0x7f982c0ff170 2023-11-16T16:38:46.375 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.372+0000 7f9828ff9700 1 --2- 172.21.15.1:0/3871300337 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f98140a1150 0x7f98140a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:46.375 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.372+0000 7f982affd700 1 --2- 172.21.15.1:0/3871300337 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f98140a1150 0x7f98140a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:46.375 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.372+0000 7f9828ff9700 1 -- 172.21.15.1:0/3871300337 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f981c0cca10 con 0x7f982c0ff170 2023-11-16T16:38:46.375 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.372+0000 7f982affd700 1 --2- 172.21.15.1:0/3871300337 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f98140a1150 0x7f98140a3600 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9820005950 tx=0x7f982000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:46.376 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.375+0000 7f9828ff9700 1 -- 172.21.15.1:0/3871300337 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f981c096160 con 0x7f982c0ff170 2023-11-16T16:38:46.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:46 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/2411021681' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:38:46.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:46 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/2185440086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:38:46.576 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.575+0000 7f9831e58700 1 -- 172.21.15.1:0/3871300337 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f982c19ebb0 con 0x7f982c0ff170 2023-11-16T16:38:46.576 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.576+0000 7f9828ff9700 1 -- 172.21.15.1:0/3871300337 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f981c099780 con 0x7f982c0ff170 2023-11-16T16:38:46.577 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2023-11-16T16:38:46.579 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.578+0000 7f9831e58700 1 -- 172.21.15.1:0/3871300337 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f98140a1150 msgr2=0x7f98140a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:46.579 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.578+0000 7f9831e58700 1 --2- 172.21.15.1:0/3871300337 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f98140a1150 0x7f98140a3600 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9820005950 tx=0x7f982000a300 comp rx=0 tx=0).stop 2023-11-16T16:38:46.579 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.578+0000 7f9831e58700 1 -- 172.21.15.1:0/3871300337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f982c0ff170 msgr2=0x7f982c196080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:46.579 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.578+0000 7f9831e58700 1 --2- 172.21.15.1:0/3871300337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f982c0ff170 0x7f982c196080 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f981c00b530 tx=0x7f981c00b840 comp rx=0 tx=0).stop 2023-11-16T16:38:46.580 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.578+0000 7f9831e58700 1 -- 172.21.15.1:0/3871300337 shutdown_connections 2023-11-16T16:38:46.580 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.578+0000 7f9831e58700 1 --2- 172.21.15.1:0/3871300337 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f98140a1150 0x7f98140a3600 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:46.580 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.578+0000 7f9831e58700 1 --2- 172.21.15.1:0/3871300337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f982c100db0 0x7f982c19d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:46.580 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.578+0000 7f9831e58700 1 --2- 172.21.15.1:0/3871300337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f982c0ff170 0x7f982c196080 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:46.580 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.578+0000 7f9831e58700 1 --2- 172.21.15.1:0/3871300337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f982c100400 0x7f982c1965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:46.580 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.578+0000 7f9831e58700 1 -- 172.21.15.1:0/3871300337 >> 172.21.15.1:0/3871300337 conn(0x7f982c0fa750 msgr2=0x7f982c067dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:46.580 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.578+0000 7f9831e58700 1 -- 172.21.15.1:0/3871300337 shutdown_connections 2023-11-16T16:38:46.580 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.578+0000 7f9831e58700 1 -- 172.21.15.1:0/3871300337 wait complete. 2023-11-16T16:38:46.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:46 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/2411021681' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:38:46.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:46 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/2185440086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:38:46.590 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:46 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/2411021681' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:38:46.590 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:46 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/2185440086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:38:46.591 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-16T16:38:46.722 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.719+0000 7f07fb445700 1 -- 172.21.15.1:0/3149666701 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07f40ff170 msgr2=0x7f07f40ff580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:46.722 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.719+0000 7f07fb445700 1 --2- 172.21.15.1:0/3149666701 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07f40ff170 0x7f07f40ff580 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f07e8009990 tx=0x7f07e8009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:46.723 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.719+0000 7f07fb445700 1 -- 172.21.15.1:0/3149666701 shutdown_connections 2023-11-16T16:38:46.723 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.719+0000 7f07fb445700 1 --2- 172.21.15.1:0/3149666701 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07f4100db0 0x7f07f4107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:46.723 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.719+0000 7f07fb445700 1 --2- 172.21.15.1:0/3149666701 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07f40ff170 0x7f07f40ff580 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:46.723 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.719+0000 7f07fb445700 1 --2- 172.21.15.1:0/3149666701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f4100400 0x7f07f4100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:46.723 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.719+0000 7f07fb445700 1 -- 172.21.15.1:0/3149666701 >> 172.21.15.1:0/3149666701 conn(0x7f07f40fa750 msgr2=0x7f07f40fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:46.723 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.719+0000 7f07fb445700 1 -- 172.21.15.1:0/3149666701 shutdown_connections 2023-11-16T16:38:46.723 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.719+0000 7f07fb445700 1 -- 172.21.15.1:0/3149666701 wait complete. 2023-11-16T16:38:46.723 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.720+0000 7f07fb445700 1 Processor -- start 2023-11-16T16:38:46.723 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.720+0000 7f07fb445700 1 -- start start 2023-11-16T16:38:46.723 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.720+0000 7f07fb445700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07f40ff170 0x7f07f41981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:46.723 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.720+0000 7f07fb445700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07f4100400 0x7f07f41986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:46.723 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.720+0000 7f07fb445700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f4100db0 0x7f07f419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:46.724 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.720+0000 7f07fb445700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f419fd20 con 0x7f07f4100400 2023-11-16T16:38:46.724 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.720+0000 7f07fb445700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f419fe60 con 0x7f07f4100db0 2023-11-16T16:38:46.724 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.720+0000 7f07fb445700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f419fff0 con 0x7f07f40ff170 2023-11-16T16:38:46.724 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.720+0000 7f07f89e0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07f4100400 0x7f07f41986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:46.724 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.720+0000 7f07f89e0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07f4100400 0x7f07f41986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42636/0 (socket says 172.21.15.1:42636) 2023-11-16T16:38:46.724 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.720+0000 7f07f89e0700 1 -- 172.21.15.1:0/224227025 learned_addr learned my addr 172.21.15.1:0/224227025 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:46.724 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.720+0000 7f07f99e2700 1 --2- 172.21.15.1:0/224227025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f4100db0 0x7f07f419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:46.724 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.720+0000 7f07f91e1700 1 --2- 172.21.15.1:0/224227025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07f40ff170 0x7f07f41981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:46.724 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.721+0000 7f07f89e0700 1 -- 172.21.15.1:0/224227025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07f40ff170 msgr2=0x7f07f41981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:46.724 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.721+0000 7f07f89e0700 1 --2- 172.21.15.1:0/224227025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07f40ff170 0x7f07f41981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:46.724 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.721+0000 7f07f89e0700 1 -- 172.21.15.1:0/224227025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f4100db0 msgr2=0x7f07f419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:46.725 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.721+0000 7f07f89e0700 1 --2- 172.21.15.1:0/224227025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f4100db0 0x7f07f419f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:46.725 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.721+0000 7f07f89e0700 1 -- 172.21.15.1:0/224227025 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e80095f0 con 0x7f07f4100400 2023-11-16T16:38:46.725 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.721+0000 7f07f89e0700 1 --2- 172.21.15.1:0/224227025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07f4100400 0x7f07f41986e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f07f000e9a0 tx=0x7f07f000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:46.725 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.721+0000 7f07e67fc700 1 -- 172.21.15.1:0/224227025 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f0009790 con 0x7f07f4100400 2023-11-16T16:38:46.725 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.721+0000 7f07fb445700 1 -- 172.21.15.1:0/224227025 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07f410de60 con 0x7f07f4100400 2023-11-16T16:38:46.725 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.721+0000 7f07e67fc700 1 -- 172.21.15.1:0/224227025 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07f0010430 con 0x7f07f4100400 2023-11-16T16:38:46.725 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.721+0000 7f07e67fc700 1 -- 172.21.15.1:0/224227025 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f0018680 con 0x7f07f4100400 2023-11-16T16:38:46.725 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.721+0000 7f07fb445700 1 -- 172.21.15.1:0/224227025 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07f410dcd0 con 0x7f07f4100400 2023-11-16T16:38:46.725 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.722+0000 7f07fb445700 1 -- 172.21.15.1:0/224227025 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07f4106190 con 0x7f07f4100400 2023-11-16T16:38:46.726 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.723+0000 7f07e67fc700 1 -- 172.21.15.1:0/224227025 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f07f00105a0 con 0x7f07f4100400 2023-11-16T16:38:46.726 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.723+0000 7f07e67fc700 1 --2- 172.21.15.1:0/224227025 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f07e00a1130 0x7f07e00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:46.726 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.724+0000 7f07e67fc700 1 -- 172.21.15.1:0/224227025 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f07f0014070 con 0x7f07f4100400 2023-11-16T16:38:46.726 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.724+0000 7f07f91e1700 1 --2- 172.21.15.1:0/224227025 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f07e00a1130 0x7f07e00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:46.726 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.724+0000 7f07f91e1700 1 --2- 172.21.15.1:0/224227025 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f07e00a1130 0x7f07e00a35e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f07e800b2e0 tx=0x7f07e8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:46.727 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.727+0000 7f07e67fc700 1 -- 172.21.15.1:0/224227025 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f07f0096410 con 0x7f07f4100400 2023-11-16T16:38:46.928 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.926+0000 7f07fb445700 1 -- 172.21.15.1:0/224227025 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f07f4061700 con 0x7f07f4100400 2023-11-16T16:38:46.928 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.926+0000 7f07e67fc700 1 -- 172.21.15.1:0/224227025 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7f07f0099a30 con 0x7f07f4100400 2023-11-16T16:38:46.928 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2023-11-16T16:38:46.931 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.928+0000 7f07fb445700 1 -- 172.21.15.1:0/224227025 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f07e00a1130 msgr2=0x7f07e00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:46.931 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.928+0000 7f07fb445700 1 --2- 172.21.15.1:0/224227025 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f07e00a1130 0x7f07e00a35e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f07e800b2e0 tx=0x7f07e8005fb0 comp rx=0 tx=0).stop 2023-11-16T16:38:46.931 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.928+0000 7f07fb445700 1 -- 172.21.15.1:0/224227025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07f4100400 msgr2=0x7f07f41986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:46.931 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.928+0000 7f07fb445700 1 --2- 172.21.15.1:0/224227025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07f4100400 0x7f07f41986e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f07f000e9a0 tx=0x7f07f000ecb0 comp rx=0 tx=0).stop 2023-11-16T16:38:46.931 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.929+0000 7f07fb445700 1 -- 172.21.15.1:0/224227025 shutdown_connections 2023-11-16T16:38:46.931 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.929+0000 7f07fb445700 1 --2- 172.21.15.1:0/224227025 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f07e00a1130 0x7f07e00a35e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:46.931 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.929+0000 7f07fb445700 1 --2- 172.21.15.1:0/224227025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f4100db0 0x7f07f419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:46.931 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.929+0000 7f07fb445700 1 --2- 172.21.15.1:0/224227025 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07f40ff170 0x7f07f41981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:46.931 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.929+0000 7f07fb445700 1 --2- 172.21.15.1:0/224227025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07f4100400 0x7f07f41986e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:46.932 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.929+0000 7f07fb445700 1 -- 172.21.15.1:0/224227025 >> 172.21.15.1:0/224227025 conn(0x7f07f40fa750 msgr2=0x7f07f40fca60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:46.932 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.929+0000 7f07fb445700 1 -- 172.21.15.1:0/224227025 shutdown_connections 2023-11-16T16:38:46.932 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:46.929+0000 7f07fb445700 1 -- 172.21.15.1:0/224227025 wait complete. 2023-11-16T16:38:46.943 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-16T16:38:47.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.070+0000 7f011d38f700 1 -- 172.21.15.1:0/2779362578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01180ff170 msgr2=0x7f01180ff580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:47.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.070+0000 7f011d38f700 1 --2- 172.21.15.1:0/2779362578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01180ff170 0x7f01180ff580 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f010c009990 tx=0x7f010c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:47.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.070+0000 7f011d38f700 1 -- 172.21.15.1:0/2779362578 shutdown_connections 2023-11-16T16:38:47.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.070+0000 7f011d38f700 1 --2- 172.21.15.1:0/2779362578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0118100db0 0x7f0118107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:47.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.070+0000 7f011d38f700 1 --2- 172.21.15.1:0/2779362578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01180ff170 0x7f01180ff580 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:47.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.070+0000 7f011d38f700 1 --2- 172.21.15.1:0/2779362578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0118100400 0x7f0118100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:47.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.070+0000 7f011d38f700 1 -- 172.21.15.1:0/2779362578 >> 172.21.15.1:0/2779362578 conn(0x7f01180fa750 msgr2=0x7f01180fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:47.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.070+0000 7f011d38f700 1 -- 172.21.15.1:0/2779362578 shutdown_connections 2023-11-16T16:38:47.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.070+0000 7f011d38f700 1 -- 172.21.15.1:0/2779362578 wait complete. 2023-11-16T16:38:47.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.070+0000 7f011d38f700 1 Processor -- start 2023-11-16T16:38:47.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.070+0000 7f011d38f700 1 -- start start 2023-11-16T16:38:47.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.071+0000 7f011d38f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01180ff170 0x7f0118194db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:47.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.071+0000 7f011d38f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0118100400 0x7f01181952f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:47.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.071+0000 7f011d38f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0118100db0 0x7f011819c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:47.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.071+0000 7f011d38f700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011819c8f0 con 0x7f0118100400 2023-11-16T16:38:47.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.071+0000 7f011d38f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011819ca60 con 0x7f0118100db0 2023-11-16T16:38:47.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.071+0000 7f011d38f700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011819cc20 con 0x7f01180ff170 2023-11-16T16:38:47.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.071+0000 7f01167fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0118100400 0x7f01181952f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:47.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.071+0000 7f01167fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0118100400 0x7f01181952f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42650/0 (socket says 172.21.15.1:42650) 2023-11-16T16:38:47.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.071+0000 7f01167fc700 1 -- 172.21.15.1:0/1434591359 learned_addr learned my addr 172.21.15.1:0/1434591359 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:47.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.071+0000 7f01177fe700 1 --2- 172.21.15.1:0/1434591359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0118100db0 0x7f011819c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:47.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.071+0000 7f0116ffd700 1 --2- 172.21.15.1:0/1434591359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01180ff170 0x7f0118194db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:47.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.071+0000 7f01167fc700 1 -- 172.21.15.1:0/1434591359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01180ff170 msgr2=0x7f0118194db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:47.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.071+0000 7f01167fc700 1 --2- 172.21.15.1:0/1434591359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01180ff170 0x7f0118194db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:47.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.071+0000 7f01167fc700 1 -- 172.21.15.1:0/1434591359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0118100db0 msgr2=0x7f011819c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:47.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.071+0000 7f01167fc700 1 --2- 172.21.15.1:0/1434591359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0118100db0 0x7f011819c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:47.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.071+0000 7f01167fc700 1 -- 172.21.15.1:0/1434591359 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f010c0095f0 con 0x7f0118100400 2023-11-16T16:38:47.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.071+0000 7f0116ffd700 1 --2- 172.21.15.1:0/1434591359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01180ff170 0x7f0118194db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:38:47.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.071+0000 7f01167fc700 1 --2- 172.21.15.1:0/1434591359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0118100400 0x7f01181952f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f010800c9b0 tx=0x7f010800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:47.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.071+0000 7f00fffff700 1 -- 172.21.15.1:0/1434591359 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0108007b00 con 0x7f0118100400 2023-11-16T16:38:47.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.071+0000 7f011d38f700 1 -- 172.21.15.1:0/1434591359 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f011810ab60 con 0x7f0118100400 2023-11-16T16:38:47.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.071+0000 7f00fffff700 1 -- 172.21.15.1:0/1434591359 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0108007c60 con 0x7f0118100400 2023-11-16T16:38:47.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.071+0000 7f00fffff700 1 -- 172.21.15.1:0/1434591359 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0108018660 con 0x7f0118100400 2023-11-16T16:38:47.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.071+0000 7f011d38f700 1 -- 172.21.15.1:0/1434591359 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f011810a9d0 con 0x7f0118100400 2023-11-16T16:38:47.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.072+0000 7f011d38f700 1 -- 172.21.15.1:0/1434591359 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0118106140 con 0x7f0118100400 2023-11-16T16:38:47.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.073+0000 7f00fffff700 1 -- 172.21.15.1:0/1434591359 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f010800f450 con 0x7f0118100400 2023-11-16T16:38:47.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.074+0000 7f00fffff700 1 --2- 172.21.15.1:0/1434591359 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f01000a1150 0x7f01000a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:47.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.074+0000 7f00fffff700 1 -- 172.21.15.1:0/1434591359 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f01080ccb30 con 0x7f0118100400 2023-11-16T16:38:47.080 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.074+0000 7f0116ffd700 1 --2- 172.21.15.1:0/1434591359 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f01000a1150 0x7f01000a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:47.080 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.075+0000 7f0116ffd700 1 --2- 172.21.15.1:0/1434591359 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f01000a1150 0x7f01000a3600 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f010c006010 tx=0x7f010c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:47.080 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.077+0000 7f00fffff700 1 -- 172.21.15.1:0/1434591359 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0108096310 con 0x7f0118100400 2023-11-16T16:38:47.283 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.281+0000 7f011d38f700 1 -- 172.21.15.1:0/1434591359 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0118061af0 con 0x7f0118100400 2023-11-16T16:38:47.283 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.281+0000 7f00fffff700 1 -- 172.21.15.1:0/1434591359 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0108099930 con 0x7f0118100400 2023-11-16T16:38:47.283 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2023-11-16T16:38:47.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.283+0000 7f011d38f700 1 -- 172.21.15.1:0/1434591359 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f01000a1150 msgr2=0x7f01000a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:47.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.284+0000 7f011d38f700 1 --2- 172.21.15.1:0/1434591359 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f01000a1150 0x7f01000a3600 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f010c006010 tx=0x7f010c00b540 comp rx=0 tx=0).stop 2023-11-16T16:38:47.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.284+0000 7f011d38f700 1 -- 172.21.15.1:0/1434591359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0118100400 msgr2=0x7f01181952f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:47.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.284+0000 7f011d38f700 1 --2- 172.21.15.1:0/1434591359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0118100400 0x7f01181952f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f010800c9b0 tx=0x7f010800ccc0 comp rx=0 tx=0).stop 2023-11-16T16:38:47.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.284+0000 7f011d38f700 1 -- 172.21.15.1:0/1434591359 shutdown_connections 2023-11-16T16:38:47.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.284+0000 7f011d38f700 1 --2- 172.21.15.1:0/1434591359 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f01000a1150 0x7f01000a3600 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:47.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.284+0000 7f011d38f700 1 --2- 172.21.15.1:0/1434591359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0118100db0 0x7f011819c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:47.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.284+0000 7f011d38f700 1 --2- 172.21.15.1:0/1434591359 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f01180ff170 0x7f0118194db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:47.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.284+0000 7f011d38f700 1 --2- 172.21.15.1:0/1434591359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0118100400 0x7f01181952f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:47.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.284+0000 7f011d38f700 1 -- 172.21.15.1:0/1434591359 >> 172.21.15.1:0/1434591359 conn(0x7f01180fa750 msgr2=0x7f01180fca40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:47.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.284+0000 7f011d38f700 1 -- 172.21.15.1:0/1434591359 shutdown_connections 2023-11-16T16:38:47.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.284+0000 7f011d38f700 1 -- 172.21.15.1:0/1434591359 wait complete. 2023-11-16T16:38:47.299 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:38:47.428 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.423+0000 7f7b4aa9d700 1 -- 172.21.15.1:0/3743447982 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b440ff1c0 msgr2=0x7f7b440ff5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:47.428 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.423+0000 7f7b4aa9d700 1 --2- 172.21.15.1:0/3743447982 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b440ff1c0 0x7f7b440ff5d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7b38009990 tx=0x7f7b38009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:47.428 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.423+0000 7f7b4aa9d700 1 -- 172.21.15.1:0/3743447982 shutdown_connections 2023-11-16T16:38:47.428 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.423+0000 7f7b4aa9d700 1 --2- 172.21.15.1:0/3743447982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b44100d40 0x7f7b44107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:47.428 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.423+0000 7f7b4aa9d700 1 --2- 172.21.15.1:0/3743447982 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b440ff1c0 0x7f7b440ff5d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:47.428 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.423+0000 7f7b4aa9d700 1 --2- 172.21.15.1:0/3743447982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b441003b0 0x7f7b44100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:47.429 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.423+0000 7f7b4aa9d700 1 -- 172.21.15.1:0/3743447982 >> 172.21.15.1:0/3743447982 conn(0x7f7b440fa7c0 msgr2=0x7f7b440fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:47.429 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.423+0000 7f7b4aa9d700 1 -- 172.21.15.1:0/3743447982 shutdown_connections 2023-11-16T16:38:47.429 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.423+0000 7f7b4aa9d700 1 -- 172.21.15.1:0/3743447982 wait complete. 2023-11-16T16:38:47.429 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.423+0000 7f7b4aa9d700 1 Processor -- start 2023-11-16T16:38:47.429 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.423+0000 7f7b4aa9d700 1 -- start start 2023-11-16T16:38:47.429 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.424+0000 7f7b4aa9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b440ff1c0 0x7f7b44103150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:47.429 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.424+0000 7f7b4aa9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b441003b0 0x7f7b44101790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:47.429 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.424+0000 7f7b4aa9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b44100d40 0x7f7b44101cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:47.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.424+0000 7f7b4aa9d700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b44102470 con 0x7f7b44100d40 2023-11-16T16:38:47.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.424+0000 7f7b4aa9d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b441025e0 con 0x7f7b440ff1c0 2023-11-16T16:38:47.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.424+0000 7f7b4aa9d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b441a00f0 con 0x7f7b441003b0 2023-11-16T16:38:47.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.424+0000 7f7b4903a700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b44100d40 0x7f7b44101cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:47.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.424+0000 7f7b4903a700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b44100d40 0x7f7b44101cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42666/0 (socket says 172.21.15.1:42666) 2023-11-16T16:38:47.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.424+0000 7f7b4903a700 1 -- 172.21.15.1:0/3252609596 learned_addr learned my addr 172.21.15.1:0/3252609596 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:47.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.424+0000 7f7b43fff700 1 --2- 172.21.15.1:0/3252609596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b441003b0 0x7f7b44101790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:47.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.424+0000 7f7b48839700 1 --2- 172.21.15.1:0/3252609596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b440ff1c0 0x7f7b44103150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:47.431 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.424+0000 7f7b4903a700 1 -- 172.21.15.1:0/3252609596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b441003b0 msgr2=0x7f7b44101790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:47.431 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.424+0000 7f7b4903a700 1 --2- 172.21.15.1:0/3252609596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b441003b0 0x7f7b44101790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:47.431 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.424+0000 7f7b4903a700 1 -- 172.21.15.1:0/3252609596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b440ff1c0 msgr2=0x7f7b44103150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:47.431 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.424+0000 7f7b4903a700 1 --2- 172.21.15.1:0/3252609596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b440ff1c0 0x7f7b44103150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:47.431 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.424+0000 7f7b4903a700 1 -- 172.21.15.1:0/3252609596 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b380095f0 con 0x7f7b44100d40 2023-11-16T16:38:47.431 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.424+0000 7f7b4903a700 1 --2- 172.21.15.1:0/3252609596 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b44100d40 0x7f7b44101cd0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c007ee0 tx=0x7f7b3c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:47.431 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.424+0000 7f7b41ffb700 1 -- 172.21.15.1:0/3252609596 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c017070 con 0x7f7b44100d40 2023-11-16T16:38:47.431 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.425+0000 7f7b41ffb700 1 -- 172.21.15.1:0/3252609596 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b3c00f040 con 0x7f7b44100d40 2023-11-16T16:38:47.432 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.424+0000 7f7b4aa9d700 1 -- 172.21.15.1:0/3252609596 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b4410ab20 con 0x7f7b44100d40 2023-11-16T16:38:47.432 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.425+0000 7f7b41ffb700 1 -- 172.21.15.1:0/3252609596 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c005550 con 0x7f7b44100d40 2023-11-16T16:38:47.432 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.425+0000 7f7b4aa9d700 1 -- 172.21.15.1:0/3252609596 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b4410a990 con 0x7f7b44100d40 2023-11-16T16:38:47.432 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.426+0000 7f7b4aa9d700 1 -- 172.21.15.1:0/3252609596 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b44067af0 con 0x7f7b44100d40 2023-11-16T16:38:47.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.427+0000 7f7b41ffb700 1 -- 172.21.15.1:0/3252609596 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f7b3c00db20 con 0x7f7b44100d40 2023-11-16T16:38:47.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.427+0000 7f7b41ffb700 1 --2- 172.21.15.1:0/3252609596 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f7b340a1150 0x7f7b340a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:47.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.427+0000 7f7b41ffb700 1 -- 172.21.15.1:0/3252609596 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7b3c0cbff0 con 0x7f7b44100d40 2023-11-16T16:38:47.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.427+0000 7f7b48839700 1 --2- 172.21.15.1:0/3252609596 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f7b340a1150 0x7f7b340a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:47.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.428+0000 7f7b48839700 1 --2- 172.21.15.1:0/3252609596 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f7b340a1150 0x7f7b340a3600 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7b3800b2e0 tx=0x7f7b3800b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:47.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.431+0000 7f7b41ffb700 1 -- 172.21.15.1:0/3252609596 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7b3c0fe730 con 0x7f7b44100d40 2023-11-16T16:38:47.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:47 smithi026 ceph-mon[100112]: pgmap v308: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2023-11-16T16:38:47.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:47 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1098654048' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:38:47.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:47 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/3871300337' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:38:47.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:47 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/224227025' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:38:47.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:47 smithi053 ceph-mon[100638]: pgmap v308: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2023-11-16T16:38:47.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:47 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1098654048' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:38:47.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:47 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/3871300337' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:38:47.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:47 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/224227025' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:38:47.675 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.673+0000 7f7b4aa9d700 1 -- 172.21.15.1:0/3252609596 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b441a0c00 con 0x7f7b44100d40 2023-11-16T16:38:47.676 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.674+0000 7f7b41ffb700 1 -- 172.21.15.1:0/3252609596 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10774 (secure 0 0 0) 0x7f7b3c095740 con 0x7f7b44100d40 2023-11-16T16:38:47.676 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:38:47.676 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2587563684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2587563684},{"type":"v1","addr":"172.21.15.1:6835","nonce":2587563684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/277936168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":277936168},{"type":"v1","addr":"172.21.15.1:6837","nonce":277936168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/1172575045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1172575045},{"type":"v1","addr":"172.21.15.1:6839","nonce":1172575045}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/2631175899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2631175899},{"type":"v1","addr":"172.21.15.1:6841","nonce":2631175899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3693905267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3693905267},{"type":"v1","addr":"172.21.15.53:6833","nonce":3693905267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/164668033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":164668033},{"type":"v1","addr":"172.21.15.26:6833","nonce":164668033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-16T16:37:13.918943+0000","modified":"2023-11-16T16:37:19.292898+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2140542192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2140542192},{"type":"v1","addr":"172.21.15.53:6835","nonce":2140542192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1010760275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1010760275},{"type":"v1","addr":"172.21.15.26:6835","nonce":1010760275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3197060302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3197060302},{"type":"v1","addr":"172.21.15.53:6837","nonce":3197060302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6837/1453504863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1453504863},{"type":"v1","addr":"172.21.15.26:6837","nonce":1453504863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.53:6839/1044753024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1044753024},{"type":"v1","addr":"172.21.15.53:6839","nonce":1044753024}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6839/2192205949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2192205949},{"type":"v1","addr":"172.21.15.26:6839","nonce":2192205949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-16T16:38:47.679 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.676+0000 7f7b4aa9d700 1 -- 172.21.15.1:0/3252609596 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f7b340a1150 msgr2=0x7f7b340a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:47.679 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.676+0000 7f7b4aa9d700 1 --2- 172.21.15.1:0/3252609596 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f7b340a1150 0x7f7b340a3600 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7b3800b2e0 tx=0x7f7b3800b5b0 comp rx=0 tx=0).stop 2023-11-16T16:38:47.679 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.676+0000 7f7b4aa9d700 1 -- 172.21.15.1:0/3252609596 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b44100d40 msgr2=0x7f7b44101cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:47.679 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.676+0000 7f7b4aa9d700 1 --2- 172.21.15.1:0/3252609596 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b44100d40 0x7f7b44101cd0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c007ee0 tx=0x7f7b3c00d3b0 comp rx=0 tx=0).stop 2023-11-16T16:38:47.679 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.676+0000 7f7b4aa9d700 1 -- 172.21.15.1:0/3252609596 shutdown_connections 2023-11-16T16:38:47.679 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.676+0000 7f7b4aa9d700 1 --2- 172.21.15.1:0/3252609596 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f7b340a1150 0x7f7b340a3600 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:47.679 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.676+0000 7f7b4aa9d700 1 --2- 172.21.15.1:0/3252609596 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7b44100d40 0x7f7b44101cd0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:47.680 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.676+0000 7f7b4aa9d700 1 --2- 172.21.15.1:0/3252609596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b440ff1c0 0x7f7b44103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:47.680 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.676+0000 7f7b4aa9d700 1 --2- 172.21.15.1:0/3252609596 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7b441003b0 0x7f7b44101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:47.680 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.676+0000 7f7b4aa9d700 1 -- 172.21.15.1:0/3252609596 >> 172.21.15.1:0/3252609596 conn(0x7f7b440fa7c0 msgr2=0x7f7b44067e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:47.680 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.676+0000 7f7b4aa9d700 1 -- 172.21.15.1:0/3252609596 shutdown_connections 2023-11-16T16:38:47.680 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.676+0000 7f7b4aa9d700 1 -- 172.21.15.1:0/3252609596 wait complete. 2023-11-16T16:38:47.680 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 23 2023-11-16T16:38:47.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:47 smithi001 ceph-mon[98800]: pgmap v308: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2023-11-16T16:38:47.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:47 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1098654048' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:38:47.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:47 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3871300337' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:38:47.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:47 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/224227025' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:38:47.691 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-16T16:38:47.819 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.814+0000 7fbd79721700 1 -- 172.21.15.1:0/927949902 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd740ff430 msgr2=0x7fbd740ff8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:47.820 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.814+0000 7fbd79721700 1 --2- 172.21.15.1:0/927949902 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd740ff430 0x7fbd740ff8a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009990 tx=0x7fbd68009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:47.820 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.814+0000 7fbd79721700 1 -- 172.21.15.1:0/927949902 shutdown_connections 2023-11-16T16:38:47.820 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.814+0000 7fbd79721700 1 --2- 172.21.15.1:0/927949902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd740ffde0 0x7fbd74107690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:47.820 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.814+0000 7fbd79721700 1 --2- 172.21.15.1:0/927949902 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd74100870 0x7fbd74100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:47.820 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.814+0000 7fbd79721700 1 --2- 172.21.15.1:0/927949902 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd740ff430 0x7fbd740ff8a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:47.820 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.814+0000 7fbd79721700 1 -- 172.21.15.1:0/927949902 >> 172.21.15.1:0/927949902 conn(0x7fbd740f9f70 msgr2=0x7fbd740fc3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:47.820 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.814+0000 7fbd79721700 1 -- 172.21.15.1:0/927949902 shutdown_connections 2023-11-16T16:38:47.821 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.814+0000 7fbd79721700 1 -- 172.21.15.1:0/927949902 wait complete. 2023-11-16T16:38:47.821 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.815+0000 7fbd79721700 1 Processor -- start 2023-11-16T16:38:47.821 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.815+0000 7fbd79721700 1 -- start start 2023-11-16T16:38:47.821 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.815+0000 7fbd79721700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd740ff430 0x7fbd740feeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:47.821 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.815+0000 7fbd79721700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd740ffde0 0x7fbd7419adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:47.821 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.815+0000 7fbd79721700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd74100870 0x7fbd7419d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:47.821 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.815+0000 7fbd79721700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7419dbe0 con 0x7fbd74100870 2023-11-16T16:38:47.821 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.815+0000 7fbd79721700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7419dd50 con 0x7fbd740ffde0 2023-11-16T16:38:47.822 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.815+0000 7fbd79721700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7419df10 con 0x7fbd740ff430 2023-11-16T16:38:47.822 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.816+0000 7fbd737fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd74100870 0x7fbd7419d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:47.822 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.816+0000 7fbd72ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd740ff430 0x7fbd740feeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:47.822 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.816+0000 7fbd737fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd74100870 0x7fbd7419d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42680/0 (socket says 172.21.15.1:42680) 2023-11-16T16:38:47.822 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.816+0000 7fbd727fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd740ffde0 0x7fbd7419adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:47.822 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.816+0000 7fbd737fe700 1 -- 172.21.15.1:0/842505632 learned_addr learned my addr 172.21.15.1:0/842505632 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:47.822 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.816+0000 7fbd72ffd700 1 -- 172.21.15.1:0/842505632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd740ffde0 msgr2=0x7fbd7419adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:47.822 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.816+0000 7fbd72ffd700 1 --2- 172.21.15.1:0/842505632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd740ffde0 0x7fbd7419adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:47.822 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.816+0000 7fbd72ffd700 1 -- 172.21.15.1:0/842505632 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd74100870 msgr2=0x7fbd7419d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:47.823 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.816+0000 7fbd72ffd700 1 --2- 172.21.15.1:0/842505632 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd74100870 0x7fbd7419d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:47.823 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.816+0000 7fbd72ffd700 1 -- 172.21.15.1:0/842505632 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd680095f0 con 0x7fbd740ff430 2023-11-16T16:38:47.823 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.816+0000 7fbd737fe700 1 --2- 172.21.15.1:0/842505632 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd74100870 0x7fbd7419d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:38:47.823 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.816+0000 7fbd727fc700 1 --2- 172.21.15.1:0/842505632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd740ffde0 0x7fbd7419adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:38:47.823 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.816+0000 7fbd72ffd700 1 --2- 172.21.15.1:0/842505632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd740ff430 0x7fbd740feeb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbd6400e9f0 tx=0x7fbd6400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:47.823 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.816+0000 7fbd5bfff700 1 -- 172.21.15.1:0/842505632 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd64004220 con 0x7fbd740ff430 2023-11-16T16:38:47.823 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.816+0000 7fbd79721700 1 -- 172.21.15.1:0/842505632 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd7410ac40 con 0x7fbd740ff430 2023-11-16T16:38:47.823 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.817+0000 7fbd5bfff700 1 -- 172.21.15.1:0/842505632 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd64004cf0 con 0x7fbd740ff430 2023-11-16T16:38:47.824 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.817+0000 7fbd79721700 1 -- 172.21.15.1:0/842505632 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd7410aab0 con 0x7fbd740ff430 2023-11-16T16:38:47.824 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.817+0000 7fbd5bfff700 1 -- 172.21.15.1:0/842505632 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6400ce30 con 0x7fbd740ff430 2023-11-16T16:38:47.824 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.817+0000 7fbd79721700 1 -- 172.21.15.1:0/842505632 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd7419e6b0 con 0x7fbd740ff430 2023-11-16T16:38:47.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.819+0000 7fbd5bfff700 1 -- 172.21.15.1:0/842505632 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fbd64004750 con 0x7fbd740ff430 2023-11-16T16:38:47.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.819+0000 7fbd5bfff700 1 --2- 172.21.15.1:0/842505632 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbd5c0a1150 0x7fbd5c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:47.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.820+0000 7fbd5bfff700 1 -- 172.21.15.1:0/842505632 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbd64014070 con 0x7fbd740ff430 2023-11-16T16:38:47.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.820+0000 7fbd727fc700 1 --2- 172.21.15.1:0/842505632 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbd5c0a1150 0x7fbd5c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:47.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.820+0000 7fbd727fc700 1 --2- 172.21.15.1:0/842505632 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbd5c0a1150 0x7fbd5c0a3600 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbd740fec20 tx=0x7fbd68005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:47.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:47.823+0000 7fbd5bfff700 1 -- 172.21.15.1:0/842505632 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbd64096280 con 0x7fbd740ff430 2023-11-16T16:38:48.029 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.028+0000 7fbd79721700 1 -- 172.21.15.1:0/842505632 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd741060e0 con 0x7fbd740ff430 2023-11-16T16:38:48.032 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.030+0000 7fbd5bfff700 1 -- 172.21.15.1:0/842505632 <== mon.1 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22469 (secure 0 0 0) 0x7fbd64078e60 con 0x7fbd740ff430 2023-11-16T16:38:48.032 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:38:48.033 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":86,"fsid":"35d9b010-849d-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:30:06.217778+0000","modified":"2023-11-16T16:38:03.179244+0000","last_up_change":"2023-11-16T16:36:31.735403+0000","last_in_change":"2023-11-16T16:36:14.083021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:33:15.916723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-16T16:37:12.158031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-16T16:37:13.272343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a8480507-5275-4e52-953b-626110beb436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6803","nonce":2469671609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6805","nonce":2469671609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6809","nonce":2469671609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6807","nonce":2469671609}]},"public_addr":"172.21.15.1:6803/2469671609","cluster_addr":"172.21.15.1:6805/2469671609","heartbeat_back_addr":"172.21.15.1:6809/2469671609","heartbeat_front_addr":"172.21.15.1:6807/2469671609","state":["exists","up"]},{"osd":1,"uuid":"1d011db2-99d1-4575-bdc5-51f88a308fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6801","nonce":247994796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6803","nonce":247994796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6807","nonce":247994796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6805","nonce":247994796}]},"public_addr":"172.21.15.26:6801/247994796","cluster_addr":"172.21.15.26:6803/247994796","heartbeat_back_addr":"172.21.15.26:6807/247994796","heartbeat_front_addr":"172.21.15.26:6805/247994796","state":["exists","up"]},{"osd":2,"uuid":"4d7a2bd1-3c8e-4546-a243-f0354473b031","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6801","nonce":1659316290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6803","nonce":1659316290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6807","nonce":1659316290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6805","nonce":1659316290}]},"public_addr":"172.21.15.53:6801/1659316290","cluster_addr":"172.21.15.53:6803/1659316290","heartbeat_back_addr":"172.21.15.53:6807/1659316290","heartbeat_front_addr":"172.21.15.53:6805/1659316290","state":["exists","up"]},{"osd":3,"uuid":"9c290fd9-e121-4a81-9687-b0eb93debd86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6811","nonce":2728987281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6813","nonce":2728987281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6817","nonce":2728987281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6815","nonce":2728987281}]},"public_addr":"172.21.15.1:6811/2728987281","cluster_addr":"172.21.15.1:6813/2728987281","heartbeat_back_addr":"172.21.15.1:6817/2728987281","heartbeat_front_addr":"172.21.15.1:6815/2728987281","state":["exists","up"]},{"osd":4,"uuid":"533226bc-cc57-4f15-9e29-820294d57e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6809","nonce":2954740024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6811","nonce":2954740024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6815","nonce":2954740024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6813","nonce":2954740024}]},"public_addr":"172.21.15.26:6809/2954740024","cluster_addr":"172.21.15.26:6811/2954740024","heartbeat_back_addr":"172.21.15.26:6815/2954740024","heartbeat_front_addr":"172.21.15.26:6813/2954740024","state":["exists","up"]},{"osd":5,"uuid":"d3ab25eb-e1d4-4557-bc89-efc9072401ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6809","nonce":3966666318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6811","nonce":3966666318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6815","nonce":3966666318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6813","nonce":3966666318}]},"public_addr":"172.21.15.53:6809/3966666318","cluster_addr":"172.21.15.53:6811/3966666318","heartbeat_back_addr":"172.21.15.53:6815/3966666318","heartbeat_front_addr":"172.21.15.53:6813/3966666318","state":["exists","up"]},{"osd":6,"uuid":"180f506e-afb8-4043-b0b4-2dc86f6d4bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6819","nonce":3364032646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6821","nonce":3364032646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6825","nonce":3364032646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6823","nonce":3364032646}]},"public_addr":"172.21.15.1:6819/3364032646","cluster_addr":"172.21.15.1:6821/3364032646","heartbeat_back_addr":"172.21.15.1:6825/3364032646","heartbeat_front_addr":"172.21.15.1:6823/3364032646","state":["exists","up"]},{"osd":7,"uuid":"0d837bac-194f-498d-b7fb-b3147e992329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6817","nonce":3766901391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6819","nonce":3766901391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6823","nonce":3766901391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6821","nonce":3766901391}]},"public_addr":"172.21.15.26:6817/3766901391","cluster_addr":"172.21.15.26:6819/3766901391","heartbeat_back_addr":"172.21.15.26:6823/3766901391","heartbeat_front_addr":"172.21.15.26:6821/3766901391","state":["exists","up"]},{"osd":8,"uuid":"933f231a-7454-47b0-9291-f1fa64f0baec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6817","nonce":3224335871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6819","nonce":3224335871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6823","nonce":3224335871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6821","nonce":3224335871}]},"public_addr":"172.21.15.53:6817/3224335871","cluster_addr":"172.21.15.53:6819/3224335871","heartbeat_back_addr":"172.21.15.53:6823/3224335871","heartbeat_front_addr":"172.21.15.53:6821/3224335871","state":["exists","up"]},{"osd":9,"uuid":"245fcbb3-c651-4703-ab80-309dc98bd983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6827","nonce":1391199844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6829","nonce":1391199844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6833","nonce":1391199844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6831","nonce":1391199844}]},"public_addr":"172.21.15.1:6827/1391199844","cluster_addr":"172.21.15.1:6829/1391199844","heartbeat_back_addr":"172.21.15.1:6833/1391199844","heartbeat_front_addr":"172.21.15.1:6831/1391199844","state":["exists","up"]},{"osd":10,"uuid":"2c090e7a-5240-4d52-ae4f-e94fe05383d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6825","nonce":1773545447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6827","nonce":1773545447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6831","nonce":1773545447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6829","nonce":1773545447}]},"public_addr":"172.21.15.26:6825/1773545447","cluster_addr":"172.21.15.26:6827/1773545447","heartbeat_back_addr":"172.21.15.26:6831/1773545447","heartbeat_front_addr":"172.21.15.26:6829/1773545447","state":["exists","up"]},{"osd":11,"uuid":"4511e32d-e469-45e1-8310-0ff400cb7b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6825","nonce":2564669546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6827","nonce":2564669546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6831","nonce":2564669546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6829","nonce":2564669546}]},"public_addr":"172.21.15.53:6825/2564669546","cluster_addr":"172.21.15.53:6827/2564669546","heartbeat_back_addr":"172.21.15.53:6831/2564669546","heartbeat_front_addr":"172.21.15.53:6829/2564669546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:28.140250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:49.885461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:12.521487+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:34.926197+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:56.815594+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:18.387751+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:40.587106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:02.319370+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:23.832298+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:46.338040+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:08.730414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:30.050070+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1494336948":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6801/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6800/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6800/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/1641976669":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/2895923638":"2023-11-17T16:30:35.064289+0000","172.21.15.1:6801/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/78811711":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/4131957708":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/344403698":"2023-11-17T16:30:35.064289+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:38:48.035 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.032+0000 7fbd79721700 1 -- 172.21.15.1:0/842505632 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbd5c0a1150 msgr2=0x7fbd5c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:48.035 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.032+0000 7fbd79721700 1 --2- 172.21.15.1:0/842505632 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbd5c0a1150 0x7fbd5c0a3600 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbd740fec20 tx=0x7fbd68005fb0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.035 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.032+0000 7fbd79721700 1 -- 172.21.15.1:0/842505632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd740ff430 msgr2=0x7fbd740feeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:48.035 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.032+0000 7fbd79721700 1 --2- 172.21.15.1:0/842505632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd740ff430 0x7fbd740feeb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbd6400e9f0 tx=0x7fbd6400ed00 comp rx=0 tx=0).stop 2023-11-16T16:38:48.035 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.032+0000 7fbd79721700 1 -- 172.21.15.1:0/842505632 shutdown_connections 2023-11-16T16:38:48.035 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.032+0000 7fbd79721700 1 --2- 172.21.15.1:0/842505632 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbd5c0a1150 0x7fbd5c0a3600 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.035 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.032+0000 7fbd79721700 1 --2- 172.21.15.1:0/842505632 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd74100870 0x7fbd7419d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.036 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.032+0000 7fbd79721700 1 --2- 172.21.15.1:0/842505632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbd740ff430 0x7fbd740feeb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.036 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.032+0000 7fbd79721700 1 --2- 172.21.15.1:0/842505632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd740ffde0 0x7fbd7419adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.036 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.032+0000 7fbd79721700 1 -- 172.21.15.1:0/842505632 >> 172.21.15.1:0/842505632 conn(0x7fbd740f9f70 msgr2=0x7fbd740fc150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:48.036 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.032+0000 7fbd79721700 1 -- 172.21.15.1:0/842505632 shutdown_connections 2023-11-16T16:38:48.036 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.032+0000 7fbd79721700 1 -- 172.21.15.1:0/842505632 wait complete. 2023-11-16T16:38:48.047 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-11-16T16:37:13.918943+0000', 'modified': '2023-11-16T16:37:19.292898+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24521, 'mds_1': 24493, 'mds_2': 24482}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24463': {'gid': 24463, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.53:6835/2140542192', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6834', 'nonce': 2140542192}, {'type': 'v1', 'addr': '172.21.15.53:6835', 'nonce': 2140542192}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24467': {'gid': 24467, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.26:6835/1010760275', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 1010760275}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 1010760275}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24472': {'gid': 24472, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.53:6837/3197060302', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6836', 'nonce': 3197060302}, {'type': 'v1', 'addr': '172.21.15.53:6837', 'nonce': 3197060302}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24482': {'gid': 24482, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.26:6837/1453504863', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6836', 'nonce': 1453504863}, {'type': 'v1', 'addr': '172.21.15.26:6837', 'nonce': 1453504863}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24493': {'gid': 24493, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.53:6839/1044753024', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6838', 'nonce': 1044753024}, {'type': 'v1', 'addr': '172.21.15.53:6839', 'nonce': 1044753024}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24521': {'gid': 24521, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6839/2192205949', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6838', 'nonce': 2192205949}, {'type': 'v1', 'addr': '172.21.15.26:6839', 'nonce': 2192205949}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-16T16:38:48.048 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-11-16T16:38:48.048 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-11-16T16:38:48.178 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.172+0000 7f2aad667700 1 -- 172.21.15.1:0/192910234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2aa80ff110 msgr2=0x7f2aa80ff520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:48.178 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.173+0000 7f2aad667700 1 --2- 172.21.15.1:0/192910234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2aa80ff110 0x7f2aa80ff520 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c009990 tx=0x7f2a9c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.178 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.173+0000 7f2aad667700 1 -- 172.21.15.1:0/192910234 shutdown_connections 2023-11-16T16:38:48.179 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.173+0000 7f2aad667700 1 --2- 172.21.15.1:0/192910234 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2aa8100c90 0x7f2aa8107550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.179 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.173+0000 7f2aad667700 1 --2- 172.21.15.1:0/192910234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2aa80ff110 0x7f2aa80ff520 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.179 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.173+0000 7f2aad667700 1 --2- 172.21.15.1:0/192910234 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2aa8100300 0x7f2aa8100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.179 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.173+0000 7f2aad667700 1 -- 172.21.15.1:0/192910234 >> 172.21.15.1:0/192910234 conn(0x7f2aa80fa6f0 msgr2=0x7f2aa80fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:48.179 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.173+0000 7f2aad667700 1 -- 172.21.15.1:0/192910234 shutdown_connections 2023-11-16T16:38:48.179 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.173+0000 7f2aad667700 1 -- 172.21.15.1:0/192910234 wait complete. 2023-11-16T16:38:48.179 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.173+0000 7f2aad667700 1 Processor -- start 2023-11-16T16:38:48.179 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.173+0000 7f2aad667700 1 -- start start 2023-11-16T16:38:48.180 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.174+0000 7f2aad667700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2aa80ff110 0x7f2aa8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.180 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.174+0000 7f2aad667700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2aa8100300 0x7f2aa819ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.180 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.174+0000 7f2aad667700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2aa8100c90 0x7f2aa819d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.180 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.174+0000 7f2aad667700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa819da70 con 0x7f2aa8100c90 2023-11-16T16:38:48.180 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.174+0000 7f2aad667700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa819dbe0 con 0x7f2aa8100300 2023-11-16T16:38:48.180 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.174+0000 7f2aad667700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa819dda0 con 0x7f2aa80ff110 2023-11-16T16:38:48.180 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.174+0000 7f2aa77fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2aa8100c90 0x7f2aa819d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.180 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.174+0000 7f2aa77fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2aa8100c90 0x7f2aa819d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42694/0 (socket says 172.21.15.1:42694) 2023-11-16T16:38:48.181 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.174+0000 7f2aa77fe700 1 -- 172.21.15.1:0/2110240541 learned_addr learned my addr 172.21.15.1:0/2110240541 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:48.181 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.174+0000 7f2aa6ffd700 1 --2- 172.21.15.1:0/2110240541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2aa80ff110 0x7f2aa8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.181 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.174+0000 7f2aa67fc700 1 --2- 172.21.15.1:0/2110240541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2aa8100300 0x7f2aa819ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.181 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.174+0000 7f2aa77fe700 1 -- 172.21.15.1:0/2110240541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2aa80ff110 msgr2=0x7f2aa8068930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:48.181 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.174+0000 7f2aa77fe700 1 --2- 172.21.15.1:0/2110240541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2aa80ff110 0x7f2aa8068930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.181 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.174+0000 7f2aa77fe700 1 -- 172.21.15.1:0/2110240541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2aa8100300 msgr2=0x7f2aa819ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:48.181 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.174+0000 7f2aa77fe700 1 --2- 172.21.15.1:0/2110240541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2aa8100300 0x7f2aa819ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.181 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.174+0000 7f2aa77fe700 1 -- 172.21.15.1:0/2110240541 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a9c0095f0 con 0x7f2aa8100c90 2023-11-16T16:38:48.182 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.174+0000 7f2aa67fc700 1 --2- 172.21.15.1:0/2110240541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2aa8100300 0x7f2aa819ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:38:48.182 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.174+0000 7f2aa77fe700 1 --2- 172.21.15.1:0/2110240541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2aa8100c90 0x7f2aa819d2d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0007d70 tx=0x7f2aa000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:48.182 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.174+0000 7f2a8ffff700 1 -- 172.21.15.1:0/2110240541 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa000ad40 con 0x7f2aa8100c90 2023-11-16T16:38:48.182 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.174+0000 7f2aad667700 1 -- 172.21.15.1:0/2110240541 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aa810aa80 con 0x7f2aa8100c90 2023-11-16T16:38:48.182 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.174+0000 7f2a8ffff700 1 -- 172.21.15.1:0/2110240541 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2aa00128e0 con 0x7f2aa8100c90 2023-11-16T16:38:48.182 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.174+0000 7f2a8ffff700 1 -- 172.21.15.1:0/2110240541 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa00116b0 con 0x7f2aa8100c90 2023-11-16T16:38:48.182 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.174+0000 7f2aad667700 1 -- 172.21.15.1:0/2110240541 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aa810a8f0 con 0x7f2aa8100c90 2023-11-16T16:38:48.182 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.176+0000 7f2a8ffff700 1 -- 172.21.15.1:0/2110240541 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f2aa0012420 con 0x7f2aa8100c90 2023-11-16T16:38:48.190 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.177+0000 7f2a8ffff700 1 --2- 172.21.15.1:0/2110240541 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f2a900a1150 0x7f2a900a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.191 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.177+0000 7f2a8ffff700 1 -- 172.21.15.1:0/2110240541 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2aa00ccba0 con 0x7f2aa8100c90 2023-11-16T16:38:48.191 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.177+0000 7f2aa6ffd700 1 --2- 172.21.15.1:0/2110240541 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f2a900a1150 0x7f2a900a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.191 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.177+0000 7f2aa6ffd700 1 --2- 172.21.15.1:0/2110240541 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f2a900a1150 0x7f2a900a3600 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c00b5c0 tx=0x7f2a9c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:48.191 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.178+0000 7f2ab97f3b80 1 Processor -- start 2023-11-16T16:38:48.191 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.178+0000 7f2ab97f3b80 1 -- start start 2023-11-16T16:38:48.191 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.178+0000 7f2ab97f3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563c60a5bb0 0x5563c60a5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.191 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.178+0000 7f2ab97f3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563c60a6500 0x5563c60a6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.191 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.178+0000 7f2ab97f3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563c60a70c0 0x5563c60a7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.178+0000 7f2ab97f3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563c5fab010 con 0x5563c60a5bb0 2023-11-16T16:38:48.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.178+0000 7f2ab97f3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563c5ffd850 con 0x5563c60a6500 2023-11-16T16:38:48.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.178+0000 7f2ab97f3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563c5ff6ce0 con 0x5563c60a70c0 2023-11-16T16:38:48.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.178+0000 7f2aa6ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563c60a5bb0 0x5563c60a5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.178+0000 7f2aa6ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563c60a5bb0 0x5563c60a5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42698/0 (socket says 172.21.15.1:42698) 2023-11-16T16:38:48.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.178+0000 7f2aa6ffd700 1 -- 172.21.15.1:0/2936397575 learned_addr learned my addr 172.21.15.1:0/2936397575 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:48.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.179+0000 7f2aa67fc700 1 --2- 172.21.15.1:0/2936397575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563c60a6500 0x5563c60a6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.179+0000 7f2aa77fe700 1 --2- 172.21.15.1:0/2936397575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563c60a70c0 0x5563c60a7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.193 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.179+0000 7f2aa6ffd700 1 -- 172.21.15.1:0/2936397575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563c60a70c0 msgr2=0x5563c60a7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:48.193 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.179+0000 7f2aa6ffd700 1 --2- 172.21.15.1:0/2936397575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563c60a70c0 0x5563c60a7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.193 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.179+0000 7f2aa6ffd700 1 -- 172.21.15.1:0/2936397575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563c60a6500 msgr2=0x5563c60a6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:48.193 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.179+0000 7f2aa6ffd700 1 --2- 172.21.15.1:0/2936397575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563c60a6500 0x5563c60a6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.193 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.179+0000 7f2aa6ffd700 1 -- 172.21.15.1:0/2936397575 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563c5e785e0 con 0x5563c60a5bb0 2023-11-16T16:38:48.193 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.179+0000 7f2aa6ffd700 1 --2- 172.21.15.1:0/2936397575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563c60a5bb0 0x5563c60a5fc0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c015000 tx=0x7f2a9c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a15fa2e49617ff2 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:48.193 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.179+0000 7f2a8d7fa700 1 -- 172.21.15.1:0/2936397575 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c005a00 con 0x5563c60a5bb0 2023-11-16T16:38:48.193 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.179+0000 7f2a8d7fa700 1 -- 172.21.15.1:0/2936397575 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a9c01e040 con 0x5563c60a5bb0 2023-11-16T16:38:48.194 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.179+0000 7f2a8d7fa700 1 -- 172.21.15.1:0/2936397575 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c0048c0 con 0x5563c60a5bb0 2023-11-16T16:38:48.194 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.180+0000 7f2ab97f3b80 1 -- 172.21.15.1:0/2936397575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563c60a5bb0 msgr2=0x5563c60a5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:48.194 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.180+0000 7f2ab97f3b80 1 --2- 172.21.15.1:0/2936397575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563c60a5bb0 0x5563c60a5fc0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c015000 tx=0x7f2a9c020000 comp rx=0 tx=0).stop 2023-11-16T16:38:48.194 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.180+0000 7f2ab97f3b80 1 -- 172.21.15.1:0/2936397575 shutdown_connections 2023-11-16T16:38:48.194 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.180+0000 7f2ab97f3b80 1 --2- 172.21.15.1:0/2936397575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563c60a70c0 0x5563c60a7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.194 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.180+0000 7f2ab97f3b80 1 --2- 172.21.15.1:0/2936397575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563c60a5bb0 0x5563c60a5fc0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.194 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.180+0000 7f2ab97f3b80 1 --2- 172.21.15.1:0/2936397575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563c60a6500 0x5563c60a6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.194 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.180+0000 7f2ab97f3b80 1 -- 172.21.15.1:0/2936397575 >> 172.21.15.1:0/2936397575 conn(0x5563c609b310 msgr2=0x5563c60a0620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:48.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.180+0000 7f2ab97f3b80 1 -- 172.21.15.1:0/2936397575 shutdown_connections 2023-11-16T16:38:48.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.180+0000 7f2ab97f3b80 1 -- 172.21.15.1:0/2936397575 wait complete. 2023-11-16T16:38:48.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.180+0000 7f2ab97f3b80 1 -- start start 2023-11-16T16:38:48.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.180+0000 7f2ab97f3b80 1 Processor -- start 2023-11-16T16:38:48.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.180+0000 7f2ab97f3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563c60a5bb0 0x5563c60af560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.180+0000 7f2ab97f3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563c60a6500 0x5563c60afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.180+0000 7f2ab97f3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563c60a70c0 0x5563c60b6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.180+0000 7f2ab97f3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563c5f98890 con 0x5563c60a5bb0 2023-11-16T16:38:48.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.180+0000 7f2ab97f3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563c5f98530 con 0x5563c60a6500 2023-11-16T16:38:48.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.180+0000 7f2ab97f3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563c5e7a8d0 con 0x5563c60a70c0 2023-11-16T16:38:48.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.180+0000 7f2ab97f3b80 10 client.0 _ll_register_callbacks cb 0x5563c5fb6730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:38:48.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.180+0000 7f2aa6ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563c60a5bb0 0x5563c60af560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.180+0000 7f2aa6ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563c60a5bb0 0x5563c60af560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42704/0 (socket says 172.21.15.1:42704) 2023-11-16T16:38:48.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.180+0000 7f2aa6ffd700 1 -- 172.21.15.1:0/3090080161 learned_addr learned my addr 172.21.15.1:0/3090080161 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:48.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.181+0000 7f2aa67fc700 1 --2- 172.21.15.1:0/3090080161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563c60a6500 0x5563c60afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.181+0000 7f2aa77fe700 1 --2- 172.21.15.1:0/3090080161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563c60a70c0 0x5563c60b6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.197 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.181+0000 7f2aa6ffd700 1 -- 172.21.15.1:0/3090080161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563c60a70c0 msgr2=0x5563c60b6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:48.197 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.181+0000 7f2aa6ffd700 1 --2- 172.21.15.1:0/3090080161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563c60a70c0 0x5563c60b6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.197 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.181+0000 7f2aa6ffd700 1 -- 172.21.15.1:0/3090080161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563c60a6500 msgr2=0x5563c60afaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:48.197 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.181+0000 7f2aa6ffd700 1 --2- 172.21.15.1:0/3090080161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563c60a6500 0x5563c60afaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.197 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.181+0000 7f2aa6ffd700 1 -- 172.21.15.1:0/3090080161 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aa8107e80 con 0x5563c60a5bb0 2023-11-16T16:38:48.197 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.181+0000 7f2aa6ffd700 1 --2- 172.21.15.1:0/3090080161 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563c60a5bb0 0x5563c60af560 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c000c00 tx=0x7f2a9c0092b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d715649a1ead4fa0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:48.197 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.181+0000 7f2a8aff5700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:38:48.197 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.181+0000 7f2a8aff5700 1 -- 172.21.15.1:0/3090080161 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c003890 con 0x5563c60a5bb0 2023-11-16T16:38:48.197 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.181+0000 7f2a8aff5700 1 -- 172.21.15.1:0/3090080161 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2a9c0039f0 con 0x5563c60a5bb0 2023-11-16T16:38:48.198 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.181+0000 7f2ab97f3b80 1 -- 172.21.15.1:0/3090080161 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5563c6098ad0 con 0x5563c60a5bb0 2023-11-16T16:38:48.198 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.181+0000 7f2a8aff5700 1 -- 172.21.15.1:0/3090080161 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a9c003bb0 con 0x5563c60a5bb0 2023-11-16T16:38:48.198 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.181+0000 7f2ab97f3b80 10 client.15222 fetch_fsmap learned FSMap version 23 2023-11-16T16:38:48.198 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.181+0000 7f2ab97f3b80 1 -- 172.21.15.1:0/3090080161 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5563c5fbd140 con 0x5563c60a5bb0 2023-11-16T16:38:48.198 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.181+0000 7f2a8aff5700 1 -- 172.21.15.1:0/3090080161 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5563c6098ad0 con 0x5563c60a5bb0 2023-11-16T16:38:48.198 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.182+0000 7f2a8aff5700 1 -- 172.21.15.1:0/3090080161 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2a9c00f460 con 0x5563c60a5bb0 2023-11-16T16:38:48.198 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.182+0000 7f2ab97f3b80 10 client.15222 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:38:48.198 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.182+0000 7f2ab97f3b80 10 client.15222 resolve_mds: resolved name 'h' to daemon mds.h 2023-11-16T16:38:48.199 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.182+0000 7f2ab97f3b80 20 client.15222 populate_metadata read hostname 'smithi001' 2023-11-16T16:38:48.199 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.182+0000 7f2ab97f3b80 1 --2- 172.21.15.1:0/3090080161 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x5563c60ba2b0 0x5563c60b9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.199 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.182+0000 7f2ab97f3b80 4 client.15222 mds_command: new command op to 24482 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:38:48.199 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.182+0000 7f2ab97f3b80 1 -- 172.21.15.1:0/3090080161 --> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5563c5fb9050 con 0x5563c60ba2b0 2023-11-16T16:38:48.199 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.182+0000 7f2aa67fc700 1 --2- 172.21.15.1:0/3090080161 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x5563c60ba2b0 0x5563c60b9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.199 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.182+0000 7f2a8aff5700 10 client.15222 ms_handle_connect on v2:172.21.15.26:6836/1453504863 2023-11-16T16:38:48.199 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.182+0000 7f2aa67fc700 1 --2- 172.21.15.1:0/3090080161 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x5563c60ba2b0 0x5563c60b9a10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=be47642508e4f3b6 server_cookie=40f20b9121e3d421 in_seq=0 out_seq=0 2023-11-16T16:38:48.199 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.183+0000 7f2a8aff5700 1 -- 172.21.15.1:0/3090080161 <== mds.2 v2:172.21.15.26:6836/1453504863 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5563c5fb9050 con 0x5563c60ba2b0 2023-11-16T16:38:48.199 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.183+0000 7f2a8aff5700 10 client.15222 handle_command_reply: tid=0 2023-11-16T16:38:48.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.183+0000 7f2ab97f3b80 1 client.15222 shutdown 2023-11-16T16:38:48.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.183+0000 7f2ab97f3b80 1 -- 172.21.15.1:0/3090080161 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563c60a5bb0 msgr2=0x5563c60af560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:48.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.183+0000 7f2ab97f3b80 1 --2- 172.21.15.1:0/3090080161 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563c60a5bb0 0x5563c60af560 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c000c00 tx=0x7f2a9c0092b0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.183+0000 7f2ab97f3b80 1 -- 172.21.15.1:0/3090080161 shutdown_connections 2023-11-16T16:38:48.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.183+0000 7f2ab97f3b80 1 --2- 172.21.15.1:0/3090080161 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x5563c60ba2b0 0x5563c60b9a10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.183+0000 7f2ab97f3b80 1 --2- 172.21.15.1:0/3090080161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563c60a70c0 0x5563c60b6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.183+0000 7f2ab97f3b80 1 --2- 172.21.15.1:0/3090080161 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563c60a5bb0 0x5563c60af560 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.183+0000 7f2ab97f3b80 1 --2- 172.21.15.1:0/3090080161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563c60a6500 0x5563c60afaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.183+0000 7f2a8aff5700 0 client.15222 ms_handle_reset on v2:172.21.15.26:6836/1453504863 2023-11-16T16:38:48.201 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.183+0000 7f2ab97f3b80 1 -- 172.21.15.1:0/3090080161 >> 172.21.15.1:0/3090080161 conn(0x5563c609b310 msgr2=0x5563c609fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:48.201 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.183+0000 7f2ab97f3b80 1 -- 172.21.15.1:0/3090080161 shutdown_connections 2023-11-16T16:38:48.201 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.183+0000 7f2ab97f3b80 1 -- 172.21.15.1:0/3090080161 wait complete. 2023-11-16T16:38:48.201 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.184+0000 7f2ab97f3b80 20 client.15222 trim_cache size 0 max 16384 2023-11-16T16:38:48.210 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.201+0000 7f2ab97f3b80 1 Processor -- start 2023-11-16T16:38:48.210 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.201+0000 7f2ab97f3b80 1 -- start start 2023-11-16T16:38:48.210 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.201+0000 7f2ab97f3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563c60a5bb0 0x5563c60ad460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.201+0000 7f2ab97f3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563c60a6500 0x5563c60b6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.201+0000 7f2ab97f3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563c60a70c0 0x5563c60a9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.202+0000 7f2ab97f3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563c60a79b0 con 0x5563c60a6500 2023-11-16T16:38:48.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.202+0000 7f2ab97f3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563c60bb570 con 0x5563c60a70c0 2023-11-16T16:38:48.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.202+0000 7f2ab97f3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563c5fa0730 con 0x5563c60a5bb0 2023-11-16T16:38:48.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.202+0000 7f2aa67fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563c60a6500 0x5563c60b6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.202+0000 7f2aa67fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563c60a6500 0x5563c60b6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42714/0 (socket says 172.21.15.1:42714) 2023-11-16T16:38:48.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.202+0000 7f2aa67fc700 1 -- 172.21.15.1:0/749958427 learned_addr learned my addr 172.21.15.1:0/749958427 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:48.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.202+0000 7f2aa77fe700 1 --2- 172.21.15.1:0/749958427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563c60a70c0 0x5563c60a9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.212 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.202+0000 7f2aa67fc700 1 -- 172.21.15.1:0/749958427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563c60a5bb0 msgr2=0x5563c60ad460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:48.212 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.202+0000 7f2aa6ffd700 1 --2- 172.21.15.1:0/749958427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563c60a5bb0 0x5563c60ad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.212 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.202+0000 7f2aa67fc700 1 --2- 172.21.15.1:0/749958427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563c60a5bb0 0x5563c60ad460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.212 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.202+0000 7f2aa67fc700 1 -- 172.21.15.1:0/749958427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563c60a70c0 msgr2=0x5563c60a9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:48.212 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.202+0000 7f2aa67fc700 1 --2- 172.21.15.1:0/749958427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563c60a70c0 0x5563c60a9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.212 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.202+0000 7f2aa67fc700 1 -- 172.21.15.1:0/749958427 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563c5fbce90 con 0x5563c60a6500 2023-11-16T16:38:48.212 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.202+0000 7f2aa77fe700 1 --2- 172.21.15.1:0/749958427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563c60a70c0 0x5563c60a9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:38:48.212 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.202+0000 7f2aa67fc700 1 --2- 172.21.15.1:0/749958427 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563c60a6500 0x5563c60b6aa0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x5563c60b05c0 tx=0x7f2a980041d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98a6c888f8d67aba server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:48.212 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.202+0000 7f2a8dffb700 1 -- 172.21.15.1:0/749958427 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a98017040 con 0x5563c60a6500 2023-11-16T16:38:48.213 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.202+0000 7f2a8dffb700 1 -- 172.21.15.1:0/749958427 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a9801d610 con 0x5563c60a6500 2023-11-16T16:38:48.213 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.202+0000 7f2a8dffb700 1 -- 172.21.15.1:0/749958427 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a98011980 con 0x5563c60a6500 2023-11-16T16:38:48.213 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.203+0000 7f2ab97f3b80 1 -- 172.21.15.1:0/749958427 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563c60a6500 msgr2=0x5563c60b6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:48.213 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.203+0000 7f2ab97f3b80 1 --2- 172.21.15.1:0/749958427 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563c60a6500 0x5563c60b6aa0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x5563c60b05c0 tx=0x7f2a980041d0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.213 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.203+0000 7f2ab97f3b80 1 -- 172.21.15.1:0/749958427 shutdown_connections 2023-11-16T16:38:48.213 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.203+0000 7f2ab97f3b80 1 --2- 172.21.15.1:0/749958427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563c60a70c0 0x5563c60a9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.213 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.203+0000 7f2ab97f3b80 1 --2- 172.21.15.1:0/749958427 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563c60a5bb0 0x5563c60ad460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.213 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.203+0000 7f2ab97f3b80 1 --2- 172.21.15.1:0/749958427 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563c60a6500 0x5563c60b6aa0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.213 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.203+0000 7f2ab97f3b80 1 -- 172.21.15.1:0/749958427 >> 172.21.15.1:0/749958427 conn(0x5563c609b310 msgr2=0x5563c60c2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:48.214 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.203+0000 7f2ab97f3b80 1 -- 172.21.15.1:0/749958427 shutdown_connections 2023-11-16T16:38:48.214 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.203+0000 7f2ab97f3b80 1 -- 172.21.15.1:0/749958427 wait complete. 2023-11-16T16:38:48.214 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.203+0000 7f2ab97f3b80 1 -- start start 2023-11-16T16:38:48.214 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.203+0000 7f2ab97f3b80 1 Processor -- start 2023-11-16T16:38:48.214 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.203+0000 7f2ab97f3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563c60a5bb0 0x5563c60abae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.214 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.203+0000 7f2ab97f3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563c60a6500 0x5563c60ac020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.214 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.203+0000 7f2ab97f3b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563c60a70c0 0x5563c6112130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.214 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.203+0000 7f2ab97f3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563c6112670 con 0x5563c60a5bb0 2023-11-16T16:38:48.214 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.203+0000 7f2ab97f3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563c61127b0 con 0x5563c60a6500 2023-11-16T16:38:48.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.203+0000 7f2ab97f3b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563c6112940 con 0x5563c60a70c0 2023-11-16T16:38:48.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.203+0000 7f2ab97f3b80 10 client.0 _ll_register_callbacks cb 0x7f2a9c0053f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:38:48.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.203+0000 7f2aa6ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563c60a5bb0 0x5563c60abae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.203+0000 7f2aa6ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563c60a5bb0 0x5563c60abae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42718/0 (socket says 172.21.15.1:42718) 2023-11-16T16:38:48.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.203+0000 7f2aa6ffd700 1 -- 172.21.15.1:0/2861701009 learned_addr learned my addr 172.21.15.1:0/2861701009 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:48.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.204+0000 7f2aa67fc700 1 --2- 172.21.15.1:0/2861701009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563c60a6500 0x5563c60ac020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.204+0000 7f2aa77fe700 1 --2- 172.21.15.1:0/2861701009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563c60a70c0 0x5563c6112130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.204+0000 7f2aa6ffd700 1 -- 172.21.15.1:0/2861701009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563c60a70c0 msgr2=0x5563c6112130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:48.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.204+0000 7f2aa6ffd700 1 --2- 172.21.15.1:0/2861701009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563c60a70c0 0x5563c6112130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.204+0000 7f2aa6ffd700 1 -- 172.21.15.1:0/2861701009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563c60a6500 msgr2=0x5563c60ac020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:48.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.204+0000 7f2aa6ffd700 1 --2- 172.21.15.1:0/2861701009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563c60a6500 0x5563c60ac020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.204+0000 7f2aa6ffd700 1 -- 172.21.15.1:0/2861701009 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a98003b00 con 0x5563c60a5bb0 2023-11-16T16:38:48.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.204+0000 7f2aa67fc700 1 --2- 172.21.15.1:0/2861701009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563c60a6500 0x5563c60ac020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:38:48.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.204+0000 7f2aa6ffd700 1 --2- 172.21.15.1:0/2861701009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563c60a5bb0 0x5563c60abae0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c0054c0 tx=0x7f2a9c005320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6601a96b562e6ace server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:48.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.204+0000 7f2a8aff5700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:38:48.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.204+0000 7f2a8aff5700 1 -- 172.21.15.1:0/2861701009 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c004a20 con 0x5563c60a5bb0 2023-11-16T16:38:48.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.204+0000 7f2a8aff5700 1 -- 172.21.15.1:0/2861701009 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2a9c00fa10 con 0x5563c60a5bb0 2023-11-16T16:38:48.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.204+0000 7f2ab97f3b80 1 -- 172.21.15.1:0/2861701009 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a9c01e040 con 0x5563c60a5bb0 2023-11-16T16:38:48.217 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.204+0000 7f2a8aff5700 1 -- 172.21.15.1:0/2861701009 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a9c003a90 con 0x5563c60a5bb0 2023-11-16T16:38:48.217 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.204+0000 7f2ab97f3b80 10 client.15228 fetch_fsmap learned FSMap version 23 2023-11-16T16:38:48.217 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.204+0000 7f2ab97f3b80 1 -- 172.21.15.1:0/2861701009 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5563c5e0dc10 con 0x5563c60a5bb0 2023-11-16T16:38:48.217 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.204+0000 7f2a8aff5700 1 -- 172.21.15.1:0/2861701009 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a9c01e040 con 0x5563c60a5bb0 2023-11-16T16:38:48.217 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.205+0000 7f2a8aff5700 1 -- 172.21.15.1:0/2861701009 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2a9c018d00 con 0x5563c60a5bb0 2023-11-16T16:38:48.217 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.205+0000 7f2ab97f3b80 10 client.15228 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:38:48.217 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.205+0000 7f2ab97f3b80 10 client.15228 resolve_mds: resolved name 'h' to daemon mds.h 2023-11-16T16:38:48.217 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.205+0000 7f2ab97f3b80 20 client.15228 populate_metadata read hostname 'smithi001' 2023-11-16T16:38:48.217 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.205+0000 7f2ab97f3b80 1 --2- 172.21.15.1:0/2861701009 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x5563c6117320 0x5563c6113bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.217 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.205+0000 7f2ab97f3b80 4 client.15228 mds_command: new command op to 24482 tid=0[{"prefix": "status"}] 2023-11-16T16:38:48.218 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.205+0000 7f2ab97f3b80 1 -- 172.21.15.1:0/2861701009 --> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5563c5fc5610 con 0x5563c6117320 2023-11-16T16:38:48.218 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.205+0000 7f2aa67fc700 1 --2- 172.21.15.1:0/2861701009 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x5563c6117320 0x5563c6113bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.218 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.205+0000 7f2a8aff5700 10 client.15228 ms_handle_connect on v2:172.21.15.26:6836/1453504863 2023-11-16T16:38:48.218 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.205+0000 7f2aa67fc700 1 --2- 172.21.15.1:0/2861701009 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x5563c6117320 0x5563c6113bf0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=446ac16667d4665 server_cookie=841a4e0525896f7e in_seq=0 out_seq=0 2023-11-16T16:38:48.218 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.206+0000 7f2a8aff5700 1 -- 172.21.15.1:0/2861701009 <== mds.2 v2:172.21.15.26:6836/1453504863 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+319 (crc 0 0 0) 0x5563c5fc5610 con 0x5563c6117320 2023-11-16T16:38:48.218 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.206+0000 7f2a8aff5700 10 client.15228 handle_command_reply: tid=0 2023-11-16T16:38:48.218 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.206+0000 7f2ab97f3b80 1 client.15228 shutdown 2023-11-16T16:38:48.218 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.206+0000 7f2ab97f3b80 1 -- 172.21.15.1:0/2861701009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563c60a5bb0 msgr2=0x5563c60abae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:48.218 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.206+0000 7f2ab97f3b80 1 --2- 172.21.15.1:0/2861701009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563c60a5bb0 0x5563c60abae0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c0054c0 tx=0x7f2a9c005320 comp rx=0 tx=0).stop 2023-11-16T16:38:48.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.206+0000 7f2ab97f3b80 1 -- 172.21.15.1:0/2861701009 shutdown_connections 2023-11-16T16:38:48.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.206+0000 7f2ab97f3b80 1 --2- 172.21.15.1:0/2861701009 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x5563c6117320 0x5563c6113bf0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.206+0000 7f2ab97f3b80 1 --2- 172.21.15.1:0/2861701009 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5563c60a70c0 0x5563c6112130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.206+0000 7f2ab97f3b80 1 --2- 172.21.15.1:0/2861701009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5563c60a5bb0 0x5563c60abae0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.206+0000 7f2a8aff5700 0 client.15228 ms_handle_reset on v2:172.21.15.26:6836/1453504863 2023-11-16T16:38:48.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.206+0000 7f2ab97f3b80 1 --2- 172.21.15.1:0/2861701009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5563c60a6500 0x5563c60ac020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.206+0000 7f2ab97f3b80 1 -- 172.21.15.1:0/2861701009 >> 172.21.15.1:0/2861701009 conn(0x5563c609b310 msgr2=0x5563c60ad460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:48.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.206+0000 7f2ab97f3b80 1 -- 172.21.15.1:0/2861701009 shutdown_connections 2023-11-16T16:38:48.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.206+0000 7f2ab97f3b80 1 -- 172.21.15.1:0/2861701009 wait complete. 2023-11-16T16:38:48.220 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.206+0000 7f2ab97f3b80 20 client.15228 trim_cache size 0 max 16384 2023-11-16T16:38:48.220 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:38:48.220 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "35d9b010-849d-11ee-8dbe-212e2dc638e7", 2023-11-16T16:38:48.220 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 2, 2023-11-16T16:38:48.220 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24482, 2023-11-16T16:38:48.220 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2023-11-16T16:38:48.220 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2023-11-16T16:38:48.221 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2023-11-16T16:38:48.221 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 89.9221565, 2023-11-16T16:38:48.221 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 23, 2023-11-16T16:38:48.221 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 86, 2023-11-16T16:38:48.221 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 84, 2023-11-16T16:38:48.221 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 108.183796633 2023-11-16T16:38:48.221 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:38:48.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.206+0000 7f2aad667700 1 -- 172.21.15.1:0/2110240541 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f2a900a1150 msgr2=0x7f2a900a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:48.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.206+0000 7f2aad667700 1 --2- 172.21.15.1:0/2110240541 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f2a900a1150 0x7f2a900a3600 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c00b5c0 tx=0x7f2a9c005960 comp rx=0 tx=0).stop 2023-11-16T16:38:48.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.206+0000 7f2aad667700 1 -- 172.21.15.1:0/2110240541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2aa8100c90 msgr2=0x7f2aa819d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:48.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.206+0000 7f2aad667700 1 --2- 172.21.15.1:0/2110240541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2aa8100c90 0x7f2aa819d2d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0007d70 tx=0x7f2aa000a340 comp rx=0 tx=0).stop 2023-11-16T16:38:48.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.206+0000 7f2aad667700 1 -- 172.21.15.1:0/2110240541 shutdown_connections 2023-11-16T16:38:48.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.206+0000 7f2aad667700 1 --2- 172.21.15.1:0/2110240541 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f2a900a1150 0x7f2a900a3600 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.206+0000 7f2aad667700 1 --2- 172.21.15.1:0/2110240541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2aa8100c90 0x7f2aa819d2d0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.206+0000 7f2aad667700 1 --2- 172.21.15.1:0/2110240541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2aa80ff110 0x7f2aa8068930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.224 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.206+0000 7f2aad667700 1 --2- 172.21.15.1:0/2110240541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2aa8100300 0x7f2aa819ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.224 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.206+0000 7f2aad667700 1 -- 172.21.15.1:0/2110240541 >> 172.21.15.1:0/2110240541 conn(0x7f2aa80fa6f0 msgr2=0x7f2aa80fc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:48.224 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.207+0000 7f2aad667700 1 -- 172.21.15.1:0/2110240541 shutdown_connections 2023-11-16T16:38:48.224 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.207+0000 7f2aad667700 1 -- 172.21.15.1:0/2110240541 wait complete. 2023-11-16T16:38:48.224 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-11-16T16:38:48.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.350+0000 7f57a9179700 1 -- 172.21.15.1:0/1587184927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57a4101db0 msgr2=0x7f57a4108670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:48.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.350+0000 7f57a9179700 1 --2- 172.21.15.1:0/1587184927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57a4101db0 0x7f57a4108670 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f579c00b0a0 tx=0x7f579c00b3b0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.350+0000 7f57a9179700 1 -- 172.21.15.1:0/1587184927 shutdown_connections 2023-11-16T16:38:48.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.350+0000 7f57a9179700 1 --2- 172.21.15.1:0/1587184927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57a4101db0 0x7f57a4108670 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.350+0000 7f57a9179700 1 --2- 172.21.15.1:0/1587184927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57a4100230 0x7f57a4100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.350+0000 7f57a9179700 1 --2- 172.21.15.1:0/1587184927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57a4101420 0x7f57a4101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.356 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.350+0000 7f57a9179700 1 -- 172.21.15.1:0/1587184927 >> 172.21.15.1:0/1587184927 conn(0x7f57a40fb810 msgr2=0x7f57a40fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:48.356 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.350+0000 7f57a9179700 1 -- 172.21.15.1:0/1587184927 shutdown_connections 2023-11-16T16:38:48.356 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.350+0000 7f57a9179700 1 -- 172.21.15.1:0/1587184927 wait complete. 2023-11-16T16:38:48.356 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.350+0000 7f57a9179700 1 Processor -- start 2023-11-16T16:38:48.356 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.351+0000 7f57a9179700 1 -- start start 2023-11-16T16:38:48.356 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.351+0000 7f57a9179700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57a4100230 0x7f57a4196050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.356 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.351+0000 7f57a9179700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57a4101420 0x7f57a4196590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.357 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.351+0000 7f57a9179700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57a4101db0 0x7f57a419d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.357 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.351+0000 7f57a9179700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a419dbd0 con 0x7f57a4100230 2023-11-16T16:38:48.357 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.351+0000 7f57a9179700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a419dd10 con 0x7f57a4101db0 2023-11-16T16:38:48.357 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.351+0000 7f57a9179700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a419dea0 con 0x7f57a4101420 2023-11-16T16:38:48.357 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.351+0000 7f57a2d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57a4100230 0x7f57a4196050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.357 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.351+0000 7f57a2d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57a4100230 0x7f57a4196050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42732/0 (socket says 172.21.15.1:42732) 2023-11-16T16:38:48.358 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.351+0000 7f57a2d9d700 1 -- 172.21.15.1:0/3486272305 learned_addr learned my addr 172.21.15.1:0/3486272305 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:48.358 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.351+0000 7f57a259c700 1 --2- 172.21.15.1:0/3486272305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57a4101420 0x7f57a4196590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.358 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.351+0000 7f57a359e700 1 --2- 172.21.15.1:0/3486272305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57a4101db0 0x7f57a419d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.358 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.351+0000 7f57a2d9d700 1 -- 172.21.15.1:0/3486272305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57a4101420 msgr2=0x7f57a4196590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:48.358 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.351+0000 7f57a2d9d700 1 --2- 172.21.15.1:0/3486272305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57a4101420 0x7f57a4196590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.358 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.351+0000 7f57a2d9d700 1 -- 172.21.15.1:0/3486272305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57a4101db0 msgr2=0x7f57a419d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:48.358 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.351+0000 7f57a2d9d700 1 --2- 172.21.15.1:0/3486272305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57a4101db0 0x7f57a419d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.359 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.351+0000 7f57a2d9d700 1 -- 172.21.15.1:0/3486272305 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579c009d00 con 0x7f57a4100230 2023-11-16T16:38:48.359 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.351+0000 7f57a259c700 1 --2- 172.21.15.1:0/3486272305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57a4101420 0x7f57a4196590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:38:48.359 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.351+0000 7f57a2d9d700 1 --2- 172.21.15.1:0/3486272305 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57a4100230 0x7f57a4196050 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f579800d980 tx=0x7f579800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:48.359 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.352+0000 7f578bfff700 1 -- 172.21.15.1:0/3486272305 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579800de60 con 0x7f57a4100230 2023-11-16T16:38:48.359 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.352+0000 7f57a9179700 1 -- 172.21.15.1:0/3486272305 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a410bba0 con 0x7f57a4100230 2023-11-16T16:38:48.359 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.352+0000 7f578bfff700 1 -- 172.21.15.1:0/3486272305 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57980044e0 con 0x7f57a4100230 2023-11-16T16:38:48.360 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.352+0000 7f578bfff700 1 -- 172.21.15.1:0/3486272305 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5798003be0 con 0x7f57a4100230 2023-11-16T16:38:48.360 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.352+0000 7f57a9179700 1 -- 172.21.15.1:0/3486272305 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a410ba10 con 0x7f57a4100230 2023-11-16T16:38:48.368 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.354+0000 7f578bfff700 1 -- 172.21.15.1:0/3486272305 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f5798009bb0 con 0x7f57a4100230 2023-11-16T16:38:48.369 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.354+0000 7f578bfff700 1 --2- 172.21.15.1:0/3486272305 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f578c0a1150 0x7f578c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.369 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.354+0000 7f578bfff700 1 -- 172.21.15.1:0/3486272305 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f57980cc6a0 con 0x7f57a4100230 2023-11-16T16:38:48.369 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.354+0000 7f57a259c700 1 --2- 172.21.15.1:0/3486272305 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f578c0a1150 0x7f578c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.369 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.355+0000 7f57a259c700 1 --2- 172.21.15.1:0/3486272305 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f578c0a1150 0x7f578c0a3600 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5794009cb0 tx=0x7f5794009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:48.369 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.355+0000 7f57b5305b80 1 Processor -- start 2023-11-16T16:38:48.369 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.355+0000 7f57b5305b80 1 -- start start 2023-11-16T16:38:48.369 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.355+0000 7f57b5305b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a966dcbb0 0x564a966dcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.369 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.355+0000 7f57b5305b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a966dd500 0x564a966ddae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.370 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.355+0000 7f57b5305b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a966de0c0 0x564a966de530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.370 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.356+0000 7f57b5305b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a965e2010 con 0x564a966de0c0 2023-11-16T16:38:48.370 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.356+0000 7f57b5305b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a96634850 con 0x564a966dd500 2023-11-16T16:38:48.370 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.356+0000 7f57b5305b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a9662dce0 con 0x564a966dcbb0 2023-11-16T16:38:48.370 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.356+0000 7f57a359e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a966de0c0 0x564a966de530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.370 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.356+0000 7f57a359e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a966de0c0 0x564a966de530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42744/0 (socket says 172.21.15.1:42744) 2023-11-16T16:38:48.370 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.356+0000 7f57a359e700 1 -- 172.21.15.1:0/452035796 learned_addr learned my addr 172.21.15.1:0/452035796 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:48.370 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.356+0000 7f57a2d9d700 1 --2- 172.21.15.1:0/452035796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a966dcbb0 0x564a966dcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.371 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.356+0000 7f57a259c700 1 --2- 172.21.15.1:0/452035796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a966dd500 0x564a966ddae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.371 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.356+0000 7f57a359e700 1 -- 172.21.15.1:0/452035796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a966dcbb0 msgr2=0x564a966dcfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:48.371 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.356+0000 7f57a359e700 1 --2- 172.21.15.1:0/452035796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a966dcbb0 0x564a966dcfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.371 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.356+0000 7f57a359e700 1 -- 172.21.15.1:0/452035796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a966dd500 msgr2=0x564a966ddae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:48.371 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.356+0000 7f57a359e700 1 --2- 172.21.15.1:0/452035796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a966dd500 0x564a966ddae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.371 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.356+0000 7f57a359e700 1 -- 172.21.15.1:0/452035796 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a964af5e0 con 0x564a966de0c0 2023-11-16T16:38:48.371 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.356+0000 7f57a259c700 1 --2- 172.21.15.1:0/452035796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a966dd500 0x564a966ddae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:38:48.371 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.356+0000 7f57a359e700 1 --2- 172.21.15.1:0/452035796 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a966de0c0 0x564a966de530 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f579c015000 tx=0x7f579c009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d65a91ac4ebccdba server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:48.371 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.356+0000 7f57897fa700 1 -- 172.21.15.1:0/452035796 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c00e050 con 0x564a966de0c0 2023-11-16T16:38:48.372 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.356+0000 7f57897fa700 1 -- 172.21.15.1:0/452035796 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f579c009460 con 0x564a966de0c0 2023-11-16T16:38:48.372 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.356+0000 7f57897fa700 1 -- 172.21.15.1:0/452035796 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c005040 con 0x564a966de0c0 2023-11-16T16:38:48.372 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.357+0000 7f57b5305b80 1 -- 172.21.15.1:0/452035796 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a966de0c0 msgr2=0x564a966de530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:48.372 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.357+0000 7f57b5305b80 1 --2- 172.21.15.1:0/452035796 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a966de0c0 0x564a966de530 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f579c015000 tx=0x7f579c009c60 comp rx=0 tx=0).stop 2023-11-16T16:38:48.372 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.357+0000 7f57b5305b80 1 -- 172.21.15.1:0/452035796 shutdown_connections 2023-11-16T16:38:48.372 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.357+0000 7f57b5305b80 1 --2- 172.21.15.1:0/452035796 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a966de0c0 0x564a966de530 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.372 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.357+0000 7f57b5305b80 1 --2- 172.21.15.1:0/452035796 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a966dcbb0 0x564a966dcfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.372 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.357+0000 7f57b5305b80 1 --2- 172.21.15.1:0/452035796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a966dd500 0x564a966ddae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.372 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.357+0000 7f57b5305b80 1 -- 172.21.15.1:0/452035796 >> 172.21.15.1:0/452035796 conn(0x564a966d2310 msgr2=0x564a966d7620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:48.373 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.357+0000 7f57b5305b80 1 -- 172.21.15.1:0/452035796 shutdown_connections 2023-11-16T16:38:48.373 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.357+0000 7f57b5305b80 1 -- 172.21.15.1:0/452035796 wait complete. 2023-11-16T16:38:48.373 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.357+0000 7f57b5305b80 1 -- start start 2023-11-16T16:38:48.373 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.357+0000 7f57b5305b80 1 Processor -- start 2023-11-16T16:38:48.373 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.358+0000 7f57b5305b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a966dcbb0 0x564a966e64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.373 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.358+0000 7f57b5305b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a966dd500 0x564a966e6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.373 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.358+0000 7f57b5305b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a966de0c0 0x564a966eda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.373 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.358+0000 7f57b5305b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a965cf890 con 0x564a966de0c0 2023-11-16T16:38:48.374 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.358+0000 7f57b5305b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a965cf530 con 0x564a966dcbb0 2023-11-16T16:38:48.374 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.358+0000 7f57b5305b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a964b18d0 con 0x564a966dd500 2023-11-16T16:38:48.374 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.358+0000 7f57b5305b80 10 client.0 _ll_register_callbacks cb 0x564a965ed730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:38:48.374 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.358+0000 7f57a359e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a966de0c0 0x564a966eda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.374 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.358+0000 7f57a359e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a966de0c0 0x564a966eda60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42750/0 (socket says 172.21.15.1:42750) 2023-11-16T16:38:48.374 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.358+0000 7f57a359e700 1 -- 172.21.15.1:0/1651538780 learned_addr learned my addr 172.21.15.1:0/1651538780 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:48.374 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.358+0000 7f57a259c700 1 --2- 172.21.15.1:0/1651538780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a966dd500 0x564a966e6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.374 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.358+0000 7f57a359e700 1 -- 172.21.15.1:0/1651538780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a966dd500 msgr2=0x564a966e6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:48.375 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.358+0000 7f57a2d9d700 1 --2- 172.21.15.1:0/1651538780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a966dcbb0 0x564a966e64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.375 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.358+0000 7f57a359e700 1 --2- 172.21.15.1:0/1651538780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a966dd500 0x564a966e6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.375 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.358+0000 7f57a359e700 1 -- 172.21.15.1:0/1651538780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a966dcbb0 msgr2=0x564a966e64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:48.375 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.358+0000 7f57a359e700 1 --2- 172.21.15.1:0/1651538780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a966dcbb0 0x564a966e64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.375 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.358+0000 7f57a359e700 1 -- 172.21.15.1:0/1651538780 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a4108fa0 con 0x564a966de0c0 2023-11-16T16:38:48.375 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.358+0000 7f57a259c700 1 --2- 172.21.15.1:0/1651538780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a966dd500 0x564a966e6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:38:48.375 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.358+0000 7f57a359e700 1 --2- 172.21.15.1:0/1651538780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a966de0c0 0x564a966eda60 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f579c0079f0 tx=0x7f579c00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59eb442861e3d00a server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:48.375 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.358+0000 7f5786ff5700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:38:48.376 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.359+0000 7f5786ff5700 1 -- 172.21.15.1:0/1651538780 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c01b840 con 0x564a966de0c0 2023-11-16T16:38:48.376 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.359+0000 7f5786ff5700 1 -- 172.21.15.1:0/1651538780 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f579c007670 con 0x564a966de0c0 2023-11-16T16:38:48.376 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.359+0000 7f57b5305b80 1 -- 172.21.15.1:0/1651538780 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a966cfad0 con 0x564a966de0c0 2023-11-16T16:38:48.376 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.359+0000 7f5786ff5700 1 -- 172.21.15.1:0/1651538780 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f579c01b9a0 con 0x564a966de0c0 2023-11-16T16:38:48.376 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.359+0000 7f57b5305b80 10 client.15240 fetch_fsmap learned FSMap version 23 2023-11-16T16:38:48.376 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.359+0000 7f57b5305b80 1 -- 172.21.15.1:0/1651538780 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x564a965f4140 con 0x564a966de0c0 2023-11-16T16:38:48.376 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.359+0000 7f5786ff5700 1 -- 172.21.15.1:0/1651538780 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564a966cfad0 con 0x564a966de0c0 2023-11-16T16:38:48.376 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.359+0000 7f5786ff5700 1 -- 172.21.15.1:0/1651538780 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f579c00b0d0 con 0x564a966de0c0 2023-11-16T16:38:48.377 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.359+0000 7f57b5305b80 10 client.15240 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:38:48.377 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.359+0000 7f57b5305b80 10 client.15240 resolve_mds: resolved name 'i' to daemon mds.i 2023-11-16T16:38:48.377 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.359+0000 7f57b5305b80 20 client.15240 populate_metadata read hostname 'smithi001' 2023-11-16T16:38:48.377 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.359+0000 7f57b5305b80 1 --2- 172.21.15.1:0/1651538780 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x564a966f1250 0x564a966f0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.377 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.359+0000 7f57b5305b80 4 client.15240 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:38:48.377 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.359+0000 7f57b5305b80 1 -- 172.21.15.1:0/1651538780 --> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a965f0050 con 0x564a966f1250 2023-11-16T16:38:48.377 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.360+0000 7f57a2d9d700 1 --2- 172.21.15.1:0/1651538780 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x564a966f1250 0x564a966f0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.377 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.360+0000 7f57a2d9d700 1 --2- 172.21.15.1:0/1651538780 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x564a966f1250 0x564a966f0990 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1c55b64c0d0ff4af server_cookie=9a3b9d8b96dc5692 in_seq=0 out_seq=0 2023-11-16T16:38:48.378 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.360+0000 7f5786ff5700 10 client.15240 ms_handle_connect on v2:172.21.15.53:6838/1044753024 2023-11-16T16:38:48.378 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.361+0000 7f5786ff5700 1 -- 172.21.15.1:0/1651538780 <== mds.1 v2:172.21.15.53:6838/1044753024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564a965f0050 con 0x564a966f1250 2023-11-16T16:38:48.378 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.361+0000 7f5786ff5700 10 client.15240 handle_command_reply: tid=0 2023-11-16T16:38:48.378 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.361+0000 7f57b5305b80 1 client.15240 shutdown 2023-11-16T16:38:48.378 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.361+0000 7f57b5305b80 1 -- 172.21.15.1:0/1651538780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a966de0c0 msgr2=0x564a966eda60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:48.378 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.361+0000 7f57b5305b80 1 --2- 172.21.15.1:0/1651538780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a966de0c0 0x564a966eda60 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f579c0079f0 tx=0x7f579c00bd30 comp rx=0 tx=0).stop 2023-11-16T16:38:48.378 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.361+0000 7f57b5305b80 1 -- 172.21.15.1:0/1651538780 shutdown_connections 2023-11-16T16:38:48.378 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.361+0000 7f57b5305b80 1 --2- 172.21.15.1:0/1651538780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a966de0c0 0x564a966eda60 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.378 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.361+0000 7f57b5305b80 1 --2- 172.21.15.1:0/1651538780 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x564a966f1250 0x564a966f0990 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.379 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.361+0000 7f57b5305b80 1 --2- 172.21.15.1:0/1651538780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a966dcbb0 0x564a966e64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.379 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.361+0000 7f5786ff5700 0 client.15240 ms_handle_reset on v2:172.21.15.53:6838/1044753024 2023-11-16T16:38:48.379 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.361+0000 7f57b5305b80 1 --2- 172.21.15.1:0/1651538780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a966dd500 0x564a966e6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.379 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.361+0000 7f57b5305b80 1 -- 172.21.15.1:0/1651538780 >> 172.21.15.1:0/1651538780 conn(0x564a966d2310 msgr2=0x564a966d6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:48.379 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.361+0000 7f57b5305b80 1 -- 172.21.15.1:0/1651538780 shutdown_connections 2023-11-16T16:38:48.379 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.361+0000 7f57b5305b80 1 -- 172.21.15.1:0/1651538780 wait complete. 2023-11-16T16:38:48.379 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.361+0000 7f57b5305b80 20 client.15240 trim_cache size 0 max 16384 2023-11-16T16:38:48.388 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.379+0000 7f57b5305b80 1 Processor -- start 2023-11-16T16:38:48.388 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.379+0000 7f57b5305b80 1 -- start start 2023-11-16T16:38:48.388 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.379+0000 7f57b5305b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a966dcbb0 0x564a966f0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.389 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.379+0000 7f57b5305b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a966dd500 0x564a966ed980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.389 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.379+0000 7f57b5305b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a966de0c0 0x564a966f9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.389 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.379+0000 7f57b5305b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a965f0050 con 0x564a966dcbb0 2023-11-16T16:38:48.389 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.379+0000 7f57b5305b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a966d2120 con 0x564a966de0c0 2023-11-16T16:38:48.389 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.379+0000 7f57b5305b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a96539ed0 con 0x564a966dd500 2023-11-16T16:38:48.389 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.379+0000 7f57a2d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a966dcbb0 0x564a966f0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.389 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.380+0000 7f57a2d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a966dcbb0 0x564a966f0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42754/0 (socket says 172.21.15.1:42754) 2023-11-16T16:38:48.389 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.380+0000 7f57a2d9d700 1 -- 172.21.15.1:0/1386012292 learned_addr learned my addr 172.21.15.1:0/1386012292 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:48.389 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.380+0000 7f57a359e700 1 --2- 172.21.15.1:0/1386012292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a966de0c0 0x564a966f9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.390 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.380+0000 7f57a259c700 1 --2- 172.21.15.1:0/1386012292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a966dd500 0x564a966ed980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.390 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.380+0000 7f57a2d9d700 1 -- 172.21.15.1:0/1386012292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a966dd500 msgr2=0x564a966ed980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:48.390 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.380+0000 7f57a2d9d700 1 --2- 172.21.15.1:0/1386012292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a966dd500 0x564a966ed980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.390 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.380+0000 7f57a2d9d700 1 -- 172.21.15.1:0/1386012292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a966de0c0 msgr2=0x564a966f9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:48.390 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.380+0000 7f57a2d9d700 1 --2- 172.21.15.1:0/1386012292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a966de0c0 0x564a966f9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.390 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.380+0000 7f57a2d9d700 1 -- 172.21.15.1:0/1386012292 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a965f3e90 con 0x564a966dcbb0 2023-11-16T16:38:48.390 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.380+0000 7f57a359e700 1 --2- 172.21.15.1:0/1386012292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a966de0c0 0x564a966f9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:38:48.390 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.380+0000 7f57a2d9d700 1 --2- 172.21.15.1:0/1386012292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a966dcbb0 0x564a966f0990 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5798000c00 tx=0x7f57980ca070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca32d510f5bca9cf server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:48.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.380+0000 7f5789ffb700 1 -- 172.21.15.1:0/1386012292 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579809dae0 con 0x564a966dcbb0 2023-11-16T16:38:48.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.380+0000 7f5789ffb700 1 -- 172.21.15.1:0/1386012292 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57980a1f00 con 0x564a966dcbb0 2023-11-16T16:38:48.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.380+0000 7f5789ffb700 1 -- 172.21.15.1:0/1386012292 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5798099200 con 0x564a966dcbb0 2023-11-16T16:38:48.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.381+0000 7f57b5305b80 1 -- 172.21.15.1:0/1386012292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a966dcbb0 msgr2=0x564a966f0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:48.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.381+0000 7f57b5305b80 1 --2- 172.21.15.1:0/1386012292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a966dcbb0 0x564a966f0990 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5798000c00 tx=0x7f57980ca070 comp rx=0 tx=0).stop 2023-11-16T16:38:48.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.381+0000 7f57b5305b80 1 -- 172.21.15.1:0/1386012292 shutdown_connections 2023-11-16T16:38:48.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.381+0000 7f57b5305b80 1 --2- 172.21.15.1:0/1386012292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a966de0c0 0x564a966f9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.381+0000 7f57b5305b80 1 --2- 172.21.15.1:0/1386012292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a966dcbb0 0x564a966f0990 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.381+0000 7f57b5305b80 1 --2- 172.21.15.1:0/1386012292 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a966dd500 0x564a966ed980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.381+0000 7f57b5305b80 1 -- 172.21.15.1:0/1386012292 >> 172.21.15.1:0/1386012292 conn(0x564a966d2310 msgr2=0x564a966d75f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:48.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.381+0000 7f57b5305b80 1 -- 172.21.15.1:0/1386012292 shutdown_connections 2023-11-16T16:38:48.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.381+0000 7f57b5305b80 1 -- 172.21.15.1:0/1386012292 wait complete. 2023-11-16T16:38:48.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.381+0000 7f57b5305b80 1 -- start start 2023-11-16T16:38:48.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.381+0000 7f57b5305b80 1 Processor -- start 2023-11-16T16:38:48.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.381+0000 7f57b5305b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a966dcbb0 0x564a966e8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.381+0000 7f57b5305b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a966dd500 0x564a966e8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.381+0000 7f57b5305b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a966de0c0 0x564a967450f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.381+0000 7f57b5305b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a966e9310 con 0x564a966dcbb0 2023-11-16T16:38:48.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.381+0000 7f57b5305b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a966e9480 con 0x564a966de0c0 2023-11-16T16:38:48.393 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.381+0000 7f57b5305b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a96745630 con 0x564a966dd500 2023-11-16T16:38:48.393 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.381+0000 7f57b5305b80 10 client.0 _ll_register_callbacks cb 0x7f5798095790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:38:48.393 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.381+0000 7f57a2d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a966dcbb0 0x564a966e8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.393 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.381+0000 7f57a2d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a966dcbb0 0x564a966e8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42760/0 (socket says 172.21.15.1:42760) 2023-11-16T16:38:48.393 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.381+0000 7f57a2d9d700 1 -- 172.21.15.1:0/2359608115 learned_addr learned my addr 172.21.15.1:0/2359608115 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:48.393 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.382+0000 7f57a359e700 1 --2- 172.21.15.1:0/2359608115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a966de0c0 0x564a967450f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.393 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.382+0000 7f57a259c700 1 --2- 172.21.15.1:0/2359608115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a966dd500 0x564a966e8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.393 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.382+0000 7f57a2d9d700 1 -- 172.21.15.1:0/2359608115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a966dd500 msgr2=0x564a966e8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:48.393 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.382+0000 7f57a2d9d700 1 --2- 172.21.15.1:0/2359608115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a966dd500 0x564a966e8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.382+0000 7f57a2d9d700 1 -- 172.21.15.1:0/2359608115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a966de0c0 msgr2=0x564a967450f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:48.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.382+0000 7f57a2d9d700 1 --2- 172.21.15.1:0/2359608115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a966de0c0 0x564a967450f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.382+0000 7f57a2d9d700 1 -- 172.21.15.1:0/2359608115 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a410ba10 con 0x564a966dcbb0 2023-11-16T16:38:48.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.382+0000 7f57a259c700 1 --2- 172.21.15.1:0/2359608115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a966dd500 0x564a966e8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:38:48.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.382+0000 7f57a359e700 1 --2- 172.21.15.1:0/2359608115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a966de0c0 0x564a967450f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:38:48.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.382+0000 7f57a2d9d700 1 --2- 172.21.15.1:0/2359608115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a966dcbb0 0x564a966e8890 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f57980980a0 tx=0x7f5798097e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31df27b89bd59b49 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:48.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.382+0000 7f5786ff5700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:38:48.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.382+0000 7f5786ff5700 1 -- 172.21.15.1:0/2359608115 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5798097bf0 con 0x564a966dcbb0 2023-11-16T16:38:48.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.382+0000 7f5786ff5700 1 -- 172.21.15.1:0/2359608115 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f579809dae0 con 0x564a966dcbb0 2023-11-16T16:38:48.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.382+0000 7f57b5305b80 1 -- 172.21.15.1:0/2359608115 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f579c009460 con 0x564a966dcbb0 2023-11-16T16:38:48.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.382+0000 7f5786ff5700 1 -- 172.21.15.1:0/2359608115 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57980a1a70 con 0x564a966dcbb0 2023-11-16T16:38:48.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.382+0000 7f57b5305b80 10 client.15246 fetch_fsmap learned FSMap version 23 2023-11-16T16:38:48.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.382+0000 7f57b5305b80 1 -- 172.21.15.1:0/2359608115 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x564a96444c10 con 0x564a966dcbb0 2023-11-16T16:38:48.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.382+0000 7f5786ff5700 1 -- 172.21.15.1:0/2359608115 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f579807c080 con 0x564a966dcbb0 2023-11-16T16:38:48.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.382+0000 7f5786ff5700 1 -- 172.21.15.1:0/2359608115 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5798084400 con 0x564a966dcbb0 2023-11-16T16:38:48.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.382+0000 7f57b5305b80 10 client.15246 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:38:48.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.382+0000 7f57b5305b80 10 client.15246 resolve_mds: resolved name 'i' to daemon mds.i 2023-11-16T16:38:48.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.382+0000 7f57b5305b80 20 client.15246 populate_metadata read hostname 'smithi001' 2023-11-16T16:38:48.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.383+0000 7f57b5305b80 1 --2- 172.21.15.1:0/2359608115 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x564a9674a0a0 0x564a96746a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.383+0000 7f57b5305b80 4 client.15246 mds_command: new command op to 24493 tid=0[{"prefix": "status"}] 2023-11-16T16:38:48.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.383+0000 7f57b5305b80 1 -- 172.21.15.1:0/2359608115 --> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564a965fc610 con 0x564a9674a0a0 2023-11-16T16:38:48.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.383+0000 7f57a259c700 1 --2- 172.21.15.1:0/2359608115 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x564a9674a0a0 0x564a96746a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.383+0000 7f5786ff5700 10 client.15246 ms_handle_connect on v2:172.21.15.53:6838/1044753024 2023-11-16T16:38:48.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.383+0000 7f57a259c700 1 --2- 172.21.15.1:0/2359608115 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x564a9674a0a0 0x564a96746a30 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=73a6ba61bf93c534 server_cookie=c027a9ddd766c81e in_seq=0 out_seq=0 2023-11-16T16:38:48.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.383+0000 7f5786ff5700 1 -- 172.21.15.1:0/2359608115 <== mds.1 v2:172.21.15.53:6838/1044753024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x564a965fc610 con 0x564a9674a0a0 2023-11-16T16:38:48.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.383+0000 7f5786ff5700 10 client.15246 handle_command_reply: tid=0 2023-11-16T16:38:48.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.383+0000 7f57b5305b80 1 client.15246 shutdown 2023-11-16T16:38:48.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.384+0000 7f57b5305b80 1 -- 172.21.15.1:0/2359608115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a966dcbb0 msgr2=0x564a966e8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:48.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.384+0000 7f57b5305b80 1 --2- 172.21.15.1:0/2359608115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a966dcbb0 0x564a966e8890 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f57980980a0 tx=0x7f5798097e50 comp rx=0 tx=0).stop 2023-11-16T16:38:48.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.384+0000 7f57b5305b80 1 -- 172.21.15.1:0/2359608115 shutdown_connections 2023-11-16T16:38:48.397 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.384+0000 7f57b5305b80 1 --2- 172.21.15.1:0/2359608115 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x564a9674a0a0 0x564a96746a30 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.397 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.384+0000 7f57b5305b80 1 --2- 172.21.15.1:0/2359608115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a966de0c0 0x564a967450f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.397 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.384+0000 7f57b5305b80 1 --2- 172.21.15.1:0/2359608115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a966dcbb0 0x564a966e8890 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.397 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.384+0000 7f57b5305b80 1 --2- 172.21.15.1:0/2359608115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a966dd500 0x564a966e8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.397 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.384+0000 7f5786ff5700 0 client.15246 ms_handle_reset on v2:172.21.15.53:6838/1044753024 2023-11-16T16:38:48.397 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.384+0000 7f57b5305b80 1 -- 172.21.15.1:0/2359608115 >> 172.21.15.1:0/2359608115 conn(0x564a966d2310 msgr2=0x564a966f9fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:48.397 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.384+0000 7f57b5305b80 1 -- 172.21.15.1:0/2359608115 shutdown_connections 2023-11-16T16:38:48.397 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.384+0000 7f57b5305b80 1 -- 172.21.15.1:0/2359608115 wait complete. 2023-11-16T16:38:48.397 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.384+0000 7f57b5305b80 20 client.15246 trim_cache size 0 max 16384 2023-11-16T16:38:48.398 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:38:48.398 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "35d9b010-849d-11ee-8dbe-212e2dc638e7", 2023-11-16T16:38:48.398 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 1, 2023-11-16T16:38:48.398 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24493, 2023-11-16T16:38:48.398 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2023-11-16T16:38:48.398 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2023-11-16T16:38:48.398 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2023-11-16T16:38:48.399 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 91.108051381999999, 2023-11-16T16:38:48.399 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 23, 2023-11-16T16:38:48.399 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 86, 2023-11-16T16:38:48.399 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 84, 2023-11-16T16:38:48.399 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 106.085455466 2023-11-16T16:38:48.399 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:38:48.400 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.384+0000 7f57a9179700 1 -- 172.21.15.1:0/3486272305 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f578c0a1150 msgr2=0x7f578c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:48.400 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.384+0000 7f57a9179700 1 --2- 172.21.15.1:0/3486272305 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f578c0a1150 0x7f578c0a3600 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5794009cb0 tx=0x7f5794009260 comp rx=0 tx=0).stop 2023-11-16T16:38:48.400 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.384+0000 7f57a9179700 1 -- 172.21.15.1:0/3486272305 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57a4100230 msgr2=0x7f57a4196050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:48.400 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.384+0000 7f57a9179700 1 --2- 172.21.15.1:0/3486272305 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57a4100230 0x7f57a4196050 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f579800d980 tx=0x7f579800dd40 comp rx=0 tx=0).stop 2023-11-16T16:38:48.400 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.384+0000 7f57a9179700 1 -- 172.21.15.1:0/3486272305 shutdown_connections 2023-11-16T16:38:48.401 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.384+0000 7f57a9179700 1 --2- 172.21.15.1:0/3486272305 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f578c0a1150 0x7f578c0a3600 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.401 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.384+0000 7f57a9179700 1 --2- 172.21.15.1:0/3486272305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57a4101db0 0x7f57a419d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.401 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.384+0000 7f57a9179700 1 --2- 172.21.15.1:0/3486272305 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57a4100230 0x7f57a4196050 secure :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f579800d980 tx=0x7f579800dd40 comp rx=0 tx=0).stop 2023-11-16T16:38:48.401 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.384+0000 7f57a9179700 1 --2- 172.21.15.1:0/3486272305 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f57a4101420 0x7f57a4196590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.401 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.384+0000 7f57a9179700 1 -- 172.21.15.1:0/3486272305 >> 172.21.15.1:0/3486272305 conn(0x7f57a40fb810 msgr2=0x7f57a4104860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:48.401 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.384+0000 7f57a9179700 1 -- 172.21.15.1:0/3486272305 shutdown_connections 2023-11-16T16:38:48.401 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.384+0000 7f57a9179700 1 -- 172.21.15.1:0/3486272305 wait complete. 2023-11-16T16:38:48.401 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:48 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1434591359' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:38:48.402 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:48 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3252609596' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:38:48.402 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:48 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/842505632' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:38:48.403 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-16T16:38:48.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:48 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1434591359' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:38:48.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:48 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/3252609596' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:38:48.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:48 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/842505632' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:38:48.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.528+0000 7faca7952700 1 -- 172.21.15.1:0/3393376812 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faca00734d0 msgr2=0x7faca0073940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:48.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.528+0000 7faca7952700 1 --2- 172.21.15.1:0/3393376812 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faca00734d0 0x7faca0073940 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fac90009990 tx=0x7fac90009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.528+0000 7faca7952700 1 -- 172.21.15.1:0/3393376812 shutdown_connections 2023-11-16T16:38:48.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.528+0000 7faca7952700 1 --2- 172.21.15.1:0/3393376812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faca01080e0 0x7faca010a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.528+0000 7faca7952700 1 --2- 172.21.15.1:0/3393376812 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faca0074ab0 0x7faca0072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.528+0000 7faca7952700 1 --2- 172.21.15.1:0/3393376812 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faca00734d0 0x7faca0073940 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.528+0000 7faca7952700 1 -- 172.21.15.1:0/3393376812 >> 172.21.15.1:0/3393376812 conn(0x7faca00fb810 msgr2=0x7faca00fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:48.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.528+0000 7faca7952700 1 -- 172.21.15.1:0/3393376812 shutdown_connections 2023-11-16T16:38:48.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.528+0000 7faca7952700 1 -- 172.21.15.1:0/3393376812 wait complete. 2023-11-16T16:38:48.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.529+0000 7faca7952700 1 Processor -- start 2023-11-16T16:38:48.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.529+0000 7faca7952700 1 -- start start 2023-11-16T16:38:48.535 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.529+0000 7faca7952700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faca00734d0 0x7faca0198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.535 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.529+0000 7faca7952700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faca0074ab0 0x7faca0198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.535 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.529+0000 7faca7952700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faca01080e0 0x7faca019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.535 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.529+0000 7faca7952700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca019fca0 con 0x7faca00734d0 2023-11-16T16:38:48.535 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.529+0000 7faca7952700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca019fde0 con 0x7faca01080e0 2023-11-16T16:38:48.535 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.529+0000 7faca7952700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca019ff70 con 0x7faca0074ab0 2023-11-16T16:38:48.535 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.529+0000 7faca56ee700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faca00734d0 0x7faca0198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.536 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.529+0000 7faca56ee700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faca00734d0 0x7faca0198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42774/0 (socket says 172.21.15.1:42774) 2023-11-16T16:38:48.536 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.529+0000 7faca56ee700 1 -- 172.21.15.1:0/2728870921 learned_addr learned my addr 172.21.15.1:0/2728870921 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:48.536 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.529+0000 7faca4eed700 1 --2- 172.21.15.1:0/2728870921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faca0074ab0 0x7faca0198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.536 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.529+0000 7faca5eef700 1 --2- 172.21.15.1:0/2728870921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faca01080e0 0x7faca019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.536 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.529+0000 7faca56ee700 1 -- 172.21.15.1:0/2728870921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faca0074ab0 msgr2=0x7faca0198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:48.536 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.529+0000 7faca56ee700 1 --2- 172.21.15.1:0/2728870921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faca0074ab0 0x7faca0198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.536 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.529+0000 7faca56ee700 1 -- 172.21.15.1:0/2728870921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faca01080e0 msgr2=0x7faca019f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:48.536 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.529+0000 7faca56ee700 1 --2- 172.21.15.1:0/2728870921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faca01080e0 0x7faca019f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.529+0000 7faca56ee700 1 -- 172.21.15.1:0/2728870921 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac900095f0 con 0x7faca00734d0 2023-11-16T16:38:48.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.530+0000 7faca56ee700 1 --2- 172.21.15.1:0/2728870921 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faca00734d0 0x7faca0198120 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fac9c00e9a0 tx=0x7fac9c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:48.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.530+0000 7fac967fc700 1 -- 172.21.15.1:0/2728870921 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c009790 con 0x7faca00734d0 2023-11-16T16:38:48.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.530+0000 7faca7952700 1 -- 172.21.15.1:0/2728870921 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faca010dd70 con 0x7faca00734d0 2023-11-16T16:38:48.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.530+0000 7fac967fc700 1 -- 172.21.15.1:0/2728870921 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac9c010430 con 0x7faca00734d0 2023-11-16T16:38:48.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.530+0000 7fac967fc700 1 -- 172.21.15.1:0/2728870921 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c018680 con 0x7faca00734d0 2023-11-16T16:38:48.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.530+0000 7faca7952700 1 -- 172.21.15.1:0/2728870921 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca010dbe0 con 0x7faca00734d0 2023-11-16T16:38:48.546 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.532+0000 7fac967fc700 1 -- 172.21.15.1:0/2728870921 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fac9c0105a0 con 0x7faca00734d0 2023-11-16T16:38:48.546 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.533+0000 7fac967fc700 1 --2- 172.21.15.1:0/2728870921 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fac8c0a1150 0x7fac8c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.546 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.534+0000 7faca4eed700 1 --2- 172.21.15.1:0/2728870921 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fac8c0a1150 0x7fac8c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.547 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.534+0000 7fac967fc700 1 -- 172.21.15.1:0/2728870921 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fac9c014070 con 0x7faca00734d0 2023-11-16T16:38:48.547 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.534+0000 7facb3adeb80 1 Processor -- start 2023-11-16T16:38:48.547 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.534+0000 7facb3adeb80 1 -- start start 2023-11-16T16:38:48.547 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.534+0000 7faca4eed700 1 --2- 172.21.15.1:0/2728870921 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fac8c0a1150 0x7fac8c0a3600 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fac9000b2e0 tx=0x7fac90005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:48.547 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.534+0000 7facb3adeb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd09328bb0 0x55fd09328fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.547 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.534+0000 7facb3adeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fd09329500 0x55fd09329ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.547 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.534+0000 7facb3adeb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd0932a0c0 0x55fd0932a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.547 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.534+0000 7facb3adeb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd0922e010 con 0x55fd09328bb0 2023-11-16T16:38:48.547 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.534+0000 7facb3adeb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd09280850 con 0x55fd0932a0c0 2023-11-16T16:38:48.548 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.534+0000 7facb3adeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd09279ce0 con 0x55fd09329500 2023-11-16T16:38:48.548 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.534+0000 7faca56ee700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd09328bb0 0x55fd09328fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.548 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.534+0000 7faca56ee700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd09328bb0 0x55fd09328fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42782/0 (socket says 172.21.15.1:42782) 2023-11-16T16:38:48.548 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.534+0000 7faca56ee700 1 -- 172.21.15.1:0/3321457662 learned_addr learned my addr 172.21.15.1:0/3321457662 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:48.548 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.534+0000 7faca4eed700 1 --2- 172.21.15.1:0/3321457662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fd09329500 0x55fd09329ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.548 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.535+0000 7faca5eef700 1 --2- 172.21.15.1:0/3321457662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd0932a0c0 0x55fd0932a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.548 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.535+0000 7faca56ee700 1 -- 172.21.15.1:0/3321457662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fd09329500 msgr2=0x55fd09329ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:48.548 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.535+0000 7faca56ee700 1 --2- 172.21.15.1:0/3321457662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fd09329500 0x55fd09329ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.549 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.535+0000 7faca56ee700 1 -- 172.21.15.1:0/3321457662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd0932a0c0 msgr2=0x55fd0932a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:48.549 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.535+0000 7faca56ee700 1 --2- 172.21.15.1:0/3321457662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd0932a0c0 0x55fd0932a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.549 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.535+0000 7faca56ee700 1 -- 172.21.15.1:0/3321457662 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd090fb5e0 con 0x55fd09328bb0 2023-11-16T16:38:48.549 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.535+0000 7faca4eed700 1 --2- 172.21.15.1:0/3321457662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fd09329500 0x55fd09329ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:38:48.549 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.535+0000 7faca56ee700 1 --2- 172.21.15.1:0/3321457662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd09328bb0 0x55fd09328fc0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fac9c099e70 tx=0x7fac9c099940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da063b9bb6517f39 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:48.549 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.535+0000 7fac8b7fe700 1 -- 172.21.15.1:0/3321457662 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c0a2330 con 0x55fd09328bb0 2023-11-16T16:38:48.549 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.535+0000 7fac8b7fe700 1 -- 172.21.15.1:0/3321457662 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac9c098280 con 0x55fd09328bb0 2023-11-16T16:38:48.549 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.535+0000 7fac8b7fe700 1 -- 172.21.15.1:0/3321457662 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c0a2100 con 0x55fd09328bb0 2023-11-16T16:38:48.549 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.536+0000 7facb3adeb80 1 -- 172.21.15.1:0/3321457662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd09328bb0 msgr2=0x55fd09328fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:48.550 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.536+0000 7facb3adeb80 1 --2- 172.21.15.1:0/3321457662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd09328bb0 0x55fd09328fc0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fac9c099e70 tx=0x7fac9c099940 comp rx=0 tx=0).stop 2023-11-16T16:38:48.550 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.536+0000 7facb3adeb80 1 -- 172.21.15.1:0/3321457662 shutdown_connections 2023-11-16T16:38:48.550 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.536+0000 7facb3adeb80 1 --2- 172.21.15.1:0/3321457662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd0932a0c0 0x55fd0932a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.550 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.536+0000 7facb3adeb80 1 --2- 172.21.15.1:0/3321457662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd09328bb0 0x55fd09328fc0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.550 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.536+0000 7facb3adeb80 1 --2- 172.21.15.1:0/3321457662 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fd09329500 0x55fd09329ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.550 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.536+0000 7facb3adeb80 1 -- 172.21.15.1:0/3321457662 >> 172.21.15.1:0/3321457662 conn(0x55fd0931e310 msgr2=0x55fd09323620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:48.550 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.536+0000 7facb3adeb80 1 -- 172.21.15.1:0/3321457662 shutdown_connections 2023-11-16T16:38:48.551 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.536+0000 7facb3adeb80 1 -- 172.21.15.1:0/3321457662 wait complete. 2023-11-16T16:38:48.551 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.536+0000 7facb3adeb80 1 -- start start 2023-11-16T16:38:48.551 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.536+0000 7facb3adeb80 1 Processor -- start 2023-11-16T16:38:48.551 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.537+0000 7facb3adeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fd09328bb0 0x55fd09332600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.551 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.537+0000 7facb3adeb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd09329500 0x55fd09332b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.551 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.537+0000 7facb3adeb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd0932a0c0 0x55fd09339b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.551 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.537+0000 7facb3adeb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd0921b890 con 0x55fd0932a0c0 2023-11-16T16:38:48.551 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.537+0000 7facb3adeb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd0921b530 con 0x55fd09329500 2023-11-16T16:38:48.551 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.537+0000 7facb3adeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd090fd8d0 con 0x55fd09328bb0 2023-11-16T16:38:48.552 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.537+0000 7facb3adeb80 10 client.0 _ll_register_callbacks cb 0x55fd09239730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:38:48.552 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.537+0000 7faca5eef700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd0932a0c0 0x55fd09339b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.552 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.537+0000 7faca5eef700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd0932a0c0 0x55fd09339b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42792/0 (socket says 172.21.15.1:42792) 2023-11-16T16:38:48.552 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.537+0000 7faca56ee700 1 --2- 172.21.15.1:0/3261151360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fd09328bb0 0x55fd09332600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.552 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.537+0000 7faca5eef700 1 -- 172.21.15.1:0/3261151360 learned_addr learned my addr 172.21.15.1:0/3261151360 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:48.552 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.537+0000 7faca4eed700 1 --2- 172.21.15.1:0/3261151360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd09329500 0x55fd09332b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.552 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.537+0000 7faca5eef700 1 -- 172.21.15.1:0/3261151360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fd09328bb0 msgr2=0x55fd09332600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:48.552 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.537+0000 7faca5eef700 1 --2- 172.21.15.1:0/3261151360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fd09328bb0 0x55fd09332600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.553 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.537+0000 7faca5eef700 1 -- 172.21.15.1:0/3261151360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd09329500 msgr2=0x55fd09332b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:48.553 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.537+0000 7faca5eef700 1 --2- 172.21.15.1:0/3261151360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd09329500 0x55fd09332b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.553 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.537+0000 7faca5eef700 1 -- 172.21.15.1:0/3261151360 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca010dbe0 con 0x55fd0932a0c0 2023-11-16T16:38:48.553 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.538+0000 7faca5eef700 1 --2- 172.21.15.1:0/3261151360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd0932a0c0 0x55fd09339b80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fac98007eb0 tx=0x7fac98007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf0cbd06de6c9de5 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:48.553 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.538+0000 7fac88ff9700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:38:48.553 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.538+0000 7fac88ff9700 1 -- 172.21.15.1:0/3261151360 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9800e040 con 0x55fd0932a0c0 2023-11-16T16:38:48.553 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.538+0000 7fac88ff9700 1 -- 172.21.15.1:0/3261151360 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fac980138c0 con 0x55fd0932a0c0 2023-11-16T16:38:48.553 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.538+0000 7facb3adeb80 1 -- 172.21.15.1:0/3261151360 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd0931bad0 con 0x55fd0932a0c0 2023-11-16T16:38:48.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.538+0000 7fac88ff9700 1 -- 172.21.15.1:0/3261151360 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac98013a80 con 0x55fd0932a0c0 2023-11-16T16:38:48.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.538+0000 7facb3adeb80 10 client.15258 fetch_fsmap learned FSMap version 23 2023-11-16T16:38:48.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.538+0000 7facb3adeb80 1 -- 172.21.15.1:0/3261151360 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55fd09240140 con 0x55fd0932a0c0 2023-11-16T16:38:48.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.538+0000 7fac88ff9700 1 -- 172.21.15.1:0/3261151360 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fd0931bad0 con 0x55fd0932a0c0 2023-11-16T16:38:48.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.538+0000 7fac88ff9700 1 -- 172.21.15.1:0/3261151360 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fac98015390 con 0x55fd0932a0c0 2023-11-16T16:38:48.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.538+0000 7facb3adeb80 10 client.15258 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:38:48.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.538+0000 7facb3adeb80 10 client.15258 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-16T16:38:48.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.538+0000 7facb3adeb80 20 client.15258 populate_metadata read hostname 'smithi001' 2023-11-16T16:38:48.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.538+0000 7facb3adeb80 1 --2- 172.21.15.1:0/3261151360 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55fd0933d900 0x55fd0933cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.538+0000 7facb3adeb80 4 client.15258 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:38:48.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.538+0000 7facb3adeb80 1 -- 172.21.15.1:0/3261151360 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd0923c050 con 0x55fd0933d900 2023-11-16T16:38:48.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.539+0000 7faca56ee700 1 --2- 172.21.15.1:0/3261151360 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55fd0933d900 0x55fd0933cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.539+0000 7faca56ee700 1 --2- 172.21.15.1:0/3261151360 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55fd0933d900 0x55fd0933cce0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5528b9c2f4d5d951 server_cookie=435f32b6b07957f5 in_seq=0 out_seq=0 2023-11-16T16:38:48.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.539+0000 7fac88ff9700 10 client.15258 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:38:48.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.540+0000 7fac88ff9700 1 -- 172.21.15.1:0/3261151360 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fd0923c050 con 0x55fd0933d900 2023-11-16T16:38:48.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.540+0000 7fac88ff9700 10 client.15258 handle_command_reply: tid=0 2023-11-16T16:38:48.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.540+0000 7facb3adeb80 1 client.15258 shutdown 2023-11-16T16:38:48.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.540+0000 7facb3adeb80 1 -- 172.21.15.1:0/3261151360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd0932a0c0 msgr2=0x55fd09339b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:48.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.540+0000 7facb3adeb80 1 --2- 172.21.15.1:0/3261151360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd0932a0c0 0x55fd09339b80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fac98007eb0 tx=0x7fac98007ee0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.540+0000 7facb3adeb80 1 -- 172.21.15.1:0/3261151360 shutdown_connections 2023-11-16T16:38:48.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.540+0000 7facb3adeb80 1 --2- 172.21.15.1:0/3261151360 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55fd0933d900 0x55fd0933cce0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.540+0000 7facb3adeb80 1 --2- 172.21.15.1:0/3261151360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd0932a0c0 0x55fd09339b80 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.540+0000 7fac88ff9700 0 client.15258 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:38:48.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.540+0000 7facb3adeb80 1 --2- 172.21.15.1:0/3261151360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fd09328bb0 0x55fd09332600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.540+0000 7facb3adeb80 1 --2- 172.21.15.1:0/3261151360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd09329500 0x55fd09332b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.540+0000 7facb3adeb80 1 -- 172.21.15.1:0/3261151360 >> 172.21.15.1:0/3261151360 conn(0x55fd0931e310 msgr2=0x55fd09322d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:48.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.540+0000 7facb3adeb80 1 -- 172.21.15.1:0/3261151360 shutdown_connections 2023-11-16T16:38:48.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.540+0000 7facb3adeb80 1 -- 172.21.15.1:0/3261151360 wait complete. 2023-11-16T16:38:48.557 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.540+0000 7facb3adeb80 20 client.15258 trim_cache size 0 max 16384 2023-11-16T16:38:48.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.558+0000 7facb3adeb80 1 Processor -- start 2023-11-16T16:38:48.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.558+0000 7facb3adeb80 1 -- start start 2023-11-16T16:38:48.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.558+0000 7facb3adeb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd09328bb0 0x55fd0932a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.567 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.558+0000 7facb3adeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fd09329500 0x55fd09339b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.567 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.558+0000 7facb3adeb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd0932a0c0 0x55fd0933cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.567 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.558+0000 7facb3adeb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd093321d0 con 0x55fd09328bb0 2023-11-16T16:38:48.567 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.558+0000 7facb3adeb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd093298d0 con 0x55fd0932a0c0 2023-11-16T16:38:48.567 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.558+0000 7facb3adeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd0931df00 con 0x55fd09329500 2023-11-16T16:38:48.567 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.558+0000 7faca56ee700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd09328bb0 0x55fd0932a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.567 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.558+0000 7faca56ee700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd09328bb0 0x55fd0932a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42796/0 (socket says 172.21.15.1:42796) 2023-11-16T16:38:48.567 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.558+0000 7faca56ee700 1 -- 172.21.15.1:0/2406830693 learned_addr learned my addr 172.21.15.1:0/2406830693 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:48.567 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.558+0000 7faca56ee700 1 -- 172.21.15.1:0/2406830693 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fd09329500 msgr2=0x55fd09339b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:48.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.558+0000 7faca56ee700 1 --2- 172.21.15.1:0/2406830693 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fd09329500 0x55fd09339b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.558+0000 7faca56ee700 1 -- 172.21.15.1:0/2406830693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd0932a0c0 msgr2=0x55fd0933cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:48.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.558+0000 7faca56ee700 1 --2- 172.21.15.1:0/2406830693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd0932a0c0 0x55fd0933cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.558+0000 7faca56ee700 1 -- 172.21.15.1:0/2406830693 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd0923fe90 con 0x55fd09328bb0 2023-11-16T16:38:48.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.559+0000 7faca56ee700 1 --2- 172.21.15.1:0/2406830693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd09328bb0 0x55fd0932a490 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fac9c09a540 tx=0x7fac9c099de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef465bfb2b6bcf20 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:48.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.559+0000 7fac8bfff700 1 -- 172.21.15.1:0/2406830693 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c010880 con 0x55fd09328bb0 2023-11-16T16:38:48.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.559+0000 7fac8bfff700 1 -- 172.21.15.1:0/2406830693 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac9c0109e0 con 0x55fd09328bb0 2023-11-16T16:38:48.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.559+0000 7fac8bfff700 1 -- 172.21.15.1:0/2406830693 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c0ade20 con 0x55fd09328bb0 2023-11-16T16:38:48.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.559+0000 7facb3adeb80 1 -- 172.21.15.1:0/2406830693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd09328bb0 msgr2=0x55fd0932a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:48.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.559+0000 7facb3adeb80 1 --2- 172.21.15.1:0/2406830693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd09328bb0 0x55fd0932a490 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fac9c09a540 tx=0x7fac9c099de0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.559+0000 7facb3adeb80 1 -- 172.21.15.1:0/2406830693 shutdown_connections 2023-11-16T16:38:48.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.559+0000 7facb3adeb80 1 --2- 172.21.15.1:0/2406830693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd0932a0c0 0x55fd0933cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.559+0000 7facb3adeb80 1 --2- 172.21.15.1:0/2406830693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd09328bb0 0x55fd0932a490 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.559+0000 7facb3adeb80 1 --2- 172.21.15.1:0/2406830693 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fd09329500 0x55fd09339b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.559+0000 7facb3adeb80 1 -- 172.21.15.1:0/2406830693 >> 172.21.15.1:0/2406830693 conn(0x55fd0931e310 msgr2=0x55fd09330520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:48.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.559+0000 7facb3adeb80 1 -- 172.21.15.1:0/2406830693 shutdown_connections 2023-11-16T16:38:48.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.559+0000 7facb3adeb80 1 -- 172.21.15.1:0/2406830693 wait complete. 2023-11-16T16:38:48.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.560+0000 7facb3adeb80 1 -- start start 2023-11-16T16:38:48.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.560+0000 7facb3adeb80 1 Processor -- start 2023-11-16T16:38:48.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.560+0000 7facb3adeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fd09328bb0 0x55fd093399a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.560+0000 7facb3adeb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd09329500 0x55fd09336440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.560+0000 7facb3adeb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd0932a0c0 0x55fd09336980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.560+0000 7facb3adeb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd09336fd0 con 0x55fd09329500 2023-11-16T16:38:48.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.560+0000 7facb3adeb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd09391780 con 0x55fd0932a0c0 2023-11-16T16:38:48.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.560+0000 7facb3adeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd093918c0 con 0x55fd09328bb0 2023-11-16T16:38:48.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.560+0000 7facb3adeb80 10 client.0 _ll_register_callbacks cb 0x7fac98007810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:38:48.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.560+0000 7faca4eed700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd09329500 0x55fd09336440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.560+0000 7faca4eed700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd09329500 0x55fd09336440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42804/0 (socket says 172.21.15.1:42804) 2023-11-16T16:38:48.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.560+0000 7faca4eed700 1 -- 172.21.15.1:0/1973709857 learned_addr learned my addr 172.21.15.1:0/1973709857 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:48.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.561+0000 7faca56ee700 1 --2- 172.21.15.1:0/1973709857 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fd09328bb0 0x55fd093399a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.561+0000 7faca5eef700 1 --2- 172.21.15.1:0/1973709857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd0932a0c0 0x55fd09336980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.561+0000 7faca4eed700 1 -- 172.21.15.1:0/1973709857 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fd09328bb0 msgr2=0x55fd093399a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:48.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.561+0000 7faca4eed700 1 --2- 172.21.15.1:0/1973709857 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fd09328bb0 0x55fd093399a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.561+0000 7faca4eed700 1 -- 172.21.15.1:0/1973709857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd0932a0c0 msgr2=0x55fd09336980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:38:48.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.561+0000 7faca4eed700 1 --2- 172.21.15.1:0/1973709857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd0932a0c0 0x55fd09336980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.561+0000 7faca4eed700 1 -- 172.21.15.1:0/1973709857 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fd090fb5e0 con 0x55fd09329500 2023-11-16T16:38:48.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.561+0000 7faca4eed700 1 --2- 172.21.15.1:0/1973709857 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd09329500 0x55fd09336440 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fac90009990 tx=0x7fac90009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0d45a99e7c21327 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:48.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.561+0000 7fac88ff9700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:38:48.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.561+0000 7fac88ff9700 1 -- 172.21.15.1:0/1973709857 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9000f660 con 0x55fd09329500 2023-11-16T16:38:48.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.561+0000 7facb3adeb80 1 -- 172.21.15.1:0/1973709857 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac9c098280 con 0x55fd09329500 2023-11-16T16:38:48.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.561+0000 7fac88ff9700 1 -- 172.21.15.1:0/1973709857 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fac9000f7c0 con 0x55fd09329500 2023-11-16T16:38:48.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.561+0000 7fac88ff9700 1 -- 172.21.15.1:0/1973709857 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac90004e80 con 0x55fd09329500 2023-11-16T16:38:48.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.561+0000 7facb3adeb80 10 client.15264 fetch_fsmap learned FSMap version 23 2023-11-16T16:38:48.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.561+0000 7facb3adeb80 1 -- 172.21.15.1:0/1973709857 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55fd09090c10 con 0x55fd09329500 2023-11-16T16:38:48.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.561+0000 7fac88ff9700 1 -- 172.21.15.1:0/1973709857 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac9c098280 con 0x55fd09329500 2023-11-16T16:38:48.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.562+0000 7fac88ff9700 1 -- 172.21.15.1:0/1973709857 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fac90032a40 con 0x55fd09329500 2023-11-16T16:38:48.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.562+0000 7facb3adeb80 10 client.15264 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:38:48.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.562+0000 7facb3adeb80 10 client.15264 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-16T16:38:48.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.562+0000 7facb3adeb80 20 client.15264 populate_metadata read hostname 'smithi001' 2023-11-16T16:38:48.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.562+0000 7facb3adeb80 1 --2- 172.21.15.1:0/1973709857 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55fd09396690 0x55fd09393020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:48.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.562+0000 7facb3adeb80 4 client.15264 mds_command: new command op to 24521 tid=0[{"prefix": "status"}] 2023-11-16T16:38:48.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.562+0000 7facb3adeb80 1 -- 172.21.15.1:0/1973709857 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fd09248610 con 0x55fd09396690 2023-11-16T16:38:48.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.562+0000 7faca56ee700 1 --2- 172.21.15.1:0/1973709857 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55fd09396690 0x55fd09393020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:48.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.562+0000 7fac88ff9700 10 client.15264 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:38:48.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.562+0000 7faca56ee700 1 --2- 172.21.15.1:0/1973709857 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55fd09396690 0x55fd09393020 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b781c9fa7627db server_cookie=1ecee32a460ccb0a in_seq=0 out_seq=0 2023-11-16T16:38:48.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.563+0000 7fac88ff9700 1 -- 172.21.15.1:0/1973709857 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55fd09248610 con 0x55fd09396690 2023-11-16T16:38:48.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.563+0000 7fac88ff9700 10 client.15264 handle_command_reply: tid=0 2023-11-16T16:38:48.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.563+0000 7facb3adeb80 1 client.15264 shutdown 2023-11-16T16:38:48.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.563+0000 7facb3adeb80 1 -- 172.21.15.1:0/1973709857 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd09329500 msgr2=0x55fd09336440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:48.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.563+0000 7facb3adeb80 1 --2- 172.21.15.1:0/1973709857 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd09329500 0x55fd09336440 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fac90009990 tx=0x7fac90009310 comp rx=0 tx=0).stop 2023-11-16T16:38:48.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.563+0000 7facb3adeb80 1 -- 172.21.15.1:0/1973709857 shutdown_connections 2023-11-16T16:38:48.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.563+0000 7facb3adeb80 1 --2- 172.21.15.1:0/1973709857 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55fd09396690 0x55fd09393020 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.563+0000 7facb3adeb80 1 --2- 172.21.15.1:0/1973709857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd0932a0c0 0x55fd09336980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.563+0000 7fac88ff9700 0 client.15264 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:38:48.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.563+0000 7facb3adeb80 1 --2- 172.21.15.1:0/1973709857 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fd09328bb0 0x55fd093399a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.563+0000 7facb3adeb80 1 --2- 172.21.15.1:0/1973709857 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd09329500 0x55fd09336440 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.563+0000 7facb3adeb80 1 -- 172.21.15.1:0/1973709857 >> 172.21.15.1:0/1973709857 conn(0x55fd0931e310 msgr2=0x55fd0933cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:48.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.563+0000 7facb3adeb80 1 -- 172.21.15.1:0/1973709857 shutdown_connections 2023-11-16T16:38:48.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.563+0000 7facb3adeb80 1 -- 172.21.15.1:0/1973709857 wait complete. 2023-11-16T16:38:48.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.563+0000 7facb3adeb80 20 client.15264 trim_cache size 0 max 16384 2023-11-16T16:38:48.575 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:38:48.575 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "35d9b010-849d-11ee-8dbe-212e2dc638e7", 2023-11-16T16:38:48.575 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 0, 2023-11-16T16:38:48.575 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24521, 2023-11-16T16:38:48.575 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2023-11-16T16:38:48.575 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2023-11-16T16:38:48.575 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2023-11-16T16:38:48.576 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 94.641827380999999, 2023-11-16T16:38:48.576 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 23, 2023-11-16T16:38:48.576 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 86, 2023-11-16T16:38:48.576 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 84, 2023-11-16T16:38:48.576 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 99.383975126999999 2023-11-16T16:38:48.576 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:38:48.578 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.563+0000 7faca7952700 1 -- 172.21.15.1:0/2728870921 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fac8c0a1150 msgr2=0x7fac8c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:48.578 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.563+0000 7faca7952700 1 --2- 172.21.15.1:0/2728870921 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fac8c0a1150 0x7fac8c0a3600 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fac9000b2e0 tx=0x7fac90005fb0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.578 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.563+0000 7faca7952700 1 -- 172.21.15.1:0/2728870921 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faca00734d0 msgr2=0x7faca0198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:48.578 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.563+0000 7faca7952700 1 --2- 172.21.15.1:0/2728870921 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faca00734d0 0x7faca0198120 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fac9c00e9a0 tx=0x7fac9c00ecb0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.578 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.564+0000 7faca7952700 1 -- 172.21.15.1:0/2728870921 shutdown_connections 2023-11-16T16:38:48.578 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.564+0000 7faca7952700 1 --2- 172.21.15.1:0/2728870921 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fac8c0a1150 0x7fac8c0a3600 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.578 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.564+0000 7faca7952700 1 --2- 172.21.15.1:0/2728870921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faca01080e0 0x7faca019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.578 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.564+0000 7faca7952700 1 --2- 172.21.15.1:0/2728870921 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faca00734d0 0x7faca0198120 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.579 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.564+0000 7faca7952700 1 --2- 172.21.15.1:0/2728870921 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7faca0074ab0 0x7faca0198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:48.579 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.564+0000 7faca7952700 1 -- 172.21.15.1:0/2728870921 >> 172.21.15.1:0/2728870921 conn(0x7faca00fb810 msgr2=0x7faca00fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:48.579 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.564+0000 7faca7952700 1 -- 172.21.15.1:0/2728870921 shutdown_connections 2023-11-16T16:38:48.579 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:48.564+0000 7faca7952700 1 -- 172.21.15.1:0/2728870921 wait complete. 2023-11-16T16:38:48.579 INFO:teuthology.run_tasks:Running task kclient... 2023-11-16T16:38:48.602 INFO:tasks.kclient:Mounting kernel clients... 2023-11-16T16:38:48.602 INFO:tasks.kclient:config is {'client.0': {}} 2023-11-16T16:38:48.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:38:48.602 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2023-11-16T16:38:48.603 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:48 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1434591359' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:38:48.603 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:48 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/3252609596' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:38:48.603 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:48 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/842505632' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:38:48.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:38:48.621 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link delete ceph-brx 2023-11-16T16:38:48.691 INFO:teuthology.orchestra.run.smithi001.stderr:Cannot find device "ceph-brx" 2023-11-16T16:38:48.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-16T16:38:48.693 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2023-11-16T16:38:48.693 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-16T16:38:48.694 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-16T16:38:48.694 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph fs ls 2023-11-16T16:38:49.385 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:38:49.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:49 smithi026 ceph-mon[100112]: pgmap v309: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2023-11-16T16:38:49.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:49 smithi053 ceph-mon[100638]: pgmap v309: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2023-11-16T16:38:49.596 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:49 smithi001 ceph-mon[98800]: pgmap v309: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2023-11-16T16:38:50.903 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:50.899+0000 7f39dc4f3700 1 -- 172.21.15.1:0/284312762 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39d40ff460 msgr2=0x7f39d40ff830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:50.903 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:50.899+0000 7f39dc4f3700 1 --2- 172.21.15.1:0/284312762 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39d40ff460 0x7f39d40ff830 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f39d00099c0 tx=0x7f39d0009cd0 comp rx=0 tx=0).stop 2023-11-16T16:38:50.903 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:50.899+0000 7f39dc4f3700 1 -- 172.21.15.1:0/284312762 shutdown_connections 2023-11-16T16:38:50.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:50.899+0000 7f39dc4f3700 1 --2- 172.21.15.1:0/284312762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39d4104500 0x7f39d410c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:50.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:50.899+0000 7f39dc4f3700 1 --2- 172.21.15.1:0/284312762 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39d40ff460 0x7f39d40ff830 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:50.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:50.899+0000 7f39dc4f3700 1 --2- 172.21.15.1:0/284312762 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39d40ffe00 0x7f39d4103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:50.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:50.899+0000 7f39dc4f3700 1 -- 172.21.15.1:0/284312762 >> 172.21.15.1:0/284312762 conn(0x7f39d40770a0 msgr2=0x7f39d40774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:50.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:50.899+0000 7f39dc4f3700 1 -- 172.21.15.1:0/284312762 shutdown_connections 2023-11-16T16:38:50.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:50.899+0000 7f39dc4f3700 1 -- 172.21.15.1:0/284312762 wait complete. 2023-11-16T16:38:50.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:50.900+0000 7f39dc4f3700 1 Processor -- start 2023-11-16T16:38:50.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:50.900+0000 7f39dc4f3700 1 -- start start 2023-11-16T16:38:50.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:50.900+0000 7f39dc4f3700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39d40ff460 0x7f39d4197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:50.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:50.900+0000 7f39dc4f3700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39d40ffe00 0x7f39d4197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:50.905 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:50.900+0000 7f39dc4f3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39d4104500 0x7f39d41a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:50.905 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:50.900+0000 7f39dc4f3700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d4198090 con 0x7f39d40ff460 2023-11-16T16:38:50.905 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:50.900+0000 7f39dc4f3700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d41a2800 con 0x7f39d4104500 2023-11-16T16:38:50.905 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:50.900+0000 7f39dc4f3700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d41a2970 con 0x7f39d40ffe00 2023-11-16T16:38:50.905 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:50.900+0000 7f39da28f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39d40ff460 0x7f39d4197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:50.905 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:50.900+0000 7f39da28f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39d40ff460 0x7f39d4197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42814/0 (socket says 172.21.15.1:42814) 2023-11-16T16:38:50.905 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:50.900+0000 7f39da28f700 1 -- 172.21.15.1:0/1816626311 learned_addr learned my addr 172.21.15.1:0/1816626311 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:50.905 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:50.900+0000 7f39d9a8e700 1 --2- 172.21.15.1:0/1816626311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39d40ffe00 0x7f39d4197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:50.905 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:50.900+0000 7f39da28f700 1 -- 172.21.15.1:0/1816626311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39d40ffe00 msgr2=0x7f39d4197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:50.906 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:50.900+0000 7f39da28f700 1 --2- 172.21.15.1:0/1816626311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39d40ffe00 0x7f39d4197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:50.906 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:50.900+0000 7f39da28f700 1 -- 172.21.15.1:0/1816626311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39d4104500 msgr2=0x7f39d41a22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:50.906 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:50.900+0000 7f39da28f700 1 --2- 172.21.15.1:0/1816626311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39d4104500 0x7f39d41a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:50.906 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:50.900+0000 7f39da28f700 1 -- 172.21.15.1:0/1816626311 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d00095f0 con 0x7f39d40ff460 2023-11-16T16:38:50.906 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:50.900+0000 7f39da28f700 1 --2- 172.21.15.1:0/1816626311 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39d40ff460 0x7f39d4197240 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f39d0009990 tx=0x7f39d0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:50.906 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:50.901+0000 7f39cb7fe700 1 -- 172.21.15.1:0/1816626311 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d001d070 con 0x7f39d40ff460 2023-11-16T16:38:50.906 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:50.901+0000 7f39cb7fe700 1 -- 172.21.15.1:0/1816626311 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39d0003f90 con 0x7f39d40ff460 2023-11-16T16:38:50.906 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:50.901+0000 7f39dc4f3700 1 -- 172.21.15.1:0/1816626311 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39d410f2c0 con 0x7f39d40ff460 2023-11-16T16:38:50.906 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:50.901+0000 7f39cb7fe700 1 -- 172.21.15.1:0/1816626311 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d0022ce0 con 0x7f39d40ff460 2023-11-16T16:38:50.906 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:50.901+0000 7f39dc4f3700 1 -- 172.21.15.1:0/1816626311 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39d410f130 con 0x7f39d40ff460 2023-11-16T16:38:50.906 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:50.902+0000 7f39dc4f3700 1 -- 172.21.15.1:0/1816626311 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39d410f450 con 0x7f39d40ff460 2023-11-16T16:38:50.907 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:50.903+0000 7f39cb7fe700 1 -- 172.21.15.1:0/1816626311 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f39d0022470 con 0x7f39d40ff460 2023-11-16T16:38:50.907 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:50.903+0000 7f39cb7fe700 1 --2- 172.21.15.1:0/1816626311 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f39c00a1150 0x7f39c00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:50.907 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:50.903+0000 7f39d9a8e700 1 --2- 172.21.15.1:0/1816626311 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f39c00a1150 0x7f39c00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:50.908 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:50.904+0000 7f39cb7fe700 1 -- 172.21.15.1:0/1816626311 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f39d00cdf00 con 0x7f39d40ff460 2023-11-16T16:38:50.908 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:50.904+0000 7f39d9a8e700 1 --2- 172.21.15.1:0/1816626311 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f39c00a1150 0x7f39c00a3600 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f39d4068e60 tx=0x7f39c400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:50.909 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:50.907+0000 7f39cb7fe700 1 -- 172.21.15.1:0/1816626311 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f39d00976e0 con 0x7f39d40ff460 2023-11-16T16:38:51.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:51.149+0000 7f39dc4f3700 1 -- 172.21.15.1:0/1816626311 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f39d41a3320 con 0x7f39d40ff460 2023-11-16T16:38:51.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:51.149+0000 7f39cb7fe700 1 -- 172.21.15.1:0/1816626311 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+73 (secure 0 0 0) 0x7f39d009ad00 con 0x7f39d40ff460 2023-11-16T16:38:51.151 INFO:teuthology.orchestra.run.smithi001.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-16T16:38:51.153 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:51.151+0000 7f39dc4f3700 1 -- 172.21.15.1:0/1816626311 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f39c00a1150 msgr2=0x7f39c00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:51.153 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:51.151+0000 7f39dc4f3700 1 --2- 172.21.15.1:0/1816626311 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f39c00a1150 0x7f39c00a3600 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f39d4068e60 tx=0x7f39c400b410 comp rx=0 tx=0).stop 2023-11-16T16:38:51.153 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:51.151+0000 7f39dc4f3700 1 -- 172.21.15.1:0/1816626311 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39d40ff460 msgr2=0x7f39d4197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:51.154 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:51.151+0000 7f39dc4f3700 1 --2- 172.21.15.1:0/1816626311 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39d40ff460 0x7f39d4197240 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f39d0009990 tx=0x7f39d0009580 comp rx=0 tx=0).stop 2023-11-16T16:38:51.154 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:51.151+0000 7f39dc4f3700 1 -- 172.21.15.1:0/1816626311 shutdown_connections 2023-11-16T16:38:51.154 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:51.151+0000 7f39dc4f3700 1 --2- 172.21.15.1:0/1816626311 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f39c00a1150 0x7f39c00a3600 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:51.154 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:51.151+0000 7f39dc4f3700 1 --2- 172.21.15.1:0/1816626311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39d4104500 0x7f39d41a22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:51.154 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:51.151+0000 7f39dc4f3700 1 --2- 172.21.15.1:0/1816626311 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39d40ff460 0x7f39d4197240 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:51.154 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:51.151+0000 7f39dc4f3700 1 --2- 172.21.15.1:0/1816626311 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f39d40ffe00 0x7f39d4197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:51.154 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:51.151+0000 7f39dc4f3700 1 -- 172.21.15.1:0/1816626311 >> 172.21.15.1:0/1816626311 conn(0x7f39d40770a0 msgr2=0x7f39d40fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:51.154 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:51.151+0000 7f39dc4f3700 1 -- 172.21.15.1:0/1816626311 shutdown_connections 2023-11-16T16:38:51.155 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:51.151+0000 7f39dc4f3700 1 -- 172.21.15.1:0/1816626311 wait complete. 2023-11-16T16:38:51.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:51 smithi001 ceph-mon[98800]: pgmap v310: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2023-11-16T16:38:51.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:51 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1816626311' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-11-16T16:38:51.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:51 smithi026 ceph-mon[100112]: pgmap v310: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2023-11-16T16:38:51.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:51 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1816626311' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-11-16T16:38:51.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:51 smithi053 ceph-mon[100638]: pgmap v310: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2023-11-16T16:38:51.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:51 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1816626311' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-11-16T16:38:51.650 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-16T16:38:51.650 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-16T16:38:51.650 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi001.front.sepia.ceph.com 2023-11-16T16:38:51.650 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-16T16:38:51.650 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-16T16:38:51.650 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-16T16:38:51.650 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-16T16:38:51.650 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-16T16:38:51.650 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-16T16:38:51.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:38:51.651 DEBUG:teuthology.orchestra.run.smithi001:> ip addr 2023-11-16T16:38:51.669 INFO:teuthology.orchestra.run.smithi001.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-16T16:38:51.669 INFO:teuthology.orchestra.run.smithi001.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-16T16:38:51.669 INFO:teuthology.orchestra.run.smithi001.stdout: inet 127.0.0.1/8 scope host lo 2023-11-16T16:38:51.670 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2023-11-16T16:38:51.670 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 ::1/128 scope host 2023-11-16T16:38:51.670 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2023-11-16T16:38:51.670 INFO:teuthology.orchestra.run.smithi001.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-16T16:38:51.670 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7a brd ff:ff:ff:ff:ff:ff 2023-11-16T16:38:51.670 INFO:teuthology.orchestra.run.smithi001.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-16T16:38:51.670 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7b brd ff:ff:ff:ff:ff:ff 2023-11-16T16:38:51.670 INFO:teuthology.orchestra.run.smithi001.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-16T16:38:51.670 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e8 brd ff:ff:ff:ff:ff:ff 2023-11-16T16:38:51.670 INFO:teuthology.orchestra.run.smithi001.stdout: inet 172.21.15.1/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-16T16:38:51.670 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft 41747sec preferred_lft 41747sec 2023-11-16T16:38:51.671 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::ec4:7aff:febd:15e8/64 scope link 2023-11-16T16:38:51.671 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2023-11-16T16:38:51.671 INFO:teuthology.orchestra.run.smithi001.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-16T16:38:51.671 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e9 brd ff:ff:ff:ff:ff:ff 2023-11-16T16:38:51.671 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-16T16:38:51.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:38:51.671 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec bash -c ' 2023-11-16T16:38:51.671 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2023-11-16T16:38:51.672 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link add name ceph-brx type bridge 2023-11-16T16:38:51.672 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip addr flush dev ceph-brx 2023-11-16T16:38:51.672 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set ceph-brx up 2023-11-16T16:38:51.672 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-16T16:38:51.672 DEBUG:teuthology.orchestra.run.smithi001:> ') 2023-11-16T16:38:51.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:38:51.842 DEBUG:teuthology.orchestra.run.smithi001:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-16T16:38:51.872 INFO:teuthology.orchestra.run.smithi001.stdout:1 2023-11-16T16:38:51.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:38:51.873 DEBUG:teuthology.orchestra.run.smithi001:> route 2023-11-16T16:38:51.944 INFO:teuthology.orchestra.run.smithi001.stdout:Kernel IP routing table 2023-11-16T16:38:51.944 INFO:teuthology.orchestra.run.smithi001.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-16T16:38:51.944 INFO:teuthology.orchestra.run.smithi001.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-16T16:38:51.944 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-16T16:38:51.945 INFO:teuthology.orchestra.run.smithi001.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-16T16:38:51.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:38:51.945 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec bash -c ' 2023-11-16T16:38:51.945 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2023-11-16T16:38:51.945 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-16T16:38:51.945 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-16T16:38:51.945 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-16T16:38:51.945 DEBUG:teuthology.orchestra.run.smithi001:> ') 2023-11-16T16:38:52.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:38:52.191 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2023-11-16T16:38:52.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:38:52.209 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list-id 2023-11-16T16:38:52.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:38:52.266 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec bash -c ' 2023-11-16T16:38:52.266 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2023-11-16T16:38:52.266 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-16T16:38:52.266 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-16T16:38:52.266 DEBUG:teuthology.orchestra.run.smithi001:> ') 2023-11-16T16:38:52.363 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-16T16:38:52.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:38:52.363 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec bash -c ' 2023-11-16T16:38:52.363 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2023-11-16T16:38:52.363 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-16T16:38:52.363 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-16T16:38:52.363 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-16T16:38:52.363 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-16T16:38:52.363 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-16T16:38:52.364 DEBUG:teuthology.orchestra.run.smithi001:> ') 2023-11-16T16:38:52.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:38:52.573 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec bash -c ' 2023-11-16T16:38:52.573 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2023-11-16T16:38:52.573 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set brx.0 up 2023-11-16T16:38:52.573 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set dev brx.0 master ceph-brx 2023-11-16T16:38:52.574 DEBUG:teuthology.orchestra.run.smithi001:> ') 2023-11-16T16:38:52.673 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-16T16:38:52.673 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-16T16:38:52.690 INFO:teuthology.orchestra.run.smithi001.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-16T16:38:52.691 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-16T16:38:52.691 DEBUG:teuthology.orchestra.run.smithi001:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-16T16:38:52.748 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-11-16T16:38:52.748 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-11-16T16:38:52.748 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/_nogroup/sv_1/bdad94b8-b08e-45e7-b7ae-3497461cc4bb 2023-11-16T16:38:52.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:38:52.748 DEBUG:teuthology.orchestra.run.smithi001:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/_nogroup/sv_1/bdad94b8-b08e-45e7-b7ae-3497461cc4bb /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2023-11-16T16:38:52.842 INFO:teuthology.orchestra.run.smithi001.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2023-11-16T16:38:52.842 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync". 2023-11-16T16:38:53.029 INFO:teuthology.orchestra.run.smithi001.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2023-11-16T16:38:53.029 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync". 2023-11-16T16:38:53.029 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: resolved to: "172.21.15.1:6789,172.21.15.26:6789,172.21.15.53:6789" 2023-11-16T16:38:53.029 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: trying mount with new device syntax: 0@35d9b010-849d-11ee-8dbe-212e2dc638e7.cephfs=/volumes/_nogroup/sv_1/bdad94b8-b08e-45e7-b7ae-3497461cc4bb 2023-11-16T16:38:53.029 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync,name=0,key=0,mon_addr=172.21.15.1:6789/172.21.15.26:6789/172.21.15.53:6789" will pass to kernel 2023-11-16T16:38:53.030 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: v2 mount support check returned 2 2023-11-16T16:38:53.030 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-16T16:38:53.030 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.1:6789,172.21.15.26:6789,172.21.15.53:6789:/volumes/_nogroup/sv_1/bdad94b8-b08e-45e7-b7ae-3497461cc4bb 2023-11-16T16:38:53.030 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync,name=0,key=0,mds_namespace=cephfs,fsid=35d9b010-849d-11ee-8dbe-212e2dc638e7" will pass to kernel 2023-11-16T16:38:53.032 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-16T16:38:53.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:38:53.033 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-16T16:38:53.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:38:53.063 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-16T16:38:53.137 INFO:teuthology.orchestra.run.smithi001.stdout:client24953 2023-11-16T16:38:53.138 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2023-11-16T16:38:53.138 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-16T16:38:53.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:38:53.197 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-16T16:38:53.271 INFO:teuthology.orchestra.run.smithi001.stdout:client24953 2023-11-16T16:38:53.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:38:53.271 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-16T16:38:53.345 INFO:teuthology.orchestra.run.smithi001.stdout:client24953 2023-11-16T16:38:53.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:38:53.346 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7.client24953/status') 2023-11-16T16:38:53.427 INFO:teuthology.orchestra.run.smithi001.stdout:instance: client.24953 (3)172.21.15.1:0/1464597211 2023-11-16T16:38:53.427 INFO:teuthology.orchestra.run.smithi001.stdout:blocklisted: false 2023-11-16T16:38:53.428 INFO:teuthology.orchestra.run.smithi001.stderr:0+1 records in 2023-11-16T16:38:53.428 INFO:teuthology.orchestra.run.smithi001.stderr:0+1 records out 2023-11-16T16:38:53.428 INFO:teuthology.orchestra.run.smithi001.stderr:70 bytes copied, 2.3821e-05 s, 2.9 MB/s 2023-11-16T16:38:53.430 INFO:teuthology.run_tasks:Running task check-counter... 2023-11-16T16:38:53.442 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-11-16T16:38:53.484 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-16T16:38:53.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:53 smithi026 ceph-mon[100112]: pgmap v311: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-16T16:38:53.509 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:53 smithi001 ceph-mon[98800]: pgmap v311: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-16T16:38:53.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:53 smithi053 ceph-mon[100638]: pgmap v311: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-11-16T16:38:53.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.610+0000 7fd979df6700 1 -- 172.21.15.1:0/2833859372 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd974103660 msgr2=0x7fd974105a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:53.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.610+0000 7fd979df6700 1 --2- 172.21.15.1:0/2833859372 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd974103660 0x7fd974105a40 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd964009990 tx=0x7fd964009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:53.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.610+0000 7fd979df6700 1 -- 172.21.15.1:0/2833859372 shutdown_connections 2023-11-16T16:38:53.616 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.610+0000 7fd979df6700 1 --2- 172.21.15.1:0/2833859372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd974105f80 0x7fd9741083e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:53.616 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.610+0000 7fd979df6700 1 --2- 172.21.15.1:0/2833859372 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd974100d40 0x7fd974103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:53.616 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.610+0000 7fd979df6700 1 --2- 172.21.15.1:0/2833859372 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd974103660 0x7fd974105a40 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:53.616 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.610+0000 7fd979df6700 1 -- 172.21.15.1:0/2833859372 >> 172.21.15.1:0/2833859372 conn(0x7fd9740fa7c0 msgr2=0x7fd9740fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:53.616 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.610+0000 7fd979df6700 1 -- 172.21.15.1:0/2833859372 shutdown_connections 2023-11-16T16:38:53.616 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.610+0000 7fd979df6700 1 -- 172.21.15.1:0/2833859372 wait complete. 2023-11-16T16:38:53.616 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.610+0000 7fd979df6700 1 Processor -- start 2023-11-16T16:38:53.617 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.610+0000 7fd979df6700 1 -- start start 2023-11-16T16:38:53.617 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.611+0000 7fd979df6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd974100d40 0x7fd974196010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:53.617 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.611+0000 7fd979df6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd974103660 0x7fd974196550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:53.617 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.611+0000 7fd979df6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd974105f80 0x7fd97419d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:53.617 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.611+0000 7fd979df6700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97419db90 con 0x7fd974105f80 2023-11-16T16:38:53.617 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.611+0000 7fd979df6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97419dcd0 con 0x7fd974100d40 2023-11-16T16:38:53.617 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.611+0000 7fd979df6700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97419de60 con 0x7fd974103660 2023-11-16T16:38:53.617 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.611+0000 7fd973fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd974105f80 0x7fd97419d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:53.618 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.611+0000 7fd973fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd974105f80 0x7fd97419d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42840/0 (socket says 172.21.15.1:42840) 2023-11-16T16:38:53.618 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.611+0000 7fd973fff700 1 -- 172.21.15.1:0/3845352824 learned_addr learned my addr 172.21.15.1:0/3845352824 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:53.618 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.611+0000 7fd9737fe700 1 --2- 172.21.15.1:0/3845352824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd974100d40 0x7fd974196010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:53.618 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.611+0000 7fd972ffd700 1 --2- 172.21.15.1:0/3845352824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd974103660 0x7fd974196550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:53.618 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.611+0000 7fd973fff700 1 -- 172.21.15.1:0/3845352824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd974103660 msgr2=0x7fd974196550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:53.618 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.611+0000 7fd973fff700 1 --2- 172.21.15.1:0/3845352824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd974103660 0x7fd974196550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:53.619 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.611+0000 7fd973fff700 1 -- 172.21.15.1:0/3845352824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd974100d40 msgr2=0x7fd974196010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:53.619 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.611+0000 7fd973fff700 1 --2- 172.21.15.1:0/3845352824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd974100d40 0x7fd974196010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:53.619 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.611+0000 7fd973fff700 1 -- 172.21.15.1:0/3845352824 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9640095f0 con 0x7fd974105f80 2023-11-16T16:38:53.619 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.611+0000 7fd973fff700 1 --2- 172.21.15.1:0/3845352824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd974105f80 0x7fd97419d5c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd96c007d70 tx=0x7fd96c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:53.619 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.611+0000 7fd970ff9700 1 -- 172.21.15.1:0/3845352824 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c00ac90 con 0x7fd974105f80 2023-11-16T16:38:53.619 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.612+0000 7fd979df6700 1 -- 172.21.15.1:0/3845352824 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd97410bcd0 con 0x7fd974105f80 2023-11-16T16:38:53.619 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.612+0000 7fd970ff9700 1 -- 172.21.15.1:0/3845352824 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd96c00adf0 con 0x7fd974105f80 2023-11-16T16:38:53.619 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.612+0000 7fd970ff9700 1 -- 172.21.15.1:0/3845352824 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c0115e0 con 0x7fd974105f80 2023-11-16T16:38:53.620 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.612+0000 7fd979df6700 1 -- 172.21.15.1:0/3845352824 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd97410bb40 con 0x7fd974105f80 2023-11-16T16:38:53.620 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.612+0000 7fd979df6700 1 -- 172.21.15.1:0/3845352824 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9740fc330 con 0x7fd974105f80 2023-11-16T16:38:53.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.614+0000 7fd970ff9700 1 -- 172.21.15.1:0/3845352824 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fd96c012420 con 0x7fd974105f80 2023-11-16T16:38:53.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.614+0000 7fd970ff9700 1 --2- 172.21.15.1:0/3845352824 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd9600a1150 0x7fd9600a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:53.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.614+0000 7fd9737fe700 1 --2- 172.21.15.1:0/3845352824 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd9600a1150 0x7fd9600a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:53.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.614+0000 7fd970ff9700 1 -- 172.21.15.1:0/3845352824 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd96c0ccf00 con 0x7fd974105f80 2023-11-16T16:38:53.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.615+0000 7fd9737fe700 1 --2- 172.21.15.1:0/3845352824 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd9600a1150 0x7fd9600a3600 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd95c005950 tx=0x7fd95c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:53.622 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.618+0000 7fd970ff9700 1 -- 172.21.15.1:0/3845352824 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd96c096650 con 0x7fd974105f80 2023-11-16T16:38:53.818 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.816+0000 7fd979df6700 1 -- 172.21.15.1:0/3845352824 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd97402cfe0 con 0x7fd974105f80 2023-11-16T16:38:53.820 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.818+0000 7fd970ff9700 1 -- 172.21.15.1:0/3845352824 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22469 (secure 0 0 0) 0x7fd96c099c70 con 0x7fd974105f80 2023-11-16T16:38:53.820 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:38:53.820 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":86,"fsid":"35d9b010-849d-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:30:06.217778+0000","modified":"2023-11-16T16:38:03.179244+0000","last_up_change":"2023-11-16T16:36:31.735403+0000","last_in_change":"2023-11-16T16:36:14.083021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:33:15.916723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-16T16:37:12.158031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-16T16:37:13.272343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a8480507-5275-4e52-953b-626110beb436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6803","nonce":2469671609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6805","nonce":2469671609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6809","nonce":2469671609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6807","nonce":2469671609}]},"public_addr":"172.21.15.1:6803/2469671609","cluster_addr":"172.21.15.1:6805/2469671609","heartbeat_back_addr":"172.21.15.1:6809/2469671609","heartbeat_front_addr":"172.21.15.1:6807/2469671609","state":["exists","up"]},{"osd":1,"uuid":"1d011db2-99d1-4575-bdc5-51f88a308fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6801","nonce":247994796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6803","nonce":247994796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6807","nonce":247994796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6805","nonce":247994796}]},"public_addr":"172.21.15.26:6801/247994796","cluster_addr":"172.21.15.26:6803/247994796","heartbeat_back_addr":"172.21.15.26:6807/247994796","heartbeat_front_addr":"172.21.15.26:6805/247994796","state":["exists","up"]},{"osd":2,"uuid":"4d7a2bd1-3c8e-4546-a243-f0354473b031","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6801","nonce":1659316290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6803","nonce":1659316290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6807","nonce":1659316290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6805","nonce":1659316290}]},"public_addr":"172.21.15.53:6801/1659316290","cluster_addr":"172.21.15.53:6803/1659316290","heartbeat_back_addr":"172.21.15.53:6807/1659316290","heartbeat_front_addr":"172.21.15.53:6805/1659316290","state":["exists","up"]},{"osd":3,"uuid":"9c290fd9-e121-4a81-9687-b0eb93debd86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6811","nonce":2728987281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6813","nonce":2728987281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6817","nonce":2728987281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6815","nonce":2728987281}]},"public_addr":"172.21.15.1:6811/2728987281","cluster_addr":"172.21.15.1:6813/2728987281","heartbeat_back_addr":"172.21.15.1:6817/2728987281","heartbeat_front_addr":"172.21.15.1:6815/2728987281","state":["exists","up"]},{"osd":4,"uuid":"533226bc-cc57-4f15-9e29-820294d57e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6809","nonce":2954740024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6811","nonce":2954740024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6815","nonce":2954740024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6813","nonce":2954740024}]},"public_addr":"172.21.15.26:6809/2954740024","cluster_addr":"172.21.15.26:6811/2954740024","heartbeat_back_addr":"172.21.15.26:6815/2954740024","heartbeat_front_addr":"172.21.15.26:6813/2954740024","state":["exists","up"]},{"osd":5,"uuid":"d3ab25eb-e1d4-4557-bc89-efc9072401ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6809","nonce":3966666318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6811","nonce":3966666318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6815","nonce":3966666318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6813","nonce":3966666318}]},"public_addr":"172.21.15.53:6809/3966666318","cluster_addr":"172.21.15.53:6811/3966666318","heartbeat_back_addr":"172.21.15.53:6815/3966666318","heartbeat_front_addr":"172.21.15.53:6813/3966666318","state":["exists","up"]},{"osd":6,"uuid":"180f506e-afb8-4043-b0b4-2dc86f6d4bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6819","nonce":3364032646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6821","nonce":3364032646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6825","nonce":3364032646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6823","nonce":3364032646}]},"public_addr":"172.21.15.1:6819/3364032646","cluster_addr":"172.21.15.1:6821/3364032646","heartbeat_back_addr":"172.21.15.1:6825/3364032646","heartbeat_front_addr":"172.21.15.1:6823/3364032646","state":["exists","up"]},{"osd":7,"uuid":"0d837bac-194f-498d-b7fb-b3147e992329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6817","nonce":3766901391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6819","nonce":3766901391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6823","nonce":3766901391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6821","nonce":3766901391}]},"public_addr":"172.21.15.26:6817/3766901391","cluster_addr":"172.21.15.26:6819/3766901391","heartbeat_back_addr":"172.21.15.26:6823/3766901391","heartbeat_front_addr":"172.21.15.26:6821/3766901391","state":["exists","up"]},{"osd":8,"uuid":"933f231a-7454-47b0-9291-f1fa64f0baec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6817","nonce":3224335871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6819","nonce":3224335871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6823","nonce":3224335871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6821","nonce":3224335871}]},"public_addr":"172.21.15.53:6817/3224335871","cluster_addr":"172.21.15.53:6819/3224335871","heartbeat_back_addr":"172.21.15.53:6823/3224335871","heartbeat_front_addr":"172.21.15.53:6821/3224335871","state":["exists","up"]},{"osd":9,"uuid":"245fcbb3-c651-4703-ab80-309dc98bd983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6827","nonce":1391199844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6829","nonce":1391199844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6833","nonce":1391199844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6831","nonce":1391199844}]},"public_addr":"172.21.15.1:6827/1391199844","cluster_addr":"172.21.15.1:6829/1391199844","heartbeat_back_addr":"172.21.15.1:6833/1391199844","heartbeat_front_addr":"172.21.15.1:6831/1391199844","state":["exists","up"]},{"osd":10,"uuid":"2c090e7a-5240-4d52-ae4f-e94fe05383d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6825","nonce":1773545447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6827","nonce":1773545447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6831","nonce":1773545447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6829","nonce":1773545447}]},"public_addr":"172.21.15.26:6825/1773545447","cluster_addr":"172.21.15.26:6827/1773545447","heartbeat_back_addr":"172.21.15.26:6831/1773545447","heartbeat_front_addr":"172.21.15.26:6829/1773545447","state":["exists","up"]},{"osd":11,"uuid":"4511e32d-e469-45e1-8310-0ff400cb7b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6825","nonce":2564669546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6827","nonce":2564669546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6831","nonce":2564669546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6829","nonce":2564669546}]},"public_addr":"172.21.15.53:6825/2564669546","cluster_addr":"172.21.15.53:6827/2564669546","heartbeat_back_addr":"172.21.15.53:6831/2564669546","heartbeat_front_addr":"172.21.15.53:6829/2564669546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:28.140250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:49.885461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:12.521487+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:34.926197+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:56.815594+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:18.387751+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:40.587106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:02.319370+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:23.832298+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:46.338040+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:08.730414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:30.050070+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1494336948":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6801/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6800/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6800/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/1641976669":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/2895923638":"2023-11-17T16:30:35.064289+0000","172.21.15.1:6801/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/78811711":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/4131957708":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/344403698":"2023-11-17T16:30:35.064289+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:38:53.822 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.820+0000 7fd979df6700 1 -- 172.21.15.1:0/3845352824 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd9600a1150 msgr2=0x7fd9600a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:53.822 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.820+0000 7fd979df6700 1 --2- 172.21.15.1:0/3845352824 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd9600a1150 0x7fd9600a3600 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd95c005950 tx=0x7fd95c00b410 comp rx=0 tx=0).stop 2023-11-16T16:38:53.823 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.820+0000 7fd979df6700 1 -- 172.21.15.1:0/3845352824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd974105f80 msgr2=0x7fd97419d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:53.823 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.820+0000 7fd979df6700 1 --2- 172.21.15.1:0/3845352824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd974105f80 0x7fd97419d5c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd96c007d70 tx=0x7fd96c00a340 comp rx=0 tx=0).stop 2023-11-16T16:38:53.823 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.820+0000 7fd979df6700 1 -- 172.21.15.1:0/3845352824 shutdown_connections 2023-11-16T16:38:53.823 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.820+0000 7fd979df6700 1 --2- 172.21.15.1:0/3845352824 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd9600a1150 0x7fd9600a3600 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:53.823 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.820+0000 7fd979df6700 1 --2- 172.21.15.1:0/3845352824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd974105f80 0x7fd97419d5c0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:53.823 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.820+0000 7fd979df6700 1 --2- 172.21.15.1:0/3845352824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd974100d40 0x7fd974196010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:53.823 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.820+0000 7fd979df6700 1 --2- 172.21.15.1:0/3845352824 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd974103660 0x7fd974196550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:53.823 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.820+0000 7fd979df6700 1 -- 172.21.15.1:0/3845352824 >> 172.21.15.1:0/3845352824 conn(0x7fd9740fa7c0 msgr2=0x7fd9741019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:53.824 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.820+0000 7fd979df6700 1 -- 172.21.15.1:0/3845352824 shutdown_connections 2023-11-16T16:38:53.824 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:53.820+0000 7fd979df6700 1 -- 172.21.15.1:0/3845352824 wait complete. 2023-11-16T16:38:53.835 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-16T16:33:15.916723+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-16T16:37:12.158031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-16T16:37:13.272343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-16T16:38:53.836 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-16T16:38:54.005 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.001+0000 7f6424384700 1 -- 172.21.15.1:0/1325909023 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f641c068b00 msgr2=0x7f641c102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:54.005 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.001+0000 7f6424384700 1 --2- 172.21.15.1:0/1325909023 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f641c068b00 0x7f641c102e00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6418009990 tx=0x7f6418009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:54.005 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.001+0000 7f6424384700 1 -- 172.21.15.1:0/1325909023 shutdown_connections 2023-11-16T16:38:54.005 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.001+0000 7f6424384700 1 --2- 172.21.15.1:0/1325909023 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f641c105e50 0x7f641c108300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:54.006 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.001+0000 7f6424384700 1 --2- 172.21.15.1:0/1325909023 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f641c068b00 0x7f641c102e00 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:54.006 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.001+0000 7f6424384700 1 --2- 172.21.15.1:0/1325909023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f641c103340 0x7f641c1057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:54.006 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.001+0000 7f6424384700 1 -- 172.21.15.1:0/1325909023 >> 172.21.15.1:0/1325909023 conn(0x7f641c0fa750 msgr2=0x7f641c0fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:54.006 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.002+0000 7f6424384700 1 -- 172.21.15.1:0/1325909023 shutdown_connections 2023-11-16T16:38:54.006 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.002+0000 7f6424384700 1 -- 172.21.15.1:0/1325909023 wait complete. 2023-11-16T16:38:54.006 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.002+0000 7f6424384700 1 Processor -- start 2023-11-16T16:38:54.006 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.002+0000 7f6424384700 1 -- start start 2023-11-16T16:38:54.006 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.002+0000 7f6424384700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f641c068b00 0x7f641c100760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:54.006 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.002+0000 7f6424384700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f641c103340 0x7f641c0feda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:54.006 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.002+0000 7f6424384700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f641c105e50 0x7f641c0ff2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:54.006 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.002+0000 7f6424384700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f641c0ffa80 con 0x7f641c068b00 2023-11-16T16:38:54.006 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.002+0000 7f6424384700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f641c0ffbf0 con 0x7f641c105e50 2023-11-16T16:38:54.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.002+0000 7f6424384700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f641c1a0000 con 0x7f641c103340 2023-11-16T16:38:54.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.002+0000 7f6422120700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f641c068b00 0x7f641c100760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:54.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.002+0000 7f6422120700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f641c068b00 0x7f641c100760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42850/0 (socket says 172.21.15.1:42850) 2023-11-16T16:38:54.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.002+0000 7f6422120700 1 -- 172.21.15.1:0/1385721019 learned_addr learned my addr 172.21.15.1:0/1385721019 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:54.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.003+0000 7f642191f700 1 --2- 172.21.15.1:0/1385721019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f641c103340 0x7f641c0feda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:54.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.003+0000 7f6422921700 1 --2- 172.21.15.1:0/1385721019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f641c105e50 0x7f641c0ff2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:54.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.003+0000 7f6422120700 1 -- 172.21.15.1:0/1385721019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f641c103340 msgr2=0x7f641c0feda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:54.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.003+0000 7f6422120700 1 --2- 172.21.15.1:0/1385721019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f641c103340 0x7f641c0feda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:54.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.003+0000 7f6422120700 1 -- 172.21.15.1:0/1385721019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f641c105e50 msgr2=0x7f641c0ff2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:54.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.003+0000 7f6422120700 1 --2- 172.21.15.1:0/1385721019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f641c105e50 0x7f641c0ff2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:54.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.003+0000 7f6422120700 1 -- 172.21.15.1:0/1385721019 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64180095f0 con 0x7f641c068b00 2023-11-16T16:38:54.008 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.003+0000 7f6422921700 1 --2- 172.21.15.1:0/1385721019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f641c105e50 0x7f641c0ff2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:38:54.008 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.003+0000 7f6422120700 1 --2- 172.21.15.1:0/1385721019 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f641c068b00 0x7f641c100760 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f641800b680 tx=0x7f6418005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:54.008 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.003+0000 7f64137fe700 1 -- 172.21.15.1:0/1385721019 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641801d070 con 0x7f641c068b00 2023-11-16T16:38:54.008 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.003+0000 7f64137fe700 1 -- 172.21.15.1:0/1385721019 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6418004ca0 con 0x7f641c068b00 2023-11-16T16:38:54.008 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.003+0000 7f6424384700 1 -- 172.21.15.1:0/1385721019 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f641c10bbd0 con 0x7f641c068b00 2023-11-16T16:38:54.008 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.003+0000 7f64137fe700 1 -- 172.21.15.1:0/1385721019 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64180055f0 con 0x7f641c068b00 2023-11-16T16:38:54.008 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.003+0000 7f6424384700 1 -- 172.21.15.1:0/1385721019 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f641c10ba40 con 0x7f641c068b00 2023-11-16T16:38:54.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.004+0000 7f6424384700 1 -- 172.21.15.1:0/1385721019 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f641c0fc2c0 con 0x7f641c068b00 2023-11-16T16:38:54.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.005+0000 7f64137fe700 1 -- 172.21.15.1:0/1385721019 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f641800fc70 con 0x7f641c068b00 2023-11-16T16:38:54.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.006+0000 7f64137fe700 1 --2- 172.21.15.1:0/1385721019 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f64080a1150 0x7f64080a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:54.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.006+0000 7f64137fe700 1 -- 172.21.15.1:0/1385721019 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f64180ce1a0 con 0x7f641c068b00 2023-11-16T16:38:54.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.006+0000 7f642191f700 1 --2- 172.21.15.1:0/1385721019 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f64080a1150 0x7f64080a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:54.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.006+0000 7f642191f700 1 --2- 172.21.15.1:0/1385721019 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f64080a1150 0x7f64080a3600 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f640c009b30 tx=0x7f640c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:54.011 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.009+0000 7f64137fe700 1 -- 172.21.15.1:0/1385721019 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6418097980 con 0x7f641c068b00 2023-11-16T16:38:54.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.213+0000 7f6424384700 1 -- 172.21.15.1:0/1385721019 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f641c067a50 con 0x7f641c068b00 2023-11-16T16:38:54.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.214+0000 7f64137fe700 1 -- 172.21.15.1:0/1385721019 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f641809afa0 con 0x7f641c068b00 2023-11-16T16:38:54.216 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2023-11-16T16:38:54.218 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.216+0000 7f6424384700 1 -- 172.21.15.1:0/1385721019 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f64080a1150 msgr2=0x7f64080a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:54.218 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.216+0000 7f6424384700 1 --2- 172.21.15.1:0/1385721019 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f64080a1150 0x7f64080a3600 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f640c009b30 tx=0x7f640c009260 comp rx=0 tx=0).stop 2023-11-16T16:38:54.218 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.216+0000 7f6424384700 1 -- 172.21.15.1:0/1385721019 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f641c068b00 msgr2=0x7f641c100760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:54.218 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.216+0000 7f6424384700 1 --2- 172.21.15.1:0/1385721019 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f641c068b00 0x7f641c100760 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f641800b680 tx=0x7f6418005ef0 comp rx=0 tx=0).stop 2023-11-16T16:38:54.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.216+0000 7f6424384700 1 -- 172.21.15.1:0/1385721019 shutdown_connections 2023-11-16T16:38:54.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.216+0000 7f6424384700 1 --2- 172.21.15.1:0/1385721019 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f64080a1150 0x7f64080a3600 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:54.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.216+0000 7f6424384700 1 --2- 172.21.15.1:0/1385721019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f641c105e50 0x7f641c0ff2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:54.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.216+0000 7f6424384700 1 --2- 172.21.15.1:0/1385721019 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f641c068b00 0x7f641c100760 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:54.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.216+0000 7f6424384700 1 --2- 172.21.15.1:0/1385721019 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f641c103340 0x7f641c0feda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:54.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.216+0000 7f6424384700 1 -- 172.21.15.1:0/1385721019 >> 172.21.15.1:0/1385721019 conn(0x7f641c0fa750 msgr2=0x7f641c101680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:54.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.216+0000 7f6424384700 1 -- 172.21.15.1:0/1385721019 shutdown_connections 2023-11-16T16:38:54.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.216+0000 7f6424384700 1 -- 172.21.15.1:0/1385721019 wait complete. 2023-11-16T16:38:54.231 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-16T16:38:54.360 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.358+0000 7fef242a5700 1 -- 172.21.15.1:0/4138021300 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef1c0ff460 msgr2=0x7fef1c0ff8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:54.360 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.358+0000 7fef242a5700 1 --2- 172.21.15.1:0/4138021300 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef1c0ff460 0x7fef1c0ff8d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fef180099c0 tx=0x7fef18009cd0 comp rx=0 tx=0).stop 2023-11-16T16:38:54.360 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.358+0000 7fef242a5700 1 -- 172.21.15.1:0/4138021300 shutdown_connections 2023-11-16T16:38:54.361 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.358+0000 7fef242a5700 1 --2- 172.21.15.1:0/4138021300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef1c0ffe10 0x7fef1c1076c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:54.361 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.358+0000 7fef242a5700 1 --2- 172.21.15.1:0/4138021300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef1c100770 0x7fef1c100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:54.361 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.358+0000 7fef242a5700 1 --2- 172.21.15.1:0/4138021300 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef1c0ff460 0x7fef1c0ff8d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:54.361 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.358+0000 7fef242a5700 1 -- 172.21.15.1:0/4138021300 >> 172.21.15.1:0/4138021300 conn(0x7fef1c0f9f00 msgr2=0x7fef1c0fc350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:54.361 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.358+0000 7fef242a5700 1 -- 172.21.15.1:0/4138021300 shutdown_connections 2023-11-16T16:38:54.361 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.358+0000 7fef242a5700 1 -- 172.21.15.1:0/4138021300 wait complete. 2023-11-16T16:38:54.364 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.359+0000 7fef242a5700 1 Processor -- start 2023-11-16T16:38:54.364 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.359+0000 7fef242a5700 1 -- start start 2023-11-16T16:38:54.364 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.359+0000 7fef242a5700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef1c0ff460 0x7fef1c194ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:54.364 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.359+0000 7fef242a5700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef1c0ffe10 0x7fef1c1953e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:54.364 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.359+0000 7fef242a5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef1c100770 0x7fef1c19c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:54.364 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.359+0000 7fef242a5700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c19ca20 con 0x7fef1c0ff460 2023-11-16T16:38:54.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.359+0000 7fef242a5700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c19cb60 con 0x7fef1c100770 2023-11-16T16:38:54.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.359+0000 7fef242a5700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c19ccf0 con 0x7fef1c0ffe10 2023-11-16T16:38:54.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.359+0000 7fef22041700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef1c0ff460 0x7fef1c194ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:54.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.359+0000 7fef22041700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef1c0ff460 0x7fef1c194ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42864/0 (socket says 172.21.15.1:42864) 2023-11-16T16:38:54.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.359+0000 7fef22041700 1 -- 172.21.15.1:0/697398979 learned_addr learned my addr 172.21.15.1:0/697398979 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:54.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.360+0000 7fef22842700 1 --2- 172.21.15.1:0/697398979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef1c100770 0x7fef1c19c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:54.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.360+0000 7fef21840700 1 --2- 172.21.15.1:0/697398979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef1c0ffe10 0x7fef1c1953e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:54.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.360+0000 7fef22041700 1 -- 172.21.15.1:0/697398979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef1c0ffe10 msgr2=0x7fef1c1953e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:54.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.360+0000 7fef22041700 1 --2- 172.21.15.1:0/697398979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef1c0ffe10 0x7fef1c1953e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:54.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.360+0000 7fef22041700 1 -- 172.21.15.1:0/697398979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef1c100770 msgr2=0x7fef1c19c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:54.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.360+0000 7fef22041700 1 --2- 172.21.15.1:0/697398979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef1c100770 0x7fef1c19c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:54.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.360+0000 7fef22041700 1 -- 172.21.15.1:0/697398979 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef180095f0 con 0x7fef1c0ff460 2023-11-16T16:38:54.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.360+0000 7fef21840700 1 --2- 172.21.15.1:0/697398979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef1c0ffe10 0x7fef1c1953e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:38:54.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.360+0000 7fef22842700 1 --2- 172.21.15.1:0/697398979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef1c100770 0x7fef1c19c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:38:54.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.360+0000 7fef22041700 1 --2- 172.21.15.1:0/697398979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef1c0ff460 0x7fef1c194ea0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fef1000d950 tx=0x7fef1000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:54.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.360+0000 7fef0f7fe700 1 -- 172.21.15.1:0/697398979 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef10009b40 con 0x7fef1c0ff460 2023-11-16T16:38:54.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.360+0000 7fef0f7fe700 1 -- 172.21.15.1:0/697398979 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef10009ca0 con 0x7fef1c0ff460 2023-11-16T16:38:54.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.360+0000 7fef242a5700 1 -- 172.21.15.1:0/697398979 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef1c10ab60 con 0x7fef1c0ff460 2023-11-16T16:38:54.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.360+0000 7fef0f7fe700 1 -- 172.21.15.1:0/697398979 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1000f620 con 0x7fef1c0ff460 2023-11-16T16:38:54.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.360+0000 7fef242a5700 1 -- 172.21.15.1:0/697398979 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef1c10a9d0 con 0x7fef1c0ff460 2023-11-16T16:38:54.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.361+0000 7fef242a5700 1 -- 172.21.15.1:0/697398979 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef1c106200 con 0x7fef1c0ff460 2023-11-16T16:38:54.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.362+0000 7fef0f7fe700 1 -- 172.21.15.1:0/697398979 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fef10010460 con 0x7fef1c0ff460 2023-11-16T16:38:54.367 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.363+0000 7fef0f7fe700 1 --2- 172.21.15.1:0/697398979 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fef080a1150 0x7fef080a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:54.367 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.363+0000 7fef21840700 1 --2- 172.21.15.1:0/697398979 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fef080a1150 0x7fef080a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:54.367 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.363+0000 7fef0f7fe700 1 -- 172.21.15.1:0/697398979 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fef100ccca0 con 0x7fef1c0ff460 2023-11-16T16:38:54.367 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.363+0000 7fef21840700 1 --2- 172.21.15.1:0/697398979 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fef080a1150 0x7fef080a3600 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fef180098e0 tx=0x7fef18000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:54.369 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.367+0000 7fef0f7fe700 1 -- 172.21.15.1:0/697398979 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fef100963b0 con 0x7fef1c0ff460 2023-11-16T16:38:54.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:54 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/3845352824' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:38:54.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:54 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1385721019' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:38:54.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.566+0000 7fef242a5700 1 -- 172.21.15.1:0/697398979 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fef1c061700 con 0x7fef1c0ff460 2023-11-16T16:38:54.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.567+0000 7fef0f7fe700 1 -- 172.21.15.1:0/697398979 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7fef10020020 con 0x7fef1c0ff460 2023-11-16T16:38:54.569 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2023-11-16T16:38:54.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.569+0000 7fef242a5700 1 -- 172.21.15.1:0/697398979 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fef080a1150 msgr2=0x7fef080a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:54.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.569+0000 7fef242a5700 1 --2- 172.21.15.1:0/697398979 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fef080a1150 0x7fef080a3600 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fef180098e0 tx=0x7fef18000bc0 comp rx=0 tx=0).stop 2023-11-16T16:38:54.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.569+0000 7fef242a5700 1 -- 172.21.15.1:0/697398979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef1c0ff460 msgr2=0x7fef1c194ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:54.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.569+0000 7fef242a5700 1 --2- 172.21.15.1:0/697398979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef1c0ff460 0x7fef1c194ea0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fef1000d950 tx=0x7fef1000dd10 comp rx=0 tx=0).stop 2023-11-16T16:38:54.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.569+0000 7fef242a5700 1 -- 172.21.15.1:0/697398979 shutdown_connections 2023-11-16T16:38:54.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.569+0000 7fef242a5700 1 --2- 172.21.15.1:0/697398979 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fef080a1150 0x7fef080a3600 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:54.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.569+0000 7fef242a5700 1 --2- 172.21.15.1:0/697398979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef1c100770 0x7fef1c19c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:54.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.569+0000 7fef242a5700 1 --2- 172.21.15.1:0/697398979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef1c0ff460 0x7fef1c194ea0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:54.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.569+0000 7fef242a5700 1 --2- 172.21.15.1:0/697398979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef1c0ffe10 0x7fef1c1953e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:54.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.569+0000 7fef242a5700 1 -- 172.21.15.1:0/697398979 >> 172.21.15.1:0/697398979 conn(0x7fef1c0f9f00 msgr2=0x7fef1c0fc200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:54.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.569+0000 7fef242a5700 1 -- 172.21.15.1:0/697398979 shutdown_connections 2023-11-16T16:38:54.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.569+0000 7fef242a5700 1 -- 172.21.15.1:0/697398979 wait complete. 2023-11-16T16:38:54.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:54 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/3845352824' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:38:54.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:54 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1385721019' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:38:54.582 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:54 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3845352824' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:38:54.582 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:54 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1385721019' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:38:54.583 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-16T16:38:54.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.712+0000 7fc934d1e700 1 -- 172.21.15.1:0/1283460421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc930100d90 msgr2=0x7fc930103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:54.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.712+0000 7fc934d1e700 1 --2- 172.21.15.1:0/1283460421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc930100d90 0x7fc930103170 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc9200098f0 tx=0x7fc920009c00 comp rx=0 tx=0).stop 2023-11-16T16:38:54.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.712+0000 7fc934d1e700 1 -- 172.21.15.1:0/1283460421 shutdown_connections 2023-11-16T16:38:54.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.712+0000 7fc934d1e700 1 --2- 172.21.15.1:0/1283460421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc930105fd0 0x7fc9301083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:54.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.712+0000 7fc934d1e700 1 --2- 172.21.15.1:0/1283460421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc930100d90 0x7fc930103170 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:54.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.712+0000 7fc934d1e700 1 --2- 172.21.15.1:0/1283460421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9301036b0 0x7fc930105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:54.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.712+0000 7fc934d1e700 1 -- 172.21.15.1:0/1283460421 >> 172.21.15.1:0/1283460421 conn(0x7fc9300fa750 msgr2=0x7fc9300fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:54.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.712+0000 7fc934d1e700 1 -- 172.21.15.1:0/1283460421 shutdown_connections 2023-11-16T16:38:54.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.712+0000 7fc934d1e700 1 -- 172.21.15.1:0/1283460421 wait complete. 2023-11-16T16:38:54.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.713+0000 7fc934d1e700 1 Processor -- start 2023-11-16T16:38:54.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.713+0000 7fc934d1e700 1 -- start start 2023-11-16T16:38:54.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.713+0000 7fc934d1e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc930100d90 0x7fc930196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:54.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.713+0000 7fc934d1e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9301036b0 0x7fc930196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:54.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.713+0000 7fc934d1e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc930105fd0 0x7fc93019d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:54.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.713+0000 7fc934d1e700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93019db80 con 0x7fc9301036b0 2023-11-16T16:38:54.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.713+0000 7fc934d1e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93019dcc0 con 0x7fc930100d90 2023-11-16T16:38:54.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.713+0000 7fc934d1e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93019de50 con 0x7fc930105fd0 2023-11-16T16:38:54.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.713+0000 7fc92dd9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9301036b0 0x7fc930196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:54.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.713+0000 7fc92dd9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9301036b0 0x7fc930196540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42878/0 (socket says 172.21.15.1:42878) 2023-11-16T16:38:54.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.713+0000 7fc92dd9b700 1 -- 172.21.15.1:0/2931934379 learned_addr learned my addr 172.21.15.1:0/2931934379 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:54.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.713+0000 7fc92ed9d700 1 --2- 172.21.15.1:0/2931934379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc930105fd0 0x7fc93019d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:54.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.713+0000 7fc92e59c700 1 --2- 172.21.15.1:0/2931934379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc930100d90 0x7fc930196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:54.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.713+0000 7fc92dd9b700 1 -- 172.21.15.1:0/2931934379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc930105fd0 msgr2=0x7fc93019d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:54.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.713+0000 7fc92dd9b700 1 --2- 172.21.15.1:0/2931934379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc930105fd0 0x7fc93019d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:54.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.713+0000 7fc92dd9b700 1 -- 172.21.15.1:0/2931934379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc930100d90 msgr2=0x7fc930196000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:54.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.713+0000 7fc92dd9b700 1 --2- 172.21.15.1:0/2931934379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc930100d90 0x7fc930196000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:54.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.713+0000 7fc92dd9b700 1 -- 172.21.15.1:0/2931934379 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc920009520 con 0x7fc9301036b0 2023-11-16T16:38:54.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.713+0000 7fc92ed9d700 1 --2- 172.21.15.1:0/2931934379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc930105fd0 0x7fc93019d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:38:54.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.713+0000 7fc92e59c700 1 --2- 172.21.15.1:0/2931934379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc930100d90 0x7fc930196000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:38:54.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.714+0000 7fc92dd9b700 1 --2- 172.21.15.1:0/2931934379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9301036b0 0x7fc930196540 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc92400e9c0 tx=0x7fc92400ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:54.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.714+0000 7fc91f7fe700 1 -- 172.21.15.1:0/2931934379 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92400cbd0 con 0x7fc9301036b0 2023-11-16T16:38:54.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.714+0000 7fc91f7fe700 1 -- 172.21.15.1:0/2931934379 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc92400cd30 con 0x7fc9301036b0 2023-11-16T16:38:54.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.714+0000 7fc91f7fe700 1 -- 172.21.15.1:0/2931934379 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc924005580 con 0x7fc9301036b0 2023-11-16T16:38:54.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.714+0000 7fc934d1e700 1 -- 172.21.15.1:0/2931934379 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc93010bbe0 con 0x7fc9301036b0 2023-11-16T16:38:54.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.714+0000 7fc934d1e700 1 -- 172.21.15.1:0/2931934379 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc93010ba50 con 0x7fc9301036b0 2023-11-16T16:38:54.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.715+0000 7fc934d1e700 1 -- 172.21.15.1:0/2931934379 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9300fc2c0 con 0x7fc9301036b0 2023-11-16T16:38:54.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.716+0000 7fc91f7fe700 1 -- 172.21.15.1:0/2931934379 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fc924004750 con 0x7fc9301036b0 2023-11-16T16:38:54.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.716+0000 7fc91f7fe700 1 --2- 172.21.15.1:0/2931934379 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc9180a11d0 0x7fc9180a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:54.720 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.717+0000 7fc91f7fe700 1 -- 172.21.15.1:0/2931934379 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc924014070 con 0x7fc9301036b0 2023-11-16T16:38:54.720 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.717+0000 7fc92e59c700 1 --2- 172.21.15.1:0/2931934379 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc9180a11d0 0x7fc9180a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:54.720 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.717+0000 7fc92e59c700 1 --2- 172.21.15.1:0/2931934379 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc9180a11d0 0x7fc9180a3680 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc9200098c0 tx=0x7fc920019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:54.722 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.720+0000 7fc91f7fe700 1 -- 172.21.15.1:0/2931934379 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc924096430 con 0x7fc9301036b0 2023-11-16T16:38:54.925 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.924+0000 7fc934d1e700 1 -- 172.21.15.1:0/2931934379 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc930061700 con 0x7fc9301036b0 2023-11-16T16:38:54.926 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.924+0000 7fc91f7fe700 1 -- 172.21.15.1:0/2931934379 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc924099a50 con 0x7fc9301036b0 2023-11-16T16:38:54.926 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2023-11-16T16:38:54.928 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.926+0000 7fc934d1e700 1 -- 172.21.15.1:0/2931934379 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc9180a11d0 msgr2=0x7fc9180a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:54.929 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.926+0000 7fc934d1e700 1 --2- 172.21.15.1:0/2931934379 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc9180a11d0 0x7fc9180a3680 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc9200098c0 tx=0x7fc920019040 comp rx=0 tx=0).stop 2023-11-16T16:38:54.929 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.926+0000 7fc934d1e700 1 -- 172.21.15.1:0/2931934379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9301036b0 msgr2=0x7fc930196540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:54.929 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.926+0000 7fc934d1e700 1 --2- 172.21.15.1:0/2931934379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9301036b0 0x7fc930196540 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc92400e9c0 tx=0x7fc92400ed80 comp rx=0 tx=0).stop 2023-11-16T16:38:54.929 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.926+0000 7fc934d1e700 1 -- 172.21.15.1:0/2931934379 shutdown_connections 2023-11-16T16:38:54.929 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.926+0000 7fc934d1e700 1 --2- 172.21.15.1:0/2931934379 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fc9180a11d0 0x7fc9180a3680 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:54.929 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.926+0000 7fc934d1e700 1 --2- 172.21.15.1:0/2931934379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc930105fd0 0x7fc93019d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:54.929 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.926+0000 7fc934d1e700 1 --2- 172.21.15.1:0/2931934379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc930100d90 0x7fc930196000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:54.929 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.926+0000 7fc934d1e700 1 --2- 172.21.15.1:0/2931934379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9301036b0 0x7fc930196540 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:54.929 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.926+0000 7fc934d1e700 1 -- 172.21.15.1:0/2931934379 >> 172.21.15.1:0/2931934379 conn(0x7fc9300fa750 msgr2=0x7fc9301019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:54.929 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.926+0000 7fc934d1e700 1 -- 172.21.15.1:0/2931934379 shutdown_connections 2023-11-16T16:38:54.929 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:54.927+0000 7fc934d1e700 1 -- 172.21.15.1:0/2931934379 wait complete. 2023-11-16T16:38:54.941 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-16T16:38:55.069 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.065+0000 7fbec3b8f700 1 -- 172.21.15.1:0/3413787133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbebc0ff1c0 msgr2=0x7fbebc0ff5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:55.070 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.065+0000 7fbec3b8f700 1 --2- 172.21.15.1:0/3413787133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbebc0ff1c0 0x7fbebc0ff5d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8009990 tx=0x7fbeb8009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:55.070 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.065+0000 7fbec3b8f700 1 -- 172.21.15.1:0/3413787133 shutdown_connections 2023-11-16T16:38:55.070 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.065+0000 7fbec3b8f700 1 --2- 172.21.15.1:0/3413787133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbebc100d40 0x7fbebc107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:55.070 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.065+0000 7fbec3b8f700 1 --2- 172.21.15.1:0/3413787133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbebc0ff1c0 0x7fbebc0ff5d0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:55.070 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.065+0000 7fbec3b8f700 1 --2- 172.21.15.1:0/3413787133 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbebc1003b0 0x7fbebc100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:55.070 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.065+0000 7fbec3b8f700 1 -- 172.21.15.1:0/3413787133 >> 172.21.15.1:0/3413787133 conn(0x7fbebc0fa7c0 msgr2=0x7fbebc0fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:55.070 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.065+0000 7fbec3b8f700 1 -- 172.21.15.1:0/3413787133 shutdown_connections 2023-11-16T16:38:55.070 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.066+0000 7fbec3b8f700 1 -- 172.21.15.1:0/3413787133 wait complete. 2023-11-16T16:38:55.070 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.066+0000 7fbec3b8f700 1 Processor -- start 2023-11-16T16:38:55.070 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.066+0000 7fbec3b8f700 1 -- start start 2023-11-16T16:38:55.070 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.066+0000 7fbec3b8f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbebc0ff1c0 0x7fbebc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:55.071 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.066+0000 7fbec3b8f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbebc1003b0 0x7fbebc19ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:55.071 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.066+0000 7fbec3b8f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbebc100d40 0x7fbebc19d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:55.071 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.066+0000 7fbec3b8f700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc19db60 con 0x7fbebc0ff1c0 2023-11-16T16:38:55.071 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.066+0000 7fbec3b8f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc19dcd0 con 0x7fbebc100d40 2023-11-16T16:38:55.071 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.066+0000 7fbec3b8f700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc19de90 con 0x7fbebc1003b0 2023-11-16T16:38:55.071 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.066+0000 7fbec192b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbebc0ff1c0 0x7fbebc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:55.071 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.066+0000 7fbec192b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbebc0ff1c0 0x7fbebc068930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42890/0 (socket says 172.21.15.1:42890) 2023-11-16T16:38:55.071 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.066+0000 7fbec192b700 1 -- 172.21.15.1:0/1054159300 learned_addr learned my addr 172.21.15.1:0/1054159300 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:55.071 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.066+0000 7fbec112a700 1 --2- 172.21.15.1:0/1054159300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbebc1003b0 0x7fbebc19ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:55.071 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.067+0000 7fbec212c700 1 --2- 172.21.15.1:0/1054159300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbebc100d40 0x7fbebc19d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:55.071 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.067+0000 7fbec192b700 1 -- 172.21.15.1:0/1054159300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbebc1003b0 msgr2=0x7fbebc19ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:55.072 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.067+0000 7fbec192b700 1 --2- 172.21.15.1:0/1054159300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbebc1003b0 0x7fbebc19ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:55.072 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.067+0000 7fbec192b700 1 -- 172.21.15.1:0/1054159300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbebc100d40 msgr2=0x7fbebc19d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:55.072 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.067+0000 7fbec192b700 1 --2- 172.21.15.1:0/1054159300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbebc100d40 0x7fbebc19d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:55.072 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.067+0000 7fbec192b700 1 -- 172.21.15.1:0/1054159300 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb80095f0 con 0x7fbebc0ff1c0 2023-11-16T16:38:55.072 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.067+0000 7fbec212c700 1 --2- 172.21.15.1:0/1054159300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbebc100d40 0x7fbebc19d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:38:55.072 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.067+0000 7fbec192b700 1 --2- 172.21.15.1:0/1054159300 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbebc0ff1c0 0x7fbebc068930 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8006010 tx=0x7fbeb8005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:55.072 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.067+0000 7fbeaeffd700 1 -- 172.21.15.1:0/1054159300 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb801d070 con 0x7fbebc0ff1c0 2023-11-16T16:38:55.072 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.067+0000 7fbeaeffd700 1 -- 172.21.15.1:0/1054159300 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbeb80040e0 con 0x7fbebc0ff1c0 2023-11-16T16:38:55.072 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.067+0000 7fbec3b8f700 1 -- 172.21.15.1:0/1054159300 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbebc10ab20 con 0x7fbebc0ff1c0 2023-11-16T16:38:55.072 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.067+0000 7fbeaeffd700 1 -- 172.21.15.1:0/1054159300 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb8022e70 con 0x7fbebc0ff1c0 2023-11-16T16:38:55.072 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.067+0000 7fbec3b8f700 1 -- 172.21.15.1:0/1054159300 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbebc10a990 con 0x7fbebc0ff1c0 2023-11-16T16:38:55.073 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.068+0000 7fbec3b8f700 1 -- 172.21.15.1:0/1054159300 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbebc106140 con 0x7fbebc0ff1c0 2023-11-16T16:38:55.073 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.070+0000 7fbeaeffd700 1 -- 172.21.15.1:0/1054159300 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fbeb8022570 con 0x7fbebc0ff1c0 2023-11-16T16:38:55.073 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.070+0000 7fbeaeffd700 1 --2- 172.21.15.1:0/1054159300 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbea80a1150 0x7fbea80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:55.073 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.070+0000 7fbeaeffd700 1 -- 172.21.15.1:0/1054159300 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbeb80cdfc0 con 0x7fbebc0ff1c0 2023-11-16T16:38:55.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.070+0000 7fbec112a700 1 --2- 172.21.15.1:0/1054159300 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbea80a1150 0x7fbea80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:55.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.071+0000 7fbec112a700 1 --2- 172.21.15.1:0/1054159300 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbea80a1150 0x7fbea80a3600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0005950 tx=0x7fbeb00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:55.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.074+0000 7fbeaeffd700 1 -- 172.21.15.1:0/1054159300 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbeb8097840 con 0x7fbebc0ff1c0 2023-11-16T16:38:55.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.275+0000 7fbec3b8f700 1 -- 172.21.15.1:0/1054159300 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbebc061700 con 0x7fbebc0ff1c0 2023-11-16T16:38:55.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.276+0000 7fbeaeffd700 1 -- 172.21.15.1:0/1054159300 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22469 (secure 0 0 0) 0x7fbeb809ae60 con 0x7fbebc0ff1c0 2023-11-16T16:38:55.279 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:38:55.279 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":86,"fsid":"35d9b010-849d-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:30:06.217778+0000","modified":"2023-11-16T16:38:03.179244+0000","last_up_change":"2023-11-16T16:36:31.735403+0000","last_in_change":"2023-11-16T16:36:14.083021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:33:15.916723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-16T16:37:12.158031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-16T16:37:13.272343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a8480507-5275-4e52-953b-626110beb436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6803","nonce":2469671609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6805","nonce":2469671609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6809","nonce":2469671609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6807","nonce":2469671609}]},"public_addr":"172.21.15.1:6803/2469671609","cluster_addr":"172.21.15.1:6805/2469671609","heartbeat_back_addr":"172.21.15.1:6809/2469671609","heartbeat_front_addr":"172.21.15.1:6807/2469671609","state":["exists","up"]},{"osd":1,"uuid":"1d011db2-99d1-4575-bdc5-51f88a308fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6801","nonce":247994796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6803","nonce":247994796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6807","nonce":247994796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6805","nonce":247994796}]},"public_addr":"172.21.15.26:6801/247994796","cluster_addr":"172.21.15.26:6803/247994796","heartbeat_back_addr":"172.21.15.26:6807/247994796","heartbeat_front_addr":"172.21.15.26:6805/247994796","state":["exists","up"]},{"osd":2,"uuid":"4d7a2bd1-3c8e-4546-a243-f0354473b031","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6801","nonce":1659316290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6803","nonce":1659316290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6807","nonce":1659316290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6805","nonce":1659316290}]},"public_addr":"172.21.15.53:6801/1659316290","cluster_addr":"172.21.15.53:6803/1659316290","heartbeat_back_addr":"172.21.15.53:6807/1659316290","heartbeat_front_addr":"172.21.15.53:6805/1659316290","state":["exists","up"]},{"osd":3,"uuid":"9c290fd9-e121-4a81-9687-b0eb93debd86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6811","nonce":2728987281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6813","nonce":2728987281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6817","nonce":2728987281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6815","nonce":2728987281}]},"public_addr":"172.21.15.1:6811/2728987281","cluster_addr":"172.21.15.1:6813/2728987281","heartbeat_back_addr":"172.21.15.1:6817/2728987281","heartbeat_front_addr":"172.21.15.1:6815/2728987281","state":["exists","up"]},{"osd":4,"uuid":"533226bc-cc57-4f15-9e29-820294d57e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6809","nonce":2954740024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6811","nonce":2954740024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6815","nonce":2954740024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6813","nonce":2954740024}]},"public_addr":"172.21.15.26:6809/2954740024","cluster_addr":"172.21.15.26:6811/2954740024","heartbeat_back_addr":"172.21.15.26:6815/2954740024","heartbeat_front_addr":"172.21.15.26:6813/2954740024","state":["exists","up"]},{"osd":5,"uuid":"d3ab25eb-e1d4-4557-bc89-efc9072401ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6809","nonce":3966666318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6811","nonce":3966666318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6815","nonce":3966666318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6813","nonce":3966666318}]},"public_addr":"172.21.15.53:6809/3966666318","cluster_addr":"172.21.15.53:6811/3966666318","heartbeat_back_addr":"172.21.15.53:6815/3966666318","heartbeat_front_addr":"172.21.15.53:6813/3966666318","state":["exists","up"]},{"osd":6,"uuid":"180f506e-afb8-4043-b0b4-2dc86f6d4bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6819","nonce":3364032646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6821","nonce":3364032646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6825","nonce":3364032646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6823","nonce":3364032646}]},"public_addr":"172.21.15.1:6819/3364032646","cluster_addr":"172.21.15.1:6821/3364032646","heartbeat_back_addr":"172.21.15.1:6825/3364032646","heartbeat_front_addr":"172.21.15.1:6823/3364032646","state":["exists","up"]},{"osd":7,"uuid":"0d837bac-194f-498d-b7fb-b3147e992329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6817","nonce":3766901391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6819","nonce":3766901391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6823","nonce":3766901391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6821","nonce":3766901391}]},"public_addr":"172.21.15.26:6817/3766901391","cluster_addr":"172.21.15.26:6819/3766901391","heartbeat_back_addr":"172.21.15.26:6823/3766901391","heartbeat_front_addr":"172.21.15.26:6821/3766901391","state":["exists","up"]},{"osd":8,"uuid":"933f231a-7454-47b0-9291-f1fa64f0baec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6817","nonce":3224335871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6819","nonce":3224335871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6823","nonce":3224335871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6821","nonce":3224335871}]},"public_addr":"172.21.15.53:6817/3224335871","cluster_addr":"172.21.15.53:6819/3224335871","heartbeat_back_addr":"172.21.15.53:6823/3224335871","heartbeat_front_addr":"172.21.15.53:6821/3224335871","state":["exists","up"]},{"osd":9,"uuid":"245fcbb3-c651-4703-ab80-309dc98bd983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6827","nonce":1391199844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6829","nonce":1391199844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6833","nonce":1391199844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6831","nonce":1391199844}]},"public_addr":"172.21.15.1:6827/1391199844","cluster_addr":"172.21.15.1:6829/1391199844","heartbeat_back_addr":"172.21.15.1:6833/1391199844","heartbeat_front_addr":"172.21.15.1:6831/1391199844","state":["exists","up"]},{"osd":10,"uuid":"2c090e7a-5240-4d52-ae4f-e94fe05383d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6825","nonce":1773545447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6827","nonce":1773545447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6831","nonce":1773545447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6829","nonce":1773545447}]},"public_addr":"172.21.15.26:6825/1773545447","cluster_addr":"172.21.15.26:6827/1773545447","heartbeat_back_addr":"172.21.15.26:6831/1773545447","heartbeat_front_addr":"172.21.15.26:6829/1773545447","state":["exists","up"]},{"osd":11,"uuid":"4511e32d-e469-45e1-8310-0ff400cb7b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6825","nonce":2564669546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6827","nonce":2564669546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6831","nonce":2564669546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6829","nonce":2564669546}]},"public_addr":"172.21.15.53:6825/2564669546","cluster_addr":"172.21.15.53:6827/2564669546","heartbeat_back_addr":"172.21.15.53:6831/2564669546","heartbeat_front_addr":"172.21.15.53:6829/2564669546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:28.140250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:49.885461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:12.521487+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:34.926197+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:56.815594+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:18.387751+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:40.587106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:02.319370+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:23.832298+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:46.338040+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:08.730414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:30.050070+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1494336948":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6801/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6800/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6800/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/1641976669":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/2895923638":"2023-11-17T16:30:35.064289+0000","172.21.15.1:6801/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/78811711":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/4131957708":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/344403698":"2023-11-17T16:30:35.064289+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:38:55.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.278+0000 7fbec3b8f700 1 -- 172.21.15.1:0/1054159300 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbea80a1150 msgr2=0x7fbea80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:55.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.278+0000 7fbec3b8f700 1 --2- 172.21.15.1:0/1054159300 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbea80a1150 0x7fbea80a3600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0005950 tx=0x7fbeb00058e0 comp rx=0 tx=0).stop 2023-11-16T16:38:55.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.279+0000 7fbec3b8f700 1 -- 172.21.15.1:0/1054159300 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbebc0ff1c0 msgr2=0x7fbebc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:55.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.279+0000 7fbec3b8f700 1 --2- 172.21.15.1:0/1054159300 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbebc0ff1c0 0x7fbebc068930 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8006010 tx=0x7fbeb8005ef0 comp rx=0 tx=0).stop 2023-11-16T16:38:55.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.279+0000 7fbec3b8f700 1 -- 172.21.15.1:0/1054159300 shutdown_connections 2023-11-16T16:38:55.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.279+0000 7fbec3b8f700 1 --2- 172.21.15.1:0/1054159300 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbea80a1150 0x7fbea80a3600 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:55.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.279+0000 7fbec3b8f700 1 --2- 172.21.15.1:0/1054159300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbebc100d40 0x7fbebc19d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:55.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.279+0000 7fbec3b8f700 1 --2- 172.21.15.1:0/1054159300 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbebc0ff1c0 0x7fbebc068930 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:55.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.279+0000 7fbec3b8f700 1 --2- 172.21.15.1:0/1054159300 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbebc1003b0 0x7fbebc19ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:55.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.279+0000 7fbec3b8f700 1 -- 172.21.15.1:0/1054159300 >> 172.21.15.1:0/1054159300 conn(0x7fbebc0fa7c0 msgr2=0x7fbebc0fcaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:55.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.279+0000 7fbec3b8f700 1 -- 172.21.15.1:0/1054159300 shutdown_connections 2023-11-16T16:38:55.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.279+0000 7fbec3b8f700 1 -- 172.21.15.1:0/1054159300 wait complete. 2023-11-16T16:38:55.293 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-16T16:33:15.916723+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-16T16:37:12.158031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-16T16:37:13.272343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-16T16:38:55.293 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-16T16:38:55.424 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.420+0000 7fca2ece4700 1 -- 172.21.15.1:0/254765986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca280ff1c0 msgr2=0x7fca280ff5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:55.424 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.420+0000 7fca2ece4700 1 --2- 172.21.15.1:0/254765986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca280ff1c0 0x7fca280ff5d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fca1c009990 tx=0x7fca1c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:55.424 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.420+0000 7fca2ece4700 1 -- 172.21.15.1:0/254765986 shutdown_connections 2023-11-16T16:38:55.425 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.420+0000 7fca2ece4700 1 --2- 172.21.15.1:0/254765986 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca28100d40 0x7fca28107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:55.425 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.420+0000 7fca2ece4700 1 --2- 172.21.15.1:0/254765986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca280ff1c0 0x7fca280ff5d0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:55.425 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.420+0000 7fca2ece4700 1 --2- 172.21.15.1:0/254765986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca281003b0 0x7fca28100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:55.425 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.420+0000 7fca2ece4700 1 -- 172.21.15.1:0/254765986 >> 172.21.15.1:0/254765986 conn(0x7fca280fa800 msgr2=0x7fca280fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:55.425 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.420+0000 7fca2ece4700 1 -- 172.21.15.1:0/254765986 shutdown_connections 2023-11-16T16:38:55.425 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.420+0000 7fca2ece4700 1 -- 172.21.15.1:0/254765986 wait complete. 2023-11-16T16:38:55.425 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.421+0000 7fca2ece4700 1 Processor -- start 2023-11-16T16:38:55.425 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.421+0000 7fca2ece4700 1 -- start start 2023-11-16T16:38:55.425 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.421+0000 7fca2ece4700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca280ff1c0 0x7fca28069950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:55.425 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.421+0000 7fca2ece4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca281003b0 0x7fca2819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:55.426 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.421+0000 7fca2ece4700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca28100d40 0x7fca2819d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:55.426 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.421+0000 7fca2ece4700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2819db40 con 0x7fca28100d40 2023-11-16T16:38:55.426 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.421+0000 7fca2ece4700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2819dcb0 con 0x7fca281003b0 2023-11-16T16:38:55.426 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.421+0000 7fca2ece4700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2819de70 con 0x7fca280ff1c0 2023-11-16T16:38:55.426 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.421+0000 7fca2d281700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca28100d40 0x7fca2819d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:55.426 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.421+0000 7fca2d281700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca28100d40 0x7fca2819d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42908/0 (socket says 172.21.15.1:42908) 2023-11-16T16:38:55.426 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.421+0000 7fca2d281700 1 -- 172.21.15.1:0/2438290354 learned_addr learned my addr 172.21.15.1:0/2438290354 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:55.426 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.421+0000 7fca2ca80700 1 --2- 172.21.15.1:0/2438290354 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca280ff1c0 0x7fca28069950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:55.426 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.421+0000 7fca27fff700 1 --2- 172.21.15.1:0/2438290354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca281003b0 0x7fca2819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:55.427 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.421+0000 7fca2d281700 1 -- 172.21.15.1:0/2438290354 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca280ff1c0 msgr2=0x7fca28069950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:55.427 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.421+0000 7fca2d281700 1 --2- 172.21.15.1:0/2438290354 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca280ff1c0 0x7fca28069950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:55.427 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.421+0000 7fca2d281700 1 -- 172.21.15.1:0/2438290354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca281003b0 msgr2=0x7fca2819ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:55.427 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.421+0000 7fca2d281700 1 --2- 172.21.15.1:0/2438290354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca281003b0 0x7fca2819ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:55.427 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.421+0000 7fca2d281700 1 -- 172.21.15.1:0/2438290354 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca1c0095f0 con 0x7fca28100d40 2023-11-16T16:38:55.427 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.422+0000 7fca2d281700 1 --2- 172.21.15.1:0/2438290354 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca28100d40 0x7fca2819d3a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fca2000ada0 tx=0x7fca2000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:55.427 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.422+0000 7fca25ffb700 1 -- 172.21.15.1:0/2438290354 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2000f070 con 0x7fca28100d40 2023-11-16T16:38:55.427 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.422+0000 7fca25ffb700 1 -- 172.21.15.1:0/2438290354 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca20007b30 con 0x7fca28100d40 2023-11-16T16:38:55.427 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.422+0000 7fca25ffb700 1 -- 172.21.15.1:0/2438290354 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca200053e0 con 0x7fca28100d40 2023-11-16T16:38:55.427 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.422+0000 7fca2ece4700 1 -- 172.21.15.1:0/2438290354 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca2810ab80 con 0x7fca28100d40 2023-11-16T16:38:55.427 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.422+0000 7fca2ece4700 1 -- 172.21.15.1:0/2438290354 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca2810a9f0 con 0x7fca28100d40 2023-11-16T16:38:55.428 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.423+0000 7fca2ece4700 1 -- 172.21.15.1:0/2438290354 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca2819e510 con 0x7fca28100d40 2023-11-16T16:38:55.428 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.424+0000 7fca25ffb700 1 -- 172.21.15.1:0/2438290354 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fca2000c980 con 0x7fca28100d40 2023-11-16T16:38:55.429 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.425+0000 7fca25ffb700 1 --2- 172.21.15.1:0/2438290354 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fca180a1150 0x7fca180a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:55.429 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.425+0000 7fca2ca80700 1 --2- 172.21.15.1:0/2438290354 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fca180a1150 0x7fca180a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:55.429 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.425+0000 7fca25ffb700 1 -- 172.21.15.1:0/2438290354 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fca200cbee0 con 0x7fca28100d40 2023-11-16T16:38:55.429 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.425+0000 7fca2ca80700 1 --2- 172.21.15.1:0/2438290354 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fca180a1150 0x7fca180a3600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fca1c00b5c0 tx=0x7fca1c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:55.431 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.429+0000 7fca25ffb700 1 -- 172.21.15.1:0/2438290354 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fca200956c0 con 0x7fca28100d40 2023-11-16T16:38:55.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:55 smithi026 ceph-mon[100112]: pgmap v312: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:38:55.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:55 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/697398979' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:38:55.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:55 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/2931934379' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:38:55.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:55 smithi053 ceph-mon[100638]: pgmap v312: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:38:55.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:55 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/697398979' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:38:55.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:55 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/2931934379' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:38:55.629 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.628+0000 7fca2ece4700 1 -- 172.21.15.1:0/2438290354 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fca28106100 con 0x7fca28100d40 2023-11-16T16:38:55.630 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.628+0000 7fca25ffb700 1 -- 172.21.15.1:0/2438290354 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7fca20098ce0 con 0x7fca28100d40 2023-11-16T16:38:55.630 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2023-11-16T16:38:55.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.630+0000 7fca2ece4700 1 -- 172.21.15.1:0/2438290354 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fca180a1150 msgr2=0x7fca180a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:55.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.630+0000 7fca2ece4700 1 --2- 172.21.15.1:0/2438290354 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fca180a1150 0x7fca180a3600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fca1c00b5c0 tx=0x7fca1c005fd0 comp rx=0 tx=0).stop 2023-11-16T16:38:55.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.630+0000 7fca2ece4700 1 -- 172.21.15.1:0/2438290354 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca28100d40 msgr2=0x7fca2819d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:55.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.630+0000 7fca2ece4700 1 --2- 172.21.15.1:0/2438290354 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca28100d40 0x7fca2819d3a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fca2000ada0 tx=0x7fca2000c330 comp rx=0 tx=0).stop 2023-11-16T16:38:55.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.630+0000 7fca2ece4700 1 -- 172.21.15.1:0/2438290354 shutdown_connections 2023-11-16T16:38:55.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.630+0000 7fca2ece4700 1 --2- 172.21.15.1:0/2438290354 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fca180a1150 0x7fca180a3600 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:55.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.630+0000 7fca2ece4700 1 --2- 172.21.15.1:0/2438290354 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca28100d40 0x7fca2819d3a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:55.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.630+0000 7fca2ece4700 1 --2- 172.21.15.1:0/2438290354 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca280ff1c0 0x7fca28069950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:55.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.630+0000 7fca2ece4700 1 --2- 172.21.15.1:0/2438290354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca281003b0 0x7fca2819ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:55.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.630+0000 7fca2ece4700 1 -- 172.21.15.1:0/2438290354 >> 172.21.15.1:0/2438290354 conn(0x7fca280fa800 msgr2=0x7fca280fca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:55.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.630+0000 7fca2ece4700 1 -- 172.21.15.1:0/2438290354 shutdown_connections 2023-11-16T16:38:55.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.631+0000 7fca2ece4700 1 -- 172.21.15.1:0/2438290354 wait complete. 2023-11-16T16:38:55.644 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:55 smithi001 ceph-mon[98800]: pgmap v312: 97 pgs: 97 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:38:55.644 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:55 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/697398979' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:38:55.644 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:55 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/2931934379' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:38:55.645 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-16T16:38:55.776 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.772+0000 7f855ceb9700 1 -- 172.21.15.1:0/1339010876 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8558101e10 msgr2=0x7f85581086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:55.777 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.772+0000 7f855ceb9700 1 --2- 172.21.15.1:0/1339010876 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8558101e10 0x7f85581086d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f855000b0a0 tx=0x7f855000b3b0 comp rx=0 tx=0).stop 2023-11-16T16:38:55.777 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.772+0000 7f855ceb9700 1 -- 172.21.15.1:0/1339010876 shutdown_connections 2023-11-16T16:38:55.777 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.772+0000 7f855ceb9700 1 --2- 172.21.15.1:0/1339010876 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8558101e10 0x7f85581086d0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:55.777 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.772+0000 7f855ceb9700 1 --2- 172.21.15.1:0/1339010876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8558100290 0x7f85581006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:55.777 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.772+0000 7f855ceb9700 1 --2- 172.21.15.1:0/1339010876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8558101480 0x7f85581018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:55.777 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.772+0000 7f855ceb9700 1 -- 172.21.15.1:0/1339010876 >> 172.21.15.1:0/1339010876 conn(0x7f85580fb8b0 msgr2=0x7f85580fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:55.777 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.772+0000 7f855ceb9700 1 -- 172.21.15.1:0/1339010876 shutdown_connections 2023-11-16T16:38:55.777 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.772+0000 7f855ceb9700 1 -- 172.21.15.1:0/1339010876 wait complete. 2023-11-16T16:38:55.777 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.773+0000 7f855ceb9700 1 Processor -- start 2023-11-16T16:38:55.777 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.773+0000 7f855ceb9700 1 -- start start 2023-11-16T16:38:55.777 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.773+0000 7f855ceb9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8558100290 0x7f85580748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:55.778 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.773+0000 7f855ceb9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8558101480 0x7f8558072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:55.778 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.773+0000 7f855ceb9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8558101e10 0x7f8558073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:55.778 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.773+0000 7f855ceb9700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8558073bc0 con 0x7f8558100290 2023-11-16T16:38:55.778 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.773+0000 7f855ceb9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8558073d30 con 0x7f8558101e10 2023-11-16T16:38:55.778 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.773+0000 7f855ceb9700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85581a00d0 con 0x7f8558101480 2023-11-16T16:38:55.778 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.773+0000 7f855659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8558100290 0x7f85580748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:55.778 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.773+0000 7f855659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8558100290 0x7f85580748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42918/0 (socket says 172.21.15.1:42918) 2023-11-16T16:38:55.778 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.773+0000 7f855659c700 1 -- 172.21.15.1:0/3494619518 learned_addr learned my addr 172.21.15.1:0/3494619518 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:55.778 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.773+0000 7f8556d9d700 1 --2- 172.21.15.1:0/3494619518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8558101e10 0x7f8558073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:55.778 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.773+0000 7f8555d9b700 1 --2- 172.21.15.1:0/3494619518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8558101480 0x7f8558072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:55.779 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.774+0000 7f855659c700 1 -- 172.21.15.1:0/3494619518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8558101480 msgr2=0x7f8558072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:55.779 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.774+0000 7f855659c700 1 --2- 172.21.15.1:0/3494619518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8558101480 0x7f8558072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:55.779 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.774+0000 7f855659c700 1 -- 172.21.15.1:0/3494619518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8558101e10 msgr2=0x7f8558073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:55.779 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.774+0000 7f855659c700 1 --2- 172.21.15.1:0/3494619518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8558101e10 0x7f8558073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:55.779 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.774+0000 7f855659c700 1 -- 172.21.15.1:0/3494619518 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8550009d00 con 0x7f8558100290 2023-11-16T16:38:55.779 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.774+0000 7f855659c700 1 --2- 172.21.15.1:0/3494619518 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8558100290 0x7f85580748a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f854c00d980 tx=0x7f854c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:55.779 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.774+0000 7f85477fe700 1 -- 172.21.15.1:0/3494619518 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854c004020 con 0x7f8558100290 2023-11-16T16:38:55.779 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.774+0000 7f85477fe700 1 -- 172.21.15.1:0/3494619518 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f854c009730 con 0x7f8558100290 2023-11-16T16:38:55.779 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.774+0000 7f855ceb9700 1 -- 172.21.15.1:0/3494619518 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f855810bc20 con 0x7f8558100290 2023-11-16T16:38:55.779 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.774+0000 7f85477fe700 1 -- 172.21.15.1:0/3494619518 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854c003a80 con 0x7f8558100290 2023-11-16T16:38:55.779 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.774+0000 7f855ceb9700 1 -- 172.21.15.1:0/3494619518 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f855810ba90 con 0x7f8558100290 2023-11-16T16:38:55.779 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.775+0000 7f855ceb9700 1 -- 172.21.15.1:0/3494619518 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8558107820 con 0x7f8558100290 2023-11-16T16:38:55.780 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.776+0000 7f85477fe700 1 -- 172.21.15.1:0/3494619518 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f854c0098a0 con 0x7f8558100290 2023-11-16T16:38:55.780 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.776+0000 7f85477fe700 1 --2- 172.21.15.1:0/3494619518 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f85400a1150 0x7f85400a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:55.781 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.777+0000 7f85477fe700 1 -- 172.21.15.1:0/3494619518 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f854c0cc5e0 con 0x7f8558100290 2023-11-16T16:38:55.781 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.777+0000 7f8555d9b700 1 --2- 172.21.15.1:0/3494619518 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f85400a1150 0x7f85400a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:55.781 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.777+0000 7f8555d9b700 1 --2- 172.21.15.1:0/3494619518 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f85400a1150 0x7f85400a3600 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8548009bf0 tx=0x7f8548009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:55.782 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.781+0000 7f85477fe700 1 -- 172.21.15.1:0/3494619518 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f854c095d30 con 0x7f8558100290 2023-11-16T16:38:55.982 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.980+0000 7f855ceb9700 1 -- 172.21.15.1:0/3494619518 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8558061700 con 0x7f8558100290 2023-11-16T16:38:55.982 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.980+0000 7f85477fe700 1 -- 172.21.15.1:0/3494619518 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7f854c099350 con 0x7f8558100290 2023-11-16T16:38:55.982 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2023-11-16T16:38:55.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.983+0000 7f855ceb9700 1 -- 172.21.15.1:0/3494619518 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f85400a1150 msgr2=0x7f85400a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:55.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.983+0000 7f855ceb9700 1 --2- 172.21.15.1:0/3494619518 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f85400a1150 0x7f85400a3600 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8548009bf0 tx=0x7f8548009260 comp rx=0 tx=0).stop 2023-11-16T16:38:55.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.983+0000 7f855ceb9700 1 -- 172.21.15.1:0/3494619518 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8558100290 msgr2=0x7f85580748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:55.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.983+0000 7f855ceb9700 1 --2- 172.21.15.1:0/3494619518 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8558100290 0x7f85580748a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f854c00d980 tx=0x7f854c00dc90 comp rx=0 tx=0).stop 2023-11-16T16:38:55.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.983+0000 7f855ceb9700 1 -- 172.21.15.1:0/3494619518 shutdown_connections 2023-11-16T16:38:55.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.983+0000 7f855ceb9700 1 --2- 172.21.15.1:0/3494619518 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f85400a1150 0x7f85400a3600 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:55.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.983+0000 7f855ceb9700 1 --2- 172.21.15.1:0/3494619518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8558101e10 0x7f8558073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:55.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.983+0000 7f855ceb9700 1 --2- 172.21.15.1:0/3494619518 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8558100290 0x7f85580748a0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:55.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.983+0000 7f855ceb9700 1 --2- 172.21.15.1:0/3494619518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8558101480 0x7f8558072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:55.986 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.983+0000 7f855ceb9700 1 -- 172.21.15.1:0/3494619518 >> 172.21.15.1:0/3494619518 conn(0x7f85580fb8b0 msgr2=0x7f8558104960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:55.986 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.983+0000 7f855ceb9700 1 -- 172.21.15.1:0/3494619518 shutdown_connections 2023-11-16T16:38:55.986 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:55.983+0000 7f855ceb9700 1 -- 172.21.15.1:0/3494619518 wait complete. 2023-11-16T16:38:55.997 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-16T16:38:56.126 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.124+0000 7fbcb3ed6700 1 -- 172.21.15.1:0/3796885225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbcac068b00 msgr2=0x7fbcac102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:56.126 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.124+0000 7fbcb3ed6700 1 --2- 172.21.15.1:0/3796885225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbcac068b00 0x7fbcac102e00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fbca8009990 tx=0x7fbca8009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:56.126 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.124+0000 7fbcb3ed6700 1 -- 172.21.15.1:0/3796885225 shutdown_connections 2023-11-16T16:38:56.126 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.124+0000 7fbcb3ed6700 1 --2- 172.21.15.1:0/3796885225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcac105e50 0x7fbcac108300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:56.126 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.124+0000 7fbcb3ed6700 1 --2- 172.21.15.1:0/3796885225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbcac068b00 0x7fbcac102e00 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:56.126 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.124+0000 7fbcb3ed6700 1 --2- 172.21.15.1:0/3796885225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcac103340 0x7fbcac1057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:56.126 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.124+0000 7fbcb3ed6700 1 -- 172.21.15.1:0/3796885225 >> 172.21.15.1:0/3796885225 conn(0x7fbcac0fa750 msgr2=0x7fbcac0fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:56.126 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.124+0000 7fbcb3ed6700 1 -- 172.21.15.1:0/3796885225 shutdown_connections 2023-11-16T16:38:56.127 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.124+0000 7fbcb3ed6700 1 -- 172.21.15.1:0/3796885225 wait complete. 2023-11-16T16:38:56.127 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.124+0000 7fbcb3ed6700 1 Processor -- start 2023-11-16T16:38:56.127 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.124+0000 7fbcb3ed6700 1 -- start start 2023-11-16T16:38:56.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.125+0000 7fbcb3ed6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcac068b00 0x7fbcac100760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:56.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.125+0000 7fbcb3ed6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbcac103340 0x7fbcac0feda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:56.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.125+0000 7fbcb3ed6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcac105e50 0x7fbcac0ff2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:56.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.125+0000 7fbcb3ed6700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac0ffa80 con 0x7fbcac103340 2023-11-16T16:38:56.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.125+0000 7fbcb3ed6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac0ffbf0 con 0x7fbcac105e50 2023-11-16T16:38:56.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.125+0000 7fbcb3ed6700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac19ff90 con 0x7fbcac068b00 2023-11-16T16:38:56.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.125+0000 7fbcb1471700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbcac103340 0x7fbcac0feda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:56.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.125+0000 7fbcb1471700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbcac103340 0x7fbcac0feda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42934/0 (socket says 172.21.15.1:42934) 2023-11-16T16:38:56.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.125+0000 7fbcb1471700 1 -- 172.21.15.1:0/1065218116 learned_addr learned my addr 172.21.15.1:0/1065218116 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:56.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.125+0000 7fbcb1c72700 1 --2- 172.21.15.1:0/1065218116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcac068b00 0x7fbcac100760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:56.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.125+0000 7fbcb2473700 1 --2- 172.21.15.1:0/1065218116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcac105e50 0x7fbcac0ff2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:56.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.125+0000 7fbcb1471700 1 -- 172.21.15.1:0/1065218116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcac068b00 msgr2=0x7fbcac100760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:56.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.125+0000 7fbcb1471700 1 --2- 172.21.15.1:0/1065218116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcac068b00 0x7fbcac100760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:56.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.125+0000 7fbcb1471700 1 -- 172.21.15.1:0/1065218116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcac105e50 msgr2=0x7fbcac0ff2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:56.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.125+0000 7fbcb1471700 1 --2- 172.21.15.1:0/1065218116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcac105e50 0x7fbcac0ff2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:56.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.125+0000 7fbcb1471700 1 -- 172.21.15.1:0/1065218116 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca80095f0 con 0x7fbcac103340 2023-11-16T16:38:56.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.125+0000 7fbcb1471700 1 --2- 172.21.15.1:0/1065218116 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbcac103340 0x7fbcac0feda0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c00c9b0 tx=0x7fbc9c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:56.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.125+0000 7fbca2ffd700 1 -- 172.21.15.1:0/1065218116 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c007b00 con 0x7fbcac103340 2023-11-16T16:38:56.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.125+0000 7fbca2ffd700 1 -- 172.21.15.1:0/1065218116 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc9c007c60 con 0x7fbcac103340 2023-11-16T16:38:56.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.125+0000 7fbcb3ed6700 1 -- 172.21.15.1:0/1065218116 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcac10bbd0 con 0x7fbcac103340 2023-11-16T16:38:56.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.125+0000 7fbca2ffd700 1 -- 172.21.15.1:0/1065218116 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c018660 con 0x7fbcac103340 2023-11-16T16:38:56.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.125+0000 7fbcb3ed6700 1 -- 172.21.15.1:0/1065218116 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcac10ba40 con 0x7fbcac103340 2023-11-16T16:38:56.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.126+0000 7fbcb3ed6700 1 -- 172.21.15.1:0/1065218116 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcac0fc2c0 con 0x7fbcac103340 2023-11-16T16:38:56.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.128+0000 7fbca2ffd700 1 -- 172.21.15.1:0/1065218116 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fbc9c018880 con 0x7fbcac103340 2023-11-16T16:38:56.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.128+0000 7fbca2ffd700 1 --2- 172.21.15.1:0/1065218116 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbc980a11d0 0x7fbc980a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:56.132 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.128+0000 7fbca2ffd700 1 -- 172.21.15.1:0/1065218116 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbc9c0cc950 con 0x7fbcac103340 2023-11-16T16:38:56.132 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.128+0000 7fbcb1c72700 1 --2- 172.21.15.1:0/1065218116 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbc980a11d0 0x7fbc980a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:56.132 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.129+0000 7fbcb1c72700 1 --2- 172.21.15.1:0/1065218116 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbc980a11d0 0x7fbc980a3680 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbca8006010 tx=0x7fbca800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:56.134 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.132+0000 7fbca2ffd700 1 -- 172.21.15.1:0/1065218116 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbc9c0960a0 con 0x7fbcac103340 2023-11-16T16:38:56.336 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.335+0000 7fbcb3ed6700 1 -- 172.21.15.1:0/1065218116 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbcac061700 con 0x7fbcac103340 2023-11-16T16:38:56.337 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.335+0000 7fbca2ffd700 1 -- 172.21.15.1:0/1065218116 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbc9c0996c0 con 0x7fbcac103340 2023-11-16T16:38:56.337 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2023-11-16T16:38:56.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.337+0000 7fbcb3ed6700 1 -- 172.21.15.1:0/1065218116 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbc980a11d0 msgr2=0x7fbc980a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:56.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.337+0000 7fbcb3ed6700 1 --2- 172.21.15.1:0/1065218116 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbc980a11d0 0x7fbc980a3680 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbca8006010 tx=0x7fbca800b260 comp rx=0 tx=0).stop 2023-11-16T16:38:56.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.337+0000 7fbcb3ed6700 1 -- 172.21.15.1:0/1065218116 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbcac103340 msgr2=0x7fbcac0feda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:56.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.337+0000 7fbcb3ed6700 1 --2- 172.21.15.1:0/1065218116 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbcac103340 0x7fbcac0feda0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c00c9b0 tx=0x7fbc9c00cd70 comp rx=0 tx=0).stop 2023-11-16T16:38:56.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.337+0000 7fbcb3ed6700 1 -- 172.21.15.1:0/1065218116 shutdown_connections 2023-11-16T16:38:56.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.337+0000 7fbcb3ed6700 1 --2- 172.21.15.1:0/1065218116 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbc980a11d0 0x7fbc980a3680 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:56.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.337+0000 7fbcb3ed6700 1 --2- 172.21.15.1:0/1065218116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcac105e50 0x7fbcac0ff2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:56.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.337+0000 7fbcb3ed6700 1 --2- 172.21.15.1:0/1065218116 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbcac068b00 0x7fbcac100760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:56.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.337+0000 7fbcb3ed6700 1 --2- 172.21.15.1:0/1065218116 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbcac103340 0x7fbcac0feda0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:56.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.338+0000 7fbcb3ed6700 1 -- 172.21.15.1:0/1065218116 >> 172.21.15.1:0/1065218116 conn(0x7fbcac0fa750 msgr2=0x7fbcac1016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:56.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.338+0000 7fbcb3ed6700 1 -- 172.21.15.1:0/1065218116 shutdown_connections 2023-11-16T16:38:56.341 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.338+0000 7fbcb3ed6700 1 -- 172.21.15.1:0/1065218116 wait complete. 2023-11-16T16:38:56.351 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:56 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1054159300' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:38:56.351 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:56 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/2438290354' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:38:56.351 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:56 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3494619518' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:38:56.352 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-11-16T16:38:56.352 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:38:56.477 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.475+0000 7f2c218f7700 1 -- 172.21.15.1:0/2835715349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c1c100df0 msgr2=0x7f2c1c107690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:56.477 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.475+0000 7f2c218f7700 1 --2- 172.21.15.1:0/2835715349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c1c100df0 0x7f2c1c107690 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2c1400b0a0 tx=0x7f2c1400b3b0 comp rx=0 tx=0).stop 2023-11-16T16:38:56.477 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.475+0000 7f2c218f7700 1 -- 172.21.15.1:0/2835715349 shutdown_connections 2023-11-16T16:38:56.477 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.475+0000 7f2c218f7700 1 --2- 172.21.15.1:0/2835715349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c1c100df0 0x7f2c1c107690 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:56.478 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.475+0000 7f2c218f7700 1 --2- 172.21.15.1:0/2835715349 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c1c0ff1b0 0x7f2c1c0ff5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:56.478 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.475+0000 7f2c218f7700 1 --2- 172.21.15.1:0/2835715349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c1c100440 0x7f2c1c1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:56.478 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.475+0000 7f2c218f7700 1 -- 172.21.15.1:0/2835715349 >> 172.21.15.1:0/2835715349 conn(0x7f2c1c0fa7d0 msgr2=0x7f2c1c0fcc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:56.478 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.475+0000 7f2c218f7700 1 -- 172.21.15.1:0/2835715349 shutdown_connections 2023-11-16T16:38:56.478 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.475+0000 7f2c218f7700 1 -- 172.21.15.1:0/2835715349 wait complete. 2023-11-16T16:38:56.480 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.476+0000 7f2c218f7700 1 Processor -- start 2023-11-16T16:38:56.480 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.476+0000 7f2c218f7700 1 -- start start 2023-11-16T16:38:56.480 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.477+0000 7f2c218f7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c1c0ff1b0 0x7f2c1c069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:56.480 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.477+0000 7f2c218f7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c1c100440 0x7f2c1c19aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:56.480 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.477+0000 7f2c218f7700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c1c100df0 0x7f2c1c19d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:56.480 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.477+0000 7f2c218f7700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1c19dd10 con 0x7f2c1c100df0 2023-11-16T16:38:56.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.477+0000 7f2c218f7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1c19de80 con 0x7f2c1c100440 2023-11-16T16:38:56.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.477+0000 7f2c218f7700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1c19e040 con 0x7f2c1c0ff1b0 2023-11-16T16:38:56.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.477+0000 7f2c1b7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c1c100df0 0x7f2c1c19d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:56.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.477+0000 7f2c1b7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c1c100df0 0x7f2c1c19d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42950/0 (socket says 172.21.15.1:42950) 2023-11-16T16:38:56.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.477+0000 7f2c1b7fe700 1 -- 172.21.15.1:0/3008440254 learned_addr learned my addr 172.21.15.1:0/3008440254 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:56.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.477+0000 7f2c1affd700 1 --2- 172.21.15.1:0/3008440254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c1c0ff1b0 0x7f2c1c069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:56.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.477+0000 7f2c1a7fc700 1 --2- 172.21.15.1:0/3008440254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c1c100440 0x7f2c1c19aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:56.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.477+0000 7f2c1b7fe700 1 -- 172.21.15.1:0/3008440254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c1c0ff1b0 msgr2=0x7f2c1c069cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:56.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.477+0000 7f2c1b7fe700 1 --2- 172.21.15.1:0/3008440254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c1c0ff1b0 0x7f2c1c069cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:56.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.477+0000 7f2c1b7fe700 1 -- 172.21.15.1:0/3008440254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c1c100440 msgr2=0x7f2c1c19aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:56.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.477+0000 7f2c1b7fe700 1 --2- 172.21.15.1:0/3008440254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c1c100440 0x7f2c1c19aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:56.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.477+0000 7f2c1b7fe700 1 -- 172.21.15.1:0/3008440254 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c14009d00 con 0x7f2c1c100df0 2023-11-16T16:38:56.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.477+0000 7f2c1b7fe700 1 --2- 172.21.15.1:0/3008440254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c1c100df0 0x7f2c1c19d540 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2c14009fd0 tx=0x7f2c1400bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:56.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.477+0000 7f2c208f5700 1 -- 172.21.15.1:0/3008440254 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1400e040 con 0x7f2c1c100df0 2023-11-16T16:38:56.482 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.478+0000 7f2c208f5700 1 -- 172.21.15.1:0/3008440254 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c140094c0 con 0x7f2c1c100df0 2023-11-16T16:38:56.482 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.478+0000 7f2c208f5700 1 -- 172.21.15.1:0/3008440254 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c140052a0 con 0x7f2c1c100df0 2023-11-16T16:38:56.482 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.478+0000 7f2c218f7700 1 -- 172.21.15.1:0/3008440254 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c1c10ac00 con 0x7f2c1c100df0 2023-11-16T16:38:56.482 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.478+0000 7f2c218f7700 1 -- 172.21.15.1:0/3008440254 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c1c10aa70 con 0x7f2c1c100df0 2023-11-16T16:38:56.482 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.478+0000 7f2c218f7700 1 -- 172.21.15.1:0/3008440254 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c1c067f40 con 0x7f2c1c100df0 2023-11-16T16:38:56.483 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.480+0000 7f2c208f5700 1 -- 172.21.15.1:0/3008440254 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f2c1401d470 con 0x7f2c1c100df0 2023-11-16T16:38:56.483 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.480+0000 7f2c208f5700 1 --2- 172.21.15.1:0/3008440254 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f2c080a11d0 0x7f2c080a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:56.483 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.480+0000 7f2c1affd700 1 --2- 172.21.15.1:0/3008440254 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f2c080a11d0 0x7f2c080a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:56.483 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.480+0000 7f2c208f5700 1 -- 172.21.15.1:0/3008440254 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2c140cd9e0 con 0x7f2c1c100df0 2023-11-16T16:38:56.483 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.481+0000 7f2c1affd700 1 --2- 172.21.15.1:0/3008440254 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f2c080a11d0 0x7f2c080a3680 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c009c60 tx=0x7f2c0c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:56.486 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.485+0000 7f2c208f5700 1 -- 172.21.15.1:0/3008440254 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2c140971c0 con 0x7f2c1c100df0 2023-11-16T16:38:56.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:56 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1054159300' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:38:56.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:56 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/2438290354' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:38:56.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:56 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/3494619518' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:38:56.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:56 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1054159300' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:38:56.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:56 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/2438290354' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:38:56.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:56 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/3494619518' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:38:56.737 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.735+0000 7f2c218f7700 1 -- 172.21.15.1:0/3008440254 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c1c106120 con 0x7f2c1c100df0 2023-11-16T16:38:56.737 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.736+0000 7f2c208f5700 1 -- 172.21.15.1:0/3008440254 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10774 (secure 0 0 0) 0x7f2c1409a7e0 con 0x7f2c1c100df0 2023-11-16T16:38:56.738 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:38:56.738 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2587563684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2587563684},{"type":"v1","addr":"172.21.15.1:6835","nonce":2587563684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/277936168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":277936168},{"type":"v1","addr":"172.21.15.1:6837","nonce":277936168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/1172575045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1172575045},{"type":"v1","addr":"172.21.15.1:6839","nonce":1172575045}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/2631175899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2631175899},{"type":"v1","addr":"172.21.15.1:6841","nonce":2631175899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3693905267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3693905267},{"type":"v1","addr":"172.21.15.53:6833","nonce":3693905267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/164668033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":164668033},{"type":"v1","addr":"172.21.15.26:6833","nonce":164668033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-16T16:37:13.918943+0000","modified":"2023-11-16T16:37:19.292898+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2140542192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2140542192},{"type":"v1","addr":"172.21.15.53:6835","nonce":2140542192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1010760275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1010760275},{"type":"v1","addr":"172.21.15.26:6835","nonce":1010760275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3197060302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3197060302},{"type":"v1","addr":"172.21.15.53:6837","nonce":3197060302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6837/1453504863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1453504863},{"type":"v1","addr":"172.21.15.26:6837","nonce":1453504863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.53:6839/1044753024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1044753024},{"type":"v1","addr":"172.21.15.53:6839","nonce":1044753024}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6839/2192205949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2192205949},{"type":"v1","addr":"172.21.15.26:6839","nonce":2192205949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-16T16:38:56.740 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.738+0000 7f2c218f7700 1 -- 172.21.15.1:0/3008440254 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f2c080a11d0 msgr2=0x7f2c080a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:56.740 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.738+0000 7f2c218f7700 1 --2- 172.21.15.1:0/3008440254 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f2c080a11d0 0x7f2c080a3680 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c009c60 tx=0x7f2c0c009260 comp rx=0 tx=0).stop 2023-11-16T16:38:56.740 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.738+0000 7f2c218f7700 1 -- 172.21.15.1:0/3008440254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c1c100df0 msgr2=0x7f2c1c19d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:56.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.738+0000 7f2c218f7700 1 --2- 172.21.15.1:0/3008440254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c1c100df0 0x7f2c1c19d540 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2c14009fd0 tx=0x7f2c1400bce0 comp rx=0 tx=0).stop 2023-11-16T16:38:56.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.738+0000 7f2c218f7700 1 -- 172.21.15.1:0/3008440254 shutdown_connections 2023-11-16T16:38:56.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.738+0000 7f2c218f7700 1 --2- 172.21.15.1:0/3008440254 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f2c080a11d0 0x7f2c080a3680 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:56.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.738+0000 7f2c218f7700 1 --2- 172.21.15.1:0/3008440254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2c1c100df0 0x7f2c1c19d540 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:56.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.738+0000 7f2c218f7700 1 --2- 172.21.15.1:0/3008440254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2c1c0ff1b0 0x7f2c1c069cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:56.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.738+0000 7f2c218f7700 1 --2- 172.21.15.1:0/3008440254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c1c100440 0x7f2c1c19aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:56.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.738+0000 7f2c218f7700 1 -- 172.21.15.1:0/3008440254 >> 172.21.15.1:0/3008440254 conn(0x7f2c1c0fa7d0 msgr2=0x7f2c1c0fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:56.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.738+0000 7f2c218f7700 1 -- 172.21.15.1:0/3008440254 shutdown_connections 2023-11-16T16:38:56.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.738+0000 7f2c218f7700 1 -- 172.21.15.1:0/3008440254 wait complete. 2023-11-16T16:38:56.741 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 23 2023-11-16T16:38:56.752 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-11-16T16:38:56.753 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-11-16T16:38:56.753 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-16T16:38:56.880 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.876+0000 7fd0b032e700 1 -- 172.21.15.1:0/4091153255 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0a81003b0 msgr2=0x7fd0a8100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:56.881 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.876+0000 7fd0b032e700 1 --2- 172.21.15.1:0/4091153255 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0a81003b0 0x7fd0a8100800 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd098009990 tx=0x7fd098009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:56.881 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.877+0000 7fd0b032e700 1 -- 172.21.15.1:0/4091153255 shutdown_connections 2023-11-16T16:38:56.881 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.877+0000 7fd0b032e700 1 --2- 172.21.15.1:0/4091153255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0a8100d40 0x7fd0a8107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:56.881 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.877+0000 7fd0b032e700 1 --2- 172.21.15.1:0/4091153255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0a80ff1c0 0x7fd0a80ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:56.881 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.877+0000 7fd0b032e700 1 --2- 172.21.15.1:0/4091153255 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0a81003b0 0x7fd0a8100800 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:56.881 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.877+0000 7fd0b032e700 1 -- 172.21.15.1:0/4091153255 >> 172.21.15.1:0/4091153255 conn(0x7fd0a80fa7c0 msgr2=0x7fd0a80fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:56.881 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.877+0000 7fd0b032e700 1 -- 172.21.15.1:0/4091153255 shutdown_connections 2023-11-16T16:38:56.881 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.877+0000 7fd0b032e700 1 -- 172.21.15.1:0/4091153255 wait complete. 2023-11-16T16:38:56.881 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.877+0000 7fd0b032e700 1 Processor -- start 2023-11-16T16:38:56.881 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.877+0000 7fd0b032e700 1 -- start start 2023-11-16T16:38:56.881 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.877+0000 7fd0b032e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0a80ff1c0 0x7fd0a8103150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:56.882 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.877+0000 7fd0b032e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0a81003b0 0x7fd0a8101790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:56.882 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.877+0000 7fd0b032e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0a8100d40 0x7fd0a8101cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:56.882 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.877+0000 7fd0b032e700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a8102470 con 0x7fd0a81003b0 2023-11-16T16:38:56.882 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.877+0000 7fd0b032e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a81025e0 con 0x7fd0a8100d40 2023-11-16T16:38:56.882 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.877+0000 7fd0b032e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a81a0110 con 0x7fd0a80ff1c0 2023-11-16T16:38:56.882 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.877+0000 7fd0ad8c9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0a81003b0 0x7fd0a8101790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:56.882 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.878+0000 7fd0ad8c9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0a81003b0 0x7fd0a8101790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42962/0 (socket says 172.21.15.1:42962) 2023-11-16T16:38:56.882 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.878+0000 7fd0ad8c9700 1 -- 172.21.15.1:0/700659641 learned_addr learned my addr 172.21.15.1:0/700659641 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:56.882 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.878+0000 7fd0ae0ca700 1 --2- 172.21.15.1:0/700659641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0a80ff1c0 0x7fd0a8103150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:56.882 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.878+0000 7fd0ae8cb700 1 --2- 172.21.15.1:0/700659641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0a8100d40 0x7fd0a8101cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:56.882 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.878+0000 7fd0ad8c9700 1 -- 172.21.15.1:0/700659641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0a80ff1c0 msgr2=0x7fd0a8103150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:56.883 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.878+0000 7fd0ad8c9700 1 --2- 172.21.15.1:0/700659641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0a80ff1c0 0x7fd0a8103150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:56.883 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.878+0000 7fd0ad8c9700 1 -- 172.21.15.1:0/700659641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0a8100d40 msgr2=0x7fd0a8101cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:56.883 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.878+0000 7fd0ad8c9700 1 --2- 172.21.15.1:0/700659641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0a8100d40 0x7fd0a8101cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:56.883 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.878+0000 7fd0ad8c9700 1 -- 172.21.15.1:0/700659641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0980095f0 con 0x7fd0a81003b0 2023-11-16T16:38:56.883 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.878+0000 7fd0ad8c9700 1 --2- 172.21.15.1:0/700659641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0a81003b0 0x7fd0a8101790 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd09800b620 tx=0x7fd098009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:56.883 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.878+0000 7fd09f7fe700 1 -- 172.21.15.1:0/700659641 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09801d070 con 0x7fd0a81003b0 2023-11-16T16:38:56.883 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.878+0000 7fd09f7fe700 1 -- 172.21.15.1:0/700659641 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd098021410 con 0x7fd0a81003b0 2023-11-16T16:38:56.883 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.878+0000 7fd0b032e700 1 -- 172.21.15.1:0/700659641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0a810ab70 con 0x7fd0a81003b0 2023-11-16T16:38:56.883 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.878+0000 7fd09f7fe700 1 -- 172.21.15.1:0/700659641 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd098017650 con 0x7fd0a81003b0 2023-11-16T16:38:56.883 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.878+0000 7fd0b032e700 1 -- 172.21.15.1:0/700659641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0a810a9e0 con 0x7fd0a81003b0 2023-11-16T16:38:56.884 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.879+0000 7fd0b032e700 1 -- 172.21.15.1:0/700659641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0a81a06b0 con 0x7fd0a81003b0 2023-11-16T16:38:56.884 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.880+0000 7fd09f7fe700 1 -- 172.21.15.1:0/700659641 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fd098021580 con 0x7fd0a81003b0 2023-11-16T16:38:56.884 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.881+0000 7fd09f7fe700 1 --2- 172.21.15.1:0/700659641 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd0940a1150 0x7fd0940a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:56.884 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.881+0000 7fd0ae0ca700 1 --2- 172.21.15.1:0/700659641 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd0940a1150 0x7fd0940a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:56.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.881+0000 7fd09f7fe700 1 -- 172.21.15.1:0/700659641 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd0980cd790 con 0x7fd0a81003b0 2023-11-16T16:38:56.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.881+0000 7fd0ae0ca700 1 --2- 172.21.15.1:0/700659641 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd0940a1150 0x7fd0940a3600 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8100210 tx=0x7fd0a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:56.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:56.885+0000 7fd09f7fe700 1 -- 172.21.15.1:0/700659641 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd098096f30 con 0x7fd0a81003b0 2023-11-16T16:38:57.085 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.083+0000 7fd0b032e700 1 -- 172.21.15.1:0/700659641 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd0a81a0d20 con 0x7fd0940a1150 2023-11-16T16:38:57.098 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.096+0000 7fd09f7fe700 1 -- 172.21.15.1:0/700659641 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344244 (secure 0 0 0) 0x7fd0a81a0d20 con 0x7fd0940a1150 2023-11-16T16:38:57.102 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:38:57.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.100+0000 7fd0b032e700 1 -- 172.21.15.1:0/700659641 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd0940a1150 msgr2=0x7fd0940a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:57.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.100+0000 7fd0b032e700 1 --2- 172.21.15.1:0/700659641 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd0940a1150 0x7fd0940a3600 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8100210 tx=0x7fd0a4009260 comp rx=0 tx=0).stop 2023-11-16T16:38:57.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.100+0000 7fd0b032e700 1 -- 172.21.15.1:0/700659641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0a81003b0 msgr2=0x7fd0a8101790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:57.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.100+0000 7fd0b032e700 1 --2- 172.21.15.1:0/700659641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0a81003b0 0x7fd0a8101790 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd09800b620 tx=0x7fd098009580 comp rx=0 tx=0).stop 2023-11-16T16:38:57.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.100+0000 7fd0b032e700 1 -- 172.21.15.1:0/700659641 shutdown_connections 2023-11-16T16:38:57.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.100+0000 7fd0b032e700 1 --2- 172.21.15.1:0/700659641 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fd0940a1150 0x7fd0940a3600 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:57.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.100+0000 7fd0b032e700 1 --2- 172.21.15.1:0/700659641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0a8100d40 0x7fd0a8101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:57.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.100+0000 7fd0b032e700 1 --2- 172.21.15.1:0/700659641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd0a80ff1c0 0x7fd0a8103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:57.106 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.100+0000 7fd0b032e700 1 --2- 172.21.15.1:0/700659641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0a81003b0 0x7fd0a8101790 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:57.106 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.100+0000 7fd0b032e700 1 -- 172.21.15.1:0/700659641 >> 172.21.15.1:0/700659641 conn(0x7fd0a80fa7c0 msgr2=0x7fd0a8067de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:57.106 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.100+0000 7fd0b032e700 1 -- 172.21.15.1:0/700659641 shutdown_connections 2023-11-16T16:38:57.106 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.100+0000 7fd0b032e700 1 -- 172.21.15.1:0/700659641 wait complete. 2023-11-16T16:38:57.106 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2023-11-16T16:38:57.114 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":313,"stamp":"2023-11-16T16:38:55.507087+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659450,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":745,"num_read_kb":496,"num_write":345,"num_write_kb":3058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180848,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":782,"ondisk_log_size":782,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":80336,"kb_used_data":11376,"kb_used_omap":0,"kb_used_meta":68864,"kb_avail":1124861488,"statfs":{"total":1151940427776,"available":1151858163712,"internally_reserved":0,"allocated":11649024,"data_stored":7327915,"data_compressed":176451,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":0,"internal_metadata":70516736},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6015,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":93,"num_read_kb":53,"num_write":5,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003024"},"pg_stats":[{"pgid":"3.3f","version":"86'8","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.516764+0000","last_change":"2023-11-16T16:37:13.941551+0000","last_active":"2023-11-16T16:38:31.516764+0000","last_peered":"2023-11-16T16:38:31.516764+0000","last_clean":"2023-11-16T16:38:31.516764+0000","last_became_active":"2023-11-16T16:37:13.941190+0000","last_became_peered":"2023-11-16T16:37:13.941190+0000","last_unstale":"2023-11-16T16:38:31.516764+0000","last_undegraded":"2023-11-16T16:38:31.516764+0000","last_fullsized":"2023-11-16T16:38:31.516764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:42:03.561127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.510062+0000","last_change":"2023-11-16T16:37:13.942300+0000","last_active":"2023-11-16T16:38:31.510062+0000","last_peered":"2023-11-16T16:38:31.510062+0000","last_clean":"2023-11-16T16:38:31.510062+0000","last_became_active":"2023-11-16T16:37:13.941966+0000","last_became_peered":"2023-11-16T16:37:13.941966+0000","last_unstale":"2023-11-16T16:38:31.510062+0000","last_undegraded":"2023-11-16T16:38:31.510062+0000","last_fullsized":"2023-11-16T16:38:31.510062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:12:41.529896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.518118+0000","last_change":"2023-11-16T16:37:13.943461+0000","last_active":"2023-11-16T16:38:31.518118+0000","last_peered":"2023-11-16T16:38:31.518118+0000","last_clean":"2023-11-16T16:38:31.518118+0000","last_became_active":"2023-11-16T16:37:13.943165+0000","last_became_peered":"2023-11-16T16:37:13.943165+0000","last_unstale":"2023-11-16T16:38:31.518118+0000","last_undegraded":"2023-11-16T16:38:31.518118+0000","last_fullsized":"2023-11-16T16:38:31.518118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:17:44.567499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"86'8","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.526363+0000","last_change":"2023-11-16T16:37:13.940205+0000","last_active":"2023-11-16T16:38:31.526363+0000","last_peered":"2023-11-16T16:38:31.526363+0000","last_clean":"2023-11-16T16:38:31.526363+0000","last_became_active":"2023-11-16T16:37:13.939819+0000","last_became_peered":"2023-11-16T16:37:13.939819+0000","last_unstale":"2023-11-16T16:38:31.526363+0000","last_undegraded":"2023-11-16T16:38:31.526363+0000","last_fullsized":"2023-11-16T16:38:31.526363+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:52:42.183240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.508231+0000","last_change":"2023-11-16T16:37:13.939819+0000","last_active":"2023-11-16T16:38:31.508231+0000","last_peered":"2023-11-16T16:38:31.508231+0000","last_clean":"2023-11-16T16:38:31.508231+0000","last_became_active":"2023-11-16T16:37:13.939367+0000","last_became_peered":"2023-11-16T16:37:13.939367+0000","last_unstale":"2023-11-16T16:38:31.508231+0000","last_undegraded":"2023-11-16T16:38:31.508231+0000","last_fullsized":"2023-11-16T16:38:31.508231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:17:30.563282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.509876+0000","last_change":"2023-11-16T16:37:13.946613+0000","last_active":"2023-11-16T16:38:31.509876+0000","last_peered":"2023-11-16T16:38:31.509876+0000","last_clean":"2023-11-16T16:38:31.509876+0000","last_became_active":"2023-11-16T16:37:13.946281+0000","last_became_peered":"2023-11-16T16:37:13.946281+0000","last_unstale":"2023-11-16T16:38:31.509876+0000","last_undegraded":"2023-11-16T16:38:31.509876+0000","last_fullsized":"2023-11-16T16:38:31.509876+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:29:42.599960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.528658+0000","last_change":"2023-11-16T16:37:13.936899+0000","last_active":"2023-11-16T16:38:31.528658+0000","last_peered":"2023-11-16T16:38:31.528658+0000","last_clean":"2023-11-16T16:38:31.528658+0000","last_became_active":"2023-11-16T16:37:13.936543+0000","last_became_peered":"2023-11-16T16:37:13.936543+0000","last_unstale":"2023-11-16T16:38:31.528658+0000","last_undegraded":"2023-11-16T16:38:31.528658+0000","last_fullsized":"2023-11-16T16:38:31.528658+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:43:46.820689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"86'9","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.518738+0000","last_change":"2023-11-16T16:38:03.189467+0000","last_active":"2023-11-16T16:38:31.518738+0000","last_peered":"2023-11-16T16:38:31.518738+0000","last_clean":"2023-11-16T16:38:31.518738+0000","last_became_active":"2023-11-16T16:38:03.189017+0000","last_became_peered":"2023-11-16T16:38:03.189017+0000","last_unstale":"2023-11-16T16:38:31.518738+0000","last_undegraded":"2023-11-16T16:38:31.518738+0000","last_fullsized":"2023-11-16T16:38:31.518738+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:47:50.088436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"86'11","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.519665+0000","last_change":"2023-11-16T16:37:13.934105+0000","last_active":"2023-11-16T16:38:31.519665+0000","last_peered":"2023-11-16T16:38:31.519665+0000","last_clean":"2023-11-16T16:38:31.519665+0000","last_became_active":"2023-11-16T16:37:13.933887+0000","last_became_peered":"2023-11-16T16:37:13.933887+0000","last_unstale":"2023-11-16T16:38:31.519665+0000","last_undegraded":"2023-11-16T16:38:31.519665+0000","last_fullsized":"2023-11-16T16:38:31.519665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:53:26.890511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"86'16","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.519996+0000","last_change":"2023-11-16T16:38:03.191855+0000","last_active":"2023-11-16T16:38:31.519996+0000","last_peered":"2023-11-16T16:38:31.519996+0000","last_clean":"2023-11-16T16:38:31.519996+0000","last_became_active":"2023-11-16T16:38:03.191398+0000","last_became_peered":"2023-11-16T16:38:03.191398+0000","last_unstale":"2023-11-16T16:38:31.519996+0000","last_undegraded":"2023-11-16T16:38:31.519996+0000","last_fullsized":"2023-11-16T16:38:31.519996+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:09:44.431927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.529451+0000","last_change":"2023-11-16T16:37:13.941597+0000","last_active":"2023-11-16T16:38:31.529451+0000","last_peered":"2023-11-16T16:38:31.529451+0000","last_clean":"2023-11-16T16:38:31.529451+0000","last_became_active":"2023-11-16T16:37:13.941253+0000","last_became_peered":"2023-11-16T16:37:13.941253+0000","last_unstale":"2023-11-16T16:38:31.529451+0000","last_undegraded":"2023-11-16T16:38:31.529451+0000","last_fullsized":"2023-11-16T16:38:31.529451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:13:35.256952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.509176+0000","last_change":"2023-11-16T16:37:13.941673+0000","last_active":"2023-11-16T16:38:31.509176+0000","last_peered":"2023-11-16T16:38:31.509176+0000","last_clean":"2023-11-16T16:38:31.509176+0000","last_became_active":"2023-11-16T16:37:13.941345+0000","last_became_peered":"2023-11-16T16:37:13.941345+0000","last_unstale":"2023-11-16T16:38:31.509176+0000","last_undegraded":"2023-11-16T16:38:31.509176+0000","last_fullsized":"2023-11-16T16:38:31.509176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:17:51.018141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"86'10","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.510670+0000","last_change":"2023-11-16T16:37:13.947991+0000","last_active":"2023-11-16T16:38:31.510670+0000","last_peered":"2023-11-16T16:38:31.510670+0000","last_clean":"2023-11-16T16:38:31.510670+0000","last_became_active":"2023-11-16T16:37:13.947454+0000","last_became_peered":"2023-11-16T16:37:13.947454+0000","last_unstale":"2023-11-16T16:38:31.510670+0000","last_undegraded":"2023-11-16T16:38:31.510670+0000","last_fullsized":"2023-11-16T16:38:31.510670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:19:19.825458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.510762+0000","last_change":"2023-11-16T16:37:13.941502+0000","last_active":"2023-11-16T16:38:31.510762+0000","last_peered":"2023-11-16T16:38:31.510762+0000","last_clean":"2023-11-16T16:38:31.510762+0000","last_became_active":"2023-11-16T16:37:13.941191+0000","last_became_peered":"2023-11-16T16:37:13.941191+0000","last_unstale":"2023-11-16T16:38:31.510762+0000","last_undegraded":"2023-11-16T16:38:31.510762+0000","last_fullsized":"2023-11-16T16:38:31.510762+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:19:35.239602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.517384+0000","last_change":"2023-11-16T16:37:13.949198+0000","last_active":"2023-11-16T16:38:31.517384+0000","last_peered":"2023-11-16T16:38:31.517384+0000","last_clean":"2023-11-16T16:38:31.517384+0000","last_became_active":"2023-11-16T16:37:13.947147+0000","last_became_peered":"2023-11-16T16:37:13.947147+0000","last_unstale":"2023-11-16T16:38:31.517384+0000","last_undegraded":"2023-11-16T16:38:31.517384+0000","last_fullsized":"2023-11-16T16:38:31.517384+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:55:04.616796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.526473+0000","last_change":"2023-11-16T16:37:13.943915+0000","last_active":"2023-11-16T16:38:31.526473+0000","last_peered":"2023-11-16T16:38:31.526473+0000","last_clean":"2023-11-16T16:38:31.526473+0000","last_became_active":"2023-11-16T16:37:13.942995+0000","last_became_peered":"2023-11-16T16:37:13.942995+0000","last_unstale":"2023-11-16T16:38:31.526473+0000","last_undegraded":"2023-11-16T16:38:31.526473+0000","last_fullsized":"2023-11-16T16:38:31.526473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:23:05.411045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"86'8","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.516988+0000","last_change":"2023-11-16T16:37:13.941816+0000","last_active":"2023-11-16T16:38:31.516988+0000","last_peered":"2023-11-16T16:38:31.516988+0000","last_clean":"2023-11-16T16:38:31.516988+0000","last_became_active":"2023-11-16T16:37:13.941541+0000","last_became_peered":"2023-11-16T16:37:13.941541+0000","last_unstale":"2023-11-16T16:38:31.516988+0000","last_undegraded":"2023-11-16T16:38:31.516988+0000","last_fullsized":"2023-11-16T16:38:31.516988+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:53:11.142308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"86'10","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.518736+0000","last_change":"2023-11-16T16:37:13.940639+0000","last_active":"2023-11-16T16:38:31.518736+0000","last_peered":"2023-11-16T16:38:31.518736+0000","last_clean":"2023-11-16T16:38:31.518736+0000","last_became_active":"2023-11-16T16:37:13.940323+0000","last_became_peered":"2023-11-16T16:37:13.940323+0000","last_unstale":"2023-11-16T16:38:31.518736+0000","last_undegraded":"2023-11-16T16:38:31.518736+0000","last_fullsized":"2023-11-16T16:38:31.518736+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:32:03.882320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.513674+0000","last_change":"2023-11-16T16:37:13.934802+0000","last_active":"2023-11-16T16:38:31.513674+0000","last_peered":"2023-11-16T16:38:31.513674+0000","last_clean":"2023-11-16T16:38:31.513674+0000","last_became_active":"2023-11-16T16:37:13.934496+0000","last_became_peered":"2023-11-16T16:37:13.934496+0000","last_unstale":"2023-11-16T16:38:31.513674+0000","last_undegraded":"2023-11-16T16:38:31.513674+0000","last_fullsized":"2023-11-16T16:38:31.513674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:15:58.249424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185585+0000","last_change":"2023-11-16T16:37:13.499284+0000","last_active":"2023-11-16T16:38:03.185585+0000","last_peered":"2023-11-16T16:38:03.185585+0000","last_clean":"2023-11-16T16:38:03.185585+0000","last_became_active":"2023-11-16T16:37:13.498931+0000","last_became_peered":"2023-11-16T16:37:13.498931+0000","last_unstale":"2023-11-16T16:38:03.185585+0000","last_undegraded":"2023-11-16T16:38:03.185585+0000","last_fullsized":"2023-11-16T16:38:03.185585+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:53:33.061352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.514671+0000","last_change":"2023-11-16T16:37:13.939631+0000","last_active":"2023-11-16T16:38:31.514671+0000","last_peered":"2023-11-16T16:38:31.514671+0000","last_clean":"2023-11-16T16:38:31.514671+0000","last_became_active":"2023-11-16T16:37:13.939318+0000","last_became_peered":"2023-11-16T16:37:13.939318+0000","last_unstale":"2023-11-16T16:38:31.514671+0000","last_undegraded":"2023-11-16T16:38:31.514671+0000","last_fullsized":"2023-11-16T16:38:31.514671+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:11:33.421798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.183008+0000","last_change":"2023-11-16T16:37:13.495972+0000","last_active":"2023-11-16T16:38:03.183008+0000","last_peered":"2023-11-16T16:38:03.183008+0000","last_clean":"2023-11-16T16:38:03.183008+0000","last_became_active":"2023-11-16T16:37:13.495610+0000","last_became_peered":"2023-11-16T16:37:13.495610+0000","last_unstale":"2023-11-16T16:38:03.183008+0000","last_undegraded":"2023-11-16T16:38:03.183008+0000","last_fullsized":"2023-11-16T16:38:03.183008+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:47:20.115622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.518372+0000","last_change":"2023-11-16T16:37:13.935236+0000","last_active":"2023-11-16T16:38:31.518372+0000","last_peered":"2023-11-16T16:38:31.518372+0000","last_clean":"2023-11-16T16:38:31.518372+0000","last_became_active":"2023-11-16T16:37:13.934920+0000","last_became_peered":"2023-11-16T16:37:13.934920+0000","last_unstale":"2023-11-16T16:38:31.518372+0000","last_undegraded":"2023-11-16T16:38:31.518372+0000","last_fullsized":"2023-11-16T16:38:31.518372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:04:59.207169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"84'4","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.197657+0000","last_change":"2023-11-16T16:38:03.197657+0000","last_active":"2023-11-16T16:38:03.197657+0000","last_peered":"2023-11-16T16:38:03.197657+0000","last_clean":"2023-11-16T16:38:03.197657+0000","last_became_active":"2023-11-16T16:38:03.190952+0000","last_became_peered":"2023-11-16T16:38:03.190952+0000","last_unstale":"2023-11-16T16:38:03.197657+0000","last_undegraded":"2023-11-16T16:38:03.197657+0000","last_fullsized":"2023-11-16T16:38:03.197657+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:04:50.256412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.524482+0000","last_change":"2023-11-16T16:37:13.940838+0000","last_active":"2023-11-16T16:38:31.524482+0000","last_peered":"2023-11-16T16:38:31.524482+0000","last_clean":"2023-11-16T16:38:31.524482+0000","last_became_active":"2023-11-16T16:37:13.940538+0000","last_became_peered":"2023-11-16T16:37:13.940538+0000","last_unstale":"2023-11-16T16:38:31.524482+0000","last_undegraded":"2023-11-16T16:38:31.524482+0000","last_fullsized":"2023-11-16T16:38:31.524482+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:47:36.031628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.190408+0000","last_change":"2023-11-16T16:37:13.495114+0000","last_active":"2023-11-16T16:38:03.190408+0000","last_peered":"2023-11-16T16:38:03.190408+0000","last_clean":"2023-11-16T16:38:03.190408+0000","last_became_active":"2023-11-16T16:37:13.494807+0000","last_became_peered":"2023-11-16T16:37:13.494807+0000","last_unstale":"2023-11-16T16:38:03.190408+0000","last_undegraded":"2023-11-16T16:38:03.190408+0000","last_fullsized":"2023-11-16T16:38:03.190408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:35:31.040933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"86'15","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.519316+0000","last_change":"2023-11-16T16:37:13.943652+0000","last_active":"2023-11-16T16:38:31.519316+0000","last_peered":"2023-11-16T16:38:31.519316+0000","last_clean":"2023-11-16T16:38:31.519316+0000","last_became_active":"2023-11-16T16:37:13.943298+0000","last_became_peered":"2023-11-16T16:37:13.943298+0000","last_unstale":"2023-11-16T16:38:31.519316+0000","last_undegraded":"2023-11-16T16:38:31.519316+0000","last_fullsized":"2023-11-16T16:38:31.519316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:17:55.154731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'8","reported_seq":128,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:54.451248+0000","last_change":"2023-11-16T16:37:13.509801+0000","last_active":"2023-11-16T16:38:54.451248+0000","last_peered":"2023-11-16T16:38:54.451248+0000","last_clean":"2023-11-16T16:38:54.451248+0000","last_became_active":"2023-11-16T16:37:13.509389+0000","last_became_peered":"2023-11-16T16:37:13.509389+0000","last_unstale":"2023-11-16T16:38:54.451248+0000","last_undegraded":"2023-11-16T16:38:54.451248+0000","last_fullsized":"2023-11-16T16:38:54.451248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:46:55.751281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1423,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.512699+0000","last_change":"2023-11-16T16:37:13.933871+0000","last_active":"2023-11-16T16:38:31.512699+0000","last_peered":"2023-11-16T16:38:31.512699+0000","last_clean":"2023-11-16T16:38:31.512699+0000","last_became_active":"2023-11-16T16:37:13.933549+0000","last_became_peered":"2023-11-16T16:37:13.933549+0000","last_unstale":"2023-11-16T16:38:31.512699+0000","last_undegraded":"2023-11-16T16:38:31.512699+0000","last_fullsized":"2023-11-16T16:38:31.512699+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:35:18.854524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"84'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.187212+0000","last_change":"2023-11-16T16:37:13.501339+0000","last_active":"2023-11-16T16:38:03.187212+0000","last_peered":"2023-11-16T16:38:03.187212+0000","last_clean":"2023-11-16T16:38:03.187212+0000","last_became_active":"2023-11-16T16:37:13.500949+0000","last_became_peered":"2023-11-16T16:37:13.500949+0000","last_unstale":"2023-11-16T16:38:03.187212+0000","last_undegraded":"2023-11-16T16:38:03.187212+0000","last_fullsized":"2023-11-16T16:38:03.187212+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:56:34.503681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"86'8","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.514559+0000","last_change":"2023-11-16T16:38:03.192595+0000","last_active":"2023-11-16T16:38:31.514559+0000","last_peered":"2023-11-16T16:38:31.514559+0000","last_clean":"2023-11-16T16:38:31.514559+0000","last_became_active":"2023-11-16T16:38:03.192208+0000","last_became_peered":"2023-11-16T16:38:03.192208+0000","last_unstale":"2023-11-16T16:38:31.514559+0000","last_undegraded":"2023-11-16T16:38:31.514559+0000","last_fullsized":"2023-11-16T16:38:31.514559+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:30:24.627049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"84'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184789+0000","last_change":"2023-11-16T16:37:13.505481+0000","last_active":"2023-11-16T16:38:03.184789+0000","last_peered":"2023-11-16T16:38:03.184789+0000","last_clean":"2023-11-16T16:38:03.184789+0000","last_became_active":"2023-11-16T16:37:13.505085+0000","last_became_peered":"2023-11-16T16:37:13.505085+0000","last_unstale":"2023-11-16T16:38:03.184789+0000","last_undegraded":"2023-11-16T16:38:03.184789+0000","last_fullsized":"2023-11-16T16:38:03.184789+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:21:13.673154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"86'8","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.509656+0000","last_change":"2023-11-16T16:37:13.943985+0000","last_active":"2023-11-16T16:38:31.509656+0000","last_peered":"2023-11-16T16:38:31.509656+0000","last_clean":"2023-11-16T16:38:31.509656+0000","last_became_active":"2023-11-16T16:37:13.943628+0000","last_became_peered":"2023-11-16T16:37:13.943628+0000","last_unstale":"2023-11-16T16:38:31.509656+0000","last_undegraded":"2023-11-16T16:38:31.509656+0000","last_fullsized":"2023-11-16T16:38:31.509656+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:01:16.596531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184704+0000","last_change":"2023-11-16T16:37:13.498648+0000","last_active":"2023-11-16T16:38:03.184704+0000","last_peered":"2023-11-16T16:38:03.184704+0000","last_clean":"2023-11-16T16:38:03.184704+0000","last_became_active":"2023-11-16T16:37:13.497390+0000","last_became_peered":"2023-11-16T16:37:13.497390+0000","last_unstale":"2023-11-16T16:38:03.184704+0000","last_undegraded":"2023-11-16T16:38:03.184704+0000","last_fullsized":"2023-11-16T16:38:03.184704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:20:50.541938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"86'11","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.518643+0000","last_change":"2023-11-16T16:37:13.942337+0000","last_active":"2023-11-16T16:38:31.518643+0000","last_peered":"2023-11-16T16:38:31.518643+0000","last_clean":"2023-11-16T16:38:31.518643+0000","last_became_active":"2023-11-16T16:37:13.941982+0000","last_became_peered":"2023-11-16T16:37:13.941982+0000","last_unstale":"2023-11-16T16:38:31.518643+0000","last_undegraded":"2023-11-16T16:38:31.518643+0000","last_fullsized":"2023-11-16T16:38:31.518643+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:09:19.949618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'5","reported_seq":120,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:54.450706+0000","last_change":"2023-11-16T16:37:13.503690+0000","last_active":"2023-11-16T16:38:54.450706+0000","last_peered":"2023-11-16T16:38:54.450706+0000","last_clean":"2023-11-16T16:38:54.450706+0000","last_became_active":"2023-11-16T16:37:13.503203+0000","last_became_peered":"2023-11-16T16:37:13.503203+0000","last_unstale":"2023-11-16T16:38:54.450706+0000","last_undegraded":"2023-11-16T16:38:54.450706+0000","last_fullsized":"2023-11-16T16:38:54.450706+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:47:38.415047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"86'10","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.519722+0000","last_change":"2023-11-16T16:37:13.943975+0000","last_active":"2023-11-16T16:38:31.519722+0000","last_peered":"2023-11-16T16:38:31.519722+0000","last_clean":"2023-11-16T16:38:31.519722+0000","last_became_active":"2023-11-16T16:37:13.943611+0000","last_became_peered":"2023-11-16T16:37:13.943611+0000","last_unstale":"2023-11-16T16:38:31.519722+0000","last_undegraded":"2023-11-16T16:38:31.519722+0000","last_fullsized":"2023-11-16T16:38:31.519722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:37:02.660396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'2","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.344191+0000","last_change":"2023-11-16T16:38:03.344191+0000","last_active":"2023-11-16T16:38:03.344191+0000","last_peered":"2023-11-16T16:38:03.344191+0000","last_clean":"2023-11-16T16:38:03.344191+0000","last_became_active":"2023-11-16T16:38:03.189976+0000","last_became_peered":"2023-11-16T16:38:03.189976+0000","last_unstale":"2023-11-16T16:38:03.344191+0000","last_undegraded":"2023-11-16T16:38:03.344191+0000","last_fullsized":"2023-11-16T16:38:03.344191+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:04:56.889493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.513627+0000","last_change":"2023-11-16T16:37:13.949031+0000","last_active":"2023-11-16T16:38:31.513627+0000","last_peered":"2023-11-16T16:38:31.513627+0000","last_clean":"2023-11-16T16:38:31.513627+0000","last_became_active":"2023-11-16T16:37:13.946923+0000","last_became_peered":"2023-11-16T16:37:13.946923+0000","last_unstale":"2023-11-16T16:38:31.513627+0000","last_undegraded":"2023-11-16T16:38:31.513627+0000","last_fullsized":"2023-11-16T16:38:31.513627+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:04:43.521722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185398+0000","last_change":"2023-11-16T16:37:13.495003+0000","last_active":"2023-11-16T16:38:03.185398+0000","last_peered":"2023-11-16T16:38:03.185398+0000","last_clean":"2023-11-16T16:38:03.185398+0000","last_became_active":"2023-11-16T16:37:13.494563+0000","last_became_peered":"2023-11-16T16:37:13.494563+0000","last_unstale":"2023-11-16T16:38:03.185398+0000","last_undegraded":"2023-11-16T16:38:03.185398+0000","last_fullsized":"2023-11-16T16:38:03.185398+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:36:36.403721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"86'11","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.529044+0000","last_change":"2023-11-16T16:37:13.943736+0000","last_active":"2023-11-16T16:38:31.529044+0000","last_peered":"2023-11-16T16:38:31.529044+0000","last_clean":"2023-11-16T16:38:31.529044+0000","last_became_active":"2023-11-16T16:37:13.943391+0000","last_became_peered":"2023-11-16T16:37:13.943391+0000","last_unstale":"2023-11-16T16:38:31.529044+0000","last_undegraded":"2023-11-16T16:38:31.529044+0000","last_fullsized":"2023-11-16T16:38:31.529044+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:27:01.809877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'34","reported_seq":257,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:55.463894+0000","last_change":"2023-11-16T16:37:13.506508+0000","last_active":"2023-11-16T16:38:55.463894+0000","last_peered":"2023-11-16T16:38:55.463894+0000","last_clean":"2023-11-16T16:38:55.463894+0000","last_became_active":"2023-11-16T16:37:13.506045+0000","last_became_peered":"2023-11-16T16:37:13.506045+0000","last_unstale":"2023-11-16T16:38:55.463894+0000","last_undegraded":"2023-11-16T16:38:55.463894+0000","last_fullsized":"2023-11-16T16:38:55.463894+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:57:44.753422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64473,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":205,"num_read_kb":165,"num_write":35,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.525885+0000","last_change":"2023-11-16T16:37:13.939717+0000","last_active":"2023-11-16T16:38:31.525885+0000","last_peered":"2023-11-16T16:38:31.525885+0000","last_clean":"2023-11-16T16:38:31.525885+0000","last_became_active":"2023-11-16T16:37:13.939369+0000","last_became_peered":"2023-11-16T16:37:13.939369+0000","last_unstale":"2023-11-16T16:38:31.525885+0000","last_undegraded":"2023-11-16T16:38:31.525885+0000","last_fullsized":"2023-11-16T16:38:31.525885+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:10:31.454459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.188492+0000","last_change":"2023-11-16T16:37:13.506056+0000","last_active":"2023-11-16T16:38:03.188492+0000","last_peered":"2023-11-16T16:38:03.188492+0000","last_clean":"2023-11-16T16:38:03.188492+0000","last_became_active":"2023-11-16T16:37:13.505707+0000","last_became_peered":"2023-11-16T16:37:13.505707+0000","last_unstale":"2023-11-16T16:38:03.188492+0000","last_undegraded":"2023-11-16T16:38:03.188492+0000","last_fullsized":"2023-11-16T16:38:03.188492+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:29:47.242470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.525703+0000","last_change":"2023-11-16T16:37:13.947764+0000","last_active":"2023-11-16T16:38:31.525703+0000","last_peered":"2023-11-16T16:38:31.525703+0000","last_clean":"2023-11-16T16:38:31.525703+0000","last_became_active":"2023-11-16T16:37:13.947510+0000","last_became_peered":"2023-11-16T16:37:13.947510+0000","last_unstale":"2023-11-16T16:38:31.525703+0000","last_undegraded":"2023-11-16T16:38:31.525703+0000","last_fullsized":"2023-11-16T16:38:31.525703+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:41:59.672529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184939+0000","last_change":"2023-11-16T16:37:13.498178+0000","last_active":"2023-11-16T16:38:03.184939+0000","last_peered":"2023-11-16T16:38:03.184939+0000","last_clean":"2023-11-16T16:38:03.184939+0000","last_became_active":"2023-11-16T16:37:13.497312+0000","last_became_peered":"2023-11-16T16:37:13.497312+0000","last_unstale":"2023-11-16T16:38:03.184939+0000","last_undegraded":"2023-11-16T16:38:03.184939+0000","last_fullsized":"2023-11-16T16:38:03.184939+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:54:40.375590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"86'15","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.529728+0000","last_change":"2023-11-16T16:37:13.947854+0000","last_active":"2023-11-16T16:38:31.529728+0000","last_peered":"2023-11-16T16:38:31.529728+0000","last_clean":"2023-11-16T16:38:31.529728+0000","last_became_active":"2023-11-16T16:37:13.947530+0000","last_became_peered":"2023-11-16T16:37:13.947530+0000","last_unstale":"2023-11-16T16:38:31.529728+0000","last_undegraded":"2023-11-16T16:38:31.529728+0000","last_fullsized":"2023-11-16T16:38:31.529728+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:41:00.205641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'3","reported_seq":118,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:52.446665+0000","last_change":"2023-11-16T16:37:13.500127+0000","last_active":"2023-11-16T16:38:52.446665+0000","last_peered":"2023-11-16T16:38:52.446665+0000","last_clean":"2023-11-16T16:38:52.446665+0000","last_became_active":"2023-11-16T16:37:13.499758+0000","last_became_peered":"2023-11-16T16:37:13.499758+0000","last_unstale":"2023-11-16T16:38:52.446665+0000","last_undegraded":"2023-11-16T16:38:52.446665+0000","last_fullsized":"2023-11-16T16:38:52.446665+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:39:00.268144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.528126+0000","last_change":"2023-11-16T16:37:13.940962+0000","last_active":"2023-11-16T16:38:31.528126+0000","last_peered":"2023-11-16T16:38:31.528126+0000","last_clean":"2023-11-16T16:38:31.528126+0000","last_became_active":"2023-11-16T16:37:13.940450+0000","last_became_peered":"2023-11-16T16:37:13.940450+0000","last_unstale":"2023-11-16T16:38:31.528126+0000","last_undegraded":"2023-11-16T16:38:31.528126+0000","last_fullsized":"2023-11-16T16:38:31.528126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:28:45.946454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'15","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.228174+0000","last_change":"2023-11-16T16:38:03.228174+0000","last_active":"2023-11-16T16:38:03.228174+0000","last_peered":"2023-11-16T16:38:03.228174+0000","last_clean":"2023-11-16T16:38:03.228174+0000","last_became_active":"2023-11-16T16:38:03.192020+0000","last_became_peered":"2023-11-16T16:38:03.192020+0000","last_unstale":"2023-11-16T16:38:03.228174+0000","last_undegraded":"2023-11-16T16:38:03.228174+0000","last_fullsized":"2023-11-16T16:38:03.228174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:54:01.231005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.518623+0000","last_change":"2023-11-16T16:37:13.944878+0000","last_active":"2023-11-16T16:38:31.518623+0000","last_peered":"2023-11-16T16:38:31.518623+0000","last_clean":"2023-11-16T16:38:31.518623+0000","last_became_active":"2023-11-16T16:37:13.944580+0000","last_became_peered":"2023-11-16T16:37:13.944580+0000","last_unstale":"2023-11-16T16:38:31.518623+0000","last_undegraded":"2023-11-16T16:38:31.518623+0000","last_fullsized":"2023-11-16T16:38:31.518623+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:00:01.106243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.190628+0000","last_change":"2023-11-16T16:37:13.499139+0000","last_active":"2023-11-16T16:38:03.190628+0000","last_peered":"2023-11-16T16:38:03.190628+0000","last_clean":"2023-11-16T16:38:03.190628+0000","last_became_active":"2023-11-16T16:37:13.498755+0000","last_became_peered":"2023-11-16T16:37:13.498755+0000","last_unstale":"2023-11-16T16:38:03.190628+0000","last_undegraded":"2023-11-16T16:38:03.190628+0000","last_fullsized":"2023-11-16T16:38:03.190628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T16:55:06.065509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.517260+0000","last_change":"2023-11-16T16:37:13.946187+0000","last_active":"2023-11-16T16:38:31.517260+0000","last_peered":"2023-11-16T16:38:31.517260+0000","last_clean":"2023-11-16T16:38:31.517260+0000","last_became_active":"2023-11-16T16:37:13.945887+0000","last_became_peered":"2023-11-16T16:37:13.945887+0000","last_unstale":"2023-11-16T16:38:31.517260+0000","last_undegraded":"2023-11-16T16:38:31.517260+0000","last_fullsized":"2023-11-16T16:38:31.517260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:41:31.726749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185150+0000","last_change":"2023-11-16T16:37:13.507150+0000","last_active":"2023-11-16T16:38:03.185150+0000","last_peered":"2023-11-16T16:38:03.185150+0000","last_clean":"2023-11-16T16:38:03.185150+0000","last_became_active":"2023-11-16T16:37:13.506814+0000","last_became_peered":"2023-11-16T16:37:13.506814+0000","last_unstale":"2023-11-16T16:38:03.185150+0000","last_undegraded":"2023-11-16T16:38:03.185150+0000","last_fullsized":"2023-11-16T16:38:03.185150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:54:45.031046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.506758+0000","last_change":"2023-11-16T16:37:13.947866+0000","last_active":"2023-11-16T16:38:31.506758+0000","last_peered":"2023-11-16T16:38:31.506758+0000","last_clean":"2023-11-16T16:38:31.506758+0000","last_became_active":"2023-11-16T16:37:13.947544+0000","last_became_peered":"2023-11-16T16:37:13.947544+0000","last_unstale":"2023-11-16T16:38:31.506758+0000","last_undegraded":"2023-11-16T16:38:31.506758+0000","last_fullsized":"2023-11-16T16:38:31.506758+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:23:27.220130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"84'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184212+0000","last_change":"2023-11-16T16:37:13.497800+0000","last_active":"2023-11-16T16:38:03.184212+0000","last_peered":"2023-11-16T16:38:03.184212+0000","last_clean":"2023-11-16T16:38:03.184212+0000","last_became_active":"2023-11-16T16:37:13.497414+0000","last_became_peered":"2023-11-16T16:37:13.497414+0000","last_unstale":"2023-11-16T16:38:03.184212+0000","last_undegraded":"2023-11-16T16:38:03.184212+0000","last_fullsized":"2023-11-16T16:38:03.184212+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:17:42.179250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.509186+0000","last_change":"2023-11-16T16:37:13.946780+0000","last_active":"2023-11-16T16:38:31.509186+0000","last_peered":"2023-11-16T16:38:31.509186+0000","last_clean":"2023-11-16T16:38:31.509186+0000","last_became_active":"2023-11-16T16:37:13.946441+0000","last_became_peered":"2023-11-16T16:37:13.946441+0000","last_unstale":"2023-11-16T16:38:31.509186+0000","last_undegraded":"2023-11-16T16:38:31.509186+0000","last_fullsized":"2023-11-16T16:38:31.509186+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:46:10.856707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"86'8","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.519819+0000","last_change":"2023-11-16T16:38:03.191887+0000","last_active":"2023-11-16T16:38:31.519819+0000","last_peered":"2023-11-16T16:38:31.519819+0000","last_clean":"2023-11-16T16:38:31.519819+0000","last_became_active":"2023-11-16T16:38:03.191402+0000","last_became_peered":"2023-11-16T16:38:03.191402+0000","last_unstale":"2023-11-16T16:38:31.519819+0000","last_undegraded":"2023-11-16T16:38:31.519819+0000","last_fullsized":"2023-11-16T16:38:31.519819+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:47:26.769740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"84'4","reported_seq":117,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:52.446018+0000","last_change":"2023-11-16T16:37:13.498129+0000","last_active":"2023-11-16T16:38:52.446018+0000","last_peered":"2023-11-16T16:38:52.446018+0000","last_clean":"2023-11-16T16:38:52.446018+0000","last_became_active":"2023-11-16T16:37:13.497765+0000","last_became_peered":"2023-11-16T16:37:13.497765+0000","last_unstale":"2023-11-16T16:38:52.446018+0000","last_undegraded":"2023-11-16T16:38:52.446018+0000","last_fullsized":"2023-11-16T16:38:52.446018+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:15:51.118817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":95,"num_read_kb":95,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"86'8","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.513701+0000","last_change":"2023-11-16T16:37:13.938417+0000","last_active":"2023-11-16T16:38:31.513701+0000","last_peered":"2023-11-16T16:38:31.513701+0000","last_clean":"2023-11-16T16:38:31.513701+0000","last_became_active":"2023-11-16T16:37:13.938114+0000","last_became_peered":"2023-11-16T16:37:13.938114+0000","last_unstale":"2023-11-16T16:38:31.513701+0000","last_undegraded":"2023-11-16T16:38:31.513701+0000","last_fullsized":"2023-11-16T16:38:31.513701+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:07:00.909431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"84'3","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.183990+0000","last_change":"2023-11-16T16:37:13.499045+0000","last_active":"2023-11-16T16:38:03.183990+0000","last_peered":"2023-11-16T16:38:03.183990+0000","last_clean":"2023-11-16T16:38:03.183990+0000","last_became_active":"2023-11-16T16:37:13.498598+0000","last_became_peered":"2023-11-16T16:37:13.498598+0000","last_unstale":"2023-11-16T16:38:03.183990+0000","last_undegraded":"2023-11-16T16:38:03.183990+0000","last_fullsized":"2023-11-16T16:38:03.183990+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:29:46.898930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":349,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.183816+0000","last_change":"2023-11-16T16:36:14.090213+0000","last_active":"2023-11-16T16:38:03.183816+0000","last_peered":"2023-11-16T16:38:03.183816+0000","last_clean":"2023-11-16T16:38:03.183816+0000","last_became_active":"2023-11-16T16:36:14.089742+0000","last_became_peered":"2023-11-16T16:36:14.089742+0000","last_unstale":"2023-11-16T16:38:03.183816+0000","last_undegraded":"2023-11-16T16:38:03.183816+0000","last_fullsized":"2023-11-16T16:38:03.183816+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:33:16.178716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:33:16.178716+0000","last_clean_scrub_stamp":"2023-11-16T16:33:16.178716+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:34:52.910521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.511992+0000","last_change":"2023-11-16T16:37:13.945380+0000","last_active":"2023-11-16T16:38:31.511992+0000","last_peered":"2023-11-16T16:38:31.511992+0000","last_clean":"2023-11-16T16:38:31.511992+0000","last_became_active":"2023-11-16T16:37:13.945038+0000","last_became_peered":"2023-11-16T16:37:13.945038+0000","last_unstale":"2023-11-16T16:38:31.511992+0000","last_undegraded":"2023-11-16T16:38:31.511992+0000","last_fullsized":"2023-11-16T16:38:31.511992+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:29:11.812243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"84'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184055+0000","last_change":"2023-11-16T16:37:13.497586+0000","last_active":"2023-11-16T16:38:03.184055+0000","last_peered":"2023-11-16T16:38:03.184055+0000","last_clean":"2023-11-16T16:38:03.184055+0000","last_became_active":"2023-11-16T16:37:13.497233+0000","last_became_peered":"2023-11-16T16:37:13.497233+0000","last_unstale":"2023-11-16T16:38:03.184055+0000","last_undegraded":"2023-11-16T16:38:03.184055+0000","last_fullsized":"2023-11-16T16:38:03.184055+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:47:21.097628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"84'1","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.191390+0000","last_change":"2023-11-16T16:37:13.500621+0000","last_active":"2023-11-16T16:38:03.191390+0000","last_peered":"2023-11-16T16:38:03.191390+0000","last_clean":"2023-11-16T16:38:03.191390+0000","last_became_active":"2023-11-16T16:37:13.500265+0000","last_became_peered":"2023-11-16T16:37:13.500265+0000","last_unstale":"2023-11-16T16:38:03.191390+0000","last_undegraded":"2023-11-16T16:38:03.191390+0000","last_fullsized":"2023-11-16T16:38:03.191390+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:22:59.748224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.509709+0000","last_change":"2023-11-16T16:37:13.930992+0000","last_active":"2023-11-16T16:38:31.509709+0000","last_peered":"2023-11-16T16:38:31.509709+0000","last_clean":"2023-11-16T16:38:31.509709+0000","last_became_active":"2023-11-16T16:37:13.930641+0000","last_became_peered":"2023-11-16T16:37:13.930641+0000","last_unstale":"2023-11-16T16:38:31.509709+0000","last_undegraded":"2023-11-16T16:38:31.509709+0000","last_fullsized":"2023-11-16T16:38:31.509709+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:24:49.314642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'6","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185439+0000","last_change":"2023-11-16T16:37:13.498637+0000","last_active":"2023-11-16T16:38:03.185439+0000","last_peered":"2023-11-16T16:38:03.185439+0000","last_clean":"2023-11-16T16:38:03.185439+0000","last_became_active":"2023-11-16T16:37:13.498341+0000","last_became_peered":"2023-11-16T16:37:13.498341+0000","last_unstale":"2023-11-16T16:38:03.185439+0000","last_undegraded":"2023-11-16T16:38:03.185439+0000","last_fullsized":"2023-11-16T16:38:03.185439+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:30:04.558815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.526478+0000","last_change":"2023-11-16T16:37:13.944142+0000","last_active":"2023-11-16T16:38:31.526478+0000","last_peered":"2023-11-16T16:38:31.526478+0000","last_clean":"2023-11-16T16:38:31.526478+0000","last_became_active":"2023-11-16T16:37:13.943758+0000","last_became_peered":"2023-11-16T16:37:13.943758+0000","last_unstale":"2023-11-16T16:38:31.526478+0000","last_undegraded":"2023-11-16T16:38:31.526478+0000","last_fullsized":"2023-11-16T16:38:31.526478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:57:58.601449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184558+0000","last_change":"2023-11-16T16:37:13.496900+0000","last_active":"2023-11-16T16:38:03.184558+0000","last_peered":"2023-11-16T16:38:03.184558+0000","last_clean":"2023-11-16T16:38:03.184558+0000","last_became_active":"2023-11-16T16:37:13.496528+0000","last_became_peered":"2023-11-16T16:37:13.496528+0000","last_unstale":"2023-11-16T16:38:03.184558+0000","last_undegraded":"2023-11-16T16:38:03.184558+0000","last_fullsized":"2023-11-16T16:38:03.184558+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:56:42.304030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.507960+0000","last_change":"2023-11-16T16:37:13.943104+0000","last_active":"2023-11-16T16:38:31.507960+0000","last_peered":"2023-11-16T16:38:31.507960+0000","last_clean":"2023-11-16T16:38:31.507960+0000","last_became_active":"2023-11-16T16:37:13.942774+0000","last_became_peered":"2023-11-16T16:37:13.942774+0000","last_unstale":"2023-11-16T16:38:31.507960+0000","last_undegraded":"2023-11-16T16:38:31.507960+0000","last_fullsized":"2023-11-16T16:38:31.507960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:24:11.162929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.188270+0000","last_change":"2023-11-16T16:37:13.505994+0000","last_active":"2023-11-16T16:38:03.188270+0000","last_peered":"2023-11-16T16:38:03.188270+0000","last_clean":"2023-11-16T16:38:03.188270+0000","last_became_active":"2023-11-16T16:37:13.505663+0000","last_became_peered":"2023-11-16T16:37:13.505663+0000","last_unstale":"2023-11-16T16:38:03.188270+0000","last_undegraded":"2023-11-16T16:38:03.188270+0000","last_fullsized":"2023-11-16T16:38:03.188270+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:06:13.680491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.479124+0000","last_change":"2023-11-16T16:37:13.941970+0000","last_active":"2023-11-16T16:38:31.479124+0000","last_peered":"2023-11-16T16:38:31.479124+0000","last_clean":"2023-11-16T16:38:31.479124+0000","last_became_active":"2023-11-16T16:37:13.941649+0000","last_became_peered":"2023-11-16T16:37:13.941649+0000","last_unstale":"2023-11-16T16:38:31.479124+0000","last_undegraded":"2023-11-16T16:38:31.479124+0000","last_fullsized":"2023-11-16T16:38:31.479124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:23:17.814240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.182332+0000","last_change":"2023-11-16T16:37:13.503429+0000","last_active":"2023-11-16T16:38:03.182332+0000","last_peered":"2023-11-16T16:38:03.182332+0000","last_clean":"2023-11-16T16:38:03.182332+0000","last_became_active":"2023-11-16T16:37:13.503114+0000","last_became_peered":"2023-11-16T16:37:13.503114+0000","last_unstale":"2023-11-16T16:38:03.182332+0000","last_undegraded":"2023-11-16T16:38:03.182332+0000","last_fullsized":"2023-11-16T16:38:03.182332+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:57:19.910282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.513996+0000","last_change":"2023-11-16T16:37:13.941225+0000","last_active":"2023-11-16T16:38:31.513996+0000","last_peered":"2023-11-16T16:38:31.513996+0000","last_clean":"2023-11-16T16:38:31.513996+0000","last_became_active":"2023-11-16T16:37:13.940964+0000","last_became_peered":"2023-11-16T16:37:13.940964+0000","last_unstale":"2023-11-16T16:38:31.513996+0000","last_undegraded":"2023-11-16T16:38:31.513996+0000","last_fullsized":"2023-11-16T16:38:31.513996+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:21:58.992812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.188151+0000","last_change":"2023-11-16T16:37:13.510154+0000","last_active":"2023-11-16T16:38:03.188151+0000","last_peered":"2023-11-16T16:38:03.188151+0000","last_clean":"2023-11-16T16:38:03.188151+0000","last_became_active":"2023-11-16T16:37:13.509790+0000","last_became_peered":"2023-11-16T16:37:13.509790+0000","last_unstale":"2023-11-16T16:38:03.188151+0000","last_undegraded":"2023-11-16T16:38:03.188151+0000","last_fullsized":"2023-11-16T16:38:03.188151+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:09:32.999016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"86'8","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.509433+0000","last_change":"2023-11-16T16:37:13.942203+0000","last_active":"2023-11-16T16:38:31.509433+0000","last_peered":"2023-11-16T16:38:31.509433+0000","last_clean":"2023-11-16T16:38:31.509433+0000","last_became_active":"2023-11-16T16:37:13.941766+0000","last_became_peered":"2023-11-16T16:37:13.941766+0000","last_unstale":"2023-11-16T16:38:31.509433+0000","last_undegraded":"2023-11-16T16:38:31.509433+0000","last_fullsized":"2023-11-16T16:38:31.509433+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:50:04.530728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.190859+0000","last_change":"2023-11-16T16:37:13.509854+0000","last_active":"2023-11-16T16:38:03.190859+0000","last_peered":"2023-11-16T16:38:03.190859+0000","last_clean":"2023-11-16T16:38:03.190859+0000","last_became_active":"2023-11-16T16:37:13.509466+0000","last_became_peered":"2023-11-16T16:37:13.509466+0000","last_unstale":"2023-11-16T16:38:03.190859+0000","last_undegraded":"2023-11-16T16:38:03.190859+0000","last_fullsized":"2023-11-16T16:38:03.190859+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:38:13.537860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.529514+0000","last_change":"2023-11-16T16:37:13.947906+0000","last_active":"2023-11-16T16:38:31.529514+0000","last_peered":"2023-11-16T16:38:31.529514+0000","last_clean":"2023-11-16T16:38:31.529514+0000","last_became_active":"2023-11-16T16:37:13.947602+0000","last_became_peered":"2023-11-16T16:37:13.947602+0000","last_unstale":"2023-11-16T16:38:31.529514+0000","last_undegraded":"2023-11-16T16:38:31.529514+0000","last_fullsized":"2023-11-16T16:38:31.529514+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:16:45.611613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.188948+0000","last_change":"2023-11-16T16:37:13.506108+0000","last_active":"2023-11-16T16:38:03.188948+0000","last_peered":"2023-11-16T16:38:03.188948+0000","last_clean":"2023-11-16T16:38:03.188948+0000","last_became_active":"2023-11-16T16:37:13.505785+0000","last_became_peered":"2023-11-16T16:37:13.505785+0000","last_unstale":"2023-11-16T16:38:03.188948+0000","last_undegraded":"2023-11-16T16:38:03.188948+0000","last_fullsized":"2023-11-16T16:38:03.188948+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:11:02.793557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.525780+0000","last_change":"2023-11-16T16:37:13.935863+0000","last_active":"2023-11-16T16:38:31.525780+0000","last_peered":"2023-11-16T16:38:31.525780+0000","last_clean":"2023-11-16T16:38:31.525780+0000","last_became_active":"2023-11-16T16:37:13.935399+0000","last_became_peered":"2023-11-16T16:37:13.935399+0000","last_unstale":"2023-11-16T16:38:31.525780+0000","last_undegraded":"2023-11-16T16:38:31.525780+0000","last_fullsized":"2023-11-16T16:38:31.525780+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:45:56.613743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184907+0000","last_change":"2023-11-16T16:37:13.498367+0000","last_active":"2023-11-16T16:38:03.184907+0000","last_peered":"2023-11-16T16:38:03.184907+0000","last_clean":"2023-11-16T16:38:03.184907+0000","last_became_active":"2023-11-16T16:37:13.497039+0000","last_became_peered":"2023-11-16T16:37:13.497039+0000","last_unstale":"2023-11-16T16:38:03.184907+0000","last_undegraded":"2023-11-16T16:38:03.184907+0000","last_fullsized":"2023-11-16T16:38:03.184907+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:57:46.571219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.528964+0000","last_change":"2023-11-16T16:37:13.940617+0000","last_active":"2023-11-16T16:38:31.528964+0000","last_peered":"2023-11-16T16:38:31.528964+0000","last_clean":"2023-11-16T16:38:31.528964+0000","last_became_active":"2023-11-16T16:37:13.940264+0000","last_became_peered":"2023-11-16T16:37:13.940264+0000","last_unstale":"2023-11-16T16:38:31.528964+0000","last_undegraded":"2023-11-16T16:38:31.528964+0000","last_fullsized":"2023-11-16T16:38:31.528964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:59:36.630848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.193922+0000","last_change":"2023-11-16T16:38:03.193922+0000","last_active":"2023-11-16T16:38:03.193922+0000","last_peered":"2023-11-16T16:38:03.193922+0000","last_clean":"2023-11-16T16:38:03.193922+0000","last_became_active":"2023-11-16T16:38:03.190364+0000","last_became_peered":"2023-11-16T16:38:03.190364+0000","last_unstale":"2023-11-16T16:38:03.193922+0000","last_undegraded":"2023-11-16T16:38:03.193922+0000","last_fullsized":"2023-11-16T16:38:03.193922+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:59:04.681132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.529517+0000","last_change":"2023-11-16T16:37:13.943096+0000","last_active":"2023-11-16T16:38:31.529517+0000","last_peered":"2023-11-16T16:38:31.529517+0000","last_clean":"2023-11-16T16:38:31.529517+0000","last_became_active":"2023-11-16T16:37:13.942745+0000","last_became_peered":"2023-11-16T16:37:13.942745+0000","last_unstale":"2023-11-16T16:38:31.529517+0000","last_undegraded":"2023-11-16T16:38:31.529517+0000","last_fullsized":"2023-11-16T16:38:31.529517+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:02:12.721195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.515101+0000","last_change":"2023-11-16T16:37:13.938087+0000","last_active":"2023-11-16T16:38:31.515101+0000","last_peered":"2023-11-16T16:38:31.515101+0000","last_clean":"2023-11-16T16:38:31.515101+0000","last_became_active":"2023-11-16T16:37:13.937780+0000","last_became_peered":"2023-11-16T16:37:13.937780+0000","last_unstale":"2023-11-16T16:38:31.515101+0000","last_undegraded":"2023-11-16T16:38:31.515101+0000","last_fullsized":"2023-11-16T16:38:31.515101+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:05:14.399850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"86'12","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.519834+0000","last_change":"2023-11-16T16:37:13.939000+0000","last_active":"2023-11-16T16:38:31.519834+0000","last_peered":"2023-11-16T16:38:31.519834+0000","last_clean":"2023-11-16T16:38:31.519834+0000","last_became_active":"2023-11-16T16:37:13.938585+0000","last_became_peered":"2023-11-16T16:37:13.938585+0000","last_unstale":"2023-11-16T16:38:31.519834+0000","last_undegraded":"2023-11-16T16:38:31.519834+0000","last_fullsized":"2023-11-16T16:38:31.519834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:13:17.753666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"86'8","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.513239+0000","last_change":"2023-11-16T16:37:13.937696+0000","last_active":"2023-11-16T16:38:31.513239+0000","last_peered":"2023-11-16T16:38:31.513239+0000","last_clean":"2023-11-16T16:38:31.513239+0000","last_became_active":"2023-11-16T16:37:13.936920+0000","last_became_peered":"2023-11-16T16:37:13.936920+0000","last_unstale":"2023-11-16T16:38:31.513239+0000","last_undegraded":"2023-11-16T16:38:31.513239+0000","last_fullsized":"2023-11-16T16:38:31.513239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:55:00.163322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"86'7","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.517747+0000","last_change":"2023-11-16T16:38:03.189679+0000","last_active":"2023-11-16T16:38:31.517747+0000","last_peered":"2023-11-16T16:38:31.517747+0000","last_clean":"2023-11-16T16:38:31.517747+0000","last_became_active":"2023-11-16T16:38:03.189292+0000","last_became_peered":"2023-11-16T16:38:03.189292+0000","last_unstale":"2023-11-16T16:38:31.517747+0000","last_undegraded":"2023-11-16T16:38:31.517747+0000","last_fullsized":"2023-11-16T16:38:31.517747+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:24:48.303402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.507243+0000","last_change":"2023-11-16T16:37:13.949940+0000","last_active":"2023-11-16T16:38:31.507243+0000","last_peered":"2023-11-16T16:38:31.507243+0000","last_clean":"2023-11-16T16:38:31.507243+0000","last_became_active":"2023-11-16T16:37:13.949464+0000","last_became_peered":"2023-11-16T16:37:13.949464+0000","last_unstale":"2023-11-16T16:38:31.507243+0000","last_undegraded":"2023-11-16T16:38:31.507243+0000","last_fullsized":"2023-11-16T16:38:31.507243+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:05:15.768098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.514662+0000","last_change":"2023-11-16T16:37:13.940534+0000","last_active":"2023-11-16T16:38:31.514662+0000","last_peered":"2023-11-16T16:38:31.514662+0000","last_clean":"2023-11-16T16:38:31.514662+0000","last_became_active":"2023-11-16T16:37:13.940174+0000","last_became_peered":"2023-11-16T16:37:13.940174+0000","last_unstale":"2023-11-16T16:38:31.514662+0000","last_undegraded":"2023-11-16T16:38:31.514662+0000","last_fullsized":"2023-11-16T16:38:31.514662+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:29:35.708635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"86'15","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.516894+0000","last_change":"2023-11-16T16:37:13.937519+0000","last_active":"2023-11-16T16:38:31.516894+0000","last_peered":"2023-11-16T16:38:31.516894+0000","last_clean":"2023-11-16T16:38:31.516894+0000","last_became_active":"2023-11-16T16:37:13.937175+0000","last_became_peered":"2023-11-16T16:37:13.937175+0000","last_unstale":"2023-11-16T16:38:31.516894+0000","last_undegraded":"2023-11-16T16:38:31.516894+0000","last_fullsized":"2023-11-16T16:38:31.516894+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:25:23.849158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.503336+0000","last_change":"2023-11-16T16:37:13.951506+0000","last_active":"2023-11-16T16:38:31.503336+0000","last_peered":"2023-11-16T16:38:31.503336+0000","last_clean":"2023-11-16T16:38:31.503336+0000","last_became_active":"2023-11-16T16:37:13.951192+0000","last_became_peered":"2023-11-16T16:37:13.951192+0000","last_unstale":"2023-11-16T16:38:31.503336+0000","last_undegraded":"2023-11-16T16:38:31.503336+0000","last_fullsized":"2023-11-16T16:38:31.503336+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:17:38.223843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.514590+0000","last_change":"2023-11-16T16:37:13.935614+0000","last_active":"2023-11-16T16:38:31.514590+0000","last_peered":"2023-11-16T16:38:31.514590+0000","last_clean":"2023-11-16T16:38:31.514590+0000","last_became_active":"2023-11-16T16:37:13.935237+0000","last_became_peered":"2023-11-16T16:37:13.935237+0000","last_unstale":"2023-11-16T16:38:31.514590+0000","last_undegraded":"2023-11-16T16:38:31.514590+0000","last_fullsized":"2023-11-16T16:38:31.514590+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:30:01.772328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"86'8","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.514616+0000","last_change":"2023-11-16T16:37:13.939387+0000","last_active":"2023-11-16T16:38:31.514616+0000","last_peered":"2023-11-16T16:38:31.514616+0000","last_clean":"2023-11-16T16:38:31.514616+0000","last_became_active":"2023-11-16T16:37:13.939082+0000","last_became_peered":"2023-11-16T16:37:13.939082+0000","last_unstale":"2023-11-16T16:38:31.514616+0000","last_undegraded":"2023-11-16T16:38:31.514616+0000","last_fullsized":"2023-11-16T16:38:31.514616+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:28:19.026881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"86'10","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.527049+0000","last_change":"2023-11-16T16:37:13.936381+0000","last_active":"2023-11-16T16:38:31.527049+0000","last_peered":"2023-11-16T16:38:31.527049+0000","last_clean":"2023-11-16T16:38:31.527049+0000","last_became_active":"2023-11-16T16:37:13.936077+0000","last_became_peered":"2023-11-16T16:37:13.936077+0000","last_unstale":"2023-11-16T16:38:31.527049+0000","last_undegraded":"2023-11-16T16:38:31.527049+0000","last_fullsized":"2023-11-16T16:38:31.527049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:20:31.015344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"86'11","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.526052+0000","last_change":"2023-11-16T16:37:13.934005+0000","last_active":"2023-11-16T16:38:31.526052+0000","last_peered":"2023-11-16T16:38:31.526052+0000","last_clean":"2023-11-16T16:38:31.526052+0000","last_became_active":"2023-11-16T16:37:13.933633+0000","last_became_peered":"2023-11-16T16:37:13.933633+0000","last_unstale":"2023-11-16T16:38:31.526052+0000","last_undegraded":"2023-11-16T16:38:31.526052+0000","last_fullsized":"2023-11-16T16:38:31.526052+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:41:36.990242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"86'8","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.526966+0000","last_change":"2023-11-16T16:37:13.946495+0000","last_active":"2023-11-16T16:38:31.526966+0000","last_peered":"2023-11-16T16:38:31.526966+0000","last_clean":"2023-11-16T16:38:31.526966+0000","last_became_active":"2023-11-16T16:37:13.946148+0000","last_became_peered":"2023-11-16T16:37:13.946148+0000","last_unstale":"2023-11-16T16:38:31.526966+0000","last_undegraded":"2023-11-16T16:38:31.526966+0000","last_fullsized":"2023-11-16T16:38:31.526966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:25:08.609723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":3542208,"data_compressed":211753,"data_compressed_allocated":270336,"data_compressed_original":3522560,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68836,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":609,"num_read_kb":377,"num_write":162,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":200811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416416,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6104,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93739048,"statfs":{"total":95995035648,"available":95988785152,"internally_reserved":0,"allocated":933888,"data_stored":447021,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58599999999999997,"15min":0.58499999999999996},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":2.6150000000000002,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.624,"15min":0.625},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.65100000000000002}]},{"osd":1,"last update":"Thu Nov 16 16:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.64600000000000002,"15min":0.63800000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89400000000000002,"5min":1.026,"15min":1.026},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.66700000000000004,"15min":0.66500000000000004},"min":{"1min":0.44700000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.97499999999999998,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.63400000000000001}]},{"osd":2,"last update":"Thu Nov 16 16:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.58799999999999997,"15min":0.59499999999999997},"min":{"1min":0.41699999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84699999999999998,"5min":1.087,"15min":1.087},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56100000000000005,"15min":0.56200000000000006},"min":{"1min":0.31900000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.86499999999999999,"5min":0.88,"15min":0.88},"last":0.38300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":4,"last update":"Thu Nov 16 16:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.746,"15min":0.748},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.94199999999999995,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.755,"15min":0.75700000000000001},"min":{"1min":0.54500000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0389999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.66400000000000003}]},{"osd":5,"last update":"Thu Nov 16 16:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.69999999999999996,"15min":0.70599999999999996},"min":{"1min":0.379,"5min":0.37,"15min":0.37},"max":{"1min":0.90400000000000003,"5min":1.722,"15min":1.722},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.73399999999999999,"15min":0.74299999999999999},"min":{"1min":0.436,"5min":0.378,"15min":0.378},"max":{"1min":0.97399999999999998,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.53500000000000003}]},{"osd":6,"last update":"Thu Nov 16 16:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.76000000000000001,"15min":0.76600000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0740000000000001,"5min":1.516,"15min":1.516},"last":0.58699999999999997},{"interface":"front","average":{"1min":1.254,"5min":0.91800000000000004,"15min":0.86199999999999999},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":11.007,"5min":11.007,"15min":11.007},"last":0.56399999999999995}]},{"osd":7,"last update":"Thu Nov 16 16:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.61099999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0249999999999999,"5min":1.399,"15min":1.399},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.1279999999999999,"5min":1.363,"15min":1.363},"last":0.748}]},{"osd":8,"last update":"Thu Nov 16 16:38:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.70599999999999996}]},{"osd":9,"last update":"Thu Nov 16 16:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.86399999999999999,"15min":0.86499999999999999},"min":{"1min":0.64700000000000002,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.399,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.347,"5min":0.95399999999999996,"15min":0.88800000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":10.83,"5min":10.83,"15min":10.83},"last":0.77200000000000002}]},{"osd":10,"last update":"Thu Nov 16 16:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.89300000000000002,"15min":0.89200000000000002},"min":{"1min":0.65400000000000003,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.169,"5min":1.427,"15min":1.427},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.89100000000000001,"15min":0.89800000000000002},"min":{"1min":0.65000000000000002,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.073,"5min":1.863,"15min":1.863},"last":0.78000000000000003}]}]},{"osd":10,"up_from":72,"seq":309237645348,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6564,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738588,"statfs":{"total":95995035648,"available":95988314112,"internally_reserved":0,"allocated":1077248,"data_stored":1102490,"data_compressed":41378,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:38:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.78700000000000003}]},{"osd":1,"last update":"Thu Nov 16 16:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.49199999999999999,"15min":0.49299999999999999},"min":{"1min":0.34399999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.67900000000000005,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.52800000000000002,"15min":0.52700000000000002},"min":{"1min":0.33300000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.86499999999999999,"5min":0.879,"15min":0.879},"last":0.41099999999999998}]},{"osd":2,"last update":"Thu Nov 16 16:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64300000000000002,"15min":0.64100000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89300000000000002,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.70899999999999996,"15min":0.71599999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.80400000000000005,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.53100000000000003}]},{"osd":3,"last update":"Thu Nov 16 16:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65900000000000003,"15min":0.66500000000000004},"min":{"1min":0.44,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82399999999999995,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.625,"5min":0.67300000000000004,"15min":0.68100000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.85699999999999998,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.59099999999999997}]},{"osd":4,"last update":"Thu Nov 16 16:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63700000000000001,"15min":0.63500000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.94399999999999995,"5min":1.141,"15min":1.141},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.58299999999999996,"15min":0.57199999999999995},"min":{"1min":0.48199999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.91600000000000004,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.38200000000000001}]},{"osd":5,"last update":"Thu Nov 16 16:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.752,"15min":0.754},"min":{"1min":0.57699999999999996,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.95499999999999996,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.754,"15min":0.75900000000000001},"min":{"1min":0.58399999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.94099999999999995,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.46999999999999997}]},{"osd":6,"last update":"Thu Nov 16 16:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.76200000000000001,"15min":0.76800000000000002},"min":{"1min":0.498,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0029999999999999,"5min":1.282,"15min":1.282},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.77700000000000002,"15min":0.77600000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0149999999999999,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.70999999999999996}]},{"osd":7,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.505}]},{"osd":8,"last update":"Thu Nov 16 16:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.83699999999999997,"15min":0.83499999999999996},"min":{"1min":0.60199999999999998,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0760000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.61599999999999999,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0169999999999999,"5min":1.5,"15min":1.5},"last":0.625}]},{"osd":9,"last update":"Thu Nov 16 16:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0289999999999999,"5min":1.337,"15min":1.337},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81799999999999995,"15min":0.81899999999999995},"min":{"1min":0.53000000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0369999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.69199999999999995}]},{"osd":11,"last update":"Thu Nov 16 16:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.83899999999999997,"15min":0.84199999999999997},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.0900000000000001,"5min":1.2,"15min":1.2},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.83799999999999997,"15min":0.83099999999999996},"min":{"1min":0.67000000000000004,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.1180000000000001,"5min":1.236,"15min":1.236},"last":0.65400000000000003}]}]},{"osd":9,"up_from":67,"seq":287762808871,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6240,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":93738912,"statfs":{"total":95995035648,"available":95988645888,"internally_reserved":0,"allocated":942080,"data_stored":447145,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.44400000000000001,"15min":0.46100000000000002},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.47199999999999998,"15min":0.48199999999999998},"min":{"1min":0.28199999999999997,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.40400000000000003}]},{"osd":1,"last update":"Thu Nov 16 16:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59099999999999997,"15min":0.60599999999999998},"min":{"1min":0.34699999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.60099999999999998,"15min":0.627},"min":{"1min":0.41899999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.60199999999999998}]},{"osd":2,"last update":"Thu Nov 16 16:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.623,"15min":0.64300000000000002},"min":{"1min":0.42499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.62,"15min":0.63800000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77600000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.66900000000000004}]},{"osd":3,"last update":"Thu Nov 16 16:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54100000000000004,"15min":0.56100000000000005},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":0.496},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53700000000000003,"15min":0.56799999999999995},"min":{"1min":0.26900000000000002,"5min":0.25,"15min":0.25},"max":{"1min":2.2490000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":0.47499999999999998}]},{"osd":4,"last update":"Thu Nov 16 16:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.64400000000000002,"15min":0.67500000000000004},"min":{"1min":0.40400000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83899999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67000000000000004,"15min":0.68899999999999995},"min":{"1min":0.45500000000000002,"5min":0.318,"15min":0.318},"max":{"1min":2.2269999999999999,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":0.69099999999999995}]},{"osd":5,"last update":"Thu Nov 16 16:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.67700000000000005,"15min":0.70299999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.82799999999999996,"5min":1.056,"15min":1.056},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.70199999999999996,"15min":0.73299999999999998},"min":{"1min":0.39800000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.68000000000000005}]},{"osd":6,"last update":"Thu Nov 16 16:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.61899999999999999,"15min":0.66200000000000003},"min":{"1min":0.32000000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.67000000000000004,"15min":0.71599999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.53500000000000003}]},{"osd":7,"last update":"Thu Nov 16 16:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.751,"15min":0.77200000000000002},"min":{"1min":0.44700000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":2.153,"5min":2.153,"15min":2.153},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.75,"15min":0.78800000000000003},"min":{"1min":0.55200000000000005,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.76800000000000002}]},{"osd":8,"last update":"Thu Nov 16 16:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.78000000000000003,"15min":0.81699999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.88600000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.746},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.78000000000000003,"15min":0.80900000000000005},"min":{"1min":0.54300000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.876,"5min":1.089,"15min":1.089},"last":0.79700000000000004}]},{"osd":10,"last update":"Thu Nov 16 16:38:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.73699999999999999,"15min":0.747},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.91700000000000004,"5min":1.028,"15min":1.028},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.76100000000000001,"15min":0.76300000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.1819999999999999,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":0.86899999999999999}]}]},{"osd":8,"up_from":59,"seq":253403070509,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6424,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738728,"statfs":{"total":95995035648,"available":95988457472,"internally_reserved":0,"allocated":933888,"data_stored":446953,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57599999999999996,"15min":0.58199999999999996},"min":{"1min":0.40000000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.72099999999999997,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.438,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.72699999999999998,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.80100000000000005}]},{"osd":1,"last update":"Thu Nov 16 16:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.65200000000000002,"15min":0.625},"min":{"1min":0.34100000000000003,"5min":0.318,"15min":0.318},"max":{"1min":0.94899999999999995,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.628,"15min":0.60499999999999998},"min":{"1min":0.46100000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.81299999999999994,"5min":1.383,"15min":1.383},"last":0.64200000000000002}]},{"osd":2,"last update":"Thu Nov 16 16:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.52400000000000002,"15min":0.51700000000000002},"min":{"1min":0.40400000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.86499999999999999,"5min":1.056,"15min":1.056},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.53200000000000003,"15min":0.52600000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.51100000000000001}]},{"osd":3,"last update":"Thu Nov 16 16:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.621,"15min":0.624},"min":{"1min":0.39800000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.83099999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.68300000000000005,"15min":0.67700000000000005},"min":{"1min":0.48099999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.99299999999999999,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.83499999999999996}]},{"osd":4,"last update":"Thu Nov 16 16:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.68100000000000005,"15min":0.64100000000000001},"min":{"1min":0.36899999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.093,"5min":4.0599999999999996,"15min":4.0599999999999996},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.72999999999999998,"15min":0.70799999999999996},"min":{"1min":0.45000000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1200000000000001,"5min":1.167,"15min":1.167},"last":0.71299999999999997}]},{"osd":5,"last update":"Thu Nov 16 16:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.61699999999999999,"15min":0.61499999999999999},"min":{"1min":0.36899999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.89400000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.40000000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.91500000000000004,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.67000000000000004}]},{"osd":6,"last update":"Thu Nov 16 16:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.76000000000000001,"15min":0.75},"min":{"1min":0.53500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1020000000000001,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.75,"15min":0.752},"min":{"1min":0.54000000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.98799999999999999,"5min":1.173,"15min":1.173},"last":0.82399999999999995}]},{"osd":7,"last update":"Thu Nov 16 16:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.78500000000000003,"15min":0.76900000000000002},"min":{"1min":0.49199999999999999,"5min":0.433,"15min":0.433},"max":{"1min":1.05,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.77100000000000002,"15min":0.75800000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0349999999999999,"5min":1.369,"15min":1.369},"last":0.754}]},{"osd":9,"last update":"Thu Nov 16 16:37:55 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.73799999999999999,"15min":0.72999999999999998},"min":{"1min":0.54000000000000004,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.018,"5min":1.036,"15min":1.036},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.875,"5min":0.76900000000000002,"15min":0.751},"min":{"1min":0.58699999999999997,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.95799999999999996}]},{"osd":10,"last update":"Thu Nov 16 16:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.87,"15min":0.85999999999999999},"min":{"1min":0.79700000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.127,"5min":3.5720000000000001,"15min":3.5720000000000001},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.90600000000000003,"15min":0.89200000000000002},"min":{"1min":0.82599999999999996,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.149,"5min":3.5920000000000001,"15min":3.5920000000000001},"last":0.93400000000000005}]},{"osd":11,"last update":"Thu Nov 16 16:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.90900000000000003}]}]},{"osd":7,"up_from":52,"seq":223338299441,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6544,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738608,"statfs":{"total":95995035648,"available":95988334592,"internally_reserved":0,"allocated":991232,"data_stored":1039363,"data_compressed":41378,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:36:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55800000000000005}]},{"osd":1,"last update":"Thu Nov 16 16:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.42999999999999999,"15min":0.41499999999999998},"min":{"1min":0.313,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.44800000000000001,"15min":0.42999999999999999},"min":{"1min":0.33100000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.66800000000000004,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.64600000000000002}]},{"osd":2,"last update":"Thu Nov 16 16:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.58299999999999996,"15min":0.56499999999999995},"min":{"1min":0.45600000000000002,"5min":0.316,"15min":0.316},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.59699999999999998,"15min":0.57699999999999996},"min":{"1min":0.52200000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.73599999999999999}]},{"osd":3,"last update":"Thu Nov 16 16:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58299999999999996,"15min":0.57299999999999995},"min":{"1min":0.44,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.79300000000000004,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.67500000000000004,"15min":0.63700000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":5.6100000000000003,"5min":5.6100000000000003,"15min":5.6100000000000003},"last":0.47899999999999998}]},{"osd":4,"last update":"Thu Nov 16 16:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.54700000000000004,"15min":0.52100000000000002},"min":{"1min":0.371,"5min":0.317,"15min":0.317},"max":{"1min":0.77800000000000002,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.51100000000000001,"15min":0.48699999999999999},"min":{"1min":0.379,"5min":0.311,"15min":0.311},"max":{"1min":0.72799999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.61099999999999999}]},{"osd":5,"last update":"Thu Nov 16 16:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.66800000000000004,"15min":0.65200000000000002},"min":{"1min":0.47599999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.66600000000000004,"15min":0.64100000000000001},"min":{"1min":0.50900000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.69499999999999995}]},{"osd":6,"last update":"Thu Nov 16 16:38:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":0.81599999999999995,"15min":0.73299999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":11.647,"5min":11.647,"15min":11.647},"last":0.752},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.73099999999999998,"15min":0.69099999999999995},"min":{"1min":0.51100000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":4.9550000000000001,"5min":4.9550000000000001,"15min":4.9550000000000001},"last":0.50700000000000001}]},{"osd":8,"last update":"Thu Nov 16 16:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.75700000000000001,"15min":0.75},"min":{"1min":0.60499999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.0289999999999999,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.76700000000000002,"15min":0.76500000000000001},"min":{"1min":0.56200000000000006,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.069,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.82999999999999996}]},{"osd":9,"last update":"Thu Nov 16 16:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.79700000000000004,"15min":0.78000000000000003},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.98099999999999998,"5min":4.7370000000000001,"15min":4.7370000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.83099999999999996,"15min":0.81799999999999995},"min":{"1min":0.54500000000000004,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.97699999999999998,"5min":4.7809999999999997,"15min":4.7809999999999997},"last":0.80700000000000005}]},{"osd":10,"last update":"Thu Nov 16 16:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.80400000000000005,"15min":0.80100000000000005},"min":{"1min":0.64300000000000002,"5min":0.443,"15min":0.443},"max":{"1min":1.026,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.76400000000000001,"15min":0.75800000000000001},"min":{"1min":0.64000000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0509999999999999,"5min":1.054,"15min":1.054},"last":0.84099999999999997}]},{"osd":11,"last update":"Thu Nov 16 16:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.873}]}]},{"osd":6,"up_from":47,"seq":201863462965,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6356,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738796,"statfs":{"total":95995035648,"available":95988527104,"internally_reserved":0,"allocated":929792,"data_stored":446540,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.42599999999999999,"15min":0.41599999999999998},"min":{"1min":0.29199999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.43099999999999999,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.45900000000000002,"15min":0.441},"min":{"1min":0.29699999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.56499999999999995,"5min":3.472,"15min":3.472},"last":0.36099999999999999}]},{"osd":1,"last update":"Thu Nov 16 16:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.64400000000000002,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.63700000000000001,"15min":0.59699999999999998},"min":{"1min":0.40699999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.65400000000000003,"5min":4.4669999999999996,"15min":4.4669999999999996},"last":0.443}]},{"osd":2,"last update":"Thu Nov 16 16:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.63600000000000001,"15min":0.63200000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.68300000000000005,"5min":3.5369999999999999,"15min":3.5369999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.64900000000000002,"15min":0.629},"min":{"1min":0.443,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.64500000000000002,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.51700000000000002}]},{"osd":3,"last update":"Thu Nov 16 16:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.504,"15min":0.48299999999999998},"min":{"1min":0.28999999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.46600000000000003,"5min":3.3700000000000001,"15min":3.3700000000000001},"last":0.434},{"interface":"front","average":{"1min":0.434,"5min":0.53600000000000003,"15min":0.53400000000000003},"min":{"1min":0.35099999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.50900000000000001,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.502}]},{"osd":4,"last update":"Thu Nov 16 16:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.66800000000000004,"15min":0.60799999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.75900000000000001,"5min":4.4900000000000002,"15min":4.4900000000000002},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.68899999999999995,"15min":0.627},"min":{"1min":0.379,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.69399999999999995,"5min":4.4509999999999996,"15min":4.4509999999999996},"last":0.47299999999999998}]},{"osd":5,"last update":"Thu Nov 16 16:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.66900000000000004,"15min":0.67500000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.71599999999999997,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.71099999999999997,"15min":0.70299999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.75,"5min":3.5019999999999998,"15min":3.5019999999999998},"last":0.59099999999999997}]},{"osd":7,"last update":"Thu Nov 16 16:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.70899999999999996,"15min":0.70499999999999996},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":2.452,"5min":2.452,"15min":2.452},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.73999999999999999,"15min":0.71799999999999997},"min":{"1min":0.54400000000000004,"5min":0.436,"15min":0.436},"max":{"1min":3.4449999999999998,"5min":3.4449999999999998,"15min":3.4449999999999998},"last":0.60999999999999999}]},{"osd":8,"last update":"Thu Nov 16 16:38:32 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.752,"15min":0.76900000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":2.3919999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.77800000000000002,"15min":0.78300000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":3.488,"5min":3.488,"15min":3.488},"last":0.64600000000000002}]},{"osd":9,"last update":"Thu Nov 16 16:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.77300000000000002,"15min":0.76700000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.68700000000000006,"5min":6.1539999999999999,"15min":6.1539999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.79800000000000004,"15min":0.76800000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.67800000000000005,"5min":8.2460000000000004,"15min":8.2460000000000004},"last":0.57799999999999996}]},{"osd":10,"last update":"Thu Nov 16 16:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.81200000000000006,"15min":0.81000000000000005},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":3.5209999999999999,"5min":3.5209999999999999,"15min":3.5209999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.82499999999999996,"15min":0.82399999999999995},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":3.419,"5min":3.419,"15min":3.419},"last":0.65600000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7440,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":93737712,"statfs":{"total":95995035648,"available":95987417088,"internally_reserved":0,"allocated":991232,"data_stored":505103,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:38:06 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.48499999999999999,"15min":0.45800000000000002},"min":{"1min":0.34899999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.67000000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52100000000000002,"15min":0.48799999999999999},"min":{"1min":0.311,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.83499999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.68799999999999994}]},{"osd":2,"last update":"Thu Nov 16 16:38:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.58299999999999996,"15min":0.51100000000000001},"min":{"1min":0.45500000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.58199999999999996,"15min":0.51700000000000002},"min":{"1min":0.46400000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.85199999999999998,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.55000000000000004}]},{"osd":3,"last update":"Thu Nov 16 16:38:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58599999999999997,"15min":0.55800000000000005},"min":{"1min":0.33900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.93700000000000006,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60999999999999999,"15min":0.57899999999999996},"min":{"1min":0.38100000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.89700000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.65000000000000002}]},{"osd":4,"last update":"Thu Nov 16 16:38:06 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.46300000000000002,"15min":0.43099999999999999},"min":{"1min":0.33700000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.79500000000000004,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55000000000000004,"15min":0.48499999999999999},"min":{"1min":0.35499999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.82899999999999996,"5min":1.242,"15min":1.242},"last":0.84399999999999997}]},{"osd":5,"last update":"Thu Nov 16 16:38:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.67300000000000004,"15min":0.623},"min":{"1min":0.55100000000000005,"5min":0.38,"15min":0.38},"max":{"1min":0.90700000000000003,"5min":1.004,"15min":1.004},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.65000000000000002,"15min":0.60699999999999998},"min":{"1min":0.53700000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93899999999999995,"5min":1.004,"15min":1.004},"last":0.621}]},{"osd":6,"last update":"Thu Nov 16 16:38:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.69599999999999995,"15min":0.67200000000000004},"min":{"1min":0.51900000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.02,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69299999999999995,"15min":0.66700000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.82599999999999996}]},{"osd":7,"last update":"Thu Nov 16 16:38:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65300000000000002,"15min":0.63500000000000001},"min":{"1min":0.39500000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.0569999999999999,"5min":1.077,"15min":1.077},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.67900000000000005},"min":{"1min":0.42599999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.018,"5min":1.143,"15min":1.143},"last":1.034}]},{"osd":8,"last update":"Thu Nov 16 16:38:30 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.76800000000000002,"15min":0.75800000000000001},"min":{"1min":0.56799999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0289999999999999,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.75,"15min":0.73999999999999999},"min":{"1min":0.52800000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0640000000000001,"5min":1.145,"15min":1.145},"last":0.877}]},{"osd":9,"last update":"Thu Nov 16 16:37:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.77900000000000003,"15min":0.79900000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.87,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.84599999999999997,"15min":0.86399999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.97299999999999998,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.94999999999999996}]},{"osd":10,"last update":"Thu Nov 16 16:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.83999999999999997,"15min":0.84999999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.99199999999999999,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.81599999999999995,"15min":0.81999999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.98099999999999998,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.92400000000000004}]},{"osd":11,"last update":"Thu Nov 16 16:38:15 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.96999999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7272,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":6336,"kb_avail":93737880,"statfs":{"total":95995035648,"available":95987589120,"internally_reserved":0,"allocated":950272,"data_stored":447805,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":6488064},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Nov 16 16:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.54700000000000004,"15min":0.48199999999999998},"min":{"1min":0.38400000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":3.6869999999999998,"5min":3.6869999999999998,"15min":3.6869999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.57999999999999996,"15min":0.48799999999999999},"min":{"1min":0.40600000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":6.6059999999999999,"5min":6.6059999999999999,"15min":6.6059999999999999},"last":0.51800000000000002}]},{"osd":2,"last update":"Thu Nov 16 16:38:27 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.63300000000000001,"15min":0.53900000000000003},"min":{"1min":0.39700000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":6.4000000000000004,"5min":6.4000000000000004,"15min":6.4000000000000004},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.60799999999999998,"15min":0.54600000000000004},"min":{"1min":0.42599999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":3.7829999999999999,"5min":3.7829999999999999,"15min":3.7829999999999999},"last":0.60699999999999998}]},{"osd":3,"last update":"Thu Nov 16 16:38:15 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":9.0139999999999993,"5min":9.0139999999999993,"15min":9.0139999999999993},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":3.6400000000000001,"5min":3.6400000000000001,"15min":3.6400000000000001},"last":0.38600000000000001}]},{"osd":4,"last update":"Thu Nov 16 16:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.61499999999999999,"15min":0.56299999999999994},"min":{"1min":0.44,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":3.8109999999999999,"5min":3.8109999999999999,"15min":3.8109999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.871,"5min":0.64900000000000002,"15min":0.57199999999999995},"min":{"1min":0.42099999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":6.577,"5min":6.577,"15min":6.577},"last":0.70899999999999996}]},{"osd":5,"last update":"Thu Nov 16 16:38:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":0.65700000000000003,"15min":0.58299999999999996},"min":{"1min":0.47899999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":9.0690000000000008,"5min":9.0690000000000008,"15min":9.0690000000000008},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.70599999999999996,"15min":0.64100000000000001},"min":{"1min":0.38800000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":6.4539999999999997,"5min":6.4539999999999997,"15min":6.4539999999999997},"last":0.75800000000000001}]},{"osd":6,"last update":"Thu Nov 16 16:38:50 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.623,"15min":0.58899999999999997},"min":{"1min":0.314,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68300000000000005,"5min":6.5190000000000001,"15min":6.5190000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.502,"5min":0.622,"15min":0.57199999999999995},"min":{"1min":0.313,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.65600000000000003,"5min":9.141,"15min":9.141},"last":0.54200000000000004}]},{"osd":7,"last update":"Thu Nov 16 16:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.68400000000000005,"15min":0.67600000000000005},"min":{"1min":0.495,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.74199999999999999,"15min":0.73199999999999998},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.56399999999999995}]},{"osd":8,"last update":"Thu Nov 16 16:38:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.78900000000000003,"15min":0.80900000000000005},"min":{"1min":0.496,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":3.8410000000000002,"5min":3.8410000000000002,"15min":3.8410000000000002},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.86099999999999999,"15min":0.86199999999999999},"min":{"1min":0.57099999999999995,"5min":0.498,"15min":0.498},"max":{"1min":6.4870000000000001,"5min":6.4870000000000001,"15min":6.4870000000000001},"last":0.78500000000000003}]},{"osd":9,"last update":"Thu Nov 16 16:37:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.85499999999999998,"15min":0.86199999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":6.6349999999999998,"5min":6.6349999999999998,"15min":6.6349999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.79000000000000004,"15min":0.76600000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":9.1690000000000005,"5min":9.1690000000000005,"15min":9.1690000000000005},"last":0.59199999999999997}]},{"osd":10,"last update":"Thu Nov 16 16:38:15 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.78500000000000003,"15min":0.78100000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":3.7229999999999999,"5min":3.7229999999999999,"15min":3.7229999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.88300000000000001,"15min":0.86499999999999999},"min":{"1min":0.55400000000000005,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":6.5460000000000003,"5min":6.5460000000000003,"15min":6.5460000000000003},"last":0.80500000000000005}]},{"osd":11,"last update":"Thu Nov 16 16:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.99299999999999999,"15min":1.0409999999999999},"min":{"1min":0.56399999999999995,"5min":0.502,"15min":0.502},"max":{"1min":0.86399999999999999,"5min":9.1039999999999992,"15min":9.1039999999999992},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.83499999999999996,"15min":0.85499999999999998},"min":{"1min":0.53500000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.91200000000000003,"5min":3.7530000000000001,"15min":3.7530000000000001},"last":0.74199999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7532,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":6528,"kb_avail":93737620,"statfs":{"total":95995035648,"available":95987322880,"internally_reserved":0,"allocated":1019904,"data_stored":512951,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":6684672},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:38:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.52700000000000002,"15min":0.52300000000000002},"min":{"1min":0.372,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.88300000000000001,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.50700000000000001,"15min":0.49099999999999999},"min":{"1min":0.35999999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.81200000000000006,"5min":1.006,"15min":1.006},"last":0.47899999999999998}]},{"osd":1,"last update":"Thu Nov 16 16:38:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.55500000000000005,"15min":0.52300000000000002},"min":{"1min":0.40400000000000003,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.60099999999999998,"15min":0.56499999999999995},"min":{"1min":0.46700000000000003,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.91800000000000004,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.70399999999999996}]},{"osd":3,"last update":"Thu Nov 16 16:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.66300000000000003,"15min":0.60699999999999998},"min":{"1min":0.38800000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.94199999999999995,"5min":6.9740000000000002,"15min":6.9740000000000002},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59299999999999997,"15min":0.56299999999999994},"min":{"1min":0.39200000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.86699999999999999,"5min":1.03,"15min":1.03},"last":0.625}]},{"osd":4,"last update":"Thu Nov 16 16:38:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.69699999999999995}]},{"osd":5,"last update":"Thu Nov 16 16:38:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.51700000000000002,"15min":0.49099999999999999},"min":{"1min":0.376,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.83199999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.48799999999999999,"15min":0.46400000000000002},"min":{"1min":0.39100000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.71699999999999997,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.55700000000000005}]},{"osd":6,"last update":"Thu Nov 16 16:38:50 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.70399999999999996,"15min":0.66200000000000003},"min":{"1min":0.52100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0509999999999999,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.66700000000000004,"15min":0.621},"min":{"1min":0.45400000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.97199999999999998,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.67500000000000004}]},{"osd":7,"last update":"Thu Nov 16 16:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.64300000000000002,"15min":0.64400000000000002},"min":{"1min":0.41599999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70199999999999996,"15min":0.70399999999999996},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.92700000000000005,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.57699999999999996}]},{"osd":8,"last update":"Thu Nov 16 16:38:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.59599999999999997,"15min":0.57599999999999996},"min":{"1min":0.35299999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.84199999999999997,"5min":1.008,"15min":1.008},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.63300000000000001,"15min":0.627},"min":{"1min":0.38500000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.82499999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.59799999999999998}]},{"osd":9,"last update":"Thu Nov 16 16:37:54 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.86799999999999999,"15min":0.86699999999999999},"min":{"1min":0.47499999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.6080000000000001,"5min":4.524,"15min":4.524},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.751,"15min":0.73299999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.6200000000000001,"5min":2.6200000000000001,"15min":2.6200000000000001},"last":0.79000000000000004}]},{"osd":10,"last update":"Thu Nov 16 16:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.73399999999999999,"15min":0.72499999999999998},"min":{"1min":0.55100000000000005,"5min":0.497,"15min":0.497},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.72599999999999998,"15min":0.72099999999999997},"min":{"1min":0.61499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.92500000000000004,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.75800000000000001}]},{"osd":11,"last update":"Thu Nov 16 16:38:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.874},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.80500000000000005}]}]},{"osd":3,"up_from":26,"seq":111669149761,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6688,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":93738464,"statfs":{"total":95995035648,"available":95988187136,"internally_reserved":0,"allocated":942080,"data_stored":446526,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:38:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.374,"15min":0.35199999999999998},"min":{"1min":0.28599999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.40100000000000002,"15min":0.36499999999999999},"min":{"1min":0.309,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.36399999999999999}]},{"osd":1,"last update":"Thu Nov 16 16:38:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56100000000000005,"15min":0.55100000000000005},"min":{"1min":0.371,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56399999999999995,"15min":0.55900000000000005},"min":{"1min":0.40300000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.84099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.65400000000000003}]},{"osd":2,"last update":"Thu Nov 16 16:38:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.59199999999999997,"15min":0.56100000000000005},"min":{"1min":0.439,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.91400000000000003,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57199999999999995,"15min":0.54900000000000004},"min":{"1min":0.34699999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.53000000000000003}]},{"osd":4,"last update":"Thu Nov 16 16:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56299999999999994,"15min":0.54900000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79900000000000004,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60599999999999998,"15min":0.59599999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.56200000000000006}]},{"osd":5,"last update":"Thu Nov 16 16:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59899999999999998,"15min":0.58099999999999996},"min":{"1min":0.39600000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95299999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.61899999999999999,"15min":0.59599999999999997},"min":{"1min":0.47399999999999998,"5min":0.37,"15min":0.37},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.55200000000000005}]},{"osd":6,"last update":"Thu Nov 16 16:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51800000000000002,"15min":0.498},"min":{"1min":0.35599999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.72399999999999998,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53200000000000003,"15min":0.51400000000000001},"min":{"1min":0.377,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.75800000000000001,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.44600000000000001}]},{"osd":7,"last update":"Thu Nov 16 16:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.65300000000000002,"15min":0.64900000000000002},"min":{"1min":0.503,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.68700000000000006,"15min":0.68500000000000005},"min":{"1min":0.42699999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.68400000000000005}]},{"osd":8,"last update":"Thu Nov 16 16:36:27 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.63200000000000001}]},{"osd":9,"last update":"Thu Nov 16 16:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.73799999999999999,"15min":0.76300000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80500000000000005,"5min":1.024,"15min":1.024},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.68999999999999995,"15min":0.70899999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90300000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.495}]},{"osd":10,"last update":"Thu Nov 16 16:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.73299999999999998,"15min":0.74299999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.74099999999999999,"15min":0.751},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.97599999999999998,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.70799999999999996}]},{"osd":11,"last update":"Thu Nov 16 16:38:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.60799999999999998}]}]},{"osd":4,"up_from":34,"seq":146028888126,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6488,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738664,"statfs":{"total":95995035648,"available":95988391936,"internally_reserved":0,"allocated":933888,"data_stored":446923,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.52400000000000002,"15min":0.502},"min":{"1min":0.38200000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.52900000000000003,"15min":0.51500000000000001},"min":{"1min":0.40899999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.378}]},{"osd":1,"last update":"Thu Nov 16 16:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.41099999999999998,"15min":0.39300000000000002},"min":{"1min":0.33700000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.67900000000000005,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.37},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.434,"15min":0.40999999999999998},"min":{"1min":0.378,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.748,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.39700000000000002}]},{"osd":2,"last update":"Thu Nov 16 16:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.77300000000000002,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.623,"15min":0.59299999999999997},"min":{"1min":0.53000000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.81999999999999995,"5min":1.056,"15min":1.056},"last":0.80300000000000005}]},{"osd":3,"last update":"Thu Nov 16 16:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.59199999999999997,"15min":0.55000000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57299999999999995,"15min":0.55300000000000005},"min":{"1min":0.40100000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.51500000000000001}]},{"osd":5,"last update":"Thu Nov 16 16:38:28 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.623,"15min":0.627},"min":{"1min":0.40100000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.76500000000000001,"5min":1.075,"15min":1.075},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.65700000000000003,"15min":0.63200000000000001},"min":{"1min":0.49099999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.89700000000000002,"5min":1.093,"15min":1.093},"last":0.84499999999999997}]},{"osd":6,"last update":"Thu Nov 16 16:38:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65000000000000002,"15min":0.60899999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.85399999999999998,"5min":1.004,"15min":1.004},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.61599999999999999,"15min":0.59799999999999998},"min":{"1min":0.433,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77300000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.46999999999999997}]},{"osd":7,"last update":"Thu Nov 16 16:38:13 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.61499999999999999,"15min":0.56299999999999994},"min":{"1min":0.45400000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.99399999999999999,"5min":1.123,"15min":1.123},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.67300000000000004,"15min":0.621},"min":{"1min":0.51600000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.99199999999999999,"5min":1.069,"15min":1.069},"last":0.78400000000000003}]},{"osd":8,"last update":"Thu Nov 16 16:38:28 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.76400000000000001,"15min":0.73199999999999998},"min":{"1min":0.53700000000000003,"5min":0.495,"15min":0.495},"max":{"1min":0.98699999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.77000000000000002,"15min":0.73899999999999999},"min":{"1min":0.51700000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.98199999999999998,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.82799999999999996}]},{"osd":9,"last update":"Thu Nov 16 16:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.79200000000000004,"15min":0.78900000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.94099999999999995,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.72099999999999997,"15min":0.71799999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94199999999999995,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.56299999999999994}]},{"osd":10,"last update":"Thu Nov 16 16:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.68200000000000005,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0109999999999999,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.753},{"interface":"front","average":{"1min":0.872,"5min":0.83999999999999997,"15min":0.83399999999999996},"min":{"1min":0.73399999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.74299999999999999}]},{"osd":11,"last update":"Thu Nov 16 16:38:13 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.86499999999999999}]}]},{"osd":5,"up_from":39,"seq":167503724603,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6684,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738468,"statfs":{"total":95995035648,"available":95988191232,"internally_reserved":0,"allocated":1003520,"data_stored":1039095,"data_compressed":41378,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.55500000000000005,"15min":0.51300000000000001},"min":{"1min":0.38100000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":4.2380000000000004,"5min":4.2380000000000004,"15min":4.2380000000000004},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.57499999999999996,"15min":0.54000000000000004},"min":{"1min":0.42899999999999999,"5min":0.311,"15min":0.311},"max":{"1min":3.5590000000000002,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":0.58599999999999997}]},{"osd":1,"last update":"Thu Nov 16 16:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.61299999999999999,"15min":0.59399999999999997},"min":{"1min":0.55600000000000005,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.997,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.60199999999999998,"15min":0.58999999999999997},"min":{"1min":0.42699999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84099999999999997,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.56899999999999995}]},{"osd":2,"last update":"Thu Nov 16 16:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.443,"15min":0.42899999999999999},"min":{"1min":0.32700000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.70399999999999996,"5min":1.417,"15min":1.417},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.46400000000000002,"15min":0.44},"min":{"1min":0.33700000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.70799999999999996,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.35699999999999998}]},{"osd":3,"last update":"Thu Nov 16 16:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.60299999999999998,"15min":0.56200000000000006},"min":{"1min":0.38,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":3.3079999999999998,"5min":3.3079999999999998,"15min":3.3079999999999998},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.67300000000000004,"15min":0.621},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":5.343,"5min":5.343,"15min":5.343},"last":0.47699999999999998}]},{"osd":4,"last update":"Thu Nov 16 16:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.69999999999999996,"15min":0.63600000000000001},"min":{"1min":0.46000000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85599999999999998,"5min":5.5789999999999997,"15min":5.5789999999999997},"last":0.628},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.62,"15min":0.59799999999999998},"min":{"1min":0.437,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84499999999999997,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.52900000000000003}]},{"osd":6,"last update":"Thu Nov 16 16:38:48 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.68700000000000006,"15min":0.63900000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.996,"5min":3.274,"15min":3.274},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.68300000000000005,"15min":0.64700000000000002},"min":{"1min":0.40100000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.0109999999999999,"5min":3.472,"15min":3.472},"last":0.50600000000000001}]},{"osd":7,"last update":"Thu Nov 16 16:38:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.72199999999999998,"15min":0.71599999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0449999999999999,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.73799999999999999,"15min":0.73199999999999998},"min":{"1min":0.48099999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0580000000000001,"5min":1.544,"15min":1.544},"last":0.66900000000000004}]},{"osd":8,"last update":"Thu Nov 16 16:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.66500000000000004,"15min":0.65100000000000002},"min":{"1min":0.48699999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.92500000000000004,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.746,"15min":0.73199999999999998},"min":{"1min":0.59499999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.69399999999999995}]},{"osd":9,"last update":"Thu Nov 16 16:38:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.90200000000000002,"15min":0.82799999999999996},"min":{"1min":0.52400000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1699999999999999,"5min":5.5679999999999996,"15min":5.5679999999999996},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.82099999999999995,"15min":0.77100000000000002},"min":{"1min":0.56399999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0640000000000001,"5min":5.3230000000000004,"15min":5.3230000000000004},"last":0.71299999999999997}]},{"osd":10,"last update":"Thu Nov 16 16:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.80000000000000004,"15min":0.78200000000000003},"min":{"1min":0.63900000000000001,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.80200000000000005,"15min":0.78800000000000003},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.755}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33928,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":58776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":66624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-16T16:38:57.120 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-16T16:38:57.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.279+0000 7fca5519e700 1 -- 172.21.15.1:0/2857696592 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca50100230 msgr2=0x7fca501006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:57.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.279+0000 7fca5519e700 1 --2- 172.21.15.1:0/2857696592 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca50100230 0x7fca501006a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fca38009990 tx=0x7fca38009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:57.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.279+0000 7fca5519e700 1 -- 172.21.15.1:0/2857696592 shutdown_connections 2023-11-16T16:38:57.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.279+0000 7fca5519e700 1 --2- 172.21.15.1:0/2857696592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca50100be0 0x7fca50107480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:57.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.279+0000 7fca5519e700 1 --2- 172.21.15.1:0/2857696592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca500fefa0 0x7fca500ff3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:57.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.279+0000 7fca5519e700 1 --2- 172.21.15.1:0/2857696592 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca50100230 0x7fca501006a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:57.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.279+0000 7fca5519e700 1 -- 172.21.15.1:0/2857696592 >> 172.21.15.1:0/2857696592 conn(0x7fca500fa5e0 msgr2=0x7fca500fc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:57.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.279+0000 7fca5519e700 1 -- 172.21.15.1:0/2857696592 shutdown_connections 2023-11-16T16:38:57.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.279+0000 7fca5519e700 1 -- 172.21.15.1:0/2857696592 wait complete. 2023-11-16T16:38:57.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.280+0000 7fca5519e700 1 Processor -- start 2023-11-16T16:38:57.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.280+0000 7fca5519e700 1 -- start start 2023-11-16T16:38:57.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.280+0000 7fca5519e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca500fefa0 0x7fca50068950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:57.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.280+0000 7fca5519e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca50100230 0x7fca5019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:57.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.280+0000 7fca5519e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca50100be0 0x7fca5019d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:57.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.280+0000 7fca5519e700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5019d7e0 con 0x7fca50100be0 2023-11-16T16:38:57.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.280+0000 7fca5519e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5019d950 con 0x7fca500fefa0 2023-11-16T16:38:57.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.280+0000 7fca5519e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5019db10 con 0x7fca50100230 2023-11-16T16:38:57.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.280+0000 7fca5499d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca50100be0 0x7fca5019d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:57.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.280+0000 7fca5499d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca50100be0 0x7fca5019d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42978/0 (socket says 172.21.15.1:42978) 2023-11-16T16:38:57.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.280+0000 7fca5499d700 1 -- 172.21.15.1:0/1240719827 learned_addr learned my addr 172.21.15.1:0/1240719827 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:57.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.280+0000 7fca4ffff700 1 --2- 172.21.15.1:0/1240719827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca500fefa0 0x7fca50068950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:57.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.280+0000 7fca4f7fe700 1 --2- 172.21.15.1:0/1240719827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca50100230 0x7fca5019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:57.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.280+0000 7fca5499d700 1 -- 172.21.15.1:0/1240719827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca50100230 msgr2=0x7fca5019a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:57.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.280+0000 7fca5499d700 1 --2- 172.21.15.1:0/1240719827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca50100230 0x7fca5019a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:57.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.280+0000 7fca5499d700 1 -- 172.21.15.1:0/1240719827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca500fefa0 msgr2=0x7fca50068950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:57.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.280+0000 7fca5499d700 1 --2- 172.21.15.1:0/1240719827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca500fefa0 0x7fca50068950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:57.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.281+0000 7fca5499d700 1 -- 172.21.15.1:0/1240719827 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca380095f0 con 0x7fca50100be0 2023-11-16T16:38:57.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.281+0000 7fca4ffff700 1 --2- 172.21.15.1:0/1240719827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca500fefa0 0x7fca50068950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:38:57.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.281+0000 7fca5499d700 1 --2- 172.21.15.1:0/1240719827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca50100be0 0x7fca5019d010 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fca48007d70 tx=0x7fca4800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:57.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.281+0000 7fca4d7fa700 1 -- 172.21.15.1:0/1240719827 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4800ac90 con 0x7fca50100be0 2023-11-16T16:38:57.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.281+0000 7fca4d7fa700 1 -- 172.21.15.1:0/1240719827 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca4800adf0 con 0x7fca50100be0 2023-11-16T16:38:57.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.281+0000 7fca5519e700 1 -- 172.21.15.1:0/1240719827 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca5010a900 con 0x7fca50100be0 2023-11-16T16:38:57.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.281+0000 7fca4d7fa700 1 -- 172.21.15.1:0/1240719827 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca480115e0 con 0x7fca50100be0 2023-11-16T16:38:57.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.281+0000 7fca5519e700 1 -- 172.21.15.1:0/1240719827 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca5010a770 con 0x7fca50100be0 2023-11-16T16:38:57.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.282+0000 7fca5519e700 1 -- 172.21.15.1:0/1240719827 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca50105ed0 con 0x7fca50100be0 2023-11-16T16:38:57.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.283+0000 7fca4d7fa700 1 -- 172.21.15.1:0/1240719827 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fca48012420 con 0x7fca50100be0 2023-11-16T16:38:57.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.283+0000 7fca4d7fa700 1 --2- 172.21.15.1:0/1240719827 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fca3c0a1150 0x7fca3c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:57.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.284+0000 7fca4ffff700 1 --2- 172.21.15.1:0/1240719827 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fca3c0a1150 0x7fca3c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:57.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.284+0000 7fca4d7fa700 1 -- 172.21.15.1:0/1240719827 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fca480ccb00 con 0x7fca50100be0 2023-11-16T16:38:57.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.284+0000 7fca4ffff700 1 --2- 172.21.15.1:0/1240719827 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fca3c0a1150 0x7fca3c0a3600 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fca40005950 tx=0x7fca4000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:57.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.288+0000 7fca4d7fa700 1 -- 172.21.15.1:0/1240719827 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fca48096250 con 0x7fca50100be0 2023-11-16T16:38:57.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:57 smithi001 ceph-mon[98800]: pgmap v313: 97 pgs: 97 active+clean; 644 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 853 B/s wr, 8 op/s 2023-11-16T16:38:57.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:57 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1065218116' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:38:57.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:57 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3008440254' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:38:57.487 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.485+0000 7fca5519e700 1 -- 172.21.15.1:0/1240719827 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fca50061c80 con 0x7fca3c0a1150 2023-11-16T16:38:57.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.497+0000 7fca4d7fa700 1 -- 172.21.15.1:0/1240719827 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344244 (secure 0 0 0) 0x7fca50061c80 con 0x7fca3c0a1150 2023-11-16T16:38:57.503 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:38:57.505 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.501+0000 7fca5519e700 1 -- 172.21.15.1:0/1240719827 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fca3c0a1150 msgr2=0x7fca3c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:57.505 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.501+0000 7fca5519e700 1 --2- 172.21.15.1:0/1240719827 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fca3c0a1150 0x7fca3c0a3600 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fca40005950 tx=0x7fca4000b410 comp rx=0 tx=0).stop 2023-11-16T16:38:57.505 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.501+0000 7fca5519e700 1 -- 172.21.15.1:0/1240719827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca50100be0 msgr2=0x7fca5019d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:57.505 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.501+0000 7fca5519e700 1 --2- 172.21.15.1:0/1240719827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca50100be0 0x7fca5019d010 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fca48007d70 tx=0x7fca4800a340 comp rx=0 tx=0).stop 2023-11-16T16:38:57.505 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.501+0000 7fca5519e700 1 -- 172.21.15.1:0/1240719827 shutdown_connections 2023-11-16T16:38:57.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.501+0000 7fca5519e700 1 --2- 172.21.15.1:0/1240719827 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fca3c0a1150 0x7fca3c0a3600 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:57.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.501+0000 7fca5519e700 1 --2- 172.21.15.1:0/1240719827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca50100be0 0x7fca5019d010 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:57.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.501+0000 7fca5519e700 1 --2- 172.21.15.1:0/1240719827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca500fefa0 0x7fca50068950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:57.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.501+0000 7fca5519e700 1 --2- 172.21.15.1:0/1240719827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fca50100230 0x7fca5019a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:57.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.501+0000 7fca5519e700 1 -- 172.21.15.1:0/1240719827 >> 172.21.15.1:0/1240719827 conn(0x7fca500fa5e0 msgr2=0x7fca500fc760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:57.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.501+0000 7fca5519e700 1 -- 172.21.15.1:0/1240719827 shutdown_connections 2023-11-16T16:38:57.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.501+0000 7fca5519e700 1 -- 172.21.15.1:0/1240719827 wait complete. 2023-11-16T16:38:57.506 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2023-11-16T16:38:57.507 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:57 smithi026 ceph-mon[100112]: pgmap v313: 97 pgs: 97 active+clean; 644 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 853 B/s wr, 8 op/s 2023-11-16T16:38:57.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:57 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1065218116' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:38:57.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:57 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/3008440254' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:38:57.516 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":313,"stamp":"2023-11-16T16:38:55.507087+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659450,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":745,"num_read_kb":496,"num_write":345,"num_write_kb":3058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180848,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":782,"ondisk_log_size":782,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":80336,"kb_used_data":11376,"kb_used_omap":0,"kb_used_meta":68864,"kb_avail":1124861488,"statfs":{"total":1151940427776,"available":1151858163712,"internally_reserved":0,"allocated":11649024,"data_stored":7327915,"data_compressed":176451,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":0,"internal_metadata":70516736},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6015,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":93,"num_read_kb":53,"num_write":5,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003024"},"pg_stats":[{"pgid":"3.3f","version":"86'8","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.516764+0000","last_change":"2023-11-16T16:37:13.941551+0000","last_active":"2023-11-16T16:38:31.516764+0000","last_peered":"2023-11-16T16:38:31.516764+0000","last_clean":"2023-11-16T16:38:31.516764+0000","last_became_active":"2023-11-16T16:37:13.941190+0000","last_became_peered":"2023-11-16T16:37:13.941190+0000","last_unstale":"2023-11-16T16:38:31.516764+0000","last_undegraded":"2023-11-16T16:38:31.516764+0000","last_fullsized":"2023-11-16T16:38:31.516764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:42:03.561127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.510062+0000","last_change":"2023-11-16T16:37:13.942300+0000","last_active":"2023-11-16T16:38:31.510062+0000","last_peered":"2023-11-16T16:38:31.510062+0000","last_clean":"2023-11-16T16:38:31.510062+0000","last_became_active":"2023-11-16T16:37:13.941966+0000","last_became_peered":"2023-11-16T16:37:13.941966+0000","last_unstale":"2023-11-16T16:38:31.510062+0000","last_undegraded":"2023-11-16T16:38:31.510062+0000","last_fullsized":"2023-11-16T16:38:31.510062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:12:41.529896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.518118+0000","last_change":"2023-11-16T16:37:13.943461+0000","last_active":"2023-11-16T16:38:31.518118+0000","last_peered":"2023-11-16T16:38:31.518118+0000","last_clean":"2023-11-16T16:38:31.518118+0000","last_became_active":"2023-11-16T16:37:13.943165+0000","last_became_peered":"2023-11-16T16:37:13.943165+0000","last_unstale":"2023-11-16T16:38:31.518118+0000","last_undegraded":"2023-11-16T16:38:31.518118+0000","last_fullsized":"2023-11-16T16:38:31.518118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:17:44.567499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"86'8","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.526363+0000","last_change":"2023-11-16T16:37:13.940205+0000","last_active":"2023-11-16T16:38:31.526363+0000","last_peered":"2023-11-16T16:38:31.526363+0000","last_clean":"2023-11-16T16:38:31.526363+0000","last_became_active":"2023-11-16T16:37:13.939819+0000","last_became_peered":"2023-11-16T16:37:13.939819+0000","last_unstale":"2023-11-16T16:38:31.526363+0000","last_undegraded":"2023-11-16T16:38:31.526363+0000","last_fullsized":"2023-11-16T16:38:31.526363+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:52:42.183240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.508231+0000","last_change":"2023-11-16T16:37:13.939819+0000","last_active":"2023-11-16T16:38:31.508231+0000","last_peered":"2023-11-16T16:38:31.508231+0000","last_clean":"2023-11-16T16:38:31.508231+0000","last_became_active":"2023-11-16T16:37:13.939367+0000","last_became_peered":"2023-11-16T16:37:13.939367+0000","last_unstale":"2023-11-16T16:38:31.508231+0000","last_undegraded":"2023-11-16T16:38:31.508231+0000","last_fullsized":"2023-11-16T16:38:31.508231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:17:30.563282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.509876+0000","last_change":"2023-11-16T16:37:13.946613+0000","last_active":"2023-11-16T16:38:31.509876+0000","last_peered":"2023-11-16T16:38:31.509876+0000","last_clean":"2023-11-16T16:38:31.509876+0000","last_became_active":"2023-11-16T16:37:13.946281+0000","last_became_peered":"2023-11-16T16:37:13.946281+0000","last_unstale":"2023-11-16T16:38:31.509876+0000","last_undegraded":"2023-11-16T16:38:31.509876+0000","last_fullsized":"2023-11-16T16:38:31.509876+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:29:42.599960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.528658+0000","last_change":"2023-11-16T16:37:13.936899+0000","last_active":"2023-11-16T16:38:31.528658+0000","last_peered":"2023-11-16T16:38:31.528658+0000","last_clean":"2023-11-16T16:38:31.528658+0000","last_became_active":"2023-11-16T16:37:13.936543+0000","last_became_peered":"2023-11-16T16:37:13.936543+0000","last_unstale":"2023-11-16T16:38:31.528658+0000","last_undegraded":"2023-11-16T16:38:31.528658+0000","last_fullsized":"2023-11-16T16:38:31.528658+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:43:46.820689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"86'9","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.518738+0000","last_change":"2023-11-16T16:38:03.189467+0000","last_active":"2023-11-16T16:38:31.518738+0000","last_peered":"2023-11-16T16:38:31.518738+0000","last_clean":"2023-11-16T16:38:31.518738+0000","last_became_active":"2023-11-16T16:38:03.189017+0000","last_became_peered":"2023-11-16T16:38:03.189017+0000","last_unstale":"2023-11-16T16:38:31.518738+0000","last_undegraded":"2023-11-16T16:38:31.518738+0000","last_fullsized":"2023-11-16T16:38:31.518738+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:47:50.088436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"86'11","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.519665+0000","last_change":"2023-11-16T16:37:13.934105+0000","last_active":"2023-11-16T16:38:31.519665+0000","last_peered":"2023-11-16T16:38:31.519665+0000","last_clean":"2023-11-16T16:38:31.519665+0000","last_became_active":"2023-11-16T16:37:13.933887+0000","last_became_peered":"2023-11-16T16:37:13.933887+0000","last_unstale":"2023-11-16T16:38:31.519665+0000","last_undegraded":"2023-11-16T16:38:31.519665+0000","last_fullsized":"2023-11-16T16:38:31.519665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:53:26.890511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"86'16","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.519996+0000","last_change":"2023-11-16T16:38:03.191855+0000","last_active":"2023-11-16T16:38:31.519996+0000","last_peered":"2023-11-16T16:38:31.519996+0000","last_clean":"2023-11-16T16:38:31.519996+0000","last_became_active":"2023-11-16T16:38:03.191398+0000","last_became_peered":"2023-11-16T16:38:03.191398+0000","last_unstale":"2023-11-16T16:38:31.519996+0000","last_undegraded":"2023-11-16T16:38:31.519996+0000","last_fullsized":"2023-11-16T16:38:31.519996+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:09:44.431927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.529451+0000","last_change":"2023-11-16T16:37:13.941597+0000","last_active":"2023-11-16T16:38:31.529451+0000","last_peered":"2023-11-16T16:38:31.529451+0000","last_clean":"2023-11-16T16:38:31.529451+0000","last_became_active":"2023-11-16T16:37:13.941253+0000","last_became_peered":"2023-11-16T16:37:13.941253+0000","last_unstale":"2023-11-16T16:38:31.529451+0000","last_undegraded":"2023-11-16T16:38:31.529451+0000","last_fullsized":"2023-11-16T16:38:31.529451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:13:35.256952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.509176+0000","last_change":"2023-11-16T16:37:13.941673+0000","last_active":"2023-11-16T16:38:31.509176+0000","last_peered":"2023-11-16T16:38:31.509176+0000","last_clean":"2023-11-16T16:38:31.509176+0000","last_became_active":"2023-11-16T16:37:13.941345+0000","last_became_peered":"2023-11-16T16:37:13.941345+0000","last_unstale":"2023-11-16T16:38:31.509176+0000","last_undegraded":"2023-11-16T16:38:31.509176+0000","last_fullsized":"2023-11-16T16:38:31.509176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:17:51.018141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"86'10","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.510670+0000","last_change":"2023-11-16T16:37:13.947991+0000","last_active":"2023-11-16T16:38:31.510670+0000","last_peered":"2023-11-16T16:38:31.510670+0000","last_clean":"2023-11-16T16:38:31.510670+0000","last_became_active":"2023-11-16T16:37:13.947454+0000","last_became_peered":"2023-11-16T16:37:13.947454+0000","last_unstale":"2023-11-16T16:38:31.510670+0000","last_undegraded":"2023-11-16T16:38:31.510670+0000","last_fullsized":"2023-11-16T16:38:31.510670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:19:19.825458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.510762+0000","last_change":"2023-11-16T16:37:13.941502+0000","last_active":"2023-11-16T16:38:31.510762+0000","last_peered":"2023-11-16T16:38:31.510762+0000","last_clean":"2023-11-16T16:38:31.510762+0000","last_became_active":"2023-11-16T16:37:13.941191+0000","last_became_peered":"2023-11-16T16:37:13.941191+0000","last_unstale":"2023-11-16T16:38:31.510762+0000","last_undegraded":"2023-11-16T16:38:31.510762+0000","last_fullsized":"2023-11-16T16:38:31.510762+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:19:35.239602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.517384+0000","last_change":"2023-11-16T16:37:13.949198+0000","last_active":"2023-11-16T16:38:31.517384+0000","last_peered":"2023-11-16T16:38:31.517384+0000","last_clean":"2023-11-16T16:38:31.517384+0000","last_became_active":"2023-11-16T16:37:13.947147+0000","last_became_peered":"2023-11-16T16:37:13.947147+0000","last_unstale":"2023-11-16T16:38:31.517384+0000","last_undegraded":"2023-11-16T16:38:31.517384+0000","last_fullsized":"2023-11-16T16:38:31.517384+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:55:04.616796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.526473+0000","last_change":"2023-11-16T16:37:13.943915+0000","last_active":"2023-11-16T16:38:31.526473+0000","last_peered":"2023-11-16T16:38:31.526473+0000","last_clean":"2023-11-16T16:38:31.526473+0000","last_became_active":"2023-11-16T16:37:13.942995+0000","last_became_peered":"2023-11-16T16:37:13.942995+0000","last_unstale":"2023-11-16T16:38:31.526473+0000","last_undegraded":"2023-11-16T16:38:31.526473+0000","last_fullsized":"2023-11-16T16:38:31.526473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:23:05.411045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"86'8","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.516988+0000","last_change":"2023-11-16T16:37:13.941816+0000","last_active":"2023-11-16T16:38:31.516988+0000","last_peered":"2023-11-16T16:38:31.516988+0000","last_clean":"2023-11-16T16:38:31.516988+0000","last_became_active":"2023-11-16T16:37:13.941541+0000","last_became_peered":"2023-11-16T16:37:13.941541+0000","last_unstale":"2023-11-16T16:38:31.516988+0000","last_undegraded":"2023-11-16T16:38:31.516988+0000","last_fullsized":"2023-11-16T16:38:31.516988+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:53:11.142308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"86'10","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.518736+0000","last_change":"2023-11-16T16:37:13.940639+0000","last_active":"2023-11-16T16:38:31.518736+0000","last_peered":"2023-11-16T16:38:31.518736+0000","last_clean":"2023-11-16T16:38:31.518736+0000","last_became_active":"2023-11-16T16:37:13.940323+0000","last_became_peered":"2023-11-16T16:37:13.940323+0000","last_unstale":"2023-11-16T16:38:31.518736+0000","last_undegraded":"2023-11-16T16:38:31.518736+0000","last_fullsized":"2023-11-16T16:38:31.518736+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:32:03.882320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.513674+0000","last_change":"2023-11-16T16:37:13.934802+0000","last_active":"2023-11-16T16:38:31.513674+0000","last_peered":"2023-11-16T16:38:31.513674+0000","last_clean":"2023-11-16T16:38:31.513674+0000","last_became_active":"2023-11-16T16:37:13.934496+0000","last_became_peered":"2023-11-16T16:37:13.934496+0000","last_unstale":"2023-11-16T16:38:31.513674+0000","last_undegraded":"2023-11-16T16:38:31.513674+0000","last_fullsized":"2023-11-16T16:38:31.513674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:15:58.249424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185585+0000","last_change":"2023-11-16T16:37:13.499284+0000","last_active":"2023-11-16T16:38:03.185585+0000","last_peered":"2023-11-16T16:38:03.185585+0000","last_clean":"2023-11-16T16:38:03.185585+0000","last_became_active":"2023-11-16T16:37:13.498931+0000","last_became_peered":"2023-11-16T16:37:13.498931+0000","last_unstale":"2023-11-16T16:38:03.185585+0000","last_undegraded":"2023-11-16T16:38:03.185585+0000","last_fullsized":"2023-11-16T16:38:03.185585+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:53:33.061352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.514671+0000","last_change":"2023-11-16T16:37:13.939631+0000","last_active":"2023-11-16T16:38:31.514671+0000","last_peered":"2023-11-16T16:38:31.514671+0000","last_clean":"2023-11-16T16:38:31.514671+0000","last_became_active":"2023-11-16T16:37:13.939318+0000","last_became_peered":"2023-11-16T16:37:13.939318+0000","last_unstale":"2023-11-16T16:38:31.514671+0000","last_undegraded":"2023-11-16T16:38:31.514671+0000","last_fullsized":"2023-11-16T16:38:31.514671+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:11:33.421798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.183008+0000","last_change":"2023-11-16T16:37:13.495972+0000","last_active":"2023-11-16T16:38:03.183008+0000","last_peered":"2023-11-16T16:38:03.183008+0000","last_clean":"2023-11-16T16:38:03.183008+0000","last_became_active":"2023-11-16T16:37:13.495610+0000","last_became_peered":"2023-11-16T16:37:13.495610+0000","last_unstale":"2023-11-16T16:38:03.183008+0000","last_undegraded":"2023-11-16T16:38:03.183008+0000","last_fullsized":"2023-11-16T16:38:03.183008+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:47:20.115622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.518372+0000","last_change":"2023-11-16T16:37:13.935236+0000","last_active":"2023-11-16T16:38:31.518372+0000","last_peered":"2023-11-16T16:38:31.518372+0000","last_clean":"2023-11-16T16:38:31.518372+0000","last_became_active":"2023-11-16T16:37:13.934920+0000","last_became_peered":"2023-11-16T16:37:13.934920+0000","last_unstale":"2023-11-16T16:38:31.518372+0000","last_undegraded":"2023-11-16T16:38:31.518372+0000","last_fullsized":"2023-11-16T16:38:31.518372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:04:59.207169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"84'4","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.197657+0000","last_change":"2023-11-16T16:38:03.197657+0000","last_active":"2023-11-16T16:38:03.197657+0000","last_peered":"2023-11-16T16:38:03.197657+0000","last_clean":"2023-11-16T16:38:03.197657+0000","last_became_active":"2023-11-16T16:38:03.190952+0000","last_became_peered":"2023-11-16T16:38:03.190952+0000","last_unstale":"2023-11-16T16:38:03.197657+0000","last_undegraded":"2023-11-16T16:38:03.197657+0000","last_fullsized":"2023-11-16T16:38:03.197657+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:04:50.256412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.524482+0000","last_change":"2023-11-16T16:37:13.940838+0000","last_active":"2023-11-16T16:38:31.524482+0000","last_peered":"2023-11-16T16:38:31.524482+0000","last_clean":"2023-11-16T16:38:31.524482+0000","last_became_active":"2023-11-16T16:37:13.940538+0000","last_became_peered":"2023-11-16T16:37:13.940538+0000","last_unstale":"2023-11-16T16:38:31.524482+0000","last_undegraded":"2023-11-16T16:38:31.524482+0000","last_fullsized":"2023-11-16T16:38:31.524482+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:47:36.031628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.190408+0000","last_change":"2023-11-16T16:37:13.495114+0000","last_active":"2023-11-16T16:38:03.190408+0000","last_peered":"2023-11-16T16:38:03.190408+0000","last_clean":"2023-11-16T16:38:03.190408+0000","last_became_active":"2023-11-16T16:37:13.494807+0000","last_became_peered":"2023-11-16T16:37:13.494807+0000","last_unstale":"2023-11-16T16:38:03.190408+0000","last_undegraded":"2023-11-16T16:38:03.190408+0000","last_fullsized":"2023-11-16T16:38:03.190408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:35:31.040933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"86'15","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.519316+0000","last_change":"2023-11-16T16:37:13.943652+0000","last_active":"2023-11-16T16:38:31.519316+0000","last_peered":"2023-11-16T16:38:31.519316+0000","last_clean":"2023-11-16T16:38:31.519316+0000","last_became_active":"2023-11-16T16:37:13.943298+0000","last_became_peered":"2023-11-16T16:37:13.943298+0000","last_unstale":"2023-11-16T16:38:31.519316+0000","last_undegraded":"2023-11-16T16:38:31.519316+0000","last_fullsized":"2023-11-16T16:38:31.519316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:17:55.154731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'8","reported_seq":128,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:54.451248+0000","last_change":"2023-11-16T16:37:13.509801+0000","last_active":"2023-11-16T16:38:54.451248+0000","last_peered":"2023-11-16T16:38:54.451248+0000","last_clean":"2023-11-16T16:38:54.451248+0000","last_became_active":"2023-11-16T16:37:13.509389+0000","last_became_peered":"2023-11-16T16:37:13.509389+0000","last_unstale":"2023-11-16T16:38:54.451248+0000","last_undegraded":"2023-11-16T16:38:54.451248+0000","last_fullsized":"2023-11-16T16:38:54.451248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:46:55.751281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1423,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.512699+0000","last_change":"2023-11-16T16:37:13.933871+0000","last_active":"2023-11-16T16:38:31.512699+0000","last_peered":"2023-11-16T16:38:31.512699+0000","last_clean":"2023-11-16T16:38:31.512699+0000","last_became_active":"2023-11-16T16:37:13.933549+0000","last_became_peered":"2023-11-16T16:37:13.933549+0000","last_unstale":"2023-11-16T16:38:31.512699+0000","last_undegraded":"2023-11-16T16:38:31.512699+0000","last_fullsized":"2023-11-16T16:38:31.512699+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:35:18.854524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"84'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.187212+0000","last_change":"2023-11-16T16:37:13.501339+0000","last_active":"2023-11-16T16:38:03.187212+0000","last_peered":"2023-11-16T16:38:03.187212+0000","last_clean":"2023-11-16T16:38:03.187212+0000","last_became_active":"2023-11-16T16:37:13.500949+0000","last_became_peered":"2023-11-16T16:37:13.500949+0000","last_unstale":"2023-11-16T16:38:03.187212+0000","last_undegraded":"2023-11-16T16:38:03.187212+0000","last_fullsized":"2023-11-16T16:38:03.187212+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:56:34.503681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"86'8","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.514559+0000","last_change":"2023-11-16T16:38:03.192595+0000","last_active":"2023-11-16T16:38:31.514559+0000","last_peered":"2023-11-16T16:38:31.514559+0000","last_clean":"2023-11-16T16:38:31.514559+0000","last_became_active":"2023-11-16T16:38:03.192208+0000","last_became_peered":"2023-11-16T16:38:03.192208+0000","last_unstale":"2023-11-16T16:38:31.514559+0000","last_undegraded":"2023-11-16T16:38:31.514559+0000","last_fullsized":"2023-11-16T16:38:31.514559+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:30:24.627049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"84'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184789+0000","last_change":"2023-11-16T16:37:13.505481+0000","last_active":"2023-11-16T16:38:03.184789+0000","last_peered":"2023-11-16T16:38:03.184789+0000","last_clean":"2023-11-16T16:38:03.184789+0000","last_became_active":"2023-11-16T16:37:13.505085+0000","last_became_peered":"2023-11-16T16:37:13.505085+0000","last_unstale":"2023-11-16T16:38:03.184789+0000","last_undegraded":"2023-11-16T16:38:03.184789+0000","last_fullsized":"2023-11-16T16:38:03.184789+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:21:13.673154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"86'8","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.509656+0000","last_change":"2023-11-16T16:37:13.943985+0000","last_active":"2023-11-16T16:38:31.509656+0000","last_peered":"2023-11-16T16:38:31.509656+0000","last_clean":"2023-11-16T16:38:31.509656+0000","last_became_active":"2023-11-16T16:37:13.943628+0000","last_became_peered":"2023-11-16T16:37:13.943628+0000","last_unstale":"2023-11-16T16:38:31.509656+0000","last_undegraded":"2023-11-16T16:38:31.509656+0000","last_fullsized":"2023-11-16T16:38:31.509656+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:01:16.596531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184704+0000","last_change":"2023-11-16T16:37:13.498648+0000","last_active":"2023-11-16T16:38:03.184704+0000","last_peered":"2023-11-16T16:38:03.184704+0000","last_clean":"2023-11-16T16:38:03.184704+0000","last_became_active":"2023-11-16T16:37:13.497390+0000","last_became_peered":"2023-11-16T16:37:13.497390+0000","last_unstale":"2023-11-16T16:38:03.184704+0000","last_undegraded":"2023-11-16T16:38:03.184704+0000","last_fullsized":"2023-11-16T16:38:03.184704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:20:50.541938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"86'11","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.518643+0000","last_change":"2023-11-16T16:37:13.942337+0000","last_active":"2023-11-16T16:38:31.518643+0000","last_peered":"2023-11-16T16:38:31.518643+0000","last_clean":"2023-11-16T16:38:31.518643+0000","last_became_active":"2023-11-16T16:37:13.941982+0000","last_became_peered":"2023-11-16T16:37:13.941982+0000","last_unstale":"2023-11-16T16:38:31.518643+0000","last_undegraded":"2023-11-16T16:38:31.518643+0000","last_fullsized":"2023-11-16T16:38:31.518643+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:09:19.949618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'5","reported_seq":120,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:54.450706+0000","last_change":"2023-11-16T16:37:13.503690+0000","last_active":"2023-11-16T16:38:54.450706+0000","last_peered":"2023-11-16T16:38:54.450706+0000","last_clean":"2023-11-16T16:38:54.450706+0000","last_became_active":"2023-11-16T16:37:13.503203+0000","last_became_peered":"2023-11-16T16:37:13.503203+0000","last_unstale":"2023-11-16T16:38:54.450706+0000","last_undegraded":"2023-11-16T16:38:54.450706+0000","last_fullsized":"2023-11-16T16:38:54.450706+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:47:38.415047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"86'10","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.519722+0000","last_change":"2023-11-16T16:37:13.943975+0000","last_active":"2023-11-16T16:38:31.519722+0000","last_peered":"2023-11-16T16:38:31.519722+0000","last_clean":"2023-11-16T16:38:31.519722+0000","last_became_active":"2023-11-16T16:37:13.943611+0000","last_became_peered":"2023-11-16T16:37:13.943611+0000","last_unstale":"2023-11-16T16:38:31.519722+0000","last_undegraded":"2023-11-16T16:38:31.519722+0000","last_fullsized":"2023-11-16T16:38:31.519722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:37:02.660396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'2","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.344191+0000","last_change":"2023-11-16T16:38:03.344191+0000","last_active":"2023-11-16T16:38:03.344191+0000","last_peered":"2023-11-16T16:38:03.344191+0000","last_clean":"2023-11-16T16:38:03.344191+0000","last_became_active":"2023-11-16T16:38:03.189976+0000","last_became_peered":"2023-11-16T16:38:03.189976+0000","last_unstale":"2023-11-16T16:38:03.344191+0000","last_undegraded":"2023-11-16T16:38:03.344191+0000","last_fullsized":"2023-11-16T16:38:03.344191+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:04:56.889493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.513627+0000","last_change":"2023-11-16T16:37:13.949031+0000","last_active":"2023-11-16T16:38:31.513627+0000","last_peered":"2023-11-16T16:38:31.513627+0000","last_clean":"2023-11-16T16:38:31.513627+0000","last_became_active":"2023-11-16T16:37:13.946923+0000","last_became_peered":"2023-11-16T16:37:13.946923+0000","last_unstale":"2023-11-16T16:38:31.513627+0000","last_undegraded":"2023-11-16T16:38:31.513627+0000","last_fullsized":"2023-11-16T16:38:31.513627+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:04:43.521722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185398+0000","last_change":"2023-11-16T16:37:13.495003+0000","last_active":"2023-11-16T16:38:03.185398+0000","last_peered":"2023-11-16T16:38:03.185398+0000","last_clean":"2023-11-16T16:38:03.185398+0000","last_became_active":"2023-11-16T16:37:13.494563+0000","last_became_peered":"2023-11-16T16:37:13.494563+0000","last_unstale":"2023-11-16T16:38:03.185398+0000","last_undegraded":"2023-11-16T16:38:03.185398+0000","last_fullsized":"2023-11-16T16:38:03.185398+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:36:36.403721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"86'11","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.529044+0000","last_change":"2023-11-16T16:37:13.943736+0000","last_active":"2023-11-16T16:38:31.529044+0000","last_peered":"2023-11-16T16:38:31.529044+0000","last_clean":"2023-11-16T16:38:31.529044+0000","last_became_active":"2023-11-16T16:37:13.943391+0000","last_became_peered":"2023-11-16T16:37:13.943391+0000","last_unstale":"2023-11-16T16:38:31.529044+0000","last_undegraded":"2023-11-16T16:38:31.529044+0000","last_fullsized":"2023-11-16T16:38:31.529044+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:27:01.809877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'34","reported_seq":257,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:55.463894+0000","last_change":"2023-11-16T16:37:13.506508+0000","last_active":"2023-11-16T16:38:55.463894+0000","last_peered":"2023-11-16T16:38:55.463894+0000","last_clean":"2023-11-16T16:38:55.463894+0000","last_became_active":"2023-11-16T16:37:13.506045+0000","last_became_peered":"2023-11-16T16:37:13.506045+0000","last_unstale":"2023-11-16T16:38:55.463894+0000","last_undegraded":"2023-11-16T16:38:55.463894+0000","last_fullsized":"2023-11-16T16:38:55.463894+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:57:44.753422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64473,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":205,"num_read_kb":165,"num_write":35,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.525885+0000","last_change":"2023-11-16T16:37:13.939717+0000","last_active":"2023-11-16T16:38:31.525885+0000","last_peered":"2023-11-16T16:38:31.525885+0000","last_clean":"2023-11-16T16:38:31.525885+0000","last_became_active":"2023-11-16T16:37:13.939369+0000","last_became_peered":"2023-11-16T16:37:13.939369+0000","last_unstale":"2023-11-16T16:38:31.525885+0000","last_undegraded":"2023-11-16T16:38:31.525885+0000","last_fullsized":"2023-11-16T16:38:31.525885+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:10:31.454459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.188492+0000","last_change":"2023-11-16T16:37:13.506056+0000","last_active":"2023-11-16T16:38:03.188492+0000","last_peered":"2023-11-16T16:38:03.188492+0000","last_clean":"2023-11-16T16:38:03.188492+0000","last_became_active":"2023-11-16T16:37:13.505707+0000","last_became_peered":"2023-11-16T16:37:13.505707+0000","last_unstale":"2023-11-16T16:38:03.188492+0000","last_undegraded":"2023-11-16T16:38:03.188492+0000","last_fullsized":"2023-11-16T16:38:03.188492+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:29:47.242470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.525703+0000","last_change":"2023-11-16T16:37:13.947764+0000","last_active":"2023-11-16T16:38:31.525703+0000","last_peered":"2023-11-16T16:38:31.525703+0000","last_clean":"2023-11-16T16:38:31.525703+0000","last_became_active":"2023-11-16T16:37:13.947510+0000","last_became_peered":"2023-11-16T16:37:13.947510+0000","last_unstale":"2023-11-16T16:38:31.525703+0000","last_undegraded":"2023-11-16T16:38:31.525703+0000","last_fullsized":"2023-11-16T16:38:31.525703+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:41:59.672529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184939+0000","last_change":"2023-11-16T16:37:13.498178+0000","last_active":"2023-11-16T16:38:03.184939+0000","last_peered":"2023-11-16T16:38:03.184939+0000","last_clean":"2023-11-16T16:38:03.184939+0000","last_became_active":"2023-11-16T16:37:13.497312+0000","last_became_peered":"2023-11-16T16:37:13.497312+0000","last_unstale":"2023-11-16T16:38:03.184939+0000","last_undegraded":"2023-11-16T16:38:03.184939+0000","last_fullsized":"2023-11-16T16:38:03.184939+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:54:40.375590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"86'15","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.529728+0000","last_change":"2023-11-16T16:37:13.947854+0000","last_active":"2023-11-16T16:38:31.529728+0000","last_peered":"2023-11-16T16:38:31.529728+0000","last_clean":"2023-11-16T16:38:31.529728+0000","last_became_active":"2023-11-16T16:37:13.947530+0000","last_became_peered":"2023-11-16T16:37:13.947530+0000","last_unstale":"2023-11-16T16:38:31.529728+0000","last_undegraded":"2023-11-16T16:38:31.529728+0000","last_fullsized":"2023-11-16T16:38:31.529728+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:41:00.205641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'3","reported_seq":118,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:52.446665+0000","last_change":"2023-11-16T16:37:13.500127+0000","last_active":"2023-11-16T16:38:52.446665+0000","last_peered":"2023-11-16T16:38:52.446665+0000","last_clean":"2023-11-16T16:38:52.446665+0000","last_became_active":"2023-11-16T16:37:13.499758+0000","last_became_peered":"2023-11-16T16:37:13.499758+0000","last_unstale":"2023-11-16T16:38:52.446665+0000","last_undegraded":"2023-11-16T16:38:52.446665+0000","last_fullsized":"2023-11-16T16:38:52.446665+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:39:00.268144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.528126+0000","last_change":"2023-11-16T16:37:13.940962+0000","last_active":"2023-11-16T16:38:31.528126+0000","last_peered":"2023-11-16T16:38:31.528126+0000","last_clean":"2023-11-16T16:38:31.528126+0000","last_became_active":"2023-11-16T16:37:13.940450+0000","last_became_peered":"2023-11-16T16:37:13.940450+0000","last_unstale":"2023-11-16T16:38:31.528126+0000","last_undegraded":"2023-11-16T16:38:31.528126+0000","last_fullsized":"2023-11-16T16:38:31.528126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:28:45.946454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'15","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.228174+0000","last_change":"2023-11-16T16:38:03.228174+0000","last_active":"2023-11-16T16:38:03.228174+0000","last_peered":"2023-11-16T16:38:03.228174+0000","last_clean":"2023-11-16T16:38:03.228174+0000","last_became_active":"2023-11-16T16:38:03.192020+0000","last_became_peered":"2023-11-16T16:38:03.192020+0000","last_unstale":"2023-11-16T16:38:03.228174+0000","last_undegraded":"2023-11-16T16:38:03.228174+0000","last_fullsized":"2023-11-16T16:38:03.228174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:54:01.231005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.518623+0000","last_change":"2023-11-16T16:37:13.944878+0000","last_active":"2023-11-16T16:38:31.518623+0000","last_peered":"2023-11-16T16:38:31.518623+0000","last_clean":"2023-11-16T16:38:31.518623+0000","last_became_active":"2023-11-16T16:37:13.944580+0000","last_became_peered":"2023-11-16T16:37:13.944580+0000","last_unstale":"2023-11-16T16:38:31.518623+0000","last_undegraded":"2023-11-16T16:38:31.518623+0000","last_fullsized":"2023-11-16T16:38:31.518623+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:00:01.106243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.190628+0000","last_change":"2023-11-16T16:37:13.499139+0000","last_active":"2023-11-16T16:38:03.190628+0000","last_peered":"2023-11-16T16:38:03.190628+0000","last_clean":"2023-11-16T16:38:03.190628+0000","last_became_active":"2023-11-16T16:37:13.498755+0000","last_became_peered":"2023-11-16T16:37:13.498755+0000","last_unstale":"2023-11-16T16:38:03.190628+0000","last_undegraded":"2023-11-16T16:38:03.190628+0000","last_fullsized":"2023-11-16T16:38:03.190628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T16:55:06.065509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.517260+0000","last_change":"2023-11-16T16:37:13.946187+0000","last_active":"2023-11-16T16:38:31.517260+0000","last_peered":"2023-11-16T16:38:31.517260+0000","last_clean":"2023-11-16T16:38:31.517260+0000","last_became_active":"2023-11-16T16:37:13.945887+0000","last_became_peered":"2023-11-16T16:37:13.945887+0000","last_unstale":"2023-11-16T16:38:31.517260+0000","last_undegraded":"2023-11-16T16:38:31.517260+0000","last_fullsized":"2023-11-16T16:38:31.517260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:41:31.726749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185150+0000","last_change":"2023-11-16T16:37:13.507150+0000","last_active":"2023-11-16T16:38:03.185150+0000","last_peered":"2023-11-16T16:38:03.185150+0000","last_clean":"2023-11-16T16:38:03.185150+0000","last_became_active":"2023-11-16T16:37:13.506814+0000","last_became_peered":"2023-11-16T16:37:13.506814+0000","last_unstale":"2023-11-16T16:38:03.185150+0000","last_undegraded":"2023-11-16T16:38:03.185150+0000","last_fullsized":"2023-11-16T16:38:03.185150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:54:45.031046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.506758+0000","last_change":"2023-11-16T16:37:13.947866+0000","last_active":"2023-11-16T16:38:31.506758+0000","last_peered":"2023-11-16T16:38:31.506758+0000","last_clean":"2023-11-16T16:38:31.506758+0000","last_became_active":"2023-11-16T16:37:13.947544+0000","last_became_peered":"2023-11-16T16:37:13.947544+0000","last_unstale":"2023-11-16T16:38:31.506758+0000","last_undegraded":"2023-11-16T16:38:31.506758+0000","last_fullsized":"2023-11-16T16:38:31.506758+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:23:27.220130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"84'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184212+0000","last_change":"2023-11-16T16:37:13.497800+0000","last_active":"2023-11-16T16:38:03.184212+0000","last_peered":"2023-11-16T16:38:03.184212+0000","last_clean":"2023-11-16T16:38:03.184212+0000","last_became_active":"2023-11-16T16:37:13.497414+0000","last_became_peered":"2023-11-16T16:37:13.497414+0000","last_unstale":"2023-11-16T16:38:03.184212+0000","last_undegraded":"2023-11-16T16:38:03.184212+0000","last_fullsized":"2023-11-16T16:38:03.184212+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:17:42.179250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.509186+0000","last_change":"2023-11-16T16:37:13.946780+0000","last_active":"2023-11-16T16:38:31.509186+0000","last_peered":"2023-11-16T16:38:31.509186+0000","last_clean":"2023-11-16T16:38:31.509186+0000","last_became_active":"2023-11-16T16:37:13.946441+0000","last_became_peered":"2023-11-16T16:37:13.946441+0000","last_unstale":"2023-11-16T16:38:31.509186+0000","last_undegraded":"2023-11-16T16:38:31.509186+0000","last_fullsized":"2023-11-16T16:38:31.509186+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:46:10.856707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"86'8","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.519819+0000","last_change":"2023-11-16T16:38:03.191887+0000","last_active":"2023-11-16T16:38:31.519819+0000","last_peered":"2023-11-16T16:38:31.519819+0000","last_clean":"2023-11-16T16:38:31.519819+0000","last_became_active":"2023-11-16T16:38:03.191402+0000","last_became_peered":"2023-11-16T16:38:03.191402+0000","last_unstale":"2023-11-16T16:38:31.519819+0000","last_undegraded":"2023-11-16T16:38:31.519819+0000","last_fullsized":"2023-11-16T16:38:31.519819+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:47:26.769740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"84'4","reported_seq":117,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:52.446018+0000","last_change":"2023-11-16T16:37:13.498129+0000","last_active":"2023-11-16T16:38:52.446018+0000","last_peered":"2023-11-16T16:38:52.446018+0000","last_clean":"2023-11-16T16:38:52.446018+0000","last_became_active":"2023-11-16T16:37:13.497765+0000","last_became_peered":"2023-11-16T16:37:13.497765+0000","last_unstale":"2023-11-16T16:38:52.446018+0000","last_undegraded":"2023-11-16T16:38:52.446018+0000","last_fullsized":"2023-11-16T16:38:52.446018+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:15:51.118817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":95,"num_read_kb":95,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"86'8","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.513701+0000","last_change":"2023-11-16T16:37:13.938417+0000","last_active":"2023-11-16T16:38:31.513701+0000","last_peered":"2023-11-16T16:38:31.513701+0000","last_clean":"2023-11-16T16:38:31.513701+0000","last_became_active":"2023-11-16T16:37:13.938114+0000","last_became_peered":"2023-11-16T16:37:13.938114+0000","last_unstale":"2023-11-16T16:38:31.513701+0000","last_undegraded":"2023-11-16T16:38:31.513701+0000","last_fullsized":"2023-11-16T16:38:31.513701+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:07:00.909431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"84'3","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.183990+0000","last_change":"2023-11-16T16:37:13.499045+0000","last_active":"2023-11-16T16:38:03.183990+0000","last_peered":"2023-11-16T16:38:03.183990+0000","last_clean":"2023-11-16T16:38:03.183990+0000","last_became_active":"2023-11-16T16:37:13.498598+0000","last_became_peered":"2023-11-16T16:37:13.498598+0000","last_unstale":"2023-11-16T16:38:03.183990+0000","last_undegraded":"2023-11-16T16:38:03.183990+0000","last_fullsized":"2023-11-16T16:38:03.183990+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:29:46.898930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":349,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.183816+0000","last_change":"2023-11-16T16:36:14.090213+0000","last_active":"2023-11-16T16:38:03.183816+0000","last_peered":"2023-11-16T16:38:03.183816+0000","last_clean":"2023-11-16T16:38:03.183816+0000","last_became_active":"2023-11-16T16:36:14.089742+0000","last_became_peered":"2023-11-16T16:36:14.089742+0000","last_unstale":"2023-11-16T16:38:03.183816+0000","last_undegraded":"2023-11-16T16:38:03.183816+0000","last_fullsized":"2023-11-16T16:38:03.183816+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:33:16.178716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:33:16.178716+0000","last_clean_scrub_stamp":"2023-11-16T16:33:16.178716+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:34:52.910521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.511992+0000","last_change":"2023-11-16T16:37:13.945380+0000","last_active":"2023-11-16T16:38:31.511992+0000","last_peered":"2023-11-16T16:38:31.511992+0000","last_clean":"2023-11-16T16:38:31.511992+0000","last_became_active":"2023-11-16T16:37:13.945038+0000","last_became_peered":"2023-11-16T16:37:13.945038+0000","last_unstale":"2023-11-16T16:38:31.511992+0000","last_undegraded":"2023-11-16T16:38:31.511992+0000","last_fullsized":"2023-11-16T16:38:31.511992+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:29:11.812243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"84'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184055+0000","last_change":"2023-11-16T16:37:13.497586+0000","last_active":"2023-11-16T16:38:03.184055+0000","last_peered":"2023-11-16T16:38:03.184055+0000","last_clean":"2023-11-16T16:38:03.184055+0000","last_became_active":"2023-11-16T16:37:13.497233+0000","last_became_peered":"2023-11-16T16:37:13.497233+0000","last_unstale":"2023-11-16T16:38:03.184055+0000","last_undegraded":"2023-11-16T16:38:03.184055+0000","last_fullsized":"2023-11-16T16:38:03.184055+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:47:21.097628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"84'1","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.191390+0000","last_change":"2023-11-16T16:37:13.500621+0000","last_active":"2023-11-16T16:38:03.191390+0000","last_peered":"2023-11-16T16:38:03.191390+0000","last_clean":"2023-11-16T16:38:03.191390+0000","last_became_active":"2023-11-16T16:37:13.500265+0000","last_became_peered":"2023-11-16T16:37:13.500265+0000","last_unstale":"2023-11-16T16:38:03.191390+0000","last_undegraded":"2023-11-16T16:38:03.191390+0000","last_fullsized":"2023-11-16T16:38:03.191390+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:22:59.748224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.509709+0000","last_change":"2023-11-16T16:37:13.930992+0000","last_active":"2023-11-16T16:38:31.509709+0000","last_peered":"2023-11-16T16:38:31.509709+0000","last_clean":"2023-11-16T16:38:31.509709+0000","last_became_active":"2023-11-16T16:37:13.930641+0000","last_became_peered":"2023-11-16T16:37:13.930641+0000","last_unstale":"2023-11-16T16:38:31.509709+0000","last_undegraded":"2023-11-16T16:38:31.509709+0000","last_fullsized":"2023-11-16T16:38:31.509709+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:24:49.314642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'6","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185439+0000","last_change":"2023-11-16T16:37:13.498637+0000","last_active":"2023-11-16T16:38:03.185439+0000","last_peered":"2023-11-16T16:38:03.185439+0000","last_clean":"2023-11-16T16:38:03.185439+0000","last_became_active":"2023-11-16T16:37:13.498341+0000","last_became_peered":"2023-11-16T16:37:13.498341+0000","last_unstale":"2023-11-16T16:38:03.185439+0000","last_undegraded":"2023-11-16T16:38:03.185439+0000","last_fullsized":"2023-11-16T16:38:03.185439+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:30:04.558815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.526478+0000","last_change":"2023-11-16T16:37:13.944142+0000","last_active":"2023-11-16T16:38:31.526478+0000","last_peered":"2023-11-16T16:38:31.526478+0000","last_clean":"2023-11-16T16:38:31.526478+0000","last_became_active":"2023-11-16T16:37:13.943758+0000","last_became_peered":"2023-11-16T16:37:13.943758+0000","last_unstale":"2023-11-16T16:38:31.526478+0000","last_undegraded":"2023-11-16T16:38:31.526478+0000","last_fullsized":"2023-11-16T16:38:31.526478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:57:58.601449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184558+0000","last_change":"2023-11-16T16:37:13.496900+0000","last_active":"2023-11-16T16:38:03.184558+0000","last_peered":"2023-11-16T16:38:03.184558+0000","last_clean":"2023-11-16T16:38:03.184558+0000","last_became_active":"2023-11-16T16:37:13.496528+0000","last_became_peered":"2023-11-16T16:37:13.496528+0000","last_unstale":"2023-11-16T16:38:03.184558+0000","last_undegraded":"2023-11-16T16:38:03.184558+0000","last_fullsized":"2023-11-16T16:38:03.184558+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:56:42.304030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.507960+0000","last_change":"2023-11-16T16:37:13.943104+0000","last_active":"2023-11-16T16:38:31.507960+0000","last_peered":"2023-11-16T16:38:31.507960+0000","last_clean":"2023-11-16T16:38:31.507960+0000","last_became_active":"2023-11-16T16:37:13.942774+0000","last_became_peered":"2023-11-16T16:37:13.942774+0000","last_unstale":"2023-11-16T16:38:31.507960+0000","last_undegraded":"2023-11-16T16:38:31.507960+0000","last_fullsized":"2023-11-16T16:38:31.507960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:24:11.162929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.188270+0000","last_change":"2023-11-16T16:37:13.505994+0000","last_active":"2023-11-16T16:38:03.188270+0000","last_peered":"2023-11-16T16:38:03.188270+0000","last_clean":"2023-11-16T16:38:03.188270+0000","last_became_active":"2023-11-16T16:37:13.505663+0000","last_became_peered":"2023-11-16T16:37:13.505663+0000","last_unstale":"2023-11-16T16:38:03.188270+0000","last_undegraded":"2023-11-16T16:38:03.188270+0000","last_fullsized":"2023-11-16T16:38:03.188270+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:06:13.680491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.479124+0000","last_change":"2023-11-16T16:37:13.941970+0000","last_active":"2023-11-16T16:38:31.479124+0000","last_peered":"2023-11-16T16:38:31.479124+0000","last_clean":"2023-11-16T16:38:31.479124+0000","last_became_active":"2023-11-16T16:37:13.941649+0000","last_became_peered":"2023-11-16T16:37:13.941649+0000","last_unstale":"2023-11-16T16:38:31.479124+0000","last_undegraded":"2023-11-16T16:38:31.479124+0000","last_fullsized":"2023-11-16T16:38:31.479124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:23:17.814240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.182332+0000","last_change":"2023-11-16T16:37:13.503429+0000","last_active":"2023-11-16T16:38:03.182332+0000","last_peered":"2023-11-16T16:38:03.182332+0000","last_clean":"2023-11-16T16:38:03.182332+0000","last_became_active":"2023-11-16T16:37:13.503114+0000","last_became_peered":"2023-11-16T16:37:13.503114+0000","last_unstale":"2023-11-16T16:38:03.182332+0000","last_undegraded":"2023-11-16T16:38:03.182332+0000","last_fullsized":"2023-11-16T16:38:03.182332+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:57:19.910282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.513996+0000","last_change":"2023-11-16T16:37:13.941225+0000","last_active":"2023-11-16T16:38:31.513996+0000","last_peered":"2023-11-16T16:38:31.513996+0000","last_clean":"2023-11-16T16:38:31.513996+0000","last_became_active":"2023-11-16T16:37:13.940964+0000","last_became_peered":"2023-11-16T16:37:13.940964+0000","last_unstale":"2023-11-16T16:38:31.513996+0000","last_undegraded":"2023-11-16T16:38:31.513996+0000","last_fullsized":"2023-11-16T16:38:31.513996+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:21:58.992812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.188151+0000","last_change":"2023-11-16T16:37:13.510154+0000","last_active":"2023-11-16T16:38:03.188151+0000","last_peered":"2023-11-16T16:38:03.188151+0000","last_clean":"2023-11-16T16:38:03.188151+0000","last_became_active":"2023-11-16T16:37:13.509790+0000","last_became_peered":"2023-11-16T16:37:13.509790+0000","last_unstale":"2023-11-16T16:38:03.188151+0000","last_undegraded":"2023-11-16T16:38:03.188151+0000","last_fullsized":"2023-11-16T16:38:03.188151+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:09:32.999016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"86'8","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.509433+0000","last_change":"2023-11-16T16:37:13.942203+0000","last_active":"2023-11-16T16:38:31.509433+0000","last_peered":"2023-11-16T16:38:31.509433+0000","last_clean":"2023-11-16T16:38:31.509433+0000","last_became_active":"2023-11-16T16:37:13.941766+0000","last_became_peered":"2023-11-16T16:37:13.941766+0000","last_unstale":"2023-11-16T16:38:31.509433+0000","last_undegraded":"2023-11-16T16:38:31.509433+0000","last_fullsized":"2023-11-16T16:38:31.509433+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:50:04.530728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.190859+0000","last_change":"2023-11-16T16:37:13.509854+0000","last_active":"2023-11-16T16:38:03.190859+0000","last_peered":"2023-11-16T16:38:03.190859+0000","last_clean":"2023-11-16T16:38:03.190859+0000","last_became_active":"2023-11-16T16:37:13.509466+0000","last_became_peered":"2023-11-16T16:37:13.509466+0000","last_unstale":"2023-11-16T16:38:03.190859+0000","last_undegraded":"2023-11-16T16:38:03.190859+0000","last_fullsized":"2023-11-16T16:38:03.190859+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:38:13.537860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.529514+0000","last_change":"2023-11-16T16:37:13.947906+0000","last_active":"2023-11-16T16:38:31.529514+0000","last_peered":"2023-11-16T16:38:31.529514+0000","last_clean":"2023-11-16T16:38:31.529514+0000","last_became_active":"2023-11-16T16:37:13.947602+0000","last_became_peered":"2023-11-16T16:37:13.947602+0000","last_unstale":"2023-11-16T16:38:31.529514+0000","last_undegraded":"2023-11-16T16:38:31.529514+0000","last_fullsized":"2023-11-16T16:38:31.529514+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:16:45.611613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.188948+0000","last_change":"2023-11-16T16:37:13.506108+0000","last_active":"2023-11-16T16:38:03.188948+0000","last_peered":"2023-11-16T16:38:03.188948+0000","last_clean":"2023-11-16T16:38:03.188948+0000","last_became_active":"2023-11-16T16:37:13.505785+0000","last_became_peered":"2023-11-16T16:37:13.505785+0000","last_unstale":"2023-11-16T16:38:03.188948+0000","last_undegraded":"2023-11-16T16:38:03.188948+0000","last_fullsized":"2023-11-16T16:38:03.188948+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:11:02.793557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.525780+0000","last_change":"2023-11-16T16:37:13.935863+0000","last_active":"2023-11-16T16:38:31.525780+0000","last_peered":"2023-11-16T16:38:31.525780+0000","last_clean":"2023-11-16T16:38:31.525780+0000","last_became_active":"2023-11-16T16:37:13.935399+0000","last_became_peered":"2023-11-16T16:37:13.935399+0000","last_unstale":"2023-11-16T16:38:31.525780+0000","last_undegraded":"2023-11-16T16:38:31.525780+0000","last_fullsized":"2023-11-16T16:38:31.525780+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:45:56.613743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184907+0000","last_change":"2023-11-16T16:37:13.498367+0000","last_active":"2023-11-16T16:38:03.184907+0000","last_peered":"2023-11-16T16:38:03.184907+0000","last_clean":"2023-11-16T16:38:03.184907+0000","last_became_active":"2023-11-16T16:37:13.497039+0000","last_became_peered":"2023-11-16T16:37:13.497039+0000","last_unstale":"2023-11-16T16:38:03.184907+0000","last_undegraded":"2023-11-16T16:38:03.184907+0000","last_fullsized":"2023-11-16T16:38:03.184907+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:57:46.571219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.528964+0000","last_change":"2023-11-16T16:37:13.940617+0000","last_active":"2023-11-16T16:38:31.528964+0000","last_peered":"2023-11-16T16:38:31.528964+0000","last_clean":"2023-11-16T16:38:31.528964+0000","last_became_active":"2023-11-16T16:37:13.940264+0000","last_became_peered":"2023-11-16T16:37:13.940264+0000","last_unstale":"2023-11-16T16:38:31.528964+0000","last_undegraded":"2023-11-16T16:38:31.528964+0000","last_fullsized":"2023-11-16T16:38:31.528964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:59:36.630848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.193922+0000","last_change":"2023-11-16T16:38:03.193922+0000","last_active":"2023-11-16T16:38:03.193922+0000","last_peered":"2023-11-16T16:38:03.193922+0000","last_clean":"2023-11-16T16:38:03.193922+0000","last_became_active":"2023-11-16T16:38:03.190364+0000","last_became_peered":"2023-11-16T16:38:03.190364+0000","last_unstale":"2023-11-16T16:38:03.193922+0000","last_undegraded":"2023-11-16T16:38:03.193922+0000","last_fullsized":"2023-11-16T16:38:03.193922+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:59:04.681132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.529517+0000","last_change":"2023-11-16T16:37:13.943096+0000","last_active":"2023-11-16T16:38:31.529517+0000","last_peered":"2023-11-16T16:38:31.529517+0000","last_clean":"2023-11-16T16:38:31.529517+0000","last_became_active":"2023-11-16T16:37:13.942745+0000","last_became_peered":"2023-11-16T16:37:13.942745+0000","last_unstale":"2023-11-16T16:38:31.529517+0000","last_undegraded":"2023-11-16T16:38:31.529517+0000","last_fullsized":"2023-11-16T16:38:31.529517+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:02:12.721195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.515101+0000","last_change":"2023-11-16T16:37:13.938087+0000","last_active":"2023-11-16T16:38:31.515101+0000","last_peered":"2023-11-16T16:38:31.515101+0000","last_clean":"2023-11-16T16:38:31.515101+0000","last_became_active":"2023-11-16T16:37:13.937780+0000","last_became_peered":"2023-11-16T16:37:13.937780+0000","last_unstale":"2023-11-16T16:38:31.515101+0000","last_undegraded":"2023-11-16T16:38:31.515101+0000","last_fullsized":"2023-11-16T16:38:31.515101+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:05:14.399850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"86'12","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.519834+0000","last_change":"2023-11-16T16:37:13.939000+0000","last_active":"2023-11-16T16:38:31.519834+0000","last_peered":"2023-11-16T16:38:31.519834+0000","last_clean":"2023-11-16T16:38:31.519834+0000","last_became_active":"2023-11-16T16:37:13.938585+0000","last_became_peered":"2023-11-16T16:37:13.938585+0000","last_unstale":"2023-11-16T16:38:31.519834+0000","last_undegraded":"2023-11-16T16:38:31.519834+0000","last_fullsized":"2023-11-16T16:38:31.519834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:13:17.753666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"86'8","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.513239+0000","last_change":"2023-11-16T16:37:13.937696+0000","last_active":"2023-11-16T16:38:31.513239+0000","last_peered":"2023-11-16T16:38:31.513239+0000","last_clean":"2023-11-16T16:38:31.513239+0000","last_became_active":"2023-11-16T16:37:13.936920+0000","last_became_peered":"2023-11-16T16:37:13.936920+0000","last_unstale":"2023-11-16T16:38:31.513239+0000","last_undegraded":"2023-11-16T16:38:31.513239+0000","last_fullsized":"2023-11-16T16:38:31.513239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:55:00.163322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"86'7","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.517747+0000","last_change":"2023-11-16T16:38:03.189679+0000","last_active":"2023-11-16T16:38:31.517747+0000","last_peered":"2023-11-16T16:38:31.517747+0000","last_clean":"2023-11-16T16:38:31.517747+0000","last_became_active":"2023-11-16T16:38:03.189292+0000","last_became_peered":"2023-11-16T16:38:03.189292+0000","last_unstale":"2023-11-16T16:38:31.517747+0000","last_undegraded":"2023-11-16T16:38:31.517747+0000","last_fullsized":"2023-11-16T16:38:31.517747+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:24:48.303402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.507243+0000","last_change":"2023-11-16T16:37:13.949940+0000","last_active":"2023-11-16T16:38:31.507243+0000","last_peered":"2023-11-16T16:38:31.507243+0000","last_clean":"2023-11-16T16:38:31.507243+0000","last_became_active":"2023-11-16T16:37:13.949464+0000","last_became_peered":"2023-11-16T16:37:13.949464+0000","last_unstale":"2023-11-16T16:38:31.507243+0000","last_undegraded":"2023-11-16T16:38:31.507243+0000","last_fullsized":"2023-11-16T16:38:31.507243+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:05:15.768098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.514662+0000","last_change":"2023-11-16T16:37:13.940534+0000","last_active":"2023-11-16T16:38:31.514662+0000","last_peered":"2023-11-16T16:38:31.514662+0000","last_clean":"2023-11-16T16:38:31.514662+0000","last_became_active":"2023-11-16T16:37:13.940174+0000","last_became_peered":"2023-11-16T16:37:13.940174+0000","last_unstale":"2023-11-16T16:38:31.514662+0000","last_undegraded":"2023-11-16T16:38:31.514662+0000","last_fullsized":"2023-11-16T16:38:31.514662+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:29:35.708635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"86'15","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.516894+0000","last_change":"2023-11-16T16:37:13.937519+0000","last_active":"2023-11-16T16:38:31.516894+0000","last_peered":"2023-11-16T16:38:31.516894+0000","last_clean":"2023-11-16T16:38:31.516894+0000","last_became_active":"2023-11-16T16:37:13.937175+0000","last_became_peered":"2023-11-16T16:37:13.937175+0000","last_unstale":"2023-11-16T16:38:31.516894+0000","last_undegraded":"2023-11-16T16:38:31.516894+0000","last_fullsized":"2023-11-16T16:38:31.516894+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:25:23.849158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.503336+0000","last_change":"2023-11-16T16:37:13.951506+0000","last_active":"2023-11-16T16:38:31.503336+0000","last_peered":"2023-11-16T16:38:31.503336+0000","last_clean":"2023-11-16T16:38:31.503336+0000","last_became_active":"2023-11-16T16:37:13.951192+0000","last_became_peered":"2023-11-16T16:37:13.951192+0000","last_unstale":"2023-11-16T16:38:31.503336+0000","last_undegraded":"2023-11-16T16:38:31.503336+0000","last_fullsized":"2023-11-16T16:38:31.503336+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:17:38.223843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.514590+0000","last_change":"2023-11-16T16:37:13.935614+0000","last_active":"2023-11-16T16:38:31.514590+0000","last_peered":"2023-11-16T16:38:31.514590+0000","last_clean":"2023-11-16T16:38:31.514590+0000","last_became_active":"2023-11-16T16:37:13.935237+0000","last_became_peered":"2023-11-16T16:37:13.935237+0000","last_unstale":"2023-11-16T16:38:31.514590+0000","last_undegraded":"2023-11-16T16:38:31.514590+0000","last_fullsized":"2023-11-16T16:38:31.514590+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:30:01.772328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"86'8","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.514616+0000","last_change":"2023-11-16T16:37:13.939387+0000","last_active":"2023-11-16T16:38:31.514616+0000","last_peered":"2023-11-16T16:38:31.514616+0000","last_clean":"2023-11-16T16:38:31.514616+0000","last_became_active":"2023-11-16T16:37:13.939082+0000","last_became_peered":"2023-11-16T16:37:13.939082+0000","last_unstale":"2023-11-16T16:38:31.514616+0000","last_undegraded":"2023-11-16T16:38:31.514616+0000","last_fullsized":"2023-11-16T16:38:31.514616+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:28:19.026881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"86'10","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.527049+0000","last_change":"2023-11-16T16:37:13.936381+0000","last_active":"2023-11-16T16:38:31.527049+0000","last_peered":"2023-11-16T16:38:31.527049+0000","last_clean":"2023-11-16T16:38:31.527049+0000","last_became_active":"2023-11-16T16:37:13.936077+0000","last_became_peered":"2023-11-16T16:37:13.936077+0000","last_unstale":"2023-11-16T16:38:31.527049+0000","last_undegraded":"2023-11-16T16:38:31.527049+0000","last_fullsized":"2023-11-16T16:38:31.527049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:20:31.015344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"86'11","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.526052+0000","last_change":"2023-11-16T16:37:13.934005+0000","last_active":"2023-11-16T16:38:31.526052+0000","last_peered":"2023-11-16T16:38:31.526052+0000","last_clean":"2023-11-16T16:38:31.526052+0000","last_became_active":"2023-11-16T16:37:13.933633+0000","last_became_peered":"2023-11-16T16:37:13.933633+0000","last_unstale":"2023-11-16T16:38:31.526052+0000","last_undegraded":"2023-11-16T16:38:31.526052+0000","last_fullsized":"2023-11-16T16:38:31.526052+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:41:36.990242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"86'8","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.526966+0000","last_change":"2023-11-16T16:37:13.946495+0000","last_active":"2023-11-16T16:38:31.526966+0000","last_peered":"2023-11-16T16:38:31.526966+0000","last_clean":"2023-11-16T16:38:31.526966+0000","last_became_active":"2023-11-16T16:37:13.946148+0000","last_became_peered":"2023-11-16T16:37:13.946148+0000","last_unstale":"2023-11-16T16:38:31.526966+0000","last_undegraded":"2023-11-16T16:38:31.526966+0000","last_fullsized":"2023-11-16T16:38:31.526966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:25:08.609723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":3542208,"data_compressed":211753,"data_compressed_allocated":270336,"data_compressed_original":3522560,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68836,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":609,"num_read_kb":377,"num_write":162,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":200811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416416,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6104,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93739048,"statfs":{"total":95995035648,"available":95988785152,"internally_reserved":0,"allocated":933888,"data_stored":447021,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58599999999999997,"15min":0.58499999999999996},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":2.6150000000000002,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.624,"15min":0.625},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.65100000000000002}]},{"osd":1,"last update":"Thu Nov 16 16:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.64600000000000002,"15min":0.63800000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89400000000000002,"5min":1.026,"15min":1.026},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.66700000000000004,"15min":0.66500000000000004},"min":{"1min":0.44700000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.97499999999999998,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.63400000000000001}]},{"osd":2,"last update":"Thu Nov 16 16:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.58799999999999997,"15min":0.59499999999999997},"min":{"1min":0.41699999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84699999999999998,"5min":1.087,"15min":1.087},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56100000000000005,"15min":0.56200000000000006},"min":{"1min":0.31900000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.86499999999999999,"5min":0.88,"15min":0.88},"last":0.38300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":4,"last update":"Thu Nov 16 16:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.746,"15min":0.748},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.94199999999999995,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.755,"15min":0.75700000000000001},"min":{"1min":0.54500000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0389999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.66400000000000003}]},{"osd":5,"last update":"Thu Nov 16 16:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.69999999999999996,"15min":0.70599999999999996},"min":{"1min":0.379,"5min":0.37,"15min":0.37},"max":{"1min":0.90400000000000003,"5min":1.722,"15min":1.722},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.73399999999999999,"15min":0.74299999999999999},"min":{"1min":0.436,"5min":0.378,"15min":0.378},"max":{"1min":0.97399999999999998,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.53500000000000003}]},{"osd":6,"last update":"Thu Nov 16 16:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.76000000000000001,"15min":0.76600000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0740000000000001,"5min":1.516,"15min":1.516},"last":0.58699999999999997},{"interface":"front","average":{"1min":1.254,"5min":0.91800000000000004,"15min":0.86199999999999999},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":11.007,"5min":11.007,"15min":11.007},"last":0.56399999999999995}]},{"osd":7,"last update":"Thu Nov 16 16:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.61099999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0249999999999999,"5min":1.399,"15min":1.399},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.1279999999999999,"5min":1.363,"15min":1.363},"last":0.748}]},{"osd":8,"last update":"Thu Nov 16 16:38:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.70599999999999996}]},{"osd":9,"last update":"Thu Nov 16 16:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.86399999999999999,"15min":0.86499999999999999},"min":{"1min":0.64700000000000002,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.399,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.347,"5min":0.95399999999999996,"15min":0.88800000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":10.83,"5min":10.83,"15min":10.83},"last":0.77200000000000002}]},{"osd":10,"last update":"Thu Nov 16 16:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.89300000000000002,"15min":0.89200000000000002},"min":{"1min":0.65400000000000003,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.169,"5min":1.427,"15min":1.427},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.89100000000000001,"15min":0.89800000000000002},"min":{"1min":0.65000000000000002,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.073,"5min":1.863,"15min":1.863},"last":0.78000000000000003}]}]},{"osd":10,"up_from":72,"seq":309237645348,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6564,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738588,"statfs":{"total":95995035648,"available":95988314112,"internally_reserved":0,"allocated":1077248,"data_stored":1102490,"data_compressed":41378,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:38:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.78700000000000003}]},{"osd":1,"last update":"Thu Nov 16 16:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.49199999999999999,"15min":0.49299999999999999},"min":{"1min":0.34399999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.67900000000000005,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.52800000000000002,"15min":0.52700000000000002},"min":{"1min":0.33300000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.86499999999999999,"5min":0.879,"15min":0.879},"last":0.41099999999999998}]},{"osd":2,"last update":"Thu Nov 16 16:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64300000000000002,"15min":0.64100000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89300000000000002,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.70899999999999996,"15min":0.71599999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.80400000000000005,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.53100000000000003}]},{"osd":3,"last update":"Thu Nov 16 16:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65900000000000003,"15min":0.66500000000000004},"min":{"1min":0.44,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82399999999999995,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.625,"5min":0.67300000000000004,"15min":0.68100000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.85699999999999998,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.59099999999999997}]},{"osd":4,"last update":"Thu Nov 16 16:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63700000000000001,"15min":0.63500000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.94399999999999995,"5min":1.141,"15min":1.141},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.58299999999999996,"15min":0.57199999999999995},"min":{"1min":0.48199999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.91600000000000004,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.38200000000000001}]},{"osd":5,"last update":"Thu Nov 16 16:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.752,"15min":0.754},"min":{"1min":0.57699999999999996,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.95499999999999996,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.754,"15min":0.75900000000000001},"min":{"1min":0.58399999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.94099999999999995,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.46999999999999997}]},{"osd":6,"last update":"Thu Nov 16 16:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.76200000000000001,"15min":0.76800000000000002},"min":{"1min":0.498,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0029999999999999,"5min":1.282,"15min":1.282},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.77700000000000002,"15min":0.77600000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0149999999999999,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.70999999999999996}]},{"osd":7,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.505}]},{"osd":8,"last update":"Thu Nov 16 16:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.83699999999999997,"15min":0.83499999999999996},"min":{"1min":0.60199999999999998,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0760000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.61599999999999999,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0169999999999999,"5min":1.5,"15min":1.5},"last":0.625}]},{"osd":9,"last update":"Thu Nov 16 16:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0289999999999999,"5min":1.337,"15min":1.337},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81799999999999995,"15min":0.81899999999999995},"min":{"1min":0.53000000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0369999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.69199999999999995}]},{"osd":11,"last update":"Thu Nov 16 16:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.83899999999999997,"15min":0.84199999999999997},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.0900000000000001,"5min":1.2,"15min":1.2},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.83799999999999997,"15min":0.83099999999999996},"min":{"1min":0.67000000000000004,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.1180000000000001,"5min":1.236,"15min":1.236},"last":0.65400000000000003}]}]},{"osd":9,"up_from":67,"seq":287762808871,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6240,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":93738912,"statfs":{"total":95995035648,"available":95988645888,"internally_reserved":0,"allocated":942080,"data_stored":447145,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.44400000000000001,"15min":0.46100000000000002},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.47199999999999998,"15min":0.48199999999999998},"min":{"1min":0.28199999999999997,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.40400000000000003}]},{"osd":1,"last update":"Thu Nov 16 16:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59099999999999997,"15min":0.60599999999999998},"min":{"1min":0.34699999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.60099999999999998,"15min":0.627},"min":{"1min":0.41899999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.60199999999999998}]},{"osd":2,"last update":"Thu Nov 16 16:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.623,"15min":0.64300000000000002},"min":{"1min":0.42499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.62,"15min":0.63800000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77600000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.66900000000000004}]},{"osd":3,"last update":"Thu Nov 16 16:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54100000000000004,"15min":0.56100000000000005},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":0.496},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53700000000000003,"15min":0.56799999999999995},"min":{"1min":0.26900000000000002,"5min":0.25,"15min":0.25},"max":{"1min":2.2490000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":0.47499999999999998}]},{"osd":4,"last update":"Thu Nov 16 16:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.64400000000000002,"15min":0.67500000000000004},"min":{"1min":0.40400000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83899999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67000000000000004,"15min":0.68899999999999995},"min":{"1min":0.45500000000000002,"5min":0.318,"15min":0.318},"max":{"1min":2.2269999999999999,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":0.69099999999999995}]},{"osd":5,"last update":"Thu Nov 16 16:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.67700000000000005,"15min":0.70299999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.82799999999999996,"5min":1.056,"15min":1.056},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.70199999999999996,"15min":0.73299999999999998},"min":{"1min":0.39800000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.68000000000000005}]},{"osd":6,"last update":"Thu Nov 16 16:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.61899999999999999,"15min":0.66200000000000003},"min":{"1min":0.32000000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.67000000000000004,"15min":0.71599999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.53500000000000003}]},{"osd":7,"last update":"Thu Nov 16 16:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.751,"15min":0.77200000000000002},"min":{"1min":0.44700000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":2.153,"5min":2.153,"15min":2.153},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.75,"15min":0.78800000000000003},"min":{"1min":0.55200000000000005,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.76800000000000002}]},{"osd":8,"last update":"Thu Nov 16 16:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.78000000000000003,"15min":0.81699999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.88600000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.746},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.78000000000000003,"15min":0.80900000000000005},"min":{"1min":0.54300000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.876,"5min":1.089,"15min":1.089},"last":0.79700000000000004}]},{"osd":10,"last update":"Thu Nov 16 16:38:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.73699999999999999,"15min":0.747},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.91700000000000004,"5min":1.028,"15min":1.028},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.76100000000000001,"15min":0.76300000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.1819999999999999,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":0.86899999999999999}]}]},{"osd":8,"up_from":59,"seq":253403070509,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6424,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738728,"statfs":{"total":95995035648,"available":95988457472,"internally_reserved":0,"allocated":933888,"data_stored":446953,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57599999999999996,"15min":0.58199999999999996},"min":{"1min":0.40000000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.72099999999999997,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.438,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.72699999999999998,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.80100000000000005}]},{"osd":1,"last update":"Thu Nov 16 16:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.65200000000000002,"15min":0.625},"min":{"1min":0.34100000000000003,"5min":0.318,"15min":0.318},"max":{"1min":0.94899999999999995,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.628,"15min":0.60499999999999998},"min":{"1min":0.46100000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.81299999999999994,"5min":1.383,"15min":1.383},"last":0.64200000000000002}]},{"osd":2,"last update":"Thu Nov 16 16:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.52400000000000002,"15min":0.51700000000000002},"min":{"1min":0.40400000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.86499999999999999,"5min":1.056,"15min":1.056},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.53200000000000003,"15min":0.52600000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.51100000000000001}]},{"osd":3,"last update":"Thu Nov 16 16:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.621,"15min":0.624},"min":{"1min":0.39800000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.83099999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.68300000000000005,"15min":0.67700000000000005},"min":{"1min":0.48099999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.99299999999999999,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.83499999999999996}]},{"osd":4,"last update":"Thu Nov 16 16:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.68100000000000005,"15min":0.64100000000000001},"min":{"1min":0.36899999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.093,"5min":4.0599999999999996,"15min":4.0599999999999996},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.72999999999999998,"15min":0.70799999999999996},"min":{"1min":0.45000000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1200000000000001,"5min":1.167,"15min":1.167},"last":0.71299999999999997}]},{"osd":5,"last update":"Thu Nov 16 16:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.61699999999999999,"15min":0.61499999999999999},"min":{"1min":0.36899999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.89400000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.40000000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.91500000000000004,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.67000000000000004}]},{"osd":6,"last update":"Thu Nov 16 16:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.76000000000000001,"15min":0.75},"min":{"1min":0.53500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1020000000000001,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.75,"15min":0.752},"min":{"1min":0.54000000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.98799999999999999,"5min":1.173,"15min":1.173},"last":0.82399999999999995}]},{"osd":7,"last update":"Thu Nov 16 16:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.78500000000000003,"15min":0.76900000000000002},"min":{"1min":0.49199999999999999,"5min":0.433,"15min":0.433},"max":{"1min":1.05,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.77100000000000002,"15min":0.75800000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0349999999999999,"5min":1.369,"15min":1.369},"last":0.754}]},{"osd":9,"last update":"Thu Nov 16 16:37:55 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.73799999999999999,"15min":0.72999999999999998},"min":{"1min":0.54000000000000004,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.018,"5min":1.036,"15min":1.036},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.875,"5min":0.76900000000000002,"15min":0.751},"min":{"1min":0.58699999999999997,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.95799999999999996}]},{"osd":10,"last update":"Thu Nov 16 16:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.87,"15min":0.85999999999999999},"min":{"1min":0.79700000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.127,"5min":3.5720000000000001,"15min":3.5720000000000001},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.90600000000000003,"15min":0.89200000000000002},"min":{"1min":0.82599999999999996,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.149,"5min":3.5920000000000001,"15min":3.5920000000000001},"last":0.93400000000000005}]},{"osd":11,"last update":"Thu Nov 16 16:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.90900000000000003}]}]},{"osd":7,"up_from":52,"seq":223338299441,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6544,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738608,"statfs":{"total":95995035648,"available":95988334592,"internally_reserved":0,"allocated":991232,"data_stored":1039363,"data_compressed":41378,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:36:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55800000000000005}]},{"osd":1,"last update":"Thu Nov 16 16:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.42999999999999999,"15min":0.41499999999999998},"min":{"1min":0.313,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.44800000000000001,"15min":0.42999999999999999},"min":{"1min":0.33100000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.66800000000000004,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.64600000000000002}]},{"osd":2,"last update":"Thu Nov 16 16:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.58299999999999996,"15min":0.56499999999999995},"min":{"1min":0.45600000000000002,"5min":0.316,"15min":0.316},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.59699999999999998,"15min":0.57699999999999996},"min":{"1min":0.52200000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.73599999999999999}]},{"osd":3,"last update":"Thu Nov 16 16:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58299999999999996,"15min":0.57299999999999995},"min":{"1min":0.44,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.79300000000000004,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.67500000000000004,"15min":0.63700000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":5.6100000000000003,"5min":5.6100000000000003,"15min":5.6100000000000003},"last":0.47899999999999998}]},{"osd":4,"last update":"Thu Nov 16 16:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.54700000000000004,"15min":0.52100000000000002},"min":{"1min":0.371,"5min":0.317,"15min":0.317},"max":{"1min":0.77800000000000002,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.51100000000000001,"15min":0.48699999999999999},"min":{"1min":0.379,"5min":0.311,"15min":0.311},"max":{"1min":0.72799999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.61099999999999999}]},{"osd":5,"last update":"Thu Nov 16 16:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.66800000000000004,"15min":0.65200000000000002},"min":{"1min":0.47599999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.66600000000000004,"15min":0.64100000000000001},"min":{"1min":0.50900000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.69499999999999995}]},{"osd":6,"last update":"Thu Nov 16 16:38:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":0.81599999999999995,"15min":0.73299999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":11.647,"5min":11.647,"15min":11.647},"last":0.752},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.73099999999999998,"15min":0.69099999999999995},"min":{"1min":0.51100000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":4.9550000000000001,"5min":4.9550000000000001,"15min":4.9550000000000001},"last":0.50700000000000001}]},{"osd":8,"last update":"Thu Nov 16 16:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.75700000000000001,"15min":0.75},"min":{"1min":0.60499999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.0289999999999999,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.76700000000000002,"15min":0.76500000000000001},"min":{"1min":0.56200000000000006,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.069,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.82999999999999996}]},{"osd":9,"last update":"Thu Nov 16 16:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.79700000000000004,"15min":0.78000000000000003},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.98099999999999998,"5min":4.7370000000000001,"15min":4.7370000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.83099999999999996,"15min":0.81799999999999995},"min":{"1min":0.54500000000000004,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.97699999999999998,"5min":4.7809999999999997,"15min":4.7809999999999997},"last":0.80700000000000005}]},{"osd":10,"last update":"Thu Nov 16 16:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.80400000000000005,"15min":0.80100000000000005},"min":{"1min":0.64300000000000002,"5min":0.443,"15min":0.443},"max":{"1min":1.026,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.76400000000000001,"15min":0.75800000000000001},"min":{"1min":0.64000000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0509999999999999,"5min":1.054,"15min":1.054},"last":0.84099999999999997}]},{"osd":11,"last update":"Thu Nov 16 16:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.873}]}]},{"osd":6,"up_from":47,"seq":201863462965,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6356,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738796,"statfs":{"total":95995035648,"available":95988527104,"internally_reserved":0,"allocated":929792,"data_stored":446540,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.42599999999999999,"15min":0.41599999999999998},"min":{"1min":0.29199999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.43099999999999999,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.45900000000000002,"15min":0.441},"min":{"1min":0.29699999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.56499999999999995,"5min":3.472,"15min":3.472},"last":0.36099999999999999}]},{"osd":1,"last update":"Thu Nov 16 16:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.64400000000000002,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.63700000000000001,"15min":0.59699999999999998},"min":{"1min":0.40699999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.65400000000000003,"5min":4.4669999999999996,"15min":4.4669999999999996},"last":0.443}]},{"osd":2,"last update":"Thu Nov 16 16:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.63600000000000001,"15min":0.63200000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.68300000000000005,"5min":3.5369999999999999,"15min":3.5369999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.64900000000000002,"15min":0.629},"min":{"1min":0.443,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.64500000000000002,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.51700000000000002}]},{"osd":3,"last update":"Thu Nov 16 16:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.504,"15min":0.48299999999999998},"min":{"1min":0.28999999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.46600000000000003,"5min":3.3700000000000001,"15min":3.3700000000000001},"last":0.434},{"interface":"front","average":{"1min":0.434,"5min":0.53600000000000003,"15min":0.53400000000000003},"min":{"1min":0.35099999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.50900000000000001,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.502}]},{"osd":4,"last update":"Thu Nov 16 16:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.66800000000000004,"15min":0.60799999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.75900000000000001,"5min":4.4900000000000002,"15min":4.4900000000000002},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.68899999999999995,"15min":0.627},"min":{"1min":0.379,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.69399999999999995,"5min":4.4509999999999996,"15min":4.4509999999999996},"last":0.47299999999999998}]},{"osd":5,"last update":"Thu Nov 16 16:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.66900000000000004,"15min":0.67500000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.71599999999999997,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.71099999999999997,"15min":0.70299999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.75,"5min":3.5019999999999998,"15min":3.5019999999999998},"last":0.59099999999999997}]},{"osd":7,"last update":"Thu Nov 16 16:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.70899999999999996,"15min":0.70499999999999996},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":2.452,"5min":2.452,"15min":2.452},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.73999999999999999,"15min":0.71799999999999997},"min":{"1min":0.54400000000000004,"5min":0.436,"15min":0.436},"max":{"1min":3.4449999999999998,"5min":3.4449999999999998,"15min":3.4449999999999998},"last":0.60999999999999999}]},{"osd":8,"last update":"Thu Nov 16 16:38:32 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.752,"15min":0.76900000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":2.3919999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.77800000000000002,"15min":0.78300000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":3.488,"5min":3.488,"15min":3.488},"last":0.64600000000000002}]},{"osd":9,"last update":"Thu Nov 16 16:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.77300000000000002,"15min":0.76700000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.68700000000000006,"5min":6.1539999999999999,"15min":6.1539999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.79800000000000004,"15min":0.76800000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.67800000000000005,"5min":8.2460000000000004,"15min":8.2460000000000004},"last":0.57799999999999996}]},{"osd":10,"last update":"Thu Nov 16 16:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.81200000000000006,"15min":0.81000000000000005},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":3.5209999999999999,"5min":3.5209999999999999,"15min":3.5209999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.82499999999999996,"15min":0.82399999999999995},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":3.419,"5min":3.419,"15min":3.419},"last":0.65600000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7440,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":93737712,"statfs":{"total":95995035648,"available":95987417088,"internally_reserved":0,"allocated":991232,"data_stored":505103,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:38:06 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.48499999999999999,"15min":0.45800000000000002},"min":{"1min":0.34899999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.67000000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52100000000000002,"15min":0.48799999999999999},"min":{"1min":0.311,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.83499999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.68799999999999994}]},{"osd":2,"last update":"Thu Nov 16 16:38:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.58299999999999996,"15min":0.51100000000000001},"min":{"1min":0.45500000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.58199999999999996,"15min":0.51700000000000002},"min":{"1min":0.46400000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.85199999999999998,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.55000000000000004}]},{"osd":3,"last update":"Thu Nov 16 16:38:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58599999999999997,"15min":0.55800000000000005},"min":{"1min":0.33900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.93700000000000006,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60999999999999999,"15min":0.57899999999999996},"min":{"1min":0.38100000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.89700000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.65000000000000002}]},{"osd":4,"last update":"Thu Nov 16 16:38:06 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.46300000000000002,"15min":0.43099999999999999},"min":{"1min":0.33700000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.79500000000000004,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55000000000000004,"15min":0.48499999999999999},"min":{"1min":0.35499999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.82899999999999996,"5min":1.242,"15min":1.242},"last":0.84399999999999997}]},{"osd":5,"last update":"Thu Nov 16 16:38:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.67300000000000004,"15min":0.623},"min":{"1min":0.55100000000000005,"5min":0.38,"15min":0.38},"max":{"1min":0.90700000000000003,"5min":1.004,"15min":1.004},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.65000000000000002,"15min":0.60699999999999998},"min":{"1min":0.53700000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93899999999999995,"5min":1.004,"15min":1.004},"last":0.621}]},{"osd":6,"last update":"Thu Nov 16 16:38:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.69599999999999995,"15min":0.67200000000000004},"min":{"1min":0.51900000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.02,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69299999999999995,"15min":0.66700000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.82599999999999996}]},{"osd":7,"last update":"Thu Nov 16 16:38:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65300000000000002,"15min":0.63500000000000001},"min":{"1min":0.39500000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.0569999999999999,"5min":1.077,"15min":1.077},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.67900000000000005},"min":{"1min":0.42599999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.018,"5min":1.143,"15min":1.143},"last":1.034}]},{"osd":8,"last update":"Thu Nov 16 16:38:30 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.76800000000000002,"15min":0.75800000000000001},"min":{"1min":0.56799999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0289999999999999,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.75,"15min":0.73999999999999999},"min":{"1min":0.52800000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0640000000000001,"5min":1.145,"15min":1.145},"last":0.877}]},{"osd":9,"last update":"Thu Nov 16 16:37:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.77900000000000003,"15min":0.79900000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.87,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.84599999999999997,"15min":0.86399999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.97299999999999998,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.94999999999999996}]},{"osd":10,"last update":"Thu Nov 16 16:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.83999999999999997,"15min":0.84999999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.99199999999999999,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.81599999999999995,"15min":0.81999999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.98099999999999998,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.92400000000000004}]},{"osd":11,"last update":"Thu Nov 16 16:38:15 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.96999999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7272,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":6336,"kb_avail":93737880,"statfs":{"total":95995035648,"available":95987589120,"internally_reserved":0,"allocated":950272,"data_stored":447805,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":6488064},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Nov 16 16:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.54700000000000004,"15min":0.48199999999999998},"min":{"1min":0.38400000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":3.6869999999999998,"5min":3.6869999999999998,"15min":3.6869999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.57999999999999996,"15min":0.48799999999999999},"min":{"1min":0.40600000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":6.6059999999999999,"5min":6.6059999999999999,"15min":6.6059999999999999},"last":0.51800000000000002}]},{"osd":2,"last update":"Thu Nov 16 16:38:27 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.63300000000000001,"15min":0.53900000000000003},"min":{"1min":0.39700000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":6.4000000000000004,"5min":6.4000000000000004,"15min":6.4000000000000004},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.60799999999999998,"15min":0.54600000000000004},"min":{"1min":0.42599999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":3.7829999999999999,"5min":3.7829999999999999,"15min":3.7829999999999999},"last":0.60699999999999998}]},{"osd":3,"last update":"Thu Nov 16 16:38:15 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":9.0139999999999993,"5min":9.0139999999999993,"15min":9.0139999999999993},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":3.6400000000000001,"5min":3.6400000000000001,"15min":3.6400000000000001},"last":0.38600000000000001}]},{"osd":4,"last update":"Thu Nov 16 16:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.61499999999999999,"15min":0.56299999999999994},"min":{"1min":0.44,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":3.8109999999999999,"5min":3.8109999999999999,"15min":3.8109999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.871,"5min":0.64900000000000002,"15min":0.57199999999999995},"min":{"1min":0.42099999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":6.577,"5min":6.577,"15min":6.577},"last":0.70899999999999996}]},{"osd":5,"last update":"Thu Nov 16 16:38:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":0.65700000000000003,"15min":0.58299999999999996},"min":{"1min":0.47899999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":9.0690000000000008,"5min":9.0690000000000008,"15min":9.0690000000000008},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.70599999999999996,"15min":0.64100000000000001},"min":{"1min":0.38800000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":6.4539999999999997,"5min":6.4539999999999997,"15min":6.4539999999999997},"last":0.75800000000000001}]},{"osd":6,"last update":"Thu Nov 16 16:38:50 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.623,"15min":0.58899999999999997},"min":{"1min":0.314,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68300000000000005,"5min":6.5190000000000001,"15min":6.5190000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.502,"5min":0.622,"15min":0.57199999999999995},"min":{"1min":0.313,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.65600000000000003,"5min":9.141,"15min":9.141},"last":0.54200000000000004}]},{"osd":7,"last update":"Thu Nov 16 16:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.68400000000000005,"15min":0.67600000000000005},"min":{"1min":0.495,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.74199999999999999,"15min":0.73199999999999998},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.56399999999999995}]},{"osd":8,"last update":"Thu Nov 16 16:38:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.78900000000000003,"15min":0.80900000000000005},"min":{"1min":0.496,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":3.8410000000000002,"5min":3.8410000000000002,"15min":3.8410000000000002},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.86099999999999999,"15min":0.86199999999999999},"min":{"1min":0.57099999999999995,"5min":0.498,"15min":0.498},"max":{"1min":6.4870000000000001,"5min":6.4870000000000001,"15min":6.4870000000000001},"last":0.78500000000000003}]},{"osd":9,"last update":"Thu Nov 16 16:37:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.85499999999999998,"15min":0.86199999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":6.6349999999999998,"5min":6.6349999999999998,"15min":6.6349999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.79000000000000004,"15min":0.76600000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":9.1690000000000005,"5min":9.1690000000000005,"15min":9.1690000000000005},"last":0.59199999999999997}]},{"osd":10,"last update":"Thu Nov 16 16:38:15 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.78500000000000003,"15min":0.78100000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":3.7229999999999999,"5min":3.7229999999999999,"15min":3.7229999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.88300000000000001,"15min":0.86499999999999999},"min":{"1min":0.55400000000000005,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":6.5460000000000003,"5min":6.5460000000000003,"15min":6.5460000000000003},"last":0.80500000000000005}]},{"osd":11,"last update":"Thu Nov 16 16:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.99299999999999999,"15min":1.0409999999999999},"min":{"1min":0.56399999999999995,"5min":0.502,"15min":0.502},"max":{"1min":0.86399999999999999,"5min":9.1039999999999992,"15min":9.1039999999999992},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.83499999999999996,"15min":0.85499999999999998},"min":{"1min":0.53500000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.91200000000000003,"5min":3.7530000000000001,"15min":3.7530000000000001},"last":0.74199999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7532,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":6528,"kb_avail":93737620,"statfs":{"total":95995035648,"available":95987322880,"internally_reserved":0,"allocated":1019904,"data_stored":512951,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":6684672},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:38:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.52700000000000002,"15min":0.52300000000000002},"min":{"1min":0.372,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.88300000000000001,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.50700000000000001,"15min":0.49099999999999999},"min":{"1min":0.35999999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.81200000000000006,"5min":1.006,"15min":1.006},"last":0.47899999999999998}]},{"osd":1,"last update":"Thu Nov 16 16:38:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.55500000000000005,"15min":0.52300000000000002},"min":{"1min":0.40400000000000003,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.60099999999999998,"15min":0.56499999999999995},"min":{"1min":0.46700000000000003,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.91800000000000004,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.70399999999999996}]},{"osd":3,"last update":"Thu Nov 16 16:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.66300000000000003,"15min":0.60699999999999998},"min":{"1min":0.38800000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.94199999999999995,"5min":6.9740000000000002,"15min":6.9740000000000002},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59299999999999997,"15min":0.56299999999999994},"min":{"1min":0.39200000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.86699999999999999,"5min":1.03,"15min":1.03},"last":0.625}]},{"osd":4,"last update":"Thu Nov 16 16:38:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.69699999999999995}]},{"osd":5,"last update":"Thu Nov 16 16:38:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.51700000000000002,"15min":0.49099999999999999},"min":{"1min":0.376,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.83199999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.48799999999999999,"15min":0.46400000000000002},"min":{"1min":0.39100000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.71699999999999997,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.55700000000000005}]},{"osd":6,"last update":"Thu Nov 16 16:38:50 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.70399999999999996,"15min":0.66200000000000003},"min":{"1min":0.52100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0509999999999999,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.66700000000000004,"15min":0.621},"min":{"1min":0.45400000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.97199999999999998,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.67500000000000004}]},{"osd":7,"last update":"Thu Nov 16 16:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.64300000000000002,"15min":0.64400000000000002},"min":{"1min":0.41599999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70199999999999996,"15min":0.70399999999999996},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.92700000000000005,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.57699999999999996}]},{"osd":8,"last update":"Thu Nov 16 16:38:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.59599999999999997,"15min":0.57599999999999996},"min":{"1min":0.35299999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.84199999999999997,"5min":1.008,"15min":1.008},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.63300000000000001,"15min":0.627},"min":{"1min":0.38500000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.82499999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.59799999999999998}]},{"osd":9,"last update":"Thu Nov 16 16:37:54 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.86799999999999999,"15min":0.86699999999999999},"min":{"1min":0.47499999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.6080000000000001,"5min":4.524,"15min":4.524},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.751,"15min":0.73299999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.6200000000000001,"5min":2.6200000000000001,"15min":2.6200000000000001},"last":0.79000000000000004}]},{"osd":10,"last update":"Thu Nov 16 16:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.73399999999999999,"15min":0.72499999999999998},"min":{"1min":0.55100000000000005,"5min":0.497,"15min":0.497},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.72599999999999998,"15min":0.72099999999999997},"min":{"1min":0.61499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.92500000000000004,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.75800000000000001}]},{"osd":11,"last update":"Thu Nov 16 16:38:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.874},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.80500000000000005}]}]},{"osd":3,"up_from":26,"seq":111669149761,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6688,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":93738464,"statfs":{"total":95995035648,"available":95988187136,"internally_reserved":0,"allocated":942080,"data_stored":446526,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:38:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.374,"15min":0.35199999999999998},"min":{"1min":0.28599999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.40100000000000002,"15min":0.36499999999999999},"min":{"1min":0.309,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.36399999999999999}]},{"osd":1,"last update":"Thu Nov 16 16:38:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56100000000000005,"15min":0.55100000000000005},"min":{"1min":0.371,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56399999999999995,"15min":0.55900000000000005},"min":{"1min":0.40300000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.84099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.65400000000000003}]},{"osd":2,"last update":"Thu Nov 16 16:38:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.59199999999999997,"15min":0.56100000000000005},"min":{"1min":0.439,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.91400000000000003,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57199999999999995,"15min":0.54900000000000004},"min":{"1min":0.34699999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.53000000000000003}]},{"osd":4,"last update":"Thu Nov 16 16:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56299999999999994,"15min":0.54900000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79900000000000004,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60599999999999998,"15min":0.59599999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.56200000000000006}]},{"osd":5,"last update":"Thu Nov 16 16:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59899999999999998,"15min":0.58099999999999996},"min":{"1min":0.39600000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95299999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.61899999999999999,"15min":0.59599999999999997},"min":{"1min":0.47399999999999998,"5min":0.37,"15min":0.37},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.55200000000000005}]},{"osd":6,"last update":"Thu Nov 16 16:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51800000000000002,"15min":0.498},"min":{"1min":0.35599999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.72399999999999998,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53200000000000003,"15min":0.51400000000000001},"min":{"1min":0.377,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.75800000000000001,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.44600000000000001}]},{"osd":7,"last update":"Thu Nov 16 16:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.65300000000000002,"15min":0.64900000000000002},"min":{"1min":0.503,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.68700000000000006,"15min":0.68500000000000005},"min":{"1min":0.42699999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.68400000000000005}]},{"osd":8,"last update":"Thu Nov 16 16:36:27 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.63200000000000001}]},{"osd":9,"last update":"Thu Nov 16 16:37:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.73799999999999999,"15min":0.76300000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80500000000000005,"5min":1.024,"15min":1.024},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.68999999999999995,"15min":0.70899999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90300000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.495}]},{"osd":10,"last update":"Thu Nov 16 16:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.73299999999999998,"15min":0.74299999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.74099999999999999,"15min":0.751},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.97599999999999998,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.70799999999999996}]},{"osd":11,"last update":"Thu Nov 16 16:38:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.60799999999999998}]}]},{"osd":4,"up_from":34,"seq":146028888126,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6488,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738664,"statfs":{"total":95995035648,"available":95988391936,"internally_reserved":0,"allocated":933888,"data_stored":446923,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.52400000000000002,"15min":0.502},"min":{"1min":0.38200000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.52900000000000003,"15min":0.51500000000000001},"min":{"1min":0.40899999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.378}]},{"osd":1,"last update":"Thu Nov 16 16:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.41099999999999998,"15min":0.39300000000000002},"min":{"1min":0.33700000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.67900000000000005,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.37},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.434,"15min":0.40999999999999998},"min":{"1min":0.378,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.748,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.39700000000000002}]},{"osd":2,"last update":"Thu Nov 16 16:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.77300000000000002,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.623,"15min":0.59299999999999997},"min":{"1min":0.53000000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.81999999999999995,"5min":1.056,"15min":1.056},"last":0.80300000000000005}]},{"osd":3,"last update":"Thu Nov 16 16:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.59199999999999997,"15min":0.55000000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57299999999999995,"15min":0.55300000000000005},"min":{"1min":0.40100000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.51500000000000001}]},{"osd":5,"last update":"Thu Nov 16 16:38:28 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.623,"15min":0.627},"min":{"1min":0.40100000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.76500000000000001,"5min":1.075,"15min":1.075},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.65700000000000003,"15min":0.63200000000000001},"min":{"1min":0.49099999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.89700000000000002,"5min":1.093,"15min":1.093},"last":0.84499999999999997}]},{"osd":6,"last update":"Thu Nov 16 16:38:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65000000000000002,"15min":0.60899999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.85399999999999998,"5min":1.004,"15min":1.004},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.61599999999999999,"15min":0.59799999999999998},"min":{"1min":0.433,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77300000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.46999999999999997}]},{"osd":7,"last update":"Thu Nov 16 16:38:13 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.61499999999999999,"15min":0.56299999999999994},"min":{"1min":0.45400000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.99399999999999999,"5min":1.123,"15min":1.123},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.67300000000000004,"15min":0.621},"min":{"1min":0.51600000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.99199999999999999,"5min":1.069,"15min":1.069},"last":0.78400000000000003}]},{"osd":8,"last update":"Thu Nov 16 16:38:28 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.76400000000000001,"15min":0.73199999999999998},"min":{"1min":0.53700000000000003,"5min":0.495,"15min":0.495},"max":{"1min":0.98699999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.77000000000000002,"15min":0.73899999999999999},"min":{"1min":0.51700000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.98199999999999998,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.82799999999999996}]},{"osd":9,"last update":"Thu Nov 16 16:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.79200000000000004,"15min":0.78900000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.94099999999999995,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.72099999999999997,"15min":0.71799999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94199999999999995,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.56299999999999994}]},{"osd":10,"last update":"Thu Nov 16 16:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.68200000000000005,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0109999999999999,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.753},{"interface":"front","average":{"1min":0.872,"5min":0.83999999999999997,"15min":0.83399999999999996},"min":{"1min":0.73399999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.74299999999999999}]},{"osd":11,"last update":"Thu Nov 16 16:38:13 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.86499999999999999}]}]},{"osd":5,"up_from":39,"seq":167503724603,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6684,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738468,"statfs":{"total":95995035648,"available":95988191232,"internally_reserved":0,"allocated":1003520,"data_stored":1039095,"data_compressed":41378,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.55500000000000005,"15min":0.51300000000000001},"min":{"1min":0.38100000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":4.2380000000000004,"5min":4.2380000000000004,"15min":4.2380000000000004},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.57499999999999996,"15min":0.54000000000000004},"min":{"1min":0.42899999999999999,"5min":0.311,"15min":0.311},"max":{"1min":3.5590000000000002,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":0.58599999999999997}]},{"osd":1,"last update":"Thu Nov 16 16:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.61299999999999999,"15min":0.59399999999999997},"min":{"1min":0.55600000000000005,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.997,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.60199999999999998,"15min":0.58999999999999997},"min":{"1min":0.42699999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84099999999999997,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.56899999999999995}]},{"osd":2,"last update":"Thu Nov 16 16:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.443,"15min":0.42899999999999999},"min":{"1min":0.32700000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.70399999999999996,"5min":1.417,"15min":1.417},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.46400000000000002,"15min":0.44},"min":{"1min":0.33700000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.70799999999999996,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.35699999999999998}]},{"osd":3,"last update":"Thu Nov 16 16:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.60299999999999998,"15min":0.56200000000000006},"min":{"1min":0.38,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":3.3079999999999998,"5min":3.3079999999999998,"15min":3.3079999999999998},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.67300000000000004,"15min":0.621},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":5.343,"5min":5.343,"15min":5.343},"last":0.47699999999999998}]},{"osd":4,"last update":"Thu Nov 16 16:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.69999999999999996,"15min":0.63600000000000001},"min":{"1min":0.46000000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85599999999999998,"5min":5.5789999999999997,"15min":5.5789999999999997},"last":0.628},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.62,"15min":0.59799999999999998},"min":{"1min":0.437,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84499999999999997,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.52900000000000003}]},{"osd":6,"last update":"Thu Nov 16 16:38:48 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.68700000000000006,"15min":0.63900000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.996,"5min":3.274,"15min":3.274},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.68300000000000005,"15min":0.64700000000000002},"min":{"1min":0.40100000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.0109999999999999,"5min":3.472,"15min":3.472},"last":0.50600000000000001}]},{"osd":7,"last update":"Thu Nov 16 16:38:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.72199999999999998,"15min":0.71599999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0449999999999999,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.73799999999999999,"15min":0.73199999999999998},"min":{"1min":0.48099999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0580000000000001,"5min":1.544,"15min":1.544},"last":0.66900000000000004}]},{"osd":8,"last update":"Thu Nov 16 16:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.66500000000000004,"15min":0.65100000000000002},"min":{"1min":0.48699999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.92500000000000004,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.746,"15min":0.73199999999999998},"min":{"1min":0.59499999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.69399999999999995}]},{"osd":9,"last update":"Thu Nov 16 16:38:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.90200000000000002,"15min":0.82799999999999996},"min":{"1min":0.52400000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1699999999999999,"5min":5.5679999999999996,"15min":5.5679999999999996},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.82099999999999995,"15min":0.77100000000000002},"min":{"1min":0.56399999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0640000000000001,"5min":5.3230000000000004,"15min":5.3230000000000004},"last":0.71299999999999997}]},{"osd":10,"last update":"Thu Nov 16 16:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.80000000000000004,"15min":0.78200000000000003},"min":{"1min":0.63900000000000001,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.80200000000000005,"15min":0.78800000000000003},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.755}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33928,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":58776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":66624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-16T16:38:57.522 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-11-16T16:38:57.522 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-16T16:38:57.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:57 smithi053 ceph-mon[100638]: pgmap v313: 97 pgs: 97 active+clean; 644 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 853 B/s wr, 8 op/s 2023-11-16T16:38:57.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:57 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1065218116' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:38:57.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:57 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/3008440254' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:38:57.684 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.680+0000 7f6709b20700 1 -- 172.21.15.1:0/3571696455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67041018b0 msgr2=0x7f6704101cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:57.684 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.680+0000 7f6709b20700 1 --2- 172.21.15.1:0/3571696455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67041018b0 0x7f6704101cc0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f66f4009990 tx=0x7f66f4009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:57.684 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.680+0000 7f6709b20700 1 -- 172.21.15.1:0/3571696455 shutdown_connections 2023-11-16T16:38:57.684 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.680+0000 7f6709b20700 1 --2- 172.21.15.1:0/3571696455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67040fecb0 0x7f6704108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:57.685 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.680+0000 7f6709b20700 1 --2- 172.21.15.1:0/3571696455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67041018b0 0x7f6704101cc0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:57.685 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.680+0000 7f6709b20700 1 --2- 172.21.15.1:0/3571696455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67040fe320 0x7f67040fe770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:57.685 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.680+0000 7f6709b20700 1 -- 172.21.15.1:0/3571696455 >> 172.21.15.1:0/3571696455 conn(0x7f67040f9f50 msgr2=0x7f67040fc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:57.685 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.680+0000 7f6709b20700 1 -- 172.21.15.1:0/3571696455 shutdown_connections 2023-11-16T16:38:57.685 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.680+0000 7f6709b20700 1 -- 172.21.15.1:0/3571696455 wait complete. 2023-11-16T16:38:57.685 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.681+0000 7f6709b20700 1 Processor -- start 2023-11-16T16:38:57.685 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.681+0000 7f6709b20700 1 -- start start 2023-11-16T16:38:57.685 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.681+0000 7f6709b20700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67040fe320 0x7f6704100fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:57.685 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.681+0000 7f6709b20700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67040fecb0 0x7f67040ff5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:57.685 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.681+0000 7f6709b20700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67041018b0 0x7f67040ffb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:57.685 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.681+0000 7f6709b20700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67041002d0 con 0x7f67041018b0 2023-11-16T16:38:57.685 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.681+0000 7f6709b20700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6704100440 con 0x7f67040fe320 2023-11-16T16:38:57.685 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.681+0000 7f6709b20700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67041a0080 con 0x7f67040fecb0 2023-11-16T16:38:57.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.681+0000 7f6703fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67041018b0 0x7f67040ffb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:57.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.682+0000 7f6703fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67041018b0 0x7f67040ffb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42996/0 (socket says 172.21.15.1:42996) 2023-11-16T16:38:57.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.682+0000 7f6703fff700 1 -- 172.21.15.1:0/546419433 learned_addr learned my addr 172.21.15.1:0/546419433 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:57.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.682+0000 7f67037fe700 1 --2- 172.21.15.1:0/546419433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67040fe320 0x7f6704100fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:57.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.682+0000 7f6702ffd700 1 --2- 172.21.15.1:0/546419433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67040fecb0 0x7f67040ff5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:57.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.682+0000 7f6703fff700 1 -- 172.21.15.1:0/546419433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67040fecb0 msgr2=0x7f67040ff5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:57.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.682+0000 7f6703fff700 1 --2- 172.21.15.1:0/546419433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67040fecb0 0x7f67040ff5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:57.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.682+0000 7f6703fff700 1 -- 172.21.15.1:0/546419433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67040fe320 msgr2=0x7f6704100fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:57.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.682+0000 7f6703fff700 1 --2- 172.21.15.1:0/546419433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67040fe320 0x7f6704100fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:57.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.682+0000 7f6703fff700 1 -- 172.21.15.1:0/546419433 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f40095f0 con 0x7f67041018b0 2023-11-16T16:38:57.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.682+0000 7f6703fff700 1 --2- 172.21.15.1:0/546419433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67041018b0 0x7f67040ffb30 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f66fc00ada0 tx=0x7f66fc00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:57.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.682+0000 7f6700ff9700 1 -- 172.21.15.1:0/546419433 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66fc00f070 con 0x7f67041018b0 2023-11-16T16:38:57.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.682+0000 7f6700ff9700 1 -- 172.21.15.1:0/546419433 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66fc007b30 con 0x7f67041018b0 2023-11-16T16:38:57.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.682+0000 7f6709b20700 1 -- 172.21.15.1:0/546419433 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f670410bc20 con 0x7f67041018b0 2023-11-16T16:38:57.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.682+0000 7f6700ff9700 1 -- 172.21.15.1:0/546419433 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66fc0053e0 con 0x7f67041018b0 2023-11-16T16:38:57.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.682+0000 7f6709b20700 1 -- 172.21.15.1:0/546419433 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f670410ba90 con 0x7f67041018b0 2023-11-16T16:38:57.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.683+0000 7f6709b20700 1 -- 172.21.15.1:0/546419433 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6704067260 con 0x7f67041018b0 2023-11-16T16:38:57.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.684+0000 7f6700ff9700 1 -- 172.21.15.1:0/546419433 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f66fc00c980 con 0x7f67041018b0 2023-11-16T16:38:57.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.685+0000 7f6700ff9700 1 --2- 172.21.15.1:0/546419433 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f66ec0a1150 0x7f66ec0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:57.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.685+0000 7f67037fe700 1 --2- 172.21.15.1:0/546419433 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f66ec0a1150 0x7f66ec0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:57.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.685+0000 7f6700ff9700 1 -- 172.21.15.1:0/546419433 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f66fc0cbeb0 con 0x7f67041018b0 2023-11-16T16:38:57.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.685+0000 7f67037fe700 1 --2- 172.21.15.1:0/546419433 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f66ec0a1150 0x7f66ec0a3600 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f66f400b5c0 tx=0x7f66f4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:57.691 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.689+0000 7f6700ff9700 1 -- 172.21.15.1:0/546419433 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f66fc095690 con 0x7f67041018b0 2023-11-16T16:38:57.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.886+0000 7f6709b20700 1 -- 172.21.15.1:0/546419433 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6704060f30 con 0x7f66ec0a1150 2023-11-16T16:38:57.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.898+0000 7f6700ff9700 1 -- 172.21.15.1:0/546419433 <== mgr.14152 v2:172.21.15.1:6800/2490729488 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344111 (secure 0 0 0) 0x7f6704060f30 con 0x7f66ec0a1150 2023-11-16T16:38:57.904 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:38:57.907 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.902+0000 7f6709b20700 1 -- 172.21.15.1:0/546419433 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f66ec0a1150 msgr2=0x7f66ec0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:57.907 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.902+0000 7f6709b20700 1 --2- 172.21.15.1:0/546419433 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f66ec0a1150 0x7f66ec0a3600 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f66f400b5c0 tx=0x7f66f4005fd0 comp rx=0 tx=0).stop 2023-11-16T16:38:57.907 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.902+0000 7f6709b20700 1 -- 172.21.15.1:0/546419433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67041018b0 msgr2=0x7f67040ffb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:57.907 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.902+0000 7f6709b20700 1 --2- 172.21.15.1:0/546419433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67041018b0 0x7f67040ffb30 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f66fc00ada0 tx=0x7f66fc00c330 comp rx=0 tx=0).stop 2023-11-16T16:38:57.908 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.902+0000 7f6709b20700 1 -- 172.21.15.1:0/546419433 shutdown_connections 2023-11-16T16:38:57.908 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.902+0000 7f6709b20700 1 --2- 172.21.15.1:0/546419433 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f66ec0a1150 0x7f66ec0a3600 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:57.908 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.902+0000 7f6709b20700 1 --2- 172.21.15.1:0/546419433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67041018b0 0x7f67040ffb30 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:57.908 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.902+0000 7f6709b20700 1 --2- 172.21.15.1:0/546419433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67040fe320 0x7f6704100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:57.908 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.902+0000 7f6709b20700 1 --2- 172.21.15.1:0/546419433 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f67040fecb0 0x7f67040ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:57.908 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.902+0000 7f6709b20700 1 -- 172.21.15.1:0/546419433 >> 172.21.15.1:0/546419433 conn(0x7f67040f9f50 msgr2=0x7f67040fc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:57.908 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.902+0000 7f6709b20700 1 -- 172.21.15.1:0/546419433 shutdown_connections 2023-11-16T16:38:57.908 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:57.902+0000 7f6709b20700 1 -- 172.21.15.1:0/546419433 wait complete. 2023-11-16T16:38:57.908 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2023-11-16T16:38:57.916 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2023-11-16T16:38:57.507580+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659450,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":745,"num_read_kb":496,"num_write":345,"num_write_kb":3058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1180848,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":782,"ondisk_log_size":782,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":80340,"kb_used_data":11380,"kb_used_omap":0,"kb_used_meta":68864,"kb_avail":1124861484,"statfs":{"total":1151940427776,"available":1151858159616,"internally_reserved":0,"allocated":11653120,"data_stored":7333612,"data_compressed":176451,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":0,"internal_metadata":70516736},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6015,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":38,"num_write":5,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003056"},"pg_stats":[{"pgid":"3.3f","version":"86'8","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.516764+0000","last_change":"2023-11-16T16:37:13.941551+0000","last_active":"2023-11-16T16:38:31.516764+0000","last_peered":"2023-11-16T16:38:31.516764+0000","last_clean":"2023-11-16T16:38:31.516764+0000","last_became_active":"2023-11-16T16:37:13.941190+0000","last_became_peered":"2023-11-16T16:37:13.941190+0000","last_unstale":"2023-11-16T16:38:31.516764+0000","last_undegraded":"2023-11-16T16:38:31.516764+0000","last_fullsized":"2023-11-16T16:38:31.516764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:42:03.561127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.510062+0000","last_change":"2023-11-16T16:37:13.942300+0000","last_active":"2023-11-16T16:38:31.510062+0000","last_peered":"2023-11-16T16:38:31.510062+0000","last_clean":"2023-11-16T16:38:31.510062+0000","last_became_active":"2023-11-16T16:37:13.941966+0000","last_became_peered":"2023-11-16T16:37:13.941966+0000","last_unstale":"2023-11-16T16:38:31.510062+0000","last_undegraded":"2023-11-16T16:38:31.510062+0000","last_fullsized":"2023-11-16T16:38:31.510062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:12:41.529896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.518118+0000","last_change":"2023-11-16T16:37:13.943461+0000","last_active":"2023-11-16T16:38:31.518118+0000","last_peered":"2023-11-16T16:38:31.518118+0000","last_clean":"2023-11-16T16:38:31.518118+0000","last_became_active":"2023-11-16T16:37:13.943165+0000","last_became_peered":"2023-11-16T16:37:13.943165+0000","last_unstale":"2023-11-16T16:38:31.518118+0000","last_undegraded":"2023-11-16T16:38:31.518118+0000","last_fullsized":"2023-11-16T16:38:31.518118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:17:44.567499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"86'8","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.526363+0000","last_change":"2023-11-16T16:37:13.940205+0000","last_active":"2023-11-16T16:38:31.526363+0000","last_peered":"2023-11-16T16:38:31.526363+0000","last_clean":"2023-11-16T16:38:31.526363+0000","last_became_active":"2023-11-16T16:37:13.939819+0000","last_became_peered":"2023-11-16T16:37:13.939819+0000","last_unstale":"2023-11-16T16:38:31.526363+0000","last_undegraded":"2023-11-16T16:38:31.526363+0000","last_fullsized":"2023-11-16T16:38:31.526363+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:52:42.183240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.508231+0000","last_change":"2023-11-16T16:37:13.939819+0000","last_active":"2023-11-16T16:38:31.508231+0000","last_peered":"2023-11-16T16:38:31.508231+0000","last_clean":"2023-11-16T16:38:31.508231+0000","last_became_active":"2023-11-16T16:37:13.939367+0000","last_became_peered":"2023-11-16T16:37:13.939367+0000","last_unstale":"2023-11-16T16:38:31.508231+0000","last_undegraded":"2023-11-16T16:38:31.508231+0000","last_fullsized":"2023-11-16T16:38:31.508231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:17:30.563282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.509876+0000","last_change":"2023-11-16T16:37:13.946613+0000","last_active":"2023-11-16T16:38:31.509876+0000","last_peered":"2023-11-16T16:38:31.509876+0000","last_clean":"2023-11-16T16:38:31.509876+0000","last_became_active":"2023-11-16T16:37:13.946281+0000","last_became_peered":"2023-11-16T16:37:13.946281+0000","last_unstale":"2023-11-16T16:38:31.509876+0000","last_undegraded":"2023-11-16T16:38:31.509876+0000","last_fullsized":"2023-11-16T16:38:31.509876+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:29:42.599960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.528658+0000","last_change":"2023-11-16T16:37:13.936899+0000","last_active":"2023-11-16T16:38:31.528658+0000","last_peered":"2023-11-16T16:38:31.528658+0000","last_clean":"2023-11-16T16:38:31.528658+0000","last_became_active":"2023-11-16T16:37:13.936543+0000","last_became_peered":"2023-11-16T16:37:13.936543+0000","last_unstale":"2023-11-16T16:38:31.528658+0000","last_undegraded":"2023-11-16T16:38:31.528658+0000","last_fullsized":"2023-11-16T16:38:31.528658+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:43:46.820689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"86'9","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.518738+0000","last_change":"2023-11-16T16:38:03.189467+0000","last_active":"2023-11-16T16:38:31.518738+0000","last_peered":"2023-11-16T16:38:31.518738+0000","last_clean":"2023-11-16T16:38:31.518738+0000","last_became_active":"2023-11-16T16:38:03.189017+0000","last_became_peered":"2023-11-16T16:38:03.189017+0000","last_unstale":"2023-11-16T16:38:31.518738+0000","last_undegraded":"2023-11-16T16:38:31.518738+0000","last_fullsized":"2023-11-16T16:38:31.518738+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:47:50.088436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"86'11","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.519665+0000","last_change":"2023-11-16T16:37:13.934105+0000","last_active":"2023-11-16T16:38:31.519665+0000","last_peered":"2023-11-16T16:38:31.519665+0000","last_clean":"2023-11-16T16:38:31.519665+0000","last_became_active":"2023-11-16T16:37:13.933887+0000","last_became_peered":"2023-11-16T16:37:13.933887+0000","last_unstale":"2023-11-16T16:38:31.519665+0000","last_undegraded":"2023-11-16T16:38:31.519665+0000","last_fullsized":"2023-11-16T16:38:31.519665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:53:26.890511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"86'16","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.519996+0000","last_change":"2023-11-16T16:38:03.191855+0000","last_active":"2023-11-16T16:38:31.519996+0000","last_peered":"2023-11-16T16:38:31.519996+0000","last_clean":"2023-11-16T16:38:31.519996+0000","last_became_active":"2023-11-16T16:38:03.191398+0000","last_became_peered":"2023-11-16T16:38:03.191398+0000","last_unstale":"2023-11-16T16:38:31.519996+0000","last_undegraded":"2023-11-16T16:38:31.519996+0000","last_fullsized":"2023-11-16T16:38:31.519996+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:09:44.431927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.529451+0000","last_change":"2023-11-16T16:37:13.941597+0000","last_active":"2023-11-16T16:38:31.529451+0000","last_peered":"2023-11-16T16:38:31.529451+0000","last_clean":"2023-11-16T16:38:31.529451+0000","last_became_active":"2023-11-16T16:37:13.941253+0000","last_became_peered":"2023-11-16T16:37:13.941253+0000","last_unstale":"2023-11-16T16:38:31.529451+0000","last_undegraded":"2023-11-16T16:38:31.529451+0000","last_fullsized":"2023-11-16T16:38:31.529451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:13:35.256952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.509176+0000","last_change":"2023-11-16T16:37:13.941673+0000","last_active":"2023-11-16T16:38:31.509176+0000","last_peered":"2023-11-16T16:38:31.509176+0000","last_clean":"2023-11-16T16:38:31.509176+0000","last_became_active":"2023-11-16T16:37:13.941345+0000","last_became_peered":"2023-11-16T16:37:13.941345+0000","last_unstale":"2023-11-16T16:38:31.509176+0000","last_undegraded":"2023-11-16T16:38:31.509176+0000","last_fullsized":"2023-11-16T16:38:31.509176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:17:51.018141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"86'10","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.510670+0000","last_change":"2023-11-16T16:37:13.947991+0000","last_active":"2023-11-16T16:38:31.510670+0000","last_peered":"2023-11-16T16:38:31.510670+0000","last_clean":"2023-11-16T16:38:31.510670+0000","last_became_active":"2023-11-16T16:37:13.947454+0000","last_became_peered":"2023-11-16T16:37:13.947454+0000","last_unstale":"2023-11-16T16:38:31.510670+0000","last_undegraded":"2023-11-16T16:38:31.510670+0000","last_fullsized":"2023-11-16T16:38:31.510670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:19:19.825458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.510762+0000","last_change":"2023-11-16T16:37:13.941502+0000","last_active":"2023-11-16T16:38:31.510762+0000","last_peered":"2023-11-16T16:38:31.510762+0000","last_clean":"2023-11-16T16:38:31.510762+0000","last_became_active":"2023-11-16T16:37:13.941191+0000","last_became_peered":"2023-11-16T16:37:13.941191+0000","last_unstale":"2023-11-16T16:38:31.510762+0000","last_undegraded":"2023-11-16T16:38:31.510762+0000","last_fullsized":"2023-11-16T16:38:31.510762+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:19:35.239602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.517384+0000","last_change":"2023-11-16T16:37:13.949198+0000","last_active":"2023-11-16T16:38:31.517384+0000","last_peered":"2023-11-16T16:38:31.517384+0000","last_clean":"2023-11-16T16:38:31.517384+0000","last_became_active":"2023-11-16T16:37:13.947147+0000","last_became_peered":"2023-11-16T16:37:13.947147+0000","last_unstale":"2023-11-16T16:38:31.517384+0000","last_undegraded":"2023-11-16T16:38:31.517384+0000","last_fullsized":"2023-11-16T16:38:31.517384+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:55:04.616796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.526473+0000","last_change":"2023-11-16T16:37:13.943915+0000","last_active":"2023-11-16T16:38:31.526473+0000","last_peered":"2023-11-16T16:38:31.526473+0000","last_clean":"2023-11-16T16:38:31.526473+0000","last_became_active":"2023-11-16T16:37:13.942995+0000","last_became_peered":"2023-11-16T16:37:13.942995+0000","last_unstale":"2023-11-16T16:38:31.526473+0000","last_undegraded":"2023-11-16T16:38:31.526473+0000","last_fullsized":"2023-11-16T16:38:31.526473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:23:05.411045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"86'8","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.516988+0000","last_change":"2023-11-16T16:37:13.941816+0000","last_active":"2023-11-16T16:38:31.516988+0000","last_peered":"2023-11-16T16:38:31.516988+0000","last_clean":"2023-11-16T16:38:31.516988+0000","last_became_active":"2023-11-16T16:37:13.941541+0000","last_became_peered":"2023-11-16T16:37:13.941541+0000","last_unstale":"2023-11-16T16:38:31.516988+0000","last_undegraded":"2023-11-16T16:38:31.516988+0000","last_fullsized":"2023-11-16T16:38:31.516988+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:53:11.142308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"86'10","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.518736+0000","last_change":"2023-11-16T16:37:13.940639+0000","last_active":"2023-11-16T16:38:31.518736+0000","last_peered":"2023-11-16T16:38:31.518736+0000","last_clean":"2023-11-16T16:38:31.518736+0000","last_became_active":"2023-11-16T16:37:13.940323+0000","last_became_peered":"2023-11-16T16:37:13.940323+0000","last_unstale":"2023-11-16T16:38:31.518736+0000","last_undegraded":"2023-11-16T16:38:31.518736+0000","last_fullsized":"2023-11-16T16:38:31.518736+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:32:03.882320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.513674+0000","last_change":"2023-11-16T16:37:13.934802+0000","last_active":"2023-11-16T16:38:31.513674+0000","last_peered":"2023-11-16T16:38:31.513674+0000","last_clean":"2023-11-16T16:38:31.513674+0000","last_became_active":"2023-11-16T16:37:13.934496+0000","last_became_peered":"2023-11-16T16:37:13.934496+0000","last_unstale":"2023-11-16T16:38:31.513674+0000","last_undegraded":"2023-11-16T16:38:31.513674+0000","last_fullsized":"2023-11-16T16:38:31.513674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:15:58.249424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'9","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185585+0000","last_change":"2023-11-16T16:37:13.499284+0000","last_active":"2023-11-16T16:38:03.185585+0000","last_peered":"2023-11-16T16:38:03.185585+0000","last_clean":"2023-11-16T16:38:03.185585+0000","last_became_active":"2023-11-16T16:37:13.498931+0000","last_became_peered":"2023-11-16T16:37:13.498931+0000","last_unstale":"2023-11-16T16:38:03.185585+0000","last_undegraded":"2023-11-16T16:38:03.185585+0000","last_fullsized":"2023-11-16T16:38:03.185585+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:53:33.061352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.514671+0000","last_change":"2023-11-16T16:37:13.939631+0000","last_active":"2023-11-16T16:38:31.514671+0000","last_peered":"2023-11-16T16:38:31.514671+0000","last_clean":"2023-11-16T16:38:31.514671+0000","last_became_active":"2023-11-16T16:37:13.939318+0000","last_became_peered":"2023-11-16T16:37:13.939318+0000","last_unstale":"2023-11-16T16:38:31.514671+0000","last_undegraded":"2023-11-16T16:38:31.514671+0000","last_fullsized":"2023-11-16T16:38:31.514671+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:11:33.421798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.183008+0000","last_change":"2023-11-16T16:37:13.495972+0000","last_active":"2023-11-16T16:38:03.183008+0000","last_peered":"2023-11-16T16:38:03.183008+0000","last_clean":"2023-11-16T16:38:03.183008+0000","last_became_active":"2023-11-16T16:37:13.495610+0000","last_became_peered":"2023-11-16T16:37:13.495610+0000","last_unstale":"2023-11-16T16:38:03.183008+0000","last_undegraded":"2023-11-16T16:38:03.183008+0000","last_fullsized":"2023-11-16T16:38:03.183008+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:47:20.115622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.518372+0000","last_change":"2023-11-16T16:37:13.935236+0000","last_active":"2023-11-16T16:38:31.518372+0000","last_peered":"2023-11-16T16:38:31.518372+0000","last_clean":"2023-11-16T16:38:31.518372+0000","last_became_active":"2023-11-16T16:37:13.934920+0000","last_became_peered":"2023-11-16T16:37:13.934920+0000","last_unstale":"2023-11-16T16:38:31.518372+0000","last_undegraded":"2023-11-16T16:38:31.518372+0000","last_fullsized":"2023-11-16T16:38:31.518372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:04:59.207169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"84'4","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.197657+0000","last_change":"2023-11-16T16:38:03.197657+0000","last_active":"2023-11-16T16:38:03.197657+0000","last_peered":"2023-11-16T16:38:03.197657+0000","last_clean":"2023-11-16T16:38:03.197657+0000","last_became_active":"2023-11-16T16:38:03.190952+0000","last_became_peered":"2023-11-16T16:38:03.190952+0000","last_unstale":"2023-11-16T16:38:03.197657+0000","last_undegraded":"2023-11-16T16:38:03.197657+0000","last_fullsized":"2023-11-16T16:38:03.197657+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:04:50.256412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.524482+0000","last_change":"2023-11-16T16:37:13.940838+0000","last_active":"2023-11-16T16:38:31.524482+0000","last_peered":"2023-11-16T16:38:31.524482+0000","last_clean":"2023-11-16T16:38:31.524482+0000","last_became_active":"2023-11-16T16:37:13.940538+0000","last_became_peered":"2023-11-16T16:37:13.940538+0000","last_unstale":"2023-11-16T16:38:31.524482+0000","last_undegraded":"2023-11-16T16:38:31.524482+0000","last_fullsized":"2023-11-16T16:38:31.524482+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:47:36.031628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.190408+0000","last_change":"2023-11-16T16:37:13.495114+0000","last_active":"2023-11-16T16:38:03.190408+0000","last_peered":"2023-11-16T16:38:03.190408+0000","last_clean":"2023-11-16T16:38:03.190408+0000","last_became_active":"2023-11-16T16:37:13.494807+0000","last_became_peered":"2023-11-16T16:37:13.494807+0000","last_unstale":"2023-11-16T16:38:03.190408+0000","last_undegraded":"2023-11-16T16:38:03.190408+0000","last_fullsized":"2023-11-16T16:38:03.190408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:35:31.040933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"86'15","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.519316+0000","last_change":"2023-11-16T16:37:13.943652+0000","last_active":"2023-11-16T16:38:31.519316+0000","last_peered":"2023-11-16T16:38:31.519316+0000","last_clean":"2023-11-16T16:38:31.519316+0000","last_became_active":"2023-11-16T16:37:13.943298+0000","last_became_peered":"2023-11-16T16:37:13.943298+0000","last_unstale":"2023-11-16T16:38:31.519316+0000","last_undegraded":"2023-11-16T16:38:31.519316+0000","last_fullsized":"2023-11-16T16:38:31.519316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:17:55.154731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'8","reported_seq":128,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:54.451248+0000","last_change":"2023-11-16T16:37:13.509801+0000","last_active":"2023-11-16T16:38:54.451248+0000","last_peered":"2023-11-16T16:38:54.451248+0000","last_clean":"2023-11-16T16:38:54.451248+0000","last_became_active":"2023-11-16T16:37:13.509389+0000","last_became_peered":"2023-11-16T16:37:13.509389+0000","last_unstale":"2023-11-16T16:38:54.451248+0000","last_undegraded":"2023-11-16T16:38:54.451248+0000","last_fullsized":"2023-11-16T16:38:54.451248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:46:55.751281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1423,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.512699+0000","last_change":"2023-11-16T16:37:13.933871+0000","last_active":"2023-11-16T16:38:31.512699+0000","last_peered":"2023-11-16T16:38:31.512699+0000","last_clean":"2023-11-16T16:38:31.512699+0000","last_became_active":"2023-11-16T16:37:13.933549+0000","last_became_peered":"2023-11-16T16:37:13.933549+0000","last_unstale":"2023-11-16T16:38:31.512699+0000","last_undegraded":"2023-11-16T16:38:31.512699+0000","last_fullsized":"2023-11-16T16:38:31.512699+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:35:18.854524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"84'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.187212+0000","last_change":"2023-11-16T16:37:13.501339+0000","last_active":"2023-11-16T16:38:03.187212+0000","last_peered":"2023-11-16T16:38:03.187212+0000","last_clean":"2023-11-16T16:38:03.187212+0000","last_became_active":"2023-11-16T16:37:13.500949+0000","last_became_peered":"2023-11-16T16:37:13.500949+0000","last_unstale":"2023-11-16T16:38:03.187212+0000","last_undegraded":"2023-11-16T16:38:03.187212+0000","last_fullsized":"2023-11-16T16:38:03.187212+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:56:34.503681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"86'8","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.514559+0000","last_change":"2023-11-16T16:38:03.192595+0000","last_active":"2023-11-16T16:38:31.514559+0000","last_peered":"2023-11-16T16:38:31.514559+0000","last_clean":"2023-11-16T16:38:31.514559+0000","last_became_active":"2023-11-16T16:38:03.192208+0000","last_became_peered":"2023-11-16T16:38:03.192208+0000","last_unstale":"2023-11-16T16:38:31.514559+0000","last_undegraded":"2023-11-16T16:38:31.514559+0000","last_fullsized":"2023-11-16T16:38:31.514559+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:30:24.627049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"84'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184789+0000","last_change":"2023-11-16T16:37:13.505481+0000","last_active":"2023-11-16T16:38:03.184789+0000","last_peered":"2023-11-16T16:38:03.184789+0000","last_clean":"2023-11-16T16:38:03.184789+0000","last_became_active":"2023-11-16T16:37:13.505085+0000","last_became_peered":"2023-11-16T16:37:13.505085+0000","last_unstale":"2023-11-16T16:38:03.184789+0000","last_undegraded":"2023-11-16T16:38:03.184789+0000","last_fullsized":"2023-11-16T16:38:03.184789+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:21:13.673154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"86'8","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.509656+0000","last_change":"2023-11-16T16:37:13.943985+0000","last_active":"2023-11-16T16:38:31.509656+0000","last_peered":"2023-11-16T16:38:31.509656+0000","last_clean":"2023-11-16T16:38:31.509656+0000","last_became_active":"2023-11-16T16:37:13.943628+0000","last_became_peered":"2023-11-16T16:37:13.943628+0000","last_unstale":"2023-11-16T16:38:31.509656+0000","last_undegraded":"2023-11-16T16:38:31.509656+0000","last_fullsized":"2023-11-16T16:38:31.509656+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:01:16.596531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"84'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184704+0000","last_change":"2023-11-16T16:37:13.498648+0000","last_active":"2023-11-16T16:38:03.184704+0000","last_peered":"2023-11-16T16:38:03.184704+0000","last_clean":"2023-11-16T16:38:03.184704+0000","last_became_active":"2023-11-16T16:37:13.497390+0000","last_became_peered":"2023-11-16T16:37:13.497390+0000","last_unstale":"2023-11-16T16:38:03.184704+0000","last_undegraded":"2023-11-16T16:38:03.184704+0000","last_fullsized":"2023-11-16T16:38:03.184704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:20:50.541938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"86'11","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.518643+0000","last_change":"2023-11-16T16:37:13.942337+0000","last_active":"2023-11-16T16:38:31.518643+0000","last_peered":"2023-11-16T16:38:31.518643+0000","last_clean":"2023-11-16T16:38:31.518643+0000","last_became_active":"2023-11-16T16:37:13.941982+0000","last_became_peered":"2023-11-16T16:37:13.941982+0000","last_unstale":"2023-11-16T16:38:31.518643+0000","last_undegraded":"2023-11-16T16:38:31.518643+0000","last_fullsized":"2023-11-16T16:38:31.518643+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:09:19.949618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'5","reported_seq":120,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:54.450706+0000","last_change":"2023-11-16T16:37:13.503690+0000","last_active":"2023-11-16T16:38:54.450706+0000","last_peered":"2023-11-16T16:38:54.450706+0000","last_clean":"2023-11-16T16:38:54.450706+0000","last_became_active":"2023-11-16T16:37:13.503203+0000","last_became_peered":"2023-11-16T16:37:13.503203+0000","last_unstale":"2023-11-16T16:38:54.450706+0000","last_undegraded":"2023-11-16T16:38:54.450706+0000","last_fullsized":"2023-11-16T16:38:54.450706+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:47:38.415047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"86'10","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.519722+0000","last_change":"2023-11-16T16:37:13.943975+0000","last_active":"2023-11-16T16:38:31.519722+0000","last_peered":"2023-11-16T16:38:31.519722+0000","last_clean":"2023-11-16T16:38:31.519722+0000","last_became_active":"2023-11-16T16:37:13.943611+0000","last_became_peered":"2023-11-16T16:37:13.943611+0000","last_unstale":"2023-11-16T16:38:31.519722+0000","last_undegraded":"2023-11-16T16:38:31.519722+0000","last_fullsized":"2023-11-16T16:38:31.519722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:37:02.660396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"84'2","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.344191+0000","last_change":"2023-11-16T16:38:03.344191+0000","last_active":"2023-11-16T16:38:03.344191+0000","last_peered":"2023-11-16T16:38:03.344191+0000","last_clean":"2023-11-16T16:38:03.344191+0000","last_became_active":"2023-11-16T16:38:03.189976+0000","last_became_peered":"2023-11-16T16:38:03.189976+0000","last_unstale":"2023-11-16T16:38:03.344191+0000","last_undegraded":"2023-11-16T16:38:03.344191+0000","last_fullsized":"2023-11-16T16:38:03.344191+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:04:56.889493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.513627+0000","last_change":"2023-11-16T16:37:13.949031+0000","last_active":"2023-11-16T16:38:31.513627+0000","last_peered":"2023-11-16T16:38:31.513627+0000","last_clean":"2023-11-16T16:38:31.513627+0000","last_became_active":"2023-11-16T16:37:13.946923+0000","last_became_peered":"2023-11-16T16:37:13.946923+0000","last_unstale":"2023-11-16T16:38:31.513627+0000","last_undegraded":"2023-11-16T16:38:31.513627+0000","last_fullsized":"2023-11-16T16:38:31.513627+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:04:43.521722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185398+0000","last_change":"2023-11-16T16:37:13.495003+0000","last_active":"2023-11-16T16:38:03.185398+0000","last_peered":"2023-11-16T16:38:03.185398+0000","last_clean":"2023-11-16T16:38:03.185398+0000","last_became_active":"2023-11-16T16:37:13.494563+0000","last_became_peered":"2023-11-16T16:37:13.494563+0000","last_unstale":"2023-11-16T16:38:03.185398+0000","last_undegraded":"2023-11-16T16:38:03.185398+0000","last_fullsized":"2023-11-16T16:38:03.185398+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:36:36.403721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"86'11","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.529044+0000","last_change":"2023-11-16T16:37:13.943736+0000","last_active":"2023-11-16T16:38:31.529044+0000","last_peered":"2023-11-16T16:38:31.529044+0000","last_clean":"2023-11-16T16:38:31.529044+0000","last_became_active":"2023-11-16T16:37:13.943391+0000","last_became_peered":"2023-11-16T16:37:13.943391+0000","last_unstale":"2023-11-16T16:38:31.529044+0000","last_undegraded":"2023-11-16T16:38:31.529044+0000","last_fullsized":"2023-11-16T16:38:31.529044+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:27:01.809877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'34","reported_seq":257,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:55.463894+0000","last_change":"2023-11-16T16:37:13.506508+0000","last_active":"2023-11-16T16:38:55.463894+0000","last_peered":"2023-11-16T16:38:55.463894+0000","last_clean":"2023-11-16T16:38:55.463894+0000","last_became_active":"2023-11-16T16:37:13.506045+0000","last_became_peered":"2023-11-16T16:37:13.506045+0000","last_unstale":"2023-11-16T16:38:55.463894+0000","last_undegraded":"2023-11-16T16:38:55.463894+0000","last_fullsized":"2023-11-16T16:38:55.463894+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:57:44.753422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64473,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":205,"num_read_kb":165,"num_write":35,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.525885+0000","last_change":"2023-11-16T16:37:13.939717+0000","last_active":"2023-11-16T16:38:31.525885+0000","last_peered":"2023-11-16T16:38:31.525885+0000","last_clean":"2023-11-16T16:38:31.525885+0000","last_became_active":"2023-11-16T16:37:13.939369+0000","last_became_peered":"2023-11-16T16:37:13.939369+0000","last_unstale":"2023-11-16T16:38:31.525885+0000","last_undegraded":"2023-11-16T16:38:31.525885+0000","last_fullsized":"2023-11-16T16:38:31.525885+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:10:31.454459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"84'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.188492+0000","last_change":"2023-11-16T16:37:13.506056+0000","last_active":"2023-11-16T16:38:03.188492+0000","last_peered":"2023-11-16T16:38:03.188492+0000","last_clean":"2023-11-16T16:38:03.188492+0000","last_became_active":"2023-11-16T16:37:13.505707+0000","last_became_peered":"2023-11-16T16:37:13.505707+0000","last_unstale":"2023-11-16T16:38:03.188492+0000","last_undegraded":"2023-11-16T16:38:03.188492+0000","last_fullsized":"2023-11-16T16:38:03.188492+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:29:47.242470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.525703+0000","last_change":"2023-11-16T16:37:13.947764+0000","last_active":"2023-11-16T16:38:31.525703+0000","last_peered":"2023-11-16T16:38:31.525703+0000","last_clean":"2023-11-16T16:38:31.525703+0000","last_became_active":"2023-11-16T16:37:13.947510+0000","last_became_peered":"2023-11-16T16:37:13.947510+0000","last_unstale":"2023-11-16T16:38:31.525703+0000","last_undegraded":"2023-11-16T16:38:31.525703+0000","last_fullsized":"2023-11-16T16:38:31.525703+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:41:59.672529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"84'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184939+0000","last_change":"2023-11-16T16:37:13.498178+0000","last_active":"2023-11-16T16:38:03.184939+0000","last_peered":"2023-11-16T16:38:03.184939+0000","last_clean":"2023-11-16T16:38:03.184939+0000","last_became_active":"2023-11-16T16:37:13.497312+0000","last_became_peered":"2023-11-16T16:37:13.497312+0000","last_unstale":"2023-11-16T16:38:03.184939+0000","last_undegraded":"2023-11-16T16:38:03.184939+0000","last_fullsized":"2023-11-16T16:38:03.184939+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:54:40.375590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"86'15","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.529728+0000","last_change":"2023-11-16T16:37:13.947854+0000","last_active":"2023-11-16T16:38:31.529728+0000","last_peered":"2023-11-16T16:38:31.529728+0000","last_clean":"2023-11-16T16:38:31.529728+0000","last_became_active":"2023-11-16T16:37:13.947530+0000","last_became_peered":"2023-11-16T16:37:13.947530+0000","last_unstale":"2023-11-16T16:38:31.529728+0000","last_undegraded":"2023-11-16T16:38:31.529728+0000","last_fullsized":"2023-11-16T16:38:31.529728+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:41:00.205641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'3","reported_seq":118,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:52.446665+0000","last_change":"2023-11-16T16:37:13.500127+0000","last_active":"2023-11-16T16:38:52.446665+0000","last_peered":"2023-11-16T16:38:52.446665+0000","last_clean":"2023-11-16T16:38:52.446665+0000","last_became_active":"2023-11-16T16:37:13.499758+0000","last_became_peered":"2023-11-16T16:37:13.499758+0000","last_unstale":"2023-11-16T16:38:52.446665+0000","last_undegraded":"2023-11-16T16:38:52.446665+0000","last_fullsized":"2023-11-16T16:38:52.446665+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:39:00.268144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.528126+0000","last_change":"2023-11-16T16:37:13.940962+0000","last_active":"2023-11-16T16:38:31.528126+0000","last_peered":"2023-11-16T16:38:31.528126+0000","last_clean":"2023-11-16T16:38:31.528126+0000","last_became_active":"2023-11-16T16:37:13.940450+0000","last_became_peered":"2023-11-16T16:37:13.940450+0000","last_unstale":"2023-11-16T16:38:31.528126+0000","last_undegraded":"2023-11-16T16:38:31.528126+0000","last_fullsized":"2023-11-16T16:38:31.528126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:28:45.946454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"84'15","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.228174+0000","last_change":"2023-11-16T16:38:03.228174+0000","last_active":"2023-11-16T16:38:03.228174+0000","last_peered":"2023-11-16T16:38:03.228174+0000","last_clean":"2023-11-16T16:38:03.228174+0000","last_became_active":"2023-11-16T16:38:03.192020+0000","last_became_peered":"2023-11-16T16:38:03.192020+0000","last_unstale":"2023-11-16T16:38:03.228174+0000","last_undegraded":"2023-11-16T16:38:03.228174+0000","last_fullsized":"2023-11-16T16:38:03.228174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:54:01.231005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.518623+0000","last_change":"2023-11-16T16:37:13.944878+0000","last_active":"2023-11-16T16:38:31.518623+0000","last_peered":"2023-11-16T16:38:31.518623+0000","last_clean":"2023-11-16T16:38:31.518623+0000","last_became_active":"2023-11-16T16:37:13.944580+0000","last_became_peered":"2023-11-16T16:37:13.944580+0000","last_unstale":"2023-11-16T16:38:31.518623+0000","last_undegraded":"2023-11-16T16:38:31.518623+0000","last_fullsized":"2023-11-16T16:38:31.518623+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:00:01.106243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.190628+0000","last_change":"2023-11-16T16:37:13.499139+0000","last_active":"2023-11-16T16:38:03.190628+0000","last_peered":"2023-11-16T16:38:03.190628+0000","last_clean":"2023-11-16T16:38:03.190628+0000","last_became_active":"2023-11-16T16:37:13.498755+0000","last_became_peered":"2023-11-16T16:37:13.498755+0000","last_unstale":"2023-11-16T16:38:03.190628+0000","last_undegraded":"2023-11-16T16:38:03.190628+0000","last_fullsized":"2023-11-16T16:38:03.190628+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T16:55:06.065509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.517260+0000","last_change":"2023-11-16T16:37:13.946187+0000","last_active":"2023-11-16T16:38:31.517260+0000","last_peered":"2023-11-16T16:38:31.517260+0000","last_clean":"2023-11-16T16:38:31.517260+0000","last_became_active":"2023-11-16T16:37:13.945887+0000","last_became_peered":"2023-11-16T16:37:13.945887+0000","last_unstale":"2023-11-16T16:38:31.517260+0000","last_undegraded":"2023-11-16T16:38:31.517260+0000","last_fullsized":"2023-11-16T16:38:31.517260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:41:31.726749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185150+0000","last_change":"2023-11-16T16:37:13.507150+0000","last_active":"2023-11-16T16:38:03.185150+0000","last_peered":"2023-11-16T16:38:03.185150+0000","last_clean":"2023-11-16T16:38:03.185150+0000","last_became_active":"2023-11-16T16:37:13.506814+0000","last_became_peered":"2023-11-16T16:37:13.506814+0000","last_unstale":"2023-11-16T16:38:03.185150+0000","last_undegraded":"2023-11-16T16:38:03.185150+0000","last_fullsized":"2023-11-16T16:38:03.185150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:54:45.031046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.506758+0000","last_change":"2023-11-16T16:37:13.947866+0000","last_active":"2023-11-16T16:38:31.506758+0000","last_peered":"2023-11-16T16:38:31.506758+0000","last_clean":"2023-11-16T16:38:31.506758+0000","last_became_active":"2023-11-16T16:37:13.947544+0000","last_became_peered":"2023-11-16T16:37:13.947544+0000","last_unstale":"2023-11-16T16:38:31.506758+0000","last_undegraded":"2023-11-16T16:38:31.506758+0000","last_fullsized":"2023-11-16T16:38:31.506758+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:23:27.220130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"84'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184212+0000","last_change":"2023-11-16T16:37:13.497800+0000","last_active":"2023-11-16T16:38:03.184212+0000","last_peered":"2023-11-16T16:38:03.184212+0000","last_clean":"2023-11-16T16:38:03.184212+0000","last_became_active":"2023-11-16T16:37:13.497414+0000","last_became_peered":"2023-11-16T16:37:13.497414+0000","last_unstale":"2023-11-16T16:38:03.184212+0000","last_undegraded":"2023-11-16T16:38:03.184212+0000","last_fullsized":"2023-11-16T16:38:03.184212+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:17:42.179250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.509186+0000","last_change":"2023-11-16T16:37:13.946780+0000","last_active":"2023-11-16T16:38:31.509186+0000","last_peered":"2023-11-16T16:38:31.509186+0000","last_clean":"2023-11-16T16:38:31.509186+0000","last_became_active":"2023-11-16T16:37:13.946441+0000","last_became_peered":"2023-11-16T16:37:13.946441+0000","last_unstale":"2023-11-16T16:38:31.509186+0000","last_undegraded":"2023-11-16T16:38:31.509186+0000","last_fullsized":"2023-11-16T16:38:31.509186+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:46:10.856707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"86'8","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.519819+0000","last_change":"2023-11-16T16:38:03.191887+0000","last_active":"2023-11-16T16:38:31.519819+0000","last_peered":"2023-11-16T16:38:31.519819+0000","last_clean":"2023-11-16T16:38:31.519819+0000","last_became_active":"2023-11-16T16:38:03.191402+0000","last_became_peered":"2023-11-16T16:38:03.191402+0000","last_unstale":"2023-11-16T16:38:31.519819+0000","last_undegraded":"2023-11-16T16:38:31.519819+0000","last_fullsized":"2023-11-16T16:38:31.519819+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:47:26.769740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"84'4","reported_seq":117,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:52.446018+0000","last_change":"2023-11-16T16:37:13.498129+0000","last_active":"2023-11-16T16:38:52.446018+0000","last_peered":"2023-11-16T16:38:52.446018+0000","last_clean":"2023-11-16T16:38:52.446018+0000","last_became_active":"2023-11-16T16:37:13.497765+0000","last_became_peered":"2023-11-16T16:37:13.497765+0000","last_unstale":"2023-11-16T16:38:52.446018+0000","last_undegraded":"2023-11-16T16:38:52.446018+0000","last_fullsized":"2023-11-16T16:38:52.446018+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:15:51.118817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":95,"num_read_kb":95,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"86'8","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.513701+0000","last_change":"2023-11-16T16:37:13.938417+0000","last_active":"2023-11-16T16:38:31.513701+0000","last_peered":"2023-11-16T16:38:31.513701+0000","last_clean":"2023-11-16T16:38:31.513701+0000","last_became_active":"2023-11-16T16:37:13.938114+0000","last_became_peered":"2023-11-16T16:37:13.938114+0000","last_unstale":"2023-11-16T16:38:31.513701+0000","last_undegraded":"2023-11-16T16:38:31.513701+0000","last_fullsized":"2023-11-16T16:38:31.513701+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:07:00.909431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"84'3","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.183990+0000","last_change":"2023-11-16T16:37:13.499045+0000","last_active":"2023-11-16T16:38:03.183990+0000","last_peered":"2023-11-16T16:38:03.183990+0000","last_clean":"2023-11-16T16:38:03.183990+0000","last_became_active":"2023-11-16T16:37:13.498598+0000","last_became_peered":"2023-11-16T16:37:13.498598+0000","last_unstale":"2023-11-16T16:38:03.183990+0000","last_undegraded":"2023-11-16T16:38:03.183990+0000","last_fullsized":"2023-11-16T16:38:03.183990+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:29:46.898930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":349,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.183816+0000","last_change":"2023-11-16T16:36:14.090213+0000","last_active":"2023-11-16T16:38:03.183816+0000","last_peered":"2023-11-16T16:38:03.183816+0000","last_clean":"2023-11-16T16:38:03.183816+0000","last_became_active":"2023-11-16T16:36:14.089742+0000","last_became_peered":"2023-11-16T16:36:14.089742+0000","last_unstale":"2023-11-16T16:38:03.183816+0000","last_undegraded":"2023-11-16T16:38:03.183816+0000","last_fullsized":"2023-11-16T16:38:03.183816+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:33:16.178716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:33:16.178716+0000","last_clean_scrub_stamp":"2023-11-16T16:33:16.178716+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:34:52.910521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.511992+0000","last_change":"2023-11-16T16:37:13.945380+0000","last_active":"2023-11-16T16:38:31.511992+0000","last_peered":"2023-11-16T16:38:31.511992+0000","last_clean":"2023-11-16T16:38:31.511992+0000","last_became_active":"2023-11-16T16:37:13.945038+0000","last_became_peered":"2023-11-16T16:37:13.945038+0000","last_unstale":"2023-11-16T16:38:31.511992+0000","last_undegraded":"2023-11-16T16:38:31.511992+0000","last_fullsized":"2023-11-16T16:38:31.511992+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:29:11.812243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"84'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184055+0000","last_change":"2023-11-16T16:37:13.497586+0000","last_active":"2023-11-16T16:38:03.184055+0000","last_peered":"2023-11-16T16:38:03.184055+0000","last_clean":"2023-11-16T16:38:03.184055+0000","last_became_active":"2023-11-16T16:37:13.497233+0000","last_became_peered":"2023-11-16T16:37:13.497233+0000","last_unstale":"2023-11-16T16:38:03.184055+0000","last_undegraded":"2023-11-16T16:38:03.184055+0000","last_fullsized":"2023-11-16T16:38:03.184055+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:47:21.097628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"84'1","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.191390+0000","last_change":"2023-11-16T16:37:13.500621+0000","last_active":"2023-11-16T16:38:03.191390+0000","last_peered":"2023-11-16T16:38:03.191390+0000","last_clean":"2023-11-16T16:38:03.191390+0000","last_became_active":"2023-11-16T16:37:13.500265+0000","last_became_peered":"2023-11-16T16:37:13.500265+0000","last_unstale":"2023-11-16T16:38:03.191390+0000","last_undegraded":"2023-11-16T16:38:03.191390+0000","last_fullsized":"2023-11-16T16:38:03.191390+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:22:59.748224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.509709+0000","last_change":"2023-11-16T16:37:13.930992+0000","last_active":"2023-11-16T16:38:31.509709+0000","last_peered":"2023-11-16T16:38:31.509709+0000","last_clean":"2023-11-16T16:38:31.509709+0000","last_became_active":"2023-11-16T16:37:13.930641+0000","last_became_peered":"2023-11-16T16:37:13.930641+0000","last_unstale":"2023-11-16T16:38:31.509709+0000","last_undegraded":"2023-11-16T16:38:31.509709+0000","last_fullsized":"2023-11-16T16:38:31.509709+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:24:49.314642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'6","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.185439+0000","last_change":"2023-11-16T16:37:13.498637+0000","last_active":"2023-11-16T16:38:03.185439+0000","last_peered":"2023-11-16T16:38:03.185439+0000","last_clean":"2023-11-16T16:38:03.185439+0000","last_became_active":"2023-11-16T16:37:13.498341+0000","last_became_peered":"2023-11-16T16:37:13.498341+0000","last_unstale":"2023-11-16T16:38:03.185439+0000","last_undegraded":"2023-11-16T16:38:03.185439+0000","last_fullsized":"2023-11-16T16:38:03.185439+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:30:04.558815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.526478+0000","last_change":"2023-11-16T16:37:13.944142+0000","last_active":"2023-11-16T16:38:31.526478+0000","last_peered":"2023-11-16T16:38:31.526478+0000","last_clean":"2023-11-16T16:38:31.526478+0000","last_became_active":"2023-11-16T16:37:13.943758+0000","last_became_peered":"2023-11-16T16:37:13.943758+0000","last_unstale":"2023-11-16T16:38:31.526478+0000","last_undegraded":"2023-11-16T16:38:31.526478+0000","last_fullsized":"2023-11-16T16:38:31.526478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:57:58.601449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"84'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184558+0000","last_change":"2023-11-16T16:37:13.496900+0000","last_active":"2023-11-16T16:38:03.184558+0000","last_peered":"2023-11-16T16:38:03.184558+0000","last_clean":"2023-11-16T16:38:03.184558+0000","last_became_active":"2023-11-16T16:37:13.496528+0000","last_became_peered":"2023-11-16T16:37:13.496528+0000","last_unstale":"2023-11-16T16:38:03.184558+0000","last_undegraded":"2023-11-16T16:38:03.184558+0000","last_fullsized":"2023-11-16T16:38:03.184558+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T18:56:42.304030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.507960+0000","last_change":"2023-11-16T16:37:13.943104+0000","last_active":"2023-11-16T16:38:31.507960+0000","last_peered":"2023-11-16T16:38:31.507960+0000","last_clean":"2023-11-16T16:38:31.507960+0000","last_became_active":"2023-11-16T16:37:13.942774+0000","last_became_peered":"2023-11-16T16:37:13.942774+0000","last_unstale":"2023-11-16T16:38:31.507960+0000","last_undegraded":"2023-11-16T16:38:31.507960+0000","last_fullsized":"2023-11-16T16:38:31.507960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:24:11.162929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.188270+0000","last_change":"2023-11-16T16:37:13.505994+0000","last_active":"2023-11-16T16:38:03.188270+0000","last_peered":"2023-11-16T16:38:03.188270+0000","last_clean":"2023-11-16T16:38:03.188270+0000","last_became_active":"2023-11-16T16:37:13.505663+0000","last_became_peered":"2023-11-16T16:37:13.505663+0000","last_unstale":"2023-11-16T16:38:03.188270+0000","last_undegraded":"2023-11-16T16:38:03.188270+0000","last_fullsized":"2023-11-16T16:38:03.188270+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:06:13.680491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.479124+0000","last_change":"2023-11-16T16:37:13.941970+0000","last_active":"2023-11-16T16:38:31.479124+0000","last_peered":"2023-11-16T16:38:31.479124+0000","last_clean":"2023-11-16T16:38:31.479124+0000","last_became_active":"2023-11-16T16:37:13.941649+0000","last_became_peered":"2023-11-16T16:37:13.941649+0000","last_unstale":"2023-11-16T16:38:31.479124+0000","last_undegraded":"2023-11-16T16:38:31.479124+0000","last_fullsized":"2023-11-16T16:38:31.479124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:23:17.814240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.182332+0000","last_change":"2023-11-16T16:37:13.503429+0000","last_active":"2023-11-16T16:38:03.182332+0000","last_peered":"2023-11-16T16:38:03.182332+0000","last_clean":"2023-11-16T16:38:03.182332+0000","last_became_active":"2023-11-16T16:37:13.503114+0000","last_became_peered":"2023-11-16T16:37:13.503114+0000","last_unstale":"2023-11-16T16:38:03.182332+0000","last_undegraded":"2023-11-16T16:38:03.182332+0000","last_fullsized":"2023-11-16T16:38:03.182332+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:57:19.910282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.513996+0000","last_change":"2023-11-16T16:37:13.941225+0000","last_active":"2023-11-16T16:38:31.513996+0000","last_peered":"2023-11-16T16:38:31.513996+0000","last_clean":"2023-11-16T16:38:31.513996+0000","last_became_active":"2023-11-16T16:37:13.940964+0000","last_became_peered":"2023-11-16T16:37:13.940964+0000","last_unstale":"2023-11-16T16:38:31.513996+0000","last_undegraded":"2023-11-16T16:38:31.513996+0000","last_fullsized":"2023-11-16T16:38:31.513996+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:21:58.992812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'1","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.188151+0000","last_change":"2023-11-16T16:37:13.510154+0000","last_active":"2023-11-16T16:38:03.188151+0000","last_peered":"2023-11-16T16:38:03.188151+0000","last_clean":"2023-11-16T16:38:03.188151+0000","last_became_active":"2023-11-16T16:37:13.509790+0000","last_became_peered":"2023-11-16T16:37:13.509790+0000","last_unstale":"2023-11-16T16:38:03.188151+0000","last_undegraded":"2023-11-16T16:38:03.188151+0000","last_fullsized":"2023-11-16T16:38:03.188151+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:09:32.999016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"86'8","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.509433+0000","last_change":"2023-11-16T16:37:13.942203+0000","last_active":"2023-11-16T16:38:31.509433+0000","last_peered":"2023-11-16T16:38:31.509433+0000","last_clean":"2023-11-16T16:38:31.509433+0000","last_became_active":"2023-11-16T16:37:13.941766+0000","last_became_peered":"2023-11-16T16:37:13.941766+0000","last_unstale":"2023-11-16T16:38:31.509433+0000","last_undegraded":"2023-11-16T16:38:31.509433+0000","last_fullsized":"2023-11-16T16:38:31.509433+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:50:04.530728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.190859+0000","last_change":"2023-11-16T16:37:13.509854+0000","last_active":"2023-11-16T16:38:03.190859+0000","last_peered":"2023-11-16T16:38:03.190859+0000","last_clean":"2023-11-16T16:38:03.190859+0000","last_became_active":"2023-11-16T16:37:13.509466+0000","last_became_peered":"2023-11-16T16:37:13.509466+0000","last_unstale":"2023-11-16T16:38:03.190859+0000","last_undegraded":"2023-11-16T16:38:03.190859+0000","last_fullsized":"2023-11-16T16:38:03.190859+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:38:13.537860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.529514+0000","last_change":"2023-11-16T16:37:13.947906+0000","last_active":"2023-11-16T16:38:31.529514+0000","last_peered":"2023-11-16T16:38:31.529514+0000","last_clean":"2023-11-16T16:38:31.529514+0000","last_became_active":"2023-11-16T16:37:13.947602+0000","last_became_peered":"2023-11-16T16:37:13.947602+0000","last_unstale":"2023-11-16T16:38:31.529514+0000","last_undegraded":"2023-11-16T16:38:31.529514+0000","last_fullsized":"2023-11-16T16:38:31.529514+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:16:45.611613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.188948+0000","last_change":"2023-11-16T16:37:13.506108+0000","last_active":"2023-11-16T16:38:03.188948+0000","last_peered":"2023-11-16T16:38:03.188948+0000","last_clean":"2023-11-16T16:38:03.188948+0000","last_became_active":"2023-11-16T16:37:13.505785+0000","last_became_peered":"2023-11-16T16:37:13.505785+0000","last_unstale":"2023-11-16T16:38:03.188948+0000","last_undegraded":"2023-11-16T16:38:03.188948+0000","last_fullsized":"2023-11-16T16:38:03.188948+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:11:02.793557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.525780+0000","last_change":"2023-11-16T16:37:13.935863+0000","last_active":"2023-11-16T16:38:31.525780+0000","last_peered":"2023-11-16T16:38:31.525780+0000","last_clean":"2023-11-16T16:38:31.525780+0000","last_became_active":"2023-11-16T16:37:13.935399+0000","last_became_peered":"2023-11-16T16:37:13.935399+0000","last_unstale":"2023-11-16T16:38:31.525780+0000","last_undegraded":"2023-11-16T16:38:31.525780+0000","last_fullsized":"2023-11-16T16:38:31.525780+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:45:56.613743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.184907+0000","last_change":"2023-11-16T16:37:13.498367+0000","last_active":"2023-11-16T16:38:03.184907+0000","last_peered":"2023-11-16T16:38:03.184907+0000","last_clean":"2023-11-16T16:38:03.184907+0000","last_became_active":"2023-11-16T16:37:13.497039+0000","last_became_peered":"2023-11-16T16:37:13.497039+0000","last_unstale":"2023-11-16T16:38:03.184907+0000","last_undegraded":"2023-11-16T16:38:03.184907+0000","last_fullsized":"2023-11-16T16:38:03.184907+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:57:46.571219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'7","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.528964+0000","last_change":"2023-11-16T16:37:13.940617+0000","last_active":"2023-11-16T16:38:31.528964+0000","last_peered":"2023-11-16T16:38:31.528964+0000","last_clean":"2023-11-16T16:38:31.528964+0000","last_became_active":"2023-11-16T16:37:13.940264+0000","last_became_peered":"2023-11-16T16:37:13.940264+0000","last_unstale":"2023-11-16T16:38:31.528964+0000","last_undegraded":"2023-11-16T16:38:31.528964+0000","last_fullsized":"2023-11-16T16:38:31.528964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T20:59:36.630848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:03.193922+0000","last_change":"2023-11-16T16:38:03.193922+0000","last_active":"2023-11-16T16:38:03.193922+0000","last_peered":"2023-11-16T16:38:03.193922+0000","last_clean":"2023-11-16T16:38:03.193922+0000","last_became_active":"2023-11-16T16:38:03.190364+0000","last_became_peered":"2023-11-16T16:38:03.190364+0000","last_unstale":"2023-11-16T16:38:03.193922+0000","last_undegraded":"2023-11-16T16:38:03.193922+0000","last_fullsized":"2023-11-16T16:38:03.193922+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:12.473444+0000","last_clean_scrub_stamp":"2023-11-16T16:37:12.473444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:59:04.681132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.529517+0000","last_change":"2023-11-16T16:37:13.943096+0000","last_active":"2023-11-16T16:38:31.529517+0000","last_peered":"2023-11-16T16:38:31.529517+0000","last_clean":"2023-11-16T16:38:31.529517+0000","last_became_active":"2023-11-16T16:37:13.942745+0000","last_became_peered":"2023-11-16T16:37:13.942745+0000","last_unstale":"2023-11-16T16:38:31.529517+0000","last_undegraded":"2023-11-16T16:38:31.529517+0000","last_fullsized":"2023-11-16T16:38:31.529517+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:02:12.721195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.515101+0000","last_change":"2023-11-16T16:37:13.938087+0000","last_active":"2023-11-16T16:38:31.515101+0000","last_peered":"2023-11-16T16:38:31.515101+0000","last_clean":"2023-11-16T16:38:31.515101+0000","last_became_active":"2023-11-16T16:37:13.937780+0000","last_became_peered":"2023-11-16T16:37:13.937780+0000","last_unstale":"2023-11-16T16:38:31.515101+0000","last_undegraded":"2023-11-16T16:38:31.515101+0000","last_fullsized":"2023-11-16T16:38:31.515101+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:05:14.399850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"86'12","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.519834+0000","last_change":"2023-11-16T16:37:13.939000+0000","last_active":"2023-11-16T16:38:31.519834+0000","last_peered":"2023-11-16T16:38:31.519834+0000","last_clean":"2023-11-16T16:38:31.519834+0000","last_became_active":"2023-11-16T16:37:13.938585+0000","last_became_peered":"2023-11-16T16:37:13.938585+0000","last_unstale":"2023-11-16T16:38:31.519834+0000","last_undegraded":"2023-11-16T16:38:31.519834+0000","last_fullsized":"2023-11-16T16:38:31.519834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T01:13:17.753666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"86'8","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.513239+0000","last_change":"2023-11-16T16:37:13.937696+0000","last_active":"2023-11-16T16:38:31.513239+0000","last_peered":"2023-11-16T16:38:31.513239+0000","last_clean":"2023-11-16T16:38:31.513239+0000","last_became_active":"2023-11-16T16:37:13.936920+0000","last_became_peered":"2023-11-16T16:37:13.936920+0000","last_unstale":"2023-11-16T16:38:31.513239+0000","last_undegraded":"2023-11-16T16:38:31.513239+0000","last_fullsized":"2023-11-16T16:38:31.513239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:55:00.163322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"86'7","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.517747+0000","last_change":"2023-11-16T16:38:03.189679+0000","last_active":"2023-11-16T16:38:31.517747+0000","last_peered":"2023-11-16T16:38:31.517747+0000","last_clean":"2023-11-16T16:38:31.517747+0000","last_became_active":"2023-11-16T16:38:03.189292+0000","last_became_peered":"2023-11-16T16:38:03.189292+0000","last_unstale":"2023-11-16T16:38:31.517747+0000","last_undegraded":"2023-11-16T16:38:31.517747+0000","last_fullsized":"2023-11-16T16:38:31.517747+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T00:24:48.303402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.507243+0000","last_change":"2023-11-16T16:37:13.949940+0000","last_active":"2023-11-16T16:38:31.507243+0000","last_peered":"2023-11-16T16:38:31.507243+0000","last_clean":"2023-11-16T16:38:31.507243+0000","last_became_active":"2023-11-16T16:37:13.949464+0000","last_became_peered":"2023-11-16T16:37:13.949464+0000","last_unstale":"2023-11-16T16:38:31.507243+0000","last_undegraded":"2023-11-16T16:38:31.507243+0000","last_fullsized":"2023-11-16T16:38:31.507243+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:05:15.768098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.514662+0000","last_change":"2023-11-16T16:37:13.940534+0000","last_active":"2023-11-16T16:38:31.514662+0000","last_peered":"2023-11-16T16:38:31.514662+0000","last_clean":"2023-11-16T16:38:31.514662+0000","last_became_active":"2023-11-16T16:37:13.940174+0000","last_became_peered":"2023-11-16T16:37:13.940174+0000","last_unstale":"2023-11-16T16:38:31.514662+0000","last_undegraded":"2023-11-16T16:38:31.514662+0000","last_fullsized":"2023-11-16T16:38:31.514662+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T23:29:35.708635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"86'15","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.516894+0000","last_change":"2023-11-16T16:37:13.937519+0000","last_active":"2023-11-16T16:38:31.516894+0000","last_peered":"2023-11-16T16:38:31.516894+0000","last_clean":"2023-11-16T16:38:31.516894+0000","last_became_active":"2023-11-16T16:37:13.937175+0000","last_became_peered":"2023-11-16T16:37:13.937175+0000","last_unstale":"2023-11-16T16:38:31.516894+0000","last_undegraded":"2023-11-16T16:38:31.516894+0000","last_fullsized":"2023-11-16T16:38:31.516894+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T17:25:23.849158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.503336+0000","last_change":"2023-11-16T16:37:13.951506+0000","last_active":"2023-11-16T16:38:31.503336+0000","last_peered":"2023-11-16T16:38:31.503336+0000","last_clean":"2023-11-16T16:38:31.503336+0000","last_became_active":"2023-11-16T16:37:13.951192+0000","last_became_peered":"2023-11-16T16:37:13.951192+0000","last_unstale":"2023-11-16T16:38:31.503336+0000","last_undegraded":"2023-11-16T16:38:31.503336+0000","last_fullsized":"2023-11-16T16:38:31.503336+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T22:17:38.223843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.514590+0000","last_change":"2023-11-16T16:37:13.935614+0000","last_active":"2023-11-16T16:38:31.514590+0000","last_peered":"2023-11-16T16:38:31.514590+0000","last_clean":"2023-11-16T16:38:31.514590+0000","last_became_active":"2023-11-16T16:37:13.935237+0000","last_became_peered":"2023-11-16T16:37:13.935237+0000","last_unstale":"2023-11-16T16:38:31.514590+0000","last_undegraded":"2023-11-16T16:38:31.514590+0000","last_fullsized":"2023-11-16T16:38:31.514590+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T02:30:01.772328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"86'8","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.514616+0000","last_change":"2023-11-16T16:37:13.939387+0000","last_active":"2023-11-16T16:38:31.514616+0000","last_peered":"2023-11-16T16:38:31.514616+0000","last_clean":"2023-11-16T16:38:31.514616+0000","last_became_active":"2023-11-16T16:37:13.939082+0000","last_became_peered":"2023-11-16T16:37:13.939082+0000","last_unstale":"2023-11-16T16:38:31.514616+0000","last_undegraded":"2023-11-16T16:38:31.514616+0000","last_fullsized":"2023-11-16T16:38:31.514616+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T19:28:19.026881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"86'10","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.527049+0000","last_change":"2023-11-16T16:37:13.936381+0000","last_active":"2023-11-16T16:38:31.527049+0000","last_peered":"2023-11-16T16:38:31.527049+0000","last_clean":"2023-11-16T16:38:31.527049+0000","last_became_active":"2023-11-16T16:37:13.936077+0000","last_became_peered":"2023-11-16T16:37:13.936077+0000","last_unstale":"2023-11-16T16:38:31.527049+0000","last_undegraded":"2023-11-16T16:38:31.527049+0000","last_fullsized":"2023-11-16T16:38:31.527049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T03:20:31.015344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"86'11","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.526052+0000","last_change":"2023-11-16T16:37:13.934005+0000","last_active":"2023-11-16T16:38:31.526052+0000","last_peered":"2023-11-16T16:38:31.526052+0000","last_clean":"2023-11-16T16:38:31.526052+0000","last_became_active":"2023-11-16T16:37:13.933633+0000","last_became_peered":"2023-11-16T16:37:13.933633+0000","last_unstale":"2023-11-16T16:38:31.526052+0000","last_undegraded":"2023-11-16T16:38:31.526052+0000","last_fullsized":"2023-11-16T16:38:31.526052+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-17T21:41:36.990242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"86'8","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-16T16:38:31.526966+0000","last_change":"2023-11-16T16:37:13.946495+0000","last_active":"2023-11-16T16:38:31.526966+0000","last_peered":"2023-11-16T16:38:31.526966+0000","last_clean":"2023-11-16T16:38:31.526966+0000","last_became_active":"2023-11-16T16:37:13.946148+0000","last_became_peered":"2023-11-16T16:37:13.946148+0000","last_unstale":"2023-11-16T16:38:31.526966+0000","last_undegraded":"2023-11-16T16:38:31.526966+0000","last_fullsized":"2023-11-16T16:38:31.526966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-16T16:37:13.477527+0000","last_clean_scrub_stamp":"2023-11-16T16:37:13.477527+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-18T04:25:08.609723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":3542208,"data_compressed":211753,"data_compressed_allocated":270336,"data_compressed_original":3522560,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68836,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":609,"num_read_kb":377,"num_write":162,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":206508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416416,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6104,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93739048,"statfs":{"total":95995035648,"available":95988785152,"internally_reserved":0,"allocated":933888,"data_stored":447021,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58599999999999997,"15min":0.58499999999999996},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":2.6150000000000002,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.624,"15min":0.625},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.65100000000000002}]},{"osd":1,"last update":"Thu Nov 16 16:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.64600000000000002,"15min":0.63800000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89400000000000002,"5min":1.026,"15min":1.026},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.66700000000000004,"15min":0.66500000000000004},"min":{"1min":0.44700000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.97499999999999998,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.63400000000000001}]},{"osd":2,"last update":"Thu Nov 16 16:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.58799999999999997,"15min":0.59499999999999997},"min":{"1min":0.41699999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84699999999999998,"5min":1.087,"15min":1.087},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56100000000000005,"15min":0.56200000000000006},"min":{"1min":0.31900000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.86499999999999999,"5min":0.88,"15min":0.88},"last":0.38300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":4,"last update":"Thu Nov 16 16:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.746,"15min":0.748},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.94199999999999995,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.755,"15min":0.75700000000000001},"min":{"1min":0.54500000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0389999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.66400000000000003}]},{"osd":5,"last update":"Thu Nov 16 16:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.69999999999999996,"15min":0.70599999999999996},"min":{"1min":0.379,"5min":0.37,"15min":0.37},"max":{"1min":0.90400000000000003,"5min":1.722,"15min":1.722},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.73399999999999999,"15min":0.74299999999999999},"min":{"1min":0.436,"5min":0.378,"15min":0.378},"max":{"1min":0.97399999999999998,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.53500000000000003}]},{"osd":6,"last update":"Thu Nov 16 16:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.76000000000000001,"15min":0.76600000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0740000000000001,"5min":1.516,"15min":1.516},"last":0.58699999999999997},{"interface":"front","average":{"1min":1.254,"5min":0.91800000000000004,"15min":0.86199999999999999},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":11.007,"5min":11.007,"15min":11.007},"last":0.56399999999999995}]},{"osd":7,"last update":"Thu Nov 16 16:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.61099999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0249999999999999,"5min":1.399,"15min":1.399},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.1279999999999999,"5min":1.363,"15min":1.363},"last":0.748}]},{"osd":8,"last update":"Thu Nov 16 16:38:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.70599999999999996}]},{"osd":9,"last update":"Thu Nov 16 16:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.86399999999999999,"15min":0.86499999999999999},"min":{"1min":0.64700000000000002,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.399,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.347,"5min":0.95399999999999996,"15min":0.88800000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":10.83,"5min":10.83,"15min":10.83},"last":0.77200000000000002}]},{"osd":10,"last update":"Thu Nov 16 16:38:33 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.89300000000000002,"15min":0.89200000000000002},"min":{"1min":0.65400000000000003,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.169,"5min":1.427,"15min":1.427},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.89100000000000001,"15min":0.89800000000000002},"min":{"1min":0.65000000000000002,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.073,"5min":1.863,"15min":1.863},"last":0.78000000000000003}]}]},{"osd":10,"up_from":72,"seq":309237645348,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6564,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738588,"statfs":{"total":95995035648,"available":95988314112,"internally_reserved":0,"allocated":1077248,"data_stored":1102490,"data_compressed":41378,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:38:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.78700000000000003}]},{"osd":1,"last update":"Thu Nov 16 16:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.49199999999999999,"15min":0.49299999999999999},"min":{"1min":0.34399999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.67900000000000005,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.52800000000000002,"15min":0.52700000000000002},"min":{"1min":0.33300000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.86499999999999999,"5min":0.879,"15min":0.879},"last":0.41099999999999998}]},{"osd":2,"last update":"Thu Nov 16 16:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64300000000000002,"15min":0.64100000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89300000000000002,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.70899999999999996,"15min":0.71599999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.80400000000000005,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.53100000000000003}]},{"osd":3,"last update":"Thu Nov 16 16:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65900000000000003,"15min":0.66500000000000004},"min":{"1min":0.44,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82399999999999995,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.625,"5min":0.67300000000000004,"15min":0.68100000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.85699999999999998,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.59099999999999997}]},{"osd":4,"last update":"Thu Nov 16 16:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63700000000000001,"15min":0.63500000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.94399999999999995,"5min":1.141,"15min":1.141},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.58299999999999996,"15min":0.57199999999999995},"min":{"1min":0.48199999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.91600000000000004,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.38200000000000001}]},{"osd":5,"last update":"Thu Nov 16 16:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.752,"15min":0.754},"min":{"1min":0.57699999999999996,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.95499999999999996,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.754,"15min":0.75900000000000001},"min":{"1min":0.58399999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.94099999999999995,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.46999999999999997}]},{"osd":6,"last update":"Thu Nov 16 16:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.76200000000000001,"15min":0.76800000000000002},"min":{"1min":0.498,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0029999999999999,"5min":1.282,"15min":1.282},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.77700000000000002,"15min":0.77600000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0149999999999999,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.70999999999999996}]},{"osd":7,"last update":"Thu Nov 16 16:37:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.505}]},{"osd":8,"last update":"Thu Nov 16 16:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.83699999999999997,"15min":0.83499999999999996},"min":{"1min":0.60199999999999998,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0760000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.61599999999999999,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0169999999999999,"5min":1.5,"15min":1.5},"last":0.625}]},{"osd":9,"last update":"Thu Nov 16 16:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0289999999999999,"5min":1.337,"15min":1.337},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81799999999999995,"15min":0.81899999999999995},"min":{"1min":0.53000000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0369999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.69199999999999995}]},{"osd":11,"last update":"Thu Nov 16 16:38:38 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.83899999999999997,"15min":0.84199999999999997},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.0900000000000001,"5min":1.2,"15min":1.2},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.83799999999999997,"15min":0.83099999999999996},"min":{"1min":0.67000000000000004,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.1180000000000001,"5min":1.236,"15min":1.236},"last":0.65400000000000003}]}]},{"osd":9,"up_from":67,"seq":287762808871,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6240,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":93738912,"statfs":{"total":95995035648,"available":95988645888,"internally_reserved":0,"allocated":942080,"data_stored":447145,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.44400000000000001,"15min":0.46100000000000002},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.47199999999999998,"15min":0.48199999999999998},"min":{"1min":0.28199999999999997,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.40400000000000003}]},{"osd":1,"last update":"Thu Nov 16 16:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59099999999999997,"15min":0.60599999999999998},"min":{"1min":0.34699999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.60099999999999998,"15min":0.627},"min":{"1min":0.41899999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.60199999999999998}]},{"osd":2,"last update":"Thu Nov 16 16:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.623,"15min":0.64300000000000002},"min":{"1min":0.42499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.62,"15min":0.63800000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77600000000000002,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.66900000000000004}]},{"osd":3,"last update":"Thu Nov 16 16:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54100000000000004,"15min":0.56100000000000005},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":0.496},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53700000000000003,"15min":0.56799999999999995},"min":{"1min":0.26900000000000002,"5min":0.25,"15min":0.25},"max":{"1min":2.2490000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":0.47499999999999998}]},{"osd":4,"last update":"Thu Nov 16 16:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.64400000000000002,"15min":0.67500000000000004},"min":{"1min":0.40400000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83899999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67000000000000004,"15min":0.68899999999999995},"min":{"1min":0.45500000000000002,"5min":0.318,"15min":0.318},"max":{"1min":2.2269999999999999,"5min":2.2269999999999999,"15min":2.2269999999999999},"last":0.69099999999999995}]},{"osd":5,"last update":"Thu Nov 16 16:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.67700000000000005,"15min":0.70299999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.82799999999999996,"5min":1.056,"15min":1.056},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.70199999999999996,"15min":0.73299999999999998},"min":{"1min":0.39800000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.68000000000000005}]},{"osd":6,"last update":"Thu Nov 16 16:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.61899999999999999,"15min":0.66200000000000003},"min":{"1min":0.32000000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.67000000000000004,"15min":0.71599999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.53500000000000003}]},{"osd":7,"last update":"Thu Nov 16 16:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.751,"15min":0.77200000000000002},"min":{"1min":0.44700000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":2.153,"5min":2.153,"15min":2.153},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.75,"15min":0.78800000000000003},"min":{"1min":0.55200000000000005,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.76800000000000002}]},{"osd":8,"last update":"Thu Nov 16 16:38:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.78000000000000003,"15min":0.81699999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.88600000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.746},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.78000000000000003,"15min":0.80900000000000005},"min":{"1min":0.54300000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.876,"5min":1.089,"15min":1.089},"last":0.79700000000000004}]},{"osd":10,"last update":"Thu Nov 16 16:38:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.73699999999999999,"15min":0.747},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.91700000000000004,"5min":1.028,"15min":1.028},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.76100000000000001,"15min":0.76300000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.1819999999999999,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":0.86899999999999999}]}]},{"osd":8,"up_from":59,"seq":253403070510,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6424,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738728,"statfs":{"total":95995035648,"available":95988457472,"internally_reserved":0,"allocated":933888,"data_stored":446953,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57599999999999996,"15min":0.58199999999999996},"min":{"1min":0.40000000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.72099999999999997,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.438,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.72699999999999998,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.57899999999999996}]},{"osd":1,"last update":"Thu Nov 16 16:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.65200000000000002,"15min":0.625},"min":{"1min":0.34100000000000003,"5min":0.318,"15min":0.318},"max":{"1min":0.94899999999999995,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.628,"15min":0.60499999999999998},"min":{"1min":0.46100000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.81299999999999994,"5min":1.383,"15min":1.383},"last":0.89900000000000002}]},{"osd":2,"last update":"Thu Nov 16 16:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.52400000000000002,"15min":0.51700000000000002},"min":{"1min":0.40400000000000003,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.86499999999999999,"5min":1.056,"15min":1.056},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.53200000000000003,"15min":0.52600000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.65800000000000003}]},{"osd":3,"last update":"Thu Nov 16 16:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.621,"15min":0.624},"min":{"1min":0.39800000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.83099999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.68300000000000005,"15min":0.67700000000000005},"min":{"1min":0.48099999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.99299999999999999,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.872}]},{"osd":4,"last update":"Thu Nov 16 16:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.68100000000000005,"15min":0.64100000000000001},"min":{"1min":0.36899999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.093,"5min":4.0599999999999996,"15min":4.0599999999999996},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.72999999999999998,"15min":0.70799999999999996},"min":{"1min":0.45000000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1200000000000001,"5min":1.167,"15min":1.167},"last":0.97499999999999998}]},{"osd":5,"last update":"Thu Nov 16 16:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.61699999999999999,"15min":0.61499999999999999},"min":{"1min":0.36899999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.89400000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.40000000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.91500000000000004,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.83199999999999996}]},{"osd":6,"last update":"Thu Nov 16 16:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.76000000000000001,"15min":0.75},"min":{"1min":0.53500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1020000000000001,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.75,"15min":0.752},"min":{"1min":0.54000000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.98799999999999999,"5min":1.173,"15min":1.173},"last":0.85299999999999998}]},{"osd":7,"last update":"Thu Nov 16 16:38:31 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.78500000000000003,"15min":0.76900000000000002},"min":{"1min":0.49199999999999999,"5min":0.433,"15min":0.433},"max":{"1min":1.05,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.77100000000000002,"15min":0.75800000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0349999999999999,"5min":1.369,"15min":1.369},"last":0.88900000000000001}]},{"osd":9,"last update":"Thu Nov 16 16:37:55 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.73799999999999999,"15min":0.72999999999999998},"min":{"1min":0.54000000000000004,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.018,"5min":1.036,"15min":1.036},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.875,"5min":0.76900000000000002,"15min":0.751},"min":{"1min":0.58699999999999997,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":1.069}]},{"osd":10,"last update":"Thu Nov 16 16:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.87,"15min":0.85999999999999999},"min":{"1min":0.79700000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.127,"5min":3.5720000000000001,"15min":3.5720000000000001},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.90600000000000003,"15min":0.89200000000000002},"min":{"1min":0.82599999999999996,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.149,"5min":3.5920000000000001,"15min":3.5920000000000001},"last":1.125}]},{"osd":11,"last update":"Thu Nov 16 16:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":1.006},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":1.099}]}]},{"osd":7,"up_from":52,"seq":223338299441,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6544,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738608,"statfs":{"total":95995035648,"available":95988334592,"internally_reserved":0,"allocated":991232,"data_stored":1039363,"data_compressed":41378,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:36:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55800000000000005}]},{"osd":1,"last update":"Thu Nov 16 16:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.42999999999999999,"15min":0.41499999999999998},"min":{"1min":0.313,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.44800000000000001,"15min":0.42999999999999999},"min":{"1min":0.33100000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.66800000000000004,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.64600000000000002}]},{"osd":2,"last update":"Thu Nov 16 16:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.58299999999999996,"15min":0.56499999999999995},"min":{"1min":0.45600000000000002,"5min":0.316,"15min":0.316},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.59699999999999998,"15min":0.57699999999999996},"min":{"1min":0.52200000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.73599999999999999}]},{"osd":3,"last update":"Thu Nov 16 16:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58299999999999996,"15min":0.57299999999999995},"min":{"1min":0.44,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.79300000000000004,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.67500000000000004,"15min":0.63700000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":5.6100000000000003,"5min":5.6100000000000003,"15min":5.6100000000000003},"last":0.47899999999999998}]},{"osd":4,"last update":"Thu Nov 16 16:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.54700000000000004,"15min":0.52100000000000002},"min":{"1min":0.371,"5min":0.317,"15min":0.317},"max":{"1min":0.77800000000000002,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.51100000000000001,"15min":0.48699999999999999},"min":{"1min":0.379,"5min":0.311,"15min":0.311},"max":{"1min":0.72799999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.61099999999999999}]},{"osd":5,"last update":"Thu Nov 16 16:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.66800000000000004,"15min":0.65200000000000002},"min":{"1min":0.47599999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.66600000000000004,"15min":0.64100000000000001},"min":{"1min":0.50900000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.69499999999999995}]},{"osd":6,"last update":"Thu Nov 16 16:38:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":0.81599999999999995,"15min":0.73299999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":11.647,"5min":11.647,"15min":11.647},"last":0.752},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.73099999999999998,"15min":0.69099999999999995},"min":{"1min":0.51100000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":4.9550000000000001,"5min":4.9550000000000001,"15min":4.9550000000000001},"last":0.50700000000000001}]},{"osd":8,"last update":"Thu Nov 16 16:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.75700000000000001,"15min":0.75},"min":{"1min":0.60499999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.0289999999999999,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.76700000000000002,"15min":0.76500000000000001},"min":{"1min":0.56200000000000006,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.069,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.82999999999999996}]},{"osd":9,"last update":"Thu Nov 16 16:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.79700000000000004,"15min":0.78000000000000003},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.98099999999999998,"5min":4.7370000000000001,"15min":4.7370000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.83099999999999996,"15min":0.81799999999999995},"min":{"1min":0.54500000000000004,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.97699999999999998,"5min":4.7809999999999997,"15min":4.7809999999999997},"last":0.80700000000000005}]},{"osd":10,"last update":"Thu Nov 16 16:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.80400000000000005,"15min":0.80100000000000005},"min":{"1min":0.64300000000000002,"5min":0.443,"15min":0.443},"max":{"1min":1.026,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.76400000000000001,"15min":0.75800000000000001},"min":{"1min":0.64000000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0509999999999999,"5min":1.054,"15min":1.054},"last":0.84099999999999997}]},{"osd":11,"last update":"Thu Nov 16 16:38:17 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.873}]}]},{"osd":6,"up_from":47,"seq":201863462965,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6356,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738796,"statfs":{"total":95995035648,"available":95988527104,"internally_reserved":0,"allocated":929792,"data_stored":446540,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.42599999999999999,"15min":0.41599999999999998},"min":{"1min":0.29199999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.43099999999999999,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.45900000000000002,"15min":0.441},"min":{"1min":0.29699999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.56499999999999995,"5min":3.472,"15min":3.472},"last":0.36099999999999999}]},{"osd":1,"last update":"Thu Nov 16 16:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.58199999999999996,"15min":0.57799999999999996},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.64400000000000002,"5min":3.3290000000000002,"15min":3.3290000000000002},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.63700000000000001,"15min":0.59699999999999998},"min":{"1min":0.40699999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.65400000000000003,"5min":4.4669999999999996,"15min":4.4669999999999996},"last":0.443}]},{"osd":2,"last update":"Thu Nov 16 16:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.63600000000000001,"15min":0.63200000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.68300000000000005,"5min":3.5369999999999999,"15min":3.5369999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.64900000000000002,"15min":0.629},"min":{"1min":0.443,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.64500000000000002,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.51700000000000002}]},{"osd":3,"last update":"Thu Nov 16 16:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.504,"15min":0.48299999999999998},"min":{"1min":0.28999999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.46600000000000003,"5min":3.3700000000000001,"15min":3.3700000000000001},"last":0.434},{"interface":"front","average":{"1min":0.434,"5min":0.53600000000000003,"15min":0.53400000000000003},"min":{"1min":0.35099999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.50900000000000001,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.502}]},{"osd":4,"last update":"Thu Nov 16 16:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.66800000000000004,"15min":0.60799999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.75900000000000001,"5min":4.4900000000000002,"15min":4.4900000000000002},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.68899999999999995,"15min":0.627},"min":{"1min":0.379,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.69399999999999995,"5min":4.4509999999999996,"15min":4.4509999999999996},"last":0.47299999999999998}]},{"osd":5,"last update":"Thu Nov 16 16:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.66900000000000004,"15min":0.67500000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.71599999999999997,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.71099999999999997,"15min":0.70299999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.75,"5min":3.5019999999999998,"15min":3.5019999999999998},"last":0.59099999999999997}]},{"osd":7,"last update":"Thu Nov 16 16:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.70899999999999996,"15min":0.70499999999999996},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":2.452,"5min":2.452,"15min":2.452},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.73999999999999999,"15min":0.71799999999999997},"min":{"1min":0.54400000000000004,"5min":0.436,"15min":0.436},"max":{"1min":3.4449999999999998,"5min":3.4449999999999998,"15min":3.4449999999999998},"last":0.60999999999999999}]},{"osd":8,"last update":"Thu Nov 16 16:38:32 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.752,"15min":0.76900000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":2.3919999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.77800000000000002,"15min":0.78300000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":3.488,"5min":3.488,"15min":3.488},"last":0.64600000000000002}]},{"osd":9,"last update":"Thu Nov 16 16:38:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.77300000000000002,"15min":0.76700000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.68700000000000006,"5min":6.1539999999999999,"15min":6.1539999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.79800000000000004,"15min":0.76800000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.67800000000000005,"5min":8.2460000000000004,"15min":8.2460000000000004},"last":0.57799999999999996}]},{"osd":10,"last update":"Thu Nov 16 16:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.81200000000000006,"15min":0.81000000000000005},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":3.5209999999999999,"5min":3.5209999999999999,"15min":3.5209999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.82499999999999996,"15min":0.82399999999999995},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":3.419,"5min":3.419,"15min":3.419},"last":0.65600000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7444,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":93737708,"statfs":{"total":95995035648,"available":95987412992,"internally_reserved":0,"allocated":995328,"data_stored":510800,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:38:06 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.48499999999999999,"15min":0.45800000000000002},"min":{"1min":0.34899999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.67000000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52100000000000002,"15min":0.48799999999999999},"min":{"1min":0.311,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.83499999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.48299999999999998}]},{"osd":2,"last update":"Thu Nov 16 16:38:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.58299999999999996,"15min":0.51100000000000001},"min":{"1min":0.45500000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.58199999999999996,"15min":0.51700000000000002},"min":{"1min":0.46400000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.85199999999999998,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.745}]},{"osd":3,"last update":"Thu Nov 16 16:38:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58599999999999997,"15min":0.55800000000000005},"min":{"1min":0.33900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.93700000000000006,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60999999999999999,"15min":0.57899999999999996},"min":{"1min":0.38100000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.89700000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.66400000000000003}]},{"osd":4,"last update":"Thu Nov 16 16:38:06 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.46300000000000002,"15min":0.43099999999999999},"min":{"1min":0.33700000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.79500000000000004,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.503},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55000000000000004,"15min":0.48499999999999999},"min":{"1min":0.35499999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.82899999999999996,"5min":1.242,"15min":1.242},"last":0.59399999999999997}]},{"osd":5,"last update":"Thu Nov 16 16:38:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.67300000000000004,"15min":0.623},"min":{"1min":0.55100000000000005,"5min":0.38,"15min":0.38},"max":{"1min":0.90700000000000003,"5min":1.004,"15min":1.004},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.65000000000000002,"15min":0.60699999999999998},"min":{"1min":0.53700000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93899999999999995,"5min":1.004,"15min":1.004},"last":0.79600000000000004}]},{"osd":6,"last update":"Thu Nov 16 16:38:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.69599999999999995,"15min":0.67200000000000004},"min":{"1min":0.51900000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.02,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69299999999999995,"15min":0.66700000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.72099999999999997}]},{"osd":7,"last update":"Thu Nov 16 16:38:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65300000000000002,"15min":0.63500000000000001},"min":{"1min":0.39500000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.0569999999999999,"5min":1.077,"15min":1.077},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.67900000000000005},"min":{"1min":0.42599999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.018,"5min":1.143,"15min":1.143},"last":0.73399999999999999}]},{"osd":8,"last update":"Thu Nov 16 16:38:30 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.76800000000000002,"15min":0.75800000000000001},"min":{"1min":0.56799999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0289999999999999,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.75,"15min":0.73999999999999999},"min":{"1min":0.52800000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0640000000000001,"5min":1.145,"15min":1.145},"last":0.85299999999999998}]},{"osd":9,"last update":"Thu Nov 16 16:38:55 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.76200000000000001,"15min":0.79300000000000004},"min":{"1min":0.45600000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.004,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83799999999999997,"15min":0.86099999999999999},"min":{"1min":0.55300000000000005,"5min":0.498,"15min":0.498},"max":{"1min":1.0980000000000001,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.82699999999999996}]},{"osd":10,"last update":"Thu Nov 16 16:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.83999999999999997,"15min":0.84999999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.99199999999999999,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.81599999999999995,"15min":0.81999999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.98099999999999998,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.70699999999999996}]},{"osd":11,"last update":"Thu Nov 16 16:38:15 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.871},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.89700000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7272,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":6336,"kb_avail":93737880,"statfs":{"total":95995035648,"available":95987589120,"internally_reserved":0,"allocated":950272,"data_stored":447805,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":6488064},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Nov 16 16:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.54700000000000004,"15min":0.48199999999999998},"min":{"1min":0.38400000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":3.6869999999999998,"5min":3.6869999999999998,"15min":3.6869999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.57999999999999996,"15min":0.48799999999999999},"min":{"1min":0.40600000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":6.6059999999999999,"5min":6.6059999999999999,"15min":6.6059999999999999},"last":0.60599999999999998}]},{"osd":2,"last update":"Thu Nov 16 16:38:27 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.63300000000000001,"15min":0.53900000000000003},"min":{"1min":0.39700000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":6.4000000000000004,"5min":6.4000000000000004,"15min":6.4000000000000004},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.60799999999999998,"15min":0.54600000000000004},"min":{"1min":0.42599999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":3.7829999999999999,"5min":3.7829999999999999,"15min":3.7829999999999999},"last":0.81200000000000006}]},{"osd":3,"last update":"Thu Nov 16 16:38:15 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":9.0139999999999993,"5min":9.0139999999999993,"15min":9.0139999999999993},"last":0.501},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":3.6400000000000001,"5min":3.6400000000000001,"15min":3.6400000000000001},"last":0.51900000000000002}]},{"osd":4,"last update":"Thu Nov 16 16:38:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.61499999999999999,"15min":0.56299999999999994},"min":{"1min":0.44,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":3.8109999999999999,"5min":3.8109999999999999,"15min":3.8109999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.871,"5min":0.64900000000000002,"15min":0.57199999999999995},"min":{"1min":0.42099999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":6.577,"5min":6.577,"15min":6.577},"last":0.54700000000000004}]},{"osd":5,"last update":"Thu Nov 16 16:38:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":0.65700000000000003,"15min":0.58299999999999996},"min":{"1min":0.47899999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":9.0690000000000008,"5min":9.0690000000000008,"15min":9.0690000000000008},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.70599999999999996,"15min":0.64100000000000001},"min":{"1min":0.38800000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":6.4539999999999997,"5min":6.4539999999999997,"15min":6.4539999999999997},"last":0.749}]},{"osd":6,"last update":"Thu Nov 16 16:38:50 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.623,"15min":0.58899999999999997},"min":{"1min":0.314,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68300000000000005,"5min":6.5190000000000001,"15min":6.5190000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.622,"15min":0.57199999999999995},"min":{"1min":0.313,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.65600000000000003,"5min":9.141,"15min":9.141},"last":0.58799999999999997}]},{"osd":7,"last update":"Thu Nov 16 16:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.68400000000000005,"15min":0.67600000000000005},"min":{"1min":0.495,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.74199999999999999,"15min":0.73199999999999998},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.76700000000000002}]},{"osd":8,"last update":"Thu Nov 16 16:38:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.78900000000000003,"15min":0.80900000000000005},"min":{"1min":0.496,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":3.8410000000000002,"5min":3.8410000000000002,"15min":3.8410000000000002},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.86099999999999999,"15min":0.86199999999999999},"min":{"1min":0.57099999999999995,"5min":0.498,"15min":0.498},"max":{"1min":6.4870000000000001,"5min":6.4870000000000001,"15min":6.4870000000000001},"last":0.67400000000000004}]},{"osd":9,"last update":"Thu Nov 16 16:37:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.85499999999999998,"15min":0.86199999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":6.6349999999999998,"5min":6.6349999999999998,"15min":6.6349999999999998},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.79000000000000004,"15min":0.76600000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":9.1690000000000005,"5min":9.1690000000000005,"15min":9.1690000000000005},"last":0.68100000000000005}]},{"osd":10,"last update":"Thu Nov 16 16:38:15 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.78500000000000003,"15min":0.78100000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":3.7229999999999999,"5min":3.7229999999999999,"15min":3.7229999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.88300000000000001,"15min":0.86499999999999999},"min":{"1min":0.55400000000000005,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":6.5460000000000003,"5min":6.5460000000000003,"15min":6.5460000000000003},"last":0.71299999999999997}]},{"osd":11,"last update":"Thu Nov 16 16:38:35 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.99299999999999999,"15min":1.0409999999999999},"min":{"1min":0.56399999999999995,"5min":0.502,"15min":0.502},"max":{"1min":0.86399999999999999,"5min":9.1039999999999992,"15min":9.1039999999999992},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.83499999999999996,"15min":0.85499999999999998},"min":{"1min":0.53500000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.91200000000000003,"5min":3.7530000000000001,"15min":3.7530000000000001},"last":0.874}]}]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7532,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":6528,"kb_avail":93737620,"statfs":{"total":95995035648,"available":95987322880,"internally_reserved":0,"allocated":1019904,"data_stored":512951,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":6684672},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:38:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.52700000000000002,"15min":0.52300000000000002},"min":{"1min":0.372,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.88300000000000001,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.50700000000000001,"15min":0.49099999999999999},"min":{"1min":0.35999999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.81200000000000006,"5min":1.006,"15min":1.006},"last":0.47899999999999998}]},{"osd":1,"last update":"Thu Nov 16 16:38:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.55500000000000005,"15min":0.52300000000000002},"min":{"1min":0.40400000000000003,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.60099999999999998,"15min":0.56499999999999995},"min":{"1min":0.46700000000000003,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.91800000000000004,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.70399999999999996}]},{"osd":3,"last update":"Thu Nov 16 16:38:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.66300000000000003,"15min":0.60699999999999998},"min":{"1min":0.38800000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.94199999999999995,"5min":6.9740000000000002,"15min":6.9740000000000002},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59299999999999997,"15min":0.56299999999999994},"min":{"1min":0.39200000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.86699999999999999,"5min":1.03,"15min":1.03},"last":0.625}]},{"osd":4,"last update":"Thu Nov 16 16:38:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.69699999999999995}]},{"osd":5,"last update":"Thu Nov 16 16:38:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.51700000000000002,"15min":0.49099999999999999},"min":{"1min":0.376,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.83199999999999996,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.48799999999999999,"15min":0.46400000000000002},"min":{"1min":0.39100000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.71699999999999997,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.55700000000000005}]},{"osd":6,"last update":"Thu Nov 16 16:38:50 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.70399999999999996,"15min":0.66200000000000003},"min":{"1min":0.52100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0509999999999999,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.66700000000000004,"15min":0.621},"min":{"1min":0.45400000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.97199999999999998,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.67500000000000004}]},{"osd":7,"last update":"Thu Nov 16 16:37:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.64300000000000002,"15min":0.64400000000000002},"min":{"1min":0.41599999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70199999999999996,"15min":0.70399999999999996},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.92700000000000005,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.57699999999999996}]},{"osd":8,"last update":"Thu Nov 16 16:38:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.59599999999999997,"15min":0.57599999999999996},"min":{"1min":0.35299999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.84199999999999997,"5min":1.008,"15min":1.008},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.63300000000000001,"15min":0.627},"min":{"1min":0.38500000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.82499999999999996,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.59799999999999998}]},{"osd":9,"last update":"Thu Nov 16 16:37:54 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.86799999999999999,"15min":0.86699999999999999},"min":{"1min":0.47499999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.6080000000000001,"5min":4.524,"15min":4.524},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.751,"15min":0.73299999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.6200000000000001,"5min":2.6200000000000001,"15min":2.6200000000000001},"last":0.79000000000000004}]},{"osd":10,"last update":"Thu Nov 16 16:38:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.73399999999999999,"15min":0.72499999999999998},"min":{"1min":0.55100000000000005,"5min":0.497,"15min":0.497},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.72599999999999998,"15min":0.72099999999999997},"min":{"1min":0.61499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.92500000000000004,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.75800000000000001}]},{"osd":11,"last update":"Thu Nov 16 16:38:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.874},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.80500000000000005}]}]},{"osd":3,"up_from":26,"seq":111669149762,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6688,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":93738464,"statfs":{"total":95995035648,"available":95988187136,"internally_reserved":0,"allocated":942080,"data_stored":446526,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:38:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.374,"15min":0.35199999999999998},"min":{"1min":0.28599999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.40100000000000002,"15min":0.36499999999999999},"min":{"1min":0.309,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.28599999999999998}]},{"osd":1,"last update":"Thu Nov 16 16:38:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56100000000000005,"15min":0.55100000000000005},"min":{"1min":0.371,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.56399999999999995,"15min":0.55900000000000005},"min":{"1min":0.40300000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.84099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.71099999999999997}]},{"osd":2,"last update":"Thu Nov 16 16:38:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.59199999999999997,"15min":0.56100000000000005},"min":{"1min":0.439,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.91400000000000003,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57199999999999995,"15min":0.54900000000000004},"min":{"1min":0.34699999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.53900000000000003}]},{"osd":4,"last update":"Thu Nov 16 16:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56299999999999994,"15min":0.54900000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79900000000000004,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60599999999999998,"15min":0.59599999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.60199999999999998}]},{"osd":5,"last update":"Thu Nov 16 16:38:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59899999999999998,"15min":0.58099999999999996},"min":{"1min":0.39600000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95299999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.61899999999999999,"15min":0.59599999999999997},"min":{"1min":0.47399999999999998,"5min":0.37,"15min":0.37},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.67300000000000004}]},{"osd":6,"last update":"Thu Nov 16 16:38:55 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.52800000000000002,"15min":0.501},"min":{"1min":0.373,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.52900000000000003,"15min":0.51300000000000001},"min":{"1min":0.34799999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.52100000000000002}]},{"osd":7,"last update":"Thu Nov 16 16:38:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.65300000000000002,"15min":0.64900000000000002},"min":{"1min":0.503,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.68700000000000006,"15min":0.68500000000000005},"min":{"1min":0.42699999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.69399999999999995}]},{"osd":8,"last update":"Thu Nov 16 16:36:27 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.63200000000000001}]},{"osd":9,"last update":"Thu Nov 16 16:38:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.69999999999999996,"15min":0.75},"min":{"1min":0.38500000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.89700000000000002,"5min":1.024,"15min":1.024},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.65300000000000002,"15min":0.69699999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.72699999999999998,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.46899999999999997}]},{"osd":10,"last update":"Thu Nov 16 16:38:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.73299999999999998,"15min":0.74299999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.752},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.74099999999999999,"15min":0.751},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.97599999999999998,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.77300000000000002}]},{"osd":11,"last update":"Thu Nov 16 16:38:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.72799999999999998}]}]},{"osd":4,"up_from":34,"seq":146028888126,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6488,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738664,"statfs":{"total":95995035648,"available":95988391936,"internally_reserved":0,"allocated":933888,"data_stored":446923,"data_compressed":5813,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.52400000000000002,"15min":0.502},"min":{"1min":0.38200000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.52900000000000003,"15min":0.51500000000000001},"min":{"1min":0.40899999999999997,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.378}]},{"osd":1,"last update":"Thu Nov 16 16:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.41099999999999998,"15min":0.39300000000000002},"min":{"1min":0.33700000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.67900000000000005,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.37},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.434,"15min":0.40999999999999998},"min":{"1min":0.378,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.748,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.39700000000000002}]},{"osd":2,"last update":"Thu Nov 16 16:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.77300000000000002,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.623,"15min":0.59299999999999997},"min":{"1min":0.53000000000000003,"5min":0.371,"15min":0.371},"max":{"1min":0.81999999999999995,"5min":1.056,"15min":1.056},"last":0.80300000000000005}]},{"osd":3,"last update":"Thu Nov 16 16:38:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.59199999999999997,"15min":0.55000000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57299999999999995,"15min":0.55300000000000005},"min":{"1min":0.40100000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.51500000000000001}]},{"osd":5,"last update":"Thu Nov 16 16:38:28 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.623,"15min":0.627},"min":{"1min":0.40100000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.76500000000000001,"5min":1.075,"15min":1.075},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.65700000000000003,"15min":0.63200000000000001},"min":{"1min":0.49099999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.89700000000000002,"5min":1.093,"15min":1.093},"last":0.84499999999999997}]},{"osd":6,"last update":"Thu Nov 16 16:38:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65000000000000002,"15min":0.60899999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.85399999999999998,"5min":1.004,"15min":1.004},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.61599999999999999,"15min":0.59799999999999998},"min":{"1min":0.433,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77300000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.46999999999999997}]},{"osd":7,"last update":"Thu Nov 16 16:38:13 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.61499999999999999,"15min":0.56299999999999994},"min":{"1min":0.45400000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.99399999999999999,"5min":1.123,"15min":1.123},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.67300000000000004,"15min":0.621},"min":{"1min":0.51600000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.99199999999999999,"5min":1.069,"15min":1.069},"last":0.78400000000000003}]},{"osd":8,"last update":"Thu Nov 16 16:38:28 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.76400000000000001,"15min":0.73199999999999998},"min":{"1min":0.53700000000000003,"5min":0.495,"15min":0.495},"max":{"1min":0.98699999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.77000000000000002,"15min":0.73899999999999999},"min":{"1min":0.51700000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.98199999999999998,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.82799999999999996}]},{"osd":9,"last update":"Thu Nov 16 16:37:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.79200000000000004,"15min":0.78900000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.94099999999999995,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.72099999999999997,"15min":0.71799999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94199999999999995,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.56299999999999994}]},{"osd":10,"last update":"Thu Nov 16 16:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.68200000000000005,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0109999999999999,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.753},{"interface":"front","average":{"1min":0.872,"5min":0.83999999999999997,"15min":0.83399999999999996},"min":{"1min":0.73399999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.74299999999999999}]},{"osd":11,"last update":"Thu Nov 16 16:38:13 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.86499999999999999}]}]},{"osd":5,"up_from":39,"seq":167503724603,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6684,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738468,"statfs":{"total":95995035648,"available":95988191232,"internally_reserved":0,"allocated":1003520,"data_stored":1039095,"data_compressed":41378,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Nov 16 16:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.55500000000000005,"15min":0.51300000000000001},"min":{"1min":0.38100000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":4.2380000000000004,"5min":4.2380000000000004,"15min":4.2380000000000004},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.57499999999999996,"15min":0.54000000000000004},"min":{"1min":0.42899999999999999,"5min":0.311,"15min":0.311},"max":{"1min":3.5590000000000002,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":0.58599999999999997}]},{"osd":1,"last update":"Thu Nov 16 16:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.61299999999999999,"15min":0.59399999999999997},"min":{"1min":0.55600000000000005,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.997,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.60199999999999998,"15min":0.58999999999999997},"min":{"1min":0.42699999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84099999999999997,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.56899999999999995}]},{"osd":2,"last update":"Thu Nov 16 16:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.443,"15min":0.42899999999999999},"min":{"1min":0.32700000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.70399999999999996,"5min":1.417,"15min":1.417},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.46400000000000002,"15min":0.44},"min":{"1min":0.33700000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.70799999999999996,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.35699999999999998}]},{"osd":3,"last update":"Thu Nov 16 16:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.60299999999999998,"15min":0.56200000000000006},"min":{"1min":0.38,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":3.3079999999999998,"5min":3.3079999999999998,"15min":3.3079999999999998},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.67300000000000004,"15min":0.621},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":5.343,"5min":5.343,"15min":5.343},"last":0.47699999999999998}]},{"osd":4,"last update":"Thu Nov 16 16:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.69999999999999996,"15min":0.63600000000000001},"min":{"1min":0.46000000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85599999999999998,"5min":5.5789999999999997,"15min":5.5789999999999997},"last":0.628},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.62,"15min":0.59799999999999998},"min":{"1min":0.437,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84499999999999997,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.52900000000000003}]},{"osd":6,"last update":"Thu Nov 16 16:38:48 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.68700000000000006,"15min":0.63900000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.996,"5min":3.274,"15min":3.274},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.68300000000000005,"15min":0.64700000000000002},"min":{"1min":0.40100000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.0109999999999999,"5min":3.472,"15min":3.472},"last":0.50600000000000001}]},{"osd":7,"last update":"Thu Nov 16 16:38:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.72199999999999998,"15min":0.71599999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0449999999999999,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.73799999999999999,"15min":0.73199999999999998},"min":{"1min":0.48099999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0580000000000001,"5min":1.544,"15min":1.544},"last":0.66900000000000004}]},{"osd":8,"last update":"Thu Nov 16 16:38:29 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.66500000000000004,"15min":0.65100000000000002},"min":{"1min":0.48699999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.92500000000000004,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.746,"15min":0.73199999999999998},"min":{"1min":0.59499999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.69399999999999995}]},{"osd":9,"last update":"Thu Nov 16 16:38:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.90200000000000002,"15min":0.82799999999999996},"min":{"1min":0.52400000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1699999999999999,"5min":5.5679999999999996,"15min":5.5679999999999996},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.82099999999999995,"15min":0.77100000000000002},"min":{"1min":0.56399999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0640000000000001,"5min":5.3230000000000004,"15min":5.3230000000000004},"last":0.71299999999999997}]},{"osd":10,"last update":"Thu Nov 16 16:38:16 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.80000000000000004,"15min":0.78200000000000003},"min":{"1min":0.63900000000000001,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.80200000000000005,"15min":0.78800000000000003},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.755}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33928,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35565,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":66624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-16T16:38:57.922 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-16T16:38:58.085 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.081+0000 7f3d6eaab700 1 -- 172.21.15.1:0/3598922687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d680ff1c0 msgr2=0x7f3d680ff5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:58.085 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.081+0000 7f3d6eaab700 1 --2- 172.21.15.1:0/3598922687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d680ff1c0 0x7f3d680ff5d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c009990 tx=0x7f3d5c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:58.085 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.081+0000 7f3d6eaab700 1 -- 172.21.15.1:0/3598922687 shutdown_connections 2023-11-16T16:38:58.085 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.081+0000 7f3d6eaab700 1 --2- 172.21.15.1:0/3598922687 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d68100d40 0x7f3d68107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:58.085 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.081+0000 7f3d6eaab700 1 --2- 172.21.15.1:0/3598922687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d680ff1c0 0x7f3d680ff5d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:58.085 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.081+0000 7f3d6eaab700 1 --2- 172.21.15.1:0/3598922687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d681003b0 0x7f3d68100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:58.085 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.081+0000 7f3d6eaab700 1 -- 172.21.15.1:0/3598922687 >> 172.21.15.1:0/3598922687 conn(0x7f3d680fa7c0 msgr2=0x7f3d680fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:58.085 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.081+0000 7f3d6eaab700 1 -- 172.21.15.1:0/3598922687 shutdown_connections 2023-11-16T16:38:58.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.081+0000 7f3d6eaab700 1 -- 172.21.15.1:0/3598922687 wait complete. 2023-11-16T16:38:58.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.081+0000 7f3d6eaab700 1 Processor -- start 2023-11-16T16:38:58.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.081+0000 7f3d6eaab700 1 -- start start 2023-11-16T16:38:58.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.081+0000 7f3d6eaab700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d680ff1c0 0x7f3d68103150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:58.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.081+0000 7f3d6eaab700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d681003b0 0x7f3d68101790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:58.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.081+0000 7f3d6eaab700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d68100d40 0x7f3d68101cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:58.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.081+0000 7f3d6eaab700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d68102470 con 0x7f3d681003b0 2023-11-16T16:38:58.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.081+0000 7f3d6eaab700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d681025e0 con 0x7f3d680ff1c0 2023-11-16T16:38:58.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.082+0000 7f3d6eaab700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d681a0140 con 0x7f3d68100d40 2023-11-16T16:38:58.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.082+0000 7f3d67fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d681003b0 0x7f3d68101790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:58.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.082+0000 7f3d67fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d681003b0 0x7f3d68101790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43010/0 (socket says 172.21.15.1:43010) 2023-11-16T16:38:58.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.082+0000 7f3d67fff700 1 -- 172.21.15.1:0/4052677904 learned_addr learned my addr 172.21.15.1:0/4052677904 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:58.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.082+0000 7f3d6c847700 1 --2- 172.21.15.1:0/4052677904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d680ff1c0 0x7f3d68103150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:58.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.082+0000 7f3d6d048700 1 --2- 172.21.15.1:0/4052677904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d68100d40 0x7f3d68101cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:58.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.082+0000 7f3d67fff700 1 -- 172.21.15.1:0/4052677904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d68100d40 msgr2=0x7f3d68101cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:58.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.082+0000 7f3d67fff700 1 --2- 172.21.15.1:0/4052677904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d68100d40 0x7f3d68101cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:58.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.082+0000 7f3d67fff700 1 -- 172.21.15.1:0/4052677904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d680ff1c0 msgr2=0x7f3d68103150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:58.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.082+0000 7f3d67fff700 1 --2- 172.21.15.1:0/4052677904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d680ff1c0 0x7f3d68103150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:58.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.082+0000 7f3d67fff700 1 -- 172.21.15.1:0/4052677904 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d5c0095f0 con 0x7f3d681003b0 2023-11-16T16:38:58.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.082+0000 7f3d67fff700 1 --2- 172.21.15.1:0/4052677904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d681003b0 0x7f3d68101790 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3d5400c9b0 tx=0x7f3d5400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:58.088 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.082+0000 7f3d65ffb700 1 -- 172.21.15.1:0/4052677904 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d54007b00 con 0x7f3d681003b0 2023-11-16T16:38:58.088 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.082+0000 7f3d65ffb700 1 -- 172.21.15.1:0/4052677904 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d54007c60 con 0x7f3d681003b0 2023-11-16T16:38:58.088 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.082+0000 7f3d6eaab700 1 -- 172.21.15.1:0/4052677904 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d6810aba0 con 0x7f3d681003b0 2023-11-16T16:38:58.088 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.082+0000 7f3d65ffb700 1 -- 172.21.15.1:0/4052677904 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d54018660 con 0x7f3d681003b0 2023-11-16T16:38:58.088 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.083+0000 7f3d6eaab700 1 -- 172.21.15.1:0/4052677904 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d6810aa10 con 0x7f3d681003b0 2023-11-16T16:38:58.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.083+0000 7f3d6eaab700 1 -- 172.21.15.1:0/4052677904 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d681a0890 con 0x7f3d681003b0 2023-11-16T16:38:58.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.084+0000 7f3d65ffb700 1 -- 172.21.15.1:0/4052677904 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f3d54018880 con 0x7f3d681003b0 2023-11-16T16:38:58.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.085+0000 7f3d65ffb700 1 --2- 172.21.15.1:0/4052677904 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f3d580a1150 0x7f3d580a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:58.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.085+0000 7f3d65ffb700 1 -- 172.21.15.1:0/4052677904 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3d540cc9d0 con 0x7f3d681003b0 2023-11-16T16:38:58.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.085+0000 7f3d6c847700 1 --2- 172.21.15.1:0/4052677904 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f3d580a1150 0x7f3d580a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:58.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.086+0000 7f3d6c847700 1 --2- 172.21.15.1:0/4052677904 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f3d580a1150 0x7f3d580a3600 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c0036c0 tx=0x7f3d5c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:58.091 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.089+0000 7f3d65ffb700 1 -- 172.21.15.1:0/4052677904 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3d54096120 con 0x7f3d681003b0 2023-11-16T16:38:58.293 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.291+0000 7f3d6eaab700 1 -- 172.21.15.1:0/4052677904 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d681a0f30 con 0x7f3d681003b0 2023-11-16T16:38:58.295 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.292+0000 7f3d65ffb700 1 -- 172.21.15.1:0/4052677904 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22469 (secure 0 0 0) 0x7f3d54099740 con 0x7f3d681003b0 2023-11-16T16:38:58.295 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:38:58.296 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":86,"fsid":"35d9b010-849d-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:30:06.217778+0000","modified":"2023-11-16T16:38:03.179244+0000","last_up_change":"2023-11-16T16:36:31.735403+0000","last_in_change":"2023-11-16T16:36:14.083021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:33:15.916723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-16T16:37:12.158031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-16T16:37:13.272343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a8480507-5275-4e52-953b-626110beb436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6803","nonce":2469671609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6805","nonce":2469671609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6809","nonce":2469671609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6807","nonce":2469671609}]},"public_addr":"172.21.15.1:6803/2469671609","cluster_addr":"172.21.15.1:6805/2469671609","heartbeat_back_addr":"172.21.15.1:6809/2469671609","heartbeat_front_addr":"172.21.15.1:6807/2469671609","state":["exists","up"]},{"osd":1,"uuid":"1d011db2-99d1-4575-bdc5-51f88a308fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6801","nonce":247994796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6803","nonce":247994796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6807","nonce":247994796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6805","nonce":247994796}]},"public_addr":"172.21.15.26:6801/247994796","cluster_addr":"172.21.15.26:6803/247994796","heartbeat_back_addr":"172.21.15.26:6807/247994796","heartbeat_front_addr":"172.21.15.26:6805/247994796","state":["exists","up"]},{"osd":2,"uuid":"4d7a2bd1-3c8e-4546-a243-f0354473b031","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6801","nonce":1659316290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6803","nonce":1659316290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6807","nonce":1659316290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6805","nonce":1659316290}]},"public_addr":"172.21.15.53:6801/1659316290","cluster_addr":"172.21.15.53:6803/1659316290","heartbeat_back_addr":"172.21.15.53:6807/1659316290","heartbeat_front_addr":"172.21.15.53:6805/1659316290","state":["exists","up"]},{"osd":3,"uuid":"9c290fd9-e121-4a81-9687-b0eb93debd86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6811","nonce":2728987281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6813","nonce":2728987281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6817","nonce":2728987281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6815","nonce":2728987281}]},"public_addr":"172.21.15.1:6811/2728987281","cluster_addr":"172.21.15.1:6813/2728987281","heartbeat_back_addr":"172.21.15.1:6817/2728987281","heartbeat_front_addr":"172.21.15.1:6815/2728987281","state":["exists","up"]},{"osd":4,"uuid":"533226bc-cc57-4f15-9e29-820294d57e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6809","nonce":2954740024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6811","nonce":2954740024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6815","nonce":2954740024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6813","nonce":2954740024}]},"public_addr":"172.21.15.26:6809/2954740024","cluster_addr":"172.21.15.26:6811/2954740024","heartbeat_back_addr":"172.21.15.26:6815/2954740024","heartbeat_front_addr":"172.21.15.26:6813/2954740024","state":["exists","up"]},{"osd":5,"uuid":"d3ab25eb-e1d4-4557-bc89-efc9072401ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6809","nonce":3966666318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6811","nonce":3966666318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6815","nonce":3966666318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6813","nonce":3966666318}]},"public_addr":"172.21.15.53:6809/3966666318","cluster_addr":"172.21.15.53:6811/3966666318","heartbeat_back_addr":"172.21.15.53:6815/3966666318","heartbeat_front_addr":"172.21.15.53:6813/3966666318","state":["exists","up"]},{"osd":6,"uuid":"180f506e-afb8-4043-b0b4-2dc86f6d4bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6819","nonce":3364032646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6821","nonce":3364032646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6825","nonce":3364032646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6823","nonce":3364032646}]},"public_addr":"172.21.15.1:6819/3364032646","cluster_addr":"172.21.15.1:6821/3364032646","heartbeat_back_addr":"172.21.15.1:6825/3364032646","heartbeat_front_addr":"172.21.15.1:6823/3364032646","state":["exists","up"]},{"osd":7,"uuid":"0d837bac-194f-498d-b7fb-b3147e992329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6817","nonce":3766901391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6819","nonce":3766901391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6823","nonce":3766901391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6821","nonce":3766901391}]},"public_addr":"172.21.15.26:6817/3766901391","cluster_addr":"172.21.15.26:6819/3766901391","heartbeat_back_addr":"172.21.15.26:6823/3766901391","heartbeat_front_addr":"172.21.15.26:6821/3766901391","state":["exists","up"]},{"osd":8,"uuid":"933f231a-7454-47b0-9291-f1fa64f0baec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6817","nonce":3224335871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6819","nonce":3224335871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6823","nonce":3224335871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6821","nonce":3224335871}]},"public_addr":"172.21.15.53:6817/3224335871","cluster_addr":"172.21.15.53:6819/3224335871","heartbeat_back_addr":"172.21.15.53:6823/3224335871","heartbeat_front_addr":"172.21.15.53:6821/3224335871","state":["exists","up"]},{"osd":9,"uuid":"245fcbb3-c651-4703-ab80-309dc98bd983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6827","nonce":1391199844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6829","nonce":1391199844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6833","nonce":1391199844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6831","nonce":1391199844}]},"public_addr":"172.21.15.1:6827/1391199844","cluster_addr":"172.21.15.1:6829/1391199844","heartbeat_back_addr":"172.21.15.1:6833/1391199844","heartbeat_front_addr":"172.21.15.1:6831/1391199844","state":["exists","up"]},{"osd":10,"uuid":"2c090e7a-5240-4d52-ae4f-e94fe05383d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6825","nonce":1773545447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6827","nonce":1773545447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6831","nonce":1773545447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6829","nonce":1773545447}]},"public_addr":"172.21.15.26:6825/1773545447","cluster_addr":"172.21.15.26:6827/1773545447","heartbeat_back_addr":"172.21.15.26:6831/1773545447","heartbeat_front_addr":"172.21.15.26:6829/1773545447","state":["exists","up"]},{"osd":11,"uuid":"4511e32d-e469-45e1-8310-0ff400cb7b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6825","nonce":2564669546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6827","nonce":2564669546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6831","nonce":2564669546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6829","nonce":2564669546}]},"public_addr":"172.21.15.53:6825/2564669546","cluster_addr":"172.21.15.53:6827/2564669546","heartbeat_back_addr":"172.21.15.53:6831/2564669546","heartbeat_front_addr":"172.21.15.53:6829/2564669546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:28.140250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:49.885461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:12.521487+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:34.926197+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:56.815594+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:18.387751+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:40.587106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:02.319370+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:23.832298+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:46.338040+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:08.730414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:30.050070+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1494336948":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6801/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6800/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6800/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/1641976669":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/2895923638":"2023-11-17T16:30:35.064289+0000","172.21.15.1:6801/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/78811711":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/4131957708":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/344403698":"2023-11-17T16:30:35.064289+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:38:58.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.294+0000 7f3d6eaab700 1 -- 172.21.15.1:0/4052677904 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f3d580a1150 msgr2=0x7f3d580a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:58.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.294+0000 7f3d6eaab700 1 --2- 172.21.15.1:0/4052677904 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f3d580a1150 0x7f3d580a3600 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c0036c0 tx=0x7f3d5c00b540 comp rx=0 tx=0).stop 2023-11-16T16:38:58.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.294+0000 7f3d6eaab700 1 -- 172.21.15.1:0/4052677904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d681003b0 msgr2=0x7f3d68101790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:58.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.294+0000 7f3d6eaab700 1 --2- 172.21.15.1:0/4052677904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d681003b0 0x7f3d68101790 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3d5400c9b0 tx=0x7f3d5400cd70 comp rx=0 tx=0).stop 2023-11-16T16:38:58.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.295+0000 7f3d6eaab700 1 -- 172.21.15.1:0/4052677904 shutdown_connections 2023-11-16T16:38:58.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.295+0000 7f3d6eaab700 1 --2- 172.21.15.1:0/4052677904 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f3d580a1150 0x7f3d580a3600 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:58.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.295+0000 7f3d6eaab700 1 --2- 172.21.15.1:0/4052677904 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3d68100d40 0x7f3d68101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:58.298 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.295+0000 7f3d6eaab700 1 --2- 172.21.15.1:0/4052677904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d680ff1c0 0x7f3d68103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:58.298 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.295+0000 7f3d6eaab700 1 --2- 172.21.15.1:0/4052677904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d681003b0 0x7f3d68101790 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:58.298 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.295+0000 7f3d6eaab700 1 -- 172.21.15.1:0/4052677904 >> 172.21.15.1:0/4052677904 conn(0x7f3d680fa7c0 msgr2=0x7f3d68067dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:58.298 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.295+0000 7f3d6eaab700 1 -- 172.21.15.1:0/4052677904 shutdown_connections 2023-11-16T16:38:58.298 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.295+0000 7f3d6eaab700 1 -- 172.21.15.1:0/4052677904 wait complete. 2023-11-16T16:38:58.310 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-16T16:33:15.916723+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-16T16:37:12.158031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-16T16:37:13.272343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-16T16:38:58.310 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-16T16:38:58.442 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.437+0000 7fecef06f700 1 -- 172.21.15.1:0/2626705758 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fece8100d30 msgr2=0x7fece8103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:58.442 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.437+0000 7fecef06f700 1 --2- 172.21.15.1:0/2626705758 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fece8100d30 0x7fece8103110 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fecdc009990 tx=0x7fecdc009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:58.442 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.437+0000 7fecef06f700 1 -- 172.21.15.1:0/2626705758 shutdown_connections 2023-11-16T16:38:58.443 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.437+0000 7fecef06f700 1 --2- 172.21.15.1:0/2626705758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fece8105f70 0x7fece81083d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:58.443 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.437+0000 7fecef06f700 1 --2- 172.21.15.1:0/2626705758 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fece8100d30 0x7fece8103110 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:58.443 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.437+0000 7fecef06f700 1 --2- 172.21.15.1:0/2626705758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fece8103650 0x7fece8105a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:58.443 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.437+0000 7fecef06f700 1 -- 172.21.15.1:0/2626705758 >> 172.21.15.1:0/2626705758 conn(0x7fece80fa7b0 msgr2=0x7fece80fcc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:58.443 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.437+0000 7fecef06f700 1 -- 172.21.15.1:0/2626705758 shutdown_connections 2023-11-16T16:38:58.443 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.437+0000 7fecef06f700 1 -- 172.21.15.1:0/2626705758 wait complete. 2023-11-16T16:38:58.443 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.437+0000 7fecef06f700 1 Processor -- start 2023-11-16T16:38:58.443 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.437+0000 7fecef06f700 1 -- start start 2023-11-16T16:38:58.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.437+0000 7fecef06f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fece8100d30 0x7fece8195fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:58.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.437+0000 7fecef06f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fece8103650 0x7fece8196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:58.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.437+0000 7fecef06f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fece8105f70 0x7fece819d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:58.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.437+0000 7fecef06f700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece819db40 con 0x7fece8100d30 2023-11-16T16:38:58.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.437+0000 7fecef06f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece819dc80 con 0x7fece8105f70 2023-11-16T16:38:58.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.438+0000 7fecef06f700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece819de10 con 0x7fece8103650 2023-11-16T16:38:58.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.438+0000 7fecece0b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fece8100d30 0x7fece8195fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:58.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.438+0000 7fecece0b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fece8100d30 0x7fece8195fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43022/0 (socket says 172.21.15.1:43022) 2023-11-16T16:38:58.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.438+0000 7fecece0b700 1 -- 172.21.15.1:0/3846338770 learned_addr learned my addr 172.21.15.1:0/3846338770 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:58.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.438+0000 7fece7fff700 1 --2- 172.21.15.1:0/3846338770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fece8103650 0x7fece8196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:58.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.438+0000 7feced60c700 1 --2- 172.21.15.1:0/3846338770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fece8105f70 0x7fece819d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:58.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.438+0000 7fecece0b700 1 -- 172.21.15.1:0/3846338770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fece8103650 msgr2=0x7fece8196500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:58.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.438+0000 7fecece0b700 1 --2- 172.21.15.1:0/3846338770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fece8103650 0x7fece8196500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:58.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.438+0000 7fecece0b700 1 -- 172.21.15.1:0/3846338770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fece8105f70 msgr2=0x7fece819d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:58.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.438+0000 7fecece0b700 1 --2- 172.21.15.1:0/3846338770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fece8105f70 0x7fece819d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:58.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.438+0000 7fecece0b700 1 -- 172.21.15.1:0/3846338770 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecdc0095f0 con 0x7fece8100d30 2023-11-16T16:38:58.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.438+0000 7feced60c700 1 --2- 172.21.15.1:0/3846338770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fece8105f70 0x7fece819d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:38:58.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.438+0000 7fece7fff700 1 --2- 172.21.15.1:0/3846338770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fece8103650 0x7fece8196500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:38:58.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.438+0000 7fecece0b700 1 --2- 172.21.15.1:0/3846338770 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fece8100d30 0x7fece8195fc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fecdc004730 tx=0x7fecdc005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:58.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.438+0000 7fece5ffb700 1 -- 172.21.15.1:0/3846338770 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecdc01d070 con 0x7fece8100d30 2023-11-16T16:38:58.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.438+0000 7fece5ffb700 1 -- 172.21.15.1:0/3846338770 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fecdc004cd0 con 0x7fece8100d30 2023-11-16T16:38:58.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.438+0000 7fece5ffb700 1 -- 172.21.15.1:0/3846338770 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecdc005610 con 0x7fece8100d30 2023-11-16T16:38:58.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.438+0000 7fecef06f700 1 -- 172.21.15.1:0/3846338770 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fece810bc60 con 0x7fece8100d30 2023-11-16T16:38:58.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.438+0000 7fecef06f700 1 -- 172.21.15.1:0/3846338770 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fece810bad0 con 0x7fece8100d30 2023-11-16T16:38:58.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.439+0000 7fecef06f700 1 -- 172.21.15.1:0/3846338770 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fece80fc320 con 0x7fece8100d30 2023-11-16T16:38:58.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.440+0000 7fece5ffb700 1 -- 172.21.15.1:0/3846338770 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fecdc00fc70 con 0x7fece8100d30 2023-11-16T16:38:58.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.441+0000 7fece5ffb700 1 --2- 172.21.15.1:0/3846338770 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fecd80a1150 0x7fecd80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:58.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.441+0000 7fece7fff700 1 --2- 172.21.15.1:0/3846338770 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fecd80a1150 0x7fecd80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:58.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.441+0000 7fece5ffb700 1 -- 172.21.15.1:0/3846338770 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fecdc0cddb0 con 0x7fece8100d30 2023-11-16T16:38:58.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.442+0000 7fece7fff700 1 --2- 172.21.15.1:0/3846338770 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fecd80a1150 0x7fecd80a3600 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fecd4009cb0 tx=0x7fecd4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:58.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.445+0000 7fece5ffb700 1 -- 172.21.15.1:0/3846338770 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fecdc05a730 con 0x7fece8100d30 2023-11-16T16:38:58.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:58 smithi026 ceph-mon[100112]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:38:58.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:58 smithi026 ceph-mon[100112]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:38:58.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:58 smithi053 ceph-mon[100638]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:38:58.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:58 smithi053 ceph-mon[100638]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:38:58.646 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.644+0000 7fecef06f700 1 -- 172.21.15.1:0/3846338770 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fece8061700 con 0x7fece8100d30 2023-11-16T16:38:58.647 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.645+0000 7fece5ffb700 1 -- 172.21.15.1:0/3846338770 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7fecdc097550 con 0x7fece8100d30 2023-11-16T16:38:58.647 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2023-11-16T16:38:58.649 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.647+0000 7fecef06f700 1 -- 172.21.15.1:0/3846338770 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fecd80a1150 msgr2=0x7fecd80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:58.649 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.647+0000 7fecef06f700 1 --2- 172.21.15.1:0/3846338770 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fecd80a1150 0x7fecd80a3600 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fecd4009cb0 tx=0x7fecd4009260 comp rx=0 tx=0).stop 2023-11-16T16:38:58.649 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.647+0000 7fecef06f700 1 -- 172.21.15.1:0/3846338770 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fece8100d30 msgr2=0x7fece8195fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:58.650 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.647+0000 7fecef06f700 1 --2- 172.21.15.1:0/3846338770 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fece8100d30 0x7fece8195fc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fecdc004730 tx=0x7fecdc005ef0 comp rx=0 tx=0).stop 2023-11-16T16:38:58.650 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.647+0000 7fecef06f700 1 -- 172.21.15.1:0/3846338770 shutdown_connections 2023-11-16T16:38:58.650 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.647+0000 7fecef06f700 1 --2- 172.21.15.1:0/3846338770 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fecd80a1150 0x7fecd80a3600 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:58.650 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.647+0000 7fecef06f700 1 --2- 172.21.15.1:0/3846338770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fece8105f70 0x7fece819d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:58.650 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.647+0000 7fecef06f700 1 --2- 172.21.15.1:0/3846338770 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fece8100d30 0x7fece8195fc0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:58.650 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.647+0000 7fecef06f700 1 --2- 172.21.15.1:0/3846338770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fece8103650 0x7fece8196500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:58.651 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.647+0000 7fecef06f700 1 -- 172.21.15.1:0/3846338770 >> 172.21.15.1:0/3846338770 conn(0x7fece80fa7b0 msgr2=0x7fece8101990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:58.651 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.647+0000 7fecef06f700 1 -- 172.21.15.1:0/3846338770 shutdown_connections 2023-11-16T16:38:58.651 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.647+0000 7fecef06f700 1 -- 172.21.15.1:0/3846338770 wait complete. 2023-11-16T16:38:58.660 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:58 smithi001 ceph-mon[98800]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:38:58.660 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:58 smithi001 ceph-mon[98800]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:38:58.661 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-16T16:38:58.794 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.789+0000 7fa5a02b1700 1 -- 172.21.15.1:0/929595780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa598068ac0 msgr2=0x7fa598102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:58.794 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.789+0000 7fa5a02b1700 1 --2- 172.21.15.1:0/929595780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa598068ac0 0x7fa598102e40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa588009990 tx=0x7fa588009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:58.794 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.789+0000 7fa5a02b1700 1 -- 172.21.15.1:0/929595780 shutdown_connections 2023-11-16T16:38:58.794 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.789+0000 7fa5a02b1700 1 --2- 172.21.15.1:0/929595780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa598105e50 0x7fa598108300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:58.794 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.789+0000 7fa5a02b1700 1 --2- 172.21.15.1:0/929595780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa598068ac0 0x7fa598102e40 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:58.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.789+0000 7fa5a02b1700 1 --2- 172.21.15.1:0/929595780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa598103380 0x7fa5981057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:58.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.789+0000 7fa5a02b1700 1 -- 172.21.15.1:0/929595780 >> 172.21.15.1:0/929595780 conn(0x7fa5980fa750 msgr2=0x7fa5980fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:58.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.789+0000 7fa5a02b1700 1 -- 172.21.15.1:0/929595780 shutdown_connections 2023-11-16T16:38:58.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.789+0000 7fa5a02b1700 1 -- 172.21.15.1:0/929595780 wait complete. 2023-11-16T16:38:58.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.790+0000 7fa5a02b1700 1 Processor -- start 2023-11-16T16:38:58.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.790+0000 7fa5a02b1700 1 -- start start 2023-11-16T16:38:58.796 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.790+0000 7fa5a02b1700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa598068ac0 0x7fa598195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:58.796 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.790+0000 7fa5a02b1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa598103380 0x7fa5981964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:58.796 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.790+0000 7fa5a02b1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa598105e50 0x7fa59819d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:58.796 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.790+0000 7fa5a02b1700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59819db30 con 0x7fa598103380 2023-11-16T16:38:58.796 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.790+0000 7fa5a02b1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59819dc70 con 0x7fa598105e50 2023-11-16T16:38:58.796 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.790+0000 7fa5a02b1700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59819de00 con 0x7fa598068ac0 2023-11-16T16:38:58.796 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.790+0000 7fa59d84c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa598103380 0x7fa5981964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:58.796 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.790+0000 7fa59d84c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa598103380 0x7fa5981964f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43040/0 (socket says 172.21.15.1:43040) 2023-11-16T16:38:58.796 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.790+0000 7fa59d84c700 1 -- 172.21.15.1:0/2776563771 learned_addr learned my addr 172.21.15.1:0/2776563771 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:58.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.790+0000 7fa59e04d700 1 --2- 172.21.15.1:0/2776563771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa598068ac0 0x7fa598195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:58.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.790+0000 7fa59e84e700 1 --2- 172.21.15.1:0/2776563771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa598105e50 0x7fa59819d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:58.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.790+0000 7fa59d84c700 1 -- 172.21.15.1:0/2776563771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa598068ac0 msgr2=0x7fa598195fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:58.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.790+0000 7fa59d84c700 1 --2- 172.21.15.1:0/2776563771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa598068ac0 0x7fa598195fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:58.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.790+0000 7fa59d84c700 1 -- 172.21.15.1:0/2776563771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa598105e50 msgr2=0x7fa59819d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:58.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.790+0000 7fa59d84c700 1 --2- 172.21.15.1:0/2776563771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa598105e50 0x7fa59819d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:58.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.790+0000 7fa59d84c700 1 -- 172.21.15.1:0/2776563771 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5880095f0 con 0x7fa598103380 2023-11-16T16:38:58.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.791+0000 7fa59d84c700 1 --2- 172.21.15.1:0/2776563771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa598103380 0x7fa5981964f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa59400e9a0 tx=0x7fa59400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:58.798 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.791+0000 7fa58f7fe700 1 -- 172.21.15.1:0/2776563771 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa594009790 con 0x7fa598103380 2023-11-16T16:38:58.798 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.791+0000 7fa58f7fe700 1 -- 172.21.15.1:0/2776563771 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa59400cbd0 con 0x7fa598103380 2023-11-16T16:38:58.798 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.791+0000 7fa5a02b1700 1 -- 172.21.15.1:0/2776563771 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa59810bc30 con 0x7fa598103380 2023-11-16T16:38:58.798 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.791+0000 7fa58f7fe700 1 -- 172.21.15.1:0/2776563771 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa594018680 con 0x7fa598103380 2023-11-16T16:38:58.798 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.791+0000 7fa5a02b1700 1 -- 172.21.15.1:0/2776563771 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa59810baa0 con 0x7fa598103380 2023-11-16T16:38:58.798 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.792+0000 7fa5a02b1700 1 -- 172.21.15.1:0/2776563771 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5980fc2c0 con 0x7fa598103380 2023-11-16T16:38:58.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.793+0000 7fa58f7fe700 1 -- 172.21.15.1:0/2776563771 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fa59400cd40 con 0x7fa598103380 2023-11-16T16:38:58.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.793+0000 7fa58f7fe700 1 --2- 172.21.15.1:0/2776563771 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fa5840a1150 0x7fa5840a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:58.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.793+0000 7fa59e04d700 1 --2- 172.21.15.1:0/2776563771 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fa5840a1150 0x7fa5840a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:58.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.793+0000 7fa58f7fe700 1 -- 172.21.15.1:0/2776563771 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa594014070 con 0x7fa598103380 2023-11-16T16:38:58.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.794+0000 7fa59e04d700 1 --2- 172.21.15.1:0/2776563771 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fa5840a1150 0x7fa5840a3600 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa58800b2e0 tx=0x7fa588005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:58.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.798+0000 7fa58f7fe700 1 -- 172.21.15.1:0/2776563771 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa594051730 con 0x7fa598103380 2023-11-16T16:38:59.000 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.998+0000 7fa5a02b1700 1 -- 172.21.15.1:0/2776563771 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa598061c00 con 0x7fa598103380 2023-11-16T16:38:59.000 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:58.998+0000 7fa58f7fe700 1 -- 172.21.15.1:0/2776563771 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa594099e70 con 0x7fa598103380 2023-11-16T16:38:59.000 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2023-11-16T16:38:59.003 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.000+0000 7fa5a02b1700 1 -- 172.21.15.1:0/2776563771 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fa5840a1150 msgr2=0x7fa5840a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:59.003 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.000+0000 7fa5a02b1700 1 --2- 172.21.15.1:0/2776563771 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fa5840a1150 0x7fa5840a3600 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa58800b2e0 tx=0x7fa588005fb0 comp rx=0 tx=0).stop 2023-11-16T16:38:59.003 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.000+0000 7fa5a02b1700 1 -- 172.21.15.1:0/2776563771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa598103380 msgr2=0x7fa5981964f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:59.003 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.000+0000 7fa5a02b1700 1 --2- 172.21.15.1:0/2776563771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa598103380 0x7fa5981964f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa59400e9a0 tx=0x7fa59400ed60 comp rx=0 tx=0).stop 2023-11-16T16:38:59.003 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.000+0000 7fa5a02b1700 1 -- 172.21.15.1:0/2776563771 shutdown_connections 2023-11-16T16:38:59.003 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.000+0000 7fa5a02b1700 1 --2- 172.21.15.1:0/2776563771 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fa5840a1150 0x7fa5840a3600 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:59.003 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.000+0000 7fa5a02b1700 1 --2- 172.21.15.1:0/2776563771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa598105e50 0x7fa59819d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:59.003 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.000+0000 7fa5a02b1700 1 --2- 172.21.15.1:0/2776563771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa598068ac0 0x7fa598195fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:59.004 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.000+0000 7fa5a02b1700 1 --2- 172.21.15.1:0/2776563771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa598103380 0x7fa5981964f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:59.004 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.000+0000 7fa5a02b1700 1 -- 172.21.15.1:0/2776563771 >> 172.21.15.1:0/2776563771 conn(0x7fa5980fa750 msgr2=0x7fa5981016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:59.004 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.000+0000 7fa5a02b1700 1 -- 172.21.15.1:0/2776563771 shutdown_connections 2023-11-16T16:38:59.004 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.000+0000 7fa5a02b1700 1 -- 172.21.15.1:0/2776563771 wait complete. 2023-11-16T16:38:59.015 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-16T16:38:59.148 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.142+0000 7fad3ada9700 1 -- 172.21.15.1:0/3217741775 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad340ff170 msgr2=0x7fad340ff580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:59.149 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.142+0000 7fad3ada9700 1 --2- 172.21.15.1:0/3217741775 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad340ff170 0x7fad340ff580 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fad28009990 tx=0x7fad28009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:59.149 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.142+0000 7fad3ada9700 1 -- 172.21.15.1:0/3217741775 shutdown_connections 2023-11-16T16:38:59.149 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.142+0000 7fad3ada9700 1 --2- 172.21.15.1:0/3217741775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad34100db0 0x7fad34107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:59.149 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.142+0000 7fad3ada9700 1 --2- 172.21.15.1:0/3217741775 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad340ff170 0x7fad340ff580 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:59.150 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.142+0000 7fad3ada9700 1 --2- 172.21.15.1:0/3217741775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad34100400 0x7fad34100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:59.150 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.142+0000 7fad3ada9700 1 -- 172.21.15.1:0/3217741775 >> 172.21.15.1:0/3217741775 conn(0x7fad340fa750 msgr2=0x7fad340fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:59.150 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.142+0000 7fad3ada9700 1 -- 172.21.15.1:0/3217741775 shutdown_connections 2023-11-16T16:38:59.150 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.142+0000 7fad3ada9700 1 -- 172.21.15.1:0/3217741775 wait complete. 2023-11-16T16:38:59.150 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.143+0000 7fad3ada9700 1 Processor -- start 2023-11-16T16:38:59.150 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.143+0000 7fad3ada9700 1 -- start start 2023-11-16T16:38:59.150 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.143+0000 7fad3ada9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad340ff170 0x7fad34193db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:59.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.143+0000 7fad3ada9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad34100400 0x7fad341942f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:59.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.143+0000 7fad3ada9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad34100db0 0x7fad3419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:59.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.143+0000 7fad3ada9700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3419b930 con 0x7fad340ff170 2023-11-16T16:38:59.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.143+0000 7fad3ada9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3419ba70 con 0x7fad34100db0 2023-11-16T16:38:59.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.143+0000 7fad3ada9700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3419bc00 con 0x7fad34100400 2023-11-16T16:38:59.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.143+0000 7fad38b45700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad340ff170 0x7fad34193db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:59.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.143+0000 7fad38b45700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad340ff170 0x7fad34193db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43050/0 (socket says 172.21.15.1:43050) 2023-11-16T16:38:59.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.143+0000 7fad38b45700 1 -- 172.21.15.1:0/751739240 learned_addr learned my addr 172.21.15.1:0/751739240 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:59.152 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.143+0000 7fad33fff700 1 --2- 172.21.15.1:0/751739240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad34100400 0x7fad341942f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:59.152 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.143+0000 7fad38b45700 1 -- 172.21.15.1:0/751739240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad34100400 msgr2=0x7fad341942f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:59.152 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.143+0000 7fad38b45700 1 --2- 172.21.15.1:0/751739240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad34100400 0x7fad341942f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:59.152 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.143+0000 7fad39346700 1 --2- 172.21.15.1:0/751739240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad34100db0 0x7fad3419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:59.152 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.143+0000 7fad38b45700 1 -- 172.21.15.1:0/751739240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad34100db0 msgr2=0x7fad3419b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:59.152 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.143+0000 7fad38b45700 1 --2- 172.21.15.1:0/751739240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad34100db0 0x7fad3419b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:59.152 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.143+0000 7fad38b45700 1 -- 172.21.15.1:0/751739240 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad280095f0 con 0x7fad340ff170 2023-11-16T16:38:59.153 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.143+0000 7fad38b45700 1 --2- 172.21.15.1:0/751739240 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad340ff170 0x7fad34193db0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fad28004730 tx=0x7fad28005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:59.153 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.144+0000 7fad31ffb700 1 -- 172.21.15.1:0/751739240 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2801d070 con 0x7fad340ff170 2023-11-16T16:38:59.153 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.144+0000 7fad31ffb700 1 -- 172.21.15.1:0/751739240 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad28004cd0 con 0x7fad340ff170 2023-11-16T16:38:59.153 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.144+0000 7fad3ada9700 1 -- 172.21.15.1:0/751739240 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad34109a70 con 0x7fad340ff170 2023-11-16T16:38:59.153 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.144+0000 7fad31ffb700 1 -- 172.21.15.1:0/751739240 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad28005610 con 0x7fad340ff170 2023-11-16T16:38:59.153 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.144+0000 7fad3ada9700 1 -- 172.21.15.1:0/751739240 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad341098e0 con 0x7fad340ff170 2023-11-16T16:38:59.153 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.145+0000 7fad3ada9700 1 -- 172.21.15.1:0/751739240 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad3419c0b0 con 0x7fad340ff170 2023-11-16T16:38:59.154 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.146+0000 7fad31ffb700 1 -- 172.21.15.1:0/751739240 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fad2800fc70 con 0x7fad340ff170 2023-11-16T16:38:59.155 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.147+0000 7fad31ffb700 1 --2- 172.21.15.1:0/751739240 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fad240a1150 0x7fad240a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:59.155 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.147+0000 7fad31ffb700 1 -- 172.21.15.1:0/751739240 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fad280ce260 con 0x7fad340ff170 2023-11-16T16:38:59.155 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.147+0000 7fad33fff700 1 --2- 172.21.15.1:0/751739240 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fad240a1150 0x7fad240a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:59.155 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.147+0000 7fad33fff700 1 --2- 172.21.15.1:0/751739240 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fad240a1150 0x7fad240a3600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fad2000d440 tx=0x7fad2000f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:59.155 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.150+0000 7fad31ffb700 1 -- 172.21.15.1:0/751739240 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fad28097a00 con 0x7fad340ff170 2023-11-16T16:38:59.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.353+0000 7fad3ada9700 1 -- 172.21.15.1:0/751739240 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fad341061b0 con 0x7fad340ff170 2023-11-16T16:38:59.356 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.354+0000 7fad31ffb700 1 -- 172.21.15.1:0/751739240 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7fad2809b020 con 0x7fad340ff170 2023-11-16T16:38:59.356 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2023-11-16T16:38:59.358 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.356+0000 7fad3ada9700 1 -- 172.21.15.1:0/751739240 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fad240a1150 msgr2=0x7fad240a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:59.358 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.356+0000 7fad3ada9700 1 --2- 172.21.15.1:0/751739240 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fad240a1150 0x7fad240a3600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fad2000d440 tx=0x7fad2000f040 comp rx=0 tx=0).stop 2023-11-16T16:38:59.359 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.356+0000 7fad3ada9700 1 -- 172.21.15.1:0/751739240 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad340ff170 msgr2=0x7fad34193db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:59.359 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.356+0000 7fad3ada9700 1 --2- 172.21.15.1:0/751739240 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad340ff170 0x7fad34193db0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fad28004730 tx=0x7fad28005ef0 comp rx=0 tx=0).stop 2023-11-16T16:38:59.359 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.356+0000 7fad3ada9700 1 -- 172.21.15.1:0/751739240 shutdown_connections 2023-11-16T16:38:59.359 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.356+0000 7fad3ada9700 1 --2- 172.21.15.1:0/751739240 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fad240a1150 0x7fad240a3600 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:59.359 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.356+0000 7fad3ada9700 1 --2- 172.21.15.1:0/751739240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad34100db0 0x7fad3419b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:59.359 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.356+0000 7fad3ada9700 1 --2- 172.21.15.1:0/751739240 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad340ff170 0x7fad34193db0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:59.359 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.356+0000 7fad3ada9700 1 --2- 172.21.15.1:0/751739240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fad34100400 0x7fad341942f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:59.359 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.356+0000 7fad3ada9700 1 -- 172.21.15.1:0/751739240 >> 172.21.15.1:0/751739240 conn(0x7fad340fa750 msgr2=0x7fad340fca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:59.360 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.356+0000 7fad3ada9700 1 -- 172.21.15.1:0/751739240 shutdown_connections 2023-11-16T16:38:59.360 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.356+0000 7fad3ada9700 1 -- 172.21.15.1:0/751739240 wait complete. 2023-11-16T16:38:59.369 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:59 smithi001 ceph-mon[98800]: pgmap v314: 97 pgs: 97 active+clean; 644 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2023-11-16T16:38:59.369 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:59 smithi001 ceph-mon[98800]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:38:59.369 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:59 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/4052677904' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:38:59.370 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:59 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3846338770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:38:59.370 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:38:59 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/2776563771' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:38:59.371 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:38:59.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.495+0000 7fbfcf62e700 1 -- 172.21.15.1:0/1389400689 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfc80ffde0 msgr2=0x7fbfc81076b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:59.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.495+0000 7fbfcf62e700 1 --2- 172.21.15.1:0/1389400689 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfc80ffde0 0x7fbfc81076b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000b0a0 tx=0x7fbfc000b3b0 comp rx=0 tx=0).stop 2023-11-16T16:38:59.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.495+0000 7fbfcf62e700 1 -- 172.21.15.1:0/1389400689 shutdown_connections 2023-11-16T16:38:59.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.495+0000 7fbfcf62e700 1 --2- 172.21.15.1:0/1389400689 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfc80ffde0 0x7fbfc81076b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:59.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.495+0000 7fbfcf62e700 1 --2- 172.21.15.1:0/1389400689 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfc81007b0 0x7fbfc8100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:59.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.495+0000 7fbfcf62e700 1 --2- 172.21.15.1:0/1389400689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc80ff450 0x7fbfc80ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:59.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.495+0000 7fbfcf62e700 1 -- 172.21.15.1:0/1389400689 >> 172.21.15.1:0/1389400689 conn(0x7fbfc80f9f50 msgr2=0x7fbfc80fc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:59.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.496+0000 7fbfcf62e700 1 -- 172.21.15.1:0/1389400689 shutdown_connections 2023-11-16T16:38:59.502 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.496+0000 7fbfcf62e700 1 -- 172.21.15.1:0/1389400689 wait complete. 2023-11-16T16:38:59.502 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.496+0000 7fbfcf62e700 1 Processor -- start 2023-11-16T16:38:59.502 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.496+0000 7fbfcf62e700 1 -- start start 2023-11-16T16:38:59.502 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.497+0000 7fbfcf62e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfc80ff450 0x7fbfc8195010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:59.502 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.497+0000 7fbfcf62e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfc80ffde0 0x7fbfc8195550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:59.502 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.497+0000 7fbfcf62e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc81007b0 0x7fbfc819c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:59.502 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.497+0000 7fbfcf62e700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc819cb90 con 0x7fbfc80ffde0 2023-11-16T16:38:59.503 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.497+0000 7fbfcf62e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc819ccd0 con 0x7fbfc81007b0 2023-11-16T16:38:59.503 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.497+0000 7fbfcf62e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc819ce60 con 0x7fbfc80ff450 2023-11-16T16:38:59.503 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.497+0000 7fbfccbc9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfc80ffde0 0x7fbfc8195550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:59.503 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.497+0000 7fbfccbc9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfc80ffde0 0x7fbfc8195550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43066/0 (socket says 172.21.15.1:43066) 2023-11-16T16:38:59.503 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.497+0000 7fbfccbc9700 1 -- 172.21.15.1:0/1741867817 learned_addr learned my addr 172.21.15.1:0/1741867817 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:59.503 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.497+0000 7fbfcd3ca700 1 --2- 172.21.15.1:0/1741867817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfc80ff450 0x7fbfc8195010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:59.503 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.497+0000 7fbfcdbcb700 1 --2- 172.21.15.1:0/1741867817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc81007b0 0x7fbfc819c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:59.503 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.497+0000 7fbfccbc9700 1 -- 172.21.15.1:0/1741867817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfc80ff450 msgr2=0x7fbfc8195010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:59.503 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.497+0000 7fbfccbc9700 1 --2- 172.21.15.1:0/1741867817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfc80ff450 0x7fbfc8195010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:59.504 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.497+0000 7fbfccbc9700 1 -- 172.21.15.1:0/1741867817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc81007b0 msgr2=0x7fbfc819c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:59.504 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.497+0000 7fbfccbc9700 1 --2- 172.21.15.1:0/1741867817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc81007b0 0x7fbfc819c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:59.504 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.497+0000 7fbfccbc9700 1 -- 172.21.15.1:0/1741867817 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc0009d00 con 0x7fbfc80ffde0 2023-11-16T16:38:59.504 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.497+0000 7fbfccbc9700 1 --2- 172.21.15.1:0/1741867817 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfc80ffde0 0x7fbfc8195550 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbfc400eab0 tx=0x7fbfc400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:59.504 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.497+0000 7fbfba7fc700 1 -- 172.21.15.1:0/1741867817 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4013070 con 0x7fbfc80ffde0 2023-11-16T16:38:59.504 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.497+0000 7fbfcf62e700 1 -- 172.21.15.1:0/1741867817 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc810ab30 con 0x7fbfc80ffde0 2023-11-16T16:38:59.504 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.497+0000 7fbfba7fc700 1 -- 172.21.15.1:0/1741867817 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfc400be80 con 0x7fbfc80ffde0 2023-11-16T16:38:59.504 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.497+0000 7fbfba7fc700 1 -- 172.21.15.1:0/1741867817 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc40163b0 con 0x7fbfc80ffde0 2023-11-16T16:38:59.504 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.497+0000 7fbfcf62e700 1 -- 172.21.15.1:0/1741867817 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc810a9a0 con 0x7fbfc80ffde0 2023-11-16T16:38:59.505 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.498+0000 7fbfcf62e700 1 -- 172.21.15.1:0/1741867817 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfc80ff030 con 0x7fbfc80ffde0 2023-11-16T16:38:59.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.500+0000 7fbfba7fc700 1 -- 172.21.15.1:0/1741867817 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fbfc40175a0 con 0x7fbfc80ffde0 2023-11-16T16:38:59.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.500+0000 7fbfba7fc700 1 --2- 172.21.15.1:0/1741867817 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbfb40a1150 0x7fbfb40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:59.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.500+0000 7fbfba7fc700 1 -- 172.21.15.1:0/1741867817 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbfc40ce1b0 con 0x7fbfc80ffde0 2023-11-16T16:38:59.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.500+0000 7fbfcd3ca700 1 --2- 172.21.15.1:0/1741867817 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbfb40a1150 0x7fbfb40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:59.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.501+0000 7fbfcd3ca700 1 --2- 172.21.15.1:0/1741867817 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbfb40a1150 0x7fbfb40a3600 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8067b80 tx=0x7fbfbc007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:59.507 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.504+0000 7fbfba7fc700 1 -- 172.21.15.1:0/1741867817 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbfc4097900 con 0x7fbfc80ffde0 2023-11-16T16:38:59.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:59 smithi026 ceph-mon[100112]: pgmap v314: 97 pgs: 97 active+clean; 644 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2023-11-16T16:38:59.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:59 smithi026 ceph-mon[100112]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:38:59.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:59 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/4052677904' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:38:59.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:59 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/3846338770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:38:59.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:38:59 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/2776563771' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:38:59.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:59 smithi053 ceph-mon[100638]: pgmap v314: 97 pgs: 97 active+clean; 644 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2023-11-16T16:38:59.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:59 smithi053 ceph-mon[100638]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-16T16:38:59.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:59 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/4052677904' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:38:59.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:59 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/3846338770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:38:59.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:38:59 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/2776563771' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:38:59.751 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.749+0000 7fbfcf62e700 1 -- 172.21.15.1:0/1741867817 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbfc8105f00 con 0x7fbfc80ffde0 2023-11-16T16:38:59.752 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.750+0000 7fbfba7fc700 1 -- 172.21.15.1:0/1741867817 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10774 (secure 0 0 0) 0x7fbfc409af20 con 0x7fbfc80ffde0 2023-11-16T16:38:59.752 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:38:59.752 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2587563684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2587563684},{"type":"v1","addr":"172.21.15.1:6835","nonce":2587563684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/277936168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":277936168},{"type":"v1","addr":"172.21.15.1:6837","nonce":277936168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/1172575045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1172575045},{"type":"v1","addr":"172.21.15.1:6839","nonce":1172575045}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/2631175899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2631175899},{"type":"v1","addr":"172.21.15.1:6841","nonce":2631175899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3693905267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3693905267},{"type":"v1","addr":"172.21.15.53:6833","nonce":3693905267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/164668033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":164668033},{"type":"v1","addr":"172.21.15.26:6833","nonce":164668033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-16T16:37:13.918943+0000","modified":"2023-11-16T16:37:19.292898+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2140542192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2140542192},{"type":"v1","addr":"172.21.15.53:6835","nonce":2140542192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1010760275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1010760275},{"type":"v1","addr":"172.21.15.26:6835","nonce":1010760275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3197060302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3197060302},{"type":"v1","addr":"172.21.15.53:6837","nonce":3197060302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6837/1453504863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1453504863},{"type":"v1","addr":"172.21.15.26:6837","nonce":1453504863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.53:6839/1044753024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1044753024},{"type":"v1","addr":"172.21.15.53:6839","nonce":1044753024}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6839/2192205949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2192205949},{"type":"v1","addr":"172.21.15.26:6839","nonce":2192205949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-16T16:38:59.755 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.752+0000 7fbfcf62e700 1 -- 172.21.15.1:0/1741867817 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbfb40a1150 msgr2=0x7fbfb40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:59.755 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.752+0000 7fbfcf62e700 1 --2- 172.21.15.1:0/1741867817 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbfb40a1150 0x7fbfb40a3600 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8067b80 tx=0x7fbfbc007400 comp rx=0 tx=0).stop 2023-11-16T16:38:59.755 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.752+0000 7fbfcf62e700 1 -- 172.21.15.1:0/1741867817 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfc80ffde0 msgr2=0x7fbfc8195550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:59.755 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.752+0000 7fbfcf62e700 1 --2- 172.21.15.1:0/1741867817 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfc80ffde0 0x7fbfc8195550 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbfc400eab0 tx=0x7fbfc400edc0 comp rx=0 tx=0).stop 2023-11-16T16:38:59.755 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.752+0000 7fbfcf62e700 1 -- 172.21.15.1:0/1741867817 shutdown_connections 2023-11-16T16:38:59.755 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.752+0000 7fbfcf62e700 1 --2- 172.21.15.1:0/1741867817 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fbfb40a1150 0x7fbfb40a3600 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:59.756 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.752+0000 7fbfcf62e700 1 --2- 172.21.15.1:0/1741867817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc81007b0 0x7fbfc819c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:59.756 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.752+0000 7fbfcf62e700 1 --2- 172.21.15.1:0/1741867817 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbfc80ff450 0x7fbfc8195010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:59.756 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.752+0000 7fbfcf62e700 1 --2- 172.21.15.1:0/1741867817 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfc80ffde0 0x7fbfc8195550 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:59.756 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.752+0000 7fbfcf62e700 1 -- 172.21.15.1:0/1741867817 >> 172.21.15.1:0/1741867817 conn(0x7fbfc80f9f50 msgr2=0x7fbfc80fc180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:59.756 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.752+0000 7fbfcf62e700 1 -- 172.21.15.1:0/1741867817 shutdown_connections 2023-11-16T16:38:59.756 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.752+0000 7fbfcf62e700 1 -- 172.21.15.1:0/1741867817 wait complete. 2023-11-16T16:38:59.756 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 23 2023-11-16T16:38:59.767 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-16T16:38:59.893 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.891+0000 7f5d333ba700 1 -- 172.21.15.1:0/369713702 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d2c100340 msgr2=0x7f5d2c100750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:59.893 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.891+0000 7f5d333ba700 1 --2- 172.21.15.1:0/369713702 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d2c100340 0x7f5d2c100750 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5d28009990 tx=0x7f5d28009ca0 comp rx=0 tx=0).stop 2023-11-16T16:38:59.893 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.891+0000 7f5d333ba700 1 -- 172.21.15.1:0/369713702 shutdown_connections 2023-11-16T16:38:59.893 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.891+0000 7f5d333ba700 1 --2- 172.21.15.1:0/369713702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d2c101ec0 0x7f5d2c108780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:59.893 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.891+0000 7f5d333ba700 1 --2- 172.21.15.1:0/369713702 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d2c100340 0x7f5d2c100750 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:59.893 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.891+0000 7f5d333ba700 1 --2- 172.21.15.1:0/369713702 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d2c101530 0x7f5d2c101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:59.894 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.891+0000 7f5d333ba700 1 -- 172.21.15.1:0/369713702 >> 172.21.15.1:0/369713702 conn(0x7f5d2c0fb940 msgr2=0x7f5d2c0fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:38:59.894 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.891+0000 7f5d333ba700 1 -- 172.21.15.1:0/369713702 shutdown_connections 2023-11-16T16:38:59.894 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.891+0000 7f5d333ba700 1 -- 172.21.15.1:0/369713702 wait complete. 2023-11-16T16:38:59.896 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.892+0000 7f5d333ba700 1 Processor -- start 2023-11-16T16:38:59.896 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.892+0000 7f5d333ba700 1 -- start start 2023-11-16T16:38:59.896 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.892+0000 7f5d333ba700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d2c100340 0x7f5d2c196050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:59.896 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.892+0000 7f5d333ba700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d2c101530 0x7f5d2c196590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:59.896 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.892+0000 7f5d333ba700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d2c101ec0 0x7f5d2c19d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:59.897 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.892+0000 7f5d333ba700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c19dbd0 con 0x7f5d2c101ec0 2023-11-16T16:38:59.897 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.892+0000 7f5d333ba700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c19dd10 con 0x7f5d2c100340 2023-11-16T16:38:59.897 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.892+0000 7f5d333ba700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c19dea0 con 0x7f5d2c101530 2023-11-16T16:38:59.897 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.892+0000 7f5d31957700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d2c101ec0 0x7f5d2c19d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:59.897 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.892+0000 7f5d31957700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d2c101ec0 0x7f5d2c19d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43082/0 (socket says 172.21.15.1:43082) 2023-11-16T16:38:59.897 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.892+0000 7f5d31957700 1 -- 172.21.15.1:0/732622619 learned_addr learned my addr 172.21.15.1:0/732622619 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:38:59.897 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.892+0000 7f5d31156700 1 --2- 172.21.15.1:0/732622619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d2c100340 0x7f5d2c196050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:59.897 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.892+0000 7f5d30955700 1 --2- 172.21.15.1:0/732622619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d2c101530 0x7f5d2c196590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:59.897 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.892+0000 7f5d31957700 1 -- 172.21.15.1:0/732622619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d2c101530 msgr2=0x7f5d2c196590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:59.897 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.892+0000 7f5d31957700 1 --2- 172.21.15.1:0/732622619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d2c101530 0x7f5d2c196590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:59.897 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.892+0000 7f5d31957700 1 -- 172.21.15.1:0/732622619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d2c100340 msgr2=0x7f5d2c196050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:38:59.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.892+0000 7f5d31957700 1 --2- 172.21.15.1:0/732622619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d2c100340 0x7f5d2c196050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:38:59.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.892+0000 7f5d31957700 1 -- 172.21.15.1:0/732622619 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d280095f0 con 0x7f5d2c101ec0 2023-11-16T16:38:59.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.892+0000 7f5d31156700 1 --2- 172.21.15.1:0/732622619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d2c100340 0x7f5d2c196050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:38:59.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.892+0000 7f5d30955700 1 --2- 172.21.15.1:0/732622619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d2c101530 0x7f5d2c196590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:38:59.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.893+0000 7f5d31957700 1 --2- 172.21.15.1:0/732622619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d2c101ec0 0x7f5d2c19d600 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5d24007ee0 tx=0x7f5d2400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:59.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.893+0000 7f5d227fc700 1 -- 172.21.15.1:0/732622619 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d24017070 con 0x7f5d2c101ec0 2023-11-16T16:38:59.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.893+0000 7f5d333ba700 1 -- 172.21.15.1:0/732622619 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d2c10bc30 con 0x7f5d2c101ec0 2023-11-16T16:38:59.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.893+0000 7f5d227fc700 1 -- 172.21.15.1:0/732622619 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d2400f040 con 0x7f5d2c101ec0 2023-11-16T16:38:59.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.893+0000 7f5d227fc700 1 -- 172.21.15.1:0/732622619 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d24005740 con 0x7f5d2c101ec0 2023-11-16T16:38:59.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.893+0000 7f5d333ba700 1 -- 172.21.15.1:0/732622619 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d2c10baa0 con 0x7f5d2c101ec0 2023-11-16T16:38:59.899 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.894+0000 7f5d333ba700 1 -- 172.21.15.1:0/732622619 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d2c067f40 con 0x7f5d2c101ec0 2023-11-16T16:38:59.899 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.896+0000 7f5d227fc700 1 -- 172.21.15.1:0/732622619 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f5d240058a0 con 0x7f5d2c101ec0 2023-11-16T16:38:59.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.896+0000 7f5d227fc700 1 --2- 172.21.15.1:0/732622619 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f5d180a1150 0x7f5d180a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:38:59.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.896+0000 7f5d31156700 1 --2- 172.21.15.1:0/732622619 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f5d180a1150 0x7f5d180a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:38:59.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.896+0000 7f5d227fc700 1 -- 172.21.15.1:0/732622619 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5d240cbff0 con 0x7f5d2c101ec0 2023-11-16T16:38:59.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.897+0000 7f5d31156700 1 --2- 172.21.15.1:0/732622619 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f5d180a1150 0x7f5d180a3600 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5d28006010 tx=0x7f5d2800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:38:59.901 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:38:59.900+0000 7f5d227fc700 1 -- 172.21.15.1:0/732622619 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5d24095740 con 0x7f5d2c101ec0 2023-11-16T16:39:00.104 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.102+0000 7f5d333ba700 1 -- 172.21.15.1:0/732622619 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d2c107240 con 0x7f5d2c101ec0 2023-11-16T16:39:00.106 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.104+0000 7f5d227fc700 1 -- 172.21.15.1:0/732622619 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22469 (secure 0 0 0) 0x7f5d24098d60 con 0x7f5d2c101ec0 2023-11-16T16:39:00.106 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:39:00.106 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":86,"fsid":"35d9b010-849d-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:30:06.217778+0000","modified":"2023-11-16T16:38:03.179244+0000","last_up_change":"2023-11-16T16:36:31.735403+0000","last_in_change":"2023-11-16T16:36:14.083021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:33:15.916723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-16T16:37:12.158031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-16T16:37:13.272343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a8480507-5275-4e52-953b-626110beb436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6803","nonce":2469671609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6805","nonce":2469671609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6809","nonce":2469671609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6807","nonce":2469671609}]},"public_addr":"172.21.15.1:6803/2469671609","cluster_addr":"172.21.15.1:6805/2469671609","heartbeat_back_addr":"172.21.15.1:6809/2469671609","heartbeat_front_addr":"172.21.15.1:6807/2469671609","state":["exists","up"]},{"osd":1,"uuid":"1d011db2-99d1-4575-bdc5-51f88a308fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6801","nonce":247994796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6803","nonce":247994796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6807","nonce":247994796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6805","nonce":247994796}]},"public_addr":"172.21.15.26:6801/247994796","cluster_addr":"172.21.15.26:6803/247994796","heartbeat_back_addr":"172.21.15.26:6807/247994796","heartbeat_front_addr":"172.21.15.26:6805/247994796","state":["exists","up"]},{"osd":2,"uuid":"4d7a2bd1-3c8e-4546-a243-f0354473b031","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6801","nonce":1659316290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6803","nonce":1659316290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6807","nonce":1659316290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6805","nonce":1659316290}]},"public_addr":"172.21.15.53:6801/1659316290","cluster_addr":"172.21.15.53:6803/1659316290","heartbeat_back_addr":"172.21.15.53:6807/1659316290","heartbeat_front_addr":"172.21.15.53:6805/1659316290","state":["exists","up"]},{"osd":3,"uuid":"9c290fd9-e121-4a81-9687-b0eb93debd86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6811","nonce":2728987281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6813","nonce":2728987281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6817","nonce":2728987281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6815","nonce":2728987281}]},"public_addr":"172.21.15.1:6811/2728987281","cluster_addr":"172.21.15.1:6813/2728987281","heartbeat_back_addr":"172.21.15.1:6817/2728987281","heartbeat_front_addr":"172.21.15.1:6815/2728987281","state":["exists","up"]},{"osd":4,"uuid":"533226bc-cc57-4f15-9e29-820294d57e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6809","nonce":2954740024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6811","nonce":2954740024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6815","nonce":2954740024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6813","nonce":2954740024}]},"public_addr":"172.21.15.26:6809/2954740024","cluster_addr":"172.21.15.26:6811/2954740024","heartbeat_back_addr":"172.21.15.26:6815/2954740024","heartbeat_front_addr":"172.21.15.26:6813/2954740024","state":["exists","up"]},{"osd":5,"uuid":"d3ab25eb-e1d4-4557-bc89-efc9072401ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6809","nonce":3966666318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6811","nonce":3966666318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6815","nonce":3966666318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6813","nonce":3966666318}]},"public_addr":"172.21.15.53:6809/3966666318","cluster_addr":"172.21.15.53:6811/3966666318","heartbeat_back_addr":"172.21.15.53:6815/3966666318","heartbeat_front_addr":"172.21.15.53:6813/3966666318","state":["exists","up"]},{"osd":6,"uuid":"180f506e-afb8-4043-b0b4-2dc86f6d4bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6819","nonce":3364032646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6821","nonce":3364032646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6825","nonce":3364032646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6823","nonce":3364032646}]},"public_addr":"172.21.15.1:6819/3364032646","cluster_addr":"172.21.15.1:6821/3364032646","heartbeat_back_addr":"172.21.15.1:6825/3364032646","heartbeat_front_addr":"172.21.15.1:6823/3364032646","state":["exists","up"]},{"osd":7,"uuid":"0d837bac-194f-498d-b7fb-b3147e992329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6817","nonce":3766901391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6819","nonce":3766901391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6823","nonce":3766901391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6821","nonce":3766901391}]},"public_addr":"172.21.15.26:6817/3766901391","cluster_addr":"172.21.15.26:6819/3766901391","heartbeat_back_addr":"172.21.15.26:6823/3766901391","heartbeat_front_addr":"172.21.15.26:6821/3766901391","state":["exists","up"]},{"osd":8,"uuid":"933f231a-7454-47b0-9291-f1fa64f0baec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6817","nonce":3224335871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6819","nonce":3224335871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6823","nonce":3224335871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6821","nonce":3224335871}]},"public_addr":"172.21.15.53:6817/3224335871","cluster_addr":"172.21.15.53:6819/3224335871","heartbeat_back_addr":"172.21.15.53:6823/3224335871","heartbeat_front_addr":"172.21.15.53:6821/3224335871","state":["exists","up"]},{"osd":9,"uuid":"245fcbb3-c651-4703-ab80-309dc98bd983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6827","nonce":1391199844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6829","nonce":1391199844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6833","nonce":1391199844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6831","nonce":1391199844}]},"public_addr":"172.21.15.1:6827/1391199844","cluster_addr":"172.21.15.1:6829/1391199844","heartbeat_back_addr":"172.21.15.1:6833/1391199844","heartbeat_front_addr":"172.21.15.1:6831/1391199844","state":["exists","up"]},{"osd":10,"uuid":"2c090e7a-5240-4d52-ae4f-e94fe05383d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6825","nonce":1773545447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6827","nonce":1773545447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6831","nonce":1773545447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6829","nonce":1773545447}]},"public_addr":"172.21.15.26:6825/1773545447","cluster_addr":"172.21.15.26:6827/1773545447","heartbeat_back_addr":"172.21.15.26:6831/1773545447","heartbeat_front_addr":"172.21.15.26:6829/1773545447","state":["exists","up"]},{"osd":11,"uuid":"4511e32d-e469-45e1-8310-0ff400cb7b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6825","nonce":2564669546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6827","nonce":2564669546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6831","nonce":2564669546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6829","nonce":2564669546}]},"public_addr":"172.21.15.53:6825/2564669546","cluster_addr":"172.21.15.53:6827/2564669546","heartbeat_back_addr":"172.21.15.53:6831/2564669546","heartbeat_front_addr":"172.21.15.53:6829/2564669546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:28.140250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:49.885461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:12.521487+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:34.926197+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:56.815594+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:18.387751+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:40.587106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:02.319370+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:23.832298+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:46.338040+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:08.730414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:30.050070+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1494336948":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6801/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6800/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6800/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/1641976669":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/2895923638":"2023-11-17T16:30:35.064289+0000","172.21.15.1:6801/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/78811711":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/4131957708":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/344403698":"2023-11-17T16:30:35.064289+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:39:00.108 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.106+0000 7f5d333ba700 1 -- 172.21.15.1:0/732622619 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f5d180a1150 msgr2=0x7f5d180a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:00.108 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.106+0000 7f5d333ba700 1 --2- 172.21.15.1:0/732622619 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f5d180a1150 0x7f5d180a3600 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5d28006010 tx=0x7f5d2800b540 comp rx=0 tx=0).stop 2023-11-16T16:39:00.108 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.106+0000 7f5d333ba700 1 -- 172.21.15.1:0/732622619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d2c101ec0 msgr2=0x7f5d2c19d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:00.108 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.106+0000 7f5d333ba700 1 --2- 172.21.15.1:0/732622619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d2c101ec0 0x7f5d2c19d600 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5d24007ee0 tx=0x7f5d2400d3b0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.109 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.106+0000 7f5d333ba700 1 -- 172.21.15.1:0/732622619 shutdown_connections 2023-11-16T16:39:00.109 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.106+0000 7f5d333ba700 1 --2- 172.21.15.1:0/732622619 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f5d180a1150 0x7f5d180a3600 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.109 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.106+0000 7f5d333ba700 1 --2- 172.21.15.1:0/732622619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d2c101ec0 0x7f5d2c19d600 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.109 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.106+0000 7f5d333ba700 1 --2- 172.21.15.1:0/732622619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d2c100340 0x7f5d2c196050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.109 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.106+0000 7f5d333ba700 1 --2- 172.21.15.1:0/732622619 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d2c101530 0x7f5d2c196590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.109 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.106+0000 7f5d333ba700 1 -- 172.21.15.1:0/732622619 >> 172.21.15.1:0/732622619 conn(0x7f5d2c0fb940 msgr2=0x7f5d2c0fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:00.109 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.106+0000 7f5d333ba700 1 -- 172.21.15.1:0/732622619 shutdown_connections 2023-11-16T16:39:00.109 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.106+0000 7f5d333ba700 1 -- 172.21.15.1:0/732622619 wait complete. 2023-11-16T16:39:00.121 DEBUG:tasks.fwd_scrub:Yielding 2023-11-16T16:39:00.121 INFO:teuthology.run_tasks:Running task exec... 2023-11-16T16:39:00.130 INFO:teuthology.task.exec:Executing custom commands... 2023-11-16T16:39:00.131 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi001.front.sepia.ceph.com 2023-11-16T16:39:00.131 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2023-11-16T16:39:00.131 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-11-16T16:39:00.131 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-16T16:39:00.131 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-16T16:39:00.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.265+0000 7f28d5b4d700 1 -- 172.21.15.1:0/1000883535 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28d0105f00 msgr2=0x7f28d0108360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:00.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.265+0000 7f28d5b4d700 1 --2- 172.21.15.1:0/1000883535 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28d0105f00 0x7f28d0108360 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f28c800b0a0 tx=0x7f28c800b3b0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.265+0000 7f28d5b4d700 1 -- 172.21.15.1:0/1000883535 shutdown_connections 2023-11-16T16:39:00.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.265+0000 7f28d5b4d700 1 --2- 172.21.15.1:0/1000883535 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28d0105f00 0x7f28d0108360 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.265+0000 7f28d5b4d700 1 --2- 172.21.15.1:0/1000883535 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28d0100cc0 0x7f28d01030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.265+0000 7f28d5b4d700 1 --2- 172.21.15.1:0/1000883535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28d01035e0 0x7f28d01059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.265+0000 7f28d5b4d700 1 -- 172.21.15.1:0/1000883535 >> 172.21.15.1:0/1000883535 conn(0x7f28d00fa720 msgr2=0x7f28d00fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:00.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.265+0000 7f28d5b4d700 1 -- 172.21.15.1:0/1000883535 shutdown_connections 2023-11-16T16:39:00.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.265+0000 7f28d5b4d700 1 -- 172.21.15.1:0/1000883535 wait complete. 2023-11-16T16:39:00.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.265+0000 7f28d5b4d700 1 Processor -- start 2023-11-16T16:39:00.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.266+0000 7f28d5b4d700 1 -- start start 2023-11-16T16:39:00.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.266+0000 7f28d5b4d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28d0100cc0 0x7f28d0100630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.266+0000 7f28d5b4d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28d01035e0 0x7f28d00fec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.266+0000 7f28d5b4d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28d0105f00 0x7f28d00ff1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.266+0000 7f28d5b4d700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d00ff950 con 0x7f28d0105f00 2023-11-16T16:39:00.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.266+0000 7f28d5b4d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d00ffac0 con 0x7f28d0100cc0 2023-11-16T16:39:00.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.266+0000 7f28d5b4d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d01a00a0 con 0x7f28d01035e0 2023-11-16T16:39:00.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.266+0000 7f28cffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28d0105f00 0x7f28d00ff1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:00.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.266+0000 7f28cffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28d0105f00 0x7f28d00ff1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43096/0 (socket says 172.21.15.1:43096) 2023-11-16T16:39:00.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.266+0000 7f28cffff700 1 -- 172.21.15.1:0/1356608954 learned_addr learned my addr 172.21.15.1:0/1356608954 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:00.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.266+0000 7f28cf7fe700 1 --2- 172.21.15.1:0/1356608954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28d0100cc0 0x7f28d0100630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:00.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.266+0000 7f28ceffd700 1 --2- 172.21.15.1:0/1356608954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28d01035e0 0x7f28d00fec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:00.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.266+0000 7f28cffff700 1 -- 172.21.15.1:0/1356608954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28d01035e0 msgr2=0x7f28d00fec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:00.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.266+0000 7f28cffff700 1 --2- 172.21.15.1:0/1356608954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28d01035e0 0x7f28d00fec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.266+0000 7f28cffff700 1 -- 172.21.15.1:0/1356608954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28d0100cc0 msgr2=0x7f28d0100630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:00.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.266+0000 7f28cffff700 1 --2- 172.21.15.1:0/1356608954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28d0100cc0 0x7f28d0100630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.266+0000 7f28cffff700 1 -- 172.21.15.1:0/1356608954 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c8009d00 con 0x7f28d0105f00 2023-11-16T16:39:00.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.267+0000 7f28cffff700 1 --2- 172.21.15.1:0/1356608954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28d0105f00 0x7f28d00ff1b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f28c800b070 tx=0x7f28c8007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:00.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.267+0000 7f28ccff9700 1 -- 172.21.15.1:0/1356608954 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c800e040 con 0x7f28d0105f00 2023-11-16T16:39:00.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.267+0000 7f28d5b4d700 1 -- 172.21.15.1:0/1356608954 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28d010bba0 con 0x7f28d0105f00 2023-11-16T16:39:00.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.267+0000 7f28ccff9700 1 -- 172.21.15.1:0/1356608954 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28c801d470 con 0x7f28d0105f00 2023-11-16T16:39:00.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.267+0000 7f28ccff9700 1 -- 172.21.15.1:0/1356608954 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c801b660 con 0x7f28d0105f00 2023-11-16T16:39:00.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.267+0000 7f28d5b4d700 1 -- 172.21.15.1:0/1356608954 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28d010ba10 con 0x7f28d0105f00 2023-11-16T16:39:00.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.268+0000 7f28d5b4d700 1 -- 172.21.15.1:0/1356608954 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28d00fc290 con 0x7f28d0105f00 2023-11-16T16:39:00.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.269+0000 7f28ccff9700 1 -- 172.21.15.1:0/1356608954 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f28c801d5e0 con 0x7f28d0105f00 2023-11-16T16:39:00.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.269+0000 7f14a0e08700 1 -- 172.21.15.1:0/99629513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f149c0ff1b0 msgr2=0x7f149c0ff5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:00.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.269+0000 7f14a0e08700 1 --2- 172.21.15.1:0/99629513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f149c0ff1b0 0x7f149c0ff5c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f148c009990 tx=0x7f148c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.269+0000 7f14a0e08700 1 -- 172.21.15.1:0/99629513 shutdown_connections 2023-11-16T16:39:00.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.269+0000 7f14a0e08700 1 --2- 172.21.15.1:0/99629513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f149c100d30 0x7f149c1075f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.269+0000 7f14a0e08700 1 --2- 172.21.15.1:0/99629513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f149c0ff1b0 0x7f149c0ff5c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.269+0000 7f14a0e08700 1 --2- 172.21.15.1:0/99629513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f149c1003a0 0x7f149c1007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.269+0000 7f14a0e08700 1 -- 172.21.15.1:0/99629513 >> 172.21.15.1:0/99629513 conn(0x7f149c0fa7b0 msgr2=0x7f149c0fcc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:00.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.270+0000 7f14a0e08700 1 -- 172.21.15.1:0/99629513 shutdown_connections 2023-11-16T16:39:00.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.270+0000 7f14a0e08700 1 -- 172.21.15.1:0/99629513 wait complete. 2023-11-16T16:39:00.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.270+0000 7f28ccff9700 1 --2- 172.21.15.1:0/1356608954 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f28b80a1150 0x7f28b80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.270+0000 7f28cf7fe700 1 --2- 172.21.15.1:0/1356608954 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f28b80a1150 0x7f28b80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:00.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.270+0000 7f28ccff9700 1 -- 172.21.15.1:0/1356608954 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f28c80cdc30 con 0x7f28d0105f00 2023-11-16T16:39:00.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.270+0000 7f28cf7fe700 1 --2- 172.21.15.1:0/1356608954 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f28b80a1150 0x7f28b80a3600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f28c000ac50 tx=0x7f28c000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:00.277 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.270+0000 7f14a0e08700 1 Processor -- start 2023-11-16T16:39:00.277 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.271+0000 7f14a0e08700 1 -- start start 2023-11-16T16:39:00.277 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.271+0000 7f14a0e08700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f149c0ff1b0 0x7f149c193c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.277 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.271+0000 7f14a0e08700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f149c1003a0 0x7f149c199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.277 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.271+0000 7f14a0e08700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f149c100d30 0x7f149c1941d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.277 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.271+0000 7f14a0e08700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149c1ac8e0 con 0x7f149c100d30 2023-11-16T16:39:00.277 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.271+0000 7f14a0e08700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149c1aca20 con 0x7f149c0ff1b0 2023-11-16T16:39:00.277 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.271+0000 7f14a0e08700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149c1acb60 con 0x7f149c1003a0 2023-11-16T16:39:00.277 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.271+0000 7f149ad9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f149c100d30 0x7f149c1941d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:00.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.271+0000 7f149ad9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f149c100d30 0x7f149c1941d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43110/0 (socket says 172.21.15.1:43110) 2023-11-16T16:39:00.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.271+0000 7f149ad9d700 1 -- 172.21.15.1:0/3778043521 learned_addr learned my addr 172.21.15.1:0/3778043521 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:00.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.271+0000 7f149a59c700 1 --2- 172.21.15.1:0/3778043521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f149c0ff1b0 0x7f149c193c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:00.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.271+0000 7f1499d9b700 1 --2- 172.21.15.1:0/3778043521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f149c1003a0 0x7f149c199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:00.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.271+0000 7f149ad9d700 1 -- 172.21.15.1:0/3778043521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f149c1003a0 msgr2=0x7f149c199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:00.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.271+0000 7f149ad9d700 1 --2- 172.21.15.1:0/3778043521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f149c1003a0 0x7f149c199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.271+0000 7f149ad9d700 1 -- 172.21.15.1:0/3778043521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f149c0ff1b0 msgr2=0x7f149c193c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:00.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.271+0000 7f149ad9d700 1 --2- 172.21.15.1:0/3778043521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f149c0ff1b0 0x7f149c193c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.271+0000 7f149ad9d700 1 -- 172.21.15.1:0/3778043521 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f148c0095f0 con 0x7f149c100d30 2023-11-16T16:39:00.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.271+0000 7f149ad9d700 1 --2- 172.21.15.1:0/3778043521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f149c100d30 0x7f149c1941d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f149400ef30 tx=0x7f149400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:00.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.274+0000 7f148b7fe700 1 -- 172.21.15.1:0/3778043521 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1494018070 con 0x7f149c100d30 2023-11-16T16:39:00.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.274+0000 7f148b7fe700 1 -- 172.21.15.1:0/3778043521 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f149400f040 con 0x7f149c100d30 2023-11-16T16:39:00.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.274+0000 7f14a0e08700 1 -- 172.21.15.1:0/3778043521 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f149c10bc20 con 0x7f149c100d30 2023-11-16T16:39:00.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.274+0000 7f148b7fe700 1 -- 172.21.15.1:0/3778043521 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149400cd50 con 0x7f149c100d30 2023-11-16T16:39:00.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.274+0000 7f14a0e08700 1 -- 172.21.15.1:0/3778043521 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f149c10ba90 con 0x7f149c100d30 2023-11-16T16:39:00.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.274+0000 7f28ccff9700 1 -- 172.21.15.1:0/1356608954 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f28c8097410 con 0x7f28d0105f00 2023-11-16T16:39:00.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.276+0000 7f148b7fe700 1 -- 172.21.15.1:0/3778043521 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f1494003a00 con 0x7f149c100d30 2023-11-16T16:39:00.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.277+0000 7f148b7fe700 1 --2- 172.21.15.1:0/3778043521 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f14840a1150 0x7f14840a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.277+0000 7f148b7fe700 1 -- 172.21.15.1:0/3778043521 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f14940cc4f0 con 0x7f149c100d30 2023-11-16T16:39:00.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.277+0000 7f149a59c700 1 --2- 172.21.15.1:0/3778043521 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f14840a1150 0x7f14840a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:00.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.278+0000 7f149a59c700 1 --2- 172.21.15.1:0/3778043521 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f14840a1150 0x7f14840a3600 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f148c006010 tx=0x7f148c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:00.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.278+0000 7f14acf94b80 1 Processor -- start 2023-11-16T16:39:00.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.278+0000 7f14acf94b80 1 -- start start 2023-11-16T16:39:00.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.279+0000 7f14acf94b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b1648c80 0x55a2b1649090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.279+0000 7f14acf94b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2b16495d0 0x55a2b1649bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.279+0000 7f14acf94b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a2b164a190 0x55a2b164a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.279+0000 7f14acf94b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b154e0e0 con 0x55a2b164a190 2023-11-16T16:39:00.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.279+0000 7f14acf94b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b15a0920 con 0x55a2b16495d0 2023-11-16T16:39:00.283 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.279+0000 7f14acf94b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b1599db0 con 0x55a2b1648c80 2023-11-16T16:39:00.283 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.279+0000 7f149ad9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a2b164a190 0x55a2b164a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:00.283 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.279+0000 7f149ad9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a2b164a190 0x55a2b164a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43118/0 (socket says 172.21.15.1:43118) 2023-11-16T16:39:00.283 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.279+0000 7f149ad9d700 1 -- 172.21.15.1:0/1958939590 learned_addr learned my addr 172.21.15.1:0/1958939590 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:00.283 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.279+0000 7f1499d9b700 1 --2- 172.21.15.1:0/1958939590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2b16495d0 0x55a2b1649bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:00.283 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.279+0000 7f149a59c700 1 --2- 172.21.15.1:0/1958939590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b1648c80 0x55a2b1649090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:00.283 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.279+0000 7f149ad9d700 1 -- 172.21.15.1:0/1958939590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b1648c80 msgr2=0x55a2b1649090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:00.283 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.279+0000 7f149ad9d700 1 --2- 172.21.15.1:0/1958939590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b1648c80 0x55a2b1649090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.283 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.279+0000 7f149ad9d700 1 -- 172.21.15.1:0/1958939590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2b16495d0 msgr2=0x55a2b1649bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:00.283 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.279+0000 7f149ad9d700 1 --2- 172.21.15.1:0/1958939590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2b16495d0 0x55a2b1649bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.283 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.279+0000 7f149ad9d700 1 -- 172.21.15.1:0/1958939590 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2b141b6b0 con 0x55a2b164a190 2023-11-16T16:39:00.283 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.279+0000 7f149ad9d700 1 --2- 172.21.15.1:0/1958939590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a2b164a190 0x55a2b164a600 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1494096400 tx=0x7f1494099550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7cc7324800863e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:00.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.279+0000 7f1488ff9700 1 -- 172.21.15.1:0/1958939590 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1494099990 con 0x55a2b164a190 2023-11-16T16:39:00.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.280+0000 7f1488ff9700 1 -- 172.21.15.1:0/1958939590 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f149409d9c0 con 0x55a2b164a190 2023-11-16T16:39:00.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.280+0000 7f1488ff9700 1 -- 172.21.15.1:0/1958939590 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1494098e80 con 0x55a2b164a190 2023-11-16T16:39:00.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.280+0000 7f14acf94b80 1 -- 172.21.15.1:0/1958939590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a2b164a190 msgr2=0x55a2b164a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:00.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.280+0000 7f14acf94b80 1 --2- 172.21.15.1:0/1958939590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a2b164a190 0x55a2b164a600 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1494096400 tx=0x7f1494099550 comp rx=0 tx=0).stop 2023-11-16T16:39:00.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.280+0000 7f14acf94b80 1 -- 172.21.15.1:0/1958939590 shutdown_connections 2023-11-16T16:39:00.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.280+0000 7f14acf94b80 1 --2- 172.21.15.1:0/1958939590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a2b164a190 0x55a2b164a600 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.280+0000 7f14acf94b80 1 --2- 172.21.15.1:0/1958939590 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b1648c80 0x55a2b1649090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.280+0000 7f14acf94b80 1 --2- 172.21.15.1:0/1958939590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2b16495d0 0x55a2b1649bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.280+0000 7f14acf94b80 1 -- 172.21.15.1:0/1958939590 >> 172.21.15.1:0/1958939590 conn(0x55a2b163e3e0 msgr2=0x55a2b16436f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:00.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.280+0000 7f14acf94b80 1 -- 172.21.15.1:0/1958939590 shutdown_connections 2023-11-16T16:39:00.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.280+0000 7f14acf94b80 1 -- 172.21.15.1:0/1958939590 wait complete. 2023-11-16T16:39:00.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.281+0000 7f14acf94b80 1 -- start start 2023-11-16T16:39:00.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.281+0000 7f14acf94b80 1 Processor -- start 2023-11-16T16:39:00.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.281+0000 7f14acf94b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2b1648c80 0x55a2b1652710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.281+0000 7f14acf94b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a2b16495d0 0x55a2b1652c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.281+0000 7f14acf94b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b164a190 0x55a2b1659c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.281+0000 7f14acf94b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b153b960 con 0x55a2b16495d0 2023-11-16T16:39:00.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.281+0000 7f14acf94b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b153b600 con 0x55a2b1648c80 2023-11-16T16:39:00.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.281+0000 7f14acf94b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b141d9a0 con 0x55a2b164a190 2023-11-16T16:39:00.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.281+0000 7f14acf94b80 10 client.0 _ll_register_callbacks cb 0x55a2b1559800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:39:00.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.281+0000 7f1499d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a2b16495d0 0x55a2b1652c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:00.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.281+0000 7f1499d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a2b16495d0 0x55a2b1652c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43122/0 (socket says 172.21.15.1:43122) 2023-11-16T16:39:00.290 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.281+0000 7f1499d9b700 1 -- 172.21.15.1:0/3235190706 learned_addr learned my addr 172.21.15.1:0/3235190706 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:00.290 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.281+0000 7f149ad9d700 1 --2- 172.21.15.1:0/3235190706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b164a190 0x55a2b1659c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:00.290 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.281+0000 7f149a59c700 1 --2- 172.21.15.1:0/3235190706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2b1648c80 0x55a2b1652710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:00.290 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.282+0000 7f1499d9b700 1 -- 172.21.15.1:0/3235190706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b164a190 msgr2=0x55a2b1659c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:00.290 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.282+0000 7f1499d9b700 1 --2- 172.21.15.1:0/3235190706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b164a190 0x55a2b1659c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.290 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.282+0000 7f1499d9b700 1 -- 172.21.15.1:0/3235190706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2b1648c80 msgr2=0x55a2b1652710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:00.290 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.282+0000 7f1499d9b700 1 --2- 172.21.15.1:0/3235190706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2b1648c80 0x55a2b1652710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.290 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.282+0000 7f1499d9b700 1 -- 172.21.15.1:0/3235190706 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f149c10ba90 con 0x55a2b16495d0 2023-11-16T16:39:00.290 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.282+0000 7f1499d9b700 1 --2- 172.21.15.1:0/3235190706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a2b16495d0 0x55a2b1652c50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1490010810 tx=0x7f1490010b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6556c126b017d11 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:00.290 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.282+0000 7f1481ffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:39:00.291 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.282+0000 7f1481ffb700 1 -- 172.21.15.1:0/3235190706 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149000e920 con 0x55a2b16495d0 2023-11-16T16:39:00.291 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.282+0000 7f14acf94b80 1 -- 172.21.15.1:0/3235190706 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a2b163bba0 con 0x55a2b16495d0 2023-11-16T16:39:00.291 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.282+0000 7f1481ffb700 1 -- 172.21.15.1:0/3235190706 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1490017070 con 0x55a2b16495d0 2023-11-16T16:39:00.291 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.282+0000 7f1481ffb700 1 -- 172.21.15.1:0/3235190706 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f149000ea80 con 0x55a2b16495d0 2023-11-16T16:39:00.291 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.282+0000 7f14acf94b80 10 client.15393 fetch_fsmap learned FSMap version 23 2023-11-16T16:39:00.291 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.282+0000 7f14acf94b80 1 -- 172.21.15.1:0/3235190706 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55a2b1560210 con 0x55a2b16495d0 2023-11-16T16:39:00.291 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.282+0000 7f1481ffb700 1 -- 172.21.15.1:0/3235190706 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a2b163bba0 con 0x55a2b16495d0 2023-11-16T16:39:00.291 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.282+0000 7f1481ffb700 1 -- 172.21.15.1:0/3235190706 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1490011440 con 0x55a2b16495d0 2023-11-16T16:39:00.291 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.283+0000 7f14acf94b80 10 client.15393 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:39:00.291 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.283+0000 7f14acf94b80 10 client.15393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:39:00.291 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.283+0000 7f14acf94b80 20 client.15393 populate_metadata read hostname 'smithi001' 2023-11-16T16:39:00.291 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.283+0000 7f14acf94b80 1 --2- 172.21.15.1:0/3235190706 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55a2b165e4e0 0x55a2b16608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.283+0000 7f14acf94b80 4 client.15393 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:39:00.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.283+0000 7f14acf94b80 1 -- 172.21.15.1:0/3235190706 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a2b155c120 con 0x55a2b165e4e0 2023-11-16T16:39:00.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.283+0000 7f149a59c700 1 --2- 172.21.15.1:0/3235190706 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55a2b165e4e0 0x55a2b16608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:00.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.283+0000 7f1481ffb700 10 client.15393 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:39:00.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.283+0000 7f149a59c700 1 --2- 172.21.15.1:0/3235190706 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55a2b165e4e0 0x55a2b16608f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57361db8e6dba288 server_cookie=c117f8dc954a1bcb in_seq=0 out_seq=0 2023-11-16T16:39:00.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.284+0000 7f1481ffb700 1 -- 172.21.15.1:0/3235190706 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a2b155c120 con 0x55a2b165e4e0 2023-11-16T16:39:00.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.284+0000 7f1481ffb700 10 client.15393 handle_command_reply: tid=0 2023-11-16T16:39:00.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.284+0000 7f14acf94b80 1 client.15393 shutdown 2023-11-16T16:39:00.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.284+0000 7f14acf94b80 1 -- 172.21.15.1:0/3235190706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a2b16495d0 msgr2=0x55a2b1652c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:00.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.284+0000 7f14acf94b80 1 --2- 172.21.15.1:0/3235190706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a2b16495d0 0x55a2b1652c50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1490010810 tx=0x7f1490010b20 comp rx=0 tx=0).stop 2023-11-16T16:39:00.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.284+0000 7f14acf94b80 1 -- 172.21.15.1:0/3235190706 shutdown_connections 2023-11-16T16:39:00.293 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.284+0000 7f14acf94b80 1 --2- 172.21.15.1:0/3235190706 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55a2b165e4e0 0x55a2b16608f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.293 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.284+0000 7f14acf94b80 1 --2- 172.21.15.1:0/3235190706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b164a190 0x55a2b1659c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.293 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.284+0000 7f14acf94b80 1 --2- 172.21.15.1:0/3235190706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2b1648c80 0x55a2b1652710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.293 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.284+0000 7f1481ffb700 0 client.15393 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:39:00.293 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.284+0000 7f14acf94b80 1 --2- 172.21.15.1:0/3235190706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a2b16495d0 0x55a2b1652c50 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.293 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.284+0000 7f14acf94b80 1 -- 172.21.15.1:0/3235190706 >> 172.21.15.1:0/3235190706 conn(0x55a2b163e3e0 msgr2=0x55a2b1642e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:00.293 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.284+0000 7f14acf94b80 1 -- 172.21.15.1:0/3235190706 shutdown_connections 2023-11-16T16:39:00.293 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.284+0000 7f14acf94b80 1 -- 172.21.15.1:0/3235190706 wait complete. 2023-11-16T16:39:00.293 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.284+0000 7f14acf94b80 20 client.15393 trim_cache size 0 max 16384 2023-11-16T16:39:00.304 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.302+0000 7f14acf94b80 1 Processor -- start 2023-11-16T16:39:00.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.302+0000 7f14acf94b80 1 -- start start 2023-11-16T16:39:00.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.302+0000 7f14acf94b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2b1648c80 0x55a2b16524c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.302+0000 7f14acf94b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b16495d0 0x55a2b1659bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.302+0000 7f14acf94b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a2b164a190 0x55a2b1652a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.302+0000 7f14acf94b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b165b980 con 0x55a2b164a190 2023-11-16T16:39:00.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.302+0000 7f14acf94b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b164a010 con 0x55a2b1648c80 2023-11-16T16:39:00.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.302+0000 7f14acf94b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b1651bc0 con 0x55a2b16495d0 2023-11-16T16:39:00.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.302+0000 7f149ad9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a2b164a190 0x55a2b1652a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:00.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.302+0000 7f149ad9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a2b164a190 0x55a2b1652a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43132/0 (socket says 172.21.15.1:43132) 2023-11-16T16:39:00.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.302+0000 7f149ad9d700 1 -- 172.21.15.1:0/3227004466 learned_addr learned my addr 172.21.15.1:0/3227004466 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:00.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.302+0000 7f149a59c700 1 --2- 172.21.15.1:0/3227004466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2b1648c80 0x55a2b16524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:00.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.302+0000 7f1499d9b700 1 --2- 172.21.15.1:0/3227004466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b16495d0 0x55a2b1659bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:00.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.302+0000 7f149ad9d700 1 -- 172.21.15.1:0/3227004466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b16495d0 msgr2=0x55a2b1659bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:00.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.302+0000 7f149ad9d700 1 --2- 172.21.15.1:0/3227004466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b16495d0 0x55a2b1659bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.302+0000 7f149ad9d700 1 -- 172.21.15.1:0/3227004466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2b1648c80 msgr2=0x55a2b16524c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:00.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.302+0000 7f149ad9d700 1 --2- 172.21.15.1:0/3227004466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2b1648c80 0x55a2b16524c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.302+0000 7f149ad9d700 1 -- 172.21.15.1:0/3227004466 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2b155ff60 con 0x55a2b164a190 2023-11-16T16:39:00.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.302+0000 7f149a59c700 1 --2- 172.21.15.1:0/3227004466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2b1648c80 0x55a2b16524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:39:00.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.302+0000 7f149ad9d700 1 --2- 172.21.15.1:0/3227004466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a2b164a190 0x55a2b1652a00 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1494099920 tx=0x7f14940092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a29d8b8ba14d50 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:00.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.302+0000 7f14897fa700 1 -- 172.21.15.1:0/3227004466 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1494099990 con 0x55a2b164a190 2023-11-16T16:39:00.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.303+0000 7f14897fa700 1 -- 172.21.15.1:0/3227004466 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14940a1d50 con 0x55a2b164a190 2023-11-16T16:39:00.307 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.303+0000 7f14897fa700 1 -- 172.21.15.1:0/3227004466 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1494099e50 con 0x55a2b164a190 2023-11-16T16:39:00.311 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.303+0000 7f14acf94b80 1 -- 172.21.15.1:0/3227004466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a2b164a190 msgr2=0x55a2b1652a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:00.311 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.303+0000 7f14acf94b80 1 --2- 172.21.15.1:0/3227004466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a2b164a190 0x55a2b1652a00 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1494099920 tx=0x7f14940092e0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.311 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.303+0000 7f14acf94b80 1 -- 172.21.15.1:0/3227004466 shutdown_connections 2023-11-16T16:39:00.311 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.303+0000 7f14acf94b80 1 --2- 172.21.15.1:0/3227004466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a2b164a190 0x55a2b1652a00 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.311 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.303+0000 7f14acf94b80 1 --2- 172.21.15.1:0/3227004466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2b1648c80 0x55a2b16524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.311 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.303+0000 7f14acf94b80 1 --2- 172.21.15.1:0/3227004466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b16495d0 0x55a2b1659bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.311 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.303+0000 7f14acf94b80 1 -- 172.21.15.1:0/3227004466 >> 172.21.15.1:0/3227004466 conn(0x55a2b163e3e0 msgr2=0x55a2b164d040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:00.311 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.303+0000 7f14acf94b80 1 -- 172.21.15.1:0/3227004466 shutdown_connections 2023-11-16T16:39:00.311 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.303+0000 7f14acf94b80 1 -- 172.21.15.1:0/3227004466 wait complete. 2023-11-16T16:39:00.311 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.304+0000 7f14acf94b80 1 -- start start 2023-11-16T16:39:00.311 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.304+0000 7f14acf94b80 1 Processor -- start 2023-11-16T16:39:00.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.304+0000 7f14acf94b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2b1648c80 0x55a2b16598f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.304+0000 7f14acf94b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b16495d0 0x55a2b1656390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.304+0000 7f14acf94b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a2b164a190 0x55a2b16568d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.304+0000 7f14acf94b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b1656e10 con 0x55a2b164a190 2023-11-16T16:39:00.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.304+0000 7f14acf94b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b1656f80 con 0x55a2b1648c80 2023-11-16T16:39:00.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.304+0000 7f14acf94b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2b16b2390 con 0x55a2b16495d0 2023-11-16T16:39:00.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.304+0000 7f14acf94b80 10 client.0 _ll_register_callbacks cb 0x7f14900103d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:39:00.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.304+0000 7f149ad9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a2b164a190 0x55a2b16568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:00.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.304+0000 7f149ad9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a2b164a190 0x55a2b16568d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43138/0 (socket says 172.21.15.1:43138) 2023-11-16T16:39:00.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.304+0000 7f149ad9d700 1 -- 172.21.15.1:0/4177033098 learned_addr learned my addr 172.21.15.1:0/4177033098 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:00.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.304+0000 7f1499d9b700 1 --2- 172.21.15.1:0/4177033098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b16495d0 0x55a2b1656390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:00.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.304+0000 7f149a59c700 1 --2- 172.21.15.1:0/4177033098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2b1648c80 0x55a2b16598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:00.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.304+0000 7f149ad9d700 1 -- 172.21.15.1:0/4177033098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b16495d0 msgr2=0x55a2b1656390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:00.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.304+0000 7f149ad9d700 1 --2- 172.21.15.1:0/4177033098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b16495d0 0x55a2b1656390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.304+0000 7f149ad9d700 1 -- 172.21.15.1:0/4177033098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2b1648c80 msgr2=0x55a2b16598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:00.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.304+0000 7f149ad9d700 1 --2- 172.21.15.1:0/4177033098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2b1648c80 0x55a2b16598f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.304+0000 7f149ad9d700 1 -- 172.21.15.1:0/4177033098 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2b141b6b0 con 0x55a2b164a190 2023-11-16T16:39:00.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.304+0000 7f1499d9b700 1 --2- 172.21.15.1:0/4177033098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b16495d0 0x55a2b1656390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:39:00.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.304+0000 7f149a59c700 1 --2- 172.21.15.1:0/4177033098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2b1648c80 0x55a2b16598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:39:00.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.305+0000 7f149ad9d700 1 --2- 172.21.15.1:0/4177033098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a2b164a190 0x55a2b16568d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f149400c7a0 tx=0x7f1494097b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb6d964dc1d4e38 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:00.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.305+0000 7f1481ffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:39:00.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.305+0000 7f1481ffb700 1 -- 172.21.15.1:0/4177033098 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14940c5020 con 0x55a2b164a190 2023-11-16T16:39:00.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.305+0000 7f1481ffb700 1 -- 172.21.15.1:0/4177033098 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f14940ca070 con 0x55a2b164a190 2023-11-16T16:39:00.314 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.305+0000 7f14acf94b80 1 -- 172.21.15.1:0/4177033098 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f149409d9c0 con 0x55a2b164a190 2023-11-16T16:39:00.314 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.305+0000 7f1481ffb700 1 -- 172.21.15.1:0/4177033098 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14940ca230 con 0x55a2b164a190 2023-11-16T16:39:00.314 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.305+0000 7f14acf94b80 10 client.15399 fetch_fsmap learned FSMap version 23 2023-11-16T16:39:00.314 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.305+0000 7f14acf94b80 1 -- 172.21.15.1:0/4177033098 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55a2b13b0ce0 con 0x55a2b164a190 2023-11-16T16:39:00.314 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.305+0000 7f1481ffb700 1 -- 172.21.15.1:0/4177033098 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1494065020 con 0x55a2b164a190 2023-11-16T16:39:00.314 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.305+0000 7f1481ffb700 1 -- 172.21.15.1:0/4177033098 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f14940cc0b0 con 0x55a2b164a190 2023-11-16T16:39:00.314 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.305+0000 7f14acf94b80 10 client.15399 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:39:00.314 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.305+0000 7f14acf94b80 10 client.15399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:39:00.314 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.305+0000 7f14acf94b80 20 client.15399 populate_metadata read hostname 'smithi001' 2023-11-16T16:39:00.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.305+0000 7f14acf94b80 1 --2- 172.21.15.1:0/4177033098 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55a2b16b38e0 0x55a2b16b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.305+0000 7f14acf94b80 4 client.15399 mds_command: new command op to 24521 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-16T16:39:00.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.305+0000 7f14acf94b80 1 -- 172.21.15.1:0/4177033098 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55a2b15686e0 con 0x55a2b16b38e0 2023-11-16T16:39:00.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.306+0000 7f149a59c700 1 --2- 172.21.15.1:0/4177033098 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55a2b16b38e0 0x55a2b16b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:00.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.306+0000 7f1481ffb700 10 client.15399 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:39:00.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.306+0000 7f149a59c700 1 --2- 172.21.15.1:0/4177033098 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55a2b16b38e0 0x55a2b16b5cf0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb6b92eb994abe31 server_cookie=e6c415538b9ba37b in_seq=0 out_seq=0 2023-11-16T16:39:00.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.307+0000 7f1481ffb700 1 -- 172.21.15.1:0/4177033098 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a2b15686e0 con 0x55a2b16b38e0 2023-11-16T16:39:00.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.307+0000 7f1481ffb700 10 client.15399 handle_command_reply: tid=0 2023-11-16T16:39:00.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.307+0000 7f14acf94b80 1 client.15399 shutdown 2023-11-16T16:39:00.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.307+0000 7f14acf94b80 1 -- 172.21.15.1:0/4177033098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a2b164a190 msgr2=0x55a2b16568d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:00.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.307+0000 7f14acf94b80 1 --2- 172.21.15.1:0/4177033098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a2b164a190 0x55a2b16568d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f149400c7a0 tx=0x7f1494097b90 comp rx=0 tx=0).stop 2023-11-16T16:39:00.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.308+0000 7f14acf94b80 1 -- 172.21.15.1:0/4177033098 shutdown_connections 2023-11-16T16:39:00.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.308+0000 7f14acf94b80 1 --2- 172.21.15.1:0/4177033098 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55a2b16b38e0 0x55a2b16b5cf0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.316 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.308+0000 7f14acf94b80 1 --2- 172.21.15.1:0/4177033098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a2b164a190 0x55a2b16568d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.316 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.308+0000 7f14acf94b80 1 --2- 172.21.15.1:0/4177033098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a2b1648c80 0x55a2b16598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.316 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.308+0000 7f14acf94b80 1 --2- 172.21.15.1:0/4177033098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a2b16495d0 0x55a2b1656390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.316 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.308+0000 7f14acf94b80 1 -- 172.21.15.1:0/4177033098 >> 172.21.15.1:0/4177033098 conn(0x55a2b163e3e0 msgr2=0x55a2b16524c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:00.316 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.308+0000 7f1481ffb700 0 client.15399 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:39:00.316 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.308+0000 7f14acf94b80 1 -- 172.21.15.1:0/4177033098 shutdown_connections 2023-11-16T16:39:00.316 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.308+0000 7f14acf94b80 1 -- 172.21.15.1:0/4177033098 wait complete. 2023-11-16T16:39:00.316 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.308+0000 7f14acf94b80 20 client.15399 trim_cache size 0 max 16384 2023-11-16T16:39:00.316 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:39:00.316 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2023-11-16T16:39:00.317 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "b9b2b2e4-79c5-46f1-807f-44024b6eba20", 2023-11-16T16:39:00.317 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2023-11-16T16:39:00.317 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:39:00.317 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.308+0000 7f14a0e08700 1 -- 172.21.15.1:0/3778043521 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f14840a1150 msgr2=0x7f14840a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:00.318 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.308+0000 7f14a0e08700 1 --2- 172.21.15.1:0/3778043521 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f14840a1150 0x7f14840a3600 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f148c006010 tx=0x7f148c005960 comp rx=0 tx=0).stop 2023-11-16T16:39:00.318 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.308+0000 7f14a0e08700 1 -- 172.21.15.1:0/3778043521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f149c100d30 msgr2=0x7f149c1941d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:00.318 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.308+0000 7f14a0e08700 1 --2- 172.21.15.1:0/3778043521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f149c100d30 0x7f149c1941d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f149400ef30 tx=0x7f149400c3a0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.318 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.308+0000 7f14a0e08700 1 -- 172.21.15.1:0/3778043521 shutdown_connections 2023-11-16T16:39:00.318 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.308+0000 7f14a0e08700 1 --2- 172.21.15.1:0/3778043521 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f14840a1150 0x7f14840a3600 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.318 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.308+0000 7f14a0e08700 1 --2- 172.21.15.1:0/3778043521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f149c100d30 0x7f149c1941d0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.318 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.308+0000 7f14a0e08700 1 --2- 172.21.15.1:0/3778043521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f149c0ff1b0 0x7f149c193c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.318 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.308+0000 7f14a0e08700 1 --2- 172.21.15.1:0/3778043521 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f149c1003a0 0x7f149c199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.318 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.308+0000 7f14a0e08700 1 -- 172.21.15.1:0/3778043521 >> 172.21.15.1:0/3778043521 conn(0x7f149c0fa7b0 msgr2=0x7f149c0fc990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:00.318 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.309+0000 7f14a0e08700 1 -- 172.21.15.1:0/3778043521 shutdown_connections 2023-11-16T16:39:00.318 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.309+0000 7f14a0e08700 1 -- 172.21.15.1:0/3778043521 wait complete. 2023-11-16T16:39:00.323 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-16T16:39:00.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:00 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/751739240' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:39:00.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:00 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1741867817' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:39:00.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:00 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/732622619' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:39:00.451 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.449+0000 7f1ecafa8700 1 -- 172.21.15.1:0/2382690096 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ec4105760 msgr2=0x7f1ec4107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:00.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.449+0000 7f1ecafa8700 1 --2- 172.21.15.1:0/2382690096 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ec4105760 0x7f1ec4107b40 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0009990 tx=0x7f1eb0009ca0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.450+0000 7f1ecafa8700 1 -- 172.21.15.1:0/2382690096 shutdown_connections 2023-11-16T16:39:00.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.450+0000 7f1ecafa8700 1 --2- 172.21.15.1:0/2382690096 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ec4108080 0x7f1ec410a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.450+0000 7f1ecafa8700 1 --2- 172.21.15.1:0/2382690096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ec4068ae0 0x7f1ec4105220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.450+0000 7f1ecafa8700 1 --2- 172.21.15.1:0/2382690096 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ec4105760 0x7f1ec4107b40 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.450+0000 7f1ecafa8700 1 -- 172.21.15.1:0/2382690096 >> 172.21.15.1:0/2382690096 conn(0x7f1ec40fa750 msgr2=0x7f1ec40fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:00.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.450+0000 7f1ecafa8700 1 -- 172.21.15.1:0/2382690096 shutdown_connections 2023-11-16T16:39:00.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.450+0000 7f1ecafa8700 1 -- 172.21.15.1:0/2382690096 wait complete. 2023-11-16T16:39:00.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.450+0000 7f1ecafa8700 1 Processor -- start 2023-11-16T16:39:00.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.451+0000 7f1ecafa8700 1 -- start start 2023-11-16T16:39:00.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.451+0000 7f1ecafa8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ec4068ae0 0x7f1ec41981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.451+0000 7f1ecafa8700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ec4105760 0x7f1ec4198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.451+0000 7f1ecafa8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ec4108080 0x7f1ec419f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.451+0000 7f1ecafa8700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec419fd70 con 0x7f1ec4108080 2023-11-16T16:39:00.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.451+0000 7f1ecafa8700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec419feb0 con 0x7f1ec4068ae0 2023-11-16T16:39:00.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.451+0000 7f1ecafa8700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec41a0040 con 0x7f1ec4105760 2023-11-16T16:39:00.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.451+0000 7f1ec9545700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ec4108080 0x7f1ec419f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:00.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.451+0000 7f1ec8d44700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ec4068ae0 0x7f1ec41981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:00.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.451+0000 7f1ec9545700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ec4108080 0x7f1ec419f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43152/0 (socket says 172.21.15.1:43152) 2023-11-16T16:39:00.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.451+0000 7f1ec9545700 1 -- 172.21.15.1:0/2363349011 learned_addr learned my addr 172.21.15.1:0/2363349011 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:00.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.451+0000 7f1ec3fff700 1 --2- 172.21.15.1:0/2363349011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ec4105760 0x7f1ec4198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:00.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.451+0000 7f1ec9545700 1 -- 172.21.15.1:0/2363349011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ec4105760 msgr2=0x7f1ec4198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:00.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.451+0000 7f1ec9545700 1 --2- 172.21.15.1:0/2363349011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ec4105760 0x7f1ec4198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.451+0000 7f1ec9545700 1 -- 172.21.15.1:0/2363349011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ec4068ae0 msgr2=0x7f1ec41981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:00.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.451+0000 7f1ec9545700 1 --2- 172.21.15.1:0/2363349011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ec4068ae0 0x7f1ec41981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.451+0000 7f1ec9545700 1 -- 172.21.15.1:0/2363349011 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb00095f0 con 0x7f1ec4108080 2023-11-16T16:39:00.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.451+0000 7f1ec9545700 1 --2- 172.21.15.1:0/2363349011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ec4108080 0x7f1ec419f7a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc007ee0 tx=0x7f1ebc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:00.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.452+0000 7f1ec1ffb700 1 -- 172.21.15.1:0/2363349011 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc017070 con 0x7f1ec4108080 2023-11-16T16:39:00.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.452+0000 7f1ec1ffb700 1 -- 172.21.15.1:0/2363349011 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ebc00f040 con 0x7f1ec4108080 2023-11-16T16:39:00.456 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.452+0000 7f1ecafa8700 1 -- 172.21.15.1:0/2363349011 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ec410ddf0 con 0x7f1ec4108080 2023-11-16T16:39:00.456 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.452+0000 7f1ec1ffb700 1 -- 172.21.15.1:0/2363349011 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc005740 con 0x7f1ec4108080 2023-11-16T16:39:00.456 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.452+0000 7f1ecafa8700 1 -- 172.21.15.1:0/2363349011 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ec410dc60 con 0x7f1ec4108080 2023-11-16T16:39:00.456 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.454+0000 7f1ec1ffb700 1 -- 172.21.15.1:0/2363349011 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f1ebc00dc60 con 0x7f1ec4108080 2023-11-16T16:39:00.457 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.454+0000 7f1ec1ffb700 1 --2- 172.21.15.1:0/2363349011 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f1eb40a11d0 0x7f1eb40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.457 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.455+0000 7f1ec1ffb700 1 -- 172.21.15.1:0/2363349011 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1ebc0cc100 con 0x7f1ec4108080 2023-11-16T16:39:00.457 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.455+0000 7f1ec8d44700 1 --2- 172.21.15.1:0/2363349011 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f1eb40a11d0 0x7f1eb40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:00.457 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.455+0000 7f1ec8d44700 1 --2- 172.21.15.1:0/2363349011 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f1eb40a11d0 0x7f1eb40a3680 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8005950 tx=0x7f1eb800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:00.460 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.457+0000 7f1ed7134b80 1 Processor -- start 2023-11-16T16:39:00.460 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.457+0000 7f1ed7134b80 1 -- start start 2023-11-16T16:39:00.460 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.457+0000 7f1ed7134b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a45864dc00 0x55a45864e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.460 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.457+0000 7f1ed7134b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a45864e550 0x55a45864eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.460 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.457+0000 7f1ed7134b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45864f110 0x55a45864f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.460 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.457+0000 7f1ed7134b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a458553060 con 0x55a45864f110 2023-11-16T16:39:00.460 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.457+0000 7f1ed7134b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4585a58a0 con 0x55a45864e550 2023-11-16T16:39:00.460 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.457+0000 7f1ed7134b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a45859ed30 con 0x55a45864dc00 2023-11-16T16:39:00.461 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.457+0000 7f1ec9545700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45864f110 0x55a45864f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:00.461 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.457+0000 7f1ec9545700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45864f110 0x55a45864f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43160/0 (socket says 172.21.15.1:43160) 2023-11-16T16:39:00.461 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.457+0000 7f1ec9545700 1 -- 172.21.15.1:0/1215279058 learned_addr learned my addr 172.21.15.1:0/1215279058 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:00.461 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.457+0000 7f1ec3fff700 1 --2- 172.21.15.1:0/1215279058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a45864e550 0x55a45864eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:00.461 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.457+0000 7f1ec8d44700 1 --2- 172.21.15.1:0/1215279058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a45864dc00 0x55a45864e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:00.461 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.458+0000 7f1ec3fff700 1 -- 172.21.15.1:0/1215279058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a45864dc00 msgr2=0x55a45864e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:00.461 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.458+0000 7f1ec3fff700 1 --2- 172.21.15.1:0/1215279058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a45864dc00 0x55a45864e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.461 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.458+0000 7f1ec3fff700 1 -- 172.21.15.1:0/1215279058 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45864f110 msgr2=0x55a45864f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:00.461 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.458+0000 7f1ec3fff700 1 --2- 172.21.15.1:0/1215279058 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45864f110 0x55a45864f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.461 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.458+0000 7f1ec3fff700 1 -- 172.21.15.1:0/1215279058 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a458420630 con 0x55a45864e550 2023-11-16T16:39:00.461 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.458+0000 7f1ec8d44700 1 --2- 172.21.15.1:0/1215279058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a45864dc00 0x55a45864e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:39:00.461 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.458+0000 7f1ec9545700 1 --2- 172.21.15.1:0/1215279058 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45864f110 0x55a45864f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:39:00.461 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.458+0000 7f1ec3fff700 1 --2- 172.21.15.1:0/1215279058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a45864e550 0x55a45864eb30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1eb00059d0 tx=0x7f1eb0005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2997f7dec9fbcef server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:00.462 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.458+0000 7f1eaeffd700 1 -- 172.21.15.1:0/1215279058 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0005a00 con 0x55a45864e550 2023-11-16T16:39:00.462 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.458+0000 7f1eaeffd700 1 -- 172.21.15.1:0/1215279058 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1eb001d040 con 0x55a45864e550 2023-11-16T16:39:00.462 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.458+0000 7f1eaeffd700 1 -- 172.21.15.1:0/1215279058 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0005310 con 0x55a45864e550 2023-11-16T16:39:00.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.459+0000 7f1ed7134b80 1 -- 172.21.15.1:0/1215279058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a45864e550 msgr2=0x55a45864eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:00.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.459+0000 7f1ed7134b80 1 --2- 172.21.15.1:0/1215279058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a45864e550 0x55a45864eb30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1eb00059d0 tx=0x7f1eb0005e80 comp rx=0 tx=0).stop 2023-11-16T16:39:00.466 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.459+0000 7f1ed7134b80 1 -- 172.21.15.1:0/1215279058 shutdown_connections 2023-11-16T16:39:00.466 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.459+0000 7f1ed7134b80 1 --2- 172.21.15.1:0/1215279058 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45864f110 0x55a45864f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.466 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.459+0000 7f1ed7134b80 1 --2- 172.21.15.1:0/1215279058 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a45864dc00 0x55a45864e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.466 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.459+0000 7f1ed7134b80 1 --2- 172.21.15.1:0/1215279058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a45864e550 0x55a45864eb30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.466 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.459+0000 7f1ed7134b80 1 -- 172.21.15.1:0/1215279058 >> 172.21.15.1:0/1215279058 conn(0x55a458643360 msgr2=0x55a458648670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:00.466 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.459+0000 7f1ed7134b80 1 -- 172.21.15.1:0/1215279058 shutdown_connections 2023-11-16T16:39:00.466 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.459+0000 7f1ed7134b80 1 -- 172.21.15.1:0/1215279058 wait complete. 2023-11-16T16:39:00.466 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.459+0000 7f1ed7134b80 1 -- start start 2023-11-16T16:39:00.466 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.459+0000 7f1ed7134b80 1 Processor -- start 2023-11-16T16:39:00.466 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.460+0000 7f1ed7134b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a45864dc00 0x55a4586575b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.466 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.460+0000 7f1ed7134b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a45864e550 0x55a458657af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.466 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.460+0000 7f1ed7134b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45864f110 0x55a45865eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.467 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.460+0000 7f1ed7134b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4585408e0 con 0x55a45864f110 2023-11-16T16:39:00.467 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.460+0000 7f1ed7134b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a458540580 con 0x55a45864e550 2023-11-16T16:39:00.467 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.460+0000 7f1ed7134b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a458422920 con 0x55a45864dc00 2023-11-16T16:39:00.467 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.460+0000 7f1ed7134b80 10 client.0 _ll_register_callbacks cb 0x55a45855e780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:39:00.467 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.460+0000 7f1ec9545700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45864f110 0x55a45865eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:00.467 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.460+0000 7f1ec9545700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45864f110 0x55a45865eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43166/0 (socket says 172.21.15.1:43166) 2023-11-16T16:39:00.467 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.460+0000 7f1ec9545700 1 -- 172.21.15.1:0/1381550405 learned_addr learned my addr 172.21.15.1:0/1381550405 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:00.467 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.460+0000 7f1ec8d44700 1 --2- 172.21.15.1:0/1381550405 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a45864dc00 0x55a4586575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:00.467 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.460+0000 7f1ec9545700 1 -- 172.21.15.1:0/1381550405 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a45864dc00 msgr2=0x55a4586575b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:00.467 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.460+0000 7f1ec9545700 1 --2- 172.21.15.1:0/1381550405 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a45864dc00 0x55a4586575b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.467 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.460+0000 7f1ec9545700 1 -- 172.21.15.1:0/1381550405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a45864e550 msgr2=0x55a458657af0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-16T16:39:00.467 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.460+0000 7f1ec9545700 1 --2- 172.21.15.1:0/1381550405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a45864e550 0x55a458657af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.467 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.460+0000 7f1ec9545700 1 -- 172.21.15.1:0/1381550405 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ec410b1c0 con 0x55a45864f110 2023-11-16T16:39:00.468 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.460+0000 7f1ec9545700 1 --2- 172.21.15.1:0/1381550405 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45864f110 0x55a45865eb30 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc099410 tx=0x7f1ebc004c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a877d7e906235e59 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:00.468 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.460+0000 7f1ea3fff700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:39:00.468 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.461+0000 7f1ea3fff700 1 -- 172.21.15.1:0/1381550405 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc0a1920 con 0x55a45864f110 2023-11-16T16:39:00.468 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.461+0000 7f1ea3fff700 1 -- 172.21.15.1:0/1381550405 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1ebc0840b0 con 0x55a45864f110 2023-11-16T16:39:00.468 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.461+0000 7f1ed7134b80 1 -- 172.21.15.1:0/1381550405 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a458640b20 con 0x55a45864f110 2023-11-16T16:39:00.468 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.461+0000 7f1ea3fff700 1 -- 172.21.15.1:0/1381550405 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ebc0058a0 con 0x55a45864f110 2023-11-16T16:39:00.468 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.461+0000 7f1ed7134b80 10 client.15411 fetch_fsmap learned FSMap version 23 2023-11-16T16:39:00.468 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.461+0000 7f1ed7134b80 1 -- 172.21.15.1:0/1381550405 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55a458565190 con 0x55a45864f110 2023-11-16T16:39:00.468 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.461+0000 7f1ea3fff700 1 -- 172.21.15.1:0/1381550405 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a458640b20 con 0x55a45864f110 2023-11-16T16:39:00.468 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.461+0000 7f1ea3fff700 1 -- 172.21.15.1:0/1381550405 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1ebc09d0b0 con 0x55a45864f110 2023-11-16T16:39:00.468 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.461+0000 7f1ed7134b80 10 client.15411 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:39:00.468 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.461+0000 7f1ed7134b80 10 client.15411 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:39:00.468 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.461+0000 7f1ed7134b80 20 client.15411 populate_metadata read hostname 'smithi001' 2023-11-16T16:39:00.469 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.461+0000 7f1ed7134b80 1 --2- 172.21.15.1:0/1381550405 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55a4586632c0 0x55a4586656d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.469 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.461+0000 7f1ed7134b80 4 client.15411 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:39:00.469 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.461+0000 7f1ed7134b80 1 -- 172.21.15.1:0/1381550405 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a4585610a0 con 0x55a4586632c0 2023-11-16T16:39:00.469 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.461+0000 7f1ec8d44700 1 --2- 172.21.15.1:0/1381550405 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55a4586632c0 0x55a4586656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:00.469 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.462+0000 7f1ea3fff700 10 client.15411 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:39:00.469 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.462+0000 7f1ec8d44700 1 --2- 172.21.15.1:0/1381550405 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55a4586632c0 0x55a4586656d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2ddddf61cb9bf5c server_cookie=eb8b186343b3795c in_seq=0 out_seq=0 2023-11-16T16:39:00.469 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.462+0000 7f1ea3fff700 1 -- 172.21.15.1:0/1381550405 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a4585610a0 con 0x55a4586632c0 2023-11-16T16:39:00.469 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.462+0000 7f1ea3fff700 10 client.15411 handle_command_reply: tid=0 2023-11-16T16:39:00.469 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.462+0000 7f1ed7134b80 1 client.15411 shutdown 2023-11-16T16:39:00.469 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.463+0000 7f1ed7134b80 1 -- 172.21.15.1:0/1381550405 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45864f110 msgr2=0x55a45865eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:00.469 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.463+0000 7f1ed7134b80 1 --2- 172.21.15.1:0/1381550405 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45864f110 0x55a45865eb30 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc099410 tx=0x7f1ebc004c40 comp rx=0 tx=0).stop 2023-11-16T16:39:00.469 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.463+0000 7f1ed7134b80 1 -- 172.21.15.1:0/1381550405 shutdown_connections 2023-11-16T16:39:00.469 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.463+0000 7f1ed7134b80 1 --2- 172.21.15.1:0/1381550405 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55a4586632c0 0x55a4586656d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.470 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.463+0000 7f1ed7134b80 1 --2- 172.21.15.1:0/1381550405 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45864f110 0x55a45865eb30 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.470 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.463+0000 7f1ea3fff700 0 client.15411 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:39:00.470 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.463+0000 7f1ed7134b80 1 --2- 172.21.15.1:0/1381550405 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a45864dc00 0x55a4586575b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.470 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.463+0000 7f1ed7134b80 1 --2- 172.21.15.1:0/1381550405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a45864e550 0x55a458657af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.470 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.463+0000 7f1ed7134b80 1 -- 172.21.15.1:0/1381550405 >> 172.21.15.1:0/1381550405 conn(0x55a458643360 msgr2=0x55a458647de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:00.470 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.463+0000 7f1ed7134b80 1 -- 172.21.15.1:0/1381550405 shutdown_connections 2023-11-16T16:39:00.470 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.463+0000 7f1ed7134b80 1 -- 172.21.15.1:0/1381550405 wait complete. 2023-11-16T16:39:00.470 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.463+0000 7f1ed7134b80 20 client.15411 trim_cache size 0 max 16384 2023-11-16T16:39:00.484 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.480+0000 7f1ed7134b80 1 Processor -- start 2023-11-16T16:39:00.484 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.480+0000 7f1ed7134b80 1 -- start start 2023-11-16T16:39:00.484 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.481+0000 7f1ed7134b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45864dc00 0x55a458651fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.484 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.481+0000 7f1ed7134b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a45864e550 0x55a45865eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.484 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.481+0000 7f1ed7134b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a45864f110 0x55a458652500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.485 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.481+0000 7f1ed7134b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a45864efd0 con 0x55a45864dc00 2023-11-16T16:39:00.485 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.481+0000 7f1ed7134b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4586584b0 con 0x55a45864e550 2023-11-16T16:39:00.485 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.481+0000 7f1ed7134b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a458656a90 con 0x55a45864f110 2023-11-16T16:39:00.485 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.481+0000 7f1ec8d44700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45864dc00 0x55a458651fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:00.485 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.481+0000 7f1ec8d44700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45864dc00 0x55a458651fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43170/0 (socket says 172.21.15.1:43170) 2023-11-16T16:39:00.485 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.481+0000 7f1ec8d44700 1 -- 172.21.15.1:0/832134887 learned_addr learned my addr 172.21.15.1:0/832134887 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:00.485 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.481+0000 7f1ec9545700 1 --2- 172.21.15.1:0/832134887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a45864f110 0x55a458652500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:00.485 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.481+0000 7f1ec3fff700 1 --2- 172.21.15.1:0/832134887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a45864e550 0x55a45865eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:00.485 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.481+0000 7f1ec8d44700 1 -- 172.21.15.1:0/832134887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a45864f110 msgr2=0x55a458652500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:00.485 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.481+0000 7f1ec8d44700 1 --2- 172.21.15.1:0/832134887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a45864f110 0x55a458652500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.485 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.481+0000 7f1ec8d44700 1 -- 172.21.15.1:0/832134887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a45864e550 msgr2=0x55a45865eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:00.485 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.481+0000 7f1ec8d44700 1 --2- 172.21.15.1:0/832134887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a45864e550 0x55a45865eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.485 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.481+0000 7f1ec8d44700 1 -- 172.21.15.1:0/832134887 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a458564ee0 con 0x55a45864dc00 2023-11-16T16:39:00.485 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.481+0000 7f1ec3fff700 1 --2- 172.21.15.1:0/832134887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a45864e550 0x55a45865eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:39:00.486 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.481+0000 7f1ec8d44700 1 --2- 172.21.15.1:0/832134887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45864dc00 0x55a458651fc0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x55a458665fe0 tx=0x7f1eb8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be190a4fc389cf2 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:00.486 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.481+0000 7f1eaf7fe700 1 -- 172.21.15.1:0/832134887 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb802b070 con 0x55a45864dc00 2023-11-16T16:39:00.486 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.482+0000 7f1eaf7fe700 1 -- 172.21.15.1:0/832134887 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1eb8015070 con 0x55a45864dc00 2023-11-16T16:39:00.486 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.482+0000 7f1eaf7fe700 1 -- 172.21.15.1:0/832134887 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8025070 con 0x55a45864dc00 2023-11-16T16:39:00.486 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.482+0000 7f1ed7134b80 1 -- 172.21.15.1:0/832134887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45864dc00 msgr2=0x55a458651fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:00.486 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.482+0000 7f1ed7134b80 1 --2- 172.21.15.1:0/832134887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45864dc00 0x55a458651fc0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x55a458665fe0 tx=0x7f1eb8013000 comp rx=0 tx=0).stop 2023-11-16T16:39:00.486 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.482+0000 7f1ed7134b80 1 -- 172.21.15.1:0/832134887 shutdown_connections 2023-11-16T16:39:00.486 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.482+0000 7f1ed7134b80 1 --2- 172.21.15.1:0/832134887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a45864f110 0x55a458652500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.486 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.482+0000 7f1ed7134b80 1 --2- 172.21.15.1:0/832134887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45864dc00 0x55a458651fc0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.486 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.482+0000 7f1ed7134b80 1 --2- 172.21.15.1:0/832134887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a45864e550 0x55a45865eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.486 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.482+0000 7f1ed7134b80 1 -- 172.21.15.1:0/832134887 >> 172.21.15.1:0/832134887 conn(0x55a458643360 msgr2=0x55a458656290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:00.486 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.482+0000 7f1ed7134b80 1 -- 172.21.15.1:0/832134887 shutdown_connections 2023-11-16T16:39:00.486 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.482+0000 7f1ed7134b80 1 -- 172.21.15.1:0/832134887 wait complete. 2023-11-16T16:39:00.487 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.483+0000 7f1ed7134b80 1 -- start start 2023-11-16T16:39:00.487 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.483+0000 7f1ed7134b80 1 Processor -- start 2023-11-16T16:39:00.490 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.483+0000 7f1ed7134b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45864dc00 0x55a45865e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.490 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.483+0000 7f1ed7134b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a45864e550 0x55a45865e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.490 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.483+0000 7f1ed7134b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a45864f110 0x55a45865adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.490 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.483+0000 7f1ed7134b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a45865b350 con 0x55a45864dc00 2023-11-16T16:39:00.490 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.483+0000 7f1ed7134b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a45865b4c0 con 0x55a45864e550 2023-11-16T16:39:00.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.483+0000 7f1ed7134b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a45865b680 con 0x55a45864f110 2023-11-16T16:39:00.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.483+0000 7f1ed7134b80 10 client.0 _ll_register_callbacks cb 0x7f1ebc099010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:39:00.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.483+0000 7f1ec8d44700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45864dc00 0x55a45865e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:00.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.483+0000 7f1ec8d44700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45864dc00 0x55a45865e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43176/0 (socket says 172.21.15.1:43176) 2023-11-16T16:39:00.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.483+0000 7f1ec8d44700 1 -- 172.21.15.1:0/2388270935 learned_addr learned my addr 172.21.15.1:0/2388270935 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:00.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.483+0000 7f1ec9545700 1 --2- 172.21.15.1:0/2388270935 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a45864f110 0x55a45865adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:00.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.484+0000 7f1ec3fff700 1 --2- 172.21.15.1:0/2388270935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a45864e550 0x55a45865e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:00.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.484+0000 7f1ec8d44700 1 -- 172.21.15.1:0/2388270935 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a45864f110 msgr2=0x55a45865adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:00.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.484+0000 7f1ec8d44700 1 --2- 172.21.15.1:0/2388270935 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a45864f110 0x55a45865adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.484+0000 7f1ec8d44700 1 -- 172.21.15.1:0/2388270935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a45864e550 msgr2=0x55a45865e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:00.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.484+0000 7f1ec8d44700 1 --2- 172.21.15.1:0/2388270935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a45864e550 0x55a45865e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.484+0000 7f1ec8d44700 1 -- 172.21.15.1:0/2388270935 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb800be90 con 0x55a45864dc00 2023-11-16T16:39:00.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.484+0000 7f1ec8d44700 1 --2- 172.21.15.1:0/2388270935 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45864dc00 0x55a45865e320 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1eb800b9b0 tx=0x7f1eb8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a161c625aeab2a11 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:00.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.484+0000 7f1ea3fff700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:39:00.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.484+0000 7f1ea3fff700 1 -- 172.21.15.1:0/2388270935 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb802b040 con 0x55a45864dc00 2023-11-16T16:39:00.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.484+0000 7f1ea3fff700 1 -- 172.21.15.1:0/2388270935 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1eb8017070 con 0x55a45864dc00 2023-11-16T16:39:00.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.484+0000 7f1ed7134b80 1 -- 172.21.15.1:0/2388270935 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1eb001d040 con 0x55a45864dc00 2023-11-16T16:39:00.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.484+0000 7f1ea3fff700 1 -- 172.21.15.1:0/2388270935 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1eb8025040 con 0x55a45864dc00 2023-11-16T16:39:00.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.484+0000 7f1ed7134b80 10 client.15417 fetch_fsmap learned FSMap version 23 2023-11-16T16:39:00.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.484+0000 7f1ed7134b80 1 -- 172.21.15.1:0/2388270935 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55a4583b5c60 con 0x55a45864dc00 2023-11-16T16:39:00.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.484+0000 7f1ea3fff700 1 -- 172.21.15.1:0/2388270935 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1eb001d040 con 0x55a45864dc00 2023-11-16T16:39:00.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.485+0000 7f1ea3fff700 1 -- 172.21.15.1:0/2388270935 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1eb8027b70 con 0x55a45864dc00 2023-11-16T16:39:00.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.485+0000 7f1ed7134b80 10 client.15417 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:39:00.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.485+0000 7f1ed7134b80 10 client.15417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:39:00.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.485+0000 7f1ed7134b80 20 client.15417 populate_metadata read hostname 'smithi001' 2023-11-16T16:39:00.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.485+0000 7f1ed7134b80 1 --2- 172.21.15.1:0/2388270935 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55a4586b85b0 0x55a4586ba9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:00.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.485+0000 7f1ed7134b80 4 client.15417 mds_command: new command op to 24521 tid=0[{"prefix": "scrub status"}] 2023-11-16T16:39:00.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.485+0000 7f1ed7134b80 1 -- 172.21.15.1:0/2388270935 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a45856d660 con 0x55a4586b85b0 2023-11-16T16:39:00.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.485+0000 7f1ec3fff700 1 --2- 172.21.15.1:0/2388270935 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55a4586b85b0 0x55a4586ba9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:00.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.485+0000 7f1ea3fff700 10 client.15417 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:39:00.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.485+0000 7f1ec3fff700 1 --2- 172.21.15.1:0/2388270935 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55a4586b85b0 0x55a4586ba9c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb8f6d7fbc8146a5 server_cookie=71fbb7cfc1d44f4a in_seq=0 out_seq=0 2023-11-16T16:39:00.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.486+0000 7f1ea3fff700 1 -- 172.21.15.1:0/2388270935 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55a45856d660 con 0x55a4586b85b0 2023-11-16T16:39:00.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.486+0000 7f1ea3fff700 10 client.15417 handle_command_reply: tid=0 2023-11-16T16:39:00.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.486+0000 7f1ed7134b80 1 client.15417 shutdown 2023-11-16T16:39:00.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.486+0000 7f1ed7134b80 1 -- 172.21.15.1:0/2388270935 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45864dc00 msgr2=0x55a45865e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:00.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.486+0000 7f1ed7134b80 1 --2- 172.21.15.1:0/2388270935 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45864dc00 0x55a45865e320 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1eb800b9b0 tx=0x7f1eb8004750 comp rx=0 tx=0).stop 2023-11-16T16:39:00.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.486+0000 7f1ed7134b80 1 -- 172.21.15.1:0/2388270935 shutdown_connections 2023-11-16T16:39:00.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.486+0000 7f1ed7134b80 1 --2- 172.21.15.1:0/2388270935 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55a4586b85b0 0x55a4586ba9c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.486+0000 7f1ed7134b80 1 --2- 172.21.15.1:0/2388270935 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a45864f110 0x55a45865adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.486+0000 7f1ea3fff700 0 client.15417 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:39:00.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.486+0000 7f1ed7134b80 1 --2- 172.21.15.1:0/2388270935 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a45864dc00 0x55a45865e320 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.486+0000 7f1ed7134b80 1 --2- 172.21.15.1:0/2388270935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a45864e550 0x55a45865e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.486+0000 7f1ed7134b80 1 -- 172.21.15.1:0/2388270935 >> 172.21.15.1:0/2388270935 conn(0x55a458643360 msgr2=0x55a458651fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:00.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.486+0000 7f1ed7134b80 1 -- 172.21.15.1:0/2388270935 shutdown_connections 2023-11-16T16:39:00.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.486+0000 7f1ed7134b80 1 -- 172.21.15.1:0/2388270935 wait complete. 2023-11-16T16:39:00.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.486+0000 7f1ed7134b80 20 client.15417 trim_cache size 0 max 16384 2023-11-16T16:39:00.495 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:39:00.495 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-16T16:39:00.495 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2023-11-16T16:39:00.495 INFO:teuthology.orchestra.run.smithi001.stdout: "b9b2b2e4-79c5-46f1-807f-44024b6eba20": { 2023-11-16T16:39:00.495 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2023-11-16T16:39:00.495 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "b9b2b2e4-79c5-46f1-807f-44024b6eba20", 2023-11-16T16:39:00.495 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2023-11-16T16:39:00.495 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:39:00.495 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:39:00.495 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:39:00.496 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.486+0000 7f1ecafa8700 1 -- 172.21.15.1:0/2363349011 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f1eb40a11d0 msgr2=0x7f1eb40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:00.496 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.486+0000 7f1ecafa8700 1 --2- 172.21.15.1:0/2363349011 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f1eb40a11d0 0x7f1eb40a3680 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8005950 tx=0x7f1eb800b410 comp rx=0 tx=0).stop 2023-11-16T16:39:00.496 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.486+0000 7f1ecafa8700 1 -- 172.21.15.1:0/2363349011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ec4108080 msgr2=0x7f1ec419f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:00.496 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.486+0000 7f1ecafa8700 1 --2- 172.21.15.1:0/2363349011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ec4108080 0x7f1ec419f7a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc007ee0 tx=0x7f1ebc00d3b0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.496 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.487+0000 7f1ecafa8700 1 -- 172.21.15.1:0/2363349011 shutdown_connections 2023-11-16T16:39:00.496 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.487+0000 7f1ecafa8700 1 --2- 172.21.15.1:0/2363349011 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f1eb40a11d0 0x7f1eb40a3680 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.496 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.487+0000 7f1ecafa8700 1 --2- 172.21.15.1:0/2363349011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ec4108080 0x7f1ec419f7a0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.496 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.487+0000 7f1ecafa8700 1 --2- 172.21.15.1:0/2363349011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ec4068ae0 0x7f1ec41981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.487+0000 7f1ecafa8700 1 --2- 172.21.15.1:0/2363349011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1ec4105760 0x7f1ec4198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:00.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.487+0000 7f1ecafa8700 1 -- 172.21.15.1:0/2363349011 >> 172.21.15.1:0/2363349011 conn(0x7f1ec40fa750 msgr2=0x7f1ec4106bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:00.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.487+0000 7f1ecafa8700 1 -- 172.21.15.1:0/2363349011 shutdown_connections 2023-11-16T16:39:00.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.487+0000 7f1ecafa8700 1 -- 172.21.15.1:0/2363349011 wait complete. 2023-11-16T16:39:00.500 INFO:tasks.cephfs.filesystem:scrub status for tag:b9b2b2e4-79c5-46f1-807f-44024b6eba20 - {'path': '/', 'tag': 'b9b2b2e4-79c5-46f1-807f-44024b6eba20', 'options': 'recursive,force'} 2023-11-16T16:39:00.550 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:00.548+0000 7f28d5b4d700 1 -- 172.21.15.1:0/1356608954 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f28d0061980 con 0x7f28d0105f00 2023-11-16T16:39:00.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:00 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/751739240' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:39:00.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:00 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1741867817' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:39:00.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:00 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/732622619' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:39:00.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:00 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/751739240' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:39:00.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:00 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1741867817' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:39:00.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:00 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/732622619' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:39:01.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.271+0000 7f28ccff9700 1 -- 172.21.15.1:0/1356608954 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 19) v1 ==== 135731+0+0 (secure 0 0 0) 0x7f28c80977e0 con 0x7f28d0105f00 2023-11-16T16:39:01.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.274+0000 7f28ccff9700 1 -- 172.21.15.1:0/1356608954 <== mon.0 v2:172.21.15.1:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7f28c8055b70 con 0x7f28d0105f00 2023-11-16T16:39:01.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.277+0000 7f28d5b4d700 1 -- 172.21.15.1:0/1356608954 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f28b80a1150 msgr2=0x7f28b80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:01.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.277+0000 7f28d5b4d700 1 --2- 172.21.15.1:0/1356608954 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f28b80a1150 0x7f28b80a3600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f28c000ac50 tx=0x7f28c000a380 comp rx=0 tx=0).stop 2023-11-16T16:39:01.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.277+0000 7f28d5b4d700 1 -- 172.21.15.1:0/1356608954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28d0105f00 msgr2=0x7f28d00ff1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:01.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.277+0000 7f28d5b4d700 1 --2- 172.21.15.1:0/1356608954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28d0105f00 0x7f28d00ff1b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f28c800b070 tx=0x7f28c8007870 comp rx=0 tx=0).stop 2023-11-16T16:39:01.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.277+0000 7f28d5b4d700 1 -- 172.21.15.1:0/1356608954 shutdown_connections 2023-11-16T16:39:01.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.277+0000 7f28d5b4d700 1 --2- 172.21.15.1:0/1356608954 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f28b80a1150 0x7f28b80a3600 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:01.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.277+0000 7f28d5b4d700 1 --2- 172.21.15.1:0/1356608954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28d0105f00 0x7f28d00ff1b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:01.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.277+0000 7f28d5b4d700 1 --2- 172.21.15.1:0/1356608954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28d0100cc0 0x7f28d0100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:01.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.277+0000 7f28d5b4d700 1 --2- 172.21.15.1:0/1356608954 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28d01035e0 0x7f28d00fec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:01.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.277+0000 7f28d5b4d700 1 -- 172.21.15.1:0/1356608954 >> 172.21.15.1:0/1356608954 conn(0x7f28d00fa720 msgr2=0x7f28d0101920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:01.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.277+0000 7f28d5b4d700 1 -- 172.21.15.1:0/1356608954 shutdown_connections 2023-11-16T16:39:01.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.277+0000 7f28d5b4d700 1 -- 172.21.15.1:0/1356608954 wait complete. 2023-11-16T16:39:01.294 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2023-11-16T16:39:01.436 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.431+0000 7f3e74446700 1 -- 172.21.15.1:0/2864347982 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e6c103680 msgr2=0x7f3e6c105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:01.436 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.431+0000 7f3e74446700 1 --2- 172.21.15.1:0/2864347982 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e6c103680 0x7f3e6c105a60 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c009990 tx=0x7f3e5c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:39:01.436 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.431+0000 7f3e74446700 1 -- 172.21.15.1:0/2864347982 shutdown_connections 2023-11-16T16:39:01.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.431+0000 7f3e74446700 1 --2- 172.21.15.1:0/2864347982 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e6c105fa0 0x7f3e6c108380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:01.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.431+0000 7f3e74446700 1 --2- 172.21.15.1:0/2864347982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e6c100d60 0x7f3e6c103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:01.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.431+0000 7f3e74446700 1 --2- 172.21.15.1:0/2864347982 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e6c103680 0x7f3e6c105a60 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:01.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.431+0000 7f3e74446700 1 -- 172.21.15.1:0/2864347982 >> 172.21.15.1:0/2864347982 conn(0x7f3e6c0fa720 msgr2=0x7f3e6c0fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:01.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.431+0000 7f3e74446700 1 -- 172.21.15.1:0/2864347982 shutdown_connections 2023-11-16T16:39:01.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.431+0000 7f3e74446700 1 -- 172.21.15.1:0/2864347982 wait complete. 2023-11-16T16:39:01.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.432+0000 7f3e74446700 1 Processor -- start 2023-11-16T16:39:01.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.432+0000 7f3e74446700 1 -- start start 2023-11-16T16:39:01.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.432+0000 7f3e74446700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e6c100d60 0x7f3e6c195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:01.438 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.432+0000 7f3e74446700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e6c103680 0x7f3e6c1964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:01.438 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.432+0000 7f3e74446700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e6c105fa0 0x7f3e6c19d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:01.438 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.432+0000 7f3e74446700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c19db30 con 0x7f3e6c100d60 2023-11-16T16:39:01.438 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.432+0000 7f3e74446700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c19dc70 con 0x7f3e6c105fa0 2023-11-16T16:39:01.438 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.432+0000 7f3e74446700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c19de00 con 0x7f3e6c103680 2023-11-16T16:39:01.438 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.432+0000 7f3e721e2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e6c100d60 0x7f3e6c195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:01.438 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.432+0000 7f3e719e1700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e6c103680 0x7f3e6c1964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:01.438 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.432+0000 7f3e721e2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e6c100d60 0x7f3e6c195fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43220/0 (socket says 172.21.15.1:43220) 2023-11-16T16:39:01.438 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.432+0000 7f3e721e2700 1 -- 172.21.15.1:0/3234421099 learned_addr learned my addr 172.21.15.1:0/3234421099 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:01.439 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.432+0000 7f3e729e3700 1 --2- 172.21.15.1:0/3234421099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e6c105fa0 0x7f3e6c19d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:01.439 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.433+0000 7f3e721e2700 1 -- 172.21.15.1:0/3234421099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e6c103680 msgr2=0x7f3e6c1964f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:01.439 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.433+0000 7f3e721e2700 1 --2- 172.21.15.1:0/3234421099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e6c103680 0x7f3e6c1964f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:01.439 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.433+0000 7f3e721e2700 1 -- 172.21.15.1:0/3234421099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e6c105fa0 msgr2=0x7f3e6c19d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:01.439 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.433+0000 7f3e721e2700 1 --2- 172.21.15.1:0/3234421099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e6c105fa0 0x7f3e6c19d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:01.439 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.433+0000 7f3e721e2700 1 -- 172.21.15.1:0/3234421099 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e5c0095f0 con 0x7f3e6c100d60 2023-11-16T16:39:01.439 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.433+0000 7f3e721e2700 1 --2- 172.21.15.1:0/3234421099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e6c100d60 0x7f3e6c195fb0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3e6800c920 tx=0x7f3e6800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:01.439 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.433+0000 7f3e637fe700 1 -- 172.21.15.1:0/3234421099 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e68004a30 con 0x7f3e6c100d60 2023-11-16T16:39:01.439 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.433+0000 7f3e637fe700 1 -- 172.21.15.1:0/3234421099 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e680054e0 con 0x7f3e6c100d60 2023-11-16T16:39:01.440 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.433+0000 7f3e74446700 1 -- 172.21.15.1:0/3234421099 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e6c10bc00 con 0x7f3e6c100d60 2023-11-16T16:39:01.440 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.433+0000 7f3e637fe700 1 -- 172.21.15.1:0/3234421099 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e68007bd0 con 0x7f3e6c100d60 2023-11-16T16:39:01.440 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.433+0000 7f3e74446700 1 -- 172.21.15.1:0/3234421099 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e6c10ba70 con 0x7f3e6c100d60 2023-11-16T16:39:01.440 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.434+0000 7f3e74446700 1 -- 172.21.15.1:0/3234421099 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e6c0fc290 con 0x7f3e6c100d60 2023-11-16T16:39:01.441 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.436+0000 7f3e637fe700 1 -- 172.21.15.1:0/3234421099 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 19) v1 ==== 135731+0+0 (secure 0 0 0) 0x7f3e68007d50 con 0x7f3e6c100d60 2023-11-16T16:39:01.441 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.436+0000 7f3e637fe700 1 --2- 172.21.15.1:0/3234421099 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f3e580a11a0 0x7f3e580a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:01.441 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.436+0000 7f3e719e1700 1 -- 172.21.15.1:0/3234421099 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f3e580a11a0 msgr2=0x7f3e580a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/2490729488 2023-11-16T16:39:01.441 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.436+0000 7f3e637fe700 1 -- 172.21.15.1:0/3234421099 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3e680cc8f0 con 0x7f3e6c100d60 2023-11-16T16:39:01.441 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.436+0000 7f3e719e1700 1 --2- 172.21.15.1:0/3234421099 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f3e580a11a0 0x7f3e580a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-16T16:39:01.442 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.440+0000 7f3e637fe700 1 -- 172.21.15.1:0/3234421099 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3e68096000 con 0x7f3e6c100d60 2023-11-16T16:39:01.565 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:39:01 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: ignoring --setuser ceph since I am not root 2023-11-16T16:39:01.565 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:39:01 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: ignoring --setgroup ceph since I am not root 2023-11-16T16:39:01.565 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:39:01 smithi026 ceph-mgr[101590]: -- 172.21.15.26:0/1888937140 <== mon.2 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558443f89520 con 0x558443f5e800 2023-11-16T16:39:01.565 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:39:01 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:39:01.467+0000 7fc6c56c6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-16T16:39:01.565 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:01 smithi026 ceph-mon[100112]: pgmap v315: 97 pgs: 97 active+clean; 644 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 938 B/s wr, 6 op/s 2023-11-16T16:39:01.565 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:01 smithi026 ceph-mon[100112]: scrub queued for path: / 2023-11-16T16:39:01.566 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:01 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:39:01.566 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:01 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:39:01.566 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:01 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:39:01.566 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:01 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1356608954' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-11-16T16:39:01.566 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:01 smithi053 ceph-mon[100638]: pgmap v315: 97 pgs: 97 active+clean; 644 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 938 B/s wr, 6 op/s 2023-11-16T16:39:01.567 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:01 smithi053 ceph-mon[100638]: scrub queued for path: / 2023-11-16T16:39:01.567 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:01 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:39:01.567 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:01 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:39:01.567 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:01 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:39:01.567 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:01 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1356608954' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-11-16T16:39:01.567 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:39:01 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: ignoring --setuser ceph since I am not root 2023-11-16T16:39:01.567 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:39:01 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: ignoring --setgroup ceph since I am not root 2023-11-16T16:39:01.567 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:39:01 smithi053 ceph-mgr[102777]: -- 172.21.15.53:0/309283595 <== mon.1 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a9f9085520 con 0x55a9f905ac00 2023-11-16T16:39:01.568 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:39:01 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:39:01.466+0000 7f614933d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-16T16:39:01.568 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:39:01 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:39:01.563+0000 7f614933d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-16T16:39:01.571 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:01 smithi001 ceph-mon[98800]: pgmap v315: 97 pgs: 97 active+clean; 644 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 938 B/s wr, 6 op/s 2023-11-16T16:39:01.571 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:01 smithi001 ceph-mon[98800]: scrub queued for path: / 2023-11-16T16:39:01.571 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:01 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:39:01.572 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:01 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:39:01.572 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:01 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:39:01.572 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:01 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1356608954' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-11-16T16:39:01.572 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:39:01 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: ignoring --setuser ceph since I am not root 2023-11-16T16:39:01.572 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:39:01 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: ignoring --setgroup ceph since I am not root 2023-11-16T16:39:01.572 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:39:01 smithi001 ceph-mgr[99067]: -- 172.21.15.1:0/1696081912 <== mon.0 v2:172.21.15.1:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56111a8bd520 con 0x56111a893000 2023-11-16T16:39:01.572 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:39:01 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:39:01.472+0000 7f4ba2b8b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-16T16:39:01.572 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:39:01 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:39:01.569+0000 7f4ba2b8b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-16T16:39:01.639 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.637+0000 7f3e719e1700 1 -- 172.21.15.1:0/3234421099 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f3e580a11a0 msgr2=0x7f3e580a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/2490729488 2023-11-16T16:39:01.639 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.637+0000 7f3e719e1700 1 --2- 172.21.15.1:0/3234421099 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f3e580a11a0 0x7f3e580a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-16T16:39:01.643 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.642+0000 7f3e74446700 1 -- 172.21.15.1:0/3234421099 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f3e6c0619b0 con 0x7f3e6c100d60 2023-11-16T16:39:01.648 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.646+0000 7f3e637fe700 1 -- 172.21.15.1:0/3234421099 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3e6801d020 con 0x7f3e6c100d60 2023-11-16T16:39:01.651 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.648+0000 7f3e74446700 1 -- 172.21.15.1:0/3234421099 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f3e580a11a0 msgr2=0x7f3e580a3650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-16T16:39:01.651 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.648+0000 7f3e74446700 1 --2- 172.21.15.1:0/3234421099 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f3e580a11a0 0x7f3e580a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:01.651 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.648+0000 7f3e74446700 1 -- 172.21.15.1:0/3234421099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e6c100d60 msgr2=0x7f3e6c195fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:01.651 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.648+0000 7f3e74446700 1 --2- 172.21.15.1:0/3234421099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e6c100d60 0x7f3e6c195fb0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3e6800c920 tx=0x7f3e6800cc30 comp rx=0 tx=0).stop 2023-11-16T16:39:01.651 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.648+0000 7f3e74446700 1 -- 172.21.15.1:0/3234421099 shutdown_connections 2023-11-16T16:39:01.651 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.648+0000 7f3e74446700 1 --2- 172.21.15.1:0/3234421099 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f3e580a11a0 0x7f3e580a3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:01.651 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.648+0000 7f3e74446700 1 --2- 172.21.15.1:0/3234421099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e6c105fa0 0x7f3e6c19d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:01.652 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.648+0000 7f3e74446700 1 --2- 172.21.15.1:0/3234421099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e6c100d60 0x7f3e6c195fb0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:01.652 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.648+0000 7f3e74446700 1 --2- 172.21.15.1:0/3234421099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3e6c103680 0x7f3e6c1964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:01.652 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.648+0000 7f3e74446700 1 -- 172.21.15.1:0/3234421099 >> 172.21.15.1:0/3234421099 conn(0x7f3e6c0fa720 msgr2=0x7f3e6c1019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:01.652 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.648+0000 7f3e74446700 1 -- 172.21.15.1:0/3234421099 shutdown_connections 2023-11-16T16:39:01.652 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.649+0000 7f3e74446700 1 -- 172.21.15.1:0/3234421099 wait complete. 2023-11-16T16:39:01.664 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2023-11-16T16:39:01.796 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.790+0000 7fef3a50f700 1 -- 172.21.15.1:0/2370375597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef34100cc0 msgr2=0x7fef34107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:01.796 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.790+0000 7fef3a50f700 1 --2- 172.21.15.1:0/2370375597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef34100cc0 0x7fef34107580 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fef2c00b0a0 tx=0x7fef2c00b3b0 comp rx=0 tx=0).stop 2023-11-16T16:39:01.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.790+0000 7fef3a50f700 1 -- 172.21.15.1:0/2370375597 shutdown_connections 2023-11-16T16:39:01.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.790+0000 7fef3a50f700 1 --2- 172.21.15.1:0/2370375597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef34100cc0 0x7fef34107580 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:01.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.790+0000 7fef3a50f700 1 --2- 172.21.15.1:0/2370375597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef340ff140 0x7fef340ff550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:01.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.790+0000 7fef3a50f700 1 --2- 172.21.15.1:0/2370375597 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef34100330 0x7fef34100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:01.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.791+0000 7fef3a50f700 1 -- 172.21.15.1:0/2370375597 >> 172.21.15.1:0/2370375597 conn(0x7fef340fa720 msgr2=0x7fef340fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:01.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.791+0000 7fef3a50f700 1 -- 172.21.15.1:0/2370375597 shutdown_connections 2023-11-16T16:39:01.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.791+0000 7fef3a50f700 1 -- 172.21.15.1:0/2370375597 wait complete. 2023-11-16T16:39:01.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.791+0000 7fef3a50f700 1 Processor -- start 2023-11-16T16:39:01.798 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.791+0000 7fef3a50f700 1 -- start start 2023-11-16T16:39:01.798 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.791+0000 7fef3a50f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef340ff140 0x7fef34198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:01.798 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.791+0000 7fef3a50f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef34100330 0x7fef34198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:01.798 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.792+0000 7fef3a50f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef34100cc0 0x7fef3419f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:01.798 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.792+0000 7fef3a50f700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3419fe80 con 0x7fef34100cc0 2023-11-16T16:39:01.798 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.792+0000 7fef3a50f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3419ffc0 con 0x7fef34100330 2023-11-16T16:39:01.798 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.792+0000 7fef3a50f700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef341a0150 con 0x7fef340ff140 2023-11-16T16:39:01.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.792+0000 7fef38aac700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef34100cc0 0x7fef3419f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:01.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.792+0000 7fef33fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef340ff140 0x7fef34198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:01.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.792+0000 7fef337fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef34100330 0x7fef34198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:01.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.792+0000 7fef38aac700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef34100cc0 0x7fef3419f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43312/0 (socket says 172.21.15.1:43312) 2023-11-16T16:39:01.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.792+0000 7fef33fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef340ff140 0x7fef34198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:39990/0 (socket says 172.21.15.1:39990) 2023-11-16T16:39:01.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.792+0000 7fef38aac700 1 -- 172.21.15.1:0/2492793236 learned_addr learned my addr 172.21.15.1:0/2492793236 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:01.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.792+0000 7fef337fe700 1 -- 172.21.15.1:0/2492793236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef340ff140 msgr2=0x7fef34198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:01.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.792+0000 7fef337fe700 1 --2- 172.21.15.1:0/2492793236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef340ff140 0x7fef34198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:01.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.792+0000 7fef337fe700 1 -- 172.21.15.1:0/2492793236 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef34100cc0 msgr2=0x7fef3419f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:01.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.792+0000 7fef337fe700 1 --2- 172.21.15.1:0/2492793236 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef34100cc0 0x7fef3419f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:01.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.792+0000 7fef337fe700 1 -- 172.21.15.1:0/2492793236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef2c009d00 con 0x7fef34100330 2023-11-16T16:39:01.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.792+0000 7fef38aac700 1 --2- 172.21.15.1:0/2492793236 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef34100cc0 0x7fef3419f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:39:01.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.792+0000 7fef337fe700 1 --2- 172.21.15.1:0/2492793236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef34100330 0x7fef34198840 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fef2800e9f0 tx=0x7fef2800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:01.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.792+0000 7fef317fa700 1 -- 172.21.15.1:0/2492793236 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2800c9c0 con 0x7fef34100330 2023-11-16T16:39:01.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.792+0000 7fef3a50f700 1 -- 172.21.15.1:0/2492793236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef3410aab0 con 0x7fef34100330 2023-11-16T16:39:01.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.792+0000 7fef317fa700 1 -- 172.21.15.1:0/2492793236 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef28010430 con 0x7fef34100330 2023-11-16T16:39:01.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.792+0000 7fef317fa700 1 -- 172.21.15.1:0/2492793236 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef28018680 con 0x7fef34100330 2023-11-16T16:39:01.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.793+0000 7fef3a50f700 1 -- 172.21.15.1:0/2492793236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef3410a920 con 0x7fef34100330 2023-11-16T16:39:01.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.793+0000 7fef3a50f700 1 -- 172.21.15.1:0/2492793236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef341a0890 con 0x7fef34100330 2023-11-16T16:39:01.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.795+0000 7fef317fa700 1 -- 172.21.15.1:0/2492793236 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) v1 ==== 135731+0+0 (secure 0 0 0) 0x7fef28010860 con 0x7fef34100330 2023-11-16T16:39:01.802 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.795+0000 7fef317fa700 1 --2- 172.21.15.1:0/2492793236 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fef1c0a11a0 0x7fef1c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:01.802 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.795+0000 7fef33fff700 1 -- 172.21.15.1:0/2492793236 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fef1c0a11a0 msgr2=0x7fef1c0a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/2490729488 2023-11-16T16:39:01.803 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.795+0000 7fef317fa700 1 -- 172.21.15.1:0/2492793236 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fef28014070 con 0x7fef34100330 2023-11-16T16:39:01.803 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.795+0000 7fef33fff700 1 --2- 172.21.15.1:0/2492793236 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fef1c0a11a0 0x7fef1c0a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-16T16:39:01.803 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.799+0000 7fef317fa700 1 -- 172.21.15.1:0/2492793236 <== mon.2 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fef280961b0 con 0x7fef34100330 2023-11-16T16:39:01.830 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:39:01 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:39:01.673+0000 7f614933d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-16T16:39:01.831 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:39:01 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:39:01.783+0000 7f614933d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-16T16:39:01.882 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:39:01 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:39:01.681+0000 7f4ba2b8b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-16T16:39:01.882 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:39:01 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:39:01.798+0000 7f4ba2b8b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-16T16:39:01.992 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:39:01 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:39:01.563+0000 7fc6c56c6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-16T16:39:01.992 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:39:01 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:39:01.671+0000 7fc6c56c6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-16T16:39:01.992 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:39:01 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:39:01.782+0000 7fc6c56c6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-16T16:39:01.997 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.996+0000 7fef33fff700 1 -- 172.21.15.1:0/2492793236 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fef1c0a11a0 msgr2=0x7fef1c0a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/2490729488 2023-11-16T16:39:01.998 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:01.996+0000 7fef33fff700 1 --2- 172.21.15.1:0/2492793236 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fef1c0a11a0 0x7fef1c0a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-16T16:39:02.006 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.004+0000 7fef3a50f700 1 -- 172.21.15.1:0/2492793236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fef341a0f30 con 0x7fef34100330 2023-11-16T16:39:02.011 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.009+0000 7fef317fa700 1 -- 172.21.15.1:0/2492793236 <== mon.2 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7fef280997d0 con 0x7fef34100330 2023-11-16T16:39:02.014 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.011+0000 7fef3a50f700 1 -- 172.21.15.1:0/2492793236 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fef1c0a11a0 msgr2=0x7fef1c0a3650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-16T16:39:02.014 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.011+0000 7fef3a50f700 1 --2- 172.21.15.1:0/2492793236 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fef1c0a11a0 0x7fef1c0a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:02.014 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.011+0000 7fef3a50f700 1 -- 172.21.15.1:0/2492793236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef34100330 msgr2=0x7fef34198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:02.014 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.011+0000 7fef3a50f700 1 --2- 172.21.15.1:0/2492793236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef34100330 0x7fef34198840 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fef2800e9f0 tx=0x7fef2800ed00 comp rx=0 tx=0).stop 2023-11-16T16:39:02.014 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.011+0000 7fef3a50f700 1 -- 172.21.15.1:0/2492793236 shutdown_connections 2023-11-16T16:39:02.014 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.011+0000 7fef3a50f700 1 --2- 172.21.15.1:0/2492793236 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7fef1c0a11a0 0x7fef1c0a3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:02.014 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.011+0000 7fef3a50f700 1 --2- 172.21.15.1:0/2492793236 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef34100cc0 0x7fef3419f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:02.014 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.011+0000 7fef3a50f700 1 --2- 172.21.15.1:0/2492793236 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fef340ff140 0x7fef34198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:02.015 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.011+0000 7fef3a50f700 1 --2- 172.21.15.1:0/2492793236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef34100330 0x7fef34198840 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:02.015 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.011+0000 7fef3a50f700 1 -- 172.21.15.1:0/2492793236 >> 172.21.15.1:0/2492793236 conn(0x7fef340fa720 msgr2=0x7fef340680f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:02.015 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.011+0000 7fef3a50f700 1 -- 172.21.15.1:0/2492793236 shutdown_connections 2023-11-16T16:39:02.015 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.011+0000 7fef3a50f700 1 -- 172.21.15.1:0/2492793236 wait complete. 2023-11-16T16:39:02.026 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1M' 2023-11-16T16:39:02.163 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.155+0000 7f8fbf3fc700 1 -- 172.21.15.1:0/870868018 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fb80ffe10 msgr2=0x7f8fb8100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:02.163 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.155+0000 7f8fbf3fc700 1 --2- 172.21.15.1:0/870868018 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fb80ffe10 0x7f8fb8100280 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000b0a0 tx=0x7f8fb000b3b0 comp rx=0 tx=0).stop 2023-11-16T16:39:02.163 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.155+0000 7f8fbf3fc700 1 -- 172.21.15.1:0/870868018 shutdown_connections 2023-11-16T16:39:02.164 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.155+0000 7f8fbf3fc700 1 --2- 172.21.15.1:0/870868018 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fb80ffe10 0x7f8fb8100280 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:02.164 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.155+0000 7f8fbf3fc700 1 --2- 172.21.15.1:0/870868018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fb80feb10 0x7f8fb80fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:02.164 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.155+0000 7f8fbf3fc700 1 --2- 172.21.15.1:0/870868018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fb80ff460 0x7f8fb80ff8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:02.164 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.155+0000 7f8fbf3fc700 1 -- 172.21.15.1:0/870868018 >> 172.21.15.1:0/870868018 conn(0x7f8fb80fa720 msgr2=0x7f8fb80fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:02.164 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.155+0000 7f8fbf3fc700 1 -- 172.21.15.1:0/870868018 shutdown_connections 2023-11-16T16:39:02.164 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.155+0000 7f8fbf3fc700 1 -- 172.21.15.1:0/870868018 wait complete. 2023-11-16T16:39:02.164 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.156+0000 7f8fbf3fc700 1 Processor -- start 2023-11-16T16:39:02.164 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.156+0000 7f8fbf3fc700 1 -- start start 2023-11-16T16:39:02.165 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.156+0000 7f8fbf3fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fb80feb10 0x7f8fb81982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:02.165 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.156+0000 7f8fbf3fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fb80ff460 0x7f8fb8198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:02.165 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.156+0000 7f8fbf3fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fb80ffe10 0x7f8fb819f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:02.165 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.156+0000 7f8fbf3fc700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb819fe60 con 0x7f8fb80ff460 2023-11-16T16:39:02.165 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.156+0000 7f8fbf3fc700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb819ffa0 con 0x7f8fb80feb10 2023-11-16T16:39:02.165 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.156+0000 7f8fbf3fc700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb81a0130 con 0x7f8fb80ffe10 2023-11-16T16:39:02.165 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.156+0000 7f8fbc997700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fb80ff460 0x7f8fb8198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:02.165 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.156+0000 7f8fbc997700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fb80ff460 0x7f8fb8198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43374/0 (socket says 172.21.15.1:43374) 2023-11-16T16:39:02.166 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.156+0000 7f8fbc997700 1 -- 172.21.15.1:0/3906335793 learned_addr learned my addr 172.21.15.1:0/3906335793 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:02.166 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.156+0000 7f8fbd198700 1 --2- 172.21.15.1:0/3906335793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fb80feb10 0x7f8fb81982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:02.166 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.156+0000 7f8fbd999700 1 --2- 172.21.15.1:0/3906335793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fb80ffe10 0x7f8fb819f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:02.166 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.157+0000 7f8fbc997700 1 -- 172.21.15.1:0/3906335793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fb80ffe10 msgr2=0x7f8fb819f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:02.166 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.157+0000 7f8fbc997700 1 --2- 172.21.15.1:0/3906335793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fb80ffe10 0x7f8fb819f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:02.166 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.157+0000 7f8fbc997700 1 -- 172.21.15.1:0/3906335793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fb80feb10 msgr2=0x7f8fb81982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:02.166 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.157+0000 7f8fbc997700 1 --2- 172.21.15.1:0/3906335793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fb80feb10 0x7f8fb81982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:02.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.157+0000 7f8fbc997700 1 -- 172.21.15.1:0/3906335793 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb0009d00 con 0x7f8fb80ff460 2023-11-16T16:39:02.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.157+0000 7f8fbd198700 1 --2- 172.21.15.1:0/3906335793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fb80feb10 0x7f8fb81982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:39:02.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.157+0000 7f8fbc997700 1 --2- 172.21.15.1:0/3906335793 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fb80ff460 0x7f8fb8198820 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8fa800e9a0 tx=0x7f8fa800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:02.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.157+0000 7f8fae7fc700 1 -- 172.21.15.1:0/3906335793 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8009790 con 0x7f8fb80ff460 2023-11-16T16:39:02.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.157+0000 7f8fae7fc700 1 -- 172.21.15.1:0/3906335793 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fa8010430 con 0x7f8fb80ff460 2023-11-16T16:39:02.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.157+0000 7f8fbf3fc700 1 -- 172.21.15.1:0/3906335793 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fb810dd90 con 0x7f8fb80ff460 2023-11-16T16:39:02.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.157+0000 7f8fae7fc700 1 -- 172.21.15.1:0/3906335793 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8018680 con 0x7f8fb80ff460 2023-11-16T16:39:02.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.157+0000 7f8fbf3fc700 1 -- 172.21.15.1:0/3906335793 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fb810dc00 con 0x7f8fb80ff460 2023-11-16T16:39:02.168 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.158+0000 7f8fbf3fc700 1 -- 172.21.15.1:0/3906335793 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fb81098f0 con 0x7f8fb80ff460 2023-11-16T16:39:02.168 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.159+0000 7f8fae7fc700 1 -- 172.21.15.1:0/3906335793 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 19) v1 ==== 135731+0+0 (secure 0 0 0) 0x7f8fa80105a0 con 0x7f8fb80ff460 2023-11-16T16:39:02.168 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.160+0000 7f8fae7fc700 1 --2- 172.21.15.1:0/3906335793 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8fa40a11a0 0x7f8fa40a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:02.168 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.160+0000 7f8fbd198700 1 -- 172.21.15.1:0/3906335793 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8fa40a11a0 msgr2=0x7f8fa40a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/2490729488 2023-11-16T16:39:02.168 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.160+0000 7f8fae7fc700 1 -- 172.21.15.1:0/3906335793 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8fa8014070 con 0x7f8fb80ff460 2023-11-16T16:39:02.168 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.160+0000 7f8fbd198700 1 --2- 172.21.15.1:0/3906335793 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8fa40a11a0 0x7f8fa40a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-16T16:39:02.169 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.164+0000 7f8fae7fc700 1 -- 172.21.15.1:0/3906335793 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8fa8051730 con 0x7f8fb80ff460 2023-11-16T16:39:02.182 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:39:02 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:39:02.008+0000 7f4ba2b8b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-16T16:39:02.258 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:39:01 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:39:01.990+0000 7fc6c56c6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-16T16:39:02.278 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:39:01 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:39:01.991+0000 7f614933d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-16T16:39:02.362 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.361+0000 7f8fbd198700 1 -- 172.21.15.1:0/3906335793 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8fa40a11a0 msgr2=0x7f8fa40a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/2490729488 2023-11-16T16:39:02.362 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.361+0000 7f8fbd198700 1 --2- 172.21.15.1:0/3906335793 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8fa40a11a0 0x7f8fa40a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-16T16:39:02.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.393+0000 7f8fbf3fc700 1 -- 172.21.15.1:0/3906335793 --> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f8fb8061a30 con 0x7f8fa40a11a0 2023-11-16T16:39:02.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:02 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1356608954' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-11-16T16:39:02.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:02 smithi053 ceph-mon[100638]: mgrmap e19: x(active, since 7m), standbys: y, z 2023-11-16T16:39:02.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:02 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/3234421099' entity='client.admin' 2023-11-16T16:39:02.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:02 smithi053 ceph-mon[100638]: from='client.? ' entity='client.admin' 2023-11-16T16:39:02.581 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:39:02 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:39:02.385+0000 7f614933d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-16T16:39:02.582 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:39:02 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:39:02.493+0000 7f614933d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-16T16:39:02.646 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:39:02 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:39:02.385+0000 7fc6c56c6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-16T16:39:02.647 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:39:02 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:39:02.493+0000 7fc6c56c6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-16T16:39:02.647 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:02 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1356608954' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-11-16T16:39:02.647 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:02 smithi026 ceph-mon[100112]: mgrmap e19: x(active, since 7m), standbys: y, z 2023-11-16T16:39:02.647 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:02 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/3234421099' entity='client.admin' 2023-11-16T16:39:02.648 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:02 smithi026 ceph-mon[100112]: from='client.? ' entity='client.admin' 2023-11-16T16:39:02.682 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:39:02 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:39:02.415+0000 7f4ba2b8b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-16T16:39:02.682 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:39:02 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:39:02.525+0000 7f4ba2b8b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-16T16:39:02.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:02 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1356608954' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-11-16T16:39:02.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:02 smithi001 ceph-mon[98800]: mgrmap e19: x(active, since 7m), standbys: y, z 2023-11-16T16:39:02.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:02 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3234421099' entity='client.admin' 2023-11-16T16:39:02.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:02 smithi001 ceph-mon[98800]: from='client.? ' entity='client.admin' 2023-11-16T16:39:02.763 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.762+0000 7f8fbd198700 1 -- 172.21.15.1:0/3906335793 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8fa40a11a0 msgr2=0x7f8fa40a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/2490729488 2023-11-16T16:39:02.763 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:02.762+0000 7f8fbd198700 1 --2- 172.21.15.1:0/3906335793 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8fa40a11a0 0x7f8fa40a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-16T16:39:03.008 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:39:02 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:39:02.645+0000 7fc6c56c6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-16T16:39:03.008 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:39:02 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:39:02.760+0000 7fc6c56c6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-16T16:39:03.080 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:39:02 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:39:02.644+0000 7f614933d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-16T16:39:03.081 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:39:02 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:39:02.759+0000 7f614933d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-16T16:39:03.182 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:39:02 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:39:02.683+0000 7f4ba2b8b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-16T16:39:03.182 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:39:02 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:39:02.799+0000 7f4ba2b8b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-16T16:39:03.564 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:03.563+0000 7f8fbd198700 1 -- 172.21.15.1:0/3906335793 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8fa40a11a0 msgr2=0x7f8fa40a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/2490729488 2023-11-16T16:39:03.565 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:03.563+0000 7f8fbd198700 1 --2- 172.21.15.1:0/3906335793 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8fa40a11a0 0x7f8fa40a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-16T16:39:03.581 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:39:03 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:39:03.268+0000 7f614933d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-16T16:39:03.683 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:39:03 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:39:03.312+0000 7f4ba2b8b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-16T16:39:03.758 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:39:03 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:39:03.270+0000 7fc6c56c6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-16T16:39:04.250 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:39:03 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:39:03.954+0000 7f614933d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-16T16:39:04.250 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:39:04 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:39:04.146+0000 7f614933d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-16T16:39:04.251 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:39:03 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:39:03.957+0000 7fc6c56c6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-16T16:39:04.251 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:39:04 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:39:04.149+0000 7fc6c56c6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-16T16:39:04.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:04 smithi001 ceph-mon[98800]: scrub summary: idle 2023-11-16T16:39:04.432 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:39:04 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:39:03.999+0000 7f4ba2b8b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-16T16:39:04.433 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:39:04 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:39:04.192+0000 7f4ba2b8b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-16T16:39:04.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:04 smithi026 ceph-mon[100112]: scrub summary: idle 2023-11-16T16:39:04.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:04 smithi053 ceph-mon[100638]: scrub summary: idle 2023-11-16T16:39:05.165 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:05.164+0000 7f8fbd198700 1 -- 172.21.15.1:0/3906335793 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8fa40a11a0 msgr2=0x7f8fa40a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/2490729488 2023-11-16T16:39:05.166 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:05.164+0000 7f8fbd198700 1 --2- 172.21.15.1:0/3906335793 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8fa40a11a0 0x7f8fa40a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-11-16T16:39:05.432 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:39:05 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:39:05.162+0000 7f4ba2b8b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-16T16:39:05.432 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:39:05 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:39:05.275+0000 7f4ba2b8b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-16T16:39:05.439 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:39:05 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:39:05.109+0000 7f614933d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-16T16:39:05.439 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:39:05 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:39:05.220+0000 7f614933d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-16T16:39:05.441 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:39:05 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:39:05.112+0000 7fc6c56c6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-16T16:39:05.441 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:39:05 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:39:05.223+0000 7fc6c56c6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-16T16:39:05.757 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:39:05 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:39:05.440+0000 7fc6c56c6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-16T16:39:05.787 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:39:05 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:39:05.437+0000 7f614933d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-16T16:39:05.854 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:39:05 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:39:05.497+0000 7f4ba2b8b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-16T16:39:06.040 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:39:05 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:39:05.785+0000 7f614933d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-16T16:39:06.043 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:39:05 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:39:05.787+0000 7fc6c56c6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-16T16:39:06.110 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:39:05 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:39:05.852+0000 7f4ba2b8b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-16T16:39:06.331 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:39:06 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:39:06.039+0000 7f614933d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-16T16:39:06.331 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:39:06 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:39:06.220+0000 7f614933d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-16T16:39:06.432 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:39:06 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:39:06.108+0000 7f4ba2b8b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-16T16:39:06.432 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:39:06 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:39:06.293+0000 7f4ba2b8b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-16T16:39:06.508 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:39:06 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:39:06.042+0000 7fc6c56c6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-16T16:39:06.508 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:39:06 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:39:06.223+0000 7fc6c56c6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-16T16:39:06.831 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:39:06 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:39:06.547+0000 7f614933d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-16T16:39:06.932 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:39:06 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:39:06.629+0000 7f4ba2b8b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-16T16:39:07.007 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:39:06 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:39:06.551+0000 7fc6c56c6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-16T16:39:07.758 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:39:07 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:39:07.375+0000 7fc6c56c6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-16T16:39:07.758 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:39:07 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:39:07.532+0000 7fc6c56c6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-16T16:39:07.770 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:39:07 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:39:07.368+0000 7f614933d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-16T16:39:07.770 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:39:07 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:39:07.525+0000 7f614933d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-16T16:39:07.876 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:39:07 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:39:07.465+0000 7f4ba2b8b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-16T16:39:07.876 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:39:07 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:39:07.626+0000 7f4ba2b8b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-16T16:39:08.081 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:39:07 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:39:07.769+0000 7f614933d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-16T16:39:08.081 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:39:07 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:39:07.905+0000 7f614933d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-16T16:39:08.183 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:39:07 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:39:07.773+0000 7fc6c56c6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-16T16:39:08.184 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:39:07 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:39:07.909+0000 7fc6c56c6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-16T16:39:08.184 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:39:07 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:39:07.874+0000 7f4ba2b8b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-16T16:39:08.184 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:39:08 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:39:08.014+0000 7f4ba2b8b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-16T16:39:08.369 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:08.367+0000 7f8fbd198700 1 -- 172.21.15.1:0/3906335793 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8fa40a11a0 msgr2=0x7f8fa40a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/2490729488 2023-11-16T16:39:08.369 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:08.367+0000 7f8fbd198700 1 --2- 172.21.15.1:0/3906335793 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8fa40a11a0 0x7f8fa40a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-11-16T16:39:08.508 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:39:08 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:39:08.179+0000 7fc6c56c6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-16T16:39:08.580 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:39:08 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:39:08.175+0000 7f614933d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-16T16:39:08.682 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:39:08 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:39:08.292+0000 7f4ba2b8b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-16T16:39:09.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:09.286+0000 7f8fae7fc700 1 -- 172.21.15.1:0/3906335793 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 20) v1 ==== 135731+0+0 (secure 0 0 0) 0x7f8fa8010880 con 0x7f8fb80ff460 2023-11-16T16:39:09.372 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:09.370+0000 7f8fae7fc700 1 -- 172.21.15.1:0/3906335793 <== mon.0 v2:172.21.15.1:3300/0 8 ==== mgrmap(e 21) v1 ==== 135400+0+0 (secure 0 0 0) 0x7f8fa80ca6b0 con 0x7f8fb80ff460 2023-11-16T16:39:09.372 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:09.371+0000 7f8fae7fc700 1 -- 172.21.15.1:0/3906335793 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8fa40a11a0 msgr2=0x7f8fa40a3650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-16T16:39:09.372 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:09.371+0000 7f8fae7fc700 1 --2- 172.21.15.1:0/3906335793 >> [v2:172.21.15.1:6800/2490729488,v1:172.21.15.1:6801/2490729488] conn(0x7f8fa40a11a0 0x7f8fa40a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:09.508 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:39:09 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y[101566]: 2023-11-16T16:39:09.213+0000 7fc6c56c6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-16T16:39:09.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:09 smithi026 ceph-mon[100112]: Standby manager daemon z restarted 2023-11-16T16:39:09.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:09 smithi026 ceph-mon[100112]: Standby manager daemon z started 2023-11-16T16:39:09.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:09 smithi026 ceph-mon[100112]: Standby manager daemon y restarted 2023-11-16T16:39:09.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:09 smithi026 ceph-mon[100112]: Standby manager daemon y started 2023-11-16T16:39:09.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:09 smithi026 ceph-mon[100112]: from='mgr.? 172.21.15.53:0/4114538377' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-11-16T16:39:09.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:09 smithi026 ceph-mon[100112]: from='mgr.? 172.21.15.53:0/4114538377' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-16T16:39:09.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:09 smithi026 ceph-mon[100112]: from='mgr.? 172.21.15.26:0/4266249722' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-11-16T16:39:09.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:09 smithi026 ceph-mon[100112]: from='mgr.? 172.21.15.53:0/4114538377' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-11-16T16:39:09.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:09 smithi026 ceph-mon[100112]: from='mgr.? 172.21.15.53:0/4114538377' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-16T16:39:09.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:09 smithi026 ceph-mon[100112]: from='mgr.? 172.21.15.26:0/4266249722' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-16T16:39:09.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:09 smithi026 ceph-mon[100112]: from='mgr.? 172.21.15.26:0/4266249722' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-11-16T16:39:09.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:09 smithi026 ceph-mon[100112]: from='mgr.? 172.21.15.26:0/4266249722' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-16T16:39:09.532 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:39:09 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z[102753]: 2023-11-16T16:39:09.211+0000 7f614933d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-16T16:39:09.532 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:09 smithi053 ceph-mon[100638]: Standby manager daemon z restarted 2023-11-16T16:39:09.532 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:09 smithi053 ceph-mon[100638]: Standby manager daemon z started 2023-11-16T16:39:09.532 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:09 smithi053 ceph-mon[100638]: Standby manager daemon y restarted 2023-11-16T16:39:09.532 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:09 smithi053 ceph-mon[100638]: Standby manager daemon y started 2023-11-16T16:39:09.532 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:09 smithi053 ceph-mon[100638]: from='mgr.? 172.21.15.53:0/4114538377' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-11-16T16:39:09.532 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:09 smithi053 ceph-mon[100638]: from='mgr.? 172.21.15.53:0/4114538377' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-16T16:39:09.532 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:09 smithi053 ceph-mon[100638]: from='mgr.? 172.21.15.26:0/4266249722' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-11-16T16:39:09.533 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:09 smithi053 ceph-mon[100638]: from='mgr.? 172.21.15.53:0/4114538377' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-11-16T16:39:09.533 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:09 smithi053 ceph-mon[100638]: from='mgr.? 172.21.15.53:0/4114538377' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-16T16:39:09.533 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:09 smithi053 ceph-mon[100638]: from='mgr.? 172.21.15.26:0/4266249722' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-16T16:39:09.533 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:09 smithi053 ceph-mon[100638]: from='mgr.? 172.21.15.26:0/4266249722' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-11-16T16:39:09.533 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:09 smithi053 ceph-mon[100638]: from='mgr.? 172.21.15.26:0/4266249722' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-16T16:39:09.583 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:39:09 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:39:09.346+0000 7f4ba2b8b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-16T16:39:09.584 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:39:09 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:39:09.580+0000 7f4afefe3700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-16T16:39:09.584 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:39:09 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:39:09.580+0000 7f4afefe3700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-16T16:39:09.584 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:39:09 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:39:09.580+0000 7f4afefe3700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-16T16:39:09.584 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:39:09 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:39:09.580+0000 7f4afefe3700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-16T16:39:09.584 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:39:09 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:39:09.580+0000 7f4afefe3700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-16T16:39:09.584 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:39:09 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:39:09.580+0000 7f4b01fe9700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-16T16:39:09.584 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:39:09 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:39:09.580+0000 7f4b01fe9700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-16T16:39:09.585 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:39:09 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:39:09.580+0000 7f4b01fe9700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-16T16:39:09.585 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:09 smithi001 ceph-mon[98800]: Standby manager daemon z restarted 2023-11-16T16:39:09.585 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:09 smithi001 ceph-mon[98800]: Standby manager daemon z started 2023-11-16T16:39:09.585 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:09 smithi001 ceph-mon[98800]: Standby manager daemon y restarted 2023-11-16T16:39:09.586 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:09 smithi001 ceph-mon[98800]: Standby manager daemon y started 2023-11-16T16:39:09.586 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:09 smithi001 ceph-mon[98800]: from='mgr.? 172.21.15.53:0/4114538377' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-11-16T16:39:09.586 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:09 smithi001 ceph-mon[98800]: from='mgr.? 172.21.15.53:0/4114538377' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-16T16:39:09.586 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:09 smithi001 ceph-mon[98800]: from='mgr.? 172.21.15.26:0/4266249722' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-11-16T16:39:09.586 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:09 smithi001 ceph-mon[98800]: from='mgr.? 172.21.15.53:0/4114538377' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-11-16T16:39:09.586 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:09 smithi001 ceph-mon[98800]: from='mgr.? 172.21.15.53:0/4114538377' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-16T16:39:09.587 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:09 smithi001 ceph-mon[98800]: from='mgr.? 172.21.15.26:0/4266249722' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-16T16:39:09.587 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:09 smithi001 ceph-mon[98800]: from='mgr.? 172.21.15.26:0/4266249722' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-11-16T16:39:09.587 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:09 smithi001 ceph-mon[98800]: from='mgr.? 172.21.15.26:0/4266249722' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-16T16:39:09.932 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:39:09 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:39:09.580+0000 7f4b01fe9700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-16T16:39:09.932 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:39:09 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:39:09.580+0000 7f4b01fe9700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-16T16:39:10.381 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.379+0000 7f8fae7fc700 1 -- 172.21.15.1:0/3906335793 <== mon.0 v2:172.21.15.1:3300/0 9 ==== mgrmap(e 22) v1 ==== 135527+0+0 (secure 0 0 0) 0x7f8fa8096650 con 0x7f8fb80ff460 2023-11-16T16:39:10.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.380+0000 7f8fae7fc700 1 --2- 172.21.15.1:0/3906335793 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f8fa4027e90 0x7f8fa4028260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:10.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.380+0000 7f8fae7fc700 1 -- 172.21.15.1:0/3906335793 --> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f8fb8061a30 con 0x7f8fa4027e90 2023-11-16T16:39:10.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.381+0000 7f8fbd198700 1 --2- 172.21.15.1:0/3906335793 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f8fa4027e90 0x7f8fa4028260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:10.385 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.383+0000 7f8fbd198700 1 --2- 172.21.15.1:0/3906335793 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f8fa4027e90 0x7f8fa4028260 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009c30 tx=0x7f8fb4009260 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:10.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.452+0000 7f8fae7fc700 1 -- 172.21.15.1:0/3906335793 <== mgr.15432 v2:172.21.15.1:6800/4199419371 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f8fb8061a30 con 0x7f8fa4027e90 2023-11-16T16:39:10.453 INFO:teuthology.orchestra.run.smithi001.stdout:Schedule set for path / 2023-11-16T16:39:10.456 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.454+0000 7f8fbf3fc700 1 -- 172.21.15.1:0/3906335793 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f8fa4027e90 msgr2=0x7f8fa4028260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:10.456 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.454+0000 7f8fbf3fc700 1 --2- 172.21.15.1:0/3906335793 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f8fa4027e90 0x7f8fa4028260 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009c30 tx=0x7f8fb4009260 comp rx=0 tx=0).stop 2023-11-16T16:39:10.456 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.454+0000 7f8fbf3fc700 1 -- 172.21.15.1:0/3906335793 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fb80ff460 msgr2=0x7f8fb8198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:10.456 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.454+0000 7f8fbf3fc700 1 --2- 172.21.15.1:0/3906335793 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fb80ff460 0x7f8fb8198820 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8fa800e9a0 tx=0x7f8fa800ecb0 comp rx=0 tx=0).stop 2023-11-16T16:39:10.456 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.454+0000 7f8fbd999700 1 -- 172.21.15.1:0/3906335793 reap_dead start 2023-11-16T16:39:10.456 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.454+0000 7f8fbf3fc700 1 -- 172.21.15.1:0/3906335793 shutdown_connections 2023-11-16T16:39:10.456 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.454+0000 7f8fbf3fc700 1 -- 172.21.15.1:0/3906335793 >> 172.21.15.1:0/3906335793 conn(0x7f8fb80fa720 msgr2=0x7f8fb80fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:10.456 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.454+0000 7f8fbf3fc700 1 -- 172.21.15.1:0/3906335793 shutdown_connections 2023-11-16T16:39:10.456 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.454+0000 7f8fbf3fc700 1 -- 172.21.15.1:0/3906335793 wait complete. 2023-11-16T16:39:10.468 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6M3h' 2023-11-16T16:39:10.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: mgrmap e20: x(active, since 8m), standbys: y, z 2023-11-16T16:39:10.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: Active manager daemon x restarted 2023-11-16T16:39:10.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: Activating manager daemon x 2023-11-16T16:39:10.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: osdmap e87: 12 total, 12 up, 12 in 2023-11-16T16:39:10.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: mgrmap e21: x(active, starting, since 0.0122417s), standbys: y, z 2023-11-16T16:39:10.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T16:39:10.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:39:10.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:39:10.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-11-16T16:39:10.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-11-16T16:39:10.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-11-16T16:39:10.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-11-16T16:39:10.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-11-16T16:39:10.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-11-16T16:39:10.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-16T16:39:10.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-16T16:39:10.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-16T16:39:10.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-16T16:39:10.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-16T16:39:10.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-16T16:39:10.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-11-16T16:39:10.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-11-16T16:39:10.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-11-16T16:39:10.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-16T16:39:10.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-16T16:39:10.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-16T16:39:10.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-16T16:39:10.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-16T16:39:10.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-16T16:39:10.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-16T16:39:10.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-16T16:39:10.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-16T16:39:10.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-16T16:39:10.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-16T16:39:10.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-16T16:39:10.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-11-16T16:39:10.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-11-16T16:39:10.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-11-16T16:39:10.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: Manager daemon x is now available 2023-11-16T16:39:10.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:39:10.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:39:10.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-11-16T16:39:10.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:10 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-11-16T16:39:10.608 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.604+0000 7f38ab2b6700 1 -- 172.21.15.1:0/2800509821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38a4100350 msgr2=0x7f38a41007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:10.608 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.604+0000 7f38ab2b6700 1 --2- 172.21.15.1:0/2800509821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38a4100350 0x7f38a41007a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f38a0009990 tx=0x7f38a0009ca0 comp rx=0 tx=0).stop 2023-11-16T16:39:10.608 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.604+0000 7f38ab2b6700 1 -- 172.21.15.1:0/2800509821 shutdown_connections 2023-11-16T16:39:10.608 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.604+0000 7f38ab2b6700 1 --2- 172.21.15.1:0/2800509821 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38a4100ce0 0x7f38a41075a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:10.608 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.604+0000 7f38ab2b6700 1 --2- 172.21.15.1:0/2800509821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38a40ff160 0x7f38a40ff570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:10.608 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.604+0000 7f38ab2b6700 1 --2- 172.21.15.1:0/2800509821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38a4100350 0x7f38a41007a0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:10.608 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.604+0000 7f38ab2b6700 1 -- 172.21.15.1:0/2800509821 >> 172.21.15.1:0/2800509821 conn(0x7f38a40fa780 msgr2=0x7f38a40fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:10.608 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.604+0000 7f38ab2b6700 1 -- 172.21.15.1:0/2800509821 shutdown_connections 2023-11-16T16:39:10.609 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.604+0000 7f38ab2b6700 1 -- 172.21.15.1:0/2800509821 wait complete. 2023-11-16T16:39:10.609 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.605+0000 7f38ab2b6700 1 Processor -- start 2023-11-16T16:39:10.609 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.605+0000 7f38ab2b6700 1 -- start start 2023-11-16T16:39:10.609 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.605+0000 7f38ab2b6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38a40ff160 0x7f38a4196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:10.609 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.605+0000 7f38ab2b6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38a4100350 0x7f38a4196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:10.609 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.605+0000 7f38ab2b6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38a4100ce0 0x7f38a419d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:10.609 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.605+0000 7f38ab2b6700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a419db80 con 0x7f38a4100ce0 2023-11-16T16:39:10.609 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.605+0000 7f38ab2b6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a419dcc0 con 0x7f38a40ff160 2023-11-16T16:39:10.609 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.605+0000 7f38ab2b6700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a419de50 con 0x7f38a4100350 2023-11-16T16:39:10.609 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.605+0000 7f38a9853700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38a4100ce0 0x7f38a419d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:10.609 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.605+0000 7f38a9052700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38a40ff160 0x7f38a4196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:10.610 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.605+0000 7f38a9853700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38a4100ce0 0x7f38a419d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43680/0 (socket says 172.21.15.1:43680) 2023-11-16T16:39:10.610 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.605+0000 7f38a9853700 1 -- 172.21.15.1:0/1111053656 learned_addr learned my addr 172.21.15.1:0/1111053656 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:10.610 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.605+0000 7f38a8851700 1 --2- 172.21.15.1:0/1111053656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38a4100350 0x7f38a4196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:10.610 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.605+0000 7f38a9853700 1 -- 172.21.15.1:0/1111053656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38a4100350 msgr2=0x7f38a4196540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:10.610 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.605+0000 7f38a9853700 1 --2- 172.21.15.1:0/1111053656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38a4100350 0x7f38a4196540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:10.610 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.605+0000 7f38a9853700 1 -- 172.21.15.1:0/1111053656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38a40ff160 msgr2=0x7f38a4196000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:10.610 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.605+0000 7f38a9853700 1 --2- 172.21.15.1:0/1111053656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38a40ff160 0x7f38a4196000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:10.610 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.606+0000 7f38a9853700 1 -- 172.21.15.1:0/1111053656 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a00095f0 con 0x7f38a4100ce0 2023-11-16T16:39:10.610 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.606+0000 7f38a9853700 1 --2- 172.21.15.1:0/1111053656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38a4100ce0 0x7f38a419d5b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f389c007ee0 tx=0x7f389c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:10.610 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.606+0000 7f389a7fc700 1 -- 172.21.15.1:0/1111053656 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c00dd20 con 0x7f38a4100ce0 2023-11-16T16:39:10.610 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.606+0000 7f389a7fc700 1 -- 172.21.15.1:0/1111053656 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f389c00f040 con 0x7f38a4100ce0 2023-11-16T16:39:10.610 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.606+0000 7f38ab2b6700 1 -- 172.21.15.1:0/1111053656 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a410aac0 con 0x7f38a4100ce0 2023-11-16T16:39:10.611 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.606+0000 7f389a7fc700 1 -- 172.21.15.1:0/1111053656 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c0048a0 con 0x7f38a4100ce0 2023-11-16T16:39:10.611 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.606+0000 7f38ab2b6700 1 -- 172.21.15.1:0/1111053656 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a410a930 con 0x7f38a4100ce0 2023-11-16T16:39:10.611 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.607+0000 7f38ab2b6700 1 -- 172.21.15.1:0/1111053656 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a419e540 con 0x7f38a4100ce0 2023-11-16T16:39:10.611 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.608+0000 7f389a7fc700 1 -- 172.21.15.1:0/1111053656 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 22) v1 ==== 135527+0+0 (secure 0 0 0) 0x7f389c003a00 con 0x7f38a4100ce0 2023-11-16T16:39:10.612 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.609+0000 7f389a7fc700 1 --2- 172.21.15.1:0/1111053656 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f38900a0fd0 0x7f38900a3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:10.612 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.609+0000 7f38a9052700 1 --2- 172.21.15.1:0/1111053656 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f38900a0fd0 0x7f38900a3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:10.612 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.609+0000 7f389a7fc700 1 -- 172.21.15.1:0/1111053656 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f389c0cc690 con 0x7f38a4100ce0 2023-11-16T16:39:10.612 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.609+0000 7f38a9052700 1 --2- 172.21.15.1:0/1111053656 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f38900a0fd0 0x7f38900a3480 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f38a41001b0 tx=0x7f389400b410 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:10.614 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.612+0000 7f389a7fc700 1 -- 172.21.15.1:0/1111053656 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f389c095f80 con 0x7f38a4100ce0 2023-11-16T16:39:10.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: mgrmap e20: x(active, since 8m), standbys: y, z 2023-11-16T16:39:10.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: Active manager daemon x restarted 2023-11-16T16:39:10.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: Activating manager daemon x 2023-11-16T16:39:10.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: osdmap e87: 12 total, 12 up, 12 in 2023-11-16T16:39:10.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: mgrmap e21: x(active, starting, since 0.0122417s), standbys: y, z 2023-11-16T16:39:10.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T16:39:10.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:39:10.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:39:10.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-11-16T16:39:10.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-11-16T16:39:10.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-11-16T16:39:10.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-11-16T16:39:10.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-11-16T16:39:10.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-11-16T16:39:10.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-16T16:39:10.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-16T16:39:10.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-16T16:39:10.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-16T16:39:10.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-16T16:39:10.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-16T16:39:10.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-11-16T16:39:10.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-11-16T16:39:10.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-11-16T16:39:10.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-16T16:39:10.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-16T16:39:10.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-16T16:39:10.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-16T16:39:10.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-16T16:39:10.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-16T16:39:10.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-16T16:39:10.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-16T16:39:10.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-16T16:39:10.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-16T16:39:10.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-16T16:39:10.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-16T16:39:10.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-11-16T16:39:10.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-11-16T16:39:10.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-11-16T16:39:10.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: Manager daemon x is now available 2023-11-16T16:39:10.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:39:10.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:39:10.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-11-16T16:39:10.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:10 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-11-16T16:39:10.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: mgrmap e20: x(active, since 8m), standbys: y, z 2023-11-16T16:39:10.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: Active manager daemon x restarted 2023-11-16T16:39:10.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: Activating manager daemon x 2023-11-16T16:39:10.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: osdmap e87: 12 total, 12 up, 12 in 2023-11-16T16:39:10.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: mgrmap e21: x(active, starting, since 0.0122417s), standbys: y, z 2023-11-16T16:39:10.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-16T16:39:10.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-16T16:39:10.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-16T16:39:10.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-11-16T16:39:10.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-11-16T16:39:10.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-11-16T16:39:10.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-11-16T16:39:10.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-11-16T16:39:10.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-11-16T16:39:10.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-16T16:39:10.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-16T16:39:10.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-16T16:39:10.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-16T16:39:10.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-16T16:39:10.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-16T16:39:10.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-11-16T16:39:10.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-11-16T16:39:10.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-11-16T16:39:10.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-16T16:39:10.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-16T16:39:10.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-16T16:39:10.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-16T16:39:10.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-16T16:39:10.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-16T16:39:10.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-16T16:39:10.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-16T16:39:10.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-16T16:39:10.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-16T16:39:10.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-16T16:39:10.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-16T16:39:10.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-11-16T16:39:10.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-11-16T16:39:10.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-11-16T16:39:10.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: Manager daemon x is now available 2023-11-16T16:39:10.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:39:10.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:39:10.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-11-16T16:39:10.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:10 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-11-16T16:39:10.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.843+0000 7f38ab2b6700 1 -- 172.21.15.1:0/1111053656 --> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}) v1 -- 0x7f38a4069420 con 0x7f38900a0fd0 2023-11-16T16:39:10.901 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.899+0000 7f389a7fc700 1 -- 172.21.15.1:0/1111053656 <== mgr.15432 v2:172.21.15.1:6800/4199419371 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f38a4069420 con 0x7f38900a0fd0 2023-11-16T16:39:10.901 INFO:teuthology.orchestra.run.smithi001.stdout:Retention added to path / 2023-11-16T16:39:10.903 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.901+0000 7f38ab2b6700 1 -- 172.21.15.1:0/1111053656 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f38900a0fd0 msgr2=0x7f38900a3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:10.903 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.901+0000 7f38ab2b6700 1 --2- 172.21.15.1:0/1111053656 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f38900a0fd0 0x7f38900a3480 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f38a41001b0 tx=0x7f389400b410 comp rx=0 tx=0).stop 2023-11-16T16:39:10.903 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.901+0000 7f38ab2b6700 1 -- 172.21.15.1:0/1111053656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38a4100ce0 msgr2=0x7f38a419d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:10.903 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.901+0000 7f38ab2b6700 1 --2- 172.21.15.1:0/1111053656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38a4100ce0 0x7f38a419d5b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f389c007ee0 tx=0x7f389c00d3b0 comp rx=0 tx=0).stop 2023-11-16T16:39:10.903 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.901+0000 7f38ab2b6700 1 -- 172.21.15.1:0/1111053656 shutdown_connections 2023-11-16T16:39:10.903 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.901+0000 7f38ab2b6700 1 --2- 172.21.15.1:0/1111053656 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f38900a0fd0 0x7f38900a3480 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:10.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.901+0000 7f38ab2b6700 1 --2- 172.21.15.1:0/1111053656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38a4100ce0 0x7f38a419d5b0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:10.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.901+0000 7f38ab2b6700 1 --2- 172.21.15.1:0/1111053656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38a40ff160 0x7f38a4196000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:10.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.901+0000 7f38ab2b6700 1 --2- 172.21.15.1:0/1111053656 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f38a4100350 0x7f38a4196540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:10.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.901+0000 7f38ab2b6700 1 -- 172.21.15.1:0/1111053656 >> 172.21.15.1:0/1111053656 conn(0x7f38a40fa780 msgr2=0x7f38a4068d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:10.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.901+0000 7f38ab2b6700 1 -- 172.21.15.1:0/1111053656 shutdown_connections 2023-11-16T16:39:10.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:10.901+0000 7f38ab2b6700 1 -- 172.21.15.1:0/1111053656 wait complete. 2023-11-16T16:39:10.915 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-11-16T16:39:11.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.044+0000 7f2a1195e700 1 -- 172.21.15.1:0/2976159813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a0c105f00 msgr2=0x7f2a0c108360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:11.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.044+0000 7f2a1195e700 1 --2- 172.21.15.1:0/2976159813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a0c105f00 0x7f2a0c108360 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2a0400b0a0 tx=0x7f2a0400b3b0 comp rx=0 tx=0).stop 2023-11-16T16:39:11.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.044+0000 7f2a1195e700 1 -- 172.21.15.1:0/2976159813 shutdown_connections 2023-11-16T16:39:11.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.044+0000 7f2a1195e700 1 --2- 172.21.15.1:0/2976159813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a0c105f00 0x7f2a0c108360 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:11.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.044+0000 7f2a1195e700 1 --2- 172.21.15.1:0/2976159813 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a0c100cc0 0x7f2a0c1030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:11.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.044+0000 7f2a1195e700 1 --2- 172.21.15.1:0/2976159813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a0c1035e0 0x7f2a0c1059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:11.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.044+0000 7f2a1195e700 1 -- 172.21.15.1:0/2976159813 >> 172.21.15.1:0/2976159813 conn(0x7f2a0c0fa720 msgr2=0x7f2a0c0fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:11.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.044+0000 7f2a1195e700 1 -- 172.21.15.1:0/2976159813 shutdown_connections 2023-11-16T16:39:11.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.044+0000 7f2a1195e700 1 -- 172.21.15.1:0/2976159813 wait complete. 2023-11-16T16:39:11.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.045+0000 7f2a1195e700 1 Processor -- start 2023-11-16T16:39:11.049 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.045+0000 7f2a1195e700 1 -- start start 2023-11-16T16:39:11.049 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.045+0000 7f2a1195e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a0c100cc0 0x7f2a0c196130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:11.049 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.045+0000 7f2a1195e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a0c1035e0 0x7f2a0c196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:11.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.045+0000 7f2a1195e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a0c105f00 0x7f2a0c19d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:11.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.045+0000 7f2a1195e700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c19dcb0 con 0x7f2a0c100cc0 2023-11-16T16:39:11.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.045+0000 7f2a1195e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c19ddf0 con 0x7f2a0c1035e0 2023-11-16T16:39:11.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.045+0000 7f2a1195e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c19df80 con 0x7f2a0c105f00 2023-11-16T16:39:11.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.046+0000 7f2a0affd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a0c100cc0 0x7f2a0c196130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:11.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.046+0000 7f2a0affd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a0c100cc0 0x7f2a0c196130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43690/0 (socket says 172.21.15.1:43690) 2023-11-16T16:39:11.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.046+0000 7f2a0a7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a0c1035e0 0x7f2a0c196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:11.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.046+0000 7f2a0affd700 1 -- 172.21.15.1:0/1736645421 learned_addr learned my addr 172.21.15.1:0/1736645421 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:11.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.046+0000 7f2a0b7fe700 1 --2- 172.21.15.1:0/1736645421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a0c105f00 0x7f2a0c19d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:11.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.046+0000 7f2a0affd700 1 -- 172.21.15.1:0/1736645421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a0c105f00 msgr2=0x7f2a0c19d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:11.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.046+0000 7f2a0affd700 1 --2- 172.21.15.1:0/1736645421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a0c105f00 0x7f2a0c19d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:11.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.046+0000 7f2a0affd700 1 -- 172.21.15.1:0/1736645421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a0c1035e0 msgr2=0x7f2a0c196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:11.051 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.046+0000 7f2a0affd700 1 --2- 172.21.15.1:0/1736645421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a0c1035e0 0x7f2a0c196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:11.051 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.046+0000 7f2a0affd700 1 -- 172.21.15.1:0/1736645421 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a04009d00 con 0x7f2a0c100cc0 2023-11-16T16:39:11.051 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.046+0000 7f2a0a7fc700 1 --2- 172.21.15.1:0/1736645421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a0c1035e0 0x7f2a0c196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:39:11.051 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.046+0000 7f2a0b7fe700 1 --2- 172.21.15.1:0/1736645421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a0c105f00 0x7f2a0c19d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:39:11.051 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.046+0000 7f2a0affd700 1 --2- 172.21.15.1:0/1736645421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a0c100cc0 0x7f2a0c196130 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f29fc00e9a0 tx=0x7f29fc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:11.051 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.046+0000 7f2a1095c700 1 -- 172.21.15.1:0/1736645421 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc009790 con 0x7f2a0c100cc0 2023-11-16T16:39:11.051 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.046+0000 7f2a1095c700 1 -- 172.21.15.1:0/1736645421 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29fc010430 con 0x7f2a0c100cc0 2023-11-16T16:39:11.051 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.046+0000 7f2a1195e700 1 -- 172.21.15.1:0/1736645421 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a0c10bbf0 con 0x7f2a0c100cc0 2023-11-16T16:39:11.051 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.046+0000 7f2a1095c700 1 -- 172.21.15.1:0/1736645421 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc018680 con 0x7f2a0c100cc0 2023-11-16T16:39:11.051 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.046+0000 7f2a1195e700 1 -- 172.21.15.1:0/1736645421 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a0c10ba60 con 0x7f2a0c100cc0 2023-11-16T16:39:11.051 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.047+0000 7f2a1195e700 1 -- 172.21.15.1:0/1736645421 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a0c067c00 con 0x7f2a0c100cc0 2023-11-16T16:39:11.052 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.049+0000 7f2a1095c700 1 -- 172.21.15.1:0/1736645421 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 22) v1 ==== 135527+0+0 (secure 0 0 0) 0x7f29fc0105a0 con 0x7f2a0c100cc0 2023-11-16T16:39:11.052 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.049+0000 7f2a1095c700 1 --2- 172.21.15.1:0/1736645421 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f29f40a0f50 0x7f29f40a3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:11.052 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.049+0000 7f2a1095c700 1 -- 172.21.15.1:0/1736645421 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f29fc014070 con 0x7f2a0c100cc0 2023-11-16T16:39:11.052 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.049+0000 7f2a0a7fc700 1 --2- 172.21.15.1:0/1736645421 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f29f40a0f50 0x7f29f40a3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:11.053 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.050+0000 7f2a0a7fc700 1 --2- 172.21.15.1:0/1736645421 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f29f40a0f50 0x7f29f40a3400 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2a00011490 tx=0x7f2a00009260 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:11.058 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.056+0000 7f2a1095c700 1 -- 172.21.15.1:0/1736645421 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f29fc096300 con 0x7f2a0c100cc0 2023-11-16T16:39:11.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.284+0000 7f2a1195e700 1 -- 172.21.15.1:0/1736645421 --> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a0c102810 con 0x7f29f40a0f50 2023-11-16T16:39:11.294 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.292+0000 7f2a1095c700 1 -- 172.21.15.1:0/1736645421 <== mgr.15432 v2:172.21.15.1:6800/4199419371 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+283 (secure 0 0 0) 0x7f2a0c102810 con 0x7f29f40a0f50 2023-11-16T16:39:11.294 INFO:teuthology.orchestra.run.smithi001.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1M", "retention": {"h": 3, "M": 6}, "start": "2023-11-16T00:00:00", "created": "2023-11-16T16:39:10", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2023-11-16T16:39:11.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.294+0000 7f2a1195e700 1 -- 172.21.15.1:0/1736645421 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f29f40a0f50 msgr2=0x7f29f40a3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:11.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.294+0000 7f2a1195e700 1 --2- 172.21.15.1:0/1736645421 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f29f40a0f50 0x7f29f40a3400 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2a00011490 tx=0x7f2a00009260 comp rx=0 tx=0).stop 2023-11-16T16:39:11.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.295+0000 7f2a1195e700 1 -- 172.21.15.1:0/1736645421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a0c100cc0 msgr2=0x7f2a0c196130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:11.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.295+0000 7f2a1195e700 1 --2- 172.21.15.1:0/1736645421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a0c100cc0 0x7f2a0c196130 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f29fc00e9a0 tx=0x7f29fc00ecb0 comp rx=0 tx=0).stop 2023-11-16T16:39:11.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.295+0000 7f2a1195e700 1 -- 172.21.15.1:0/1736645421 shutdown_connections 2023-11-16T16:39:11.298 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.295+0000 7f2a1195e700 1 --2- 172.21.15.1:0/1736645421 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f29f40a0f50 0x7f29f40a3400 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:11.298 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.295+0000 7f2a1195e700 1 --2- 172.21.15.1:0/1736645421 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2a0c105f00 0x7f2a0c19d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:11.298 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.295+0000 7f2a1195e700 1 --2- 172.21.15.1:0/1736645421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a0c100cc0 0x7f2a0c196130 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:11.298 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.295+0000 7f2a1195e700 1 --2- 172.21.15.1:0/1736645421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a0c1035e0 0x7f2a0c196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:11.298 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.295+0000 7f2a1195e700 1 -- 172.21.15.1:0/1736645421 >> 172.21.15.1:0/1736645421 conn(0x7f2a0c0fa720 msgr2=0x7f2a0c102100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:11.298 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.295+0000 7f2a1195e700 1 -- 172.21.15.1:0/1736645421 shutdown_connections 2023-11-16T16:39:11.299 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.295+0000 7f2a1195e700 1 -- 172.21.15.1:0/1736645421 wait complete. 2023-11-16T16:39:11.313 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-11-16T16:39:11.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.446+0000 7fd436c21700 1 -- 172.21.15.1:0/2971772964 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd430101dc0 msgr2=0x7fd430108680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:11.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.446+0000 7fd436c21700 1 --2- 172.21.15.1:0/2971772964 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd430101dc0 0x7fd430108680 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd42800b0a0 tx=0x7fd42800b3b0 comp rx=0 tx=0).stop 2023-11-16T16:39:11.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.446+0000 7fd436c21700 1 -- 172.21.15.1:0/2971772964 shutdown_connections 2023-11-16T16:39:11.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.446+0000 7fd436c21700 1 --2- 172.21.15.1:0/2971772964 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd430101dc0 0x7fd430108680 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:11.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.446+0000 7fd436c21700 1 --2- 172.21.15.1:0/2971772964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd430100240 0x7fd430100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:11.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.446+0000 7fd436c21700 1 --2- 172.21.15.1:0/2971772964 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd430101430 0x7fd430101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:11.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.446+0000 7fd436c21700 1 -- 172.21.15.1:0/2971772964 >> 172.21.15.1:0/2971772964 conn(0x7fd4300fb880 msgr2=0x7fd4300fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:11.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.446+0000 7fd436c21700 1 -- 172.21.15.1:0/2971772964 shutdown_connections 2023-11-16T16:39:11.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.446+0000 7fd436c21700 1 -- 172.21.15.1:0/2971772964 wait complete. 2023-11-16T16:39:11.451 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.446+0000 7fd436c21700 1 Processor -- start 2023-11-16T16:39:11.451 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.447+0000 7fd436c21700 1 -- start start 2023-11-16T16:39:11.451 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.447+0000 7fd436c21700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd430100240 0x7fd4301960a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:11.451 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.447+0000 7fd436c21700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd430101430 0x7fd4301965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:11.451 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.447+0000 7fd436c21700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd430101dc0 0x7fd43019d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:11.451 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.447+0000 7fd436c21700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43019dc20 con 0x7fd430100240 2023-11-16T16:39:11.451 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.447+0000 7fd436c21700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43019dd60 con 0x7fd430101dc0 2023-11-16T16:39:11.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.447+0000 7fd436c21700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43019def0 con 0x7fd430101430 2023-11-16T16:39:11.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.447+0000 7fd4349bd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd430100240 0x7fd4301960a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:11.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.447+0000 7fd4349bd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd430100240 0x7fd4301960a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43704/0 (socket says 172.21.15.1:43704) 2023-11-16T16:39:11.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.447+0000 7fd4349bd700 1 -- 172.21.15.1:0/3049592026 learned_addr learned my addr 172.21.15.1:0/3049592026 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:11.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.447+0000 7fd42ffff700 1 --2- 172.21.15.1:0/3049592026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd430101430 0x7fd4301965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:11.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.447+0000 7fd4351be700 1 --2- 172.21.15.1:0/3049592026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd430101dc0 0x7fd43019d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:11.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.447+0000 7fd4349bd700 1 -- 172.21.15.1:0/3049592026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd430101430 msgr2=0x7fd4301965e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:11.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.447+0000 7fd4349bd700 1 --2- 172.21.15.1:0/3049592026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd430101430 0x7fd4301965e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:11.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.447+0000 7fd4349bd700 1 -- 172.21.15.1:0/3049592026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd430101dc0 msgr2=0x7fd43019d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:11.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.447+0000 7fd4349bd700 1 --2- 172.21.15.1:0/3049592026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd430101dc0 0x7fd43019d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:11.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.447+0000 7fd4349bd700 1 -- 172.21.15.1:0/3049592026 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd428009d00 con 0x7fd430100240 2023-11-16T16:39:11.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.447+0000 7fd4351be700 1 --2- 172.21.15.1:0/3049592026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd430101dc0 0x7fd43019d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:39:11.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.448+0000 7fd4349bd700 1 --2- 172.21.15.1:0/3049592026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd430100240 0x7fd4301960a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd42400d980 tx=0x7fd42400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:11.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.448+0000 7fd42dffb700 1 -- 172.21.15.1:0/3049592026 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42400b950 con 0x7fd430100240 2023-11-16T16:39:11.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.448+0000 7fd436c21700 1 -- 172.21.15.1:0/3049592026 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd43010bbb0 con 0x7fd430100240 2023-11-16T16:39:11.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.448+0000 7fd42dffb700 1 -- 172.21.15.1:0/3049592026 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd424007620 con 0x7fd430100240 2023-11-16T16:39:11.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.448+0000 7fd42dffb700 1 -- 172.21.15.1:0/3049592026 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42400f3c0 con 0x7fd430100240 2023-11-16T16:39:11.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.448+0000 7fd436c21700 1 -- 172.21.15.1:0/3049592026 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd43010ba20 con 0x7fd430100240 2023-11-16T16:39:11.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.449+0000 7fd436c21700 1 -- 172.21.15.1:0/3049592026 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd430107720 con 0x7fd430100240 2023-11-16T16:39:11.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.450+0000 7fd42dffb700 1 -- 172.21.15.1:0/3049592026 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 22) v1 ==== 135527+0+0 (secure 0 0 0) 0x7fd424010640 con 0x7fd430100240 2023-11-16T16:39:11.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.450+0000 7fd42dffb700 1 --2- 172.21.15.1:0/3049592026 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fd4200a0f50 0x7fd4200a3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:11.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.451+0000 7fd42dffb700 1 -- 172.21.15.1:0/3049592026 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd4240cc540 con 0x7fd430100240 2023-11-16T16:39:11.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.451+0000 7fd42ffff700 1 --2- 172.21.15.1:0/3049592026 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fd4200a0f50 0x7fd4200a3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:11.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.451+0000 7fd42ffff700 1 --2- 172.21.15.1:0/3049592026 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fd4200a0f50 0x7fd4200a3400 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd41c009be0 tx=0x7fd41c009260 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:11.457 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.455+0000 7fd42dffb700 1 -- 172.21.15.1:0/3049592026 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd424095e30 con 0x7fd430100240 2023-11-16T16:39:11.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:11 smithi001 ceph-mon[98800]: mgrmap e22: x(active, since 1.01768s), standbys: y, z 2023-11-16T16:39:11.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:11 smithi001 ceph-mon[98800]: [16/Nov/2023:16:39:10] ENGINE Bus STARTING 2023-11-16T16:39:11.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:11 smithi001 ceph-mon[98800]: [16/Nov/2023:16:39:10] ENGINE Serving on http://172.21.15.1:8765 2023-11-16T16:39:11.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:11 smithi001 ceph-mon[98800]: [16/Nov/2023:16:39:10] ENGINE Serving on https://172.21.15.1:7150 2023-11-16T16:39:11.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:11 smithi001 ceph-mon[98800]: [16/Nov/2023:16:39:10] ENGINE Bus STARTED 2023-11-16T16:39:11.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:11 smithi001 ceph-mon[98800]: from='client.15498 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:39:11.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:11 smithi001 ceph-mon[98800]: from='client.15504 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:39:11.692 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.691+0000 7fd436c21700 1 -- 172.21.15.1:0/3049592026 --> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fd430061a60 con 0x7fd4200a0f50 2023-11-16T16:39:11.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.698+0000 7fd42dffb700 1 -- 172.21.15.1:0/3049592026 <== mgr.15432 v2:172.21.15.1:6800/4199419371 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7fd430061a60 con 0x7fd4200a0f50 2023-11-16T16:39:11.700 INFO:teuthology.orchestra.run.smithi001.stdout:/ 1M 6M3h 2023-11-16T16:39:11.702 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.700+0000 7fd436c21700 1 -- 172.21.15.1:0/3049592026 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fd4200a0f50 msgr2=0x7fd4200a3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:11.702 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.700+0000 7fd436c21700 1 --2- 172.21.15.1:0/3049592026 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fd4200a0f50 0x7fd4200a3400 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd41c009be0 tx=0x7fd41c009260 comp rx=0 tx=0).stop 2023-11-16T16:39:11.702 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.700+0000 7fd436c21700 1 -- 172.21.15.1:0/3049592026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd430100240 msgr2=0x7fd4301960a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:11.702 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.700+0000 7fd436c21700 1 --2- 172.21.15.1:0/3049592026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd430100240 0x7fd4301960a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd42400d980 tx=0x7fd42400dc90 comp rx=0 tx=0).stop 2023-11-16T16:39:11.702 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.700+0000 7fd436c21700 1 -- 172.21.15.1:0/3049592026 shutdown_connections 2023-11-16T16:39:11.702 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.700+0000 7fd436c21700 1 --2- 172.21.15.1:0/3049592026 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fd4200a0f50 0x7fd4200a3400 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:11.702 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.700+0000 7fd436c21700 1 --2- 172.21.15.1:0/3049592026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd430101dc0 0x7fd43019d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:11.703 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.700+0000 7fd436c21700 1 --2- 172.21.15.1:0/3049592026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd430100240 0x7fd4301960a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:11.703 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.700+0000 7fd436c21700 1 --2- 172.21.15.1:0/3049592026 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd430101430 0x7fd4301965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:11.703 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.700+0000 7fd436c21700 1 -- 172.21.15.1:0/3049592026 >> 172.21.15.1:0/3049592026 conn(0x7fd4300fb880 msgr2=0x7fd430104860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:11.703 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.700+0000 7fd436c21700 1 -- 172.21.15.1:0/3049592026 shutdown_connections 2023-11-16T16:39:11.703 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:11.700+0000 7fd436c21700 1 -- 172.21.15.1:0/3049592026 wait complete. 2023-11-16T16:39:11.714 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2023-11-16T16:39:11.742 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2023-11-16T16:39:11.767 INFO:teuthology.run_tasks:Running task workunit... 2023-11-16T16:39:11.791 INFO:tasks.workunit:Pulling workunits from ref 5e4a743ad7fd1422ebd6c4dc950a21a303279207 2023-11-16T16:39:11.791 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-11-16T16:39:11.791 INFO:tasks.workunit:timeout=3h 2023-11-16T16:39:11.791 INFO:tasks.workunit:cleanup=True 2023-11-16T16:39:11.791 DEBUG:teuthology.orchestra.run.smithi001:> stat -- /home/ubuntu/cephtest/mnt.0 2023-11-16T16:39:11.792 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:11 smithi026 ceph-mon[100112]: mgrmap e22: x(active, since 1.01768s), standbys: y, z 2023-11-16T16:39:11.792 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:11 smithi026 ceph-mon[100112]: [16/Nov/2023:16:39:10] ENGINE Bus STARTING 2023-11-16T16:39:11.792 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:11 smithi026 ceph-mon[100112]: [16/Nov/2023:16:39:10] ENGINE Serving on http://172.21.15.1:8765 2023-11-16T16:39:11.792 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:11 smithi026 ceph-mon[100112]: [16/Nov/2023:16:39:10] ENGINE Serving on https://172.21.15.1:7150 2023-11-16T16:39:11.792 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:11 smithi026 ceph-mon[100112]: [16/Nov/2023:16:39:10] ENGINE Bus STARTED 2023-11-16T16:39:11.792 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:11 smithi026 ceph-mon[100112]: from='client.15498 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:39:11.793 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:11 smithi026 ceph-mon[100112]: from='client.15504 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:39:11.811 INFO:teuthology.orchestra.run.smithi001.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-11-16T16:39:11.811 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-11-16T16:39:11.811 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 7ch/124d Inode: 1099511628283 Links: 2 2023-11-16T16:39:11.811 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-16T16:39:11.811 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:cephfs_t:s0 2023-11-16T16:39:11.811 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2023-11-16 16:37:20.516375233 +0000 2023-11-16T16:39:11.811 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2023-11-16 16:37:20.516375233 +0000 2023-11-16T16:39:11.811 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2023-11-16 16:38:53.058926863 +0000 2023-11-16T16:39:11.811 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2023-11-16 16:37:20.516375233 +0000 2023-11-16T16:39:11.811 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-11-16T16:39:11.812 DEBUG:teuthology.orchestra.run.smithi001:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-11-16T16:39:11.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:11 smithi053 ceph-mon[100638]: mgrmap e22: x(active, since 1.01768s), standbys: y, z 2023-11-16T16:39:11.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:11 smithi053 ceph-mon[100638]: [16/Nov/2023:16:39:10] ENGINE Bus STARTING 2023-11-16T16:39:11.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:11 smithi053 ceph-mon[100638]: [16/Nov/2023:16:39:10] ENGINE Serving on http://172.21.15.1:8765 2023-11-16T16:39:11.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:11 smithi053 ceph-mon[100638]: [16/Nov/2023:16:39:10] ENGINE Serving on https://172.21.15.1:7150 2023-11-16T16:39:11.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:11 smithi053 ceph-mon[100638]: [16/Nov/2023:16:39:10] ENGINE Bus STARTED 2023-11-16T16:39:11.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:11 smithi053 ceph-mon[100638]: from='client.15498 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:39:11.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:11 smithi053 ceph-mon[100638]: from='client.15504 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:39:11.883 DEBUG:teuthology.orchestra.run.smithi001:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 5e4a743ad7fd1422ebd6c4dc950a21a303279207 2023-11-16T16:39:11.944 INFO:tasks.workunit.client.0.smithi001.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-11-16T16:39:12.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:12 smithi001 ceph-mon[98800]: pgmap v4: 97 pgs: 97 active+clean; 1.0 MiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:39:12.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:12 smithi001 ceph-mon[98800]: from='client.15510 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:39:12.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:12 smithi026 ceph-mon[100112]: pgmap v4: 97 pgs: 97 active+clean; 1.0 MiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:39:12.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:12 smithi026 ceph-mon[100112]: from='client.15510 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:39:12.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:12 smithi053 ceph-mon[100638]: pgmap v4: 97 pgs: 97 active+clean; 1.0 MiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:39:12.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:12 smithi053 ceph-mon[100638]: from='client.15510 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:39:13.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:13 smithi001 ceph-mon[98800]: mgrmap e23: x(active, since 3s), standbys: y, z 2023-11-16T16:39:13.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:13 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:13.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:13 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:13.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:13 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:13.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:13 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:13.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:13 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:13.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:13 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:13.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:13 smithi026 ceph-mon[100112]: mgrmap e23: x(active, since 3s), standbys: y, z 2023-11-16T16:39:13.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:13 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:13.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:13 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:13.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:13 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:13.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:13 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:13.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:13 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:13.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:13 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:13.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:13 smithi053 ceph-mon[100638]: mgrmap e23: x(active, since 3s), standbys: y, z 2023-11-16T16:39:13.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:13 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:13.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:13 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:13.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:13 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:13.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:13 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:13.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:13 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:13.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:13 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:15.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:14 smithi001 ceph-mon[98800]: pgmap v5: 97 pgs: 97 active+clean; 1.0 MiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:39:15.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:14 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:15.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:14 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:15.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:14 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:15.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:14 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:39:15.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:14 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:15.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:14 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:39:15.184 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:14 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:39:15.184 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:14 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:39:15.184 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:14 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:39:15.184 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:14 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:39:15.184 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:14 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:39:15.184 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:14 smithi001 ceph-mon[98800]: Adjusting osd_memory_target on smithi053 to 197.2M 2023-11-16T16:39:15.184 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:14 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:39:15.184 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:14 smithi001 ceph-mon[98800]: Unable to set osd_memory_target on smithi053 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-11-16T16:39:15.184 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:14 smithi001 ceph-mon[98800]: Adjusting osd_memory_target on smithi026 to 197.2M 2023-11-16T16:39:15.185 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:14 smithi001 ceph-mon[98800]: Unable to set osd_memory_target on smithi026 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-11-16T16:39:15.185 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:14 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:15.185 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:14 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:15.185 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:14 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:39:15.185 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:14 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:39:15.185 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:14 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:39:15.185 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:14 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:39:15.185 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:14 smithi001 ceph-mon[98800]: Adjusting osd_memory_target on smithi001 to 197.2M 2023-11-16T16:39:15.185 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:14 smithi001 ceph-mon[98800]: Unable to set osd_memory_target on smithi001 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-11-16T16:39:15.186 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:14 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:39:15.186 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:14 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:39:15.186 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:14 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:39:15.186 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:14 smithi001 ceph-mon[98800]: Updating smithi001:/etc/ceph/ceph.conf 2023-11-16T16:39:15.186 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:14 smithi001 ceph-mon[98800]: Updating smithi026:/etc/ceph/ceph.conf 2023-11-16T16:39:15.186 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:14 smithi001 ceph-mon[98800]: Updating smithi053:/etc/ceph/ceph.conf 2023-11-16T16:39:15.186 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:14 smithi001 ceph-mon[98800]: Updating smithi001:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:39:15.186 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:14 smithi001 ceph-mon[98800]: Updating smithi026:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:39:15.186 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:14 smithi001 ceph-mon[98800]: Updating smithi053:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:39:15.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:14 smithi026 ceph-mon[100112]: pgmap v5: 97 pgs: 97 active+clean; 1.0 MiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:39:15.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:14 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:15.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:14 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:15.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:14 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:15.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:14 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:39:15.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:14 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:15.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:14 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:39:15.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:14 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:39:15.259 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:14 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:39:15.260 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:14 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:39:15.260 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:14 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:39:15.260 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:14 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:39:15.260 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:14 smithi026 ceph-mon[100112]: Adjusting osd_memory_target on smithi053 to 197.2M 2023-11-16T16:39:15.260 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:14 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:39:15.260 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:14 smithi026 ceph-mon[100112]: Unable to set osd_memory_target on smithi053 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-11-16T16:39:15.260 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:14 smithi026 ceph-mon[100112]: Adjusting osd_memory_target on smithi026 to 197.2M 2023-11-16T16:39:15.260 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:14 smithi026 ceph-mon[100112]: Unable to set osd_memory_target on smithi026 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-11-16T16:39:15.261 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:14 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:15.261 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:14 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:15.261 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:14 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:39:15.261 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:14 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:39:15.261 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:14 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:39:15.261 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:14 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:39:15.261 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:14 smithi026 ceph-mon[100112]: Adjusting osd_memory_target on smithi001 to 197.2M 2023-11-16T16:39:15.261 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:14 smithi026 ceph-mon[100112]: Unable to set osd_memory_target on smithi001 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-11-16T16:39:15.261 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:14 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:39:15.262 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:14 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:39:15.262 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:14 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:39:15.262 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:14 smithi026 ceph-mon[100112]: Updating smithi001:/etc/ceph/ceph.conf 2023-11-16T16:39:15.262 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:14 smithi026 ceph-mon[100112]: Updating smithi026:/etc/ceph/ceph.conf 2023-11-16T16:39:15.262 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:14 smithi026 ceph-mon[100112]: Updating smithi053:/etc/ceph/ceph.conf 2023-11-16T16:39:15.262 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:14 smithi026 ceph-mon[100112]: Updating smithi001:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:39:15.262 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:14 smithi026 ceph-mon[100112]: Updating smithi026:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:39:15.262 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:14 smithi026 ceph-mon[100112]: Updating smithi053:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:39:15.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:14 smithi053 ceph-mon[100638]: pgmap v5: 97 pgs: 97 active+clean; 1.0 MiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:39:15.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:14 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:15.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:14 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:15.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:14 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:15.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:14 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:39:15.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:14 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:15.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:14 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:39:15.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:14 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:39:15.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:14 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:39:15.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:14 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:39:15.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:14 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:39:15.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:14 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:39:15.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:14 smithi053 ceph-mon[100638]: Adjusting osd_memory_target on smithi053 to 197.2M 2023-11-16T16:39:15.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:14 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:39:15.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:14 smithi053 ceph-mon[100638]: Unable to set osd_memory_target on smithi053 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-11-16T16:39:15.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:14 smithi053 ceph-mon[100638]: Adjusting osd_memory_target on smithi026 to 197.2M 2023-11-16T16:39:15.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:14 smithi053 ceph-mon[100638]: Unable to set osd_memory_target on smithi026 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-11-16T16:39:15.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:14 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:15.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:14 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:15.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:14 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:39:15.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:14 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:39:15.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:14 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:39:15.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:14 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:39:15.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:14 smithi053 ceph-mon[100638]: Adjusting osd_memory_target on smithi001 to 197.2M 2023-11-16T16:39:15.334 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:14 smithi053 ceph-mon[100638]: Unable to set osd_memory_target on smithi001 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-11-16T16:39:15.334 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:14 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:39:15.334 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:14 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:39:15.334 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:14 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:39:15.334 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:14 smithi053 ceph-mon[100638]: Updating smithi001:/etc/ceph/ceph.conf 2023-11-16T16:39:15.334 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:14 smithi053 ceph-mon[100638]: Updating smithi026:/etc/ceph/ceph.conf 2023-11-16T16:39:15.334 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:14 smithi053 ceph-mon[100638]: Updating smithi053:/etc/ceph/ceph.conf 2023-11-16T16:39:15.334 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:14 smithi053 ceph-mon[100638]: Updating smithi001:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:39:15.335 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:14 smithi053 ceph-mon[100638]: Updating smithi026:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:39:15.335 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:14 smithi053 ceph-mon[100638]: Updating smithi053:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.conf 2023-11-16T16:39:16.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:16 smithi053 ceph-mon[100638]: Updating smithi001:/etc/ceph/ceph.client.admin.keyring 2023-11-16T16:39:16.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:16 smithi053 ceph-mon[100638]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2023-11-16T16:39:16.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:16 smithi053 ceph-mon[100638]: Updating smithi026:/etc/ceph/ceph.client.admin.keyring 2023-11-16T16:39:16.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:16 smithi053 ceph-mon[100638]: Updating smithi001:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-16T16:39:16.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:16 smithi053 ceph-mon[100638]: Updating smithi053:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-16T16:39:16.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:16 smithi053 ceph-mon[100638]: Updating smithi026:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-16T16:39:16.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:16 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:16.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:16 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:16.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:16 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:16.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:16 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:16.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:16 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:16.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:16 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:16.332 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:16 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:16.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:16 smithi053 ceph-mon[100638]: Service mds.all has no related daemon already placed. 2023-11-16T16:39:16.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:16 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:39:16.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:16 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:39:16.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:16 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:39:16.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:16 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:39:16.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:16 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:39:16.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:16 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:39:16.333 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:16 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:16.334 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:16 smithi053 ceph-mon[100638]: Service mds.all has no related daemon already placed. 2023-11-16T16:39:16.334 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:16 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:39:16.334 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:16 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:39:16.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:16 smithi001 ceph-mon[98800]: Updating smithi001:/etc/ceph/ceph.client.admin.keyring 2023-11-16T16:39:16.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:16 smithi001 ceph-mon[98800]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2023-11-16T16:39:16.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:16 smithi001 ceph-mon[98800]: Updating smithi026:/etc/ceph/ceph.client.admin.keyring 2023-11-16T16:39:16.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:16 smithi001 ceph-mon[98800]: Updating smithi001:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-16T16:39:16.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:16 smithi001 ceph-mon[98800]: Updating smithi053:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-16T16:39:16.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:16 smithi001 ceph-mon[98800]: Updating smithi026:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-16T16:39:16.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:16 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:16.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:16 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:16.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:16 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:16.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:16 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:16.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:16 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:16.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:16 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:16.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:16 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:16.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:16 smithi001 ceph-mon[98800]: Service mds.all has no related daemon already placed. 2023-11-16T16:39:16.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:16 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:39:16.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:16 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:39:16.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:16 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:39:16.434 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:16 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:39:16.435 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:16 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:39:16.435 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:16 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:39:16.435 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:16 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:16.435 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:16 smithi001 ceph-mon[98800]: Service mds.all has no related daemon already placed. 2023-11-16T16:39:16.435 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:16 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:39:16.435 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:16 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:39:16.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:16 smithi026 ceph-mon[100112]: Updating smithi001:/etc/ceph/ceph.client.admin.keyring 2023-11-16T16:39:16.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:16 smithi026 ceph-mon[100112]: Updating smithi053:/etc/ceph/ceph.client.admin.keyring 2023-11-16T16:39:16.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:16 smithi026 ceph-mon[100112]: Updating smithi026:/etc/ceph/ceph.client.admin.keyring 2023-11-16T16:39:16.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:16 smithi026 ceph-mon[100112]: Updating smithi001:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-16T16:39:16.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:16 smithi026 ceph-mon[100112]: Updating smithi053:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-16T16:39:16.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:16 smithi026 ceph-mon[100112]: Updating smithi026:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/config/ceph.client.admin.keyring 2023-11-16T16:39:16.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:16 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:16.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:16 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:16.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:16 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:16.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:16 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:16.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:16 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:16.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:16 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:16.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:16 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:16.510 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:16 smithi026 ceph-mon[100112]: Service mds.all has no related daemon already placed. 2023-11-16T16:39:16.510 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:16 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:39:16.510 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:16 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:39:16.510 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:16 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:39:16.510 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:16 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:39:16.510 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:16 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:39:16.510 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:16 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:39:16.510 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:16 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:39:16.510 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:16 smithi026 ceph-mon[100112]: Service mds.all has no related daemon already placed. 2023-11-16T16:39:16.510 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:16 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:39:16.510 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:16 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:39:17.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:17 smithi053 ceph-mon[100638]: pgmap v6: 97 pgs: 97 active+clean; 1.3 MiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 93 KiB/s wr, 10 op/s 2023-11-16T16:39:17.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:17 smithi001 ceph-mon[98800]: pgmap v6: 97 pgs: 97 active+clean; 1.3 MiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 93 KiB/s wr, 10 op/s 2023-11-16T16:39:17.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:17 smithi026 ceph-mon[100112]: pgmap v6: 97 pgs: 97 active+clean; 1.3 MiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 93 KiB/s wr, 10 op/s 2023-11-16T16:39:19.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:19 smithi053 ceph-mon[100638]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 132 KiB/s wr, 51 op/s 2023-11-16T16:39:19.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:19 smithi001 ceph-mon[98800]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 132 KiB/s wr, 51 op/s 2023-11-16T16:39:19.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:19 smithi026 ceph-mon[100112]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 132 KiB/s wr, 51 op/s 2023-11-16T16:39:21.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:21 smithi053 ceph-mon[100638]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 103 KiB/s wr, 39 op/s 2023-11-16T16:39:21.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:21 smithi001 ceph-mon[98800]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 103 KiB/s wr, 39 op/s 2023-11-16T16:39:21.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:21 smithi026 ceph-mon[100112]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 103 KiB/s wr, 39 op/s 2023-11-16T16:39:23.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:23 smithi053 ceph-mon[100638]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 84 KiB/s wr, 35 op/s 2023-11-16T16:39:23.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:23 smithi001 ceph-mon[98800]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 84 KiB/s wr, 35 op/s 2023-11-16T16:39:23.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:23 smithi026 ceph-mon[100112]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 84 KiB/s wr, 35 op/s 2023-11-16T16:39:25.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:25 smithi053 ceph-mon[100638]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 77 KiB/s wr, 32 op/s 2023-11-16T16:39:25.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:25 smithi001 ceph-mon[98800]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 77 KiB/s wr, 32 op/s 2023-11-16T16:39:25.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:25 smithi026 ceph-mon[100112]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 77 KiB/s wr, 32 op/s 2023-11-16T16:39:27.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:27 smithi053 ceph-mon[100638]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 77 KiB/s wr, 33 op/s 2023-11-16T16:39:27.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:27 smithi001 ceph-mon[98800]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 77 KiB/s wr, 33 op/s 2023-11-16T16:39:27.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:27 smithi026 ceph-mon[100112]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 77 KiB/s wr, 33 op/s 2023-11-16T16:39:29.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:29 smithi053 ceph-mon[100638]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 38 KiB/s wr, 30 op/s 2023-11-16T16:39:29.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:29 smithi001 ceph-mon[98800]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 38 KiB/s wr, 30 op/s 2023-11-16T16:39:29.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:29 smithi026 ceph-mon[100112]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 38 KiB/s wr, 30 op/s 2023-11-16T16:39:30.501 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-16T16:39:30.659 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.655+0000 7fda3878a700 1 -- 172.21.15.1:0/3925114197 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda3010a720 msgr2=0x7fda3010cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:30.659 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.655+0000 7fda3878a700 1 --2- 172.21.15.1:0/3925114197 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda3010a720 0x7fda3010cb80 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fda2800b0a0 tx=0x7fda2800b3b0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.659 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.655+0000 7fda3878a700 1 -- 172.21.15.1:0/3925114197 shutdown_connections 2023-11-16T16:39:30.659 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.655+0000 7fda3878a700 1 --2- 172.21.15.1:0/3925114197 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda3010a720 0x7fda3010cb80 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.659 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.655+0000 7fda3878a700 1 --2- 172.21.15.1:0/3925114197 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda301013e0 0x7fda301017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.660 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.655+0000 7fda3878a700 1 --2- 172.21.15.1:0/3925114197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda30101cf0 0x7fda3010a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.660 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.655+0000 7fda3878a700 1 -- 172.21.15.1:0/3925114197 >> 172.21.15.1:0/3925114197 conn(0x7fda300763a0 msgr2=0x7fda300767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:30.660 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.655+0000 7fda3878a700 1 -- 172.21.15.1:0/3925114197 shutdown_connections 2023-11-16T16:39:30.660 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.655+0000 7fda3878a700 1 -- 172.21.15.1:0/3925114197 wait complete. 2023-11-16T16:39:30.660 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.656+0000 7fda3878a700 1 Processor -- start 2023-11-16T16:39:30.660 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.656+0000 7fda3878a700 1 -- start start 2023-11-16T16:39:30.660 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.656+0000 7fda3878a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda301013e0 0x7fda30196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:30.660 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.656+0000 7fda3878a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda30101cf0 0x7fda30196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:30.660 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.656+0000 7fda3878a700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda3010a720 0x7fda3019adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:30.660 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.656+0000 7fda3878a700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3019b400 con 0x7fda3010a720 2023-11-16T16:39:30.660 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.656+0000 7fda3878a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3019b570 con 0x7fda301013e0 2023-11-16T16:39:30.660 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.656+0000 7fda3878a700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3019b730 con 0x7fda30101cf0 2023-11-16T16:39:30.660 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.656+0000 7fda36d27700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda3010a720 0x7fda3019adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:30.661 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.656+0000 7fda36d27700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda3010a720 0x7fda3019adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43722/0 (socket says 172.21.15.1:43722) 2023-11-16T16:39:30.661 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.656+0000 7fda36d27700 1 -- 172.21.15.1:0/353960241 learned_addr learned my addr 172.21.15.1:0/353960241 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:30.661 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.656+0000 7fda36526700 1 --2- 172.21.15.1:0/353960241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda301013e0 0x7fda30196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:30.661 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.656+0000 7fda35d25700 1 --2- 172.21.15.1:0/353960241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda30101cf0 0x7fda30196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:30.661 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.657+0000 7fda36d27700 1 -- 172.21.15.1:0/353960241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda30101cf0 msgr2=0x7fda30196960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:30.661 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.657+0000 7fda36d27700 1 --2- 172.21.15.1:0/353960241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda30101cf0 0x7fda30196960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.661 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.657+0000 7fda36d27700 1 -- 172.21.15.1:0/353960241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda301013e0 msgr2=0x7fda30196420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:30.661 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.657+0000 7fda36d27700 1 --2- 172.21.15.1:0/353960241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda301013e0 0x7fda30196420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.661 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.657+0000 7fda36d27700 1 -- 172.21.15.1:0/353960241 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda28009d00 con 0x7fda3010a720 2023-11-16T16:39:30.661 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.657+0000 7fda36d27700 1 --2- 172.21.15.1:0/353960241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda3010a720 0x7fda3019adc0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fda2800b780 tx=0x7fda28007990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:30.661 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.657+0000 7fda277fe700 1 -- 172.21.15.1:0/353960241 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2800e040 con 0x7fda3010a720 2023-11-16T16:39:30.661 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.657+0000 7fda3878a700 1 -- 172.21.15.1:0/353960241 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda301101b0 con 0x7fda3010a720 2023-11-16T16:39:30.662 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.657+0000 7fda277fe700 1 -- 172.21.15.1:0/353960241 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda2801d470 con 0x7fda3010a720 2023-11-16T16:39:30.662 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.657+0000 7fda277fe700 1 -- 172.21.15.1:0/353960241 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda280052a0 con 0x7fda3010a720 2023-11-16T16:39:30.662 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.657+0000 7fda3878a700 1 -- 172.21.15.1:0/353960241 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda30110020 con 0x7fda3010a720 2023-11-16T16:39:30.665 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.659+0000 7fda277fe700 1 -- 172.21.15.1:0/353960241 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 23) v1 ==== 135890+0+0 (secure 0 0 0) 0x7fda2801d5e0 con 0x7fda3010a720 2023-11-16T16:39:30.665 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.660+0000 7fda277fe700 1 --2- 172.21.15.1:0/353960241 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fda1c0a1350 0x7fda1c0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:30.665 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.660+0000 7fda277fe700 1 -- 172.21.15.1:0/353960241 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fda280ce2b0 con 0x7fda3010a720 2023-11-16T16:39:30.665 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.660+0000 7fda36526700 1 --2- 172.21.15.1:0/353960241 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fda1c0a1350 0x7fda1c0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:30.665 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.660+0000 7fda36526700 1 --2- 172.21.15.1:0/353960241 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fda1c0a1350 0x7fda1c0a3800 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fda2c009c60 tx=0x7fda2c009260 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:30.665 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.661+0000 7fda44916b80 1 Processor -- start 2023-11-16T16:39:30.665 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.661+0000 7fda44916b80 1 -- start start 2023-11-16T16:39:30.665 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.661+0000 7fda44916b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55698d545f70 0x55698d546340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:30.665 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.661+0000 7fda44916b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55698d546880 0x55698d54a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:30.666 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.661+0000 7fda44916b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55698d54acc0 0x55698d54b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:30.666 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.661+0000 7fda44916b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55698d447060 con 0x55698d545f70 2023-11-16T16:39:30.666 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.661+0000 7fda44916b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55698d4998a0 con 0x55698d546880 2023-11-16T16:39:30.666 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.661+0000 7fda44916b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55698d492d30 con 0x55698d54acc0 2023-11-16T16:39:30.666 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.661+0000 7fda36526700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55698d545f70 0x55698d546340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:30.666 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.661+0000 7fda36526700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55698d545f70 0x55698d546340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43726/0 (socket says 172.21.15.1:43726) 2023-11-16T16:39:30.666 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.661+0000 7fda36526700 1 -- 172.21.15.1:0/3659287169 learned_addr learned my addr 172.21.15.1:0/3659287169 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:30.666 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.661+0000 7fda35d25700 1 --2- 172.21.15.1:0/3659287169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55698d546880 0x55698d54a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:30.666 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.661+0000 7fda36d27700 1 --2- 172.21.15.1:0/3659287169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55698d54acc0 0x55698d54b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:30.666 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.662+0000 7fda36526700 1 -- 172.21.15.1:0/3659287169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55698d54acc0 msgr2=0x55698d54b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:30.666 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.662+0000 7fda36526700 1 --2- 172.21.15.1:0/3659287169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55698d54acc0 0x55698d54b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.666 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.662+0000 7fda36526700 1 -- 172.21.15.1:0/3659287169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55698d546880 msgr2=0x55698d54a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:30.667 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.662+0000 7fda36526700 1 --2- 172.21.15.1:0/3659287169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55698d546880 0x55698d54a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.667 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.662+0000 7fda36526700 1 -- 172.21.15.1:0/3659287169 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55698d314630 con 0x55698d545f70 2023-11-16T16:39:30.667 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.662+0000 7fda36526700 1 --2- 172.21.15.1:0/3659287169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55698d545f70 0x55698d546340 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00e4e0 tx=0x7fda2c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ee3e864adce548a server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:30.667 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.662+0000 7fda24ff9700 1 -- 172.21.15.1:0/3659287169 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c00ee00 con 0x55698d545f70 2023-11-16T16:39:30.667 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.662+0000 7fda24ff9700 1 -- 172.21.15.1:0/3659287169 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda2c021070 con 0x55698d545f70 2023-11-16T16:39:30.667 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.662+0000 7fda24ff9700 1 -- 172.21.15.1:0/3659287169 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c01d7b0 con 0x55698d545f70 2023-11-16T16:39:30.667 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.663+0000 7fda44916b80 1 -- 172.21.15.1:0/3659287169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55698d545f70 msgr2=0x55698d546340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:30.667 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.663+0000 7fda44916b80 1 --2- 172.21.15.1:0/3659287169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55698d545f70 0x55698d546340 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00e4e0 tx=0x7fda2c014000 comp rx=0 tx=0).stop 2023-11-16T16:39:30.667 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.663+0000 7fda44916b80 1 -- 172.21.15.1:0/3659287169 shutdown_connections 2023-11-16T16:39:30.667 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.663+0000 7fda44916b80 1 --2- 172.21.15.1:0/3659287169 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55698d54acc0 0x55698d54b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.667 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.663+0000 7fda44916b80 1 --2- 172.21.15.1:0/3659287169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55698d545f70 0x55698d546340 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.667 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.663+0000 7fda44916b80 1 --2- 172.21.15.1:0/3659287169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55698d546880 0x55698d54a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.667 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.663+0000 7fda44916b80 1 -- 172.21.15.1:0/3659287169 >> 172.21.15.1:0/3659287169 conn(0x55698d537360 msgr2=0x55698d53cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:30.668 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.663+0000 7fda44916b80 1 -- 172.21.15.1:0/3659287169 shutdown_connections 2023-11-16T16:39:30.668 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.663+0000 7fda44916b80 1 -- 172.21.15.1:0/3659287169 wait complete. 2023-11-16T16:39:30.668 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.663+0000 7fda44916b80 1 -- start start 2023-11-16T16:39:30.668 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.663+0000 7fda44916b80 1 Processor -- start 2023-11-16T16:39:30.671 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.663+0000 7fda44916b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55698d545f70 0x55698d552450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:30.671 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.664+0000 7fda44916b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55698d546880 0x55698d54f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:30.672 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.664+0000 7fda44916b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55698d54acc0 0x55698d54fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:30.672 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.664+0000 7fda44916b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55698d4348e0 con 0x55698d546880 2023-11-16T16:39:30.672 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.664+0000 7fda44916b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55698d434580 con 0x55698d545f70 2023-11-16T16:39:30.672 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.664+0000 7fda44916b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55698d316920 con 0x55698d54acc0 2023-11-16T16:39:30.672 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.664+0000 7fda44916b80 10 client.0 _ll_register_callbacks cb 0x55698d452780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:39:30.672 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.664+0000 7fda35d25700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55698d546880 0x55698d54f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:30.672 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.664+0000 7fda35d25700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55698d546880 0x55698d54f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43734/0 (socket says 172.21.15.1:43734) 2023-11-16T16:39:30.672 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.664+0000 7fda35d25700 1 -- 172.21.15.1:0/1706945946 learned_addr learned my addr 172.21.15.1:0/1706945946 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:30.672 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.664+0000 7fda36526700 1 --2- 172.21.15.1:0/1706945946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55698d545f70 0x55698d552450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:30.672 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.664+0000 7fda36d27700 1 --2- 172.21.15.1:0/1706945946 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55698d54acc0 0x55698d54fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:30.672 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.664+0000 7fda35d25700 1 -- 172.21.15.1:0/1706945946 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55698d54acc0 msgr2=0x55698d54fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:30.672 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.664+0000 7fda35d25700 1 --2- 172.21.15.1:0/1706945946 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55698d54acc0 0x55698d54fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.672 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.664+0000 7fda35d25700 1 -- 172.21.15.1:0/1706945946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55698d545f70 msgr2=0x55698d552450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:30.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.664+0000 7fda35d25700 1 --2- 172.21.15.1:0/1706945946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55698d545f70 0x55698d552450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.664+0000 7fda35d25700 1 -- 172.21.15.1:0/1706945946 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda2c017070 con 0x55698d546880 2023-11-16T16:39:30.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.664+0000 7fda35d25700 1 --2- 172.21.15.1:0/1706945946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55698d546880 0x55698d54f740 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fda20012770 tx=0x7fda20012a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=635be0843c742051 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:30.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.664+0000 7fda19ffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:39:30.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.664+0000 7fda19ffb700 1 -- 172.21.15.1:0/1706945946 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda20017070 con 0x55698d546880 2023-11-16T16:39:30.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.664+0000 7fda44916b80 1 -- 172.21.15.1:0/1706945946 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55698d534b20 con 0x55698d546880 2023-11-16T16:39:30.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.664+0000 7fda19ffb700 1 -- 172.21.15.1:0/1706945946 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fda2000f480 con 0x55698d546880 2023-11-16T16:39:30.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.665+0000 7fda19ffb700 1 -- 172.21.15.1:0/1706945946 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda200122e0 con 0x55698d546880 2023-11-16T16:39:30.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.665+0000 7fda44916b80 10 client.15522 fetch_fsmap learned FSMap version 23 2023-11-16T16:39:30.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.665+0000 7fda44916b80 1 -- 172.21.15.1:0/1706945946 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55698d459190 con 0x55698d546880 2023-11-16T16:39:30.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.665+0000 7fda19ffb700 1 -- 172.21.15.1:0/1706945946 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda20028080 con 0x55698d546880 2023-11-16T16:39:30.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.665+0000 7fda19ffb700 1 -- 172.21.15.1:0/1706945946 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fda20010830 con 0x55698d546880 2023-11-16T16:39:30.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.665+0000 7fda44916b80 10 client.15522 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:39:30.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.665+0000 7fda44916b80 10 client.15522 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:39:30.674 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.665+0000 7fda44916b80 20 client.15522 populate_metadata read hostname 'smithi001' 2023-11-16T16:39:30.674 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.665+0000 7fda44916b80 1 --2- 172.21.15.1:0/1706945946 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55698d55fce0 0x55698d5620f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:30.674 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.665+0000 7fda44916b80 4 client.15522 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:39:30.674 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.665+0000 7fda44916b80 1 -- 172.21.15.1:0/1706945946 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55698d4550a0 con 0x55698d55fce0 2023-11-16T16:39:30.674 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.665+0000 7fda36526700 1 --2- 172.21.15.1:0/1706945946 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55698d55fce0 0x55698d5620f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:30.674 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.666+0000 7fda36526700 1 --2- 172.21.15.1:0/1706945946 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55698d55fce0 0x55698d5620f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44a71184d6432ad1 server_cookie=426676ad205a1fb7 in_seq=0 out_seq=0 2023-11-16T16:39:30.674 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.666+0000 7fda19ffb700 10 client.15522 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:39:30.674 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.666+0000 7fda19ffb700 1 -- 172.21.15.1:0/1706945946 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55698d4550a0 con 0x55698d55fce0 2023-11-16T16:39:30.674 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.666+0000 7fda19ffb700 10 client.15522 handle_command_reply: tid=0 2023-11-16T16:39:30.674 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.666+0000 7fda44916b80 1 client.15522 shutdown 2023-11-16T16:39:30.674 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.667+0000 7fda44916b80 1 -- 172.21.15.1:0/1706945946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55698d546880 msgr2=0x55698d54f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:30.674 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.667+0000 7fda44916b80 1 --2- 172.21.15.1:0/1706945946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55698d546880 0x55698d54f740 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fda20012770 tx=0x7fda20012a80 comp rx=0 tx=0).stop 2023-11-16T16:39:30.674 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.667+0000 7fda44916b80 1 -- 172.21.15.1:0/1706945946 shutdown_connections 2023-11-16T16:39:30.675 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.667+0000 7fda44916b80 1 --2- 172.21.15.1:0/1706945946 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55698d55fce0 0x55698d5620f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.675 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.667+0000 7fda44916b80 1 --2- 172.21.15.1:0/1706945946 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55698d54acc0 0x55698d54fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.675 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.667+0000 7fda19ffb700 0 client.15522 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:39:30.675 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.667+0000 7fda44916b80 1 --2- 172.21.15.1:0/1706945946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55698d545f70 0x55698d552450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.675 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.667+0000 7fda44916b80 1 --2- 172.21.15.1:0/1706945946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55698d546880 0x55698d54f740 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.675 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.667+0000 7fda44916b80 1 -- 172.21.15.1:0/1706945946 >> 172.21.15.1:0/1706945946 conn(0x55698d537360 msgr2=0x55698d53bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:30.675 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.667+0000 7fda44916b80 1 -- 172.21.15.1:0/1706945946 shutdown_connections 2023-11-16T16:39:30.675 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.667+0000 7fda44916b80 1 -- 172.21.15.1:0/1706945946 wait complete. 2023-11-16T16:39:30.675 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.667+0000 7fda44916b80 20 client.15522 trim_cache size 0 max 16384 2023-11-16T16:39:30.693 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.684+0000 7fda44916b80 1 Processor -- start 2023-11-16T16:39:30.711 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.684+0000 7fda44916b80 1 -- start start 2023-11-16T16:39:30.711 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.684+0000 7fda44916b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55698d545f70 0x55698d5495b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:30.712 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.684+0000 7fda44916b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55698d546880 0x55698d55c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:30.712 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.684+0000 7fda44916b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55698d54acc0 0x55698d551ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:30.712 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.684+0000 7fda44916b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55698d550cb0 con 0x55698d546880 2023-11-16T16:39:30.712 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.684+0000 7fda44916b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55698d53d950 con 0x55698d545f70 2023-11-16T16:39:30.712 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.684+0000 7fda44916b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55698d54dcc0 con 0x55698d54acc0 2023-11-16T16:39:30.712 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.684+0000 7fda35d25700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55698d546880 0x55698d55c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:30.712 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.684+0000 7fda35d25700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55698d546880 0x55698d55c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43742/0 (socket says 172.21.15.1:43742) 2023-11-16T16:39:30.713 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.684+0000 7fda35d25700 1 -- 172.21.15.1:0/2593298113 learned_addr learned my addr 172.21.15.1:0/2593298113 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:30.713 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.684+0000 7fda36526700 1 --2- 172.21.15.1:0/2593298113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55698d545f70 0x55698d5495b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:30.713 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.684+0000 7fda36d27700 1 --2- 172.21.15.1:0/2593298113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55698d54acc0 0x55698d551ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:30.713 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.684+0000 7fda35d25700 1 -- 172.21.15.1:0/2593298113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55698d54acc0 msgr2=0x55698d551ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:30.713 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.684+0000 7fda35d25700 1 --2- 172.21.15.1:0/2593298113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55698d54acc0 0x55698d551ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.713 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.684+0000 7fda35d25700 1 -- 172.21.15.1:0/2593298113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55698d545f70 msgr2=0x55698d5495b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:30.713 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.684+0000 7fda35d25700 1 --2- 172.21.15.1:0/2593298113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55698d545f70 0x55698d5495b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.684+0000 7fda35d25700 1 -- 172.21.15.1:0/2593298113 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55698d458ee0 con 0x55698d546880 2023-11-16T16:39:30.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.685+0000 7fda35d25700 1 --2- 172.21.15.1:0/2593298113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55698d546880 0x55698d55c050 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fda20010be0 tx=0x7fda20012e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad184ea7796d5a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:30.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.685+0000 7fda257fa700 1 -- 172.21.15.1:0/2593298113 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda20017070 con 0x55698d546880 2023-11-16T16:39:30.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.685+0000 7fda257fa700 1 -- 172.21.15.1:0/2593298113 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55698d534b20 con 0x55698d546880 2023-11-16T16:39:30.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.685+0000 7fda257fa700 1 -- 172.21.15.1:0/2593298113 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2000fac0 con 0x55698d546880 2023-11-16T16:39:30.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.685+0000 7fda44916b80 1 -- 172.21.15.1:0/2593298113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55698d546880 msgr2=0x55698d55c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:30.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.685+0000 7fda44916b80 1 --2- 172.21.15.1:0/2593298113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55698d546880 0x55698d55c050 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fda20010be0 tx=0x7fda20012e80 comp rx=0 tx=0).stop 2023-11-16T16:39:30.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.685+0000 7fda44916b80 1 -- 172.21.15.1:0/2593298113 shutdown_connections 2023-11-16T16:39:30.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.685+0000 7fda44916b80 1 --2- 172.21.15.1:0/2593298113 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55698d54acc0 0x55698d551ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.685+0000 7fda44916b80 1 --2- 172.21.15.1:0/2593298113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55698d545f70 0x55698d5495b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.685+0000 7fda44916b80 1 --2- 172.21.15.1:0/2593298113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55698d546880 0x55698d55c050 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.685+0000 7fda44916b80 1 -- 172.21.15.1:0/2593298113 >> 172.21.15.1:0/2593298113 conn(0x55698d537360 msgr2=0x55698d54b090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:30.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.685+0000 7fda44916b80 1 -- 172.21.15.1:0/2593298113 shutdown_connections 2023-11-16T16:39:30.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.685+0000 7fda44916b80 1 -- 172.21.15.1:0/2593298113 wait complete. 2023-11-16T16:39:30.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.686+0000 7fda44916b80 1 -- start start 2023-11-16T16:39:30.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.686+0000 7fda44916b80 1 Processor -- start 2023-11-16T16:39:30.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.686+0000 7fda44916b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55698d545f70 0x55698d552f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:30.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.686+0000 7fda44916b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55698d546880 0x55698d5534c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:30.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.686+0000 7fda44916b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55698d54acc0 0x55698d5acc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:30.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.686+0000 7fda44916b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55698d5ad180 con 0x55698d545f70 2023-11-16T16:39:30.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.686+0000 7fda44916b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55698d5ad2f0 con 0x55698d546880 2023-11-16T16:39:30.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.686+0000 7fda44916b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55698d5ad4b0 con 0x55698d54acc0 2023-11-16T16:39:30.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.686+0000 7fda44916b80 10 client.0 _ll_register_callbacks cb 0x7fda20009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:39:30.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.686+0000 7fda36526700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55698d545f70 0x55698d552f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:30.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.686+0000 7fda36526700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55698d545f70 0x55698d552f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43746/0 (socket says 172.21.15.1:43746) 2023-11-16T16:39:30.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.686+0000 7fda36526700 1 -- 172.21.15.1:0/4100573088 learned_addr learned my addr 172.21.15.1:0/4100573088 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:30.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.686+0000 7fda35d25700 1 --2- 172.21.15.1:0/4100573088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55698d546880 0x55698d5534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:30.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.686+0000 7fda36526700 1 -- 172.21.15.1:0/4100573088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55698d54acc0 msgr2=0x55698d5acc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:30.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.686+0000 7fda36526700 1 --2- 172.21.15.1:0/4100573088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55698d54acc0 0x55698d5acc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.686+0000 7fda36526700 1 -- 172.21.15.1:0/4100573088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55698d546880 msgr2=0x55698d5534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:30.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.686+0000 7fda36526700 1 --2- 172.21.15.1:0/4100573088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55698d546880 0x55698d5534c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.686+0000 7fda36526700 1 -- 172.21.15.1:0/4100573088 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55698d459190 con 0x55698d545f70 2023-11-16T16:39:30.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.686+0000 7fda35d25700 1 --2- 172.21.15.1:0/4100573088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55698d546880 0x55698d5534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:39:30.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.686+0000 7fda36526700 1 --2- 172.21.15.1:0/4100573088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55698d545f70 0x55698d552f80 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fda2c009ec0 tx=0x7fda2c00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=243f75fd73b97c42 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:30.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.686+0000 7fda19ffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:39:30.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.687+0000 7fda19ffb700 1 -- 172.21.15.1:0/4100573088 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c033ba0 con 0x55698d545f70 2023-11-16T16:39:30.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.687+0000 7fda19ffb700 1 -- 172.21.15.1:0/4100573088 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fda2c033d00 con 0x55698d545f70 2023-11-16T16:39:30.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.687+0000 7fda44916b80 1 -- 172.21.15.1:0/4100573088 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda2c021070 con 0x55698d545f70 2023-11-16T16:39:30.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.687+0000 7fda19ffb700 1 -- 172.21.15.1:0/4100573088 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda2c005240 con 0x55698d545f70 2023-11-16T16:39:30.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.687+0000 7fda44916b80 10 client.15528 fetch_fsmap learned FSMap version 23 2023-11-16T16:39:30.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.687+0000 7fda44916b80 1 -- 172.21.15.1:0/4100573088 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55698d2a9c60 con 0x55698d545f70 2023-11-16T16:39:30.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.687+0000 7fda19ffb700 1 -- 172.21.15.1:0/4100573088 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda2c021070 con 0x55698d545f70 2023-11-16T16:39:30.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.687+0000 7fda19ffb700 1 -- 172.21.15.1:0/4100573088 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fda2c003710 con 0x55698d545f70 2023-11-16T16:39:30.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.687+0000 7fda44916b80 10 client.15528 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:39:30.720 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.687+0000 7fda44916b80 10 client.15528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:39:30.720 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.687+0000 7fda44916b80 20 client.15528 populate_metadata read hostname 'smithi001' 2023-11-16T16:39:30.720 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.687+0000 7fda44916b80 1 --2- 172.21.15.1:0/4100573088 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55698d55fce0 0x55698d5b6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:30.720 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.687+0000 7fda44916b80 4 client.15528 mds_command: new command op to 24521 tid=0[{"prefix": "scrub status"}] 2023-11-16T16:39:30.720 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.687+0000 7fda44916b80 1 -- 172.21.15.1:0/4100573088 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55698d461660 con 0x55698d55fce0 2023-11-16T16:39:30.720 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.687+0000 7fda35d25700 1 --2- 172.21.15.1:0/4100573088 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55698d55fce0 0x55698d5b6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:30.720 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.688+0000 7fda19ffb700 10 client.15528 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:39:30.721 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.688+0000 7fda35d25700 1 --2- 172.21.15.1:0/4100573088 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55698d55fce0 0x55698d5b6b60 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c06e20bdfaa3c57 server_cookie=c9dd4f7c19a8cd65 in_seq=0 out_seq=0 2023-11-16T16:39:30.721 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.688+0000 7fda19ffb700 1 -- 172.21.15.1:0/4100573088 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55698d461660 con 0x55698d55fce0 2023-11-16T16:39:30.721 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.688+0000 7fda19ffb700 10 client.15528 handle_command_reply: tid=0 2023-11-16T16:39:30.721 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.688+0000 7fda44916b80 1 client.15528 shutdown 2023-11-16T16:39:30.721 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.688+0000 7fda44916b80 1 -- 172.21.15.1:0/4100573088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55698d545f70 msgr2=0x55698d552f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:30.721 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.688+0000 7fda44916b80 1 --2- 172.21.15.1:0/4100573088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55698d545f70 0x55698d552f80 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fda2c009ec0 tx=0x7fda2c00ed90 comp rx=0 tx=0).stop 2023-11-16T16:39:30.721 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.688+0000 7fda44916b80 1 -- 172.21.15.1:0/4100573088 shutdown_connections 2023-11-16T16:39:30.722 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.688+0000 7fda44916b80 1 --2- 172.21.15.1:0/4100573088 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55698d55fce0 0x55698d5b6b60 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.722 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.688+0000 7fda44916b80 1 --2- 172.21.15.1:0/4100573088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55698d54acc0 0x55698d5acc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.722 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.688+0000 7fda44916b80 1 --2- 172.21.15.1:0/4100573088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55698d545f70 0x55698d552f80 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.722 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.688+0000 7fda19ffb700 0 client.15528 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:39:30.722 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.688+0000 7fda44916b80 1 --2- 172.21.15.1:0/4100573088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55698d546880 0x55698d5534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.722 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.688+0000 7fda44916b80 1 -- 172.21.15.1:0/4100573088 >> 172.21.15.1:0/4100573088 conn(0x55698d537360 msgr2=0x55698d549e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:30.722 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.688+0000 7fda44916b80 1 -- 172.21.15.1:0/4100573088 shutdown_connections 2023-11-16T16:39:30.722 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.688+0000 7fda44916b80 1 -- 172.21.15.1:0/4100573088 wait complete. 2023-11-16T16:39:30.723 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.688+0000 7fda44916b80 20 client.15528 trim_cache size 0 max 16384 2023-11-16T16:39:30.723 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:39:30.723 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2023-11-16T16:39:30.723 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2023-11-16T16:39:30.723 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:39:30.725 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.689+0000 7fda3878a700 1 -- 172.21.15.1:0/353960241 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fda1c0a1350 msgr2=0x7fda1c0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:30.725 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.689+0000 7fda3878a700 1 --2- 172.21.15.1:0/353960241 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fda1c0a1350 0x7fda1c0a3800 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fda2c009c60 tx=0x7fda2c009260 comp rx=0 tx=0).stop 2023-11-16T16:39:30.725 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.689+0000 7fda3878a700 1 -- 172.21.15.1:0/353960241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda3010a720 msgr2=0x7fda3019adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:30.725 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.689+0000 7fda3878a700 1 --2- 172.21.15.1:0/353960241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda3010a720 0x7fda3019adc0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fda2800b780 tx=0x7fda28007990 comp rx=0 tx=0).stop 2023-11-16T16:39:30.725 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.689+0000 7fda3878a700 1 -- 172.21.15.1:0/353960241 shutdown_connections 2023-11-16T16:39:30.725 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.689+0000 7fda3878a700 1 --2- 172.21.15.1:0/353960241 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fda1c0a1350 0x7fda1c0a3800 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.725 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.689+0000 7fda3878a700 1 --2- 172.21.15.1:0/353960241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda3010a720 0x7fda3019adc0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.725 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.689+0000 7fda3878a700 1 --2- 172.21.15.1:0/353960241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda301013e0 0x7fda30196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.726 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.689+0000 7fda3878a700 1 --2- 172.21.15.1:0/353960241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fda30101cf0 0x7fda30196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.726 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.689+0000 7fda3878a700 1 -- 172.21.15.1:0/353960241 >> 172.21.15.1:0/353960241 conn(0x7fda300763a0 msgr2=0x7fda30102180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:30.726 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.689+0000 7fda3878a700 1 -- 172.21.15.1:0/353960241 shutdown_connections 2023-11-16T16:39:30.726 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.689+0000 7fda3878a700 1 -- 172.21.15.1:0/353960241 wait complete. 2023-11-16T16:39:30.726 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-16T16:39:30.727 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:39:30.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.850+0000 7f60c5910700 1 -- 172.21.15.1:0/2704648792 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60c00fe850 msgr2=0x7f60c00fecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:30.853 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.850+0000 7f60c5910700 1 --2- 172.21.15.1:0/2704648792 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60c00fe850 0x7f60c00fecc0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f60b0009990 tx=0x7f60b0009ca0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.853 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.850+0000 7f60c5910700 1 -- 172.21.15.1:0/2704648792 shutdown_connections 2023-11-16T16:39:30.853 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.850+0000 7f60c5910700 1 --2- 172.21.15.1:0/2704648792 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60c00ff200 0x7f60c010d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.853 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.850+0000 7f60c5910700 1 --2- 172.21.15.1:0/2704648792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60c01063f0 0x7f60c01067c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.853 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.850+0000 7f60c5910700 1 --2- 172.21.15.1:0/2704648792 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60c00fe850 0x7f60c00fecc0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.854 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.850+0000 7f60c5910700 1 -- 172.21.15.1:0/2704648792 >> 172.21.15.1:0/2704648792 conn(0x7f60c00758c0 msgr2=0x7f60c0075cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:30.854 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.850+0000 7f60c5910700 1 -- 172.21.15.1:0/2704648792 shutdown_connections 2023-11-16T16:39:30.854 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.850+0000 7f60c5910700 1 -- 172.21.15.1:0/2704648792 wait complete. 2023-11-16T16:39:30.854 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.851+0000 7f60c5910700 1 Processor -- start 2023-11-16T16:39:30.856 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.851+0000 7f60c5910700 1 -- start start 2023-11-16T16:39:30.856 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.851+0000 7f60c5910700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60c00fe850 0x7f60c00ffa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:30.856 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.851+0000 7f60c5910700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60c00ff200 0x7f60c00fff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:30.856 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.851+0000 7f60c5910700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60c01063f0 0x7f60c01a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:30.857 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.851+0000 7f60c5910700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c01007a0 con 0x7f60c00fe850 2023-11-16T16:39:30.857 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.851+0000 7f60c5910700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c01a4bd0 con 0x7f60c00ff200 2023-11-16T16:39:30.857 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.851+0000 7f60c5910700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c01a4d10 con 0x7f60c01063f0 2023-11-16T16:39:30.857 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.851+0000 7f60beffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60c00fe850 0x7f60c00ffa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:30.857 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.852+0000 7f60beffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60c00fe850 0x7f60c00ffa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43758/0 (socket says 172.21.15.1:43758) 2023-11-16T16:39:30.857 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.852+0000 7f60beffd700 1 -- 172.21.15.1:0/3418598255 learned_addr learned my addr 172.21.15.1:0/3418598255 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:30.857 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.852+0000 7f60beffd700 1 -- 172.21.15.1:0/3418598255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60c01063f0 msgr2=0x7f60c01a4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:30.858 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.852+0000 7f60beffd700 1 --2- 172.21.15.1:0/3418598255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60c01063f0 0x7f60c01a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.858 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.852+0000 7f60beffd700 1 -- 172.21.15.1:0/3418598255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60c00ff200 msgr2=0x7f60c00fff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:30.858 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.852+0000 7f60be7fc700 1 --2- 172.21.15.1:0/3418598255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60c00ff200 0x7f60c00fff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:30.858 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.852+0000 7f60beffd700 1 --2- 172.21.15.1:0/3418598255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60c00ff200 0x7f60c00fff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:30.858 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.852+0000 7f60beffd700 1 -- 172.21.15.1:0/3418598255 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b00095f0 con 0x7f60c00fe850 2023-11-16T16:39:30.858 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.852+0000 7f60beffd700 1 --2- 172.21.15.1:0/3418598255 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60c00fe850 0x7f60c00ffa30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f60a800d950 tx=0x7f60a800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:30.858 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.852+0000 7f60c490e700 1 -- 172.21.15.1:0/3418598255 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a8009b40 con 0x7f60c00fe850 2023-11-16T16:39:30.859 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.852+0000 7f60c5910700 1 -- 172.21.15.1:0/3418598255 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c01103f0 con 0x7f60c00fe850 2023-11-16T16:39:30.859 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.852+0000 7f60c490e700 1 -- 172.21.15.1:0/3418598255 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60a8009ca0 con 0x7f60c00fe850 2023-11-16T16:39:30.859 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.852+0000 7f60c490e700 1 -- 172.21.15.1:0/3418598255 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a800f620 con 0x7f60c00fe850 2023-11-16T16:39:30.859 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.852+0000 7f60c5910700 1 -- 172.21.15.1:0/3418598255 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c0110260 con 0x7f60c00fe850 2023-11-16T16:39:30.859 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.853+0000 7f60c5910700 1 -- 172.21.15.1:0/3418598255 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60c0110580 con 0x7f60c00fe850 2023-11-16T16:39:30.860 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.855+0000 7f60c490e700 1 -- 172.21.15.1:0/3418598255 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 23) v1 ==== 135890+0+0 (secure 0 0 0) 0x7f60a8010460 con 0x7f60c00fe850 2023-11-16T16:39:30.860 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.855+0000 7f60c490e700 1 --2- 172.21.15.1:0/3418598255 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f60ac0a1350 0x7f60ac0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:30.860 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.855+0000 7f60c490e700 1 -- 172.21.15.1:0/3418598255 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f60a80cd0f0 con 0x7f60c00fe850 2023-11-16T16:39:30.861 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.855+0000 7f60be7fc700 1 --2- 172.21.15.1:0/3418598255 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f60ac0a1350 0x7f60ac0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:30.861 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.856+0000 7f60be7fc700 1 --2- 172.21.15.1:0/3418598255 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f60ac0a1350 0x7f60ac0a3800 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f60b0003730 tx=0x7f60b00036a0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:30.861 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:30.858+0000 7f60c490e700 1 -- 172.21.15.1:0/3418598255 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f60a8096440 con 0x7f60c00fe850 2023-11-16T16:39:31.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.104+0000 7f60c5910700 1 -- 172.21.15.1:0/3418598255 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60c0067260 con 0x7f60c00fe850 2023-11-16T16:39:31.106 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.104+0000 7f60c490e700 1 -- 172.21.15.1:0/3418598255 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10774 (secure 0 0 0) 0x7f60a8020090 con 0x7f60c00fe850 2023-11-16T16:39:31.106 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:39:31.107 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2587563684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2587563684},{"type":"v1","addr":"172.21.15.1:6835","nonce":2587563684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/277936168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":277936168},{"type":"v1","addr":"172.21.15.1:6837","nonce":277936168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/1172575045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1172575045},{"type":"v1","addr":"172.21.15.1:6839","nonce":1172575045}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/2631175899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2631175899},{"type":"v1","addr":"172.21.15.1:6841","nonce":2631175899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3693905267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3693905267},{"type":"v1","addr":"172.21.15.53:6833","nonce":3693905267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/164668033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":164668033},{"type":"v1","addr":"172.21.15.26:6833","nonce":164668033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-16T16:37:13.918943+0000","modified":"2023-11-16T16:37:19.292898+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2140542192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2140542192},{"type":"v1","addr":"172.21.15.53:6835","nonce":2140542192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1010760275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1010760275},{"type":"v1","addr":"172.21.15.26:6835","nonce":1010760275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3197060302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3197060302},{"type":"v1","addr":"172.21.15.53:6837","nonce":3197060302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6837/1453504863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1453504863},{"type":"v1","addr":"172.21.15.26:6837","nonce":1453504863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.53:6839/1044753024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1044753024},{"type":"v1","addr":"172.21.15.53:6839","nonce":1044753024}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6839/2192205949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2192205949},{"type":"v1","addr":"172.21.15.26:6839","nonce":2192205949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-16T16:39:31.108 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.106+0000 7f60c5910700 1 -- 172.21.15.1:0/3418598255 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f60ac0a1350 msgr2=0x7f60ac0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:31.109 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.106+0000 7f60c5910700 1 --2- 172.21.15.1:0/3418598255 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f60ac0a1350 0x7f60ac0a3800 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f60b0003730 tx=0x7f60b00036a0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.109 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.106+0000 7f60c5910700 1 -- 172.21.15.1:0/3418598255 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60c00fe850 msgr2=0x7f60c00ffa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:31.109 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.106+0000 7f60c5910700 1 --2- 172.21.15.1:0/3418598255 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60c00fe850 0x7f60c00ffa30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f60a800d950 tx=0x7f60a800dd10 comp rx=0 tx=0).stop 2023-11-16T16:39:31.109 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.106+0000 7f60c5910700 1 -- 172.21.15.1:0/3418598255 shutdown_connections 2023-11-16T16:39:31.109 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.106+0000 7f60c5910700 1 --2- 172.21.15.1:0/3418598255 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f60ac0a1350 0x7f60ac0a3800 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.109 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.106+0000 7f60c5910700 1 --2- 172.21.15.1:0/3418598255 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f60c01063f0 0x7f60c01a4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.110 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.106+0000 7f60c5910700 1 --2- 172.21.15.1:0/3418598255 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60c00fe850 0x7f60c00ffa30 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.110 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.106+0000 7f60c5910700 1 --2- 172.21.15.1:0/3418598255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60c00ff200 0x7f60c00fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.110 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.106+0000 7f60c5910700 1 -- 172.21.15.1:0/3418598255 >> 172.21.15.1:0/3418598255 conn(0x7f60c00758c0 msgr2=0x7f60c00fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:31.110 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.107+0000 7f60c5910700 1 -- 172.21.15.1:0/3418598255 shutdown_connections 2023-11-16T16:39:31.110 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.107+0000 7f60c5910700 1 -- 172.21.15.1:0/3418598255 wait complete. 2023-11-16T16:39:31.111 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 23 2023-11-16T16:39:31.120 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-11-16T16:39:31.249 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.247+0000 7fa518ab7700 1 -- 172.21.15.1:0/2872603207 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa514105250 msgr2=0x7fa514105620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:31.249 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.247+0000 7fa518ab7700 1 --2- 172.21.15.1:0/2872603207 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa514105250 0x7fa514105620 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa504009990 tx=0x7fa504009ca0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.249 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.247+0000 7fa518ab7700 1 -- 172.21.15.1:0/2872603207 shutdown_connections 2023-11-16T16:39:31.249 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.247+0000 7fa518ab7700 1 --2- 172.21.15.1:0/2872603207 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa514100420 0x7fa51410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.249 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.247+0000 7fa518ab7700 1 --2- 172.21.15.1:0/2872603207 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa514105250 0x7fa514105620 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.249 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.247+0000 7fa518ab7700 1 --2- 172.21.15.1:0/2872603207 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5140ffa70 0x7fa5140ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.250 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.247+0000 7fa518ab7700 1 -- 172.21.15.1:0/2872603207 >> 172.21.15.1:0/2872603207 conn(0x7fa5140763a0 msgr2=0x7fa5140767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:31.250 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.247+0000 7fa518ab7700 1 -- 172.21.15.1:0/2872603207 shutdown_connections 2023-11-16T16:39:31.250 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.247+0000 7fa518ab7700 1 -- 172.21.15.1:0/2872603207 wait complete. 2023-11-16T16:39:31.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.247+0000 7fa518ab7700 1 Processor -- start 2023-11-16T16:39:31.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.248+0000 7fa518ab7700 1 -- start start 2023-11-16T16:39:31.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.248+0000 7fa518ab7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5140ffa70 0x7fa514194ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.248+0000 7fa518ab7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa514100420 0x7fa514195410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.248+0000 7fa518ab7700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa514105250 0x7fa514195950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.248+0000 7fa518ab7700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5141a4710 con 0x7fa514105250 2023-11-16T16:39:31.253 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.248+0000 7fa518ab7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5141a4880 con 0x7fa5140ffa70 2023-11-16T16:39:31.253 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.248+0000 7fa518ab7700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5141a4a40 con 0x7fa514100420 2023-11-16T16:39:31.253 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.248+0000 7fa512d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa514105250 0x7fa514195950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.253 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.248+0000 7fa512d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa514105250 0x7fa514195950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43776/0 (socket says 172.21.15.1:43776) 2023-11-16T16:39:31.253 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.248+0000 7fa512d9d700 1 -- 172.21.15.1:0/2545296876 learned_addr learned my addr 172.21.15.1:0/2545296876 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:31.253 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.248+0000 7fa511d9b700 1 --2- 172.21.15.1:0/2545296876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa514100420 0x7fa514195410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.254 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.248+0000 7fa512d9d700 1 -- 172.21.15.1:0/2545296876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa514100420 msgr2=0x7fa514195410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:31.254 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.248+0000 7fa512d9d700 1 --2- 172.21.15.1:0/2545296876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa514100420 0x7fa514195410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.254 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.248+0000 7fa51259c700 1 --2- 172.21.15.1:0/2545296876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5140ffa70 0x7fa514194ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.254 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.248+0000 7fa512d9d700 1 -- 172.21.15.1:0/2545296876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5140ffa70 msgr2=0x7fa514194ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:31.254 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.248+0000 7fa512d9d700 1 --2- 172.21.15.1:0/2545296876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5140ffa70 0x7fa514194ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.248+0000 7fa512d9d700 1 -- 172.21.15.1:0/2545296876 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5040095f0 con 0x7fa514105250 2023-11-16T16:39:31.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.248+0000 7fa51259c700 1 --2- 172.21.15.1:0/2545296876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5140ffa70 0x7fa514194ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:39:31.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.248+0000 7fa512d9d700 1 --2- 172.21.15.1:0/2545296876 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa514105250 0x7fa514195950 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa50c007ee0 tx=0x7fa50c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:31.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.248+0000 7fa5037fe700 1 -- 172.21.15.1:0/2545296876 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50c00ddd0 con 0x7fa514105250 2023-11-16T16:39:31.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.249+0000 7fa5037fe700 1 -- 172.21.15.1:0/2545296876 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa50c00f040 con 0x7fa514105250 2023-11-16T16:39:31.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.249+0000 7fa5037fe700 1 -- 172.21.15.1:0/2545296876 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50c015400 con 0x7fa514105250 2023-11-16T16:39:31.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.249+0000 7fa518ab7700 1 -- 172.21.15.1:0/2545296876 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa51410f1b0 con 0x7fa514105250 2023-11-16T16:39:31.256 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.249+0000 7fa518ab7700 1 -- 172.21.15.1:0/2545296876 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa51410f020 con 0x7fa514105250 2023-11-16T16:39:31.261 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.251+0000 7fa5037fe700 1 -- 172.21.15.1:0/2545296876 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 23) v1 ==== 135890+0+0 (secure 0 0 0) 0x7fa50c004020 con 0x7fa514105250 2023-11-16T16:39:31.262 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.251+0000 7fa5037fe700 1 --2- 172.21.15.1:0/2545296876 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fa4fc0a1350 0x7fa4fc0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.262 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.251+0000 7fa5037fe700 1 -- 172.21.15.1:0/2545296876 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fa50c0cc810 con 0x7fa514105250 2023-11-16T16:39:31.262 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.251+0000 7fa51259c700 1 --2- 172.21.15.1:0/2545296876 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fa4fc0a1350 0x7fa4fc0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.262 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.252+0000 7fa51259c700 1 --2- 172.21.15.1:0/2545296876 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fa4fc0a1350 0x7fa4fc0a3800 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa50400b5c0 tx=0x7fa504005fb0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:31.262 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.252+0000 7fa524c43b80 1 Processor -- start 2023-11-16T16:39:31.262 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.252+0000 7fa524c43b80 1 -- start start 2023-11-16T16:39:31.263 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.253+0000 7fa524c43b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640afb45b60 0x5640afb45f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.263 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.253+0000 7fa524c43b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640afb46470 0x5640afb4a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.263 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.253+0000 7fa524c43b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640afb4a8b0 0x5640afb4ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.263 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.253+0000 7fa524c43b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640afa4b060 con 0x5640afb46470 2023-11-16T16:39:31.263 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.253+0000 7fa524c43b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640afa9d8a0 con 0x5640afb4a8b0 2023-11-16T16:39:31.263 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.253+0000 7fa524c43b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640afa96d30 con 0x5640afb45b60 2023-11-16T16:39:31.263 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.253+0000 7fa511d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640afb46470 0x5640afb4a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.264 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.253+0000 7fa511d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640afb46470 0x5640afb4a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43782/0 (socket says 172.21.15.1:43782) 2023-11-16T16:39:31.264 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.253+0000 7fa511d9b700 1 -- 172.21.15.1:0/2942673258 learned_addr learned my addr 172.21.15.1:0/2942673258 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:31.264 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.253+0000 7fa512d9d700 1 --2- 172.21.15.1:0/2942673258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640afb4a8b0 0x5640afb4ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.264 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.253+0000 7fa51259c700 1 --2- 172.21.15.1:0/2942673258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640afb45b60 0x5640afb45f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.264 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.253+0000 7fa511d9b700 1 -- 172.21.15.1:0/2942673258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640afb45b60 msgr2=0x5640afb45f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:31.264 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.253+0000 7fa511d9b700 1 --2- 172.21.15.1:0/2942673258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640afb45b60 0x5640afb45f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.264 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.253+0000 7fa511d9b700 1 -- 172.21.15.1:0/2942673258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640afb4a8b0 msgr2=0x5640afb4ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:31.264 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.253+0000 7fa511d9b700 1 --2- 172.21.15.1:0/2942673258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640afb4a8b0 0x5640afb4ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.265 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.253+0000 7fa511d9b700 1 -- 172.21.15.1:0/2942673258 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640af918630 con 0x5640afb46470 2023-11-16T16:39:31.265 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.253+0000 7fa51259c700 1 --2- 172.21.15.1:0/2942673258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640afb45b60 0x5640afb45f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:39:31.265 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.253+0000 7fa511d9b700 1 --2- 172.21.15.1:0/2942673258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640afb46470 0x5640afb4a370 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa508008c00 tx=0x7fa508008f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2053ec1c4a47ff0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:31.265 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.253+0000 7fa500ff9700 1 -- 172.21.15.1:0/2942673258 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50800ecf0 con 0x5640afb46470 2023-11-16T16:39:31.265 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.254+0000 7fa500ff9700 1 -- 172.21.15.1:0/2942673258 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa50800ee50 con 0x5640afb46470 2023-11-16T16:39:31.265 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.254+0000 7fa500ff9700 1 -- 172.21.15.1:0/2942673258 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa508010430 con 0x5640afb46470 2023-11-16T16:39:31.265 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.254+0000 7fa524c43b80 1 -- 172.21.15.1:0/2942673258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640afb46470 msgr2=0x5640afb4a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:31.266 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.254+0000 7fa524c43b80 1 --2- 172.21.15.1:0/2942673258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640afb46470 0x5640afb4a370 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa508008c00 tx=0x7fa508008f10 comp rx=0 tx=0).stop 2023-11-16T16:39:31.266 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.254+0000 7fa524c43b80 1 -- 172.21.15.1:0/2942673258 shutdown_connections 2023-11-16T16:39:31.266 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.254+0000 7fa524c43b80 1 --2- 172.21.15.1:0/2942673258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640afb4a8b0 0x5640afb4ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.266 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.254+0000 7fa524c43b80 1 --2- 172.21.15.1:0/2942673258 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640afb45b60 0x5640afb45f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.266 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.254+0000 7fa524c43b80 1 --2- 172.21.15.1:0/2942673258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640afb46470 0x5640afb4a370 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.266 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.254+0000 7fa524c43b80 1 -- 172.21.15.1:0/2942673258 >> 172.21.15.1:0/2942673258 conn(0x5640afb3b360 msgr2=0x5640afb40b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:31.266 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.254+0000 7fa524c43b80 1 -- 172.21.15.1:0/2942673258 shutdown_connections 2023-11-16T16:39:31.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.254+0000 7fa524c43b80 1 -- 172.21.15.1:0/2942673258 wait complete. 2023-11-16T16:39:31.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.255+0000 7fa524c43b80 1 -- start start 2023-11-16T16:39:31.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.255+0000 7fa524c43b80 1 Processor -- start 2023-11-16T16:39:31.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.255+0000 7fa524c43b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640afb45b60 0x5640afb51fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.255+0000 7fa524c43b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640afb46470 0x5640afb4f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.255+0000 7fa524c43b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640afb4a8b0 0x5640afb4f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.255+0000 7fa524c43b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640afa388e0 con 0x5640afb45b60 2023-11-16T16:39:31.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.255+0000 7fa524c43b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640afa38580 con 0x5640afb46470 2023-11-16T16:39:31.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.255+0000 7fa524c43b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640af91a920 con 0x5640afb4a8b0 2023-11-16T16:39:31.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.255+0000 7fa524c43b80 10 client.0 _ll_register_callbacks cb 0x5640afa56780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:39:31.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.255+0000 7fa51259c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640afb45b60 0x5640afb51fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.255+0000 7fa51259c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640afb45b60 0x5640afb51fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43786/0 (socket says 172.21.15.1:43786) 2023-11-16T16:39:31.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.255+0000 7fa51259c700 1 -- 172.21.15.1:0/3537600566 learned_addr learned my addr 172.21.15.1:0/3537600566 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:31.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.255+0000 7fa511d9b700 1 --2- 172.21.15.1:0/3537600566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640afb46470 0x5640afb4f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.255+0000 7fa512d9d700 1 --2- 172.21.15.1:0/3537600566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640afb4a8b0 0x5640afb4f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.256+0000 7fa51259c700 1 -- 172.21.15.1:0/3537600566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640afb4a8b0 msgr2=0x5640afb4f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:31.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.256+0000 7fa51259c700 1 --2- 172.21.15.1:0/3537600566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640afb4a8b0 0x5640afb4f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.256+0000 7fa51259c700 1 -- 172.21.15.1:0/3537600566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640afb46470 msgr2=0x5640afb4f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:31.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.256+0000 7fa51259c700 1 --2- 172.21.15.1:0/3537600566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640afb46470 0x5640afb4f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.256+0000 7fa51259c700 1 -- 172.21.15.1:0/3537600566 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa508008830 con 0x5640afb45b60 2023-11-16T16:39:31.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.256+0000 7fa511d9b700 1 --2- 172.21.15.1:0/3537600566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640afb46470 0x5640afb4f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:39:31.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.256+0000 7fa51259c700 1 --2- 172.21.15.1:0/3537600566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640afb45b60 0x5640afb51fa0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa504005b40 tx=0x7fa504009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17072df65498c607 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:31.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.256+0000 7fa4f9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:39:31.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.256+0000 7fa4f9ffb700 1 -- 172.21.15.1:0/3537600566 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50401d070 con 0x5640afb45b60 2023-11-16T16:39:31.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.256+0000 7fa4f9ffb700 1 -- 172.21.15.1:0/3537600566 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fa504005550 con 0x5640afb45b60 2023-11-16T16:39:31.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.256+0000 7fa524c43b80 1 -- 172.21.15.1:0/3537600566 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5640afb38b20 con 0x5640afb45b60 2023-11-16T16:39:31.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.256+0000 7fa4f9ffb700 1 -- 172.21.15.1:0/3537600566 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa504005710 con 0x5640afb45b60 2023-11-16T16:39:31.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.256+0000 7fa524c43b80 10 client.15546 fetch_fsmap learned FSMap version 23 2023-11-16T16:39:31.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.256+0000 7fa524c43b80 1 -- 172.21.15.1:0/3537600566 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5640afa5d190 con 0x5640afb45b60 2023-11-16T16:39:31.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.256+0000 7fa4f9ffb700 1 -- 172.21.15.1:0/3537600566 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa50403a020 con 0x5640afb45b60 2023-11-16T16:39:31.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.256+0000 7fa4f9ffb700 1 -- 172.21.15.1:0/3537600566 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa50402d570 con 0x5640afb45b60 2023-11-16T16:39:31.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.257+0000 7fa524c43b80 10 client.15546 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:39:31.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.257+0000 7fa524c43b80 10 client.15546 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-11-16T16:39:31.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.257+0000 7fa524c43b80 20 client.15546 populate_metadata read hostname 'smithi001' 2023-11-16T16:39:31.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.257+0000 7fa524c43b80 1 --2- 172.21.15.1:0/3537600566 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x5640afb5f810 0x5640afb61c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.257+0000 7fa524c43b80 4 client.15546 mds_command: new command op to 24482 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:39:31.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.257+0000 7fa524c43b80 1 -- 172.21.15.1:0/3537600566 --> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5640afa590a0 con 0x5640afb5f810 2023-11-16T16:39:31.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.257+0000 7fa511d9b700 1 --2- 172.21.15.1:0/3537600566 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x5640afb5f810 0x5640afb61c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.257+0000 7fa511d9b700 1 --2- 172.21.15.1:0/3537600566 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x5640afb5f810 0x5640afb61c20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e4b09da77bbe05a2 server_cookie=ba2007b73137eda7 in_seq=0 out_seq=0 2023-11-16T16:39:31.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.257+0000 7fa4f9ffb700 10 client.15546 ms_handle_connect on v2:172.21.15.26:6836/1453504863 2023-11-16T16:39:31.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.258+0000 7fa4f9ffb700 1 -- 172.21.15.1:0/3537600566 <== mds.2 v2:172.21.15.26:6836/1453504863 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5640afa590a0 con 0x5640afb5f810 2023-11-16T16:39:31.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.258+0000 7fa4f9ffb700 10 client.15546 handle_command_reply: tid=0 2023-11-16T16:39:31.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.258+0000 7fa524c43b80 1 client.15546 shutdown 2023-11-16T16:39:31.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.258+0000 7fa524c43b80 1 -- 172.21.15.1:0/3537600566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640afb45b60 msgr2=0x5640afb51fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:31.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.258+0000 7fa524c43b80 1 --2- 172.21.15.1:0/3537600566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640afb45b60 0x5640afb51fa0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa504005b40 tx=0x7fa504009ca0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.258+0000 7fa524c43b80 1 -- 172.21.15.1:0/3537600566 shutdown_connections 2023-11-16T16:39:31.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.258+0000 7fa524c43b80 1 --2- 172.21.15.1:0/3537600566 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x5640afb5f810 0x5640afb61c20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.258+0000 7fa524c43b80 1 --2- 172.21.15.1:0/3537600566 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640afb4a8b0 0x5640afb4f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.258+0000 7fa524c43b80 1 --2- 172.21.15.1:0/3537600566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640afb45b60 0x5640afb51fa0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.258+0000 7fa524c43b80 1 --2- 172.21.15.1:0/3537600566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640afb46470 0x5640afb4f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.258+0000 7fa4f9ffb700 0 client.15546 ms_handle_reset on v2:172.21.15.26:6836/1453504863 2023-11-16T16:39:31.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.258+0000 7fa524c43b80 1 -- 172.21.15.1:0/3537600566 >> 172.21.15.1:0/3537600566 conn(0x5640afb3b360 msgr2=0x5640afb3fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:31.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.258+0000 7fa524c43b80 1 -- 172.21.15.1:0/3537600566 shutdown_connections 2023-11-16T16:39:31.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.258+0000 7fa524c43b80 1 -- 172.21.15.1:0/3537600566 wait complete. 2023-11-16T16:39:31.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.258+0000 7fa524c43b80 20 client.15546 trim_cache size 0 max 16384 2023-11-16T16:39:31.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.271+0000 7fa524c43b80 1 Processor -- start 2023-11-16T16:39:31.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.271+0000 7fa524c43b80 1 -- start start 2023-11-16T16:39:31.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.271+0000 7fa524c43b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640afb45b60 0x5640afb4ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.271+0000 7fa524c43b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640afb46470 0x5640afb569d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.272+0000 7fa524c43b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640afb4a8b0 0x5640afb51b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.272+0000 7fa524c43b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640afb38730 con 0x5640afb4a8b0 2023-11-16T16:39:31.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.272+0000 7fa524c43b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640afb46bc0 con 0x5640afb45b60 2023-11-16T16:39:31.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.272+0000 7fa524c43b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640afb46970 con 0x5640afb46470 2023-11-16T16:39:31.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.272+0000 7fa512d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640afb4a8b0 0x5640afb51b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.272+0000 7fa512d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640afb4a8b0 0x5640afb51b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43798/0 (socket says 172.21.15.1:43798) 2023-11-16T16:39:31.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.272+0000 7fa512d9d700 1 -- 172.21.15.1:0/3487594309 learned_addr learned my addr 172.21.15.1:0/3487594309 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:31.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.272+0000 7fa512d9d700 1 -- 172.21.15.1:0/3487594309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640afb46470 msgr2=0x5640afb569d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:31.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.272+0000 7fa512d9d700 1 --2- 172.21.15.1:0/3487594309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640afb46470 0x5640afb569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.272+0000 7fa512d9d700 1 -- 172.21.15.1:0/3487594309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640afb45b60 msgr2=0x5640afb4ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:31.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.272+0000 7fa512d9d700 1 --2- 172.21.15.1:0/3487594309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640afb45b60 0x5640afb4ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.272+0000 7fa512d9d700 1 -- 172.21.15.1:0/3487594309 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640afb3cb00 con 0x5640afb4a8b0 2023-11-16T16:39:31.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.272+0000 7fa512d9d700 1 --2- 172.21.15.1:0/3487594309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640afb4a8b0 0x5640afb51b40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa50c0c9f40 tx=0x7fa50c084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec51d58e302cdc8 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:31.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.272+0000 7fa5017fa700 1 -- 172.21.15.1:0/3487594309 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50c0990e0 con 0x5640afb4a8b0 2023-11-16T16:39:31.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.272+0000 7fa5017fa700 1 -- 172.21.15.1:0/3487594309 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa50c099780 con 0x5640afb4a8b0 2023-11-16T16:39:31.283 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.272+0000 7fa5017fa700 1 -- 172.21.15.1:0/3487594309 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50c0ca3c0 con 0x5640afb4a8b0 2023-11-16T16:39:31.283 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.273+0000 7fa524c43b80 1 -- 172.21.15.1:0/3487594309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640afb4a8b0 msgr2=0x5640afb51b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:31.283 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.273+0000 7fa524c43b80 1 --2- 172.21.15.1:0/3487594309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640afb4a8b0 0x5640afb51b40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa50c0c9f40 tx=0x7fa50c084040 comp rx=0 tx=0).stop 2023-11-16T16:39:31.283 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.273+0000 7fa524c43b80 1 -- 172.21.15.1:0/3487594309 shutdown_connections 2023-11-16T16:39:31.283 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.273+0000 7fa524c43b80 1 --2- 172.21.15.1:0/3487594309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640afb4a8b0 0x5640afb51b40 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.283 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.273+0000 7fa524c43b80 1 --2- 172.21.15.1:0/3487594309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640afb45b60 0x5640afb4ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.283 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.273+0000 7fa524c43b80 1 --2- 172.21.15.1:0/3487594309 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640afb46470 0x5640afb569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.283 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.273+0000 7fa524c43b80 1 -- 172.21.15.1:0/3487594309 >> 172.21.15.1:0/3487594309 conn(0x5640afb3b360 msgr2=0x5640afb4b440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:31.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.273+0000 7fa524c43b80 1 -- 172.21.15.1:0/3487594309 shutdown_connections 2023-11-16T16:39:31.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.273+0000 7fa524c43b80 1 -- 172.21.15.1:0/3487594309 wait complete. 2023-11-16T16:39:31.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.273+0000 7fa524c43b80 1 -- start start 2023-11-16T16:39:31.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.273+0000 7fa524c43b80 1 Processor -- start 2023-11-16T16:39:31.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.273+0000 7fa524c43b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640afb45b60 0x5640afb4eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.273+0000 7fa524c43b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640afb46470 0x5640afb4f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.273+0000 7fa524c43b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640afb4a8b0 0x5640afb4d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.273+0000 7fa524c43b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640afb56d50 con 0x5640afb4a8b0 2023-11-16T16:39:31.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.273+0000 7fa524c43b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640afb56ec0 con 0x5640afb46470 2023-11-16T16:39:31.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.273+0000 7fa524c43b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640afb4df30 con 0x5640afb45b60 2023-11-16T16:39:31.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.273+0000 7fa524c43b80 10 client.0 _ll_register_callbacks cb 0x7fa50400bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:39:31.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.274+0000 7fa512d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640afb4a8b0 0x5640afb4d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.274+0000 7fa512d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640afb4a8b0 0x5640afb4d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43804/0 (socket says 172.21.15.1:43804) 2023-11-16T16:39:31.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.274+0000 7fa512d9d700 1 -- 172.21.15.1:0/3109178628 learned_addr learned my addr 172.21.15.1:0/3109178628 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:31.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.274+0000 7fa51259c700 1 --2- 172.21.15.1:0/3109178628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640afb45b60 0x5640afb4eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.274+0000 7fa511d9b700 1 --2- 172.21.15.1:0/3109178628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640afb46470 0x5640afb4f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.274+0000 7fa512d9d700 1 -- 172.21.15.1:0/3109178628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640afb45b60 msgr2=0x5640afb4eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:31.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.274+0000 7fa512d9d700 1 --2- 172.21.15.1:0/3109178628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640afb45b60 0x5640afb4eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.274+0000 7fa512d9d700 1 -- 172.21.15.1:0/3109178628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640afb46470 msgr2=0x5640afb4f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:31.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.274+0000 7fa512d9d700 1 --2- 172.21.15.1:0/3109178628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640afb46470 0x5640afb4f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.274+0000 7fa512d9d700 1 -- 172.21.15.1:0/3109178628 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa51410f020 con 0x5640afb4a8b0 2023-11-16T16:39:31.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.274+0000 7fa511d9b700 1 --2- 172.21.15.1:0/3109178628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640afb46470 0x5640afb4f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:39:31.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.274+0000 7fa512d9d700 1 --2- 172.21.15.1:0/3109178628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640afb4a8b0 0x5640afb4d9f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa50c095c80 tx=0x7fa50c0c9ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f805296455d5228 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:31.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.274+0000 7fa4f9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:39:31.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.274+0000 7fa4f9ffb700 1 -- 172.21.15.1:0/3109178628 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50c09ade0 con 0x5640afb4a8b0 2023-11-16T16:39:31.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.274+0000 7fa4f9ffb700 1 -- 172.21.15.1:0/3109178628 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fa50c09d7b0 con 0x5640afb4a8b0 2023-11-16T16:39:31.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.274+0000 7fa524c43b80 1 -- 172.21.15.1:0/3109178628 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa50800ee50 con 0x5640afb4a8b0 2023-11-16T16:39:31.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.274+0000 7fa4f9ffb700 1 -- 172.21.15.1:0/3109178628 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa50c0ca370 con 0x5640afb4a8b0 2023-11-16T16:39:31.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.274+0000 7fa524c43b80 10 client.15552 fetch_fsmap learned FSMap version 23 2023-11-16T16:39:31.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.274+0000 7fa524c43b80 1 -- 172.21.15.1:0/3109178628 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5640af8c4de0 con 0x5640afb4a8b0 2023-11-16T16:39:31.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.274+0000 7fa4f9ffb700 1 -- 172.21.15.1:0/3109178628 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa50c063020 con 0x5640afb4a8b0 2023-11-16T16:39:31.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.275+0000 7fa4f9ffb700 1 -- 172.21.15.1:0/3109178628 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa50c0b6030 con 0x5640afb4a8b0 2023-11-16T16:39:31.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.275+0000 7fa524c43b80 10 client.15552 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:39:31.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.275+0000 7fa524c43b80 10 client.15552 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-11-16T16:39:31.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.275+0000 7fa524c43b80 20 client.15552 populate_metadata read hostname 'smithi001' 2023-11-16T16:39:31.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.275+0000 7fa524c43b80 1 --2- 172.21.15.1:0/3109178628 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x5640afbb3ee0 0x5640afbb62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.275+0000 7fa524c43b80 4 client.15552 mds_command: new command op to 24482 tid=0[{"prefix": "damage ls"}] 2023-11-16T16:39:31.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.275+0000 7fa524c43b80 1 -- 172.21.15.1:0/3109178628 --> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5640afa65660 con 0x5640afbb3ee0 2023-11-16T16:39:31.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.275+0000 7fa51259c700 1 --2- 172.21.15.1:0/3109178628 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x5640afbb3ee0 0x5640afbb62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.275+0000 7fa51259c700 1 --2- 172.21.15.1:0/3109178628 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x5640afbb3ee0 0x5640afbb62c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e4ef582da752829e server_cookie=2e0422dd6b0a1f07 in_seq=0 out_seq=0 2023-11-16T16:39:31.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.275+0000 7fa4f9ffb700 10 client.15552 ms_handle_connect on v2:172.21.15.26:6836/1453504863 2023-11-16T16:39:31.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.275+0000 7fa4f9ffb700 1 -- 172.21.15.1:0/3109178628 <== mds.2 v2:172.21.15.26:6836/1453504863 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5640afa65660 con 0x5640afbb3ee0 2023-11-16T16:39:31.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.275+0000 7fa4f9ffb700 10 client.15552 handle_command_reply: tid=0 2023-11-16T16:39:31.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.276+0000 7fa524c43b80 1 client.15552 shutdown 2023-11-16T16:39:31.290 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.276+0000 7fa524c43b80 1 -- 172.21.15.1:0/3109178628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640afb4a8b0 msgr2=0x5640afb4d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:31.290 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.276+0000 7fa524c43b80 1 --2- 172.21.15.1:0/3109178628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640afb4a8b0 0x5640afb4d9f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa50c095c80 tx=0x7fa50c0c9ed0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.290 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.276+0000 7fa524c43b80 1 -- 172.21.15.1:0/3109178628 shutdown_connections 2023-11-16T16:39:31.290 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.276+0000 7fa524c43b80 1 --2- 172.21.15.1:0/3109178628 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x5640afbb3ee0 0x5640afbb62c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.290 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.276+0000 7fa524c43b80 1 --2- 172.21.15.1:0/3109178628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640afb4a8b0 0x5640afb4d9f0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.290 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.276+0000 7fa4f9ffb700 0 client.15552 ms_handle_reset on v2:172.21.15.26:6836/1453504863 2023-11-16T16:39:31.290 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.276+0000 7fa524c43b80 1 --2- 172.21.15.1:0/3109178628 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640afb45b60 0x5640afb4eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.291 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.276+0000 7fa524c43b80 1 --2- 172.21.15.1:0/3109178628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640afb46470 0x5640afb4f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.291 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.276+0000 7fa524c43b80 1 -- 172.21.15.1:0/3109178628 >> 172.21.15.1:0/3109178628 conn(0x5640afb3b360 msgr2=0x5640afb488f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:31.291 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.276+0000 7fa524c43b80 1 -- 172.21.15.1:0/3109178628 shutdown_connections 2023-11-16T16:39:31.291 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.276+0000 7fa524c43b80 1 -- 172.21.15.1:0/3109178628 wait complete. 2023-11-16T16:39:31.291 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.276+0000 7fa524c43b80 20 client.15552 trim_cache size 0 max 16384 2023-11-16T16:39:31.291 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2023-11-16T16:39:31.293 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.276+0000 7fa518ab7700 1 -- 172.21.15.1:0/2545296876 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fa4fc0a1350 msgr2=0x7fa4fc0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:31.293 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.276+0000 7fa518ab7700 1 --2- 172.21.15.1:0/2545296876 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fa4fc0a1350 0x7fa4fc0a3800 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa50400b5c0 tx=0x7fa504005fb0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.293 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.276+0000 7fa518ab7700 1 -- 172.21.15.1:0/2545296876 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa514105250 msgr2=0x7fa514195950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:31.293 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.276+0000 7fa518ab7700 1 --2- 172.21.15.1:0/2545296876 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa514105250 0x7fa514195950 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa50c007ee0 tx=0x7fa50c00d3b0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.293 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.276+0000 7fa518ab7700 1 -- 172.21.15.1:0/2545296876 shutdown_connections 2023-11-16T16:39:31.293 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.276+0000 7fa518ab7700 1 --2- 172.21.15.1:0/2545296876 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fa4fc0a1350 0x7fa4fc0a3800 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.294 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.276+0000 7fa518ab7700 1 --2- 172.21.15.1:0/2545296876 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa514105250 0x7fa514195950 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.294 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.276+0000 7fa518ab7700 1 --2- 172.21.15.1:0/2545296876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5140ffa70 0x7fa514194ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.294 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.276+0000 7fa518ab7700 1 --2- 172.21.15.1:0/2545296876 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa514100420 0x7fa514195410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.294 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.276+0000 7fa518ab7700 1 -- 172.21.15.1:0/2545296876 >> 172.21.15.1:0/2545296876 conn(0x7fa5140763a0 msgr2=0x7fa5140fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:31.294 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.276+0000 7fa518ab7700 1 -- 172.21.15.1:0/2545296876 shutdown_connections 2023-11-16T16:39:31.294 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.276+0000 7fa518ab7700 1 -- 172.21.15.1:0/2545296876 wait complete. 2023-11-16T16:39:31.295 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-11-16T16:39:31.423 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.421+0000 7f0f0908c700 1 -- 172.21.15.1:0/3084347044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f0410af70 msgr2=0x7f0f0410d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:31.423 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.421+0000 7f0f0908c700 1 --2- 172.21.15.1:0/3084347044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f0410af70 0x7f0f0410d350 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00b0d0 tx=0x7f0efc00b3e0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.424 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.421+0000 7f0f0908c700 1 -- 172.21.15.1:0/3084347044 shutdown_connections 2023-11-16T16:39:31.424 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.421+0000 7f0f0908c700 1 --2- 172.21.15.1:0/3084347044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f0410af70 0x7f0f0410d350 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.424 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.421+0000 7f0f0908c700 1 --2- 172.21.15.1:0/3084347044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f040ff6b0 0x7f0f040ffa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.424 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.421+0000 7f0f0908c700 1 --2- 172.21.15.1:0/3084347044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f040fffc0 0x7f0f0410aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.424 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.421+0000 7f0f0908c700 1 -- 172.21.15.1:0/3084347044 >> 172.21.15.1:0/3084347044 conn(0x7f0f04074c60 msgr2=0x7f0f04075060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:31.425 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.421+0000 7f0f0908c700 1 -- 172.21.15.1:0/3084347044 shutdown_connections 2023-11-16T16:39:31.425 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.421+0000 7f0f0908c700 1 -- 172.21.15.1:0/3084347044 wait complete. 2023-11-16T16:39:31.427 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.422+0000 7f0f0908c700 1 Processor -- start 2023-11-16T16:39:31.427 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.422+0000 7f0f0908c700 1 -- start start 2023-11-16T16:39:31.427 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.423+0000 7f0f0908c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f040ff6b0 0x7f0f041965a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.427 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.423+0000 7f0f0908c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f040fffc0 0x7f0f04196ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.428 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.423+0000 7f0f0908c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f0410af70 0x7f0f0419af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.428 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.423+0000 7f0f0908c700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0419b580 con 0x7f0f040fffc0 2023-11-16T16:39:31.428 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.423+0000 7f0f0908c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0419b6f0 con 0x7f0f0410af70 2023-11-16T16:39:31.428 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.423+0000 7f0f0908c700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0419b8b0 con 0x7f0f040ff6b0 2023-11-16T16:39:31.428 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.423+0000 7f0f0259c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f040fffc0 0x7f0f04196ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.428 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.423+0000 7f0f0259c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f040fffc0 0x7f0f04196ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43816/0 (socket says 172.21.15.1:43816) 2023-11-16T16:39:31.429 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.423+0000 7f0f0259c700 1 -- 172.21.15.1:0/1403769418 learned_addr learned my addr 172.21.15.1:0/1403769418 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:31.429 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.423+0000 7f0f0359e700 1 --2- 172.21.15.1:0/1403769418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f0410af70 0x7f0f0419af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.429 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.423+0000 7f0f0259c700 1 -- 172.21.15.1:0/1403769418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f040ff6b0 msgr2=0x7f0f041965a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:31.429 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.423+0000 7f0f02d9d700 1 --2- 172.21.15.1:0/1403769418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f040ff6b0 0x7f0f041965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.429 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.423+0000 7f0f0259c700 1 --2- 172.21.15.1:0/1403769418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f040ff6b0 0x7f0f041965a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.423+0000 7f0f0259c700 1 -- 172.21.15.1:0/1403769418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f0410af70 msgr2=0x7f0f0419af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:31.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.423+0000 7f0f0259c700 1 --2- 172.21.15.1:0/1403769418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f0410af70 0x7f0f0419af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.423+0000 7f0f0259c700 1 -- 172.21.15.1:0/1403769418 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0efc009d00 con 0x7f0f040fffc0 2023-11-16T16:39:31.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.423+0000 7f0f0359e700 1 --2- 172.21.15.1:0/1403769418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f0410af70 0x7f0f0419af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:39:31.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.423+0000 7f0f02d9d700 1 --2- 172.21.15.1:0/1403769418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f040ff6b0 0x7f0f041965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:39:31.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.423+0000 7f0f0259c700 1 --2- 172.21.15.1:0/1403769418 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f040fffc0 0x7f0f04196ae0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0ef800c9b0 tx=0x7f0ef800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:31.431 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.424+0000 7f0eebfff700 1 -- 172.21.15.1:0/1403769418 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8007b00 con 0x7f0f040fffc0 2023-11-16T16:39:31.431 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.424+0000 7f0eebfff700 1 -- 172.21.15.1:0/1403769418 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ef8007c60 con 0x7f0f040fffc0 2023-11-16T16:39:31.431 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.424+0000 7f0f0908c700 1 -- 172.21.15.1:0/1403769418 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f04110520 con 0x7f0f040fffc0 2023-11-16T16:39:31.431 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.424+0000 7f0eebfff700 1 -- 172.21.15.1:0/1403769418 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8018660 con 0x7f0f040fffc0 2023-11-16T16:39:31.431 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.424+0000 7f0f0908c700 1 -- 172.21.15.1:0/1403769418 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f04110390 con 0x7f0f040fffc0 2023-11-16T16:39:31.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.426+0000 7f0eebfff700 1 -- 172.21.15.1:0/1403769418 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 23) v1 ==== 135890+0+0 (secure 0 0 0) 0x7f0ef8018880 con 0x7f0f040fffc0 2023-11-16T16:39:31.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.426+0000 7f0eebfff700 1 --2- 172.21.15.1:0/1403769418 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f0eec0a1350 0x7f0eec0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.427+0000 7f0f02d9d700 1 --2- 172.21.15.1:0/1403769418 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f0eec0a1350 0x7f0eec0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.427+0000 7f0eebfff700 1 -- 172.21.15.1:0/1403769418 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f0ef80ccfd0 con 0x7f0f040fffc0 2023-11-16T16:39:31.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.427+0000 7f0f02d9d700 1 --2- 172.21.15.1:0/1403769418 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f0eec0a1350 0x7f0eec0a3800 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4005fd0 tx=0x7f0ef4005dc0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:31.436 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.427+0000 7f0f15218b80 1 Processor -- start 2023-11-16T16:39:31.436 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.427+0000 7f0f15218b80 1 -- start start 2023-11-16T16:39:31.436 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.427+0000 7f0f15218b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b21859b60 0x564b21859f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.436 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.428+0000 7f0f15218b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b2185a470 0x564b2185e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.436 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.428+0000 7f0f15218b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b2185e8b0 0x564b2185ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.436 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.428+0000 7f0f15218b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b2175f060 con 0x564b21859b60 2023-11-16T16:39:31.436 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.428+0000 7f0f15218b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b217b18a0 con 0x564b2185a470 2023-11-16T16:39:31.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.428+0000 7f0f15218b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b217aad30 con 0x564b2185e8b0 2023-11-16T16:39:31.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.428+0000 7f0f02d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b21859b60 0x564b21859f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.428+0000 7f0f02d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b21859b60 0x564b21859f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43822/0 (socket says 172.21.15.1:43822) 2023-11-16T16:39:31.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.428+0000 7f0f02d9d700 1 -- 172.21.15.1:0/2189155768 learned_addr learned my addr 172.21.15.1:0/2189155768 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:31.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.428+0000 7f0f0259c700 1 --2- 172.21.15.1:0/2189155768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b2185a470 0x564b2185e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.428+0000 7f0f02d9d700 1 -- 172.21.15.1:0/2189155768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b2185e8b0 msgr2=0x564b2185ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:31.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.428+0000 7f0f02d9d700 1 --2- 172.21.15.1:0/2189155768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b2185e8b0 0x564b2185ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.438 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.428+0000 7f0f02d9d700 1 -- 172.21.15.1:0/2189155768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b2185a470 msgr2=0x564b2185e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:31.438 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.428+0000 7f0f02d9d700 1 --2- 172.21.15.1:0/2189155768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b2185a470 0x564b2185e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.438 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.428+0000 7f0f02d9d700 1 -- 172.21.15.1:0/2189155768 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b2162c630 con 0x564b21859b60 2023-11-16T16:39:31.438 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.428+0000 7f0f0259c700 1 --2- 172.21.15.1:0/2189155768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b2185a470 0x564b2185e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:39:31.438 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.428+0000 7f0f02d9d700 1 --2- 172.21.15.1:0/2189155768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b21859b60 0x564b21859f30 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0ef400db90 tx=0x7f0ef4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82037a535cd96fe server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:31.438 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.428+0000 7f0ee97fa700 1 -- 172.21.15.1:0/2189155768 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef400f930 con 0x564b21859b60 2023-11-16T16:39:31.438 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.428+0000 7f0ee97fa700 1 -- 172.21.15.1:0/2189155768 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ef401c450 con 0x564b21859b60 2023-11-16T16:39:31.439 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.428+0000 7f0ee97fa700 1 -- 172.21.15.1:0/2189155768 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef401b610 con 0x564b21859b60 2023-11-16T16:39:31.439 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.429+0000 7f0f15218b80 1 -- 172.21.15.1:0/2189155768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b21859b60 msgr2=0x564b21859f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:31.439 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.429+0000 7f0f15218b80 1 --2- 172.21.15.1:0/2189155768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b21859b60 0x564b21859f30 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0ef400db90 tx=0x7f0ef4010000 comp rx=0 tx=0).stop 2023-11-16T16:39:31.439 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.429+0000 7f0f15218b80 1 -- 172.21.15.1:0/2189155768 shutdown_connections 2023-11-16T16:39:31.439 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.429+0000 7f0f15218b80 1 --2- 172.21.15.1:0/2189155768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b2185e8b0 0x564b2185ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.439 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.429+0000 7f0f15218b80 1 --2- 172.21.15.1:0/2189155768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b21859b60 0x564b21859f30 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.439 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.429+0000 7f0f15218b80 1 --2- 172.21.15.1:0/2189155768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b2185a470 0x564b2185e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.440 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.429+0000 7f0f15218b80 1 -- 172.21.15.1:0/2189155768 >> 172.21.15.1:0/2189155768 conn(0x564b2184f360 msgr2=0x564b21854b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:31.440 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.429+0000 7f0f15218b80 1 -- 172.21.15.1:0/2189155768 shutdown_connections 2023-11-16T16:39:31.440 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.429+0000 7f0f15218b80 1 -- 172.21.15.1:0/2189155768 wait complete. 2023-11-16T16:39:31.440 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.429+0000 7f0f15218b80 1 -- start start 2023-11-16T16:39:31.440 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.430+0000 7f0f15218b80 1 Processor -- start 2023-11-16T16:39:31.440 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.430+0000 7f0f15218b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b21859b60 0x564b21866050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.440 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.430+0000 7f0f15218b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b2185a470 0x564b218632d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.441 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.430+0000 7f0f15218b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b2185e8b0 0x564b21863950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.441 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.430+0000 7f0f15218b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b2174c8e0 con 0x564b21859b60 2023-11-16T16:39:31.441 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.430+0000 7f0f15218b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b2174c580 con 0x564b2185e8b0 2023-11-16T16:39:31.441 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.430+0000 7f0f15218b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b2162e920 con 0x564b2185a470 2023-11-16T16:39:31.441 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.430+0000 7f0f15218b80 10 client.0 _ll_register_callbacks cb 0x564b2176a780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:39:31.441 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.430+0000 7f0f02d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b21859b60 0x564b21866050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.442 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.430+0000 7f0f02d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b21859b60 0x564b21866050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43828/0 (socket says 172.21.15.1:43828) 2023-11-16T16:39:31.442 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.430+0000 7f0f02d9d700 1 -- 172.21.15.1:0/2417031835 learned_addr learned my addr 172.21.15.1:0/2417031835 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:31.442 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.430+0000 7f0f0359e700 1 --2- 172.21.15.1:0/2417031835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b2185e8b0 0x564b21863950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.442 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:31 smithi001 ceph-mon[98800]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:39:31.442 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:31 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3418598255' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:39:31.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.430+0000 7f0f0259c700 1 --2- 172.21.15.1:0/2417031835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b2185a470 0x564b218632d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.430+0000 7f0f0359e700 1 -- 172.21.15.1:0/2417031835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b2185a470 msgr2=0x564b218632d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:31.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.430+0000 7f0f0359e700 1 --2- 172.21.15.1:0/2417031835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b2185a470 0x564b218632d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.430+0000 7f0f0359e700 1 -- 172.21.15.1:0/2417031835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b21859b60 msgr2=0x564b21866050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:31.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.430+0000 7f0f0359e700 1 --2- 172.21.15.1:0/2417031835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b21859b60 0x564b21866050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.430+0000 7f0f0359e700 1 -- 172.21.15.1:0/2417031835 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ef400d840 con 0x564b2185e8b0 2023-11-16T16:39:31.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.431+0000 7f0f0259c700 1 --2- 172.21.15.1:0/2417031835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b2185a470 0x564b218632d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:39:31.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.431+0000 7f0f0359e700 1 --2- 172.21.15.1:0/2417031835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b2185e8b0 0x564b21863950 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00b820 tx=0x7f0efc009300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8baef58368379d56 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:31.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.431+0000 7f0ee6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-11-16T16:39:31.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.431+0000 7f0ee6ff5700 1 -- 172.21.15.1:0/2417031835 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0efc00e050 con 0x564b2185e8b0 2023-11-16T16:39:31.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.431+0000 7f0ee6ff5700 1 -- 172.21.15.1:0/2417031835 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f0efc009470 con 0x564b2185e8b0 2023-11-16T16:39:31.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.431+0000 7f0f15218b80 1 -- 172.21.15.1:0/2417031835 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564b2184cb20 con 0x564b2185e8b0 2023-11-16T16:39:31.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.431+0000 7f0ee6ff5700 1 -- 172.21.15.1:0/2417031835 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0efc00ceb0 con 0x564b2185e8b0 2023-11-16T16:39:31.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.431+0000 7f0f15218b80 10 client.25184 fetch_fsmap learned FSMap version 23 2023-11-16T16:39:31.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.431+0000 7f0f15218b80 1 -- 172.21.15.1:0/2417031835 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x564b21771190 con 0x564b2185e8b0 2023-11-16T16:39:31.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.431+0000 7f0ee6ff5700 1 -- 172.21.15.1:0/2417031835 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564b2184cb20 con 0x564b2185e8b0 2023-11-16T16:39:31.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.431+0000 7f0ee6ff5700 1 -- 172.21.15.1:0/2417031835 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0efc00b100 con 0x564b2185e8b0 2023-11-16T16:39:31.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.432+0000 7f0f15218b80 10 client.25184 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:39:31.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.432+0000 7f0f15218b80 10 client.25184 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-11-16T16:39:31.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.432+0000 7f0f15218b80 20 client.25184 populate_metadata read hostname 'smithi001' 2023-11-16T16:39:31.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.432+0000 7f0f15218b80 1 --2- 172.21.15.1:0/2417031835 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x564b21873840 0x564b21875c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.432+0000 7f0f15218b80 4 client.25184 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:39:31.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.432+0000 7f0f15218b80 1 -- 172.21.15.1:0/2417031835 --> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564b2176d0a0 con 0x564b21873840 2023-11-16T16:39:31.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.432+0000 7f0f02d9d700 1 --2- 172.21.15.1:0/2417031835 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x564b21873840 0x564b21875c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.432+0000 7f0ee6ff5700 10 client.25184 ms_handle_connect on v2:172.21.15.53:6838/1044753024 2023-11-16T16:39:31.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.432+0000 7f0f02d9d700 1 --2- 172.21.15.1:0/2417031835 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x564b21873840 0x564b21875c20 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b854e10eb7b6859b server_cookie=ca6847865ec1e07d in_seq=0 out_seq=0 2023-11-16T16:39:31.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.433+0000 7f0ee6ff5700 1 -- 172.21.15.1:0/2417031835 <== mds.1 v2:172.21.15.53:6838/1044753024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564b2176d0a0 con 0x564b21873840 2023-11-16T16:39:31.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.433+0000 7f0ee6ff5700 10 client.25184 handle_command_reply: tid=0 2023-11-16T16:39:31.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.433+0000 7f0f15218b80 1 client.25184 shutdown 2023-11-16T16:39:31.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.434+0000 7f0f15218b80 1 -- 172.21.15.1:0/2417031835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b2185e8b0 msgr2=0x564b21863950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:31.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.434+0000 7f0f15218b80 1 --2- 172.21.15.1:0/2417031835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b2185e8b0 0x564b21863950 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00b820 tx=0x7f0efc009300 comp rx=0 tx=0).stop 2023-11-16T16:39:31.450 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.434+0000 7f0f15218b80 1 -- 172.21.15.1:0/2417031835 shutdown_connections 2023-11-16T16:39:31.450 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.434+0000 7f0f15218b80 1 --2- 172.21.15.1:0/2417031835 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x564b21873840 0x564b21875c20 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.450 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.434+0000 7f0f15218b80 1 --2- 172.21.15.1:0/2417031835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b2185e8b0 0x564b21863950 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.450 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.434+0000 7f0ee6ff5700 0 client.25184 ms_handle_reset on v2:172.21.15.53:6838/1044753024 2023-11-16T16:39:31.450 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.434+0000 7f0f15218b80 1 --2- 172.21.15.1:0/2417031835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b21859b60 0x564b21866050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.450 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.434+0000 7f0f15218b80 1 --2- 172.21.15.1:0/2417031835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b2185a470 0x564b218632d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.451 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.434+0000 7f0f15218b80 1 -- 172.21.15.1:0/2417031835 >> 172.21.15.1:0/2417031835 conn(0x564b2184f360 msgr2=0x564b21853de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:31.451 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.434+0000 7f0f15218b80 1 -- 172.21.15.1:0/2417031835 shutdown_connections 2023-11-16T16:39:31.451 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.434+0000 7f0f15218b80 1 -- 172.21.15.1:0/2417031835 wait complete. 2023-11-16T16:39:31.451 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.434+0000 7f0f15218b80 20 client.25184 trim_cache size 0 max 16384 2023-11-16T16:39:31.456 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.447+0000 7f0f15218b80 1 Processor -- start 2023-11-16T16:39:31.457 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.447+0000 7f0f15218b80 1 -- start start 2023-11-16T16:39:31.457 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.447+0000 7f0f15218b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b21859b60 0x564b2186a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.457 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.447+0000 7f0f15218b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b2185a470 0x564b2186fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.457 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.447+0000 7f0f15218b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b2185e8b0 0x564b2185cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.457 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.447+0000 7f0f15218b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b2184f170 con 0x564b2185a470 2023-11-16T16:39:31.457 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.447+0000 7f0f15218b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b21855950 con 0x564b21859b60 2023-11-16T16:39:31.457 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.447+0000 7f0f15218b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b21861850 con 0x564b2185e8b0 2023-11-16T16:39:31.458 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.447+0000 7f0f0259c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b2185a470 0x564b2186fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.458 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.447+0000 7f0f0259c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b2185a470 0x564b2186fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43838/0 (socket says 172.21.15.1:43838) 2023-11-16T16:39:31.458 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.447+0000 7f0f0259c700 1 -- 172.21.15.1:0/373686698 learned_addr learned my addr 172.21.15.1:0/373686698 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:31.458 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.447+0000 7f0f02d9d700 1 --2- 172.21.15.1:0/373686698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b21859b60 0x564b2186a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.458 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.447+0000 7f0f0359e700 1 --2- 172.21.15.1:0/373686698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b2185e8b0 0x564b2185cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.458 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.447+0000 7f0f0259c700 1 -- 172.21.15.1:0/373686698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b2185e8b0 msgr2=0x564b2185cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:31.459 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.447+0000 7f0f0259c700 1 --2- 172.21.15.1:0/373686698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b2185e8b0 0x564b2185cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.459 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.447+0000 7f0f0259c700 1 -- 172.21.15.1:0/373686698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b21859b60 msgr2=0x564b2186a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:31.459 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.447+0000 7f0f0259c700 1 --2- 172.21.15.1:0/373686698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b21859b60 0x564b2186a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.459 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.447+0000 7f0f0259c700 1 -- 172.21.15.1:0/373686698 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b21770ee0 con 0x564b2185a470 2023-11-16T16:39:31.459 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.447+0000 7f0f0359e700 1 --2- 172.21.15.1:0/373686698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b2185e8b0 0x564b2185cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:39:31.459 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.448+0000 7f0f0259c700 1 --2- 172.21.15.1:0/373686698 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b2185a470 0x564b2186fc40 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0ef800cfd0 tx=0x7f0ef8099a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0326cb4bc368b99 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:31.459 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.448+0000 7f0ee9ffb700 1 -- 172.21.15.1:0/373686698 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef809df10 con 0x564b2185a470 2023-11-16T16:39:31.460 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.448+0000 7f0ee9ffb700 1 -- 172.21.15.1:0/373686698 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ef80a2330 con 0x564b2185a470 2023-11-16T16:39:31.460 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.448+0000 7f0ee9ffb700 1 -- 172.21.15.1:0/373686698 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef80993d0 con 0x564b2185a470 2023-11-16T16:39:31.460 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.448+0000 7f0f15218b80 1 -- 172.21.15.1:0/373686698 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b2185a470 msgr2=0x564b2186fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:31.460 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.448+0000 7f0f15218b80 1 --2- 172.21.15.1:0/373686698 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b2185a470 0x564b2186fc40 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0ef800cfd0 tx=0x7f0ef8099a20 comp rx=0 tx=0).stop 2023-11-16T16:39:31.460 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.448+0000 7f0f15218b80 1 -- 172.21.15.1:0/373686698 shutdown_connections 2023-11-16T16:39:31.460 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.448+0000 7f0f15218b80 1 --2- 172.21.15.1:0/373686698 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b2185e8b0 0x564b2185cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.460 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.448+0000 7f0f15218b80 1 --2- 172.21.15.1:0/373686698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b21859b60 0x564b2186a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.461 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.448+0000 7f0f15218b80 1 --2- 172.21.15.1:0/373686698 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b2185a470 0x564b2186fc40 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.461 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.448+0000 7f0f15218b80 1 -- 172.21.15.1:0/373686698 >> 172.21.15.1:0/373686698 conn(0x564b2184f360 msgr2=0x564b2185ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:31.461 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.448+0000 7f0f15218b80 1 -- 172.21.15.1:0/373686698 shutdown_connections 2023-11-16T16:39:31.461 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.448+0000 7f0f15218b80 1 -- 172.21.15.1:0/373686698 wait complete. 2023-11-16T16:39:31.461 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.449+0000 7f0f15218b80 1 -- start start 2023-11-16T16:39:31.461 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.449+0000 7f0f15218b80 1 Processor -- start 2023-11-16T16:39:31.461 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.449+0000 7f0f15218b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b21859b60 0x564b2185f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.462 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.449+0000 7f0f15218b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b2185a470 0x564b2185fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.462 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.449+0000 7f0f15218b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b2185e8b0 0x564b21860140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.462 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.449+0000 7f0f15218b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b21860680 con 0x564b21859b60 2023-11-16T16:39:31.462 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.449+0000 7f0f15218b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b218607f0 con 0x564b2185a470 2023-11-16T16:39:31.462 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.449+0000 7f0f15218b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b218609b0 con 0x564b2185e8b0 2023-11-16T16:39:31.462 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.449+0000 7f0f15218b80 10 client.0 _ll_register_callbacks cb 0x7f0efc009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:39:31.462 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.449+0000 7f0f02d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b21859b60 0x564b2185f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.463 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.449+0000 7f0f02d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b21859b60 0x564b2185f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43842/0 (socket says 172.21.15.1:43842) 2023-11-16T16:39:31.463 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.449+0000 7f0f02d9d700 1 -- 172.21.15.1:0/2714018972 learned_addr learned my addr 172.21.15.1:0/2714018972 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:31.463 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.449+0000 7f0f0259c700 1 --2- 172.21.15.1:0/2714018972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b2185a470 0x564b2185fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.463 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.450+0000 7f0f0359e700 1 --2- 172.21.15.1:0/2714018972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b2185e8b0 0x564b21860140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.463 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.450+0000 7f0f02d9d700 1 -- 172.21.15.1:0/2714018972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b2185e8b0 msgr2=0x564b21860140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:31.463 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.450+0000 7f0f02d9d700 1 --2- 172.21.15.1:0/2714018972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b2185e8b0 0x564b21860140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.463 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.450+0000 7f0f02d9d700 1 -- 172.21.15.1:0/2714018972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b2185a470 msgr2=0x564b2185fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:31.464 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.450+0000 7f0f02d9d700 1 --2- 172.21.15.1:0/2714018972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b2185a470 0x564b2185fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.464 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.450+0000 7f0f02d9d700 1 -- 172.21.15.1:0/2714018972 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f04110390 con 0x564b21859b60 2023-11-16T16:39:31.464 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.450+0000 7f0f0259c700 1 --2- 172.21.15.1:0/2714018972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b2185a470 0x564b2185fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:39:31.464 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.450+0000 7f0f0359e700 1 --2- 172.21.15.1:0/2714018972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b2185e8b0 0x564b21860140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:39:31.464 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.450+0000 7f0f02d9d700 1 --2- 172.21.15.1:0/2714018972 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b21859b60 0x564b2185f6c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4000c00 tx=0x7f0ef401c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c9a1630ddfb043 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:31.464 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.450+0000 7f0ee6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:39:31.464 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.450+0000 7f0ee6ff5700 1 -- 172.21.15.1:0/2714018972 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef402b040 con 0x564b21859b60 2023-11-16T16:39:31.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.450+0000 7f0ee6ff5700 1 -- 172.21.15.1:0/2714018972 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f0ef401c890 con 0x564b21859b60 2023-11-16T16:39:31.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.450+0000 7f0f15218b80 1 -- 172.21.15.1:0/2714018972 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ef401c450 con 0x564b21859b60 2023-11-16T16:39:31.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.450+0000 7f0ee6ff5700 1 -- 172.21.15.1:0/2714018972 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ef4034040 con 0x564b21859b60 2023-11-16T16:39:31.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.450+0000 7f0f15218b80 10 client.15570 fetch_fsmap learned FSMap version 23 2023-11-16T16:39:31.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.450+0000 7f0f15218b80 1 -- 172.21.15.1:0/2714018972 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x564b215c1c60 con 0x564b21859b60 2023-11-16T16:39:31.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.450+0000 7f0ee6ff5700 1 -- 172.21.15.1:0/2714018972 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ef401c450 con 0x564b21859b60 2023-11-16T16:39:31.466 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.451+0000 7f0ee6ff5700 1 -- 172.21.15.1:0/2714018972 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0ef400fd00 con 0x564b21859b60 2023-11-16T16:39:31.466 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.451+0000 7f0f15218b80 10 client.15570 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:39:31.466 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.451+0000 7f0f15218b80 10 client.15570 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-11-16T16:39:31.466 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.451+0000 7f0f15218b80 20 client.15570 populate_metadata read hostname 'smithi001' 2023-11-16T16:39:31.466 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.451+0000 7f0f15218b80 1 --2- 172.21.15.1:0/2714018972 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x564b21873840 0x564b218ca070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.466 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.451+0000 7f0f15218b80 4 client.15570 mds_command: new command op to 24493 tid=0[{"prefix": "damage ls"}] 2023-11-16T16:39:31.466 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.451+0000 7f0f15218b80 1 -- 172.21.15.1:0/2714018972 --> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564b21779660 con 0x564b21873840 2023-11-16T16:39:31.467 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.451+0000 7f0f0259c700 1 --2- 172.21.15.1:0/2714018972 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x564b21873840 0x564b218ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.467 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.451+0000 7f0ee6ff5700 10 client.15570 ms_handle_connect on v2:172.21.15.53:6838/1044753024 2023-11-16T16:39:31.467 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.451+0000 7f0f0259c700 1 --2- 172.21.15.1:0/2714018972 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x564b21873840 0x564b218ca070 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=68d045f26001035b server_cookie=701f874f8b5c86bc in_seq=0 out_seq=0 2023-11-16T16:39:31.467 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.452+0000 7f0ee6ff5700 1 -- 172.21.15.1:0/2714018972 <== mds.1 v2:172.21.15.53:6838/1044753024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564b21779660 con 0x564b21873840 2023-11-16T16:39:31.467 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.452+0000 7f0ee6ff5700 10 client.15570 handle_command_reply: tid=0 2023-11-16T16:39:31.467 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.452+0000 7f0f15218b80 1 client.15570 shutdown 2023-11-16T16:39:31.467 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.452+0000 7f0f15218b80 1 -- 172.21.15.1:0/2714018972 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b21859b60 msgr2=0x564b2185f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:31.468 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.452+0000 7f0f15218b80 1 --2- 172.21.15.1:0/2714018972 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b21859b60 0x564b2185f6c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4000c00 tx=0x7f0ef401c670 comp rx=0 tx=0).stop 2023-11-16T16:39:31.468 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.452+0000 7f0f15218b80 1 -- 172.21.15.1:0/2714018972 shutdown_connections 2023-11-16T16:39:31.468 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.452+0000 7f0f15218b80 1 --2- 172.21.15.1:0/2714018972 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564b2185e8b0 0x564b21860140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.468 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.452+0000 7f0f15218b80 1 --2- 172.21.15.1:0/2714018972 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b21859b60 0x564b2185f6c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.468 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.452+0000 7f0f15218b80 1 --2- 172.21.15.1:0/2714018972 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x564b21873840 0x564b218ca070 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.468 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.452+0000 7f0f15218b80 1 --2- 172.21.15.1:0/2714018972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b2185a470 0x564b2185fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.469 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.452+0000 7f0ee6ff5700 0 client.15570 ms_handle_reset on v2:172.21.15.53:6838/1044753024 2023-11-16T16:39:31.469 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.452+0000 7f0f15218b80 1 -- 172.21.15.1:0/2714018972 >> 172.21.15.1:0/2714018972 conn(0x564b2184f360 msgr2=0x564b2185cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:31.469 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.452+0000 7f0f15218b80 1 -- 172.21.15.1:0/2714018972 shutdown_connections 2023-11-16T16:39:31.469 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.452+0000 7f0f15218b80 1 -- 172.21.15.1:0/2714018972 wait complete. 2023-11-16T16:39:31.469 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.452+0000 7f0f15218b80 20 client.15570 trim_cache size 0 max 16384 2023-11-16T16:39:31.469 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2023-11-16T16:39:31.471 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.452+0000 7f0f0908c700 1 -- 172.21.15.1:0/1403769418 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f0eec0a1350 msgr2=0x7f0eec0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:31.471 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.452+0000 7f0f0908c700 1 --2- 172.21.15.1:0/1403769418 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f0eec0a1350 0x7f0eec0a3800 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4005fd0 tx=0x7f0ef4005dc0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.471 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.452+0000 7f0f0908c700 1 -- 172.21.15.1:0/1403769418 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f040fffc0 msgr2=0x7f0f04196ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:31.471 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.452+0000 7f0f0908c700 1 --2- 172.21.15.1:0/1403769418 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f040fffc0 0x7f0f04196ae0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0ef800c9b0 tx=0x7f0ef800cd70 comp rx=0 tx=0).stop 2023-11-16T16:39:31.471 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.452+0000 7f0f0908c700 1 -- 172.21.15.1:0/1403769418 shutdown_connections 2023-11-16T16:39:31.472 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.452+0000 7f0f0908c700 1 --2- 172.21.15.1:0/1403769418 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f0eec0a1350 0x7f0eec0a3800 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.472 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.452+0000 7f0f0908c700 1 --2- 172.21.15.1:0/1403769418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f0410af70 0x7f0f0419af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.472 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.452+0000 7f0f0908c700 1 --2- 172.21.15.1:0/1403769418 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0f040ff6b0 0x7f0f041965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.472 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.452+0000 7f0f0908c700 1 --2- 172.21.15.1:0/1403769418 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f040fffc0 0x7f0f04196ae0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.472 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.452+0000 7f0f0908c700 1 -- 172.21.15.1:0/1403769418 >> 172.21.15.1:0/1403769418 conn(0x7f0f04074c60 msgr2=0x7f0f041029d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:31.472 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.453+0000 7f0f0908c700 1 -- 172.21.15.1:0/1403769418 shutdown_connections 2023-11-16T16:39:31.472 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.453+0000 7f0f0908c700 1 -- 172.21.15.1:0/1403769418 wait complete. 2023-11-16T16:39:31.473 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-16T16:39:31.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:31 smithi053 ceph-mon[100638]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:39:31.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:31 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/3418598255' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:39:31.603 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.599+0000 7fc19afd0700 1 -- 172.21.15.1:0/2090522226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1941014b0 msgr2=0x7fc194101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:31.603 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.599+0000 7fc19afd0700 1 --2- 172.21.15.1:0/2090522226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1941014b0 0x7fc194101880 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc188009990 tx=0x7fc188009ca0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.603 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.599+0000 7fc19afd0700 1 -- 172.21.15.1:0/2090522226 shutdown_connections 2023-11-16T16:39:31.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.599+0000 7fc19afd0700 1 --2- 172.21.15.1:0/2090522226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc19410a880 0x7fc19410ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.599+0000 7fc19afd0700 1 --2- 172.21.15.1:0/2090522226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1941014b0 0x7fc194101880 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.599+0000 7fc19afd0700 1 --2- 172.21.15.1:0/2090522226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc194101dc0 0x7fc19410a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.599+0000 7fc19afd0700 1 -- 172.21.15.1:0/2090522226 >> 172.21.15.1:0/2090522226 conn(0x7fc1940763a0 msgr2=0x7fc1940767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:31.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.599+0000 7fc19afd0700 1 -- 172.21.15.1:0/2090522226 shutdown_connections 2023-11-16T16:39:31.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.599+0000 7fc19afd0700 1 -- 172.21.15.1:0/2090522226 wait complete. 2023-11-16T16:39:31.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.600+0000 7fc19afd0700 1 Processor -- start 2023-11-16T16:39:31.605 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.600+0000 7fc19afd0700 1 -- start start 2023-11-16T16:39:31.605 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.600+0000 7fc19afd0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1941014b0 0x7fc194196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.605 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.600+0000 7fc19afd0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc194101dc0 0x7fc194196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.605 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.600+0000 7fc19afd0700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc19410a880 0x7fc19419adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.605 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.600+0000 7fc19afd0700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19419b3f0 con 0x7fc1941014b0 2023-11-16T16:39:31.605 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.600+0000 7fc19afd0700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19419b560 con 0x7fc194101dc0 2023-11-16T16:39:31.605 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.600+0000 7fc19afd0700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19419b720 con 0x7fc19410a880 2023-11-16T16:39:31.606 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.600+0000 7fc198d6c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1941014b0 0x7fc194196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.606 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.600+0000 7fc198d6c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1941014b0 0x7fc194196410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43856/0 (socket says 172.21.15.1:43856) 2023-11-16T16:39:31.606 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.600+0000 7fc198d6c700 1 -- 172.21.15.1:0/3958608688 learned_addr learned my addr 172.21.15.1:0/3958608688 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:31.606 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.600+0000 7fc193fff700 1 --2- 172.21.15.1:0/3958608688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc194101dc0 0x7fc194196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.606 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.600+0000 7fc19956d700 1 --2- 172.21.15.1:0/3958608688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc19410a880 0x7fc19419adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.606 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.600+0000 7fc198d6c700 1 -- 172.21.15.1:0/3958608688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc19410a880 msgr2=0x7fc19419adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:31.606 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.600+0000 7fc198d6c700 1 --2- 172.21.15.1:0/3958608688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc19410a880 0x7fc19419adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.606 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.600+0000 7fc198d6c700 1 -- 172.21.15.1:0/3958608688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc194101dc0 msgr2=0x7fc194196950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:31.607 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.600+0000 7fc198d6c700 1 --2- 172.21.15.1:0/3958608688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc194101dc0 0x7fc194196950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.607 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.600+0000 7fc198d6c700 1 -- 172.21.15.1:0/3958608688 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1880095f0 con 0x7fc1941014b0 2023-11-16T16:39:31.607 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.600+0000 7fc19956d700 1 --2- 172.21.15.1:0/3958608688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc19410a880 0x7fc19419adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:39:31.607 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.600+0000 7fc193fff700 1 --2- 172.21.15.1:0/3958608688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc194101dc0 0x7fc194196950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:39:31.607 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.600+0000 7fc198d6c700 1 --2- 172.21.15.1:0/3958608688 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1941014b0 0x7fc194196410 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc1880095c0 tx=0x7fc18800f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:31.607 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.601+0000 7fc191ffb700 1 -- 172.21.15.1:0/3958608688 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18801d070 con 0x7fc1941014b0 2023-11-16T16:39:31.607 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.601+0000 7fc191ffb700 1 -- 172.21.15.1:0/3958608688 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc18800fc00 con 0x7fc1941014b0 2023-11-16T16:39:31.608 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.601+0000 7fc19afd0700 1 -- 172.21.15.1:0/3958608688 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1941102f0 con 0x7fc1941014b0 2023-11-16T16:39:31.608 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.601+0000 7fc191ffb700 1 -- 172.21.15.1:0/3958608688 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc188017650 con 0x7fc1941014b0 2023-11-16T16:39:31.608 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.601+0000 7fc19afd0700 1 -- 172.21.15.1:0/3958608688 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc194110160 con 0x7fc1941014b0 2023-11-16T16:39:31.614 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.603+0000 7fc191ffb700 1 -- 172.21.15.1:0/3958608688 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 23) v1 ==== 135890+0+0 (secure 0 0 0) 0x7fc188022470 con 0x7fc1941014b0 2023-11-16T16:39:31.614 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.603+0000 7fc191ffb700 1 --2- 172.21.15.1:0/3958608688 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fc1840a13d0 0x7fc1840a3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.614 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.604+0000 7fc191ffb700 1 -- 172.21.15.1:0/3958608688 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc1880ce270 con 0x7fc1941014b0 2023-11-16T16:39:31.614 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.604+0000 7fc193fff700 1 --2- 172.21.15.1:0/3958608688 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fc1840a13d0 0x7fc1840a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.614 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.604+0000 7fc193fff700 1 --2- 172.21.15.1:0/3958608688 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fc1840a13d0 0x7fc1840a3880 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc194197a20 tx=0x7fc180005f50 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:31.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.604+0000 7fc1a715cb80 1 Processor -- start 2023-11-16T16:39:31.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.604+0000 7fc1a715cb80 1 -- start start 2023-11-16T16:39:31.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.605+0000 7fc1a715cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562cab95cb60 0x562cab95cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.605+0000 7fc1a715cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cab95d470 0x562cab961370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.605+0000 7fc1a715cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562cab9618b0 0x562cab961d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.605+0000 7fc1a715cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cab862060 con 0x562cab9618b0 2023-11-16T16:39:31.616 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.605+0000 7fc1a715cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cab8b48a0 con 0x562cab95d470 2023-11-16T16:39:31.616 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.605+0000 7fc1a715cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cab8add30 con 0x562cab95cb60 2023-11-16T16:39:31.616 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.605+0000 7fc19956d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562cab9618b0 0x562cab961d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.616 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.605+0000 7fc19956d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562cab9618b0 0x562cab961d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43868/0 (socket says 172.21.15.1:43868) 2023-11-16T16:39:31.616 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.605+0000 7fc19956d700 1 -- 172.21.15.1:0/2426833235 learned_addr learned my addr 172.21.15.1:0/2426833235 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:31.616 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.605+0000 7fc193fff700 1 --2- 172.21.15.1:0/2426833235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cab95d470 0x562cab961370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.616 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.605+0000 7fc198d6c700 1 --2- 172.21.15.1:0/2426833235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562cab95cb60 0x562cab95cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.617 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.605+0000 7fc19956d700 1 -- 172.21.15.1:0/2426833235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562cab95cb60 msgr2=0x562cab95cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:31.617 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.605+0000 7fc19956d700 1 --2- 172.21.15.1:0/2426833235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562cab95cb60 0x562cab95cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.617 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.605+0000 7fc19956d700 1 -- 172.21.15.1:0/2426833235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cab95d470 msgr2=0x562cab961370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:31.617 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.605+0000 7fc19956d700 1 --2- 172.21.15.1:0/2426833235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cab95d470 0x562cab961370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.617 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.605+0000 7fc19956d700 1 -- 172.21.15.1:0/2426833235 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562cab72f630 con 0x562cab9618b0 2023-11-16T16:39:31.617 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.605+0000 7fc193fff700 1 --2- 172.21.15.1:0/2426833235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cab95d470 0x562cab961370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:39:31.617 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.605+0000 7fc19956d700 1 --2- 172.21.15.1:0/2426833235 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562cab9618b0 0x562cab961d00 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009d40 tx=0x7fc18c0077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ef2cf0f322630f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:31.617 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.606+0000 7fc17effd700 1 -- 172.21.15.1:0/2426833235 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c014600 con 0x562cab9618b0 2023-11-16T16:39:31.618 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.606+0000 7fc17effd700 1 -- 172.21.15.1:0/2426833235 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc18c014c20 con 0x562cab9618b0 2023-11-16T16:39:31.618 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.606+0000 7fc17effd700 1 -- 172.21.15.1:0/2426833235 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c013b70 con 0x562cab9618b0 2023-11-16T16:39:31.618 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.606+0000 7fc1a715cb80 1 -- 172.21.15.1:0/2426833235 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562cab9618b0 msgr2=0x562cab961d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:31.618 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.606+0000 7fc1a715cb80 1 --2- 172.21.15.1:0/2426833235 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562cab9618b0 0x562cab961d00 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009d40 tx=0x7fc18c0077e0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.618 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.606+0000 7fc1a715cb80 1 -- 172.21.15.1:0/2426833235 shutdown_connections 2023-11-16T16:39:31.618 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.606+0000 7fc1a715cb80 1 --2- 172.21.15.1:0/2426833235 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562cab9618b0 0x562cab961d00 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.619 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.606+0000 7fc1a715cb80 1 --2- 172.21.15.1:0/2426833235 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562cab95cb60 0x562cab95cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.619 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.606+0000 7fc1a715cb80 1 --2- 172.21.15.1:0/2426833235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cab95d470 0x562cab961370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.619 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.606+0000 7fc1a715cb80 1 -- 172.21.15.1:0/2426833235 >> 172.21.15.1:0/2426833235 conn(0x562cab952360 msgr2=0x562cab957b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:31.619 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.606+0000 7fc1a715cb80 1 -- 172.21.15.1:0/2426833235 shutdown_connections 2023-11-16T16:39:31.619 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.606+0000 7fc1a715cb80 1 -- 172.21.15.1:0/2426833235 wait complete. 2023-11-16T16:39:31.619 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.607+0000 7fc1a715cb80 1 -- start start 2023-11-16T16:39:31.619 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.607+0000 7fc1a715cb80 1 Processor -- start 2023-11-16T16:39:31.620 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.607+0000 7fc1a715cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cab95cb60 0x562cab968f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.620 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.607+0000 7fc1a715cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562cab95d470 0x562cab966210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.620 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.607+0000 7fc1a715cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562cab9618b0 0x562cab9667d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.620 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.607+0000 7fc1a715cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cab84f8e0 con 0x562cab95d470 2023-11-16T16:39:31.620 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.607+0000 7fc1a715cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cab84f580 con 0x562cab95cb60 2023-11-16T16:39:31.620 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.607+0000 7fc1a715cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cab731920 con 0x562cab9618b0 2023-11-16T16:39:31.620 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.607+0000 7fc1a715cb80 10 client.0 _ll_register_callbacks cb 0x562cab86d780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:39:31.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.607+0000 7fc193fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562cab95d470 0x562cab966210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.608+0000 7fc193fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562cab95d470 0x562cab966210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43872/0 (socket says 172.21.15.1:43872) 2023-11-16T16:39:31.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.608+0000 7fc193fff700 1 -- 172.21.15.1:0/2912212533 learned_addr learned my addr 172.21.15.1:0/2912212533 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:31.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.608+0000 7fc19956d700 1 --2- 172.21.15.1:0/2912212533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562cab9618b0 0x562cab9667d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.608+0000 7fc198d6c700 1 --2- 172.21.15.1:0/2912212533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cab95cb60 0x562cab968f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.608+0000 7fc193fff700 1 -- 172.21.15.1:0/2912212533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562cab9618b0 msgr2=0x562cab9667d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:31.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.608+0000 7fc193fff700 1 --2- 172.21.15.1:0/2912212533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562cab9618b0 0x562cab9667d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.622 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.608+0000 7fc193fff700 1 -- 172.21.15.1:0/2912212533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cab95cb60 msgr2=0x562cab968f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:31.622 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.608+0000 7fc193fff700 1 --2- 172.21.15.1:0/2912212533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cab95cb60 0x562cab968f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.622 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.608+0000 7fc193fff700 1 -- 172.21.15.1:0/2912212533 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc18c009980 con 0x562cab95d470 2023-11-16T16:39:31.622 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.608+0000 7fc193fff700 1 --2- 172.21.15.1:0/2912212533 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562cab95d470 0x562cab966210 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc1800109e0 tx=0x7fc180004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b48694cc0d0e2d55 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:31.622 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.608+0000 7fc173fff700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:39:31.622 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.608+0000 7fc173fff700 1 -- 172.21.15.1:0/2912212533 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc180004a30 con 0x562cab95d470 2023-11-16T16:39:31.622 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.608+0000 7fc173fff700 1 -- 172.21.15.1:0/2912212533 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc180004b90 con 0x562cab95d470 2023-11-16T16:39:31.623 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.608+0000 7fc1a715cb80 1 -- 172.21.15.1:0/2912212533 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562cab94fb20 con 0x562cab95d470 2023-11-16T16:39:31.623 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.608+0000 7fc173fff700 1 -- 172.21.15.1:0/2912212533 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc180004d50 con 0x562cab95d470 2023-11-16T16:39:31.623 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.608+0000 7fc1a715cb80 10 client.15582 fetch_fsmap learned FSMap version 23 2023-11-16T16:39:31.623 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.608+0000 7fc1a715cb80 1 -- 172.21.15.1:0/2912212533 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x562cab874190 con 0x562cab95d470 2023-11-16T16:39:31.623 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.608+0000 7fc173fff700 1 -- 172.21.15.1:0/2912212533 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562cab94fb20 con 0x562cab95d470 2023-11-16T16:39:31.623 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.609+0000 7fc173fff700 1 -- 172.21.15.1:0/2912212533 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc180023030 con 0x562cab95d470 2023-11-16T16:39:31.623 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.609+0000 7fc1a715cb80 10 client.15582 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:39:31.624 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.609+0000 7fc1a715cb80 10 client.15582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:39:31.624 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.609+0000 7fc1a715cb80 20 client.15582 populate_metadata read hostname 'smithi001' 2023-11-16T16:39:31.624 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.609+0000 7fc1a715cb80 1 --2- 172.21.15.1:0/2912212533 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x562cab976750 0x562cab978b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.624 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.609+0000 7fc1a715cb80 4 client.15582 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:39:31.624 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.609+0000 7fc1a715cb80 1 -- 172.21.15.1:0/2912212533 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562cab8700a0 con 0x562cab976750 2023-11-16T16:39:31.624 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.609+0000 7fc198d6c700 1 --2- 172.21.15.1:0/2912212533 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x562cab976750 0x562cab978b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.624 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.609+0000 7fc173fff700 10 client.15582 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:39:31.625 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.609+0000 7fc198d6c700 1 --2- 172.21.15.1:0/2912212533 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x562cab976750 0x562cab978b60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=649f0e317a3f1e44 server_cookie=5bc1b6f726a0eec3 in_seq=0 out_seq=0 2023-11-16T16:39:31.625 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.610+0000 7fc173fff700 1 -- 172.21.15.1:0/2912212533 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562cab8700a0 con 0x562cab976750 2023-11-16T16:39:31.625 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.610+0000 7fc173fff700 10 client.15582 handle_command_reply: tid=0 2023-11-16T16:39:31.625 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.610+0000 7fc1a715cb80 1 client.15582 shutdown 2023-11-16T16:39:31.625 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.610+0000 7fc1a715cb80 1 -- 172.21.15.1:0/2912212533 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562cab95d470 msgr2=0x562cab966210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:31.625 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.610+0000 7fc1a715cb80 1 --2- 172.21.15.1:0/2912212533 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562cab95d470 0x562cab966210 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc1800109e0 tx=0x7fc180004750 comp rx=0 tx=0).stop 2023-11-16T16:39:31.625 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.610+0000 7fc1a715cb80 1 -- 172.21.15.1:0/2912212533 shutdown_connections 2023-11-16T16:39:31.626 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.610+0000 7fc1a715cb80 1 --2- 172.21.15.1:0/2912212533 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x562cab976750 0x562cab978b60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.626 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.610+0000 7fc1a715cb80 1 --2- 172.21.15.1:0/2912212533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562cab9618b0 0x562cab9667d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.626 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.610+0000 7fc1a715cb80 1 --2- 172.21.15.1:0/2912212533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cab95cb60 0x562cab968f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.626 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.610+0000 7fc173fff700 0 client.15582 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:39:31.626 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.610+0000 7fc1a715cb80 1 --2- 172.21.15.1:0/2912212533 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562cab95d470 0x562cab966210 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.626 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.610+0000 7fc1a715cb80 1 -- 172.21.15.1:0/2912212533 >> 172.21.15.1:0/2912212533 conn(0x562cab952360 msgr2=0x562cab962d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:31.626 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.611+0000 7fc1a715cb80 1 -- 172.21.15.1:0/2912212533 shutdown_connections 2023-11-16T16:39:31.627 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.611+0000 7fc1a715cb80 1 -- 172.21.15.1:0/2912212533 wait complete. 2023-11-16T16:39:31.627 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.611+0000 7fc1a715cb80 20 client.15582 trim_cache size 0 max 16384 2023-11-16T16:39:31.632 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.624+0000 7fc1a715cb80 1 Processor -- start 2023-11-16T16:39:31.632 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.624+0000 7fc1a715cb80 1 -- start start 2023-11-16T16:39:31.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.624+0000 7fc1a715cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562cab95cb60 0x562cab957c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.624+0000 7fc1a715cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562cab95d470 0x562cab96d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.624+0000 7fc1a715cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cab9618b0 0x562cab968b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.624+0000 7fc1a715cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cab960140 con 0x562cab95cb60 2023-11-16T16:39:31.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.624+0000 7fc1a715cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cab951ee0 con 0x562cab9618b0 2023-11-16T16:39:31.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.624+0000 7fc1a715cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cab94f9a0 con 0x562cab95d470 2023-11-16T16:39:31.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.624+0000 7fc198d6c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562cab95cb60 0x562cab957c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.624+0000 7fc198d6c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562cab95cb60 0x562cab957c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43878/0 (socket says 172.21.15.1:43878) 2023-11-16T16:39:31.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.624+0000 7fc198d6c700 1 -- 172.21.15.1:0/4265375608 learned_addr learned my addr 172.21.15.1:0/4265375608 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:31.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.624+0000 7fc19956d700 1 --2- 172.21.15.1:0/4265375608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cab9618b0 0x562cab968b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.624+0000 7fc198d6c700 1 -- 172.21.15.1:0/4265375608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562cab95d470 msgr2=0x562cab96d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:31.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.624+0000 7fc198d6c700 1 --2- 172.21.15.1:0/4265375608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562cab95d470 0x562cab96d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.624+0000 7fc198d6c700 1 -- 172.21.15.1:0/4265375608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cab9618b0 msgr2=0x562cab968b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:31.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.624+0000 7fc198d6c700 1 --2- 172.21.15.1:0/4265375608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cab9618b0 0x562cab968b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.624+0000 7fc198d6c700 1 -- 172.21.15.1:0/4265375608 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562cab956820 con 0x562cab95cb60 2023-11-16T16:39:31.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.624+0000 7fc19956d700 1 --2- 172.21.15.1:0/4265375608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cab9618b0 0x562cab968b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:39:31.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.624+0000 7fc198d6c700 1 --2- 172.21.15.1:0/4265375608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562cab95cb60 0x562cab957c40 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc18809b200 tx=0x7fc188009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299b8f2b213b1168 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:31.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.624+0000 7fc17f7fe700 1 -- 172.21.15.1:0/4265375608 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1880ce040 con 0x562cab95cb60 2023-11-16T16:39:31.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.625+0000 7fc17f7fe700 1 -- 172.21.15.1:0/4265375608 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc18809f2a0 con 0x562cab95cb60 2023-11-16T16:39:31.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.625+0000 7fc17f7fe700 1 -- 172.21.15.1:0/4265375608 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18809b730 con 0x562cab95cb60 2023-11-16T16:39:31.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.625+0000 7fc1a715cb80 1 -- 172.21.15.1:0/4265375608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562cab95cb60 msgr2=0x562cab957c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:31.636 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.625+0000 7fc1a715cb80 1 --2- 172.21.15.1:0/4265375608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562cab95cb60 0x562cab957c40 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc18809b200 tx=0x7fc188009ca0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.636 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.625+0000 7fc1a715cb80 1 -- 172.21.15.1:0/4265375608 shutdown_connections 2023-11-16T16:39:31.636 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.625+0000 7fc1a715cb80 1 --2- 172.21.15.1:0/4265375608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cab9618b0 0x562cab968b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.636 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.625+0000 7fc1a715cb80 1 --2- 172.21.15.1:0/4265375608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562cab95cb60 0x562cab957c40 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.636 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.625+0000 7fc1a715cb80 1 --2- 172.21.15.1:0/4265375608 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562cab95d470 0x562cab96d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.636 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.625+0000 7fc1a715cb80 1 -- 172.21.15.1:0/4265375608 >> 172.21.15.1:0/4265375608 conn(0x562cab952360 msgr2=0x562cab972b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:31.636 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.625+0000 7fc1a715cb80 1 -- 172.21.15.1:0/4265375608 shutdown_connections 2023-11-16T16:39:31.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.625+0000 7fc1a715cb80 1 -- 172.21.15.1:0/4265375608 wait complete. 2023-11-16T16:39:31.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.625+0000 7fc1a715cb80 1 -- start start 2023-11-16T16:39:31.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.626+0000 7fc1a715cb80 1 Processor -- start 2023-11-16T16:39:31.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.626+0000 7fc1a715cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562cab95cb60 0x562cab96b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.626+0000 7fc1a715cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cab95d470 0x562cab96b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.626+0000 7fc1a715cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562cab9618b0 0x562cab96bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.626+0000 7fc1a715cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cab9c9740 con 0x562cab95cb60 2023-11-16T16:39:31.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.626+0000 7fc1a715cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cab9c98b0 con 0x562cab95d470 2023-11-16T16:39:31.638 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.626+0000 7fc1a715cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cab9c9a70 con 0x562cab9618b0 2023-11-16T16:39:31.638 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.626+0000 7fc1a715cb80 10 client.0 _ll_register_callbacks cb 0x7fc18000edf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:39:31.638 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.626+0000 7fc198d6c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562cab95cb60 0x562cab96b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.638 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.626+0000 7fc198d6c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562cab95cb60 0x562cab96b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43884/0 (socket says 172.21.15.1:43884) 2023-11-16T16:39:31.638 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.626+0000 7fc198d6c700 1 -- 172.21.15.1:0/2644832098 learned_addr learned my addr 172.21.15.1:0/2644832098 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:31.638 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.626+0000 7fc193fff700 1 --2- 172.21.15.1:0/2644832098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cab95d470 0x562cab96b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.639 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.626+0000 7fc19956d700 1 --2- 172.21.15.1:0/2644832098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562cab9618b0 0x562cab96bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.639 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.626+0000 7fc198d6c700 1 -- 172.21.15.1:0/2644832098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562cab9618b0 msgr2=0x562cab96bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:31.639 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.626+0000 7fc198d6c700 1 --2- 172.21.15.1:0/2644832098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562cab9618b0 0x562cab96bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.639 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.626+0000 7fc198d6c700 1 -- 172.21.15.1:0/2644832098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cab95d470 msgr2=0x562cab96b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:31.639 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.626+0000 7fc198d6c700 1 --2- 172.21.15.1:0/2644832098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cab95d470 0x562cab96b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.639 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.626+0000 7fc198d6c700 1 -- 172.21.15.1:0/2644832098 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc194110160 con 0x562cab95cb60 2023-11-16T16:39:31.639 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.627+0000 7fc198d6c700 1 --2- 172.21.15.1:0/2644832098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562cab95cb60 0x562cab96b240 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc188000c00 tx=0x7fc1880cb8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d9794193e9a4c4a server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:31.640 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.627+0000 7fc173fff700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:39:31.640 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.627+0000 7fc173fff700 1 -- 172.21.15.1:0/2644832098 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18809a960 con 0x562cab95cb60 2023-11-16T16:39:31.640 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.627+0000 7fc173fff700 1 -- 172.21.15.1:0/2644832098 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc1880a3630 con 0x562cab95cb60 2023-11-16T16:39:31.640 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.627+0000 7fc1a715cb80 1 -- 172.21.15.1:0/2644832098 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc18c014c20 con 0x562cab95cb60 2023-11-16T16:39:31.640 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.627+0000 7fc173fff700 1 -- 172.21.15.1:0/2644832098 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc188099720 con 0x562cab95cb60 2023-11-16T16:39:31.640 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.627+0000 7fc1a715cb80 10 client.15588 fetch_fsmap learned FSMap version 23 2023-11-16T16:39:31.640 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.627+0000 7fc1a715cb80 1 -- 172.21.15.1:0/2644832098 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x562cab6dbde0 con 0x562cab95cb60 2023-11-16T16:39:31.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.627+0000 7fc173fff700 1 -- 172.21.15.1:0/2644832098 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1880cb030 con 0x562cab95cb60 2023-11-16T16:39:31.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.627+0000 7fc173fff700 1 -- 172.21.15.1:0/2644832098 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc188022720 con 0x562cab95cb60 2023-11-16T16:39:31.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.627+0000 7fc1a715cb80 10 client.15588 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:39:31.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.627+0000 7fc1a715cb80 10 client.15588 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:39:31.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.627+0000 7fc1a715cb80 20 client.15588 populate_metadata read hostname 'smithi001' 2023-11-16T16:39:31.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.627+0000 7fc1a715cb80 1 --2- 172.21.15.1:0/2644832098 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x562cab9cac10 0x562cab9ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:31.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.627+0000 7fc1a715cb80 4 client.15588 mds_command: new command op to 24521 tid=0[{"prefix": "damage ls"}] 2023-11-16T16:39:31.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.627+0000 7fc1a715cb80 1 -- 172.21.15.1:0/2644832098 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562cab87c660 con 0x562cab9cac10 2023-11-16T16:39:31.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.628+0000 7fc193fff700 1 --2- 172.21.15.1:0/2644832098 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x562cab9cac10 0x562cab9ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:31.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.628+0000 7fc173fff700 10 client.15588 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:39:31.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.628+0000 7fc193fff700 1 --2- 172.21.15.1:0/2644832098 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x562cab9cac10 0x562cab9ccff0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11606939dbb76c3e server_cookie=8fd32f902464ceb2 in_seq=0 out_seq=0 2023-11-16T16:39:31.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.628+0000 7fc173fff700 1 -- 172.21.15.1:0/2644832098 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562cab87c660 con 0x562cab9cac10 2023-11-16T16:39:31.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.628+0000 7fc173fff700 10 client.15588 handle_command_reply: tid=0 2023-11-16T16:39:31.643 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.628+0000 7fc1a715cb80 1 client.15588 shutdown 2023-11-16T16:39:31.643 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.628+0000 7fc1a715cb80 1 -- 172.21.15.1:0/2644832098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562cab95cb60 msgr2=0x562cab96b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:31.643 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.628+0000 7fc1a715cb80 1 --2- 172.21.15.1:0/2644832098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562cab95cb60 0x562cab96b240 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc188000c00 tx=0x7fc1880cb8b0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.643 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.628+0000 7fc1a715cb80 1 -- 172.21.15.1:0/2644832098 shutdown_connections 2023-11-16T16:39:31.643 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.628+0000 7fc1a715cb80 1 --2- 172.21.15.1:0/2644832098 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x562cab9cac10 0x562cab9ccff0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.643 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.628+0000 7fc1a715cb80 1 --2- 172.21.15.1:0/2644832098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562cab9618b0 0x562cab96bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.643 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.628+0000 7fc173fff700 0 client.15588 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:39:31.644 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.628+0000 7fc1a715cb80 1 --2- 172.21.15.1:0/2644832098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562cab95cb60 0x562cab96b240 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.644 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.628+0000 7fc1a715cb80 1 --2- 172.21.15.1:0/2644832098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cab95d470 0x562cab96b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.644 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.628+0000 7fc1a715cb80 1 -- 172.21.15.1:0/2644832098 >> 172.21.15.1:0/2644832098 conn(0x562cab952360 msgr2=0x562cab960740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:31.644 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.628+0000 7fc1a715cb80 1 -- 172.21.15.1:0/2644832098 shutdown_connections 2023-11-16T16:39:31.644 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.628+0000 7fc1a715cb80 1 -- 172.21.15.1:0/2644832098 wait complete. 2023-11-16T16:39:31.644 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.629+0000 7fc1a715cb80 20 client.15588 trim_cache size 0 max 16384 2023-11-16T16:39:31.645 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2023-11-16T16:39:31.646 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.629+0000 7fc19afd0700 1 -- 172.21.15.1:0/3958608688 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fc1840a13d0 msgr2=0x7fc1840a3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:31.646 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.629+0000 7fc19afd0700 1 --2- 172.21.15.1:0/3958608688 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fc1840a13d0 0x7fc1840a3880 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc194197a20 tx=0x7fc180005f50 comp rx=0 tx=0).stop 2023-11-16T16:39:31.646 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.629+0000 7fc19afd0700 1 -- 172.21.15.1:0/3958608688 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1941014b0 msgr2=0x7fc194196410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:31.646 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.629+0000 7fc19afd0700 1 --2- 172.21.15.1:0/3958608688 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1941014b0 0x7fc194196410 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc1880095c0 tx=0x7fc18800f710 comp rx=0 tx=0).stop 2023-11-16T16:39:31.647 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.629+0000 7fc19afd0700 1 -- 172.21.15.1:0/3958608688 shutdown_connections 2023-11-16T16:39:31.647 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.629+0000 7fc19afd0700 1 --2- 172.21.15.1:0/3958608688 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fc1840a13d0 0x7fc1840a3880 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.647 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.629+0000 7fc19afd0700 1 --2- 172.21.15.1:0/3958608688 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc19410a880 0x7fc19419adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.647 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.629+0000 7fc19afd0700 1 --2- 172.21.15.1:0/3958608688 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1941014b0 0x7fc194196410 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.647 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.629+0000 7fc19afd0700 1 --2- 172.21.15.1:0/3958608688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc194101dc0 0x7fc194196950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:31.647 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.629+0000 7fc19afd0700 1 -- 172.21.15.1:0/3958608688 >> 172.21.15.1:0/3958608688 conn(0x7fc1940763a0 msgr2=0x7fc194102250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:31.647 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.629+0000 7fc19afd0700 1 -- 172.21.15.1:0/3958608688 shutdown_connections 2023-11-16T16:39:31.648 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:31.629+0000 7fc19afd0700 1 -- 172.21.15.1:0/3958608688 wait complete. 2023-11-16T16:39:31.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:31 smithi026 ceph-mon[100112]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:39:31.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:31 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/3418598255' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:39:32.648 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-16T16:39:32.649 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-16T16:39:32.796 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.792+0000 7f9a1f471700 1 -- 172.21.15.1:0/2940901055 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a18104100 msgr2=0x7f9a181044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:32.796 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.792+0000 7f9a1f471700 1 --2- 172.21.15.1:0/2940901055 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a18104100 0x7f9a181044d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9a140099c0 tx=0x7f9a14009cd0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.796 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.792+0000 7f9a1f471700 1 -- 172.21.15.1:0/2940901055 shutdown_connections 2023-11-16T16:39:32.796 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.792+0000 7f9a1f471700 1 --2- 172.21.15.1:0/2940901055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a181002e0 0x7f9a1810bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.796 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.792+0000 7f9a1f471700 1 --2- 172.21.15.1:0/2940901055 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a18104100 0x7f9a181044d0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.796 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.792+0000 7f9a1f471700 1 --2- 172.21.15.1:0/2940901055 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a180ff930 0x7f9a180ffda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.792+0000 7f9a1f471700 1 -- 172.21.15.1:0/2940901055 >> 172.21.15.1:0/2940901055 conn(0x7f9a18075880 msgr2=0x7f9a18075c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:32.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.792+0000 7f9a1f471700 1 -- 172.21.15.1:0/2940901055 shutdown_connections 2023-11-16T16:39:32.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.792+0000 7f9a1f471700 1 -- 172.21.15.1:0/2940901055 wait complete. 2023-11-16T16:39:32.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.792+0000 7f9a1f471700 1 Processor -- start 2023-11-16T16:39:32.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.793+0000 7f9a1f471700 1 -- start start 2023-11-16T16:39:32.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.793+0000 7f9a1f471700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a180ff930 0x7f9a18197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:32.798 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.793+0000 7f9a1f471700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a181002e0 0x7f9a18197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:32.798 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.793+0000 7f9a1f471700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a18104100 0x7f9a181a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:32.798 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.793+0000 7f9a1f471700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a181a2800 con 0x7f9a180ff930 2023-11-16T16:39:32.798 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.793+0000 7f9a1f471700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a181a2940 con 0x7f9a18104100 2023-11-16T16:39:32.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.793+0000 7f9a1f471700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a181a2a80 con 0x7f9a181002e0 2023-11-16T16:39:32.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.793+0000 7f9a1d20d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a180ff930 0x7f9a18197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:32.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.793+0000 7f9a1d20d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a180ff930 0x7f9a18197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43898/0 (socket says 172.21.15.1:43898) 2023-11-16T16:39:32.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.793+0000 7f9a1d20d700 1 -- 172.21.15.1:0/1349722737 learned_addr learned my addr 172.21.15.1:0/1349722737 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:32.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.793+0000 7f9a1ca0c700 1 --2- 172.21.15.1:0/1349722737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a181002e0 0x7f9a18197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:32.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.793+0000 7f9a1d20d700 1 -- 172.21.15.1:0/1349722737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a181002e0 msgr2=0x7f9a18197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:32.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.793+0000 7f9a1da0e700 1 --2- 172.21.15.1:0/1349722737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a18104100 0x7f9a181a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:32.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.793+0000 7f9a1d20d700 1 --2- 172.21.15.1:0/1349722737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a181002e0 0x7f9a18197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.793+0000 7f9a1d20d700 1 -- 172.21.15.1:0/1349722737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a18104100 msgr2=0x7f9a181a22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:32.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.793+0000 7f9a1d20d700 1 --2- 172.21.15.1:0/1349722737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a18104100 0x7f9a181a22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.793+0000 7f9a1d20d700 1 -- 172.21.15.1:0/1349722737 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a140095f0 con 0x7f9a180ff930 2023-11-16T16:39:32.802 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.793+0000 7f9a1d20d700 1 --2- 172.21.15.1:0/1349722737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a180ff930 0x7f9a18197240 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9a1400b580 tx=0x7f9a1400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:32.802 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.794+0000 7f9a0e7fc700 1 -- 172.21.15.1:0/1349722737 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1401d070 con 0x7f9a180ff930 2023-11-16T16:39:32.802 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.794+0000 7f9a0e7fc700 1 -- 172.21.15.1:0/1349722737 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a140054f0 con 0x7f9a180ff930 2023-11-16T16:39:32.802 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.794+0000 7f9a1f471700 1 -- 172.21.15.1:0/1349722737 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a1810f2b0 con 0x7f9a180ff930 2023-11-16T16:39:32.803 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.794+0000 7f9a0e7fc700 1 -- 172.21.15.1:0/1349722737 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a14004bd0 con 0x7f9a180ff930 2023-11-16T16:39:32.803 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.794+0000 7f9a1f471700 1 -- 172.21.15.1:0/1349722737 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a1810f120 con 0x7f9a180ff930 2023-11-16T16:39:32.810 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.796+0000 7f9a0e7fc700 1 -- 172.21.15.1:0/1349722737 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 23) v1 ==== 135890+0+0 (secure 0 0 0) 0x7f9a1400fc30 con 0x7f9a180ff930 2023-11-16T16:39:32.810 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.796+0000 7f9a0e7fc700 1 --2- 172.21.15.1:0/1349722737 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f9a040a1350 0x7f9a040a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:32.810 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.797+0000 7f9a0e7fc700 1 -- 172.21.15.1:0/1349722737 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f9a140cdcf0 con 0x7f9a180ff930 2023-11-16T16:39:32.810 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.797+0000 7f9a1ca0c700 1 --2- 172.21.15.1:0/1349722737 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f9a040a1350 0x7f9a040a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:32.810 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.797+0000 7f9a1ca0c700 1 --2- 172.21.15.1:0/1349722737 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f9a040a1350 0x7f9a040a3800 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9a180ff900 tx=0x7f9a08009260 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:32.810 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.797+0000 7f9a2b5fdb80 1 Processor -- start 2023-11-16T16:39:32.810 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.798+0000 7f9a2b5fdb80 1 -- start start 2023-11-16T16:39:32.810 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.798+0000 7f9a2b5fdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573a90acff0 0x5573a90ad3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:32.810 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.798+0000 7f9a2b5fdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573a90ad900 0x5573a90b1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:32.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.798+0000 7f9a2b5fdb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573a90b1d40 0x5573a90b2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:32.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.798+0000 7f9a2b5fdb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573a8fae0e0 con 0x5573a90b1d40 2023-11-16T16:39:32.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.798+0000 7f9a2b5fdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573a9000920 con 0x5573a90ad900 2023-11-16T16:39:32.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.798+0000 7f9a2b5fdb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573a8ff9db0 con 0x5573a90acff0 2023-11-16T16:39:32.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.798+0000 7f9a1da0e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573a90b1d40 0x5573a90b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:32.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.798+0000 7f9a1da0e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573a90b1d40 0x5573a90b2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43910/0 (socket says 172.21.15.1:43910) 2023-11-16T16:39:32.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.798+0000 7f9a1da0e700 1 -- 172.21.15.1:0/463353265 learned_addr learned my addr 172.21.15.1:0/463353265 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:32.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.798+0000 7f9a1ca0c700 1 --2- 172.21.15.1:0/463353265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573a90ad900 0x5573a90b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:32.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.798+0000 7f9a1d20d700 1 --2- 172.21.15.1:0/463353265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573a90acff0 0x5573a90ad3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:32.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.798+0000 7f9a1da0e700 1 -- 172.21.15.1:0/463353265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573a90acff0 msgr2=0x5573a90ad3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:32.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.798+0000 7f9a1da0e700 1 --2- 172.21.15.1:0/463353265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573a90acff0 0x5573a90ad3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.798+0000 7f9a1da0e700 1 -- 172.21.15.1:0/463353265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573a90ad900 msgr2=0x5573a90b1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:32.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.798+0000 7f9a1da0e700 1 --2- 172.21.15.1:0/463353265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573a90ad900 0x5573a90b1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.798+0000 7f9a1da0e700 1 -- 172.21.15.1:0/463353265 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573a8e7b6b0 con 0x5573a90b1d40 2023-11-16T16:39:32.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.798+0000 7f9a1da0e700 1 --2- 172.21.15.1:0/463353265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573a90b1d40 0x5573a90b2190 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9a1000d1d0 tx=0x7f9a1000d4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d7d9b93ad1a2d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:32.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.799+0000 7f9a037fe700 1 -- 172.21.15.1:0/463353265 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a100123f0 con 0x5573a90b1d40 2023-11-16T16:39:32.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.799+0000 7f9a037fe700 1 -- 172.21.15.1:0/463353265 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a10012a10 con 0x5573a90b1d40 2023-11-16T16:39:32.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.799+0000 7f9a037fe700 1 -- 172.21.15.1:0/463353265 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a100119d0 con 0x5573a90b1d40 2023-11-16T16:39:32.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.799+0000 7f9a2b5fdb80 1 -- 172.21.15.1:0/463353265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573a90b1d40 msgr2=0x5573a90b2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:32.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.799+0000 7f9a2b5fdb80 1 --2- 172.21.15.1:0/463353265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573a90b1d40 0x5573a90b2190 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9a1000d1d0 tx=0x7f9a1000d4e0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.799+0000 7f9a2b5fdb80 1 -- 172.21.15.1:0/463353265 shutdown_connections 2023-11-16T16:39:32.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.799+0000 7f9a2b5fdb80 1 --2- 172.21.15.1:0/463353265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573a90b1d40 0x5573a90b2190 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.799+0000 7f9a2b5fdb80 1 --2- 172.21.15.1:0/463353265 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573a90acff0 0x5573a90ad3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.799+0000 7f9a2b5fdb80 1 --2- 172.21.15.1:0/463353265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573a90ad900 0x5573a90b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.799+0000 7f9a2b5fdb80 1 -- 172.21.15.1:0/463353265 >> 172.21.15.1:0/463353265 conn(0x5573a909e3e0 msgr2=0x5573a90a3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:32.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.799+0000 7f9a2b5fdb80 1 -- 172.21.15.1:0/463353265 shutdown_connections 2023-11-16T16:39:32.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.800+0000 7f9a2b5fdb80 1 -- 172.21.15.1:0/463353265 wait complete. 2023-11-16T16:39:32.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.800+0000 7f9a2b5fdb80 1 -- start start 2023-11-16T16:39:32.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.800+0000 7f9a2b5fdb80 1 Processor -- start 2023-11-16T16:39:32.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.800+0000 7f9a2b5fdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573a90acff0 0x5573a90b94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:32.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.800+0000 7f9a2b5fdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573a90ad900 0x5573a90b6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:32.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.800+0000 7f9a2b5fdb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573a90b1d40 0x5573a90b6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:32.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.800+0000 7f9a2b5fdb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573a8f9b960 con 0x5573a90b1d40 2023-11-16T16:39:32.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.800+0000 7f9a2b5fdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573a8f9b600 con 0x5573a90acff0 2023-11-16T16:39:32.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.800+0000 7f9a2b5fdb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573a8e7d9a0 con 0x5573a90ad900 2023-11-16T16:39:32.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.801+0000 7f9a2b5fdb80 10 client.0 _ll_register_callbacks cb 0x5573a8fb9800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:39:32.814 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.801+0000 7f9a1da0e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573a90b1d40 0x5573a90b6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:32.814 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.801+0000 7f9a1da0e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573a90b1d40 0x5573a90b6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43916/0 (socket says 172.21.15.1:43916) 2023-11-16T16:39:32.814 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.801+0000 7f9a1da0e700 1 -- 172.21.15.1:0/1857636633 learned_addr learned my addr 172.21.15.1:0/1857636633 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:32.814 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.801+0000 7f9a1d20d700 1 --2- 172.21.15.1:0/1857636633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573a90acff0 0x5573a90b94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:32.814 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.801+0000 7f9a1ca0c700 1 --2- 172.21.15.1:0/1857636633 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573a90ad900 0x5573a90b6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:32.814 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.801+0000 7f9a1da0e700 1 -- 172.21.15.1:0/1857636633 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573a90ad900 msgr2=0x5573a90b6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:32.814 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.801+0000 7f9a1da0e700 1 --2- 172.21.15.1:0/1857636633 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573a90ad900 0x5573a90b6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.814 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.801+0000 7f9a1da0e700 1 -- 172.21.15.1:0/1857636633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573a90acff0 msgr2=0x5573a90b94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:32.814 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.801+0000 7f9a1da0e700 1 --2- 172.21.15.1:0/1857636633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573a90acff0 0x5573a90b94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.814 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.801+0000 7f9a1da0e700 1 -- 172.21.15.1:0/1857636633 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a10007cd0 con 0x5573a90b1d40 2023-11-16T16:39:32.814 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.801+0000 7f9a1da0e700 1 --2- 172.21.15.1:0/1857636633 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573a90b1d40 0x5573a90b6dd0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9a10009010 tx=0x7f9a10012f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=141761e545db131a server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:32.814 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.801+0000 7f9a00ff9700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:39:32.815 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.801+0000 7f9a00ff9700 1 -- 172.21.15.1:0/1857636633 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a10012680 con 0x5573a90b1d40 2023-11-16T16:39:32.815 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.801+0000 7f9a00ff9700 1 -- 172.21.15.1:0/1857636633 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f9a100127e0 con 0x5573a90b1d40 2023-11-16T16:39:32.815 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.801+0000 7f9a2b5fdb80 1 -- 172.21.15.1:0/1857636633 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5573a909bba0 con 0x5573a90b1d40 2023-11-16T16:39:32.815 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.801+0000 7f9a00ff9700 1 -- 172.21.15.1:0/1857636633 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a10004b60 con 0x5573a90b1d40 2023-11-16T16:39:32.815 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.801+0000 7f9a2b5fdb80 10 client.15600 fetch_fsmap learned FSMap version 23 2023-11-16T16:39:32.815 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.801+0000 7f9a2b5fdb80 1 -- 172.21.15.1:0/1857636633 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5573a8fc0210 con 0x5573a90b1d40 2023-11-16T16:39:32.815 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.802+0000 7f9a00ff9700 1 -- 172.21.15.1:0/1857636633 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a10032020 con 0x5573a90b1d40 2023-11-16T16:39:32.815 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.802+0000 7f9a00ff9700 1 -- 172.21.15.1:0/1857636633 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9a10005030 con 0x5573a90b1d40 2023-11-16T16:39:32.815 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.802+0000 7f9a2b5fdb80 10 client.15600 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:39:32.815 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.802+0000 7f9a2b5fdb80 10 client.15600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:39:32.815 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.802+0000 7f9a2b5fdb80 20 client.15600 populate_metadata read hostname 'smithi001' 2023-11-16T16:39:32.816 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.802+0000 7f9a2b5fdb80 1 --2- 172.21.15.1:0/1857636633 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5573a90c6c90 0x5573a90c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:32.816 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.802+0000 7f9a2b5fdb80 4 client.15600 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:39:32.816 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.802+0000 7f9a2b5fdb80 1 -- 172.21.15.1:0/1857636633 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5573a8fbc120 con 0x5573a90c6c90 2023-11-16T16:39:32.816 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.802+0000 7f9a1d20d700 1 --2- 172.21.15.1:0/1857636633 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5573a90c6c90 0x5573a90c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:32.816 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.803+0000 7f9a00ff9700 10 client.15600 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:39:32.816 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.803+0000 7f9a1d20d700 1 --2- 172.21.15.1:0/1857636633 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5573a90c6c90 0x5573a90c90a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ade3432e0f14b4da server_cookie=4d17b3d9e9e3e0b1 in_seq=0 out_seq=0 2023-11-16T16:39:32.816 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.803+0000 7f9a00ff9700 1 -- 172.21.15.1:0/1857636633 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5573a8fbc120 con 0x5573a90c6c90 2023-11-16T16:39:32.816 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.803+0000 7f9a00ff9700 10 client.15600 handle_command_reply: tid=0 2023-11-16T16:39:32.816 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.803+0000 7f9a2b5fdb80 1 client.15600 shutdown 2023-11-16T16:39:32.816 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.803+0000 7f9a2b5fdb80 1 -- 172.21.15.1:0/1857636633 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573a90b1d40 msgr2=0x5573a90b6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:32.816 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.803+0000 7f9a2b5fdb80 1 --2- 172.21.15.1:0/1857636633 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573a90b1d40 0x5573a90b6dd0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9a10009010 tx=0x7f9a10012f60 comp rx=0 tx=0).stop 2023-11-16T16:39:32.816 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.803+0000 7f9a2b5fdb80 1 -- 172.21.15.1:0/1857636633 shutdown_connections 2023-11-16T16:39:32.817 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.803+0000 7f9a2b5fdb80 1 --2- 172.21.15.1:0/1857636633 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5573a90c6c90 0x5573a90c90a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.817 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.803+0000 7f9a2b5fdb80 1 --2- 172.21.15.1:0/1857636633 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573a90b1d40 0x5573a90b6dd0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.817 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.803+0000 7f9a00ff9700 0 client.15600 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:39:32.817 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.803+0000 7f9a2b5fdb80 1 --2- 172.21.15.1:0/1857636633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573a90acff0 0x5573a90b94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.817 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.803+0000 7f9a2b5fdb80 1 --2- 172.21.15.1:0/1857636633 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573a90ad900 0x5573a90b6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.817 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.803+0000 7f9a2b5fdb80 1 -- 172.21.15.1:0/1857636633 >> 172.21.15.1:0/1857636633 conn(0x5573a909e3e0 msgr2=0x5573a90b31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:32.817 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.804+0000 7f9a2b5fdb80 1 -- 172.21.15.1:0/1857636633 shutdown_connections 2023-11-16T16:39:32.817 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.804+0000 7f9a2b5fdb80 1 -- 172.21.15.1:0/1857636633 wait complete. 2023-11-16T16:39:32.817 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.804+0000 7f9a2b5fdb80 20 client.15600 trim_cache size 0 max 16384 2023-11-16T16:39:32.824 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.821+0000 7f9a2b5fdb80 1 Processor -- start 2023-11-16T16:39:32.824 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.821+0000 7f9a2b5fdb80 1 -- start start 2023-11-16T16:39:32.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.821+0000 7f9a2b5fdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573a90acff0 0x5573a90a3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:32.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.821+0000 7f9a2b5fdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573a90ad900 0x5573a90bde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:32.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.821+0000 7f9a2b5fdb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573a90b1d40 0x5573a90b9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:32.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.821+0000 7f9a2b5fdb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573a90ae050 con 0x5573a90b1d40 2023-11-16T16:39:32.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.821+0000 7f9a2b5fdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573a90b7160 con 0x5573a90ad900 2023-11-16T16:39:32.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.821+0000 7f9a2b5fdb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573a90b4cb0 con 0x5573a90acff0 2023-11-16T16:39:32.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.821+0000 7f9a1da0e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573a90b1d40 0x5573a90b9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:32.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.821+0000 7f9a1da0e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573a90b1d40 0x5573a90b9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43924/0 (socket says 172.21.15.1:43924) 2023-11-16T16:39:32.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.821+0000 7f9a1da0e700 1 -- 172.21.15.1:0/3479594262 learned_addr learned my addr 172.21.15.1:0/3479594262 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:32.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.821+0000 7f9a1ca0c700 1 --2- 172.21.15.1:0/3479594262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573a90ad900 0x5573a90bde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:32.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.821+0000 7f9a1da0e700 1 -- 172.21.15.1:0/3479594262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573a90acff0 msgr2=0x5573a90a3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:32.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.821+0000 7f9a1da0e700 1 --2- 172.21.15.1:0/3479594262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573a90acff0 0x5573a90a3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.821+0000 7f9a1da0e700 1 -- 172.21.15.1:0/3479594262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573a90ad900 msgr2=0x5573a90bde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:32.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.821+0000 7f9a1da0e700 1 --2- 172.21.15.1:0/3479594262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573a90ad900 0x5573a90bde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.821+0000 7f9a1da0e700 1 -- 172.21.15.1:0/3479594262 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573a90a28a0 con 0x5573a90b1d40 2023-11-16T16:39:32.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.821+0000 7f9a1da0e700 1 --2- 172.21.15.1:0/3479594262 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573a90b1d40 0x5573a90b9070 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9a100074c0 tx=0x7f9a100060a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=136c8889a73e29d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:32.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.822+0000 7f9a03fff700 1 -- 172.21.15.1:0/3479594262 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a100058c0 con 0x5573a90b1d40 2023-11-16T16:39:32.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.822+0000 7f9a03fff700 1 -- 172.21.15.1:0/3479594262 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a10032070 con 0x5573a90b1d40 2023-11-16T16:39:32.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.822+0000 7f9a03fff700 1 -- 172.21.15.1:0/3479594262 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a100257a0 con 0x5573a90b1d40 2023-11-16T16:39:32.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.822+0000 7f9a2b5fdb80 1 -- 172.21.15.1:0/3479594262 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573a90b1d40 msgr2=0x5573a90b9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:32.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.822+0000 7f9a2b5fdb80 1 --2- 172.21.15.1:0/3479594262 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573a90b1d40 0x5573a90b9070 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9a100074c0 tx=0x7f9a100060a0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.822+0000 7f9a2b5fdb80 1 -- 172.21.15.1:0/3479594262 shutdown_connections 2023-11-16T16:39:32.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.822+0000 7f9a2b5fdb80 1 --2- 172.21.15.1:0/3479594262 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573a90b1d40 0x5573a90b9070 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.822+0000 7f9a2b5fdb80 1 --2- 172.21.15.1:0/3479594262 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573a90acff0 0x5573a90a3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.822+0000 7f9a2b5fdb80 1 --2- 172.21.15.1:0/3479594262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573a90ad900 0x5573a90bde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.822+0000 7f9a2b5fdb80 1 -- 172.21.15.1:0/3479594262 >> 172.21.15.1:0/3479594262 conn(0x5573a909e3e0 msgr2=0x5573a90a3600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:32.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.822+0000 7f9a2b5fdb80 1 -- 172.21.15.1:0/3479594262 shutdown_connections 2023-11-16T16:39:32.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.822+0000 7f9a2b5fdb80 1 -- 172.21.15.1:0/3479594262 wait complete. 2023-11-16T16:39:32.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.823+0000 7f9a2b5fdb80 1 -- start start 2023-11-16T16:39:32.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.823+0000 7f9a2b5fdb80 1 Processor -- start 2023-11-16T16:39:32.831 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.823+0000 7f9a2b5fdb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573a90acff0 0x5573a9114a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:32.831 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.823+0000 7f9a2b5fdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573a90ad900 0x5573a9114f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:32.831 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.823+0000 7f9a2b5fdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573a90b1d40 0x5573a910edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:32.831 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.823+0000 7f9a2b5fdb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573a910f3d0 con 0x5573a90acff0 2023-11-16T16:39:32.831 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.823+0000 7f9a2b5fdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573a910f540 con 0x5573a90ad900 2023-11-16T16:39:32.831 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.823+0000 7f9a2b5fdb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573a910f700 con 0x5573a90b1d40 2023-11-16T16:39:32.831 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.823+0000 7f9a2b5fdb80 10 client.0 _ll_register_callbacks cb 0x7f9a080099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:39:32.831 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.823+0000 7f9a1d20d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573a90acff0 0x5573a9114a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:32.831 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.823+0000 7f9a1d20d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573a90acff0 0x5573a9114a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43926/0 (socket says 172.21.15.1:43926) 2023-11-16T16:39:32.831 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.823+0000 7f9a1d20d700 1 -- 172.21.15.1:0/862550316 learned_addr learned my addr 172.21.15.1:0/862550316 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:32.831 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.823+0000 7f9a1ca0c700 1 --2- 172.21.15.1:0/862550316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573a90ad900 0x5573a9114f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:32.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.823+0000 7f9a1da0e700 1 --2- 172.21.15.1:0/862550316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573a90b1d40 0x5573a910edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:32.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.823+0000 7f9a1d20d700 1 -- 172.21.15.1:0/862550316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573a90b1d40 msgr2=0x5573a910edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:32.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.823+0000 7f9a1d20d700 1 --2- 172.21.15.1:0/862550316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573a90b1d40 0x5573a910edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.824+0000 7f9a1d20d700 1 -- 172.21.15.1:0/862550316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573a90ad900 msgr2=0x5573a9114f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:32.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.824+0000 7f9a1d20d700 1 --2- 172.21.15.1:0/862550316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573a90ad900 0x5573a9114f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.824+0000 7f9a1d20d700 1 -- 172.21.15.1:0/862550316 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5573a8fc0210 con 0x5573a90acff0 2023-11-16T16:39:32.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.824+0000 7f9a1ca0c700 1 --2- 172.21.15.1:0/862550316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573a90ad900 0x5573a9114f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:39:32.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.824+0000 7f9a1da0e700 1 --2- 172.21.15.1:0/862550316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573a90b1d40 0x5573a910edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:39:32.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.824+0000 7f9a1d20d700 1 --2- 172.21.15.1:0/862550316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573a90acff0 0x5573a9114a50 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9a14009fd0 tx=0x7f9a1409ae70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=120419a04e4cde07 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:32.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.824+0000 7f9a00ff9700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:39:32.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.824+0000 7f9a00ff9700 1 -- 172.21.15.1:0/862550316 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1409b150 con 0x5573a90acff0 2023-11-16T16:39:32.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.824+0000 7f9a00ff9700 1 -- 172.21.15.1:0/862550316 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f9a1409a1e0 con 0x5573a90acff0 2023-11-16T16:39:32.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.824+0000 7f9a2b5fdb80 1 -- 172.21.15.1:0/862550316 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9a10012a10 con 0x5573a90acff0 2023-11-16T16:39:32.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.824+0000 7f9a00ff9700 1 -- 172.21.15.1:0/862550316 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a1409a3c0 con 0x5573a90acff0 2023-11-16T16:39:32.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.824+0000 7f9a2b5fdb80 10 client.15606 fetch_fsmap learned FSMap version 23 2023-11-16T16:39:32.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.824+0000 7f9a2b5fdb80 1 -- 172.21.15.1:0/862550316 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5573a8e27e60 con 0x5573a90acff0 2023-11-16T16:39:32.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.824+0000 7f9a00ff9700 1 -- 172.21.15.1:0/862550316 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a140c9080 con 0x5573a90acff0 2023-11-16T16:39:32.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.824+0000 7f9a00ff9700 1 -- 172.21.15.1:0/862550316 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9a140cbc80 con 0x5573a90acff0 2023-11-16T16:39:32.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.824+0000 7f9a2b5fdb80 10 client.15606 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:39:32.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.824+0000 7f9a2b5fdb80 10 client.15606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:39:32.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.824+0000 7f9a2b5fdb80 20 client.15606 populate_metadata read hostname 'smithi001' 2023-11-16T16:39:32.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.824+0000 7f9a2b5fdb80 1 --2- 172.21.15.1:0/862550316 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5573a90c6c90 0x5573a911e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:32.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.824+0000 7f9a2b5fdb80 4 client.15606 mds_command: new command op to 24521 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-16T16:39:32.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.824+0000 7f9a2b5fdb80 1 -- 172.21.15.1:0/862550316 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5573a8fc86e0 con 0x5573a90c6c90 2023-11-16T16:39:32.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.825+0000 7f9a1ca0c700 1 --2- 172.21.15.1:0/862550316 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5573a90c6c90 0x5573a911e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:32.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.825+0000 7f9a00ff9700 10 client.15606 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:39:32.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.825+0000 7f9a1ca0c700 1 --2- 172.21.15.1:0/862550316 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5573a90c6c90 0x5573a911e560 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98f7b1e4f7a403a5 server_cookie=5273ac5c86a48369 in_seq=0 out_seq=0 2023-11-16T16:39:32.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.826+0000 7f9a00ff9700 1 -- 172.21.15.1:0/862550316 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5573a8fc86e0 con 0x5573a90c6c90 2023-11-16T16:39:32.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.826+0000 7f9a00ff9700 10 client.15606 handle_command_reply: tid=0 2023-11-16T16:39:32.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.826+0000 7f9a2b5fdb80 1 client.15606 shutdown 2023-11-16T16:39:32.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.827+0000 7f9a2b5fdb80 1 -- 172.21.15.1:0/862550316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573a90acff0 msgr2=0x5573a9114a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:32.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.827+0000 7f9a2b5fdb80 1 --2- 172.21.15.1:0/862550316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573a90acff0 0x5573a9114a50 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9a14009fd0 tx=0x7f9a1409ae70 comp rx=0 tx=0).stop 2023-11-16T16:39:32.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.827+0000 7f9a2b5fdb80 1 -- 172.21.15.1:0/862550316 shutdown_connections 2023-11-16T16:39:32.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.827+0000 7f9a2b5fdb80 1 --2- 172.21.15.1:0/862550316 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5573a90c6c90 0x5573a911e560 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.827+0000 7f9a2b5fdb80 1 --2- 172.21.15.1:0/862550316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573a90b1d40 0x5573a910edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.827+0000 7f9a2b5fdb80 1 --2- 172.21.15.1:0/862550316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573a90acff0 0x5573a9114a50 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.827+0000 7f9a2b5fdb80 1 --2- 172.21.15.1:0/862550316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573a90ad900 0x5573a9114f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.827+0000 7f9a00ff9700 0 client.15606 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:39:32.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.827+0000 7f9a2b5fdb80 1 -- 172.21.15.1:0/862550316 >> 172.21.15.1:0/862550316 conn(0x5573a909e3e0 msgr2=0x5573a90b0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:32.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.827+0000 7f9a2b5fdb80 1 -- 172.21.15.1:0/862550316 shutdown_connections 2023-11-16T16:39:32.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.827+0000 7f9a2b5fdb80 1 -- 172.21.15.1:0/862550316 wait complete. 2023-11-16T16:39:32.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.827+0000 7f9a2b5fdb80 20 client.15606 trim_cache size 0 max 16384 2023-11-16T16:39:32.835 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:39:32.835 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2023-11-16T16:39:32.835 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "3dd6b4ce-2833-4a83-93bf-28b49034e19c", 2023-11-16T16:39:32.836 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2023-11-16T16:39:32.836 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:39:32.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.827+0000 7f9a1f471700 1 -- 172.21.15.1:0/1349722737 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f9a040a1350 msgr2=0x7f9a040a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:32.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.827+0000 7f9a1f471700 1 --2- 172.21.15.1:0/1349722737 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f9a040a1350 0x7f9a040a3800 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9a180ff900 tx=0x7f9a08009260 comp rx=0 tx=0).stop 2023-11-16T16:39:32.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.827+0000 7f9a1f471700 1 -- 172.21.15.1:0/1349722737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a180ff930 msgr2=0x7f9a18197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:32.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.827+0000 7f9a1f471700 1 --2- 172.21.15.1:0/1349722737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a180ff930 0x7f9a18197240 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9a1400b580 tx=0x7f9a1400f740 comp rx=0 tx=0).stop 2023-11-16T16:39:32.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.828+0000 7f9a1f471700 1 -- 172.21.15.1:0/1349722737 shutdown_connections 2023-11-16T16:39:32.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.828+0000 7f9a1f471700 1 --2- 172.21.15.1:0/1349722737 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f9a040a1350 0x7f9a040a3800 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.828+0000 7f9a1f471700 1 --2- 172.21.15.1:0/1349722737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a18104100 0x7f9a181a22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.828+0000 7f9a1f471700 1 --2- 172.21.15.1:0/1349722737 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a180ff930 0x7f9a18197240 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.828+0000 7f9a1f471700 1 --2- 172.21.15.1:0/1349722737 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9a181002e0 0x7f9a18197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.828+0000 7f9a1f471700 1 -- 172.21.15.1:0/1349722737 >> 172.21.15.1:0/1349722737 conn(0x7f9a18075880 msgr2=0x7f9a180fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:32.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.828+0000 7f9a1f471700 1 -- 172.21.15.1:0/1349722737 shutdown_connections 2023-11-16T16:39:32.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.828+0000 7f9a1f471700 1 -- 172.21.15.1:0/1349722737 wait complete. 2023-11-16T16:39:32.842 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-16T16:39:32.971 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.967+0000 7f013ec89700 1 -- 172.21.15.1:0/1012976367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01380ff410 msgr2=0x7f01380ff7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:32.972 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.967+0000 7f013ec89700 1 --2- 172.21.15.1:0/1012976367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01380ff410 0x7f01380ff7e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f012c0099c0 tx=0x7f012c009cd0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.972 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.968+0000 7f013ec89700 1 -- 172.21.15.1:0/1012976367 shutdown_connections 2023-11-16T16:39:32.972 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.968+0000 7f013ec89700 1 --2- 172.21.15.1:0/1012976367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0138104450 0x7f013810c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.972 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.968+0000 7f013ec89700 1 --2- 172.21.15.1:0/1012976367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01380ff410 0x7f01380ff7e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.972 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.968+0000 7f013ec89700 1 --2- 172.21.15.1:0/1012976367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01380ffdb0 0x7f0138103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.972 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.968+0000 7f013ec89700 1 -- 172.21.15.1:0/1012976367 >> 172.21.15.1:0/1012976367 conn(0x7f0138076370 msgr2=0x7f0138076770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:32.972 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.968+0000 7f013ec89700 1 -- 172.21.15.1:0/1012976367 shutdown_connections 2023-11-16T16:39:32.972 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.968+0000 7f013ec89700 1 -- 172.21.15.1:0/1012976367 wait complete. 2023-11-16T16:39:32.972 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.968+0000 7f013ec89700 1 Processor -- start 2023-11-16T16:39:32.972 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.968+0000 7f013ec89700 1 -- start start 2023-11-16T16:39:32.972 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.968+0000 7f013ec89700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01380ff410 0x7f0138100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:32.972 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.968+0000 7f013ec89700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01380ffdb0 0x7f0138101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:32.973 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.968+0000 7f013ec89700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0138104450 0x7f01381a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:32.973 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.968+0000 7f013ec89700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0138101b80 con 0x7f01380ff410 2023-11-16T16:39:32.973 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.968+0000 7f013ec89700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01381a4b40 con 0x7f01380ffdb0 2023-11-16T16:39:32.973 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.968+0000 7f013ec89700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01381a4c80 con 0x7f0138104450 2023-11-16T16:39:32.973 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.969+0000 7f013ca25700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01380ff410 0x7f0138100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:32.973 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.969+0000 7f013ca25700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01380ff410 0x7f0138100e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43940/0 (socket says 172.21.15.1:43940) 2023-11-16T16:39:32.973 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.969+0000 7f013ca25700 1 -- 172.21.15.1:0/1399683851 learned_addr learned my addr 172.21.15.1:0/1399683851 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:32.973 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.969+0000 7f013d226700 1 --2- 172.21.15.1:0/1399683851 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0138104450 0x7f01381a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:32.973 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.969+0000 7f0137fff700 1 --2- 172.21.15.1:0/1399683851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01380ffdb0 0x7f0138101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:32.973 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.969+0000 7f013ca25700 1 -- 172.21.15.1:0/1399683851 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0138104450 msgr2=0x7f01381a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:32.974 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.969+0000 7f013ca25700 1 --2- 172.21.15.1:0/1399683851 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0138104450 0x7f01381a4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.974 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.969+0000 7f013ca25700 1 -- 172.21.15.1:0/1399683851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01380ffdb0 msgr2=0x7f0138101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:32.974 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.969+0000 7f013ca25700 1 --2- 172.21.15.1:0/1399683851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01380ffdb0 0x7f0138101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.974 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.969+0000 7f013ca25700 1 -- 172.21.15.1:0/1399683851 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f012c0095f0 con 0x7f01380ff410 2023-11-16T16:39:32.974 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.969+0000 7f013ca25700 1 --2- 172.21.15.1:0/1399683851 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01380ff410 0x7f0138100e10 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f012c009990 tx=0x7f012c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:32.974 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.969+0000 7f0135ffb700 1 -- 172.21.15.1:0/1399683851 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c01d070 con 0x7f01380ff410 2023-11-16T16:39:32.974 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.969+0000 7f0135ffb700 1 -- 172.21.15.1:0/1399683851 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f012c004010 con 0x7f01380ff410 2023-11-16T16:39:32.974 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.969+0000 7f013ec89700 1 -- 172.21.15.1:0/1399683851 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f013810f210 con 0x7f01380ff410 2023-11-16T16:39:32.974 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.969+0000 7f0135ffb700 1 -- 172.21.15.1:0/1399683851 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c022d40 con 0x7f01380ff410 2023-11-16T16:39:32.974 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.970+0000 7f013ec89700 1 -- 172.21.15.1:0/1399683851 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f013810f080 con 0x7f01380ff410 2023-11-16T16:39:32.977 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.971+0000 7f0135ffb700 1 -- 172.21.15.1:0/1399683851 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 23) v1 ==== 135890+0+0 (secure 0 0 0) 0x7f012c022470 con 0x7f01380ff410 2023-11-16T16:39:32.977 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.972+0000 7f0135ffb700 1 --2- 172.21.15.1:0/1399683851 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f01280a1350 0x7f01280a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:32.977 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.972+0000 7f0135ffb700 1 -- 172.21.15.1:0/1399683851 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f012c0ce2b0 con 0x7f01380ff410 2023-11-16T16:39:32.978 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.972+0000 7f0137fff700 1 --2- 172.21.15.1:0/1399683851 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f01280a1350 0x7f01280a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:32.978 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.973+0000 7f0137fff700 1 --2- 172.21.15.1:0/1399683851 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f01280a1350 0x7f01280a3800 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0124005950 tx=0x7f01240058e0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:32.978 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.973+0000 7f014ae15b80 1 Processor -- start 2023-11-16T16:39:32.978 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.973+0000 7f014ae15b80 1 -- start start 2023-11-16T16:39:32.978 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.973+0000 7f014ae15b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560392fd8b60 0x560392fd8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:32.978 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.973+0000 7f014ae15b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560392fd9470 0x560392fdd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:32.978 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.973+0000 7f014ae15b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560392fdd8b0 0x560392fddd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:32.978 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.973+0000 7f014ae15b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560392ede060 con 0x560392fd9470 2023-11-16T16:39:32.978 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.973+0000 7f014ae15b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560392f308a0 con 0x560392fdd8b0 2023-11-16T16:39:32.979 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.973+0000 7f014ae15b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560392f29d30 con 0x560392fd8b60 2023-11-16T16:39:32.979 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.974+0000 7f0137fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560392fd9470 0x560392fdd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:32.979 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.974+0000 7f0137fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560392fd9470 0x560392fdd370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43950/0 (socket says 172.21.15.1:43950) 2023-11-16T16:39:32.979 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.974+0000 7f0137fff700 1 -- 172.21.15.1:0/19399513 learned_addr learned my addr 172.21.15.1:0/19399513 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:32.979 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.974+0000 7f013ca25700 1 --2- 172.21.15.1:0/19399513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560392fd8b60 0x560392fd8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:32.979 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.974+0000 7f013d226700 1 --2- 172.21.15.1:0/19399513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560392fdd8b0 0x560392fddd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:32.979 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.974+0000 7f0137fff700 1 -- 172.21.15.1:0/19399513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560392fd8b60 msgr2=0x560392fd8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:32.979 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.974+0000 7f0137fff700 1 --2- 172.21.15.1:0/19399513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560392fd8b60 0x560392fd8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.979 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.974+0000 7f0137fff700 1 -- 172.21.15.1:0/19399513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560392fdd8b0 msgr2=0x560392fddd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:32.979 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.974+0000 7f0137fff700 1 --2- 172.21.15.1:0/19399513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560392fdd8b0 0x560392fddd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.979 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.974+0000 7f0137fff700 1 -- 172.21.15.1:0/19399513 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560392dab630 con 0x560392fd9470 2023-11-16T16:39:32.979 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.974+0000 7f013d226700 1 --2- 172.21.15.1:0/19399513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560392fdd8b0 0x560392fddd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:39:32.980 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.974+0000 7f0137fff700 1 --2- 172.21.15.1:0/19399513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560392fd9470 0x560392fdd370 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f012400fce0 tx=0x7f0124012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c09467e99e8544 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:32.980 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.974+0000 7f0122ffd700 1 -- 172.21.15.1:0/19399513 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0124011ad0 con 0x560392fd9470 2023-11-16T16:39:32.980 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.974+0000 7f0122ffd700 1 -- 172.21.15.1:0/19399513 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0124011c30 con 0x560392fd9470 2023-11-16T16:39:32.980 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.974+0000 7f0122ffd700 1 -- 172.21.15.1:0/19399513 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012401d610 con 0x560392fd9470 2023-11-16T16:39:32.980 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.975+0000 7f014ae15b80 1 -- 172.21.15.1:0/19399513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560392fd9470 msgr2=0x560392fdd370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:32.980 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.975+0000 7f014ae15b80 1 --2- 172.21.15.1:0/19399513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560392fd9470 0x560392fdd370 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f012400fce0 tx=0x7f0124012000 comp rx=0 tx=0).stop 2023-11-16T16:39:32.980 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.975+0000 7f014ae15b80 1 -- 172.21.15.1:0/19399513 shutdown_connections 2023-11-16T16:39:32.980 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.975+0000 7f014ae15b80 1 --2- 172.21.15.1:0/19399513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560392fdd8b0 0x560392fddd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.980 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.975+0000 7f014ae15b80 1 --2- 172.21.15.1:0/19399513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560392fd8b60 0x560392fd8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.980 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.975+0000 7f014ae15b80 1 --2- 172.21.15.1:0/19399513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560392fd9470 0x560392fdd370 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.980 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.975+0000 7f014ae15b80 1 -- 172.21.15.1:0/19399513 >> 172.21.15.1:0/19399513 conn(0x560392fce360 msgr2=0x560392fd3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:32.981 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.975+0000 7f014ae15b80 1 -- 172.21.15.1:0/19399513 shutdown_connections 2023-11-16T16:39:32.981 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.975+0000 7f014ae15b80 1 -- 172.21.15.1:0/19399513 wait complete. 2023-11-16T16:39:32.981 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.975+0000 7f014ae15b80 1 -- start start 2023-11-16T16:39:32.981 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.976+0000 7f014ae15b80 1 Processor -- start 2023-11-16T16:39:32.984 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.976+0000 7f014ae15b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560392fd8b60 0x560392fe4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:32.984 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.976+0000 7f014ae15b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560392fd9470 0x560392fe22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:32.984 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.976+0000 7f014ae15b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560392fdd8b0 0x560392fe28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:32.984 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.976+0000 7f014ae15b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560392ecb8e0 con 0x560392fd8b60 2023-11-16T16:39:32.984 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.976+0000 7f014ae15b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560392ecb580 con 0x560392fd9470 2023-11-16T16:39:32.984 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.976+0000 7f014ae15b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560392dad920 con 0x560392fdd8b0 2023-11-16T16:39:32.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.976+0000 7f013ca25700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560392fd8b60 0x560392fe4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:32.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.976+0000 7f014ae15b80 10 client.0 _ll_register_callbacks cb 0x560392ee9780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:39:32.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.976+0000 7f013ca25700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560392fd8b60 0x560392fe4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43954/0 (socket says 172.21.15.1:43954) 2023-11-16T16:39:32.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.976+0000 7f013ca25700 1 -- 172.21.15.1:0/3157539349 learned_addr learned my addr 172.21.15.1:0/3157539349 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:32.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.976+0000 7f013d226700 1 --2- 172.21.15.1:0/3157539349 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560392fdd8b0 0x560392fe28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:32.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.976+0000 7f0137fff700 1 --2- 172.21.15.1:0/3157539349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560392fd9470 0x560392fe22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:32.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.977+0000 7f013ca25700 1 -- 172.21.15.1:0/3157539349 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560392fdd8b0 msgr2=0x560392fe28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:32.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.977+0000 7f013ca25700 1 --2- 172.21.15.1:0/3157539349 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560392fdd8b0 0x560392fe28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.977+0000 7f013ca25700 1 -- 172.21.15.1:0/3157539349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560392fd9470 msgr2=0x560392fe22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:32.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.977+0000 7f013ca25700 1 --2- 172.21.15.1:0/3157539349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560392fd9470 0x560392fe22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.977+0000 7f013ca25700 1 -- 172.21.15.1:0/3157539349 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f012400f990 con 0x560392fd8b60 2023-11-16T16:39:32.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.977+0000 7f013d226700 1 --2- 172.21.15.1:0/3157539349 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560392fdd8b0 0x560392fe28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:39:32.986 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.977+0000 7f013ca25700 1 --2- 172.21.15.1:0/3157539349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560392fd8b60 0x560392fe4fe0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f012c09af50 tx=0x7f012c0cbe80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf42aaa9e69d2484 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:32.986 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.977+0000 7f0117fff700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:39:32.986 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.977+0000 7f0117fff700 1 -- 172.21.15.1:0/3157539349 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c09a940 con 0x560392fd8b60 2023-11-16T16:39:32.986 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.977+0000 7f0117fff700 1 -- 172.21.15.1:0/3157539349 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f012c09f280 con 0x560392fd8b60 2023-11-16T16:39:32.986 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.977+0000 7f014ae15b80 1 -- 172.21.15.1:0/3157539349 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560392fcbb20 con 0x560392fd8b60 2023-11-16T16:39:32.986 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.977+0000 7f0117fff700 1 -- 172.21.15.1:0/3157539349 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f012c0a3610 con 0x560392fd8b60 2023-11-16T16:39:32.986 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.977+0000 7f014ae15b80 10 client.15618 fetch_fsmap learned FSMap version 23 2023-11-16T16:39:32.986 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.977+0000 7f014ae15b80 1 -- 172.21.15.1:0/3157539349 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x560392ef0190 con 0x560392fd8b60 2023-11-16T16:39:32.986 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.977+0000 7f0117fff700 1 -- 172.21.15.1:0/3157539349 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560392fcbb20 con 0x560392fd8b60 2023-11-16T16:39:32.986 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.977+0000 7f0117fff700 1 -- 172.21.15.1:0/3157539349 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f012c097040 con 0x560392fd8b60 2023-11-16T16:39:32.986 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.977+0000 7f014ae15b80 10 client.15618 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:39:32.987 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.977+0000 7f014ae15b80 10 client.15618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:39:32.987 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.977+0000 7f014ae15b80 20 client.15618 populate_metadata read hostname 'smithi001' 2023-11-16T16:39:32.987 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.977+0000 7f014ae15b80 1 --2- 172.21.15.1:0/3157539349 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x560392ff2440 0x560392ff4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:32.987 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.977+0000 7f014ae15b80 4 client.15618 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:39:32.987 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.977+0000 7f014ae15b80 1 -- 172.21.15.1:0/3157539349 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560392eec0a0 con 0x560392ff2440 2023-11-16T16:39:32.987 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.978+0000 7f0137fff700 1 --2- 172.21.15.1:0/3157539349 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x560392ff2440 0x560392ff4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:32.987 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.978+0000 7f0137fff700 1 --2- 172.21.15.1:0/3157539349 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x560392ff2440 0x560392ff4820 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a178baac746eac89 server_cookie=81b72c153a83cdab in_seq=0 out_seq=0 2023-11-16T16:39:32.987 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.978+0000 7f0117fff700 10 client.15618 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:39:32.987 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.979+0000 7f0117fff700 1 -- 172.21.15.1:0/3157539349 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560392eec0a0 con 0x560392ff2440 2023-11-16T16:39:32.987 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.979+0000 7f0117fff700 10 client.15618 handle_command_reply: tid=0 2023-11-16T16:39:32.987 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.979+0000 7f014ae15b80 1 client.15618 shutdown 2023-11-16T16:39:32.987 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.979+0000 7f014ae15b80 1 -- 172.21.15.1:0/3157539349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560392fd8b60 msgr2=0x560392fe4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:32.988 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.979+0000 7f014ae15b80 1 --2- 172.21.15.1:0/3157539349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560392fd8b60 0x560392fe4fe0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f012c09af50 tx=0x7f012c0cbe80 comp rx=0 tx=0).stop 2023-11-16T16:39:32.988 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.979+0000 7f014ae15b80 1 -- 172.21.15.1:0/3157539349 shutdown_connections 2023-11-16T16:39:32.988 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.979+0000 7f014ae15b80 1 --2- 172.21.15.1:0/3157539349 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x560392ff2440 0x560392ff4820 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.988 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.979+0000 7f014ae15b80 1 --2- 172.21.15.1:0/3157539349 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560392fdd8b0 0x560392fe28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.988 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.979+0000 7f0117fff700 0 client.15618 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:39:32.988 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.979+0000 7f014ae15b80 1 --2- 172.21.15.1:0/3157539349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560392fd8b60 0x560392fe4fe0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.988 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.979+0000 7f014ae15b80 1 --2- 172.21.15.1:0/3157539349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560392fd9470 0x560392fe22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:32.988 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.979+0000 7f014ae15b80 1 -- 172.21.15.1:0/3157539349 >> 172.21.15.1:0/3157539349 conn(0x560392fce360 msgr2=0x560392fded60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:32.988 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.979+0000 7f014ae15b80 1 -- 172.21.15.1:0/3157539349 shutdown_connections 2023-11-16T16:39:32.988 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.979+0000 7f014ae15b80 1 -- 172.21.15.1:0/3157539349 wait complete. 2023-11-16T16:39:32.988 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.979+0000 7f014ae15b80 20 client.15618 trim_cache size 0 max 16384 2023-11-16T16:39:33.000 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.996+0000 7f014ae15b80 1 Processor -- start 2023-11-16T16:39:33.000 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.996+0000 7f014ae15b80 1 -- start start 2023-11-16T16:39:33.000 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.996+0000 7f014ae15b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560392fd8b60 0x560392fe99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:33.000 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.996+0000 7f014ae15b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560392fd9470 0x560392fe4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:33.001 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.996+0000 7f014ae15b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560392fdd8b0 0x560392fd3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:33.001 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.996+0000 7f014ae15b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560392fe3880 con 0x560392fd8b60 2023-11-16T16:39:33.001 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.996+0000 7f014ae15b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560392fdc200 con 0x560392fd9470 2023-11-16T16:39:33.001 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.996+0000 7f014ae15b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560392fd9950 con 0x560392fdd8b0 2023-11-16T16:39:33.001 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.996+0000 7f013ca25700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560392fd8b60 0x560392fe99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:33.001 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.997+0000 7f013ca25700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560392fd8b60 0x560392fe99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43962/0 (socket says 172.21.15.1:43962) 2023-11-16T16:39:33.001 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.997+0000 7f013ca25700 1 -- 172.21.15.1:0/3459614484 learned_addr learned my addr 172.21.15.1:0/3459614484 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:33.001 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.997+0000 7f0137fff700 1 --2- 172.21.15.1:0/3459614484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560392fd9470 0x560392fe4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:33.001 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.997+0000 7f013d226700 1 --2- 172.21.15.1:0/3459614484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560392fdd8b0 0x560392fd3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:33.001 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.997+0000 7f013ca25700 1 -- 172.21.15.1:0/3459614484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560392fdd8b0 msgr2=0x560392fd3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:33.001 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.997+0000 7f013ca25700 1 --2- 172.21.15.1:0/3459614484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560392fdd8b0 0x560392fd3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:33.001 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.997+0000 7f013ca25700 1 -- 172.21.15.1:0/3459614484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560392fd9470 msgr2=0x560392fe4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:33.001 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.997+0000 7f013ca25700 1 --2- 172.21.15.1:0/3459614484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560392fd9470 0x560392fe4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:33.002 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.997+0000 7f013ca25700 1 -- 172.21.15.1:0/3459614484 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560392fddf50 con 0x560392fd8b60 2023-11-16T16:39:33.002 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.997+0000 7f013ca25700 1 --2- 172.21.15.1:0/3459614484 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560392fd8b60 0x560392fe99a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f012c097c70 tx=0x7f012c099800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6362574b8d1a8c83 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:33.002 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.997+0000 7f01237fe700 1 -- 172.21.15.1:0/3459614484 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c0cb830 con 0x560392fd8b60 2023-11-16T16:39:33.002 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.997+0000 7f01237fe700 1 -- 172.21.15.1:0/3459614484 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f012c09f280 con 0x560392fd8b60 2023-11-16T16:39:33.002 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.997+0000 7f01237fe700 1 -- 172.21.15.1:0/3459614484 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c0a33e0 con 0x560392fd8b60 2023-11-16T16:39:33.002 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.998+0000 7f014ae15b80 1 -- 172.21.15.1:0/3459614484 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560392fd8b60 msgr2=0x560392fe99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:33.002 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.998+0000 7f014ae15b80 1 --2- 172.21.15.1:0/3459614484 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560392fd8b60 0x560392fe99a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f012c097c70 tx=0x7f012c099800 comp rx=0 tx=0).stop 2023-11-16T16:39:33.002 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.998+0000 7f014ae15b80 1 -- 172.21.15.1:0/3459614484 shutdown_connections 2023-11-16T16:39:33.002 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.998+0000 7f014ae15b80 1 --2- 172.21.15.1:0/3459614484 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560392fdd8b0 0x560392fd3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:33.002 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.998+0000 7f014ae15b80 1 --2- 172.21.15.1:0/3459614484 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560392fd8b60 0x560392fe99a0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:33.002 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.998+0000 7f014ae15b80 1 --2- 172.21.15.1:0/3459614484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560392fd9470 0x560392fe4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:33.002 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.998+0000 7f014ae15b80 1 -- 172.21.15.1:0/3459614484 >> 172.21.15.1:0/3459614484 conn(0x560392fce360 msgr2=0x560392feebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:33.003 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.998+0000 7f014ae15b80 1 -- 172.21.15.1:0/3459614484 shutdown_connections 2023-11-16T16:39:33.003 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.998+0000 7f014ae15b80 1 -- 172.21.15.1:0/3459614484 wait complete. 2023-11-16T16:39:33.003 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.998+0000 7f014ae15b80 1 -- start start 2023-11-16T16:39:33.003 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.998+0000 7f014ae15b80 1 Processor -- start 2023-11-16T16:39:33.006 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.999+0000 7f014ae15b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560392fd8b60 0x560392fe7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:33.006 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.999+0000 7f014ae15b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560392fd9470 0x560392fe7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:33.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.999+0000 7f014ae15b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560392fdd8b0 0x560392fe8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:33.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.999+0000 7f014ae15b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560393045400 con 0x560392fdd8b0 2023-11-16T16:39:33.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.999+0000 7f014ae15b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560393045570 con 0x560392fd9470 2023-11-16T16:39:33.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.999+0000 7f014ae15b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560393045730 con 0x560392fd8b60 2023-11-16T16:39:33.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.999+0000 7f014ae15b80 10 client.0 _ll_register_callbacks cb 0x7f012c09b040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:39:33.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.999+0000 7f013d226700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560392fdd8b0 0x560392fe8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:33.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.999+0000 7f013d226700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560392fdd8b0 0x560392fe8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43972/0 (socket says 172.21.15.1:43972) 2023-11-16T16:39:33.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.999+0000 7f013d226700 1 -- 172.21.15.1:0/2882380432 learned_addr learned my addr 172.21.15.1:0/2882380432 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:39:33.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.999+0000 7f013ca25700 1 --2- 172.21.15.1:0/2882380432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560392fd8b60 0x560392fe7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:33.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.999+0000 7f0137fff700 1 --2- 172.21.15.1:0/2882380432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560392fd9470 0x560392fe7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:33.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.999+0000 7f013d226700 1 -- 172.21.15.1:0/2882380432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560392fd8b60 msgr2=0x560392fe7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:33.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.999+0000 7f013d226700 1 --2- 172.21.15.1:0/2882380432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560392fd8b60 0x560392fe7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:33.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.999+0000 7f013d226700 1 -- 172.21.15.1:0/2882380432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560392fd9470 msgr2=0x560392fe7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:39:33.008 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.999+0000 7f013d226700 1 --2- 172.21.15.1:0/2882380432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560392fd9470 0x560392fe7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:33.008 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.999+0000 7f013d226700 1 -- 172.21.15.1:0/2882380432 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560392ef0190 con 0x560392fdd8b0 2023-11-16T16:39:33.008 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.999+0000 7f013d226700 1 --2- 172.21.15.1:0/2882380432 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560392fdd8b0 0x560392fe8410 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0130008930 tx=0x7f0130008c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=970d0c02d6ed685a server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:39:33.008 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:32.999+0000 7f0117fff700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:39:33.008 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.000+0000 7f0117fff700 1 -- 172.21.15.1:0/2882380432 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013000f050 con 0x560392fdd8b0 2023-11-16T16:39:33.008 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.000+0000 7f0117fff700 1 -- 172.21.15.1:0/2882380432 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f01300138f0 con 0x560392fdd8b0 2023-11-16T16:39:33.008 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.000+0000 7f014ae15b80 1 -- 172.21.15.1:0/2882380432 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0124011c30 con 0x560392fdd8b0 2023-11-16T16:39:33.008 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.000+0000 7f0117fff700 1 -- 172.21.15.1:0/2882380432 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0130013ab0 con 0x560392fdd8b0 2023-11-16T16:39:33.008 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.000+0000 7f014ae15b80 10 client.15624 fetch_fsmap learned FSMap version 23 2023-11-16T16:39:33.008 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.000+0000 7f014ae15b80 1 -- 172.21.15.1:0/2882380432 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x560392d57de0 con 0x560392fdd8b0 2023-11-16T16:39:33.008 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.000+0000 7f0117fff700 1 -- 172.21.15.1:0/2882380432 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0124011c30 con 0x560392fdd8b0 2023-11-16T16:39:33.008 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.000+0000 7f0117fff700 1 -- 172.21.15.1:0/2882380432 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f01300173c0 con 0x560392fdd8b0 2023-11-16T16:39:33.008 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.000+0000 7f014ae15b80 10 client.15624 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:39:33.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.000+0000 7f014ae15b80 10 client.15624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:39:33.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.000+0000 7f014ae15b80 20 client.15624 populate_metadata read hostname 'smithi001' 2023-11-16T16:39:33.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.000+0000 7f014ae15b80 1 --2- 172.21.15.1:0/2882380432 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x560393046f30 0x560393049340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:39:33.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.000+0000 7f014ae15b80 4 client.15624 mds_command: new command op to 24521 tid=0[{"prefix": "scrub status"}] 2023-11-16T16:39:33.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.000+0000 7f014ae15b80 1 -- 172.21.15.1:0/2882380432 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560392ef8660 con 0x560393046f30 2023-11-16T16:39:33.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.000+0000 7f013ca25700 1 --2- 172.21.15.1:0/2882380432 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x560393046f30 0x560393049340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:39:33.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.001+0000 7f0117fff700 10 client.15624 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:39:33.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.001+0000 7f013ca25700 1 --2- 172.21.15.1:0/2882380432 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x560393046f30 0x560393049340 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=face40f08addaa6b server_cookie=5d6bdadfb22f4b2d in_seq=0 out_seq=0 2023-11-16T16:39:33.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.001+0000 7f0117fff700 1 -- 172.21.15.1:0/2882380432 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560392ef8660 con 0x560393046f30 2023-11-16T16:39:33.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.001+0000 7f0117fff700 10 client.15624 handle_command_reply: tid=0 2023-11-16T16:39:33.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.001+0000 7f014ae15b80 1 client.15624 shutdown 2023-11-16T16:39:33.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.001+0000 7f014ae15b80 1 -- 172.21.15.1:0/2882380432 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560392fdd8b0 msgr2=0x560392fe8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:33.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.001+0000 7f014ae15b80 1 --2- 172.21.15.1:0/2882380432 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560392fdd8b0 0x560392fe8410 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0130008930 tx=0x7f0130008c40 comp rx=0 tx=0).stop 2023-11-16T16:39:33.010 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.001+0000 7f014ae15b80 1 -- 172.21.15.1:0/2882380432 shutdown_connections 2023-11-16T16:39:33.010 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.001+0000 7f014ae15b80 1 --2- 172.21.15.1:0/2882380432 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x560393046f30 0x560393049340 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:33.010 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.001+0000 7f014ae15b80 1 --2- 172.21.15.1:0/2882380432 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560392fdd8b0 0x560392fe8410 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:33.010 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.001+0000 7f0117fff700 0 client.15624 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:39:33.010 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.001+0000 7f014ae15b80 1 --2- 172.21.15.1:0/2882380432 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560392fd8b60 0x560392fe7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:33.010 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.001+0000 7f014ae15b80 1 --2- 172.21.15.1:0/2882380432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560392fd9470 0x560392fe7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:33.010 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.001+0000 7f014ae15b80 1 -- 172.21.15.1:0/2882380432 >> 172.21.15.1:0/2882380432 conn(0x560392fce360 msgr2=0x560392fdc830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:33.010 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.001+0000 7f014ae15b80 1 -- 172.21.15.1:0/2882380432 shutdown_connections 2023-11-16T16:39:33.010 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.001+0000 7f014ae15b80 1 -- 172.21.15.1:0/2882380432 wait complete. 2023-11-16T16:39:33.010 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.002+0000 7f014ae15b80 20 client.15624 trim_cache size 0 max 16384 2023-11-16T16:39:33.011 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:39:33.011 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-16T16:39:33.011 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2023-11-16T16:39:33.011 INFO:teuthology.orchestra.run.smithi001.stdout: "3dd6b4ce-2833-4a83-93bf-28b49034e19c": { 2023-11-16T16:39:33.011 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2023-11-16T16:39:33.011 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "3dd6b4ce-2833-4a83-93bf-28b49034e19c", 2023-11-16T16:39:33.011 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2023-11-16T16:39:33.011 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:39:33.012 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:39:33.012 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:39:33.012 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.002+0000 7f013ec89700 1 -- 172.21.15.1:0/1399683851 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f01280a1350 msgr2=0x7f01280a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:33.012 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.002+0000 7f013ec89700 1 --2- 172.21.15.1:0/1399683851 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f01280a1350 0x7f01280a3800 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0124005950 tx=0x7f01240058e0 comp rx=0 tx=0).stop 2023-11-16T16:39:33.013 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.002+0000 7f013ec89700 1 -- 172.21.15.1:0/1399683851 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01380ff410 msgr2=0x7f0138100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:39:33.013 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.002+0000 7f013ec89700 1 --2- 172.21.15.1:0/1399683851 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01380ff410 0x7f0138100e10 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f012c009990 tx=0x7f012c009580 comp rx=0 tx=0).stop 2023-11-16T16:39:33.013 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.002+0000 7f013ec89700 1 -- 172.21.15.1:0/1399683851 shutdown_connections 2023-11-16T16:39:33.013 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.002+0000 7f013ec89700 1 --2- 172.21.15.1:0/1399683851 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f01280a1350 0x7f01280a3800 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:33.013 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.002+0000 7f013ec89700 1 --2- 172.21.15.1:0/1399683851 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0138104450 0x7f01381a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:33.013 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.002+0000 7f013ec89700 1 --2- 172.21.15.1:0/1399683851 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01380ff410 0x7f0138100e10 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:33.013 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.002+0000 7f013ec89700 1 --2- 172.21.15.1:0/1399683851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01380ffdb0 0x7f0138101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:39:33.013 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.002+0000 7f013ec89700 1 -- 172.21.15.1:0/1399683851 >> 172.21.15.1:0/1399683851 conn(0x7f0138076370 msgr2=0x7f0138068f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:39:33.013 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.002+0000 7f013ec89700 1 -- 172.21.15.1:0/1399683851 shutdown_connections 2023-11-16T16:39:33.013 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:39:33.002+0000 7f013ec89700 1 -- 172.21.15.1:0/1399683851 wait complete. 2023-11-16T16:39:33.016 INFO:tasks.cephfs.filesystem:scrub status for tag:3dd6b4ce-2833-4a83-93bf-28b49034e19c - {'path': '/', 'tag': '3dd6b4ce-2833-4a83-93bf-28b49034e19c', 'options': 'recursive,force'} 2023-11-16T16:39:33.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:33 smithi053 ceph-mon[100638]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:39:33.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:33 smithi053 ceph-mon[100638]: scrub queued for path: / 2023-11-16T16:39:33.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:33 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:39:33.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:33 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:39:33.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:33 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:39:33.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:33 smithi001 ceph-mon[98800]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:39:33.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:33 smithi001 ceph-mon[98800]: scrub queued for path: / 2023-11-16T16:39:33.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:33 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:39:33.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:33 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:39:33.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:33 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:39:33.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:33 smithi026 ceph-mon[100112]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:39:33.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:33 smithi026 ceph-mon[100112]: scrub queued for path: / 2023-11-16T16:39:33.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:33 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:39:33.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:33 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:39:33.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:33 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:39:34.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:34 smithi053 ceph-mon[100638]: scrub summary: idle 2023-11-16T16:39:34.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:34 smithi001 ceph-mon[98800]: scrub summary: idle 2023-11-16T16:39:34.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:34 smithi026 ceph-mon[100112]: scrub summary: idle 2023-11-16T16:39:35.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:35 smithi053 ceph-mon[100638]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:39:35.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:35 smithi001 ceph-mon[98800]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:39:35.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:35 smithi026 ceph-mon[100112]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:39:37.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:37 smithi053 ceph-mon[100638]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 7 op/s 2023-11-16T16:39:37.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:37 smithi001 ceph-mon[98800]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 7 op/s 2023-11-16T16:39:37.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:37 smithi026 ceph-mon[100112]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 7 op/s 2023-11-16T16:39:39.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:39 smithi053 ceph-mon[100638]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2023-11-16T16:39:39.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:39 smithi001 ceph-mon[98800]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2023-11-16T16:39:39.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:39 smithi026 ceph-mon[100112]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2023-11-16T16:39:41.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:41 smithi053 ceph-mon[100638]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:39:41.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:41 smithi001 ceph-mon[98800]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:39:41.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:41 smithi026 ceph-mon[100112]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:39:43.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:43 smithi053 ceph-mon[100638]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-16T16:39:43.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:43 smithi001 ceph-mon[98800]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-16T16:39:43.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:43 smithi026 ceph-mon[100112]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-16T16:39:45.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:45 smithi053 ceph-mon[100638]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:39:45.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:45 smithi001 ceph-mon[98800]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:39:45.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:45 smithi026 ceph-mon[100112]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:39:47.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:47 smithi053 ceph-mon[100638]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 7 op/s 2023-11-16T16:39:47.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:47 smithi001 ceph-mon[98800]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 7 op/s 2023-11-16T16:39:47.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:47 smithi026 ceph-mon[100112]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 7 op/s 2023-11-16T16:39:49.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:49 smithi001 ceph-mon[98800]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 682 B/s wr, 5 op/s 2023-11-16T16:39:49.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:49 smithi026 ceph-mon[100112]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 682 B/s wr, 5 op/s 2023-11-16T16:39:49.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:49 smithi053 ceph-mon[100638]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 682 B/s wr, 5 op/s 2023-11-16T16:39:50.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:50 smithi001 ceph-mon[98800]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2023-11-16T16:39:50.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:50 smithi026 ceph-mon[100112]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2023-11-16T16:39:50.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:50 smithi053 ceph-mon[100638]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2023-11-16T16:39:53.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:53 smithi026 ceph-mon[100112]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2023-11-16T16:39:53.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:53 smithi053 ceph-mon[100638]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2023-11-16T16:39:53.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:53 smithi001 ceph-mon[98800]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2023-11-16T16:39:55.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:55 smithi026 ceph-mon[100112]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2023-11-16T16:39:55.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:55 smithi053 ceph-mon[100638]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2023-11-16T16:39:55.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:55 smithi001 ceph-mon[98800]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2023-11-16T16:39:57.509 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:57 smithi026 ceph-mon[100112]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2023-11-16T16:39:57.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:57 smithi053 ceph-mon[100638]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2023-11-16T16:39:57.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:57 smithi001 ceph-mon[98800]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2023-11-16T16:39:59.510 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:39:59 smithi026 ceph-mon[100112]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-11-16T16:40:02.971 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:39:59 smithi001 ceph-mon[98800]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-11-16T16:40:02.971 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:00 smithi001 ceph-mon[98800]: overall HEALTH_OK 2023-11-16T16:40:02.971 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:00 smithi026 ceph-mon[100112]: overall HEALTH_OK 2023-11-16T16:40:02.971 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:01 smithi026 ceph-mon[100112]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:40:02.971 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:40:00 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:40:00.911+0000 7f4adfca5700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-16T16:40:02.971 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:40:00 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:40:00.911+0000 7f4adfca5700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-16T16:40:02.972 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:40:00 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:40:00.911+0000 7f4adfca5700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-16T16:40:02.972 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:40:00 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:40:00.911+0000 7f4adfca5700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-16T16:40:02.972 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:40:00 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:40:00.911+0000 7f4adfca5700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-16T16:40:02.972 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:39:59 smithi053 ceph-mon[100638]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-11-16T16:40:02.972 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:00 smithi053 ceph-mon[100638]: overall HEALTH_OK 2023-11-16T16:40:02.972 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:01 smithi053 ceph-mon[100638]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:40:02.972 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:01 smithi001 ceph-mon[98800]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:40:03.017 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-16T16:40:03.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.218+0000 7fdda3fc9700 1 -- 172.21.15.1:0/3159826162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd9c0ff2c0 msgr2=0x7fdd9c10b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:03.220 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.218+0000 7fdda3fc9700 1 --2- 172.21.15.1:0/3159826162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd9c0ff2c0 0x7fdd9c10b750 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fdd9400b0a0 tx=0x7fdd9400b3b0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.220 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.218+0000 7fdda3fc9700 1 -- 172.21.15.1:0/3159826162 shutdown_connections 2023-11-16T16:40:03.220 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.218+0000 7fdda3fc9700 1 --2- 172.21.15.1:0/3159826162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd9c0ff2c0 0x7fdd9c10b750 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.220 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.218+0000 7fdda3fc9700 1 --2- 172.21.15.1:0/3159826162 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd9c104920 0x7fdd9c104cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.220 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.218+0000 7fdda3fc9700 1 --2- 172.21.15.1:0/3159826162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd9c0fe910 0x7fdd9c0fed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.220 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.218+0000 7fdda3fc9700 1 -- 172.21.15.1:0/3159826162 >> 172.21.15.1:0/3159826162 conn(0x7fdd9c075880 msgr2=0x7fdd9c075c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:03.220 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.218+0000 7fdda3fc9700 1 -- 172.21.15.1:0/3159826162 shutdown_connections 2023-11-16T16:40:03.220 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.218+0000 7fdda3fc9700 1 -- 172.21.15.1:0/3159826162 wait complete. 2023-11-16T16:40:03.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.219+0000 7fdda3fc9700 1 Processor -- start 2023-11-16T16:40:03.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.219+0000 7fdda3fc9700 1 -- start start 2023-11-16T16:40:03.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.220+0000 7fdda3fc9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd9c0fe910 0x7fdd9c10ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:03.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.220+0000 7fdda3fc9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd9c0ff2c0 0x7fdd9c10d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:03.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.220+0000 7fdda3fc9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd9c104920 0x7fdd9c1a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:03.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.220+0000 7fdda3fc9700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9c10dbe0 con 0x7fdd9c0fe910 2023-11-16T16:40:03.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.220+0000 7fdda3fc9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9c1a4b00 con 0x7fdd9c0ff2c0 2023-11-16T16:40:03.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.220+0000 7fdda3fc9700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9c1a4c40 con 0x7fdd9c104920 2023-11-16T16:40:03.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.220+0000 7fdda1d65700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd9c0fe910 0x7fdd9c10ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:03.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.220+0000 7fdda1d65700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd9c0fe910 0x7fdd9c10ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43998/0 (socket says 172.21.15.1:43998) 2023-11-16T16:40:03.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.220+0000 7fdda1d65700 1 -- 172.21.15.1:0/1995158357 learned_addr learned my addr 172.21.15.1:0/1995158357 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:03.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.220+0000 7fdda2566700 1 --2- 172.21.15.1:0/1995158357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd9c104920 0x7fdd9c1a45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:03.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.220+0000 7fdda1564700 1 --2- 172.21.15.1:0/1995158357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd9c0ff2c0 0x7fdd9c10d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:03.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.220+0000 7fdda1d65700 1 -- 172.21.15.1:0/1995158357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd9c104920 msgr2=0x7fdd9c1a45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:03.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.220+0000 7fdda1d65700 1 --2- 172.21.15.1:0/1995158357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd9c104920 0x7fdd9c1a45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.220+0000 7fdda1d65700 1 -- 172.21.15.1:0/1995158357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd9c0ff2c0 msgr2=0x7fdd9c10d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:03.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.220+0000 7fdda1d65700 1 --2- 172.21.15.1:0/1995158357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd9c0ff2c0 0x7fdd9c10d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.220+0000 7fdda1d65700 1 -- 172.21.15.1:0/1995158357 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd94009d00 con 0x7fdd9c0fe910 2023-11-16T16:40:03.224 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.220+0000 7fdda2566700 1 --2- 172.21.15.1:0/1995158357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd9c104920 0x7fdd9c1a45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:40:03.224 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.221+0000 7fdda1d65700 1 --2- 172.21.15.1:0/1995158357 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd9c0fe910 0x7fdd9c10ce70 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fdd9800e970 tx=0x7fdd9800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:03.224 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.221+0000 7fdd92ffd700 1 -- 172.21.15.1:0/1995158357 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd98009790 con 0x7fdd9c0fe910 2023-11-16T16:40:03.224 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.221+0000 7fdda3fc9700 1 -- 172.21.15.1:0/1995158357 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd9c110360 con 0x7fdd9c0fe910 2023-11-16T16:40:03.224 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.221+0000 7fdd92ffd700 1 -- 172.21.15.1:0/1995158357 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd980044e0 con 0x7fdd9c0fe910 2023-11-16T16:40:03.224 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.221+0000 7fdd92ffd700 1 -- 172.21.15.1:0/1995158357 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd98003e30 con 0x7fdd9c0fe910 2023-11-16T16:40:03.224 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.221+0000 7fdda3fc9700 1 -- 172.21.15.1:0/1995158357 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd9c1101d0 con 0x7fdd9c0fe910 2023-11-16T16:40:03.225 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.224+0000 7fdd92ffd700 1 -- 172.21.15.1:0/1995158357 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 24) v1 ==== 135947+0+0 (secure 0 0 0) 0x7fdd98004000 con 0x7fdd9c0fe910 2023-11-16T16:40:03.226 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.225+0000 7fdd92ffd700 1 --2- 172.21.15.1:0/1995158357 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fdd880a13e0 0x7fdd880a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:03.226 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.225+0000 7fdda1564700 1 --2- 172.21.15.1:0/1995158357 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fdd880a13e0 0x7fdd880a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:03.226 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.225+0000 7fdd92ffd700 1 -- 172.21.15.1:0/1995158357 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fdd98014070 con 0x7fdd9c0fe910 2023-11-16T16:40:03.227 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.226+0000 7fdda1564700 1 --2- 172.21.15.1:0/1995158357 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fdd880a13e0 0x7fdd880a3890 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c10e660 tx=0x7fdd8c00b5b0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:03.227 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.226+0000 7fddb0155b80 1 Processor -- start 2023-11-16T16:40:03.228 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.227+0000 7fddb0155b80 1 -- start start 2023-11-16T16:40:03.229 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.227+0000 7fddb0155b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b579a5b60 0x563b579a5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:03.229 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.227+0000 7fddb0155b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b579a6470 0x563b579aa370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:03.229 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.227+0000 7fddb0155b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b579aa8b0 0x563b579aad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:03.229 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.227+0000 7fddb0155b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b578ab060 con 0x563b579a6470 2023-11-16T16:40:03.229 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.227+0000 7fddb0155b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b578fd8a0 con 0x563b579a5b60 2023-11-16T16:40:03.229 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.227+0000 7fddb0155b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b578f6d30 con 0x563b579aa8b0 2023-11-16T16:40:03.229 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.227+0000 7fdda1564700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b579a6470 0x563b579aa370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:03.230 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.227+0000 7fdda1564700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b579a6470 0x563b579aa370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44006/0 (socket says 172.21.15.1:44006) 2023-11-16T16:40:03.230 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.227+0000 7fdda1564700 1 -- 172.21.15.1:0/2231641283 learned_addr learned my addr 172.21.15.1:0/2231641283 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:03.230 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.228+0000 7fdda1d65700 1 --2- 172.21.15.1:0/2231641283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b579a5b60 0x563b579a5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:03.230 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.228+0000 7fdda2566700 1 --2- 172.21.15.1:0/2231641283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b579aa8b0 0x563b579aad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:03.230 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.228+0000 7fdda1564700 1 -- 172.21.15.1:0/2231641283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b579aa8b0 msgr2=0x563b579aad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:03.230 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.228+0000 7fdda1564700 1 --2- 172.21.15.1:0/2231641283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b579aa8b0 0x563b579aad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.231 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.228+0000 7fdda1564700 1 -- 172.21.15.1:0/2231641283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b579a5b60 msgr2=0x563b579a5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:03.231 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.228+0000 7fdda1564700 1 --2- 172.21.15.1:0/2231641283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b579a5b60 0x563b579a5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.231 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.228+0000 7fdda1564700 1 -- 172.21.15.1:0/2231641283 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b57778630 con 0x563b579a6470 2023-11-16T16:40:03.231 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.231+0000 7fdda1564700 1 --2- 172.21.15.1:0/2231641283 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b579a6470 0x563b579aa370 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c00fce0 tx=0x7fdd8c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f209adb2ed3ba6 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:03.232 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.232+0000 7fdd87fff700 1 -- 172.21.15.1:0/2231641283 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c011ad0 con 0x563b579a6470 2023-11-16T16:40:03.232 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.232+0000 7fdd87fff700 1 -- 172.21.15.1:0/2231641283 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd8c011c30 con 0x563b579a6470 2023-11-16T16:40:03.232 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.232+0000 7fdd87fff700 1 -- 172.21.15.1:0/2231641283 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c01d610 con 0x563b579a6470 2023-11-16T16:40:03.233 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.233+0000 7fddb0155b80 1 -- 172.21.15.1:0/2231641283 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b579a6470 msgr2=0x563b579aa370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:03.234 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.233+0000 7fddb0155b80 1 --2- 172.21.15.1:0/2231641283 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b579a6470 0x563b579aa370 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c00fce0 tx=0x7fdd8c012000 comp rx=0 tx=0).stop 2023-11-16T16:40:03.234 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.233+0000 7fddb0155b80 1 -- 172.21.15.1:0/2231641283 shutdown_connections 2023-11-16T16:40:03.234 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.233+0000 7fddb0155b80 1 --2- 172.21.15.1:0/2231641283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b579aa8b0 0x563b579aad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.234 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.233+0000 7fddb0155b80 1 --2- 172.21.15.1:0/2231641283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b579a5b60 0x563b579a5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.234 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.233+0000 7fddb0155b80 1 --2- 172.21.15.1:0/2231641283 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b579a6470 0x563b579aa370 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.234 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.233+0000 7fddb0155b80 1 -- 172.21.15.1:0/2231641283 >> 172.21.15.1:0/2231641283 conn(0x563b5799b360 msgr2=0x563b579a0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:03.234 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.233+0000 7fddb0155b80 1 -- 172.21.15.1:0/2231641283 shutdown_connections 2023-11-16T16:40:03.234 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.233+0000 7fddb0155b80 1 -- 172.21.15.1:0/2231641283 wait complete. 2023-11-16T16:40:03.236 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.233+0000 7fddb0155b80 1 -- start start 2023-11-16T16:40:03.236 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.234+0000 7fddb0155b80 1 Processor -- start 2023-11-16T16:40:03.236 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.234+0000 7fddb0155b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b579a5b60 0x563b579b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:03.236 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.234+0000 7fddb0155b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b579a6470 0x563b579af280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:03.236 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.234+0000 7fddb0155b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b579aa8b0 0x563b579af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:03.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.234+0000 7fddb0155b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b578988e0 con 0x563b579aa8b0 2023-11-16T16:40:03.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.234+0000 7fddb0155b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b57898580 con 0x563b579a5b60 2023-11-16T16:40:03.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.234+0000 7fddb0155b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b5777a920 con 0x563b579a6470 2023-11-16T16:40:03.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.234+0000 7fddb0155b80 10 client.0 _ll_register_callbacks cb 0x563b578b6780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:40:03.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.234+0000 7fdda1d65700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b579a5b60 0x563b579b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:03.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.234+0000 7fdda2566700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b579aa8b0 0x563b579af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:03.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.234+0000 7fdda2566700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b579aa8b0 0x563b579af8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44012/0 (socket says 172.21.15.1:44012) 2023-11-16T16:40:03.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.234+0000 7fdda2566700 1 -- 172.21.15.1:0/233859054 learned_addr learned my addr 172.21.15.1:0/233859054 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:03.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.235+0000 7fdda2566700 1 -- 172.21.15.1:0/233859054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b579a6470 msgr2=0x563b579af280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:03.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.235+0000 7fdda2566700 1 --2- 172.21.15.1:0/233859054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b579a6470 0x563b579af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.235+0000 7fdda2566700 1 -- 172.21.15.1:0/233859054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b579a5b60 msgr2=0x563b579b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:03.238 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.235+0000 7fdda2566700 1 --2- 172.21.15.1:0/233859054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b579a5b60 0x563b579b2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.238 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.235+0000 7fdda2566700 1 -- 172.21.15.1:0/233859054 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd8c00f9c0 con 0x563b579aa8b0 2023-11-16T16:40:03.238 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.235+0000 7fdda1d65700 1 --2- 172.21.15.1:0/233859054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b579a5b60 0x563b579b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:40:03.238 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.235+0000 7fdda2566700 1 --2- 172.21.15.1:0/233859054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b579aa8b0 0x563b579af8b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fdd94015000 tx=0x7fdd94004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa6de0429d8ed55 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:03.238 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.235+0000 7fdd857fa700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:40:03.238 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.235+0000 7fdd857fa700 1 -- 172.21.15.1:0/233859054 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9400e040 con 0x563b579aa8b0 2023-11-16T16:40:03.238 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.235+0000 7fdd857fa700 1 -- 172.21.15.1:0/233859054 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fdd94022070 con 0x563b579aa8b0 2023-11-16T16:40:03.238 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.235+0000 7fddb0155b80 1 -- 172.21.15.1:0/233859054 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563b57998b20 con 0x563b579aa8b0 2023-11-16T16:40:03.238 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.235+0000 7fdd857fa700 1 -- 172.21.15.1:0/233859054 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd94004e80 con 0x563b579aa8b0 2023-11-16T16:40:03.238 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.236+0000 7fddb0155b80 10 client.15642 fetch_fsmap learned FSMap version 23 2023-11-16T16:40:03.238 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.236+0000 7fddb0155b80 1 -- 172.21.15.1:0/233859054 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x563b578bd190 con 0x563b579aa8b0 2023-11-16T16:40:03.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.236+0000 7fdd857fa700 1 -- 172.21.15.1:0/233859054 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563b57998b20 con 0x563b579aa8b0 2023-11-16T16:40:03.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.236+0000 7fdd857fa700 1 -- 172.21.15.1:0/233859054 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdd9400b0d0 con 0x563b579aa8b0 2023-11-16T16:40:03.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.236+0000 7fddb0155b80 10 client.15642 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:40:03.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.236+0000 7fddb0155b80 10 client.15642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:40:03.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.236+0000 7fddb0155b80 20 client.15642 populate_metadata read hostname 'smithi001' 2023-11-16T16:40:03.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.236+0000 7fddb0155b80 1 --2- 172.21.15.1:0/233859054 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x563b579bf8b0 0x563b579c1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:03.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.236+0000 7fddb0155b80 4 client.15642 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:40:03.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.236+0000 7fddb0155b80 1 -- 172.21.15.1:0/233859054 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563b578b90a0 con 0x563b579bf8b0 2023-11-16T16:40:03.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.236+0000 7fdda1d65700 1 --2- 172.21.15.1:0/233859054 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x563b579bf8b0 0x563b579c1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:03.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.237+0000 7fdd857fa700 10 client.15642 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:40:03.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.237+0000 7fdda1d65700 1 --2- 172.21.15.1:0/233859054 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x563b579bf8b0 0x563b579c1cc0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=333f98f85baa1933 server_cookie=64e45e16862a65bd in_seq=0 out_seq=0 2023-11-16T16:40:03.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.237+0000 7fdd857fa700 1 -- 172.21.15.1:0/233859054 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563b578b90a0 con 0x563b579bf8b0 2023-11-16T16:40:03.242 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.237+0000 7fdd857fa700 10 client.15642 handle_command_reply: tid=0 2023-11-16T16:40:03.242 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.238+0000 7fddb0155b80 1 client.15642 shutdown 2023-11-16T16:40:03.242 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.238+0000 7fddb0155b80 1 -- 172.21.15.1:0/233859054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b579aa8b0 msgr2=0x563b579af8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:03.242 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.238+0000 7fddb0155b80 1 --2- 172.21.15.1:0/233859054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b579aa8b0 0x563b579af8b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fdd94015000 tx=0x7fdd94004a50 comp rx=0 tx=0).stop 2023-11-16T16:40:03.242 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.238+0000 7fddb0155b80 1 -- 172.21.15.1:0/233859054 shutdown_connections 2023-11-16T16:40:03.242 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.238+0000 7fddb0155b80 1 --2- 172.21.15.1:0/233859054 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x563b579bf8b0 0x563b579c1cc0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.242 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.238+0000 7fddb0155b80 1 --2- 172.21.15.1:0/233859054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b579aa8b0 0x563b579af8b0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.242 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.238+0000 7fddb0155b80 1 --2- 172.21.15.1:0/233859054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b579a5b60 0x563b579b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.242 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.238+0000 7fddb0155b80 1 --2- 172.21.15.1:0/233859054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b579a6470 0x563b579af280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.242 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.238+0000 7fddb0155b80 1 -- 172.21.15.1:0/233859054 >> 172.21.15.1:0/233859054 conn(0x563b5799b360 msgr2=0x563b5799fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:03.242 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.238+0000 7fdd857fa700 0 client.15642 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:40:03.242 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.238+0000 7fddb0155b80 1 -- 172.21.15.1:0/233859054 shutdown_connections 2023-11-16T16:40:03.243 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.238+0000 7fddb0155b80 1 -- 172.21.15.1:0/233859054 wait complete. 2023-11-16T16:40:03.243 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.238+0000 7fddb0155b80 20 client.15642 trim_cache size 0 max 16384 2023-11-16T16:40:03.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.271+0000 7fddb0155b80 1 Processor -- start 2023-11-16T16:40:03.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.271+0000 7fddb0155b80 1 -- start start 2023-11-16T16:40:03.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.271+0000 7fddb0155b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b579a5b60 0x563b579b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:03.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.271+0000 7fddb0155b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b579a6470 0x563b579a0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:03.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.271+0000 7fddb0155b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b579aa8b0 0x563b579a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:03.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.272+0000 7fddb0155b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b579b0860 con 0x563b579aa8b0 2023-11-16T16:40:03.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.272+0000 7fddb0155b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b5799aee0 con 0x563b579a5b60 2023-11-16T16:40:03.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.272+0000 7fddb0155b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b579a1950 con 0x563b579a6470 2023-11-16T16:40:03.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.272+0000 7fdda2566700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b579aa8b0 0x563b579a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:03.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.272+0000 7fdda2566700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b579aa8b0 0x563b579a0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44022/0 (socket says 172.21.15.1:44022) 2023-11-16T16:40:03.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.272+0000 7fdda2566700 1 -- 172.21.15.1:0/670581283 learned_addr learned my addr 172.21.15.1:0/670581283 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:03.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.272+0000 7fdda1d65700 1 --2- 172.21.15.1:0/670581283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b579a5b60 0x563b579b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:03.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.272+0000 7fdda1564700 1 --2- 172.21.15.1:0/670581283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b579a6470 0x563b579a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:03.277 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.272+0000 7fdda1d65700 1 -- 172.21.15.1:0/670581283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b579a6470 msgr2=0x563b579a0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:03.277 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.272+0000 7fdda1d65700 1 --2- 172.21.15.1:0/670581283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b579a6470 0x563b579a0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.272+0000 7fdda1d65700 1 -- 172.21.15.1:0/670581283 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b579aa8b0 msgr2=0x563b579a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:03.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.272+0000 7fdda1d65700 1 --2- 172.21.15.1:0/670581283 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b579aa8b0 0x563b579a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.272+0000 7fdda1d65700 1 -- 172.21.15.1:0/670581283 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b578bcee0 con 0x563b579a5b60 2023-11-16T16:40:03.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.272+0000 7fdda2566700 1 --2- 172.21.15.1:0/670581283 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b579aa8b0 0x563b579a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:40:03.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.272+0000 7fdda1564700 1 --2- 172.21.15.1:0/670581283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b579a6470 0x563b579a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:40:03.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.272+0000 7fdda1d65700 1 --2- 172.21.15.1:0/670581283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b579a5b60 0x563b579b69e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdd980ca0e0 tx=0x7fdd980cf000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d05a1f14581d0eb server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:03.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.273+0000 7fdd90ff9700 1 -- 172.21.15.1:0/670581283 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd980992f0 con 0x563b579a5b60 2023-11-16T16:40:03.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.273+0000 7fdd90ff9700 1 -- 172.21.15.1:0/670581283 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd98099990 con 0x563b579a5b60 2023-11-16T16:40:03.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.273+0000 7fdd90ff9700 1 -- 172.21.15.1:0/670581283 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd98004340 con 0x563b579a5b60 2023-11-16T16:40:03.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.274+0000 7fddb0155b80 1 -- 172.21.15.1:0/670581283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b579a5b60 msgr2=0x563b579b69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:03.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.274+0000 7fddb0155b80 1 --2- 172.21.15.1:0/670581283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b579a5b60 0x563b579b69e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdd980ca0e0 tx=0x7fdd980cf000 comp rx=0 tx=0).stop 2023-11-16T16:40:03.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.274+0000 7fddb0155b80 1 -- 172.21.15.1:0/670581283 shutdown_connections 2023-11-16T16:40:03.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.274+0000 7fddb0155b80 1 --2- 172.21.15.1:0/670581283 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b579aa8b0 0x563b579a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.274+0000 7fddb0155b80 1 --2- 172.21.15.1:0/670581283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b579a5b60 0x563b579b69e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.274+0000 7fddb0155b80 1 --2- 172.21.15.1:0/670581283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b579a6470 0x563b579a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.274+0000 7fddb0155b80 1 -- 172.21.15.1:0/670581283 >> 172.21.15.1:0/670581283 conn(0x563b5799b360 msgr2=0x563b579a9970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:03.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.274+0000 7fddb0155b80 1 -- 172.21.15.1:0/670581283 shutdown_connections 2023-11-16T16:40:03.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.274+0000 7fddb0155b80 1 -- 172.21.15.1:0/670581283 wait complete. 2023-11-16T16:40:03.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.274+0000 7fddb0155b80 1 -- start start 2023-11-16T16:40:03.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.274+0000 7fddb0155b80 1 Processor -- start 2023-11-16T16:40:03.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.275+0000 7fddb0155b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b579a5b60 0x563b579b4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:03.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.275+0000 7fddb0155b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b579a6470 0x563b579b4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:03.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.275+0000 7fddb0155b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b579aa8b0 0x563b579b4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:03.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.275+0000 7fddb0155b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b57a12770 con 0x563b579aa8b0 2023-11-16T16:40:03.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.275+0000 7fddb0155b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b57a128b0 con 0x563b579a5b60 2023-11-16T16:40:03.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.275+0000 7fddb0155b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b57a12a20 con 0x563b579a6470 2023-11-16T16:40:03.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.275+0000 7fddb0155b80 10 client.0 _ll_register_callbacks cb 0x7fdd9400bdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:40:03.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.275+0000 7fdda1d65700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b579a5b60 0x563b579b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:03.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.275+0000 7fdda1d65700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b579a5b60 0x563b579b4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.1:38210/0 (socket says 172.21.15.1:38210) 2023-11-16T16:40:03.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.275+0000 7fdda1d65700 1 -- 172.21.15.1:0/4006220571 learned_addr learned my addr 172.21.15.1:0/4006220571 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:03.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.275+0000 7fdda2566700 1 --2- 172.21.15.1:0/4006220571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b579aa8b0 0x563b579b4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:03.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.275+0000 7fdda1d65700 1 -- 172.21.15.1:0/4006220571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b579a6470 msgr2=0x563b579b4590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-16T16:40:03.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.275+0000 7fdda1d65700 1 --2- 172.21.15.1:0/4006220571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b579a6470 0x563b579b4590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.275+0000 7fdda1d65700 1 -- 172.21.15.1:0/4006220571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b579aa8b0 msgr2=0x563b579b4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:03.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.275+0000 7fdda1d65700 1 --2- 172.21.15.1:0/4006220571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b579aa8b0 0x563b579b4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.275+0000 7fdda1d65700 1 -- 172.21.15.1:0/4006220571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b57778630 con 0x563b579a5b60 2023-11-16T16:40:03.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.276+0000 7fdda1d65700 1 --2- 172.21.15.1:0/4006220571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b579a5b60 0x563b579b4050 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdd98096400 tx=0x7fdd9800ef70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f270cd74cdc20da4 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:03.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.276+0000 7fdd857fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-11-16T16:40:03.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.276+0000 7fdd857fa700 1 -- 172.21.15.1:0/4006220571 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd980b7070 con 0x563b579a5b60 2023-11-16T16:40:03.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.276+0000 7fdd857fa700 1 -- 172.21.15.1:0/4006220571 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fdd98084040 con 0x563b579a5b60 2023-11-16T16:40:03.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.276+0000 7fddb0155b80 1 -- 172.21.15.1:0/4006220571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdd8c011c30 con 0x563b579a5b60 2023-11-16T16:40:03.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.276+0000 7fdd857fa700 1 -- 172.21.15.1:0/4006220571 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd980994d0 con 0x563b579a5b60 2023-11-16T16:40:03.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.276+0000 7fddb0155b80 10 client.25253 fetch_fsmap learned FSMap version 23 2023-11-16T16:40:03.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.276+0000 7fddb0155b80 1 -- 172.21.15.1:0/4006220571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x563b578bd190 con 0x563b579a5b60 2023-11-16T16:40:03.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.276+0000 7fdd857fa700 1 -- 172.21.15.1:0/4006220571 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd8c011c30 con 0x563b579a5b60 2023-11-16T16:40:03.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.277+0000 7fdd857fa700 1 -- 172.21.15.1:0/4006220571 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdd98099bf0 con 0x563b579a5b60 2023-11-16T16:40:03.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.277+0000 7fddb0155b80 10 client.25253 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:40:03.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.277+0000 7fddb0155b80 10 client.25253 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:40:03.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.277+0000 7fddb0155b80 20 client.25253 populate_metadata read hostname 'smithi001' 2023-11-16T16:40:03.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.277+0000 7fddb0155b80 1 --2- 172.21.15.1:0/4006220571 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x563b57a13da0 0x563b57a161b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:03.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.277+0000 7fddb0155b80 4 client.25253 mds_command: new command op to 24521 tid=0[{"prefix": "scrub status"}] 2023-11-16T16:40:03.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.277+0000 7fddb0155b80 1 -- 172.21.15.1:0/4006220571 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563b578c5660 con 0x563b57a13da0 2023-11-16T16:40:03.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.277+0000 7fdda1564700 1 --2- 172.21.15.1:0/4006220571 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x563b57a13da0 0x563b57a161b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:03.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.277+0000 7fdd857fa700 10 client.25253 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:40:03.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.277+0000 7fdda1564700 1 --2- 172.21.15.1:0/4006220571 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x563b57a13da0 0x563b57a161b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb0f48d564b31b48 server_cookie=97c529bbc35bab91 in_seq=0 out_seq=0 2023-11-16T16:40:03.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.278+0000 7fdd857fa700 1 -- 172.21.15.1:0/4006220571 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563b578c5660 con 0x563b57a13da0 2023-11-16T16:40:03.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.278+0000 7fdd857fa700 10 client.25253 handle_command_reply: tid=0 2023-11-16T16:40:03.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.278+0000 7fddb0155b80 1 client.25253 shutdown 2023-11-16T16:40:03.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.278+0000 7fddb0155b80 1 -- 172.21.15.1:0/4006220571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b579a5b60 msgr2=0x563b579b4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:03.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.278+0000 7fddb0155b80 1 --2- 172.21.15.1:0/4006220571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b579a5b60 0x563b579b4050 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdd98096400 tx=0x7fdd9800ef70 comp rx=0 tx=0).stop 2023-11-16T16:40:03.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.278+0000 7fddb0155b80 1 -- 172.21.15.1:0/4006220571 shutdown_connections 2023-11-16T16:40:03.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.278+0000 7fddb0155b80 1 --2- 172.21.15.1:0/4006220571 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x563b57a13da0 0x563b57a161b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.278+0000 7fddb0155b80 1 --2- 172.21.15.1:0/4006220571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b579aa8b0 0x563b579b4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.278+0000 7fddb0155b80 1 --2- 172.21.15.1:0/4006220571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b579a5b60 0x563b579b4050 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.278+0000 7fddb0155b80 1 --2- 172.21.15.1:0/4006220571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563b579a6470 0x563b579b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.278+0000 7fddb0155b80 1 -- 172.21.15.1:0/4006220571 >> 172.21.15.1:0/4006220571 conn(0x563b5799b360 msgr2=0x563b579a4960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:03.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.278+0000 7fdd857fa700 0 client.25253 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:40:03.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.278+0000 7fddb0155b80 1 -- 172.21.15.1:0/4006220571 shutdown_connections 2023-11-16T16:40:03.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.278+0000 7fddb0155b80 1 -- 172.21.15.1:0/4006220571 wait complete. 2023-11-16T16:40:03.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.278+0000 7fddb0155b80 20 client.25253 trim_cache size 0 max 16384 2023-11-16T16:40:03.285 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:40:03.285 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2023-11-16T16:40:03.285 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2023-11-16T16:40:03.286 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:40:03.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.279+0000 7fdda3fc9700 1 -- 172.21.15.1:0/1995158357 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fdd880a13e0 msgr2=0x7fdd880a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:03.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.279+0000 7fdda3fc9700 1 --2- 172.21.15.1:0/1995158357 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fdd880a13e0 0x7fdd880a3890 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c10e660 tx=0x7fdd8c00b5b0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.279+0000 7fdda3fc9700 1 -- 172.21.15.1:0/1995158357 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd9c0fe910 msgr2=0x7fdd9c10ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:03.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.279+0000 7fdda3fc9700 1 --2- 172.21.15.1:0/1995158357 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd9c0fe910 0x7fdd9c10ce70 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fdd9800e970 tx=0x7fdd9800ed30 comp rx=0 tx=0).stop 2023-11-16T16:40:03.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.279+0000 7fdda3fc9700 1 -- 172.21.15.1:0/1995158357 shutdown_connections 2023-11-16T16:40:03.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.279+0000 7fdda3fc9700 1 --2- 172.21.15.1:0/1995158357 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fdd880a13e0 0x7fdd880a3890 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.279+0000 7fdda3fc9700 1 --2- 172.21.15.1:0/1995158357 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdd9c104920 0x7fdd9c1a45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.279+0000 7fdda3fc9700 1 --2- 172.21.15.1:0/1995158357 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd9c0fe910 0x7fdd9c10ce70 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.279+0000 7fdda3fc9700 1 --2- 172.21.15.1:0/1995158357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd9c0ff2c0 0x7fdd9c10d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.279+0000 7fdda3fc9700 1 -- 172.21.15.1:0/1995158357 >> 172.21.15.1:0/1995158357 conn(0x7fdd9c075880 msgr2=0x7fdd9c0fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:03.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.279+0000 7fdda3fc9700 1 -- 172.21.15.1:0/1995158357 shutdown_connections 2023-11-16T16:40:03.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.279+0000 7fdda3fc9700 1 -- 172.21.15.1:0/1995158357 wait complete. 2023-11-16T16:40:03.297 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-16T16:40:03.297 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:40:03.331 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:03 smithi001 ceph-mon[98800]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:40:03.331 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:03 smithi001 ceph-mon[98800]: mgrmap e24: x(active, since 52s), standbys: y, z 2023-11-16T16:40:03.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.493+0000 7f5fdda9c700 1 -- 172.21.15.1:0/3966639980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fd80ff460 msgr2=0x7f5fd80ff830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:03.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.493+0000 7f5fdda9c700 1 --2- 172.21.15.1:0/3966639980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fd80ff460 0x7f5fd80ff830 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5fc80099c0 tx=0x7f5fc8009cd0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.493+0000 7f5fdda9c700 1 -- 172.21.15.1:0/3966639980 shutdown_connections 2023-11-16T16:40:03.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.493+0000 7f5fdda9c700 1 --2- 172.21.15.1:0/3966639980 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fd81044a0 0x7f5fd810c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.493+0000 7f5fdda9c700 1 --2- 172.21.15.1:0/3966639980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fd80ff460 0x7f5fd80ff830 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.493+0000 7f5fdda9c700 1 --2- 172.21.15.1:0/3966639980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fd80ffe00 0x7f5fd8103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.493+0000 7f5fdda9c700 1 -- 172.21.15.1:0/3966639980 >> 172.21.15.1:0/3966639980 conn(0x7f5fd8076310 msgr2=0x7f5fd8076710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:03.495 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.494+0000 7f5fdda9c700 1 -- 172.21.15.1:0/3966639980 shutdown_connections 2023-11-16T16:40:03.495 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.494+0000 7f5fdda9c700 1 -- 172.21.15.1:0/3966639980 wait complete. 2023-11-16T16:40:03.495 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.494+0000 7f5fdda9c700 1 Processor -- start 2023-11-16T16:40:03.495 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.495+0000 7f5fdda9c700 1 -- start start 2023-11-16T16:40:03.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.495+0000 7f5fdda9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fd80ff460 0x7f5fd81973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:03.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.495+0000 7f5fdda9c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fd80ffe00 0x7f5fd8197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:03.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.495+0000 7f5fdda9c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fd81044a0 0x7f5fd81a2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:03.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.495+0000 7f5fdda9c700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd8198250 con 0x7f5fd81044a0 2023-11-16T16:40:03.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.495+0000 7f5fdda9c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd81a2990 con 0x7f5fd80ff460 2023-11-16T16:40:03.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.495+0000 7f5fdda9c700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd81a2b50 con 0x7f5fd80ffe00 2023-11-16T16:40:03.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.495+0000 7f5fd7fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fd81044a0 0x7f5fd81a2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:03.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.495+0000 7f5fd7fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fd81044a0 0x7f5fd81a2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44040/0 (socket says 172.21.15.1:44040) 2023-11-16T16:40:03.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.495+0000 7f5fd7fff700 1 -- 172.21.15.1:0/1783190761 learned_addr learned my addr 172.21.15.1:0/1783190761 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:03.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.495+0000 7f5fd77fe700 1 --2- 172.21.15.1:0/1783190761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fd80ff460 0x7f5fd81973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:03.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.495+0000 7f5fd6ffd700 1 --2- 172.21.15.1:0/1783190761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fd80ffe00 0x7f5fd8197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:03.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.496+0000 7f5fd7fff700 1 -- 172.21.15.1:0/1783190761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fd80ffe00 msgr2=0x7f5fd8197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:03.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.496+0000 7f5fd7fff700 1 --2- 172.21.15.1:0/1783190761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fd80ffe00 0x7f5fd8197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.496+0000 7f5fd7fff700 1 -- 172.21.15.1:0/1783190761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fd80ff460 msgr2=0x7f5fd81973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:03.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.496+0000 7f5fd7fff700 1 --2- 172.21.15.1:0/1783190761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fd80ff460 0x7f5fd81973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:03.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.496+0000 7f5fd7fff700 1 -- 172.21.15.1:0/1783190761 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc80095f0 con 0x7f5fd81044a0 2023-11-16T16:40:03.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.496+0000 7f5fd7fff700 1 --2- 172.21.15.1:0/1783190761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fd81044a0 0x7f5fd81a2450 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5fd000b9e0 tx=0x7f5fd000bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:03.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.496+0000 7f5fd4ff9700 1 -- 172.21.15.1:0/1783190761 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd000d8e0 con 0x7f5fd81044a0 2023-11-16T16:40:03.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.496+0000 7f5fdda9c700 1 -- 172.21.15.1:0/1783190761 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd810f390 con 0x7f5fd81044a0 2023-11-16T16:40:03.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.496+0000 7f5fd4ff9700 1 -- 172.21.15.1:0/1783190761 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fd0009880 con 0x7f5fd81044a0 2023-11-16T16:40:03.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.496+0000 7f5fd4ff9700 1 -- 172.21.15.1:0/1783190761 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd00143d0 con 0x7f5fd81044a0 2023-11-16T16:40:03.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.496+0000 7f5fdda9c700 1 -- 172.21.15.1:0/1783190761 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd810f200 con 0x7f5fd81044a0 2023-11-16T16:40:03.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.497+0000 7f5fdda9c700 1 -- 172.21.15.1:0/1783190761 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fd810f520 con 0x7f5fd81044a0 2023-11-16T16:40:03.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.499+0000 7f5fd4ff9700 1 -- 172.21.15.1:0/1783190761 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 24) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f5fd000da40 con 0x7f5fd81044a0 2023-11-16T16:40:03.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.500+0000 7f5fd4ff9700 1 --2- 172.21.15.1:0/1783190761 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5fc00a13e0 0x7f5fc00a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:03.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.500+0000 7f5fd4ff9700 1 -- 172.21.15.1:0/1783190761 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f5fd0011070 con 0x7f5fd81044a0 2023-11-16T16:40:03.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.500+0000 7f5fd77fe700 1 --2- 172.21.15.1:0/1783190761 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5fc00a13e0 0x7f5fc00a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:03.502 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.501+0000 7f5fd77fe700 1 --2- 172.21.15.1:0/1783190761 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5fc00a13e0 0x7f5fc00a3890 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8009990 tx=0x7f5fc8000bc0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:03.507 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.507+0000 7f5fd4ff9700 1 -- 172.21.15.1:0/1783190761 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5fd004e730 con 0x7f5fd81044a0 2023-11-16T16:40:03.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:03 smithi026 ceph-mon[100112]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:40:03.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:03 smithi026 ceph-mon[100112]: mgrmap e24: x(active, since 52s), standbys: y, z 2023-11-16T16:40:03.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:03 smithi053 ceph-mon[100638]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:40:03.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:03 smithi053 ceph-mon[100638]: mgrmap e24: x(active, since 52s), standbys: y, z 2023-11-16T16:40:03.996 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.995+0000 7f5fdda9c700 1 -- 172.21.15.1:0/1783190761 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5fd8067260 con 0x7f5fd81044a0 2023-11-16T16:40:03.997 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.996+0000 7f5fd4ff9700 1 -- 172.21.15.1:0/1783190761 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10774 (secure 0 0 0) 0x7f5fd0099790 con 0x7f5fd81044a0 2023-11-16T16:40:03.998 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:40:03.998 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2587563684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2587563684},{"type":"v1","addr":"172.21.15.1:6835","nonce":2587563684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/277936168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":277936168},{"type":"v1","addr":"172.21.15.1:6837","nonce":277936168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/1172575045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1172575045},{"type":"v1","addr":"172.21.15.1:6839","nonce":1172575045}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/2631175899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2631175899},{"type":"v1","addr":"172.21.15.1:6841","nonce":2631175899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3693905267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3693905267},{"type":"v1","addr":"172.21.15.53:6833","nonce":3693905267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/164668033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":164668033},{"type":"v1","addr":"172.21.15.26:6833","nonce":164668033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-16T16:37:13.918943+0000","modified":"2023-11-16T16:37:19.292898+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2140542192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2140542192},{"type":"v1","addr":"172.21.15.53:6835","nonce":2140542192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1010760275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1010760275},{"type":"v1","addr":"172.21.15.26:6835","nonce":1010760275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3197060302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3197060302},{"type":"v1","addr":"172.21.15.53:6837","nonce":3197060302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6837/1453504863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1453504863},{"type":"v1","addr":"172.21.15.26:6837","nonce":1453504863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.53:6839/1044753024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1044753024},{"type":"v1","addr":"172.21.15.53:6839","nonce":1044753024}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6839/2192205949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2192205949},{"type":"v1","addr":"172.21.15.26:6839","nonce":2192205949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-16T16:40:04.000 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.999+0000 7f5fdda9c700 1 -- 172.21.15.1:0/1783190761 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5fc00a13e0 msgr2=0x7f5fc00a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:04.001 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.999+0000 7f5fdda9c700 1 --2- 172.21.15.1:0/1783190761 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5fc00a13e0 0x7f5fc00a3890 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8009990 tx=0x7f5fc8000bc0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.001 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.999+0000 7f5fdda9c700 1 -- 172.21.15.1:0/1783190761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fd81044a0 msgr2=0x7f5fd81a2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:04.001 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.999+0000 7f5fdda9c700 1 --2- 172.21.15.1:0/1783190761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fd81044a0 0x7f5fd81a2450 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5fd000b9e0 tx=0x7f5fd000bcf0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.001 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.999+0000 7f5fdda9c700 1 -- 172.21.15.1:0/1783190761 shutdown_connections 2023-11-16T16:40:04.001 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.999+0000 7f5fdda9c700 1 --2- 172.21.15.1:0/1783190761 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5fc00a13e0 0x7f5fc00a3890 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.001 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.999+0000 7f5fdda9c700 1 --2- 172.21.15.1:0/1783190761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5fd81044a0 0x7f5fd81a2450 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.001 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.999+0000 7f5fdda9c700 1 --2- 172.21.15.1:0/1783190761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fd80ff460 0x7f5fd81973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.002 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.999+0000 7f5fdda9c700 1 --2- 172.21.15.1:0/1783190761 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5fd80ffe00 0x7f5fd8197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.002 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.999+0000 7f5fdda9c700 1 -- 172.21.15.1:0/1783190761 >> 172.21.15.1:0/1783190761 conn(0x7f5fd8076310 msgr2=0x7f5fd80fcaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:04.002 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.999+0000 7f5fdda9c700 1 -- 172.21.15.1:0/1783190761 shutdown_connections 2023-11-16T16:40:04.002 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:03.999+0000 7f5fdda9c700 1 -- 172.21.15.1:0/1783190761 wait complete. 2023-11-16T16:40:04.002 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 23 2023-11-16T16:40:04.017 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-11-16T16:40:04.254 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.253+0000 7f5e4951b700 1 -- 172.21.15.1:0/3770283151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e441002a0 msgr2=0x7f5e4410bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:04.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.253+0000 7f5e4951b700 1 --2- 172.21.15.1:0/3770283151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e441002a0 0x7f5e4410bfd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c009f90 tx=0x7f5e3c00b330 comp rx=0 tx=0).stop 2023-11-16T16:40:04.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.253+0000 7f5e4951b700 1 -- 172.21.15.1:0/3770283151 shutdown_connections 2023-11-16T16:40:04.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.253+0000 7f5e4951b700 1 --2- 172.21.15.1:0/3770283151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e441002a0 0x7f5e4410bfd0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.253+0000 7f5e4951b700 1 --2- 172.21.15.1:0/3770283151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e44104140 0x7f5e44104510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.253+0000 7f5e4951b700 1 --2- 172.21.15.1:0/3770283151 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e440ff8f0 0x7f5e440ffd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.253+0000 7f5e4951b700 1 -- 172.21.15.1:0/3770283151 >> 172.21.15.1:0/3770283151 conn(0x7f5e44075880 msgr2=0x7f5e44075c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:04.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.253+0000 7f5e4951b700 1 -- 172.21.15.1:0/3770283151 shutdown_connections 2023-11-16T16:40:04.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.253+0000 7f5e4951b700 1 -- 172.21.15.1:0/3770283151 wait complete. 2023-11-16T16:40:04.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.254+0000 7f5e4951b700 1 Processor -- start 2023-11-16T16:40:04.257 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.254+0000 7f5e4951b700 1 -- start start 2023-11-16T16:40:04.257 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.254+0000 7f5e4951b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e440ff8f0 0x7f5e4410c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.257 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.254+0000 7f5e4951b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e441002a0 0x7f5e4410ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.257 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.255+0000 7f5e4951b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e44104140 0x7f5e441a45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.257 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.255+0000 7f5e4951b700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4410d4e0 con 0x7f5e441002a0 2023-11-16T16:40:04.257 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.255+0000 7f5e4951b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e441a4b10 con 0x7f5e440ff8f0 2023-11-16T16:40:04.257 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.255+0000 7f5e4951b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e441a4c50 con 0x7f5e44104140 2023-11-16T16:40:04.257 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.255+0000 7f5e427fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e441002a0 0x7f5e4410ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.258 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.255+0000 7f5e427fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e441002a0 0x7f5e4410ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44052/0 (socket says 172.21.15.1:44052) 2023-11-16T16:40:04.258 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.255+0000 7f5e427fc700 1 -- 172.21.15.1:0/1741704049 learned_addr learned my addr 172.21.15.1:0/1741704049 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:04.258 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.255+0000 7f5e42ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e440ff8f0 0x7f5e4410c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.258 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.255+0000 7f5e437fe700 1 --2- 172.21.15.1:0/1741704049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e44104140 0x7f5e441a45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.258 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.255+0000 7f5e427fc700 1 -- 172.21.15.1:0/1741704049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e44104140 msgr2=0x7f5e441a45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:04.258 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.255+0000 7f5e427fc700 1 --2- 172.21.15.1:0/1741704049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e44104140 0x7f5e441a45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.258 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.255+0000 7f5e427fc700 1 -- 172.21.15.1:0/1741704049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e440ff8f0 msgr2=0x7f5e4410c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:04.258 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.255+0000 7f5e427fc700 1 --2- 172.21.15.1:0/1741704049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e440ff8f0 0x7f5e4410c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.258 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.255+0000 7f5e427fc700 1 -- 172.21.15.1:0/1741704049 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3c009c40 con 0x7f5e441002a0 2023-11-16T16:40:04.258 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.255+0000 7f5e427fc700 1 --2- 172.21.15.1:0/1741704049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e441002a0 0x7f5e4410ccb0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5e3800e9c0 tx=0x7f5e3800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:04.258 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.256+0000 7f5e2bfff700 1 -- 172.21.15.1:0/1741704049 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3800cbd0 con 0x7f5e441002a0 2023-11-16T16:40:04.258 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.256+0000 7f5e4951b700 1 -- 172.21.15.1:0/1741704049 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e441103c0 con 0x7f5e441002a0 2023-11-16T16:40:04.259 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.256+0000 7f5e2bfff700 1 -- 172.21.15.1:0/1741704049 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e3800eeb0 con 0x7f5e441002a0 2023-11-16T16:40:04.260 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.256+0000 7f5e2bfff700 1 -- 172.21.15.1:0/1741704049 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e38005580 con 0x7f5e441002a0 2023-11-16T16:40:04.260 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.256+0000 7f5e4951b700 1 -- 172.21.15.1:0/1741704049 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e44110230 con 0x7f5e441002a0 2023-11-16T16:40:04.260 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.260+0000 7f5e2bfff700 1 -- 172.21.15.1:0/1741704049 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 24) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f5e38004750 con 0x7f5e441002a0 2023-11-16T16:40:04.261 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.260+0000 7f5e2bfff700 1 --2- 172.21.15.1:0/1741704049 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5e2c0a13e0 0x7f5e2c0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.261 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.261+0000 7f5e556a7b80 1 Processor -- start 2023-11-16T16:40:04.261 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.261+0000 7f5e2bfff700 1 -- 172.21.15.1:0/1741704049 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f5e38014070 con 0x7f5e441002a0 2023-11-16T16:40:04.263 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.261+0000 7f5e42ffd700 1 --2- 172.21.15.1:0/1741704049 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5e2c0a13e0 0x7f5e2c0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.263 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.261+0000 7f5e556a7b80 1 -- start start 2023-11-16T16:40:04.263 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.261+0000 7f5e556a7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9b897b60 0x55fe9b897f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.263 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.261+0000 7f5e556a7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe9b898470 0x55fe9b89c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.263 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.261+0000 7f5e556a7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9b89c8b0 0x55fe9b89cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.264 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.261+0000 7f5e556a7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe9b79d060 con 0x55fe9b89c8b0 2023-11-16T16:40:04.264 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.261+0000 7f5e556a7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe9b7ef8a0 con 0x55fe9b898470 2023-11-16T16:40:04.264 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.261+0000 7f5e556a7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe9b7e8d30 con 0x55fe9b897b60 2023-11-16T16:40:04.264 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.262+0000 7f5e437fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9b89c8b0 0x55fe9b89cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.264 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.262+0000 7f5e437fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9b89c8b0 0x55fe9b89cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44060/0 (socket says 172.21.15.1:44060) 2023-11-16T16:40:04.264 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.262+0000 7f5e437fe700 1 -- 172.21.15.1:0/3364170011 learned_addr learned my addr 172.21.15.1:0/3364170011 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:04.264 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.262+0000 7f5e427fc700 1 --2- 172.21.15.1:0/3364170011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe9b898470 0x55fe9b89c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.264 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.262+0000 7f5e437fe700 1 -- 172.21.15.1:0/3364170011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9b897b60 msgr2=0x55fe9b897f30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-16T16:40:04.264 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.262+0000 7f5e437fe700 1 --2- 172.21.15.1:0/3364170011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9b897b60 0x55fe9b897f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.264 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.262+0000 7f5e437fe700 1 -- 172.21.15.1:0/3364170011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe9b898470 msgr2=0x55fe9b89c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:04.264 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.262+0000 7f5e437fe700 1 --2- 172.21.15.1:0/3364170011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe9b898470 0x55fe9b89c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.264 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.262+0000 7f5e437fe700 1 -- 172.21.15.1:0/3364170011 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe9b66a630 con 0x55fe9b89c8b0 2023-11-16T16:40:04.264 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.262+0000 7f5e437fe700 1 --2- 172.21.15.1:0/3364170011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9b89c8b0 0x55fe9b89cd00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c015000 tx=0x7f5e3c009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d0cbeb44b53227 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:04.265 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.262+0000 7f5e297fa700 1 -- 172.21.15.1:0/3364170011 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c00e030 con 0x55fe9b89c8b0 2023-11-16T16:40:04.265 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.263+0000 7f5e297fa700 1 -- 172.21.15.1:0/3364170011 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e3c007d00 con 0x55fe9b89c8b0 2023-11-16T16:40:04.265 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.263+0000 7f5e297fa700 1 -- 172.21.15.1:0/3364170011 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c00bce0 con 0x55fe9b89c8b0 2023-11-16T16:40:04.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.264+0000 7f5e556a7b80 1 -- 172.21.15.1:0/3364170011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9b89c8b0 msgr2=0x55fe9b89cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:04.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.264+0000 7f5e42ffd700 1 --2- 172.21.15.1:0/1741704049 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5e2c0a13e0 0x7f5e2c0a3890 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5e34009d00 tx=0x7f5e34009260 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:04.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.264+0000 7f5e556a7b80 1 --2- 172.21.15.1:0/3364170011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9b89c8b0 0x55fe9b89cd00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c015000 tx=0x7f5e3c009300 comp rx=0 tx=0).stop 2023-11-16T16:40:04.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.264+0000 7f5e556a7b80 1 -- 172.21.15.1:0/3364170011 shutdown_connections 2023-11-16T16:40:04.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.264+0000 7f5e556a7b80 1 --2- 172.21.15.1:0/3364170011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9b89c8b0 0x55fe9b89cd00 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.264+0000 7f5e556a7b80 1 --2- 172.21.15.1:0/3364170011 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9b897b60 0x55fe9b897f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.264+0000 7f5e556a7b80 1 --2- 172.21.15.1:0/3364170011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe9b898470 0x55fe9b89c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.264+0000 7f5e556a7b80 1 -- 172.21.15.1:0/3364170011 >> 172.21.15.1:0/3364170011 conn(0x55fe9b88d360 msgr2=0x55fe9b892b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:04.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.264+0000 7f5e556a7b80 1 -- 172.21.15.1:0/3364170011 shutdown_connections 2023-11-16T16:40:04.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.264+0000 7f5e556a7b80 1 -- 172.21.15.1:0/3364170011 wait complete. 2023-11-16T16:40:04.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.264+0000 7f5e556a7b80 1 -- start start 2023-11-16T16:40:04.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.264+0000 7f5e556a7b80 1 Processor -- start 2023-11-16T16:40:04.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.265+0000 7f5e556a7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe9b897b60 0x55fe9b8a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.265+0000 7f5e556a7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9b898470 0x55fe9b8a1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.265+0000 7f5e556a7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9b89c8b0 0x55fe9b8a1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.265+0000 7f5e556a7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe9b78a8e0 con 0x55fe9b89c8b0 2023-11-16T16:40:04.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.265+0000 7f5e556a7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe9b78a580 con 0x55fe9b897b60 2023-11-16T16:40:04.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.265+0000 7f5e556a7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe9b66c920 con 0x55fe9b898470 2023-11-16T16:40:04.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.265+0000 7f5e556a7b80 10 client.0 _ll_register_callbacks cb 0x55fe9b7a8780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:40:04.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.265+0000 7f5e437fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9b89c8b0 0x55fe9b8a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.265+0000 7f5e437fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9b89c8b0 0x55fe9b8a1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44068/0 (socket says 172.21.15.1:44068) 2023-11-16T16:40:04.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.265+0000 7f5e437fe700 1 -- 172.21.15.1:0/3225625593 learned_addr learned my addr 172.21.15.1:0/3225625593 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:04.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.265+0000 7f5e427fc700 1 --2- 172.21.15.1:0/3225625593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9b898470 0x55fe9b8a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.265+0000 7f5e437fe700 1 -- 172.21.15.1:0/3225625593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9b898470 msgr2=0x55fe9b8a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:04.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.265+0000 7f5e437fe700 1 --2- 172.21.15.1:0/3225625593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9b898470 0x55fe9b8a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.265+0000 7f5e437fe700 1 -- 172.21.15.1:0/3225625593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe9b897b60 msgr2=0x55fe9b8a3f80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-16T16:40:04.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.265+0000 7f5e437fe700 1 --2- 172.21.15.1:0/3225625593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe9b897b60 0x55fe9b8a3f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.265+0000 7f5e437fe700 1 -- 172.21.15.1:0/3225625593 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e440fe9a0 con 0x55fe9b89c8b0 2023-11-16T16:40:04.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.266+0000 7f5e437fe700 1 --2- 172.21.15.1:0/3225625593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9b89c8b0 0x55fe9b8a1830 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c009c10 tx=0x7f5e3c0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62c26d428a1e4a45 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:04.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.266+0000 7f5e26ff5700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:40:04.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.266+0000 7f5e26ff5700 1 -- 172.21.15.1:0/3225625593 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c00e040 con 0x55fe9b89c8b0 2023-11-16T16:40:04.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.266+0000 7f5e26ff5700 1 -- 172.21.15.1:0/3225625593 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f5e3c00be00 con 0x55fe9b89c8b0 2023-11-16T16:40:04.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.266+0000 7f5e556a7b80 1 -- 172.21.15.1:0/3225625593 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe9b88ab20 con 0x55fe9b89c8b0 2023-11-16T16:40:04.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.266+0000 7f5e26ff5700 1 -- 172.21.15.1:0/3225625593 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e3c005040 con 0x55fe9b89c8b0 2023-11-16T16:40:04.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.266+0000 7f5e556a7b80 10 client.15663 fetch_fsmap learned FSMap version 23 2023-11-16T16:40:04.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.266+0000 7f5e556a7b80 1 -- 172.21.15.1:0/3225625593 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55fe9b7af190 con 0x55fe9b89c8b0 2023-11-16T16:40:04.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.266+0000 7f5e26ff5700 1 -- 172.21.15.1:0/3225625593 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fe9b88ab20 con 0x55fe9b89c8b0 2023-11-16T16:40:04.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.267+0000 7f5e26ff5700 1 -- 172.21.15.1:0/3225625593 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5e3c032070 con 0x55fe9b89c8b0 2023-11-16T16:40:04.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.267+0000 7f5e556a7b80 10 client.15663 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:40:04.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.267+0000 7f5e556a7b80 10 client.15663 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-11-16T16:40:04.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.267+0000 7f5e556a7b80 20 client.15663 populate_metadata read hostname 'smithi001' 2023-11-16T16:40:04.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.267+0000 7f5e556a7b80 1 --2- 172.21.15.1:0/3225625593 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55fe9b8b1340 0x55fe9b8b3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.267+0000 7f5e556a7b80 4 client.15663 mds_command: new command op to 24482 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:40:04.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.267+0000 7f5e556a7b80 1 -- 172.21.15.1:0/3225625593 --> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe9b7ab0a0 con 0x55fe9b8b1340 2023-11-16T16:40:04.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.267+0000 7f5e42ffd700 1 --2- 172.21.15.1:0/3225625593 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55fe9b8b1340 0x55fe9b8b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.267+0000 7f5e26ff5700 10 client.15663 ms_handle_connect on v2:172.21.15.26:6836/1453504863 2023-11-16T16:40:04.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.267+0000 7f5e42ffd700 1 --2- 172.21.15.1:0/3225625593 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55fe9b8b1340 0x55fe9b8b3720 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f43cd5e0bb0bc2b4 server_cookie=48c1aecca4707dd7 in_seq=0 out_seq=0 2023-11-16T16:40:04.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.268+0000 7f5e26ff5700 1 -- 172.21.15.1:0/3225625593 <== mds.2 v2:172.21.15.26:6836/1453504863 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fe9b7ab0a0 con 0x55fe9b8b1340 2023-11-16T16:40:04.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.268+0000 7f5e26ff5700 10 client.15663 handle_command_reply: tid=0 2023-11-16T16:40:04.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.268+0000 7f5e556a7b80 1 client.15663 shutdown 2023-11-16T16:40:04.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.268+0000 7f5e556a7b80 1 -- 172.21.15.1:0/3225625593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9b89c8b0 msgr2=0x55fe9b8a1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:04.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.268+0000 7f5e556a7b80 1 --2- 172.21.15.1:0/3225625593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9b89c8b0 0x55fe9b8a1830 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c009c10 tx=0x7f5e3c0096c0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.268+0000 7f5e556a7b80 1 -- 172.21.15.1:0/3225625593 shutdown_connections 2023-11-16T16:40:04.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.268+0000 7f5e556a7b80 1 --2- 172.21.15.1:0/3225625593 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55fe9b8b1340 0x55fe9b8b3720 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.268+0000 7f5e556a7b80 1 --2- 172.21.15.1:0/3225625593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9b89c8b0 0x55fe9b8a1830 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.268+0000 7f5e556a7b80 1 --2- 172.21.15.1:0/3225625593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe9b897b60 0x55fe9b8a3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.268+0000 7f5e556a7b80 1 --2- 172.21.15.1:0/3225625593 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9b898470 0x55fe9b8a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.268+0000 7f5e556a7b80 1 -- 172.21.15.1:0/3225625593 >> 172.21.15.1:0/3225625593 conn(0x55fe9b88d360 msgr2=0x55fe9b89dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:04.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.269+0000 7f5e26ff5700 0 client.15663 ms_handle_reset on v2:172.21.15.26:6836/1453504863 2023-11-16T16:40:04.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.269+0000 7f5e556a7b80 1 -- 172.21.15.1:0/3225625593 shutdown_connections 2023-11-16T16:40:04.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.269+0000 7f5e556a7b80 1 -- 172.21.15.1:0/3225625593 wait complete. 2023-11-16T16:40:04.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.269+0000 7f5e556a7b80 20 client.15663 trim_cache size 0 max 16384 2023-11-16T16:40:04.295 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.294+0000 7f5e556a7b80 1 Processor -- start 2023-11-16T16:40:04.296 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.294+0000 7f5e556a7b80 1 -- start start 2023-11-16T16:40:04.296 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.295+0000 7f5e556a7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe9b897b60 0x55fe9b8a8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.296 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.295+0000 7f5e556a7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9b898470 0x55fe9b8a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.296 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.295+0000 7f5e556a7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9b89c8b0 0x55fe9b89a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.296 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.295+0000 7f5e556a7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe9b8941c0 con 0x55fe9b898470 2023-11-16T16:40:04.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.295+0000 7f5e556a7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe9b898bc0 con 0x55fe9b897b60 2023-11-16T16:40:04.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.295+0000 7f5e556a7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe9b8a27e0 con 0x55fe9b89c8b0 2023-11-16T16:40:04.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.295+0000 7f5e427fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9b898470 0x55fe9b8a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.295+0000 7f5e427fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9b898470 0x55fe9b8a3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44074/0 (socket says 172.21.15.1:44074) 2023-11-16T16:40:04.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.295+0000 7f5e427fc700 1 -- 172.21.15.1:0/456327719 learned_addr learned my addr 172.21.15.1:0/456327719 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:04.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.295+0000 7f5e437fe700 1 --2- 172.21.15.1:0/456327719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9b89c8b0 0x55fe9b89a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.295+0000 7f5e42ffd700 1 --2- 172.21.15.1:0/456327719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe9b897b60 0x55fe9b8a8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.295+0000 7f5e427fc700 1 -- 172.21.15.1:0/456327719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9b89c8b0 msgr2=0x55fe9b89a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:04.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.295+0000 7f5e427fc700 1 --2- 172.21.15.1:0/456327719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9b89c8b0 0x55fe9b89a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.295+0000 7f5e427fc700 1 -- 172.21.15.1:0/456327719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe9b897b60 msgr2=0x55fe9b8a8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:04.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.295+0000 7f5e427fc700 1 --2- 172.21.15.1:0/456327719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe9b897b60 0x55fe9b8a8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.295+0000 7f5e427fc700 1 -- 172.21.15.1:0/456327719 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe9b7aeee0 con 0x55fe9b898470 2023-11-16T16:40:04.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.296+0000 7f5e427fc700 1 --2- 172.21.15.1:0/456327719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9b898470 0x55fe9b8a3bb0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5e38000c00 tx=0x7f5e38096570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f30b9373add08553 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:04.300 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.296+0000 7f5e29ffb700 1 -- 172.21.15.1:0/456327719 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3809e1e0 con 0x55fe9b898470 2023-11-16T16:40:04.300 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.296+0000 7f5e29ffb700 1 -- 172.21.15.1:0/456327719 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e380a2600 con 0x55fe9b898470 2023-11-16T16:40:04.300 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.296+0000 7f5e29ffb700 1 -- 172.21.15.1:0/456327719 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e380a23d0 con 0x55fe9b898470 2023-11-16T16:40:04.300 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.297+0000 7f5e556a7b80 1 -- 172.21.15.1:0/456327719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9b898470 msgr2=0x55fe9b8a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:04.300 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.297+0000 7f5e556a7b80 1 --2- 172.21.15.1:0/456327719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9b898470 0x55fe9b8a3bb0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5e38000c00 tx=0x7f5e38096570 comp rx=0 tx=0).stop 2023-11-16T16:40:04.300 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.297+0000 7f5e556a7b80 1 -- 172.21.15.1:0/456327719 shutdown_connections 2023-11-16T16:40:04.300 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.297+0000 7f5e556a7b80 1 --2- 172.21.15.1:0/456327719 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9b89c8b0 0x55fe9b89a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.300 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.297+0000 7f5e556a7b80 1 --2- 172.21.15.1:0/456327719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe9b897b60 0x55fe9b8a8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.300 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.297+0000 7f5e556a7b80 1 --2- 172.21.15.1:0/456327719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9b898470 0x55fe9b8a3bb0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.297+0000 7f5e556a7b80 1 -- 172.21.15.1:0/456327719 >> 172.21.15.1:0/456327719 conn(0x55fe9b88d360 msgr2=0x55fe9b8adac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:04.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.297+0000 7f5e556a7b80 1 -- 172.21.15.1:0/456327719 shutdown_connections 2023-11-16T16:40:04.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.297+0000 7f5e556a7b80 1 -- 172.21.15.1:0/456327719 wait complete. 2023-11-16T16:40:04.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.297+0000 7f5e556a7b80 1 -- start start 2023-11-16T16:40:04.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.297+0000 7f5e556a7b80 1 Processor -- start 2023-11-16T16:40:04.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.298+0000 7f5e556a7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe9b897b60 0x55fe9b8fe1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.298+0000 7f5e556a7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9b898470 0x55fe9b8fe710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.298+0000 7f5e556a7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9b89c8b0 0x55fe9b8fc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.298+0000 7f5e556a7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe9b8fec50 con 0x55fe9b89c8b0 2023-11-16T16:40:04.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.298+0000 7f5e556a7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe9b8fedc0 con 0x55fe9b897b60 2023-11-16T16:40:04.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.298+0000 7f5e556a7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe9b8fef80 con 0x55fe9b898470 2023-11-16T16:40:04.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.298+0000 7f5e556a7b80 10 client.0 _ll_register_callbacks cb 0x7f5e380091e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:40:04.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.298+0000 7f5e437fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9b89c8b0 0x55fe9b8fc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.298+0000 7f5e437fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9b89c8b0 0x55fe9b8fc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44082/0 (socket says 172.21.15.1:44082) 2023-11-16T16:40:04.302 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.298+0000 7f5e437fe700 1 -- 172.21.15.1:0/3404763100 learned_addr learned my addr 172.21.15.1:0/3404763100 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:04.302 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.298+0000 7f5e42ffd700 1 --2- 172.21.15.1:0/3404763100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe9b897b60 0x55fe9b8fe1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.302 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.298+0000 7f5e427fc700 1 --2- 172.21.15.1:0/3404763100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9b898470 0x55fe9b8fe710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.302 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.298+0000 7f5e437fe700 1 -- 172.21.15.1:0/3404763100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9b898470 msgr2=0x55fe9b8fe710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:04.302 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.298+0000 7f5e437fe700 1 --2- 172.21.15.1:0/3404763100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9b898470 0x55fe9b8fe710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.302 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.298+0000 7f5e437fe700 1 -- 172.21.15.1:0/3404763100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe9b897b60 msgr2=0x55fe9b8fe1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:04.302 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.298+0000 7f5e437fe700 1 --2- 172.21.15.1:0/3404763100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe9b897b60 0x55fe9b8fe1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.302 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.298+0000 7f5e437fe700 1 -- 172.21.15.1:0/3404763100 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e44110230 con 0x55fe9b89c8b0 2023-11-16T16:40:04.302 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.298+0000 7f5e427fc700 1 --2- 172.21.15.1:0/3404763100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9b898470 0x55fe9b8fe710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:40:04.302 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.298+0000 7f5e42ffd700 1 --2- 172.21.15.1:0/3404763100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe9b897b60 0x55fe9b8fe1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:40:04.302 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.299+0000 7f5e437fe700 1 --2- 172.21.15.1:0/3404763100 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9b89c8b0 0x55fe9b8fc810 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c00b700 tx=0x7f5e3c0060f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2843067defe4c6e server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:04.302 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.299+0000 7f5e26ff5700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:40:04.303 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.299+0000 7f5e26ff5700 1 -- 172.21.15.1:0/3404763100 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c00e020 con 0x55fe9b89c8b0 2023-11-16T16:40:04.303 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.299+0000 7f5e26ff5700 1 -- 172.21.15.1:0/3404763100 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f5e3c003d10 con 0x55fe9b89c8b0 2023-11-16T16:40:04.303 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.299+0000 7f5e556a7b80 1 -- 172.21.15.1:0/3404763100 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5e3c007d00 con 0x55fe9b89c8b0 2023-11-16T16:40:04.303 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.299+0000 7f5e26ff5700 1 -- 172.21.15.1:0/3404763100 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e3c01bdd0 con 0x55fe9b89c8b0 2023-11-16T16:40:04.303 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.299+0000 7f5e556a7b80 10 client.15669 fetch_fsmap learned FSMap version 23 2023-11-16T16:40:04.303 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.299+0000 7f5e556a7b80 1 -- 172.21.15.1:0/3404763100 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55fe9b5ffc60 con 0x55fe9b89c8b0 2023-11-16T16:40:04.304 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.299+0000 7f5e26ff5700 1 -- 172.21.15.1:0/3404763100 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e3c007d00 con 0x55fe9b89c8b0 2023-11-16T16:40:04.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.300+0000 7f5e26ff5700 1 -- 172.21.15.1:0/3404763100 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5e3c00b3a0 con 0x55fe9b89c8b0 2023-11-16T16:40:04.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.300+0000 7f5e556a7b80 10 client.15669 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:40:04.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.300+0000 7f5e556a7b80 10 client.15669 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-11-16T16:40:04.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.300+0000 7f5e556a7b80 20 client.15669 populate_metadata read hostname 'smithi001' 2023-11-16T16:40:04.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.300+0000 7f5e556a7b80 1 --2- 172.21.15.1:0/3404763100 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55fe9b91efe0 0x55fe9b9213f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.300+0000 7f5e556a7b80 4 client.15669 mds_command: new command op to 24482 tid=0[{"prefix": "damage ls"}] 2023-11-16T16:40:04.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.300+0000 7f5e556a7b80 1 -- 172.21.15.1:0/3404763100 --> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fe9b7b7660 con 0x55fe9b91efe0 2023-11-16T16:40:04.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.300+0000 7f5e42ffd700 1 --2- 172.21.15.1:0/3404763100 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55fe9b91efe0 0x55fe9b9213f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.301+0000 7f5e26ff5700 10 client.15669 ms_handle_connect on v2:172.21.15.26:6836/1453504863 2023-11-16T16:40:04.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.301+0000 7f5e42ffd700 1 --2- 172.21.15.1:0/3404763100 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55fe9b91efe0 0x55fe9b9213f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b5e5b2329e9821de server_cookie=ea371545e3d2ba9e in_seq=0 out_seq=0 2023-11-16T16:40:04.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.301+0000 7f5e26ff5700 1 -- 172.21.15.1:0/3404763100 <== mds.2 v2:172.21.15.26:6836/1453504863 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fe9b7b7660 con 0x55fe9b91efe0 2023-11-16T16:40:04.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.301+0000 7f5e26ff5700 10 client.15669 handle_command_reply: tid=0 2023-11-16T16:40:04.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.301+0000 7f5e556a7b80 1 client.15669 shutdown 2023-11-16T16:40:04.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.301+0000 7f5e556a7b80 1 -- 172.21.15.1:0/3404763100 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9b89c8b0 msgr2=0x55fe9b8fc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:04.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.301+0000 7f5e556a7b80 1 --2- 172.21.15.1:0/3404763100 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9b89c8b0 0x55fe9b8fc810 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c00b700 tx=0x7f5e3c0060f0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.301+0000 7f5e556a7b80 1 -- 172.21.15.1:0/3404763100 shutdown_connections 2023-11-16T16:40:04.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.301+0000 7f5e556a7b80 1 --2- 172.21.15.1:0/3404763100 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55fe9b91efe0 0x55fe9b9213f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.301+0000 7f5e556a7b80 1 --2- 172.21.15.1:0/3404763100 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe9b89c8b0 0x55fe9b8fc810 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.301+0000 7f5e556a7b80 1 --2- 172.21.15.1:0/3404763100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe9b897b60 0x55fe9b8fe1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.301+0000 7f5e556a7b80 1 --2- 172.21.15.1:0/3404763100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fe9b898470 0x55fe9b8fe710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.301+0000 7f5e556a7b80 1 -- 172.21.15.1:0/3404763100 >> 172.21.15.1:0/3404763100 conn(0x55fe9b88d360 msgr2=0x55fe9b8a3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:04.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.301+0000 7f5e26ff5700 0 client.15669 ms_handle_reset on v2:172.21.15.26:6836/1453504863 2023-11-16T16:40:04.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.301+0000 7f5e556a7b80 1 -- 172.21.15.1:0/3404763100 shutdown_connections 2023-11-16T16:40:04.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.301+0000 7f5e556a7b80 1 -- 172.21.15.1:0/3404763100 wait complete. 2023-11-16T16:40:04.307 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.301+0000 7f5e556a7b80 20 client.15669 trim_cache size 0 max 16384 2023-11-16T16:40:04.307 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2023-11-16T16:40:04.307 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.302+0000 7f5e4951b700 1 -- 172.21.15.1:0/1741704049 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5e2c0a13e0 msgr2=0x7f5e2c0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:04.308 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.302+0000 7f5e4951b700 1 --2- 172.21.15.1:0/1741704049 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5e2c0a13e0 0x7f5e2c0a3890 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5e34009d00 tx=0x7f5e34009260 comp rx=0 tx=0).stop 2023-11-16T16:40:04.308 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.302+0000 7f5e4951b700 1 -- 172.21.15.1:0/1741704049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e441002a0 msgr2=0x7f5e4410ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:04.308 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.302+0000 7f5e4951b700 1 --2- 172.21.15.1:0/1741704049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e441002a0 0x7f5e4410ccb0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5e3800e9c0 tx=0x7f5e3800ecd0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.308 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.302+0000 7f5e4951b700 1 -- 172.21.15.1:0/1741704049 shutdown_connections 2023-11-16T16:40:04.308 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.302+0000 7f5e4951b700 1 --2- 172.21.15.1:0/1741704049 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5e2c0a13e0 0x7f5e2c0a3890 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.308 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.302+0000 7f5e4951b700 1 --2- 172.21.15.1:0/1741704049 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e44104140 0x7f5e441a45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.308 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.302+0000 7f5e4951b700 1 --2- 172.21.15.1:0/1741704049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e440ff8f0 0x7f5e4410c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.308 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.302+0000 7f5e4951b700 1 --2- 172.21.15.1:0/1741704049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e441002a0 0x7f5e4410ccb0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.308 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.302+0000 7f5e4951b700 1 -- 172.21.15.1:0/1741704049 >> 172.21.15.1:0/1741704049 conn(0x7f5e44075880 msgr2=0x7f5e440fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:04.308 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.302+0000 7f5e4951b700 1 -- 172.21.15.1:0/1741704049 shutdown_connections 2023-11-16T16:40:04.308 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.302+0000 7f5e4951b700 1 -- 172.21.15.1:0/1741704049 wait complete. 2023-11-16T16:40:04.318 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:04 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1783190761' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:40:04.319 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-11-16T16:40:04.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:04 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1783190761' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:40:04.517 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.516+0000 7f80d1771700 1 -- 172.21.15.1:0/2561099716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f80cc1010d0 msgr2=0x7f80cc109830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:04.517 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.516+0000 7f80d1771700 1 --2- 172.21.15.1:0/2561099716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f80cc1010d0 0x7f80cc109830 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f80c00099c0 tx=0x7f80c0009cd0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.517 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.516+0000 7f80d1771700 1 -- 172.21.15.1:0/2561099716 shutdown_connections 2023-11-16T16:40:04.517 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.516+0000 7f80d1771700 1 --2- 172.21.15.1:0/2561099716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80cc109d70 0x7f80cc10c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.517 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.516+0000 7f80d1771700 1 --2- 172.21.15.1:0/2561099716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80cc1007c0 0x7f80cc100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.517 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.516+0000 7f80d1771700 1 --2- 172.21.15.1:0/2561099716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f80cc1010d0 0x7f80cc109830 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.518 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.516+0000 7f80d1771700 1 -- 172.21.15.1:0/2561099716 >> 172.21.15.1:0/2561099716 conn(0x7f80cc074c60 msgr2=0x7f80cc075060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:04.518 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.516+0000 7f80d1771700 1 -- 172.21.15.1:0/2561099716 shutdown_connections 2023-11-16T16:40:04.518 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.516+0000 7f80d1771700 1 -- 172.21.15.1:0/2561099716 wait complete. 2023-11-16T16:40:04.518 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.517+0000 7f80d1771700 1 Processor -- start 2023-11-16T16:40:04.518 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.517+0000 7f80d1771700 1 -- start start 2023-11-16T16:40:04.519 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.517+0000 7f80d1771700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80cc1007c0 0x7f80cc194340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.521 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.517+0000 7f80d1771700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f80cc1010d0 0x7f80cc194880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.521 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.517+0000 7f80d1771700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80cc109d70 0x7f80cc198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.521 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.518+0000 7f80d1771700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc199320 con 0x7f80cc1010d0 2023-11-16T16:40:04.522 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.518+0000 7f80d1771700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc199490 con 0x7f80cc109d70 2023-11-16T16:40:04.522 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.518+0000 7f80d1771700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc199650 con 0x7f80cc1007c0 2023-11-16T16:40:04.522 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.518+0000 7f80caffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80cc1007c0 0x7f80cc194340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.522 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.518+0000 7f80caffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80cc1007c0 0x7f80cc194340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:40774/0 (socket says 172.21.15.1:40774) 2023-11-16T16:40:04.522 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.518+0000 7f80caffd700 1 -- 172.21.15.1:0/942536176 learned_addr learned my addr 172.21.15.1:0/942536176 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:04.522 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.518+0000 7f80ca7fc700 1 --2- 172.21.15.1:0/942536176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f80cc1010d0 0x7f80cc194880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.522 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.518+0000 7f80cb7fe700 1 --2- 172.21.15.1:0/942536176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80cc109d70 0x7f80cc198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.522 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.518+0000 7f80caffd700 1 -- 172.21.15.1:0/942536176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80cc109d70 msgr2=0x7f80cc198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:04.522 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.518+0000 7f80caffd700 1 --2- 172.21.15.1:0/942536176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80cc109d70 0x7f80cc198ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.522 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.518+0000 7f80caffd700 1 -- 172.21.15.1:0/942536176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f80cc1010d0 msgr2=0x7f80cc194880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:04.522 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.518+0000 7f80caffd700 1 --2- 172.21.15.1:0/942536176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f80cc1010d0 0x7f80cc194880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.523 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.518+0000 7f80caffd700 1 -- 172.21.15.1:0/942536176 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c00095f0 con 0x7f80cc1007c0 2023-11-16T16:40:04.523 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.518+0000 7f80caffd700 1 --2- 172.21.15.1:0/942536176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80cc1007c0 0x7f80cc194340 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f80bc00d950 tx=0x7f80bc00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:04.523 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.519+0000 7f80b3fff700 1 -- 172.21.15.1:0/942536176 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc00f890 con 0x7f80cc1007c0 2023-11-16T16:40:04.523 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.519+0000 7f80b3fff700 1 -- 172.21.15.1:0/942536176 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80bc010460 con 0x7f80cc1007c0 2023-11-16T16:40:04.523 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.519+0000 7f80d1771700 1 -- 172.21.15.1:0/942536176 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80cc10e380 con 0x7f80cc1007c0 2023-11-16T16:40:04.523 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.519+0000 7f80b3fff700 1 -- 172.21.15.1:0/942536176 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc00e610 con 0x7f80cc1007c0 2023-11-16T16:40:04.523 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.519+0000 7f80d1771700 1 -- 172.21.15.1:0/942536176 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80cc10e1f0 con 0x7f80cc1007c0 2023-11-16T16:40:04.524 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.522+0000 7f80b3fff700 1 -- 172.21.15.1:0/942536176 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 24) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f80bc0108d0 con 0x7f80cc1007c0 2023-11-16T16:40:04.524 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.523+0000 7f80b3fff700 1 --2- 172.21.15.1:0/942536176 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f80b40c5600 0x7f80b40c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.524 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.523+0000 7f80ca7fc700 1 --2- 172.21.15.1:0/942536176 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f80b40c5600 0x7f80b40c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.524 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.523+0000 7f80b3fff700 1 -- 172.21.15.1:0/942536176 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f80bc0ccda0 con 0x7f80cc1007c0 2023-11-16T16:40:04.524 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.523+0000 7f80ca7fc700 1 --2- 172.21.15.1:0/942536176 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f80b40c5600 0x7f80b40c7ab0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f80c0000c00 tx=0x7f80c0003680 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:04.525 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.524+0000 7f80dd8fdb80 1 Processor -- start 2023-11-16T16:40:04.525 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.524+0000 7f80dd8fdb80 1 -- start start 2023-11-16T16:40:04.526 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.524+0000 7f80dd8fdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d26ed2eb60 0x55d26ed2ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.526 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.524+0000 7f80dd8fdb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d26ed2f470 0x55d26ed33370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.526 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.524+0000 7f80dd8fdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d26ed338b0 0x55d26ed33d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.526 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.524+0000 7f80dd8fdb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26ec34060 con 0x55d26ed2f470 2023-11-16T16:40:04.527 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.524+0000 7f80dd8fdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26ec868a0 con 0x55d26ed338b0 2023-11-16T16:40:04.527 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.524+0000 7f80dd8fdb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26ec7fd30 con 0x55d26ed2eb60 2023-11-16T16:40:04.527 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.524+0000 7f80ca7fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d26ed2f470 0x55d26ed33370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.527 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.524+0000 7f80ca7fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d26ed2f470 0x55d26ed33370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44102/0 (socket says 172.21.15.1:44102) 2023-11-16T16:40:04.527 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.524+0000 7f80ca7fc700 1 -- 172.21.15.1:0/4242941758 learned_addr learned my addr 172.21.15.1:0/4242941758 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:04.527 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.524+0000 7f80caffd700 1 --2- 172.21.15.1:0/4242941758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d26ed2eb60 0x55d26ed2ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.527 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.525+0000 7f80ca7fc700 1 -- 172.21.15.1:0/4242941758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d26ed2eb60 msgr2=0x55d26ed2ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:04.527 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.525+0000 7f80ca7fc700 1 --2- 172.21.15.1:0/4242941758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d26ed2eb60 0x55d26ed2ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.527 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.525+0000 7f80ca7fc700 1 -- 172.21.15.1:0/4242941758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d26ed338b0 msgr2=0x55d26ed33d00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-16T16:40:04.527 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.525+0000 7f80ca7fc700 1 --2- 172.21.15.1:0/4242941758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d26ed338b0 0x55d26ed33d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.527 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.525+0000 7f80ca7fc700 1 -- 172.21.15.1:0/4242941758 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d26eb01630 con 0x55d26ed2f470 2023-11-16T16:40:04.527 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.525+0000 7f80ca7fc700 1 --2- 172.21.15.1:0/4242941758 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d26ed2f470 0x55d26ed33370 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f80c00095c0 tx=0x7f80c001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=874533a403702fb4 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:04.528 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.525+0000 7f80b17fa700 1 -- 172.21.15.1:0/4242941758 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c002b070 con 0x55d26ed2f470 2023-11-16T16:40:04.528 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.525+0000 7f80b17fa700 1 -- 172.21.15.1:0/4242941758 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80c001e040 con 0x55d26ed2f470 2023-11-16T16:40:04.528 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.525+0000 7f80b17fa700 1 -- 172.21.15.1:0/4242941758 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c0004070 con 0x55d26ed2f470 2023-11-16T16:40:04.528 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.526+0000 7f80dd8fdb80 1 -- 172.21.15.1:0/4242941758 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d26ed2f470 msgr2=0x55d26ed33370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:04.528 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.526+0000 7f80dd8fdb80 1 --2- 172.21.15.1:0/4242941758 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d26ed2f470 0x55d26ed33370 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f80c00095c0 tx=0x7f80c001f000 comp rx=0 tx=0).stop 2023-11-16T16:40:04.531 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.530+0000 7f80dd8fdb80 1 -- 172.21.15.1:0/4242941758 shutdown_connections 2023-11-16T16:40:04.531 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.530+0000 7f80dd8fdb80 1 --2- 172.21.15.1:0/4242941758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d26ed338b0 0x55d26ed33d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.531 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.530+0000 7f80dd8fdb80 1 --2- 172.21.15.1:0/4242941758 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d26ed2eb60 0x55d26ed2ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.531 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.530+0000 7f80dd8fdb80 1 --2- 172.21.15.1:0/4242941758 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d26ed2f470 0x55d26ed33370 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.531 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.530+0000 7f80dd8fdb80 1 -- 172.21.15.1:0/4242941758 >> 172.21.15.1:0/4242941758 conn(0x55d26ed24360 msgr2=0x55d26ed29b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:04.531 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.530+0000 7f80dd8fdb80 1 -- 172.21.15.1:0/4242941758 shutdown_connections 2023-11-16T16:40:04.531 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.530+0000 7f80dd8fdb80 1 -- 172.21.15.1:0/4242941758 wait complete. 2023-11-16T16:40:04.531 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.530+0000 7f80dd8fdb80 1 -- start start 2023-11-16T16:40:04.531 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.530+0000 7f80dd8fdb80 1 Processor -- start 2023-11-16T16:40:04.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.531+0000 7f80dd8fdb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d26ed2eb60 0x55d26ed38320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.531+0000 7f80dd8fdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d26ed2f470 0x55d26ed38860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.531+0000 7f80dd8fdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d26ed338b0 0x55d26ed38da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.531+0000 7f80dd8fdb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26ec218e0 con 0x55d26ed2eb60 2023-11-16T16:40:04.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.531+0000 7f80dd8fdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26ec21580 con 0x55d26ed338b0 2023-11-16T16:40:04.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.531+0000 7f80dd8fdb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26eb03920 con 0x55d26ed2f470 2023-11-16T16:40:04.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.531+0000 7f80dd8fdb80 10 client.0 _ll_register_callbacks cb 0x55d26ec3f780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:40:04.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.531+0000 7f80ca7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d26ed2f470 0x55d26ed38860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.531+0000 7f80cb7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d26ed338b0 0x55d26ed38da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.531+0000 7f80cb7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d26ed338b0 0x55d26ed38da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.1:38294/0 (socket says 172.21.15.1:38294) 2023-11-16T16:40:04.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.531+0000 7f80ca7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d26ed2f470 0x55d26ed38860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:40792/0 (socket says 172.21.15.1:40792) 2023-11-16T16:40:04.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.531+0000 7f80cb7fe700 1 -- 172.21.15.1:0/2682572452 learned_addr learned my addr 172.21.15.1:0/2682572452 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:04.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.531+0000 7f80cb7fe700 1 -- 172.21.15.1:0/2682572452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d26ed2f470 msgr2=0x55d26ed38860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:04.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.531+0000 7f80cb7fe700 1 --2- 172.21.15.1:0/2682572452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d26ed2f470 0x55d26ed38860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.531+0000 7f80cb7fe700 1 -- 172.21.15.1:0/2682572452 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d26ed2eb60 msgr2=0x55d26ed38320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:04.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.531+0000 7f80cb7fe700 1 --2- 172.21.15.1:0/2682572452 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d26ed2eb60 0x55d26ed38320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.531+0000 7f80cb7fe700 1 -- 172.21.15.1:0/2682572452 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80cc10caa0 con 0x55d26ed338b0 2023-11-16T16:40:04.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.532+0000 7f80ca7fc700 1 --2- 172.21.15.1:0/2682572452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d26ed2f470 0x55d26ed38860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:40:04.536 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.533+0000 7f80cb7fe700 1 --2- 172.21.15.1:0/2682572452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d26ed338b0 0x55d26ed38da0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f80c4009f50 tx=0x7f80c4004320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee4264c6d8c6bd0e server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:04.536 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.533+0000 7f80aeff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-11-16T16:40:04.536 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.533+0000 7f80aeff5700 1 -- 172.21.15.1:0/2682572452 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c4019070 con 0x55d26ed338b0 2023-11-16T16:40:04.536 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.533+0000 7f80aeff5700 1 -- 172.21.15.1:0/2682572452 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f80c4004bb0 con 0x55d26ed338b0 2023-11-16T16:40:04.536 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.533+0000 7f80dd8fdb80 1 -- 172.21.15.1:0/2682572452 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d26ed21b20 con 0x55d26ed338b0 2023-11-16T16:40:04.536 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.533+0000 7f80aeff5700 1 -- 172.21.15.1:0/2682572452 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80c4004d70 con 0x55d26ed338b0 2023-11-16T16:40:04.536 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.533+0000 7f80dd8fdb80 10 client.25280 fetch_fsmap learned FSMap version 23 2023-11-16T16:40:04.536 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.533+0000 7f80dd8fdb80 1 -- 172.21.15.1:0/2682572452 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d26ec46190 con 0x55d26ed338b0 2023-11-16T16:40:04.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.534+0000 7f80aeff5700 1 -- 172.21.15.1:0/2682572452 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d26ed21b20 con 0x55d26ed338b0 2023-11-16T16:40:04.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.534+0000 7f80aeff5700 1 -- 172.21.15.1:0/2682572452 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f80c401c020 con 0x55d26ed338b0 2023-11-16T16:40:04.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.534+0000 7f80dd8fdb80 10 client.25280 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:40:04.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.534+0000 7f80dd8fdb80 10 client.25280 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-11-16T16:40:04.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.534+0000 7f80dd8fdb80 20 client.25280 populate_metadata read hostname 'smithi001' 2023-11-16T16:40:04.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.534+0000 7f80dd8fdb80 1 --2- 172.21.15.1:0/2682572452 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55d26ed4ce00 0x55d26ed4f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.534+0000 7f80dd8fdb80 4 client.25280 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:40:04.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.534+0000 7f80dd8fdb80 1 -- 172.21.15.1:0/2682572452 --> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d26ec420a0 con 0x55d26ed4ce00 2023-11-16T16:40:04.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.534+0000 7f80caffd700 1 --2- 172.21.15.1:0/2682572452 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55d26ed4ce00 0x55d26ed4f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.535+0000 7f80aeff5700 10 client.25280 ms_handle_connect on v2:172.21.15.53:6838/1044753024 2023-11-16T16:40:04.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.535+0000 7f80caffd700 1 --2- 172.21.15.1:0/2682572452 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55d26ed4ce00 0x55d26ed4f210 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b3c451e66cab65b7 server_cookie=642ca255cbfa5c36 in_seq=0 out_seq=0 2023-11-16T16:40:04.538 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.536+0000 7f80aeff5700 1 -- 172.21.15.1:0/2682572452 <== mds.1 v2:172.21.15.53:6838/1044753024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d26ec420a0 con 0x55d26ed4ce00 2023-11-16T16:40:04.539 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.536+0000 7f80aeff5700 10 client.25280 handle_command_reply: tid=0 2023-11-16T16:40:04.539 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.536+0000 7f80dd8fdb80 1 client.25280 shutdown 2023-11-16T16:40:04.539 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.536+0000 7f80dd8fdb80 1 -- 172.21.15.1:0/2682572452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d26ed338b0 msgr2=0x55d26ed38da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:04.539 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.536+0000 7f80dd8fdb80 1 --2- 172.21.15.1:0/2682572452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d26ed338b0 0x55d26ed38da0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f80c4009f50 tx=0x7f80c4004320 comp rx=0 tx=0).stop 2023-11-16T16:40:04.539 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.536+0000 7f80dd8fdb80 1 -- 172.21.15.1:0/2682572452 shutdown_connections 2023-11-16T16:40:04.539 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.536+0000 7f80dd8fdb80 1 --2- 172.21.15.1:0/2682572452 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55d26ed4ce00 0x55d26ed4f210 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.539 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.536+0000 7f80dd8fdb80 1 --2- 172.21.15.1:0/2682572452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d26ed338b0 0x55d26ed38da0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.539 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.536+0000 7f80dd8fdb80 1 --2- 172.21.15.1:0/2682572452 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d26ed2eb60 0x55d26ed38320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.539 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.536+0000 7f80dd8fdb80 1 --2- 172.21.15.1:0/2682572452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d26ed2f470 0x55d26ed38860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.539 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.536+0000 7f80dd8fdb80 1 -- 172.21.15.1:0/2682572452 >> 172.21.15.1:0/2682572452 conn(0x55d26ed24360 msgr2=0x55d26ed28de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:04.539 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.536+0000 7f80aeff5700 0 client.25280 ms_handle_reset on v2:172.21.15.53:6838/1044753024 2023-11-16T16:40:04.539 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.536+0000 7f80dd8fdb80 1 -- 172.21.15.1:0/2682572452 shutdown_connections 2023-11-16T16:40:04.540 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.536+0000 7f80dd8fdb80 1 -- 172.21.15.1:0/2682572452 wait complete. 2023-11-16T16:40:04.540 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.536+0000 7f80dd8fdb80 20 client.25280 trim_cache size 0 max 16384 2023-11-16T16:40:04.563 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.562+0000 7f80dd8fdb80 1 Processor -- start 2023-11-16T16:40:04.563 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.563+0000 7f80dd8fdb80 1 -- start start 2023-11-16T16:40:04.564 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.563+0000 7f80dd8fdb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d26ed2eb60 0x55d26ed43d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.564 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.563+0000 7f80dd8fdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d26ed2f470 0x55d26ed4ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.565 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.563+0000 7f80dd8fdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d26ed338b0 0x55d26ed31970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.565 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.563+0000 7f80dd8fdb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26ed2f970 con 0x55d26ed2eb60 2023-11-16T16:40:04.565 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.563+0000 7f80dd8fdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26ed36880 con 0x55d26ed2f470 2023-11-16T16:40:04.565 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.563+0000 7f80dd8fdb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26ed32440 con 0x55d26ed338b0 2023-11-16T16:40:04.565 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.563+0000 7f80caffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d26ed2eb60 0x55d26ed43d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.565 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.563+0000 7f80caffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d26ed2eb60 0x55d26ed43d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44114/0 (socket says 172.21.15.1:44114) 2023-11-16T16:40:04.565 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.563+0000 7f80caffd700 1 -- 172.21.15.1:0/1322298147 learned_addr learned my addr 172.21.15.1:0/1322298147 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:04.565 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.563+0000 7f80caffd700 1 -- 172.21.15.1:0/1322298147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d26ed338b0 msgr2=0x55d26ed31970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-16T16:40:04.565 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.563+0000 7f80caffd700 1 --2- 172.21.15.1:0/1322298147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d26ed338b0 0x55d26ed31970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.565 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.563+0000 7f80caffd700 1 -- 172.21.15.1:0/1322298147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d26ed2f470 msgr2=0x55d26ed4ee10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-16T16:40:04.565 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.563+0000 7f80caffd700 1 --2- 172.21.15.1:0/1322298147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d26ed2f470 0x55d26ed4ee10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.563+0000 7f80caffd700 1 -- 172.21.15.1:0/1322298147 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d26ec45ee0 con 0x55d26ed2eb60 2023-11-16T16:40:04.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.564+0000 7f80caffd700 1 --2- 172.21.15.1:0/1322298147 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d26ed2eb60 0x55d26ed43d80 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f80bc096130 tx=0x7f80bc005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87152b40e142d83f server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:04.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.564+0000 7f80b1ffb700 1 -- 172.21.15.1:0/1322298147 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc0a2100 con 0x55d26ed2eb60 2023-11-16T16:40:04.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.564+0000 7f80b1ffb700 1 -- 172.21.15.1:0/1322298147 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80bc097fd0 con 0x55d26ed2eb60 2023-11-16T16:40:04.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.564+0000 7f80b1ffb700 1 -- 172.21.15.1:0/1322298147 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc099380 con 0x55d26ed2eb60 2023-11-16T16:40:04.567 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.565+0000 7f80dd8fdb80 1 -- 172.21.15.1:0/1322298147 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d26ed2eb60 msgr2=0x55d26ed43d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:04.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.565+0000 7f80dd8fdb80 1 --2- 172.21.15.1:0/1322298147 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d26ed2eb60 0x55d26ed43d80 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f80bc096130 tx=0x7f80bc005910 comp rx=0 tx=0).stop 2023-11-16T16:40:04.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.565+0000 7f80dd8fdb80 1 -- 172.21.15.1:0/1322298147 shutdown_connections 2023-11-16T16:40:04.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.565+0000 7f80dd8fdb80 1 --2- 172.21.15.1:0/1322298147 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d26ed338b0 0x55d26ed31970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.565+0000 7f80dd8fdb80 1 --2- 172.21.15.1:0/1322298147 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d26ed2eb60 0x55d26ed43d80 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.565+0000 7f80dd8fdb80 1 --2- 172.21.15.1:0/1322298147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d26ed2f470 0x55d26ed4ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.565+0000 7f80dd8fdb80 1 -- 172.21.15.1:0/1322298147 >> 172.21.15.1:0/1322298147 conn(0x55d26ed24360 msgr2=0x55d26ed49060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:04.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.566+0000 7f80dd8fdb80 1 -- 172.21.15.1:0/1322298147 shutdown_connections 2023-11-16T16:40:04.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.566+0000 7f80dd8fdb80 1 -- 172.21.15.1:0/1322298147 wait complete. 2023-11-16T16:40:04.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.566+0000 7f80dd8fdb80 1 -- start start 2023-11-16T16:40:04.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.566+0000 7f80dd8fdb80 1 Processor -- start 2023-11-16T16:40:04.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.566+0000 7f80dd8fdb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d26ed2eb60 0x55d26ed43f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.566+0000 7f80dd8fdb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d26ed338b0 0x55d26ed35560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.566+0000 7f80dd8fdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d26ed35aa0 0x55d26ed35f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.566+0000 7f80dd8fdb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26ed9fd00 con 0x55d26ed338b0 2023-11-16T16:40:04.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.566+0000 7f80dd8fdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26ed9fe70 con 0x55d26ed35aa0 2023-11-16T16:40:04.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.566+0000 7f80dd8fdb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26eda0030 con 0x55d26ed2eb60 2023-11-16T16:40:04.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.567+0000 7f80dd8fdb80 10 client.0 _ll_register_callbacks cb 0x7f80c4003f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:40:04.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.567+0000 7f80caffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d26ed2eb60 0x55d26ed43f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.567+0000 7f80caffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d26ed2eb60 0x55d26ed43f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:40798/0 (socket says 172.21.15.1:40798) 2023-11-16T16:40:04.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.567+0000 7f80caffd700 1 -- 172.21.15.1:0/3915626981 learned_addr learned my addr 172.21.15.1:0/3915626981 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:04.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.567+0000 7f80cb7fe700 1 --2- 172.21.15.1:0/3915626981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d26ed35aa0 0x55d26ed35f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.567+0000 7f80cb7fe700 1 -- 172.21.15.1:0/3915626981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d26ed2eb60 msgr2=0x55d26ed43f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:04.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.567+0000 7f80cb7fe700 1 --2- 172.21.15.1:0/3915626981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d26ed2eb60 0x55d26ed43f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.567+0000 7f80cb7fe700 1 -- 172.21.15.1:0/3915626981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d26ed338b0 msgr2=0x55d26ed35560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:04.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.567+0000 7f80cb7fe700 1 --2- 172.21.15.1:0/3915626981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d26ed338b0 0x55d26ed35560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.567+0000 7f80cb7fe700 1 -- 172.21.15.1:0/3915626981 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80cc10e1f0 con 0x55d26ed35aa0 2023-11-16T16:40:04.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.567+0000 7f80caffd700 1 --2- 172.21.15.1:0/3915626981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d26ed2eb60 0x55d26ed43f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:40:04.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.567+0000 7f80cb7fe700 1 --2- 172.21.15.1:0/3915626981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d26ed35aa0 0x55d26ed35f10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f80c4009c70 tx=0x7f80c4005b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cab2d2861982cdc server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:04.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.567+0000 7f80aeff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-11-16T16:40:04.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.568+0000 7f80aeff5700 1 -- 172.21.15.1:0/3915626981 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c4019070 con 0x55d26ed35aa0 2023-11-16T16:40:04.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.568+0000 7f80aeff5700 1 -- 172.21.15.1:0/3915626981 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f80c4009560 con 0x55d26ed35aa0 2023-11-16T16:40:04.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.568+0000 7f80dd8fdb80 1 -- 172.21.15.1:0/3915626981 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f80c001e040 con 0x55d26ed35aa0 2023-11-16T16:40:04.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.568+0000 7f80aeff5700 1 -- 172.21.15.1:0/3915626981 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80c4015690 con 0x55d26ed35aa0 2023-11-16T16:40:04.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.568+0000 7f80dd8fdb80 10 client.25283 fetch_fsmap learned FSMap version 23 2023-11-16T16:40:04.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.568+0000 7f80dd8fdb80 1 -- 172.21.15.1:0/3915626981 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d26ea96c60 con 0x55d26ed35aa0 2023-11-16T16:40:04.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.568+0000 7f80aeff5700 1 -- 172.21.15.1:0/3915626981 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80c001e040 con 0x55d26ed35aa0 2023-11-16T16:40:04.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.568+0000 7f80aeff5700 1 -- 172.21.15.1:0/3915626981 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f80c401c050 con 0x55d26ed35aa0 2023-11-16T16:40:04.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.568+0000 7f80dd8fdb80 10 client.25283 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:40:04.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.568+0000 7f80dd8fdb80 10 client.25283 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-11-16T16:40:04.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.568+0000 7f80dd8fdb80 20 client.25283 populate_metadata read hostname 'smithi001' 2023-11-16T16:40:04.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.568+0000 7f80dd8fdb80 1 --2- 172.21.15.1:0/3915626981 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55d26eda16f0 0x55d26eda3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.568+0000 7f80dd8fdb80 4 client.25283 mds_command: new command op to 24493 tid=0[{"prefix": "damage ls"}] 2023-11-16T16:40:04.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.568+0000 7f80dd8fdb80 1 -- 172.21.15.1:0/3915626981 --> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d26ec4e660 con 0x55d26eda16f0 2023-11-16T16:40:04.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.569+0000 7f80caffd700 1 --2- 172.21.15.1:0/3915626981 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55d26eda16f0 0x55d26eda3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.569+0000 7f80aeff5700 10 client.25283 ms_handle_connect on v2:172.21.15.53:6838/1044753024 2023-11-16T16:40:04.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.569+0000 7f80caffd700 1 --2- 172.21.15.1:0/3915626981 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55d26eda16f0 0x55d26eda3b00 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ed0e40edd15c5418 server_cookie=ff81f568dc298393 in_seq=0 out_seq=0 2023-11-16T16:40:04.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.569+0000 7f80aeff5700 1 -- 172.21.15.1:0/3915626981 <== mds.1 v2:172.21.15.53:6838/1044753024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d26ec4e660 con 0x55d26eda16f0 2023-11-16T16:40:04.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.569+0000 7f80aeff5700 10 client.25283 handle_command_reply: tid=0 2023-11-16T16:40:04.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.569+0000 7f80dd8fdb80 1 client.25283 shutdown 2023-11-16T16:40:04.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.570+0000 7f80dd8fdb80 1 -- 172.21.15.1:0/3915626981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d26ed35aa0 msgr2=0x55d26ed35f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:04.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.570+0000 7f80dd8fdb80 1 --2- 172.21.15.1:0/3915626981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d26ed35aa0 0x55d26ed35f10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f80c4009c70 tx=0x7f80c4005b90 comp rx=0 tx=0).stop 2023-11-16T16:40:04.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.570+0000 7f80dd8fdb80 1 -- 172.21.15.1:0/3915626981 shutdown_connections 2023-11-16T16:40:04.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.570+0000 7f80dd8fdb80 1 --2- 172.21.15.1:0/3915626981 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55d26eda16f0 0x55d26eda3b00 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.570+0000 7f80dd8fdb80 1 --2- 172.21.15.1:0/3915626981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d26ed35aa0 0x55d26ed35f10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.570+0000 7f80dd8fdb80 1 --2- 172.21.15.1:0/3915626981 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55d26ed2eb60 0x55d26ed43f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.570+0000 7f80dd8fdb80 1 --2- 172.21.15.1:0/3915626981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d26ed338b0 0x55d26ed35560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.570+0000 7f80dd8fdb80 1 -- 172.21.15.1:0/3915626981 >> 172.21.15.1:0/3915626981 conn(0x55d26ed24360 msgr2=0x55d26ed4ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:04.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.570+0000 7f80aeff5700 0 client.25283 ms_handle_reset on v2:172.21.15.53:6838/1044753024 2023-11-16T16:40:04.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.570+0000 7f80dd8fdb80 1 -- 172.21.15.1:0/3915626981 shutdown_connections 2023-11-16T16:40:04.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.570+0000 7f80dd8fdb80 1 -- 172.21.15.1:0/3915626981 wait complete. 2023-11-16T16:40:04.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.570+0000 7f80dd8fdb80 20 client.25283 trim_cache size 0 max 16384 2023-11-16T16:40:04.576 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2023-11-16T16:40:04.576 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.570+0000 7f80d1771700 1 -- 172.21.15.1:0/942536176 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f80b40c5600 msgr2=0x7f80b40c7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:04.576 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.570+0000 7f80d1771700 1 --2- 172.21.15.1:0/942536176 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f80b40c5600 0x7f80b40c7ab0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f80c0000c00 tx=0x7f80c0003680 comp rx=0 tx=0).stop 2023-11-16T16:40:04.577 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.571+0000 7f80d1771700 1 -- 172.21.15.1:0/942536176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80cc1007c0 msgr2=0x7f80cc194340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:04.577 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.571+0000 7f80d1771700 1 --2- 172.21.15.1:0/942536176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80cc1007c0 0x7f80cc194340 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f80bc00d950 tx=0x7f80bc00dc60 comp rx=0 tx=0).stop 2023-11-16T16:40:04.577 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.571+0000 7f80d1771700 1 -- 172.21.15.1:0/942536176 shutdown_connections 2023-11-16T16:40:04.577 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.571+0000 7f80d1771700 1 --2- 172.21.15.1:0/942536176 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f80b40c5600 0x7f80b40c7ab0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.577 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.571+0000 7f80d1771700 1 --2- 172.21.15.1:0/942536176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80cc109d70 0x7f80cc198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.577 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.571+0000 7f80d1771700 1 --2- 172.21.15.1:0/942536176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f80cc1007c0 0x7f80cc194340 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.577 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.571+0000 7f80d1771700 1 --2- 172.21.15.1:0/942536176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f80cc1010d0 0x7f80cc194880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.577 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.571+0000 7f80d1771700 1 -- 172.21.15.1:0/942536176 >> 172.21.15.1:0/942536176 conn(0x7f80cc074c60 msgr2=0x7f80cc0fc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:04.577 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.571+0000 7f80d1771700 1 -- 172.21.15.1:0/942536176 shutdown_connections 2023-11-16T16:40:04.577 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.571+0000 7f80d1771700 1 -- 172.21.15.1:0/942536176 wait complete. 2023-11-16T16:40:04.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:04 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1783190761' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:40:04.588 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-16T16:40:04.786 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.784+0000 7f0073f8a700 1 -- 172.21.15.1:0/1282607272 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f006c0ffa70 msgr2=0x7f006c0ffee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:04.786 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.784+0000 7f0073f8a700 1 --2- 172.21.15.1:0/1282607272 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f006c0ffa70 0x7f006c0ffee0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f005c009990 tx=0x7f005c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.786 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.785+0000 7f0073f8a700 1 -- 172.21.15.1:0/1282607272 shutdown_connections 2023-11-16T16:40:04.786 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.785+0000 7f0073f8a700 1 --2- 172.21.15.1:0/1282607272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f006c100420 0x7f006c10bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.786 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.785+0000 7f0073f8a700 1 --2- 172.21.15.1:0/1282607272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f006c105250 0x7f006c105620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.786 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.785+0000 7f0073f8a700 1 --2- 172.21.15.1:0/1282607272 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f006c0ffa70 0x7f006c0ffee0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.786 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.785+0000 7f0073f8a700 1 -- 172.21.15.1:0/1282607272 >> 172.21.15.1:0/1282607272 conn(0x7f006c0763a0 msgr2=0x7f006c0767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:04.786 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.785+0000 7f0073f8a700 1 -- 172.21.15.1:0/1282607272 shutdown_connections 2023-11-16T16:40:04.787 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.785+0000 7f0073f8a700 1 -- 172.21.15.1:0/1282607272 wait complete. 2023-11-16T16:40:04.788 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.786+0000 7f0073f8a700 1 Processor -- start 2023-11-16T16:40:04.788 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.786+0000 7f0073f8a700 1 -- start start 2023-11-16T16:40:04.788 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.786+0000 7f0073f8a700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f006c0ffa70 0x7f006c194110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.788 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.786+0000 7f0073f8a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f006c100420 0x7f006c194650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.789 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.786+0000 7f0073f8a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f006c105250 0x7f006c198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.789 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.786+0000 7f0073f8a700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006c1990f0 con 0x7f006c0ffa70 2023-11-16T16:40:04.789 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.786+0000 7f0073f8a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006c199260 con 0x7f006c105250 2023-11-16T16:40:04.789 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.786+0000 7f0073f8a700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006c199420 con 0x7f006c100420 2023-11-16T16:40:04.789 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.786+0000 7f0071d26700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f006c0ffa70 0x7f006c194110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.789 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.786+0000 7f0071d26700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f006c0ffa70 0x7f006c194110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44130/0 (socket says 172.21.15.1:44130) 2023-11-16T16:40:04.789 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.786+0000 7f0071d26700 1 -- 172.21.15.1:0/3597406176 learned_addr learned my addr 172.21.15.1:0/3597406176 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:04.789 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.786+0000 7f0072527700 1 --2- 172.21.15.1:0/3597406176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f006c105250 0x7f006c198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.789 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.786+0000 7f0071525700 1 --2- 172.21.15.1:0/3597406176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f006c100420 0x7f006c194650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.789 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.787+0000 7f0071d26700 1 -- 172.21.15.1:0/3597406176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f006c100420 msgr2=0x7f006c194650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:04.789 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.787+0000 7f0071d26700 1 --2- 172.21.15.1:0/3597406176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f006c100420 0x7f006c194650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.789 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.787+0000 7f0071d26700 1 -- 172.21.15.1:0/3597406176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f006c105250 msgr2=0x7f006c198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:04.790 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.787+0000 7f0071d26700 1 --2- 172.21.15.1:0/3597406176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f006c105250 0x7f006c198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.790 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.787+0000 7f0071d26700 1 -- 172.21.15.1:0/3597406176 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f005c0095f0 con 0x7f006c0ffa70 2023-11-16T16:40:04.790 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.787+0000 7f0072527700 1 --2- 172.21.15.1:0/3597406176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f006c105250 0x7f006c198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:40:04.790 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.787+0000 7f0071d26700 1 --2- 172.21.15.1:0/3597406176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f006c0ffa70 0x7f006c194110 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f006800e9a0 tx=0x7f006800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:04.790 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.787+0000 7f0062ffd700 1 -- 172.21.15.1:0/3597406176 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0068009790 con 0x7f006c0ffa70 2023-11-16T16:40:04.790 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.787+0000 7f0073f8a700 1 -- 172.21.15.1:0/3597406176 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f006c10e170 con 0x7f006c0ffa70 2023-11-16T16:40:04.790 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.787+0000 7f0062ffd700 1 -- 172.21.15.1:0/3597406176 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0068010430 con 0x7f006c0ffa70 2023-11-16T16:40:04.790 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.787+0000 7f0062ffd700 1 -- 172.21.15.1:0/3597406176 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0068018680 con 0x7f006c0ffa70 2023-11-16T16:40:04.790 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.787+0000 7f0073f8a700 1 -- 172.21.15.1:0/3597406176 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f006c10dfe0 con 0x7f006c0ffa70 2023-11-16T16:40:04.791 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.791+0000 7f0062ffd700 1 -- 172.21.15.1:0/3597406176 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 24) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f00680105a0 con 0x7f006c0ffa70 2023-11-16T16:40:04.792 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.791+0000 7f0062ffd700 1 --2- 172.21.15.1:0/3597406176 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f00580a13e0 0x7f00580a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.793 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.792+0000 7f0062ffd700 1 -- 172.21.15.1:0/3597406176 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f0068014070 con 0x7f006c0ffa70 2023-11-16T16:40:04.793 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.792+0000 7f0071525700 1 --2- 172.21.15.1:0/3597406176 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f00580a13e0 0x7f00580a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.793 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.792+0000 7f0080116b80 1 Processor -- start 2023-11-16T16:40:04.794 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.792+0000 7f0080116b80 1 -- start start 2023-11-16T16:40:04.794 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.792+0000 7f0071525700 1 --2- 172.21.15.1:0/3597406176 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f00580a13e0 0x7f00580a3890 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f006c195720 tx=0x7f005c005fb0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:04.794 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.793+0000 7f0080116b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562bf3f0c370 0x562bf3f0c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.794 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.793+0000 7f0080116b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bf3f0cc80 0x562bf3f10b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.794 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.793+0000 7f0080116b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562bf3f110c0 0x562bf3f11510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.794 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.793+0000 7f0080116b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bf3e09060 con 0x562bf3f0cc80 2023-11-16T16:40:04.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.793+0000 7f0080116b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bf3e5b8a0 con 0x562bf3f0c370 2023-11-16T16:40:04.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.793+0000 7f0080116b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bf3e54d30 con 0x562bf3f110c0 2023-11-16T16:40:04.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.793+0000 7f0071525700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bf3f0cc80 0x562bf3f10b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.793+0000 7f0071525700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bf3f0cc80 0x562bf3f10b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44140/0 (socket says 172.21.15.1:44140) 2023-11-16T16:40:04.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.793+0000 7f0071525700 1 -- 172.21.15.1:0/714197177 learned_addr learned my addr 172.21.15.1:0/714197177 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:04.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.793+0000 7f0072527700 1 --2- 172.21.15.1:0/714197177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562bf3f110c0 0x562bf3f11510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.793+0000 7f0071525700 1 -- 172.21.15.1:0/714197177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562bf3f110c0 msgr2=0x562bf3f11510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:04.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.793+0000 7f0071525700 1 --2- 172.21.15.1:0/714197177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562bf3f110c0 0x562bf3f11510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.793+0000 7f0071525700 1 -- 172.21.15.1:0/714197177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562bf3f0c370 msgr2=0x562bf3f0c740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-16T16:40:04.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.793+0000 7f0071525700 1 --2- 172.21.15.1:0/714197177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562bf3f0c370 0x562bf3f0c740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.793+0000 7f0071525700 1 -- 172.21.15.1:0/714197177 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bf3cd6630 con 0x562bf3f0cc80 2023-11-16T16:40:04.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.793+0000 7f0071525700 1 --2- 172.21.15.1:0/714197177 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bf3f0cc80 0x562bf3f10b80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f005c009990 tx=0x7f005c024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58770aadfce181a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:04.798 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.794+0000 7f0057fff700 1 -- 172.21.15.1:0/714197177 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c003ce0 con 0x562bf3f0cc80 2023-11-16T16:40:04.798 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.794+0000 7f0057fff700 1 -- 172.21.15.1:0/714197177 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f005c005590 con 0x562bf3f0cc80 2023-11-16T16:40:04.798 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.794+0000 7f0057fff700 1 -- 172.21.15.1:0/714197177 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c00bbf0 con 0x562bf3f0cc80 2023-11-16T16:40:04.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.795+0000 7f0080116b80 1 -- 172.21.15.1:0/714197177 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bf3f0cc80 msgr2=0x562bf3f10b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:04.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.795+0000 7f0080116b80 1 --2- 172.21.15.1:0/714197177 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bf3f0cc80 0x562bf3f10b80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f005c009990 tx=0x7f005c024000 comp rx=0 tx=0).stop 2023-11-16T16:40:04.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.795+0000 7f0080116b80 1 -- 172.21.15.1:0/714197177 shutdown_connections 2023-11-16T16:40:04.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.795+0000 7f0080116b80 1 --2- 172.21.15.1:0/714197177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562bf3f110c0 0x562bf3f11510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.795+0000 7f0080116b80 1 --2- 172.21.15.1:0/714197177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562bf3f0c370 0x562bf3f0c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.795+0000 7f0080116b80 1 --2- 172.21.15.1:0/714197177 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bf3f0cc80 0x562bf3f10b80 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.795+0000 7f0080116b80 1 -- 172.21.15.1:0/714197177 >> 172.21.15.1:0/714197177 conn(0x562bf3ef9360 msgr2=0x562bf3efeb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:04.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.795+0000 7f0080116b80 1 -- 172.21.15.1:0/714197177 shutdown_connections 2023-11-16T16:40:04.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.795+0000 7f0080116b80 1 -- 172.21.15.1:0/714197177 wait complete. 2023-11-16T16:40:04.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.795+0000 7f0080116b80 1 -- start start 2023-11-16T16:40:04.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.795+0000 7f0080116b80 1 Processor -- start 2023-11-16T16:40:04.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.796+0000 7f0080116b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bf3f0c370 0x562bf3f15be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.796+0000 7f0080116b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562bf3f0cc80 0x562bf3f16120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.796+0000 7f0080116b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562bf3f110c0 0x562bf3f16660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.796+0000 7f0080116b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bf3df68e0 con 0x562bf3f0c370 2023-11-16T16:40:04.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.796+0000 7f0080116b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bf3df6580 con 0x562bf3f110c0 2023-11-16T16:40:04.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.796+0000 7f0080116b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bf3cd8920 con 0x562bf3f0cc80 2023-11-16T16:40:04.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.796+0000 7f0080116b80 10 client.0 _ll_register_callbacks cb 0x562bf3e14780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:40:04.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.796+0000 7f0071d26700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bf3f0c370 0x562bf3f15be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.796+0000 7f0071d26700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bf3f0c370 0x562bf3f15be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44144/0 (socket says 172.21.15.1:44144) 2023-11-16T16:40:04.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.796+0000 7f0071d26700 1 -- 172.21.15.1:0/1034689732 learned_addr learned my addr 172.21.15.1:0/1034689732 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:04.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.796+0000 7f0071d26700 1 -- 172.21.15.1:0/1034689732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562bf3f0cc80 msgr2=0x562bf3f16120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-16T16:40:04.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.796+0000 7f0071d26700 1 --2- 172.21.15.1:0/1034689732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562bf3f0cc80 0x562bf3f16120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.796+0000 7f0071d26700 1 -- 172.21.15.1:0/1034689732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562bf3f110c0 msgr2=0x562bf3f16660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-16T16:40:04.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.796+0000 7f0071d26700 1 --2- 172.21.15.1:0/1034689732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562bf3f110c0 0x562bf3f16660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.796+0000 7f0071d26700 1 -- 172.21.15.1:0/1034689732 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f006c10c8d0 con 0x562bf3f0c370 2023-11-16T16:40:04.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.797+0000 7f0071d26700 1 --2- 172.21.15.1:0/1034689732 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bf3f0c370 0x562bf3f15be0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f00680cae50 tx=0x7f0068099a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f4378cdb05f16d server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:04.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.797+0000 7f00557fa700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:40:04.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.797+0000 7f00557fa700 1 -- 172.21.15.1:0/1034689732 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00680a2140 con 0x562bf3f0c370 2023-11-16T16:40:04.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.797+0000 7f00557fa700 1 -- 172.21.15.1:0/1034689732 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f00680a23d0 con 0x562bf3f0c370 2023-11-16T16:40:04.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.797+0000 7f0080116b80 1 -- 172.21.15.1:0/1034689732 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562bf3ef6b20 con 0x562bf3f0c370 2023-11-16T16:40:04.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.797+0000 7f00557fa700 1 -- 172.21.15.1:0/1034689732 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00680ca9b0 con 0x562bf3f0c370 2023-11-16T16:40:04.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.797+0000 7f0080116b80 10 client.15693 fetch_fsmap learned FSMap version 23 2023-11-16T16:40:04.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.797+0000 7f0080116b80 1 -- 172.21.15.1:0/1034689732 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x562bf3e1b190 con 0x562bf3f0c370 2023-11-16T16:40:04.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.797+0000 7f00557fa700 1 -- 172.21.15.1:0/1034689732 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00680b8030 con 0x562bf3f0c370 2023-11-16T16:40:04.802 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.797+0000 7f00557fa700 1 -- 172.21.15.1:0/1034689732 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0068098f70 con 0x562bf3f0c370 2023-11-16T16:40:04.802 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.797+0000 7f0080116b80 10 client.15693 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:40:04.802 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.797+0000 7f0080116b80 10 client.15693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:40:04.802 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.798+0000 7f0080116b80 20 client.15693 populate_metadata read hostname 'smithi001' 2023-11-16T16:40:04.802 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.798+0000 7f0080116b80 1 --2- 172.21.15.1:0/1034689732 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x562bf3f2ea10 0x562bf3f30e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.802 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.798+0000 7f0080116b80 4 client.15693 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:40:04.802 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.798+0000 7f0080116b80 1 -- 172.21.15.1:0/1034689732 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562bf3e170a0 con 0x562bf3f2ea10 2023-11-16T16:40:04.803 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.799+0000 7f0071525700 1 --2- 172.21.15.1:0/1034689732 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x562bf3f2ea10 0x562bf3f30e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.799+0000 7f00557fa700 10 client.15693 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:40:04.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.799+0000 7f0071525700 1 --2- 172.21.15.1:0/1034689732 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x562bf3f2ea10 0x562bf3f30e80 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffe8d76c1f8ea380 server_cookie=6f7a48f9ccb2c0dc in_seq=0 out_seq=0 2023-11-16T16:40:04.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.800+0000 7f00557fa700 1 -- 172.21.15.1:0/1034689732 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562bf3e170a0 con 0x562bf3f2ea10 2023-11-16T16:40:04.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.800+0000 7f00557fa700 10 client.15693 handle_command_reply: tid=0 2023-11-16T16:40:04.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.800+0000 7f0080116b80 1 client.15693 shutdown 2023-11-16T16:40:04.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.800+0000 7f0080116b80 1 -- 172.21.15.1:0/1034689732 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bf3f0c370 msgr2=0x562bf3f15be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:04.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.800+0000 7f0080116b80 1 --2- 172.21.15.1:0/1034689732 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bf3f0c370 0x562bf3f15be0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f00680cae50 tx=0x7f0068099a10 comp rx=0 tx=0).stop 2023-11-16T16:40:04.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.800+0000 7f0080116b80 1 -- 172.21.15.1:0/1034689732 shutdown_connections 2023-11-16T16:40:04.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.801+0000 7f0080116b80 1 --2- 172.21.15.1:0/1034689732 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x562bf3f2ea10 0x562bf3f30e80 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.801+0000 7f0080116b80 1 --2- 172.21.15.1:0/1034689732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562bf3f110c0 0x562bf3f16660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.801+0000 7f0080116b80 1 --2- 172.21.15.1:0/1034689732 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bf3f0c370 0x562bf3f15be0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.801+0000 7f0080116b80 1 --2- 172.21.15.1:0/1034689732 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562bf3f0cc80 0x562bf3f16120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.805 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.801+0000 7f0080116b80 1 -- 172.21.15.1:0/1034689732 >> 172.21.15.1:0/1034689732 conn(0x562bf3ef9360 msgr2=0x562bf3f12570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:04.805 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.801+0000 7f00557fa700 0 client.15693 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:40:04.805 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.801+0000 7f0080116b80 1 -- 172.21.15.1:0/1034689732 shutdown_connections 2023-11-16T16:40:04.805 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.801+0000 7f0080116b80 1 -- 172.21.15.1:0/1034689732 wait complete. 2023-11-16T16:40:04.805 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.801+0000 7f0080116b80 20 client.15693 trim_cache size 0 max 16384 2023-11-16T16:40:04.828 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.826+0000 7f0080116b80 1 Processor -- start 2023-11-16T16:40:04.828 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.826+0000 7f0080116b80 1 -- start start 2023-11-16T16:40:04.828 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.826+0000 7f0080116b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562bf3f0c370 0x562bf3f0e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.828 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.826+0000 7f0080116b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bf3f0cc80 0x562bf3f16490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.828 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.826+0000 7f0080116b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562bf3f110c0 0x562bf3f0f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.829 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.826+0000 7f0080116b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bf3f0d400 con 0x562bf3f0cc80 2023-11-16T16:40:04.829 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.826+0000 7f0080116b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bf3f14140 con 0x562bf3f0c370 2023-11-16T16:40:04.829 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.826+0000 7f0080116b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bf3f0fdd0 con 0x562bf3f110c0 2023-11-16T16:40:04.829 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.827+0000 7f0071525700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bf3f0cc80 0x562bf3f16490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.829 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.827+0000 7f0071525700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bf3f0cc80 0x562bf3f16490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44156/0 (socket says 172.21.15.1:44156) 2023-11-16T16:40:04.829 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.827+0000 7f0071525700 1 -- 172.21.15.1:0/3357828010 learned_addr learned my addr 172.21.15.1:0/3357828010 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:04.829 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.827+0000 7f0071d26700 1 --2- 172.21.15.1:0/3357828010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562bf3f0c370 0x562bf3f0e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.829 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.827+0000 7f0071525700 1 -- 172.21.15.1:0/3357828010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562bf3f110c0 msgr2=0x562bf3f0f2c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-16T16:40:04.829 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.827+0000 7f0071525700 1 --2- 172.21.15.1:0/3357828010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562bf3f110c0 0x562bf3f0f2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.829 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.827+0000 7f0071525700 1 -- 172.21.15.1:0/3357828010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562bf3f0c370 msgr2=0x562bf3f0e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:04.829 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.827+0000 7f0071525700 1 --2- 172.21.15.1:0/3357828010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562bf3f0c370 0x562bf3f0e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.829 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.827+0000 7f0071525700 1 -- 172.21.15.1:0/3357828010 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bf3e1aee0 con 0x562bf3f0cc80 2023-11-16T16:40:04.830 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.827+0000 7f0071525700 1 --2- 172.21.15.1:0/3357828010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bf3f0cc80 0x562bf3f16490 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f005c0098e0 tx=0x7f005c00b750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d20693be16698 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:04.830 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.827+0000 7f0060ff9700 1 -- 172.21.15.1:0/3357828010 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c003c90 con 0x562bf3f0cc80 2023-11-16T16:40:04.830 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.828+0000 7f0060ff9700 1 -- 172.21.15.1:0/3357828010 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f005c03e070 con 0x562bf3f0cc80 2023-11-16T16:40:04.830 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.828+0000 7f0060ff9700 1 -- 172.21.15.1:0/3357828010 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c03e440 con 0x562bf3f0cc80 2023-11-16T16:40:04.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.829+0000 7f0080116b80 1 -- 172.21.15.1:0/3357828010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bf3f0cc80 msgr2=0x562bf3f16490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:04.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.829+0000 7f0080116b80 1 --2- 172.21.15.1:0/3357828010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bf3f0cc80 0x562bf3f16490 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f005c0098e0 tx=0x7f005c00b750 comp rx=0 tx=0).stop 2023-11-16T16:40:04.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.829+0000 7f0080116b80 1 -- 172.21.15.1:0/3357828010 shutdown_connections 2023-11-16T16:40:04.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.829+0000 7f0080116b80 1 --2- 172.21.15.1:0/3357828010 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562bf3f110c0 0x562bf3f0f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.829+0000 7f0080116b80 1 --2- 172.21.15.1:0/3357828010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562bf3f0c370 0x562bf3f0e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.829+0000 7f0080116b80 1 --2- 172.21.15.1:0/3357828010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bf3f0cc80 0x562bf3f16490 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.829+0000 7f0080116b80 1 -- 172.21.15.1:0/3357828010 >> 172.21.15.1:0/3357828010 conn(0x562bf3ef9360 msgr2=0x562bf3f30a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:04.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.829+0000 7f0080116b80 1 -- 172.21.15.1:0/3357828010 shutdown_connections 2023-11-16T16:40:04.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.829+0000 7f0080116b80 1 -- 172.21.15.1:0/3357828010 wait complete. 2023-11-16T16:40:04.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.829+0000 7f0080116b80 1 -- start start 2023-11-16T16:40:04.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.829+0000 7f0080116b80 1 Processor -- start 2023-11-16T16:40:04.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.830+0000 7f0080116b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562bf3f0c370 0x562bf3f07e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.830+0000 7f0080116b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bf3f0cc80 0x562bf3f05c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.830+0000 7f0080116b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562bf3f110c0 0x562bf3f061c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.830+0000 7f0080116b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bf3f068d0 con 0x562bf3f0cc80 2023-11-16T16:40:04.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.830+0000 7f0080116b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bf3f06a40 con 0x562bf3f0c370 2023-11-16T16:40:04.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.830+0000 7f0080116b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bf3f81970 con 0x562bf3f110c0 2023-11-16T16:40:04.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.830+0000 7f0080116b80 10 client.0 _ll_register_callbacks cb 0x562bf3e14780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:40:04.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.830+0000 7f0071525700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bf3f0cc80 0x562bf3f05c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.830+0000 7f0071525700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bf3f0cc80 0x562bf3f05c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44160/0 (socket says 172.21.15.1:44160) 2023-11-16T16:40:04.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.830+0000 7f0071525700 1 -- 172.21.15.1:0/2752799371 learned_addr learned my addr 172.21.15.1:0/2752799371 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:04.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.830+0000 7f0071d26700 1 --2- 172.21.15.1:0/2752799371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562bf3f0c370 0x562bf3f07e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.830+0000 7f0072527700 1 --2- 172.21.15.1:0/2752799371 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562bf3f110c0 0x562bf3f061c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.830+0000 7f0071525700 1 -- 172.21.15.1:0/2752799371 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562bf3f110c0 msgr2=0x562bf3f061c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:04.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.830+0000 7f0071525700 1 --2- 172.21.15.1:0/2752799371 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562bf3f110c0 0x562bf3f061c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.831+0000 7f0071525700 1 -- 172.21.15.1:0/2752799371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562bf3f0c370 msgr2=0x562bf3f07e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:04.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.831+0000 7f0071525700 1 --2- 172.21.15.1:0/2752799371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562bf3f0c370 0x562bf3f07e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.831+0000 7f0071525700 1 -- 172.21.15.1:0/2752799371 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562bf3cd6630 con 0x562bf3f0cc80 2023-11-16T16:40:04.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.831+0000 7f0071525700 1 --2- 172.21.15.1:0/2752799371 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bf3f0cc80 0x562bf3f05c80 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f005c009350 tx=0x7f005c010320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90cda36d16146495 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:04.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.831+0000 7f00557fa700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:40:04.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.831+0000 7f00557fa700 1 -- 172.21.15.1:0/2752799371 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c00b5b0 con 0x562bf3f0cc80 2023-11-16T16:40:04.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.831+0000 7f00557fa700 1 -- 172.21.15.1:0/2752799371 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f005c011e40 con 0x562bf3f0cc80 2023-11-16T16:40:04.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.831+0000 7f0080116b80 1 -- 172.21.15.1:0/2752799371 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f005c005590 con 0x562bf3f0cc80 2023-11-16T16:40:04.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.831+0000 7f00557fa700 1 -- 172.21.15.1:0/2752799371 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f005c003c90 con 0x562bf3f0cc80 2023-11-16T16:40:04.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.831+0000 7f0080116b80 10 client.15699 fetch_fsmap learned FSMap version 23 2023-11-16T16:40:04.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.831+0000 7f0080116b80 1 -- 172.21.15.1:0/2752799371 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x562bf3c6bc60 con 0x562bf3f0cc80 2023-11-16T16:40:04.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.832+0000 7f00557fa700 1 -- 172.21.15.1:0/2752799371 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f005c055070 con 0x562bf3f0cc80 2023-11-16T16:40:04.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.832+0000 7f00557fa700 1 -- 172.21.15.1:0/2752799371 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f005c00b8e0 con 0x562bf3f0cc80 2023-11-16T16:40:04.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.832+0000 7f0080116b80 10 client.15699 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:40:04.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.832+0000 7f0080116b80 10 client.15699 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:40:04.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.832+0000 7f0080116b80 20 client.15699 populate_metadata read hostname 'smithi001' 2023-11-16T16:40:04.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.832+0000 7f0080116b80 1 --2- 172.21.15.1:0/2752799371 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x562bf3f82bd0 0x562bf3f84fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:04.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.832+0000 7f0080116b80 4 client.15699 mds_command: new command op to 24521 tid=0[{"prefix": "damage ls"}] 2023-11-16T16:40:04.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.832+0000 7f0080116b80 1 -- 172.21.15.1:0/2752799371 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562bf3e23660 con 0x562bf3f82bd0 2023-11-16T16:40:04.838 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.833+0000 7f0071d26700 1 --2- 172.21.15.1:0/2752799371 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x562bf3f82bd0 0x562bf3f84fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:04.838 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.833+0000 7f0071d26700 1 --2- 172.21.15.1:0/2752799371 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x562bf3f82bd0 0x562bf3f84fe0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca5e845edd124088 server_cookie=186cb2232e940378 in_seq=0 out_seq=0 2023-11-16T16:40:04.838 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.833+0000 7f00557fa700 10 client.15699 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:40:04.838 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.834+0000 7f00557fa700 1 -- 172.21.15.1:0/2752799371 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562bf3e23660 con 0x562bf3f82bd0 2023-11-16T16:40:04.838 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.834+0000 7f00557fa700 10 client.15699 handle_command_reply: tid=0 2023-11-16T16:40:04.838 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.834+0000 7f0080116b80 1 client.15699 shutdown 2023-11-16T16:40:04.838 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.834+0000 7f0080116b80 1 -- 172.21.15.1:0/2752799371 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bf3f0cc80 msgr2=0x562bf3f05c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:04.838 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.834+0000 7f0080116b80 1 --2- 172.21.15.1:0/2752799371 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bf3f0cc80 0x562bf3f05c80 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f005c009350 tx=0x7f005c010320 comp rx=0 tx=0).stop 2023-11-16T16:40:04.838 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.834+0000 7f0080116b80 1 -- 172.21.15.1:0/2752799371 shutdown_connections 2023-11-16T16:40:04.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.834+0000 7f0080116b80 1 --2- 172.21.15.1:0/2752799371 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x562bf3f82bd0 0x562bf3f84fe0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.834+0000 7f0080116b80 1 --2- 172.21.15.1:0/2752799371 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562bf3f110c0 0x562bf3f061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.834+0000 7f00557fa700 0 client.15699 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:40:04.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.834+0000 7f0080116b80 1 --2- 172.21.15.1:0/2752799371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562bf3f0c370 0x562bf3f07e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.834+0000 7f0080116b80 1 --2- 172.21.15.1:0/2752799371 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bf3f0cc80 0x562bf3f05c80 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.834+0000 7f0080116b80 1 -- 172.21.15.1:0/2752799371 >> 172.21.15.1:0/2752799371 conn(0x562bf3ef9360 msgr2=0x562bf3f0f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:04.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.834+0000 7f0080116b80 1 -- 172.21.15.1:0/2752799371 shutdown_connections 2023-11-16T16:40:04.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.834+0000 7f0080116b80 1 -- 172.21.15.1:0/2752799371 wait complete. 2023-11-16T16:40:04.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.834+0000 7f0080116b80 20 client.15699 trim_cache size 0 max 16384 2023-11-16T16:40:04.839 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2023-11-16T16:40:04.840 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.835+0000 7f0073f8a700 1 -- 172.21.15.1:0/3597406176 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f00580a13e0 msgr2=0x7f00580a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:04.840 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.835+0000 7f0073f8a700 1 --2- 172.21.15.1:0/3597406176 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f00580a13e0 0x7f00580a3890 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f006c195720 tx=0x7f005c005fb0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.840 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.835+0000 7f0073f8a700 1 -- 172.21.15.1:0/3597406176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f006c0ffa70 msgr2=0x7f006c194110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:04.840 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.835+0000 7f0073f8a700 1 --2- 172.21.15.1:0/3597406176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f006c0ffa70 0x7f006c194110 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f006800e9a0 tx=0x7f006800ecb0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.835+0000 7f0073f8a700 1 -- 172.21.15.1:0/3597406176 shutdown_connections 2023-11-16T16:40:04.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.835+0000 7f0073f8a700 1 --2- 172.21.15.1:0/3597406176 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f00580a13e0 0x7f00580a3890 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.835+0000 7f0073f8a700 1 --2- 172.21.15.1:0/3597406176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f006c105250 0x7f006c198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.835+0000 7f0073f8a700 1 --2- 172.21.15.1:0/3597406176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f006c0ffa70 0x7f006c194110 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.835+0000 7f0073f8a700 1 --2- 172.21.15.1:0/3597406176 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f006c100420 0x7f006c194650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:04.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.835+0000 7f0073f8a700 1 -- 172.21.15.1:0/3597406176 >> 172.21.15.1:0/3597406176 conn(0x7f006c0763a0 msgr2=0x7f006c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:04.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.835+0000 7f0073f8a700 1 -- 172.21.15.1:0/3597406176 shutdown_connections 2023-11-16T16:40:04.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:04.835+0000 7f0073f8a700 1 -- 172.21.15.1:0/3597406176 wait complete. 2023-11-16T16:40:05.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:05 smithi026 ceph-mon[100112]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:40:05.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:05 smithi053 ceph-mon[100638]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:40:05.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:05 smithi001 ceph-mon[98800]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:40:05.853 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-16T16:40:05.853 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-16T16:40:06.055 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.054+0000 7f46b81bf700 1 -- 172.21.15.1:0/4051923592 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46b01005e0 msgr2=0x7f46b01009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:06.056 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.054+0000 7f46b81bf700 1 --2- 172.21.15.1:0/4051923592 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46b01005e0 0x7f46b01009b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f46ac009990 tx=0x7f46ac009ca0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.056 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.054+0000 7f46b81bf700 1 -- 172.21.15.1:0/4051923592 shutdown_connections 2023-11-16T16:40:06.056 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.054+0000 7f46b81bf700 1 --2- 172.21.15.1:0/4051923592 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46b0109b90 0x7f46b010bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.056 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.054+0000 7f46b81bf700 1 --2- 172.21.15.1:0/4051923592 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46b01005e0 0x7f46b01009b0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.056 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.054+0000 7f46b81bf700 1 --2- 172.21.15.1:0/4051923592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46b0100ef0 0x7f46b0109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.056 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.054+0000 7f46b81bf700 1 -- 172.21.15.1:0/4051923592 >> 172.21.15.1:0/4051923592 conn(0x7f46b0074a60 msgr2=0x7f46b0074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:06.056 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.055+0000 7f46b81bf700 1 -- 172.21.15.1:0/4051923592 shutdown_connections 2023-11-16T16:40:06.057 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.055+0000 7f46b81bf700 1 -- 172.21.15.1:0/4051923592 wait complete. 2023-11-16T16:40:06.058 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.055+0000 7f46b81bf700 1 Processor -- start 2023-11-16T16:40:06.058 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.056+0000 7f46b81bf700 1 -- start start 2023-11-16T16:40:06.058 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.056+0000 7f46b81bf700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46b01005e0 0x7f46b01940d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:06.058 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.056+0000 7f46b81bf700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46b0100ef0 0x7f46b0194610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:06.058 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.056+0000 7f46b81bf700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46b0109b90 0x7f46b0198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:06.059 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.056+0000 7f46b81bf700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b01990b0 con 0x7f46b0109b90 2023-11-16T16:40:06.059 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.056+0000 7f46b81bf700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b0199220 con 0x7f46b0100ef0 2023-11-16T16:40:06.059 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.056+0000 7f46b81bf700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b01993e0 con 0x7f46b01005e0 2023-11-16T16:40:06.059 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.056+0000 7f46b675c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46b0109b90 0x7f46b0198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:06.059 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.056+0000 7f46b675c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46b0109b90 0x7f46b0198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44176/0 (socket says 172.21.15.1:44176) 2023-11-16T16:40:06.059 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.056+0000 7f46b675c700 1 -- 172.21.15.1:0/4195279180 learned_addr learned my addr 172.21.15.1:0/4195279180 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:06.059 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.056+0000 7f46b575a700 1 --2- 172.21.15.1:0/4195279180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46b0100ef0 0x7f46b0194610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:06.059 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.056+0000 7f46b5f5b700 1 --2- 172.21.15.1:0/4195279180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46b01005e0 0x7f46b01940d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:06.059 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.056+0000 7f46b675c700 1 -- 172.21.15.1:0/4195279180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46b01005e0 msgr2=0x7f46b01940d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:06.059 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.056+0000 7f46b675c700 1 --2- 172.21.15.1:0/4195279180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46b01005e0 0x7f46b01940d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.060 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.056+0000 7f46b675c700 1 -- 172.21.15.1:0/4195279180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46b0100ef0 msgr2=0x7f46b0194610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:06.060 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.056+0000 7f46b675c700 1 --2- 172.21.15.1:0/4195279180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46b0100ef0 0x7f46b0194610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.060 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.056+0000 7f46b675c700 1 -- 172.21.15.1:0/4195279180 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ac0095f0 con 0x7f46b0109b90 2023-11-16T16:40:06.060 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.057+0000 7f46b675c700 1 --2- 172.21.15.1:0/4195279180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46b0109b90 0x7f46b0198a70 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f46a800f360 tx=0x7f46a800f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:06.060 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.057+0000 7f46a6ffd700 1 -- 172.21.15.1:0/4195279180 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a8019070 con 0x7f46b0109b90 2023-11-16T16:40:06.060 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.057+0000 7f46b81bf700 1 -- 172.21.15.1:0/4195279180 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46b010e180 con 0x7f46b0109b90 2023-11-16T16:40:06.060 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.057+0000 7f46a6ffd700 1 -- 172.21.15.1:0/4195279180 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46a8004bb0 con 0x7f46b0109b90 2023-11-16T16:40:06.060 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.057+0000 7f46a6ffd700 1 -- 172.21.15.1:0/4195279180 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a8010040 con 0x7f46b0109b90 2023-11-16T16:40:06.061 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.057+0000 7f46b81bf700 1 -- 172.21.15.1:0/4195279180 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46b010dff0 con 0x7f46b0109b90 2023-11-16T16:40:06.061 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.060+0000 7f46a6ffd700 1 -- 172.21.15.1:0/4195279180 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 24) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f46a8004d20 con 0x7f46b0109b90 2023-11-16T16:40:06.062 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.061+0000 7f46a6ffd700 1 --2- 172.21.15.1:0/4195279180 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f469c0a13e0 0x7f469c0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:06.062 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.061+0000 7f46b5f5b700 1 --2- 172.21.15.1:0/4195279180 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f469c0a13e0 0x7f469c0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:06.062 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.061+0000 7f46a6ffd700 1 -- 172.21.15.1:0/4195279180 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f46a80cce00 con 0x7f46b0109b90 2023-11-16T16:40:06.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.062+0000 7f46b5f5b700 1 --2- 172.21.15.1:0/4195279180 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f469c0a13e0 0x7f469c0a3890 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00b2e0 tx=0x7f46ac005fd0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:06.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.062+0000 7f46c434bb80 1 Processor -- start 2023-11-16T16:40:06.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.062+0000 7f46c434bb80 1 -- start start 2023-11-16T16:40:06.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.063+0000 7f46c434bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56245fff33f0 0x56245fff37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:06.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.063+0000 7f46c434bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56245fff3d00 0x56245fff7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:06.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.063+0000 7f46c434bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56245fff8140 0x56245fff8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:06.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.063+0000 7f46c434bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56245fef00e0 con 0x56245fff3d00 2023-11-16T16:40:06.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.063+0000 7f46c434bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56245ff42920 con 0x56245fff33f0 2023-11-16T16:40:06.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.063+0000 7f46c434bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56245ff3bdb0 con 0x56245fff8140 2023-11-16T16:40:06.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.063+0000 7f46b5f5b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56245fff33f0 0x56245fff37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:06.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.063+0000 7f46b675c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56245fff8140 0x56245fff8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:06.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.063+0000 7f46b5f5b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56245fff33f0 0x56245fff37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.1:38366/0 (socket says 172.21.15.1:38366) 2023-11-16T16:40:06.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.063+0000 7f46b5f5b700 1 -- 172.21.15.1:0/280106225 learned_addr learned my addr 172.21.15.1:0/280106225 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:06.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.063+0000 7f46b575a700 1 --2- 172.21.15.1:0/280106225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56245fff3d00 0x56245fff7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:06.065 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.063+0000 7f46b5f5b700 1 -- 172.21.15.1:0/280106225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56245fff8140 msgr2=0x56245fff8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:06.065 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.063+0000 7f46b5f5b700 1 --2- 172.21.15.1:0/280106225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56245fff8140 0x56245fff8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.065 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.063+0000 7f46b5f5b700 1 -- 172.21.15.1:0/280106225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56245fff3d00 msgr2=0x56245fff7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:06.065 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.063+0000 7f46b5f5b700 1 --2- 172.21.15.1:0/280106225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56245fff3d00 0x56245fff7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.066 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.063+0000 7f46b5f5b700 1 -- 172.21.15.1:0/280106225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56245fdbd6b0 con 0x56245fff33f0 2023-11-16T16:40:06.066 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.064+0000 7f46b5f5b700 1 --2- 172.21.15.1:0/280106225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56245fff33f0 0x56245fff37c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f46ac015000 tx=0x7f46ac020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cebcd16461be591a server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:06.066 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.064+0000 7f469bfff700 1 -- 172.21.15.1:0/280106225 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac005250 con 0x56245fff33f0 2023-11-16T16:40:06.066 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.064+0000 7f469bfff700 1 -- 172.21.15.1:0/280106225 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46ac02a040 con 0x56245fff33f0 2023-11-16T16:40:06.066 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.064+0000 7f469bfff700 1 -- 172.21.15.1:0/280106225 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac0049d0 con 0x56245fff33f0 2023-11-16T16:40:06.066 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.065+0000 7f46c434bb80 1 -- 172.21.15.1:0/280106225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56245fff33f0 msgr2=0x56245fff37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:06.066 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.065+0000 7f46c434bb80 1 --2- 172.21.15.1:0/280106225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56245fff33f0 0x56245fff37c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f46ac015000 tx=0x7f46ac020000 comp rx=0 tx=0).stop 2023-11-16T16:40:06.070 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.070+0000 7f46c434bb80 1 -- 172.21.15.1:0/280106225 shutdown_connections 2023-11-16T16:40:06.071 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.070+0000 7f46c434bb80 1 --2- 172.21.15.1:0/280106225 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56245fff8140 0x56245fff8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.071 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.070+0000 7f46c434bb80 1 --2- 172.21.15.1:0/280106225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56245fff33f0 0x56245fff37c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.071 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.070+0000 7f46c434bb80 1 --2- 172.21.15.1:0/280106225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56245fff3d00 0x56245fff7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.071 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.070+0000 7f46c434bb80 1 -- 172.21.15.1:0/280106225 >> 172.21.15.1:0/280106225 conn(0x56245ffe03e0 msgr2=0x56245ffe5bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:06.071 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.070+0000 7f46c434bb80 1 -- 172.21.15.1:0/280106225 shutdown_connections 2023-11-16T16:40:06.071 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.070+0000 7f46c434bb80 1 -- 172.21.15.1:0/280106225 wait complete. 2023-11-16T16:40:06.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.070+0000 7f46c434bb80 1 -- start start 2023-11-16T16:40:06.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.070+0000 7f46c434bb80 1 Processor -- start 2023-11-16T16:40:06.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.071+0000 7f46c434bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56245fff33f0 0x56245ffff830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:06.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.071+0000 7f46c434bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56245fff3d00 0x56245fffcb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:06.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.071+0000 7f46c434bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56245fff8140 0x56245fffd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:06.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.071+0000 7f46c434bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56245fedd960 con 0x56245fff33f0 2023-11-16T16:40:06.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.071+0000 7f46c434bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56245fedd600 con 0x56245fff3d00 2023-11-16T16:40:06.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.071+0000 7f46c434bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56245fdbf9a0 con 0x56245fff8140 2023-11-16T16:40:06.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.071+0000 7f46c434bb80 10 client.0 _ll_register_callbacks cb 0x56245fefb800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:40:06.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.071+0000 7f46b5f5b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56245fff33f0 0x56245ffff830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:06.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.071+0000 7f46b5f5b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56245fff33f0 0x56245ffff830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44186/0 (socket says 172.21.15.1:44186) 2023-11-16T16:40:06.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.071+0000 7f46b5f5b700 1 -- 172.21.15.1:0/3938283687 learned_addr learned my addr 172.21.15.1:0/3938283687 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:06.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.071+0000 7f46b675c700 1 --2- 172.21.15.1:0/3938283687 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56245fff8140 0x56245fffd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:06.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.072+0000 7f46b5f5b700 1 -- 172.21.15.1:0/3938283687 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56245fff8140 msgr2=0x56245fffd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:06.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.072+0000 7f46b5f5b700 1 --2- 172.21.15.1:0/3938283687 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56245fff8140 0x56245fffd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.072+0000 7f46b5f5b700 1 -- 172.21.15.1:0/3938283687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56245fff3d00 msgr2=0x56245fffcb20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-16T16:40:06.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.072+0000 7f46b5f5b700 1 --2- 172.21.15.1:0/3938283687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56245fff3d00 0x56245fffcb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.072+0000 7f46b5f5b700 1 -- 172.21.15.1:0/3938283687 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46b010c8a0 con 0x56245fff33f0 2023-11-16T16:40:06.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.072+0000 7f46b5f5b700 1 --2- 172.21.15.1:0/3938283687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56245fff33f0 0x56245ffff830 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00bf90 tx=0x7f46ac005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=136cc6d2472f85ed server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:06.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.072+0000 7f46997fa700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:40:06.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.072+0000 7f46997fa700 1 -- 172.21.15.1:0/3938283687 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac004a20 con 0x56245fff33f0 2023-11-16T16:40:06.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.072+0000 7f46997fa700 1 -- 172.21.15.1:0/3938283687 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f46ac004b80 con 0x56245fff33f0 2023-11-16T16:40:06.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.072+0000 7f46c434bb80 1 -- 172.21.15.1:0/3938283687 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56245ffddba0 con 0x56245fff33f0 2023-11-16T16:40:06.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.072+0000 7f46997fa700 1 -- 172.21.15.1:0/3938283687 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46ac004d40 con 0x56245fff33f0 2023-11-16T16:40:06.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.072+0000 7f46c434bb80 10 client.15711 fetch_fsmap learned FSMap version 23 2023-11-16T16:40:06.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.072+0000 7f46c434bb80 1 -- 172.21.15.1:0/3938283687 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56245ff02210 con 0x56245fff33f0 2023-11-16T16:40:06.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.073+0000 7f46997fa700 1 -- 172.21.15.1:0/3938283687 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46ac045080 con 0x56245fff33f0 2023-11-16T16:40:06.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.073+0000 7f46997fa700 1 -- 172.21.15.1:0/3938283687 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f46ac02ebc0 con 0x56245fff33f0 2023-11-16T16:40:06.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.073+0000 7f46c434bb80 10 client.15711 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:40:06.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.073+0000 7f46c434bb80 10 client.15711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:40:06.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.073+0000 7f46c434bb80 20 client.15711 populate_metadata read hostname 'smithi001' 2023-11-16T16:40:06.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.073+0000 7f46c434bb80 1 --2- 172.21.15.1:0/3938283687 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56246000cba0 0x56246000efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:06.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.073+0000 7f46c434bb80 4 client.15711 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:40:06.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.073+0000 7f46c434bb80 1 -- 172.21.15.1:0/3938283687 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56245fefe120 con 0x56246000cba0 2023-11-16T16:40:06.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.073+0000 7f46b575a700 1 --2- 172.21.15.1:0/3938283687 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56246000cba0 0x56246000efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:06.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.074+0000 7f46b575a700 1 --2- 172.21.15.1:0/3938283687 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56246000cba0 0x56246000efb0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ed5c069eb408ffb server_cookie=2603c27fe99e634b in_seq=0 out_seq=0 2023-11-16T16:40:06.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.074+0000 7f46997fa700 10 client.15711 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:40:06.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.074+0000 7f46997fa700 1 -- 172.21.15.1:0/3938283687 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56245fefe120 con 0x56246000cba0 2023-11-16T16:40:06.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.074+0000 7f46997fa700 10 client.15711 handle_command_reply: tid=0 2023-11-16T16:40:06.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.074+0000 7f46c434bb80 1 client.15711 shutdown 2023-11-16T16:40:06.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.074+0000 7f46c434bb80 1 -- 172.21.15.1:0/3938283687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56245fff33f0 msgr2=0x56245ffff830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:06.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.074+0000 7f46c434bb80 1 --2- 172.21.15.1:0/3938283687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56245fff33f0 0x56245ffff830 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00bf90 tx=0x7f46ac005280 comp rx=0 tx=0).stop 2023-11-16T16:40:06.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.075+0000 7f46c434bb80 1 -- 172.21.15.1:0/3938283687 shutdown_connections 2023-11-16T16:40:06.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.075+0000 7f46c434bb80 1 --2- 172.21.15.1:0/3938283687 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56246000cba0 0x56246000efb0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.075+0000 7f46c434bb80 1 --2- 172.21.15.1:0/3938283687 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56245fff8140 0x56245fffd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.075+0000 7f46c434bb80 1 --2- 172.21.15.1:0/3938283687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56245fff33f0 0x56245ffff830 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.075+0000 7f46c434bb80 1 --2- 172.21.15.1:0/3938283687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56245fff3d00 0x56245fffcb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.075+0000 7f46c434bb80 1 -- 172.21.15.1:0/3938283687 >> 172.21.15.1:0/3938283687 conn(0x56245ffe03e0 msgr2=0x56245fff95f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:06.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.075+0000 7f46997fa700 0 client.15711 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:40:06.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.075+0000 7f46c434bb80 1 -- 172.21.15.1:0/3938283687 shutdown_connections 2023-11-16T16:40:06.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.075+0000 7f46c434bb80 1 -- 172.21.15.1:0/3938283687 wait complete. 2023-11-16T16:40:06.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.075+0000 7f46c434bb80 20 client.15711 trim_cache size 0 max 16384 2023-11-16T16:40:06.109 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.108+0000 7f46c434bb80 1 Processor -- start 2023-11-16T16:40:06.110 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.108+0000 7f46c434bb80 1 -- start start 2023-11-16T16:40:06.110 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.109+0000 7f46c434bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56245fff33f0 0x56245fff7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:06.110 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.109+0000 7f46c434bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56245fff3d00 0x56245ffff3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:06.110 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.109+0000 7f46c434bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56245fff8140 0x56245fffc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:06.110 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.109+0000 7f46c434bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624600098c0 con 0x56245fff3d00 2023-11-16T16:40:06.110 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.109+0000 7f46c434bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56245ffe6860 con 0x56245fff33f0 2023-11-16T16:40:06.110 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.109+0000 7f46c434bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56245ffe6a40 con 0x56245fff8140 2023-11-16T16:40:06.111 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.109+0000 7f46b675c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56245fff8140 0x56245fffc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:06.111 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.109+0000 7f46b5f5b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56245fff33f0 0x56245fff7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:06.111 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.109+0000 7f46b675c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56245fff8140 0x56245fffc2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:40878/0 (socket says 172.21.15.1:40878) 2023-11-16T16:40:06.111 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.109+0000 7f46b5f5b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56245fff33f0 0x56245fff7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.1:38380/0 (socket says 172.21.15.1:38380) 2023-11-16T16:40:06.111 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.109+0000 7f46b675c700 1 -- 172.21.15.1:0/4127122422 learned_addr learned my addr 172.21.15.1:0/4127122422 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:06.112 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.109+0000 7f46b575a700 1 --2- 172.21.15.1:0/4127122422 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56245fff3d00 0x56245ffff3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:06.112 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.109+0000 7f46b5f5b700 1 -- 172.21.15.1:0/4127122422 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56245fff8140 msgr2=0x56245fffc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:06.112 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.109+0000 7f46b5f5b700 1 --2- 172.21.15.1:0/4127122422 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56245fff8140 0x56245fffc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.112 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.109+0000 7f46b5f5b700 1 -- 172.21.15.1:0/4127122422 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56245fff3d00 msgr2=0x56245ffff3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:06.112 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.109+0000 7f46b5f5b700 1 --2- 172.21.15.1:0/4127122422 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56245fff3d00 0x56245ffff3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.112 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.109+0000 7f46b5f5b700 1 -- 172.21.15.1:0/4127122422 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562460013ba0 con 0x56245fff33f0 2023-11-16T16:40:06.112 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.110+0000 7f46b5f5b700 1 --2- 172.21.15.1:0/4127122422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56245fff33f0 0x56245fff7820 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00b6f0 tx=0x7f46ac0098c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7dd8a2603c1804 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:06.112 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.110+0000 7f46a4ff9700 1 -- 172.21.15.1:0/4127122422 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac0373f0 con 0x56245fff33f0 2023-11-16T16:40:06.112 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.110+0000 7f46a4ff9700 1 -- 172.21.15.1:0/4127122422 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56245ffddba0 con 0x56245fff33f0 2023-11-16T16:40:06.112 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.110+0000 7f46a4ff9700 1 -- 172.21.15.1:0/4127122422 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac024bd0 con 0x56245fff33f0 2023-11-16T16:40:06.115 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.111+0000 7f46c434bb80 1 -- 172.21.15.1:0/4127122422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56245fff33f0 msgr2=0x56245fff7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:06.115 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.111+0000 7f46c434bb80 1 --2- 172.21.15.1:0/4127122422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56245fff33f0 0x56245fff7820 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00b6f0 tx=0x7f46ac0098c0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.116 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.111+0000 7f46c434bb80 1 -- 172.21.15.1:0/4127122422 shutdown_connections 2023-11-16T16:40:06.116 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.111+0000 7f46c434bb80 1 --2- 172.21.15.1:0/4127122422 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56245fff8140 0x56245fffc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.116 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.111+0000 7f46c434bb80 1 --2- 172.21.15.1:0/4127122422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56245fff33f0 0x56245fff7820 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.116 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.111+0000 7f46c434bb80 1 --2- 172.21.15.1:0/4127122422 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56245fff3d00 0x56245ffff3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.116 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.111+0000 7f46c434bb80 1 -- 172.21.15.1:0/4127122422 >> 172.21.15.1:0/4127122422 conn(0x56245ffe03e0 msgr2=0x56245fff7110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:06.116 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.111+0000 7f46c434bb80 1 -- 172.21.15.1:0/4127122422 shutdown_connections 2023-11-16T16:40:06.116 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.111+0000 7f46c434bb80 1 -- 172.21.15.1:0/4127122422 wait complete. 2023-11-16T16:40:06.116 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.111+0000 7f46c434bb80 1 -- start start 2023-11-16T16:40:06.116 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.111+0000 7f46c434bb80 1 Processor -- start 2023-11-16T16:40:06.116 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.112+0000 7f46c434bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56245fff33f0 0x562460003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:06.116 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.112+0000 7f46c434bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56245fff3d00 0x562460004120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:06.116 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.112+0000 7f46c434bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56245fff8140 0x5624600021c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:06.117 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.112+0000 7f46c434bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624600027a0 con 0x56245fff8140 2023-11-16T16:40:06.117 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.112+0000 7f46c434bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562460002910 con 0x56245fff33f0 2023-11-16T16:40:06.117 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.112+0000 7f46c434bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562460002ad0 con 0x56245fff3d00 2023-11-16T16:40:06.117 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.112+0000 7f46c434bb80 10 client.0 _ll_register_callbacks cb 0x7f46a000faa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:40:06.117 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.112+0000 7f46b5f5b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56245fff33f0 0x562460003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:06.117 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.112+0000 7f46b5f5b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56245fff33f0 0x562460003be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.1:38386/0 (socket says 172.21.15.1:38386) 2023-11-16T16:40:06.117 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.112+0000 7f46b5f5b700 1 -- 172.21.15.1:0/219037636 learned_addr learned my addr 172.21.15.1:0/219037636 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:06.117 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.112+0000 7f46b675c700 1 --2- 172.21.15.1:0/219037636 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56245fff8140 0x5624600021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:06.117 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.112+0000 7f46b575a700 1 --2- 172.21.15.1:0/219037636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56245fff3d00 0x562460004120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:06.117 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.113+0000 7f46b675c700 1 -- 172.21.15.1:0/219037636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56245fff3d00 msgr2=0x562460004120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:06.117 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.113+0000 7f46b675c700 1 --2- 172.21.15.1:0/219037636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56245fff3d00 0x562460004120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.118 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.113+0000 7f46b675c700 1 -- 172.21.15.1:0/219037636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56245fff33f0 msgr2=0x562460003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:06.118 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.113+0000 7f46b675c700 1 --2- 172.21.15.1:0/219037636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56245fff33f0 0x562460003be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.118 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.113+0000 7f46b675c700 1 -- 172.21.15.1:0/219037636 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56245ff02210 con 0x56245fff8140 2023-11-16T16:40:06.118 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.113+0000 7f46b575a700 1 --2- 172.21.15.1:0/219037636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56245fff3d00 0x562460004120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:40:06.118 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.113+0000 7f46b675c700 1 --2- 172.21.15.1:0/219037636 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56245fff8140 0x5624600021c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f46a80cab60 tx=0x7f46a800b550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2b58fac71044666 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:06.118 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.113+0000 7f46997fa700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:40:06.118 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.113+0000 7f46997fa700 1 -- 172.21.15.1:0/219037636 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a809dd50 con 0x56245fff8140 2023-11-16T16:40:06.118 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.113+0000 7f46997fa700 1 -- 172.21.15.1:0/219037636 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f46a80a20e0 con 0x56245fff8140 2023-11-16T16:40:06.118 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.113+0000 7f46c434bb80 1 -- 172.21.15.1:0/219037636 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f46ac02a040 con 0x56245fff8140 2023-11-16T16:40:06.118 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.113+0000 7f46997fa700 1 -- 172.21.15.1:0/219037636 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46a80b2260 con 0x56245fff8140 2023-11-16T16:40:06.118 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.113+0000 7f46c434bb80 10 client.15717 fetch_fsmap learned FSMap version 23 2023-11-16T16:40:06.119 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.113+0000 7f46c434bb80 1 -- 172.21.15.1:0/219037636 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56245fd69e60 con 0x56245fff8140 2023-11-16T16:40:06.119 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.114+0000 7f46997fa700 1 -- 172.21.15.1:0/219037636 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46a80ca030 con 0x56245fff8140 2023-11-16T16:40:06.119 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.114+0000 7f46997fa700 1 -- 172.21.15.1:0/219037636 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f46a80ca6c0 con 0x56245fff8140 2023-11-16T16:40:06.119 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.114+0000 7f46c434bb80 10 client.15717 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:40:06.119 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.114+0000 7f46c434bb80 10 client.15717 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:40:06.119 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.114+0000 7f46c434bb80 20 client.15717 populate_metadata read hostname 'smithi001' 2023-11-16T16:40:06.119 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.114+0000 7f46c434bb80 1 --2- 172.21.15.1:0/219037636 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x562460061b50 0x562460063f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:06.119 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.114+0000 7f46c434bb80 4 client.15717 mds_command: new command op to 24521 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-16T16:40:06.119 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.114+0000 7f46c434bb80 1 -- 172.21.15.1:0/219037636 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56245ff0a6e0 con 0x562460061b50 2023-11-16T16:40:06.119 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.114+0000 7f46b5f5b700 1 --2- 172.21.15.1:0/219037636 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x562460061b50 0x562460063f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:06.119 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.115+0000 7f46997fa700 10 client.15717 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:40:06.120 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.115+0000 7f46b5f5b700 1 --2- 172.21.15.1:0/219037636 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x562460061b50 0x562460063f60 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b00269171c7be761 server_cookie=46cee9ed598534bc in_seq=0 out_seq=0 2023-11-16T16:40:06.121 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.116+0000 7f46997fa700 1 -- 172.21.15.1:0/219037636 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56245ff0a6e0 con 0x562460061b50 2023-11-16T16:40:06.121 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.116+0000 7f46997fa700 10 client.15717 handle_command_reply: tid=0 2023-11-16T16:40:06.121 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.116+0000 7f46c434bb80 1 client.15717 shutdown 2023-11-16T16:40:06.121 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.116+0000 7f46c434bb80 1 -- 172.21.15.1:0/219037636 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56245fff8140 msgr2=0x5624600021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:06.121 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.116+0000 7f46c434bb80 1 --2- 172.21.15.1:0/219037636 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56245fff8140 0x5624600021c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f46a80cab60 tx=0x7f46a800b550 comp rx=0 tx=0).stop 2023-11-16T16:40:06.121 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.117+0000 7f46c434bb80 1 -- 172.21.15.1:0/219037636 shutdown_connections 2023-11-16T16:40:06.121 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.117+0000 7f46c434bb80 1 --2- 172.21.15.1:0/219037636 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x562460061b50 0x562460063f60 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.121 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.117+0000 7f46c434bb80 1 --2- 172.21.15.1:0/219037636 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56245fff8140 0x5624600021c0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.121 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.117+0000 7f46c434bb80 1 --2- 172.21.15.1:0/219037636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56245fff33f0 0x562460003be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.122 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.117+0000 7f46c434bb80 1 --2- 172.21.15.1:0/219037636 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56245fff3d00 0x562460004120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.122 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.117+0000 7f46c434bb80 1 -- 172.21.15.1:0/219037636 >> 172.21.15.1:0/219037636 conn(0x56245ffe03e0 msgr2=0x56245fff2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:06.122 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.117+0000 7f46997fa700 0 client.15717 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:40:06.122 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.117+0000 7f46c434bb80 1 -- 172.21.15.1:0/219037636 shutdown_connections 2023-11-16T16:40:06.122 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.117+0000 7f46c434bb80 1 -- 172.21.15.1:0/219037636 wait complete. 2023-11-16T16:40:06.122 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.117+0000 7f46c434bb80 20 client.15717 trim_cache size 0 max 16384 2023-11-16T16:40:06.122 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:40:06.122 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2023-11-16T16:40:06.122 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "b06a01b7-cea1-4aaa-820f-a0b39514f5f5", 2023-11-16T16:40:06.123 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2023-11-16T16:40:06.123 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:40:06.123 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.118+0000 7f46b81bf700 1 -- 172.21.15.1:0/4195279180 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f469c0a13e0 msgr2=0x7f469c0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:06.123 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.118+0000 7f46b81bf700 1 --2- 172.21.15.1:0/4195279180 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f469c0a13e0 0x7f469c0a3890 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00b2e0 tx=0x7f46ac005fd0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.124 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.118+0000 7f46b81bf700 1 -- 172.21.15.1:0/4195279180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46b0109b90 msgr2=0x7f46b0198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:06.124 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.118+0000 7f46b81bf700 1 --2- 172.21.15.1:0/4195279180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46b0109b90 0x7f46b0198a70 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f46a800f360 tx=0x7f46a800f670 comp rx=0 tx=0).stop 2023-11-16T16:40:06.124 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.118+0000 7f46b81bf700 1 -- 172.21.15.1:0/4195279180 shutdown_connections 2023-11-16T16:40:06.124 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.118+0000 7f46b81bf700 1 --2- 172.21.15.1:0/4195279180 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f469c0a13e0 0x7f469c0a3890 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.124 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.118+0000 7f46b81bf700 1 --2- 172.21.15.1:0/4195279180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46b0109b90 0x7f46b0198a70 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.124 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.118+0000 7f46b81bf700 1 --2- 172.21.15.1:0/4195279180 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f46b01005e0 0x7f46b01940d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.124 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.118+0000 7f46b81bf700 1 --2- 172.21.15.1:0/4195279180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46b0100ef0 0x7f46b0194610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.124 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.118+0000 7f46b81bf700 1 -- 172.21.15.1:0/4195279180 >> 172.21.15.1:0/4195279180 conn(0x7f46b0074a60 msgr2=0x7f46b00fc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:06.124 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.118+0000 7f46b81bf700 1 -- 172.21.15.1:0/4195279180 shutdown_connections 2023-11-16T16:40:06.124 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.118+0000 7f46b81bf700 1 -- 172.21.15.1:0/4195279180 wait complete. 2023-11-16T16:40:06.135 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-16T16:40:06.335 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.334+0000 7f8e8c6b6700 1 -- 172.21.15.1:0/2956748257 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e841002a0 msgr2=0x7f8e8410bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:06.335 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.334+0000 7f8e8c6b6700 1 --2- 172.21.15.1:0/2956748257 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e841002a0 0x7f8e8410bfd0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c00b0d0 tx=0x7f8e7c00b3e0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.335 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.334+0000 7f8e8c6b6700 1 -- 172.21.15.1:0/2956748257 shutdown_connections 2023-11-16T16:40:06.335 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.334+0000 7f8e8c6b6700 1 --2- 172.21.15.1:0/2956748257 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e841002a0 0x7f8e8410bfd0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.336 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.334+0000 7f8e8c6b6700 1 --2- 172.21.15.1:0/2956748257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e84104140 0x7f8e84104510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.336 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.334+0000 7f8e8c6b6700 1 --2- 172.21.15.1:0/2956748257 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e840ff8f0 0x7f8e840ffd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.336 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.334+0000 7f8e8c6b6700 1 -- 172.21.15.1:0/2956748257 >> 172.21.15.1:0/2956748257 conn(0x7f8e84075880 msgr2=0x7f8e84075c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:06.336 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.335+0000 7f8e8c6b6700 1 -- 172.21.15.1:0/2956748257 shutdown_connections 2023-11-16T16:40:06.339 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.338+0000 7f8e8c6b6700 1 -- 172.21.15.1:0/2956748257 wait complete. 2023-11-16T16:40:06.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.339+0000 7f8e8c6b6700 1 Processor -- start 2023-11-16T16:40:06.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.339+0000 7f8e8c6b6700 1 -- start start 2023-11-16T16:40:06.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.340+0000 7f8e8c6b6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e840ff8f0 0x7f8e84197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:06.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.340+0000 7f8e8c6b6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e841002a0 0x7f8e841978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:06.341 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.340+0000 7f8e8c6b6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e84104140 0x7f8e841a23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:06.341 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.340+0000 7f8e8c6b6700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e841981e0 con 0x7f8e840ff8f0 2023-11-16T16:40:06.341 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.340+0000 7f8e8c6b6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e841a2920 con 0x7f8e841002a0 2023-11-16T16:40:06.341 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.340+0000 7f8e8c6b6700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e841a2ae0 con 0x7f8e84104140 2023-11-16T16:40:06.342 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.340+0000 7f8e8a452700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e840ff8f0 0x7f8e84197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:06.342 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.340+0000 7f8e8a452700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e840ff8f0 0x7f8e84197360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44214/0 (socket says 172.21.15.1:44214) 2023-11-16T16:40:06.342 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.340+0000 7f8e8a452700 1 -- 172.21.15.1:0/195100303 learned_addr learned my addr 172.21.15.1:0/195100303 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:06.342 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.340+0000 7f8e8ac53700 1 --2- 172.21.15.1:0/195100303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e84104140 0x7f8e841a23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:06.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.340+0000 7f8e89c51700 1 --2- 172.21.15.1:0/195100303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e841002a0 0x7f8e841978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:06.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.340+0000 7f8e8ac53700 1 -- 172.21.15.1:0/195100303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e841002a0 msgr2=0x7f8e841978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:06.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.340+0000 7f8e8ac53700 1 --2- 172.21.15.1:0/195100303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e841002a0 0x7f8e841978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.340+0000 7f8e8ac53700 1 -- 172.21.15.1:0/195100303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e840ff8f0 msgr2=0x7f8e84197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:06.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.340+0000 7f8e8ac53700 1 --2- 172.21.15.1:0/195100303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e840ff8f0 0x7f8e84197360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.340+0000 7f8e8ac53700 1 -- 172.21.15.1:0/195100303 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e7c009d00 con 0x7f8e84104140 2023-11-16T16:40:06.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.341+0000 7f8e8a452700 1 --2- 172.21.15.1:0/195100303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e840ff8f0 0x7f8e84197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:40:06.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.341+0000 7f8e89c51700 1 --2- 172.21.15.1:0/195100303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e841002a0 0x7f8e841978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:40:06.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.341+0000 7f8e8ac53700 1 --2- 172.21.15.1:0/195100303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e84104140 0x7f8e841a23e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c00b770 tx=0x7f8e7c007870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:06.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.341+0000 7f8e7b7fe700 1 -- 172.21.15.1:0/195100303 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c00e040 con 0x7f8e84104140 2023-11-16T16:40:06.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.341+0000 7f8e7b7fe700 1 -- 172.21.15.1:0/195100303 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e7c020070 con 0x7f8e84104140 2023-11-16T16:40:06.344 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.341+0000 7f8e7b7fe700 1 -- 172.21.15.1:0/195100303 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c027980 con 0x7f8e84104140 2023-11-16T16:40:06.344 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.341+0000 7f8e8c6b6700 1 -- 172.21.15.1:0/195100303 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e8410f230 con 0x7f8e84104140 2023-11-16T16:40:06.344 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.341+0000 7f8e8c6b6700 1 -- 172.21.15.1:0/195100303 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e8410f0a0 con 0x7f8e84104140 2023-11-16T16:40:06.345 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.344+0000 7f8e7b7fe700 1 -- 172.21.15.1:0/195100303 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 24) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f8e7c01c6c0 con 0x7f8e84104140 2023-11-16T16:40:06.345 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.345+0000 7f8e7b7fe700 1 --2- 172.21.15.1:0/195100303 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f8e700a13e0 0x7f8e700a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:06.346 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.345+0000 7f8e7b7fe700 1 -- 172.21.15.1:0/195100303 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f8e7c022070 con 0x7f8e84104140 2023-11-16T16:40:06.346 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.345+0000 7f8e8a452700 1 --2- 172.21.15.1:0/195100303 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f8e700a13e0 0x7f8e700a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:06.347 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.346+0000 7f8e8a452700 1 --2- 172.21.15.1:0/195100303 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f8e700a13e0 0x7f8e700a3890 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8e80009ea0 tx=0x7f8e80009290 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:06.347 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.346+0000 7f8e98842b80 1 Processor -- start 2023-11-16T16:40:06.347 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.346+0000 7f8e98842b80 1 -- start start 2023-11-16T16:40:06.348 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.347+0000 7f8e98842b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da54b89b60 0x55da54b89f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:06.348 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.347+0000 7f8e98842b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da54b8a470 0x55da54b8e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:06.349 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.347+0000 7f8e98842b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da54b8e8b0 0x55da54b8ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:06.349 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.347+0000 7f8e98842b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da54a8f060 con 0x55da54b89b60 2023-11-16T16:40:06.349 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.347+0000 7f8e98842b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da54ae18a0 con 0x55da54b8a470 2023-11-16T16:40:06.349 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.347+0000 7f8e98842b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da54adad30 con 0x55da54b8e8b0 2023-11-16T16:40:06.349 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.347+0000 7f8e8a452700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da54b89b60 0x55da54b89f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:06.349 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.347+0000 7f8e8a452700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da54b89b60 0x55da54b89f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44222/0 (socket says 172.21.15.1:44222) 2023-11-16T16:40:06.349 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.347+0000 7f8e8a452700 1 -- 172.21.15.1:0/3385645759 learned_addr learned my addr 172.21.15.1:0/3385645759 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:06.349 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.347+0000 7f8e89c51700 1 --2- 172.21.15.1:0/3385645759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da54b8a470 0x55da54b8e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:06.349 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.347+0000 7f8e8a452700 1 -- 172.21.15.1:0/3385645759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da54b8e8b0 msgr2=0x55da54b8ed00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-16T16:40:06.349 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.347+0000 7f8e8a452700 1 --2- 172.21.15.1:0/3385645759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da54b8e8b0 0x55da54b8ed00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.349 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.347+0000 7f8e8a452700 1 -- 172.21.15.1:0/3385645759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da54b8a470 msgr2=0x55da54b8e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:06.350 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.347+0000 7f8e8a452700 1 --2- 172.21.15.1:0/3385645759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da54b8a470 0x55da54b8e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.350 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.347+0000 7f8e8a452700 1 -- 172.21.15.1:0/3385645759 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da5495c630 con 0x55da54b89b60 2023-11-16T16:40:06.350 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.347+0000 7f8e89c51700 1 --2- 172.21.15.1:0/3385645759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da54b8a470 0x55da54b8e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:40:06.350 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.347+0000 7f8e8a452700 1 --2- 172.21.15.1:0/3385645759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da54b89b60 0x55da54b89f30 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8e8000ef10 tx=0x7f8e80011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6959bd0adfaefa7d server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:06.350 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.348+0000 7f8e78ff9700 1 -- 172.21.15.1:0/3385645759 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e80015910 con 0x55da54b89b60 2023-11-16T16:40:06.350 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.348+0000 7f8e78ff9700 1 -- 172.21.15.1:0/3385645759 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e80013450 con 0x55da54b89b60 2023-11-16T16:40:06.350 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.348+0000 7f8e78ff9700 1 -- 172.21.15.1:0/3385645759 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8001f650 con 0x55da54b89b60 2023-11-16T16:40:06.353 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.349+0000 7f8e98842b80 1 -- 172.21.15.1:0/3385645759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da54b89b60 msgr2=0x55da54b89f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:06.353 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.349+0000 7f8e98842b80 1 --2- 172.21.15.1:0/3385645759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da54b89b60 0x55da54b89f30 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8e8000ef10 tx=0x7f8e80011000 comp rx=0 tx=0).stop 2023-11-16T16:40:06.353 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.349+0000 7f8e98842b80 1 -- 172.21.15.1:0/3385645759 shutdown_connections 2023-11-16T16:40:06.354 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.349+0000 7f8e98842b80 1 --2- 172.21.15.1:0/3385645759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da54b8e8b0 0x55da54b8ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.354 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.349+0000 7f8e98842b80 1 --2- 172.21.15.1:0/3385645759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da54b89b60 0x55da54b89f30 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.354 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.349+0000 7f8e98842b80 1 --2- 172.21.15.1:0/3385645759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da54b8a470 0x55da54b8e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.354 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.349+0000 7f8e98842b80 1 -- 172.21.15.1:0/3385645759 >> 172.21.15.1:0/3385645759 conn(0x55da54b7f360 msgr2=0x55da54b84b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:06.354 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.349+0000 7f8e98842b80 1 -- 172.21.15.1:0/3385645759 shutdown_connections 2023-11-16T16:40:06.354 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.349+0000 7f8e98842b80 1 -- 172.21.15.1:0/3385645759 wait complete. 2023-11-16T16:40:06.354 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.349+0000 7f8e98842b80 1 -- start start 2023-11-16T16:40:06.354 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.350+0000 7f8e98842b80 1 Processor -- start 2023-11-16T16:40:06.354 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.350+0000 7f8e98842b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da54b89b60 0x55da54b933b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:06.354 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.350+0000 7f8e98842b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da54b8a470 0x55da54b938f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:06.354 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.350+0000 7f8e98842b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da54b8e8b0 0x55da54b93e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:06.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.350+0000 7f8e98842b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da54a7c8e0 con 0x55da54b8a470 2023-11-16T16:40:06.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.350+0000 7f8e98842b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da54a7c580 con 0x55da54b89b60 2023-11-16T16:40:06.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.350+0000 7f8e98842b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da5495e920 con 0x55da54b8e8b0 2023-11-16T16:40:06.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.350+0000 7f8e98842b80 10 client.0 _ll_register_callbacks cb 0x55da54a9a780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:40:06.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.350+0000 7f8e8a452700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da54b89b60 0x55da54b933b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:06.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.350+0000 7f8e8a452700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da54b89b60 0x55da54b933b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.1:38414/0 (socket says 172.21.15.1:38414) 2023-11-16T16:40:06.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.350+0000 7f8e8a452700 1 -- 172.21.15.1:0/39121238 learned_addr learned my addr 172.21.15.1:0/39121238 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:06.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.351+0000 7f8e8a452700 1 -- 172.21.15.1:0/39121238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da54b8e8b0 msgr2=0x55da54b93e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:06.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.351+0000 7f8e8a452700 1 --2- 172.21.15.1:0/39121238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da54b8e8b0 0x55da54b93e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.351+0000 7f8e8a452700 1 -- 172.21.15.1:0/39121238 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da54b8a470 msgr2=0x55da54b938f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:06.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.351+0000 7f8e8a452700 1 --2- 172.21.15.1:0/39121238 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da54b8a470 0x55da54b938f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.351+0000 7f8e8a452700 1 -- 172.21.15.1:0/39121238 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e8000ebc0 con 0x55da54b89b60 2023-11-16T16:40:06.356 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.351+0000 7f8e8a452700 1 --2- 172.21.15.1:0/39121238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da54b89b60 0x55da54b933b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8e8000eee0 tx=0x7f8e80013a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b87b772801acb3fc server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:06.356 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.351+0000 7f8e6dffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-11-16T16:40:06.356 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.351+0000 7f8e6dffb700 1 -- 172.21.15.1:0/39121238 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e80032410 con 0x55da54b89b60 2023-11-16T16:40:06.356 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.351+0000 7f8e98842b80 1 -- 172.21.15.1:0/39121238 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55da54b7cb20 con 0x55da54b89b60 2023-11-16T16:40:06.356 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.351+0000 7f8e6dffb700 1 -- 172.21.15.1:0/39121238 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f8e80032570 con 0x55da54b89b60 2023-11-16T16:40:06.356 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.351+0000 7f8e6dffb700 1 -- 172.21.15.1:0/39121238 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e80032730 con 0x55da54b89b60 2023-11-16T16:40:06.356 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.352+0000 7f8e98842b80 10 client.25316 fetch_fsmap learned FSMap version 23 2023-11-16T16:40:06.356 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.352+0000 7f8e98842b80 1 -- 172.21.15.1:0/39121238 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55da54aa1190 con 0x55da54b89b60 2023-11-16T16:40:06.356 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.352+0000 7f8e6dffb700 1 -- 172.21.15.1:0/39121238 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55da54b7cb20 con 0x55da54b89b60 2023-11-16T16:40:06.356 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.352+0000 7f8e6dffb700 1 -- 172.21.15.1:0/39121238 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8e80039070 con 0x55da54b89b60 2023-11-16T16:40:06.356 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.352+0000 7f8e98842b80 10 client.25316 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:40:06.356 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.352+0000 7f8e98842b80 10 client.25316 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:40:06.357 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.352+0000 7f8e98842b80 20 client.25316 populate_metadata read hostname 'smithi001' 2023-11-16T16:40:06.357 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.352+0000 7f8e98842b80 1 --2- 172.21.15.1:0/39121238 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55da54bac0d0 0x55da54bae4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:06.357 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.352+0000 7f8e98842b80 4 client.25316 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:40:06.357 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.352+0000 7f8e98842b80 1 -- 172.21.15.1:0/39121238 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55da54a9d0a0 con 0x55da54bac0d0 2023-11-16T16:40:06.357 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.352+0000 7f8e89c51700 1 --2- 172.21.15.1:0/39121238 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55da54bac0d0 0x55da54bae4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:06.357 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.353+0000 7f8e6dffb700 10 client.25316 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:40:06.357 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.353+0000 7f8e89c51700 1 --2- 172.21.15.1:0/39121238 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55da54bac0d0 0x55da54bae4e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=459ddb8a66fa61e5 server_cookie=59eaaa7893125329 in_seq=0 out_seq=0 2023-11-16T16:40:06.358 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.353+0000 7f8e6dffb700 1 -- 172.21.15.1:0/39121238 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55da54a9d0a0 con 0x55da54bac0d0 2023-11-16T16:40:06.358 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.354+0000 7f8e6dffb700 10 client.25316 handle_command_reply: tid=0 2023-11-16T16:40:06.358 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.354+0000 7f8e98842b80 1 client.25316 shutdown 2023-11-16T16:40:06.359 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.354+0000 7f8e98842b80 1 -- 172.21.15.1:0/39121238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da54b89b60 msgr2=0x55da54b933b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:06.359 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.354+0000 7f8e98842b80 1 --2- 172.21.15.1:0/39121238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da54b89b60 0x55da54b933b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8e8000eee0 tx=0x7f8e80013a60 comp rx=0 tx=0).stop 2023-11-16T16:40:06.359 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.354+0000 7f8e98842b80 1 -- 172.21.15.1:0/39121238 shutdown_connections 2023-11-16T16:40:06.359 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.354+0000 7f8e98842b80 1 --2- 172.21.15.1:0/39121238 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55da54bac0d0 0x55da54bae4e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.359 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.354+0000 7f8e98842b80 1 --2- 172.21.15.1:0/39121238 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da54b8e8b0 0x55da54b93e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.359 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.354+0000 7f8e98842b80 1 --2- 172.21.15.1:0/39121238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da54b89b60 0x55da54b933b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.359 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.354+0000 7f8e6dffb700 0 client.25316 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:40:06.359 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.354+0000 7f8e98842b80 1 --2- 172.21.15.1:0/39121238 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da54b8a470 0x55da54b938f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.359 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.354+0000 7f8e98842b80 1 -- 172.21.15.1:0/39121238 >> 172.21.15.1:0/39121238 conn(0x55da54b7f360 msgr2=0x55da54b83de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:06.359 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.354+0000 7f8e98842b80 1 -- 172.21.15.1:0/39121238 shutdown_connections 2023-11-16T16:40:06.359 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.354+0000 7f8e98842b80 1 -- 172.21.15.1:0/39121238 wait complete. 2023-11-16T16:40:06.360 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.354+0000 7f8e98842b80 20 client.25316 trim_cache size 0 max 16384 2023-11-16T16:40:06.388 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.387+0000 7f8e98842b80 1 Processor -- start 2023-11-16T16:40:06.388 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.387+0000 7f8e98842b80 1 -- start start 2023-11-16T16:40:06.388 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.388+0000 7f8e98842b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da54b89b60 0x55da54b8d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:06.389 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.388+0000 7f8e98842b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da54b8a470 0x55da54b9a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:06.389 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.388+0000 7f8e98842b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da54b8e8b0 0x55da54b93c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:06.389 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.388+0000 7f8e98842b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da54baeb40 con 0x55da54b8e8b0 2023-11-16T16:40:06.389 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.388+0000 7f8e98842b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da54b85730 con 0x55da54b8a470 2023-11-16T16:40:06.389 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.388+0000 7f8e98842b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da54b8a900 con 0x55da54b89b60 2023-11-16T16:40:06.390 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.388+0000 7f8e8a452700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da54b89b60 0x55da54b8d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:06.390 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.388+0000 7f8e89c51700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da54b8a470 0x55da54b9a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:06.390 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.388+0000 7f8e8ac53700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da54b8e8b0 0x55da54b93c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:06.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.388+0000 7f8e8a452700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da54b89b60 0x55da54b8d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:40918/0 (socket says 172.21.15.1:40918) 2023-11-16T16:40:06.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.388+0000 7f8e8a452700 1 -- 172.21.15.1:0/284429771 learned_addr learned my addr 172.21.15.1:0/284429771 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:06.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.388+0000 7f8e8a452700 1 -- 172.21.15.1:0/284429771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da54b8a470 msgr2=0x55da54b9a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:06.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.388+0000 7f8e8a452700 1 --2- 172.21.15.1:0/284429771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da54b8a470 0x55da54b9a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.388+0000 7f8e8a452700 1 -- 172.21.15.1:0/284429771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da54b8e8b0 msgr2=0x55da54b93c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:06.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.388+0000 7f8e8a452700 1 --2- 172.21.15.1:0/284429771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da54b8e8b0 0x55da54b93c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.388+0000 7f8e8a452700 1 -- 172.21.15.1:0/284429771 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da54b80b00 con 0x55da54b89b60 2023-11-16T16:40:06.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.389+0000 7f8e89c51700 1 --2- 172.21.15.1:0/284429771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da54b8a470 0x55da54b9a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:40:06.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.389+0000 7f8e8ac53700 1 --2- 172.21.15.1:0/284429771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da54b8e8b0 0x55da54b93c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:40:06.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.389+0000 7f8e8a452700 1 --2- 172.21.15.1:0/284429771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da54b89b60 0x55da54b8d270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8e800048a0 tx=0x7f8e8001fa10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=475d758b742baca0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:06.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.389+0000 7f8e797fa700 1 -- 172.21.15.1:0/284429771 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8001f5f0 con 0x55da54b89b60 2023-11-16T16:40:06.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.389+0000 7f8e797fa700 1 -- 172.21.15.1:0/284429771 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e8001fb60 con 0x55da54b89b60 2023-11-16T16:40:06.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.389+0000 7f8e797fa700 1 -- 172.21.15.1:0/284429771 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e80016750 con 0x55da54b89b60 2023-11-16T16:40:06.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.390+0000 7f8e98842b80 1 -- 172.21.15.1:0/284429771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da54b89b60 msgr2=0x55da54b8d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:06.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.390+0000 7f8e98842b80 1 --2- 172.21.15.1:0/284429771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da54b89b60 0x55da54b8d270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8e800048a0 tx=0x7f8e8001fa10 comp rx=0 tx=0).stop 2023-11-16T16:40:06.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.390+0000 7f8e98842b80 1 -- 172.21.15.1:0/284429771 shutdown_connections 2023-11-16T16:40:06.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.390+0000 7f8e98842b80 1 --2- 172.21.15.1:0/284429771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da54b8e8b0 0x55da54b93c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.390+0000 7f8e98842b80 1 --2- 172.21.15.1:0/284429771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da54b89b60 0x55da54b8d270 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.390+0000 7f8e98842b80 1 --2- 172.21.15.1:0/284429771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da54b8a470 0x55da54b9a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.390+0000 7f8e98842b80 1 -- 172.21.15.1:0/284429771 >> 172.21.15.1:0/284429771 conn(0x55da54b7f360 msgr2=0x55da54b8ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:06.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.390+0000 7f8e98842b80 1 -- 172.21.15.1:0/284429771 shutdown_connections 2023-11-16T16:40:06.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.390+0000 7f8e98842b80 1 -- 172.21.15.1:0/284429771 wait complete. 2023-11-16T16:40:06.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.390+0000 7f8e98842b80 1 -- start start 2023-11-16T16:40:06.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.391+0000 7f8e98842b80 1 Processor -- start 2023-11-16T16:40:06.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.391+0000 7f8e98842b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da54b89b60 0x55da54b96980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:06.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.391+0000 7f8e98842b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da54b8a470 0x55da54b96ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:06.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.391+0000 7f8e98842b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da54b8e8b0 0x55da54bfef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:06.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.391+0000 7f8e98842b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da54b977b0 con 0x55da54b89b60 2023-11-16T16:40:06.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.391+0000 7f8e98842b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da54bff480 con 0x55da54b8a470 2023-11-16T16:40:06.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.391+0000 7f8e98842b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da54bff5c0 con 0x55da54b8e8b0 2023-11-16T16:40:06.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.391+0000 7f8e98842b80 10 client.0 _ll_register_callbacks cb 0x7f8e8000ede0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:40:06.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.391+0000 7f8e8a452700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da54b89b60 0x55da54b96980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:06.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.391+0000 7f8e8a452700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da54b89b60 0x55da54b96980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44242/0 (socket says 172.21.15.1:44242) 2023-11-16T16:40:06.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.391+0000 7f8e8a452700 1 -- 172.21.15.1:0/1719444396 learned_addr learned my addr 172.21.15.1:0/1719444396 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:06.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.391+0000 7f8e8ac53700 1 --2- 172.21.15.1:0/1719444396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da54b8e8b0 0x55da54bfef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:06.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.392+0000 7f8e8a452700 1 -- 172.21.15.1:0/1719444396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da54b8e8b0 msgr2=0x55da54bfef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:06.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.392+0000 7f8e8a452700 1 --2- 172.21.15.1:0/1719444396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da54b8e8b0 0x55da54bfef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.392+0000 7f8e8a452700 1 -- 172.21.15.1:0/1719444396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da54b8a470 msgr2=0x55da54b96ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:06.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.392+0000 7f8e8a452700 1 --2- 172.21.15.1:0/1719444396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da54b8a470 0x55da54b96ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.392+0000 7f8e8a452700 1 -- 172.21.15.1:0/1719444396 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e8410f0a0 con 0x55da54b89b60 2023-11-16T16:40:06.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.392+0000 7f8e8a452700 1 --2- 172.21.15.1:0/1719444396 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da54b89b60 0x55da54b96980 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8e80000c00 tx=0x7f8e8001f6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=220f999e541ee7e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:06.397 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.392+0000 7f8e6dffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:40:06.397 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.392+0000 7f8e6dffb700 1 -- 172.21.15.1:0/1719444396 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e80016740 con 0x55da54b89b60 2023-11-16T16:40:06.397 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.392+0000 7f8e6dffb700 1 -- 172.21.15.1:0/1719444396 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f8e80015910 con 0x55da54b89b60 2023-11-16T16:40:06.397 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.392+0000 7f8e98842b80 1 -- 172.21.15.1:0/1719444396 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e80013450 con 0x55da54b89b60 2023-11-16T16:40:06.397 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.392+0000 7f8e6dffb700 1 -- 172.21.15.1:0/1719444396 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e800168a0 con 0x55da54b89b60 2023-11-16T16:40:06.397 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.392+0000 7f8e98842b80 10 client.15732 fetch_fsmap learned FSMap version 23 2023-11-16T16:40:06.397 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.392+0000 7f8e98842b80 1 -- 172.21.15.1:0/1719444396 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55da54aa1190 con 0x55da54b89b60 2023-11-16T16:40:06.397 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.393+0000 7f8e6dffb700 1 -- 172.21.15.1:0/1719444396 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e80013450 con 0x55da54b89b60 2023-11-16T16:40:06.397 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.393+0000 7f8e6dffb700 1 -- 172.21.15.1:0/1719444396 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8e80049030 con 0x55da54b89b60 2023-11-16T16:40:06.397 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.393+0000 7f8e98842b80 10 client.15732 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:40:06.397 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.393+0000 7f8e98842b80 10 client.15732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:40:06.397 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.393+0000 7f8e98842b80 20 client.15732 populate_metadata read hostname 'smithi001' 2023-11-16T16:40:06.398 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.393+0000 7f8e98842b80 1 --2- 172.21.15.1:0/1719444396 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55da54c00720 0x55da54c02b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:06.398 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.393+0000 7f8e98842b80 4 client.15732 mds_command: new command op to 24521 tid=0[{"prefix": "scrub status"}] 2023-11-16T16:40:06.398 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.393+0000 7f8e98842b80 1 -- 172.21.15.1:0/1719444396 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55da54aa9660 con 0x55da54c00720 2023-11-16T16:40:06.398 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.393+0000 7f8e89c51700 1 --2- 172.21.15.1:0/1719444396 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55da54c00720 0x55da54c02b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:06.399 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.394+0000 7f8e6dffb700 10 client.15732 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:40:06.399 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.394+0000 7f8e89c51700 1 --2- 172.21.15.1:0/1719444396 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55da54c00720 0x55da54c02b00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54ceffc7f0baa567 server_cookie=33f7569d16d218fc in_seq=0 out_seq=0 2023-11-16T16:40:06.399 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.394+0000 7f8e6dffb700 1 -- 172.21.15.1:0/1719444396 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55da54aa9660 con 0x55da54c00720 2023-11-16T16:40:06.399 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.394+0000 7f8e6dffb700 10 client.15732 handle_command_reply: tid=0 2023-11-16T16:40:06.400 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.394+0000 7f8e98842b80 1 client.15732 shutdown 2023-11-16T16:40:06.400 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.394+0000 7f8e98842b80 1 -- 172.21.15.1:0/1719444396 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da54b89b60 msgr2=0x55da54b96980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:06.400 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.394+0000 7f8e98842b80 1 --2- 172.21.15.1:0/1719444396 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da54b89b60 0x55da54b96980 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8e80000c00 tx=0x7f8e8001f6d0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.400 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.394+0000 7f8e98842b80 1 -- 172.21.15.1:0/1719444396 shutdown_connections 2023-11-16T16:40:06.400 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.394+0000 7f8e98842b80 1 --2- 172.21.15.1:0/1719444396 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55da54c00720 0x55da54c02b00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.400 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.394+0000 7f8e98842b80 1 --2- 172.21.15.1:0/1719444396 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55da54b8e8b0 0x55da54bfef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.400 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.394+0000 7f8e98842b80 1 --2- 172.21.15.1:0/1719444396 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55da54b89b60 0x55da54b96980 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.400 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.394+0000 7f8e98842b80 1 --2- 172.21.15.1:0/1719444396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55da54b8a470 0x55da54b96ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.400 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.394+0000 7f8e98842b80 1 -- 172.21.15.1:0/1719444396 >> 172.21.15.1:0/1719444396 conn(0x55da54b7f360 msgr2=0x55da54b88960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:06.400 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.394+0000 7f8e6dffb700 0 client.15732 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:40:06.400 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.394+0000 7f8e98842b80 1 -- 172.21.15.1:0/1719444396 shutdown_connections 2023-11-16T16:40:06.400 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.395+0000 7f8e98842b80 1 -- 172.21.15.1:0/1719444396 wait complete. 2023-11-16T16:40:06.401 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.395+0000 7f8e98842b80 20 client.15732 trim_cache size 0 max 16384 2023-11-16T16:40:06.401 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:40:06.401 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-16T16:40:06.401 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2023-11-16T16:40:06.401 INFO:teuthology.orchestra.run.smithi001.stdout: "b06a01b7-cea1-4aaa-820f-a0b39514f5f5": { 2023-11-16T16:40:06.401 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2023-11-16T16:40:06.401 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "b06a01b7-cea1-4aaa-820f-a0b39514f5f5", 2023-11-16T16:40:06.401 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2023-11-16T16:40:06.401 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:40:06.402 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:40:06.402 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:40:06.402 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.395+0000 7f8e8c6b6700 1 -- 172.21.15.1:0/195100303 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f8e700a13e0 msgr2=0x7f8e700a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:06.403 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.395+0000 7f8e8c6b6700 1 --2- 172.21.15.1:0/195100303 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f8e700a13e0 0x7f8e700a3890 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8e80009ea0 tx=0x7f8e80009290 comp rx=0 tx=0).stop 2023-11-16T16:40:06.403 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.395+0000 7f8e8c6b6700 1 -- 172.21.15.1:0/195100303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e84104140 msgr2=0x7f8e841a23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:06.403 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.395+0000 7f8e8c6b6700 1 --2- 172.21.15.1:0/195100303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e84104140 0x7f8e841a23e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c00b770 tx=0x7f8e7c007870 comp rx=0 tx=0).stop 2023-11-16T16:40:06.403 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.395+0000 7f8e8c6b6700 1 -- 172.21.15.1:0/195100303 shutdown_connections 2023-11-16T16:40:06.403 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.395+0000 7f8e8c6b6700 1 --2- 172.21.15.1:0/195100303 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f8e700a13e0 0x7f8e700a3890 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.403 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.395+0000 7f8e8c6b6700 1 --2- 172.21.15.1:0/195100303 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8e84104140 0x7f8e841a23e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.403 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.395+0000 7f8e8c6b6700 1 --2- 172.21.15.1:0/195100303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e840ff8f0 0x7f8e84197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.403 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.395+0000 7f8e8c6b6700 1 --2- 172.21.15.1:0/195100303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e841002a0 0x7f8e841978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:06.403 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.395+0000 7f8e8c6b6700 1 -- 172.21.15.1:0/195100303 >> 172.21.15.1:0/195100303 conn(0x7f8e84075880 msgr2=0x7f8e840fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:06.403 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.395+0000 7f8e8c6b6700 1 -- 172.21.15.1:0/195100303 shutdown_connections 2023-11-16T16:40:06.403 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:06.395+0000 7f8e8c6b6700 1 -- 172.21.15.1:0/195100303 wait complete. 2023-11-16T16:40:06.413 INFO:tasks.cephfs.filesystem:scrub status for tag:b06a01b7-cea1-4aaa-820f-a0b39514f5f5 - {'path': '/', 'tag': 'b06a01b7-cea1-4aaa-820f-a0b39514f5f5', 'options': 'recursive,force'} 2023-11-16T16:40:06.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:06 smithi001 ceph-mon[98800]: scrub queued for path: / 2023-11-16T16:40:06.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:06 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:40:06.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:06 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:40:06.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:06 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:40:06.510 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:06 smithi026 ceph-mon[100112]: scrub queued for path: / 2023-11-16T16:40:06.510 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:06 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:40:06.510 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:06 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:40:06.510 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:06 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:40:06.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:06 smithi053 ceph-mon[100638]: scrub queued for path: / 2023-11-16T16:40:06.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:06 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:40:06.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:06 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:40:06.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:06 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:40:07.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:07 smithi026 ceph-mon[100112]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 20 op/s 2023-11-16T16:40:07.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:07 smithi053 ceph-mon[100638]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 20 op/s 2023-11-16T16:40:07.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:07 smithi001 ceph-mon[98800]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 20 op/s 2023-11-16T16:40:09.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:09 smithi026 ceph-mon[100112]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 18 op/s 2023-11-16T16:40:09.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:09 smithi026 ceph-mon[100112]: scrub summary: idle 2023-11-16T16:40:09.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:09 smithi053 ceph-mon[100638]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 18 op/s 2023-11-16T16:40:09.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:09 smithi053 ceph-mon[100638]: scrub summary: idle 2023-11-16T16:40:09.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:09 smithi001 ceph-mon[98800]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 18 op/s 2023-11-16T16:40:09.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:09 smithi001 ceph-mon[98800]: scrub summary: idle 2023-11-16T16:40:11.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:11 smithi026 ceph-mon[100112]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-11-16T16:40:11.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:11 smithi053 ceph-mon[100638]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-11-16T16:40:11.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:11 smithi001 ceph-mon[98800]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-11-16T16:40:13.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:13 smithi026 ceph-mon[100112]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2023-11-16T16:40:13.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:13 smithi053 ceph-mon[100638]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2023-11-16T16:40:13.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:13 smithi001 ceph-mon[98800]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2023-11-16T16:40:15.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:15 smithi053 ceph-mon[100638]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-11-16T16:40:15.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:15 smithi001 ceph-mon[98800]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-11-16T16:40:15.757 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:15 smithi026 ceph-mon[100112]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-11-16T16:40:16.135 INFO:tasks.workunit.client.0.smithi001.stderr:Updating files: 93% (11266/12074) Updating files: 94% (11350/12074) Updating files: 95% (11471/12074) Updating files: 96% (11592/12074) Updating files: 97% (11712/12074) Updating files: 98% (11833/12074) Updating files: 99% (11954/12074) Updating files: 100% (12074/12074) Updating files: 100% (12074/12074), done. 2023-11-16T16:40:16.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:16 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:40:16.630 INFO:tasks.workunit.client.0.smithi001.stderr:Note: switching to '5e4a743ad7fd1422ebd6c4dc950a21a303279207'. 2023-11-16T16:40:16.630 INFO:tasks.workunit.client.0.smithi001.stderr: 2023-11-16T16:40:16.630 INFO:tasks.workunit.client.0.smithi001.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-11-16T16:40:16.630 INFO:tasks.workunit.client.0.smithi001.stderr:changes and commit them, and you can discard any commits you make in this 2023-11-16T16:40:16.630 INFO:tasks.workunit.client.0.smithi001.stderr:state without impacting any branches by switching back to a branch. 2023-11-16T16:40:16.630 INFO:tasks.workunit.client.0.smithi001.stderr: 2023-11-16T16:40:16.631 INFO:tasks.workunit.client.0.smithi001.stderr:If you want to create a new branch to retain commits you create, you may 2023-11-16T16:40:16.631 INFO:tasks.workunit.client.0.smithi001.stderr:do so (now or later) by using -c with the switch command. Example: 2023-11-16T16:40:16.631 INFO:tasks.workunit.client.0.smithi001.stderr: 2023-11-16T16:40:16.631 INFO:tasks.workunit.client.0.smithi001.stderr: git switch -c 2023-11-16T16:40:16.631 INFO:tasks.workunit.client.0.smithi001.stderr: 2023-11-16T16:40:16.631 INFO:tasks.workunit.client.0.smithi001.stderr:Or undo this operation with: 2023-11-16T16:40:16.631 INFO:tasks.workunit.client.0.smithi001.stderr: 2023-11-16T16:40:16.631 INFO:tasks.workunit.client.0.smithi001.stderr: git switch - 2023-11-16T16:40:16.631 INFO:tasks.workunit.client.0.smithi001.stderr: 2023-11-16T16:40:16.631 INFO:tasks.workunit.client.0.smithi001.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-11-16T16:40:16.631 INFO:tasks.workunit.client.0.smithi001.stderr: 2023-11-16T16:40:16.632 INFO:tasks.workunit.client.0.smithi001.stderr:HEAD is now at 5e4a743ad7f doc/cephfs: Update docs to match remove functionality and respective vxattrs 2023-11-16T16:40:16.636 DEBUG:teuthology.orchestra.run.smithi001:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-11-16T16:40:16.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:16 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:40:16.706 INFO:tasks.workunit.client.0.smithi001.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-11-16T16:40:16.708 INFO:tasks.workunit.client.0.smithi001.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-11-16T16:40:16.708 INFO:tasks.workunit.client.0.smithi001.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-11-16T16:40:16.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:16 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:40:17.241 INFO:tasks.workunit.client.0.smithi001.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-11-16T16:40:17.295 INFO:tasks.workunit.client.0.smithi001.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-11-16T16:40:17.334 INFO:tasks.workunit.client.0.smithi001.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-11-16T16:40:17.336 INFO:tasks.workunit.client.0.smithi001.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-11-16T16:40:17.336 INFO:tasks.workunit.client.0.smithi001.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-11-16T16:40:17.373 INFO:tasks.workunit.client.0.smithi001.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-11-16T16:40:17.377 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2023-11-16T16:40:17.377 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-11-16T16:40:17.435 INFO:tasks.workunit:Running workunits matching fs/misc on client.0... 2023-11-16T16:40:17.437 INFO:tasks.workunit:Running workunit fs/misc/acl.sh... 2023-11-16T16:40:17.437 DEBUG:teuthology.orchestra.run.smithi001:workunit test fs/misc/acl.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/acl.sh 2023-11-16T16:40:17.499 INFO:tasks.workunit.client.0.smithi001.stderr:+ set -e 2023-11-16T16:40:17.499 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir -p testdir 2023-11-16T16:40:17.502 INFO:tasks.workunit.client.0.smithi001.stderr:+ cd testdir 2023-11-16T16:40:17.502 INFO:tasks.workunit.client.0.smithi001.stderr:+ set +e 2023-11-16T16:40:17.503 INFO:tasks.workunit.client.0.smithi001.stderr:+ setfacl -d -m u:nobody:rw . 2023-11-16T16:40:17.509 INFO:tasks.workunit.client.0.smithi001.stderr:+ test 0 '!=' 0 2023-11-16T16:40:17.509 INFO:tasks.workunit.client.0.smithi001.stderr:+ set -e 2023-11-16T16:40:17.510 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=0 2023-11-16T16:40:17.510 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 0 -lt 100 ']' 2023-11-16T16:40:17.510 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 0 + 1 2023-11-16T16:40:17.510 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=1 2023-11-16T16:40:17.510 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:17.512 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:17.513 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:17.517 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:17.517 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:17.517 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:17.525 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:17 smithi001 ceph-mon[98800]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2023-11-16T16:40:17.526 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:17 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:40:17.526 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:17 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:40:17.526 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:17 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:40:17.526 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:17 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:40:17.526 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:17 smithi001 ceph-mon[98800]: Service mds.all has no related daemon already placed. 2023-11-16T16:40:17.526 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:17 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:40:17.526 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:17 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:40:17.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:17 smithi053 ceph-mon[100638]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2023-11-16T16:40:17.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:17 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:40:17.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:17 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:40:17.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:17 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:40:17.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:17 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:40:17.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:17 smithi053 ceph-mon[100638]: Service mds.all has no related daemon already placed. 2023-11-16T16:40:17.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:17 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:40:17.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:17 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:40:17.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:17 smithi026 ceph-mon[100112]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2023-11-16T16:40:17.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:17 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:40:17.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:17 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:40:17.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:17 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:40:17.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:17 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:40:17.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:17 smithi026 ceph-mon[100112]: Service mds.all has no related daemon already placed. 2023-11-16T16:40:17.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:17 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:40:17.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:17 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:40:18.212 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:18.213 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:18.359 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:18.359 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:18.369 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:18.369 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:18.369 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 1 -lt 100 ']' 2023-11-16T16:40:18.370 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 1 + 1 2023-11-16T16:40:18.405 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=2 2023-11-16T16:40:18.405 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:18.436 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:18.436 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:18.437 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:18.437 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:18.437 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:18.838 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:18.838 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:18.968 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:18.968 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:18.979 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:18.979 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:18.979 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -lt 100 ']' 2023-11-16T16:40:18.979 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 2 + 1 2023-11-16T16:40:19.014 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=3 2023-11-16T16:40:19.014 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:19.039 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:19.040 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:19.040 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:19.040 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:19.041 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:19.413 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:19.413 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:19.550 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:19 smithi001 ceph-mon[98800]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2023-11-16T16:40:19.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:19 smithi053 ceph-mon[100638]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2023-11-16T16:40:19.592 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:19.593 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:19.619 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:19.620 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:19.620 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 3 -lt 100 ']' 2023-11-16T16:40:19.620 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 3 + 1 2023-11-16T16:40:19.655 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=4 2023-11-16T16:40:19.655 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:19.673 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:19.673 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:19.674 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:19.674 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:19.674 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:19.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:19 smithi026 ceph-mon[100112]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2023-11-16T16:40:20.104 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:20.104 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:20.201 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:20.201 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:20.213 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:20.213 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:20.213 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 4 -lt 100 ']' 2023-11-16T16:40:20.213 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 4 + 1 2023-11-16T16:40:20.247 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=5 2023-11-16T16:40:20.247 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:20.264 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:20.264 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:20.265 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:20.265 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:20.265 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:20.612 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:20.613 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:20.716 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:20.716 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:20.727 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:20.728 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:20.728 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 5 -lt 100 ']' 2023-11-16T16:40:20.728 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 5 + 1 2023-11-16T16:40:20.766 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=6 2023-11-16T16:40:20.766 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:20.788 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:20.788 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:20.790 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:20.790 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:20.790 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:21.137 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:21.137 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:21.232 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:21.232 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:21.246 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:21.246 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:21.246 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 6 -lt 100 ']' 2023-11-16T16:40:21.246 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 6 + 1 2023-11-16T16:40:21.280 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=7 2023-11-16T16:40:21.280 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:21.380 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:21.380 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:21.381 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:21.381 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:21.381 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:21.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:21 smithi001 ceph-mon[98800]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2023-11-16T16:40:21.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:21 smithi053 ceph-mon[100638]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2023-11-16T16:40:21.754 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:21.754 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:21.757 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:21 smithi026 ceph-mon[100112]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2023-11-16T16:40:21.849 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:21.849 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:21.861 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:21.861 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:21.861 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 7 -lt 100 ']' 2023-11-16T16:40:21.862 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 7 + 1 2023-11-16T16:40:21.901 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=8 2023-11-16T16:40:21.901 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:21.980 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:21.980 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:21.981 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:21.981 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:21.982 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:22.337 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:22.337 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:22.450 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:22.450 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:22.462 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:22.462 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:22.462 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 8 -lt 100 ']' 2023-11-16T16:40:22.463 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 8 + 1 2023-11-16T16:40:22.496 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=9 2023-11-16T16:40:22.497 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:22.521 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:22.521 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:22.522 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:22.522 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:22.522 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:22.912 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:22.912 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:23.025 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:23.026 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:23.035 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:23.035 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:23.036 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 9 -lt 100 ']' 2023-11-16T16:40:23.036 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 9 + 1 2023-11-16T16:40:23.071 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=10 2023-11-16T16:40:23.071 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:23.096 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:23.096 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:23.097 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:23.097 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:23.098 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:23.520 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:23.520 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:23.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:23 smithi053 ceph-mon[100638]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2023-11-16T16:40:23.641 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:23.641 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:23.654 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:23.654 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:23.654 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 10 -lt 100 ']' 2023-11-16T16:40:23.654 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 10 + 1 2023-11-16T16:40:23.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:23 smithi001 ceph-mon[98800]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2023-11-16T16:40:23.688 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=11 2023-11-16T16:40:23.688 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:23.712 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:23.713 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:23.713 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:23.713 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:23.714 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:23.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:23 smithi026 ceph-mon[100112]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2023-11-16T16:40:24.120 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:24.120 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:24.240 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:24.240 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:24.252 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:24.252 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:24.252 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 11 -lt 100 ']' 2023-11-16T16:40:24.252 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 11 + 1 2023-11-16T16:40:24.288 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=12 2023-11-16T16:40:24.288 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:24.313 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:24.313 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:24.314 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:24.314 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:24.314 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:24.753 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:24.754 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:24.898 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:24.898 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:24.911 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:24.911 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:24.911 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 12 -lt 100 ']' 2023-11-16T16:40:24.911 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 12 + 1 2023-11-16T16:40:24.946 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=13 2023-11-16T16:40:24.946 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:24.964 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:24.964 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:24.965 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:24.965 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:24.965 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:25.345 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:25.345 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:25.440 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:25.441 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:25.453 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:25.453 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:25.453 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 13 -lt 100 ']' 2023-11-16T16:40:25.453 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 13 + 1 2023-11-16T16:40:25.493 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=14 2023-11-16T16:40:25.493 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:25.521 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:25.521 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:25.522 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:25.522 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:25.522 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:25.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:25 smithi053 ceph-mon[100638]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.6 KiB/s wr, 5 op/s 2023-11-16T16:40:25.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:25 smithi001 ceph-mon[98800]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.6 KiB/s wr, 5 op/s 2023-11-16T16:40:25.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:25 smithi026 ceph-mon[100112]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.6 KiB/s wr, 5 op/s 2023-11-16T16:40:25.928 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:25.928 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:26.040 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:26.040 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:26.054 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:26.055 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:26.055 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 14 -lt 100 ']' 2023-11-16T16:40:26.055 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 14 + 1 2023-11-16T16:40:26.087 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=15 2023-11-16T16:40:26.087 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:26.106 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:26.106 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:26.107 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:26.107 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:26.107 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:26.520 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:26.520 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:26.648 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:26.648 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:26.661 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:26.661 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:26.662 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 15 -lt 100 ']' 2023-11-16T16:40:26.662 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 15 + 1 2023-11-16T16:40:26.696 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=16 2023-11-16T16:40:26.696 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:26.721 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:26.721 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:26.722 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:26.722 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:26.722 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:27.111 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:27.112 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:27.233 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:27.233 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:27.245 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:27.246 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:27.246 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 16 -lt 100 ']' 2023-11-16T16:40:27.246 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 16 + 1 2023-11-16T16:40:27.288 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=17 2023-11-16T16:40:27.288 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:27.305 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:27.305 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:27.306 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:27.306 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:27.307 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:27.557 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:27 smithi001 ceph-mon[98800]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 9 op/s 2023-11-16T16:40:27.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:27 smithi053 ceph-mon[100638]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 9 op/s 2023-11-16T16:40:27.661 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:27.662 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:27.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:27 smithi026 ceph-mon[100112]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 9 op/s 2023-11-16T16:40:27.782 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:27.782 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:27.793 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:27.794 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:27.794 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 17 -lt 100 ']' 2023-11-16T16:40:27.794 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 17 + 1 2023-11-16T16:40:27.829 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=18 2023-11-16T16:40:27.829 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:27.855 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:27.855 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:27.856 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:27.856 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:27.856 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:28.203 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:28.203 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:28.406 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:28.406 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:28.421 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:28.422 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:28.422 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 18 -lt 100 ']' 2023-11-16T16:40:28.422 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 18 + 1 2023-11-16T16:40:28.454 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=19 2023-11-16T16:40:28.454 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:28.473 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:28.473 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:28.474 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:28.474 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:28.474 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:28.828 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:28.828 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:28.932 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:28.932 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:28.944 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:28.944 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:28.944 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 19 -lt 100 ']' 2023-11-16T16:40:28.945 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 19 + 1 2023-11-16T16:40:28.979 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=20 2023-11-16T16:40:28.979 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:28.998 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:28.998 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:28.999 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:28.999 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:28.999 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:29.361 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:29.361 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:29.498 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:29.498 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:29.512 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:29.513 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:29.513 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 20 -lt 100 ']' 2023-11-16T16:40:29.513 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 20 + 1 2023-11-16T16:40:29.545 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=21 2023-11-16T16:40:29.546 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:29.564 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:29.564 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:29.565 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:29.565 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:29.565 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:29.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:29 smithi053 ceph-mon[100638]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 6 op/s 2023-11-16T16:40:29.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:29 smithi001 ceph-mon[98800]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 6 op/s 2023-11-16T16:40:29.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:29 smithi026 ceph-mon[100112]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 6 op/s 2023-11-16T16:40:30.036 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:30.036 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:30.140 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:30.140 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:30.152 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:30.152 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:30.152 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 21 -lt 100 ']' 2023-11-16T16:40:30.152 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 21 + 1 2023-11-16T16:40:30.187 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=22 2023-11-16T16:40:30.187 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:30.212 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:30.212 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:30.213 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:30.213 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:30.213 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:30.661 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:30.661 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:30.766 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:30.766 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:30.779 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:30.779 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:30.779 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 22 -lt 100 ']' 2023-11-16T16:40:30.780 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 22 + 1 2023-11-16T16:40:30.812 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=23 2023-11-16T16:40:30.812 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:30.830 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:30.830 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:30.831 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:30.831 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:30.831 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:31.244 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:31.245 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:31.349 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:31.350 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:31.360 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:31.361 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:31.361 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 23 -lt 100 ']' 2023-11-16T16:40:31.361 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 23 + 1 2023-11-16T16:40:31.395 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=24 2023-11-16T16:40:31.395 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:31.422 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:31.422 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:31.423 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:31.423 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:31.424 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:31.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:31 smithi001 ceph-mon[98800]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 6 op/s 2023-11-16T16:40:31.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:31 smithi053 ceph-mon[100638]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 6 op/s 2023-11-16T16:40:31.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:31 smithi026 ceph-mon[100112]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 6 op/s 2023-11-16T16:40:31.786 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:31.786 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:31.922 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:31.923 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:31.937 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:31.937 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:31.938 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 24 -lt 100 ']' 2023-11-16T16:40:31.938 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 24 + 1 2023-11-16T16:40:31.970 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=25 2023-11-16T16:40:31.970 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:31.988 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:31.988 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:31.989 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:31.989 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:31.989 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:32.386 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:32.386 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:32.549 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:32.550 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:32.560 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:32.560 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:32.561 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 25 -lt 100 ']' 2023-11-16T16:40:32.561 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 25 + 1 2023-11-16T16:40:32.595 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=26 2023-11-16T16:40:32.595 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:32.629 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:32.629 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:32.630 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:32.630 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:32.630 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:33.035 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:33.036 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:33.172 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:33.172 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:33.187 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:33.187 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:33.187 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 26 -lt 100 ']' 2023-11-16T16:40:33.187 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 26 + 1 2023-11-16T16:40:33.220 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=27 2023-11-16T16:40:33.220 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:33.246 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:33.246 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:33.247 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:33.247 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:33.248 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:33.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:33 smithi053 ceph-mon[100638]: pgmap v44: 97 pgs: 97 active+clean; 1.5 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 9 op/s 2023-11-16T16:40:33.606 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:33 smithi001 ceph-mon[98800]: pgmap v44: 97 pgs: 97 active+clean; 1.5 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 9 op/s 2023-11-16T16:40:33.652 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:33.652 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:33.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:33 smithi026 ceph-mon[100112]: pgmap v44: 97 pgs: 97 active+clean; 1.5 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 9 op/s 2023-11-16T16:40:33.774 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:33.774 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:33.785 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:33.785 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:33.785 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 27 -lt 100 ']' 2023-11-16T16:40:33.786 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 27 + 1 2023-11-16T16:40:33.820 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=28 2023-11-16T16:40:33.820 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:33.846 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:33.846 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:33.847 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:33.847 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:33.847 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:34.277 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:34.277 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:34.431 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:34.432 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:34.443 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:34.444 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:34.444 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 28 -lt 100 ']' 2023-11-16T16:40:34.444 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 28 + 1 2023-11-16T16:40:34.453 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=29 2023-11-16T16:40:34.453 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:34.478 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:34.478 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:34.479 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:34.479 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:34.479 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:34.919 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:34.919 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:35.099 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:35.099 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:35.109 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:35.109 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:35.109 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 29 -lt 100 ']' 2023-11-16T16:40:35.110 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 29 + 1 2023-11-16T16:40:35.145 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=30 2023-11-16T16:40:35.145 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:35.163 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:35.163 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:35.164 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:35.164 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:35.165 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:35.535 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:35.535 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:35.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:35 smithi053 ceph-mon[100638]: pgmap v45: 97 pgs: 97 active+clean; 1.5 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 6 op/s 2023-11-16T16:40:35.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:35 smithi001 ceph-mon[98800]: pgmap v45: 97 pgs: 97 active+clean; 1.5 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 6 op/s 2023-11-16T16:40:35.690 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:35.690 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:35.703 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:35.704 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:35.704 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 30 -lt 100 ']' 2023-11-16T16:40:35.704 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 30 + 1 2023-11-16T16:40:35.736 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=31 2023-11-16T16:40:35.736 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:35.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:35 smithi026 ceph-mon[100112]: pgmap v45: 97 pgs: 97 active+clean; 1.5 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 6 op/s 2023-11-16T16:40:35.762 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:35.762 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:35.763 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:35.763 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:35.764 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:36.168 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:36.169 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:36.330 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:36.331 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:36.343 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:36.343 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:36.343 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 31 -lt 100 ']' 2023-11-16T16:40:36.343 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 31 + 1 2023-11-16T16:40:36.378 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=32 2023-11-16T16:40:36.378 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:36.403 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:36.403 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:36.404 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:36.404 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:36.404 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:36.414 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-16T16:40:36.935 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:36.935 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:37.005 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:37.006 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:37.027 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:37.027 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:37.027 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 32 -lt 100 ']' 2023-11-16T16:40:37.027 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 32 + 1 2023-11-16T16:40:37.091 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=33 2023-11-16T16:40:37.091 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:37.145 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:37.145 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:37.146 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:37.146 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:37.146 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:37.518 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:37.518 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:37.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:37 smithi053 ceph-mon[100638]: pgmap v46: 97 pgs: 97 active+clean; 1.6 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 10 op/s 2023-11-16T16:40:37.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:37 smithi001 ceph-mon[98800]: pgmap v46: 97 pgs: 97 active+clean; 1.6 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 10 op/s 2023-11-16T16:40:37.688 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:37.688 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:37.701 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:37.701 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:37.701 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 33 -lt 100 ']' 2023-11-16T16:40:37.701 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 33 + 1 2023-11-16T16:40:37.732 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=34 2023-11-16T16:40:37.732 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:37.757 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:37 smithi026 ceph-mon[100112]: pgmap v46: 97 pgs: 97 active+clean; 1.6 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 10 op/s 2023-11-16T16:40:37.771 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:37.771 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:37.772 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:37.772 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:37.772 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:38.451 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:38.452 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:38.631 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:38.631 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:38.661 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:38.661 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:38.662 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 34 -lt 100 ']' 2023-11-16T16:40:38.662 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 34 + 1 2023-11-16T16:40:38.724 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=35 2023-11-16T16:40:38.724 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:38.746 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:38.746 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:38.747 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:38.748 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:38.748 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:39.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.084+0000 7f130b9b9700 1 -- 172.21.15.1:0/2018799168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f13040ffa70 msgr2=0x7f13040ffee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:39.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.084+0000 7f130b9b9700 1 --2- 172.21.15.1:0/2018799168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f13040ffa70 0x7f13040ffee0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f12f4009990 tx=0x7f12f4009ca0 comp rx=0 tx=0).stop 2023-11-16T16:40:39.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.084+0000 7f130b9b9700 1 -- 172.21.15.1:0/2018799168 shutdown_connections 2023-11-16T16:40:39.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.084+0000 7f130b9b9700 1 --2- 172.21.15.1:0/2018799168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1304100420 0x7f130410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:39.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.084+0000 7f130b9b9700 1 --2- 172.21.15.1:0/2018799168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1304105250 0x7f1304105620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:39.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.084+0000 7f130b9b9700 1 --2- 172.21.15.1:0/2018799168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f13040ffa70 0x7f13040ffee0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:39.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.084+0000 7f130b9b9700 1 -- 172.21.15.1:0/2018799168 >> 172.21.15.1:0/2018799168 conn(0x7f13040763a0 msgr2=0x7f13040767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:39.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.084+0000 7f130b9b9700 1 -- 172.21.15.1:0/2018799168 shutdown_connections 2023-11-16T16:40:39.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.084+0000 7f130b9b9700 1 -- 172.21.15.1:0/2018799168 wait complete. 2023-11-16T16:40:39.088 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.085+0000 7f130b9b9700 1 Processor -- start 2023-11-16T16:40:39.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.085+0000 7f130b9b9700 1 -- start start 2023-11-16T16:40:39.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.085+0000 7f130b9b9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f13040ffa70 0x7f1304196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:39.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.085+0000 7f130b9b9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1304100420 0x7f13041968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:39.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.085+0000 7f130b9b9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1304105250 0x7f130419ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:39.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.085+0000 7f130b9b9700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130419b340 con 0x7f13040ffa70 2023-11-16T16:40:39.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.085+0000 7f130b9b9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130419b4b0 con 0x7f1304100420 2023-11-16T16:40:39.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.085+0000 7f130b9b9700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130419b670 con 0x7f1304105250 2023-11-16T16:40:39.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.086+0000 7f1309755700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f13040ffa70 0x7f1304196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:39.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.086+0000 7f1309755700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f13040ffa70 0x7f1304196360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44258/0 (socket says 172.21.15.1:44258) 2023-11-16T16:40:39.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.086+0000 7f1309755700 1 -- 172.21.15.1:0/965562726 learned_addr learned my addr 172.21.15.1:0/965562726 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:39.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.086+0000 7f1308f54700 1 --2- 172.21.15.1:0/965562726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1304100420 0x7f13041968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:39.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.086+0000 7f1309f56700 1 --2- 172.21.15.1:0/965562726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1304105250 0x7f130419ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:39.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.086+0000 7f1309755700 1 -- 172.21.15.1:0/965562726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1304105250 msgr2=0x7f130419ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:39.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.086+0000 7f1309755700 1 --2- 172.21.15.1:0/965562726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1304105250 0x7f130419ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:39.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.086+0000 7f1309755700 1 -- 172.21.15.1:0/965562726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1304100420 msgr2=0x7f13041968a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:39.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.086+0000 7f1309755700 1 --2- 172.21.15.1:0/965562726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1304100420 0x7f13041968a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:39.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.086+0000 7f1309755700 1 -- 172.21.15.1:0/965562726 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f40095f0 con 0x7f13040ffa70 2023-11-16T16:40:39.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.086+0000 7f1309755700 1 --2- 172.21.15.1:0/965562726 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f13040ffa70 0x7f1304196360 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f130000e9a0 tx=0x7f130000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:39.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.086+0000 7f12fa7fc700 1 -- 172.21.15.1:0/965562726 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1300009790 con 0x7f13040ffa70 2023-11-16T16:40:39.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.087+0000 7f12fa7fc700 1 -- 172.21.15.1:0/965562726 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1300010430 con 0x7f13040ffa70 2023-11-16T16:40:39.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.087+0000 7f130b9b9700 1 -- 172.21.15.1:0/965562726 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1304110370 con 0x7f13040ffa70 2023-11-16T16:40:39.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.087+0000 7f130b9b9700 1 -- 172.21.15.1:0/965562726 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13041101e0 con 0x7f13040ffa70 2023-11-16T16:40:39.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.087+0000 7f12fa7fc700 1 -- 172.21.15.1:0/965562726 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1300018620 con 0x7f13040ffa70 2023-11-16T16:40:39.091 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.089+0000 7f12fa7fc700 1 -- 172.21.15.1:0/965562726 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 24) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f13000105a0 con 0x7f13040ffa70 2023-11-16T16:40:39.092 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.090+0000 7f12fa7fc700 1 --2- 172.21.15.1:0/965562726 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f12f00a13e0 0x7f12f00a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:39.092 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.090+0000 7f12fa7fc700 1 -- 172.21.15.1:0/965562726 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f1300014070 con 0x7f13040ffa70 2023-11-16T16:40:39.092 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.090+0000 7f1308f54700 1 --2- 172.21.15.1:0/965562726 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f12f00a13e0 0x7f12f00a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:39.092 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.091+0000 7f1308f54700 1 --2- 172.21.15.1:0/965562726 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f12f00a13e0 0x7f12f00a3890 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1304197970 tx=0x7f12f4005fd0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:39.188 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.186+0000 7f1317b45b80 1 Processor -- start 2023-11-16T16:40:39.188 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.186+0000 7f1317b45b80 1 -- start start 2023-11-16T16:40:39.188 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.187+0000 7f1317b45b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e737809f70 0x55e73780a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:39.189 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.187+0000 7f1317b45b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e73780a880 0x55e73780e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:39.189 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.187+0000 7f1317b45b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e73780ecc0 0x55e73780f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:39.189 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.187+0000 7f1317b45b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e73770b060 con 0x55e73780a880 2023-11-16T16:40:39.189 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.187+0000 7f1317b45b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e73775d8a0 con 0x55e73780ecc0 2023-11-16T16:40:39.189 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.187+0000 7f1317b45b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e737756d30 con 0x55e737809f70 2023-11-16T16:40:39.190 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.187+0000 7f1309755700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e737809f70 0x55e73780a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:39.190 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.187+0000 7f1308f54700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e73780a880 0x55e73780e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:39.190 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.187+0000 7f1309755700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e737809f70 0x55e73780a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:40948/0 (socket says 172.21.15.1:40948) 2023-11-16T16:40:39.190 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.187+0000 7f1309755700 1 -- 172.21.15.1:0/3182749391 learned_addr learned my addr 172.21.15.1:0/3182749391 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:39.190 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.187+0000 7f1309f56700 1 --2- 172.21.15.1:0/3182749391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e73780ecc0 0x55e73780f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:39.190 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.187+0000 7f1309755700 1 -- 172.21.15.1:0/3182749391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e73780ecc0 msgr2=0x55e73780f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:39.191 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.187+0000 7f1309755700 1 --2- 172.21.15.1:0/3182749391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e73780ecc0 0x55e73780f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:39.191 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.187+0000 7f1309755700 1 -- 172.21.15.1:0/3182749391 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e73780a880 msgr2=0x55e73780e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:39.191 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.187+0000 7f1309755700 1 --2- 172.21.15.1:0/3182749391 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e73780a880 0x55e73780e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:39.191 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.187+0000 7f1309755700 1 -- 172.21.15.1:0/3182749391 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7375d8630 con 0x55e737809f70 2023-11-16T16:40:39.191 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.188+0000 7f1309755700 1 --2- 172.21.15.1:0/3182749391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e737809f70 0x55e73780a340 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1300099be0 tx=0x7f13000998c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf657c1ae1db1608 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:39.191 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.188+0000 7f12ef7fe700 1 -- 172.21.15.1:0/3182749391 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130009df10 con 0x55e737809f70 2023-11-16T16:40:39.191 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.188+0000 7f12ef7fe700 1 -- 172.21.15.1:0/3182749391 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13000a2330 con 0x55e737809f70 2023-11-16T16:40:39.191 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.188+0000 7f12ef7fe700 1 -- 172.21.15.1:0/3182749391 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13000a20e0 con 0x55e737809f70 2023-11-16T16:40:39.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.189+0000 7f1317b45b80 1 -- 172.21.15.1:0/3182749391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e737809f70 msgr2=0x55e73780a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:39.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.189+0000 7f1317b45b80 1 --2- 172.21.15.1:0/3182749391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e737809f70 0x55e73780a340 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1300099be0 tx=0x7f13000998c0 comp rx=0 tx=0).stop 2023-11-16T16:40:39.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.189+0000 7f1317b45b80 1 -- 172.21.15.1:0/3182749391 shutdown_connections 2023-11-16T16:40:39.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.189+0000 7f1317b45b80 1 --2- 172.21.15.1:0/3182749391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e73780ecc0 0x55e73780f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:39.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.189+0000 7f1317b45b80 1 --2- 172.21.15.1:0/3182749391 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e737809f70 0x55e73780a340 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:39.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.189+0000 7f1317b45b80 1 --2- 172.21.15.1:0/3182749391 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e73780a880 0x55e73780e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:39.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.189+0000 7f1317b45b80 1 -- 172.21.15.1:0/3182749391 >> 172.21.15.1:0/3182749391 conn(0x55e7377fb360 msgr2=0x55e737800b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:39.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.189+0000 7f1317b45b80 1 -- 172.21.15.1:0/3182749391 shutdown_connections 2023-11-16T16:40:39.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.189+0000 7f1317b45b80 1 -- 172.21.15.1:0/3182749391 wait complete. 2023-11-16T16:40:39.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.190+0000 7f1317b45b80 1 -- start start 2023-11-16T16:40:39.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.190+0000 7f1317b45b80 1 Processor -- start 2023-11-16T16:40:39.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.190+0000 7f1317b45b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e737809f70 0x55e737816420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:39.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.190+0000 7f1317b45b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e73780a880 0x55e737813710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:39.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.190+0000 7f1317b45b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e73780ecc0 0x55e737813d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:39.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.190+0000 7f1317b45b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7376f88e0 con 0x55e73780a880 2023-11-16T16:40:39.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.190+0000 7f1317b45b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7376f8580 con 0x55e73780ecc0 2023-11-16T16:40:39.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.190+0000 7f1317b45b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7375da920 con 0x55e737809f70 2023-11-16T16:40:39.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.191+0000 7f1317b45b80 10 client.0 _ll_register_callbacks cb 0x55e737716780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:40:39.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.191+0000 7f1308f54700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e73780a880 0x55e737813710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:39.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.191+0000 7f1308f54700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e73780a880 0x55e737813710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44272/0 (socket says 172.21.15.1:44272) 2023-11-16T16:40:39.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.191+0000 7f1308f54700 1 -- 172.21.15.1:0/2115994938 learned_addr learned my addr 172.21.15.1:0/2115994938 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:39.197 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.191+0000 7f1309f56700 1 --2- 172.21.15.1:0/2115994938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e73780ecc0 0x55e737813d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:39.197 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.191+0000 7f1309755700 1 --2- 172.21.15.1:0/2115994938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e737809f70 0x55e737816420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:39.197 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.191+0000 7f1308f54700 1 -- 172.21.15.1:0/2115994938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e737809f70 msgr2=0x55e737816420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:39.197 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.191+0000 7f1308f54700 1 --2- 172.21.15.1:0/2115994938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e737809f70 0x55e737816420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:39.197 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.191+0000 7f1308f54700 1 -- 172.21.15.1:0/2115994938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e73780ecc0 msgr2=0x55e737813d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:39.197 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.191+0000 7f1308f54700 1 --2- 172.21.15.1:0/2115994938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e73780ecc0 0x55e737813d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:39.197 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.191+0000 7f1308f54700 1 -- 172.21.15.1:0/2115994938 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13041101e0 con 0x55e73780a880 2023-11-16T16:40:39.197 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.191+0000 7f1309f56700 1 --2- 172.21.15.1:0/2115994938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e73780ecc0 0x55e737813d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:40:39.197 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.191+0000 7f1308f54700 1 --2- 172.21.15.1:0/2115994938 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e73780a880 0x55e737813710 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f12f4009990 tx=0x7f12f4020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4952677e5cefb0d server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:39.197 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.191+0000 7f12ecff9700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:40:39.197 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.192+0000 7f12ecff9700 1 -- 172.21.15.1:0/2115994938 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f402d040 con 0x55e73780a880 2023-11-16T16:40:39.198 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.192+0000 7f12ecff9700 1 -- 172.21.15.1:0/2115994938 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f12f400f460 con 0x55e73780a880 2023-11-16T16:40:39.198 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.192+0000 7f1317b45b80 1 -- 172.21.15.1:0/2115994938 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e7377f8b20 con 0x55e73780a880 2023-11-16T16:40:39.198 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.192+0000 7f12ecff9700 1 -- 172.21.15.1:0/2115994938 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12f400f620 con 0x55e73780a880 2023-11-16T16:40:39.198 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.192+0000 7f1317b45b80 10 client.15744 fetch_fsmap learned FSMap version 23 2023-11-16T16:40:39.198 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.192+0000 7f1317b45b80 1 -- 172.21.15.1:0/2115994938 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55e73771d190 con 0x55e73780a880 2023-11-16T16:40:39.198 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.192+0000 7f12ecff9700 1 -- 172.21.15.1:0/2115994938 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12f403e080 con 0x55e73780a880 2023-11-16T16:40:39.198 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.192+0000 7f12ecff9700 1 -- 172.21.15.1:0/2115994938 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f12f40327a0 con 0x55e73780a880 2023-11-16T16:40:39.198 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.192+0000 7f1317b45b80 10 client.15744 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:40:39.198 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.192+0000 7f1317b45b80 10 client.15744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:40:39.198 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.192+0000 7f1317b45b80 20 client.15744 populate_metadata read hostname 'smithi001' 2023-11-16T16:40:39.198 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.192+0000 7f1317b45b80 1 --2- 172.21.15.1:0/2115994938 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55e737823ce0 0x55e7378260f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:39.199 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.193+0000 7f1317b45b80 4 client.15744 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:40:39.199 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.193+0000 7f1317b45b80 1 -- 172.21.15.1:0/2115994938 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e7377190a0 con 0x55e737823ce0 2023-11-16T16:40:39.199 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.193+0000 7f1309755700 1 --2- 172.21.15.1:0/2115994938 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55e737823ce0 0x55e7378260f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:39.199 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.193+0000 7f12ecff9700 10 client.15744 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:40:39.199 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.193+0000 7f1309755700 1 --2- 172.21.15.1:0/2115994938 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55e737823ce0 0x55e7378260f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49258763d9225ee3 server_cookie=d10a6ae4c1bc8ff in_seq=0 out_seq=0 2023-11-16T16:40:39.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.196+0000 7f12ecff9700 1 -- 172.21.15.1:0/2115994938 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e7377190a0 con 0x55e737823ce0 2023-11-16T16:40:39.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.196+0000 7f12ecff9700 10 client.15744 handle_command_reply: tid=0 2023-11-16T16:40:39.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.196+0000 7f1317b45b80 1 client.15744 shutdown 2023-11-16T16:40:39.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.197+0000 7f1317b45b80 1 -- 172.21.15.1:0/2115994938 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e73780a880 msgr2=0x55e737813710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:39.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.197+0000 7f1317b45b80 1 --2- 172.21.15.1:0/2115994938 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e73780a880 0x55e737813710 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f12f4009990 tx=0x7f12f4020000 comp rx=0 tx=0).stop 2023-11-16T16:40:39.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.197+0000 7f1317b45b80 1 -- 172.21.15.1:0/2115994938 shutdown_connections 2023-11-16T16:40:39.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.197+0000 7f1317b45b80 1 --2- 172.21.15.1:0/2115994938 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55e737823ce0 0x55e7378260f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:39.201 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.197+0000 7f1317b45b80 1 --2- 172.21.15.1:0/2115994938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e73780ecc0 0x55e737813d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:39.201 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.197+0000 7f1317b45b80 1 --2- 172.21.15.1:0/2115994938 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e737809f70 0x55e737816420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:39.201 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.197+0000 7f1317b45b80 1 --2- 172.21.15.1:0/2115994938 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e73780a880 0x55e737813710 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:39.201 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.197+0000 7f12ecff9700 0 client.15744 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:40:39.201 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.197+0000 7f1317b45b80 1 -- 172.21.15.1:0/2115994938 >> 172.21.15.1:0/2115994938 conn(0x55e7377fb360 msgr2=0x55e7377ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:39.201 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.197+0000 7f1317b45b80 1 -- 172.21.15.1:0/2115994938 shutdown_connections 2023-11-16T16:40:39.201 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.197+0000 7f1317b45b80 1 -- 172.21.15.1:0/2115994938 wait complete. 2023-11-16T16:40:39.201 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.197+0000 7f1317b45b80 20 client.15744 trim_cache size 0 max 16384 2023-11-16T16:40:39.218 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.215+0000 7f1317b45b80 1 Processor -- start 2023-11-16T16:40:39.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.215+0000 7f1317b45b80 1 -- start start 2023-11-16T16:40:39.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.216+0000 7f1317b45b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e737809f70 0x55e73780d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:39.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.216+0000 7f1317b45b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e73780a880 0x55e737812f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:39.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.216+0000 7f1317b45b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e73780ecc0 0x55e737815fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:39.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.216+0000 7f1317b45b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7378021c0 con 0x55e73780a880 2023-11-16T16:40:39.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.216+0000 7f1317b45b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e737801780 con 0x55e737809f70 2023-11-16T16:40:39.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.216+0000 7f1317b45b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e73780afd0 con 0x55e73780ecc0 2023-11-16T16:40:39.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.216+0000 7f1308f54700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e73780a880 0x55e737812f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:39.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.216+0000 7f1308f54700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e73780a880 0x55e737812f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44282/0 (socket says 172.21.15.1:44282) 2023-11-16T16:40:39.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.216+0000 7f1308f54700 1 -- 172.21.15.1:0/670175643 learned_addr learned my addr 172.21.15.1:0/670175643 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:39.220 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.216+0000 7f1309755700 1 --2- 172.21.15.1:0/670175643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e737809f70 0x55e73780d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:39.220 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.216+0000 7f1309f56700 1 --2- 172.21.15.1:0/670175643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e73780ecc0 0x55e737815fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:39.220 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.216+0000 7f1308f54700 1 -- 172.21.15.1:0/670175643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e73780ecc0 msgr2=0x55e737815fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:39.220 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.216+0000 7f1308f54700 1 --2- 172.21.15.1:0/670175643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e73780ecc0 0x55e737815fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:39.220 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.216+0000 7f1308f54700 1 -- 172.21.15.1:0/670175643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e737809f70 msgr2=0x55e73780d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:39.220 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.216+0000 7f1308f54700 1 --2- 172.21.15.1:0/670175643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e737809f70 0x55e73780d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:39.220 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.216+0000 7f1308f54700 1 -- 172.21.15.1:0/670175643 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e73771cee0 con 0x55e73780a880 2023-11-16T16:40:39.220 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.216+0000 7f1309f56700 1 --2- 172.21.15.1:0/670175643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e73780ecc0 0x55e737815fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:40:39.220 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.216+0000 7f1309755700 1 --2- 172.21.15.1:0/670175643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e737809f70 0x55e73780d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:40:39.220 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.216+0000 7f1308f54700 1 --2- 172.21.15.1:0/670175643 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e73780a880 0x55e737812f20 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f12f400b6f0 tx=0x7f12f4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feea0f86f59e69a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:39.220 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.217+0000 7f12effff700 1 -- 172.21.15.1:0/670175643 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f402d070 con 0x55e73780a880 2023-11-16T16:40:39.220 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.217+0000 7f12effff700 1 -- 172.21.15.1:0/670175643 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e7377f8b20 con 0x55e73780a880 2023-11-16T16:40:39.221 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.217+0000 7f12effff700 1 -- 172.21.15.1:0/670175643 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f400fae0 con 0x55e73780a880 2023-11-16T16:40:39.225 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.217+0000 7f1317b45b80 1 -- 172.21.15.1:0/670175643 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e73780a880 msgr2=0x55e737812f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:39.225 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.217+0000 7f1317b45b80 1 --2- 172.21.15.1:0/670175643 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e73780a880 0x55e737812f20 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f12f400b6f0 tx=0x7f12f4009ca0 comp rx=0 tx=0).stop 2023-11-16T16:40:39.225 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.217+0000 7f1317b45b80 1 -- 172.21.15.1:0/670175643 shutdown_connections 2023-11-16T16:40:39.225 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.217+0000 7f1317b45b80 1 --2- 172.21.15.1:0/670175643 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e73780ecc0 0x55e737815fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:39.225 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.217+0000 7f1317b45b80 1 --2- 172.21.15.1:0/670175643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e737809f70 0x55e73780d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:39.225 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.217+0000 7f1317b45b80 1 --2- 172.21.15.1:0/670175643 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e73780a880 0x55e737812f20 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:39.225 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.217+0000 7f1317b45b80 1 -- 172.21.15.1:0/670175643 >> 172.21.15.1:0/670175643 conn(0x55e7377fb360 msgr2=0x55e73780de40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:39.225 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.218+0000 7f1317b45b80 1 -- 172.21.15.1:0/670175643 shutdown_connections 2023-11-16T16:40:39.225 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.218+0000 7f1317b45b80 1 -- 172.21.15.1:0/670175643 wait complete. 2023-11-16T16:40:39.225 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.218+0000 7f1317b45b80 1 -- start start 2023-11-16T16:40:39.225 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.218+0000 7f1317b45b80 1 Processor -- start 2023-11-16T16:40:39.225 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.219+0000 7f1317b45b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e737809f70 0x55e737816d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:39.226 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.219+0000 7f1317b45b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e73780a880 0x55e737817270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:39.226 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.219+0000 7f1317b45b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e73780ecc0 0x55e7378177d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:39.226 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.219+0000 7f1317b45b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e737876d00 con 0x55e73780a880 2023-11-16T16:40:39.226 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.219+0000 7f1317b45b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e737876e70 con 0x55e73780ecc0 2023-11-16T16:40:39.226 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.219+0000 7f1317b45b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e737877030 con 0x55e737809f70 2023-11-16T16:40:39.226 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.219+0000 7f1317b45b80 10 client.0 _ll_register_callbacks cb 0x7f12f4005350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:40:39.226 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.219+0000 7f1308f54700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e73780a880 0x55e737817270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:39.226 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.219+0000 7f1308f54700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e73780a880 0x55e737817270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44288/0 (socket says 172.21.15.1:44288) 2023-11-16T16:40:39.226 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.219+0000 7f1308f54700 1 -- 172.21.15.1:0/2096073064 learned_addr learned my addr 172.21.15.1:0/2096073064 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:39.226 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.219+0000 7f1309755700 1 --2- 172.21.15.1:0/2096073064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e737809f70 0x55e737816d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:39.226 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.219+0000 7f1309f56700 1 --2- 172.21.15.1:0/2096073064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e73780ecc0 0x55e7378177d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:39.226 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.219+0000 7f1308f54700 1 -- 172.21.15.1:0/2096073064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e737809f70 msgr2=0x55e737816d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:39.226 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.219+0000 7f1308f54700 1 --2- 172.21.15.1:0/2096073064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e737809f70 0x55e737816d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:39.227 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.219+0000 7f1308f54700 1 -- 172.21.15.1:0/2096073064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e73780ecc0 msgr2=0x55e7378177d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:39.227 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.219+0000 7f1308f54700 1 --2- 172.21.15.1:0/2096073064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e73780ecc0 0x55e7378177d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:39.227 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.219+0000 7f1308f54700 1 -- 172.21.15.1:0/2096073064 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e73771d190 con 0x55e73780a880 2023-11-16T16:40:39.227 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.219+0000 7f1309f56700 1 --2- 172.21.15.1:0/2096073064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e73780ecc0 0x55e7378177d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:40:39.227 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.220+0000 7f1308f54700 1 --2- 172.21.15.1:0/2096073064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e73780a880 0x55e737817270 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f12f4032d10 tx=0x7f12f4004e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b35a4d570dc9c6e server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:39.227 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.220+0000 7f12ecff9700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:40:39.227 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.220+0000 7f12ecff9700 1 -- 172.21.15.1:0/2096073064 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f402d070 con 0x55e73780a880 2023-11-16T16:40:39.227 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.220+0000 7f12ecff9700 1 -- 172.21.15.1:0/2096073064 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f12f400f460 con 0x55e73780a880 2023-11-16T16:40:39.227 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.220+0000 7f1317b45b80 1 -- 172.21.15.1:0/2096073064 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f13000a2330 con 0x55e73780a880 2023-11-16T16:40:39.227 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.220+0000 7f12ecff9700 1 -- 172.21.15.1:0/2096073064 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12f400f660 con 0x55e73780a880 2023-11-16T16:40:39.227 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.220+0000 7f1317b45b80 10 client.15750 fetch_fsmap learned FSMap version 23 2023-11-16T16:40:39.227 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.220+0000 7f1317b45b80 1 -- 172.21.15.1:0/2096073064 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55e73756dc60 con 0x55e73780a880 2023-11-16T16:40:39.227 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.220+0000 7f12ecff9700 1 -- 172.21.15.1:0/2096073064 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13000a2330 con 0x55e73780a880 2023-11-16T16:40:39.227 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.221+0000 7f12ecff9700 1 -- 172.21.15.1:0/2096073064 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f12f404d070 con 0x55e73780a880 2023-11-16T16:40:39.228 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.221+0000 7f1317b45b80 10 client.15750 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:40:39.228 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.221+0000 7f1317b45b80 10 client.15750 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:40:39.228 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.221+0000 7f1317b45b80 20 client.15750 populate_metadata read hostname 'smithi001' 2023-11-16T16:40:39.228 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.221+0000 7f1317b45b80 1 --2- 172.21.15.1:0/2096073064 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55e737878200 0x55e73787a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:39.228 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.221+0000 7f1317b45b80 4 client.15750 mds_command: new command op to 24521 tid=0[{"prefix": "scrub status"}] 2023-11-16T16:40:39.228 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.221+0000 7f1317b45b80 1 -- 172.21.15.1:0/2096073064 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e737725660 con 0x55e737878200 2023-11-16T16:40:39.228 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.221+0000 7f1309755700 1 --2- 172.21.15.1:0/2096073064 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55e737878200 0x55e73787a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:39.228 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.221+0000 7f12ecff9700 10 client.15750 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:40:39.228 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.221+0000 7f1309755700 1 --2- 172.21.15.1:0/2096073064 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55e737878200 0x55e73787a5e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c89b92fd9933940a server_cookie=94131ac8943cbae3 in_seq=0 out_seq=0 2023-11-16T16:40:39.228 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.222+0000 7f12ecff9700 1 -- 172.21.15.1:0/2096073064 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e737725660 con 0x55e737878200 2023-11-16T16:40:39.228 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.222+0000 7f12ecff9700 10 client.15750 handle_command_reply: tid=0 2023-11-16T16:40:39.228 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.222+0000 7f1317b45b80 1 client.15750 shutdown 2023-11-16T16:40:39.228 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.222+0000 7f1317b45b80 1 -- 172.21.15.1:0/2096073064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e73780a880 msgr2=0x55e737817270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:39.229 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.222+0000 7f1317b45b80 1 --2- 172.21.15.1:0/2096073064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e73780a880 0x55e737817270 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f12f4032d10 tx=0x7f12f4004e90 comp rx=0 tx=0).stop 2023-11-16T16:40:39.229 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.222+0000 7f1317b45b80 1 -- 172.21.15.1:0/2096073064 shutdown_connections 2023-11-16T16:40:39.229 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.222+0000 7f1317b45b80 1 --2- 172.21.15.1:0/2096073064 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55e737878200 0x55e73787a5e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:39.229 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.222+0000 7f1317b45b80 1 --2- 172.21.15.1:0/2096073064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e73780ecc0 0x55e7378177d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:39.229 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.222+0000 7f1317b45b80 1 --2- 172.21.15.1:0/2096073064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e737809f70 0x55e737816d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:39.229 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.222+0000 7f1317b45b80 1 --2- 172.21.15.1:0/2096073064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e73780a880 0x55e737817270 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:39.229 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.222+0000 7f12ecff9700 0 client.15750 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:40:39.229 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.222+0000 7f1317b45b80 1 -- 172.21.15.1:0/2096073064 >> 172.21.15.1:0/2096073064 conn(0x55e7377fb360 msgr2=0x55e737804960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:39.229 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.222+0000 7f1317b45b80 1 -- 172.21.15.1:0/2096073064 shutdown_connections 2023-11-16T16:40:39.229 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.222+0000 7f1317b45b80 1 -- 172.21.15.1:0/2096073064 wait complete. 2023-11-16T16:40:39.229 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.222+0000 7f1317b45b80 20 client.15750 trim_cache size 0 max 16384 2023-11-16T16:40:39.230 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:40:39.230 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2023-11-16T16:40:39.230 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2023-11-16T16:40:39.230 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:40:39.231 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.223+0000 7f130b9b9700 1 -- 172.21.15.1:0/965562726 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f12f00a13e0 msgr2=0x7f12f00a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:39.231 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.223+0000 7f130b9b9700 1 --2- 172.21.15.1:0/965562726 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f12f00a13e0 0x7f12f00a3890 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1304197970 tx=0x7f12f4005fd0 comp rx=0 tx=0).stop 2023-11-16T16:40:39.231 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.223+0000 7f130b9b9700 1 -- 172.21.15.1:0/965562726 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f13040ffa70 msgr2=0x7f1304196360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:39.231 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.223+0000 7f130b9b9700 1 --2- 172.21.15.1:0/965562726 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f13040ffa70 0x7f1304196360 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f130000e9a0 tx=0x7f130000ecb0 comp rx=0 tx=0).stop 2023-11-16T16:40:39.231 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.223+0000 7f130b9b9700 1 -- 172.21.15.1:0/965562726 shutdown_connections 2023-11-16T16:40:39.231 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.223+0000 7f130b9b9700 1 --2- 172.21.15.1:0/965562726 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f12f00a13e0 0x7f12f00a3890 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:39.231 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.223+0000 7f130b9b9700 1 --2- 172.21.15.1:0/965562726 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1304105250 0x7f130419ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:39.231 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.223+0000 7f130b9b9700 1 --2- 172.21.15.1:0/965562726 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f13040ffa70 0x7f1304196360 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:39.231 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.223+0000 7f130b9b9700 1 --2- 172.21.15.1:0/965562726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1304100420 0x7f13041968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:39.231 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.223+0000 7f130b9b9700 1 -- 172.21.15.1:0/965562726 >> 172.21.15.1:0/965562726 conn(0x7f13040763a0 msgr2=0x7f13040fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:39.231 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.223+0000 7f130b9b9700 1 -- 172.21.15.1:0/965562726 shutdown_connections 2023-11-16T16:40:39.232 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:39.223+0000 7f130b9b9700 1 -- 172.21.15.1:0/965562726 wait complete. 2023-11-16T16:40:39.300 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-16T16:40:39.301 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:40:39.443 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:39.443 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:39.564 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:39.564 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:39.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:39 smithi053 ceph-mon[100638]: pgmap v47: 97 pgs: 97 active+clean; 1.6 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 6 op/s 2023-11-16T16:40:39.584 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:39.584 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:39.584 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 35 -lt 100 ']' 2023-11-16T16:40:39.584 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 35 + 1 2023-11-16T16:40:39.632 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=36 2023-11-16T16:40:39.632 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:39.638 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:39.638 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:39.639 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:39.639 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:39.639 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:39.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:39 smithi001 ceph-mon[98800]: pgmap v47: 97 pgs: 97 active+clean; 1.6 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 6 op/s 2023-11-16T16:40:39.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:39 smithi026 ceph-mon[100112]: pgmap v47: 97 pgs: 97 active+clean; 1.6 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 6 op/s 2023-11-16T16:40:40.060 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:40.060 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:40.239 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:40.239 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:40.269 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:40.269 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:40.269 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 36 -lt 100 ']' 2023-11-16T16:40:40.269 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 36 + 1 2023-11-16T16:40:40.307 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=37 2023-11-16T16:40:40.307 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:40.354 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:40.355 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:40.356 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:40.356 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:40.356 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:41.518 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:41.518 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:41.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:41 smithi053 ceph-mon[100638]: pgmap v48: 97 pgs: 97 active+clean; 1.6 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 6 op/s 2023-11-16T16:40:41.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:41 smithi001 ceph-mon[98800]: pgmap v48: 97 pgs: 97 active+clean; 1.6 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 6 op/s 2023-11-16T16:40:41.713 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:41.713 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:41.736 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:41.737 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:41.737 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 37 -lt 100 ']' 2023-11-16T16:40:41.737 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 37 + 1 2023-11-16T16:40:41.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:41 smithi026 ceph-mon[100112]: pgmap v48: 97 pgs: 97 active+clean; 1.6 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 6 op/s 2023-11-16T16:40:42.257 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=38 2023-11-16T16:40:42.257 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:42.341 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:42.341 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:42.342 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:42.343 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:42.343 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:43.101 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:43.101 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:43.263 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:43.263 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:43.294 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:43.294 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:43.294 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 38 -lt 100 ']' 2023-11-16T16:40:43.295 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 38 + 1 2023-11-16T16:40:43.332 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=39 2023-11-16T16:40:43.332 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:43.366 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:43.367 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:43.367 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:43.368 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:43.368 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:43.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:43 smithi053 ceph-mon[100638]: pgmap v49: 97 pgs: 97 active+clean; 1.7 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 11 op/s 2023-11-16T16:40:43.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:43 smithi053 ceph-mon[100638]: mgrmap e25: x(active, since 92s), standbys: y, z 2023-11-16T16:40:43.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:43 smithi001 ceph-mon[98800]: pgmap v49: 97 pgs: 97 active+clean; 1.7 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 11 op/s 2023-11-16T16:40:43.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:43 smithi001 ceph-mon[98800]: mgrmap e25: x(active, since 92s), standbys: y, z 2023-11-16T16:40:43.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.684+0000 7fde7e447700 1 -- 172.21.15.1:0/749493381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde780ffdd0 msgr2=0x7fde78103d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:43.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.684+0000 7fde7e447700 1 --2- 172.21.15.1:0/749493381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde780ffdd0 0x7fde78103d90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fde600099c0 tx=0x7fde60009cd0 comp rx=0 tx=0).stop 2023-11-16T16:40:43.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.684+0000 7fde7e447700 1 -- 172.21.15.1:0/749493381 shutdown_connections 2023-11-16T16:40:43.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.684+0000 7fde7e447700 1 --2- 172.21.15.1:0/749493381 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde78104470 0x7fde7810c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:43.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.684+0000 7fde7e447700 1 --2- 172.21.15.1:0/749493381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde780ff430 0x7fde780ff800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:43.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.684+0000 7fde7e447700 1 --2- 172.21.15.1:0/749493381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde780ffdd0 0x7fde78103d90 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:43.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.684+0000 7fde7e447700 1 -- 172.21.15.1:0/749493381 >> 172.21.15.1:0/749493381 conn(0x7fde78077060 msgr2=0x7fde78077460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:43.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.684+0000 7fde7e447700 1 -- 172.21.15.1:0/749493381 shutdown_connections 2023-11-16T16:40:43.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.684+0000 7fde7e447700 1 -- 172.21.15.1:0/749493381 wait complete. 2023-11-16T16:40:43.690 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.685+0000 7fde7e447700 1 Processor -- start 2023-11-16T16:40:43.690 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.685+0000 7fde7e447700 1 -- start start 2023-11-16T16:40:43.690 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.685+0000 7fde7e447700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde780ff430 0x7fde78100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:43.690 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.685+0000 7fde7e447700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde780ffdd0 0x7fde78101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:43.690 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.685+0000 7fde7e447700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde78104470 0x7fde781a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:43.690 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.685+0000 7fde7e447700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde78101ba0 con 0x7fde780ff430 2023-11-16T16:40:43.690 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.685+0000 7fde7e447700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde781a4be0 con 0x7fde780ffdd0 2023-11-16T16:40:43.691 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.685+0000 7fde7e447700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde781a4d20 con 0x7fde78104470 2023-11-16T16:40:43.691 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.686+0000 7fde77fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde780ff430 0x7fde78100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:43.691 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.686+0000 7fde77fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde780ff430 0x7fde78100e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44298/0 (socket says 172.21.15.1:44298) 2023-11-16T16:40:43.691 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.686+0000 7fde77fff700 1 -- 172.21.15.1:0/214571980 learned_addr learned my addr 172.21.15.1:0/214571980 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:43.691 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.686+0000 7fde7c9e4700 1 --2- 172.21.15.1:0/214571980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde78104470 0x7fde781a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:43.691 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.686+0000 7fde6f7fe700 1 --2- 172.21.15.1:0/214571980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde780ffdd0 0x7fde78101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:43.691 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.686+0000 7fde77fff700 1 -- 172.21.15.1:0/214571980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde78104470 msgr2=0x7fde781a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:43.691 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.686+0000 7fde77fff700 1 --2- 172.21.15.1:0/214571980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde78104470 0x7fde781a46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:43.691 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.686+0000 7fde77fff700 1 -- 172.21.15.1:0/214571980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde780ffdd0 msgr2=0x7fde78101370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:43.691 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.686+0000 7fde77fff700 1 --2- 172.21.15.1:0/214571980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde780ffdd0 0x7fde78101370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:43.691 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.686+0000 7fde77fff700 1 -- 172.21.15.1:0/214571980 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde600095f0 con 0x7fde780ff430 2023-11-16T16:40:43.691 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.686+0000 7fde6f7fe700 1 --2- 172.21.15.1:0/214571980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde780ffdd0 0x7fde78101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:40:43.692 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.686+0000 7fde77fff700 1 --2- 172.21.15.1:0/214571980 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde780ff430 0x7fde78100e30 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fde6800e9a0 tx=0x7fde6800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:43.692 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.686+0000 7fde75ffb700 1 -- 172.21.15.1:0/214571980 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde68009790 con 0x7fde780ff430 2023-11-16T16:40:43.692 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.687+0000 7fde7e447700 1 -- 172.21.15.1:0/214571980 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde7810f1f0 con 0x7fde780ff430 2023-11-16T16:40:43.692 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.687+0000 7fde75ffb700 1 -- 172.21.15.1:0/214571980 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde6800cbd0 con 0x7fde780ff430 2023-11-16T16:40:43.692 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.687+0000 7fde75ffb700 1 -- 172.21.15.1:0/214571980 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde68018680 con 0x7fde780ff430 2023-11-16T16:40:43.692 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.687+0000 7fde7e447700 1 -- 172.21.15.1:0/214571980 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde7810f060 con 0x7fde780ff430 2023-11-16T16:40:43.692 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.688+0000 7fde7e447700 1 -- 172.21.15.1:0/214571980 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde7810f380 con 0x7fde780ff430 2023-11-16T16:40:43.693 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.690+0000 7fde75ffb700 1 -- 172.21.15.1:0/214571980 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fde6800cd40 con 0x7fde780ff430 2023-11-16T16:40:43.693 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.690+0000 7fde75ffb700 1 --2- 172.21.15.1:0/214571980 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fde640a12c0 0x7fde640a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:43.693 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.691+0000 7fde6f7fe700 1 --2- 172.21.15.1:0/214571980 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fde640a12c0 0x7fde640a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:43.693 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.691+0000 7fde75ffb700 1 -- 172.21.15.1:0/214571980 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fde68014070 con 0x7fde780ff430 2023-11-16T16:40:43.693 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.691+0000 7fde6f7fe700 1 --2- 172.21.15.1:0/214571980 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fde640a12c0 0x7fde640a3770 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fde600098e0 tx=0x7fde6001a040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:43.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.695+0000 7fde75ffb700 1 -- 172.21.15.1:0/214571980 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fde68096660 con 0x7fde780ff430 2023-11-16T16:40:43.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:43 smithi026 ceph-mon[100112]: pgmap v49: 97 pgs: 97 active+clean; 1.7 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 11 op/s 2023-11-16T16:40:43.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:43 smithi026 ceph-mon[100112]: mgrmap e25: x(active, since 92s), standbys: y, z 2023-11-16T16:40:43.951 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:43.951 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:43.955 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.954+0000 7fde7e447700 1 -- 172.21.15.1:0/214571980 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde78067af0 con 0x7fde780ff430 2023-11-16T16:40:43.956 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.954+0000 7fde75ffb700 1 -- 172.21.15.1:0/214571980 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10774 (secure 0 0 0) 0x7fde68099c80 con 0x7fde780ff430 2023-11-16T16:40:43.956 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:40:43.956 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2587563684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2587563684},{"type":"v1","addr":"172.21.15.1:6835","nonce":2587563684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/277936168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":277936168},{"type":"v1","addr":"172.21.15.1:6837","nonce":277936168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/1172575045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1172575045},{"type":"v1","addr":"172.21.15.1:6839","nonce":1172575045}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/2631175899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2631175899},{"type":"v1","addr":"172.21.15.1:6841","nonce":2631175899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3693905267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3693905267},{"type":"v1","addr":"172.21.15.53:6833","nonce":3693905267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/164668033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":164668033},{"type":"v1","addr":"172.21.15.26:6833","nonce":164668033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-16T16:37:13.918943+0000","modified":"2023-11-16T16:37:19.292898+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2140542192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2140542192},{"type":"v1","addr":"172.21.15.53:6835","nonce":2140542192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1010760275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1010760275},{"type":"v1","addr":"172.21.15.26:6835","nonce":1010760275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3197060302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3197060302},{"type":"v1","addr":"172.21.15.53:6837","nonce":3197060302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6837/1453504863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1453504863},{"type":"v1","addr":"172.21.15.26:6837","nonce":1453504863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.53:6839/1044753024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1044753024},{"type":"v1","addr":"172.21.15.53:6839","nonce":1044753024}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6839/2192205949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2192205949},{"type":"v1","addr":"172.21.15.26:6839","nonce":2192205949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-16T16:40:43.959 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.956+0000 7fde7e447700 1 -- 172.21.15.1:0/214571980 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fde640a12c0 msgr2=0x7fde640a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:43.959 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.956+0000 7fde7e447700 1 --2- 172.21.15.1:0/214571980 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fde640a12c0 0x7fde640a3770 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fde600098e0 tx=0x7fde6001a040 comp rx=0 tx=0).stop 2023-11-16T16:40:43.959 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.956+0000 7fde7e447700 1 -- 172.21.15.1:0/214571980 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde780ff430 msgr2=0x7fde78100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:43.959 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.956+0000 7fde7e447700 1 --2- 172.21.15.1:0/214571980 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde780ff430 0x7fde78100e30 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fde6800e9a0 tx=0x7fde6800ecb0 comp rx=0 tx=0).stop 2023-11-16T16:40:43.959 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.956+0000 7fde7e447700 1 -- 172.21.15.1:0/214571980 shutdown_connections 2023-11-16T16:40:43.959 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.957+0000 7fde7e447700 1 --2- 172.21.15.1:0/214571980 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fde640a12c0 0x7fde640a3770 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:43.959 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.957+0000 7fde7e447700 1 --2- 172.21.15.1:0/214571980 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fde78104470 0x7fde781a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:43.959 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.957+0000 7fde7e447700 1 --2- 172.21.15.1:0/214571980 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde780ff430 0x7fde78100e30 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:43.959 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.957+0000 7fde7e447700 1 --2- 172.21.15.1:0/214571980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde780ffdd0 0x7fde78101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:43.959 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.957+0000 7fde7e447700 1 -- 172.21.15.1:0/214571980 >> 172.21.15.1:0/214571980 conn(0x7fde78077060 msgr2=0x7fde78067ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:43.960 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.957+0000 7fde7e447700 1 -- 172.21.15.1:0/214571980 shutdown_connections 2023-11-16T16:40:43.960 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:43.957+0000 7fde7e447700 1 -- 172.21.15.1:0/214571980 wait complete. 2023-11-16T16:40:43.960 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 23 2023-11-16T16:40:44.016 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-11-16T16:40:44.147 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:44.147 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:44.192 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:44.192 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:44.192 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 39 -lt 100 ']' 2023-11-16T16:40:44.192 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 39 + 1 2023-11-16T16:40:44.223 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=40 2023-11-16T16:40:44.223 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:44.230 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:44.230 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:44.231 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:44.232 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:44.232 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:44.567 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:44.567 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:44.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:44 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/214571980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:40:44.656 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:44.657 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:44.667 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:44.667 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:44.668 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 40 -lt 100 ']' 2023-11-16T16:40:44.668 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 40 + 1 2023-11-16T16:40:44.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:44 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/214571980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:40:44.706 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=41 2023-11-16T16:40:44.707 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:44.745 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:44.745 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:44.746 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:44.746 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:44.747 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:44.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:44 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/214571980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:40:45.118 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:45.118 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:45.297 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:45.298 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:45.350 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:45.351 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:45.351 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 41 -lt 100 ']' 2023-11-16T16:40:45.351 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 41 + 1 2023-11-16T16:40:45.485 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=42 2023-11-16T16:40:45.485 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:45.552 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:45.553 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:45.553 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:45.553 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:45.554 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:45.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:45 smithi053 ceph-mon[100638]: pgmap v50: 97 pgs: 97 active+clean; 1.7 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 7 op/s 2023-11-16T16:40:45.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:45 smithi001 ceph-mon[98800]: pgmap v50: 97 pgs: 97 active+clean; 1.7 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 7 op/s 2023-11-16T16:40:45.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:45 smithi026 ceph-mon[100112]: pgmap v50: 97 pgs: 97 active+clean; 1.7 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 7 op/s 2023-11-16T16:40:46.310 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:46.310 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:46.863 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:46.863 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:46.936 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:46.936 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:46.936 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 42 -lt 100 ']' 2023-11-16T16:40:46.936 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 42 + 1 2023-11-16T16:40:47.048 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=43 2023-11-16T16:40:47.048 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:47.201 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:47.201 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:47.202 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:47.203 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:47.203 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:47.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.441+0000 7fd8a4235700 1 -- 172.21.15.1:0/3649423446 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd89c068940 msgr2=0x7fd89c1108b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:47.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.441+0000 7fd8a4235700 1 --2- 172.21.15.1:0/3649423446 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd89c068940 0x7fd89c1108b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd890009990 tx=0x7fd890009ca0 comp rx=0 tx=0).stop 2023-11-16T16:40:47.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.441+0000 7fd8a4235700 1 -- 172.21.15.1:0/3649423446 shutdown_connections 2023-11-16T16:40:47.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.441+0000 7fd8a4235700 1 --2- 172.21.15.1:0/3649423446 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd89c110f70 0x7fd89c1133b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:47.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.441+0000 7fd8a4235700 1 --2- 172.21.15.1:0/3649423446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd89c068030 0x7fd89c068400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:47.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.441+0000 7fd8a4235700 1 --2- 172.21.15.1:0/3649423446 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd89c068940 0x7fd89c1108b0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:47.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.441+0000 7fd8a4235700 1 -- 172.21.15.1:0/3649423446 >> 172.21.15.1:0/3649423446 conn(0x7fd89c0763a0 msgr2=0x7fd89c0767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:47.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.441+0000 7fd8a4235700 1 -- 172.21.15.1:0/3649423446 shutdown_connections 2023-11-16T16:40:47.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.442+0000 7fd8a4235700 1 -- 172.21.15.1:0/3649423446 wait complete. 2023-11-16T16:40:47.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.442+0000 7fd8a4235700 1 Processor -- start 2023-11-16T16:40:47.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.442+0000 7fd8a4235700 1 -- start start 2023-11-16T16:40:47.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.443+0000 7fd8a4235700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd89c068030 0x7fd89c1a2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:47.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.443+0000 7fd8a4235700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd89c068940 0x7fd89c1a2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:47.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.443+0000 7fd8a4235700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd89c110f70 0x7fd89c19c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:47.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.443+0000 7fd8a4235700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c19cf60 con 0x7fd89c068030 2023-11-16T16:40:47.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.443+0000 7fd8a4235700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c19d0d0 con 0x7fd89c068940 2023-11-16T16:40:47.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.443+0000 7fd8a4235700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c19d290 con 0x7fd89c110f70 2023-11-16T16:40:47.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.443+0000 7fd8a1fd1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd89c068030 0x7fd89c1a2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:47.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.443+0000 7fd8a1fd1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd89c068030 0x7fd89c1a2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44310/0 (socket says 172.21.15.1:44310) 2023-11-16T16:40:47.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.443+0000 7fd8a1fd1700 1 -- 172.21.15.1:0/1446705154 learned_addr learned my addr 172.21.15.1:0/1446705154 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:47.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.443+0000 7fd8a27d2700 1 --2- 172.21.15.1:0/1446705154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd89c110f70 0x7fd89c19c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:47.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.443+0000 7fd8a17d0700 1 --2- 172.21.15.1:0/1446705154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd89c068940 0x7fd89c1a2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:47.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.443+0000 7fd8a1fd1700 1 -- 172.21.15.1:0/1446705154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd89c110f70 msgr2=0x7fd89c19c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:47.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.443+0000 7fd8a1fd1700 1 --2- 172.21.15.1:0/1446705154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd89c110f70 0x7fd89c19c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:47.450 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.443+0000 7fd8a1fd1700 1 -- 172.21.15.1:0/1446705154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd89c068940 msgr2=0x7fd89c1a2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:47.450 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.443+0000 7fd8a1fd1700 1 --2- 172.21.15.1:0/1446705154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd89c068940 0x7fd89c1a2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:47.450 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.443+0000 7fd8a1fd1700 1 -- 172.21.15.1:0/1446705154 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8900095f0 con 0x7fd89c068030 2023-11-16T16:40:47.450 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.443+0000 7fd8a1fd1700 1 --2- 172.21.15.1:0/1446705154 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd89c068030 0x7fd89c1a2880 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd89800e9a0 tx=0x7fd89800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:47.450 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.444+0000 7fd88effd700 1 -- 172.21.15.1:0/1446705154 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd898009790 con 0x7fd89c068030 2023-11-16T16:40:47.450 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.444+0000 7fd88effd700 1 -- 172.21.15.1:0/1446705154 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd89800cbd0 con 0x7fd89c068030 2023-11-16T16:40:47.450 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.444+0000 7fd8a4235700 1 -- 172.21.15.1:0/1446705154 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd89c116aa0 con 0x7fd89c068030 2023-11-16T16:40:47.451 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.444+0000 7fd88effd700 1 -- 172.21.15.1:0/1446705154 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd898018680 con 0x7fd89c068030 2023-11-16T16:40:47.451 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.444+0000 7fd8a4235700 1 -- 172.21.15.1:0/1446705154 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd89c116910 con 0x7fd89c068030 2023-11-16T16:40:47.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.446+0000 7fd88effd700 1 -- 172.21.15.1:0/1446705154 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fd89800cd40 con 0x7fd89c068030 2023-11-16T16:40:47.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.447+0000 7fd88effd700 1 --2- 172.21.15.1:0/1446705154 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fd8880a1340 0x7fd8880a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:47.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.447+0000 7fd8a17d0700 1 --2- 172.21.15.1:0/1446705154 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fd8880a1340 0x7fd8880a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:47.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.447+0000 7fd88effd700 1 -- 172.21.15.1:0/1446705154 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd898014070 con 0x7fd89c068030 2023-11-16T16:40:47.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.448+0000 7fd8a17d0700 1 --2- 172.21.15.1:0/1446705154 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fd8880a1340 0x7fd8880a37f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd89000b2e0 tx=0x7fd890005fb0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:47.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.568+0000 7fd8b03c1b80 1 Processor -- start 2023-11-16T16:40:47.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.569+0000 7fd8b03c1b80 1 -- start start 2023-11-16T16:40:47.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.569+0000 7fd8b03c1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8069f1b60 0x55a8069f1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:47.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.569+0000 7fd8b03c1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8069f2470 0x55a8069f6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:47.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.569+0000 7fd8b03c1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8069f68b0 0x55a8069f6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:47.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.569+0000 7fd8b03c1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8068f7060 con 0x55a8069f2470 2023-11-16T16:40:47.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.569+0000 7fd8b03c1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8069498a0 con 0x55a8069f68b0 2023-11-16T16:40:47.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.569+0000 7fd8b03c1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a806942d30 con 0x55a8069f1b60 2023-11-16T16:40:47.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.569+0000 7fd8a17d0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8069f2470 0x55a8069f6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:47.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.569+0000 7fd8a17d0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8069f2470 0x55a8069f6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44318/0 (socket says 172.21.15.1:44318) 2023-11-16T16:40:47.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.569+0000 7fd8a17d0700 1 -- 172.21.15.1:0/2598790923 learned_addr learned my addr 172.21.15.1:0/2598790923 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:47.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.569+0000 7fd8a1fd1700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8069f1b60 0x55a8069f1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:47.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.569+0000 7fd8a27d2700 1 --2- 172.21.15.1:0/2598790923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8069f68b0 0x55a8069f6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:47.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.570+0000 7fd8a1fd1700 1 -- 172.21.15.1:0/2598790923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8069f68b0 msgr2=0x55a8069f6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:47.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.570+0000 7fd8a1fd1700 1 --2- 172.21.15.1:0/2598790923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8069f68b0 0x55a8069f6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:47.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.570+0000 7fd8a1fd1700 1 -- 172.21.15.1:0/2598790923 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8069f2470 msgr2=0x55a8069f6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:47.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.570+0000 7fd8a1fd1700 1 --2- 172.21.15.1:0/2598790923 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8069f2470 0x55a8069f6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:47.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.570+0000 7fd8a1fd1700 1 -- 172.21.15.1:0/2598790923 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8067c4630 con 0x55a8069f1b60 2023-11-16T16:40:47.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.570+0000 7fd8a17d0700 1 --2- 172.21.15.1:0/2598790923 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8069f2470 0x55a8069f6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:40:47.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.570+0000 7fd8a1fd1700 1 --2- 172.21.15.1:0/2598790923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8069f1b60 0x55a8069f1f30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd8980ca640 tx=0x7fd898099920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=791f4cc0875475a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:47.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.570+0000 7fd887fff700 1 -- 172.21.15.1:0/2598790923 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8980a2410 con 0x55a8069f1b60 2023-11-16T16:40:47.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.570+0000 7fd887fff700 1 -- 172.21.15.1:0/2598790923 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd89809a220 con 0x55a8069f1b60 2023-11-16T16:40:47.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.570+0000 7fd887fff700 1 -- 172.21.15.1:0/2598790923 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd898094e30 con 0x55a8069f1b60 2023-11-16T16:40:47.581 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.571+0000 7fd8b03c1b80 1 -- 172.21.15.1:0/2598790923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8069f1b60 msgr2=0x55a8069f1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:47.581 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.571+0000 7fd8b03c1b80 1 --2- 172.21.15.1:0/2598790923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8069f1b60 0x55a8069f1f30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd8980ca640 tx=0x7fd898099920 comp rx=0 tx=0).stop 2023-11-16T16:40:47.581 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.571+0000 7fd8b03c1b80 1 -- 172.21.15.1:0/2598790923 shutdown_connections 2023-11-16T16:40:47.581 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.571+0000 7fd8b03c1b80 1 --2- 172.21.15.1:0/2598790923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8069f68b0 0x55a8069f6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:47.581 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.571+0000 7fd8b03c1b80 1 --2- 172.21.15.1:0/2598790923 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8069f1b60 0x55a8069f1f30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:47.582 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.571+0000 7fd8b03c1b80 1 --2- 172.21.15.1:0/2598790923 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8069f2470 0x55a8069f6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:47.582 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.571+0000 7fd8b03c1b80 1 -- 172.21.15.1:0/2598790923 >> 172.21.15.1:0/2598790923 conn(0x55a8069e7360 msgr2=0x55a8069ecb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:47.582 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.571+0000 7fd8b03c1b80 1 -- 172.21.15.1:0/2598790923 shutdown_connections 2023-11-16T16:40:47.582 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.571+0000 7fd8b03c1b80 1 -- 172.21.15.1:0/2598790923 wait complete. 2023-11-16T16:40:47.582 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.572+0000 7fd8b03c1b80 1 -- start start 2023-11-16T16:40:47.582 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.572+0000 7fd8b03c1b80 1 Processor -- start 2023-11-16T16:40:47.582 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.572+0000 7fd8b03c1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8069f1b60 0x55a8069fe100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:47.582 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.572+0000 7fd8b03c1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8069f2470 0x55a8069fb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:47.583 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.572+0000 7fd8b03c1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8069f68b0 0x55a8069fbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:47.583 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.572+0000 7fd8b03c1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8068e48e0 con 0x55a8069f2470 2023-11-16T16:40:47.583 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.572+0000 7fd8b03c1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8068e4580 con 0x55a8069f1b60 2023-11-16T16:40:47.583 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.572+0000 7fd8b03c1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8067c6920 con 0x55a8069f68b0 2023-11-16T16:40:47.583 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.573+0000 7fd8b03c1b80 10 client.0 _ll_register_callbacks cb 0x55a806902780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:40:47.583 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.573+0000 7fd8a17d0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8069f2470 0x55a8069fb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:47.583 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.573+0000 7fd8a17d0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8069f2470 0x55a8069fb380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44322/0 (socket says 172.21.15.1:44322) 2023-11-16T16:40:47.583 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.573+0000 7fd8a17d0700 1 -- 172.21.15.1:0/241523784 learned_addr learned my addr 172.21.15.1:0/241523784 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:47.583 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.573+0000 7fd8a27d2700 1 --2- 172.21.15.1:0/241523784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8069f68b0 0x55a8069fbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:47.584 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.573+0000 7fd8a1fd1700 1 --2- 172.21.15.1:0/241523784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8069f1b60 0x55a8069fe100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:47.584 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.573+0000 7fd8a17d0700 1 -- 172.21.15.1:0/241523784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8069f68b0 msgr2=0x55a8069fbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:47.584 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.573+0000 7fd8a17d0700 1 --2- 172.21.15.1:0/241523784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8069f68b0 0x55a8069fbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:47.584 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.573+0000 7fd8a17d0700 1 -- 172.21.15.1:0/241523784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8069f1b60 msgr2=0x55a8069fe100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:47.584 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.573+0000 7fd8a17d0700 1 --2- 172.21.15.1:0/241523784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8069f1b60 0x55a8069fe100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:47.584 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.573+0000 7fd8a17d0700 1 -- 172.21.15.1:0/241523784 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd89c116910 con 0x55a8069f2470 2023-11-16T16:40:47.584 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.573+0000 7fd8a27d2700 1 --2- 172.21.15.1:0/241523784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8069f68b0 0x55a8069fbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:40:47.584 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.573+0000 7fd8a17d0700 1 --2- 172.21.15.1:0/241523784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8069f2470 0x55a8069fb380 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd890005790 tx=0x7fd89001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ee72fae7c51aa8 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:47.584 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.573+0000 7fd8857fa700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:40:47.584 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.574+0000 7fd8857fa700 1 -- 172.21.15.1:0/241523784 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89002d040 con 0x55a8069f2470 2023-11-16T16:40:47.585 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.574+0000 7fd8857fa700 1 -- 172.21.15.1:0/241523784 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd890004b10 con 0x55a8069f2470 2023-11-16T16:40:47.585 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.574+0000 7fd8b03c1b80 1 -- 172.21.15.1:0/241523784 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a8069e4b20 con 0x55a8069f2470 2023-11-16T16:40:47.585 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.574+0000 7fd8857fa700 1 -- 172.21.15.1:0/241523784 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd890004cd0 con 0x55a8069f2470 2023-11-16T16:40:47.585 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.574+0000 7fd8b03c1b80 10 client.15768 fetch_fsmap learned FSMap version 23 2023-11-16T16:40:47.585 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.574+0000 7fd8b03c1b80 1 -- 172.21.15.1:0/241523784 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55a806909190 con 0x55a8069f2470 2023-11-16T16:40:47.585 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.574+0000 7fd8857fa700 1 -- 172.21.15.1:0/241523784 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd89003e080 con 0x55a8069f2470 2023-11-16T16:40:47.585 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.574+0000 7fd8857fa700 1 -- 172.21.15.1:0/241523784 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd890032430 con 0x55a8069f2470 2023-11-16T16:40:47.585 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.575+0000 7fd8b03c1b80 10 client.15768 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:40:47.585 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.575+0000 7fd8b03c1b80 10 client.15768 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-11-16T16:40:47.585 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.575+0000 7fd8b03c1b80 20 client.15768 populate_metadata read hostname 'smithi001' 2023-11-16T16:40:47.586 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.575+0000 7fd8b03c1b80 1 --2- 172.21.15.1:0/241523784 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55a806a0b9f0 0x55a806a0de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:47.586 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.575+0000 7fd8b03c1b80 4 client.15768 mds_command: new command op to 24482 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:40:47.586 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.575+0000 7fd8b03c1b80 1 -- 172.21.15.1:0/241523784 --> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a8069050a0 con 0x55a806a0b9f0 2023-11-16T16:40:47.586 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.575+0000 7fd8a1fd1700 1 --2- 172.21.15.1:0/241523784 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55a806a0b9f0 0x55a806a0de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:47.586 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.575+0000 7fd8a1fd1700 1 --2- 172.21.15.1:0/241523784 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55a806a0b9f0 0x55a806a0de00 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=82e95b9cc157e290 server_cookie=506c6d44bb42fc64 in_seq=0 out_seq=0 2023-11-16T16:40:47.586 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.575+0000 7fd8857fa700 10 client.15768 ms_handle_connect on v2:172.21.15.26:6836/1453504863 2023-11-16T16:40:47.586 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.576+0000 7fd8857fa700 1 -- 172.21.15.1:0/241523784 <== mds.2 v2:172.21.15.26:6836/1453504863 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a8069050a0 con 0x55a806a0b9f0 2023-11-16T16:40:47.586 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.576+0000 7fd8857fa700 10 client.15768 handle_command_reply: tid=0 2023-11-16T16:40:47.586 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.576+0000 7fd8b03c1b80 1 client.15768 shutdown 2023-11-16T16:40:47.587 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.576+0000 7fd8b03c1b80 1 -- 172.21.15.1:0/241523784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8069f2470 msgr2=0x55a8069fb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:47.587 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.576+0000 7fd8b03c1b80 1 --2- 172.21.15.1:0/241523784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8069f2470 0x55a8069fb380 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd890005790 tx=0x7fd89001f000 comp rx=0 tx=0).stop 2023-11-16T16:40:47.587 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.577+0000 7fd8b03c1b80 1 -- 172.21.15.1:0/241523784 shutdown_connections 2023-11-16T16:40:47.587 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.577+0000 7fd8b03c1b80 1 --2- 172.21.15.1:0/241523784 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55a806a0b9f0 0x55a806a0de00 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:47.587 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.577+0000 7fd8b03c1b80 1 --2- 172.21.15.1:0/241523784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8069f68b0 0x55a8069fbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:47.587 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.577+0000 7fd8b03c1b80 1 --2- 172.21.15.1:0/241523784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8069f1b60 0x55a8069fe100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:47.587 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.577+0000 7fd8b03c1b80 1 --2- 172.21.15.1:0/241523784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8069f2470 0x55a8069fb380 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:47.587 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.577+0000 7fd8857fa700 0 client.15768 ms_handle_reset on v2:172.21.15.26:6836/1453504863 2023-11-16T16:40:47.587 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.577+0000 7fd8b03c1b80 1 -- 172.21.15.1:0/241523784 >> 172.21.15.1:0/241523784 conn(0x55a8069e7360 msgr2=0x55a8069ebde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:47.587 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.577+0000 7fd8b03c1b80 1 -- 172.21.15.1:0/241523784 shutdown_connections 2023-11-16T16:40:47.588 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.577+0000 7fd8b03c1b80 1 -- 172.21.15.1:0/241523784 wait complete. 2023-11-16T16:40:47.588 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.577+0000 7fd8b03c1b80 20 client.15768 trim_cache size 0 max 16384 2023-11-16T16:40:47.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:47 smithi053 ceph-mon[100638]: pgmap v51: 97 pgs: 97 active+clean; 1.8 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-11-16T16:40:47.595 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.591+0000 7fd8b03c1b80 1 Processor -- start 2023-11-16T16:40:47.595 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.591+0000 7fd8b03c1b80 1 -- start start 2023-11-16T16:40:47.596 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.591+0000 7fd8b03c1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8069f1b60 0x55a8069f51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:47.596 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.591+0000 7fd8b03c1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8069f2470 0x55a8069fdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:47.596 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.591+0000 7fd8b03c1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8069f68b0 0x55a8069fab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:47.596 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.591+0000 7fd8b03c1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8069e6ee0 con 0x55a8069f68b0 2023-11-16T16:40:47.596 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.591+0000 7fd8b03c1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8069ee430 con 0x55a8069f2470 2023-11-16T16:40:47.596 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.591+0000 7fd8b03c1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8069ed750 con 0x55a8069f1b60 2023-11-16T16:40:47.596 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.591+0000 7fd8a1fd1700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8069f1b60 0x55a8069f51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:47.596 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.591+0000 7fd8a27d2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8069f68b0 0x55a8069fab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:47.596 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.591+0000 7fd8a17d0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8069f2470 0x55a8069fdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:47.596 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.591+0000 7fd8a27d2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8069f68b0 0x55a8069fab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44334/0 (socket says 172.21.15.1:44334) 2023-11-16T16:40:47.597 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.591+0000 7fd8a1fd1700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8069f1b60 0x55a8069f51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:41012/0 (socket says 172.21.15.1:41012) 2023-11-16T16:40:47.597 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.591+0000 7fd8a27d2700 1 -- 172.21.15.1:0/1370623365 learned_addr learned my addr 172.21.15.1:0/1370623365 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:47.597 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.592+0000 7fd8a17d0700 1 -- 172.21.15.1:0/1370623365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8069f1b60 msgr2=0x55a8069f51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:47.597 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.592+0000 7fd8a17d0700 1 --2- 172.21.15.1:0/1370623365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8069f1b60 0x55a8069f51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:47.597 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.592+0000 7fd8a17d0700 1 -- 172.21.15.1:0/1370623365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8069f68b0 msgr2=0x55a8069fab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:47.597 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.592+0000 7fd8a17d0700 1 --2- 172.21.15.1:0/1370623365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8069f68b0 0x55a8069fab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:47.597 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.592+0000 7fd8a17d0700 1 -- 172.21.15.1:0/1370623365 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a806908ee0 con 0x55a8069f2470 2023-11-16T16:40:47.597 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.592+0000 7fd8a1fd1700 1 --2- 172.21.15.1:0/1370623365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8069f1b60 0x55a8069f51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:40:47.597 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.592+0000 7fd8a27d2700 1 --2- 172.21.15.1:0/1370623365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8069f68b0 0x55a8069fab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:40:47.598 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.592+0000 7fd8a17d0700 1 --2- 172.21.15.1:0/1370623365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8069f2470 0x55a8069fdca0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x55a8069e4750 tx=0x7fd890009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9404cac63d289441 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:47.598 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.592+0000 7fd88cff9700 1 -- 172.21.15.1:0/1370623365 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89002d040 con 0x55a8069f2470 2023-11-16T16:40:47.598 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.592+0000 7fd88cff9700 1 -- 172.21.15.1:0/1370623365 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a8069e4b20 con 0x55a8069f2470 2023-11-16T16:40:47.598 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.592+0000 7fd88cff9700 1 -- 172.21.15.1:0/1370623365 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89000f460 con 0x55a8069f2470 2023-11-16T16:40:47.598 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.593+0000 7fd8b03c1b80 1 -- 172.21.15.1:0/1370623365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8069f2470 msgr2=0x55a8069fdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:47.598 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.593+0000 7fd8b03c1b80 1 --2- 172.21.15.1:0/1370623365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8069f2470 0x55a8069fdca0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x55a8069e4750 tx=0x7fd890009310 comp rx=0 tx=0).stop 2023-11-16T16:40:47.598 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.593+0000 7fd8b03c1b80 1 -- 172.21.15.1:0/1370623365 shutdown_connections 2023-11-16T16:40:47.598 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.593+0000 7fd8b03c1b80 1 --2- 172.21.15.1:0/1370623365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8069f68b0 0x55a8069fab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:47.599 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.593+0000 7fd8b03c1b80 1 --2- 172.21.15.1:0/1370623365 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8069f1b60 0x55a8069f51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:47.599 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.593+0000 7fd8b03c1b80 1 --2- 172.21.15.1:0/1370623365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8069f2470 0x55a8069fdca0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:47.599 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.593+0000 7fd8b03c1b80 1 -- 172.21.15.1:0/1370623365 >> 172.21.15.1:0/1370623365 conn(0x55a8069e7360 msgr2=0x55a8069f5a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:47.599 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.593+0000 7fd8b03c1b80 1 -- 172.21.15.1:0/1370623365 shutdown_connections 2023-11-16T16:40:47.599 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.593+0000 7fd8b03c1b80 1 -- 172.21.15.1:0/1370623365 wait complete. 2023-11-16T16:40:47.599 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.593+0000 7fd8b03c1b80 1 -- start start 2023-11-16T16:40:47.599 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.593+0000 7fd8b03c1b80 1 Processor -- start 2023-11-16T16:40:47.603 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.594+0000 7fd8b03c1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8069f1b60 0x55a8069fe920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:47.603 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.594+0000 7fd8b03c1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8069f2470 0x55a8069fee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:47.603 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.594+0000 7fd8b03c1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8069f68b0 0x55a8069ff3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:47.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.594+0000 7fd8b03c1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a806a5e8a0 con 0x55a8069f2470 2023-11-16T16:40:47.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.594+0000 7fd8b03c1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a806a5e9e0 con 0x55a8069f1b60 2023-11-16T16:40:47.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.594+0000 7fd8b03c1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a806a5eb50 con 0x55a8069f68b0 2023-11-16T16:40:47.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.594+0000 7fd8b03c1b80 10 client.0 _ll_register_callbacks cb 0x55a806902780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:40:47.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.594+0000 7fd8a17d0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8069f2470 0x55a8069fee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:47.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.594+0000 7fd8a17d0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8069f2470 0x55a8069fee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44338/0 (socket says 172.21.15.1:44338) 2023-11-16T16:40:47.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.594+0000 7fd8a17d0700 1 -- 172.21.15.1:0/366906240 learned_addr learned my addr 172.21.15.1:0/366906240 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:47.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.594+0000 7fd8a1fd1700 1 --2- 172.21.15.1:0/366906240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8069f1b60 0x55a8069fe920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:47.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.594+0000 7fd8a27d2700 1 --2- 172.21.15.1:0/366906240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8069f68b0 0x55a8069ff3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:47.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.595+0000 7fd8a17d0700 1 -- 172.21.15.1:0/366906240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8069f68b0 msgr2=0x55a8069ff3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:47.605 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.595+0000 7fd8a17d0700 1 --2- 172.21.15.1:0/366906240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8069f68b0 0x55a8069ff3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:47.605 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.595+0000 7fd8a17d0700 1 -- 172.21.15.1:0/366906240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8069f1b60 msgr2=0x55a8069fe920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:47.605 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.595+0000 7fd8a17d0700 1 --2- 172.21.15.1:0/366906240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8069f1b60 0x55a8069fe920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:47.605 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.595+0000 7fd8a17d0700 1 -- 172.21.15.1:0/366906240 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8067c4630 con 0x55a8069f2470 2023-11-16T16:40:47.605 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.595+0000 7fd8a27d2700 1 --2- 172.21.15.1:0/366906240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8069f68b0 0x55a8069ff3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:40:47.605 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.595+0000 7fd8a1fd1700 1 --2- 172.21.15.1:0/366906240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8069f1b60 0x55a8069fe920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:40:47.605 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.595+0000 7fd8a17d0700 1 --2- 172.21.15.1:0/366906240 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8069f2470 0x55a8069fee60 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd8900098e0 tx=0x7fd890032a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1936b5fc491ed17 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:47.605 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.595+0000 7fd8857fa700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:40:47.605 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.595+0000 7fd8857fa700 1 -- 172.21.15.1:0/366906240 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8900047e0 con 0x55a8069f2470 2023-11-16T16:40:47.605 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.595+0000 7fd8857fa700 1 -- 172.21.15.1:0/366906240 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd890004940 con 0x55a8069f2470 2023-11-16T16:40:47.606 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.595+0000 7fd8b03c1b80 1 -- 172.21.15.1:0/366906240 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd89809a220 con 0x55a8069f2470 2023-11-16T16:40:47.606 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.595+0000 7fd8857fa700 1 -- 172.21.15.1:0/366906240 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd890004b00 con 0x55a8069f2470 2023-11-16T16:40:47.606 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.595+0000 7fd8b03c1b80 10 client.15774 fetch_fsmap learned FSMap version 23 2023-11-16T16:40:47.606 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.595+0000 7fd8b03c1b80 1 -- 172.21.15.1:0/366906240 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7fd894011920 con 0x55a8069f2470 2023-11-16T16:40:47.606 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.596+0000 7fd8857fa700 1 -- 172.21.15.1:0/366906240 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd890052020 con 0x55a8069f2470 2023-11-16T16:40:47.606 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.596+0000 7fd8857fa700 1 -- 172.21.15.1:0/366906240 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd890018400 con 0x55a8069f2470 2023-11-16T16:40:47.606 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.596+0000 7fd8b03c1b80 10 client.15774 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:40:47.606 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.596+0000 7fd8b03c1b80 10 client.15774 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-11-16T16:40:47.606 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.596+0000 7fd8b03c1b80 20 client.15774 populate_metadata read hostname 'smithi001' 2023-11-16T16:40:47.606 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.596+0000 7fd8b03c1b80 1 --2- 172.21.15.1:0/366906240 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55a806a5fd00 0x55a806a620e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:47.607 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.596+0000 7fd8b03c1b80 4 client.15774 mds_command: new command op to 24482 tid=0[{"prefix": "damage ls"}] 2023-11-16T16:40:47.607 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.596+0000 7fd8b03c1b80 1 -- 172.21.15.1:0/366906240 --> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a806911660 con 0x55a806a5fd00 2023-11-16T16:40:47.607 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.596+0000 7fd8a1fd1700 1 --2- 172.21.15.1:0/366906240 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55a806a5fd00 0x55a806a620e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:47.607 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.597+0000 7fd8857fa700 10 client.15774 ms_handle_connect on v2:172.21.15.26:6836/1453504863 2023-11-16T16:40:47.607 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.597+0000 7fd8a1fd1700 1 --2- 172.21.15.1:0/366906240 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55a806a5fd00 0x55a806a620e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ecb2dbc2f22c144c server_cookie=4f7e385391505ac6 in_seq=0 out_seq=0 2023-11-16T16:40:47.607 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.597+0000 7fd8857fa700 1 -- 172.21.15.1:0/366906240 <== mds.2 v2:172.21.15.26:6836/1453504863 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a806911660 con 0x55a806a5fd00 2023-11-16T16:40:47.607 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.597+0000 7fd8857fa700 10 client.15774 handle_command_reply: tid=0 2023-11-16T16:40:47.607 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.597+0000 7fd8b03c1b80 1 client.15774 shutdown 2023-11-16T16:40:47.607 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.597+0000 7fd8b03c1b80 1 -- 172.21.15.1:0/366906240 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8069f2470 msgr2=0x55a8069fee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:47.607 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.597+0000 7fd8b03c1b80 1 --2- 172.21.15.1:0/366906240 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8069f2470 0x55a8069fee60 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd8900098e0 tx=0x7fd890032a00 comp rx=0 tx=0).stop 2023-11-16T16:40:47.608 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.597+0000 7fd8b03c1b80 1 -- 172.21.15.1:0/366906240 shutdown_connections 2023-11-16T16:40:47.608 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.597+0000 7fd8b03c1b80 1 --2- 172.21.15.1:0/366906240 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55a806a5fd00 0x55a806a620e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:47.608 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.597+0000 7fd8b03c1b80 1 --2- 172.21.15.1:0/366906240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a8069f68b0 0x55a8069ff3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:47.608 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.597+0000 7fd8b03c1b80 1 --2- 172.21.15.1:0/366906240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a8069f1b60 0x55a8069fe920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:47.608 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.597+0000 7fd8b03c1b80 1 --2- 172.21.15.1:0/366906240 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a8069f2470 0x55a8069fee60 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:47.608 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.597+0000 7fd8857fa700 0 client.15774 ms_handle_reset on v2:172.21.15.26:6836/1453504863 2023-11-16T16:40:47.608 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.597+0000 7fd8b03c1b80 1 -- 172.21.15.1:0/366906240 >> 172.21.15.1:0/366906240 conn(0x55a8069e7360 msgr2=0x55a8069f0960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:47.608 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.597+0000 7fd8b03c1b80 1 -- 172.21.15.1:0/366906240 shutdown_connections 2023-11-16T16:40:47.608 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.597+0000 7fd8b03c1b80 1 -- 172.21.15.1:0/366906240 wait complete. 2023-11-16T16:40:47.608 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.597+0000 7fd8b03c1b80 20 client.15774 trim_cache size 0 max 16384 2023-11-16T16:40:47.609 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2023-11-16T16:40:47.610 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.598+0000 7fd8a4235700 1 -- 172.21.15.1:0/1446705154 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fd8880a1340 msgr2=0x7fd8880a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:47.610 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.598+0000 7fd8a4235700 1 --2- 172.21.15.1:0/1446705154 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fd8880a1340 0x7fd8880a37f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd89000b2e0 tx=0x7fd890005fb0 comp rx=0 tx=0).stop 2023-11-16T16:40:47.610 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.598+0000 7fd8a4235700 1 -- 172.21.15.1:0/1446705154 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd89c068030 msgr2=0x7fd89c1a2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:47.610 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.598+0000 7fd8a4235700 1 --2- 172.21.15.1:0/1446705154 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd89c068030 0x7fd89c1a2880 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd89800e9a0 tx=0x7fd89800ecb0 comp rx=0 tx=0).stop 2023-11-16T16:40:47.610 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.598+0000 7fd8a4235700 1 -- 172.21.15.1:0/1446705154 shutdown_connections 2023-11-16T16:40:47.610 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.598+0000 7fd8a4235700 1 --2- 172.21.15.1:0/1446705154 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fd8880a1340 0x7fd8880a37f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:47.610 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.598+0000 7fd8a4235700 1 --2- 172.21.15.1:0/1446705154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd89c110f70 0x7fd89c19c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:47.610 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.598+0000 7fd8a4235700 1 --2- 172.21.15.1:0/1446705154 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd89c068030 0x7fd89c1a2880 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:47.610 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.598+0000 7fd8a4235700 1 --2- 172.21.15.1:0/1446705154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd89c068940 0x7fd89c1a2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:47.611 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.598+0000 7fd8a4235700 1 -- 172.21.15.1:0/1446705154 >> 172.21.15.1:0/1446705154 conn(0x7fd89c0763a0 msgr2=0x7fd89c10e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:47.611 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.598+0000 7fd8a4235700 1 -- 172.21.15.1:0/1446705154 shutdown_connections 2023-11-16T16:40:47.611 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:47.598+0000 7fd8a4235700 1 -- 172.21.15.1:0/1446705154 wait complete. 2023-11-16T16:40:47.673 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:47 smithi001 ceph-mon[98800]: pgmap v51: 97 pgs: 97 active+clean; 1.8 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-11-16T16:40:47.674 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-11-16T16:40:47.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:47 smithi026 ceph-mon[100112]: pgmap v51: 97 pgs: 97 active+clean; 1.8 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 11 op/s 2023-11-16T16:40:49.075 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:49.076 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:49.229 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:49.229 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:49.259 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:49.260 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:49.260 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 43 -lt 100 ']' 2023-11-16T16:40:49.260 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 43 + 1 2023-11-16T16:40:49.306 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=44 2023-11-16T16:40:49.306 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:49.399 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:49.399 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:49.400 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:49.401 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:49.401 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:49.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:49 smithi053 ceph-mon[100638]: pgmap v52: 97 pgs: 97 active+clean; 1.8 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 8 op/s 2023-11-16T16:40:49.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:49 smithi001 ceph-mon[98800]: pgmap v52: 97 pgs: 97 active+clean; 1.8 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 8 op/s 2023-11-16T16:40:49.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:49 smithi026 ceph-mon[100112]: pgmap v52: 97 pgs: 97 active+clean; 1.8 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 8 op/s 2023-11-16T16:40:50.850 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:50.851 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:51.179 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:51.179 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:51.208 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:51.208 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:51.208 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 44 -lt 100 ']' 2023-11-16T16:40:51.209 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 44 + 1 2023-11-16T16:40:51.239 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=45 2023-11-16T16:40:51.239 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:51.318 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:51.318 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:51.320 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:51.320 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:51.320 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:51.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:51 smithi053 ceph-mon[100638]: pgmap v53: 97 pgs: 97 active+clean; 1.8 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 8 op/s 2023-11-16T16:40:51.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:51 smithi001 ceph-mon[98800]: pgmap v53: 97 pgs: 97 active+clean; 1.8 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 8 op/s 2023-11-16T16:40:51.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:51 smithi026 ceph-mon[100112]: pgmap v53: 97 pgs: 97 active+clean; 1.8 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 8 op/s 2023-11-16T16:40:51.958 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:51.959 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:52.162 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:52.162 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:52.174 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:52.174 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:52.174 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 45 -lt 100 ']' 2023-11-16T16:40:52.174 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 45 + 1 2023-11-16T16:40:52.231 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=46 2023-11-16T16:40:52.231 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:52.277 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:52.277 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:52.278 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:52.278 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:52.278 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:52.676 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.674+0000 7f25fa9b9700 1 -- 172.21.15.1:0/2170542274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25f41055c0 msgr2=0x7f25f410d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:52.676 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.674+0000 7f25fa9b9700 1 --2- 172.21.15.1:0/2170542274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25f41055c0 0x7f25f410d140 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f25ec00b0d0 tx=0x7f25ec00b3e0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.676 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.674+0000 7f25fa9b9700 1 -- 172.21.15.1:0/2170542274 shutdown_connections 2023-11-16T16:40:52.676 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.674+0000 7f25fa9b9700 1 --2- 172.21.15.1:0/2170542274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25f41055c0 0x7f25f410d140 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.677 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.674+0000 7f25fa9b9700 1 --2- 172.21.15.1:0/2170542274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25f4100cc0 0x7f25f4101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.677 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.674+0000 7f25fa9b9700 1 --2- 172.21.15.1:0/2170542274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25f4101660 0x7f25f4104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.677 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.674+0000 7f25fa9b9700 1 -- 172.21.15.1:0/2170542274 >> 172.21.15.1:0/2170542274 conn(0x7f25f4077560 msgr2=0x7f25f4077960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:52.677 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.674+0000 7f25fa9b9700 1 -- 172.21.15.1:0/2170542274 shutdown_connections 2023-11-16T16:40:52.677 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.674+0000 7f25fa9b9700 1 -- 172.21.15.1:0/2170542274 wait complete. 2023-11-16T16:40:52.679 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.675+0000 7f25fa9b9700 1 Processor -- start 2023-11-16T16:40:52.680 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.675+0000 7f25fa9b9700 1 -- start start 2023-11-16T16:40:52.680 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.676+0000 7f25fa9b9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25f4100cc0 0x7f25f4072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:52.680 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.676+0000 7f25fa9b9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25f4101660 0x7f25f4073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:52.680 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.676+0000 7f25fa9b9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25f41055c0 0x7f25f41a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:52.680 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.676+0000 7f25fa9b9700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f4073b60 con 0x7f25f41055c0 2023-11-16T16:40:52.681 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.676+0000 7f25fa9b9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f41a4c30 con 0x7f25f4100cc0 2023-11-16T16:40:52.681 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.676+0000 7f25fa9b9700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f41a4d70 con 0x7f25f4101660 2023-11-16T16:40:52.681 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.676+0000 7f25f8f56700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25f41055c0 0x7f25f41a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:52.681 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.676+0000 7f25f37fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25f4101660 0x7f25f4073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:52.681 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.676+0000 7f25f8f56700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25f41055c0 0x7f25f41a46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44354/0 (socket says 172.21.15.1:44354) 2023-11-16T16:40:52.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.676+0000 7f25f37fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25f4101660 0x7f25f4073330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:41034/0 (socket says 172.21.15.1:41034) 2023-11-16T16:40:52.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.676+0000 7f25f8f56700 1 -- 172.21.15.1:0/1658929942 learned_addr learned my addr 172.21.15.1:0/1658929942 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:52.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.676+0000 7f25f3fff700 1 --2- 172.21.15.1:0/1658929942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25f4100cc0 0x7f25f4072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:52.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.676+0000 7f25f8f56700 1 -- 172.21.15.1:0/1658929942 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25f4101660 msgr2=0x7f25f4073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:52.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.676+0000 7f25f8f56700 1 --2- 172.21.15.1:0/1658929942 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25f4101660 0x7f25f4073330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.676+0000 7f25f8f56700 1 -- 172.21.15.1:0/1658929942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25f4100cc0 msgr2=0x7f25f4072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:52.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.676+0000 7f25f8f56700 1 --2- 172.21.15.1:0/1658929942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25f4100cc0 0x7f25f4072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.676+0000 7f25f8f56700 1 -- 172.21.15.1:0/1658929942 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ec009d00 con 0x7f25f41055c0 2023-11-16T16:40:52.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.677+0000 7f25f8f56700 1 --2- 172.21.15.1:0/1658929942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25f41055c0 0x7f25f41a46f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f25ec007870 tx=0x7f25ec0078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:52.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.677+0000 7f25f17fa700 1 -- 172.21.15.1:0/1658929942 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ec00e040 con 0x7f25f41055c0 2023-11-16T16:40:52.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.677+0000 7f25fa9b9700 1 -- 172.21.15.1:0/1658929942 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25f4110340 con 0x7f25f41055c0 2023-11-16T16:40:52.684 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.677+0000 7f25f17fa700 1 -- 172.21.15.1:0/1658929942 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25ec005070 con 0x7f25f41055c0 2023-11-16T16:40:52.684 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.677+0000 7f25f17fa700 1 -- 172.21.15.1:0/1658929942 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ec01b6c0 con 0x7f25f41055c0 2023-11-16T16:40:52.684 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.677+0000 7f25fa9b9700 1 -- 172.21.15.1:0/1658929942 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25f41101b0 con 0x7f25f41055c0 2023-11-16T16:40:52.685 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.679+0000 7f25f17fa700 1 -- 172.21.15.1:0/1658929942 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f25ec0051e0 con 0x7f25f41055c0 2023-11-16T16:40:52.685 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.680+0000 7f25f17fa700 1 --2- 172.21.15.1:0/1658929942 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f25dc0a12c0 0x7f25dc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:52.685 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.680+0000 7f25f3fff700 1 --2- 172.21.15.1:0/1658929942 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f25dc0a12c0 0x7f25dc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:52.685 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.680+0000 7f25f17fa700 1 -- 172.21.15.1:0/1658929942 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f25ec0cde30 con 0x7f25f41055c0 2023-11-16T16:40:52.685 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.681+0000 7f25f3fff700 1 --2- 172.21.15.1:0/1658929942 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f25dc0a12c0 0x7f25dc0a3770 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f25e4009c60 tx=0x7f25e4009260 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:52.749 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.747+0000 7f2606b45b80 1 Processor -- start 2023-11-16T16:40:52.749 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.747+0000 7f2606b45b80 1 -- start start 2023-11-16T16:40:52.751 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.748+0000 7f2606b45b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578d37d6b60 0x5578d37d6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:52.751 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.748+0000 7f2606b45b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578d37d7470 0x5578d37db370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:52.752 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.748+0000 7f2606b45b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578d37db8b0 0x5578d37dbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:52.752 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.748+0000 7f2606b45b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578d36dc060 con 0x5578d37d6b60 2023-11-16T16:40:52.752 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.748+0000 7f2606b45b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578d372e8a0 con 0x5578d37db8b0 2023-11-16T16:40:52.752 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.748+0000 7f2606b45b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578d3727d30 con 0x5578d37d7470 2023-11-16T16:40:52.752 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.748+0000 7f25f3fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578d37d6b60 0x5578d37d6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:52.753 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.748+0000 7f25f3fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578d37d6b60 0x5578d37d6f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44358/0 (socket says 172.21.15.1:44358) 2023-11-16T16:40:52.753 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.748+0000 7f25f3fff700 1 -- 172.21.15.1:0/1419288282 learned_addr learned my addr 172.21.15.1:0/1419288282 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:52.753 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.748+0000 7f25f37fe700 1 --2- 172.21.15.1:0/1419288282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578d37d7470 0x5578d37db370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:52.753 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.748+0000 7f25f8f56700 1 --2- 172.21.15.1:0/1419288282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578d37db8b0 0x5578d37dbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:52.753 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.748+0000 7f25f3fff700 1 -- 172.21.15.1:0/1419288282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578d37d7470 msgr2=0x5578d37db370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:52.754 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.748+0000 7f25f3fff700 1 --2- 172.21.15.1:0/1419288282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578d37d7470 0x5578d37db370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.754 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.748+0000 7f25f3fff700 1 -- 172.21.15.1:0/1419288282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578d37db8b0 msgr2=0x5578d37dbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:52.754 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.748+0000 7f25f3fff700 1 --2- 172.21.15.1:0/1419288282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578d37db8b0 0x5578d37dbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.754 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.748+0000 7f25f3fff700 1 -- 172.21.15.1:0/1419288282 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578d35a9630 con 0x5578d37d6b60 2023-11-16T16:40:52.754 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.749+0000 7f25f3fff700 1 --2- 172.21.15.1:0/1419288282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578d37d6b60 0x5578d37d6f30 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f25e400e4e0 tx=0x7f25e4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee76b528ad07ddf2 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:52.754 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.749+0000 7f25da7fc700 1 -- 172.21.15.1:0/1419288282 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e400ee00 con 0x5578d37d6b60 2023-11-16T16:40:52.755 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.749+0000 7f25da7fc700 1 -- 172.21.15.1:0/1419288282 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25e4021070 con 0x5578d37d6b60 2023-11-16T16:40:52.755 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.749+0000 7f25da7fc700 1 -- 172.21.15.1:0/1419288282 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e401d7b0 con 0x5578d37d6b60 2023-11-16T16:40:52.755 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.750+0000 7f2606b45b80 1 -- 172.21.15.1:0/1419288282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578d37d6b60 msgr2=0x5578d37d6f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:52.755 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.750+0000 7f2606b45b80 1 --2- 172.21.15.1:0/1419288282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578d37d6b60 0x5578d37d6f30 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f25e400e4e0 tx=0x7f25e4014000 comp rx=0 tx=0).stop 2023-11-16T16:40:52.755 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.750+0000 7f2606b45b80 1 -- 172.21.15.1:0/1419288282 shutdown_connections 2023-11-16T16:40:52.756 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.750+0000 7f2606b45b80 1 --2- 172.21.15.1:0/1419288282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578d37db8b0 0x5578d37dbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.756 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.750+0000 7f2606b45b80 1 --2- 172.21.15.1:0/1419288282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578d37d6b60 0x5578d37d6f30 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.756 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.750+0000 7f2606b45b80 1 --2- 172.21.15.1:0/1419288282 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578d37d7470 0x5578d37db370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.756 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.750+0000 7f2606b45b80 1 -- 172.21.15.1:0/1419288282 >> 172.21.15.1:0/1419288282 conn(0x5578d37cc360 msgr2=0x5578d37d1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:52.756 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.750+0000 7f2606b45b80 1 -- 172.21.15.1:0/1419288282 shutdown_connections 2023-11-16T16:40:52.756 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.750+0000 7f2606b45b80 1 -- 172.21.15.1:0/1419288282 wait complete. 2023-11-16T16:40:52.770 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.768+0000 7f2606b45b80 1 -- start start 2023-11-16T16:40:52.773 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.769+0000 7f2606b45b80 1 Processor -- start 2023-11-16T16:40:52.773 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.769+0000 7f2606b45b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578d37d6b60 0x5578d37e30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:52.773 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.769+0000 7f2606b45b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578d37d7470 0x5578d37e0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:52.774 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.769+0000 7f2606b45b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578d37db8b0 0x5578d37e0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:52.774 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.769+0000 7f2606b45b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578d36c98e0 con 0x5578d37db8b0 2023-11-16T16:40:52.774 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.769+0000 7f2606b45b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578d36c9580 con 0x5578d37d6b60 2023-11-16T16:40:52.774 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.769+0000 7f2606b45b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578d35ab920 con 0x5578d37d7470 2023-11-16T16:40:52.774 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.769+0000 7f2606b45b80 10 client.0 _ll_register_callbacks cb 0x5578d36e7780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:40:52.775 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.769+0000 7f25f8f56700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578d37db8b0 0x5578d37e0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:52.775 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.769+0000 7f25f37fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578d37d7470 0x5578d37e0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:52.775 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.769+0000 7f25f3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578d37d6b60 0x5578d37e30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:52.775 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.769+0000 7f25f37fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578d37d7470 0x5578d37e0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:41048/0 (socket says 172.21.15.1:41048) 2023-11-16T16:40:52.775 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.770+0000 7f25f37fe700 1 -- 172.21.15.1:0/2211415674 learned_addr learned my addr 172.21.15.1:0/2211415674 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:52.776 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.769+0000 7f25f8f56700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578d37db8b0 0x5578d37e0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44368/0 (socket says 172.21.15.1:44368) 2023-11-16T16:40:52.776 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.770+0000 7f25f8f56700 1 -- 172.21.15.1:0/2211415674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578d37d7470 msgr2=0x5578d37e0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:52.776 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.770+0000 7f25f8f56700 1 --2- 172.21.15.1:0/2211415674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578d37d7470 0x5578d37e0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.776 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.770+0000 7f25f8f56700 1 -- 172.21.15.1:0/2211415674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578d37d6b60 msgr2=0x5578d37e30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:52.776 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.770+0000 7f25f8f56700 1 --2- 172.21.15.1:0/2211415674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578d37d6b60 0x5578d37e30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.776 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.770+0000 7f25f8f56700 1 -- 172.21.15.1:0/2211415674 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e4017070 con 0x5578d37db8b0 2023-11-16T16:40:52.777 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.770+0000 7f25f3fff700 1 --2- 172.21.15.1:0/2211415674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578d37d6b60 0x5578d37e30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:40:52.777 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.770+0000 7f25f8f56700 1 --2- 172.21.15.1:0/2211415674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578d37db8b0 0x5578d37e0a30 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f25ec0cb590 tx=0x7f25ec09a900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=606811449eecb66f server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:52.777 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.770+0000 7f25d37fe700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:40:52.777 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.770+0000 7f25d37fe700 1 -- 172.21.15.1:0/2211415674 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ec09b400 con 0x5578d37db8b0 2023-11-16T16:40:52.777 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.770+0000 7f25d37fe700 1 -- 172.21.15.1:0/2211415674 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f25ec0a3360 con 0x5578d37db8b0 2023-11-16T16:40:52.777 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.770+0000 7f2606b45b80 1 -- 172.21.15.1:0/2211415674 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5578d37c9b20 con 0x5578d37db8b0 2023-11-16T16:40:52.778 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.770+0000 7f25d37fe700 1 -- 172.21.15.1:0/2211415674 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25ec0b92f0 con 0x5578d37db8b0 2023-11-16T16:40:52.778 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.771+0000 7f2606b45b80 10 client.15786 fetch_fsmap learned FSMap version 23 2023-11-16T16:40:52.778 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.771+0000 7f2606b45b80 1 -- 172.21.15.1:0/2211415674 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5578d36ee190 con 0x5578d37db8b0 2023-11-16T16:40:52.778 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.771+0000 7f25d37fe700 1 -- 172.21.15.1:0/2211415674 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5578d37c9b20 con 0x5578d37db8b0 2023-11-16T16:40:52.778 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.771+0000 7f25d37fe700 1 -- 172.21.15.1:0/2211415674 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f25ec095bb0 con 0x5578d37db8b0 2023-11-16T16:40:52.779 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.771+0000 7f2606b45b80 10 client.15786 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:40:52.779 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.771+0000 7f2606b45b80 10 client.15786 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-11-16T16:40:52.779 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.771+0000 7f2606b45b80 20 client.15786 populate_metadata read hostname 'smithi001' 2023-11-16T16:40:52.779 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.771+0000 7f2606b45b80 1 --2- 172.21.15.1:0/2211415674 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x5578d37f06e0 0x5578d37f2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:52.779 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.771+0000 7f2606b45b80 4 client.15786 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:40:52.779 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.771+0000 7f2606b45b80 1 -- 172.21.15.1:0/2211415674 --> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5578d36ea0a0 con 0x5578d37f06e0 2023-11-16T16:40:52.780 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.771+0000 7f25f3fff700 1 --2- 172.21.15.1:0/2211415674 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x5578d37f06e0 0x5578d37f2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:52.781 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.772+0000 7f25f3fff700 1 --2- 172.21.15.1:0/2211415674 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x5578d37f06e0 0x5578d37f2af0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5e5174b6f246c93e server_cookie=c91ce58cebd4e08a in_seq=0 out_seq=0 2023-11-16T16:40:52.781 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.772+0000 7f25d37fe700 10 client.15786 ms_handle_connect on v2:172.21.15.53:6838/1044753024 2023-11-16T16:40:52.781 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.772+0000 7f25d37fe700 1 -- 172.21.15.1:0/2211415674 <== mds.1 v2:172.21.15.53:6838/1044753024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5578d36ea0a0 con 0x5578d37f06e0 2023-11-16T16:40:52.781 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.772+0000 7f25d37fe700 10 client.15786 handle_command_reply: tid=0 2023-11-16T16:40:52.782 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.772+0000 7f2606b45b80 1 client.15786 shutdown 2023-11-16T16:40:52.782 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.773+0000 7f2606b45b80 1 -- 172.21.15.1:0/2211415674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578d37db8b0 msgr2=0x5578d37e0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:52.782 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.773+0000 7f2606b45b80 1 --2- 172.21.15.1:0/2211415674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578d37db8b0 0x5578d37e0a30 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f25ec0cb590 tx=0x7f25ec09a900 comp rx=0 tx=0).stop 2023-11-16T16:40:52.782 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.773+0000 7f2606b45b80 1 -- 172.21.15.1:0/2211415674 shutdown_connections 2023-11-16T16:40:52.782 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.773+0000 7f2606b45b80 1 --2- 172.21.15.1:0/2211415674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578d37db8b0 0x5578d37e0a30 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.783 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.773+0000 7f2606b45b80 1 --2- 172.21.15.1:0/2211415674 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x5578d37f06e0 0x5578d37f2af0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.783 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.773+0000 7f2606b45b80 1 --2- 172.21.15.1:0/2211415674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578d37d6b60 0x5578d37e30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.783 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.773+0000 7f2606b45b80 1 --2- 172.21.15.1:0/2211415674 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578d37d7470 0x5578d37e0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.783 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.773+0000 7f25d37fe700 0 client.15786 ms_handle_reset on v2:172.21.15.53:6838/1044753024 2023-11-16T16:40:52.783 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.773+0000 7f2606b45b80 1 -- 172.21.15.1:0/2211415674 >> 172.21.15.1:0/2211415674 conn(0x5578d37cc360 msgr2=0x5578d37dcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:52.783 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.773+0000 7f2606b45b80 1 -- 172.21.15.1:0/2211415674 shutdown_connections 2023-11-16T16:40:52.784 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.773+0000 7f2606b45b80 1 -- 172.21.15.1:0/2211415674 wait complete. 2023-11-16T16:40:52.784 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.773+0000 7f2606b45b80 20 client.15786 trim_cache size 0 max 16384 2023-11-16T16:40:52.791 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.787+0000 7f2606b45b80 1 Processor -- start 2023-11-16T16:40:52.791 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.787+0000 7f2606b45b80 1 -- start start 2023-11-16T16:40:52.791 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.787+0000 7f2606b45b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578d37d6b60 0x5578d37d91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:52.791 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.787+0000 7f2606b45b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578d37d7470 0x5578d37e2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:52.791 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.787+0000 7f2606b45b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578d37db8b0 0x5578d37ddc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:52.792 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.787+0000 7f2606b45b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578d37d3340 con 0x5578d37d6b60 2023-11-16T16:40:52.792 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.787+0000 7f2606b45b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578d37c9750 con 0x5578d37db8b0 2023-11-16T16:40:52.792 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.787+0000 7f2606b45b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578d37da540 con 0x5578d37d7470 2023-11-16T16:40:52.792 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.787+0000 7f25f3fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578d37d6b60 0x5578d37d91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:52.792 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.787+0000 7f25f3fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578d37d6b60 0x5578d37d91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44372/0 (socket says 172.21.15.1:44372) 2023-11-16T16:40:52.793 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.787+0000 7f25f3fff700 1 -- 172.21.15.1:0/2848174558 learned_addr learned my addr 172.21.15.1:0/2848174558 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:52.793 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.788+0000 7f25f37fe700 1 --2- 172.21.15.1:0/2848174558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578d37d7470 0x5578d37e2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:52.793 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.788+0000 7f25f8f56700 1 --2- 172.21.15.1:0/2848174558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578d37db8b0 0x5578d37ddc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:52.793 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.788+0000 7f25f3fff700 1 -- 172.21.15.1:0/2848174558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578d37d7470 msgr2=0x5578d37e2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:52.793 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.788+0000 7f25f3fff700 1 --2- 172.21.15.1:0/2848174558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578d37d7470 0x5578d37e2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.793 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.788+0000 7f25f3fff700 1 -- 172.21.15.1:0/2848174558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578d37db8b0 msgr2=0x5578d37ddc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:52.794 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.788+0000 7f25f3fff700 1 --2- 172.21.15.1:0/2848174558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578d37db8b0 0x5578d37ddc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.794 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.788+0000 7f25f3fff700 1 -- 172.21.15.1:0/2848174558 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578d37cdb00 con 0x5578d37d6b60 2023-11-16T16:40:52.794 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.788+0000 7f25f37fe700 1 --2- 172.21.15.1:0/2848174558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578d37d7470 0x5578d37e2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:40:52.794 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.788+0000 7f25f8f56700 1 --2- 172.21.15.1:0/2848174558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578d37db8b0 0x5578d37ddc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:40:52.794 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.788+0000 7f25f3fff700 1 --2- 172.21.15.1:0/2848174558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578d37d6b60 0x5578d37d91a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f25e400e480 tx=0x7f25e4009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a05991562f5a3a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:52.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.788+0000 7f25daffd700 1 -- 172.21.15.1:0/2848174558 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e400ed80 con 0x5578d37d6b60 2023-11-16T16:40:52.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.788+0000 7f25daffd700 1 -- 172.21.15.1:0/2848174558 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25e4033170 con 0x5578d37d6b60 2023-11-16T16:40:52.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.788+0000 7f25daffd700 1 -- 172.21.15.1:0/2848174558 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e4003680 con 0x5578d37d6b60 2023-11-16T16:40:52.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.789+0000 7f2606b45b80 1 -- 172.21.15.1:0/2848174558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578d37d6b60 msgr2=0x5578d37d91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:52.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.789+0000 7f2606b45b80 1 --2- 172.21.15.1:0/2848174558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578d37d6b60 0x5578d37d91a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f25e400e480 tx=0x7f25e4009e50 comp rx=0 tx=0).stop 2023-11-16T16:40:52.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.789+0000 7f2606b45b80 1 -- 172.21.15.1:0/2848174558 shutdown_connections 2023-11-16T16:40:52.796 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.789+0000 7f2606b45b80 1 --2- 172.21.15.1:0/2848174558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578d37db8b0 0x5578d37ddc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.796 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.789+0000 7f2606b45b80 1 --2- 172.21.15.1:0/2848174558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578d37d6b60 0x5578d37d91a0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.796 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.789+0000 7f2606b45b80 1 --2- 172.21.15.1:0/2848174558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578d37d7470 0x5578d37e2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.796 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.789+0000 7f2606b45b80 1 -- 172.21.15.1:0/2848174558 >> 172.21.15.1:0/2848174558 conn(0x5578d37cc360 msgr2=0x5578d37d9a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:52.796 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.789+0000 7f2606b45b80 1 -- 172.21.15.1:0/2848174558 shutdown_connections 2023-11-16T16:40:52.796 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.789+0000 7f2606b45b80 1 -- 172.21.15.1:0/2848174558 wait complete. 2023-11-16T16:40:52.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.789+0000 7f2606b45b80 1 -- start start 2023-11-16T16:40:52.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.790+0000 7f2606b45b80 1 Processor -- start 2023-11-16T16:40:52.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.790+0000 7f2606b45b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578d37d6b60 0x5578d37e7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:52.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.790+0000 7f2606b45b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578d37d7470 0x5578d37e5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:52.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.790+0000 7f2606b45b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578d37db8b0 0x5578d37e5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:52.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.790+0000 7f2606b45b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578d37e65a0 con 0x5578d37d7470 2023-11-16T16:40:52.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.790+0000 7f2606b45b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578d37e6710 con 0x5578d37db8b0 2023-11-16T16:40:52.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.790+0000 7f2606b45b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578d38435a0 con 0x5578d37d6b60 2023-11-16T16:40:52.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.790+0000 7f2606b45b80 10 client.0 _ll_register_callbacks cb 0x7f25ec09ad70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:40:52.802 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.790+0000 7f25f37fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578d37d7470 0x5578d37e5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:52.802 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.790+0000 7f25f37fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578d37d7470 0x5578d37e5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44380/0 (socket says 172.21.15.1:44380) 2023-11-16T16:40:52.802 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.790+0000 7f25f37fe700 1 -- 172.21.15.1:0/640960277 learned_addr learned my addr 172.21.15.1:0/640960277 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:52.802 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.790+0000 7f25f3fff700 1 --2- 172.21.15.1:0/640960277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578d37d6b60 0x5578d37e7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:52.802 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.790+0000 7f25f8f56700 1 --2- 172.21.15.1:0/640960277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578d37db8b0 0x5578d37e5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:52.803 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.791+0000 7f25f37fe700 1 -- 172.21.15.1:0/640960277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578d37d6b60 msgr2=0x5578d37e7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:52.803 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.791+0000 7f25f37fe700 1 --2- 172.21.15.1:0/640960277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578d37d6b60 0x5578d37e7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.803 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.791+0000 7f25f37fe700 1 -- 172.21.15.1:0/640960277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578d37db8b0 msgr2=0x5578d37e5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:52.803 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.791+0000 7f25f37fe700 1 --2- 172.21.15.1:0/640960277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578d37db8b0 0x5578d37e5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.803 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.791+0000 7f25f37fe700 1 -- 172.21.15.1:0/640960277 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e402a040 con 0x5578d37d7470 2023-11-16T16:40:52.803 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.791+0000 7f25f3fff700 1 --2- 172.21.15.1:0/640960277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578d37d6b60 0x5578d37e7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:40:52.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.791+0000 7f25f8f56700 1 --2- 172.21.15.1:0/640960277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578d37db8b0 0x5578d37e5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:40:52.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.791+0000 7f25f37fe700 1 --2- 172.21.15.1:0/640960277 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578d37d7470 0x5578d37e5990 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x5578d37e3620 tx=0x7f25e8004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a1f062e6d23fe93 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:52.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.791+0000 7f25d37fe700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:40:52.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.791+0000 7f25d37fe700 1 -- 172.21.15.1:0/640960277 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e8020070 con 0x5578d37d7470 2023-11-16T16:40:52.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.791+0000 7f25d37fe700 1 -- 172.21.15.1:0/640960277 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f25e8003a90 con 0x5578d37d7470 2023-11-16T16:40:52.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.791+0000 7f2606b45b80 1 -- 172.21.15.1:0/640960277 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25e4021070 con 0x5578d37d7470 2023-11-16T16:40:52.805 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.791+0000 7f25d37fe700 1 -- 172.21.15.1:0/640960277 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25e8003c50 con 0x5578d37d7470 2023-11-16T16:40:52.805 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.791+0000 7f2606b45b80 10 client.15792 fetch_fsmap learned FSMap version 23 2023-11-16T16:40:52.805 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.791+0000 7f2606b45b80 1 -- 172.21.15.1:0/640960277 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5578d3555de0 con 0x5578d37d7470 2023-11-16T16:40:52.805 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.791+0000 7f25d37fe700 1 -- 172.21.15.1:0/640960277 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25e4021070 con 0x5578d37d7470 2023-11-16T16:40:52.805 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.792+0000 7f25d37fe700 1 -- 172.21.15.1:0/640960277 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f25e8023020 con 0x5578d37d7470 2023-11-16T16:40:52.806 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.792+0000 7f2606b45b80 10 client.15792 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:40:52.806 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.792+0000 7f2606b45b80 10 client.15792 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-11-16T16:40:52.806 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.792+0000 7f2606b45b80 20 client.15792 populate_metadata read hostname 'smithi001' 2023-11-16T16:40:52.806 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.792+0000 7f2606b45b80 1 --2- 172.21.15.1:0/640960277 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x5578d3844bf0 0x5578d3847000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:52.806 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.792+0000 7f2606b45b80 4 client.15792 mds_command: new command op to 24493 tid=0[{"prefix": "damage ls"}] 2023-11-16T16:40:52.806 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.792+0000 7f2606b45b80 1 -- 172.21.15.1:0/640960277 --> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5578d36f6660 con 0x5578d3844bf0 2023-11-16T16:40:52.807 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.792+0000 7f25f3fff700 1 --2- 172.21.15.1:0/640960277 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x5578d3844bf0 0x5578d3847000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:52.807 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.792+0000 7f25f3fff700 1 --2- 172.21.15.1:0/640960277 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x5578d3844bf0 0x5578d3847000 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d500a8f3e8a91d82 server_cookie=1710f3d3d7ac5098 in_seq=0 out_seq=0 2023-11-16T16:40:52.807 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.792+0000 7f25d37fe700 10 client.15792 ms_handle_connect on v2:172.21.15.53:6838/1044753024 2023-11-16T16:40:52.807 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.793+0000 7f25d37fe700 1 -- 172.21.15.1:0/640960277 <== mds.1 v2:172.21.15.53:6838/1044753024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5578d36f6660 con 0x5578d3844bf0 2023-11-16T16:40:52.807 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.793+0000 7f25d37fe700 10 client.15792 handle_command_reply: tid=0 2023-11-16T16:40:52.808 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.793+0000 7f2606b45b80 1 client.15792 shutdown 2023-11-16T16:40:52.808 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.793+0000 7f2606b45b80 1 -- 172.21.15.1:0/640960277 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578d37d7470 msgr2=0x5578d37e5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:52.808 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.793+0000 7f2606b45b80 1 --2- 172.21.15.1:0/640960277 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578d37d7470 0x5578d37e5990 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x5578d37e3620 tx=0x7f25e8004320 comp rx=0 tx=0).stop 2023-11-16T16:40:52.808 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.793+0000 7f2606b45b80 1 -- 172.21.15.1:0/640960277 shutdown_connections 2023-11-16T16:40:52.808 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.793+0000 7f2606b45b80 1 --2- 172.21.15.1:0/640960277 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x5578d3844bf0 0x5578d3847000 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.808 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.793+0000 7f2606b45b80 1 --2- 172.21.15.1:0/640960277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5578d37db8b0 0x5578d37e5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.809 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.793+0000 7f2606b45b80 1 --2- 172.21.15.1:0/640960277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5578d37d6b60 0x5578d37e7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.809 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.793+0000 7f25d37fe700 0 client.15792 ms_handle_reset on v2:172.21.15.53:6838/1044753024 2023-11-16T16:40:52.809 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.793+0000 7f2606b45b80 1 --2- 172.21.15.1:0/640960277 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578d37d7470 0x5578d37e5990 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.809 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.793+0000 7f2606b45b80 1 -- 172.21.15.1:0/640960277 >> 172.21.15.1:0/640960277 conn(0x5578d37cc360 msgr2=0x5578d37ddc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:52.809 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.793+0000 7f2606b45b80 1 -- 172.21.15.1:0/640960277 shutdown_connections 2023-11-16T16:40:52.810 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.793+0000 7f2606b45b80 1 -- 172.21.15.1:0/640960277 wait complete. 2023-11-16T16:40:52.810 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.793+0000 7f2606b45b80 20 client.15792 trim_cache size 0 max 16384 2023-11-16T16:40:52.810 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2023-11-16T16:40:52.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.793+0000 7f25fa9b9700 1 -- 172.21.15.1:0/1658929942 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f25dc0a12c0 msgr2=0x7f25dc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:52.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.793+0000 7f25fa9b9700 1 --2- 172.21.15.1:0/1658929942 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f25dc0a12c0 0x7f25dc0a3770 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f25e4009c60 tx=0x7f25e4009260 comp rx=0 tx=0).stop 2023-11-16T16:40:52.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.793+0000 7f25fa9b9700 1 -- 172.21.15.1:0/1658929942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25f41055c0 msgr2=0x7f25f41a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:52.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.793+0000 7f25fa9b9700 1 --2- 172.21.15.1:0/1658929942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25f41055c0 0x7f25f41a46f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f25ec007870 tx=0x7f25ec0078a0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.794+0000 7f25fa9b9700 1 -- 172.21.15.1:0/1658929942 shutdown_connections 2023-11-16T16:40:52.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.794+0000 7f25fa9b9700 1 --2- 172.21.15.1:0/1658929942 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f25dc0a12c0 0x7f25dc0a3770 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.794+0000 7f25fa9b9700 1 --2- 172.21.15.1:0/1658929942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25f41055c0 0x7f25f41a46f0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.794+0000 7f25fa9b9700 1 --2- 172.21.15.1:0/1658929942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25f4100cc0 0x7f25f4072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.794+0000 7f25fa9b9700 1 --2- 172.21.15.1:0/1658929942 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f25f4101660 0x7f25f4073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:52.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.794+0000 7f25fa9b9700 1 -- 172.21.15.1:0/1658929942 >> 172.21.15.1:0/1658929942 conn(0x7f25f4077560 msgr2=0x7f25f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:52.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.794+0000 7f25fa9b9700 1 -- 172.21.15.1:0/1658929942 shutdown_connections 2023-11-16T16:40:52.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:52.794+0000 7f25fa9b9700 1 -- 172.21.15.1:0/1658929942 wait complete. 2023-11-16T16:40:52.814 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:52.814 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:52.882 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-16T16:40:53.012 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:53.013 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:53.031 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:53.032 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:53.032 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 46 -lt 100 ']' 2023-11-16T16:40:53.032 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 46 + 1 2023-11-16T16:40:53.069 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=47 2023-11-16T16:40:53.069 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:53.076 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:53.076 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:53.077 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:53.077 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:53.077 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:53.466 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:53.467 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:53.575 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:53.576 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:53.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:53 smithi053 ceph-mon[100638]: pgmap v54: 97 pgs: 97 active+clean; 1.8 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 11 op/s 2023-11-16T16:40:53.625 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:53.625 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:53.625 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 47 -lt 100 ']' 2023-11-16T16:40:53.625 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 47 + 1 2023-11-16T16:40:53.660 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=48 2023-11-16T16:40:53.661 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:53.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:53 smithi001 ceph-mon[98800]: pgmap v54: 97 pgs: 97 active+clean; 1.8 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 11 op/s 2023-11-16T16:40:53.714 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:53.715 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:53.716 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:53.716 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:53.716 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:53.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:53 smithi026 ceph-mon[100112]: pgmap v54: 97 pgs: 97 active+clean; 1.8 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 11 op/s 2023-11-16T16:40:54.166 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:54.166 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:54.370 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:54.370 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:54.390 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:54.390 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:54.390 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 48 -lt 100 ']' 2023-11-16T16:40:54.390 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 48 + 1 2023-11-16T16:40:54.422 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=49 2023-11-16T16:40:54.423 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:54.471 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:54.471 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:54.472 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:54.472 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:54.472 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:55.175 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:55.175 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:55.505 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:55.505 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:55.527 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:55.527 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:55.527 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 49 -lt 100 ']' 2023-11-16T16:40:55.527 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 49 + 1 2023-11-16T16:40:55.572 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=50 2023-11-16T16:40:55.572 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:55.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:55 smithi053 ceph-mon[100638]: pgmap v55: 97 pgs: 97 active+clean; 1.8 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 7 op/s 2023-11-16T16:40:55.638 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:55.638 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:55.639 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:55.639 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:55.639 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:55.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:55 smithi001 ceph-mon[98800]: pgmap v55: 97 pgs: 97 active+clean; 1.8 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 7 op/s 2023-11-16T16:40:55.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:55 smithi026 ceph-mon[100112]: pgmap v55: 97 pgs: 97 active+clean; 1.8 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 7 op/s 2023-11-16T16:40:55.968 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:55.965+0000 7f3b7f7a7700 1 -- 172.21.15.1:0/789710721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b781015f0 msgr2=0x7f3b78104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:55.968 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:55.965+0000 7f3b7f7a7700 1 --2- 172.21.15.1:0/789710721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b781015f0 0x7f3b78104e80 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c0099c0 tx=0x7f3b6c009cd0 comp rx=0 tx=0).stop 2023-11-16T16:40:55.968 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:55.965+0000 7f3b7f7a7700 1 -- 172.21.15.1:0/789710721 shutdown_connections 2023-11-16T16:40:55.968 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:55.965+0000 7f3b7f7a7700 1 --2- 172.21.15.1:0/789710721 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b78105560 0x7f3b7810d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:55.969 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:55.965+0000 7f3b7f7a7700 1 --2- 172.21.15.1:0/789710721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b78100c50 0x7f3b78101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:55.969 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:55.965+0000 7f3b7f7a7700 1 --2- 172.21.15.1:0/789710721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b781015f0 0x7f3b78104e80 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:55.969 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:55.965+0000 7f3b7f7a7700 1 -- 172.21.15.1:0/789710721 >> 172.21.15.1:0/789710721 conn(0x7f3b78077500 msgr2=0x7f3b78077900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:55.969 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:55.966+0000 7f3b7f7a7700 1 -- 172.21.15.1:0/789710721 shutdown_connections 2023-11-16T16:40:55.969 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:55.966+0000 7f3b7f7a7700 1 -- 172.21.15.1:0/789710721 wait complete. 2023-11-16T16:40:55.972 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:55.966+0000 7f3b7f7a7700 1 Processor -- start 2023-11-16T16:40:55.973 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:55.966+0000 7f3b7f7a7700 1 -- start start 2023-11-16T16:40:55.973 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:55.967+0000 7f3b7f7a7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b78100c50 0x7f3b781985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:55.973 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:55.967+0000 7f3b7f7a7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b781015f0 0x7f3b78198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:55.973 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:55.967+0000 7f3b7f7a7700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b78105560 0x7f3b7819cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:55.973 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:55.967+0000 7f3b7f7a7700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7819d590 con 0x7f3b78105560 2023-11-16T16:40:55.973 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:55.967+0000 7f3b7f7a7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7819d700 con 0x7f3b78100c50 2023-11-16T16:40:55.974 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:55.967+0000 7f3b7f7a7700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7819d8c0 con 0x7f3b781015f0 2023-11-16T16:40:55.974 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:55.967+0000 7f3b7dd44700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b78105560 0x7f3b7819cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:55.974 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:55.967+0000 7f3b7dd44700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b78105560 0x7f3b7819cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44396/0 (socket says 172.21.15.1:44396) 2023-11-16T16:40:55.974 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:55.967+0000 7f3b7dd44700 1 -- 172.21.15.1:0/3260108123 learned_addr learned my addr 172.21.15.1:0/3260108123 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:55.974 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:55.967+0000 7f3b7d543700 1 --2- 172.21.15.1:0/3260108123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b78100c50 0x7f3b781985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:55.974 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:55.967+0000 7f3b7cd42700 1 --2- 172.21.15.1:0/3260108123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b781015f0 0x7f3b78198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:55.974 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:55.967+0000 7f3b7dd44700 1 -- 172.21.15.1:0/3260108123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b781015f0 msgr2=0x7f3b78198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:55.975 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:55.967+0000 7f3b7dd44700 1 --2- 172.21.15.1:0/3260108123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b781015f0 0x7f3b78198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:55.975 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:55.967+0000 7f3b7dd44700 1 -- 172.21.15.1:0/3260108123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b78100c50 msgr2=0x7f3b781985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:55.975 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:55.967+0000 7f3b7dd44700 1 --2- 172.21.15.1:0/3260108123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b78100c50 0x7f3b781985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:55.975 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:55.967+0000 7f3b7dd44700 1 -- 172.21.15.1:0/3260108123 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b70009520 con 0x7f3b78105560 2023-11-16T16:40:55.975 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:55.968+0000 7f3b7dd44700 1 --2- 172.21.15.1:0/3260108123 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b78105560 0x7f3b7819cf50 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3b7000ea80 tx=0x7f3b7000ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:55.975 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:55.968+0000 7f3b767fc700 1 -- 172.21.15.1:0/3260108123 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b700098a0 con 0x7f3b78105560 2023-11-16T16:40:55.975 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:55.968+0000 7f3b767fc700 1 -- 172.21.15.1:0/3260108123 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b7000ce10 con 0x7f3b78105560 2023-11-16T16:40:55.976 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:55.968+0000 7f3b7f7a7700 1 -- 172.21.15.1:0/3260108123 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b6c0095f0 con 0x7f3b78105560 2023-11-16T16:40:55.976 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:55.968+0000 7f3b767fc700 1 -- 172.21.15.1:0/3260108123 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b70003dc0 con 0x7f3b78105560 2023-11-16T16:40:55.976 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:55.968+0000 7f3b7f7a7700 1 -- 172.21.15.1:0/3260108123 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b781102d0 con 0x7f3b78105560 2023-11-16T16:40:55.976 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:55.970+0000 7f3b767fc700 1 -- 172.21.15.1:0/3260108123 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f3b7000c950 con 0x7f3b78105560 2023-11-16T16:40:55.977 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:55.971+0000 7f3b767fc700 1 --2- 172.21.15.1:0/3260108123 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f3b600a1340 0x7f3b600a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:55.977 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:55.971+0000 7f3b7d543700 1 --2- 172.21.15.1:0/3260108123 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f3b600a1340 0x7f3b600a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:55.977 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:55.971+0000 7f3b767fc700 1 -- 172.21.15.1:0/3260108123 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f3b70014070 con 0x7f3b78105560 2023-11-16T16:40:55.977 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:55.972+0000 7f3b7d543700 1 --2- 172.21.15.1:0/3260108123 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f3b600a1340 0x7f3b600a37f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3b6800f540 tx=0x7f3b6800b040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:56.036 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.031+0000 7f3b8b933b80 1 Processor -- start 2023-11-16T16:40:56.036 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.031+0000 7f3b8b933b80 1 -- start start 2023-11-16T16:40:56.036 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.031+0000 7f3b8b933b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606ea2e9b60 0x5606ea2e9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:56.036 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.031+0000 7f3b8b933b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606ea2ea470 0x5606ea2ee370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:56.036 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.031+0000 7f3b8b933b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606ea2ee8b0 0x5606ea2eed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:56.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.031+0000 7f3b8b933b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606ea1ef060 con 0x5606ea2e9b60 2023-11-16T16:40:56.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.031+0000 7f3b8b933b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606ea2418a0 con 0x5606ea2ea470 2023-11-16T16:40:56.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.031+0000 7f3b8b933b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606ea23ad30 con 0x5606ea2ee8b0 2023-11-16T16:40:56.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.031+0000 7f3b7d543700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606ea2e9b60 0x5606ea2e9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:56.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.031+0000 7f3b7d543700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606ea2e9b60 0x5606ea2e9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44400/0 (socket says 172.21.15.1:44400) 2023-11-16T16:40:56.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.031+0000 7f3b7d543700 1 -- 172.21.15.1:0/1047469515 learned_addr learned my addr 172.21.15.1:0/1047469515 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:56.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.031+0000 7f3b7dd44700 1 --2- 172.21.15.1:0/1047469515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606ea2ee8b0 0x5606ea2eed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:56.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.032+0000 7f3b7cd42700 1 --2- 172.21.15.1:0/1047469515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606ea2ea470 0x5606ea2ee370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:56.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.032+0000 7f3b7dd44700 1 -- 172.21.15.1:0/1047469515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606ea2ea470 msgr2=0x5606ea2ee370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:56.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.032+0000 7f3b7dd44700 1 --2- 172.21.15.1:0/1047469515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606ea2ea470 0x5606ea2ee370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:56.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.032+0000 7f3b7dd44700 1 -- 172.21.15.1:0/1047469515 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606ea2e9b60 msgr2=0x5606ea2e9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:56.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.032+0000 7f3b7dd44700 1 --2- 172.21.15.1:0/1047469515 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606ea2e9b60 0x5606ea2e9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:56.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.032+0000 7f3b7dd44700 1 -- 172.21.15.1:0/1047469515 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606ea0bc630 con 0x5606ea2ee8b0 2023-11-16T16:40:56.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.032+0000 7f3b7d543700 1 --2- 172.21.15.1:0/1047469515 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606ea2e9b60 0x5606ea2e9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:40:56.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.032+0000 7f3b7dd44700 1 --2- 172.21.15.1:0/1047469515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606ea2ee8b0 0x5606ea2eed00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3b700ca640 tx=0x7f3b70099bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50aa61f301610295 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:56.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.032+0000 7f3b5f7fe700 1 -- 172.21.15.1:0/1047469515 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b70099fc0 con 0x5606ea2ee8b0 2023-11-16T16:40:56.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.032+0000 7f3b5f7fe700 1 -- 172.21.15.1:0/1047469515 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b7009dff0 con 0x5606ea2ee8b0 2023-11-16T16:40:56.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.032+0000 7f3b5f7fe700 1 -- 172.21.15.1:0/1047469515 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7009a220 con 0x5606ea2ee8b0 2023-11-16T16:40:56.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.033+0000 7f3b8b933b80 1 -- 172.21.15.1:0/1047469515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606ea2ee8b0 msgr2=0x5606ea2eed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:56.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.033+0000 7f3b8b933b80 1 --2- 172.21.15.1:0/1047469515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606ea2ee8b0 0x5606ea2eed00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3b700ca640 tx=0x7f3b70099bd0 comp rx=0 tx=0).stop 2023-11-16T16:40:56.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.033+0000 7f3b8b933b80 1 -- 172.21.15.1:0/1047469515 shutdown_connections 2023-11-16T16:40:56.040 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.033+0000 7f3b8b933b80 1 --2- 172.21.15.1:0/1047469515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606ea2ee8b0 0x5606ea2eed00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:56.040 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.033+0000 7f3b8b933b80 1 --2- 172.21.15.1:0/1047469515 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606ea2e9b60 0x5606ea2e9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:56.040 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.033+0000 7f3b8b933b80 1 --2- 172.21.15.1:0/1047469515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606ea2ea470 0x5606ea2ee370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:56.040 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.033+0000 7f3b8b933b80 1 -- 172.21.15.1:0/1047469515 >> 172.21.15.1:0/1047469515 conn(0x5606ea2df360 msgr2=0x5606ea2e4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:56.040 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.033+0000 7f3b8b933b80 1 -- 172.21.15.1:0/1047469515 shutdown_connections 2023-11-16T16:40:56.040 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.033+0000 7f3b8b933b80 1 -- 172.21.15.1:0/1047469515 wait complete. 2023-11-16T16:40:56.040 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.034+0000 7f3b8b933b80 1 -- start start 2023-11-16T16:40:56.041 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.034+0000 7f3b8b933b80 1 Processor -- start 2023-11-16T16:40:56.044 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.035+0000 7f3b8b933b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606ea2e9b60 0x5606ea2f60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:56.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.035+0000 7f3b8b933b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606ea2ea470 0x5606ea2f3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:56.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.035+0000 7f3b8b933b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606ea2ee8b0 0x5606ea2f3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:56.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.035+0000 7f3b8b933b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606ea1dc8e0 con 0x5606ea2ea470 2023-11-16T16:40:56.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.035+0000 7f3b8b933b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606ea1dc580 con 0x5606ea2ee8b0 2023-11-16T16:40:56.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.035+0000 7f3b8b933b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606ea0be920 con 0x5606ea2e9b60 2023-11-16T16:40:56.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.035+0000 7f3b8b933b80 10 client.0 _ll_register_callbacks cb 0x5606ea1fa780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:40:56.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.035+0000 7f3b7cd42700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606ea2ea470 0x5606ea2f3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:56.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.035+0000 7f3b7cd42700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606ea2ea470 0x5606ea2f3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44408/0 (socket says 172.21.15.1:44408) 2023-11-16T16:40:56.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.035+0000 7f3b7cd42700 1 -- 172.21.15.1:0/1798342022 learned_addr learned my addr 172.21.15.1:0/1798342022 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:56.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.035+0000 7f3b7d543700 1 --2- 172.21.15.1:0/1798342022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606ea2e9b60 0x5606ea2f60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:56.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.035+0000 7f3b7dd44700 1 --2- 172.21.15.1:0/1798342022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606ea2ee8b0 0x5606ea2f3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:56.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.036+0000 7f3b7cd42700 1 -- 172.21.15.1:0/1798342022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606ea2e9b60 msgr2=0x5606ea2f60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:56.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.036+0000 7f3b7cd42700 1 --2- 172.21.15.1:0/1798342022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606ea2e9b60 0x5606ea2f60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:56.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.036+0000 7f3b7cd42700 1 -- 172.21.15.1:0/1798342022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606ea2ee8b0 msgr2=0x5606ea2f3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:56.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.036+0000 7f3b7cd42700 1 --2- 172.21.15.1:0/1798342022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606ea2ee8b0 0x5606ea2f3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:56.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.036+0000 7f3b7cd42700 1 -- 172.21.15.1:0/1798342022 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b781102d0 con 0x5606ea2ea470 2023-11-16T16:40:56.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.036+0000 7f3b7dd44700 1 --2- 172.21.15.1:0/1798342022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606ea2ee8b0 0x5606ea2f3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:40:56.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.036+0000 7f3b7cd42700 1 --2- 172.21.15.1:0/1798342022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606ea2ea470 0x5606ea2f3340 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c000c00 tx=0x7f3b6c005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff4df3e8305c52b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:56.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.036+0000 7f3b5cff9700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:40:56.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.036+0000 7f3b5cff9700 1 -- 172.21.15.1:0/1798342022 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c005590 con 0x5606ea2ea470 2023-11-16T16:40:56.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.036+0000 7f3b5cff9700 1 -- 172.21.15.1:0/1798342022 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f3b6c00b790 con 0x5606ea2ea470 2023-11-16T16:40:56.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.036+0000 7f3b8b933b80 1 -- 172.21.15.1:0/1798342022 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5606ea2dcb20 con 0x5606ea2ea470 2023-11-16T16:40:56.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.036+0000 7f3b5cff9700 1 -- 172.21.15.1:0/1798342022 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b6c0056f0 con 0x5606ea2ea470 2023-11-16T16:40:56.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.036+0000 7f3b8b933b80 10 client.15804 fetch_fsmap learned FSMap version 23 2023-11-16T16:40:56.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.037+0000 7f3b8b933b80 1 -- 172.21.15.1:0/1798342022 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5606ea201190 con 0x5606ea2ea470 2023-11-16T16:40:56.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.037+0000 7f3b5cff9700 1 -- 172.21.15.1:0/1798342022 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b6c031020 con 0x5606ea2ea470 2023-11-16T16:40:56.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.037+0000 7f3b5cff9700 1 -- 172.21.15.1:0/1798342022 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3b6c00fa00 con 0x5606ea2ea470 2023-11-16T16:40:56.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.037+0000 7f3b8b933b80 10 client.15804 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:40:56.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.037+0000 7f3b8b933b80 10 client.15804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:40:56.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.037+0000 7f3b8b933b80 20 client.15804 populate_metadata read hostname 'smithi001' 2023-11-16T16:40:56.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.037+0000 7f3b8b933b80 1 --2- 172.21.15.1:0/1798342022 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5606ea303970 0x5606ea305d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:56.049 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.037+0000 7f3b8b933b80 4 client.15804 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:40:56.049 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.037+0000 7f3b8b933b80 1 -- 172.21.15.1:0/1798342022 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5606ea1fd0a0 con 0x5606ea303970 2023-11-16T16:40:56.049 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.037+0000 7f3b7d543700 1 --2- 172.21.15.1:0/1798342022 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5606ea303970 0x5606ea305d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:56.049 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.038+0000 7f3b5cff9700 10 client.15804 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:40:56.049 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.038+0000 7f3b7d543700 1 --2- 172.21.15.1:0/1798342022 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5606ea303970 0x5606ea305d80 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=717facf2a469d243 server_cookie=b9cb196922b920c4 in_seq=0 out_seq=0 2023-11-16T16:40:56.049 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.039+0000 7f3b5cff9700 1 -- 172.21.15.1:0/1798342022 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5606ea1fd0a0 con 0x5606ea303970 2023-11-16T16:40:56.049 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.039+0000 7f3b5cff9700 10 client.15804 handle_command_reply: tid=0 2023-11-16T16:40:56.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.039+0000 7f3b8b933b80 1 client.15804 shutdown 2023-11-16T16:40:56.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.039+0000 7f3b8b933b80 1 -- 172.21.15.1:0/1798342022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606ea2ea470 msgr2=0x5606ea2f3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:56.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.039+0000 7f3b8b933b80 1 --2- 172.21.15.1:0/1798342022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606ea2ea470 0x5606ea2f3340 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c000c00 tx=0x7f3b6c005bc0 comp rx=0 tx=0).stop 2023-11-16T16:40:56.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.039+0000 7f3b8b933b80 1 -- 172.21.15.1:0/1798342022 shutdown_connections 2023-11-16T16:40:56.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.039+0000 7f3b8b933b80 1 --2- 172.21.15.1:0/1798342022 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5606ea303970 0x5606ea305d80 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:56.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.039+0000 7f3b8b933b80 1 --2- 172.21.15.1:0/1798342022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606ea2ee8b0 0x5606ea2f3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:56.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.039+0000 7f3b5cff9700 0 client.15804 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:40:56.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.039+0000 7f3b8b933b80 1 --2- 172.21.15.1:0/1798342022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606ea2e9b60 0x5606ea2f60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:56.051 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.039+0000 7f3b8b933b80 1 --2- 172.21.15.1:0/1798342022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606ea2ea470 0x5606ea2f3340 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:56.051 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.039+0000 7f3b8b933b80 1 -- 172.21.15.1:0/1798342022 >> 172.21.15.1:0/1798342022 conn(0x5606ea2df360 msgr2=0x5606ea2efd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:56.051 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.039+0000 7f3b8b933b80 1 -- 172.21.15.1:0/1798342022 shutdown_connections 2023-11-16T16:40:56.051 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.039+0000 7f3b8b933b80 1 -- 172.21.15.1:0/1798342022 wait complete. 2023-11-16T16:40:56.051 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.039+0000 7f3b8b933b80 20 client.15804 trim_cache size 0 max 16384 2023-11-16T16:40:56.061 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.056+0000 7f3b8b933b80 1 Processor -- start 2023-11-16T16:40:56.062 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.056+0000 7f3b8b933b80 1 -- start start 2023-11-16T16:40:56.062 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.057+0000 7f3b8b933b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606ea2e9b60 0x5606ea2fa9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:56.062 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.057+0000 7f3b8b933b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606ea2ea470 0x5606ea2ec970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:56.062 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.057+0000 7f3b8b933b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606ea2ee8b0 0x5606ea2eceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:56.062 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.057+0000 7f3b8b933b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606ea2eabc0 con 0x5606ea2ee8b0 2023-11-16T16:40:56.062 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.057+0000 7f3b8b933b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606ea2deee0 con 0x5606ea2ea470 2023-11-16T16:40:56.062 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.057+0000 7f3b8b933b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606ea2f18a0 con 0x5606ea2e9b60 2023-11-16T16:40:56.062 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.057+0000 7f3b7dd44700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606ea2ee8b0 0x5606ea2eceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:56.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.057+0000 7f3b7dd44700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606ea2ee8b0 0x5606ea2eceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44418/0 (socket says 172.21.15.1:44418) 2023-11-16T16:40:56.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.057+0000 7f3b7dd44700 1 -- 172.21.15.1:0/846375516 learned_addr learned my addr 172.21.15.1:0/846375516 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:56.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.057+0000 7f3b7d543700 1 --2- 172.21.15.1:0/846375516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606ea2e9b60 0x5606ea2fa9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:56.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.057+0000 7f3b7cd42700 1 --2- 172.21.15.1:0/846375516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606ea2ea470 0x5606ea2ec970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:56.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.057+0000 7f3b7dd44700 1 -- 172.21.15.1:0/846375516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606ea2e9b60 msgr2=0x5606ea2fa9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:56.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.057+0000 7f3b7dd44700 1 --2- 172.21.15.1:0/846375516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606ea2e9b60 0x5606ea2fa9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:56.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.057+0000 7f3b7dd44700 1 -- 172.21.15.1:0/846375516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606ea2ea470 msgr2=0x5606ea2ec970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:56.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.057+0000 7f3b7dd44700 1 --2- 172.21.15.1:0/846375516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606ea2ea470 0x5606ea2ec970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:56.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.057+0000 7f3b7dd44700 1 -- 172.21.15.1:0/846375516 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606ea2e0b00 con 0x5606ea2ee8b0 2023-11-16T16:40:56.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.058+0000 7f3b7dd44700 1 --2- 172.21.15.1:0/846375516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606ea2ee8b0 0x5606ea2eceb0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f3b70099f50 tx=0x7f3b7000cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ee5927a488b8e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:56.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.058+0000 7f3b5ffff700 1 -- 172.21.15.1:0/846375516 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b70099fc0 con 0x5606ea2ee8b0 2023-11-16T16:40:56.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.058+0000 7f3b5ffff700 1 -- 172.21.15.1:0/846375516 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b700a2410 con 0x5606ea2ee8b0 2023-11-16T16:40:56.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.058+0000 7f3b5ffff700 1 -- 172.21.15.1:0/846375516 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b700b82c0 con 0x5606ea2ee8b0 2023-11-16T16:40:56.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.059+0000 7f3b8b933b80 1 -- 172.21.15.1:0/846375516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606ea2ee8b0 msgr2=0x5606ea2eceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:56.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.059+0000 7f3b8b933b80 1 --2- 172.21.15.1:0/846375516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606ea2ee8b0 0x5606ea2eceb0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f3b70099f50 tx=0x7f3b7000cf80 comp rx=0 tx=0).stop 2023-11-16T16:40:56.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.059+0000 7f3b8b933b80 1 -- 172.21.15.1:0/846375516 shutdown_connections 2023-11-16T16:40:56.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.059+0000 7f3b8b933b80 1 --2- 172.21.15.1:0/846375516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606ea2ee8b0 0x5606ea2eceb0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:56.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.059+0000 7f3b8b933b80 1 --2- 172.21.15.1:0/846375516 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606ea2e9b60 0x5606ea2fa9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:56.065 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.059+0000 7f3b8b933b80 1 --2- 172.21.15.1:0/846375516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606ea2ea470 0x5606ea2ec970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:56.065 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.059+0000 7f3b8b933b80 1 -- 172.21.15.1:0/846375516 >> 172.21.15.1:0/846375516 conn(0x5606ea2df360 msgr2=0x5606ea2ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:56.065 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.059+0000 7f3b8b933b80 1 -- 172.21.15.1:0/846375516 shutdown_connections 2023-11-16T16:40:56.065 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.059+0000 7f3b8b933b80 1 -- 172.21.15.1:0/846375516 wait complete. 2023-11-16T16:40:56.065 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.059+0000 7f3b8b933b80 1 -- start start 2023-11-16T16:40:56.065 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.059+0000 7f3b8b933b80 1 Processor -- start 2023-11-16T16:40:56.069 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.060+0000 7f3b8b933b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606ea2e9b60 0x5606ea2fa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:56.069 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.060+0000 7f3b8b933b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606ea2ea470 0x5606ea2eb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:56.070 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.060+0000 7f3b8b933b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606ea2ee8b0 0x5606ea2ebcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:56.070 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.060+0000 7f3b8b933b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606ea2ec310 con 0x5606ea2ee8b0 2023-11-16T16:40:56.070 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.060+0000 7f3b8b933b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606ea2ec480 con 0x5606ea2e9b60 2023-11-16T16:40:56.070 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.060+0000 7f3b8b933b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606ea2f1af0 con 0x5606ea2ea470 2023-11-16T16:40:56.070 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.060+0000 7f3b8b933b80 10 client.0 _ll_register_callbacks cb 0x7f3b6c003c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:40:56.070 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.060+0000 7f3b7dd44700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606ea2ee8b0 0x5606ea2ebcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:56.070 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.060+0000 7f3b7d543700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606ea2e9b60 0x5606ea2fa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:56.070 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.060+0000 7f3b7cd42700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606ea2ea470 0x5606ea2eb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:56.071 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.060+0000 7f3b7d543700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606ea2e9b60 0x5606ea2fa9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.1:38606/0 (socket says 172.21.15.1:38606) 2023-11-16T16:40:56.071 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.060+0000 7f3b7d543700 1 -- 172.21.15.1:0/1475623833 learned_addr learned my addr 172.21.15.1:0/1475623833 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:56.071 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.061+0000 7f3b7d543700 1 -- 172.21.15.1:0/1475623833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606ea2ea470 msgr2=0x5606ea2eb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:56.071 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.061+0000 7f3b7d543700 1 --2- 172.21.15.1:0/1475623833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606ea2ea470 0x5606ea2eb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:56.071 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.061+0000 7f3b7d543700 1 -- 172.21.15.1:0/1475623833 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606ea2ee8b0 msgr2=0x5606ea2ebcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:56.071 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.061+0000 7f3b7d543700 1 --2- 172.21.15.1:0/1475623833 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606ea2ee8b0 0x5606ea2ebcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:56.071 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.061+0000 7f3b7d543700 1 -- 172.21.15.1:0/1475623833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5606ea0bc630 con 0x5606ea2e9b60 2023-11-16T16:40:56.071 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.061+0000 7f3b7d543700 1 --2- 172.21.15.1:0/1475623833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606ea2e9b60 0x5606ea2fa9b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3b6800fc90 tx=0x7f3b68004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=314d126c30b54d16 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:56.071 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.061+0000 7f3b5cff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-11-16T16:40:56.071 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.061+0000 7f3b5cff9700 1 -- 172.21.15.1:0/1475623833 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6800e040 con 0x5606ea2e9b60 2023-11-16T16:40:56.072 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.061+0000 7f3b8b933b80 1 -- 172.21.15.1:0/1475623833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b7009dff0 con 0x5606ea2e9b60 2023-11-16T16:40:56.072 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.061+0000 7f3b5cff9700 1 -- 172.21.15.1:0/1475623833 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f3b680155b0 con 0x5606ea2e9b60 2023-11-16T16:40:56.072 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.061+0000 7f3b5cff9700 1 -- 172.21.15.1:0/1475623833 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b68015770 con 0x5606ea2e9b60 2023-11-16T16:40:56.072 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.061+0000 7f3b8b933b80 10 client.25391 fetch_fsmap learned FSMap version 23 2023-11-16T16:40:56.072 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.061+0000 7f3b8b933b80 1 -- 172.21.15.1:0/1475623833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5606ea200ee0 con 0x5606ea2e9b60 2023-11-16T16:40:56.072 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.062+0000 7f3b5cff9700 1 -- 172.21.15.1:0/1475623833 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b6803c020 con 0x5606ea2e9b60 2023-11-16T16:40:56.072 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.062+0000 7f3b5cff9700 1 -- 172.21.15.1:0/1475623833 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3b68008c10 con 0x5606ea2e9b60 2023-11-16T16:40:56.072 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.062+0000 7f3b8b933b80 10 client.25391 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:40:56.072 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.062+0000 7f3b8b933b80 10 client.25391 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:40:56.073 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.062+0000 7f3b8b933b80 20 client.25391 populate_metadata read hostname 'smithi001' 2023-11-16T16:40:56.073 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.062+0000 7f3b8b933b80 1 --2- 172.21.15.1:0/1475623833 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5606ea357f10 0x5606ea35a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:56.073 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.062+0000 7f3b8b933b80 4 client.25391 mds_command: new command op to 24521 tid=0[{"prefix": "damage ls"}] 2023-11-16T16:40:56.073 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.062+0000 7f3b8b933b80 1 -- 172.21.15.1:0/1475623833 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5606ea209660 con 0x5606ea357f10 2023-11-16T16:40:56.073 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.062+0000 7f3b7cd42700 1 --2- 172.21.15.1:0/1475623833 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5606ea357f10 0x5606ea35a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:56.073 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.063+0000 7f3b5cff9700 10 client.25391 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:40:56.073 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.063+0000 7f3b7cd42700 1 --2- 172.21.15.1:0/1475623833 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5606ea357f10 0x5606ea35a320 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d86f56bf89b91ef server_cookie=bbe78e214ead081e in_seq=0 out_seq=0 2023-11-16T16:40:56.073 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.063+0000 7f3b5cff9700 1 -- 172.21.15.1:0/1475623833 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5606ea209660 con 0x5606ea357f10 2023-11-16T16:40:56.073 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.063+0000 7f3b5cff9700 10 client.25391 handle_command_reply: tid=0 2023-11-16T16:40:56.073 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.063+0000 7f3b8b933b80 1 client.25391 shutdown 2023-11-16T16:40:56.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.063+0000 7f3b8b933b80 1 -- 172.21.15.1:0/1475623833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606ea2e9b60 msgr2=0x5606ea2fa9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:56.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.063+0000 7f3b8b933b80 1 --2- 172.21.15.1:0/1475623833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606ea2e9b60 0x5606ea2fa9b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3b6800fc90 tx=0x7f3b68004b40 comp rx=0 tx=0).stop 2023-11-16T16:40:56.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.063+0000 7f3b8b933b80 1 -- 172.21.15.1:0/1475623833 shutdown_connections 2023-11-16T16:40:56.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.063+0000 7f3b8b933b80 1 --2- 172.21.15.1:0/1475623833 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5606ea357f10 0x5606ea35a320 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:56.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.063+0000 7f3b8b933b80 1 --2- 172.21.15.1:0/1475623833 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606ea2ee8b0 0x5606ea2ebcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:56.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.063+0000 7f3b8b933b80 1 --2- 172.21.15.1:0/1475623833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5606ea2e9b60 0x5606ea2fa9b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:56.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.063+0000 7f3b5cff9700 0 client.25391 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:40:56.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.063+0000 7f3b8b933b80 1 --2- 172.21.15.1:0/1475623833 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5606ea2ea470 0x5606ea2eb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:56.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.063+0000 7f3b8b933b80 1 -- 172.21.15.1:0/1475623833 >> 172.21.15.1:0/1475623833 conn(0x5606ea2df360 msgr2=0x5606ea2ec970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:56.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.063+0000 7f3b8b933b80 1 -- 172.21.15.1:0/1475623833 shutdown_connections 2023-11-16T16:40:56.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.063+0000 7f3b8b933b80 1 -- 172.21.15.1:0/1475623833 wait complete. 2023-11-16T16:40:56.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.063+0000 7f3b8b933b80 20 client.25391 trim_cache size 0 max 16384 2023-11-16T16:40:56.075 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2023-11-16T16:40:56.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.064+0000 7f3b7f7a7700 1 -- 172.21.15.1:0/3260108123 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f3b600a1340 msgr2=0x7f3b600a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:56.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.064+0000 7f3b7f7a7700 1 --2- 172.21.15.1:0/3260108123 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f3b600a1340 0x7f3b600a37f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3b6800f540 tx=0x7f3b6800b040 comp rx=0 tx=0).stop 2023-11-16T16:40:56.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.064+0000 7f3b7f7a7700 1 -- 172.21.15.1:0/3260108123 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b78105560 msgr2=0x7f3b7819cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:56.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.064+0000 7f3b7f7a7700 1 --2- 172.21.15.1:0/3260108123 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b78105560 0x7f3b7819cf50 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3b7000ea80 tx=0x7f3b7000ed90 comp rx=0 tx=0).stop 2023-11-16T16:40:56.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.064+0000 7f3b7f7a7700 1 -- 172.21.15.1:0/3260108123 shutdown_connections 2023-11-16T16:40:56.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.064+0000 7f3b7f7a7700 1 --2- 172.21.15.1:0/3260108123 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f3b600a1340 0x7f3b600a37f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:56.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.064+0000 7f3b7f7a7700 1 --2- 172.21.15.1:0/3260108123 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b78105560 0x7f3b7819cf50 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:56.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.064+0000 7f3b7f7a7700 1 --2- 172.21.15.1:0/3260108123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b78100c50 0x7f3b781985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:56.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.064+0000 7f3b7f7a7700 1 --2- 172.21.15.1:0/3260108123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f3b781015f0 0x7f3b78198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:56.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.064+0000 7f3b7f7a7700 1 -- 172.21.15.1:0/3260108123 >> 172.21.15.1:0/3260108123 conn(0x7f3b78077500 msgr2=0x7f3b780fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:56.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.064+0000 7f3b7f7a7700 1 -- 172.21.15.1:0/3260108123 shutdown_connections 2023-11-16T16:40:56.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:56.064+0000 7f3b7f7a7700 1 -- 172.21.15.1:0/3260108123 wait complete. 2023-11-16T16:40:56.308 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:56.308 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:56.455 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:56.455 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:56.473 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:56.473 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:56.474 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 50 -lt 100 ']' 2023-11-16T16:40:56.474 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 50 + 1 2023-11-16T16:40:56.509 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=51 2023-11-16T16:40:56.509 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:56.540 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:56.540 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:56.542 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:56.542 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:56.542 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:56.891 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:56.891 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:56.994 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:56.995 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:57.006 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:57.007 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:57.007 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 51 -lt 100 ']' 2023-11-16T16:40:57.007 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 51 + 1 2023-11-16T16:40:57.042 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=52 2023-11-16T16:40:57.042 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:57.072 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:57.072 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:57.073 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:57.073 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:57.073 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:57.158 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-16T16:40:57.159 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-16T16:40:57.541 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:57.541 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:57.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:57 smithi053 ceph-mon[100638]: pgmap v56: 97 pgs: 97 active+clean; 1.9 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 10 op/s 2023-11-16T16:40:57.586 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:57.586 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:57.591 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:57.591 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:57.592 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 52 -lt 100 ']' 2023-11-16T16:40:57.592 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 52 + 1 2023-11-16T16:40:57.605 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=53 2023-11-16T16:40:57.605 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:57.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:57 smithi001 ceph-mon[98800]: pgmap v56: 97 pgs: 97 active+clean; 1.9 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 10 op/s 2023-11-16T16:40:57.684 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:57.685 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:57.686 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:57.686 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:57.686 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:57.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:57 smithi026 ceph-mon[100112]: pgmap v56: 97 pgs: 97 active+clean; 1.9 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 10 op/s 2023-11-16T16:40:58.124 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:58.124 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:58.303 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:58.303 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:58.315 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:58.315 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:58.315 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 53 -lt 100 ']' 2023-11-16T16:40:58.315 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 53 + 1 2023-11-16T16:40:58.346 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=54 2023-11-16T16:40:58.346 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:58.389 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:58.389 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:58.390 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:58.390 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:58.390 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:59.032 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:59.033 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:59.361 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:40:59.361 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:40:59.385 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:59.386 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:40:59.386 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 54 -lt 100 ']' 2023-11-16T16:40:59.386 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 54 + 1 2023-11-16T16:40:59.421 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=55 2023-11-16T16:40:59.422 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:40:59.446 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:40:59.447 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:40:59.447 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:40:59.448 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:40:59.448 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:40:59.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:40:59 smithi001 ceph-mon[98800]: pgmap v57: 97 pgs: 97 active+clean; 1.9 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 6 op/s 2023-11-16T16:40:59.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:40:59 smithi026 ceph-mon[100112]: pgmap v57: 97 pgs: 97 active+clean; 1.9 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 6 op/s 2023-11-16T16:40:59.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.798+0000 7f5c81187700 1 -- 172.21.15.1:0/4000785901 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c7c0ff050 msgr2=0x7f5c7c0ff420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:59.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.798+0000 7f5c81187700 1 --2- 172.21.15.1:0/4000785901 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c7c0ff050 0x7f5c7c0ff420 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c009990 tx=0x7f5c6c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:40:59.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.798+0000 7f5c81187700 1 -- 172.21.15.1:0/4000785901 shutdown_connections 2023-11-16T16:40:59.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.798+0000 7f5c81187700 1 --2- 172.21.15.1:0/4000785901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c7c10cc00 0x7f5c7c10efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:59.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.798+0000 7f5c81187700 1 --2- 172.21.15.1:0/4000785901 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c7c0ff050 0x7f5c7c0ff420 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:59.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.798+0000 7f5c81187700 1 --2- 172.21.15.1:0/4000785901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c7c0ff960 0x7f5c7c10c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:59.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.798+0000 7f5c81187700 1 -- 172.21.15.1:0/4000785901 >> 172.21.15.1:0/4000785901 conn(0x7f5c7c0763a0 msgr2=0x7f5c7c0767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:59.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.798+0000 7f5c81187700 1 -- 172.21.15.1:0/4000785901 shutdown_connections 2023-11-16T16:40:59.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.798+0000 7f5c81187700 1 -- 172.21.15.1:0/4000785901 wait complete. 2023-11-16T16:40:59.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.799+0000 7f5c81187700 1 Processor -- start 2023-11-16T16:40:59.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.799+0000 7f5c81187700 1 -- start start 2023-11-16T16:40:59.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.800+0000 7f5c81187700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c7c0ff050 0x7f5c7c1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:59.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.800+0000 7f5c81187700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c7c0ff960 0x7f5c7c198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:59.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.800+0000 7f5c81187700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c7c10cc00 0x7f5c7c19cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:59.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.800+0000 7f5c81187700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c19d5a0 con 0x7f5c7c0ff960 2023-11-16T16:40:59.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.800+0000 7f5c81187700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c19d710 con 0x7f5c7c10cc00 2023-11-16T16:40:59.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.800+0000 7f5c81187700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c19d8d0 con 0x7f5c7c0ff050 2023-11-16T16:40:59.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.800+0000 7f5c7a59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c7c0ff960 0x7f5c7c198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:59.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.800+0000 7f5c7ad9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c7c0ff050 0x7f5c7c1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:59.805 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.800+0000 7f5c7a59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c7c0ff960 0x7f5c7c198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44436/0 (socket says 172.21.15.1:44436) 2023-11-16T16:40:59.805 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.800+0000 7f5c7a59c700 1 -- 172.21.15.1:0/1007629511 learned_addr learned my addr 172.21.15.1:0/1007629511 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:59.805 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.800+0000 7f5c7b59e700 1 --2- 172.21.15.1:0/1007629511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c7c10cc00 0x7f5c7c19cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:59.805 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.800+0000 7f5c7a59c700 1 -- 172.21.15.1:0/1007629511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c7c0ff050 msgr2=0x7f5c7c1985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:59.805 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.800+0000 7f5c7a59c700 1 --2- 172.21.15.1:0/1007629511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c7c0ff050 0x7f5c7c1985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:59.805 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.800+0000 7f5c7a59c700 1 -- 172.21.15.1:0/1007629511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c7c10cc00 msgr2=0x7f5c7c19cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:59.805 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.800+0000 7f5c7a59c700 1 --2- 172.21.15.1:0/1007629511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c7c10cc00 0x7f5c7c19cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:59.805 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.800+0000 7f5c7a59c700 1 -- 172.21.15.1:0/1007629511 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c6c0095f0 con 0x7f5c7c0ff960 2023-11-16T16:40:59.805 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.801+0000 7f5c7a59c700 1 --2- 172.21.15.1:0/1007629511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c7c0ff960 0x7f5c7c198b00 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000e9a0 tx=0x7f5c7000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:59.805 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.801+0000 7f5c63fff700 1 -- 172.21.15.1:0/1007629511 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c70009790 con 0x7f5c7c0ff960 2023-11-16T16:40:59.805 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.801+0000 7f5c81187700 1 -- 172.21.15.1:0/1007629511 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c7c112510 con 0x7f5c7c0ff960 2023-11-16T16:40:59.806 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.801+0000 7f5c63fff700 1 -- 172.21.15.1:0/1007629511 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c70010430 con 0x7f5c7c0ff960 2023-11-16T16:40:59.806 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.801+0000 7f5c63fff700 1 -- 172.21.15.1:0/1007629511 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c70018680 con 0x7f5c7c0ff960 2023-11-16T16:40:59.806 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.801+0000 7f5c81187700 1 -- 172.21.15.1:0/1007629511 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c7c112380 con 0x7f5c7c0ff960 2023-11-16T16:40:59.806 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.804+0000 7f5c63fff700 1 -- 172.21.15.1:0/1007629511 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f5c700105a0 con 0x7f5c7c0ff960 2023-11-16T16:40:59.807 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.804+0000 7f5c63fff700 1 --2- 172.21.15.1:0/1007629511 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5c640a12c0 0x7f5c640a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:59.807 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.804+0000 7f5c63fff700 1 -- 172.21.15.1:0/1007629511 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f5c70014070 con 0x7f5c7c0ff960 2023-11-16T16:40:59.807 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.804+0000 7f5c7ad9d700 1 --2- 172.21.15.1:0/1007629511 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5c640a12c0 0x7f5c640a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:59.807 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.805+0000 7f5c7ad9d700 1 --2- 172.21.15.1:0/1007629511 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5c640a12c0 0x7f5c640a3770 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00b2e0 tx=0x7f5c6c005fd0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:59.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:40:59 smithi053 ceph-mon[100638]: pgmap v57: 97 pgs: 97 active+clean; 1.9 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 6 op/s 2023-11-16T16:40:59.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.888+0000 7f5c8d313b80 1 Processor -- start 2023-11-16T16:40:59.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.888+0000 7f5c8d313b80 1 -- start start 2023-11-16T16:40:59.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.888+0000 7f5c8d313b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a41086be0 0x558a41086fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:59.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.888+0000 7f5c8d313b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a410874f0 0x558a4108b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:59.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.888+0000 7f5c8d313b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a4108b930 0x558a4108bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:59.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.888+0000 7f5c8d313b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a40f8c0e0 con 0x558a410874f0 2023-11-16T16:40:59.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.888+0000 7f5c8d313b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a40fde920 con 0x558a4108b930 2023-11-16T16:40:59.891 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.888+0000 7f5c8d313b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a40fd7db0 con 0x558a41086be0 2023-11-16T16:40:59.891 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.888+0000 7f5c7a59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a410874f0 0x558a4108b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:59.891 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.888+0000 7f5c7a59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a410874f0 0x558a4108b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44444/0 (socket says 172.21.15.1:44444) 2023-11-16T16:40:59.891 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.888+0000 7f5c7ad9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a41086be0 0x558a41086fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:59.891 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.888+0000 7f5c7b59e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a4108b930 0x558a4108bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:59.891 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.888+0000 7f5c7a59c700 1 -- 172.21.15.1:0/2243125460 learned_addr learned my addr 172.21.15.1:0/2243125460 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:59.894 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.889+0000 7f5c7ad9d700 1 -- 172.21.15.1:0/2243125460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a4108b930 msgr2=0x558a4108bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:59.894 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.889+0000 7f5c7ad9d700 1 --2- 172.21.15.1:0/2243125460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a4108b930 0x558a4108bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:59.894 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.889+0000 7f5c7ad9d700 1 -- 172.21.15.1:0/2243125460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a410874f0 msgr2=0x558a4108b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:59.894 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.889+0000 7f5c7ad9d700 1 --2- 172.21.15.1:0/2243125460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a410874f0 0x558a4108b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:59.895 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.889+0000 7f5c7ad9d700 1 -- 172.21.15.1:0/2243125460 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a40e596b0 con 0x558a41086be0 2023-11-16T16:40:59.895 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.889+0000 7f5c7a59c700 1 --2- 172.21.15.1:0/2243125460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a410874f0 0x558a4108b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:40:59.895 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.889+0000 7f5c7ad9d700 1 --2- 172.21.15.1:0/2243125460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a41086be0 0x558a41086fb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00b6f0 tx=0x7f5c6c020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f373fd5aba40c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:59.895 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.889+0000 7f5c617fa700 1 -- 172.21.15.1:0/2243125460 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c02b070 con 0x558a41086be0 2023-11-16T16:40:59.895 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.889+0000 7f5c617fa700 1 -- 172.21.15.1:0/2243125460 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c6c01e040 con 0x558a41086be0 2023-11-16T16:40:59.895 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.889+0000 7f5c617fa700 1 -- 172.21.15.1:0/2243125460 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c00f480 con 0x558a41086be0 2023-11-16T16:40:59.895 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.890+0000 7f5c8d313b80 1 -- 172.21.15.1:0/2243125460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a41086be0 msgr2=0x558a41086fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:59.895 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.890+0000 7f5c8d313b80 1 --2- 172.21.15.1:0/2243125460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a41086be0 0x558a41086fb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00b6f0 tx=0x7f5c6c020000 comp rx=0 tx=0).stop 2023-11-16T16:40:59.895 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.890+0000 7f5c8d313b80 1 -- 172.21.15.1:0/2243125460 shutdown_connections 2023-11-16T16:40:59.895 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.890+0000 7f5c8d313b80 1 --2- 172.21.15.1:0/2243125460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a4108b930 0x558a4108bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:59.895 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.890+0000 7f5c8d313b80 1 --2- 172.21.15.1:0/2243125460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a41086be0 0x558a41086fb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:59.895 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.890+0000 7f5c8d313b80 1 --2- 172.21.15.1:0/2243125460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a410874f0 0x558a4108b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:59.895 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.890+0000 7f5c8d313b80 1 -- 172.21.15.1:0/2243125460 >> 172.21.15.1:0/2243125460 conn(0x558a4107c3e0 msgr2=0x558a41081bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:59.896 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.890+0000 7f5c8d313b80 1 -- 172.21.15.1:0/2243125460 shutdown_connections 2023-11-16T16:40:59.896 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.890+0000 7f5c8d313b80 1 -- 172.21.15.1:0/2243125460 wait complete. 2023-11-16T16:40:59.896 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.891+0000 7f5c8d313b80 1 -- start start 2023-11-16T16:40:59.896 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.891+0000 7f5c8d313b80 1 Processor -- start 2023-11-16T16:40:59.896 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.891+0000 7f5c8d313b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a41086be0 0x558a41093100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:59.896 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.891+0000 7f5c8d313b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a410874f0 0x558a41090380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:59.896 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.891+0000 7f5c8d313b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a4108b930 0x558a410909b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:59.896 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.891+0000 7f5c8d313b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a40f79960 con 0x558a41086be0 2023-11-16T16:40:59.896 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.891+0000 7f5c8d313b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a40f79600 con 0x558a410874f0 2023-11-16T16:40:59.896 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.891+0000 7f5c8d313b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a40e5b9a0 con 0x558a4108b930 2023-11-16T16:40:59.896 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.891+0000 7f5c8d313b80 10 client.0 _ll_register_callbacks cb 0x558a40f97800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:40:59.896 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.891+0000 7f5c7ad9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a41086be0 0x558a41093100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:59.896 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.891+0000 7f5c7ad9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a41086be0 0x558a41093100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44448/0 (socket says 172.21.15.1:44448) 2023-11-16T16:40:59.896 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.891+0000 7f5c7ad9d700 1 -- 172.21.15.1:0/2944476896 learned_addr learned my addr 172.21.15.1:0/2944476896 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:59.897 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.891+0000 7f5c7b59e700 1 --2- 172.21.15.1:0/2944476896 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a4108b930 0x558a410909b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:59.897 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.891+0000 7f5c7a59c700 1 --2- 172.21.15.1:0/2944476896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a410874f0 0x558a41090380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:59.897 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.892+0000 7f5c7b59e700 1 -- 172.21.15.1:0/2944476896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a410874f0 msgr2=0x558a41090380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:59.897 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.892+0000 7f5c7b59e700 1 --2- 172.21.15.1:0/2944476896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a410874f0 0x558a41090380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:59.897 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.892+0000 7f5c7b59e700 1 -- 172.21.15.1:0/2944476896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a41086be0 msgr2=0x558a41093100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:59.897 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.892+0000 7f5c7b59e700 1 --2- 172.21.15.1:0/2944476896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a41086be0 0x558a41093100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:59.897 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.892+0000 7f5c7b59e700 1 -- 172.21.15.1:0/2944476896 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c7c10fb60 con 0x558a4108b930 2023-11-16T16:40:59.897 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.892+0000 7f5c7ad9d700 1 --2- 172.21.15.1:0/2944476896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a41086be0 0x558a41093100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:40:59.897 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.892+0000 7f5c7a59c700 1 --2- 172.21.15.1:0/2944476896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a410874f0 0x558a41090380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:40:59.897 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.892+0000 7f5c7b59e700 1 --2- 172.21.15.1:0/2944476896 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a4108b930 0x558a410909b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c101230 tx=0x7f5c74014600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be75b9a5cc0a6c8a server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:59.897 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.892+0000 7f5c5eff5700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-16T16:40:59.897 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.892+0000 7f5c5eff5700 1 -- 172.21.15.1:0/2944476896 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7400e040 con 0x558a4108b930 2023-11-16T16:40:59.897 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.892+0000 7f5c5eff5700 1 -- 172.21.15.1:0/2944476896 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f5c74004bb0 con 0x558a4108b930 2023-11-16T16:40:59.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.892+0000 7f5c8d313b80 1 -- 172.21.15.1:0/2944476896 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558a41079ba0 con 0x558a4108b930 2023-11-16T16:40:59.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.892+0000 7f5c5eff5700 1 -- 172.21.15.1:0/2944476896 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c74004d70 con 0x558a4108b930 2023-11-16T16:40:59.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.893+0000 7f5c8d313b80 10 client.25393 fetch_fsmap learned FSMap version 23 2023-11-16T16:40:59.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.893+0000 7f5c8d313b80 1 -- 172.21.15.1:0/2944476896 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x558a40f9e210 con 0x558a4108b930 2023-11-16T16:40:59.899 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.893+0000 7f5c5eff5700 1 -- 172.21.15.1:0/2944476896 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558a41079ba0 con 0x558a4108b930 2023-11-16T16:40:59.899 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.893+0000 7f5c5eff5700 1 -- 172.21.15.1:0/2944476896 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5c7401c030 con 0x558a4108b930 2023-11-16T16:40:59.899 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.893+0000 7f5c8d313b80 10 client.25393 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:40:59.899 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.893+0000 7f5c8d313b80 10 client.25393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:40:59.899 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.893+0000 7f5c8d313b80 20 client.25393 populate_metadata read hostname 'smithi001' 2023-11-16T16:40:59.899 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.893+0000 7f5c8d313b80 1 --2- 172.21.15.1:0/2944476896 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x558a410a0870 0x558a410a2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:59.899 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.893+0000 7f5c8d313b80 4 client.25393 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:40:59.899 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.893+0000 7f5c8d313b80 1 -- 172.21.15.1:0/2944476896 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558a40f9a120 con 0x558a410a0870 2023-11-16T16:40:59.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.893+0000 7f5c7ad9d700 1 --2- 172.21.15.1:0/2944476896 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x558a410a0870 0x558a410a2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:59.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.894+0000 7f5c5eff5700 10 client.25393 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:40:59.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.894+0000 7f5c7ad9d700 1 --2- 172.21.15.1:0/2944476896 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x558a410a0870 0x558a410a2c80 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=411ecb729503affe server_cookie=95927d2cb2689afa in_seq=0 out_seq=0 2023-11-16T16:40:59.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.895+0000 7f5c5eff5700 1 -- 172.21.15.1:0/2944476896 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558a40f9a120 con 0x558a410a0870 2023-11-16T16:40:59.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.895+0000 7f5c5eff5700 10 client.25393 handle_command_reply: tid=0 2023-11-16T16:40:59.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.895+0000 7f5c8d313b80 1 client.25393 shutdown 2023-11-16T16:40:59.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.895+0000 7f5c8d313b80 1 -- 172.21.15.1:0/2944476896 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a4108b930 msgr2=0x558a410909b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:59.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.895+0000 7f5c8d313b80 1 --2- 172.21.15.1:0/2944476896 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a4108b930 0x558a410909b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c101230 tx=0x7f5c74014600 comp rx=0 tx=0).stop 2023-11-16T16:40:59.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.895+0000 7f5c8d313b80 1 -- 172.21.15.1:0/2944476896 shutdown_connections 2023-11-16T16:40:59.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.895+0000 7f5c8d313b80 1 --2- 172.21.15.1:0/2944476896 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x558a410a0870 0x558a410a2c80 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:59.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.895+0000 7f5c8d313b80 1 --2- 172.21.15.1:0/2944476896 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a4108b930 0x558a410909b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:59.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.895+0000 7f5c8d313b80 1 --2- 172.21.15.1:0/2944476896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a41086be0 0x558a41093100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:59.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.895+0000 7f5c8d313b80 1 --2- 172.21.15.1:0/2944476896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a410874f0 0x558a41090380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:59.901 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.895+0000 7f5c5eff5700 0 client.25393 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:40:59.901 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.895+0000 7f5c8d313b80 1 -- 172.21.15.1:0/2944476896 >> 172.21.15.1:0/2944476896 conn(0x558a4107c3e0 msgr2=0x558a41080e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:59.901 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.895+0000 7f5c8d313b80 1 -- 172.21.15.1:0/2944476896 shutdown_connections 2023-11-16T16:40:59.901 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.895+0000 7f5c8d313b80 1 -- 172.21.15.1:0/2944476896 wait complete. 2023-11-16T16:40:59.901 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.895+0000 7f5c8d313b80 20 client.25393 trim_cache size 0 max 16384 2023-11-16T16:40:59.915 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.913+0000 7f5c8d313b80 1 Processor -- start 2023-11-16T16:40:59.915 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.913+0000 7f5c8d313b80 1 -- start start 2023-11-16T16:40:59.917 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.914+0000 7f5c8d313b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a41086be0 0x558a410860f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:59.917 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.914+0000 7f5c8d313b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a410874f0 0x558a41092ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:59.917 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.914+0000 7f5c8d313b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a4108b930 0x558a4108fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:59.917 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.914+0000 7f5c8d313b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a4108a630 con 0x558a4108b930 2023-11-16T16:40:59.917 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.914+0000 7f5c8d313b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a4109df80 con 0x558a41086be0 2023-11-16T16:40:59.917 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.914+0000 7f5c8d313b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a4107bf60 con 0x558a410874f0 2023-11-16T16:40:59.917 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.914+0000 7f5c7b59e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a4108b930 0x558a4108fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:59.917 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.914+0000 7f5c7b59e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a4108b930 0x558a4108fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44460/0 (socket says 172.21.15.1:44460) 2023-11-16T16:40:59.917 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.914+0000 7f5c7b59e700 1 -- 172.21.15.1:0/781322229 learned_addr learned my addr 172.21.15.1:0/781322229 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:59.917 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.914+0000 7f5c7a59c700 1 --2- 172.21.15.1:0/781322229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a410874f0 0x558a41092ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:59.917 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.914+0000 7f5c7ad9d700 1 --2- 172.21.15.1:0/781322229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a41086be0 0x558a410860f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:59.917 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.914+0000 7f5c7b59e700 1 -- 172.21.15.1:0/781322229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a410874f0 msgr2=0x558a41092ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:59.918 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.914+0000 7f5c7b59e700 1 --2- 172.21.15.1:0/781322229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a410874f0 0x558a41092ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:59.918 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.914+0000 7f5c7b59e700 1 -- 172.21.15.1:0/781322229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a41086be0 msgr2=0x558a410860f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:59.918 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.914+0000 7f5c7b59e700 1 --2- 172.21.15.1:0/781322229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a41086be0 0x558a410860f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:59.918 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.914+0000 7f5c7b59e700 1 -- 172.21.15.1:0/781322229 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a4107db80 con 0x558a4108b930 2023-11-16T16:40:59.918 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.914+0000 7f5c7ad9d700 1 --2- 172.21.15.1:0/781322229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a41086be0 0x558a410860f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:40:59.918 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.914+0000 7f5c7b59e700 1 --2- 172.21.15.1:0/781322229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a4108b930 0x558a4108fb90 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f5c74009310 tx=0x7f5c74009340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb316e2e47d2489 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:59.918 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.915+0000 7f5c61ffb700 1 -- 172.21.15.1:0/781322229 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7400e040 con 0x558a4108b930 2023-11-16T16:40:59.918 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.915+0000 7f5c61ffb700 1 -- 172.21.15.1:0/781322229 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c7400f040 con 0x558a4108b930 2023-11-16T16:40:59.918 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.915+0000 7f5c61ffb700 1 -- 172.21.15.1:0/781322229 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c740048a0 con 0x558a4108b930 2023-11-16T16:40:59.921 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.915+0000 7f5c8d313b80 1 -- 172.21.15.1:0/781322229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a4108b930 msgr2=0x558a4108fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:59.921 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.915+0000 7f5c8d313b80 1 --2- 172.21.15.1:0/781322229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a4108b930 0x558a4108fb90 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f5c74009310 tx=0x7f5c74009340 comp rx=0 tx=0).stop 2023-11-16T16:40:59.921 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.915+0000 7f5c8d313b80 1 -- 172.21.15.1:0/781322229 shutdown_connections 2023-11-16T16:40:59.922 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.915+0000 7f5c8d313b80 1 --2- 172.21.15.1:0/781322229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a4108b930 0x558a4108fb90 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:59.922 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.916+0000 7f5c8d313b80 1 --2- 172.21.15.1:0/781322229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a41086be0 0x558a410860f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:59.922 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.916+0000 7f5c8d313b80 1 --2- 172.21.15.1:0/781322229 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a410874f0 0x558a41092ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:59.922 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.916+0000 7f5c8d313b80 1 -- 172.21.15.1:0/781322229 >> 172.21.15.1:0/781322229 conn(0x558a4107c3e0 msgr2=0x558a410859e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:59.922 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.916+0000 7f5c8d313b80 1 -- 172.21.15.1:0/781322229 shutdown_connections 2023-11-16T16:40:59.922 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.916+0000 7f5c8d313b80 1 -- 172.21.15.1:0/781322229 wait complete. 2023-11-16T16:40:59.922 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.916+0000 7f5c8d313b80 1 -- start start 2023-11-16T16:40:59.922 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.916+0000 7f5c8d313b80 1 Processor -- start 2023-11-16T16:40:59.922 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.916+0000 7f5c8d313b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a41086be0 0x558a41097b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:59.922 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.916+0000 7f5c8d313b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a410874f0 0x558a4108cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:59.922 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.916+0000 7f5c8d313b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a4108b930 0x558a4108d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:59.923 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.916+0000 7f5c8d313b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a4108d800 con 0x558a4108b930 2023-11-16T16:40:59.923 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.916+0000 7f5c8d313b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a4108d970 con 0x558a41086be0 2023-11-16T16:40:59.923 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.916+0000 7f5c8d313b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a410f46f0 con 0x558a410874f0 2023-11-16T16:40:59.923 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.917+0000 7f5c8d313b80 10 client.0 _ll_register_callbacks cb 0x7f5c740117c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:40:59.923 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.917+0000 7f5c7b59e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a4108b930 0x558a4108d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:59.923 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.917+0000 7f5c7b59e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a4108b930 0x558a4108d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44466/0 (socket says 172.21.15.1:44466) 2023-11-16T16:40:59.923 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.917+0000 7f5c7b59e700 1 -- 172.21.15.1:0/2002970201 learned_addr learned my addr 172.21.15.1:0/2002970201 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:40:59.923 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.917+0000 7f5c7a59c700 1 --2- 172.21.15.1:0/2002970201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a410874f0 0x558a4108cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:59.923 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.917+0000 7f5c7ad9d700 1 --2- 172.21.15.1:0/2002970201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a41086be0 0x558a41097b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:59.923 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.917+0000 7f5c7b59e700 1 -- 172.21.15.1:0/2002970201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a410874f0 msgr2=0x558a4108cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:59.923 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.917+0000 7f5c7b59e700 1 --2- 172.21.15.1:0/2002970201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a410874f0 0x558a4108cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:59.923 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.917+0000 7f5c7b59e700 1 -- 172.21.15.1:0/2002970201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a41086be0 msgr2=0x558a41097b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:40:59.923 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.917+0000 7f5c7b59e700 1 --2- 172.21.15.1:0/2002970201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a41086be0 0x558a41097b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:59.924 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.917+0000 7f5c7b59e700 1 -- 172.21.15.1:0/2002970201 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a40f9e210 con 0x558a4108b930 2023-11-16T16:40:59.924 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.917+0000 7f5c7a59c700 1 --2- 172.21.15.1:0/2002970201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a410874f0 0x558a4108cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:40:59.924 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.917+0000 7f5c5eff5700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:40:59.924 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.917+0000 7f5c7b59e700 1 --2- 172.21.15.1:0/2002970201 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a4108b930 0x558a4108d0f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5c74000f80 tx=0x7f5c74006160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c0ffe5ecbcc9016 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:40:59.924 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.918+0000 7f5c5eff5700 1 -- 172.21.15.1:0/2002970201 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7400e040 con 0x558a4108b930 2023-11-16T16:40:59.924 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.918+0000 7f5c5eff5700 1 -- 172.21.15.1:0/2002970201 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f5c74014de0 con 0x558a4108b930 2023-11-16T16:40:59.924 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.918+0000 7f5c8d313b80 1 -- 172.21.15.1:0/2002970201 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c6c01e040 con 0x558a4108b930 2023-11-16T16:40:59.924 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.918+0000 7f5c5eff5700 1 -- 172.21.15.1:0/2002970201 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c74003a00 con 0x558a4108b930 2023-11-16T16:40:59.924 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.918+0000 7f5c8d313b80 10 client.15828 fetch_fsmap learned FSMap version 23 2023-11-16T16:40:59.924 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.918+0000 7f5c8d313b80 1 -- 172.21.15.1:0/2002970201 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x558a40e05e60 con 0x558a4108b930 2023-11-16T16:40:59.924 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.918+0000 7f5c5eff5700 1 -- 172.21.15.1:0/2002970201 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c7403d020 con 0x558a4108b930 2023-11-16T16:40:59.924 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.919+0000 7f5c5eff5700 1 -- 172.21.15.1:0/2002970201 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5c74014320 con 0x558a4108b930 2023-11-16T16:40:59.925 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.919+0000 7f5c8d313b80 10 client.15828 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:40:59.925 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.919+0000 7f5c8d313b80 10 client.15828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:40:59.925 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.919+0000 7f5c8d313b80 20 client.15828 populate_metadata read hostname 'smithi001' 2023-11-16T16:40:59.925 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.919+0000 7f5c8d313b80 1 --2- 172.21.15.1:0/2002970201 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x558a410f58e0 0x558a410f7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:40:59.925 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.919+0000 7f5c8d313b80 4 client.15828 mds_command: new command op to 24521 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-16T16:40:59.925 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.919+0000 7f5c8d313b80 1 -- 172.21.15.1:0/2002970201 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x558a40fa66e0 con 0x558a410f58e0 2023-11-16T16:40:59.925 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.919+0000 7f5c7ad9d700 1 --2- 172.21.15.1:0/2002970201 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x558a410f58e0 0x558a410f7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:40:59.925 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.919+0000 7f5c5eff5700 10 client.15828 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:40:59.925 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.919+0000 7f5c7ad9d700 1 --2- 172.21.15.1:0/2002970201 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x558a410f58e0 0x558a410f7cc0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa4d614053249860 server_cookie=13f8b36a89b3b26c in_seq=0 out_seq=0 2023-11-16T16:40:59.926 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.921+0000 7f5c5eff5700 1 -- 172.21.15.1:0/2002970201 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558a40fa66e0 con 0x558a410f58e0 2023-11-16T16:40:59.926 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.921+0000 7f5c5eff5700 10 client.15828 handle_command_reply: tid=0 2023-11-16T16:40:59.927 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.921+0000 7f5c8d313b80 1 client.15828 shutdown 2023-11-16T16:40:59.927 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.921+0000 7f5c8d313b80 1 -- 172.21.15.1:0/2002970201 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a4108b930 msgr2=0x558a4108d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:59.927 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.921+0000 7f5c8d313b80 1 --2- 172.21.15.1:0/2002970201 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a4108b930 0x558a4108d0f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5c74000f80 tx=0x7f5c74006160 comp rx=0 tx=0).stop 2023-11-16T16:40:59.927 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.922+0000 7f5c8d313b80 1 -- 172.21.15.1:0/2002970201 shutdown_connections 2023-11-16T16:40:59.927 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.922+0000 7f5c8d313b80 1 --2- 172.21.15.1:0/2002970201 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x558a410f58e0 0x558a410f7cc0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:59.927 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.922+0000 7f5c8d313b80 1 --2- 172.21.15.1:0/2002970201 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a4108b930 0x558a4108d0f0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:59.927 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.922+0000 7f5c8d313b80 1 --2- 172.21.15.1:0/2002970201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558a41086be0 0x558a41097b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:59.927 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.922+0000 7f5c8d313b80 1 --2- 172.21.15.1:0/2002970201 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558a410874f0 0x558a4108cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:59.927 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.922+0000 7f5c5eff5700 0 client.15828 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:40:59.927 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.922+0000 7f5c8d313b80 1 -- 172.21.15.1:0/2002970201 >> 172.21.15.1:0/2002970201 conn(0x558a4107c3e0 msgr2=0x558a41086270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:59.927 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.922+0000 7f5c8d313b80 1 -- 172.21.15.1:0/2002970201 shutdown_connections 2023-11-16T16:40:59.927 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.922+0000 7f5c8d313b80 1 -- 172.21.15.1:0/2002970201 wait complete. 2023-11-16T16:40:59.928 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.922+0000 7f5c8d313b80 20 client.15828 trim_cache size 0 max 16384 2023-11-16T16:40:59.928 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:40:59.928 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2023-11-16T16:40:59.928 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "360e3f41-0374-491f-bd62-43c1f96d0aa9", 2023-11-16T16:40:59.928 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2023-11-16T16:40:59.928 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:40:59.929 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.922+0000 7f5c81187700 1 -- 172.21.15.1:0/1007629511 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5c640a12c0 msgr2=0x7f5c640a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:59.929 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.922+0000 7f5c81187700 1 --2- 172.21.15.1:0/1007629511 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5c640a12c0 0x7f5c640a3770 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00b2e0 tx=0x7f5c6c005fd0 comp rx=0 tx=0).stop 2023-11-16T16:40:59.929 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.922+0000 7f5c81187700 1 -- 172.21.15.1:0/1007629511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c7c0ff960 msgr2=0x7f5c7c198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:40:59.929 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.922+0000 7f5c81187700 1 --2- 172.21.15.1:0/1007629511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c7c0ff960 0x7f5c7c198b00 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000e9a0 tx=0x7f5c7000ecb0 comp rx=0 tx=0).stop 2023-11-16T16:40:59.929 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.922+0000 7f5c81187700 1 -- 172.21.15.1:0/1007629511 shutdown_connections 2023-11-16T16:40:59.929 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.923+0000 7f5c81187700 1 --2- 172.21.15.1:0/1007629511 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5c640a12c0 0x7f5c640a3770 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:59.929 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.923+0000 7f5c81187700 1 --2- 172.21.15.1:0/1007629511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c7c10cc00 0x7f5c7c19cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:59.929 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.923+0000 7f5c81187700 1 --2- 172.21.15.1:0/1007629511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5c7c0ff050 0x7f5c7c1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:59.930 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.923+0000 7f5c81187700 1 --2- 172.21.15.1:0/1007629511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c7c0ff960 0x7f5c7c198b00 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:40:59.930 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.923+0000 7f5c81187700 1 -- 172.21.15.1:0/1007629511 >> 172.21.15.1:0/1007629511 conn(0x7f5c7c0763a0 msgr2=0x7f5c7c0fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:40:59.930 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.923+0000 7f5c81187700 1 -- 172.21.15.1:0/1007629511 shutdown_connections 2023-11-16T16:40:59.930 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:40:59.923+0000 7f5c81187700 1 -- 172.21.15.1:0/1007629511 wait complete. 2023-11-16T16:40:59.948 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-16T16:41:00.091 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:00.091 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:00.146 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:00.146 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:00.173 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:00.173 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:00.173 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 55 -lt 100 ']' 2023-11-16T16:41:00.173 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 55 + 1 2023-11-16T16:41:00.204 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=56 2023-11-16T16:41:00.204 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:00.250 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:00.251 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:00.251 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:00.252 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:00.252 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:00.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:00 smithi001 ceph-mon[98800]: scrub queued for path: / 2023-11-16T16:41:00.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:00 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:41:00.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:00 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:41:00.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:00 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:41:00.732 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:00.732 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:00.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:00 smithi026 ceph-mon[100112]: scrub queued for path: / 2023-11-16T16:41:00.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:00 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:41:00.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:00 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:41:00.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:00 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:41:00.809 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:00.809 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:00.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:00 smithi053 ceph-mon[100638]: scrub queued for path: / 2023-11-16T16:41:00.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:00 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:41:00.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:00 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:41:00.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:00 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:41:00.842 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:00.842 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:00.842 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 56 -lt 100 ']' 2023-11-16T16:41:00.842 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 56 + 1 2023-11-16T16:41:00.896 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=57 2023-11-16T16:41:00.896 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:00.952 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:00.952 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:00.953 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:00.953 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:00.953 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:01.420 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:01 smithi001 ceph-mon[98800]: pgmap v58: 97 pgs: 97 active+clean; 1.9 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 6 op/s 2023-11-16T16:41:01.473 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:01.474 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:01.602 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:01.602 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:01.615 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:01.615 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:01.615 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 57 -lt 100 ']' 2023-11-16T16:41:01.615 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 57 + 1 2023-11-16T16:41:01.646 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=58 2023-11-16T16:41:01.646 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:01.709 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:01.709 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:01.710 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:01.710 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:01.710 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:01.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:01 smithi026 ceph-mon[100112]: pgmap v58: 97 pgs: 97 active+clean; 1.9 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 6 op/s 2023-11-16T16:41:01.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:01 smithi053 ceph-mon[100638]: pgmap v58: 97 pgs: 97 active+clean; 1.9 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 6 op/s 2023-11-16T16:41:02.415 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:02.415 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:02.621 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:02.621 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:02.641 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:02.641 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:02.641 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 58 -lt 100 ']' 2023-11-16T16:41:02.641 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 58 + 1 2023-11-16T16:41:02.679 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=59 2023-11-16T16:41:02.679 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:02.709 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:02.709 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:02.710 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:02.710 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:02.710 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:03.041 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.039+0000 7f75c6ad1700 1 -- 172.21.15.1:0/1827025046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75c01064d0 msgr2=0x7f75c01068a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:03.041 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.039+0000 7f75c6ad1700 1 --2- 172.21.15.1:0/1827025046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75c01064d0 0x7f75c01068a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f75b0009990 tx=0x7f75b0009ca0 comp rx=0 tx=0).stop 2023-11-16T16:41:03.041 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.039+0000 7f75c6ad1700 1 -- 172.21.15.1:0/1827025046 shutdown_connections 2023-11-16T16:41:03.042 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.039+0000 7f75c6ad1700 1 --2- 172.21.15.1:0/1827025046 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75c0101390 0x7f75c010d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:03.042 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.039+0000 7f75c6ad1700 1 --2- 172.21.15.1:0/1827025046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75c01064d0 0x7f75c01068a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:03.042 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.039+0000 7f75c6ad1700 1 --2- 172.21.15.1:0/1827025046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75c01009e0 0x7f75c0100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:03.042 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.039+0000 7f75c6ad1700 1 -- 172.21.15.1:0/1827025046 >> 172.21.15.1:0/1827025046 conn(0x7f75c0077530 msgr2=0x7f75c0077930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:03.042 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.039+0000 7f75c6ad1700 1 -- 172.21.15.1:0/1827025046 shutdown_connections 2023-11-16T16:41:03.042 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.039+0000 7f75c6ad1700 1 -- 172.21.15.1:0/1827025046 wait complete. 2023-11-16T16:41:03.044 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.040+0000 7f75c6ad1700 1 Processor -- start 2023-11-16T16:41:03.044 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.040+0000 7f75c6ad1700 1 -- start start 2023-11-16T16:41:03.044 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.040+0000 7f75c6ad1700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75c01009e0 0x7f75c0072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:03.044 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.040+0000 7f75c6ad1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75c0101390 0x7f75c0073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:03.044 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.040+0000 7f75c6ad1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75c01064d0 0x7f75c01a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:03.044 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.040+0000 7f75c6ad1700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c0073b60 con 0x7f75c01064d0 2023-11-16T16:41:03.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.040+0000 7f75c6ad1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c01a4c30 con 0x7f75c0101390 2023-11-16T16:41:03.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.040+0000 7f75c6ad1700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c01a4d70 con 0x7f75c01009e0 2023-11-16T16:41:03.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.040+0000 7f75c506e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75c01064d0 0x7f75c01a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:03.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.040+0000 7f75bffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75c0101390 0x7f75c0073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:03.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.041+0000 7f75c506e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75c01064d0 0x7f75c01a46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44480/0 (socket says 172.21.15.1:44480) 2023-11-16T16:41:03.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.041+0000 7f75c506e700 1 -- 172.21.15.1:0/16306854 learned_addr learned my addr 172.21.15.1:0/16306854 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:41:03.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.041+0000 7f75bffff700 1 --2- 172.21.15.1:0/16306854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75c0101390 0x7f75c0073330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.1:38664/0 (socket says 172.21.15.1:38664) 2023-11-16T16:41:03.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.041+0000 7f75c486d700 1 --2- 172.21.15.1:0/16306854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75c01009e0 0x7f75c0072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:03.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.041+0000 7f75c506e700 1 -- 172.21.15.1:0/16306854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75c01009e0 msgr2=0x7f75c0072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:03.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.041+0000 7f75c506e700 1 --2- 172.21.15.1:0/16306854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75c01009e0 0x7f75c0072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:03.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.041+0000 7f75c506e700 1 -- 172.21.15.1:0/16306854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75c0101390 msgr2=0x7f75c0073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:03.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.041+0000 7f75c506e700 1 --2- 172.21.15.1:0/16306854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75c0101390 0x7f75c0073330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:03.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.041+0000 7f75c506e700 1 -- 172.21.15.1:0/16306854 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a8009520 con 0x7f75c01064d0 2023-11-16T16:41:03.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.041+0000 7f75c506e700 1 --2- 172.21.15.1:0/16306854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75c01064d0 0x7f75c01a46f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f75b800ef30 tx=0x7f75b800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:03.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.041+0000 7f75bdffb700 1 -- 172.21.15.1:0/16306854 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b8018070 con 0x7f75c01064d0 2023-11-16T16:41:03.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.041+0000 7f75c6ad1700 1 -- 172.21.15.1:0/16306854 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75b00095f0 con 0x7f75c01064d0 2023-11-16T16:41:03.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.041+0000 7f75bdffb700 1 -- 172.21.15.1:0/16306854 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75b800f040 con 0x7f75c01064d0 2023-11-16T16:41:03.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.041+0000 7f75bdffb700 1 -- 172.21.15.1:0/16306854 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b800cd00 con 0x7f75c01064d0 2023-11-16T16:41:03.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.042+0000 7f75c6ad1700 1 -- 172.21.15.1:0/16306854 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75c01104e0 con 0x7f75c01064d0 2023-11-16T16:41:03.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.044+0000 7f75bdffb700 1 -- 172.21.15.1:0/16306854 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f75b8003a00 con 0x7f75c01064d0 2023-11-16T16:41:03.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.045+0000 7f75bdffb700 1 --2- 172.21.15.1:0/16306854 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f75ac0a12c0 0x7f75ac0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:03.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.045+0000 7f75bdffb700 1 -- 172.21.15.1:0/16306854 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f75b80cc7c0 con 0x7f75c01064d0 2023-11-16T16:41:03.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.045+0000 7f75c486d700 1 --2- 172.21.15.1:0/16306854 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f75ac0a12c0 0x7f75ac0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:03.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.045+0000 7f75c486d700 1 --2- 172.21.15.1:0/16306854 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f75ac0a12c0 0x7f75ac0a3770 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f75b000b2e0 tx=0x7f75b0005fd0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:03.097 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.095+0000 7f75d2c5db80 1 Processor -- start 2023-11-16T16:41:03.097 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.096+0000 7f75d2c5db80 1 -- start start 2023-11-16T16:41:03.099 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.096+0000 7f75d2c5db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6cb48fb60 0x55a6cb48ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:03.099 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.096+0000 7f75d2c5db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6cb490470 0x55a6cb494370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:03.099 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.096+0000 7f75d2c5db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6cb4948b0 0x55a6cb494d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:03.099 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.096+0000 7f75d2c5db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6cb395060 con 0x55a6cb4948b0 2023-11-16T16:41:03.099 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.096+0000 7f75d2c5db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6cb3e78a0 con 0x55a6cb490470 2023-11-16T16:41:03.100 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.096+0000 7f75d2c5db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6cb3e0d30 con 0x55a6cb48fb60 2023-11-16T16:41:03.100 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.096+0000 7f75c506e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6cb4948b0 0x55a6cb494d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:03.100 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.096+0000 7f75c486d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6cb48fb60 0x55a6cb48ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:03.100 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.096+0000 7f75c506e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6cb4948b0 0x55a6cb494d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44488/0 (socket says 172.21.15.1:44488) 2023-11-16T16:41:03.100 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.096+0000 7f75c506e700 1 -- 172.21.15.1:0/1185875573 learned_addr learned my addr 172.21.15.1:0/1185875573 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:41:03.100 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.096+0000 7f75bffff700 1 --2- 172.21.15.1:0/1185875573 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6cb490470 0x55a6cb494370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:03.100 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.097+0000 7f75c486d700 1 -- 172.21.15.1:0/1185875573 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6cb490470 msgr2=0x55a6cb494370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:03.100 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.097+0000 7f75c486d700 1 --2- 172.21.15.1:0/1185875573 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6cb490470 0x55a6cb494370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:03.100 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.097+0000 7f75c486d700 1 -- 172.21.15.1:0/1185875573 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6cb4948b0 msgr2=0x55a6cb494d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:03.100 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.097+0000 7f75c486d700 1 --2- 172.21.15.1:0/1185875573 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6cb4948b0 0x55a6cb494d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:03.100 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.097+0000 7f75c486d700 1 -- 172.21.15.1:0/1185875573 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6cb262630 con 0x55a6cb48fb60 2023-11-16T16:41:03.101 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.097+0000 7f75c506e700 1 --2- 172.21.15.1:0/1185875573 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6cb4948b0 0x55a6cb494d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:41:03.101 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.097+0000 7f75c486d700 1 --2- 172.21.15.1:0/1185875573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6cb48fb60 0x55a6cb48ff30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f75b000b6f0 tx=0x7f75b0020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b44b05a581961877 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:03.101 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.097+0000 7f75b6ffd700 1 -- 172.21.15.1:0/1185875573 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b002b070 con 0x55a6cb48fb60 2023-11-16T16:41:03.101 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.097+0000 7f75b6ffd700 1 -- 172.21.15.1:0/1185875573 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75b001e040 con 0x55a6cb48fb60 2023-11-16T16:41:03.101 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.097+0000 7f75b6ffd700 1 -- 172.21.15.1:0/1185875573 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b000f6c0 con 0x55a6cb48fb60 2023-11-16T16:41:03.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.098+0000 7f75d2c5db80 1 -- 172.21.15.1:0/1185875573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6cb48fb60 msgr2=0x55a6cb48ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:03.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.098+0000 7f75d2c5db80 1 --2- 172.21.15.1:0/1185875573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6cb48fb60 0x55a6cb48ff30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f75b000b6f0 tx=0x7f75b0020000 comp rx=0 tx=0).stop 2023-11-16T16:41:03.106 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.098+0000 7f75d2c5db80 1 -- 172.21.15.1:0/1185875573 shutdown_connections 2023-11-16T16:41:03.106 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.098+0000 7f75d2c5db80 1 --2- 172.21.15.1:0/1185875573 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6cb4948b0 0x55a6cb494d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:03.106 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.098+0000 7f75d2c5db80 1 --2- 172.21.15.1:0/1185875573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6cb48fb60 0x55a6cb48ff30 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:03.106 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.098+0000 7f75d2c5db80 1 --2- 172.21.15.1:0/1185875573 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6cb490470 0x55a6cb494370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:03.106 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.098+0000 7f75d2c5db80 1 -- 172.21.15.1:0/1185875573 >> 172.21.15.1:0/1185875573 conn(0x55a6cb485360 msgr2=0x55a6cb48ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:03.106 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.098+0000 7f75d2c5db80 1 -- 172.21.15.1:0/1185875573 shutdown_connections 2023-11-16T16:41:03.106 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.098+0000 7f75d2c5db80 1 -- 172.21.15.1:0/1185875573 wait complete. 2023-11-16T16:41:03.106 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.099+0000 7f75d2c5db80 1 -- start start 2023-11-16T16:41:03.106 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.099+0000 7f75d2c5db80 1 Processor -- start 2023-11-16T16:41:03.106 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.099+0000 7f75d2c5db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6cb48fb60 0x55a6cb49c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:03.106 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.099+0000 7f75d2c5db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6cb490470 0x55a6cb499320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:03.107 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.099+0000 7f75d2c5db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6cb4948b0 0x55a6cb499a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:03.107 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.099+0000 7f75d2c5db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6cb3828e0 con 0x55a6cb4948b0 2023-11-16T16:41:03.107 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.099+0000 7f75d2c5db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6cb382580 con 0x55a6cb48fb60 2023-11-16T16:41:03.107 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.099+0000 7f75d2c5db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6cb264920 con 0x55a6cb490470 2023-11-16T16:41:03.107 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.099+0000 7f75d2c5db80 10 client.0 _ll_register_callbacks cb 0x55a6cb3a0780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:41:03.107 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.099+0000 7f75c506e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6cb4948b0 0x55a6cb499a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:03.107 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.100+0000 7f75bffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6cb490470 0x55a6cb499320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:03.107 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.100+0000 7f75c506e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6cb4948b0 0x55a6cb499a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44494/0 (socket says 172.21.15.1:44494) 2023-11-16T16:41:03.107 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.100+0000 7f75c506e700 1 -- 172.21.15.1:0/1881729840 learned_addr learned my addr 172.21.15.1:0/1881729840 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:41:03.107 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.100+0000 7f75bffff700 1 --2- 172.21.15.1:0/1881729840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6cb490470 0x55a6cb499320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:41174/0 (socket says 172.21.15.1:41174) 2023-11-16T16:41:03.107 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.100+0000 7f75c486d700 1 --2- 172.21.15.1:0/1881729840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6cb48fb60 0x55a6cb49c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:03.107 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.100+0000 7f75bffff700 1 -- 172.21.15.1:0/1881729840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6cb48fb60 msgr2=0x55a6cb49c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:03.108 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.100+0000 7f75bffff700 1 --2- 172.21.15.1:0/1881729840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6cb48fb60 0x55a6cb49c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:03.108 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.100+0000 7f75bffff700 1 -- 172.21.15.1:0/1881729840 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6cb4948b0 msgr2=0x55a6cb499a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:03.108 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.100+0000 7f75bffff700 1 --2- 172.21.15.1:0/1881729840 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6cb4948b0 0x55a6cb499a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:03.108 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.100+0000 7f75bffff700 1 -- 172.21.15.1:0/1881729840 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75c010dc00 con 0x55a6cb490470 2023-11-16T16:41:03.108 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.100+0000 7f75c506e700 1 --2- 172.21.15.1:0/1881729840 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6cb4948b0 0x55a6cb499a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:41:03.108 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.100+0000 7f75bffff700 1 --2- 172.21.15.1:0/1881729840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6cb490470 0x55a6cb499320 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f75a8012950 tx=0x7f75a8012c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b57a60f61509ebd server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:03.108 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.100+0000 7f759ffff700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-16T16:41:03.108 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.100+0000 7f759ffff700 1 -- 172.21.15.1:0/1881729840 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a8017070 con 0x55a6cb490470 2023-11-16T16:41:03.108 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.100+0000 7f759ffff700 1 -- 172.21.15.1:0/1881729840 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f75a800f830 con 0x55a6cb490470 2023-11-16T16:41:03.108 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.100+0000 7f75d2c5db80 1 -- 172.21.15.1:0/1881729840 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a6cb482b20 con 0x55a6cb490470 2023-11-16T16:41:03.108 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.101+0000 7f759ffff700 1 -- 172.21.15.1:0/1881729840 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75a80122e0 con 0x55a6cb490470 2023-11-16T16:41:03.108 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.101+0000 7f75d2c5db80 10 client.25411 fetch_fsmap learned FSMap version 23 2023-11-16T16:41:03.109 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.101+0000 7f75d2c5db80 1 -- 172.21.15.1:0/1881729840 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55a6cb3a7190 con 0x55a6cb490470 2023-11-16T16:41:03.109 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.101+0000 7f759ffff700 1 -- 172.21.15.1:0/1881729840 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75a8028080 con 0x55a6cb490470 2023-11-16T16:41:03.109 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.101+0000 7f759ffff700 1 -- 172.21.15.1:0/1881729840 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f75a8010b50 con 0x55a6cb490470 2023-11-16T16:41:03.109 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.101+0000 7f75d2c5db80 10 client.25411 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:41:03.109 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.101+0000 7f75d2c5db80 10 client.25411 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:41:03.109 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.101+0000 7f75d2c5db80 20 client.25411 populate_metadata read hostname 'smithi001' 2023-11-16T16:41:03.109 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.101+0000 7f75d2c5db80 1 --2- 172.21.15.1:0/1881729840 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55a6cb4a9910 0x55a6cb4abd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:03.109 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.101+0000 7f75d2c5db80 4 client.25411 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:41:03.109 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.101+0000 7f75d2c5db80 1 -- 172.21.15.1:0/1881729840 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a6cb3a30a0 con 0x55a6cb4a9910 2023-11-16T16:41:03.109 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.101+0000 7f75c486d700 1 --2- 172.21.15.1:0/1881729840 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55a6cb4a9910 0x55a6cb4abd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:03.109 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.102+0000 7f75c486d700 1 --2- 172.21.15.1:0/1881729840 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55a6cb4a9910 0x55a6cb4abd20 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0e1d45070eff785 server_cookie=76f028c8464c25b4 in_seq=0 out_seq=0 2023-11-16T16:41:03.110 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.102+0000 7f759ffff700 10 client.25411 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:41:03.110 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.103+0000 7f759ffff700 1 -- 172.21.15.1:0/1881729840 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a6cb3a30a0 con 0x55a6cb4a9910 2023-11-16T16:41:03.110 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.103+0000 7f759ffff700 10 client.25411 handle_command_reply: tid=0 2023-11-16T16:41:03.110 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.103+0000 7f75d2c5db80 1 client.25411 shutdown 2023-11-16T16:41:03.110 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.103+0000 7f75d2c5db80 1 -- 172.21.15.1:0/1881729840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6cb490470 msgr2=0x55a6cb499320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:03.110 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.103+0000 7f75d2c5db80 1 --2- 172.21.15.1:0/1881729840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6cb490470 0x55a6cb499320 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f75a8012950 tx=0x7f75a8012c60 comp rx=0 tx=0).stop 2023-11-16T16:41:03.110 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.103+0000 7f75d2c5db80 1 -- 172.21.15.1:0/1881729840 shutdown_connections 2023-11-16T16:41:03.110 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.103+0000 7f75d2c5db80 1 --2- 172.21.15.1:0/1881729840 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55a6cb4a9910 0x55a6cb4abd20 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:03.110 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.103+0000 7f75d2c5db80 1 --2- 172.21.15.1:0/1881729840 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6cb4948b0 0x55a6cb499a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:03.110 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.103+0000 7f75d2c5db80 1 --2- 172.21.15.1:0/1881729840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6cb48fb60 0x55a6cb49c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:03.110 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.103+0000 7f75d2c5db80 1 --2- 172.21.15.1:0/1881729840 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6cb490470 0x55a6cb499320 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:03.110 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.103+0000 7f759ffff700 0 client.25411 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:41:03.111 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.103+0000 7f75d2c5db80 1 -- 172.21.15.1:0/1881729840 >> 172.21.15.1:0/1881729840 conn(0x55a6cb485360 msgr2=0x55a6cb495d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:03.111 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.103+0000 7f75d2c5db80 1 -- 172.21.15.1:0/1881729840 shutdown_connections 2023-11-16T16:41:03.111 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.103+0000 7f75d2c5db80 1 -- 172.21.15.1:0/1881729840 wait complete. 2023-11-16T16:41:03.111 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.103+0000 7f75d2c5db80 20 client.25411 trim_cache size 0 max 16384 2023-11-16T16:41:03.124 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.121+0000 7f75d2c5db80 1 Processor -- start 2023-11-16T16:41:03.124 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.121+0000 7f75d2c5db80 1 -- start start 2023-11-16T16:41:03.124 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.122+0000 7f75d2c5db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6cb48fb60 0x55a6cb489550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:03.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.122+0000 7f75d2c5db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6cb490470 0x55a6cb4988b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:03.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.122+0000 7f75d2c5db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6cb4948b0 0x55a6cb49bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:03.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.122+0000 7f75d2c5db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6cb49c6b0 con 0x55a6cb490470 2023-11-16T16:41:03.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.122+0000 7f75d2c5db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6cb4976a0 con 0x55a6cb4948b0 2023-11-16T16:41:03.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.122+0000 7f75d2c5db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6cb493260 con 0x55a6cb48fb60 2023-11-16T16:41:03.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.122+0000 7f75bffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6cb490470 0x55a6cb4988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:03.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.122+0000 7f75bffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6cb490470 0x55a6cb4988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44500/0 (socket says 172.21.15.1:44500) 2023-11-16T16:41:03.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.122+0000 7f75bffff700 1 -- 172.21.15.1:0/3753739287 learned_addr learned my addr 172.21.15.1:0/3753739287 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:41:03.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.122+0000 7f75c486d700 1 --2- 172.21.15.1:0/3753739287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6cb48fb60 0x55a6cb489550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:03.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.122+0000 7f75c506e700 1 --2- 172.21.15.1:0/3753739287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6cb4948b0 0x55a6cb49bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:03.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.122+0000 7f75bffff700 1 -- 172.21.15.1:0/3753739287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6cb48fb60 msgr2=0x55a6cb489550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:03.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.122+0000 7f75bffff700 1 --2- 172.21.15.1:0/3753739287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6cb48fb60 0x55a6cb489550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:03.126 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.122+0000 7f75bffff700 1 -- 172.21.15.1:0/3753739287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6cb4948b0 msgr2=0x55a6cb49bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:03.126 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.122+0000 7f75bffff700 1 --2- 172.21.15.1:0/3753739287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6cb4948b0 0x55a6cb49bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:03.126 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.122+0000 7f75bffff700 1 -- 172.21.15.1:0/3753739287 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6cb3a6ee0 con 0x55a6cb490470 2023-11-16T16:41:03.126 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.122+0000 7f75c506e700 1 --2- 172.21.15.1:0/3753739287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6cb4948b0 0x55a6cb49bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:41:03.126 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.122+0000 7f75bffff700 1 --2- 172.21.15.1:0/3753739287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6cb490470 0x55a6cb4988b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f75a800ffd0 tx=0x7f75a8005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc44860b7ca636d server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:03.126 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.123+0000 7f75b77fe700 1 -- 172.21.15.1:0/3753739287 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a8017070 con 0x55a6cb490470 2023-11-16T16:41:03.126 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.123+0000 7f75b77fe700 1 -- 172.21.15.1:0/3753739287 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75a8018040 con 0x55a6cb490470 2023-11-16T16:41:03.126 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.123+0000 7f75b77fe700 1 -- 172.21.15.1:0/3753739287 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a801cbc0 con 0x55a6cb490470 2023-11-16T16:41:03.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.123+0000 7f75d2c5db80 1 -- 172.21.15.1:0/3753739287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6cb490470 msgr2=0x55a6cb4988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:03.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.123+0000 7f75d2c5db80 1 --2- 172.21.15.1:0/3753739287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6cb490470 0x55a6cb4988b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f75a800ffd0 tx=0x7f75a8005dc0 comp rx=0 tx=0).stop 2023-11-16T16:41:03.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.123+0000 7f75d2c5db80 1 -- 172.21.15.1:0/3753739287 shutdown_connections 2023-11-16T16:41:03.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.123+0000 7f75d2c5db80 1 --2- 172.21.15.1:0/3753739287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6cb4948b0 0x55a6cb49bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:03.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.123+0000 7f75d2c5db80 1 --2- 172.21.15.1:0/3753739287 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6cb48fb60 0x55a6cb489550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:03.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.123+0000 7f75d2c5db80 1 --2- 172.21.15.1:0/3753739287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6cb490470 0x55a6cb4988b0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:03.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.123+0000 7f75d2c5db80 1 -- 172.21.15.1:0/3753739287 >> 172.21.15.1:0/3753739287 conn(0x55a6cb485360 msgr2=0x55a6cb4a5b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:03.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.123+0000 7f75d2c5db80 1 -- 172.21.15.1:0/3753739287 shutdown_connections 2023-11-16T16:41:03.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.123+0000 7f75d2c5db80 1 -- 172.21.15.1:0/3753739287 wait complete. 2023-11-16T16:41:03.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.124+0000 7f75d2c5db80 1 -- start start 2023-11-16T16:41:03.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.124+0000 7f75d2c5db80 1 Processor -- start 2023-11-16T16:41:03.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.124+0000 7f75d2c5db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6cb48fb60 0x55a6cb49eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:03.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.124+0000 7f75d2c5db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6cb490470 0x55a6cb49f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:03.132 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.124+0000 7f75d2c5db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6cb4948b0 0x55a6cb4f6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:03.132 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.124+0000 7f75d2c5db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6cb4f6d10 con 0x55a6cb48fb60 2023-11-16T16:41:03.132 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.124+0000 7f75d2c5db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6cb4f6e80 con 0x55a6cb490470 2023-11-16T16:41:03.132 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.124+0000 7f75d2c5db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6cb4f7040 con 0x55a6cb4948b0 2023-11-16T16:41:03.132 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.125+0000 7f75d2c5db80 10 client.0 _ll_register_callbacks cb 0x7f75a8012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:41:03.132 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.125+0000 7f75c486d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6cb48fb60 0x55a6cb49eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:03.132 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.125+0000 7f75c486d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6cb48fb60 0x55a6cb49eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44504/0 (socket says 172.21.15.1:44504) 2023-11-16T16:41:03.132 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.125+0000 7f75c486d700 1 -- 172.21.15.1:0/2411566887 learned_addr learned my addr 172.21.15.1:0/2411566887 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:41:03.132 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.125+0000 7f75c506e700 1 --2- 172.21.15.1:0/2411566887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6cb4948b0 0x55a6cb4f6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:03.132 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.125+0000 7f75bffff700 1 --2- 172.21.15.1:0/2411566887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6cb490470 0x55a6cb49f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:03.132 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.125+0000 7f75c486d700 1 -- 172.21.15.1:0/2411566887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6cb4948b0 msgr2=0x55a6cb4f6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:03.133 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.125+0000 7f75c486d700 1 --2- 172.21.15.1:0/2411566887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6cb4948b0 0x55a6cb4f6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:03.133 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.125+0000 7f75c486d700 1 -- 172.21.15.1:0/2411566887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6cb490470 msgr2=0x55a6cb49f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:03.133 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.125+0000 7f75c486d700 1 --2- 172.21.15.1:0/2411566887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6cb490470 0x55a6cb49f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:03.133 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.125+0000 7f75c486d700 1 -- 172.21.15.1:0/2411566887 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6cb3a7190 con 0x55a6cb48fb60 2023-11-16T16:41:03.133 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.125+0000 7f75bffff700 1 --2- 172.21.15.1:0/2411566887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6cb490470 0x55a6cb49f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:41:03.133 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.125+0000 7f75c486d700 1 --2- 172.21.15.1:0/2411566887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6cb48fb60 0x55a6cb49eaf0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f75b00098e0 tx=0x7f75b0005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f523595cf2f63978 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:03.133 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.125+0000 7f759ffff700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:41:03.133 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.125+0000 7f759ffff700 1 -- 172.21.15.1:0/2411566887 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b002b070 con 0x55a6cb48fb60 2023-11-16T16:41:03.133 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.125+0000 7f759ffff700 1 -- 172.21.15.1:0/2411566887 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f75b003a0f0 con 0x55a6cb48fb60 2023-11-16T16:41:03.133 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.125+0000 7f75d2c5db80 1 -- 172.21.15.1:0/2411566887 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f75b001e040 con 0x55a6cb48fb60 2023-11-16T16:41:03.133 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.126+0000 7f759ffff700 1 -- 172.21.15.1:0/2411566887 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75b003ae60 con 0x55a6cb48fb60 2023-11-16T16:41:03.133 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.126+0000 7f75d2c5db80 10 client.15846 fetch_fsmap learned FSMap version 23 2023-11-16T16:41:03.134 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.126+0000 7f75d2c5db80 1 -- 172.21.15.1:0/2411566887 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55a6cb1f7c60 con 0x55a6cb48fb60 2023-11-16T16:41:03.134 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.126+0000 7f759ffff700 1 -- 172.21.15.1:0/2411566887 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75b001e040 con 0x55a6cb48fb60 2023-11-16T16:41:03.134 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.126+0000 7f759ffff700 1 -- 172.21.15.1:0/2411566887 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f75b0004500 con 0x55a6cb48fb60 2023-11-16T16:41:03.134 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.126+0000 7f75d2c5db80 10 client.15846 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:41:03.134 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.126+0000 7f75d2c5db80 10 client.15846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:41:03.134 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.126+0000 7f75d2c5db80 20 client.15846 populate_metadata read hostname 'smithi001' 2023-11-16T16:41:03.134 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.126+0000 7f75d2c5db80 1 --2- 172.21.15.1:0/2411566887 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55a6cb4fe3c0 0x55a6cb5007d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:03.134 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.126+0000 7f75d2c5db80 4 client.15846 mds_command: new command op to 24521 tid=0[{"prefix": "scrub status"}] 2023-11-16T16:41:03.134 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.126+0000 7f75d2c5db80 1 -- 172.21.15.1:0/2411566887 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a6cb3af660 con 0x55a6cb4fe3c0 2023-11-16T16:41:03.134 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.126+0000 7f75bffff700 1 --2- 172.21.15.1:0/2411566887 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55a6cb4fe3c0 0x55a6cb5007d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:03.134 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.127+0000 7f75bffff700 1 --2- 172.21.15.1:0/2411566887 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55a6cb4fe3c0 0x55a6cb5007d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=398839c956f38f0f server_cookie=4d569c33acf8d750 in_seq=0 out_seq=0 2023-11-16T16:41:03.134 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.127+0000 7f759ffff700 10 client.15846 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:41:03.135 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.127+0000 7f759ffff700 1 -- 172.21.15.1:0/2411566887 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55a6cb3af660 con 0x55a6cb4fe3c0 2023-11-16T16:41:03.135 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.127+0000 7f759ffff700 10 client.15846 handle_command_reply: tid=0 2023-11-16T16:41:03.135 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.127+0000 7f75d2c5db80 1 client.15846 shutdown 2023-11-16T16:41:03.135 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.127+0000 7f75d2c5db80 1 -- 172.21.15.1:0/2411566887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6cb48fb60 msgr2=0x55a6cb49eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:03.135 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.127+0000 7f75d2c5db80 1 --2- 172.21.15.1:0/2411566887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6cb48fb60 0x55a6cb49eaf0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f75b00098e0 tx=0x7f75b0005700 comp rx=0 tx=0).stop 2023-11-16T16:41:03.135 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.127+0000 7f75d2c5db80 1 -- 172.21.15.1:0/2411566887 shutdown_connections 2023-11-16T16:41:03.135 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.127+0000 7f75d2c5db80 1 --2- 172.21.15.1:0/2411566887 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55a6cb4fe3c0 0x55a6cb5007d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:03.135 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.127+0000 7f75d2c5db80 1 --2- 172.21.15.1:0/2411566887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a6cb4948b0 0x55a6cb4f6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:03.135 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.127+0000 7f75d2c5db80 1 --2- 172.21.15.1:0/2411566887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a6cb48fb60 0x55a6cb49eaf0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:03.135 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.127+0000 7f75d2c5db80 1 --2- 172.21.15.1:0/2411566887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6cb490470 0x55a6cb49f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:03.135 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.127+0000 7f759ffff700 0 client.15846 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:41:03.135 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.127+0000 7f75d2c5db80 1 -- 172.21.15.1:0/2411566887 >> 172.21.15.1:0/2411566887 conn(0x55a6cb485360 msgr2=0x55a6cb489550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:03.135 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.127+0000 7f75d2c5db80 1 -- 172.21.15.1:0/2411566887 shutdown_connections 2023-11-16T16:41:03.136 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.127+0000 7f75d2c5db80 1 -- 172.21.15.1:0/2411566887 wait complete. 2023-11-16T16:41:03.136 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.128+0000 7f75d2c5db80 20 client.15846 trim_cache size 0 max 16384 2023-11-16T16:41:03.136 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:41:03.136 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-16T16:41:03.136 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2023-11-16T16:41:03.136 INFO:teuthology.orchestra.run.smithi001.stdout: "360e3f41-0374-491f-bd62-43c1f96d0aa9": { 2023-11-16T16:41:03.136 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2023-11-16T16:41:03.136 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "360e3f41-0374-491f-bd62-43c1f96d0aa9", 2023-11-16T16:41:03.136 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2023-11-16T16:41:03.136 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:41:03.137 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:41:03.137 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:41:03.137 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.128+0000 7f75c6ad1700 1 -- 172.21.15.1:0/16306854 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f75ac0a12c0 msgr2=0x7f75ac0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:03.138 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.128+0000 7f75c6ad1700 1 --2- 172.21.15.1:0/16306854 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f75ac0a12c0 0x7f75ac0a3770 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f75b000b2e0 tx=0x7f75b0005fd0 comp rx=0 tx=0).stop 2023-11-16T16:41:03.138 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.128+0000 7f75c6ad1700 1 -- 172.21.15.1:0/16306854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75c01064d0 msgr2=0x7f75c01a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:03.138 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.128+0000 7f75c6ad1700 1 --2- 172.21.15.1:0/16306854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75c01064d0 0x7f75c01a46f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f75b800ef30 tx=0x7f75b800c3a0 comp rx=0 tx=0).stop 2023-11-16T16:41:03.138 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.128+0000 7f75c6ad1700 1 -- 172.21.15.1:0/16306854 shutdown_connections 2023-11-16T16:41:03.138 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.128+0000 7f75c6ad1700 1 --2- 172.21.15.1:0/16306854 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f75ac0a12c0 0x7f75ac0a3770 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:03.138 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.128+0000 7f75c6ad1700 1 --2- 172.21.15.1:0/16306854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f75c01064d0 0x7f75c01a46f0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:03.138 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.128+0000 7f75c6ad1700 1 --2- 172.21.15.1:0/16306854 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f75c01009e0 0x7f75c0072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:03.138 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.128+0000 7f75c6ad1700 1 --2- 172.21.15.1:0/16306854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75c0101390 0x7f75c0073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:03.138 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.128+0000 7f75c6ad1700 1 -- 172.21.15.1:0/16306854 >> 172.21.15.1:0/16306854 conn(0x7f75c0077530 msgr2=0x7f75c00fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:03.138 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.128+0000 7f75c6ad1700 1 -- 172.21.15.1:0/16306854 shutdown_connections 2023-11-16T16:41:03.138 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:03.128+0000 7f75c6ad1700 1 -- 172.21.15.1:0/16306854 wait complete. 2023-11-16T16:41:03.163 INFO:tasks.cephfs.filesystem:scrub status for tag:360e3f41-0374-491f-bd62-43c1f96d0aa9 - {'path': '/', 'tag': '360e3f41-0374-491f-bd62-43c1f96d0aa9', 'options': 'recursive,force'} 2023-11-16T16:41:03.282 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:03.282 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:03.411 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:03.411 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:03.422 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:03.422 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:03.422 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 59 -lt 100 ']' 2023-11-16T16:41:03.422 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 59 + 1 2023-11-16T16:41:03.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:03 smithi001 ceph-mon[98800]: pgmap v59: 97 pgs: 97 active+clean; 2.0 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 10 op/s 2023-11-16T16:41:03.454 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=60 2023-11-16T16:41:03.454 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:03.492 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:03.492 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:03.493 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:03.493 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:03.493 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:03.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:03 smithi026 ceph-mon[100112]: pgmap v59: 97 pgs: 97 active+clean; 2.0 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 10 op/s 2023-11-16T16:41:03.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:03 smithi053 ceph-mon[100638]: pgmap v59: 97 pgs: 97 active+clean; 2.0 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 10 op/s 2023-11-16T16:41:03.856 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:03.857 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:04.045 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:04.045 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:04.057 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:04.058 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:04.058 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 60 -lt 100 ']' 2023-11-16T16:41:04.058 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 60 + 1 2023-11-16T16:41:04.091 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=61 2023-11-16T16:41:04.091 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:04.116 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:04.117 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:04.118 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:04.118 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:04.118 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:04.582 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:04.582 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:04.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:04 smithi001 ceph-mon[98800]: scrub summary: idle 2023-11-16T16:41:04.703 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:04.704 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:04.714 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:04.714 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:04.714 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 61 -lt 100 ']' 2023-11-16T16:41:04.714 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 61 + 1 2023-11-16T16:41:04.749 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=62 2023-11-16T16:41:04.749 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:04.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:04 smithi026 ceph-mon[100112]: scrub summary: idle 2023-11-16T16:41:04.774 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:04.774 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:04.775 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:04.775 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:04.775 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:04.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:04 smithi053 ceph-mon[100638]: scrub summary: idle 2023-11-16T16:41:05.165 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:05.165 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:05.293 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:05.293 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:05.307 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:05.307 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:05.307 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 62 -lt 100 ']' 2023-11-16T16:41:05.307 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 62 + 1 2023-11-16T16:41:05.341 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=63 2023-11-16T16:41:05.341 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:05.366 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:05.366 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:05.367 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:05.368 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:05.368 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:05.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:05 smithi001 ceph-mon[98800]: pgmap v60: 97 pgs: 97 active+clean; 2.0 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 6 op/s 2023-11-16T16:41:05.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:05 smithi026 ceph-mon[100112]: pgmap v60: 97 pgs: 97 active+clean; 2.0 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 6 op/s 2023-11-16T16:41:05.790 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:05.790 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:05.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:05 smithi053 ceph-mon[100638]: pgmap v60: 97 pgs: 97 active+clean; 2.0 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 6 op/s 2023-11-16T16:41:05.920 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:05.920 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:05.939 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:05.939 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:05.939 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 63 -lt 100 ']' 2023-11-16T16:41:05.940 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 63 + 1 2023-11-16T16:41:05.974 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=64 2023-11-16T16:41:05.974 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:06.018 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:06.018 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:06.019 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:06.019 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:06.020 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:06.473 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:06.474 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:06.585 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:06.585 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:06.606 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:06.607 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:06.607 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 64 -lt 100 ']' 2023-11-16T16:41:06.607 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 64 + 1 2023-11-16T16:41:06.641 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=65 2023-11-16T16:41:06.641 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:06.669 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:06.669 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:06.670 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:06.670 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:06.671 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:06.998 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:06.998 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:07.128 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:07.128 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:07.138 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:07.139 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:07.139 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 65 -lt 100 ']' 2023-11-16T16:41:07.139 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 65 + 1 2023-11-16T16:41:07.174 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=66 2023-11-16T16:41:07.174 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:07.193 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:07.193 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:07.194 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:07.195 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:07.195 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:07.606 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:07.607 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:07.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:07 smithi001 ceph-mon[98800]: pgmap v61: 97 pgs: 97 active+clean; 2.1 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 46 KiB/s wr, 22 op/s 2023-11-16T16:41:07.710 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:07.710 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:07.724 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:07.724 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:07.724 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 66 -lt 100 ']' 2023-11-16T16:41:07.724 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 66 + 1 2023-11-16T16:41:07.757 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=67 2023-11-16T16:41:07.757 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:07.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:07 smithi026 ceph-mon[100112]: pgmap v61: 97 pgs: 97 active+clean; 2.1 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 46 KiB/s wr, 22 op/s 2023-11-16T16:41:07.782 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:07.782 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:07.783 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:07.784 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:07.784 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:07.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:07 smithi053 ceph-mon[100638]: pgmap v61: 97 pgs: 97 active+clean; 2.1 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 46 KiB/s wr, 22 op/s 2023-11-16T16:41:08.232 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:08.232 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:08.360 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:08.360 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:08.372 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:08.372 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:08.372 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 67 -lt 100 ']' 2023-11-16T16:41:08.373 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 67 + 1 2023-11-16T16:41:08.407 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=68 2023-11-16T16:41:08.407 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:08.432 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:08.432 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:08.433 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:08.434 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:08.434 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:08.790 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:08.790 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:08.893 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:08.893 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:08.907 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:08.907 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:08.907 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 68 -lt 100 ']' 2023-11-16T16:41:08.907 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 68 + 1 2023-11-16T16:41:08.940 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=69 2023-11-16T16:41:08.941 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:08.966 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:08.966 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:08.967 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:08.967 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:08.968 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:09.348 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:09.348 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:09.503 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:09.503 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:09.513 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:09.513 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:09.513 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 69 -lt 100 ']' 2023-11-16T16:41:09.514 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 69 + 1 2023-11-16T16:41:09.549 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=70 2023-11-16T16:41:09.549 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:09.567 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:09.567 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:09.568 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:09.568 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:09.568 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:09.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:09 smithi001 ceph-mon[98800]: pgmap v62: 97 pgs: 97 active+clean; 2.1 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 39 KiB/s wr, 19 op/s 2023-11-16T16:41:09.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:09 smithi026 ceph-mon[100112]: pgmap v62: 97 pgs: 97 active+clean; 2.1 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 39 KiB/s wr, 19 op/s 2023-11-16T16:41:09.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:09 smithi053 ceph-mon[100638]: pgmap v62: 97 pgs: 97 active+clean; 2.1 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 39 KiB/s wr, 19 op/s 2023-11-16T16:41:09.964 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:09.965 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:10.094 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:10.095 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:10.106 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:10.106 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:10.107 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 70 -lt 100 ']' 2023-11-16T16:41:10.107 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 70 + 1 2023-11-16T16:41:10.140 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=71 2023-11-16T16:41:10.140 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:10.170 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:10.170 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:10.171 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:10.172 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:10.172 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:10.664 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:10.664 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:10.768 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:10.768 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:10.780 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:10.780 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:10.780 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 71 -lt 100 ']' 2023-11-16T16:41:10.780 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 71 + 1 2023-11-16T16:41:10.815 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=72 2023-11-16T16:41:10.815 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:10.841 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:10.841 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:10.842 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:10.842 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:10.843 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:11.239 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:11.239 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:11.386 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:11.386 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:11.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:11 smithi001 ceph-mon[98800]: pgmap v63: 97 pgs: 97 active+clean; 2.1 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 39 KiB/s wr, 19 op/s 2023-11-16T16:41:11.440 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:11.440 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:11.440 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 72 -lt 100 ']' 2023-11-16T16:41:11.440 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 72 + 1 2023-11-16T16:41:11.473 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=73 2023-11-16T16:41:11.474 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:11.491 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:11.491 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:11.493 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:11.493 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:11.493 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:11.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:11 smithi026 ceph-mon[100112]: pgmap v63: 97 pgs: 97 active+clean; 2.1 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 39 KiB/s wr, 19 op/s 2023-11-16T16:41:11.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:11 smithi053 ceph-mon[100638]: pgmap v63: 97 pgs: 97 active+clean; 2.1 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 39 KiB/s wr, 19 op/s 2023-11-16T16:41:11.847 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:11.848 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:11.976 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:11.976 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:11.988 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:11.988 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:11.988 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 73 -lt 100 ']' 2023-11-16T16:41:11.988 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 73 + 1 2023-11-16T16:41:12.023 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=74 2023-11-16T16:41:12.024 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:12.041 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:12.041 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:12.042 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:12.042 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:12.042 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:12.431 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:12.431 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:12.544 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:12.544 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:12.557 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:12.557 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:12.557 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 74 -lt 100 ']' 2023-11-16T16:41:12.558 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 74 + 1 2023-11-16T16:41:12.590 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=75 2023-11-16T16:41:12.590 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:12.615 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:12.615 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:12.616 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:12.616 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:12.617 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:12.947 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:12.947 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:13.042 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:13.042 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:13.054 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:13.054 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:13.054 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 75 -lt 100 ']' 2023-11-16T16:41:13.055 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 75 + 1 2023-11-16T16:41:13.089 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=76 2023-11-16T16:41:13.090 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:13.116 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:13.116 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:13.117 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:13.117 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:13.118 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:13.489 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:13.490 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:13.603 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:13.603 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:13.615 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:13.615 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:13.615 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 76 -lt 100 ']' 2023-11-16T16:41:13.615 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 76 + 1 2023-11-16T16:41:13.648 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=77 2023-11-16T16:41:13.648 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:13.681 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:13.681 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:13.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:13 smithi001 ceph-mon[98800]: pgmap v64: 97 pgs: 97 active+clean; 2.2 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 49 KiB/s wr, 22 op/s 2023-11-16T16:41:13.683 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:13.683 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:13.683 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:13.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:13 smithi026 ceph-mon[100112]: pgmap v64: 97 pgs: 97 active+clean; 2.2 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 49 KiB/s wr, 22 op/s 2023-11-16T16:41:13.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:13 smithi053 ceph-mon[100638]: pgmap v64: 97 pgs: 97 active+clean; 2.2 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 49 KiB/s wr, 22 op/s 2023-11-16T16:41:14.055 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:14.056 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:14.192 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:14.192 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:14.204 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:14.204 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:14.205 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 77 -lt 100 ']' 2023-11-16T16:41:14.205 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 77 + 1 2023-11-16T16:41:14.239 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=78 2023-11-16T16:41:14.240 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:14.258 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:14.258 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:14.259 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:14.259 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:14.259 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:14.647 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:14.647 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:14.769 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:14.769 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:14.781 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:14.782 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:14.782 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 78 -lt 100 ']' 2023-11-16T16:41:14.782 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 78 + 1 2023-11-16T16:41:14.814 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=79 2023-11-16T16:41:14.815 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:14.839 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:14.839 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:14.841 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:14.841 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:14.841 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:15.205 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:15.205 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:15.359 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:15.359 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:15.373 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:15.373 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:15.373 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 79 -lt 100 ']' 2023-11-16T16:41:15.374 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 79 + 1 2023-11-16T16:41:15.406 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=80 2023-11-16T16:41:15.407 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:15.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:15 smithi001 ceph-mon[98800]: pgmap v65: 97 pgs: 97 active+clean; 2.2 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 42 KiB/s wr, 19 op/s 2023-11-16T16:41:15.445 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:15.445 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:15.447 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:15.447 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:15.447 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:15.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:15 smithi026 ceph-mon[100112]: pgmap v65: 97 pgs: 97 active+clean; 2.2 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 42 KiB/s wr, 19 op/s 2023-11-16T16:41:15.830 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:15.831 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:15.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:15 smithi053 ceph-mon[100638]: pgmap v65: 97 pgs: 97 active+clean; 2.2 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 42 KiB/s wr, 19 op/s 2023-11-16T16:41:15.976 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:15.977 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:15.989 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:15.989 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:15.989 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 80 -lt 100 ']' 2023-11-16T16:41:15.990 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 80 + 1 2023-11-16T16:41:16.022 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=81 2023-11-16T16:41:16.023 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:16.043 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:16.044 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:16.045 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:16.045 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:16.045 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:16.888 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:16.888 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:17.009 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:17.009 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:17.029 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:17.030 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:17.030 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 81 -lt 100 ']' 2023-11-16T16:41:17.030 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 81 + 1 2023-11-16T16:41:17.052 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=82 2023-11-16T16:41:17.052 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:17.064 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:17.064 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:17.065 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:17.066 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:17.066 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:17.513 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:17.513 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:17.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:17 smithi001 ceph-mon[98800]: pgmap v66: 97 pgs: 97 active+clean; 2.3 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 55 KiB/s wr, 23 op/s 2023-11-16T16:41:17.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:17 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:41:17.708 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:17.708 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:17.720 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:17.721 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:17.721 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 82 -lt 100 ']' 2023-11-16T16:41:17.721 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 82 + 1 2023-11-16T16:41:17.756 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=83 2023-11-16T16:41:17.756 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:17.757 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:17 smithi026 ceph-mon[100112]: pgmap v66: 97 pgs: 97 active+clean; 2.3 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 55 KiB/s wr, 23 op/s 2023-11-16T16:41:17.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:17 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:41:17.791 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:17.791 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:17.792 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:17.792 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:17.792 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:17.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:17 smithi053 ceph-mon[100638]: pgmap v66: 97 pgs: 97 active+clean; 2.3 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 55 KiB/s wr, 23 op/s 2023-11-16T16:41:17.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:17 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:41:18.355 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:18.355 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:18.500 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:18.500 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:18.522 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:18.522 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:18.522 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 83 -lt 100 ']' 2023-11-16T16:41:18.522 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 83 + 1 2023-11-16T16:41:18.560 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=84 2023-11-16T16:41:18.561 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:18.627 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:18.628 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:18.628 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:18.629 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:18.629 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:18.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:18 smithi001 ceph-mon[98800]: pgmap v67: 97 pgs: 97 active+clean; 2.3 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 7 op/s 2023-11-16T16:41:18.757 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:18 smithi026 ceph-mon[100112]: pgmap v67: 97 pgs: 97 active+clean; 2.3 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 7 op/s 2023-11-16T16:41:18.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:18 smithi053 ceph-mon[100638]: pgmap v67: 97 pgs: 97 active+clean; 2.3 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 7 op/s 2023-11-16T16:41:19.430 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:19.430 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:19.628 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:19.629 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:19.647 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:19.647 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:19.647 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 84 -lt 100 ']' 2023-11-16T16:41:19.647 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 84 + 1 2023-11-16T16:41:19.686 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=85 2023-11-16T16:41:19.686 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:19.815 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:19.815 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:19.816 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:19.816 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:19.816 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:20.463 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:20.463 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:20.635 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:20.636 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:20.647 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:20.647 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:20.647 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 85 -lt 100 ']' 2023-11-16T16:41:20.647 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 85 + 1 2023-11-16T16:41:20.694 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=86 2023-11-16T16:41:20.694 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:20.772 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:20.772 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:20.773 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:20.773 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:20.773 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:20.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:20 smithi001 ceph-mon[98800]: pgmap v68: 97 pgs: 97 active+clean; 2.3 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 7 op/s 2023-11-16T16:41:21.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:20 smithi026 ceph-mon[100112]: pgmap v68: 97 pgs: 97 active+clean; 2.3 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 7 op/s 2023-11-16T16:41:21.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:20 smithi053 ceph-mon[100638]: pgmap v68: 97 pgs: 97 active+clean; 2.3 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 7 op/s 2023-11-16T16:41:21.479 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:21.480 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:21.609 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:21.609 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:21.636 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:21.636 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:21.637 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 86 -lt 100 ']' 2023-11-16T16:41:21.637 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 86 + 1 2023-11-16T16:41:21.677 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=87 2023-11-16T16:41:21.677 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:21.699 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:21.699 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:21.700 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:21.700 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:21.700 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:22.229 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:22.230 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:22.408 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:22.408 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:22.419 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:22.420 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:22.420 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 87 -lt 100 ']' 2023-11-16T16:41:22.420 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 87 + 1 2023-11-16T16:41:22.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:22 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:41:22.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:22 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:41:22.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:22 smithi001 ceph-mon[98800]: pgmap v69: 97 pgs: 97 active+clean; 2.4 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 10 op/s 2023-11-16T16:41:22.452 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=88 2023-11-16T16:41:22.452 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:22.475 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:22.476 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:22.477 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:22.477 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:22.477 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:22.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:22 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:41:22.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:22 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:41:22.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:22 smithi026 ceph-mon[100112]: pgmap v69: 97 pgs: 97 active+clean; 2.4 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 10 op/s 2023-11-16T16:41:22.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:22 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:41:22.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:22 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:41:22.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:22 smithi053 ceph-mon[100638]: pgmap v69: 97 pgs: 97 active+clean; 2.4 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 10 op/s 2023-11-16T16:41:22.954 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:22.954 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:23.102 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:23.102 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:23.121 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:23.121 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:23.121 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 88 -lt 100 ']' 2023-11-16T16:41:23.121 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 88 + 1 2023-11-16T16:41:23.152 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=89 2023-11-16T16:41:23.152 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:23.209 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:23.209 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:23.210 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:23.210 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:23.210 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:24.087 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:24.088 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:24.259 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:24.259 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:24.295 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:24.295 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:24.295 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 89 -lt 100 ']' 2023-11-16T16:41:24.295 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 89 + 1 2023-11-16T16:41:24.326 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=90 2023-11-16T16:41:24.327 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:24.406 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:24.406 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:24.408 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:24.408 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:24.408 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:24.849 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:24 smithi001 ceph-mon[98800]: pgmap v70: 97 pgs: 97 active+clean; 2.4 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-11-16T16:41:25.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:24 smithi026 ceph-mon[100112]: pgmap v70: 97 pgs: 97 active+clean; 2.4 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-11-16T16:41:25.012 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:25.012 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:25.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:24 smithi053 ceph-mon[100638]: pgmap v70: 97 pgs: 97 active+clean; 2.4 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 7 op/s 2023-11-16T16:41:25.234 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:25.234 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:25.260 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:25.261 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:25.261 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 90 -lt 100 ']' 2023-11-16T16:41:25.261 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 90 + 1 2023-11-16T16:41:25.326 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=91 2023-11-16T16:41:25.326 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:25.349 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:25.349 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:25.350 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:25.350 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:25.351 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:25.895 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:25.896 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:26.042 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:26.042 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:26.061 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:26.061 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:26.061 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 91 -lt 100 ']' 2023-11-16T16:41:26.061 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 91 + 1 2023-11-16T16:41:26.101 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=92 2023-11-16T16:41:26.101 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:26.114 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:26.114 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:26.115 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:26.115 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:26.115 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:26.620 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:26.621 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:26.767 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:26.767 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:26.779 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:26.779 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:26.779 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 92 -lt 100 ']' 2023-11-16T16:41:26.779 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 92 + 1 2023-11-16T16:41:26.813 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=93 2023-11-16T16:41:26.813 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:26.864 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:26.864 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:26.865 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:26.865 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:26.865 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:26.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:26 smithi001 ceph-mon[98800]: pgmap v71: 97 pgs: 97 active+clean; 2.6 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 10 op/s 2023-11-16T16:41:26.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:26 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:26.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:26 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:41:26.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:26 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:41:27.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:26 smithi026 ceph-mon[100112]: pgmap v71: 97 pgs: 97 active+clean; 2.6 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 10 op/s 2023-11-16T16:41:27.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:26 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:27.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:26 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:41:27.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:26 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:41:27.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:26 smithi053 ceph-mon[100638]: pgmap v71: 97 pgs: 97 active+clean; 2.6 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 10 op/s 2023-11-16T16:41:27.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:26 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:41:27.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:26 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:41:27.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:26 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:41:27.270 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:27.271 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:27.417 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:27.417 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:27.427 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:27.427 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:27.427 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 93 -lt 100 ']' 2023-11-16T16:41:27.428 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 93 + 1 2023-11-16T16:41:27.463 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=94 2023-11-16T16:41:27.463 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:27.514 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:27.514 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:27.515 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:27.515 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:27.515 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:27.846 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:27 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:41:27.846 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:27 smithi001 ceph-mon[98800]: Service mds.all has no related daemon already placed. 2023-11-16T16:41:27.846 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:27 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:41:27.846 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:27 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:41:27.912 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:27.912 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:28.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:27 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:41:28.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:27 smithi026 ceph-mon[100112]: Service mds.all has no related daemon already placed. 2023-11-16T16:41:28.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:27 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:41:28.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:27 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:41:28.017 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:28.017 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:28.029 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:28.029 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:28.029 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 94 -lt 100 ']' 2023-11-16T16:41:28.029 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 94 + 1 2023-11-16T16:41:28.063 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=95 2023-11-16T16:41:28.063 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:28.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:27 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:41:28.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:27 smithi053 ceph-mon[100638]: Service mds.all has no related daemon already placed. 2023-11-16T16:41:28.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:27 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:41:28.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:27 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:41:28.088 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:28.088 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:28.089 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:28.089 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:28.089 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:28.453 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:28.454 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:28.583 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:28.584 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:28.594 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:28.594 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:28.594 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 95 -lt 100 ']' 2023-11-16T16:41:28.594 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 95 + 1 2023-11-16T16:41:28.630 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=96 2023-11-16T16:41:28.630 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:28.648 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:28.648 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:28.649 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:28.649 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:28.650 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:28.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:28 smithi001 ceph-mon[98800]: pgmap v72: 97 pgs: 97 active+clean; 2.6 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 7 op/s 2023-11-16T16:41:29.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:28 smithi026 ceph-mon[100112]: pgmap v72: 97 pgs: 97 active+clean; 2.6 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 7 op/s 2023-11-16T16:41:29.020 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:29.020 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:29.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:28 smithi053 ceph-mon[100638]: pgmap v72: 97 pgs: 97 active+clean; 2.6 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 7 op/s 2023-11-16T16:41:29.184 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:29.184 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:29.195 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:29.196 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:29.196 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 96 -lt 100 ']' 2023-11-16T16:41:29.196 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 96 + 1 2023-11-16T16:41:29.229 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=97 2023-11-16T16:41:29.229 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:29.254 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:29.254 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:29.255 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:29.256 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:29.256 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:29.712 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:29.712 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:29.842 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:29.842 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:29.852 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:29.852 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:29.852 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 97 -lt 100 ']' 2023-11-16T16:41:29.853 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 97 + 1 2023-11-16T16:41:29.887 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=98 2023-11-16T16:41:29.888 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:29.905 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:29.906 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:29.907 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:29.907 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:29.907 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:30.270 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:30.270 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:30.415 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:30.415 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:30.429 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:30.429 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:30.429 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 98 -lt 100 ']' 2023-11-16T16:41:30.429 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 98 + 1 2023-11-16T16:41:30.462 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=99 2023-11-16T16:41:30.463 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:30.487 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:30.487 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:30.488 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:30.489 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:30.489 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:30.908 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:30 smithi001 ceph-mon[98800]: pgmap v73: 97 pgs: 97 active+clean; 2.6 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 7 op/s 2023-11-16T16:41:30.953 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:30.953 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:31.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:30 smithi026 ceph-mon[100112]: pgmap v73: 97 pgs: 97 active+clean; 2.6 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 7 op/s 2023-11-16T16:41:31.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:30 smithi053 ceph-mon[100638]: pgmap v73: 97 pgs: 97 active+clean; 2.6 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 7 op/s 2023-11-16T16:41:31.123 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:31.124 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:31.135 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:31.135 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:31.136 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 99 -lt 100 ']' 2023-11-16T16:41:31.136 INFO:tasks.workunit.client.0.smithi001.stderr:++ expr 99 + 1 2023-11-16T16:41:31.171 INFO:tasks.workunit.client.0.smithi001.stderr:+ c=100 2023-11-16T16:41:31.171 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:31.196 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:31.196 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:31.198 INFO:tasks.workunit.client.0.smithi001.stderr:+ c1=2 2023-11-16T16:41:31.198 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo 3 2023-11-16T16:41:31.198 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-16T16:41:31.536 INFO:tasks.workunit.client.0.smithi001.stderr:++ getfacl d1 2023-11-16T16:41:31.536 INFO:tasks.workunit.client.0.smithi001.stderr:++ grep -c nobody:rw 2023-11-16T16:41:31.666 INFO:tasks.workunit.client.0.smithi001.stderr:+ c2=2 2023-11-16T16:41:31.667 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:31.679 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:31.679 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 2 -ne 2 ']' 2023-11-16T16:41:31.679 INFO:tasks.workunit.client.0.smithi001.stderr:+ '[' 100 -lt 100 ']' 2023-11-16T16:41:31.687 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir d1 2023-11-16T16:41:31.729 INFO:tasks.workunit.client.0.smithi001.stderr:+ setfattr -n system.posix_acl_access -v 0x02000000 d1 2023-11-16T16:41:31.736 INFO:tasks.workunit.client.0.smithi001.stderr:+ setfattr -n system.posix_acl_default -v 0x02000000 . 2023-11-16T16:41:31.739 INFO:tasks.workunit.client.0.smithi001.stderr:+ expect_failure getfattr -n system.posix_acl_access d1 2023-11-16T16:41:31.739 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n system.posix_acl_access d1 2023-11-16T16:41:31.795 INFO:tasks.workunit.client.0.smithi001.stderr:d1: system.posix_acl_access: No such attribute 2023-11-16T16:41:31.795 INFO:tasks.workunit.client.0.smithi001.stderr:+ return 0 2023-11-16T16:41:31.795 INFO:tasks.workunit.client.0.smithi001.stderr:+ expect_failure getfattr -n system.posix_acl_default . 2023-11-16T16:41:31.795 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n system.posix_acl_default . 2023-11-16T16:41:31.796 INFO:tasks.workunit.client.0.smithi001.stderr:.: system.posix_acl_default: No such attribute 2023-11-16T16:41:31.796 INFO:tasks.workunit.client.0.smithi001.stderr:+ return 0 2023-11-16T16:41:31.796 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir d1 2023-11-16T16:41:31.799 INFO:tasks.workunit.client.0.smithi001.stderr:+ cd .. 2023-11-16T16:41:31.799 INFO:tasks.workunit.client.0.smithi001.stderr:+ rmdir testdir 2023-11-16T16:41:31.821 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo OK 2023-11-16T16:41:31.822 INFO:tasks.workunit.client.0.smithi001.stdout:OK 2023-11-16T16:41:31.822 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-16T16:41:31.822 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-16T16:41:32.244 INFO:tasks.workunit:Running workunit fs/misc/chmod.sh... 2023-11-16T16:41:32.244 DEBUG:teuthology.orchestra.run.smithi001:workunit test fs/misc/chmod.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/chmod.sh 2023-11-16T16:41:32.328 INFO:tasks.workunit.client.0.smithi001.stderr:+ set -e 2023-11-16T16:41:32.328 INFO:tasks.workunit.client.0.smithi001.stderr:+ file=test_chmod.153700 2023-11-16T16:41:32.328 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo foo 2023-11-16T16:41:32.332 INFO:tasks.workunit.client.0.smithi001.stderr:+ test 0 '!=' 0 2023-11-16T16:41:32.332 INFO:tasks.workunit.client.0.smithi001.stderr:+ chmod 400 test_chmod.153700 2023-11-16T16:41:32.334 INFO:tasks.workunit.client.0.smithi001.stderr:+ test 0 '!=' 0 2023-11-16T16:41:32.334 INFO:tasks.workunit.client.0.smithi001.stderr:+ check_perms test_chmod.153700 -r-------- 2023-11-16T16:41:32.334 INFO:tasks.workunit.client.0.smithi001.stderr:+ file=test_chmod.153700 2023-11-16T16:41:32.334 INFO:tasks.workunit.client.0.smithi001.stderr:++ ls -la test_chmod.153700 2023-11-16T16:41:32.336 INFO:tasks.workunit.client.0.smithi001.stderr:+ r='-r--------. 1 ubuntu ubuntu 4 Nov 16 16:41 test_chmod.153700' 2023-11-16T16:41:32.336 INFO:tasks.workunit.client.0.smithi001.stderr:+ test 0 '!=' 0 2023-11-16T16:41:32.336 INFO:tasks.workunit.client.0.smithi001.stderr:+ perms=-r-------- 2023-11-16T16:41:32.337 INFO:tasks.workunit.client.0.smithi001.stderr:++ echo -r--------. 1 ubuntu ubuntu 4 Nov 16 16:41 test_chmod.153700 2023-11-16T16:41:32.337 INFO:tasks.workunit.client.0.smithi001.stderr:++ awk '{print $1}' 2023-11-16T16:41:32.410 INFO:tasks.workunit.client.0.smithi001.stderr:+ test -r-------- '!=' -r--------. 2023-11-16T16:41:32.411 INFO:tasks.workunit.client.0.smithi001.stderr:++ echo -r--------. 1 ubuntu ubuntu 4 Nov 16 16:41 test_chmod.153700 2023-11-16T16:41:32.411 INFO:tasks.workunit.client.0.smithi001.stderr:++ awk '{print $1}' 2023-11-16T16:41:32.413 INFO:tasks.workunit.client.0.smithi001.stderr:+ test -r--------. '!=' -r--------. 2023-11-16T16:41:32.413 INFO:tasks.workunit.client.0.smithi001.stderr:+ set +e 2023-11-16T16:41:32.413 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo bar 2023-11-16T16:41:32.413 INFO:tasks.workunit.client.0.smithi001.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/chmod.sh: line 40: test_chmod.153700: Permission denied 2023-11-16T16:41:32.413 INFO:tasks.workunit.client.0.smithi001.stderr:+ test 1 = 0 2023-11-16T16:41:32.413 INFO:tasks.workunit.client.0.smithi001.stderr:+ set -e 2023-11-16T16:41:32.414 INFO:tasks.workunit.client.0.smithi001.stderr:+ chmod 600 test_chmod.153700 2023-11-16T16:41:32.414 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo bar 2023-11-16T16:41:32.415 INFO:tasks.workunit.client.0.smithi001.stderr:+ test 0 '!=' 0 2023-11-16T16:41:32.415 INFO:tasks.workunit.client.0.smithi001.stderr:+ check_perms test_chmod.153700 -rw------- 2023-11-16T16:41:32.415 INFO:tasks.workunit.client.0.smithi001.stderr:+ file=test_chmod.153700 2023-11-16T16:41:32.415 INFO:tasks.workunit.client.0.smithi001.stderr:++ ls -la test_chmod.153700 2023-11-16T16:41:32.416 INFO:tasks.workunit.client.0.smithi001.stderr:+ r='-rw-------. 1 ubuntu ubuntu 8 Nov 16 16:41 test_chmod.153700' 2023-11-16T16:41:32.416 INFO:tasks.workunit.client.0.smithi001.stderr:+ test 0 '!=' 0 2023-11-16T16:41:32.416 INFO:tasks.workunit.client.0.smithi001.stderr:+ perms=-rw------- 2023-11-16T16:41:32.416 INFO:tasks.workunit.client.0.smithi001.stderr:++ echo -rw-------. 1 ubuntu ubuntu 8 Nov 16 16:41 test_chmod.153700 2023-11-16T16:41:32.416 INFO:tasks.workunit.client.0.smithi001.stderr:++ awk '{print $1}' 2023-11-16T16:41:32.417 INFO:tasks.workunit.client.0.smithi001.stderr:+ test -rw------- '!=' -rw-------. 2023-11-16T16:41:32.418 INFO:tasks.workunit.client.0.smithi001.stderr:++ echo -rw-------. 1 ubuntu ubuntu 8 Nov 16 16:41 test_chmod.153700 2023-11-16T16:41:32.418 INFO:tasks.workunit.client.0.smithi001.stderr:++ awk '{print $1}' 2023-11-16T16:41:32.420 INFO:tasks.workunit.client.0.smithi001.stderr:+ test -rw-------. '!=' -rw-------. 2023-11-16T16:41:32.420 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo foo 2023-11-16T16:41:32.420 INFO:tasks.workunit.client.0.smithi001.stderr:+ test 0 '!=' 0 2023-11-16T16:41:32.421 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-16T16:41:32.421 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-16T16:41:32.473 INFO:tasks.workunit:Running workunit fs/misc/dac_override.sh... 2023-11-16T16:41:32.473 DEBUG:teuthology.orchestra.run.smithi001:workunit test fs/misc/dac_override.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/dac_override.sh 2023-11-16T16:41:32.499 INFO:tasks.workunit.client.0.smithi001.stderr:+ set -e 2023-11-16T16:41:32.499 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir -p testdir 2023-11-16T16:41:32.501 INFO:tasks.workunit.client.0.smithi001.stderr:+ file=test_chmod.153759 2023-11-16T16:41:32.501 INFO:tasks.workunit.client.0.smithi001.stderr:+ echo foo 2023-11-16T16:41:32.502 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo chmod 600 testdir 2023-11-16T16:41:32.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:32 smithi026 ceph-mon[100112]: pgmap v74: 97 pgs: 97 active+clean; 2.7 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 10 op/s 2023-11-16T16:41:32.511 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:32 smithi001 ceph-mon[98800]: pgmap v74: 97 pgs: 97 active+clean; 2.7 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 10 op/s 2023-11-16T16:41:32.522 INFO:tasks.workunit.client.0.smithi001.stderr:+ expect_failure cat testdir/test_chmod.153759 2023-11-16T16:41:32.522 INFO:tasks.workunit.client.0.smithi001.stderr:+ cat testdir/test_chmod.153759 2023-11-16T16:41:32.524 INFO:tasks.workunit.client.0.smithi001.stderr:cat: testdir/test_chmod.153759: Permission denied 2023-11-16T16:41:32.524 INFO:tasks.workunit.client.0.smithi001.stderr:+ return 0 2023-11-16T16:41:32.524 INFO:tasks.workunit.client.0.smithi001.stderr:+ sudo cat testdir/test_chmod.153759 2023-11-16T16:41:32.534 INFO:tasks.workunit.client.0.smithi001.stdout:foo 2023-11-16T16:41:32.536 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-16T16:41:32.536 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-16T16:41:32.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:32 smithi053 ceph-mon[100638]: pgmap v74: 97 pgs: 97 active+clean; 2.7 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 10 op/s 2023-11-16T16:41:32.630 INFO:tasks.workunit:Running workunit fs/misc/direct_io.py... 2023-11-16T16:41:32.631 DEBUG:teuthology.orchestra.run.smithi001:workunit test fs/misc/direct_io.py> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/direct_io.py 2023-11-16T16:41:33.165 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-16T16:41:33.433 INFO:tasks.workunit.client.0.smithi001.stdout:ok 2023-11-16T16:41:33.439 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-16T16:41:33.439 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-16T16:41:33.487 INFO:tasks.workunit:Running workunit fs/misc/dirfrag.sh... 2023-11-16T16:41:33.487 DEBUG:teuthology.orchestra.run.smithi001:workunit test fs/misc/dirfrag.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/dirfrag.sh 2023-11-16T16:41:33.532 INFO:tasks.workunit.client.0.smithi001.stdout:creating folder hierarchy 2023-11-16T16:41:33.628 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.625+0000 7fea7c7fd700 1 -- 172.21.15.1:0/3349543118 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea740ff8f0 msgr2=0x7fea740ffd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:33.628 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.625+0000 7fea7c7fd700 1 --2- 172.21.15.1:0/3349543118 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea740ff8f0 0x7fea740ffd60 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fea640099c0 tx=0x7fea64009cd0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.628 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.625+0000 7fea7c7fd700 1 -- 172.21.15.1:0/3349543118 shutdown_connections 2023-11-16T16:41:33.628 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.625+0000 7fea7c7fd700 1 --2- 172.21.15.1:0/3349543118 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea741002a0 0x7fea7410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.628 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.625+0000 7fea7c7fd700 1 --2- 172.21.15.1:0/3349543118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea74104140 0x7fea74104510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.628 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.625+0000 7fea7c7fd700 1 --2- 172.21.15.1:0/3349543118 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea740ff8f0 0x7fea740ffd60 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.628 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.625+0000 7fea7c7fd700 1 -- 172.21.15.1:0/3349543118 >> 172.21.15.1:0/3349543118 conn(0x7fea74075880 msgr2=0x7fea74075c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:33.629 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.625+0000 7fea7c7fd700 1 -- 172.21.15.1:0/3349543118 shutdown_connections 2023-11-16T16:41:33.629 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.626+0000 7fea7c7fd700 1 -- 172.21.15.1:0/3349543118 wait complete. 2023-11-16T16:41:33.632 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.626+0000 7fea7c7fd700 1 Processor -- start 2023-11-16T16:41:33.632 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.626+0000 7fea7c7fd700 1 -- start start 2023-11-16T16:41:33.632 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.627+0000 7fea7c7fd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea740ff8f0 0x7fea74197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:33.632 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.627+0000 7fea7c7fd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea741002a0 0x7fea74197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:33.632 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.627+0000 7fea7c7fd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea74104140 0x7fea741a21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:33.632 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.627+0000 7fea7c7fd700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea74197fb0 con 0x7fea741002a0 2023-11-16T16:41:33.632 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.627+0000 7fea7c7fd700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea741a26f0 con 0x7fea740ff8f0 2023-11-16T16:41:33.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.627+0000 7fea7c7fd700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea741a28b0 con 0x7fea74104140 2023-11-16T16:41:33.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.627+0000 7fea79d98700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea741002a0 0x7fea74197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:33.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.627+0000 7fea7a599700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea740ff8f0 0x7fea74197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:33.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.627+0000 7fea79d98700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea741002a0 0x7fea74197670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44568/0 (socket says 172.21.15.1:44568) 2023-11-16T16:41:33.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.627+0000 7fea79d98700 1 -- 172.21.15.1:0/3670831090 learned_addr learned my addr 172.21.15.1:0/3670831090 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:41:33.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.627+0000 7fea7ad9a700 1 --2- 172.21.15.1:0/3670831090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea74104140 0x7fea741a21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:33.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.627+0000 7fea7a599700 1 -- 172.21.15.1:0/3670831090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea74104140 msgr2=0x7fea741a21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:33.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.627+0000 7fea7a599700 1 --2- 172.21.15.1:0/3670831090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea74104140 0x7fea741a21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.627+0000 7fea7a599700 1 -- 172.21.15.1:0/3670831090 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea741002a0 msgr2=0x7fea74197670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:33.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.627+0000 7fea7a599700 1 --2- 172.21.15.1:0/3670831090 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea741002a0 0x7fea74197670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.627+0000 7fea7a599700 1 -- 172.21.15.1:0/3670831090 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea70009520 con 0x7fea740ff8f0 2023-11-16T16:41:33.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.627+0000 7fea79d98700 1 --2- 172.21.15.1:0/3670831090 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea741002a0 0x7fea74197670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:41:33.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.627+0000 7fea7a599700 1 --2- 172.21.15.1:0/3670831090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea740ff8f0 0x7fea74197130 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fea7000e9f0 tx=0x7fea7000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:33.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.628+0000 7fea6b7fe700 1 -- 172.21.15.1:0/3670831090 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7000cb70 con 0x7fea740ff8f0 2023-11-16T16:41:33.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.628+0000 7fea6b7fe700 1 -- 172.21.15.1:0/3670831090 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea7000ee90 con 0x7fea740ff8f0 2023-11-16T16:41:33.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.628+0000 7fea7c7fd700 1 -- 172.21.15.1:0/3670831090 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea640095f0 con 0x7fea740ff8f0 2023-11-16T16:41:33.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.628+0000 7fea6b7fe700 1 -- 172.21.15.1:0/3670831090 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea70003cf0 con 0x7fea740ff8f0 2023-11-16T16:41:33.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.628+0000 7fea7c7fd700 1 -- 172.21.15.1:0/3670831090 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea7410f1d0 con 0x7fea740ff8f0 2023-11-16T16:41:33.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.630+0000 7fea6b7fe700 1 -- 172.21.15.1:0/3670831090 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fea7000ccd0 con 0x7fea740ff8f0 2023-11-16T16:41:33.636 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.631+0000 7fea6b7fe700 1 --2- 172.21.15.1:0/3670831090 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fea600a12c0 0x7fea600a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:33.636 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.631+0000 7fea79d98700 1 --2- 172.21.15.1:0/3670831090 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fea600a12c0 0x7fea600a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:33.636 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.631+0000 7fea6b7fe700 1 -- 172.21.15.1:0/3670831090 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fea70014070 con 0x7fea740ff8f0 2023-11-16T16:41:33.636 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.631+0000 7fea79d98700 1 --2- 172.21.15.1:0/3670831090 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fea600a12c0 0x7fea600a3770 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fea740ff0b0 tx=0x7fea6401a040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:33.670 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.662+0000 7fea88989b80 1 Processor -- start 2023-11-16T16:41:33.670 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.662+0000 7fea88989b80 1 -- start start 2023-11-16T16:41:33.670 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.662+0000 7fea88989b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4ef488b60 0x55f4ef488f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:33.671 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.662+0000 7fea88989b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4ef489470 0x55f4ef48d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:33.671 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.662+0000 7fea88989b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ef48d8b0 0x55f4ef48dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:33.671 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.662+0000 7fea88989b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4ef38e060 con 0x55f4ef489470 2023-11-16T16:41:33.671 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.662+0000 7fea88989b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4ef3e08a0 con 0x55f4ef48d8b0 2023-11-16T16:41:33.671 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.662+0000 7fea88989b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4ef3d9d30 con 0x55f4ef488b60 2023-11-16T16:41:33.671 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.662+0000 7fea79d98700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4ef489470 0x55f4ef48d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:33.671 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.662+0000 7fea7ad9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ef48d8b0 0x55f4ef48dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:33.671 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.662+0000 7fea79d98700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4ef489470 0x55f4ef48d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44576/0 (socket says 172.21.15.1:44576) 2023-11-16T16:41:33.672 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.662+0000 7fea79d98700 1 -- 172.21.15.1:0/1463157222 learned_addr learned my addr 172.21.15.1:0/1463157222 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:41:33.672 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.662+0000 7fea7a599700 1 --2- 172.21.15.1:0/1463157222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4ef488b60 0x55f4ef488f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:33.672 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.663+0000 7fea7ad9a700 1 -- 172.21.15.1:0/1463157222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4ef488b60 msgr2=0x55f4ef488f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:33.672 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.663+0000 7fea7ad9a700 1 --2- 172.21.15.1:0/1463157222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4ef488b60 0x55f4ef488f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.672 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.663+0000 7fea7ad9a700 1 -- 172.21.15.1:0/1463157222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4ef489470 msgr2=0x55f4ef48d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:33.672 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.663+0000 7fea7ad9a700 1 --2- 172.21.15.1:0/1463157222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4ef489470 0x55f4ef48d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.672 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.663+0000 7fea7ad9a700 1 -- 172.21.15.1:0/1463157222 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4ef25b630 con 0x55f4ef48d8b0 2023-11-16T16:41:33.672 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.663+0000 7fea79d98700 1 --2- 172.21.15.1:0/1463157222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4ef489470 0x55f4ef48d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:41:33.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.663+0000 7fea7ad9a700 1 --2- 172.21.15.1:0/1463157222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ef48d8b0 0x55f4ef48dd00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fea6c003ef0 tx=0x7fea6c009ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=654ac4598a183e84 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:33.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.663+0000 7fea68ff9700 1 -- 172.21.15.1:0/1463157222 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6c019070 con 0x55f4ef48d8b0 2023-11-16T16:41:33.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.663+0000 7fea68ff9700 1 -- 172.21.15.1:0/1463157222 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea6c00f040 con 0x55f4ef48d8b0 2023-11-16T16:41:33.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.663+0000 7fea68ff9700 1 -- 172.21.15.1:0/1463157222 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6c007960 con 0x55f4ef48d8b0 2023-11-16T16:41:33.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.664+0000 7fea88989b80 1 -- 172.21.15.1:0/1463157222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ef48d8b0 msgr2=0x55f4ef48dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:33.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.664+0000 7fea88989b80 1 --2- 172.21.15.1:0/1463157222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ef48d8b0 0x55f4ef48dd00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fea6c003ef0 tx=0x7fea6c009ba0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.664+0000 7fea88989b80 1 -- 172.21.15.1:0/1463157222 shutdown_connections 2023-11-16T16:41:33.674 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.664+0000 7fea88989b80 1 --2- 172.21.15.1:0/1463157222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ef48d8b0 0x55f4ef48dd00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.674 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.664+0000 7fea88989b80 1 --2- 172.21.15.1:0/1463157222 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4ef488b60 0x55f4ef488f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.674 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.664+0000 7fea88989b80 1 --2- 172.21.15.1:0/1463157222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4ef489470 0x55f4ef48d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.674 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.664+0000 7fea88989b80 1 -- 172.21.15.1:0/1463157222 >> 172.21.15.1:0/1463157222 conn(0x55f4ef47e360 msgr2=0x55f4ef483b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:33.674 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.664+0000 7fea88989b80 1 -- 172.21.15.1:0/1463157222 shutdown_connections 2023-11-16T16:41:33.675 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.664+0000 7fea88989b80 1 -- 172.21.15.1:0/1463157222 wait complete. 2023-11-16T16:41:33.675 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.664+0000 7fea88989b80 1 -- start start 2023-11-16T16:41:33.675 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.665+0000 7fea88989b80 1 Processor -- start 2023-11-16T16:41:33.675 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.665+0000 7fea88989b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4ef488b60 0x55f4ef494f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:33.675 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.665+0000 7fea88989b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ef489470 0x55f4ef492250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:33.675 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.665+0000 7fea88989b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4ef48d8b0 0x55f4ef4927f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:33.675 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.665+0000 7fea88989b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4ef37b8e0 con 0x55f4ef488b60 2023-11-16T16:41:33.675 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.665+0000 7fea88989b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4ef37b580 con 0x55f4ef489470 2023-11-16T16:41:33.676 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.665+0000 7fea88989b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4ef25d920 con 0x55f4ef48d8b0 2023-11-16T16:41:33.676 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.665+0000 7fea88989b80 10 client.0 _ll_register_callbacks cb 0x55f4ef399780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:41:33.676 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.665+0000 7fea7a599700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4ef488b60 0x55f4ef494f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:33.676 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.665+0000 7fea7a599700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4ef488b60 0x55f4ef494f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44580/0 (socket says 172.21.15.1:44580) 2023-11-16T16:41:33.676 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.665+0000 7fea7a599700 1 -- 172.21.15.1:0/2726657519 learned_addr learned my addr 172.21.15.1:0/2726657519 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:41:33.676 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.665+0000 7fea79d98700 1 --2- 172.21.15.1:0/2726657519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ef489470 0x55f4ef492250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:33.676 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.665+0000 7fea7ad9a700 1 --2- 172.21.15.1:0/2726657519 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4ef48d8b0 0x55f4ef4927f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:33.676 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.665+0000 7fea7a599700 1 -- 172.21.15.1:0/2726657519 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4ef48d8b0 msgr2=0x55f4ef4927f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:33.677 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.665+0000 7fea7a599700 1 --2- 172.21.15.1:0/2726657519 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4ef48d8b0 0x55f4ef4927f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.677 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.665+0000 7fea7a599700 1 -- 172.21.15.1:0/2726657519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ef489470 msgr2=0x55f4ef492250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:33.677 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.665+0000 7fea7a599700 1 --2- 172.21.15.1:0/2726657519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ef489470 0x55f4ef492250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.677 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.665+0000 7fea7a599700 1 -- 172.21.15.1:0/2726657519 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea6c003b50 con 0x55f4ef488b60 2023-11-16T16:41:33.677 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.665+0000 7fea79d98700 1 --2- 172.21.15.1:0/2726657519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ef489470 0x55f4ef492250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:41:33.677 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.666+0000 7fea7a599700 1 --2- 172.21.15.1:0/2726657519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4ef488b60 0x55f4ef494f60 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fea70096b10 tx=0x7fea7009a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3141d05b6aa5c2c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:33.677 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.666+0000 7fea5dffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:41:33.677 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.666+0000 7fea5dffb700 1 -- 172.21.15.1:0/2726657519 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea700a24f0 con 0x55f4ef488b60 2023-11-16T16:41:33.678 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.666+0000 7fea5dffb700 1 -- 172.21.15.1:0/2726657519 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fea700b8300 con 0x55f4ef488b60 2023-11-16T16:41:33.678 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.666+0000 7fea88989b80 1 -- 172.21.15.1:0/2726657519 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f4ef47bb20 con 0x55f4ef488b60 2023-11-16T16:41:33.678 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.666+0000 7fea5dffb700 1 -- 172.21.15.1:0/2726657519 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea700b84c0 con 0x55f4ef488b60 2023-11-16T16:41:33.678 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.666+0000 7fea88989b80 10 client.15876 fetch_fsmap learned FSMap version 23 2023-11-16T16:41:33.678 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.666+0000 7fea88989b80 1 -- 172.21.15.1:0/2726657519 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55f4ef3a0190 con 0x55f4ef488b60 2023-11-16T16:41:33.678 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.666+0000 7fea5dffb700 1 -- 172.21.15.1:0/2726657519 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f4ef47bb20 con 0x55f4ef488b60 2023-11-16T16:41:33.678 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.666+0000 7fea5dffb700 1 -- 172.21.15.1:0/2726657519 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fea70099000 con 0x55f4ef488b60 2023-11-16T16:41:33.679 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.666+0000 7fea88989b80 10 client.15876 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:41:33.679 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.666+0000 7fea88989b80 10 client.15876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:41:33.679 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.666+0000 7fea88989b80 20 client.15876 populate_metadata read hostname 'smithi001' 2023-11-16T16:41:33.679 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.666+0000 7fea88989b80 1 --2- 172.21.15.1:0/2726657519 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55f4ef4a27c0 0x55f4ef4a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:33.679 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.666+0000 7fea88989b80 4 client.15876 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:41:33.679 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.666+0000 7fea88989b80 1 -- 172.21.15.1:0/2726657519 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f4ef39c0a0 con 0x55f4ef4a27c0 2023-11-16T16:41:33.680 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.667+0000 7fea79d98700 1 --2- 172.21.15.1:0/2726657519 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55f4ef4a27c0 0x55f4ef4a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:33.680 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.667+0000 7fea79d98700 1 --2- 172.21.15.1:0/2726657519 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55f4ef4a27c0 0x55f4ef4a4ba0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3324b1990f7d58fc server_cookie=b40ccabf97755fbb in_seq=0 out_seq=0 2023-11-16T16:41:33.680 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.667+0000 7fea5dffb700 10 client.15876 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:41:33.681 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.669+0000 7fea5dffb700 1 -- 172.21.15.1:0/2726657519 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f4ef39c0a0 con 0x55f4ef4a27c0 2023-11-16T16:41:33.681 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.669+0000 7fea5dffb700 10 client.15876 handle_command_reply: tid=0 2023-11-16T16:41:33.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.669+0000 7fea88989b80 1 client.15876 shutdown 2023-11-16T16:41:33.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.669+0000 7fea88989b80 1 -- 172.21.15.1:0/2726657519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4ef488b60 msgr2=0x55f4ef494f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:33.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.669+0000 7fea88989b80 1 --2- 172.21.15.1:0/2726657519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4ef488b60 0x55f4ef494f60 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fea70096b10 tx=0x7fea7009a0a0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.669+0000 7fea88989b80 1 -- 172.21.15.1:0/2726657519 shutdown_connections 2023-11-16T16:41:33.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.669+0000 7fea88989b80 1 --2- 172.21.15.1:0/2726657519 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55f4ef4a27c0 0x55f4ef4a4ba0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.669+0000 7fea88989b80 1 --2- 172.21.15.1:0/2726657519 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4ef48d8b0 0x55f4ef4927f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.669+0000 7fea5dffb700 0 client.15876 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:41:33.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.669+0000 7fea88989b80 1 --2- 172.21.15.1:0/2726657519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4ef488b60 0x55f4ef494f60 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.669+0000 7fea88989b80 1 --2- 172.21.15.1:0/2726657519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ef489470 0x55f4ef492250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.669+0000 7fea88989b80 1 -- 172.21.15.1:0/2726657519 >> 172.21.15.1:0/2726657519 conn(0x55f4ef47e360 msgr2=0x55f4ef48ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:33.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.669+0000 7fea88989b80 1 -- 172.21.15.1:0/2726657519 shutdown_connections 2023-11-16T16:41:33.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.669+0000 7fea88989b80 1 -- 172.21.15.1:0/2726657519 wait complete. 2023-11-16T16:41:33.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.670+0000 7fea88989b80 20 client.15876 trim_cache size 0 max 16384 2023-11-16T16:41:33.693 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.687+0000 7fea88989b80 1 Processor -- start 2023-11-16T16:41:33.694 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.687+0000 7fea88989b80 1 -- start start 2023-11-16T16:41:33.694 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.687+0000 7fea88989b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4ef489470 0x55f4ef4999a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:33.694 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.687+0000 7fea88989b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ef48d8b0 0x55f4ef494b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:33.694 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.687+0000 7fea88989b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4ef48b1a0 0x55f4ef48b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:33.694 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.687+0000 7fea88989b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4ef4937c0 con 0x55f4ef489470 2023-11-16T16:41:33.694 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.687+0000 7fea88989b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4ef490800 con 0x55f4ef48d8b0 2023-11-16T16:41:33.694 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.687+0000 7fea88989b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4ef48c380 con 0x55f4ef48b1a0 2023-11-16T16:41:33.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.687+0000 7fea7a599700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4ef489470 0x55f4ef4999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:33.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.687+0000 7fea79d98700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ef48d8b0 0x55f4ef494b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:33.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.687+0000 7fea7a599700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4ef489470 0x55f4ef4999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44588/0 (socket says 172.21.15.1:44588) 2023-11-16T16:41:33.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.687+0000 7fea7a599700 1 -- 172.21.15.1:0/2441868538 learned_addr learned my addr 172.21.15.1:0/2441868538 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:41:33.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.687+0000 7fea7ad9a700 1 --2- 172.21.15.1:0/2441868538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4ef48b1a0 0x55f4ef48b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:33.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.687+0000 7fea7a599700 1 -- 172.21.15.1:0/2441868538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4ef48b1a0 msgr2=0x55f4ef48b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:33.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.687+0000 7fea7a599700 1 --2- 172.21.15.1:0/2441868538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4ef48b1a0 0x55f4ef48b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.687+0000 7fea7a599700 1 -- 172.21.15.1:0/2441868538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ef48d8b0 msgr2=0x55f4ef494b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:33.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.687+0000 7fea7a599700 1 --2- 172.21.15.1:0/2441868538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ef48d8b0 0x55f4ef494b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.687+0000 7fea7a599700 1 -- 172.21.15.1:0/2441868538 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4ef485240 con 0x55f4ef489470 2023-11-16T16:41:33.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.687+0000 7fea79d98700 1 --2- 172.21.15.1:0/2441868538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ef48d8b0 0x55f4ef494b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:41:33.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.688+0000 7fea7a599700 1 --2- 172.21.15.1:0/2441868538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4ef489470 0x55f4ef4999a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fea70099da0 tx=0x7fea700ca820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91cd961a593f5443 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:33.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.688+0000 7fea697fa700 1 -- 172.21.15.1:0/2441868538 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea700a2570 con 0x55f4ef489470 2023-11-16T16:41:33.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.688+0000 7fea697fa700 1 -- 172.21.15.1:0/2441868538 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea7009e0d0 con 0x55f4ef489470 2023-11-16T16:41:33.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.688+0000 7fea697fa700 1 -- 172.21.15.1:0/2441868538 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea700b2ad0 con 0x55f4ef489470 2023-11-16T16:41:33.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.689+0000 7fea88989b80 1 -- 172.21.15.1:0/2441868538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4ef489470 msgr2=0x55f4ef4999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:33.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.689+0000 7fea88989b80 1 --2- 172.21.15.1:0/2441868538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4ef489470 0x55f4ef4999a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fea70099da0 tx=0x7fea700ca820 comp rx=0 tx=0).stop 2023-11-16T16:41:33.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.689+0000 7fea88989b80 1 -- 172.21.15.1:0/2441868538 shutdown_connections 2023-11-16T16:41:33.697 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.689+0000 7fea88989b80 1 --2- 172.21.15.1:0/2441868538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4ef48b1a0 0x55f4ef48b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.697 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.689+0000 7fea88989b80 1 --2- 172.21.15.1:0/2441868538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4ef489470 0x55f4ef4999a0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.697 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.689+0000 7fea88989b80 1 --2- 172.21.15.1:0/2441868538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ef48d8b0 0x55f4ef494b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.697 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.689+0000 7fea88989b80 1 -- 172.21.15.1:0/2441868538 >> 172.21.15.1:0/2441868538 conn(0x55f4ef47e360 msgr2=0x55f4ef482550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:33.697 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.689+0000 7fea88989b80 1 -- 172.21.15.1:0/2441868538 shutdown_connections 2023-11-16T16:41:33.697 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.689+0000 7fea88989b80 1 -- 172.21.15.1:0/2441868538 wait complete. 2023-11-16T16:41:33.697 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.689+0000 7fea88989b80 1 -- start start 2023-11-16T16:41:33.697 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.689+0000 7fea88989b80 1 Processor -- start 2023-11-16T16:41:33.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.689+0000 7fea88989b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4ef489470 0x55f4ef4993e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:33.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.689+0000 7fea88989b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4ef48b1a0 0x55f4ef499920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:33.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.690+0000 7fea88989b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ef48d8b0 0x55f4ef497990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:33.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.690+0000 7fea88989b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4ef498010 con 0x55f4ef48b1a0 2023-11-16T16:41:33.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.690+0000 7fea88989b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4ef498180 con 0x55f4ef48d8b0 2023-11-16T16:41:33.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.690+0000 7fea88989b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4ef498340 con 0x55f4ef489470 2023-11-16T16:41:33.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.690+0000 7fea88989b80 10 client.0 _ll_register_callbacks cb 0x7fea70099a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:41:33.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.690+0000 7fea79d98700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4ef48b1a0 0x55f4ef499920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:33.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.690+0000 7fea79d98700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4ef48b1a0 0x55f4ef499920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44596/0 (socket says 172.21.15.1:44596) 2023-11-16T16:41:33.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.690+0000 7fea79d98700 1 -- 172.21.15.1:0/1889297716 learned_addr learned my addr 172.21.15.1:0/1889297716 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:41:33.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.690+0000 7fea7ad9a700 1 --2- 172.21.15.1:0/1889297716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ef48d8b0 0x55f4ef497990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:33.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.690+0000 7fea7a599700 1 --2- 172.21.15.1:0/1889297716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4ef489470 0x55f4ef4993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:33.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.690+0000 7fea79d98700 1 -- 172.21.15.1:0/1889297716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4ef489470 msgr2=0x55f4ef4993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:33.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.690+0000 7fea79d98700 1 --2- 172.21.15.1:0/1889297716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4ef489470 0x55f4ef4993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.690+0000 7fea79d98700 1 -- 172.21.15.1:0/1889297716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ef48d8b0 msgr2=0x55f4ef497990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:33.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.690+0000 7fea79d98700 1 --2- 172.21.15.1:0/1889297716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ef48d8b0 0x55f4ef497990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.700 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.690+0000 7fea79d98700 1 -- 172.21.15.1:0/1889297716 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea64003870 con 0x55f4ef48b1a0 2023-11-16T16:41:33.700 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.690+0000 7fea7a599700 1 --2- 172.21.15.1:0/1889297716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4ef489470 0x55f4ef4993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:41:33.700 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.690+0000 7fea7ad9a700 1 --2- 172.21.15.1:0/1889297716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ef48d8b0 0x55f4ef497990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:41:33.700 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.690+0000 7fea79d98700 1 --2- 172.21.15.1:0/1889297716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4ef48b1a0 0x55f4ef499920 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fea640095a0 tx=0x7fea64009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3815f6b2739763ed server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:33.700 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.690+0000 7fea5dffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:41:33.700 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.690+0000 7fea5dffb700 1 -- 172.21.15.1:0/1889297716 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6402c040 con 0x55f4ef48b1a0 2023-11-16T16:41:33.700 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.690+0000 7fea5dffb700 1 -- 172.21.15.1:0/1889297716 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fea640054c0 con 0x55f4ef48b1a0 2023-11-16T16:41:33.703 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.691+0000 7fea88989b80 1 -- 172.21.15.1:0/1889297716 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea6c00f040 con 0x55f4ef48b1a0 2023-11-16T16:41:33.704 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.691+0000 7fea5dffb700 1 -- 172.21.15.1:0/1889297716 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea64005680 con 0x55f4ef48b1a0 2023-11-16T16:41:33.704 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.691+0000 7fea88989b80 10 client.15882 fetch_fsmap learned FSMap version 23 2023-11-16T16:41:33.704 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.691+0000 7fea88989b80 1 -- 172.21.15.1:0/1889297716 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55f4ef3a0190 con 0x55f4ef48b1a0 2023-11-16T16:41:33.704 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.691+0000 7fea5dffb700 1 -- 172.21.15.1:0/1889297716 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea6c00f040 con 0x55f4ef48b1a0 2023-11-16T16:41:33.704 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.691+0000 7fea5dffb700 1 -- 172.21.15.1:0/1889297716 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fea6400f460 con 0x55f4ef48b1a0 2023-11-16T16:41:33.704 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.691+0000 7fea88989b80 10 client.15882 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:41:33.704 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.691+0000 7fea88989b80 10 client.15882 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:41:33.704 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.691+0000 7fea88989b80 20 client.15882 populate_metadata read hostname 'smithi001' 2023-11-16T16:41:33.704 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.691+0000 7fea88989b80 1 --2- 172.21.15.1:0/1889297716 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55f4ef4f77e0 0x55f4ef4f9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:33.705 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.691+0000 7fea88989b80 4 client.15882 mds_command: new command op to 24521 tid=0[{"prefix": "scrub status"}] 2023-11-16T16:41:33.705 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.691+0000 7fea88989b80 1 -- 172.21.15.1:0/1889297716 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f4ef3a8660 con 0x55f4ef4f77e0 2023-11-16T16:41:33.705 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.691+0000 7fea7a599700 1 --2- 172.21.15.1:0/1889297716 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55f4ef4f77e0 0x55f4ef4f9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:33.705 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.692+0000 7fea5dffb700 10 client.15882 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:41:33.705 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.692+0000 7fea7a599700 1 --2- 172.21.15.1:0/1889297716 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55f4ef4f77e0 0x55f4ef4f9bf0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35bdd0d3aabb93b5 server_cookie=b0c8f280ee30ded1 in_seq=0 out_seq=0 2023-11-16T16:41:33.705 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.694+0000 7fea5dffb700 1 -- 172.21.15.1:0/1889297716 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f4ef3a8660 con 0x55f4ef4f77e0 2023-11-16T16:41:33.705 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.694+0000 7fea5dffb700 10 client.15882 handle_command_reply: tid=0 2023-11-16T16:41:33.705 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.694+0000 7fea88989b80 1 client.15882 shutdown 2023-11-16T16:41:33.705 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.694+0000 7fea88989b80 1 -- 172.21.15.1:0/1889297716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4ef48b1a0 msgr2=0x55f4ef499920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:33.706 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.694+0000 7fea88989b80 1 --2- 172.21.15.1:0/1889297716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4ef48b1a0 0x55f4ef499920 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fea640095a0 tx=0x7fea64009310 comp rx=0 tx=0).stop 2023-11-16T16:41:33.706 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.694+0000 7fea88989b80 1 -- 172.21.15.1:0/1889297716 shutdown_connections 2023-11-16T16:41:33.706 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.694+0000 7fea88989b80 1 --2- 172.21.15.1:0/1889297716 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55f4ef4f77e0 0x55f4ef4f9bf0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.706 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.694+0000 7fea88989b80 1 --2- 172.21.15.1:0/1889297716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f4ef48d8b0 0x55f4ef497990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.706 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.694+0000 7fea5dffb700 0 client.15882 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:41:33.706 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.694+0000 7fea88989b80 1 --2- 172.21.15.1:0/1889297716 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f4ef489470 0x55f4ef4993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.706 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.694+0000 7fea88989b80 1 --2- 172.21.15.1:0/1889297716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f4ef48b1a0 0x55f4ef499920 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.706 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.694+0000 7fea88989b80 1 -- 172.21.15.1:0/1889297716 >> 172.21.15.1:0/1889297716 conn(0x55f4ef47e360 msgr2=0x55f4ef48b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:33.706 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.694+0000 7fea88989b80 1 -- 172.21.15.1:0/1889297716 shutdown_connections 2023-11-16T16:41:33.707 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.694+0000 7fea88989b80 1 -- 172.21.15.1:0/1889297716 wait complete. 2023-11-16T16:41:33.707 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.694+0000 7fea88989b80 20 client.15882 trim_cache size 0 max 16384 2023-11-16T16:41:33.707 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:41:33.707 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2023-11-16T16:41:33.707 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2023-11-16T16:41:33.707 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:41:33.708 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.695+0000 7fea7c7fd700 1 -- 172.21.15.1:0/3670831090 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fea600a12c0 msgr2=0x7fea600a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:33.708 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.695+0000 7fea7c7fd700 1 --2- 172.21.15.1:0/3670831090 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fea600a12c0 0x7fea600a3770 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fea740ff0b0 tx=0x7fea6401a040 comp rx=0 tx=0).stop 2023-11-16T16:41:33.709 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.695+0000 7fea7c7fd700 1 -- 172.21.15.1:0/3670831090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea740ff8f0 msgr2=0x7fea74197130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:33.709 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.695+0000 7fea7c7fd700 1 --2- 172.21.15.1:0/3670831090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea740ff8f0 0x7fea74197130 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fea7000e9f0 tx=0x7fea7000ed00 comp rx=0 tx=0).stop 2023-11-16T16:41:33.709 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.695+0000 7fea7c7fd700 1 -- 172.21.15.1:0/3670831090 shutdown_connections 2023-11-16T16:41:33.709 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.695+0000 7fea7c7fd700 1 --2- 172.21.15.1:0/3670831090 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fea600a12c0 0x7fea600a3770 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.709 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.695+0000 7fea7c7fd700 1 --2- 172.21.15.1:0/3670831090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fea74104140 0x7fea741a21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.709 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.695+0000 7fea7c7fd700 1 --2- 172.21.15.1:0/3670831090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea740ff8f0 0x7fea74197130 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.709 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.695+0000 7fea7c7fd700 1 --2- 172.21.15.1:0/3670831090 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea741002a0 0x7fea74197670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.709 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.695+0000 7fea7c7fd700 1 -- 172.21.15.1:0/3670831090 >> 172.21.15.1:0/3670831090 conn(0x7fea74075880 msgr2=0x7fea740fd120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:33.710 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.695+0000 7fea7c7fd700 1 -- 172.21.15.1:0/3670831090 shutdown_connections 2023-11-16T16:41:33.710 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.695+0000 7fea7c7fd700 1 -- 172.21.15.1:0/3670831090 wait complete. 2023-11-16T16:41:33.711 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-16T16:41:33.712 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:41:33.892 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.887+0000 7f9013f1b700 1 -- 172.21.15.1:0/3531069493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f900c0fed40 msgr2=0x7f900c0ff110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:33.892 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.887+0000 7f9013f1b700 1 --2- 172.21.15.1:0/3531069493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f900c0fed40 0x7f900c0ff110 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f90080099c0 tx=0x7f9008009cd0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.893 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.887+0000 7f9013f1b700 1 -- 172.21.15.1:0/3531069493 shutdown_connections 2023-11-16T16:41:33.893 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.887+0000 7f9013f1b700 1 --2- 172.21.15.1:0/3531069493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f900c105660 0x7f900c10d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.893 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.887+0000 7f9013f1b700 1 --2- 172.21.15.1:0/3531069493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f900c0fed40 0x7f900c0ff110 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.893 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.887+0000 7f9013f1b700 1 --2- 172.21.15.1:0/3531069493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f900c0ff6e0 0x7f900c104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.893 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.887+0000 7f9013f1b700 1 -- 172.21.15.1:0/3531069493 >> 172.21.15.1:0/3531069493 conn(0x7f900c0748b0 msgr2=0x7f900c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:33.893 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.887+0000 7f9013f1b700 1 -- 172.21.15.1:0/3531069493 shutdown_connections 2023-11-16T16:41:33.893 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.887+0000 7f9013f1b700 1 -- 172.21.15.1:0/3531069493 wait complete. 2023-11-16T16:41:33.893 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.888+0000 7f9013f1b700 1 Processor -- start 2023-11-16T16:41:33.894 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.888+0000 7f9013f1b700 1 -- start start 2023-11-16T16:41:33.894 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.888+0000 7f9013f1b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f900c0fed40 0x7f900c0ffcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:33.894 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.888+0000 7f9013f1b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f900c0ff6e0 0x7f900c1001f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:33.894 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.888+0000 7f9013f1b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f900c105660 0x7f900c100730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:33.894 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.888+0000 7f9013f1b700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c1a48e0 con 0x7f900c0fed40 2023-11-16T16:41:33.894 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.888+0000 7f9013f1b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c1a4a50 con 0x7f900c0ff6e0 2023-11-16T16:41:33.894 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.888+0000 7f9013f1b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c1a4c10 con 0x7f900c105660 2023-11-16T16:41:33.895 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.888+0000 7f9011cb7700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f900c0fed40 0x7f900c0ffcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:33.895 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.888+0000 7f9011cb7700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f900c0fed40 0x7f900c0ffcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44608/0 (socket says 172.21.15.1:44608) 2023-11-16T16:41:33.895 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.888+0000 7f9011cb7700 1 -- 172.21.15.1:0/3617439775 learned_addr learned my addr 172.21.15.1:0/3617439775 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:41:33.895 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.889+0000 7f90114b6700 1 --2- 172.21.15.1:0/3617439775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f900c0ff6e0 0x7f900c1001f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:33.895 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.889+0000 7f90124b8700 1 --2- 172.21.15.1:0/3617439775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f900c105660 0x7f900c100730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:33.895 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.889+0000 7f9011cb7700 1 -- 172.21.15.1:0/3617439775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f900c105660 msgr2=0x7f900c100730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:33.895 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.889+0000 7f9011cb7700 1 --2- 172.21.15.1:0/3617439775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f900c105660 0x7f900c100730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.895 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.889+0000 7f9011cb7700 1 -- 172.21.15.1:0/3617439775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f900c0ff6e0 msgr2=0x7f900c1001f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:33.895 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.889+0000 7f9011cb7700 1 --2- 172.21.15.1:0/3617439775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f900c0ff6e0 0x7f900c1001f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:33.896 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.889+0000 7f9011cb7700 1 -- 172.21.15.1:0/3617439775 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90080095f0 con 0x7f900c0fed40 2023-11-16T16:41:33.896 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.889+0000 7f9011cb7700 1 --2- 172.21.15.1:0/3617439775 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f900c0fed40 0x7f900c0ffcb0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f90080098e0 tx=0x7f900800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:33.896 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.889+0000 7f9002ffd700 1 -- 172.21.15.1:0/3617439775 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900801d070 con 0x7f900c0fed40 2023-11-16T16:41:33.896 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.889+0000 7f9002ffd700 1 -- 172.21.15.1:0/3617439775 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9008022470 con 0x7f900c0fed40 2023-11-16T16:41:33.896 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.889+0000 7f9002ffd700 1 -- 172.21.15.1:0/3617439775 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9008017650 con 0x7f900c0fed40 2023-11-16T16:41:33.896 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.889+0000 7f9013f1b700 1 -- 172.21.15.1:0/3617439775 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f900c110410 con 0x7f900c0fed40 2023-11-16T16:41:33.896 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.889+0000 7f9013f1b700 1 -- 172.21.15.1:0/3617439775 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f900c110280 con 0x7f900c0fed40 2023-11-16T16:41:33.896 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.890+0000 7f9013f1b700 1 -- 172.21.15.1:0/3617439775 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f900c1105a0 con 0x7f900c0fed40 2023-11-16T16:41:33.897 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.892+0000 7f9002ffd700 1 -- 172.21.15.1:0/3617439775 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f9008022850 con 0x7f900c0fed40 2023-11-16T16:41:33.897 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.892+0000 7f9002ffd700 1 --2- 172.21.15.1:0/3617439775 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f8ff80a1340 0x7f8ff80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:33.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.892+0000 7f9002ffd700 1 -- 172.21.15.1:0/3617439775 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f90080cdbf0 con 0x7f900c0fed40 2023-11-16T16:41:33.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.893+0000 7f90114b6700 1 --2- 172.21.15.1:0/3617439775 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f8ff80a1340 0x7f8ff80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:33.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.893+0000 7f90114b6700 1 --2- 172.21.15.1:0/3617439775 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f8ff80a1340 0x7f8ff80a37f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f900c1014a0 tx=0x7f8ffc009260 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:33.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:33.896+0000 7f9002ffd700 1 -- 172.21.15.1:0/3617439775 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9008097150 con 0x7f900c0fed40 2023-11-16T16:41:34.165 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.163+0000 7f9013f1b700 1 -- 172.21.15.1:0/3617439775 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f900c1011e0 con 0x7f900c0fed40 2023-11-16T16:41:34.166 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.164+0000 7f9002ffd700 1 -- 172.21.15.1:0/3617439775 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10774 (secure 0 0 0) 0x7f900809a770 con 0x7f900c0fed40 2023-11-16T16:41:34.167 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:41:34.167 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2587563684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2587563684},{"type":"v1","addr":"172.21.15.1:6835","nonce":2587563684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/277936168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":277936168},{"type":"v1","addr":"172.21.15.1:6837","nonce":277936168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/1172575045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1172575045},{"type":"v1","addr":"172.21.15.1:6839","nonce":1172575045}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/2631175899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2631175899},{"type":"v1","addr":"172.21.15.1:6841","nonce":2631175899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3693905267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3693905267},{"type":"v1","addr":"172.21.15.53:6833","nonce":3693905267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/164668033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":164668033},{"type":"v1","addr":"172.21.15.26:6833","nonce":164668033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-16T16:37:13.918943+0000","modified":"2023-11-16T16:37:19.292898+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2140542192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2140542192},{"type":"v1","addr":"172.21.15.53:6835","nonce":2140542192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1010760275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1010760275},{"type":"v1","addr":"172.21.15.26:6835","nonce":1010760275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3197060302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3197060302},{"type":"v1","addr":"172.21.15.53:6837","nonce":3197060302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6837/1453504863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1453504863},{"type":"v1","addr":"172.21.15.26:6837","nonce":1453504863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.53:6839/1044753024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1044753024},{"type":"v1","addr":"172.21.15.53:6839","nonce":1044753024}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6839/2192205949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2192205949},{"type":"v1","addr":"172.21.15.26:6839","nonce":2192205949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-16T16:41:34.169 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.166+0000 7f9013f1b700 1 -- 172.21.15.1:0/3617439775 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f8ff80a1340 msgr2=0x7f8ff80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:34.169 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.166+0000 7f9013f1b700 1 --2- 172.21.15.1:0/3617439775 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f8ff80a1340 0x7f8ff80a37f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f900c1014a0 tx=0x7f8ffc009260 comp rx=0 tx=0).stop 2023-11-16T16:41:34.169 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.166+0000 7f9013f1b700 1 -- 172.21.15.1:0/3617439775 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f900c0fed40 msgr2=0x7f900c0ffcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:34.170 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.166+0000 7f9013f1b700 1 --2- 172.21.15.1:0/3617439775 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f900c0fed40 0x7f900c0ffcb0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f90080098e0 tx=0x7f900800f690 comp rx=0 tx=0).stop 2023-11-16T16:41:34.170 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.166+0000 7f9013f1b700 1 -- 172.21.15.1:0/3617439775 shutdown_connections 2023-11-16T16:41:34.170 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.167+0000 7f9013f1b700 1 --2- 172.21.15.1:0/3617439775 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f8ff80a1340 0x7f8ff80a37f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.170 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.167+0000 7f9013f1b700 1 --2- 172.21.15.1:0/3617439775 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f900c105660 0x7f900c100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.170 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.167+0000 7f9013f1b700 1 --2- 172.21.15.1:0/3617439775 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f900c0fed40 0x7f900c0ffcb0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.170 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.167+0000 7f9013f1b700 1 --2- 172.21.15.1:0/3617439775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f900c0ff6e0 0x7f900c1001f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.167+0000 7f9013f1b700 1 -- 172.21.15.1:0/3617439775 >> 172.21.15.1:0/3617439775 conn(0x7f900c0748b0 msgr2=0x7f900c0fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:34.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.167+0000 7f9013f1b700 1 -- 172.21.15.1:0/3617439775 shutdown_connections 2023-11-16T16:41:34.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.167+0000 7f9013f1b700 1 -- 172.21.15.1:0/3617439775 wait complete. 2023-11-16T16:41:34.171 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 23 2023-11-16T16:41:34.182 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-11-16T16:41:34.321 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.319+0000 7f12282b2700 1 -- 172.21.15.1:0/2459575846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12201007c0 msgr2=0x7f1220100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:34.321 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.319+0000 7f12282b2700 1 --2- 172.21.15.1:0/2459575846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12201007c0 0x7f1220100b90 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f121c009990 tx=0x7f121c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.321 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.319+0000 7f12282b2700 1 -- 172.21.15.1:0/2459575846 shutdown_connections 2023-11-16T16:41:34.321 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.319+0000 7f12282b2700 1 --2- 172.21.15.1:0/2459575846 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1220109d70 0x7f122010c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.322 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.319+0000 7f12282b2700 1 --2- 172.21.15.1:0/2459575846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12201007c0 0x7f1220100b90 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.322 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.319+0000 7f12282b2700 1 --2- 172.21.15.1:0/2459575846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12201010d0 0x7f1220109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.322 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.319+0000 7f12282b2700 1 -- 172.21.15.1:0/2459575846 >> 172.21.15.1:0/2459575846 conn(0x7f1220074c60 msgr2=0x7f1220075060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:34.322 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.319+0000 7f12282b2700 1 -- 172.21.15.1:0/2459575846 shutdown_connections 2023-11-16T16:41:34.322 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.319+0000 7f12282b2700 1 -- 172.21.15.1:0/2459575846 wait complete. 2023-11-16T16:41:34.325 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.320+0000 7f12282b2700 1 Processor -- start 2023-11-16T16:41:34.325 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.320+0000 7f12282b2700 1 -- start start 2023-11-16T16:41:34.325 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.320+0000 7f12282b2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12201007c0 0x7f1220197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.325 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.320+0000 7f12282b2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12201010d0 0x7f1220197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.325 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.320+0000 7f12282b2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1220109d70 0x7f12201a24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.325 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.320+0000 7f12282b2700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1220198280 con 0x7f12201007c0 2023-11-16T16:41:34.325 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.320+0000 7f12282b2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12201a29f0 con 0x7f12201010d0 2023-11-16T16:41:34.326 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.320+0000 7f12282b2700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12201a2b60 con 0x7f1220109d70 2023-11-16T16:41:34.326 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.320+0000 7f122604e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12201007c0 0x7f1220197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.326 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.320+0000 7f122604e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12201007c0 0x7f1220197430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44622/0 (socket says 172.21.15.1:44622) 2023-11-16T16:41:34.326 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.320+0000 7f122604e700 1 -- 172.21.15.1:0/3171658 learned_addr learned my addr 172.21.15.1:0/3171658 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:41:34.326 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.320+0000 7f122584d700 1 --2- 172.21.15.1:0/3171658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12201010d0 0x7f1220197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.326 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.320+0000 7f122684f700 1 --2- 172.21.15.1:0/3171658 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1220109d70 0x7f12201a24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.326 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.320+0000 7f122604e700 1 -- 172.21.15.1:0/3171658 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1220109d70 msgr2=0x7f12201a24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:34.327 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.320+0000 7f122604e700 1 --2- 172.21.15.1:0/3171658 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1220109d70 0x7f12201a24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.327 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.320+0000 7f122604e700 1 -- 172.21.15.1:0/3171658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12201010d0 msgr2=0x7f1220197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:34.327 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.320+0000 7f122604e700 1 --2- 172.21.15.1:0/3171658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12201010d0 0x7f1220197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.327 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.320+0000 7f122604e700 1 -- 172.21.15.1:0/3171658 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1210009520 con 0x7f12201007c0 2023-11-16T16:41:34.327 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.320+0000 7f122684f700 1 --2- 172.21.15.1:0/3171658 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1220109d70 0x7f12201a24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:41:34.327 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.321+0000 7f122604e700 1 --2- 172.21.15.1:0/3171658 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12201007c0 0x7f1220197430 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f121c000c00 tx=0x7f121c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:34.327 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.321+0000 7f12177fe700 1 -- 172.21.15.1:0/3171658 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121c01d070 con 0x7f12201007c0 2023-11-16T16:41:34.327 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.321+0000 7f12177fe700 1 -- 172.21.15.1:0/3171658 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f121c00fb80 con 0x7f12201007c0 2023-11-16T16:41:34.328 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.321+0000 7f12282b2700 1 -- 172.21.15.1:0/3171658 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f121c0095f0 con 0x7f12201007c0 2023-11-16T16:41:34.328 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.321+0000 7f12177fe700 1 -- 172.21.15.1:0/3171658 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121c0173f0 con 0x7f12201007c0 2023-11-16T16:41:34.328 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.321+0000 7f12282b2700 1 -- 172.21.15.1:0/3171658 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f122010f470 con 0x7f12201007c0 2023-11-16T16:41:34.328 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.323+0000 7f12177fe700 1 -- 172.21.15.1:0/3171658 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f121c00fcf0 con 0x7f12201007c0 2023-11-16T16:41:34.336 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.324+0000 7f12177fe700 1 --2- 172.21.15.1:0/3171658 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f120c0a12c0 0x7f120c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.336 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.324+0000 7f122584d700 1 --2- 172.21.15.1:0/3171658 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f120c0a12c0 0x7f120c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.336 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.324+0000 7f12177fe700 1 -- 172.21.15.1:0/3171658 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f121c0cdec0 con 0x7f12201007c0 2023-11-16T16:41:34.336 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.324+0000 7f122584d700 1 --2- 172.21.15.1:0/3171658 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f120c0a12c0 0x7f120c0a3770 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f12200fff60 tx=0x7f1210009260 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:34.337 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.325+0000 7f123443eb80 1 Processor -- start 2023-11-16T16:41:34.337 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.325+0000 7f123443eb80 1 -- start start 2023-11-16T16:41:34.337 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.325+0000 7f123443eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e2b98f70 0x55f0e2b99340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.337 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.325+0000 7f123443eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0e2b99880 0x55f0e2b9d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.337 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.325+0000 7f123443eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0e2b9dcc0 0x55f0e2b9e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.337 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.325+0000 7f123443eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0e2a9a060 con 0x55f0e2b9dcc0 2023-11-16T16:41:34.337 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.325+0000 7f123443eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0e2aec8a0 con 0x55f0e2b98f70 2023-11-16T16:41:34.337 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.325+0000 7f123443eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0e2ae5d30 con 0x55f0e2b99880 2023-11-16T16:41:34.338 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.325+0000 7f122684f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0e2b9dcc0 0x55f0e2b9e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.338 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.325+0000 7f122604e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e2b98f70 0x55f0e2b99340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.338 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.325+0000 7f122604e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e2b98f70 0x55f0e2b99340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.1:38816/0 (socket says 172.21.15.1:38816) 2023-11-16T16:41:34.338 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.325+0000 7f122604e700 1 -- 172.21.15.1:0/3269474793 learned_addr learned my addr 172.21.15.1:0/3269474793 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:41:34.338 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.325+0000 7f122584d700 1 --2- 172.21.15.1:0/3269474793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0e2b99880 0x55f0e2b9d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.338 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.325+0000 7f122604e700 1 -- 172.21.15.1:0/3269474793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0e2b99880 msgr2=0x55f0e2b9d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:34.338 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.325+0000 7f122604e700 1 --2- 172.21.15.1:0/3269474793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0e2b99880 0x55f0e2b9d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.338 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.326+0000 7f122604e700 1 -- 172.21.15.1:0/3269474793 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0e2b9dcc0 msgr2=0x55f0e2b9e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:34.339 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.326+0000 7f122604e700 1 --2- 172.21.15.1:0/3269474793 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0e2b9dcc0 0x55f0e2b9e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.339 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.326+0000 7f122604e700 1 -- 172.21.15.1:0/3269474793 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0e2967630 con 0x55f0e2b98f70 2023-11-16T16:41:34.339 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.326+0000 7f122684f700 1 --2- 172.21.15.1:0/3269474793 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0e2b9dcc0 0x55f0e2b9e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:41:34.339 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.326+0000 7f122604e700 1 --2- 172.21.15.1:0/3269474793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e2b98f70 0x55f0e2b99340 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f121c0a2fe0 tx=0x7f121c00b680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39bbebbed9f5a0bb server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:34.339 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.326+0000 7f1214dac700 1 -- 172.21.15.1:0/3269474793 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121c09f180 con 0x55f0e2b98f70 2023-11-16T16:41:34.339 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.326+0000 7f1214dac700 1 -- 172.21.15.1:0/3269474793 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f121c0a3510 con 0x55f0e2b98f70 2023-11-16T16:41:34.339 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.326+0000 7f1214dac700 1 -- 172.21.15.1:0/3269474793 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121c0a3080 con 0x55f0e2b98f70 2023-11-16T16:41:34.339 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.327+0000 7f123443eb80 1 -- 172.21.15.1:0/3269474793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e2b98f70 msgr2=0x55f0e2b99340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:34.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.327+0000 7f123443eb80 1 --2- 172.21.15.1:0/3269474793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e2b98f70 0x55f0e2b99340 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f121c0a2fe0 tx=0x7f121c00b680 comp rx=0 tx=0).stop 2023-11-16T16:41:34.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.327+0000 7f123443eb80 1 -- 172.21.15.1:0/3269474793 shutdown_connections 2023-11-16T16:41:34.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.327+0000 7f123443eb80 1 --2- 172.21.15.1:0/3269474793 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0e2b9dcc0 0x55f0e2b9e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.327+0000 7f123443eb80 1 --2- 172.21.15.1:0/3269474793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e2b98f70 0x55f0e2b99340 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.327+0000 7f123443eb80 1 --2- 172.21.15.1:0/3269474793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0e2b99880 0x55f0e2b9d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.327+0000 7f123443eb80 1 -- 172.21.15.1:0/3269474793 >> 172.21.15.1:0/3269474793 conn(0x55f0e2b8a360 msgr2=0x55f0e2b8fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:34.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.327+0000 7f123443eb80 1 -- 172.21.15.1:0/3269474793 shutdown_connections 2023-11-16T16:41:34.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.327+0000 7f123443eb80 1 -- 172.21.15.1:0/3269474793 wait complete. 2023-11-16T16:41:34.341 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.327+0000 7f123443eb80 1 -- start start 2023-11-16T16:41:34.341 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.327+0000 7f123443eb80 1 Processor -- start 2023-11-16T16:41:34.341 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.327+0000 7f123443eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0e2b98f70 0x55f0e2ba5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.341 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.327+0000 7f123443eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0e2b99880 0x55f0e2ba2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.341 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.327+0000 7f123443eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e2b9dcc0 0x55f0e2ba2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.341 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.327+0000 7f123443eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0e2a878e0 con 0x55f0e2b98f70 2023-11-16T16:41:34.341 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.327+0000 7f123443eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0e2a87580 con 0x55f0e2b9dcc0 2023-11-16T16:41:34.341 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.327+0000 7f123443eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0e2969920 con 0x55f0e2b99880 2023-11-16T16:41:34.342 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.327+0000 7f122604e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0e2b98f70 0x55f0e2ba5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.342 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.327+0000 7f123443eb80 10 client.0 _ll_register_callbacks cb 0x55f0e2aa5780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:41:34.342 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.328+0000 7f122604e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0e2b98f70 0x55f0e2ba5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44636/0 (socket says 172.21.15.1:44636) 2023-11-16T16:41:34.342 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.328+0000 7f122604e700 1 -- 172.21.15.1:0/1569935155 learned_addr learned my addr 172.21.15.1:0/1569935155 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:41:34.342 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.328+0000 7f122684f700 1 --2- 172.21.15.1:0/1569935155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e2b9dcc0 0x55f0e2ba2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.342 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.328+0000 7f122584d700 1 --2- 172.21.15.1:0/1569935155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0e2b99880 0x55f0e2ba2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.342 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.328+0000 7f122604e700 1 -- 172.21.15.1:0/1569935155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0e2b99880 msgr2=0x55f0e2ba2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:34.342 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.328+0000 7f122604e700 1 --2- 172.21.15.1:0/1569935155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0e2b99880 0x55f0e2ba2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.328+0000 7f122604e700 1 -- 172.21.15.1:0/1569935155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e2b9dcc0 msgr2=0x55f0e2ba2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:34.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.328+0000 7f122604e700 1 --2- 172.21.15.1:0/1569935155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e2b9dcc0 0x55f0e2ba2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.328+0000 7f122604e700 1 -- 172.21.15.1:0/1569935155 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f122010f470 con 0x55f0e2b98f70 2023-11-16T16:41:34.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.328+0000 7f122584d700 1 --2- 172.21.15.1:0/1569935155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0e2b99880 0x55f0e2ba2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:41:34.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.328+0000 7f122684f700 1 --2- 172.21.15.1:0/1569935155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e2b9dcc0 0x55f0e2ba2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:41:34.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.328+0000 7f122604e700 1 --2- 172.21.15.1:0/1569935155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0e2b98f70 0x55f0e2ba5300 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f121c005910 tx=0x7f121c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4b5e109cebdf172 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:34.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.328+0000 7f1209ffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:41:34.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.328+0000 7f1209ffb700 1 -- 172.21.15.1:0/1569935155 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121c0b9070 con 0x55f0e2b98f70 2023-11-16T16:41:34.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.328+0000 7f1209ffb700 1 -- 172.21.15.1:0/1569935155 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f121c09f180 con 0x55f0e2b98f70 2023-11-16T16:41:34.344 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.328+0000 7f123443eb80 1 -- 172.21.15.1:0/1569935155 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f0e2b87b20 con 0x55f0e2b98f70 2023-11-16T16:41:34.344 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.328+0000 7f1209ffb700 1 -- 172.21.15.1:0/1569935155 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f121c0a3290 con 0x55f0e2b98f70 2023-11-16T16:41:34.344 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.328+0000 7f123443eb80 10 client.15900 fetch_fsmap learned FSMap version 23 2023-11-16T16:41:34.344 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.328+0000 7f123443eb80 1 -- 172.21.15.1:0/1569935155 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55f0e2aac190 con 0x55f0e2b98f70 2023-11-16T16:41:34.344 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.329+0000 7f1209ffb700 1 -- 172.21.15.1:0/1569935155 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f0e2b87b20 con 0x55f0e2b98f70 2023-11-16T16:41:34.344 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.329+0000 7f1209ffb700 1 -- 172.21.15.1:0/1569935155 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f121c07e080 con 0x55f0e2b98f70 2023-11-16T16:41:34.344 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.329+0000 7f123443eb80 10 client.15900 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:41:34.344 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.329+0000 7f123443eb80 10 client.15900 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-11-16T16:41:34.344 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.329+0000 7f123443eb80 20 client.15900 populate_metadata read hostname 'smithi001' 2023-11-16T16:41:34.345 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.329+0000 7f123443eb80 1 --2- 172.21.15.1:0/1569935155 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55f0e2bb24a0 0x55f0e2bb4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.345 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.329+0000 7f123443eb80 4 client.15900 mds_command: new command op to 24482 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:41:34.345 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.329+0000 7f123443eb80 1 -- 172.21.15.1:0/1569935155 --> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f0e2aa80a0 con 0x55f0e2bb24a0 2023-11-16T16:41:34.345 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.329+0000 7f122584d700 1 --2- 172.21.15.1:0/1569935155 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55f0e2bb24a0 0x55f0e2bb4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.345 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.329+0000 7f1209ffb700 10 client.15900 ms_handle_connect on v2:172.21.15.26:6836/1453504863 2023-11-16T16:41:34.345 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.329+0000 7f122584d700 1 --2- 172.21.15.1:0/1569935155 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55f0e2bb24a0 0x55f0e2bb4880 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e85c5e316eea4f4a server_cookie=c1066e80966b0921 in_seq=0 out_seq=0 2023-11-16T16:41:34.345 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.330+0000 7f1209ffb700 1 -- 172.21.15.1:0/1569935155 <== mds.2 v2:172.21.15.26:6836/1453504863 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f0e2aa80a0 con 0x55f0e2bb24a0 2023-11-16T16:41:34.345 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.330+0000 7f1209ffb700 10 client.15900 handle_command_reply: tid=0 2023-11-16T16:41:34.345 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.330+0000 7f123443eb80 1 client.15900 shutdown 2023-11-16T16:41:34.346 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.330+0000 7f123443eb80 1 -- 172.21.15.1:0/1569935155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0e2b98f70 msgr2=0x55f0e2ba5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:34.346 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.330+0000 7f123443eb80 1 --2- 172.21.15.1:0/1569935155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0e2b98f70 0x55f0e2ba5300 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f121c005910 tx=0x7f121c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.346 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.330+0000 7f123443eb80 1 -- 172.21.15.1:0/1569935155 shutdown_connections 2023-11-16T16:41:34.346 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.330+0000 7f123443eb80 1 --2- 172.21.15.1:0/1569935155 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55f0e2bb24a0 0x55f0e2bb4880 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.346 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.330+0000 7f123443eb80 1 --2- 172.21.15.1:0/1569935155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e2b9dcc0 0x55f0e2ba2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.346 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.330+0000 7f123443eb80 1 --2- 172.21.15.1:0/1569935155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0e2b98f70 0x55f0e2ba5300 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.346 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.330+0000 7f1209ffb700 0 client.15900 ms_handle_reset on v2:172.21.15.26:6836/1453504863 2023-11-16T16:41:34.346 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.330+0000 7f123443eb80 1 --2- 172.21.15.1:0/1569935155 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0e2b99880 0x55f0e2ba2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.347 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.330+0000 7f123443eb80 1 -- 172.21.15.1:0/1569935155 >> 172.21.15.1:0/1569935155 conn(0x55f0e2b8a360 msgr2=0x55f0e2b93960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:34.347 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.330+0000 7f123443eb80 1 -- 172.21.15.1:0/1569935155 shutdown_connections 2023-11-16T16:41:34.347 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.330+0000 7f123443eb80 1 -- 172.21.15.1:0/1569935155 wait complete. 2023-11-16T16:41:34.347 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.330+0000 7f123443eb80 20 client.15900 trim_cache size 0 max 16384 2023-11-16T16:41:34.354 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.345+0000 7f123443eb80 1 Processor -- start 2023-11-16T16:41:34.354 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.345+0000 7f123443eb80 1 -- start start 2023-11-16T16:41:34.354 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.345+0000 7f123443eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e2b98f70 0x55f0e2b8fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.354 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.345+0000 7f123443eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0e2b99880 0x55f0e2b941f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.345+0000 7f123443eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0e2b9dcc0 0x55f0e2ba4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.345+0000 7f123443eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0e2b9c570 con 0x55f0e2b9dcc0 2023-11-16T16:41:34.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.345+0000 7f123443eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0e2b99fd0 con 0x55f0e2b98f70 2023-11-16T16:41:34.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.345+0000 7f123443eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0e2b89f30 con 0x55f0e2b99880 2023-11-16T16:41:34.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.345+0000 7f122684f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0e2b9dcc0 0x55f0e2ba4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.345+0000 7f122604e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e2b98f70 0x55f0e2b8fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.345+0000 7f122684f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0e2b9dcc0 0x55f0e2ba4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44648/0 (socket says 172.21.15.1:44648) 2023-11-16T16:41:34.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.345+0000 7f122684f700 1 -- 172.21.15.1:0/3263362543 learned_addr learned my addr 172.21.15.1:0/3263362543 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:41:34.356 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.345+0000 7f122584d700 1 --2- 172.21.15.1:0/3263362543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0e2b99880 0x55f0e2b941f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.356 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.346+0000 7f122604e700 1 -- 172.21.15.1:0/3263362543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0e2b99880 msgr2=0x55f0e2b941f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:34.356 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.346+0000 7f122604e700 1 --2- 172.21.15.1:0/3263362543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0e2b99880 0x55f0e2b941f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.356 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.346+0000 7f122604e700 1 -- 172.21.15.1:0/3263362543 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0e2b9dcc0 msgr2=0x55f0e2ba4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:34.356 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.346+0000 7f122604e700 1 --2- 172.21.15.1:0/3263362543 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0e2b9dcc0 0x55f0e2ba4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.356 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.346+0000 7f122604e700 1 -- 172.21.15.1:0/3263362543 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0e2aabee0 con 0x55f0e2b98f70 2023-11-16T16:41:34.356 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.346+0000 7f122604e700 1 --2- 172.21.15.1:0/3263362543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e2b98f70 0x55f0e2b8fc50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f121c0cbc20 tx=0x7f121c009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b192589537a49d67 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:34.356 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.346+0000 7f12155ad700 1 -- 172.21.15.1:0/3263362543 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121c0b9040 con 0x55f0e2b98f70 2023-11-16T16:41:34.356 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.346+0000 7f12155ad700 1 -- 172.21.15.1:0/3263362543 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f121c07b050 con 0x55f0e2b98f70 2023-11-16T16:41:34.357 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.346+0000 7f12155ad700 1 -- 172.21.15.1:0/3263362543 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121c0a3290 con 0x55f0e2b98f70 2023-11-16T16:41:34.357 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.346+0000 7f123443eb80 1 -- 172.21.15.1:0/3263362543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e2b98f70 msgr2=0x55f0e2b8fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:34.357 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.346+0000 7f123443eb80 1 --2- 172.21.15.1:0/3263362543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e2b98f70 0x55f0e2b8fc50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f121c0cbc20 tx=0x7f121c009310 comp rx=0 tx=0).stop 2023-11-16T16:41:34.357 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.347+0000 7f123443eb80 1 -- 172.21.15.1:0/3263362543 shutdown_connections 2023-11-16T16:41:34.357 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.347+0000 7f123443eb80 1 --2- 172.21.15.1:0/3263362543 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0e2b9dcc0 0x55f0e2ba4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.357 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.347+0000 7f123443eb80 1 --2- 172.21.15.1:0/3263362543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e2b98f70 0x55f0e2b8fc50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.357 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.347+0000 7f123443eb80 1 --2- 172.21.15.1:0/3263362543 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0e2b99880 0x55f0e2b941f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.357 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.347+0000 7f123443eb80 1 -- 172.21.15.1:0/3263362543 >> 172.21.15.1:0/3263362543 conn(0x55f0e2b8a360 msgr2=0x55f0e2bb91b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:34.357 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.347+0000 7f123443eb80 1 -- 172.21.15.1:0/3263362543 shutdown_connections 2023-11-16T16:41:34.358 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.347+0000 7f123443eb80 1 -- 172.21.15.1:0/3263362543 wait complete. 2023-11-16T16:41:34.358 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.347+0000 7f123443eb80 1 -- start start 2023-11-16T16:41:34.358 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.347+0000 7f123443eb80 1 Processor -- start 2023-11-16T16:41:34.358 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.347+0000 7f123443eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0e2b98f70 0x55f0e2bfec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.358 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.347+0000 7f123443eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e2b99880 0x55f0e2bff1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.358 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.347+0000 7f123443eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0e2b9dcc0 0x55f0e2ba9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.358 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.347+0000 7f123443eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0e2ba9640 con 0x55f0e2b9dcc0 2023-11-16T16:41:34.358 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.347+0000 7f123443eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0e2ba97b0 con 0x55f0e2b99880 2023-11-16T16:41:34.358 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.347+0000 7f123443eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0e2ba9970 con 0x55f0e2b98f70 2023-11-16T16:41:34.359 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.347+0000 7f123443eb80 10 client.0 _ll_register_callbacks cb 0x7f12180091d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:41:34.359 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.347+0000 7f122684f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0e2b9dcc0 0x55f0e2ba9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.359 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.347+0000 7f122684f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0e2b9dcc0 0x55f0e2ba9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44652/0 (socket says 172.21.15.1:44652) 2023-11-16T16:41:34.359 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.347+0000 7f122684f700 1 -- 172.21.15.1:0/1450526653 learned_addr learned my addr 172.21.15.1:0/1450526653 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:41:34.359 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.347+0000 7f122584d700 1 --2- 172.21.15.1:0/1450526653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e2b99880 0x55f0e2bff1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.359 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.348+0000 7f122604e700 1 --2- 172.21.15.1:0/1450526653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0e2b98f70 0x55f0e2bfec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.359 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.348+0000 7f122684f700 1 -- 172.21.15.1:0/1450526653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0e2b98f70 msgr2=0x55f0e2bfec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:34.359 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.348+0000 7f122684f700 1 --2- 172.21.15.1:0/1450526653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0e2b98f70 0x55f0e2bfec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.360 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.348+0000 7f122684f700 1 -- 172.21.15.1:0/1450526653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e2b99880 msgr2=0x55f0e2bff1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:34.360 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.348+0000 7f122684f700 1 --2- 172.21.15.1:0/1450526653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e2b99880 0x55f0e2bff1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.360 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.348+0000 7f122684f700 1 -- 172.21.15.1:0/1450526653 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f0e2aac190 con 0x55f0e2b9dcc0 2023-11-16T16:41:34.360 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.348+0000 7f122684f700 1 --2- 172.21.15.1:0/1450526653 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0e2b9dcc0 0x55f0e2ba9060 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f1218007f50 tx=0x7f1218007f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64780bc078b89672 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:34.360 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.348+0000 7f1209ffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:41:34.360 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.348+0000 7f1209ffb700 1 -- 172.21.15.1:0/1450526653 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1218018db0 con 0x55f0e2b9dcc0 2023-11-16T16:41:34.360 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.348+0000 7f123443eb80 1 -- 172.21.15.1:0/1450526653 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f121c0a3510 con 0x55f0e2b9dcc0 2023-11-16T16:41:34.360 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.348+0000 7f1209ffb700 1 -- 172.21.15.1:0/1450526653 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f12180203e0 con 0x55f0e2b9dcc0 2023-11-16T16:41:34.360 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.348+0000 7f1209ffb700 1 -- 172.21.15.1:0/1450526653 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12180205a0 con 0x55f0e2b9dcc0 2023-11-16T16:41:34.361 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.348+0000 7f123443eb80 10 client.15906 fetch_fsmap learned FSMap version 23 2023-11-16T16:41:34.361 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.348+0000 7f123443eb80 1 -- 172.21.15.1:0/1450526653 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55f0e28fcc60 con 0x55f0e2b9dcc0 2023-11-16T16:41:34.361 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.348+0000 7f1209ffb700 1 -- 172.21.15.1:0/1450526653 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f121802d080 con 0x55f0e2b9dcc0 2023-11-16T16:41:34.361 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.349+0000 7f1209ffb700 1 -- 172.21.15.1:0/1450526653 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1218021800 con 0x55f0e2b9dcc0 2023-11-16T16:41:34.361 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.349+0000 7f123443eb80 10 client.15906 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:41:34.361 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.349+0000 7f123443eb80 10 client.15906 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-11-16T16:41:34.361 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.349+0000 7f123443eb80 20 client.15906 populate_metadata read hostname 'smithi001' 2023-11-16T16:41:34.361 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.349+0000 7f123443eb80 1 --2- 172.21.15.1:0/1450526653 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55f0e2c06bc0 0x55f0e2c08fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.361 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.349+0000 7f123443eb80 4 client.15906 mds_command: new command op to 24482 tid=0[{"prefix": "damage ls"}] 2023-11-16T16:41:34.361 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.349+0000 7f123443eb80 1 -- 172.21.15.1:0/1450526653 --> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f0e2ab4660 con 0x55f0e2c06bc0 2023-11-16T16:41:34.362 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.349+0000 7f122604e700 1 --2- 172.21.15.1:0/1450526653 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55f0e2c06bc0 0x55f0e2c08fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.362 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.349+0000 7f122604e700 1 --2- 172.21.15.1:0/1450526653 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55f0e2c06bc0 0x55f0e2c08fd0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=103f0747f2f24521 server_cookie=13fdafe81858e738 in_seq=0 out_seq=0 2023-11-16T16:41:34.362 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.349+0000 7f1209ffb700 10 client.15906 ms_handle_connect on v2:172.21.15.26:6836/1453504863 2023-11-16T16:41:34.362 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.349+0000 7f1209ffb700 1 -- 172.21.15.1:0/1450526653 <== mds.2 v2:172.21.15.26:6836/1453504863 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f0e2ab4660 con 0x55f0e2c06bc0 2023-11-16T16:41:34.362 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.349+0000 7f1209ffb700 10 client.15906 handle_command_reply: tid=0 2023-11-16T16:41:34.362 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.349+0000 7f123443eb80 1 client.15906 shutdown 2023-11-16T16:41:34.362 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.349+0000 7f123443eb80 1 -- 172.21.15.1:0/1450526653 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0e2b9dcc0 msgr2=0x55f0e2ba9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:34.362 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.349+0000 7f123443eb80 1 --2- 172.21.15.1:0/1450526653 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0e2b9dcc0 0x55f0e2ba9060 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f1218007f50 tx=0x7f1218007f80 comp rx=0 tx=0).stop 2023-11-16T16:41:34.362 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.349+0000 7f123443eb80 1 -- 172.21.15.1:0/1450526653 shutdown_connections 2023-11-16T16:41:34.363 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.349+0000 7f123443eb80 1 --2- 172.21.15.1:0/1450526653 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55f0e2c06bc0 0x55f0e2c08fd0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.363 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.349+0000 7f123443eb80 1 --2- 172.21.15.1:0/1450526653 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f0e2b9dcc0 0x55f0e2ba9060 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.363 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.349+0000 7f123443eb80 1 --2- 172.21.15.1:0/1450526653 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f0e2b98f70 0x55f0e2bfec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.363 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.349+0000 7f123443eb80 1 --2- 172.21.15.1:0/1450526653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f0e2b99880 0x55f0e2bff1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.363 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.349+0000 7f1209ffb700 0 client.15906 ms_handle_reset on v2:172.21.15.26:6836/1453504863 2023-11-16T16:41:34.363 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.349+0000 7f123443eb80 1 -- 172.21.15.1:0/1450526653 >> 172.21.15.1:0/1450526653 conn(0x55f0e2b8a360 msgr2=0x55f0e2b941f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:34.363 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.350+0000 7f123443eb80 1 -- 172.21.15.1:0/1450526653 shutdown_connections 2023-11-16T16:41:34.363 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.350+0000 7f123443eb80 1 -- 172.21.15.1:0/1450526653 wait complete. 2023-11-16T16:41:34.363 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.350+0000 7f123443eb80 20 client.15906 trim_cache size 0 max 16384 2023-11-16T16:41:34.364 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2023-11-16T16:41:34.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.350+0000 7f12282b2700 1 -- 172.21.15.1:0/3171658 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f120c0a12c0 msgr2=0x7f120c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:34.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.350+0000 7f12282b2700 1 --2- 172.21.15.1:0/3171658 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f120c0a12c0 0x7f120c0a3770 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f12200fff60 tx=0x7f1210009260 comp rx=0 tx=0).stop 2023-11-16T16:41:34.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.350+0000 7f12282b2700 1 -- 172.21.15.1:0/3171658 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12201007c0 msgr2=0x7f1220197430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:34.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.350+0000 7f12282b2700 1 --2- 172.21.15.1:0/3171658 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12201007c0 0x7f1220197430 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f121c000c00 tx=0x7f121c0047b0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.350+0000 7f12282b2700 1 -- 172.21.15.1:0/3171658 shutdown_connections 2023-11-16T16:41:34.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.350+0000 7f12282b2700 1 --2- 172.21.15.1:0/3171658 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f120c0a12c0 0x7f120c0a3770 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.350+0000 7f12282b2700 1 --2- 172.21.15.1:0/3171658 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1220109d70 0x7f12201a24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.350+0000 7f12282b2700 1 --2- 172.21.15.1:0/3171658 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12201007c0 0x7f1220197430 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.350+0000 7f12282b2700 1 --2- 172.21.15.1:0/3171658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12201010d0 0x7f1220197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.350+0000 7f12282b2700 1 -- 172.21.15.1:0/3171658 >> 172.21.15.1:0/3171658 conn(0x7f1220074c60 msgr2=0x7f12200fc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:34.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.350+0000 7f12282b2700 1 -- 172.21.15.1:0/3171658 shutdown_connections 2023-11-16T16:41:34.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.350+0000 7f12282b2700 1 -- 172.21.15.1:0/3171658 wait complete. 2023-11-16T16:41:34.367 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-11-16T16:41:34.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.493+0000 7f9b9de9b700 1 -- 172.21.15.1:0/1752995753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b980ff940 msgr2=0x7f9b9810c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:34.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.493+0000 7f9b9de9b700 1 --2- 172.21.15.1:0/1752995753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b980ff940 0x7f9b9810c6a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9b88009990 tx=0x7f9b88009ca0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.493+0000 7f9b9de9b700 1 -- 172.21.15.1:0/1752995753 shutdown_connections 2023-11-16T16:41:34.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.493+0000 7f9b9de9b700 1 --2- 172.21.15.1:0/1752995753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b9810cbe0 0x7f9b9810efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.493+0000 7f9b9de9b700 1 --2- 172.21.15.1:0/1752995753 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b980ff030 0x7f9b980ff400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.493+0000 7f9b9de9b700 1 --2- 172.21.15.1:0/1752995753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b980ff940 0x7f9b9810c6a0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.493+0000 7f9b9de9b700 1 -- 172.21.15.1:0/1752995753 >> 172.21.15.1:0/1752995753 conn(0x7f9b980763a0 msgr2=0x7f9b980767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:34.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.493+0000 7f9b9de9b700 1 -- 172.21.15.1:0/1752995753 shutdown_connections 2023-11-16T16:41:34.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.493+0000 7f9b9de9b700 1 -- 172.21.15.1:0/1752995753 wait complete. 2023-11-16T16:41:34.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.494+0000 7f9b9de9b700 1 Processor -- start 2023-11-16T16:41:34.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.494+0000 7f9b9de9b700 1 -- start start 2023-11-16T16:41:34.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.494+0000 7f9b9de9b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b980ff030 0x7f9b98198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.494+0000 7f9b9de9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b980ff940 0x7f9b98198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.494+0000 7f9b9de9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b9810cbe0 0x7f9b9819d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.494+0000 7f9b9de9b700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9819d650 con 0x7f9b980ff940 2023-11-16T16:41:34.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.494+0000 7f9b9de9b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9819d7c0 con 0x7f9b9810cbe0 2023-11-16T16:41:34.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.494+0000 7f9b9de9b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9819d980 con 0x7f9b980ff030 2023-11-16T16:41:34.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.494+0000 7f9b96ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b980ff940 0x7f9b98198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.494+0000 7f9b96ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b980ff940 0x7f9b98198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44666/0 (socket says 172.21.15.1:44666) 2023-11-16T16:41:34.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.494+0000 7f9b96ffd700 1 -- 172.21.15.1:0/2616618541 learned_addr learned my addr 172.21.15.1:0/2616618541 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:41:34.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.494+0000 7f9b97fff700 1 --2- 172.21.15.1:0/2616618541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b9810cbe0 0x7f9b9819d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.494+0000 7f9b977fe700 1 --2- 172.21.15.1:0/2616618541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b980ff030 0x7f9b98198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.495+0000 7f9b96ffd700 1 -- 172.21.15.1:0/2616618541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b980ff030 msgr2=0x7f9b98198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:34.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.495+0000 7f9b96ffd700 1 --2- 172.21.15.1:0/2616618541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b980ff030 0x7f9b98198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.495+0000 7f9b96ffd700 1 -- 172.21.15.1:0/2616618541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b9810cbe0 msgr2=0x7f9b9819d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:34.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.495+0000 7f9b96ffd700 1 --2- 172.21.15.1:0/2616618541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b9810cbe0 0x7f9b9819d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.495+0000 7f9b96ffd700 1 -- 172.21.15.1:0/2616618541 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b880095f0 con 0x7f9b980ff940 2023-11-16T16:41:34.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.495+0000 7f9b96ffd700 1 --2- 172.21.15.1:0/2616618541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b980ff940 0x7f9b98198bb0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9b88000c00 tx=0x7f9b880047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:34.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.495+0000 7f9b94ff9700 1 -- 172.21.15.1:0/2616618541 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8801d070 con 0x7f9b980ff940 2023-11-16T16:41:34.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.495+0000 7f9b94ff9700 1 -- 172.21.15.1:0/2616618541 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b88021410 con 0x7f9b980ff940 2023-11-16T16:41:34.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.495+0000 7f9b9de9b700 1 -- 172.21.15.1:0/2616618541 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b98112550 con 0x7f9b980ff940 2023-11-16T16:41:34.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.495+0000 7f9b94ff9700 1 -- 172.21.15.1:0/2616618541 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b88017650 con 0x7f9b980ff940 2023-11-16T16:41:34.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.495+0000 7f9b9de9b700 1 -- 172.21.15.1:0/2616618541 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b981123c0 con 0x7f9b980ff940 2023-11-16T16:41:34.508 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.497+0000 7f9b94ff9700 1 -- 172.21.15.1:0/2616618541 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f9b880217f0 con 0x7f9b980ff940 2023-11-16T16:41:34.508 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.498+0000 7f9b94ff9700 1 --2- 172.21.15.1:0/2616618541 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f9b800a12c0 0x7f9b800a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.508 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.498+0000 7f9b94ff9700 1 -- 172.21.15.1:0/2616618541 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f9b880ce0a0 con 0x7f9b980ff940 2023-11-16T16:41:34.508 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.498+0000 7f9b977fe700 1 --2- 172.21.15.1:0/2616618541 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f9b800a12c0 0x7f9b800a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.508 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.498+0000 7f9b977fe700 1 --2- 172.21.15.1:0/2616618541 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f9b800a12c0 0x7f9b800a3770 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c005950 tx=0x7f9b8c00b410 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:34.508 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.499+0000 7f9baa027b80 1 Processor -- start 2023-11-16T16:41:34.508 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.499+0000 7f9baa027b80 1 -- start start 2023-11-16T16:41:34.509 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.499+0000 7f9baa027b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5568579bfb60 0x5568579bff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.509 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.499+0000 7f9baa027b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568579c0470 0x5568579c4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.509 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.499+0000 7f9baa027b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568579c48b0 0x5568579c4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.509 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.499+0000 7f9baa027b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568578c5060 con 0x5568579c0470 2023-11-16T16:41:34.509 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.499+0000 7f9baa027b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568579178a0 con 0x5568579c48b0 2023-11-16T16:41:34.509 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.499+0000 7f9baa027b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556857910d30 con 0x5568579bfb60 2023-11-16T16:41:34.509 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.499+0000 7f9b96ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568579c0470 0x5568579c4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.509 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.499+0000 7f9b96ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568579c0470 0x5568579c4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44674/0 (socket says 172.21.15.1:44674) 2023-11-16T16:41:34.509 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.499+0000 7f9b96ffd700 1 -- 172.21.15.1:0/573533836 learned_addr learned my addr 172.21.15.1:0/573533836 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:41:34.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.499+0000 7f9b97fff700 1 --2- 172.21.15.1:0/573533836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568579c48b0 0x5568579c4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.499+0000 7f9b977fe700 1 --2- 172.21.15.1:0/573533836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5568579bfb60 0x5568579bff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.499+0000 7f9b96ffd700 1 -- 172.21.15.1:0/573533836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5568579bfb60 msgr2=0x5568579bff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:34.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.499+0000 7f9b96ffd700 1 --2- 172.21.15.1:0/573533836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5568579bfb60 0x5568579bff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.499+0000 7f9b96ffd700 1 -- 172.21.15.1:0/573533836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568579c48b0 msgr2=0x5568579c4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:34.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.499+0000 7f9b96ffd700 1 --2- 172.21.15.1:0/573533836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568579c48b0 0x5568579c4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.499+0000 7f9b96ffd700 1 -- 172.21.15.1:0/573533836 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556857792630 con 0x5568579c0470 2023-11-16T16:41:34.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.499+0000 7f9b96ffd700 1 --2- 172.21.15.1:0/573533836 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568579c0470 0x5568579c4370 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9b88005910 tx=0x7f9b8809b060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c59cc5f1d89f277f server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:34.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.500+0000 7f9b7dffb700 1 -- 172.21.15.1:0/573533836 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b880a10e0 con 0x5568579c0470 2023-11-16T16:41:34.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.500+0000 7f9b7dffb700 1 -- 172.21.15.1:0/573533836 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b88021e90 con 0x5568579c0470 2023-11-16T16:41:34.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.500+0000 7f9b7dffb700 1 -- 172.21.15.1:0/573533836 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8809b6d0 con 0x5568579c0470 2023-11-16T16:41:34.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.500+0000 7f9baa027b80 1 -- 172.21.15.1:0/573533836 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568579c0470 msgr2=0x5568579c4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:34.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.500+0000 7f9baa027b80 1 --2- 172.21.15.1:0/573533836 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568579c0470 0x5568579c4370 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9b88005910 tx=0x7f9b8809b060 comp rx=0 tx=0).stop 2023-11-16T16:41:34.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.500+0000 7f9baa027b80 1 -- 172.21.15.1:0/573533836 shutdown_connections 2023-11-16T16:41:34.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.500+0000 7f9baa027b80 1 --2- 172.21.15.1:0/573533836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568579c48b0 0x5568579c4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.500+0000 7f9baa027b80 1 --2- 172.21.15.1:0/573533836 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5568579bfb60 0x5568579bff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.500+0000 7f9baa027b80 1 --2- 172.21.15.1:0/573533836 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568579c0470 0x5568579c4370 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.500+0000 7f9baa027b80 1 -- 172.21.15.1:0/573533836 >> 172.21.15.1:0/573533836 conn(0x5568579b5360 msgr2=0x5568579bab40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:34.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.500+0000 7f9baa027b80 1 -- 172.21.15.1:0/573533836 shutdown_connections 2023-11-16T16:41:34.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.500+0000 7f9baa027b80 1 -- 172.21.15.1:0/573533836 wait complete. 2023-11-16T16:41:34.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.501+0000 7f9baa027b80 1 -- start start 2023-11-16T16:41:34.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.501+0000 7f9baa027b80 1 Processor -- start 2023-11-16T16:41:34.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.501+0000 7f9baa027b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5568579bfb60 0x5568579cc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.501+0000 7f9baa027b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568579c0470 0x5568579c9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.501+0000 7f9baa027b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568579c48b0 0x5568579c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.502+0000 7f9baa027b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568578b28e0 con 0x5568579c48b0 2023-11-16T16:41:34.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.502+0000 7f9baa027b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568578b2580 con 0x5568579c0470 2023-11-16T16:41:34.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.502+0000 7f9baa027b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556857794920 con 0x5568579bfb60 2023-11-16T16:41:34.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.502+0000 7f9baa027b80 10 client.0 _ll_register_callbacks cb 0x5568578d0780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:41:34.513 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.502+0000 7f9b96ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568579c0470 0x5568579c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.513 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.502+0000 7f9b96ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568579c0470 0x5568579c9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.1:38866/0 (socket says 172.21.15.1:38866) 2023-11-16T16:41:34.513 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.502+0000 7f9b96ffd700 1 -- 172.21.15.1:0/3021297782 learned_addr learned my addr 172.21.15.1:0/3021297782 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:41:34.513 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.502+0000 7f9b977fe700 1 --2- 172.21.15.1:0/3021297782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5568579bfb60 0x5568579cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.513 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.502+0000 7f9b97fff700 1 --2- 172.21.15.1:0/3021297782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568579c48b0 0x5568579c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.513 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.502+0000 7f9b96ffd700 1 -- 172.21.15.1:0/3021297782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5568579bfb60 msgr2=0x5568579cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:34.513 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.502+0000 7f9b96ffd700 1 --2- 172.21.15.1:0/3021297782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5568579bfb60 0x5568579cc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.513 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.502+0000 7f9b96ffd700 1 -- 172.21.15.1:0/3021297782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568579c48b0 msgr2=0x5568579c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:34.513 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.502+0000 7f9b96ffd700 1 --2- 172.21.15.1:0/3021297782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568579c48b0 0x5568579c97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.513 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.502+0000 7f9b96ffd700 1 -- 172.21.15.1:0/3021297782 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b981123c0 con 0x5568579c0470 2023-11-16T16:41:34.514 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.502+0000 7f9b96ffd700 1 --2- 172.21.15.1:0/3021297782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568579c0470 0x5568579c9210 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9b8809b1a0 tx=0x7f9b8809f170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e81a71f81ffa99e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:34.514 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.502+0000 7f9b76ffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-11-16T16:41:34.514 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.502+0000 7f9b76ffd700 1 -- 172.21.15.1:0/3021297782 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8809b670 con 0x5568579c0470 2023-11-16T16:41:34.514 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.503+0000 7f9b76ffd700 1 -- 172.21.15.1:0/3021297782 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f9b8809f240 con 0x5568579c0470 2023-11-16T16:41:34.514 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.503+0000 7f9baa027b80 1 -- 172.21.15.1:0/3021297782 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5568579b2b20 con 0x5568579c0470 2023-11-16T16:41:34.514 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.503+0000 7f9b76ffd700 1 -- 172.21.15.1:0/3021297782 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b880a10b0 con 0x5568579c0470 2023-11-16T16:41:34.514 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.503+0000 7f9baa027b80 10 client.25499 fetch_fsmap learned FSMap version 23 2023-11-16T16:41:34.514 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.503+0000 7f9baa027b80 1 -- 172.21.15.1:0/3021297782 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5568578d7190 con 0x5568579c0470 2023-11-16T16:41:34.514 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.503+0000 7f9b76ffd700 1 -- 172.21.15.1:0/3021297782 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b880cb030 con 0x5568579c0470 2023-11-16T16:41:34.515 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.503+0000 7f9b76ffd700 1 -- 172.21.15.1:0/3021297782 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9b88097180 con 0x5568579c0470 2023-11-16T16:41:34.515 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.503+0000 7f9baa027b80 10 client.25499 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:41:34.515 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.503+0000 7f9baa027b80 10 client.25499 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-11-16T16:41:34.515 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.503+0000 7f9baa027b80 20 client.25499 populate_metadata read hostname 'smithi001' 2023-11-16T16:41:34.515 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.503+0000 7f9baa027b80 1 --2- 172.21.15.1:0/3021297782 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x5568579d90d0 0x5568579db4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.515 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.503+0000 7f9baa027b80 4 client.25499 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:41:34.515 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.503+0000 7f9baa027b80 1 -- 172.21.15.1:0/3021297782 --> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5568578d30a0 con 0x5568579d90d0 2023-11-16T16:41:34.515 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.503+0000 7f9b977fe700 1 --2- 172.21.15.1:0/3021297782 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x5568579d90d0 0x5568579db4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.515 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.504+0000 7f9b977fe700 1 --2- 172.21.15.1:0/3021297782 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x5568579d90d0 0x5568579db4e0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e5be165aae668374 server_cookie=ff86c55ad3f6758c in_seq=0 out_seq=0 2023-11-16T16:41:34.515 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.504+0000 7f9b76ffd700 10 client.25499 ms_handle_connect on v2:172.21.15.53:6838/1044753024 2023-11-16T16:41:34.515 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.505+0000 7f9b76ffd700 1 -- 172.21.15.1:0/3021297782 <== mds.1 v2:172.21.15.53:6838/1044753024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5568578d30a0 con 0x5568579d90d0 2023-11-16T16:41:34.516 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.505+0000 7f9b76ffd700 10 client.25499 handle_command_reply: tid=0 2023-11-16T16:41:34.516 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.505+0000 7f9baa027b80 1 client.25499 shutdown 2023-11-16T16:41:34.516 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.505+0000 7f9baa027b80 1 -- 172.21.15.1:0/3021297782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568579c0470 msgr2=0x5568579c9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:34.516 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.505+0000 7f9baa027b80 1 --2- 172.21.15.1:0/3021297782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568579c0470 0x5568579c9210 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9b8809b1a0 tx=0x7f9b8809f170 comp rx=0 tx=0).stop 2023-11-16T16:41:34.516 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.505+0000 7f9baa027b80 1 -- 172.21.15.1:0/3021297782 shutdown_connections 2023-11-16T16:41:34.516 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.505+0000 7f9baa027b80 1 --2- 172.21.15.1:0/3021297782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568579c48b0 0x5568579c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.516 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.505+0000 7f9baa027b80 1 --2- 172.21.15.1:0/3021297782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5568579bfb60 0x5568579cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.516 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.505+0000 7f9baa027b80 1 --2- 172.21.15.1:0/3021297782 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x5568579d90d0 0x5568579db4e0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.516 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.505+0000 7f9baa027b80 1 --2- 172.21.15.1:0/3021297782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568579c0470 0x5568579c9210 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.516 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.505+0000 7f9baa027b80 1 -- 172.21.15.1:0/3021297782 >> 172.21.15.1:0/3021297782 conn(0x5568579b5360 msgr2=0x5568579b9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:34.517 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.505+0000 7f9b76ffd700 0 client.25499 ms_handle_reset on v2:172.21.15.53:6838/1044753024 2023-11-16T16:41:34.517 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.505+0000 7f9baa027b80 1 -- 172.21.15.1:0/3021297782 shutdown_connections 2023-11-16T16:41:34.517 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.505+0000 7f9baa027b80 1 -- 172.21.15.1:0/3021297782 wait complete. 2023-11-16T16:41:34.517 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.505+0000 7f9baa027b80 20 client.25499 trim_cache size 0 max 16384 2023-11-16T16:41:34.525 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.518+0000 7f9baa027b80 1 Processor -- start 2023-11-16T16:41:34.525 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.518+0000 7f9baa027b80 1 -- start start 2023-11-16T16:41:34.525 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.518+0000 7f9baa027b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568579bfb60 0x5568579d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.526 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.518+0000 7f9baa027b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568579c0470 0x5568579d5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.526 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.518+0000 7f9baa027b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5568579c48b0 0x5568579cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.526 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.518+0000 7f9baa027b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568578d30a0 con 0x5568579c0470 2023-11-16T16:41:34.526 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.518+0000 7f9baa027b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568579bb710 con 0x5568579bfb60 2023-11-16T16:41:34.526 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.518+0000 7f9baa027b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568579bb8b0 con 0x5568579c48b0 2023-11-16T16:41:34.526 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.518+0000 7f9b96ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568579c0470 0x5568579d5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.526 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.518+0000 7f9b96ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568579c0470 0x5568579d5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44688/0 (socket says 172.21.15.1:44688) 2023-11-16T16:41:34.526 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.518+0000 7f9b96ffd700 1 -- 172.21.15.1:0/1607248269 learned_addr learned my addr 172.21.15.1:0/1607248269 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:41:34.526 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.518+0000 7f9b977fe700 1 --2- 172.21.15.1:0/1607248269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568579bfb60 0x5568579d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.527 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.518+0000 7f9b97fff700 1 --2- 172.21.15.1:0/1607248269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5568579c48b0 0x5568579cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.527 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.518+0000 7f9b96ffd700 1 -- 172.21.15.1:0/1607248269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5568579c48b0 msgr2=0x5568579cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:34.527 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.518+0000 7f9b96ffd700 1 --2- 172.21.15.1:0/1607248269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5568579c48b0 0x5568579cbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.527 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.518+0000 7f9b96ffd700 1 -- 172.21.15.1:0/1607248269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568579bfb60 msgr2=0x5568579d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:34.527 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.518+0000 7f9b96ffd700 1 --2- 172.21.15.1:0/1607248269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568579bfb60 0x5568579d07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.527 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.518+0000 7f9b96ffd700 1 -- 172.21.15.1:0/1607248269 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568579c8d80 con 0x5568579c0470 2023-11-16T16:41:34.527 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.518+0000 7f9b97fff700 1 --2- 172.21.15.1:0/1607248269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5568579c48b0 0x5568579cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:41:34.527 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.519+0000 7f9b96ffd700 1 --2- 172.21.15.1:0/1607248269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568579c0470 0x5568579d5910 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9b88099560 tx=0x7f9b880997c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f943ee61d0b9b2e server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:34.527 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.519+0000 7f9b7e7fc700 1 -- 172.21.15.1:0/1607248269 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8800fe20 con 0x5568579c0470 2023-11-16T16:41:34.527 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.519+0000 7f9b7e7fc700 1 -- 172.21.15.1:0/1607248269 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b880cb050 con 0x5568579c0470 2023-11-16T16:41:34.528 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.519+0000 7f9b7e7fc700 1 -- 172.21.15.1:0/1607248269 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b880a33a0 con 0x5568579c0470 2023-11-16T16:41:34.528 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.519+0000 7f9baa027b80 1 -- 172.21.15.1:0/1607248269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568579c0470 msgr2=0x5568579d5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:34.528 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.519+0000 7f9baa027b80 1 --2- 172.21.15.1:0/1607248269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568579c0470 0x5568579d5910 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9b88099560 tx=0x7f9b880997c0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.528 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.519+0000 7f9baa027b80 1 -- 172.21.15.1:0/1607248269 shutdown_connections 2023-11-16T16:41:34.528 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.519+0000 7f9baa027b80 1 --2- 172.21.15.1:0/1607248269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5568579c48b0 0x5568579cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.528 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.519+0000 7f9baa027b80 1 --2- 172.21.15.1:0/1607248269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568579bfb60 0x5568579d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.528 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.519+0000 7f9baa027b80 1 --2- 172.21.15.1:0/1607248269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568579c0470 0x5568579d5910 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.528 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.519+0000 7f9baa027b80 1 -- 172.21.15.1:0/1607248269 >> 172.21.15.1:0/1607248269 conn(0x5568579b5360 msgr2=0x5568579dfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:34.528 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.520+0000 7f9baa027b80 1 -- 172.21.15.1:0/1607248269 shutdown_connections 2023-11-16T16:41:34.528 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.520+0000 7f9baa027b80 1 -- 172.21.15.1:0/1607248269 wait complete. 2023-11-16T16:41:34.529 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.520+0000 7f9baa027b80 1 -- start start 2023-11-16T16:41:34.529 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.520+0000 7f9baa027b80 1 Processor -- start 2023-11-16T16:41:34.529 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.520+0000 7f9baa027b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5568579bfb60 0x5568579cc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.529 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.520+0000 7f9baa027b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568579c0470 0x5568579cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.529 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.520+0000 7f9baa027b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568579c48b0 0x5568579cd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.529 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.520+0000 7f9baa027b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556857a2c0f0 con 0x5568579c0470 2023-11-16T16:41:34.529 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.520+0000 7f9baa027b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556857a2c260 con 0x5568579c48b0 2023-11-16T16:41:34.529 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.520+0000 7f9baa027b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556857a2c420 con 0x5568579bfb60 2023-11-16T16:41:34.529 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.520+0000 7f9baa027b80 10 client.0 _ll_register_callbacks cb 0x7f9b90009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:41:34.529 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.520+0000 7f9b96ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568579c0470 0x5568579cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.530 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.521+0000 7f9b97fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568579c48b0 0x5568579cd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.530 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.520+0000 7f9b96ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568579c0470 0x5568579cccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44694/0 (socket says 172.21.15.1:44694) 2023-11-16T16:41:34.530 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.521+0000 7f9b96ffd700 1 -- 172.21.15.1:0/2887735304 learned_addr learned my addr 172.21.15.1:0/2887735304 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:41:34.530 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.521+0000 7f9b977fe700 1 --2- 172.21.15.1:0/2887735304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5568579bfb60 0x5568579cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.530 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.521+0000 7f9b96ffd700 1 -- 172.21.15.1:0/2887735304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5568579bfb60 msgr2=0x5568579cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:34.530 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.521+0000 7f9b96ffd700 1 --2- 172.21.15.1:0/2887735304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5568579bfb60 0x5568579cc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.530 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.521+0000 7f9b96ffd700 1 -- 172.21.15.1:0/2887735304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568579c48b0 msgr2=0x5568579cd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:34.530 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.521+0000 7f9b96ffd700 1 --2- 172.21.15.1:0/2887735304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568579c48b0 0x5568579cd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.530 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.521+0000 7f9b96ffd700 1 -- 172.21.15.1:0/2887735304 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5568578d7190 con 0x5568579c0470 2023-11-16T16:41:34.531 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.521+0000 7f9b97fff700 1 --2- 172.21.15.1:0/2887735304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568579c48b0 0x5568579cd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:41:34.531 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.521+0000 7f9b96ffd700 1 --2- 172.21.15.1:0/2887735304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568579c0470 0x5568579cccd0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9b8809b1a0 tx=0x7f9b88009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9448a09807684a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:34.531 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.521+0000 7f9b76ffd700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:41:34.531 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.521+0000 7f9b76ffd700 1 -- 172.21.15.1:0/2887735304 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8809ad50 con 0x5568579c0470 2023-11-16T16:41:34.531 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.521+0000 7f9b76ffd700 1 -- 172.21.15.1:0/2887735304 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f9b880177b0 con 0x5568579c0470 2023-11-16T16:41:34.531 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.521+0000 7f9baa027b80 1 -- 172.21.15.1:0/2887735304 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b88021e90 con 0x5568579c0470 2023-11-16T16:41:34.531 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.521+0000 7f9b76ffd700 1 -- 172.21.15.1:0/2887735304 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b880a10f0 con 0x5568579c0470 2023-11-16T16:41:34.531 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.521+0000 7f9baa027b80 10 client.15924 fetch_fsmap learned FSMap version 23 2023-11-16T16:41:34.531 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.521+0000 7f9baa027b80 1 -- 172.21.15.1:0/2887735304 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55685773ede0 con 0x5568579c0470 2023-11-16T16:41:34.531 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.521+0000 7f9b76ffd700 1 -- 172.21.15.1:0/2887735304 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b88021e90 con 0x5568579c0470 2023-11-16T16:41:34.531 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.522+0000 7f9b76ffd700 1 -- 172.21.15.1:0/2887735304 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9b88097000 con 0x5568579c0470 2023-11-16T16:41:34.532 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.522+0000 7f9baa027b80 10 client.15924 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:41:34.532 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.522+0000 7f9baa027b80 10 client.15924 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-11-16T16:41:34.532 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.522+0000 7f9baa027b80 20 client.15924 populate_metadata read hostname 'smithi001' 2023-11-16T16:41:34.532 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.522+0000 7f9baa027b80 1 --2- 172.21.15.1:0/2887735304 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x556857a2d540 0x556857a2f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.532 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.522+0000 7f9baa027b80 4 client.15924 mds_command: new command op to 24493 tid=0[{"prefix": "damage ls"}] 2023-11-16T16:41:34.532 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.522+0000 7f9baa027b80 1 -- 172.21.15.1:0/2887735304 --> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5568578df660 con 0x556857a2d540 2023-11-16T16:41:34.532 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.522+0000 7f9b977fe700 1 --2- 172.21.15.1:0/2887735304 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x556857a2d540 0x556857a2f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.532 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.522+0000 7f9b76ffd700 10 client.15924 ms_handle_connect on v2:172.21.15.53:6838/1044753024 2023-11-16T16:41:34.532 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.522+0000 7f9b977fe700 1 --2- 172.21.15.1:0/2887735304 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x556857a2d540 0x556857a2f920 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=200baddb3d57f679 server_cookie=8b238f466a9ad825 in_seq=0 out_seq=0 2023-11-16T16:41:34.532 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.523+0000 7f9b76ffd700 1 -- 172.21.15.1:0/2887735304 <== mds.1 v2:172.21.15.53:6838/1044753024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5568578df660 con 0x556857a2d540 2023-11-16T16:41:34.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.523+0000 7f9b76ffd700 10 client.15924 handle_command_reply: tid=0 2023-11-16T16:41:34.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.523+0000 7f9baa027b80 1 client.15924 shutdown 2023-11-16T16:41:34.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.523+0000 7f9baa027b80 1 -- 172.21.15.1:0/2887735304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568579c0470 msgr2=0x5568579cccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:34.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.523+0000 7f9baa027b80 1 --2- 172.21.15.1:0/2887735304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568579c0470 0x5568579cccd0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9b8809b1a0 tx=0x7f9b88009310 comp rx=0 tx=0).stop 2023-11-16T16:41:34.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.523+0000 7f9baa027b80 1 -- 172.21.15.1:0/2887735304 shutdown_connections 2023-11-16T16:41:34.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.523+0000 7f9baa027b80 1 --2- 172.21.15.1:0/2887735304 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x556857a2d540 0x556857a2f920 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.523+0000 7f9baa027b80 1 --2- 172.21.15.1:0/2887735304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568579c48b0 0x5568579cd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.523+0000 7f9baa027b80 1 --2- 172.21.15.1:0/2887735304 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5568579bfb60 0x5568579cc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.523+0000 7f9baa027b80 1 --2- 172.21.15.1:0/2887735304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5568579c0470 0x5568579cccd0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.523+0000 7f9b76ffd700 0 client.15924 ms_handle_reset on v2:172.21.15.53:6838/1044753024 2023-11-16T16:41:34.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.523+0000 7f9baa027b80 1 -- 172.21.15.1:0/2887735304 >> 172.21.15.1:0/2887735304 conn(0x5568579b5360 msgr2=0x5568579d5910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:34.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.523+0000 7f9baa027b80 1 -- 172.21.15.1:0/2887735304 shutdown_connections 2023-11-16T16:41:34.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.523+0000 7f9baa027b80 1 -- 172.21.15.1:0/2887735304 wait complete. 2023-11-16T16:41:34.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.523+0000 7f9baa027b80 20 client.15924 trim_cache size 0 max 16384 2023-11-16T16:41:34.534 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2023-11-16T16:41:34.535 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.523+0000 7f9b9de9b700 1 -- 172.21.15.1:0/2616618541 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f9b800a12c0 msgr2=0x7f9b800a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:34.535 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.523+0000 7f9b9de9b700 1 --2- 172.21.15.1:0/2616618541 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f9b800a12c0 0x7f9b800a3770 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c005950 tx=0x7f9b8c00b410 comp rx=0 tx=0).stop 2023-11-16T16:41:34.535 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.523+0000 7f9b9de9b700 1 -- 172.21.15.1:0/2616618541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b980ff940 msgr2=0x7f9b98198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:34.535 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.523+0000 7f9b9de9b700 1 --2- 172.21.15.1:0/2616618541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b980ff940 0x7f9b98198bb0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9b88000c00 tx=0x7f9b880047b0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.535 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.524+0000 7f9b9de9b700 1 -- 172.21.15.1:0/2616618541 shutdown_connections 2023-11-16T16:41:34.535 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.524+0000 7f9b9de9b700 1 --2- 172.21.15.1:0/2616618541 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f9b800a12c0 0x7f9b800a3770 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.536 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.524+0000 7f9b9de9b700 1 --2- 172.21.15.1:0/2616618541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b9810cbe0 0x7f9b9819d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.536 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.524+0000 7f9b9de9b700 1 --2- 172.21.15.1:0/2616618541 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9b980ff030 0x7f9b98198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.536 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.524+0000 7f9b9de9b700 1 --2- 172.21.15.1:0/2616618541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b980ff940 0x7f9b98198bb0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.536 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.524+0000 7f9b9de9b700 1 -- 172.21.15.1:0/2616618541 >> 172.21.15.1:0/2616618541 conn(0x7f9b980763a0 msgr2=0x7f9b980fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:34.536 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.524+0000 7f9b9de9b700 1 -- 172.21.15.1:0/2616618541 shutdown_connections 2023-11-16T16:41:34.536 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.524+0000 7f9b9de9b700 1 -- 172.21.15.1:0/2616618541 wait complete. 2023-11-16T16:41:34.539 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-16T16:41:34.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.678+0000 7f5aaaafe700 1 -- 172.21.15.1:0/1867266100 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa4067dc0 msgr2=0x7f5aa4068230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:34.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.678+0000 7f5aaaafe700 1 --2- 172.21.15.1:0/1867266100 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa4067dc0 0x7f5aa4068230 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5a900099c0 tx=0x7f5a90009cd0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.678+0000 7f5aaaafe700 1 -- 172.21.15.1:0/1867266100 shutdown_connections 2023-11-16T16:41:34.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.678+0000 7f5aaaafe700 1 --2- 172.21.15.1:0/1867266100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5aa4068770 0x7f5aa410d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.678+0000 7f5aaaafe700 1 --2- 172.21.15.1:0/1867266100 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5aa41052a0 0x7f5aa4105670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.678+0000 7f5aaaafe700 1 --2- 172.21.15.1:0/1867266100 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa4067dc0 0x7f5aa4068230 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.678+0000 7f5aaaafe700 1 -- 172.21.15.1:0/1867266100 >> 172.21.15.1:0/1867266100 conn(0x7f5aa40763a0 msgr2=0x7f5aa40767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:34.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.678+0000 7f5aaaafe700 1 -- 172.21.15.1:0/1867266100 shutdown_connections 2023-11-16T16:41:34.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.678+0000 7f5aaaafe700 1 -- 172.21.15.1:0/1867266100 wait complete. 2023-11-16T16:41:34.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.678+0000 7f5aaaafe700 1 Processor -- start 2023-11-16T16:41:34.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.679+0000 7f5aaaafe700 1 -- start start 2023-11-16T16:41:34.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.679+0000 7f5aaaafe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5aa4067dc0 0x7f5aa41963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.679+0000 7f5aaaafe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5aa4068770 0x7f5aa4196910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.679+0000 7f5aaaafe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa41052a0 0x7f5aa419ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.679+0000 7f5aaaafe700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa419b3b0 con 0x7f5aa41052a0 2023-11-16T16:41:34.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.679+0000 7f5aaaafe700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa419b520 con 0x7f5aa4067dc0 2023-11-16T16:41:34.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.679+0000 7f5aaaafe700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa419b6e0 con 0x7f5aa4068770 2023-11-16T16:41:34.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.679+0000 7f5aa909b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa41052a0 0x7f5aa419ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.679+0000 7f5aa889a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5aa4067dc0 0x7f5aa41963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.679+0000 7f5aa909b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa41052a0 0x7f5aa419ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44710/0 (socket says 172.21.15.1:44710) 2023-11-16T16:41:34.684 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.679+0000 7f5aa909b700 1 -- 172.21.15.1:0/3298295712 learned_addr learned my addr 172.21.15.1:0/3298295712 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:41:34.684 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.679+0000 7f5aa3fff700 1 --2- 172.21.15.1:0/3298295712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5aa4068770 0x7f5aa4196910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.684 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.679+0000 7f5aa889a700 1 -- 172.21.15.1:0/3298295712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5aa4068770 msgr2=0x7f5aa4196910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:34.684 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.679+0000 7f5aa889a700 1 --2- 172.21.15.1:0/3298295712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5aa4068770 0x7f5aa4196910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.684 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.679+0000 7f5aa889a700 1 -- 172.21.15.1:0/3298295712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa41052a0 msgr2=0x7f5aa419ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:34.684 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.679+0000 7f5aa889a700 1 --2- 172.21.15.1:0/3298295712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa41052a0 0x7f5aa419ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.684 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.679+0000 7f5aa889a700 1 -- 172.21.15.1:0/3298295712 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a9c009c40 con 0x7f5aa4067dc0 2023-11-16T16:41:34.684 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.679+0000 7f5aa909b700 1 --2- 172.21.15.1:0/3298295712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa41052a0 0x7f5aa419ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:41:34.684 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.680+0000 7f5aa889a700 1 --2- 172.21.15.1:0/3298295712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5aa4067dc0 0x7f5aa41963d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5a9800d750 tx=0x7f5a9800da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:34.684 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.680+0000 7f5aa1ffb700 1 -- 172.21.15.1:0/3298295712 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a98004220 con 0x7f5aa4067dc0 2023-11-16T16:41:34.684 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.680+0000 7f5aaaafe700 1 -- 172.21.15.1:0/3298295712 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a900095f0 con 0x7f5aa4067dc0 2023-11-16T16:41:34.684 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.680+0000 7f5aa1ffb700 1 -- 172.21.15.1:0/3298295712 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a98004cf0 con 0x7f5aa4067dc0 2023-11-16T16:41:34.685 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.680+0000 7f5aa1ffb700 1 -- 172.21.15.1:0/3298295712 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9800f690 con 0x7f5aa4067dc0 2023-11-16T16:41:34.685 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.680+0000 7f5aaaafe700 1 -- 172.21.15.1:0/3298295712 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa4110340 con 0x7f5aa4067dc0 2023-11-16T16:41:34.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.682+0000 7f5aa1ffb700 1 -- 172.21.15.1:0/3298295712 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f5a98004750 con 0x7f5aa4067dc0 2023-11-16T16:41:34.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.683+0000 7f5aa1ffb700 1 --2- 172.21.15.1:0/3298295712 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5a940a12c0 0x7f5a940a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.683+0000 7f5aa3fff700 1 --2- 172.21.15.1:0/3298295712 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5a940a12c0 0x7f5a940a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.683+0000 7f5aa1ffb700 1 -- 172.21.15.1:0/3298295712 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f5a980cc9b0 con 0x7f5aa4067dc0 2023-11-16T16:41:34.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.684+0000 7f5ab6c8ab80 1 Processor -- start 2023-11-16T16:41:34.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.684+0000 7f5ab6c8ab80 1 -- start start 2023-11-16T16:41:34.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.684+0000 7f5ab6c8ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd7e281b60 0x55cd7e281f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.684+0000 7f5ab6c8ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd7e282470 0x55cd7e286370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.684+0000 7f5ab6c8ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd7e2868b0 0x55cd7e286d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.684+0000 7f5ab6c8ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd7e187060 con 0x55cd7e282470 2023-11-16T16:41:34.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.684+0000 7f5ab6c8ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd7e1d98a0 con 0x55cd7e2868b0 2023-11-16T16:41:34.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.684+0000 7f5ab6c8ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd7e1d2d30 con 0x55cd7e281b60 2023-11-16T16:41:34.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.684+0000 7f5aa909b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd7e2868b0 0x55cd7e286d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.684+0000 7f5aa909b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd7e2868b0 0x55cd7e286d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.1:38902/0 (socket says 172.21.15.1:38902) 2023-11-16T16:41:34.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.684+0000 7f5aa909b700 1 -- 172.21.15.1:0/2672012676 learned_addr learned my addr 172.21.15.1:0/2672012676 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:41:34.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.684+0000 7f5aa889a700 1 --2- 172.21.15.1:0/2672012676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd7e281b60 0x55cd7e281f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.685+0000 7f5aa909b700 1 -- 172.21.15.1:0/2672012676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd7e281b60 msgr2=0x55cd7e281f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:34.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.685+0000 7f5aa909b700 1 --2- 172.21.15.1:0/2672012676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd7e281b60 0x55cd7e281f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.685+0000 7f5aa909b700 1 -- 172.21.15.1:0/2672012676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd7e282470 msgr2=0x55cd7e286370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-16T16:41:34.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.685+0000 7f5aa909b700 1 --2- 172.21.15.1:0/2672012676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd7e282470 0x55cd7e286370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.685+0000 7f5aa909b700 1 -- 172.21.15.1:0/2672012676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd7e054630 con 0x55cd7e2868b0 2023-11-16T16:41:34.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.685+0000 7f5aa3fff700 1 --2- 172.21.15.1:0/3298295712 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5a940a12c0 0x7f5a940a3770 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5a90009fd0 tx=0x7f5a9001a040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:34.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.685+0000 7f5aa909b700 1 --2- 172.21.15.1:0/2672012676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd7e2868b0 0x55cd7e286d00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c00e700 tx=0x7f5a9c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37affdf527d4c5e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:34.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.685+0000 7f5a8effd700 1 -- 172.21.15.1:0/2672012676 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c009100 con 0x55cd7e2868b0 2023-11-16T16:41:34.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.685+0000 7f5a8effd700 1 -- 172.21.15.1:0/2672012676 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a9c00f040 con 0x55cd7e2868b0 2023-11-16T16:41:34.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.685+0000 7f5a8effd700 1 -- 172.21.15.1:0/2672012676 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c004280 con 0x55cd7e2868b0 2023-11-16T16:41:34.692 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.686+0000 7f5ab6c8ab80 1 -- 172.21.15.1:0/2672012676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd7e2868b0 msgr2=0x55cd7e286d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:34.692 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.686+0000 7f5ab6c8ab80 1 --2- 172.21.15.1:0/2672012676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd7e2868b0 0x55cd7e286d00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c00e700 tx=0x7f5a9c009920 comp rx=0 tx=0).stop 2023-11-16T16:41:34.693 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.686+0000 7f5ab6c8ab80 1 -- 172.21.15.1:0/2672012676 shutdown_connections 2023-11-16T16:41:34.693 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.686+0000 7f5ab6c8ab80 1 --2- 172.21.15.1:0/2672012676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd7e2868b0 0x55cd7e286d00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.693 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.686+0000 7f5ab6c8ab80 1 --2- 172.21.15.1:0/2672012676 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd7e281b60 0x55cd7e281f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.693 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.686+0000 7f5ab6c8ab80 1 --2- 172.21.15.1:0/2672012676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd7e282470 0x55cd7e286370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.693 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.686+0000 7f5ab6c8ab80 1 -- 172.21.15.1:0/2672012676 >> 172.21.15.1:0/2672012676 conn(0x55cd7e277360 msgr2=0x55cd7e27cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:34.693 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.686+0000 7f5ab6c8ab80 1 -- 172.21.15.1:0/2672012676 shutdown_connections 2023-11-16T16:41:34.693 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.686+0000 7f5ab6c8ab80 1 -- 172.21.15.1:0/2672012676 wait complete. 2023-11-16T16:41:34.693 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.687+0000 7f5ab6c8ab80 1 -- start start 2023-11-16T16:41:34.693 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.687+0000 7f5ab6c8ab80 1 Processor -- start 2023-11-16T16:41:34.693 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.687+0000 7f5ab6c8ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd7e281b60 0x55cd7e28e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.693 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.687+0000 7f5ab6c8ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd7e282470 0x55cd7e28b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.694 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.687+0000 7f5ab6c8ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd7e2868b0 0x55cd7e28bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.694 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.687+0000 7f5ab6c8ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd7e1748e0 con 0x55cd7e2868b0 2023-11-16T16:41:34.694 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.687+0000 7f5ab6c8ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd7e174580 con 0x55cd7e282470 2023-11-16T16:41:34.694 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.687+0000 7f5ab6c8ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd7e056920 con 0x55cd7e281b60 2023-11-16T16:41:34.694 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.687+0000 7f5ab6c8ab80 10 client.0 _ll_register_callbacks cb 0x55cd7e192780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:41:34.694 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.687+0000 7f5aa909b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd7e2868b0 0x55cd7e28bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.694 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.687+0000 7f5aa3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd7e282470 0x55cd7e28b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.694 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.687+0000 7f5aa3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd7e282470 0x55cd7e28b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.1:38908/0 (socket says 172.21.15.1:38908) 2023-11-16T16:41:34.694 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.687+0000 7f5aa909b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd7e2868b0 0x55cd7e28bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44724/0 (socket says 172.21.15.1:44724) 2023-11-16T16:41:34.694 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.687+0000 7f5aa3fff700 1 -- 172.21.15.1:0/3059221450 learned_addr learned my addr 172.21.15.1:0/3059221450 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:41:34.694 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.687+0000 7f5aa889a700 1 --2- 172.21.15.1:0/3059221450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd7e281b60 0x55cd7e28e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.688+0000 7f5aa909b700 1 -- 172.21.15.1:0/3059221450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd7e281b60 msgr2=0x55cd7e28e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:34.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.688+0000 7f5aa909b700 1 --2- 172.21.15.1:0/3059221450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd7e281b60 0x55cd7e28e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.688+0000 7f5aa909b700 1 -- 172.21.15.1:0/3059221450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd7e282470 msgr2=0x55cd7e28b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:34.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.688+0000 7f5aa909b700 1 --2- 172.21.15.1:0/3059221450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd7e282470 0x55cd7e28b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.688+0000 7f5aa909b700 1 -- 172.21.15.1:0/3059221450 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a9c009260 con 0x55cd7e2868b0 2023-11-16T16:41:34.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.688+0000 7f5aa3fff700 1 --2- 172.21.15.1:0/3059221450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd7e282470 0x55cd7e28b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:41:34.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.688+0000 7f5aa909b700 1 --2- 172.21.15.1:0/3059221450 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd7e2868b0 0x55cd7e28bab0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c00ecf0 tx=0x7f5a9c007b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15bacc8fdead2991 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:34.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.688+0000 7f5a83fff700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:41:34.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.688+0000 7f5a83fff700 1 -- 172.21.15.1:0/3059221450 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c004da0 con 0x55cd7e2868b0 2023-11-16T16:41:34.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.688+0000 7f5ab6c8ab80 1 -- 172.21.15.1:0/3059221450 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cd7e274b20 con 0x55cd7e2868b0 2023-11-16T16:41:34.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.688+0000 7f5a83fff700 1 -- 172.21.15.1:0/3059221450 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f5a9c003a80 con 0x55cd7e2868b0 2023-11-16T16:41:34.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.688+0000 7f5a83fff700 1 -- 172.21.15.1:0/3059221450 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a9c003c40 con 0x55cd7e2868b0 2023-11-16T16:41:34.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.688+0000 7f5ab6c8ab80 10 client.15933 fetch_fsmap learned FSMap version 23 2023-11-16T16:41:34.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.688+0000 7f5ab6c8ab80 1 -- 172.21.15.1:0/3059221450 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55cd7e199190 con 0x55cd7e2868b0 2023-11-16T16:41:34.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.689+0000 7f5a83fff700 1 -- 172.21.15.1:0/3059221450 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cd7e274b20 con 0x55cd7e2868b0 2023-11-16T16:41:34.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.689+0000 7f5a83fff700 1 -- 172.21.15.1:0/3059221450 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5a9c017530 con 0x55cd7e2868b0 2023-11-16T16:41:34.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.689+0000 7f5ab6c8ab80 10 client.15933 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:41:34.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.689+0000 7f5ab6c8ab80 10 client.15933 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:41:34.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.689+0000 7f5ab6c8ab80 20 client.15933 populate_metadata read hostname 'smithi001' 2023-11-16T16:41:34.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.689+0000 7f5ab6c8ab80 1 --2- 172.21.15.1:0/3059221450 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55cd7e29b7d0 0x55cd7e29dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.689+0000 7f5ab6c8ab80 4 client.15933 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:41:34.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.689+0000 7f5ab6c8ab80 1 -- 172.21.15.1:0/3059221450 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd7e1950a0 con 0x55cd7e29b7d0 2023-11-16T16:41:34.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.689+0000 7f5aa889a700 1 --2- 172.21.15.1:0/3059221450 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55cd7e29b7d0 0x55cd7e29dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.689+0000 7f5a83fff700 10 client.15933 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:41:34.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.689+0000 7f5aa889a700 1 --2- 172.21.15.1:0/3059221450 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55cd7e29b7d0 0x55cd7e29dbe0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5723fb849a1ce1c8 server_cookie=5fee52a32f2ef6c3 in_seq=0 out_seq=0 2023-11-16T16:41:34.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.691+0000 7f5a83fff700 1 -- 172.21.15.1:0/3059221450 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cd7e1950a0 con 0x55cd7e29b7d0 2023-11-16T16:41:34.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.691+0000 7f5a83fff700 10 client.15933 handle_command_reply: tid=0 2023-11-16T16:41:34.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.691+0000 7f5ab6c8ab80 1 client.15933 shutdown 2023-11-16T16:41:34.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.691+0000 7f5ab6c8ab80 1 -- 172.21.15.1:0/3059221450 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd7e2868b0 msgr2=0x55cd7e28bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:34.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.691+0000 7f5ab6c8ab80 1 --2- 172.21.15.1:0/3059221450 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd7e2868b0 0x55cd7e28bab0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c00ecf0 tx=0x7f5a9c007b30 comp rx=0 tx=0).stop 2023-11-16T16:41:34.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.691+0000 7f5ab6c8ab80 1 -- 172.21.15.1:0/3059221450 shutdown_connections 2023-11-16T16:41:34.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.691+0000 7f5ab6c8ab80 1 --2- 172.21.15.1:0/3059221450 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55cd7e29b7d0 0x55cd7e29dbe0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.691+0000 7f5ab6c8ab80 1 --2- 172.21.15.1:0/3059221450 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd7e2868b0 0x55cd7e28bab0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.691+0000 7f5ab6c8ab80 1 --2- 172.21.15.1:0/3059221450 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd7e281b60 0x55cd7e28e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.691+0000 7f5a83fff700 0 client.15933 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:41:34.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.691+0000 7f5ab6c8ab80 1 --2- 172.21.15.1:0/3059221450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd7e282470 0x55cd7e28b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.692+0000 7f5ab6c8ab80 1 -- 172.21.15.1:0/3059221450 >> 172.21.15.1:0/3059221450 conn(0x55cd7e277360 msgr2=0x55cd7e287d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:34.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.692+0000 7f5ab6c8ab80 1 -- 172.21.15.1:0/3059221450 shutdown_connections 2023-11-16T16:41:34.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.692+0000 7f5ab6c8ab80 1 -- 172.21.15.1:0/3059221450 wait complete. 2023-11-16T16:41:34.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.692+0000 7f5ab6c8ab80 20 client.15933 trim_cache size 0 max 16384 2023-11-16T16:41:34.709 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.706+0000 7f5ab6c8ab80 1 Processor -- start 2023-11-16T16:41:34.709 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.706+0000 7f5ab6c8ab80 1 -- start start 2023-11-16T16:41:34.709 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.706+0000 7f5ab6c8ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd7e281b60 0x55cd7e297c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.709 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.706+0000 7f5ab6c8ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd7e282470 0x55cd7e28dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.709 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.706+0000 7f5ab6c8ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd7e2868b0 0x55cd7e28e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.709 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.706+0000 7f5ab6c8ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd7e2854a0 con 0x55cd7e2868b0 2023-11-16T16:41:34.709 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.706+0000 7f5ab6c8ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd7e285200 con 0x55cd7e281b60 2023-11-16T16:41:34.709 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.706+0000 7f5ab6c8ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd7e28b330 con 0x55cd7e282470 2023-11-16T16:41:34.709 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.706+0000 7f5aa889a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd7e281b60 0x55cd7e297c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.709 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.706+0000 7f5aa909b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd7e2868b0 0x55cd7e28e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.709 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.706+0000 7f5aa3fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd7e282470 0x55cd7e28dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.710 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.706+0000 7f5aa889a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd7e281b60 0x55cd7e297c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.1:38914/0 (socket says 172.21.15.1:38914) 2023-11-16T16:41:34.710 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.706+0000 7f5aa889a700 1 -- 172.21.15.1:0/175123402 learned_addr learned my addr 172.21.15.1:0/175123402 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:41:34.710 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.707+0000 7f5aa909b700 1 -- 172.21.15.1:0/175123402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd7e282470 msgr2=0x55cd7e28dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:34.710 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.707+0000 7f5aa909b700 1 --2- 172.21.15.1:0/175123402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd7e282470 0x55cd7e28dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.710 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.707+0000 7f5aa909b700 1 -- 172.21.15.1:0/175123402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd7e281b60 msgr2=0x55cd7e297c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:34.710 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.707+0000 7f5aa909b700 1 --2- 172.21.15.1:0/175123402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd7e281b60 0x55cd7e297c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.710 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.707+0000 7f5aa909b700 1 -- 172.21.15.1:0/175123402 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd7e198ee0 con 0x55cd7e2868b0 2023-11-16T16:41:34.710 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.707+0000 7f5aa909b700 1 --2- 172.21.15.1:0/175123402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd7e2868b0 0x55cd7e28e160 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c017940 tx=0x7f5a9c006160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa47e48402ba5603 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:34.710 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.707+0000 7f5a8f7fe700 1 -- 172.21.15.1:0/175123402 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c0098a0 con 0x55cd7e2868b0 2023-11-16T16:41:34.710 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.707+0000 7f5a8f7fe700 1 -- 172.21.15.1:0/175123402 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a9c017bb0 con 0x55cd7e2868b0 2023-11-16T16:41:34.710 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.707+0000 7f5a8f7fe700 1 -- 172.21.15.1:0/175123402 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c02c930 con 0x55cd7e2868b0 2023-11-16T16:41:34.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.708+0000 7f5ab6c8ab80 1 -- 172.21.15.1:0/175123402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd7e2868b0 msgr2=0x55cd7e28e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:34.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.708+0000 7f5ab6c8ab80 1 --2- 172.21.15.1:0/175123402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd7e2868b0 0x55cd7e28e160 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c017940 tx=0x7f5a9c006160 comp rx=0 tx=0).stop 2023-11-16T16:41:34.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.708+0000 7f5ab6c8ab80 1 -- 172.21.15.1:0/175123402 shutdown_connections 2023-11-16T16:41:34.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.708+0000 7f5ab6c8ab80 1 --2- 172.21.15.1:0/175123402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd7e2868b0 0x55cd7e28e160 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.708+0000 7f5ab6c8ab80 1 --2- 172.21.15.1:0/175123402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd7e281b60 0x55cd7e297c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.708+0000 7f5ab6c8ab80 1 --2- 172.21.15.1:0/175123402 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd7e282470 0x55cd7e28dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.708+0000 7f5ab6c8ab80 1 -- 172.21.15.1:0/175123402 >> 172.21.15.1:0/175123402 conn(0x55cd7e277360 msgr2=0x55cd7e2a2540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:34.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.708+0000 7f5ab6c8ab80 1 -- 172.21.15.1:0/175123402 shutdown_connections 2023-11-16T16:41:34.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.708+0000 7f5ab6c8ab80 1 -- 172.21.15.1:0/175123402 wait complete. 2023-11-16T16:41:34.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.708+0000 7f5ab6c8ab80 1 -- start start 2023-11-16T16:41:34.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.708+0000 7f5ab6c8ab80 1 Processor -- start 2023-11-16T16:41:34.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.709+0000 7f5ab6c8ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd7e281b60 0x55cd7e290990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.709+0000 7f5ab6c8ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd7e282470 0x55cd7e290ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.709+0000 7f5ab6c8ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd7e2868b0 0x55cd7e291410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.709+0000 7f5ab6c8ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd7e2ee730 con 0x55cd7e2868b0 2023-11-16T16:41:34.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.709+0000 7f5ab6c8ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd7e2ee870 con 0x55cd7e282470 2023-11-16T16:41:34.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.709+0000 7f5ab6c8ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd7e2ee9e0 con 0x55cd7e281b60 2023-11-16T16:41:34.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.709+0000 7f5ab6c8ab80 10 client.0 _ll_register_callbacks cb 0x7f5a98005dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:41:34.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.709+0000 7f5aa909b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd7e2868b0 0x55cd7e291410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.709+0000 7f5aa3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd7e282470 0x55cd7e290ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.709+0000 7f5aa909b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd7e2868b0 0x55cd7e291410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44738/0 (socket says 172.21.15.1:44738) 2023-11-16T16:41:34.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.709+0000 7f5aa909b700 1 -- 172.21.15.1:0/2321192930 learned_addr learned my addr 172.21.15.1:0/2321192930 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:41:34.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.709+0000 7f5aa889a700 1 --2- 172.21.15.1:0/2321192930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd7e281b60 0x55cd7e290990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.709+0000 7f5aa3fff700 1 -- 172.21.15.1:0/2321192930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd7e281b60 msgr2=0x55cd7e290990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:34.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.709+0000 7f5aa3fff700 1 --2- 172.21.15.1:0/2321192930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd7e281b60 0x55cd7e290990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.709+0000 7f5aa3fff700 1 -- 172.21.15.1:0/2321192930 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd7e2868b0 msgr2=0x55cd7e291410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:34.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.709+0000 7f5aa3fff700 1 --2- 172.21.15.1:0/2321192930 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd7e2868b0 0x55cd7e291410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.709+0000 7f5aa3fff700 1 -- 172.21.15.1:0/2321192930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd7e199190 con 0x55cd7e282470 2023-11-16T16:41:34.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.709+0000 7f5aa909b700 1 --2- 172.21.15.1:0/2321192930 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd7e2868b0 0x55cd7e291410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:41:34.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.710+0000 7f5aa3fff700 1 --2- 172.21.15.1:0/2321192930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd7e282470 0x55cd7e290ed0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5a900036c0 tx=0x7f5a90009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=865c77be28032c49 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:34.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.710+0000 7f5a83fff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-11-16T16:41:34.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.710+0000 7f5a83fff700 1 -- 172.21.15.1:0/2321192930 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a90029070 con 0x55cd7e282470 2023-11-16T16:41:34.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.710+0000 7f5a83fff700 1 -- 172.21.15.1:0/2321192930 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f5a90023b80 con 0x55cd7e282470 2023-11-16T16:41:34.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.710+0000 7f5ab6c8ab80 1 -- 172.21.15.1:0/2321192930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a9c00f040 con 0x55cd7e282470 2023-11-16T16:41:34.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.710+0000 7f5a83fff700 1 -- 172.21.15.1:0/2321192930 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a9001d040 con 0x55cd7e282470 2023-11-16T16:41:34.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.710+0000 7f5ab6c8ab80 10 client.25523 fetch_fsmap learned FSMap version 23 2023-11-16T16:41:34.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.710+0000 7f5ab6c8ab80 1 -- 172.21.15.1:0/2321192930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55cd7dfe9c60 con 0x55cd7e282470 2023-11-16T16:41:34.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.710+0000 7f5a83fff700 1 -- 172.21.15.1:0/2321192930 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a9003e080 con 0x55cd7e282470 2023-11-16T16:41:34.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.710+0000 7f5a83fff700 1 -- 172.21.15.1:0/2321192930 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5a9003a400 con 0x55cd7e282470 2023-11-16T16:41:34.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.710+0000 7f5ab6c8ab80 10 client.25523 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:41:34.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.710+0000 7f5ab6c8ab80 10 client.25523 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:41:34.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.710+0000 7f5ab6c8ab80 20 client.25523 populate_metadata read hostname 'smithi001' 2023-11-16T16:41:34.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.711+0000 7f5ab6c8ab80 1 --2- 172.21.15.1:0/2321192930 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55cd7e29b7d0 0x55cd7e2f2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:34.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.711+0000 7f5ab6c8ab80 4 client.25523 mds_command: new command op to 24521 tid=0[{"prefix": "damage ls"}] 2023-11-16T16:41:34.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.711+0000 7f5ab6c8ab80 1 -- 172.21.15.1:0/2321192930 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cd7e1a1660 con 0x55cd7e29b7d0 2023-11-16T16:41:34.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.711+0000 7f5aa889a700 1 --2- 172.21.15.1:0/2321192930 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55cd7e29b7d0 0x55cd7e2f2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:34.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.711+0000 7f5aa889a700 1 --2- 172.21.15.1:0/2321192930 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55cd7e29b7d0 0x55cd7e2f2140 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9475c12a2be4b7f7 server_cookie=1d56268a10a06348 in_seq=0 out_seq=0 2023-11-16T16:41:34.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.711+0000 7f5a83fff700 10 client.25523 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:41:34.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.713+0000 7f5a83fff700 1 -- 172.21.15.1:0/2321192930 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cd7e1a1660 con 0x55cd7e29b7d0 2023-11-16T16:41:34.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.713+0000 7f5a83fff700 10 client.25523 handle_command_reply: tid=0 2023-11-16T16:41:34.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.713+0000 7f5ab6c8ab80 1 client.25523 shutdown 2023-11-16T16:41:34.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.713+0000 7f5ab6c8ab80 1 -- 172.21.15.1:0/2321192930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd7e282470 msgr2=0x55cd7e290ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:34.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.713+0000 7f5ab6c8ab80 1 --2- 172.21.15.1:0/2321192930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd7e282470 0x55cd7e290ed0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5a900036c0 tx=0x7f5a90009310 comp rx=0 tx=0).stop 2023-11-16T16:41:34.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.714+0000 7f5ab6c8ab80 1 -- 172.21.15.1:0/2321192930 shutdown_connections 2023-11-16T16:41:34.720 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.714+0000 7f5ab6c8ab80 1 --2- 172.21.15.1:0/2321192930 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55cd7e29b7d0 0x55cd7e2f2140 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.720 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.714+0000 7f5ab6c8ab80 1 --2- 172.21.15.1:0/2321192930 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cd7e2868b0 0x55cd7e291410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.720 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.714+0000 7f5ab6c8ab80 1 --2- 172.21.15.1:0/2321192930 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55cd7e281b60 0x55cd7e290990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.720 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.714+0000 7f5ab6c8ab80 1 --2- 172.21.15.1:0/2321192930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cd7e282470 0x55cd7e290ed0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.720 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.714+0000 7f5a83fff700 0 client.25523 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:41:34.720 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.714+0000 7f5ab6c8ab80 1 -- 172.21.15.1:0/2321192930 >> 172.21.15.1:0/2321192930 conn(0x55cd7e277360 msgr2=0x55cd7e297c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:34.720 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.714+0000 7f5ab6c8ab80 1 -- 172.21.15.1:0/2321192930 shutdown_connections 2023-11-16T16:41:34.720 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.714+0000 7f5ab6c8ab80 1 -- 172.21.15.1:0/2321192930 wait complete. 2023-11-16T16:41:34.720 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.714+0000 7f5ab6c8ab80 20 client.25523 trim_cache size 0 max 16384 2023-11-16T16:41:34.721 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2023-11-16T16:41:34.721 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.714+0000 7f5aaaafe700 1 -- 172.21.15.1:0/3298295712 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5a940a12c0 msgr2=0x7f5a940a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:34.721 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.714+0000 7f5aaaafe700 1 --2- 172.21.15.1:0/3298295712 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5a940a12c0 0x7f5a940a3770 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5a90009fd0 tx=0x7f5a9001a040 comp rx=0 tx=0).stop 2023-11-16T16:41:34.721 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.714+0000 7f5aaaafe700 1 -- 172.21.15.1:0/3298295712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5aa4067dc0 msgr2=0x7f5aa41963d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:34.721 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.714+0000 7f5aaaafe700 1 --2- 172.21.15.1:0/3298295712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5aa4067dc0 0x7f5aa41963d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5a9800d750 tx=0x7f5a9800da60 comp rx=0 tx=0).stop 2023-11-16T16:41:34.722 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.714+0000 7f5aaaafe700 1 -- 172.21.15.1:0/3298295712 shutdown_connections 2023-11-16T16:41:34.722 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.714+0000 7f5aaaafe700 1 --2- 172.21.15.1:0/3298295712 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5a940a12c0 0x7f5a940a3770 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.722 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.714+0000 7f5aaaafe700 1 --2- 172.21.15.1:0/3298295712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa41052a0 0x7f5aa419ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.722 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.714+0000 7f5aaaafe700 1 --2- 172.21.15.1:0/3298295712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5aa4067dc0 0x7f5aa41963d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.722 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.714+0000 7f5aaaafe700 1 --2- 172.21.15.1:0/3298295712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5aa4068770 0x7f5aa4196910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:34.722 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.714+0000 7f5aaaafe700 1 -- 172.21.15.1:0/3298295712 >> 172.21.15.1:0/3298295712 conn(0x7f5aa40763a0 msgr2=0x7f5aa40fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:34.722 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.714+0000 7f5aaaafe700 1 -- 172.21.15.1:0/3298295712 shutdown_connections 2023-11-16T16:41:34.722 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:34.715+0000 7f5aaaafe700 1 -- 172.21.15.1:0/3298295712 wait complete. 2023-11-16T16:41:34.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:34 smithi001 ceph-mon[98800]: pgmap v75: 97 pgs: 97 active+clean; 2.7 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 6 op/s 2023-11-16T16:41:34.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:34 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3617439775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:41:35.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:34 smithi026 ceph-mon[100112]: pgmap v75: 97 pgs: 97 active+clean; 2.7 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 6 op/s 2023-11-16T16:41:35.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:34 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/3617439775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:41:35.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:34 smithi053 ceph-mon[100638]: pgmap v75: 97 pgs: 97 active+clean; 2.7 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 6 op/s 2023-11-16T16:41:35.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:34 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/3617439775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:41:35.731 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-16T16:41:35.731 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-16T16:41:35.868 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.866+0000 7f49ccef2700 1 -- 172.21.15.1:0/341654393 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c80fea50 msgr2=0x7f49c80fee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:35.869 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.866+0000 7f49ccef2700 1 --2- 172.21.15.1:0/341654393 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c80fea50 0x7f49c80fee20 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f49b80099c0 tx=0x7f49b8009cd0 comp rx=0 tx=0).stop 2023-11-16T16:41:35.869 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.866+0000 7f49ccef2700 1 -- 172.21.15.1:0/341654393 shutdown_connections 2023-11-16T16:41:35.869 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.866+0000 7f49ccef2700 1 --2- 172.21.15.1:0/341654393 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49c81055f0 0x7f49c810d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:35.869 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.866+0000 7f49ccef2700 1 --2- 172.21.15.1:0/341654393 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c80fea50 0x7f49c80fee20 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:35.869 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.866+0000 7f49ccef2700 1 --2- 172.21.15.1:0/341654393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49c80ff3f0 0x7f49c8104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:35.869 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.866+0000 7f49ccef2700 1 -- 172.21.15.1:0/341654393 >> 172.21.15.1:0/341654393 conn(0x7f49c8075880 msgr2=0x7f49c8075c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:35.869 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.866+0000 7f49ccef2700 1 -- 172.21.15.1:0/341654393 shutdown_connections 2023-11-16T16:41:35.869 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.866+0000 7f49ccef2700 1 -- 172.21.15.1:0/341654393 wait complete. 2023-11-16T16:41:35.872 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.867+0000 7f49ccef2700 1 Processor -- start 2023-11-16T16:41:35.872 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.867+0000 7f49ccef2700 1 -- start start 2023-11-16T16:41:35.872 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.868+0000 7f49ccef2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49c80fea50 0x7f49c8196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:35.873 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.868+0000 7f49ccef2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49c80ff3f0 0x7f49c8196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:35.873 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.868+0000 7f49ccef2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c81055f0 0x7f49c819acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:35.873 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.868+0000 7f49ccef2700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c819b2f0 con 0x7f49c81055f0 2023-11-16T16:41:35.873 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.868+0000 7f49ccef2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c819b460 con 0x7f49c80fea50 2023-11-16T16:41:35.873 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.868+0000 7f49ccef2700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c819b620 con 0x7f49c80ff3f0 2023-11-16T16:41:35.873 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.868+0000 7f49c659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49c80fea50 0x7f49c8196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:35.873 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.868+0000 7f49c659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49c80fea50 0x7f49c8196310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.1:38934/0 (socket says 172.21.15.1:38934) 2023-11-16T16:41:35.873 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.868+0000 7f49c659c700 1 -- 172.21.15.1:0/1395005899 learned_addr learned my addr 172.21.15.1:0/1395005899 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:41:35.874 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.868+0000 7f49c6d9d700 1 --2- 172.21.15.1:0/1395005899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c81055f0 0x7f49c819acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:35.874 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.868+0000 7f49c5d9b700 1 --2- 172.21.15.1:0/1395005899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49c80ff3f0 0x7f49c8196850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:35.874 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.868+0000 7f49c659c700 1 -- 172.21.15.1:0/1395005899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49c80ff3f0 msgr2=0x7f49c8196850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:35.874 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.868+0000 7f49c659c700 1 --2- 172.21.15.1:0/1395005899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49c80ff3f0 0x7f49c8196850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:35.874 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.868+0000 7f49c659c700 1 -- 172.21.15.1:0/1395005899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c81055f0 msgr2=0x7f49c819acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:35.874 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.868+0000 7f49c659c700 1 --2- 172.21.15.1:0/1395005899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c81055f0 0x7f49c819acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:35.874 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.868+0000 7f49c659c700 1 -- 172.21.15.1:0/1395005899 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49bc009520 con 0x7f49c80fea50 2023-11-16T16:41:35.874 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.868+0000 7f49c659c700 1 --2- 172.21.15.1:0/1395005899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49c80fea50 0x7f49c8196310 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f49b800b580 tx=0x7f49b800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:35.875 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.868+0000 7f49b77fe700 1 -- 172.21.15.1:0/1395005899 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b801d070 con 0x7f49c80fea50 2023-11-16T16:41:35.875 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.868+0000 7f49ccef2700 1 -- 172.21.15.1:0/1395005899 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49b80095f0 con 0x7f49c80fea50 2023-11-16T16:41:35.875 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.868+0000 7f49b77fe700 1 -- 172.21.15.1:0/1395005899 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49b8005050 con 0x7f49c80fea50 2023-11-16T16:41:35.875 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.868+0000 7f49b77fe700 1 -- 172.21.15.1:0/1395005899 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b8004a40 con 0x7f49c80fea50 2023-11-16T16:41:35.875 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.869+0000 7f49ccef2700 1 -- 172.21.15.1:0/1395005899 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c8110330 con 0x7f49c80fea50 2023-11-16T16:41:35.883 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.871+0000 7f49b77fe700 1 -- 172.21.15.1:0/1395005899 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f49b8005430 con 0x7f49c80fea50 2023-11-16T16:41:35.884 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.871+0000 7f49b77fe700 1 --2- 172.21.15.1:0/1395005899 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f49b00a12c0 0x7f49b00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:35.884 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.872+0000 7f49b77fe700 1 -- 172.21.15.1:0/1395005899 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f49b80cdac0 con 0x7f49c80fea50 2023-11-16T16:41:35.884 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.872+0000 7f49c5d9b700 1 --2- 172.21.15.1:0/1395005899 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f49b00a12c0 0x7f49b00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:35.884 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.872+0000 7f49d907eb80 1 Processor -- start 2023-11-16T16:41:35.884 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.872+0000 7f49c5d9b700 1 --2- 172.21.15.1:0/1395005899 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f49b00a12c0 0x7f49b00a3770 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f49c8197920 tx=0x7f49bc009260 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:35.884 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.872+0000 7f49d907eb80 1 -- start start 2023-11-16T16:41:35.884 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.872+0000 7f49d907eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638725dbbe0 0x5638725dbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:35.884 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.872+0000 7f49d907eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638725dc4f0 0x5638725e03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:35.884 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.872+0000 7f49d907eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638725e0930 0x5638725e0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:35.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.872+0000 7f49d907eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638724e10e0 con 0x5638725dbbe0 2023-11-16T16:41:35.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.872+0000 7f49d907eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563872533920 con 0x5638725e0930 2023-11-16T16:41:35.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.872+0000 7f49d907eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56387252cdb0 con 0x5638725dc4f0 2023-11-16T16:41:35.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.872+0000 7f49c659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638725dbbe0 0x5638725dbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:35.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.873+0000 7f49c659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638725dbbe0 0x5638725dbfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44756/0 (socket says 172.21.15.1:44756) 2023-11-16T16:41:35.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.873+0000 7f49c659c700 1 -- 172.21.15.1:0/3126803576 learned_addr learned my addr 172.21.15.1:0/3126803576 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:41:35.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.873+0000 7f49c6d9d700 1 --2- 172.21.15.1:0/3126803576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638725e0930 0x5638725e0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:35.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.873+0000 7f49c5d9b700 1 --2- 172.21.15.1:0/3126803576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638725dc4f0 0x5638725e03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:35.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.873+0000 7f49c659c700 1 -- 172.21.15.1:0/3126803576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638725dc4f0 msgr2=0x5638725e03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:35.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.873+0000 7f49c659c700 1 --2- 172.21.15.1:0/3126803576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638725dc4f0 0x5638725e03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:35.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.873+0000 7f49c659c700 1 -- 172.21.15.1:0/3126803576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638725e0930 msgr2=0x5638725e0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:35.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.873+0000 7f49c659c700 1 --2- 172.21.15.1:0/3126803576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638725e0930 0x5638725e0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:35.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.873+0000 7f49c659c700 1 -- 172.21.15.1:0/3126803576 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638723ae6b0 con 0x5638725dbbe0 2023-11-16T16:41:35.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.873+0000 7f49c6d9d700 1 --2- 172.21.15.1:0/3126803576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638725e0930 0x5638725e0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:41:35.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.873+0000 7f49c659c700 1 --2- 172.21.15.1:0/3126803576 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638725dbbe0 0x5638725dbfb0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f49b809abc0 tx=0x7f49b809ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ac77dc7d3091787 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:35.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.873+0000 7f49b4ff9700 1 -- 172.21.15.1:0/3126803576 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b80a2ff0 con 0x5638725dbbe0 2023-11-16T16:41:35.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.873+0000 7f49b4ff9700 1 -- 172.21.15.1:0/3126803576 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49b80051c0 con 0x5638725dbbe0 2023-11-16T16:41:35.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.873+0000 7f49b4ff9700 1 -- 172.21.15.1:0/3126803576 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b809a380 con 0x5638725dbbe0 2023-11-16T16:41:35.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.874+0000 7f49d907eb80 1 -- 172.21.15.1:0/3126803576 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638725dbbe0 msgr2=0x5638725dbfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:35.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.874+0000 7f49d907eb80 1 --2- 172.21.15.1:0/3126803576 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638725dbbe0 0x5638725dbfb0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f49b809abc0 tx=0x7f49b809ac30 comp rx=0 tx=0).stop 2023-11-16T16:41:35.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.874+0000 7f49d907eb80 1 -- 172.21.15.1:0/3126803576 shutdown_connections 2023-11-16T16:41:35.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.874+0000 7f49d907eb80 1 --2- 172.21.15.1:0/3126803576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638725e0930 0x5638725e0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:35.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.874+0000 7f49d907eb80 1 --2- 172.21.15.1:0/3126803576 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638725dbbe0 0x5638725dbfb0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:35.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.874+0000 7f49d907eb80 1 --2- 172.21.15.1:0/3126803576 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638725dc4f0 0x5638725e03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:35.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.874+0000 7f49d907eb80 1 -- 172.21.15.1:0/3126803576 >> 172.21.15.1:0/3126803576 conn(0x5638725d13e0 msgr2=0x5638725d6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:35.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.874+0000 7f49d907eb80 1 -- 172.21.15.1:0/3126803576 shutdown_connections 2023-11-16T16:41:35.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.874+0000 7f49d907eb80 1 -- 172.21.15.1:0/3126803576 wait complete. 2023-11-16T16:41:35.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.874+0000 7f49d907eb80 1 -- start start 2023-11-16T16:41:35.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.875+0000 7f49d907eb80 1 Processor -- start 2023-11-16T16:41:35.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.875+0000 7f49d907eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638725dbbe0 0x5638725e8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:35.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.875+0000 7f49d907eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638725dc4f0 0x5638725e52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:35.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.875+0000 7f49d907eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638725e0930 0x5638725e58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:35.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.875+0000 7f49d907eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638724ce960 con 0x5638725dc4f0 2023-11-16T16:41:35.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.875+0000 7f49d907eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638724ce600 con 0x5638725dbbe0 2023-11-16T16:41:35.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.875+0000 7f49d907eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638723b09a0 con 0x5638725e0930 2023-11-16T16:41:35.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.875+0000 7f49d907eb80 10 client.0 _ll_register_callbacks cb 0x5638724ec800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:41:35.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.875+0000 7f49c5d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638725dc4f0 0x5638725e52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:35.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.875+0000 7f49c659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638725dbbe0 0x5638725e8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:35.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.875+0000 7f49c5d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638725dc4f0 0x5638725e52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44764/0 (socket says 172.21.15.1:44764) 2023-11-16T16:41:35.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.875+0000 7f49c5d9b700 1 -- 172.21.15.1:0/3090700322 learned_addr learned my addr 172.21.15.1:0/3090700322 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:41:35.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.875+0000 7f49c659c700 1 --2- 172.21.15.1:0/3090700322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638725dbbe0 0x5638725e8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.1:38948/0 (socket says 172.21.15.1:38948) 2023-11-16T16:41:35.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.875+0000 7f49c6d9d700 1 --2- 172.21.15.1:0/3090700322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638725e0930 0x5638725e58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:35.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.875+0000 7f49c659c700 1 -- 172.21.15.1:0/3090700322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638725e0930 msgr2=0x5638725e58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:35.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.875+0000 7f49c659c700 1 --2- 172.21.15.1:0/3090700322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638725e0930 0x5638725e58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:35.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.875+0000 7f49c659c700 1 -- 172.21.15.1:0/3090700322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638725dc4f0 msgr2=0x5638725e52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:35.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.875+0000 7f49c659c700 1 --2- 172.21.15.1:0/3090700322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638725dc4f0 0x5638725e52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:35.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.875+0000 7f49c659c700 1 -- 172.21.15.1:0/3090700322 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c8110330 con 0x5638725dbbe0 2023-11-16T16:41:35.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.875+0000 7f49c659c700 1 --2- 172.21.15.1:0/3090700322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638725dbbe0 0x5638725e8090 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f49b80cb2d0 tx=0x7f49b809ab30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc39c754f9b6ebd5 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:35.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.875+0000 7f49adffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-11-16T16:41:35.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.876+0000 7f49adffb700 1 -- 172.21.15.1:0/3090700322 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b80a2d60 con 0x5638725dbbe0 2023-11-16T16:41:35.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.876+0000 7f49d907eb80 1 -- 172.21.15.1:0/3090700322 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5638725ceba0 con 0x5638725dbbe0 2023-11-16T16:41:35.891 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.876+0000 7f49adffb700 1 -- 172.21.15.1:0/3090700322 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f49b80a2ff0 con 0x5638725dbbe0 2023-11-16T16:41:35.891 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.876+0000 7f49adffb700 1 -- 172.21.15.1:0/3090700322 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49b8004bc0 con 0x5638725dbbe0 2023-11-16T16:41:35.891 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.876+0000 7f49d907eb80 10 client.25535 fetch_fsmap learned FSMap version 23 2023-11-16T16:41:35.891 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.876+0000 7f49d907eb80 1 -- 172.21.15.1:0/3090700322 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5638724f3210 con 0x5638725dbbe0 2023-11-16T16:41:35.891 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.876+0000 7f49adffb700 1 -- 172.21.15.1:0/3090700322 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5638725ceba0 con 0x5638725dbbe0 2023-11-16T16:41:35.891 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.876+0000 7f49adffb700 1 -- 172.21.15.1:0/3090700322 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f49b800fb60 con 0x5638725dbbe0 2023-11-16T16:41:35.891 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.876+0000 7f49d907eb80 10 client.25535 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:41:35.891 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.876+0000 7f49d907eb80 10 client.25535 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:41:35.891 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.876+0000 7f49d907eb80 20 client.25535 populate_metadata read hostname 'smithi001' 2023-11-16T16:41:35.892 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.876+0000 7f49d907eb80 1 --2- 172.21.15.1:0/3090700322 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5638725f5110 0x5638725f74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:35.892 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.876+0000 7f49d907eb80 4 client.25535 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:41:35.892 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.876+0000 7f49d907eb80 1 -- 172.21.15.1:0/3090700322 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5638724ef120 con 0x5638725f5110 2023-11-16T16:41:35.892 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.876+0000 7f49c5d9b700 1 --2- 172.21.15.1:0/3090700322 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5638725f5110 0x5638725f74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:35.892 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.877+0000 7f49c5d9b700 1 --2- 172.21.15.1:0/3090700322 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5638725f5110 0x5638725f74f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8df844b11520da0 server_cookie=4373c581bd89a7b8 in_seq=0 out_seq=0 2023-11-16T16:41:35.892 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.877+0000 7f49adffb700 10 client.25535 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:41:35.892 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.878+0000 7f49adffb700 1 -- 172.21.15.1:0/3090700322 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5638724ef120 con 0x5638725f5110 2023-11-16T16:41:35.892 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.878+0000 7f49adffb700 10 client.25535 handle_command_reply: tid=0 2023-11-16T16:41:35.892 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.878+0000 7f49d907eb80 1 client.25535 shutdown 2023-11-16T16:41:35.893 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.878+0000 7f49d907eb80 1 -- 172.21.15.1:0/3090700322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638725dbbe0 msgr2=0x5638725e8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:35.893 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.878+0000 7f49d907eb80 1 --2- 172.21.15.1:0/3090700322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638725dbbe0 0x5638725e8090 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f49b80cb2d0 tx=0x7f49b809ab30 comp rx=0 tx=0).stop 2023-11-16T16:41:35.893 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.878+0000 7f49d907eb80 1 -- 172.21.15.1:0/3090700322 shutdown_connections 2023-11-16T16:41:35.893 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.878+0000 7f49d907eb80 1 --2- 172.21.15.1:0/3090700322 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5638725f5110 0x5638725f74f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:35.893 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.878+0000 7f49d907eb80 1 --2- 172.21.15.1:0/3090700322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638725e0930 0x5638725e58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:35.893 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.878+0000 7f49d907eb80 1 --2- 172.21.15.1:0/3090700322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638725dbbe0 0x5638725e8090 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:35.893 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.878+0000 7f49d907eb80 1 --2- 172.21.15.1:0/3090700322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638725dc4f0 0x5638725e52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:35.893 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.878+0000 7f49adffb700 0 client.25535 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:41:35.893 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.878+0000 7f49d907eb80 1 -- 172.21.15.1:0/3090700322 >> 172.21.15.1:0/3090700322 conn(0x5638725d13e0 msgr2=0x5638725e1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:35.894 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.878+0000 7f49d907eb80 1 -- 172.21.15.1:0/3090700322 shutdown_connections 2023-11-16T16:41:35.894 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.878+0000 7f49d907eb80 1 -- 172.21.15.1:0/3090700322 wait complete. 2023-11-16T16:41:35.894 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.878+0000 7f49d907eb80 20 client.25535 trim_cache size 0 max 16384 2023-11-16T16:41:35.903 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.896+0000 7f49d907eb80 1 Processor -- start 2023-11-16T16:41:35.903 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.896+0000 7f49d907eb80 1 -- start start 2023-11-16T16:41:35.903 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.896+0000 7f49d907eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638725dbbe0 0x5638725e5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:35.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.896+0000 7f49d907eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638725dc4f0 0x5638725d6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:35.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.896+0000 7f49d907eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638725e0930 0x5638725ec830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:35.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.896+0000 7f49d907eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638725d84b0 con 0x5638725e0930 2023-11-16T16:41:35.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.896+0000 7f49d907eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638725d0f60 con 0x5638725dc4f0 2023-11-16T16:41:35.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.896+0000 7f49d907eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638725f7b20 con 0x5638725dbbe0 2023-11-16T16:41:35.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.896+0000 7f49c6d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638725e0930 0x5638725ec830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:35.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.896+0000 7f49c5d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638725dc4f0 0x5638725d6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:35.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.896+0000 7f49c5d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638725dc4f0 0x5638725d6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.1:38958/0 (socket says 172.21.15.1:38958) 2023-11-16T16:41:35.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.896+0000 7f49c6d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638725e0930 0x5638725ec830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44774/0 (socket says 172.21.15.1:44774) 2023-11-16T16:41:35.905 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.896+0000 7f49c5d9b700 1 -- 172.21.15.1:0/2269996999 learned_addr learned my addr 172.21.15.1:0/2269996999 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:41:35.905 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.896+0000 7f49c659c700 1 --2- 172.21.15.1:0/2269996999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638725dbbe0 0x5638725e5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:35.905 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.897+0000 7f49c5d9b700 1 -- 172.21.15.1:0/2269996999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638725dbbe0 msgr2=0x5638725e5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:35.905 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.897+0000 7f49c5d9b700 1 --2- 172.21.15.1:0/2269996999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638725dbbe0 0x5638725e5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:35.905 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.897+0000 7f49c5d9b700 1 -- 172.21.15.1:0/2269996999 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638725e0930 msgr2=0x5638725ec830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:35.905 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.897+0000 7f49c5d9b700 1 --2- 172.21.15.1:0/2269996999 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638725e0930 0x5638725ec830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:35.905 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.897+0000 7f49c5d9b700 1 -- 172.21.15.1:0/2269996999 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638725d2b80 con 0x5638725dc4f0 2023-11-16T16:41:35.905 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.897+0000 7f49c5d9b700 1 --2- 172.21.15.1:0/2269996999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638725dc4f0 0x5638725d6cc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x5638725f7d60 tx=0x7f49bc014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cea8009534dfce89 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:35.905 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.897+0000 7f49b57fa700 1 -- 172.21.15.1:0/2269996999 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc023040 con 0x5638725dc4f0 2023-11-16T16:41:35.906 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.897+0000 7f49b57fa700 1 -- 172.21.15.1:0/2269996999 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49bc025040 con 0x5638725dc4f0 2023-11-16T16:41:35.906 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.897+0000 7f49b57fa700 1 -- 172.21.15.1:0/2269996999 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc01d740 con 0x5638725dc4f0 2023-11-16T16:41:35.906 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.898+0000 7f49d907eb80 1 -- 172.21.15.1:0/2269996999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638725dc4f0 msgr2=0x5638725d6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:35.906 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.898+0000 7f49d907eb80 1 --2- 172.21.15.1:0/2269996999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638725dc4f0 0x5638725d6cc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x5638725f7d60 tx=0x7f49bc014000 comp rx=0 tx=0).stop 2023-11-16T16:41:35.906 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.898+0000 7f49d907eb80 1 -- 172.21.15.1:0/2269996999 shutdown_connections 2023-11-16T16:41:35.906 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.898+0000 7f49d907eb80 1 --2- 172.21.15.1:0/2269996999 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638725e0930 0x5638725ec830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:35.906 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.898+0000 7f49d907eb80 1 --2- 172.21.15.1:0/2269996999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638725dbbe0 0x5638725e5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:35.906 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.898+0000 7f49d907eb80 1 --2- 172.21.15.1:0/2269996999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638725dc4f0 0x5638725d6cc0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:35.906 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.898+0000 7f49d907eb80 1 -- 172.21.15.1:0/2269996999 >> 172.21.15.1:0/2269996999 conn(0x5638725d13e0 msgr2=0x5638725e4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:35.907 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.898+0000 7f49d907eb80 1 -- 172.21.15.1:0/2269996999 shutdown_connections 2023-11-16T16:41:35.907 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.898+0000 7f49d907eb80 1 -- 172.21.15.1:0/2269996999 wait complete. 2023-11-16T16:41:35.907 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.898+0000 7f49d907eb80 1 -- start start 2023-11-16T16:41:35.907 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.898+0000 7f49d907eb80 1 Processor -- start 2023-11-16T16:41:35.907 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.899+0000 7f49d907eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638725dbbe0 0x5638725ec990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:35.907 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.899+0000 7f49d907eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638725dc4f0 0x5638725e8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:35.907 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.899+0000 7f49d907eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638725e0930 0x5638725e9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:35.907 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.899+0000 7f49d907eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638725e9b00 con 0x5638725dc4f0 2023-11-16T16:41:35.907 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.899+0000 7f49d907eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638725e9c70 con 0x5638725e0930 2023-11-16T16:41:35.907 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.899+0000 7f49d907eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563872648f30 con 0x5638725dbbe0 2023-11-16T16:41:35.908 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.899+0000 7f49d907eb80 10 client.0 _ll_register_callbacks cb 0x7f49c0009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:41:35.908 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.899+0000 7f49c5d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638725dc4f0 0x5638725e8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:35.908 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.899+0000 7f49c6d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638725e0930 0x5638725e9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:35.908 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.899+0000 7f49c6d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638725e0930 0x5638725e9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.1:38966/0 (socket says 172.21.15.1:38966) 2023-11-16T16:41:35.908 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.899+0000 7f49c6d9d700 1 -- 172.21.15.1:0/3220140950 learned_addr learned my addr 172.21.15.1:0/3220140950 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:41:35.908 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.899+0000 7f49c659c700 1 --2- 172.21.15.1:0/3220140950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638725dbbe0 0x5638725ec990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:35.908 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.899+0000 7f49c6d9d700 1 -- 172.21.15.1:0/3220140950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638725dbbe0 msgr2=0x5638725ec990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:35.908 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.899+0000 7f49c6d9d700 1 --2- 172.21.15.1:0/3220140950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638725dbbe0 0x5638725ec990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:35.908 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.899+0000 7f49c6d9d700 1 -- 172.21.15.1:0/3220140950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638725dc4f0 msgr2=0x5638725e8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:35.909 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.899+0000 7f49c6d9d700 1 --2- 172.21.15.1:0/3220140950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638725dc4f0 0x5638725e8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:35.909 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.899+0000 7f49c6d9d700 1 -- 172.21.15.1:0/3220140950 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49bc017070 con 0x5638725e0930 2023-11-16T16:41:35.909 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.899+0000 7f49c5d9b700 1 --2- 172.21.15.1:0/3220140950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638725dc4f0 0x5638725e8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:41:35.909 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.899+0000 7f49c6d9d700 1 --2- 172.21.15.1:0/3220140950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638725e0930 0x5638725e9420 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f49c0007fb0 tx=0x7f49c0009d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b17f8d39942c6ba server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:35.909 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.899+0000 7f49adffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-11-16T16:41:35.909 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.899+0000 7f49adffb700 1 -- 172.21.15.1:0/3220140950 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c0009be0 con 0x5638725e0930 2023-11-16T16:41:35.909 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.899+0000 7f49adffb700 1 -- 172.21.15.1:0/3220140950 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f49c0018ca0 con 0x5638725e0930 2023-11-16T16:41:35.909 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.899+0000 7f49d907eb80 1 -- 172.21.15.1:0/3220140950 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f49b80051c0 con 0x5638725e0930 2023-11-16T16:41:35.910 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.900+0000 7f49adffb700 1 -- 172.21.15.1:0/3220140950 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49c0018e60 con 0x5638725e0930 2023-11-16T16:41:35.910 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.900+0000 7f49d907eb80 10 client.25541 fetch_fsmap learned FSMap version 23 2023-11-16T16:41:35.910 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.900+0000 7f49d907eb80 1 -- 172.21.15.1:0/3220140950 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5638724f2f60 con 0x5638725e0930 2023-11-16T16:41:35.910 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.900+0000 7f49adffb700 1 -- 172.21.15.1:0/3220140950 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49c002d080 con 0x5638725e0930 2023-11-16T16:41:35.910 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.900+0000 7f49adffb700 1 -- 172.21.15.1:0/3220140950 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f49c0029620 con 0x5638725e0930 2023-11-16T16:41:35.910 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.900+0000 7f49d907eb80 10 client.25541 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:41:35.910 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.900+0000 7f49d907eb80 10 client.25541 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:41:35.911 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.900+0000 7f49d907eb80 20 client.25541 populate_metadata read hostname 'smithi001' 2023-11-16T16:41:35.911 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.900+0000 7f49d907eb80 1 --2- 172.21.15.1:0/3220140950 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56387264a780 0x56387264cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:35.911 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.900+0000 7f49d907eb80 4 client.25541 mds_command: new command op to 24521 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-16T16:41:35.911 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.900+0000 7f49d907eb80 1 -- 172.21.15.1:0/3220140950 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5638724fb6e0 con 0x56387264a780 2023-11-16T16:41:35.911 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.900+0000 7f49c659c700 1 --2- 172.21.15.1:0/3220140950 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56387264a780 0x56387264cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:35.911 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.901+0000 7f49adffb700 10 client.25541 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:41:35.911 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:35.901+0000 7f49c659c700 1 --2- 172.21.15.1:0/3220140950 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56387264a780 0x56387264cb90 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e0ff06a18a2c13f server_cookie=69cee53175184996 in_seq=0 out_seq=0 2023-11-16T16:41:36.116 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.114+0000 7f49adffb700 1 -- 172.21.15.1:0/3220140950 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5638724fb6e0 con 0x56387264a780 2023-11-16T16:41:36.116 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.114+0000 7f49adffb700 10 client.25541 handle_command_reply: tid=0 2023-11-16T16:41:36.117 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.114+0000 7f49d907eb80 1 client.25541 shutdown 2023-11-16T16:41:36.117 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.114+0000 7f49d907eb80 1 -- 172.21.15.1:0/3220140950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638725e0930 msgr2=0x5638725e9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:36.117 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.114+0000 7f49d907eb80 1 --2- 172.21.15.1:0/3220140950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638725e0930 0x5638725e9420 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f49c0007fb0 tx=0x7f49c0009d40 comp rx=0 tx=0).stop 2023-11-16T16:41:36.117 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.114+0000 7f49d907eb80 1 -- 172.21.15.1:0/3220140950 shutdown_connections 2023-11-16T16:41:36.117 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.114+0000 7f49d907eb80 1 --2- 172.21.15.1:0/3220140950 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56387264a780 0x56387264cb90 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.117 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.114+0000 7f49d907eb80 1 --2- 172.21.15.1:0/3220140950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5638725e0930 0x5638725e9420 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.117 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.114+0000 7f49d907eb80 1 --2- 172.21.15.1:0/3220140950 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5638725dbbe0 0x5638725ec990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.117 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.114+0000 7f49d907eb80 1 --2- 172.21.15.1:0/3220140950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638725dc4f0 0x5638725e8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.118 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.114+0000 7f49adffb700 0 client.25541 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:41:36.118 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.114+0000 7f49d907eb80 1 -- 172.21.15.1:0/3220140950 >> 172.21.15.1:0/3220140950 conn(0x5638725d13e0 msgr2=0x5638725da9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:36.118 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.114+0000 7f49d907eb80 1 -- 172.21.15.1:0/3220140950 shutdown_connections 2023-11-16T16:41:36.118 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.114+0000 7f49d907eb80 1 -- 172.21.15.1:0/3220140950 wait complete. 2023-11-16T16:41:36.118 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.114+0000 7f49d907eb80 20 client.25541 trim_cache size 0 max 16384 2023-11-16T16:41:36.118 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:41:36.119 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2023-11-16T16:41:36.119 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "a0702666-30d2-4c77-88ef-bed563919cc5", 2023-11-16T16:41:36.119 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2023-11-16T16:41:36.119 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:41:36.120 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.114+0000 7f49ccef2700 1 -- 172.21.15.1:0/1395005899 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f49b00a12c0 msgr2=0x7f49b00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:36.120 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.114+0000 7f49ccef2700 1 --2- 172.21.15.1:0/1395005899 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f49b00a12c0 0x7f49b00a3770 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f49c8197920 tx=0x7f49bc009260 comp rx=0 tx=0).stop 2023-11-16T16:41:36.120 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.114+0000 7f49ccef2700 1 -- 172.21.15.1:0/1395005899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49c80fea50 msgr2=0x7f49c8196310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:36.121 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.114+0000 7f49ccef2700 1 --2- 172.21.15.1:0/1395005899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49c80fea50 0x7f49c8196310 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f49b800b580 tx=0x7f49b800f690 comp rx=0 tx=0).stop 2023-11-16T16:41:36.121 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.114+0000 7f49ccef2700 1 -- 172.21.15.1:0/1395005899 shutdown_connections 2023-11-16T16:41:36.121 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.114+0000 7f49ccef2700 1 --2- 172.21.15.1:0/1395005899 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f49b00a12c0 0x7f49b00a3770 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.121 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.114+0000 7f49ccef2700 1 --2- 172.21.15.1:0/1395005899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c81055f0 0x7f49c819acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.121 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.115+0000 7f49ccef2700 1 --2- 172.21.15.1:0/1395005899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49c80fea50 0x7f49c8196310 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.121 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.115+0000 7f49ccef2700 1 --2- 172.21.15.1:0/1395005899 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f49c80ff3f0 0x7f49c8196850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.121 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.115+0000 7f49ccef2700 1 -- 172.21.15.1:0/1395005899 >> 172.21.15.1:0/1395005899 conn(0x7f49c8075880 msgr2=0x7f49c80fc490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:36.121 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.115+0000 7f49ccef2700 1 -- 172.21.15.1:0/1395005899 shutdown_connections 2023-11-16T16:41:36.121 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.115+0000 7f49ccef2700 1 -- 172.21.15.1:0/1395005899 wait complete. 2023-11-16T16:41:36.131 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-16T16:41:36.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.271+0000 7f5b9a679700 1 -- 172.21.15.1:0/2016584177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b94068770 msgr2=0x7f5b9410d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:36.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.271+0000 7f5b9a679700 1 --2- 172.21.15.1:0/2016584177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b94068770 0x7f5b9410d0e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c009f90 tx=0x7f5b8c00b330 comp rx=0 tx=0).stop 2023-11-16T16:41:36.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.271+0000 7f5b9a679700 1 -- 172.21.15.1:0/2016584177 shutdown_connections 2023-11-16T16:41:36.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.271+0000 7f5b9a679700 1 --2- 172.21.15.1:0/2016584177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b94068770 0x7f5b9410d0e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.271+0000 7f5b9a679700 1 --2- 172.21.15.1:0/2016584177 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b94105250 0x7f5b94105620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.271+0000 7f5b9a679700 1 --2- 172.21.15.1:0/2016584177 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b94067dc0 0x7f5b94068230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.271+0000 7f5b9a679700 1 -- 172.21.15.1:0/2016584177 >> 172.21.15.1:0/2016584177 conn(0x7f5b940763a0 msgr2=0x7f5b940767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:36.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.271+0000 7f5b9a679700 1 -- 172.21.15.1:0/2016584177 shutdown_connections 2023-11-16T16:41:36.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.271+0000 7f5b9a679700 1 -- 172.21.15.1:0/2016584177 wait complete. 2023-11-16T16:41:36.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.272+0000 7f5b9a679700 1 Processor -- start 2023-11-16T16:41:36.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.272+0000 7f5b9a679700 1 -- start start 2023-11-16T16:41:36.277 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.272+0000 7f5b9a679700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b94067dc0 0x7f5b94196530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:36.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.272+0000 7f5b9a679700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b94068770 0x7f5b94196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:36.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.272+0000 7f5b9a679700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b94105250 0x7f5b9419ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:36.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.272+0000 7f5b9a679700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9419b470 con 0x7f5b94105250 2023-11-16T16:41:36.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.272+0000 7f5b9a679700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9419b5e0 con 0x7f5b94068770 2023-11-16T16:41:36.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.272+0000 7f5b9a679700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9419b7a0 con 0x7f5b94067dc0 2023-11-16T16:41:36.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.272+0000 7f5b98c16700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b94105250 0x7f5b9419ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:36.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.272+0000 7f5b937fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b94068770 0x7f5b94196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:36.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.272+0000 7f5b93fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b94067dc0 0x7f5b94196530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:36.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.272+0000 7f5b937fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b94068770 0x7f5b94196a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.1:38978/0 (socket says 172.21.15.1:38978) 2023-11-16T16:41:36.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.272+0000 7f5b98c16700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b94105250 0x7f5b9419ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44794/0 (socket says 172.21.15.1:44794) 2023-11-16T16:41:36.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.272+0000 7f5b937fe700 1 -- 172.21.15.1:0/3234655103 learned_addr learned my addr 172.21.15.1:0/3234655103 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:41:36.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.273+0000 7f5b93fff700 1 -- 172.21.15.1:0/3234655103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b94068770 msgr2=0x7f5b94196a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:36.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.273+0000 7f5b93fff700 1 --2- 172.21.15.1:0/3234655103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b94068770 0x7f5b94196a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.273+0000 7f5b93fff700 1 -- 172.21.15.1:0/3234655103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b94105250 msgr2=0x7f5b9419ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:36.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.273+0000 7f5b93fff700 1 --2- 172.21.15.1:0/3234655103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b94105250 0x7f5b9419ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.273+0000 7f5b93fff700 1 -- 172.21.15.1:0/3234655103 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b880095f0 con 0x7f5b94067dc0 2023-11-16T16:41:36.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.273+0000 7f5b98c16700 1 --2- 172.21.15.1:0/3234655103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b94105250 0x7f5b9419ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:41:36.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.273+0000 7f5b937fe700 1 --2- 172.21.15.1:0/3234655103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b94068770 0x7f5b94196a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:41:36.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.273+0000 7f5b93fff700 1 --2- 172.21.15.1:0/3234655103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b94067dc0 0x7f5b94196530 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5b8400e9f0 tx=0x7f5b8400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:36.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.273+0000 7f5b917fa700 1 -- 172.21.15.1:0/3234655103 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8400c9c0 con 0x7f5b94067dc0 2023-11-16T16:41:36.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.273+0000 7f5b9a679700 1 -- 172.21.15.1:0/3234655103 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b8c009c40 con 0x7f5b94067dc0 2023-11-16T16:41:36.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.273+0000 7f5b917fa700 1 -- 172.21.15.1:0/3234655103 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b84010430 con 0x7f5b94067dc0 2023-11-16T16:41:36.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.273+0000 7f5b917fa700 1 -- 172.21.15.1:0/3234655103 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b84018680 con 0x7f5b94067dc0 2023-11-16T16:41:36.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.273+0000 7f5b9a679700 1 -- 172.21.15.1:0/3234655103 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b941103d0 con 0x7f5b94067dc0 2023-11-16T16:41:36.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.276+0000 7f5b917fa700 1 -- 172.21.15.1:0/3234655103 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f5b8400cb20 con 0x7f5b94067dc0 2023-11-16T16:41:36.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.276+0000 7f5b917fa700 1 --2- 172.21.15.1:0/3234655103 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5b7c0a12c0 0x7f5b7c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:36.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.276+0000 7f5b937fe700 1 --2- 172.21.15.1:0/3234655103 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5b7c0a12c0 0x7f5b7c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:36.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.276+0000 7f5b917fa700 1 -- 172.21.15.1:0/3234655103 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f5b84014070 con 0x7f5b94067dc0 2023-11-16T16:41:36.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.277+0000 7f5b937fe700 1 --2- 172.21.15.1:0/3234655103 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5b7c0a12c0 0x7f5b7c0a3770 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5b94197b40 tx=0x7f5b88009310 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:36.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.277+0000 7f5ba6805b80 1 Processor -- start 2023-11-16T16:41:36.290 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.277+0000 7f5ba6805b80 1 -- start start 2023-11-16T16:41:36.290 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.277+0000 7f5ba6805b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559678b4eb60 0x559678b4ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:36.290 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.277+0000 7f5ba6805b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559678b4f470 0x559678b53370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:36.290 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.277+0000 7f5ba6805b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559678b538b0 0x559678b53d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:36.290 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.277+0000 7f5ba6805b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559678a54060 con 0x559678b4f470 2023-11-16T16:41:36.290 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.277+0000 7f5ba6805b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x559678aa68a0 con 0x559678b538b0 2023-11-16T16:41:36.290 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.277+0000 7f5ba6805b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559678a9fd30 con 0x559678b4eb60 2023-11-16T16:41:36.290 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.278+0000 7f5b93fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559678b4eb60 0x559678b4ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:36.291 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.278+0000 7f5b98c16700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559678b538b0 0x559678b53d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:36.291 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.278+0000 7f5b937fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559678b4f470 0x559678b53370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:36.291 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.278+0000 7f5b98c16700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559678b538b0 0x559678b53d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.1:38988/0 (socket says 172.21.15.1:38988) 2023-11-16T16:41:36.291 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.278+0000 7f5b98c16700 1 -- 172.21.15.1:0/1608606851 learned_addr learned my addr 172.21.15.1:0/1608606851 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:41:36.291 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.278+0000 7f5b98c16700 1 -- 172.21.15.1:0/1608606851 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559678b4eb60 msgr2=0x559678b4ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:36.291 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.278+0000 7f5b98c16700 1 --2- 172.21.15.1:0/1608606851 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559678b4eb60 0x559678b4ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.291 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.278+0000 7f5b98c16700 1 -- 172.21.15.1:0/1608606851 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559678b4f470 msgr2=0x559678b53370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:36.291 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.278+0000 7f5b98c16700 1 --2- 172.21.15.1:0/1608606851 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559678b4f470 0x559678b53370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.291 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.278+0000 7f5b98c16700 1 -- 172.21.15.1:0/1608606851 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559678921630 con 0x559678b538b0 2023-11-16T16:41:36.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.278+0000 7f5b98c16700 1 --2- 172.21.15.1:0/1608606851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559678b538b0 0x559678b53d00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c006080 tx=0x7f5b8c009ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94ceac954610c7e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:36.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.278+0000 7f5b7a7fc700 1 -- 172.21.15.1:0/1608606851 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c00e040 con 0x559678b538b0 2023-11-16T16:41:36.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.278+0000 7f5b7a7fc700 1 -- 172.21.15.1:0/1608606851 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b8c007cd0 con 0x559678b538b0 2023-11-16T16:41:36.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.278+0000 7f5b7a7fc700 1 -- 172.21.15.1:0/1608606851 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c01c6a0 con 0x559678b538b0 2023-11-16T16:41:36.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.279+0000 7f5ba6805b80 1 -- 172.21.15.1:0/1608606851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559678b538b0 msgr2=0x559678b53d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:36.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.279+0000 7f5ba6805b80 1 --2- 172.21.15.1:0/1608606851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559678b538b0 0x559678b53d00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c006080 tx=0x7f5b8c009ac0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.279+0000 7f5ba6805b80 1 -- 172.21.15.1:0/1608606851 shutdown_connections 2023-11-16T16:41:36.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.279+0000 7f5ba6805b80 1 --2- 172.21.15.1:0/1608606851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559678b538b0 0x559678b53d00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.279+0000 7f5ba6805b80 1 --2- 172.21.15.1:0/1608606851 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559678b4eb60 0x559678b4ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.293 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.279+0000 7f5ba6805b80 1 --2- 172.21.15.1:0/1608606851 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559678b4f470 0x559678b53370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.293 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.279+0000 7f5ba6805b80 1 -- 172.21.15.1:0/1608606851 >> 172.21.15.1:0/1608606851 conn(0x559678b44360 msgr2=0x559678b49b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:36.293 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.279+0000 7f5ba6805b80 1 -- 172.21.15.1:0/1608606851 shutdown_connections 2023-11-16T16:41:36.293 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.280+0000 7f5ba6805b80 1 -- 172.21.15.1:0/1608606851 wait complete. 2023-11-16T16:41:36.293 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.280+0000 7f5ba6805b80 1 -- start start 2023-11-16T16:41:36.293 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.280+0000 7f5ba6805b80 1 Processor -- start 2023-11-16T16:41:36.293 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.280+0000 7f5ba6805b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559678b4eb60 0x559678b5aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:36.293 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.280+0000 7f5ba6805b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559678b4f470 0x559678b58270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:36.293 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.280+0000 7f5ba6805b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559678b538b0 0x559678b588a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:36.294 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.280+0000 7f5ba6805b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559678a418e0 con 0x559678b4eb60 2023-11-16T16:41:36.294 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.280+0000 7f5ba6805b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x559678a41580 con 0x559678b538b0 2023-11-16T16:41:36.294 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.280+0000 7f5ba6805b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559678923920 con 0x559678b4f470 2023-11-16T16:41:36.294 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.281+0000 7f5ba6805b80 10 client.0 _ll_register_callbacks cb 0x559678a5f780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:41:36.294 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.281+0000 7f5b93fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559678b4eb60 0x559678b5aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:36.294 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.281+0000 7f5b937fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559678b4f470 0x559678b58270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:36.294 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.281+0000 7f5b93fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559678b4eb60 0x559678b5aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44804/0 (socket says 172.21.15.1:44804) 2023-11-16T16:41:36.294 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.281+0000 7f5b93fff700 1 -- 172.21.15.1:0/3057548914 learned_addr learned my addr 172.21.15.1:0/3057548914 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:41:36.295 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.281+0000 7f5b98c16700 1 --2- 172.21.15.1:0/3057548914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559678b538b0 0x559678b588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:36.295 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.281+0000 7f5b937fe700 1 -- 172.21.15.1:0/3057548914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559678b538b0 msgr2=0x559678b588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:36.295 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.281+0000 7f5b937fe700 1 --2- 172.21.15.1:0/3057548914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559678b538b0 0x559678b588a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.295 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.281+0000 7f5b937fe700 1 -- 172.21.15.1:0/3057548914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559678b4eb60 msgr2=0x559678b5aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:36.295 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.281+0000 7f5b937fe700 1 --2- 172.21.15.1:0/3057548914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559678b4eb60 0x559678b5aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.295 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.281+0000 7f5b937fe700 1 -- 172.21.15.1:0/3057548914 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b9410da60 con 0x559678b4f470 2023-11-16T16:41:36.295 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.281+0000 7f5b93fff700 1 --2- 172.21.15.1:0/3057548914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559678b4eb60 0x559678b5aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:41:36.295 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.281+0000 7f5b937fe700 1 --2- 172.21.15.1:0/3057548914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559678b4f470 0x559678b58270 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5b88009930 tx=0x7f5b880047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=154faf28efaaf45d server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:36.295 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.281+0000 7f5b737fe700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-16T16:41:36.296 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.281+0000 7f5b737fe700 1 -- 172.21.15.1:0/3057548914 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b88022070 con 0x559678b4f470 2023-11-16T16:41:36.296 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.281+0000 7f5b737fe700 1 -- 172.21.15.1:0/3057548914 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f5b8801dce0 con 0x559678b4f470 2023-11-16T16:41:36.296 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.281+0000 7f5ba6805b80 1 -- 172.21.15.1:0/3057548914 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559678b41b20 con 0x559678b4f470 2023-11-16T16:41:36.296 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.282+0000 7f5b737fe700 1 -- 172.21.15.1:0/3057548914 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b8801dea0 con 0x559678b4f470 2023-11-16T16:41:36.296 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.282+0000 7f5ba6805b80 10 client.25534 fetch_fsmap learned FSMap version 23 2023-11-16T16:41:36.296 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.282+0000 7f5ba6805b80 1 -- 172.21.15.1:0/3057548914 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x559678a66190 con 0x559678b4f470 2023-11-16T16:41:36.296 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.282+0000 7f5b737fe700 1 -- 172.21.15.1:0/3057548914 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b88033020 con 0x559678b4f470 2023-11-16T16:41:36.296 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.282+0000 7f5b737fe700 1 -- 172.21.15.1:0/3057548914 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5b88003d00 con 0x559678b4f470 2023-11-16T16:41:36.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.282+0000 7f5ba6805b80 10 client.25534 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:41:36.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.282+0000 7f5ba6805b80 10 client.25534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:41:36.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.282+0000 7f5ba6805b80 20 client.25534 populate_metadata read hostname 'smithi001' 2023-11-16T16:41:36.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.282+0000 7f5ba6805b80 1 --2- 172.21.15.1:0/3057548914 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x559678b68850 0x559678b6ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:36.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.282+0000 7f5ba6805b80 4 client.25534 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:41:36.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.282+0000 7f5ba6805b80 1 -- 172.21.15.1:0/3057548914 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559678a620a0 con 0x559678b68850 2023-11-16T16:41:36.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.282+0000 7f5b93fff700 1 --2- 172.21.15.1:0/3057548914 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x559678b68850 0x559678b6ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:36.298 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.283+0000 7f5b93fff700 1 --2- 172.21.15.1:0/3057548914 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x559678b68850 0x559678b6ac60 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b39e7b6284f5d768 server_cookie=9a7e39a3bffdff6c in_seq=0 out_seq=0 2023-11-16T16:41:36.298 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.283+0000 7f5b737fe700 10 client.25534 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:41:36.298 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.285+0000 7f5b737fe700 1 -- 172.21.15.1:0/3057548914 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559678a620a0 con 0x559678b68850 2023-11-16T16:41:36.298 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.285+0000 7f5b737fe700 10 client.25534 handle_command_reply: tid=0 2023-11-16T16:41:36.298 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.285+0000 7f5ba6805b80 1 client.25534 shutdown 2023-11-16T16:41:36.298 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.285+0000 7f5ba6805b80 1 -- 172.21.15.1:0/3057548914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559678b4f470 msgr2=0x559678b58270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:36.298 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.285+0000 7f5ba6805b80 1 --2- 172.21.15.1:0/3057548914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559678b4f470 0x559678b58270 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5b88009930 tx=0x7f5b880047c0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.298 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.286+0000 7f5ba6805b80 1 -- 172.21.15.1:0/3057548914 shutdown_connections 2023-11-16T16:41:36.298 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.286+0000 7f5ba6805b80 1 --2- 172.21.15.1:0/3057548914 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x559678b68850 0x559678b6ac60 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.299 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.286+0000 7f5ba6805b80 1 --2- 172.21.15.1:0/3057548914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559678b538b0 0x559678b588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.299 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.286+0000 7f5ba6805b80 1 --2- 172.21.15.1:0/3057548914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559678b4eb60 0x559678b5aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.299 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.286+0000 7f5b737fe700 0 client.25534 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:41:36.299 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.286+0000 7f5ba6805b80 1 --2- 172.21.15.1:0/3057548914 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559678b4f470 0x559678b58270 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.299 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.286+0000 7f5ba6805b80 1 -- 172.21.15.1:0/3057548914 >> 172.21.15.1:0/3057548914 conn(0x559678b44360 msgr2=0x559678b54d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:36.299 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.286+0000 7f5ba6805b80 1 -- 172.21.15.1:0/3057548914 shutdown_connections 2023-11-16T16:41:36.299 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.286+0000 7f5ba6805b80 1 -- 172.21.15.1:0/3057548914 wait complete. 2023-11-16T16:41:36.299 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.286+0000 7f5ba6805b80 20 client.25534 trim_cache size 0 max 16384 2023-11-16T16:41:36.308 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.304+0000 7f5ba6805b80 1 Processor -- start 2023-11-16T16:41:36.309 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.304+0000 7f5ba6805b80 1 -- start start 2023-11-16T16:41:36.309 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.305+0000 7f5ba6805b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559678b4eb60 0x559678b5f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:36.309 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.305+0000 7f5ba6805b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559678b4f470 0x559678b5ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:36.309 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.305+0000 7f5ba6805b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559678b538b0 0x559678b49560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:36.309 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.305+0000 7f5ba6805b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559678b43ee0 con 0x559678b4eb60 2023-11-16T16:41:36.309 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.305+0000 7f5ba6805b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x559678b567d0 con 0x559678b538b0 2023-11-16T16:41:36.309 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.305+0000 7f5ba6805b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559678b52440 con 0x559678b4f470 2023-11-16T16:41:36.309 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.305+0000 7f5b93fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559678b4eb60 0x559678b5f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:36.310 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.305+0000 7f5b93fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559678b4eb60 0x559678b5f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44812/0 (socket says 172.21.15.1:44812) 2023-11-16T16:41:36.310 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.305+0000 7f5b93fff700 1 -- 172.21.15.1:0/3820015420 learned_addr learned my addr 172.21.15.1:0/3820015420 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:41:36.310 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.305+0000 7f5b98c16700 1 --2- 172.21.15.1:0/3820015420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559678b538b0 0x559678b49560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:36.310 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.305+0000 7f5b937fe700 1 --2- 172.21.15.1:0/3820015420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559678b4f470 0x559678b5ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:36.310 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.305+0000 7f5b93fff700 1 -- 172.21.15.1:0/3820015420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559678b4f470 msgr2=0x559678b5ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:36.310 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.305+0000 7f5b93fff700 1 --2- 172.21.15.1:0/3820015420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559678b4f470 0x559678b5ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.310 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.305+0000 7f5b93fff700 1 -- 172.21.15.1:0/3820015420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559678b538b0 msgr2=0x559678b49560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:36.310 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.305+0000 7f5b93fff700 1 --2- 172.21.15.1:0/3820015420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559678b538b0 0x559678b49560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.310 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.305+0000 7f5b93fff700 1 -- 172.21.15.1:0/3820015420 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559678a65ee0 con 0x559678b4eb60 2023-11-16T16:41:36.311 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.305+0000 7f5b93fff700 1 --2- 172.21.15.1:0/3820015420 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559678b4eb60 0x559678b5f940 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f5b84010c10 tx=0x7f5b84010c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55356ca8780d0035 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:36.311 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.305+0000 7f5b7affd700 1 -- 172.21.15.1:0/3820015420 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8409dfb0 con 0x559678b4eb60 2023-11-16T16:41:36.311 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.306+0000 7f5b7affd700 1 -- 172.21.15.1:0/3820015420 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b840a23d0 con 0x559678b4eb60 2023-11-16T16:41:36.311 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.306+0000 7f5b7affd700 1 -- 172.21.15.1:0/3820015420 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b84099470 con 0x559678b4eb60 2023-11-16T16:41:36.311 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.306+0000 7f5ba6805b80 1 -- 172.21.15.1:0/3820015420 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559678b4eb60 msgr2=0x559678b5f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:36.311 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.306+0000 7f5ba6805b80 1 --2- 172.21.15.1:0/3820015420 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559678b4eb60 0x559678b5f940 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f5b84010c10 tx=0x7f5b84010c60 comp rx=0 tx=0).stop 2023-11-16T16:41:36.311 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.306+0000 7f5ba6805b80 1 -- 172.21.15.1:0/3820015420 shutdown_connections 2023-11-16T16:41:36.311 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.306+0000 7f5ba6805b80 1 --2- 172.21.15.1:0/3820015420 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559678b538b0 0x559678b49560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.311 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.306+0000 7f5ba6805b80 1 --2- 172.21.15.1:0/3820015420 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559678b4eb60 0x559678b5f940 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.306+0000 7f5ba6805b80 1 --2- 172.21.15.1:0/3820015420 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559678b4f470 0x559678b5ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.306+0000 7f5ba6805b80 1 -- 172.21.15.1:0/3820015420 >> 172.21.15.1:0/3820015420 conn(0x559678b44360 msgr2=0x559678b64b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:36.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.306+0000 7f5ba6805b80 1 -- 172.21.15.1:0/3820015420 shutdown_connections 2023-11-16T16:41:36.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.306+0000 7f5ba6805b80 1 -- 172.21.15.1:0/3820015420 wait complete. 2023-11-16T16:41:36.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.307+0000 7f5ba6805b80 1 -- start start 2023-11-16T16:41:36.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.307+0000 7f5ba6805b80 1 Processor -- start 2023-11-16T16:41:36.316 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.307+0000 7f5ba6805b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559678b4eb60 0x559678b5f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:36.316 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.307+0000 7f5ba6805b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559678b4f470 0x559678b5f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:36.316 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.307+0000 7f5ba6805b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559678b538b0 0x559678b5d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:36.316 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.307+0000 7f5ba6805b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559678b5df00 con 0x559678b4f470 2023-11-16T16:41:36.317 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.307+0000 7f5ba6805b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x559678b5e070 con 0x559678b4eb60 2023-11-16T16:41:36.317 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.307+0000 7f5ba6805b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x559678b5e230 con 0x559678b538b0 2023-11-16T16:41:36.317 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.307+0000 7f5ba6805b80 10 client.0 _ll_register_callbacks cb 0x559678a5f780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:41:36.317 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.307+0000 7f5b937fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559678b4f470 0x559678b5f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:36.317 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.307+0000 7f5b93fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559678b4eb60 0x559678b5f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:36.317 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.307+0000 7f5b937fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559678b4f470 0x559678b5f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44820/0 (socket says 172.21.15.1:44820) 2023-11-16T16:41:36.317 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.307+0000 7f5b937fe700 1 -- 172.21.15.1:0/4144338340 learned_addr learned my addr 172.21.15.1:0/4144338340 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:41:36.317 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.308+0000 7f5b98c16700 1 --2- 172.21.15.1:0/4144338340 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559678b538b0 0x559678b5d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:36.317 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.308+0000 7f5b93fff700 1 -- 172.21.15.1:0/4144338340 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559678b538b0 msgr2=0x559678b5d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:36.317 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.308+0000 7f5b93fff700 1 --2- 172.21.15.1:0/4144338340 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559678b538b0 0x559678b5d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.318 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.308+0000 7f5b93fff700 1 -- 172.21.15.1:0/4144338340 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559678b4f470 msgr2=0x559678b5f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:41:36.318 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.308+0000 7f5b93fff700 1 --2- 172.21.15.1:0/4144338340 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559678b4f470 0x559678b5f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.318 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.308+0000 7f5b93fff700 1 -- 172.21.15.1:0/4144338340 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b941103d0 con 0x559678b4eb60 2023-11-16T16:41:36.318 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.308+0000 7f5b937fe700 1 --2- 172.21.15.1:0/4144338340 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559678b4f470 0x559678b5f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:41:36.318 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.308+0000 7f5b98c16700 1 --2- 172.21.15.1:0/4144338340 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559678b538b0 0x559678b5d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:41:36.318 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.308+0000 7f5b93fff700 1 --2- 172.21.15.1:0/4144338340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559678b4eb60 0x559678b5f030 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5b840969f0 tx=0x7f5b84098130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e130ace2008d5874 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:41:36.318 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.308+0000 7f5b737fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-11-16T16:41:36.318 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.308+0000 7f5b737fe700 1 -- 172.21.15.1:0/4144338340 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b840105a0 con 0x559678b4eb60 2023-11-16T16:41:36.318 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.308+0000 7f5b737fe700 1 -- 172.21.15.1:0/4144338340 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f5b84010860 con 0x559678b4eb60 2023-11-16T16:41:36.319 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.308+0000 7f5ba6805b80 1 -- 172.21.15.1:0/4144338340 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b8c007cd0 con 0x559678b4eb60 2023-11-16T16:41:36.319 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.308+0000 7f5b737fe700 1 -- 172.21.15.1:0/4144338340 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b840ca2f0 con 0x559678b4eb60 2023-11-16T16:41:36.319 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.308+0000 7f5ba6805b80 10 client.25559 fetch_fsmap learned FSMap version 23 2023-11-16T16:41:36.319 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.308+0000 7f5ba6805b80 1 -- 172.21.15.1:0/4144338340 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5596788b6c60 con 0x559678b4eb60 2023-11-16T16:41:36.319 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.309+0000 7f5b737fe700 1 -- 172.21.15.1:0/4144338340 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b84063020 con 0x559678b4eb60 2023-11-16T16:41:36.319 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.309+0000 7f5b737fe700 1 -- 172.21.15.1:0/4144338340 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5b84094b90 con 0x559678b4eb60 2023-11-16T16:41:36.319 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.309+0000 7f5ba6805b80 10 client.25559 fetch_fsmap finished waiting for FSMap version 23 2023-11-16T16:41:36.319 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.309+0000 7f5ba6805b80 10 client.25559 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:41:36.319 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.309+0000 7f5ba6805b80 20 client.25559 populate_metadata read hostname 'smithi001' 2023-11-16T16:41:36.320 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.309+0000 7f5ba6805b80 1 --2- 172.21.15.1:0/4144338340 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x559678bbd3d0 0x559678bbf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:41:36.320 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.309+0000 7f5ba6805b80 4 client.25559 mds_command: new command op to 24521 tid=0[{"prefix": "scrub status"}] 2023-11-16T16:41:36.320 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.309+0000 7f5ba6805b80 1 -- 172.21.15.1:0/4144338340 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559678a6e660 con 0x559678bbd3d0 2023-11-16T16:41:36.320 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.309+0000 7f5b937fe700 1 --2- 172.21.15.1:0/4144338340 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x559678bbd3d0 0x559678bbf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:41:36.320 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.309+0000 7f5b737fe700 10 client.25559 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:41:36.320 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.309+0000 7f5b937fe700 1 --2- 172.21.15.1:0/4144338340 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x559678bbd3d0 0x559678bbf7b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8e1fd2c15e37d8d server_cookie=3f34454e0493f853 in_seq=0 out_seq=0 2023-11-16T16:41:36.320 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.312+0000 7f5b737fe700 1 -- 172.21.15.1:0/4144338340 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x559678a6e660 con 0x559678bbd3d0 2023-11-16T16:41:36.320 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.312+0000 7f5b737fe700 10 client.25559 handle_command_reply: tid=0 2023-11-16T16:41:36.320 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.312+0000 7f5ba6805b80 1 client.25559 shutdown 2023-11-16T16:41:36.321 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.312+0000 7f5ba6805b80 1 -- 172.21.15.1:0/4144338340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559678b4eb60 msgr2=0x559678b5f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:36.321 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.312+0000 7f5ba6805b80 1 --2- 172.21.15.1:0/4144338340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559678b4eb60 0x559678b5f030 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5b840969f0 tx=0x7f5b84098130 comp rx=0 tx=0).stop 2023-11-16T16:41:36.321 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.312+0000 7f5ba6805b80 1 -- 172.21.15.1:0/4144338340 shutdown_connections 2023-11-16T16:41:36.321 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.312+0000 7f5ba6805b80 1 --2- 172.21.15.1:0/4144338340 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x559678bbd3d0 0x559678bbf7b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.321 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.312+0000 7f5ba6805b80 1 --2- 172.21.15.1:0/4144338340 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x559678b538b0 0x559678b5d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.321 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.312+0000 7f5b737fe700 0 client.25559 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:41:36.321 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.312+0000 7f5ba6805b80 1 --2- 172.21.15.1:0/4144338340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559678b4eb60 0x559678b5f030 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.321 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.312+0000 7f5ba6805b80 1 --2- 172.21.15.1:0/4144338340 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559678b4f470 0x559678b5f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.321 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.312+0000 7f5ba6805b80 1 -- 172.21.15.1:0/4144338340 >> 172.21.15.1:0/4144338340 conn(0x559678b44360 msgr2=0x559678b52850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:36.321 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.312+0000 7f5ba6805b80 1 -- 172.21.15.1:0/4144338340 shutdown_connections 2023-11-16T16:41:36.322 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.312+0000 7f5ba6805b80 1 -- 172.21.15.1:0/4144338340 wait complete. 2023-11-16T16:41:36.322 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.312+0000 7f5ba6805b80 20 client.25559 trim_cache size 0 max 16384 2023-11-16T16:41:36.322 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:41:36.322 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "scrub active (563 inodes in the stack)", 2023-11-16T16:41:36.322 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2023-11-16T16:41:36.322 INFO:teuthology.orchestra.run.smithi001.stdout: "a0702666-30d2-4c77-88ef-bed563919cc5": { 2023-11-16T16:41:36.322 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2023-11-16T16:41:36.323 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "a0702666-30d2-4c77-88ef-bed563919cc5", 2023-11-16T16:41:36.323 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2023-11-16T16:41:36.323 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:41:36.323 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:41:36.323 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:41:36.324 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.313+0000 7f5b9a679700 1 -- 172.21.15.1:0/3234655103 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5b7c0a12c0 msgr2=0x7f5b7c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:36.324 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.313+0000 7f5b9a679700 1 --2- 172.21.15.1:0/3234655103 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5b7c0a12c0 0x7f5b7c0a3770 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5b94197b40 tx=0x7f5b88009310 comp rx=0 tx=0).stop 2023-11-16T16:41:36.324 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.313+0000 7f5b9a679700 1 -- 172.21.15.1:0/3234655103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b94067dc0 msgr2=0x7f5b94196530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:41:36.324 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.313+0000 7f5b9a679700 1 --2- 172.21.15.1:0/3234655103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b94067dc0 0x7f5b94196530 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5b8400e9f0 tx=0x7f5b8400ed00 comp rx=0 tx=0).stop 2023-11-16T16:41:36.324 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.313+0000 7f5b9a679700 1 -- 172.21.15.1:0/3234655103 shutdown_connections 2023-11-16T16:41:36.324 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.313+0000 7f5b9a679700 1 --2- 172.21.15.1:0/3234655103 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5b7c0a12c0 0x7f5b7c0a3770 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.325 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.313+0000 7f5b9a679700 1 --2- 172.21.15.1:0/3234655103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b94105250 0x7f5b9419ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.325 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.313+0000 7f5b9a679700 1 --2- 172.21.15.1:0/3234655103 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b94067dc0 0x7f5b94196530 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.325 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.313+0000 7f5b9a679700 1 --2- 172.21.15.1:0/3234655103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b94068770 0x7f5b94196a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:41:36.325 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.313+0000 7f5b9a679700 1 -- 172.21.15.1:0/3234655103 >> 172.21.15.1:0/3234655103 conn(0x7f5b940763a0 msgr2=0x7f5b940fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:41:36.325 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.313+0000 7f5b9a679700 1 -- 172.21.15.1:0/3234655103 shutdown_connections 2023-11-16T16:41:36.325 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:41:36.313+0000 7f5b9a679700 1 -- 172.21.15.1:0/3234655103 wait complete. 2023-11-16T16:41:36.328 INFO:tasks.cephfs.filesystem:scrub status for tag:a0702666-30d2-4c77-88ef-bed563919cc5 - {'path': '/', 'tag': 'a0702666-30d2-4c77-88ef-bed563919cc5', 'options': 'recursive,force'} 2023-11-16T16:41:36.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:36 smithi001 ceph-mon[98800]: pgmap v76: 97 pgs: 97 active+clean; 5.2 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 282 KiB/s wr, 20 op/s 2023-11-16T16:41:36.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:36 smithi001 ceph-mon[98800]: scrub queued for path: / 2023-11-16T16:41:36.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:36 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:41:36.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:36 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:41:37.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:36 smithi026 ceph-mon[100112]: pgmap v76: 97 pgs: 97 active+clean; 5.2 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 282 KiB/s wr, 20 op/s 2023-11-16T16:41:37.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:36 smithi026 ceph-mon[100112]: scrub queued for path: / 2023-11-16T16:41:37.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:36 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:41:37.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:36 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:41:37.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:36 smithi053 ceph-mon[100638]: pgmap v76: 97 pgs: 97 active+clean; 5.2 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 282 KiB/s wr, 20 op/s 2023-11-16T16:41:37.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:36 smithi053 ceph-mon[100638]: scrub queued for path: / 2023-11-16T16:41:37.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:36 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:41:37.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:36 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:41:38.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:38 smithi001 ceph-mon[98800]: pgmap v77: 97 pgs: 97 active+clean; 5.2 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 271 KiB/s wr, 17 op/s 2023-11-16T16:41:39.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:38 smithi026 ceph-mon[100112]: pgmap v77: 97 pgs: 97 active+clean; 5.2 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 271 KiB/s wr, 17 op/s 2023-11-16T16:41:39.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:38 smithi053 ceph-mon[100638]: pgmap v77: 97 pgs: 97 active+clean; 5.2 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 271 KiB/s wr, 17 op/s 2023-11-16T16:41:40.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:40 smithi001 ceph-mon[98800]: pgmap v78: 97 pgs: 97 active+clean; 5.2 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 271 KiB/s wr, 17 op/s 2023-11-16T16:41:40.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:40 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:41:41.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:40 smithi026 ceph-mon[100112]: pgmap v78: 97 pgs: 97 active+clean; 5.2 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 271 KiB/s wr, 17 op/s 2023-11-16T16:41:41.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:40 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:41:41.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:40 smithi053 ceph-mon[100638]: pgmap v78: 97 pgs: 97 active+clean; 5.2 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 271 KiB/s wr, 17 op/s 2023-11-16T16:41:41.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:40 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:41:42.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:42 smithi026 ceph-mon[100112]: pgmap v79: 97 pgs: 97 active+clean; 13 MiB data, 94 MiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 978 KiB/s wr, 24 op/s 2023-11-16T16:41:42.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:42 smithi053 ceph-mon[100638]: pgmap v79: 97 pgs: 97 active+clean; 13 MiB data, 94 MiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 978 KiB/s wr, 24 op/s 2023-11-16T16:41:42.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:42 smithi001 ceph-mon[98800]: pgmap v79: 97 pgs: 97 active+clean; 13 MiB data, 94 MiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 978 KiB/s wr, 24 op/s 2023-11-16T16:41:44.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:44 smithi001 ceph-mon[98800]: pgmap v80: 97 pgs: 97 active+clean; 13 MiB data, 94 MiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 968 KiB/s wr, 21 op/s 2023-11-16T16:41:44.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:44 smithi001 ceph-mon[98800]: scrub summary: idle 2023-11-16T16:41:45.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:44 smithi026 ceph-mon[100112]: pgmap v80: 97 pgs: 97 active+clean; 13 MiB data, 94 MiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 968 KiB/s wr, 21 op/s 2023-11-16T16:41:45.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:44 smithi026 ceph-mon[100112]: scrub summary: idle 2023-11-16T16:41:45.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:44 smithi053 ceph-mon[100638]: pgmap v80: 97 pgs: 97 active+clean; 13 MiB data, 94 MiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 968 KiB/s wr, 21 op/s 2023-11-16T16:41:45.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:44 smithi053 ceph-mon[100638]: scrub summary: idle 2023-11-16T16:41:46.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:46 smithi001 ceph-mon[98800]: pgmap v81: 97 pgs: 97 active+clean; 20 MiB data, 115 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 90 op/s 2023-11-16T16:41:47.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:46 smithi026 ceph-mon[100112]: pgmap v81: 97 pgs: 97 active+clean; 20 MiB data, 115 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 90 op/s 2023-11-16T16:41:47.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:46 smithi053 ceph-mon[100638]: pgmap v81: 97 pgs: 97 active+clean; 20 MiB data, 115 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 90 op/s 2023-11-16T16:41:48.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:48 smithi001 ceph-mon[98800]: pgmap v82: 97 pgs: 97 active+clean; 20 MiB data, 115 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 76 op/s 2023-11-16T16:41:49.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:48 smithi026 ceph-mon[100112]: pgmap v82: 97 pgs: 97 active+clean; 20 MiB data, 115 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 76 op/s 2023-11-16T16:41:49.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:48 smithi053 ceph-mon[100638]: pgmap v82: 97 pgs: 97 active+clean; 20 MiB data, 115 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 76 op/s 2023-11-16T16:41:50.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:50 smithi001 ceph-mon[98800]: pgmap v83: 97 pgs: 97 active+clean; 20 MiB data, 115 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 76 op/s 2023-11-16T16:41:50.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:50 smithi001 ceph-mon[98800]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:41:51.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:50 smithi026 ceph-mon[100112]: pgmap v83: 97 pgs: 97 active+clean; 20 MiB data, 115 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 76 op/s 2023-11-16T16:41:51.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:50 smithi026 ceph-mon[100112]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:41:51.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:50 smithi053 ceph-mon[100638]: pgmap v83: 97 pgs: 97 active+clean; 20 MiB data, 115 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 76 op/s 2023-11-16T16:41:51.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:50 smithi053 ceph-mon[100638]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:41:52.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:52 smithi026 ceph-mon[100112]: pgmap v84: 97 pgs: 97 active+clean; 28 MiB data, 121 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 84 op/s 2023-11-16T16:41:52.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:52 smithi053 ceph-mon[100638]: pgmap v84: 97 pgs: 97 active+clean; 28 MiB data, 121 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 84 op/s 2023-11-16T16:41:52.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:52 smithi001 ceph-mon[98800]: pgmap v84: 97 pgs: 97 active+clean; 28 MiB data, 121 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 84 op/s 2023-11-16T16:41:54.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:54 smithi001 ceph-mon[98800]: pgmap v85: 97 pgs: 97 active+clean; 28 MiB data, 121 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 77 op/s 2023-11-16T16:41:55.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:54 smithi026 ceph-mon[100112]: pgmap v85: 97 pgs: 97 active+clean; 28 MiB data, 121 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 77 op/s 2023-11-16T16:41:55.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:54 smithi053 ceph-mon[100638]: pgmap v85: 97 pgs: 97 active+clean; 28 MiB data, 121 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 77 op/s 2023-11-16T16:41:56.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:56 smithi001 ceph-mon[98800]: pgmap v86: 97 pgs: 97 active+clean; 35 MiB data, 122 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 81 op/s 2023-11-16T16:41:57.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:56 smithi026 ceph-mon[100112]: pgmap v86: 97 pgs: 97 active+clean; 35 MiB data, 122 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 81 op/s 2023-11-16T16:41:57.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:56 smithi053 ceph-mon[100638]: pgmap v86: 97 pgs: 97 active+clean; 35 MiB data, 122 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 81 op/s 2023-11-16T16:41:58.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:41:58 smithi001 ceph-mon[98800]: pgmap v87: 97 pgs: 97 active+clean; 35 MiB data, 122 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.2 MiB/s wr, 12 op/s 2023-11-16T16:41:59.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:41:58 smithi026 ceph-mon[100112]: pgmap v87: 97 pgs: 97 active+clean; 35 MiB data, 122 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.2 MiB/s wr, 12 op/s 2023-11-16T16:41:59.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:41:58 smithi053 ceph-mon[100638]: pgmap v87: 97 pgs: 97 active+clean; 35 MiB data, 122 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.2 MiB/s wr, 12 op/s 2023-11-16T16:42:00.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:00 smithi001 ceph-mon[98800]: pgmap v88: 97 pgs: 97 active+clean; 35 MiB data, 122 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.2 MiB/s wr, 12 op/s 2023-11-16T16:42:01.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:00 smithi026 ceph-mon[100112]: pgmap v88: 97 pgs: 97 active+clean; 35 MiB data, 122 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.2 MiB/s wr, 12 op/s 2023-11-16T16:42:01.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:00 smithi053 ceph-mon[100638]: pgmap v88: 97 pgs: 97 active+clean; 35 MiB data, 122 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.2 MiB/s wr, 12 op/s 2023-11-16T16:42:02.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:02 smithi026 ceph-mon[100112]: pgmap v89: 97 pgs: 97 active+clean; 43 MiB data, 134 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 22 op/s 2023-11-16T16:42:02.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:02 smithi053 ceph-mon[100638]: pgmap v89: 97 pgs: 97 active+clean; 43 MiB data, 134 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 22 op/s 2023-11-16T16:42:02.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:02 smithi001 ceph-mon[98800]: pgmap v89: 97 pgs: 97 active+clean; 43 MiB data, 134 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 22 op/s 2023-11-16T16:42:04.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:04 smithi026 ceph-mon[100112]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:42:04.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:04 smithi026 ceph-mon[100112]: pgmap v90: 97 pgs: 97 active+clean; 43 MiB data, 134 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 14 op/s 2023-11-16T16:42:04.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:04 smithi053 ceph-mon[100638]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:42:04.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:04 smithi053 ceph-mon[100638]: pgmap v90: 97 pgs: 97 active+clean; 43 MiB data, 134 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 14 op/s 2023-11-16T16:42:04.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:04 smithi001 ceph-mon[98800]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:42:04.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:04 smithi001 ceph-mon[98800]: pgmap v90: 97 pgs: 97 active+clean; 43 MiB data, 134 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 14 op/s 2023-11-16T16:42:06.329 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-16T16:42:06.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.471+0000 7f29e3b8a700 1 -- 172.21.15.1:0/2024100138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29dc105250 msgr2=0x7f29dc105620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:06.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.471+0000 7f29e3b8a700 1 --2- 172.21.15.1:0/2024100138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29dc105250 0x7f29dc105620 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f29cc0099c0 tx=0x7f29cc009cd0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.477 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.471+0000 7f29e3b8a700 1 -- 172.21.15.1:0/2024100138 shutdown_connections 2023-11-16T16:42:06.477 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.471+0000 7f29e3b8a700 1 --2- 172.21.15.1:0/2024100138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f29dc068770 0x7f29dc10d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.477 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.471+0000 7f29e3b8a700 1 --2- 172.21.15.1:0/2024100138 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29dc105250 0x7f29dc105620 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.477 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.471+0000 7f29e3b8a700 1 --2- 172.21.15.1:0/2024100138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29dc067dc0 0x7f29dc068230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.477 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.471+0000 7f29e3b8a700 1 -- 172.21.15.1:0/2024100138 >> 172.21.15.1:0/2024100138 conn(0x7f29dc0763a0 msgr2=0x7f29dc0767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:06.478 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.471+0000 7f29e3b8a700 1 -- 172.21.15.1:0/2024100138 shutdown_connections 2023-11-16T16:42:06.478 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.471+0000 7f29e3b8a700 1 -- 172.21.15.1:0/2024100138 wait complete. 2023-11-16T16:42:06.478 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.472+0000 7f29e3b8a700 1 Processor -- start 2023-11-16T16:42:06.478 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.472+0000 7f29e3b8a700 1 -- start start 2023-11-16T16:42:06.478 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.472+0000 7f29e3b8a700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f29dc067dc0 0x7f29dc196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:06.478 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.472+0000 7f29e3b8a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29dc068770 0x7f29dc196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:06.478 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.472+0000 7f29e3b8a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29dc105250 0x7f29dc19acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:06.478 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.472+0000 7f29e3b8a700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc19b2f0 con 0x7f29dc067dc0 2023-11-16T16:42:06.479 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.472+0000 7f29e3b8a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc19b460 con 0x7f29dc105250 2023-11-16T16:42:06.479 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.472+0000 7f29e3b8a700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc19b620 con 0x7f29dc068770 2023-11-16T16:42:06.479 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.472+0000 7f29e1926700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f29dc067dc0 0x7f29dc196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:06.479 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.472+0000 7f29e1926700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f29dc067dc0 0x7f29dc196310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44832/0 (socket says 172.21.15.1:44832) 2023-11-16T16:42:06.479 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.472+0000 7f29e1926700 1 -- 172.21.15.1:0/847975337 learned_addr learned my addr 172.21.15.1:0/847975337 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:42:06.479 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.473+0000 7f29e1125700 1 --2- 172.21.15.1:0/847975337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29dc068770 0x7f29dc196850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:06.479 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.473+0000 7f29e2127700 1 --2- 172.21.15.1:0/847975337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29dc105250 0x7f29dc19acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:06.480 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.473+0000 7f29e1926700 1 -- 172.21.15.1:0/847975337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29dc068770 msgr2=0x7f29dc196850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:06.480 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.473+0000 7f29e1926700 1 --2- 172.21.15.1:0/847975337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29dc068770 0x7f29dc196850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.480 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.473+0000 7f29e1926700 1 -- 172.21.15.1:0/847975337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29dc105250 msgr2=0x7f29dc19acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:06.480 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.473+0000 7f29e1926700 1 --2- 172.21.15.1:0/847975337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29dc105250 0x7f29dc19acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.480 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.473+0000 7f29e1926700 1 -- 172.21.15.1:0/847975337 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d4009c40 con 0x7f29dc067dc0 2023-11-16T16:42:06.480 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.473+0000 7f29e1926700 1 --2- 172.21.15.1:0/847975337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f29dc067dc0 0x7f29dc196310 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f29cc000c00 tx=0x7f29cc00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:06.480 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.473+0000 7f29d2ffd700 1 -- 172.21.15.1:0/847975337 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc01d070 con 0x7f29dc067dc0 2023-11-16T16:42:06.480 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.473+0000 7f29e3b8a700 1 -- 172.21.15.1:0/847975337 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29cc0095f0 con 0x7f29dc067dc0 2023-11-16T16:42:06.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.473+0000 7f29d2ffd700 1 -- 172.21.15.1:0/847975337 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29cc005050 con 0x7f29dc067dc0 2023-11-16T16:42:06.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.473+0000 7f29d2ffd700 1 -- 172.21.15.1:0/847975337 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc004ab0 con 0x7f29dc067dc0 2023-11-16T16:42:06.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.473+0000 7f29e3b8a700 1 -- 172.21.15.1:0/847975337 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29dc1102c0 con 0x7f29dc067dc0 2023-11-16T16:42:06.490 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.475+0000 7f29d2ffd700 1 -- 172.21.15.1:0/847975337 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f29cc0051c0 con 0x7f29dc067dc0 2023-11-16T16:42:06.490 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.476+0000 7f29d2ffd700 1 --2- 172.21.15.1:0/847975337 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f29c80a12c0 0x7f29c80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:06.490 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.476+0000 7f29d2ffd700 1 -- 172.21.15.1:0/847975337 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f29cc0cd9d0 con 0x7f29dc067dc0 2023-11-16T16:42:06.490 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.476+0000 7f29e1125700 1 --2- 172.21.15.1:0/847975337 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f29c80a12c0 0x7f29c80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:06.490 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.477+0000 7f29e1125700 1 --2- 172.21.15.1:0/847975337 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f29c80a12c0 0x7f29c80a3770 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f29dc197920 tx=0x7f29d8009260 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:06.490 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.477+0000 7f29efd16b80 1 Processor -- start 2023-11-16T16:42:06.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.477+0000 7f29efd16b80 1 -- start start 2023-11-16T16:42:06.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.477+0000 7f29efd16b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f0bc9b60 0x5605f0bc9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:06.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.477+0000 7f29efd16b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f0bca470 0x5605f0bce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:06.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.477+0000 7f29efd16b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f0bce8b0 0x5605f0bced00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:06.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.477+0000 7f29efd16b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f0acf060 con 0x5605f0bce8b0 2023-11-16T16:42:06.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.477+0000 7f29efd16b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f0b218a0 con 0x5605f0bca470 2023-11-16T16:42:06.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.477+0000 7f29efd16b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f0b1ad30 con 0x5605f0bc9b60 2023-11-16T16:42:06.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.478+0000 7f29e1926700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f0bc9b60 0x5605f0bc9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:06.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.478+0000 7f29e2127700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f0bce8b0 0x5605f0bced00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:06.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.478+0000 7f29e1926700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f0bc9b60 0x5605f0bc9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:41522/0 (socket says 172.21.15.1:41522) 2023-11-16T16:42:06.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.478+0000 7f29e1926700 1 -- 172.21.15.1:0/2492665749 learned_addr learned my addr 172.21.15.1:0/2492665749 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:42:06.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.478+0000 7f29e1125700 1 --2- 172.21.15.1:0/2492665749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f0bca470 0x5605f0bce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:06.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.478+0000 7f29e1926700 1 -- 172.21.15.1:0/2492665749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f0bca470 msgr2=0x5605f0bce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:06.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.478+0000 7f29e1926700 1 --2- 172.21.15.1:0/2492665749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f0bca470 0x5605f0bce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.478+0000 7f29e1926700 1 -- 172.21.15.1:0/2492665749 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f0bce8b0 msgr2=0x5605f0bced00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:06.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.478+0000 7f29e1926700 1 --2- 172.21.15.1:0/2492665749 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f0bce8b0 0x5605f0bced00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.478+0000 7f29e1926700 1 -- 172.21.15.1:0/2492665749 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605f099c630 con 0x5605f0bc9b60 2023-11-16T16:42:06.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.478+0000 7f29e2127700 1 --2- 172.21.15.1:0/2492665749 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f0bce8b0 0x5605f0bced00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:42:06.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.478+0000 7f29e1926700 1 --2- 172.21.15.1:0/2492665749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f0bc9b60 0x5605f0bc9f30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f29cc0cb190 tx=0x7f29cc09a590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dba238d17ffb12d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:06.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.478+0000 7f29c7fff700 1 -- 172.21.15.1:0/2492665749 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc0a2f00 con 0x5605f0bc9b60 2023-11-16T16:42:06.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.478+0000 7f29c7fff700 1 -- 172.21.15.1:0/2492665749 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29cc0054a0 con 0x5605f0bc9b60 2023-11-16T16:42:06.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.478+0000 7f29c7fff700 1 -- 172.21.15.1:0/2492665749 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc09b070 con 0x5605f0bc9b60 2023-11-16T16:42:06.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.479+0000 7f29efd16b80 1 -- 172.21.15.1:0/2492665749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f0bc9b60 msgr2=0x5605f0bc9f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:06.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.479+0000 7f29efd16b80 1 --2- 172.21.15.1:0/2492665749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f0bc9b60 0x5605f0bc9f30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f29cc0cb190 tx=0x7f29cc09a590 comp rx=0 tx=0).stop 2023-11-16T16:42:06.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.479+0000 7f29efd16b80 1 -- 172.21.15.1:0/2492665749 shutdown_connections 2023-11-16T16:42:06.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.479+0000 7f29efd16b80 1 --2- 172.21.15.1:0/2492665749 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f0bce8b0 0x5605f0bced00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.479+0000 7f29efd16b80 1 --2- 172.21.15.1:0/2492665749 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f0bc9b60 0x5605f0bc9f30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.479+0000 7f29efd16b80 1 --2- 172.21.15.1:0/2492665749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f0bca470 0x5605f0bce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.479+0000 7f29efd16b80 1 -- 172.21.15.1:0/2492665749 >> 172.21.15.1:0/2492665749 conn(0x5605f0bbf360 msgr2=0x5605f0bc4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:06.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.479+0000 7f29efd16b80 1 -- 172.21.15.1:0/2492665749 shutdown_connections 2023-11-16T16:42:06.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.479+0000 7f29efd16b80 1 -- 172.21.15.1:0/2492665749 wait complete. 2023-11-16T16:42:06.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.479+0000 7f29efd16b80 1 -- start start 2023-11-16T16:42:06.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.479+0000 7f29efd16b80 1 Processor -- start 2023-11-16T16:42:06.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.480+0000 7f29efd16b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f0bc9b60 0x5605f0bd5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:06.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.480+0000 7f29efd16b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f0bca470 0x5605f0bd31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:06.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.480+0000 7f29efd16b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f0bce8b0 0x5605f0bd37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:06.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.480+0000 7f29efd16b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f0abc8e0 con 0x5605f0bca470 2023-11-16T16:42:06.495 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.480+0000 7f29efd16b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f0abc580 con 0x5605f0bce8b0 2023-11-16T16:42:06.495 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.480+0000 7f29efd16b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f099e920 con 0x5605f0bc9b60 2023-11-16T16:42:06.495 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.480+0000 7f29efd16b80 10 client.0 _ll_register_callbacks cb 0x5605f0ada780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:42:06.495 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.480+0000 7f29e1125700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f0bca470 0x5605f0bd31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:06.495 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.480+0000 7f29e1125700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f0bca470 0x5605f0bd31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44848/0 (socket says 172.21.15.1:44848) 2023-11-16T16:42:06.495 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.480+0000 7f29e1125700 1 -- 172.21.15.1:0/3906690048 learned_addr learned my addr 172.21.15.1:0/3906690048 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:42:06.495 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.480+0000 7f29e1926700 1 --2- 172.21.15.1:0/3906690048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f0bc9b60 0x5605f0bd5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:06.495 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.480+0000 7f29e2127700 1 --2- 172.21.15.1:0/3906690048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f0bce8b0 0x5605f0bd37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:06.495 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.480+0000 7f29e1125700 1 -- 172.21.15.1:0/3906690048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f0bc9b60 msgr2=0x5605f0bd5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:06.496 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.480+0000 7f29e1125700 1 --2- 172.21.15.1:0/3906690048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f0bc9b60 0x5605f0bd5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.496 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.480+0000 7f29e1125700 1 -- 172.21.15.1:0/3906690048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f0bce8b0 msgr2=0x5605f0bd37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:06.496 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.480+0000 7f29e1125700 1 --2- 172.21.15.1:0/3906690048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f0bce8b0 0x5605f0bd37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.496 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.480+0000 7f29e1125700 1 -- 172.21.15.1:0/3906690048 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29dc1102c0 con 0x5605f0bca470 2023-11-16T16:42:06.496 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.481+0000 7f29e1125700 1 --2- 172.21.15.1:0/3906690048 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f0bca470 0x5605f0bd31e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f29d80045d0 tx=0x7f29d80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d0a7400be6477eb server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:06.496 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.481+0000 7f29c57fa700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:42:06.496 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.481+0000 7f29c57fa700 1 -- 172.21.15.1:0/3906690048 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d8022070 con 0x5605f0bca470 2023-11-16T16:42:06.496 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.481+0000 7f29c57fa700 1 -- 172.21.15.1:0/3906690048 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f29d8003e20 con 0x5605f0bca470 2023-11-16T16:42:06.496 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.481+0000 7f29efd16b80 1 -- 172.21.15.1:0/3906690048 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5605f0bbcb20 con 0x5605f0bca470 2023-11-16T16:42:06.496 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.481+0000 7f29c57fa700 1 -- 172.21.15.1:0/3906690048 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29d801d620 con 0x5605f0bca470 2023-11-16T16:42:06.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.481+0000 7f29efd16b80 10 client.15987 fetch_fsmap learned FSMap version 25 2023-11-16T16:42:06.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.481+0000 7f29efd16b80 1 -- 172.21.15.1:0/3906690048 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5605f0ae1190 con 0x5605f0bca470 2023-11-16T16:42:06.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.481+0000 7f29c57fa700 1 -- 172.21.15.1:0/3906690048 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29d8033080 con 0x5605f0bca470 2023-11-16T16:42:06.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.481+0000 7f29c57fa700 1 -- 172.21.15.1:0/3906690048 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f29d8027780 con 0x5605f0bca470 2023-11-16T16:42:06.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.481+0000 7f29efd16b80 10 client.15987 fetch_fsmap finished waiting for FSMap version 25 2023-11-16T16:42:06.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.481+0000 7f29efd16b80 10 client.15987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:42:06.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.481+0000 7f29efd16b80 20 client.15987 populate_metadata read hostname 'smithi001' 2023-11-16T16:42:06.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.481+0000 7f29efd16b80 1 --2- 172.21.15.1:0/3906690048 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5605f0be3640 0x5605f0be5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:06.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.481+0000 7f29efd16b80 4 client.15987 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:42:06.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.481+0000 7f29efd16b80 1 -- 172.21.15.1:0/3906690048 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5605f0add0a0 con 0x5605f0be3640 2023-11-16T16:42:06.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.482+0000 7f29e1926700 1 --2- 172.21.15.1:0/3906690048 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5605f0be3640 0x5605f0be5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:06.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.482+0000 7f29c57fa700 10 client.15987 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:42:06.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.482+0000 7f29e1926700 1 --2- 172.21.15.1:0/3906690048 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5605f0be3640 0x5605f0be5a50 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=764150021ad3e81f server_cookie=3d76907270311af1 in_seq=0 out_seq=0 2023-11-16T16:42:06.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.483+0000 7f29c57fa700 1 -- 172.21.15.1:0/3906690048 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5605f0add0a0 con 0x5605f0be3640 2023-11-16T16:42:06.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.483+0000 7f29c57fa700 10 client.15987 handle_command_reply: tid=0 2023-11-16T16:42:06.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.483+0000 7f29efd16b80 1 client.15987 shutdown 2023-11-16T16:42:06.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.483+0000 7f29efd16b80 1 -- 172.21.15.1:0/3906690048 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f0bca470 msgr2=0x5605f0bd31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:06.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.483+0000 7f29efd16b80 1 --2- 172.21.15.1:0/3906690048 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f0bca470 0x5605f0bd31e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f29d80045d0 tx=0x7f29d80047c0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.483+0000 7f29efd16b80 1 -- 172.21.15.1:0/3906690048 shutdown_connections 2023-11-16T16:42:06.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.483+0000 7f29efd16b80 1 --2- 172.21.15.1:0/3906690048 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5605f0be3640 0x5605f0be5a50 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.483+0000 7f29efd16b80 1 --2- 172.21.15.1:0/3906690048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f0bce8b0 0x5605f0bd37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.483+0000 7f29efd16b80 1 --2- 172.21.15.1:0/3906690048 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f0bc9b60 0x5605f0bd5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.483+0000 7f29efd16b80 1 --2- 172.21.15.1:0/3906690048 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f0bca470 0x5605f0bd31e0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.483+0000 7f29c57fa700 0 client.15987 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:42:06.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.483+0000 7f29efd16b80 1 -- 172.21.15.1:0/3906690048 >> 172.21.15.1:0/3906690048 conn(0x5605f0bbf360 msgr2=0x5605f0bcfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:06.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.483+0000 7f29efd16b80 1 -- 172.21.15.1:0/3906690048 shutdown_connections 2023-11-16T16:42:06.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.483+0000 7f29efd16b80 1 -- 172.21.15.1:0/3906690048 wait complete. 2023-11-16T16:42:06.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.483+0000 7f29efd16b80 20 client.15987 trim_cache size 0 max 16384 2023-11-16T16:42:06.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.501+0000 7f29efd16b80 1 Processor -- start 2023-11-16T16:42:06.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.501+0000 7f29efd16b80 1 -- start start 2023-11-16T16:42:06.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.501+0000 7f29efd16b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f0bc9b60 0x5605f0bd5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:06.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.501+0000 7f29efd16b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f0bca470 0x5605f0bcc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:06.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.501+0000 7f29efd16b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f0bce8b0 0x5605f0bc4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:06.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.501+0000 7f29efd16b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f0add0a0 con 0x5605f0bca470 2023-11-16T16:42:06.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.501+0000 7f29efd16b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f0bdffc0 con 0x5605f0bc9b60 2023-11-16T16:42:06.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.501+0000 7f29efd16b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f0bc64a0 con 0x5605f0bce8b0 2023-11-16T16:42:06.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.501+0000 7f29e1125700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f0bca470 0x5605f0bcc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:06.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.501+0000 7f29e1125700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f0bca470 0x5605f0bcc8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44856/0 (socket says 172.21.15.1:44856) 2023-11-16T16:42:06.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.501+0000 7f29e1125700 1 -- 172.21.15.1:0/3088371741 learned_addr learned my addr 172.21.15.1:0/3088371741 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:42:06.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.501+0000 7f29e2127700 1 --2- 172.21.15.1:0/3088371741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f0bce8b0 0x5605f0bc4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:06.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.501+0000 7f29e1926700 1 --2- 172.21.15.1:0/3088371741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f0bc9b60 0x5605f0bd5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:06.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.501+0000 7f29e1125700 1 -- 172.21.15.1:0/3088371741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f0bce8b0 msgr2=0x5605f0bc4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:06.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.501+0000 7f29e1125700 1 --2- 172.21.15.1:0/3088371741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f0bce8b0 0x5605f0bc4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.501+0000 7f29e1125700 1 -- 172.21.15.1:0/3088371741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f0bc9b60 msgr2=0x5605f0bd5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:06.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.501+0000 7f29e1125700 1 --2- 172.21.15.1:0/3088371741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f0bc9b60 0x5605f0bd5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.501+0000 7f29e1125700 1 -- 172.21.15.1:0/3088371741 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605f0bcf1a0 con 0x5605f0bca470 2023-11-16T16:42:06.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.501+0000 7f29e1125700 1 --2- 172.21.15.1:0/3088371741 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f0bca470 0x5605f0bcc8f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f29d8016040 tx=0x7f29d8011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef742ac6a97bb8a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:06.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.502+0000 7f29d0dac700 1 -- 172.21.15.1:0/3088371741 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d8022070 con 0x5605f0bca470 2023-11-16T16:42:06.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.502+0000 7f29d0dac700 1 -- 172.21.15.1:0/3088371741 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5605f0bbcb20 con 0x5605f0bca470 2023-11-16T16:42:06.513 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.502+0000 7f29d0dac700 1 -- 172.21.15.1:0/3088371741 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d801d930 con 0x5605f0bca470 2023-11-16T16:42:06.513 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.502+0000 7f29efd16b80 1 -- 172.21.15.1:0/3088371741 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f0bca470 msgr2=0x5605f0bcc8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:06.513 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.502+0000 7f29efd16b80 1 --2- 172.21.15.1:0/3088371741 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f0bca470 0x5605f0bcc8f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f29d8016040 tx=0x7f29d8011ac0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.513 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.502+0000 7f29efd16b80 1 -- 172.21.15.1:0/3088371741 shutdown_connections 2023-11-16T16:42:06.513 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.502+0000 7f29efd16b80 1 --2- 172.21.15.1:0/3088371741 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f0bce8b0 0x5605f0bc4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.513 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.502+0000 7f29efd16b80 1 --2- 172.21.15.1:0/3088371741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f0bc9b60 0x5605f0bd5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.513 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.502+0000 7f29efd16b80 1 --2- 172.21.15.1:0/3088371741 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f0bca470 0x5605f0bcc8f0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.513 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.502+0000 7f29efd16b80 1 -- 172.21.15.1:0/3088371741 >> 172.21.15.1:0/3088371741 conn(0x5605f0bbf360 msgr2=0x5605f0bdf8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:06.514 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.502+0000 7f29efd16b80 1 -- 172.21.15.1:0/3088371741 shutdown_connections 2023-11-16T16:42:06.514 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.502+0000 7f29efd16b80 1 -- 172.21.15.1:0/3088371741 wait complete. 2023-11-16T16:42:06.514 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.503+0000 7f29efd16b80 1 -- start start 2023-11-16T16:42:06.514 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.503+0000 7f29efd16b80 1 Processor -- start 2023-11-16T16:42:06.514 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.503+0000 7f29efd16b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f0bc9b60 0x5605f0c34760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:06.514 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.503+0000 7f29efd16b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f0bca470 0x5605f0c34ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:06.514 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.503+0000 7f29efd16b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f0bce8b0 0x5605f0c334f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:06.514 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.503+0000 7f29efd16b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f0c33a60 con 0x5605f0bce8b0 2023-11-16T16:42:06.514 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.503+0000 7f29efd16b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f0c33bd0 con 0x5605f0bca470 2023-11-16T16:42:06.514 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.503+0000 7f29efd16b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605f0c33d90 con 0x5605f0bc9b60 2023-11-16T16:42:06.515 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.503+0000 7f29efd16b80 10 client.0 _ll_register_callbacks cb 0x7f29d80043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:42:06.515 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.504+0000 7f29e2127700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f0bce8b0 0x5605f0c334f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:06.515 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.504+0000 7f29e1926700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f0bc9b60 0x5605f0c34760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:06.515 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.504+0000 7f29e2127700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f0bce8b0 0x5605f0c334f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44864/0 (socket says 172.21.15.1:44864) 2023-11-16T16:42:06.515 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.504+0000 7f29e2127700 1 -- 172.21.15.1:0/3994936974 learned_addr learned my addr 172.21.15.1:0/3994936974 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:42:06.515 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.504+0000 7f29e1125700 1 --2- 172.21.15.1:0/3994936974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f0bca470 0x5605f0c34ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:06.515 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.504+0000 7f29e1926700 1 -- 172.21.15.1:0/3994936974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f0bca470 msgr2=0x5605f0c34ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:06.515 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.504+0000 7f29e1926700 1 --2- 172.21.15.1:0/3994936974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f0bca470 0x5605f0c34ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.516 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.504+0000 7f29e1926700 1 -- 172.21.15.1:0/3994936974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f0bce8b0 msgr2=0x5605f0c334f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:06.516 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.504+0000 7f29e1926700 1 --2- 172.21.15.1:0/3994936974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f0bce8b0 0x5605f0c334f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.516 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.504+0000 7f29e1926700 1 -- 172.21.15.1:0/3994936974 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5605f0ae1190 con 0x5605f0bc9b60 2023-11-16T16:42:06.516 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.504+0000 7f29e2127700 1 --2- 172.21.15.1:0/3994936974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f0bce8b0 0x5605f0c334f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:42:06.516 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.504+0000 7f29e1926700 1 --2- 172.21.15.1:0/3994936974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f0bc9b60 0x5605f0c34760 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f29cc09a840 tx=0x7f29cc009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7eb88c8d1178103 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:06.516 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.504+0000 7f29c57fa700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-16T16:42:06.516 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.504+0000 7f29c57fa700 1 -- 172.21.15.1:0/3994936974 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc09a4a0 con 0x5605f0bc9b60 2023-11-16T16:42:06.516 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.504+0000 7f29efd16b80 1 -- 172.21.15.1:0/3994936974 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29cc0054a0 con 0x5605f0bc9b60 2023-11-16T16:42:06.516 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.504+0000 7f29c57fa700 1 -- 172.21.15.1:0/3994936974 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f29cc0b3270 con 0x5605f0bc9b60 2023-11-16T16:42:06.516 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.504+0000 7f29c57fa700 1 -- 172.21.15.1:0/3994936974 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29cc004c10 con 0x5605f0bc9b60 2023-11-16T16:42:06.517 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.504+0000 7f29efd16b80 10 client.25558 fetch_fsmap learned FSMap version 25 2023-11-16T16:42:06.517 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.504+0000 7f29efd16b80 1 -- 172.21.15.1:0/3994936974 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5605f0948de0 con 0x5605f0bc9b60 2023-11-16T16:42:06.517 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.505+0000 7f29c57fa700 1 -- 172.21.15.1:0/3994936974 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29cc0054a0 con 0x5605f0bc9b60 2023-11-16T16:42:06.517 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.505+0000 7f29c57fa700 1 -- 172.21.15.1:0/3994936974 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 25) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f29cc09aea0 con 0x5605f0bc9b60 2023-11-16T16:42:06.517 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.505+0000 7f29efd16b80 10 client.25558 fetch_fsmap finished waiting for FSMap version 25 2023-11-16T16:42:06.517 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.505+0000 7f29efd16b80 10 client.25558 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:42:06.517 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.505+0000 7f29efd16b80 20 client.25558 populate_metadata read hostname 'smithi001' 2023-11-16T16:42:06.517 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.505+0000 7f29efd16b80 1 --2- 172.21.15.1:0/3994936974 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5605f0c3c3a0 0x5605f0c3e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:06.517 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.505+0000 7f29efd16b80 4 client.25558 mds_command: new command op to 24521 tid=0[{"prefix": "scrub status"}] 2023-11-16T16:42:06.518 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.505+0000 7f29efd16b80 1 -- 172.21.15.1:0/3994936974 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5605f0ae9660 con 0x5605f0c3c3a0 2023-11-16T16:42:06.518 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.505+0000 7f29e1125700 1 --2- 172.21.15.1:0/3994936974 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5605f0c3c3a0 0x5605f0c3e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:06.518 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.505+0000 7f29c57fa700 10 client.25558 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:42:06.518 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.505+0000 7f29e1125700 1 --2- 172.21.15.1:0/3994936974 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5605f0c3c3a0 0x5605f0c3e7b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92fc1270fa4f89fa server_cookie=6c5247626a51c154 in_seq=0 out_seq=0 2023-11-16T16:42:06.518 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.506+0000 7f29c57fa700 1 -- 172.21.15.1:0/3994936974 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5605f0ae9660 con 0x5605f0c3c3a0 2023-11-16T16:42:06.518 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.506+0000 7f29c57fa700 10 client.25558 handle_command_reply: tid=0 2023-11-16T16:42:06.518 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.506+0000 7f29efd16b80 1 client.25558 shutdown 2023-11-16T16:42:06.518 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.506+0000 7f29efd16b80 1 -- 172.21.15.1:0/3994936974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f0bc9b60 msgr2=0x5605f0c34760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:06.518 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.506+0000 7f29efd16b80 1 --2- 172.21.15.1:0/3994936974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f0bc9b60 0x5605f0c34760 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f29cc09a840 tx=0x7f29cc009310 comp rx=0 tx=0).stop 2023-11-16T16:42:06.519 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.506+0000 7f29efd16b80 1 -- 172.21.15.1:0/3994936974 shutdown_connections 2023-11-16T16:42:06.519 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.506+0000 7f29efd16b80 1 --2- 172.21.15.1:0/3994936974 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5605f0c3c3a0 0x5605f0c3e7b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.519 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.506+0000 7f29efd16b80 1 --2- 172.21.15.1:0/3994936974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5605f0bce8b0 0x5605f0c334f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.519 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.506+0000 7f29efd16b80 1 --2- 172.21.15.1:0/3994936974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5605f0bc9b60 0x5605f0c34760 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.519 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.506+0000 7f29efd16b80 1 --2- 172.21.15.1:0/3994936974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5605f0bca470 0x5605f0c34ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.519 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.506+0000 7f29c57fa700 0 client.25558 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:42:06.519 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.506+0000 7f29efd16b80 1 -- 172.21.15.1:0/3994936974 >> 172.21.15.1:0/3994936974 conn(0x5605f0bbf360 msgr2=0x5605f0bcc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:06.519 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.506+0000 7f29efd16b80 1 -- 172.21.15.1:0/3994936974 shutdown_connections 2023-11-16T16:42:06.519 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.506+0000 7f29efd16b80 1 -- 172.21.15.1:0/3994936974 wait complete. 2023-11-16T16:42:06.520 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.506+0000 7f29efd16b80 20 client.25558 trim_cache size 0 max 16384 2023-11-16T16:42:06.520 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:42:06.520 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2023-11-16T16:42:06.520 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2023-11-16T16:42:06.520 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:42:06.522 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.506+0000 7f29e3b8a700 1 -- 172.21.15.1:0/847975337 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f29c80a12c0 msgr2=0x7f29c80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:06.522 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.506+0000 7f29e3b8a700 1 --2- 172.21.15.1:0/847975337 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f29c80a12c0 0x7f29c80a3770 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f29dc197920 tx=0x7f29d8009260 comp rx=0 tx=0).stop 2023-11-16T16:42:06.522 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.506+0000 7f29e3b8a700 1 -- 172.21.15.1:0/847975337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f29dc067dc0 msgr2=0x7f29dc196310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:06.522 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.506+0000 7f29e3b8a700 1 --2- 172.21.15.1:0/847975337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f29dc067dc0 0x7f29dc196310 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f29cc000c00 tx=0x7f29cc00bfd0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.522 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.506+0000 7f29e3b8a700 1 -- 172.21.15.1:0/847975337 shutdown_connections 2023-11-16T16:42:06.523 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.506+0000 7f29e3b8a700 1 --2- 172.21.15.1:0/847975337 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f29c80a12c0 0x7f29c80a3770 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.523 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.506+0000 7f29e3b8a700 1 --2- 172.21.15.1:0/847975337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29dc105250 0x7f29dc19acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.523 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.506+0000 7f29e3b8a700 1 --2- 172.21.15.1:0/847975337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f29dc067dc0 0x7f29dc196310 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.523 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.506+0000 7f29e3b8a700 1 --2- 172.21.15.1:0/847975337 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f29dc068770 0x7f29dc196850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.523 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.506+0000 7f29e3b8a700 1 -- 172.21.15.1:0/847975337 >> 172.21.15.1:0/847975337 conn(0x7f29dc0763a0 msgr2=0x7f29dc0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:06.523 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.507+0000 7f29e3b8a700 1 -- 172.21.15.1:0/847975337 shutdown_connections 2023-11-16T16:42:06.523 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.507+0000 7f29e3b8a700 1 -- 172.21.15.1:0/847975337 wait complete. 2023-11-16T16:42:06.523 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-16T16:42:06.524 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:42:06.654 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.649+0000 7fdf85909700 1 -- 172.21.15.1:0/2329091939 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf800ff4d0 msgr2=0x7fdf80104f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:06.654 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.649+0000 7fdf85909700 1 --2- 172.21.15.1:0/2329091939 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf800ff4d0 0x7fdf80104f30 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fdf740099c0 tx=0x7fdf74009cd0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.654 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.649+0000 7fdf85909700 1 -- 172.21.15.1:0/2329091939 shutdown_connections 2023-11-16T16:42:06.654 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.649+0000 7fdf85909700 1 --2- 172.21.15.1:0/2329091939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf80105610 0x7fdf8010d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.654 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.649+0000 7fdf85909700 1 --2- 172.21.15.1:0/2329091939 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf800feb30 0x7fdf800fef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.654 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.649+0000 7fdf85909700 1 --2- 172.21.15.1:0/2329091939 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf800ff4d0 0x7fdf80104f30 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.654 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.649+0000 7fdf85909700 1 -- 172.21.15.1:0/2329091939 >> 172.21.15.1:0/2329091939 conn(0x7fdf800746b0 msgr2=0x7fdf80074ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:06.655 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.649+0000 7fdf85909700 1 -- 172.21.15.1:0/2329091939 shutdown_connections 2023-11-16T16:42:06.655 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.649+0000 7fdf85909700 1 -- 172.21.15.1:0/2329091939 wait complete. 2023-11-16T16:42:06.655 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.650+0000 7fdf85909700 1 Processor -- start 2023-11-16T16:42:06.655 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.650+0000 7fdf85909700 1 -- start start 2023-11-16T16:42:06.655 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.650+0000 7fdf85909700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf800feb30 0x7fdf801963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:06.655 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.650+0000 7fdf85909700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf800ff4d0 0x7fdf80196930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:06.655 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.650+0000 7fdf85909700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf80105610 0x7fdf8019ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:06.655 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.650+0000 7fdf85909700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8019b3d0 con 0x7fdf800feb30 2023-11-16T16:42:06.656 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.650+0000 7fdf85909700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8019b540 con 0x7fdf800ff4d0 2023-11-16T16:42:06.656 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.650+0000 7fdf85909700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8019b700 con 0x7fdf80105610 2023-11-16T16:42:06.656 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.650+0000 7fdf7effd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf800feb30 0x7fdf801963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:06.656 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.650+0000 7fdf7effd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf800feb30 0x7fdf801963f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44874/0 (socket says 172.21.15.1:44874) 2023-11-16T16:42:06.656 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.650+0000 7fdf7f7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf80105610 0x7fdf8019ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:06.656 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.650+0000 7fdf7effd700 1 -- 172.21.15.1:0/1486724791 learned_addr learned my addr 172.21.15.1:0/1486724791 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:42:06.656 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.650+0000 7fdf7e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf800ff4d0 0x7fdf80196930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:06.657 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.651+0000 7fdf7f7fe700 1 -- 172.21.15.1:0/1486724791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf800ff4d0 msgr2=0x7fdf80196930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:06.657 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.651+0000 7fdf7f7fe700 1 --2- 172.21.15.1:0/1486724791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf800ff4d0 0x7fdf80196930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.657 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.651+0000 7fdf7f7fe700 1 -- 172.21.15.1:0/1486724791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf800feb30 msgr2=0x7fdf801963f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:06.657 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.651+0000 7fdf7f7fe700 1 --2- 172.21.15.1:0/1486724791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf800feb30 0x7fdf801963f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.657 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.651+0000 7fdf7f7fe700 1 -- 172.21.15.1:0/1486724791 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf740095f0 con 0x7fdf80105610 2023-11-16T16:42:06.657 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.651+0000 7fdf7e7fc700 1 --2- 172.21.15.1:0/1486724791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf800ff4d0 0x7fdf80196930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:42:06.657 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.651+0000 7fdf7f7fe700 1 --2- 172.21.15.1:0/1486724791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf80105610 0x7fdf8019ad90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800efc0 tx=0x7fdf7800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:06.657 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.651+0000 7fdf84907700 1 -- 172.21.15.1:0/1486724791 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf78018070 con 0x7fdf80105610 2023-11-16T16:42:06.658 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.651+0000 7fdf85909700 1 -- 172.21.15.1:0/1486724791 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf801103b0 con 0x7fdf80105610 2023-11-16T16:42:06.658 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.651+0000 7fdf84907700 1 -- 172.21.15.1:0/1486724791 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf7800f040 con 0x7fdf80105610 2023-11-16T16:42:06.658 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.651+0000 7fdf84907700 1 -- 172.21.15.1:0/1486724791 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7800cd40 con 0x7fdf80105610 2023-11-16T16:42:06.658 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.651+0000 7fdf85909700 1 -- 172.21.15.1:0/1486724791 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf80110220 con 0x7fdf80105610 2023-11-16T16:42:06.658 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.652+0000 7fdf85909700 1 -- 172.21.15.1:0/1486724791 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf80110540 con 0x7fdf80105610 2023-11-16T16:42:06.659 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.653+0000 7fdf84907700 1 -- 172.21.15.1:0/1486724791 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fdf78003a00 con 0x7fdf80105610 2023-11-16T16:42:06.659 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.654+0000 7fdf84907700 1 --2- 172.21.15.1:0/1486724791 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fdf680a12c0 0x7fdf680a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:06.660 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.654+0000 7fdf7effd700 1 --2- 172.21.15.1:0/1486724791 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fdf680a12c0 0x7fdf680a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:06.660 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.654+0000 7fdf84907700 1 -- 172.21.15.1:0/1486724791 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fdf780cc9e0 con 0x7fdf80105610 2023-11-16T16:42:06.660 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.654+0000 7fdf7effd700 1 --2- 172.21.15.1:0/1486724791 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fdf680a12c0 0x7fdf680a3770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdf70009d20 tx=0x7fdf70009260 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:06.660 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.657+0000 7fdf84907700 1 -- 172.21.15.1:0/1486724791 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fdf78095e90 con 0x7fdf80105610 2023-11-16T16:42:06.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:06 smithi001 ceph-mon[98800]: pgmap v91: 97 pgs: 97 active+clean; 68 MiB data, 148 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 33 op/s 2023-11-16T16:42:06.913 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.911+0000 7fdf85909700 1 -- 172.21.15.1:0/1486724791 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf80197840 con 0x7fdf80105610 2023-11-16T16:42:06.914 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.912+0000 7fdf84907700 1 -- 172.21.15.1:0/1486724791 <== mon.1 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10778 (secure 0 0 0) 0x7fdf780994b0 con 0x7fdf80105610 2023-11-16T16:42:06.914 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:42:06.914 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2587563684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2587563684},{"type":"v1","addr":"172.21.15.1:6835","nonce":2587563684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/277936168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":277936168},{"type":"v1","addr":"172.21.15.1:6837","nonce":277936168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/1172575045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1172575045},{"type":"v1","addr":"172.21.15.1:6839","nonce":1172575045}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/2631175899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2631175899},{"type":"v1","addr":"172.21.15.1:6841","nonce":2631175899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3693905267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3693905267},{"type":"v1","addr":"172.21.15.53:6833","nonce":3693905267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/164668033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":164668033},{"type":"v1","addr":"172.21.15.26:6833","nonce":164668033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-16T16:37:13.918943+0000","modified":"2023-11-16T16:42:03.246074+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2140542192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2140542192},{"type":"v1","addr":"172.21.15.53:6835","nonce":2140542192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1010760275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1010760275},{"type":"v1","addr":"172.21.15.26:6835","nonce":1010760275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3197060302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3197060302},{"type":"v1","addr":"172.21.15.53:6837","nonce":3197060302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6837/1453504863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1453504863},{"type":"v1","addr":"172.21.15.26:6837","nonce":1453504863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.53:6839/1044753024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1044753024},{"type":"v1","addr":"172.21.15.53:6839","nonce":1044753024}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6839/2192205949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2192205949},{"type":"v1","addr":"172.21.15.26:6839","nonce":2192205949}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-16T16:42:06.917 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.914+0000 7fdf85909700 1 -- 172.21.15.1:0/1486724791 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fdf680a12c0 msgr2=0x7fdf680a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:06.917 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.914+0000 7fdf85909700 1 --2- 172.21.15.1:0/1486724791 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fdf680a12c0 0x7fdf680a3770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdf70009d20 tx=0x7fdf70009260 comp rx=0 tx=0).stop 2023-11-16T16:42:06.917 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.914+0000 7fdf85909700 1 -- 172.21.15.1:0/1486724791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf80105610 msgr2=0x7fdf8019ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:06.917 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.914+0000 7fdf85909700 1 --2- 172.21.15.1:0/1486724791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf80105610 0x7fdf8019ad90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800efc0 tx=0x7fdf7800c3a0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.917 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.914+0000 7fdf85909700 1 -- 172.21.15.1:0/1486724791 shutdown_connections 2023-11-16T16:42:06.918 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.914+0000 7fdf85909700 1 --2- 172.21.15.1:0/1486724791 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fdf680a12c0 0x7fdf680a3770 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.918 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.914+0000 7fdf85909700 1 --2- 172.21.15.1:0/1486724791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf80105610 0x7fdf8019ad90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.918 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.914+0000 7fdf85909700 1 --2- 172.21.15.1:0/1486724791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf800feb30 0x7fdf801963f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.918 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.914+0000 7fdf85909700 1 --2- 172.21.15.1:0/1486724791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf800ff4d0 0x7fdf80196930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:06.918 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.914+0000 7fdf85909700 1 -- 172.21.15.1:0/1486724791 >> 172.21.15.1:0/1486724791 conn(0x7fdf800746b0 msgr2=0x7fdf800fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:06.918 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.914+0000 7fdf85909700 1 -- 172.21.15.1:0/1486724791 shutdown_connections 2023-11-16T16:42:06.918 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:06.914+0000 7fdf85909700 1 -- 172.21.15.1:0/1486724791 wait complete. 2023-11-16T16:42:06.919 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2023-11-16T16:42:06.929 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-11-16T16:42:07.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:06 smithi026 ceph-mon[100112]: pgmap v91: 97 pgs: 97 active+clean; 68 MiB data, 148 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 33 op/s 2023-11-16T16:42:07.061 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.057+0000 7f93ec7d7700 1 -- 172.21.15.1:0/3862188444 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93e4105250 msgr2=0x7f93e4105620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:07.061 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.057+0000 7f93ec7d7700 1 --2- 172.21.15.1:0/3862188444 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93e4105250 0x7f93e4105620 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f93e00099c0 tx=0x7f93e0009cd0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.061 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.057+0000 7f93ec7d7700 1 -- 172.21.15.1:0/3862188444 shutdown_connections 2023-11-16T16:42:07.062 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.057+0000 7f93ec7d7700 1 --2- 172.21.15.1:0/3862188444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93e4068770 0x7f93e410d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.062 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.057+0000 7f93ec7d7700 1 --2- 172.21.15.1:0/3862188444 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93e4105250 0x7f93e4105620 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.062 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.057+0000 7f93ec7d7700 1 --2- 172.21.15.1:0/3862188444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93e4067dc0 0x7f93e4068230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.062 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.057+0000 7f93ec7d7700 1 -- 172.21.15.1:0/3862188444 >> 172.21.15.1:0/3862188444 conn(0x7f93e40763a0 msgr2=0x7f93e40767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:07.062 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.057+0000 7f93ec7d7700 1 -- 172.21.15.1:0/3862188444 shutdown_connections 2023-11-16T16:42:07.062 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.057+0000 7f93ec7d7700 1 -- 172.21.15.1:0/3862188444 wait complete. 2023-11-16T16:42:07.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.057+0000 7f93ec7d7700 1 Processor -- start 2023-11-16T16:42:07.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.057+0000 7f93ec7d7700 1 -- start start 2023-11-16T16:42:07.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.058+0000 7f93ec7d7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93e4067dc0 0x7f93e4196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.058+0000 7f93ec7d7700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93e4068770 0x7f93e4196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.058+0000 7f93ec7d7700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93e4105250 0x7f93e419acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.058+0000 7f93ec7d7700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e419b2f0 con 0x7f93e4105250 2023-11-16T16:42:07.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.058+0000 7f93ec7d7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e419b460 con 0x7f93e4067dc0 2023-11-16T16:42:07.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.058+0000 7f93ec7d7700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e419b620 con 0x7f93e4068770 2023-11-16T16:42:07.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.058+0000 7f93ead74700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93e4105250 0x7f93e419acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.058+0000 7f93ead74700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93e4105250 0x7f93e419acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44892/0 (socket says 172.21.15.1:44892) 2023-11-16T16:42:07.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.058+0000 7f93e9d72700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93e4068770 0x7f93e4196850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.058+0000 7f93ead74700 1 -- 172.21.15.1:0/2732617575 learned_addr learned my addr 172.21.15.1:0/2732617575 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:42:07.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.058+0000 7f93ea573700 1 --2- 172.21.15.1:0/2732617575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93e4067dc0 0x7f93e4196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.058+0000 7f93ead74700 1 -- 172.21.15.1:0/2732617575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93e4068770 msgr2=0x7f93e4196850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:07.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.058+0000 7f93ead74700 1 --2- 172.21.15.1:0/2732617575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93e4068770 0x7f93e4196850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.058+0000 7f93ead74700 1 -- 172.21.15.1:0/2732617575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93e4067dc0 msgr2=0x7f93e4196310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:07.065 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.058+0000 7f93ead74700 1 --2- 172.21.15.1:0/2732617575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93e4067dc0 0x7f93e4196310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.065 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.058+0000 7f93ead74700 1 -- 172.21.15.1:0/2732617575 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e00095f0 con 0x7f93e4105250 2023-11-16T16:42:07.065 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.058+0000 7f93ead74700 1 --2- 172.21.15.1:0/2732617575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93e4105250 0x7f93e419acb0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f93dc00ada0 tx=0x7f93dc00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:07.065 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.058+0000 7f93db7fe700 1 -- 172.21.15.1:0/2732617575 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc00f070 con 0x7f93e4105250 2023-11-16T16:42:07.065 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.058+0000 7f93db7fe700 1 -- 172.21.15.1:0/2732617575 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93dc007b20 con 0x7f93e4105250 2023-11-16T16:42:07.065 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.058+0000 7f93ec7d7700 1 -- 172.21.15.1:0/2732617575 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93e4110310 con 0x7f93e4105250 2023-11-16T16:42:07.065 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.058+0000 7f93db7fe700 1 -- 172.21.15.1:0/2732617575 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc004290 con 0x7f93e4105250 2023-11-16T16:42:07.065 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.059+0000 7f93ec7d7700 1 -- 172.21.15.1:0/2732617575 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e4110180 con 0x7f93e4105250 2023-11-16T16:42:07.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.061+0000 7f93db7fe700 1 -- 172.21.15.1:0/2732617575 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f93dc003a20 con 0x7f93e4105250 2023-11-16T16:42:07.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.061+0000 7f93db7fe700 1 --2- 172.21.15.1:0/2732617575 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f93d00a12c0 0x7f93d00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.061+0000 7f93ea573700 1 --2- 172.21.15.1:0/2732617575 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f93d00a12c0 0x7f93d00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.061+0000 7f93db7fe700 1 -- 172.21.15.1:0/2732617575 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f93dc0cbdf0 con 0x7f93e4105250 2023-11-16T16:42:07.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.062+0000 7f93ea573700 1 --2- 172.21.15.1:0/2732617575 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f93d00a12c0 0x7f93d00a3770 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009990 tx=0x7f93e0005fb0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:07.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.062+0000 7f93f8963b80 1 Processor -- start 2023-11-16T16:42:07.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.062+0000 7f93f8963b80 1 -- start start 2023-11-16T16:42:07.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.062+0000 7f93f8963b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b41ceb5080 0x55b41ceb5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.062+0000 7f93f8963b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b41cea7970 0x55b41cea7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.062+0000 7f93f8963b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b41cea8320 0x55b41cebcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.062+0000 7f93f8963b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b41cdad060 con 0x55b41ceb5080 2023-11-16T16:42:07.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.062+0000 7f93f8963b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b41cdff8a0 con 0x55b41cea8320 2023-11-16T16:42:07.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.062+0000 7f93f8963b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b41cdf8d30 con 0x55b41cea7970 2023-11-16T16:42:07.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.063+0000 7f93ea573700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b41ceb5080 0x55b41ceb5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.063+0000 7f93ea573700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b41ceb5080 0x55b41ceb5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44896/0 (socket says 172.21.15.1:44896) 2023-11-16T16:42:07.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.063+0000 7f93e9d72700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b41cea7970 0x55b41cea7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.063+0000 7f93ea573700 1 -- 172.21.15.1:0/1864924491 learned_addr learned my addr 172.21.15.1:0/1864924491 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:42:07.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.063+0000 7f93ead74700 1 --2- 172.21.15.1:0/1864924491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b41cea8320 0x55b41cebcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.063+0000 7f93ea573700 1 -- 172.21.15.1:0/1864924491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b41cea7970 msgr2=0x55b41cea7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:07.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.063+0000 7f93ea573700 1 --2- 172.21.15.1:0/1864924491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b41cea7970 0x55b41cea7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.063+0000 7f93ea573700 1 -- 172.21.15.1:0/1864924491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b41cea8320 msgr2=0x55b41cebcce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:07.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.063+0000 7f93ea573700 1 --2- 172.21.15.1:0/1864924491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b41cea8320 0x55b41cebcce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.063+0000 7f93ea573700 1 -- 172.21.15.1:0/1864924491 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b41cc7a630 con 0x55b41ceb5080 2023-11-16T16:42:07.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.063+0000 7f93ead74700 1 --2- 172.21.15.1:0/1864924491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b41cea8320 0x55b41cebcce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:42:07.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.063+0000 7f93e9d72700 1 --2- 172.21.15.1:0/1864924491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b41cea7970 0x55b41cea7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:42:07.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.063+0000 7f93ea573700 1 --2- 172.21.15.1:0/1864924491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b41ceb5080 0x55b41ceb5450 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f93e0015000 tx=0x7f93e001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c90469f95d84ad4e server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:07.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.063+0000 7f93d8ff9700 1 -- 172.21.15.1:0/1864924491 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e000b990 con 0x55b41ceb5080 2023-11-16T16:42:07.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.063+0000 7f93d8ff9700 1 -- 172.21.15.1:0/1864924491 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93e001e040 con 0x55b41ceb5080 2023-11-16T16:42:07.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.063+0000 7f93d8ff9700 1 -- 172.21.15.1:0/1864924491 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e00306e0 con 0x55b41ceb5080 2023-11-16T16:42:07.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.064+0000 7f93f8963b80 1 -- 172.21.15.1:0/1864924491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b41ceb5080 msgr2=0x55b41ceb5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:07.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.064+0000 7f93f8963b80 1 --2- 172.21.15.1:0/1864924491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b41ceb5080 0x55b41ceb5450 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f93e0015000 tx=0x7f93e001f000 comp rx=0 tx=0).stop 2023-11-16T16:42:07.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.064+0000 7f93f8963b80 1 -- 172.21.15.1:0/1864924491 shutdown_connections 2023-11-16T16:42:07.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.064+0000 7f93f8963b80 1 --2- 172.21.15.1:0/1864924491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b41cea8320 0x55b41cebcce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.064+0000 7f93f8963b80 1 --2- 172.21.15.1:0/1864924491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b41ceb5080 0x55b41ceb5450 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.064+0000 7f93f8963b80 1 --2- 172.21.15.1:0/1864924491 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b41cea7970 0x55b41cea7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.064+0000 7f93f8963b80 1 -- 172.21.15.1:0/1864924491 >> 172.21.15.1:0/1864924491 conn(0x55b41ce9d360 msgr2=0x55b41cea27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:07.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.064+0000 7f93f8963b80 1 -- 172.21.15.1:0/1864924491 shutdown_connections 2023-11-16T16:42:07.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.064+0000 7f93f8963b80 1 -- 172.21.15.1:0/1864924491 wait complete. 2023-11-16T16:42:07.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.064+0000 7f93f8963b80 1 -- start start 2023-11-16T16:42:07.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.065+0000 7f93f8963b80 1 Processor -- start 2023-11-16T16:42:07.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.065+0000 7f93f8963b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b41cea7970 0x55b41cebfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.065+0000 7f93f8963b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b41cea8320 0x55b41cec00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.065+0000 7f93f8963b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b41ceb5080 0x55b41ceb9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.065+0000 7f93f8963b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b41cd9a8e0 con 0x55b41ceb5080 2023-11-16T16:42:07.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.065+0000 7f93f8963b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b41cd9a580 con 0x55b41cea8320 2023-11-16T16:42:07.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.065+0000 7f93f8963b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b41cc7c920 con 0x55b41cea7970 2023-11-16T16:42:07.080 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.065+0000 7f93f8963b80 10 client.0 _ll_register_callbacks cb 0x55b41cdb8780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:42:07.080 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.065+0000 7f93ead74700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b41ceb5080 0x55b41ceb9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.080 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.065+0000 7f93ead74700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b41ceb5080 0x55b41ceb9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44906/0 (socket says 172.21.15.1:44906) 2023-11-16T16:42:07.080 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.065+0000 7f93ead74700 1 -- 172.21.15.1:0/3392604983 learned_addr learned my addr 172.21.15.1:0/3392604983 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:42:07.080 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.065+0000 7f93ea573700 1 --2- 172.21.15.1:0/3392604983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b41cea7970 0x55b41cebfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.080 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.065+0000 7f93e9d72700 1 --2- 172.21.15.1:0/3392604983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b41cea8320 0x55b41cec00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.080 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.065+0000 7f93ead74700 1 -- 172.21.15.1:0/3392604983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b41cea7970 msgr2=0x55b41cebfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:07.080 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.065+0000 7f93ead74700 1 --2- 172.21.15.1:0/3392604983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b41cea7970 0x55b41cebfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.081 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.065+0000 7f93ead74700 1 -- 172.21.15.1:0/3392604983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b41cea8320 msgr2=0x55b41cec00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:07.081 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.065+0000 7f93ead74700 1 --2- 172.21.15.1:0/3392604983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b41cea8320 0x55b41cec00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.081 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.065+0000 7f93ead74700 1 -- 172.21.15.1:0/3392604983 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e410d970 con 0x55b41ceb5080 2023-11-16T16:42:07.081 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.065+0000 7f93ead74700 1 --2- 172.21.15.1:0/3392604983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b41ceb5080 0x55b41ceb9e90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f93dc098e60 tx=0x7f93dc00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33d1552c99191cd0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:07.081 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.065+0000 7f93cdffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:42:07.081 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.066+0000 7f93cdffb700 1 -- 172.21.15.1:0/3392604983 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc00a0b0 con 0x55b41ceb5080 2023-11-16T16:42:07.081 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.066+0000 7f93cdffb700 1 -- 172.21.15.1:0/3392604983 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f93dc0a1320 con 0x55b41ceb5080 2023-11-16T16:42:07.081 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.066+0000 7f93f8963b80 1 -- 172.21.15.1:0/3392604983 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b41ce9ab20 con 0x55b41ceb5080 2023-11-16T16:42:07.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.066+0000 7f93cdffb700 1 -- 172.21.15.1:0/3392604983 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93dc0b7170 con 0x55b41ceb5080 2023-11-16T16:42:07.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.066+0000 7f93f8963b80 10 client.16011 fetch_fsmap learned FSMap version 25 2023-11-16T16:42:07.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.066+0000 7f93f8963b80 1 -- 172.21.15.1:0/3392604983 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b41cdbf190 con 0x55b41ceb5080 2023-11-16T16:42:07.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.066+0000 7f93cdffb700 1 -- 172.21.15.1:0/3392604983 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93dc0c7080 con 0x55b41ceb5080 2023-11-16T16:42:07.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.066+0000 7f93cdffb700 1 -- 172.21.15.1:0/3392604983 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f93dc0acc60 con 0x55b41ceb5080 2023-11-16T16:42:07.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.066+0000 7f93f8963b80 10 client.16011 fetch_fsmap finished waiting for FSMap version 25 2023-11-16T16:42:07.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.066+0000 7f93f8963b80 10 client.16011 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-11-16T16:42:07.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.066+0000 7f93f8963b80 20 client.16011 populate_metadata read hostname 'smithi001' 2023-11-16T16:42:07.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.066+0000 7f93f8963b80 1 --2- 172.21.15.1:0/3392604983 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55b41ceca0a0 0x55b41cecc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.066+0000 7f93f8963b80 4 client.16011 mds_command: new command op to 24482 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:42:07.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.066+0000 7f93f8963b80 1 -- 172.21.15.1:0/3392604983 --> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b41cdbb0a0 con 0x55b41ceca0a0 2023-11-16T16:42:07.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.066+0000 7f93ea573700 1 --2- 172.21.15.1:0/3392604983 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55b41ceca0a0 0x55b41cecc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.067+0000 7f93ea573700 1 --2- 172.21.15.1:0/3392604983 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55b41ceca0a0 0x55b41cecc4b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bb042ce13230730 server_cookie=a288c5adf8b3e235 in_seq=0 out_seq=0 2023-11-16T16:42:07.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.067+0000 7f93cdffb700 10 client.16011 ms_handle_connect on v2:172.21.15.26:6836/1453504863 2023-11-16T16:42:07.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.067+0000 7f93cdffb700 1 -- 172.21.15.1:0/3392604983 <== mds.2 v2:172.21.15.26:6836/1453504863 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b41cdbb0a0 con 0x55b41ceca0a0 2023-11-16T16:42:07.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.067+0000 7f93cdffb700 10 client.16011 handle_command_reply: tid=0 2023-11-16T16:42:07.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.068+0000 7f93f8963b80 1 client.16011 shutdown 2023-11-16T16:42:07.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.068+0000 7f93f8963b80 1 -- 172.21.15.1:0/3392604983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b41ceb5080 msgr2=0x55b41ceb9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:07.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.068+0000 7f93f8963b80 1 --2- 172.21.15.1:0/3392604983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b41ceb5080 0x55b41ceb9e90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f93dc098e60 tx=0x7f93dc00ced0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.068+0000 7f93f8963b80 1 -- 172.21.15.1:0/3392604983 shutdown_connections 2023-11-16T16:42:07.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.068+0000 7f93f8963b80 1 --2- 172.21.15.1:0/3392604983 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55b41ceca0a0 0x55b41cecc4b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.068+0000 7f93f8963b80 1 --2- 172.21.15.1:0/3392604983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b41ceb5080 0x55b41ceb9e90 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.068+0000 7f93f8963b80 1 --2- 172.21.15.1:0/3392604983 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b41cea7970 0x55b41cebfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.068+0000 7f93cdffb700 0 client.16011 ms_handle_reset on v2:172.21.15.26:6836/1453504863 2023-11-16T16:42:07.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.068+0000 7f93f8963b80 1 --2- 172.21.15.1:0/3392604983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b41cea8320 0x55b41cec00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.068+0000 7f93f8963b80 1 -- 172.21.15.1:0/3392604983 >> 172.21.15.1:0/3392604983 conn(0x55b41ce9d360 msgr2=0x55b41cea1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:07.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.068+0000 7f93f8963b80 1 -- 172.21.15.1:0/3392604983 shutdown_connections 2023-11-16T16:42:07.085 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.068+0000 7f93f8963b80 1 -- 172.21.15.1:0/3392604983 wait complete. 2023-11-16T16:42:07.085 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.068+0000 7f93f8963b80 20 client.16011 trim_cache size 0 max 16384 2023-11-16T16:42:07.086 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:06 smithi053 ceph-mon[100638]: pgmap v91: 97 pgs: 97 active+clean; 68 MiB data, 148 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 33 op/s 2023-11-16T16:42:07.093 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.081+0000 7f93f8963b80 1 Processor -- start 2023-11-16T16:42:07.093 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.081+0000 7f93f8963b80 1 -- start start 2023-11-16T16:42:07.093 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.081+0000 7f93f8963b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b41cea7970 0x55b41cea2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.093 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.081+0000 7f93f8963b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b41cea8320 0x55b41cebf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.093 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.081+0000 7f93f8963b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b41ceb5080 0x55b41cebfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.093 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.081+0000 7f93f8963b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b41ceb83f0 con 0x55b41ceb5080 2023-11-16T16:42:07.093 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.081+0000 7f93f8963b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b41ceb5ce0 con 0x55b41cea7970 2023-11-16T16:42:07.094 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.081+0000 7f93f8963b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b41cea4190 con 0x55b41cea8320 2023-11-16T16:42:07.094 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.081+0000 7f93ead74700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b41ceb5080 0x55b41cebfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.094 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.081+0000 7f93ead74700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b41ceb5080 0x55b41cebfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44914/0 (socket says 172.21.15.1:44914) 2023-11-16T16:42:07.094 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.081+0000 7f93ead74700 1 -- 172.21.15.1:0/3659606348 learned_addr learned my addr 172.21.15.1:0/3659606348 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:42:07.094 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.081+0000 7f93e9d72700 1 --2- 172.21.15.1:0/3659606348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b41cea8320 0x55b41cebf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.094 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.081+0000 7f93ea573700 1 --2- 172.21.15.1:0/3659606348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b41cea7970 0x55b41cea2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.094 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.081+0000 7f93ead74700 1 -- 172.21.15.1:0/3659606348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b41cea8320 msgr2=0x55b41cebf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:07.094 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.081+0000 7f93ead74700 1 --2- 172.21.15.1:0/3659606348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b41cea8320 0x55b41cebf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.095 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.081+0000 7f93ead74700 1 -- 172.21.15.1:0/3659606348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b41cea7970 msgr2=0x55b41cea2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:07.095 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.081+0000 7f93ead74700 1 --2- 172.21.15.1:0/3659606348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b41cea7970 0x55b41cea2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.095 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.081+0000 7f93ead74700 1 -- 172.21.15.1:0/3659606348 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b41cdbeee0 con 0x55b41ceb5080 2023-11-16T16:42:07.095 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.081+0000 7f93ea573700 1 --2- 172.21.15.1:0/3659606348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b41cea7970 0x55b41cea2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:42:07.095 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.081+0000 7f93ead74700 1 --2- 172.21.15.1:0/3659606348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b41ceb5080 0x55b41cebfca0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f93dc0c9b50 tx=0x7f93dc098d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6391fa6a7b5890aa server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:07.095 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.082+0000 7f93d97fa700 1 -- 172.21.15.1:0/3659606348 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc00a0b0 con 0x55b41ceb5080 2023-11-16T16:42:07.095 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.082+0000 7f93d97fa700 1 -- 172.21.15.1:0/3659606348 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b41ce9ab20 con 0x55b41ceb5080 2023-11-16T16:42:07.095 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.082+0000 7f93d97fa700 1 -- 172.21.15.1:0/3659606348 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc0983c0 con 0x55b41ceb5080 2023-11-16T16:42:07.095 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.082+0000 7f93f8963b80 1 -- 172.21.15.1:0/3659606348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b41ceb5080 msgr2=0x55b41cebfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:07.096 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.082+0000 7f93f8963b80 1 --2- 172.21.15.1:0/3659606348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b41ceb5080 0x55b41cebfca0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f93dc0c9b50 tx=0x7f93dc098d50 comp rx=0 tx=0).stop 2023-11-16T16:42:07.096 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.082+0000 7f93f8963b80 1 -- 172.21.15.1:0/3659606348 shutdown_connections 2023-11-16T16:42:07.096 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.082+0000 7f93f8963b80 1 --2- 172.21.15.1:0/3659606348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b41ceb5080 0x55b41cebfca0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.096 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.082+0000 7f93f8963b80 1 --2- 172.21.15.1:0/3659606348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b41cea7970 0x55b41cea2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.096 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.082+0000 7f93f8963b80 1 --2- 172.21.15.1:0/3659606348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b41cea8320 0x55b41cebf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.096 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.082+0000 7f93f8963b80 1 -- 172.21.15.1:0/3659606348 >> 172.21.15.1:0/3659606348 conn(0x55b41ce9d360 msgr2=0x55b41ceba820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:07.096 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.082+0000 7f93f8963b80 1 -- 172.21.15.1:0/3659606348 shutdown_connections 2023-11-16T16:42:07.096 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.082+0000 7f93f8963b80 1 -- 172.21.15.1:0/3659606348 wait complete. 2023-11-16T16:42:07.096 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.083+0000 7f93f8963b80 1 -- start start 2023-11-16T16:42:07.097 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.083+0000 7f93f8963b80 1 Processor -- start 2023-11-16T16:42:07.097 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.083+0000 7f93f8963b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b41cea7970 0x55b41cf11ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.097 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.083+0000 7f93f8963b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b41cea8320 0x55b41cf12430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.097 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.083+0000 7f93f8963b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b41ceb5080 0x55b41cf1cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.097 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.083+0000 7f93f8963b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b41cf12be0 con 0x55b41cea8320 2023-11-16T16:42:07.097 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.083+0000 7f93f8963b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b41cf12d50 con 0x55b41cea7970 2023-11-16T16:42:07.097 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.083+0000 7f93f8963b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b41cf1d4e0 con 0x55b41ceb5080 2023-11-16T16:42:07.097 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.083+0000 7f93f8963b80 10 client.0 _ll_register_callbacks cb 0x7f93dc098900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:42:07.098 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.083+0000 7f93e9d72700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b41cea8320 0x55b41cf12430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.098 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.083+0000 7f93e9d72700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b41cea8320 0x55b41cf12430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44918/0 (socket says 172.21.15.1:44918) 2023-11-16T16:42:07.098 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.083+0000 7f93e9d72700 1 -- 172.21.15.1:0/3050957782 learned_addr learned my addr 172.21.15.1:0/3050957782 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:42:07.098 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.083+0000 7f93ead74700 1 --2- 172.21.15.1:0/3050957782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b41ceb5080 0x55b41cf1cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.098 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.083+0000 7f93ea573700 1 --2- 172.21.15.1:0/3050957782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b41cea7970 0x55b41cf11ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.098 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.083+0000 7f93e9d72700 1 -- 172.21.15.1:0/3050957782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b41ceb5080 msgr2=0x55b41cf1cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:07.098 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.083+0000 7f93e9d72700 1 --2- 172.21.15.1:0/3050957782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b41ceb5080 0x55b41cf1cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.098 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.083+0000 7f93e9d72700 1 -- 172.21.15.1:0/3050957782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b41cea7970 msgr2=0x55b41cf11ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:07.098 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.083+0000 7f93e9d72700 1 --2- 172.21.15.1:0/3050957782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b41cea7970 0x55b41cf11ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.099 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.083+0000 7f93e9d72700 1 -- 172.21.15.1:0/3050957782 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b41cdbf190 con 0x55b41cea8320 2023-11-16T16:42:07.099 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.084+0000 7f93e9d72700 1 --2- 172.21.15.1:0/3050957782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b41cea8320 0x55b41cf12430 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x55b41cea45b0 tx=0x7f93d4012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e693b2c43b3519f server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:07.099 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.084+0000 7f93cdffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:42:07.099 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.084+0000 7f93cdffb700 1 -- 172.21.15.1:0/3050957782 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d4017040 con 0x55b41cea8320 2023-11-16T16:42:07.099 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.084+0000 7f93cdffb700 1 -- 172.21.15.1:0/3050957782 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f93d4004250 con 0x55b41cea8320 2023-11-16T16:42:07.099 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.084+0000 7f93f8963b80 1 -- 172.21.15.1:0/3050957782 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93e001e040 con 0x55b41cea8320 2023-11-16T16:42:07.099 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.084+0000 7f93cdffb700 1 -- 172.21.15.1:0/3050957782 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93d4004410 con 0x55b41cea8320 2023-11-16T16:42:07.099 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.084+0000 7f93f8963b80 10 client.16017 fetch_fsmap learned FSMap version 25 2023-11-16T16:42:07.100 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.084+0000 7f93f8963b80 1 -- 172.21.15.1:0/3050957782 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b41cc0fc60 con 0x55b41cea8320 2023-11-16T16:42:07.100 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.084+0000 7f93cdffb700 1 -- 172.21.15.1:0/3050957782 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93d402b020 con 0x55b41cea8320 2023-11-16T16:42:07.100 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.084+0000 7f93cdffb700 1 -- 172.21.15.1:0/3050957782 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f93d40054d0 con 0x55b41cea8320 2023-11-16T16:42:07.100 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.084+0000 7f93f8963b80 10 client.16017 fetch_fsmap finished waiting for FSMap version 25 2023-11-16T16:42:07.100 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.084+0000 7f93f8963b80 10 client.16017 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-11-16T16:42:07.100 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.084+0000 7f93f8963b80 20 client.16017 populate_metadata read hostname 'smithi001' 2023-11-16T16:42:07.100 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.084+0000 7f93f8963b80 1 --2- 172.21.15.1:0/3050957782 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55b41cf1ecc0 0x55b41cf210d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.100 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.084+0000 7f93f8963b80 4 client.16017 mds_command: new command op to 24482 tid=0[{"prefix": "damage ls"}] 2023-11-16T16:42:07.100 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.084+0000 7f93f8963b80 1 -- 172.21.15.1:0/3050957782 --> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b41cdc7660 con 0x55b41cf1ecc0 2023-11-16T16:42:07.100 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.085+0000 7f93ea573700 1 --2- 172.21.15.1:0/3050957782 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55b41cf1ecc0 0x55b41cf210d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.101 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.085+0000 7f93cdffb700 10 client.16017 ms_handle_connect on v2:172.21.15.26:6836/1453504863 2023-11-16T16:42:07.101 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.085+0000 7f93ea573700 1 --2- 172.21.15.1:0/3050957782 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55b41cf1ecc0 0x55b41cf210d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=63fe392b7ebf77e3 server_cookie=d12f022785b43cfa in_seq=0 out_seq=0 2023-11-16T16:42:07.101 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.085+0000 7f93cdffb700 1 -- 172.21.15.1:0/3050957782 <== mds.2 v2:172.21.15.26:6836/1453504863 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b41cdc7660 con 0x55b41cf1ecc0 2023-11-16T16:42:07.101 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.085+0000 7f93cdffb700 10 client.16017 handle_command_reply: tid=0 2023-11-16T16:42:07.101 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.085+0000 7f93f8963b80 1 client.16017 shutdown 2023-11-16T16:42:07.101 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.085+0000 7f93f8963b80 1 -- 172.21.15.1:0/3050957782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b41cea8320 msgr2=0x55b41cf12430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:07.101 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.085+0000 7f93f8963b80 1 --2- 172.21.15.1:0/3050957782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b41cea8320 0x55b41cf12430 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x55b41cea45b0 tx=0x7f93d4012bc0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.101 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.085+0000 7f93f8963b80 1 -- 172.21.15.1:0/3050957782 shutdown_connections 2023-11-16T16:42:07.101 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.085+0000 7f93f8963b80 1 --2- 172.21.15.1:0/3050957782 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55b41cf1ecc0 0x55b41cf210d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.102 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.085+0000 7f93f8963b80 1 --2- 172.21.15.1:0/3050957782 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b41ceb5080 0x55b41cf1cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.102 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.085+0000 7f93cdffb700 0 client.16017 ms_handle_reset on v2:172.21.15.26:6836/1453504863 2023-11-16T16:42:07.102 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.085+0000 7f93f8963b80 1 --2- 172.21.15.1:0/3050957782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b41cea7970 0x55b41cf11ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.102 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.085+0000 7f93f8963b80 1 --2- 172.21.15.1:0/3050957782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b41cea8320 0x55b41cf12430 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.102 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.085+0000 7f93f8963b80 1 -- 172.21.15.1:0/3050957782 >> 172.21.15.1:0/3050957782 conn(0x55b41ce9d360 msgr2=0x55b41cea6960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:07.102 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.085+0000 7f93f8963b80 1 -- 172.21.15.1:0/3050957782 shutdown_connections 2023-11-16T16:42:07.102 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.085+0000 7f93f8963b80 1 -- 172.21.15.1:0/3050957782 wait complete. 2023-11-16T16:42:07.102 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.085+0000 7f93f8963b80 20 client.16017 trim_cache size 0 max 16384 2023-11-16T16:42:07.103 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2023-11-16T16:42:07.104 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.086+0000 7f93ec7d7700 1 -- 172.21.15.1:0/2732617575 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f93d00a12c0 msgr2=0x7f93d00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:07.104 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.086+0000 7f93ec7d7700 1 --2- 172.21.15.1:0/2732617575 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f93d00a12c0 0x7f93d00a3770 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009990 tx=0x7f93e0005fb0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.104 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.086+0000 7f93ec7d7700 1 -- 172.21.15.1:0/2732617575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93e4105250 msgr2=0x7f93e419acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:07.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.086+0000 7f93ec7d7700 1 --2- 172.21.15.1:0/2732617575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93e4105250 0x7f93e419acb0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f93dc00ada0 tx=0x7f93dc00c330 comp rx=0 tx=0).stop 2023-11-16T16:42:07.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.086+0000 7f93ec7d7700 1 -- 172.21.15.1:0/2732617575 shutdown_connections 2023-11-16T16:42:07.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.086+0000 7f93ec7d7700 1 --2- 172.21.15.1:0/2732617575 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f93d00a12c0 0x7f93d00a3770 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.086+0000 7f93ec7d7700 1 --2- 172.21.15.1:0/2732617575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93e4105250 0x7f93e419acb0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.086+0000 7f93ec7d7700 1 --2- 172.21.15.1:0/2732617575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93e4067dc0 0x7f93e4196310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.086+0000 7f93ec7d7700 1 --2- 172.21.15.1:0/2732617575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f93e4068770 0x7f93e4196850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.086+0000 7f93ec7d7700 1 -- 172.21.15.1:0/2732617575 >> 172.21.15.1:0/2732617575 conn(0x7f93e40763a0 msgr2=0x7f93e40fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:07.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.086+0000 7f93ec7d7700 1 -- 172.21.15.1:0/2732617575 shutdown_connections 2023-11-16T16:42:07.106 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.086+0000 7f93ec7d7700 1 -- 172.21.15.1:0/2732617575 wait complete. 2023-11-16T16:42:07.106 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-11-16T16:42:07.236 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.234+0000 7fd314061700 1 -- 172.21.15.1:0/4267742858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd30c0ff8f0 msgr2=0x7fd30c0ffd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:07.236 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.234+0000 7fd314061700 1 --2- 172.21.15.1:0/4267742858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd30c0ff8f0 0x7fd30c0ffd60 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fd3080099c0 tx=0x7fd308009cd0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.236 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.234+0000 7fd314061700 1 -- 172.21.15.1:0/4267742858 shutdown_connections 2023-11-16T16:42:07.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.234+0000 7fd314061700 1 --2- 172.21.15.1:0/4267742858 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd30c1002a0 0x7fd30c10bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.234+0000 7fd314061700 1 --2- 172.21.15.1:0/4267742858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd30c104140 0x7fd30c104510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.234+0000 7fd314061700 1 --2- 172.21.15.1:0/4267742858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd30c0ff8f0 0x7fd30c0ffd60 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.234+0000 7fd314061700 1 -- 172.21.15.1:0/4267742858 >> 172.21.15.1:0/4267742858 conn(0x7fd30c075880 msgr2=0x7fd30c075c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:07.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.234+0000 7fd314061700 1 -- 172.21.15.1:0/4267742858 shutdown_connections 2023-11-16T16:42:07.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.234+0000 7fd314061700 1 -- 172.21.15.1:0/4267742858 wait complete. 2023-11-16T16:42:07.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.235+0000 7fd314061700 1 Processor -- start 2023-11-16T16:42:07.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.235+0000 7fd314061700 1 -- start start 2023-11-16T16:42:07.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.235+0000 7fd314061700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd30c0ff8f0 0x7fd30c1971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.235+0000 7fd314061700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd30c1002a0 0x7fd30c197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.235+0000 7fd314061700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd30c104140 0x7fd30c1a2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.235+0000 7fd314061700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c1a27a0 con 0x7fd30c104140 2023-11-16T16:42:07.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.235+0000 7fd314061700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c1a28e0 con 0x7fd30c0ff8f0 2023-11-16T16:42:07.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.235+0000 7fd314061700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c1a2a20 con 0x7fd30c1002a0 2023-11-16T16:42:07.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.235+0000 7fd3125fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd30c104140 0x7fd30c1a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.235+0000 7fd3125fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd30c104140 0x7fd30c1a2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44934/0 (socket says 172.21.15.1:44934) 2023-11-16T16:42:07.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.235+0000 7fd3125fe700 1 -- 172.21.15.1:0/423866184 learned_addr learned my addr 172.21.15.1:0/423866184 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:42:07.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.235+0000 7fd3115fc700 1 --2- 172.21.15.1:0/423866184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd30c1002a0 0x7fd30c197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.235+0000 7fd311dfd700 1 --2- 172.21.15.1:0/423866184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd30c0ff8f0 0x7fd30c1971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.235+0000 7fd3125fe700 1 -- 172.21.15.1:0/423866184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd30c1002a0 msgr2=0x7fd30c197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:07.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.235+0000 7fd3125fe700 1 --2- 172.21.15.1:0/423866184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd30c1002a0 0x7fd30c197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.235+0000 7fd3125fe700 1 -- 172.21.15.1:0/423866184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd30c0ff8f0 msgr2=0x7fd30c1971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:07.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.235+0000 7fd3125fe700 1 --2- 172.21.15.1:0/423866184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd30c0ff8f0 0x7fd30c1971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.235+0000 7fd3125fe700 1 -- 172.21.15.1:0/423866184 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3080095f0 con 0x7fd30c104140 2023-11-16T16:42:07.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.236+0000 7fd3125fe700 1 --2- 172.21.15.1:0/423866184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd30c104140 0x7fd30c1a2260 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd30400ef90 tx=0x7fd30400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:07.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.236+0000 7fd302ffd700 1 -- 172.21.15.1:0/423866184 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd304018070 con 0x7fd30c104140 2023-11-16T16:42:07.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.236+0000 7fd302ffd700 1 -- 172.21.15.1:0/423866184 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd30400f040 con 0x7fd30c104140 2023-11-16T16:42:07.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.236+0000 7fd302ffd700 1 -- 172.21.15.1:0/423866184 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30400cda0 con 0x7fd30c104140 2023-11-16T16:42:07.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.236+0000 7fd314061700 1 -- 172.21.15.1:0/423866184 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd30c10f220 con 0x7fd30c104140 2023-11-16T16:42:07.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.236+0000 7fd314061700 1 -- 172.21.15.1:0/423866184 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd30c10f090 con 0x7fd30c104140 2023-11-16T16:42:07.247 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.238+0000 7fd302ffd700 1 -- 172.21.15.1:0/423866184 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fd304003a00 con 0x7fd30c104140 2023-11-16T16:42:07.247 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.239+0000 7fd302ffd700 1 --2- 172.21.15.1:0/423866184 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fd2f80a1340 0x7fd2f80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.247 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.239+0000 7fd302ffd700 1 -- 172.21.15.1:0/423866184 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd3040cc7d0 con 0x7fd30c104140 2023-11-16T16:42:07.247 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.239+0000 7fd311dfd700 1 --2- 172.21.15.1:0/423866184 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fd2f80a1340 0x7fd2f80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.247 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.239+0000 7fd311dfd700 1 --2- 172.21.15.1:0/423866184 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fd2f80a1340 0x7fd2f80a37f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc005950 tx=0x7fd2fc00b410 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:07.247 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.239+0000 7fd3201edb80 1 Processor -- start 2023-11-16T16:42:07.247 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.240+0000 7fd3201edb80 1 -- start start 2023-11-16T16:42:07.248 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.240+0000 7fd3201edb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae8a6b3b60 0x55ae8a6b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.248 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.240+0000 7fd3201edb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8a6b4470 0x55ae8a6b8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.248 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.240+0000 7fd3201edb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ae8a6b88b0 0x55ae8a6b8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.248 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.240+0000 7fd3201edb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8a5b9060 con 0x55ae8a6b3b60 2023-11-16T16:42:07.248 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.240+0000 7fd3201edb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8a60b8a0 con 0x55ae8a6b88b0 2023-11-16T16:42:07.248 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.240+0000 7fd3201edb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8a604d30 con 0x55ae8a6b4470 2023-11-16T16:42:07.248 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.240+0000 7fd311dfd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae8a6b3b60 0x55ae8a6b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.248 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.240+0000 7fd311dfd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae8a6b3b60 0x55ae8a6b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44938/0 (socket says 172.21.15.1:44938) 2023-11-16T16:42:07.248 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.240+0000 7fd311dfd700 1 -- 172.21.15.1:0/4165773320 learned_addr learned my addr 172.21.15.1:0/4165773320 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:42:07.249 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.240+0000 7fd3115fc700 1 --2- 172.21.15.1:0/4165773320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8a6b4470 0x55ae8a6b8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.249 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.240+0000 7fd3125fe700 1 --2- 172.21.15.1:0/4165773320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ae8a6b88b0 0x55ae8a6b8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.249 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.240+0000 7fd311dfd700 1 -- 172.21.15.1:0/4165773320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8a6b4470 msgr2=0x55ae8a6b8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:07.249 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.240+0000 7fd311dfd700 1 --2- 172.21.15.1:0/4165773320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8a6b4470 0x55ae8a6b8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.249 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.240+0000 7fd311dfd700 1 -- 172.21.15.1:0/4165773320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ae8a6b88b0 msgr2=0x55ae8a6b8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:07.249 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.240+0000 7fd311dfd700 1 --2- 172.21.15.1:0/4165773320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ae8a6b88b0 0x55ae8a6b8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.249 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.240+0000 7fd311dfd700 1 -- 172.21.15.1:0/4165773320 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae8a486630 con 0x55ae8a6b3b60 2023-11-16T16:42:07.249 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.240+0000 7fd3125fe700 1 --2- 172.21.15.1:0/4165773320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ae8a6b88b0 0x55ae8a6b8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:42:07.249 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.240+0000 7fd3115fc700 1 --2- 172.21.15.1:0/4165773320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8a6b4470 0x55ae8a6b8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:42:07.249 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.240+0000 7fd311dfd700 1 --2- 172.21.15.1:0/4165773320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae8a6b3b60 0x55ae8a6b3f30 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc010e60 tx=0x7fd2fc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827e81a8f4ed059a server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:07.249 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.241+0000 7fd2f7fff700 1 -- 172.21.15.1:0/4165773320 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc011db0 con 0x55ae8a6b3b60 2023-11-16T16:42:07.249 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.241+0000 7fd2f7fff700 1 -- 172.21.15.1:0/4165773320 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2fc015070 con 0x55ae8a6b3b60 2023-11-16T16:42:07.250 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.241+0000 7fd2f7fff700 1 -- 172.21.15.1:0/4165773320 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc01c610 con 0x55ae8a6b3b60 2023-11-16T16:42:07.250 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.241+0000 7fd3201edb80 1 -- 172.21.15.1:0/4165773320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae8a6b3b60 msgr2=0x55ae8a6b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:07.250 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.241+0000 7fd3201edb80 1 --2- 172.21.15.1:0/4165773320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae8a6b3b60 0x55ae8a6b3f30 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc010e60 tx=0x7fd2fc013000 comp rx=0 tx=0).stop 2023-11-16T16:42:07.250 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.241+0000 7fd3201edb80 1 -- 172.21.15.1:0/4165773320 shutdown_connections 2023-11-16T16:42:07.250 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.241+0000 7fd3201edb80 1 --2- 172.21.15.1:0/4165773320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ae8a6b88b0 0x55ae8a6b8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.250 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.241+0000 7fd3201edb80 1 --2- 172.21.15.1:0/4165773320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae8a6b3b60 0x55ae8a6b3f30 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.250 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.241+0000 7fd3201edb80 1 --2- 172.21.15.1:0/4165773320 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8a6b4470 0x55ae8a6b8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.250 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.241+0000 7fd3201edb80 1 -- 172.21.15.1:0/4165773320 >> 172.21.15.1:0/4165773320 conn(0x55ae8a6a9360 msgr2=0x55ae8a6aeb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:07.250 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.241+0000 7fd3201edb80 1 -- 172.21.15.1:0/4165773320 shutdown_connections 2023-11-16T16:42:07.250 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.241+0000 7fd3201edb80 1 -- 172.21.15.1:0/4165773320 wait complete. 2023-11-16T16:42:07.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.242+0000 7fd3201edb80 1 -- start start 2023-11-16T16:42:07.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.242+0000 7fd3201edb80 1 Processor -- start 2023-11-16T16:42:07.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.242+0000 7fd3201edb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae8a6b3b60 0x55ae8a6bffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.242+0000 7fd3201edb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ae8a6b4470 0x55ae8a6bd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.242+0000 7fd3201edb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8a6b88b0 0x55ae8a6bd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.242+0000 7fd3201edb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8a5a68e0 con 0x55ae8a6b3b60 2023-11-16T16:42:07.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.242+0000 7fd3201edb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8a5a6580 con 0x55ae8a6b4470 2023-11-16T16:42:07.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.242+0000 7fd3201edb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8a488920 con 0x55ae8a6b88b0 2023-11-16T16:42:07.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.242+0000 7fd3201edb80 10 client.0 _ll_register_callbacks cb 0x55ae8a5c4780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:42:07.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.242+0000 7fd311dfd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae8a6b3b60 0x55ae8a6bffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.242+0000 7fd311dfd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae8a6b3b60 0x55ae8a6bffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44944/0 (socket says 172.21.15.1:44944) 2023-11-16T16:42:07.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.242+0000 7fd311dfd700 1 -- 172.21.15.1:0/2985457154 learned_addr learned my addr 172.21.15.1:0/2985457154 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:42:07.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.242+0000 7fd3125fe700 1 --2- 172.21.15.1:0/2985457154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8a6b88b0 0x55ae8a6bd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.243+0000 7fd3115fc700 1 --2- 172.21.15.1:0/2985457154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ae8a6b4470 0x55ae8a6bd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.243+0000 7fd311dfd700 1 -- 172.21.15.1:0/2985457154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8a6b88b0 msgr2=0x55ae8a6bd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:07.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.243+0000 7fd311dfd700 1 --2- 172.21.15.1:0/2985457154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8a6b88b0 0x55ae8a6bd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.243+0000 7fd311dfd700 1 -- 172.21.15.1:0/2985457154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ae8a6b4470 msgr2=0x55ae8a6bd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:07.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.243+0000 7fd311dfd700 1 --2- 172.21.15.1:0/2985457154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ae8a6b4470 0x55ae8a6bd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.243+0000 7fd311dfd700 1 -- 172.21.15.1:0/2985457154 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2fc010b10 con 0x55ae8a6b3b60 2023-11-16T16:42:07.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.243+0000 7fd311dfd700 1 --2- 172.21.15.1:0/2985457154 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae8a6b3b60 0x55ae8a6bffa0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc010e30 tx=0x7fd2fc011e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a3c83bfcceb1b21 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:07.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.243+0000 7fd2f57fa700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:42:07.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.243+0000 7fd2f57fa700 1 -- 172.21.15.1:0/2985457154 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc02c070 con 0x55ae8a6b3b60 2023-11-16T16:42:07.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.243+0000 7fd2f57fa700 1 -- 172.21.15.1:0/2985457154 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd2fc017070 con 0x55ae8a6b3b60 2023-11-16T16:42:07.253 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.243+0000 7fd3201edb80 1 -- 172.21.15.1:0/2985457154 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ae8a6a6b20 con 0x55ae8a6b3b60 2023-11-16T16:42:07.253 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.243+0000 7fd2f57fa700 1 -- 172.21.15.1:0/2985457154 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2fc010e90 con 0x55ae8a6b3b60 2023-11-16T16:42:07.253 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.243+0000 7fd3201edb80 10 client.16029 fetch_fsmap learned FSMap version 25 2023-11-16T16:42:07.253 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.243+0000 7fd3201edb80 1 -- 172.21.15.1:0/2985457154 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ae8a5cb190 con 0x55ae8a6b3b60 2023-11-16T16:42:07.253 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.243+0000 7fd2f57fa700 1 -- 172.21.15.1:0/2985457154 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2fc03d020 con 0x55ae8a6b3b60 2023-11-16T16:42:07.253 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.244+0000 7fd2f57fa700 1 -- 172.21.15.1:0/2985457154 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fd2fc004830 con 0x55ae8a6b3b60 2023-11-16T16:42:07.253 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.244+0000 7fd3201edb80 10 client.16029 fetch_fsmap finished waiting for FSMap version 25 2023-11-16T16:42:07.253 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.244+0000 7fd3201edb80 10 client.16029 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-11-16T16:42:07.253 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.244+0000 7fd3201edb80 20 client.16029 populate_metadata read hostname 'smithi001' 2023-11-16T16:42:07.253 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.244+0000 7fd3201edb80 1 --2- 172.21.15.1:0/2985457154 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55ae8a6cd650 0x55ae8a6cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.253 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.244+0000 7fd3201edb80 4 client.16029 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:42:07.253 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.244+0000 7fd3201edb80 1 -- 172.21.15.1:0/2985457154 --> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae8a5c70a0 con 0x55ae8a6cd650 2023-11-16T16:42:07.253 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.244+0000 7fd3115fc700 1 --2- 172.21.15.1:0/2985457154 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55ae8a6cd650 0x55ae8a6cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.254 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.244+0000 7fd3115fc700 1 --2- 172.21.15.1:0/2985457154 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55ae8a6cd650 0x55ae8a6cfa30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b7605f4d52a2fe31 server_cookie=2c7281eb7195c177 in_seq=0 out_seq=0 2023-11-16T16:42:07.254 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.244+0000 7fd2f57fa700 10 client.16029 ms_handle_connect on v2:172.21.15.53:6838/1044753024 2023-11-16T16:42:07.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.247+0000 7fd2f57fa700 1 -- 172.21.15.1:0/2985457154 <== mds.1 v2:172.21.15.53:6838/1044753024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ae8a5c70a0 con 0x55ae8a6cd650 2023-11-16T16:42:07.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.247+0000 7fd2f57fa700 10 client.16029 handle_command_reply: tid=0 2023-11-16T16:42:07.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.247+0000 7fd3201edb80 1 client.16029 shutdown 2023-11-16T16:42:07.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.247+0000 7fd3201edb80 1 -- 172.21.15.1:0/2985457154 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae8a6b3b60 msgr2=0x55ae8a6bffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:07.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.247+0000 7fd3201edb80 1 --2- 172.21.15.1:0/2985457154 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae8a6b3b60 0x55ae8a6bffa0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc010e30 tx=0x7fd2fc011e30 comp rx=0 tx=0).stop 2023-11-16T16:42:07.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.247+0000 7fd3201edb80 1 -- 172.21.15.1:0/2985457154 shutdown_connections 2023-11-16T16:42:07.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.247+0000 7fd3201edb80 1 --2- 172.21.15.1:0/2985457154 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8a6b88b0 0x55ae8a6bd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.247+0000 7fd3201edb80 1 --2- 172.21.15.1:0/2985457154 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae8a6b3b60 0x55ae8a6bffa0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.247+0000 7fd3201edb80 1 --2- 172.21.15.1:0/2985457154 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55ae8a6cd650 0x55ae8a6cfa30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.256 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.247+0000 7fd3201edb80 1 --2- 172.21.15.1:0/2985457154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ae8a6b4470 0x55ae8a6bd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.256 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.247+0000 7fd2f57fa700 0 client.16029 ms_handle_reset on v2:172.21.15.53:6838/1044753024 2023-11-16T16:42:07.256 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.247+0000 7fd3201edb80 1 -- 172.21.15.1:0/2985457154 >> 172.21.15.1:0/2985457154 conn(0x55ae8a6a9360 msgr2=0x55ae8a6b9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:07.256 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.247+0000 7fd3201edb80 1 -- 172.21.15.1:0/2985457154 shutdown_connections 2023-11-16T16:42:07.256 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.247+0000 7fd3201edb80 1 -- 172.21.15.1:0/2985457154 wait complete. 2023-11-16T16:42:07.256 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.247+0000 7fd3201edb80 20 client.16029 trim_cache size 0 max 16384 2023-11-16T16:42:07.264 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.260+0000 7fd3201edb80 1 Processor -- start 2023-11-16T16:42:07.264 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.260+0000 7fd3201edb80 1 -- start start 2023-11-16T16:42:07.264 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.260+0000 7fd3201edb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ae8a6b3b60 0x55ae8a6b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.264 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.260+0000 7fd3201edb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae8a6b4470 0x55ae8a6c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.264 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.260+0000 7fd3201edb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8a6b88b0 0x55ae8a6b6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.265 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.260+0000 7fd3201edb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8a6a6710 con 0x55ae8a6b4470 2023-11-16T16:42:07.265 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.260+0000 7fd3201edb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8a6a8ee0 con 0x55ae8a6b3b60 2023-11-16T16:42:07.265 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.260+0000 7fd3201edb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8a6b7420 con 0x55ae8a6b88b0 2023-11-16T16:42:07.265 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.261+0000 7fd3115fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae8a6b4470 0x55ae8a6c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.265 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.261+0000 7fd3115fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae8a6b4470 0x55ae8a6c49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44956/0 (socket says 172.21.15.1:44956) 2023-11-16T16:42:07.265 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.261+0000 7fd3115fc700 1 -- 172.21.15.1:0/507721587 learned_addr learned my addr 172.21.15.1:0/507721587 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:42:07.265 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.261+0000 7fd3125fe700 1 --2- 172.21.15.1:0/507721587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8a6b88b0 0x55ae8a6b6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.265 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.261+0000 7fd311dfd700 1 --2- 172.21.15.1:0/507721587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ae8a6b3b60 0x55ae8a6b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.265 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.261+0000 7fd3115fc700 1 -- 172.21.15.1:0/507721587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8a6b88b0 msgr2=0x55ae8a6b6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:07.265 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.261+0000 7fd3115fc700 1 --2- 172.21.15.1:0/507721587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8a6b88b0 0x55ae8a6b6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.265 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.261+0000 7fd3115fc700 1 -- 172.21.15.1:0/507721587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ae8a6b3b60 msgr2=0x55ae8a6b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:07.265 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.261+0000 7fd3115fc700 1 --2- 172.21.15.1:0/507721587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ae8a6b3b60 0x55ae8a6b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.265 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.261+0000 7fd3115fc700 1 -- 172.21.15.1:0/507721587 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae8a6aab00 con 0x55ae8a6b4470 2023-11-16T16:42:07.266 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.261+0000 7fd3115fc700 1 --2- 172.21.15.1:0/507721587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae8a6b4470 0x55ae8a6c49d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd308009fd0 tx=0x7fd308005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2285c5786eb82365 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:07.266 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.261+0000 7fd300ff9700 1 -- 172.21.15.1:0/507721587 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd308025040 con 0x55ae8a6b4470 2023-11-16T16:42:07.266 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.261+0000 7fd300ff9700 1 -- 172.21.15.1:0/507721587 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd308003c90 con 0x55ae8a6b4470 2023-11-16T16:42:07.266 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.261+0000 7fd300ff9700 1 -- 172.21.15.1:0/507721587 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30800bda0 con 0x55ae8a6b4470 2023-11-16T16:42:07.266 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.262+0000 7fd3201edb80 1 -- 172.21.15.1:0/507721587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae8a6b4470 msgr2=0x55ae8a6c49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:07.266 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.262+0000 7fd3201edb80 1 --2- 172.21.15.1:0/507721587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae8a6b4470 0x55ae8a6c49d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd308009fd0 tx=0x7fd308005040 comp rx=0 tx=0).stop 2023-11-16T16:42:07.266 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.262+0000 7fd3201edb80 1 -- 172.21.15.1:0/507721587 shutdown_connections 2023-11-16T16:42:07.266 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.262+0000 7fd3201edb80 1 --2- 172.21.15.1:0/507721587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8a6b88b0 0x55ae8a6b6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.266 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.262+0000 7fd3201edb80 1 --2- 172.21.15.1:0/507721587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ae8a6b3b60 0x55ae8a6b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.266 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.262+0000 7fd3201edb80 1 --2- 172.21.15.1:0/507721587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae8a6b4470 0x55ae8a6c49d0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.266 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.262+0000 7fd3201edb80 1 -- 172.21.15.1:0/507721587 >> 172.21.15.1:0/507721587 conn(0x55ae8a6a9360 msgr2=0x55ae8a6c9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:07.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.262+0000 7fd3201edb80 1 -- 172.21.15.1:0/507721587 shutdown_connections 2023-11-16T16:42:07.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.262+0000 7fd3201edb80 1 -- 172.21.15.1:0/507721587 wait complete. 2023-11-16T16:42:07.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.262+0000 7fd3201edb80 1 -- start start 2023-11-16T16:42:07.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.262+0000 7fd3201edb80 1 Processor -- start 2023-11-16T16:42:07.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.263+0000 7fd3201edb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ae8a6b3b60 0x55ae8a6c4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.263+0000 7fd3201edb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8a6b88b0 0x55ae8a6c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.263+0000 7fd3201edb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae8a6c2790 0x55ae8a6c2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.263+0000 7fd3201edb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8a6c3140 con 0x55ae8a6c2790 2023-11-16T16:42:07.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.263+0000 7fd3201edb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8a6c32b0 con 0x55ae8a6b3b60 2023-11-16T16:42:07.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.263+0000 7fd3201edb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae8a6c3470 con 0x55ae8a6b88b0 2023-11-16T16:42:07.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.263+0000 7fd3201edb80 10 client.0 _ll_register_callbacks cb 0x55ae8a5c4780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:42:07.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.263+0000 7fd3125fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae8a6c2790 0x55ae8a6c2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.263+0000 7fd3125fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae8a6c2790 0x55ae8a6c2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44962/0 (socket says 172.21.15.1:44962) 2023-11-16T16:42:07.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.263+0000 7fd3125fe700 1 -- 172.21.15.1:0/1759428958 learned_addr learned my addr 172.21.15.1:0/1759428958 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:42:07.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.263+0000 7fd3115fc700 1 --2- 172.21.15.1:0/1759428958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8a6b88b0 0x55ae8a6c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.263+0000 7fd311dfd700 1 --2- 172.21.15.1:0/1759428958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ae8a6b3b60 0x55ae8a6c4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.263+0000 7fd3115fc700 1 -- 172.21.15.1:0/1759428958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ae8a6b3b60 msgr2=0x55ae8a6c4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:07.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.263+0000 7fd3115fc700 1 --2- 172.21.15.1:0/1759428958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ae8a6b3b60 0x55ae8a6c4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.263+0000 7fd3115fc700 1 -- 172.21.15.1:0/1759428958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae8a6c2790 msgr2=0x55ae8a6c2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:07.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.263+0000 7fd3115fc700 1 --2- 172.21.15.1:0/1759428958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae8a6c2790 0x55ae8a6c2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.263+0000 7fd3115fc700 1 -- 172.21.15.1:0/1759428958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3080054d0 con 0x55ae8a6b88b0 2023-11-16T16:42:07.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.263+0000 7fd3125fe700 1 --2- 172.21.15.1:0/1759428958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae8a6c2790 0x55ae8a6c2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:42:07.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.264+0000 7fd3115fc700 1 --2- 172.21.15.1:0/1759428958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8a6b88b0 0x55ae8a6c48b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x55ae8a6be7a0 tx=0x7fd308005810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1757d5bfb1c56319 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:07.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.264+0000 7fd2f57fa700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-16T16:42:07.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.264+0000 7fd2f57fa700 1 -- 172.21.15.1:0/1759428958 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd308025070 con 0x55ae8a6b88b0 2023-11-16T16:42:07.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.264+0000 7fd2f57fa700 1 -- 172.21.15.1:0/1759428958 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd3080037d0 con 0x55ae8a6b88b0 2023-11-16T16:42:07.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.264+0000 7fd3201edb80 1 -- 172.21.15.1:0/1759428958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2fc015070 con 0x55ae8a6b88b0 2023-11-16T16:42:07.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.264+0000 7fd2f57fa700 1 -- 172.21.15.1:0/1759428958 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3080039b0 con 0x55ae8a6b88b0 2023-11-16T16:42:07.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.264+0000 7fd3201edb80 10 client.25600 fetch_fsmap learned FSMap version 25 2023-11-16T16:42:07.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.264+0000 7fd3201edb80 1 -- 172.21.15.1:0/1759428958 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ae8a432de0 con 0x55ae8a6b88b0 2023-11-16T16:42:07.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.264+0000 7fd2f57fa700 1 -- 172.21.15.1:0/1759428958 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2fc015070 con 0x55ae8a6b88b0 2023-11-16T16:42:07.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.264+0000 7fd2f57fa700 1 -- 172.21.15.1:0/1759428958 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 25) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fd308020070 con 0x55ae8a6b88b0 2023-11-16T16:42:07.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.264+0000 7fd3201edb80 10 client.25600 fetch_fsmap finished waiting for FSMap version 25 2023-11-16T16:42:07.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.264+0000 7fd3201edb80 10 client.25600 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-11-16T16:42:07.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.264+0000 7fd3201edb80 20 client.25600 populate_metadata read hostname 'smithi001' 2023-11-16T16:42:07.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.264+0000 7fd3201edb80 1 --2- 172.21.15.1:0/1759428958 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55ae8a72ad60 0x55ae8a72d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.264+0000 7fd3201edb80 4 client.25600 mds_command: new command op to 24493 tid=0[{"prefix": "damage ls"}] 2023-11-16T16:42:07.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.264+0000 7fd3201edb80 1 -- 172.21.15.1:0/1759428958 --> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ae8a5d3660 con 0x55ae8a72ad60 2023-11-16T16:42:07.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.265+0000 7fd311dfd700 1 --2- 172.21.15.1:0/1759428958 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55ae8a72ad60 0x55ae8a72d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.265+0000 7fd2f57fa700 10 client.25600 ms_handle_connect on v2:172.21.15.53:6838/1044753024 2023-11-16T16:42:07.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.265+0000 7fd311dfd700 1 --2- 172.21.15.1:0/1759428958 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55ae8a72ad60 0x55ae8a72d140 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=afa473cb6ccd5cae server_cookie=ce00309ea4f28d1c in_seq=0 out_seq=0 2023-11-16T16:42:07.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.267+0000 7fd2f57fa700 1 -- 172.21.15.1:0/1759428958 <== mds.1 v2:172.21.15.53:6838/1044753024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ae8a5d3660 con 0x55ae8a72ad60 2023-11-16T16:42:07.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.267+0000 7fd2f57fa700 10 client.25600 handle_command_reply: tid=0 2023-11-16T16:42:07.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.267+0000 7fd3201edb80 1 client.25600 shutdown 2023-11-16T16:42:07.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.267+0000 7fd3201edb80 1 -- 172.21.15.1:0/1759428958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8a6b88b0 msgr2=0x55ae8a6c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:07.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.267+0000 7fd3201edb80 1 --2- 172.21.15.1:0/1759428958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8a6b88b0 0x55ae8a6c48b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x55ae8a6be7a0 tx=0x7fd308005810 comp rx=0 tx=0).stop 2023-11-16T16:42:07.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.267+0000 7fd3201edb80 1 -- 172.21.15.1:0/1759428958 shutdown_connections 2023-11-16T16:42:07.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.267+0000 7fd3201edb80 1 --2- 172.21.15.1:0/1759428958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ae8a6c2790 0x55ae8a6c2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.267+0000 7fd3201edb80 1 --2- 172.21.15.1:0/1759428958 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55ae8a72ad60 0x55ae8a72d140 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.267+0000 7fd3201edb80 1 --2- 172.21.15.1:0/1759428958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ae8a6b3b60 0x55ae8a6c4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.267+0000 7fd3201edb80 1 --2- 172.21.15.1:0/1759428958 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ae8a6b88b0 0x55ae8a6c48b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.267+0000 7fd3201edb80 1 -- 172.21.15.1:0/1759428958 >> 172.21.15.1:0/1759428958 conn(0x55ae8a6a9360 msgr2=0x55ae8a6b7830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:07.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.267+0000 7fd2f57fa700 0 client.25600 ms_handle_reset on v2:172.21.15.53:6838/1044753024 2023-11-16T16:42:07.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.268+0000 7fd3201edb80 1 -- 172.21.15.1:0/1759428958 shutdown_connections 2023-11-16T16:42:07.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.268+0000 7fd3201edb80 1 -- 172.21.15.1:0/1759428958 wait complete. 2023-11-16T16:42:07.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.268+0000 7fd3201edb80 20 client.25600 trim_cache size 0 max 16384 2023-11-16T16:42:07.275 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2023-11-16T16:42:07.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.268+0000 7fd314061700 1 -- 172.21.15.1:0/423866184 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fd2f80a1340 msgr2=0x7fd2f80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:07.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.268+0000 7fd314061700 1 --2- 172.21.15.1:0/423866184 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fd2f80a1340 0x7fd2f80a37f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc005950 tx=0x7fd2fc00b410 comp rx=0 tx=0).stop 2023-11-16T16:42:07.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.268+0000 7fd314061700 1 -- 172.21.15.1:0/423866184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd30c104140 msgr2=0x7fd30c1a2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:07.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.268+0000 7fd314061700 1 --2- 172.21.15.1:0/423866184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd30c104140 0x7fd30c1a2260 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd30400ef90 tx=0x7fd30400c3a0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.268+0000 7fd314061700 1 -- 172.21.15.1:0/423866184 shutdown_connections 2023-11-16T16:42:07.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.268+0000 7fd314061700 1 --2- 172.21.15.1:0/423866184 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fd2f80a1340 0x7fd2f80a37f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.268+0000 7fd314061700 1 --2- 172.21.15.1:0/423866184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd30c104140 0x7fd30c1a2260 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.268+0000 7fd314061700 1 --2- 172.21.15.1:0/423866184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd30c0ff8f0 0x7fd30c1971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.268+0000 7fd314061700 1 --2- 172.21.15.1:0/423866184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd30c1002a0 0x7fd30c197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.268+0000 7fd314061700 1 -- 172.21.15.1:0/423866184 >> 172.21.15.1:0/423866184 conn(0x7fd30c075880 msgr2=0x7fd30c0fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:07.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.268+0000 7fd314061700 1 -- 172.21.15.1:0/423866184 shutdown_connections 2023-11-16T16:42:07.277 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.268+0000 7fd314061700 1 -- 172.21.15.1:0/423866184 wait complete. 2023-11-16T16:42:07.282 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-16T16:42:07.416 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.414+0000 7fb7cbbc9700 1 -- 172.21.15.1:0/424779163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7c4101490 msgr2=0x7fb7c4101860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:07.416 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.414+0000 7fb7cbbc9700 1 --2- 172.21.15.1:0/424779163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7c4101490 0x7fb7c4101860 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0009990 tx=0x7fb7c0009ca0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.416 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.414+0000 7fb7cbbc9700 1 -- 172.21.15.1:0/424779163 shutdown_connections 2023-11-16T16:42:07.416 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.414+0000 7fb7cbbc9700 1 --2- 172.21.15.1:0/424779163 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c410a860 0x7fb7c410cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.416 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.414+0000 7fb7cbbc9700 1 --2- 172.21.15.1:0/424779163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7c4101490 0x7fb7c4101860 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.416 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.414+0000 7fb7cbbc9700 1 --2- 172.21.15.1:0/424779163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7c4101da0 0x7fb7c410a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.416 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.414+0000 7fb7cbbc9700 1 -- 172.21.15.1:0/424779163 >> 172.21.15.1:0/424779163 conn(0x7fb7c4076370 msgr2=0x7fb7c4076770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:07.416 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.414+0000 7fb7cbbc9700 1 -- 172.21.15.1:0/424779163 shutdown_connections 2023-11-16T16:42:07.416 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.414+0000 7fb7cbbc9700 1 -- 172.21.15.1:0/424779163 wait complete. 2023-11-16T16:42:07.419 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.415+0000 7fb7cbbc9700 1 Processor -- start 2023-11-16T16:42:07.419 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.415+0000 7fb7cbbc9700 1 -- start start 2023-11-16T16:42:07.419 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.415+0000 7fb7cbbc9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c4101490 0x7fb7c4196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.419 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.415+0000 7fb7cbbc9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7c4101da0 0x7fb7c4196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.419 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.415+0000 7fb7cbbc9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7c410a860 0x7fb7c419acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.419 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.415+0000 7fb7cbbc9700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c419b2f0 con 0x7fb7c4101da0 2023-11-16T16:42:07.419 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.415+0000 7fb7cbbc9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c419b460 con 0x7fb7c410a860 2023-11-16T16:42:07.420 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.415+0000 7fb7cbbc9700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c419b620 con 0x7fb7c4101490 2023-11-16T16:42:07.420 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.415+0000 7fb7c9164700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7c4101da0 0x7fb7c4196850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.420 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.415+0000 7fb7c9965700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c4101490 0x7fb7c4196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.420 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.415+0000 7fb7c9164700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7c4101da0 0x7fb7c4196850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44974/0 (socket says 172.21.15.1:44974) 2023-11-16T16:42:07.420 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.415+0000 7fb7c9965700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c4101490 0x7fb7c4196310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:41654/0 (socket says 172.21.15.1:41654) 2023-11-16T16:42:07.420 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.415+0000 7fb7c9164700 1 -- 172.21.15.1:0/2214301072 learned_addr learned my addr 172.21.15.1:0/2214301072 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:42:07.420 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.415+0000 7fb7ca166700 1 --2- 172.21.15.1:0/2214301072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7c410a860 0x7fb7c419acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.420 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.416+0000 7fb7c9164700 1 -- 172.21.15.1:0/2214301072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c4101490 msgr2=0x7fb7c4196310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:07.420 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.416+0000 7fb7c9164700 1 --2- 172.21.15.1:0/2214301072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c4101490 0x7fb7c4196310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.420 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.416+0000 7fb7c9164700 1 -- 172.21.15.1:0/2214301072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7c410a860 msgr2=0x7fb7c419acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:07.421 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.416+0000 7fb7c9164700 1 --2- 172.21.15.1:0/2214301072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7c410a860 0x7fb7c419acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.421 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.416+0000 7fb7c9164700 1 -- 172.21.15.1:0/2214301072 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c00095f0 con 0x7fb7c4101da0 2023-11-16T16:42:07.421 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.416+0000 7fb7c9965700 1 --2- 172.21.15.1:0/2214301072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c4101490 0x7fb7c4196310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:42:07.421 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.416+0000 7fb7c9164700 1 --2- 172.21.15.1:0/2214301072 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7c4101da0 0x7fb7c4196850 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb7b800e9a0 tx=0x7fb7b800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:07.421 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.416+0000 7fb7b6ffd700 1 -- 172.21.15.1:0/2214301072 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8009790 con 0x7fb7c4101da0 2023-11-16T16:42:07.421 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.416+0000 7fb7cbbc9700 1 -- 172.21.15.1:0/2214301072 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7c41102c0 con 0x7fb7c4101da0 2023-11-16T16:42:07.421 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.416+0000 7fb7b6ffd700 1 -- 172.21.15.1:0/2214301072 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7b8010430 con 0x7fb7c4101da0 2023-11-16T16:42:07.421 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.416+0000 7fb7b6ffd700 1 -- 172.21.15.1:0/2214301072 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8018680 con 0x7fb7c4101da0 2023-11-16T16:42:07.421 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.416+0000 7fb7cbbc9700 1 -- 172.21.15.1:0/2214301072 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7c4110130 con 0x7fb7c4101da0 2023-11-16T16:42:07.428 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.418+0000 7fb7b6ffd700 1 -- 172.21.15.1:0/2214301072 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fb7b80105a0 con 0x7fb7c4101da0 2023-11-16T16:42:07.428 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.419+0000 7fb7b6ffd700 1 --2- 172.21.15.1:0/2214301072 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fb7b00a12c0 0x7fb7b00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.428 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.419+0000 7fb7b6ffd700 1 -- 172.21.15.1:0/2214301072 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fb7b8014070 con 0x7fb7c4101da0 2023-11-16T16:42:07.428 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.419+0000 7fb7c9965700 1 --2- 172.21.15.1:0/2214301072 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fb7b00a12c0 0x7fb7b00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.428 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.419+0000 7fb7c9965700 1 --2- 172.21.15.1:0/2214301072 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fb7b00a12c0 0x7fb7b00a3770 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb7c000b2e0 tx=0x7fb7c0005fd0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:07.428 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.419+0000 7fb7d7d55b80 1 Processor -- start 2023-11-16T16:42:07.428 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.420+0000 7fb7d7d55b80 1 -- start start 2023-11-16T16:42:07.428 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.420+0000 7fb7d7d55b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55849350bb60 0x55849350bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.428 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.420+0000 7fb7d7d55b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55849350c470 0x558493510370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.428 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.420+0000 7fb7d7d55b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584935108b0 0x558493510d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.428 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.420+0000 7fb7d7d55b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558493411060 con 0x5584935108b0 2023-11-16T16:42:07.429 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.420+0000 7fb7d7d55b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584934638a0 con 0x55849350bb60 2023-11-16T16:42:07.429 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.420+0000 7fb7d7d55b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55849345cd30 con 0x55849350c470 2023-11-16T16:42:07.429 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.420+0000 7fb7ca166700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584935108b0 0x558493510d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.429 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.420+0000 7fb7c9965700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55849350bb60 0x55849350bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.429 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.420+0000 7fb7c9164700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55849350c470 0x558493510370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.429 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.420+0000 7fb7ca166700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584935108b0 0x558493510d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44984/0 (socket says 172.21.15.1:44984) 2023-11-16T16:42:07.429 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.420+0000 7fb7c9965700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55849350bb60 0x55849350bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.1:39166/0 (socket says 172.21.15.1:39166) 2023-11-16T16:42:07.429 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.420+0000 7fb7ca166700 1 -- 172.21.15.1:0/3877250640 learned_addr learned my addr 172.21.15.1:0/3877250640 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:42:07.429 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.420+0000 7fb7ca166700 1 -- 172.21.15.1:0/3877250640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55849350c470 msgr2=0x558493510370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:07.429 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.420+0000 7fb7ca166700 1 --2- 172.21.15.1:0/3877250640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55849350c470 0x558493510370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.429 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.420+0000 7fb7ca166700 1 -- 172.21.15.1:0/3877250640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55849350bb60 msgr2=0x55849350bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:07.429 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.420+0000 7fb7ca166700 1 --2- 172.21.15.1:0/3877250640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55849350bb60 0x55849350bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.420+0000 7fb7ca166700 1 -- 172.21.15.1:0/3877250640 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584932de630 con 0x5584935108b0 2023-11-16T16:42:07.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.420+0000 7fb7c9164700 1 --2- 172.21.15.1:0/3877250640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55849350c470 0x558493510370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:42:07.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.420+0000 7fb7c9965700 1 --2- 172.21.15.1:0/3877250640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55849350bb60 0x55849350bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:42:07.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.421+0000 7fb7ca166700 1 --2- 172.21.15.1:0/3877250640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584935108b0 0x558493510d00 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc011680 tx=0x7fb7bc009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10336461fb397c99 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:07.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.421+0000 7fb7affff700 1 -- 172.21.15.1:0/3877250640 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc00e040 con 0x5584935108b0 2023-11-16T16:42:07.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.421+0000 7fb7affff700 1 -- 172.21.15.1:0/3877250640 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7bc00f040 con 0x5584935108b0 2023-11-16T16:42:07.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.421+0000 7fb7affff700 1 -- 172.21.15.1:0/3877250640 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc004860 con 0x5584935108b0 2023-11-16T16:42:07.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.421+0000 7fb7d7d55b80 1 -- 172.21.15.1:0/3877250640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584935108b0 msgr2=0x558493510d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:07.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.421+0000 7fb7d7d55b80 1 --2- 172.21.15.1:0/3877250640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584935108b0 0x558493510d00 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc011680 tx=0x7fb7bc009920 comp rx=0 tx=0).stop 2023-11-16T16:42:07.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.421+0000 7fb7d7d55b80 1 -- 172.21.15.1:0/3877250640 shutdown_connections 2023-11-16T16:42:07.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.421+0000 7fb7d7d55b80 1 --2- 172.21.15.1:0/3877250640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584935108b0 0x558493510d00 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.431 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.421+0000 7fb7d7d55b80 1 --2- 172.21.15.1:0/3877250640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55849350bb60 0x55849350bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.431 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.421+0000 7fb7d7d55b80 1 --2- 172.21.15.1:0/3877250640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55849350c470 0x558493510370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.431 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.421+0000 7fb7d7d55b80 1 -- 172.21.15.1:0/3877250640 >> 172.21.15.1:0/3877250640 conn(0x558493501360 msgr2=0x558493506b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:07.431 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.422+0000 7fb7d7d55b80 1 -- 172.21.15.1:0/3877250640 shutdown_connections 2023-11-16T16:42:07.431 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.422+0000 7fb7d7d55b80 1 -- 172.21.15.1:0/3877250640 wait complete. 2023-11-16T16:42:07.431 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.422+0000 7fb7d7d55b80 1 -- start start 2023-11-16T16:42:07.431 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.422+0000 7fb7d7d55b80 1 Processor -- start 2023-11-16T16:42:07.431 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.422+0000 7fb7d7d55b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55849350bb60 0x558493518030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.431 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.422+0000 7fb7d7d55b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55849350c470 0x558493515320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.431 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.422+0000 7fb7d7d55b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584935108b0 0x558493515980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.431 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.422+0000 7fb7d7d55b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584933fe8e0 con 0x55849350c470 2023-11-16T16:42:07.431 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.422+0000 7fb7d7d55b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584933fe580 con 0x5584935108b0 2023-11-16T16:42:07.432 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.422+0000 7fb7d7d55b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584932e0920 con 0x55849350bb60 2023-11-16T16:42:07.432 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.423+0000 7fb7c9164700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55849350c470 0x558493515320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.432 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.423+0000 7fb7d7d55b80 10 client.0 _ll_register_callbacks cb 0x55849341c780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:42:07.432 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.423+0000 7fb7c9164700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55849350c470 0x558493515320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44988/0 (socket says 172.21.15.1:44988) 2023-11-16T16:42:07.432 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.423+0000 7fb7c9164700 1 -- 172.21.15.1:0/2197425974 learned_addr learned my addr 172.21.15.1:0/2197425974 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:42:07.432 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.423+0000 7fb7c9965700 1 --2- 172.21.15.1:0/2197425974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55849350bb60 0x558493518030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.432 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.423+0000 7fb7ca166700 1 --2- 172.21.15.1:0/2197425974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584935108b0 0x558493515980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.432 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.423+0000 7fb7c9965700 1 -- 172.21.15.1:0/2197425974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584935108b0 msgr2=0x558493515980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:07.432 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.423+0000 7fb7c9965700 1 --2- 172.21.15.1:0/2197425974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584935108b0 0x558493515980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.432 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.423+0000 7fb7c9965700 1 -- 172.21.15.1:0/2197425974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55849350c470 msgr2=0x558493515320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:07.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.423+0000 7fb7c9965700 1 --2- 172.21.15.1:0/2197425974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55849350c470 0x558493515320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.423+0000 7fb7c9965700 1 -- 172.21.15.1:0/2197425974 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7bc011330 con 0x55849350bb60 2023-11-16T16:42:07.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.423+0000 7fb7c9164700 1 --2- 172.21.15.1:0/2197425974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55849350c470 0x558493515320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:42:07.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.423+0000 7fb7ca166700 1 --2- 172.21.15.1:0/2197425974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584935108b0 0x558493515980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:42:07.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.423+0000 7fb7c9965700 1 --2- 172.21.15.1:0/2197425974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55849350bb60 0x558493518030 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb7c000b6f0 tx=0x7fb7c001a050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c22ad3200a6dbee8 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:07.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.423+0000 7fb7ad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-16T16:42:07.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.423+0000 7fb7ad7fa700 1 -- 172.21.15.1:0/2197425974 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0034070 con 0x55849350bb60 2023-11-16T16:42:07.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.423+0000 7fb7ad7fa700 1 -- 172.21.15.1:0/2197425974 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fb7c0004830 con 0x55849350bb60 2023-11-16T16:42:07.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.423+0000 7fb7d7d55b80 1 -- 172.21.15.1:0/2197425974 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5584934feb20 con 0x55849350bb60 2023-11-16T16:42:07.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.424+0000 7fb7ad7fa700 1 -- 172.21.15.1:0/2197425974 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7c002b050 con 0x55849350bb60 2023-11-16T16:42:07.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.424+0000 7fb7d7d55b80 10 client.25612 fetch_fsmap learned FSMap version 25 2023-11-16T16:42:07.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.424+0000 7fb7d7d55b80 1 -- 172.21.15.1:0/2197425974 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558493423190 con 0x55849350bb60 2023-11-16T16:42:07.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.424+0000 7fb7ad7fa700 1 -- 172.21.15.1:0/2197425974 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5584934feb20 con 0x55849350bb60 2023-11-16T16:42:07.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.424+0000 7fb7ad7fa700 1 -- 172.21.15.1:0/2197425974 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 25) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fb7c0037020 con 0x55849350bb60 2023-11-16T16:42:07.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.424+0000 7fb7d7d55b80 10 client.25612 fetch_fsmap finished waiting for FSMap version 25 2023-11-16T16:42:07.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.424+0000 7fb7d7d55b80 10 client.25612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:42:07.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.424+0000 7fb7d7d55b80 20 client.25612 populate_metadata read hostname 'smithi001' 2023-11-16T16:42:07.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.424+0000 7fb7d7d55b80 1 --2- 172.21.15.1:0/2197425974 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x558493525920 0x558493527d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.424+0000 7fb7d7d55b80 4 client.25612 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:42:07.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.424+0000 7fb7d7d55b80 1 -- 172.21.15.1:0/2197425974 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55849341f0a0 con 0x558493525920 2023-11-16T16:42:07.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.424+0000 7fb7c9164700 1 --2- 172.21.15.1:0/2197425974 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x558493525920 0x558493527d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.425+0000 7fb7ad7fa700 10 client.25612 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:42:07.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.425+0000 7fb7c9164700 1 --2- 172.21.15.1:0/2197425974 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x558493525920 0x558493527d30 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7daad85340170bc9 server_cookie=d4c2860b0ad5cae8 in_seq=0 out_seq=0 2023-11-16T16:42:07.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.425+0000 7fb7ad7fa700 1 -- 172.21.15.1:0/2197425974 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55849341f0a0 con 0x558493525920 2023-11-16T16:42:07.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.425+0000 7fb7ad7fa700 10 client.25612 handle_command_reply: tid=0 2023-11-16T16:42:07.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.425+0000 7fb7d7d55b80 1 client.25612 shutdown 2023-11-16T16:42:07.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.425+0000 7fb7d7d55b80 1 -- 172.21.15.1:0/2197425974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55849350bb60 msgr2=0x558493518030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:07.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.425+0000 7fb7d7d55b80 1 --2- 172.21.15.1:0/2197425974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55849350bb60 0x558493518030 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb7c000b6f0 tx=0x7fb7c001a050 comp rx=0 tx=0).stop 2023-11-16T16:42:07.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.425+0000 7fb7d7d55b80 1 -- 172.21.15.1:0/2197425974 shutdown_connections 2023-11-16T16:42:07.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.426+0000 7fb7d7d55b80 1 --2- 172.21.15.1:0/2197425974 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x558493525920 0x558493527d30 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.426+0000 7fb7ad7fa700 0 client.25612 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:42:07.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.426+0000 7fb7d7d55b80 1 --2- 172.21.15.1:0/2197425974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584935108b0 0x558493515980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.426+0000 7fb7d7d55b80 1 --2- 172.21.15.1:0/2197425974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55849350bb60 0x558493518030 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.436 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.426+0000 7fb7d7d55b80 1 --2- 172.21.15.1:0/2197425974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55849350c470 0x558493515320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.436 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.426+0000 7fb7d7d55b80 1 -- 172.21.15.1:0/2197425974 >> 172.21.15.1:0/2197425974 conn(0x558493501360 msgr2=0x558493511d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:07.436 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.426+0000 7fb7d7d55b80 1 -- 172.21.15.1:0/2197425974 shutdown_connections 2023-11-16T16:42:07.436 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.426+0000 7fb7d7d55b80 1 -- 172.21.15.1:0/2197425974 wait complete. 2023-11-16T16:42:07.436 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.426+0000 7fb7d7d55b80 20 client.25612 trim_cache size 0 max 16384 2023-11-16T16:42:07.443 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.439+0000 7fb7d7d55b80 1 Processor -- start 2023-11-16T16:42:07.443 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.439+0000 7fb7d7d55b80 1 -- start start 2023-11-16T16:42:07.443 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.439+0000 7fb7d7d55b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55849350bb60 0x55849351c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.443 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.439+0000 7fb7d7d55b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55849350c470 0x558493517bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.443 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.439+0000 7fb7d7d55b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584935108b0 0x558493506560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.443 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.439+0000 7fb7d7d55b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584934fe9a0 con 0x55849350c470 2023-11-16T16:42:07.443 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.439+0000 7fb7d7d55b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558493513880 con 0x5584935108b0 2023-11-16T16:42:07.443 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.439+0000 7fb7d7d55b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55849350f450 con 0x55849350bb60 2023-11-16T16:42:07.443 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.439+0000 7fb7c9164700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55849350c470 0x558493517bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.443 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.439+0000 7fb7c9164700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55849350c470 0x558493517bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44996/0 (socket says 172.21.15.1:44996) 2023-11-16T16:42:07.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.439+0000 7fb7c9965700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55849350bb60 0x55849351c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.439+0000 7fb7c9164700 1 -- 172.21.15.1:0/1230065889 learned_addr learned my addr 172.21.15.1:0/1230065889 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:42:07.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.440+0000 7fb7ca166700 1 --2- 172.21.15.1:0/1230065889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584935108b0 0x558493506560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.440+0000 7fb7c9164700 1 -- 172.21.15.1:0/1230065889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55849350bb60 msgr2=0x55849351c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:07.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.440+0000 7fb7c9164700 1 --2- 172.21.15.1:0/1230065889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55849350bb60 0x55849351c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.440+0000 7fb7c9164700 1 -- 172.21.15.1:0/1230065889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584935108b0 msgr2=0x558493506560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:07.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.440+0000 7fb7c9164700 1 --2- 172.21.15.1:0/1230065889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584935108b0 0x558493506560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.440+0000 7fb7c9164700 1 -- 172.21.15.1:0/1230065889 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558493502b00 con 0x55849350c470 2023-11-16T16:42:07.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.440+0000 7fb7c9164700 1 --2- 172.21.15.1:0/1230065889 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55849350c470 0x558493517bd0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb7b80ca5d0 tx=0x7fb7b8010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f5b52216b6e9ca7 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:07.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.440+0000 7fb7b4dac700 1 -- 172.21.15.1:0/1230065889 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b80a2400 con 0x55849350c470 2023-11-16T16:42:07.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.440+0000 7fb7b4dac700 1 -- 172.21.15.1:0/1230065889 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7b8098350 con 0x55849350c470 2023-11-16T16:42:07.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.440+0000 7fb7b4dac700 1 -- 172.21.15.1:0/1230065889 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b80a21d0 con 0x55849350c470 2023-11-16T16:42:07.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.441+0000 7fb7d7d55b80 1 -- 172.21.15.1:0/1230065889 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55849350c470 msgr2=0x558493517bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:07.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.441+0000 7fb7d7d55b80 1 --2- 172.21.15.1:0/1230065889 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55849350c470 0x558493517bd0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb7b80ca5d0 tx=0x7fb7b8010c80 comp rx=0 tx=0).stop 2023-11-16T16:42:07.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.441+0000 7fb7d7d55b80 1 -- 172.21.15.1:0/1230065889 shutdown_connections 2023-11-16T16:42:07.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.441+0000 7fb7d7d55b80 1 --2- 172.21.15.1:0/1230065889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584935108b0 0x558493506560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.441+0000 7fb7d7d55b80 1 --2- 172.21.15.1:0/1230065889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55849350bb60 0x55849351c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.441+0000 7fb7d7d55b80 1 --2- 172.21.15.1:0/1230065889 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55849350c470 0x558493517bd0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.441+0000 7fb7d7d55b80 1 -- 172.21.15.1:0/1230065889 >> 172.21.15.1:0/1230065889 conn(0x558493501360 msgr2=0x55849350e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:07.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.441+0000 7fb7d7d55b80 1 -- 172.21.15.1:0/1230065889 shutdown_connections 2023-11-16T16:42:07.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.441+0000 7fb7d7d55b80 1 -- 172.21.15.1:0/1230065889 wait complete. 2023-11-16T16:42:07.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.441+0000 7fb7d7d55b80 1 -- start start 2023-11-16T16:42:07.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.441+0000 7fb7d7d55b80 1 Processor -- start 2023-11-16T16:42:07.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.441+0000 7fb7d7d55b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55849350bb60 0x55849351a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.442+0000 7fb7d7d55b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55849350c470 0x55849351aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.442+0000 7fb7d7d55b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584935108b0 0x55849351b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.442+0000 7fb7d7d55b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55849357cc30 con 0x55849350bb60 2023-11-16T16:42:07.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.442+0000 7fb7d7d55b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55849357cda0 con 0x55849350c470 2023-11-16T16:42:07.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.442+0000 7fb7d7d55b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55849357cf60 con 0x5584935108b0 2023-11-16T16:42:07.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.442+0000 7fb7d7d55b80 10 client.0 _ll_register_callbacks cb 0x7fb7c000f770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:42:07.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.442+0000 7fb7c9965700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55849350bb60 0x55849351a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.442+0000 7fb7c9965700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55849350bb60 0x55849351a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45000/0 (socket says 172.21.15.1:45000) 2023-11-16T16:42:07.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.442+0000 7fb7c9965700 1 -- 172.21.15.1:0/3962822545 learned_addr learned my addr 172.21.15.1:0/3962822545 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:42:07.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.442+0000 7fb7c9164700 1 --2- 172.21.15.1:0/3962822545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55849350c470 0x55849351aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.450 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.442+0000 7fb7ca166700 1 --2- 172.21.15.1:0/3962822545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584935108b0 0x55849351b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.450 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.442+0000 7fb7c9965700 1 -- 172.21.15.1:0/3962822545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584935108b0 msgr2=0x55849351b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:07.450 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.442+0000 7fb7c9965700 1 --2- 172.21.15.1:0/3962822545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584935108b0 0x55849351b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.450 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.442+0000 7fb7c9965700 1 -- 172.21.15.1:0/3962822545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55849350c470 msgr2=0x55849351aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:07.450 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.442+0000 7fb7c9965700 1 --2- 172.21.15.1:0/3962822545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55849350c470 0x55849351aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.450 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.442+0000 7fb7c9965700 1 -- 172.21.15.1:0/3962822545 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7c41102c0 con 0x55849350bb60 2023-11-16T16:42:07.450 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.442+0000 7fb7ca166700 1 --2- 172.21.15.1:0/3962822545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584935108b0 0x55849351b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:42:07.450 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.442+0000 7fb7c9965700 1 --2- 172.21.15.1:0/3962822545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55849350bb60 0x55849351a990 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fb7c00093d0 tx=0x7fb7c0026940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b373ecd6b1b6fec4 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:07.450 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.442+0000 7fb7ad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:42:07.450 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.442+0000 7fb7ad7fa700 1 -- 172.21.15.1:0/3962822545 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0034040 con 0x55849350bb60 2023-11-16T16:42:07.450 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.442+0000 7fb7ad7fa700 1 -- 172.21.15.1:0/3962822545 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fb7c000fe70 con 0x55849350bb60 2023-11-16T16:42:07.451 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.442+0000 7fb7d7d55b80 1 -- 172.21.15.1:0/3962822545 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb7bc00f040 con 0x55849350bb60 2023-11-16T16:42:07.451 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.442+0000 7fb7ad7fa700 1 -- 172.21.15.1:0/3962822545 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7c002b040 con 0x55849350bb60 2023-11-16T16:42:07.451 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.442+0000 7fb7d7d55b80 10 client.16053 fetch_fsmap learned FSMap version 25 2023-11-16T16:42:07.451 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.442+0000 7fb7d7d55b80 1 -- 172.21.15.1:0/3962822545 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55849328ade0 con 0x55849350bb60 2023-11-16T16:42:07.451 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.443+0000 7fb7ad7fa700 1 -- 172.21.15.1:0/3962822545 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7bc00f040 con 0x55849350bb60 2023-11-16T16:42:07.451 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.443+0000 7fb7ad7fa700 1 -- 172.21.15.1:0/3962822545 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fb7c000fb90 con 0x55849350bb60 2023-11-16T16:42:07.451 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.443+0000 7fb7d7d55b80 10 client.16053 fetch_fsmap finished waiting for FSMap version 25 2023-11-16T16:42:07.451 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.443+0000 7fb7d7d55b80 10 client.16053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:42:07.451 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.443+0000 7fb7d7d55b80 20 client.16053 populate_metadata read hostname 'smithi001' 2023-11-16T16:42:07.451 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.443+0000 7fb7d7d55b80 1 --2- 172.21.15.1:0/3962822545 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55849357e280 0x558493580690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:07.451 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.443+0000 7fb7d7d55b80 4 client.16053 mds_command: new command op to 24521 tid=0[{"prefix": "damage ls"}] 2023-11-16T16:42:07.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.443+0000 7fb7d7d55b80 1 -- 172.21.15.1:0/3962822545 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55849342b660 con 0x55849357e280 2023-11-16T16:42:07.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.443+0000 7fb7c9164700 1 --2- 172.21.15.1:0/3962822545 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55849357e280 0x558493580690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:07.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.443+0000 7fb7ad7fa700 10 client.16053 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:42:07.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.443+0000 7fb7c9164700 1 --2- 172.21.15.1:0/3962822545 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55849357e280 0x558493580690 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8ecdeaa0a6d1c33 server_cookie=12995869d894f39 in_seq=0 out_seq=0 2023-11-16T16:42:07.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.443+0000 7fb7ad7fa700 1 -- 172.21.15.1:0/3962822545 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55849342b660 con 0x55849357e280 2023-11-16T16:42:07.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.443+0000 7fb7ad7fa700 10 client.16053 handle_command_reply: tid=0 2023-11-16T16:42:07.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.443+0000 7fb7d7d55b80 1 client.16053 shutdown 2023-11-16T16:42:07.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.444+0000 7fb7d7d55b80 1 -- 172.21.15.1:0/3962822545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55849350bb60 msgr2=0x55849351a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:07.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.444+0000 7fb7d7d55b80 1 --2- 172.21.15.1:0/3962822545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55849350bb60 0x55849351a990 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fb7c00093d0 tx=0x7fb7c0026940 comp rx=0 tx=0).stop 2023-11-16T16:42:07.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.444+0000 7fb7d7d55b80 1 -- 172.21.15.1:0/3962822545 shutdown_connections 2023-11-16T16:42:07.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.444+0000 7fb7d7d55b80 1 --2- 172.21.15.1:0/3962822545 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55849357e280 0x558493580690 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.444+0000 7fb7d7d55b80 1 --2- 172.21.15.1:0/3962822545 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5584935108b0 0x55849351b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.444+0000 7fb7d7d55b80 1 --2- 172.21.15.1:0/3962822545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55849350bb60 0x55849351a990 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.444+0000 7fb7ad7fa700 0 client.16053 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:42:07.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.444+0000 7fb7d7d55b80 1 --2- 172.21.15.1:0/3962822545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55849350c470 0x55849351aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.444+0000 7fb7d7d55b80 1 -- 172.21.15.1:0/3962822545 >> 172.21.15.1:0/3962822545 conn(0x558493501360 msgr2=0x55849350f860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:07.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.444+0000 7fb7d7d55b80 1 -- 172.21.15.1:0/3962822545 shutdown_connections 2023-11-16T16:42:07.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.444+0000 7fb7d7d55b80 1 -- 172.21.15.1:0/3962822545 wait complete. 2023-11-16T16:42:07.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.444+0000 7fb7d7d55b80 20 client.16053 trim_cache size 0 max 16384 2023-11-16T16:42:07.453 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2023-11-16T16:42:07.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.444+0000 7fb7cbbc9700 1 -- 172.21.15.1:0/2214301072 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fb7b00a12c0 msgr2=0x7fb7b00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:07.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.444+0000 7fb7cbbc9700 1 --2- 172.21.15.1:0/2214301072 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fb7b00a12c0 0x7fb7b00a3770 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb7c000b2e0 tx=0x7fb7c0005fd0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.444+0000 7fb7cbbc9700 1 -- 172.21.15.1:0/2214301072 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7c4101da0 msgr2=0x7fb7c4196850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:07.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.444+0000 7fb7cbbc9700 1 --2- 172.21.15.1:0/2214301072 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7c4101da0 0x7fb7c4196850 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb7b800e9a0 tx=0x7fb7b800ecb0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.444+0000 7fb7cbbc9700 1 -- 172.21.15.1:0/2214301072 shutdown_connections 2023-11-16T16:42:07.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.444+0000 7fb7cbbc9700 1 --2- 172.21.15.1:0/2214301072 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fb7b00a12c0 0x7fb7b00a3770 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.444+0000 7fb7cbbc9700 1 --2- 172.21.15.1:0/2214301072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7c410a860 0x7fb7c419acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.444+0000 7fb7cbbc9700 1 --2- 172.21.15.1:0/2214301072 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7c4101490 0x7fb7c4196310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.444+0000 7fb7cbbc9700 1 --2- 172.21.15.1:0/2214301072 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7c4101da0 0x7fb7c4196850 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:07.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.444+0000 7fb7cbbc9700 1 -- 172.21.15.1:0/2214301072 >> 172.21.15.1:0/2214301072 conn(0x7fb7c4076370 msgr2=0x7fb7c4102230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:07.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.444+0000 7fb7cbbc9700 1 -- 172.21.15.1:0/2214301072 shutdown_connections 2023-11-16T16:42:07.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:07.445+0000 7fb7cbbc9700 1 -- 172.21.15.1:0/2214301072 wait complete. 2023-11-16T16:42:07.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:07 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1486724791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:42:08.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:07 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1486724791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:42:08.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:07 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1486724791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:42:08.460 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-16T16:42:08.460 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-16T16:42:08.597 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.595+0000 7f6147e2c700 1 -- 172.21.15.1:0/2993184759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61401002a0 msgr2=0x7f614010bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:08.597 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.595+0000 7f6147e2c700 1 --2- 172.21.15.1:0/2993184759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61401002a0 0x7f614010bfd0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f613800b0d0 tx=0x7f613800b3e0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.597 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.595+0000 7f6147e2c700 1 -- 172.21.15.1:0/2993184759 shutdown_connections 2023-11-16T16:42:08.597 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.595+0000 7f6147e2c700 1 --2- 172.21.15.1:0/2993184759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61401002a0 0x7f614010bfd0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.597 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.595+0000 7f6147e2c700 1 --2- 172.21.15.1:0/2993184759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6140104140 0x7f6140104510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.598 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.595+0000 7f6147e2c700 1 --2- 172.21.15.1:0/2993184759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f61400ff8f0 0x7f61400ffd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.598 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.595+0000 7f6147e2c700 1 -- 172.21.15.1:0/2993184759 >> 172.21.15.1:0/2993184759 conn(0x7f6140075880 msgr2=0x7f6140075c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:08.598 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.595+0000 7f6147e2c700 1 -- 172.21.15.1:0/2993184759 shutdown_connections 2023-11-16T16:42:08.598 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.595+0000 7f6147e2c700 1 -- 172.21.15.1:0/2993184759 wait complete. 2023-11-16T16:42:08.600 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.596+0000 7f6147e2c700 1 Processor -- start 2023-11-16T16:42:08.600 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.596+0000 7f6147e2c700 1 -- start start 2023-11-16T16:42:08.600 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.596+0000 7f6147e2c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61400ff8f0 0x7f61401963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:08.600 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.596+0000 7f6147e2c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61401002a0 0x7f61401968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:08.600 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.596+0000 7f6147e2c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6140104140 0x7f614019ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:08.600 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.596+0000 7f6147e2c700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614019b380 con 0x7f61400ff8f0 2023-11-16T16:42:08.600 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.596+0000 7f6147e2c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614019b4f0 con 0x7f61401002a0 2023-11-16T16:42:08.600 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.596+0000 7f6147e2c700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614019b6b0 con 0x7f6140104140 2023-11-16T16:42:08.600 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.596+0000 7f6145bc8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61400ff8f0 0x7f61401963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:08.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.596+0000 7f6145bc8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61400ff8f0 0x7f61401963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45016/0 (socket says 172.21.15.1:45016) 2023-11-16T16:42:08.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.596+0000 7f6145bc8700 1 -- 172.21.15.1:0/1936599336 learned_addr learned my addr 172.21.15.1:0/1936599336 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:42:08.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.596+0000 7f61463c9700 1 --2- 172.21.15.1:0/1936599336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6140104140 0x7f614019ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:08.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.596+0000 7f61453c7700 1 --2- 172.21.15.1:0/1936599336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61401002a0 0x7f61401968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:08.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.596+0000 7f6145bc8700 1 -- 172.21.15.1:0/1936599336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6140104140 msgr2=0x7f614019ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:08.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.596+0000 7f6145bc8700 1 --2- 172.21.15.1:0/1936599336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6140104140 0x7f614019ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.596+0000 7f6145bc8700 1 -- 172.21.15.1:0/1936599336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61401002a0 msgr2=0x7f61401968e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:08.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.596+0000 7f6145bc8700 1 --2- 172.21.15.1:0/1936599336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61401002a0 0x7f61401968e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.596+0000 7f6145bc8700 1 -- 172.21.15.1:0/1936599336 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6138009d00 con 0x7f61400ff8f0 2023-11-16T16:42:08.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.597+0000 7f6145bc8700 1 --2- 172.21.15.1:0/1936599336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61400ff8f0 0x7f61401963a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f613c00e9a0 tx=0x7f613c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:08.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.597+0000 7f6132ffd700 1 -- 172.21.15.1:0/1936599336 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613c009790 con 0x7f61400ff8f0 2023-11-16T16:42:08.602 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.597+0000 7f6132ffd700 1 -- 172.21.15.1:0/1936599336 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f613c00cbd0 con 0x7f61400ff8f0 2023-11-16T16:42:08.602 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.597+0000 7f6147e2c700 1 -- 172.21.15.1:0/1936599336 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f614010f1d0 con 0x7f61400ff8f0 2023-11-16T16:42:08.602 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.597+0000 7f6132ffd700 1 -- 172.21.15.1:0/1936599336 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613c018680 con 0x7f61400ff8f0 2023-11-16T16:42:08.602 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.597+0000 7f6147e2c700 1 -- 172.21.15.1:0/1936599336 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f614010f040 con 0x7f61400ff8f0 2023-11-16T16:42:08.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.600+0000 7f6132ffd700 1 -- 172.21.15.1:0/1936599336 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f613c00cd40 con 0x7f61400ff8f0 2023-11-16T16:42:08.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.600+0000 7f6132ffd700 1 --2- 172.21.15.1:0/1936599336 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f612c0a12c0 0x7f612c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:08.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.601+0000 7f61453c7700 1 --2- 172.21.15.1:0/1936599336 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f612c0a12c0 0x7f612c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:08.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.601+0000 7f6132ffd700 1 -- 172.21.15.1:0/1936599336 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f613c014070 con 0x7f61400ff8f0 2023-11-16T16:42:08.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.601+0000 7f6153fb8b80 1 Processor -- start 2023-11-16T16:42:08.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.601+0000 7f61453c7700 1 --2- 172.21.15.1:0/1936599336 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f612c0a12c0 0x7f612c0a3770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f61401979b0 tx=0x7f6134009260 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:08.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.601+0000 7f6153fb8b80 1 -- start start 2023-11-16T16:42:08.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.601+0000 7f6153fb8b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560191e29ff0 0x560191e2a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:08.605 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.601+0000 7f6153fb8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560191e2a900 0x560191e2e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:08.605 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.601+0000 7f6153fb8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560191e2ed40 0x560191e2f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:08.605 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.601+0000 7f6153fb8b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560191d2b0e0 con 0x560191e29ff0 2023-11-16T16:42:08.605 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.601+0000 7f6153fb8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560191d7d920 con 0x560191e2ed40 2023-11-16T16:42:08.605 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.601+0000 7f6153fb8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560191d76db0 con 0x560191e2a900 2023-11-16T16:42:08.605 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.602+0000 7f6145bc8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560191e29ff0 0x560191e2a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:08.605 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.602+0000 7f6145bc8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560191e29ff0 0x560191e2a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45024/0 (socket says 172.21.15.1:45024) 2023-11-16T16:42:08.605 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.602+0000 7f6145bc8700 1 -- 172.21.15.1:0/928157039 learned_addr learned my addr 172.21.15.1:0/928157039 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:42:08.605 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.602+0000 7f61453c7700 1 --2- 172.21.15.1:0/928157039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560191e2a900 0x560191e2e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:08.605 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.602+0000 7f61463c9700 1 --2- 172.21.15.1:0/928157039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560191e2ed40 0x560191e2f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:08.605 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.602+0000 7f6145bc8700 1 -- 172.21.15.1:0/928157039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560191e2a900 msgr2=0x560191e2e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:08.605 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.602+0000 7f6145bc8700 1 --2- 172.21.15.1:0/928157039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560191e2a900 0x560191e2e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.606 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.602+0000 7f6145bc8700 1 -- 172.21.15.1:0/928157039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560191e2ed40 msgr2=0x560191e2f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:08.606 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.602+0000 7f6145bc8700 1 --2- 172.21.15.1:0/928157039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560191e2ed40 0x560191e2f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.606 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.602+0000 7f6145bc8700 1 -- 172.21.15.1:0/928157039 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560191bf86b0 con 0x560191e29ff0 2023-11-16T16:42:08.606 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.602+0000 7f6145bc8700 1 --2- 172.21.15.1:0/928157039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560191e29ff0 0x560191e2a3c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f613c0a1fb0 tx=0x7f613c099a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a97b7b7f2dff8c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:08.607 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.604+0000 7f612bfff700 1 -- 172.21.15.1:0/928157039 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613c0a24e0 con 0x560191e29ff0 2023-11-16T16:42:08.607 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.604+0000 7f612bfff700 1 -- 172.21.15.1:0/928157039 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f613c09a2f0 con 0x560191e29ff0 2023-11-16T16:42:08.607 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.604+0000 7f612bfff700 1 -- 172.21.15.1:0/928157039 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613c0a22b0 con 0x560191e29ff0 2023-11-16T16:42:08.607 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.605+0000 7f6153fb8b80 1 -- 172.21.15.1:0/928157039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560191e29ff0 msgr2=0x560191e2a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:08.607 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.605+0000 7f6153fb8b80 1 --2- 172.21.15.1:0/928157039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560191e29ff0 0x560191e2a3c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f613c0a1fb0 tx=0x7f613c099a70 comp rx=0 tx=0).stop 2023-11-16T16:42:08.607 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.605+0000 7f6153fb8b80 1 -- 172.21.15.1:0/928157039 shutdown_connections 2023-11-16T16:42:08.608 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.605+0000 7f6153fb8b80 1 --2- 172.21.15.1:0/928157039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560191e2ed40 0x560191e2f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.608 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.605+0000 7f6153fb8b80 1 --2- 172.21.15.1:0/928157039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560191e29ff0 0x560191e2a3c0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.608 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.605+0000 7f6153fb8b80 1 --2- 172.21.15.1:0/928157039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560191e2a900 0x560191e2e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.608 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.605+0000 7f6153fb8b80 1 -- 172.21.15.1:0/928157039 >> 172.21.15.1:0/928157039 conn(0x560191e1b3e0 msgr2=0x560191e20bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:08.608 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.605+0000 7f6153fb8b80 1 -- 172.21.15.1:0/928157039 shutdown_connections 2023-11-16T16:42:08.608 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.605+0000 7f6153fb8b80 1 -- 172.21.15.1:0/928157039 wait complete. 2023-11-16T16:42:08.608 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.605+0000 7f6153fb8b80 1 -- start start 2023-11-16T16:42:08.608 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.606+0000 7f6153fb8b80 1 Processor -- start 2023-11-16T16:42:08.612 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.606+0000 7f6153fb8b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560191e29ff0 0x560191e36570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:08.612 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.606+0000 7f6153fb8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560191e2a900 0x560191e337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:08.612 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.606+0000 7f6153fb8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560191e2ed40 0x560191e33f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:08.612 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.606+0000 7f6153fb8b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560191d18960 con 0x560191e29ff0 2023-11-16T16:42:08.612 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.606+0000 7f6153fb8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560191d18600 con 0x560191e2ed40 2023-11-16T16:42:08.612 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.606+0000 7f6153fb8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560191bfa9a0 con 0x560191e2a900 2023-11-16T16:42:08.612 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.606+0000 7f6153fb8b80 10 client.0 _ll_register_callbacks cb 0x560191d36800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:42:08.612 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.606+0000 7f6145bc8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560191e29ff0 0x560191e36570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:08.612 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.606+0000 7f6145bc8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560191e29ff0 0x560191e36570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45030/0 (socket says 172.21.15.1:45030) 2023-11-16T16:42:08.612 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.606+0000 7f6145bc8700 1 -- 172.21.15.1:0/3652300867 learned_addr learned my addr 172.21.15.1:0/3652300867 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:42:08.612 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.606+0000 7f61453c7700 1 --2- 172.21.15.1:0/3652300867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560191e2a900 0x560191e337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:08.613 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.606+0000 7f61463c9700 1 --2- 172.21.15.1:0/3652300867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560191e2ed40 0x560191e33f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:08.613 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.606+0000 7f6145bc8700 1 -- 172.21.15.1:0/3652300867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560191e2a900 msgr2=0x560191e337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:08.613 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.606+0000 7f6145bc8700 1 --2- 172.21.15.1:0/3652300867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560191e2a900 0x560191e337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.613 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.606+0000 7f6145bc8700 1 -- 172.21.15.1:0/3652300867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560191e2ed40 msgr2=0x560191e33f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:08.613 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.606+0000 7f6145bc8700 1 --2- 172.21.15.1:0/3652300867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560191e2ed40 0x560191e33f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.613 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.606+0000 7f6145bc8700 1 -- 172.21.15.1:0/3652300867 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f614010f040 con 0x560191e29ff0 2023-11-16T16:42:08.613 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.606+0000 7f61463c9700 1 --2- 172.21.15.1:0/3652300867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560191e2ed40 0x560191e33f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:42:08.613 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.607+0000 7f6145bc8700 1 --2- 172.21.15.1:0/3652300867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560191e29ff0 0x560191e36570 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f613c0cac90 tx=0x7f613c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b2403212606aec server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:08.613 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.607+0000 7f61297fa700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:42:08.613 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.607+0000 7f61297fa700 1 -- 172.21.15.1:0/3652300867 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613c0b8470 con 0x560191e29ff0 2023-11-16T16:42:08.613 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.607+0000 7f61297fa700 1 -- 172.21.15.1:0/3652300867 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f613c0a2050 con 0x560191e29ff0 2023-11-16T16:42:08.613 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.607+0000 7f6153fb8b80 1 -- 172.21.15.1:0/3652300867 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560191e18ba0 con 0x560191e29ff0 2023-11-16T16:42:08.613 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.607+0000 7f61297fa700 1 -- 172.21.15.1:0/3652300867 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f613c0a2210 con 0x560191e29ff0 2023-11-16T16:42:08.614 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.607+0000 7f6153fb8b80 10 client.16065 fetch_fsmap learned FSMap version 25 2023-11-16T16:42:08.614 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.607+0000 7f6153fb8b80 1 -- 172.21.15.1:0/3652300867 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560191d3d210 con 0x560191e29ff0 2023-11-16T16:42:08.614 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.607+0000 7f61297fa700 1 -- 172.21.15.1:0/3652300867 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560191e18ba0 con 0x560191e29ff0 2023-11-16T16:42:08.614 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.607+0000 7f61297fa700 1 -- 172.21.15.1:0/3652300867 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f613c099180 con 0x560191e29ff0 2023-11-16T16:42:08.614 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.607+0000 7f6153fb8b80 10 client.16065 fetch_fsmap finished waiting for FSMap version 25 2023-11-16T16:42:08.614 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.607+0000 7f6153fb8b80 10 client.16065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:42:08.614 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.607+0000 7f6153fb8b80 20 client.16065 populate_metadata read hostname 'smithi001' 2023-11-16T16:42:08.614 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.607+0000 7f6153fb8b80 1 --2- 172.21.15.1:0/3652300867 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x560191e43970 0x560191e45d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:08.614 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.607+0000 7f6153fb8b80 4 client.16065 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:42:08.614 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.607+0000 7f6153fb8b80 1 -- 172.21.15.1:0/3652300867 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560191d39120 con 0x560191e43970 2023-11-16T16:42:08.614 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.608+0000 7f61453c7700 1 --2- 172.21.15.1:0/3652300867 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x560191e43970 0x560191e45d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:08.614 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.608+0000 7f61297fa700 10 client.16065 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:42:08.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.608+0000 7f61453c7700 1 --2- 172.21.15.1:0/3652300867 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x560191e43970 0x560191e45d80 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9187c08e88c8f583 server_cookie=f1184e92a13f7749 in_seq=0 out_seq=0 2023-11-16T16:42:08.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.609+0000 7f61297fa700 1 -- 172.21.15.1:0/3652300867 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560191d39120 con 0x560191e43970 2023-11-16T16:42:08.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.609+0000 7f61297fa700 10 client.16065 handle_command_reply: tid=0 2023-11-16T16:42:08.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.609+0000 7f6153fb8b80 1 client.16065 shutdown 2023-11-16T16:42:08.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.609+0000 7f6153fb8b80 1 -- 172.21.15.1:0/3652300867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560191e29ff0 msgr2=0x560191e36570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:08.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.609+0000 7f6153fb8b80 1 --2- 172.21.15.1:0/3652300867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560191e29ff0 0x560191e36570 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f613c0cac90 tx=0x7f613c00ef60 comp rx=0 tx=0).stop 2023-11-16T16:42:08.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.609+0000 7f6153fb8b80 1 -- 172.21.15.1:0/3652300867 shutdown_connections 2023-11-16T16:42:08.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.609+0000 7f6153fb8b80 1 --2- 172.21.15.1:0/3652300867 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x560191e43970 0x560191e45d80 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.609+0000 7f6153fb8b80 1 --2- 172.21.15.1:0/3652300867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560191e2ed40 0x560191e33f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.609+0000 7f6153fb8b80 1 --2- 172.21.15.1:0/3652300867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560191e29ff0 0x560191e36570 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.609+0000 7f6153fb8b80 1 --2- 172.21.15.1:0/3652300867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560191e2a900 0x560191e337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.616 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.609+0000 7f61297fa700 0 client.16065 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:42:08.616 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.609+0000 7f6153fb8b80 1 -- 172.21.15.1:0/3652300867 >> 172.21.15.1:0/3652300867 conn(0x560191e1b3e0 msgr2=0x560191e301f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:08.616 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.609+0000 7f6153fb8b80 1 -- 172.21.15.1:0/3652300867 shutdown_connections 2023-11-16T16:42:08.616 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.609+0000 7f6153fb8b80 1 -- 172.21.15.1:0/3652300867 wait complete. 2023-11-16T16:42:08.616 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.609+0000 7f6153fb8b80 20 client.16065 trim_cache size 0 max 16384 2023-11-16T16:42:08.630 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.626+0000 7f6153fb8b80 1 Processor -- start 2023-11-16T16:42:08.630 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.626+0000 7f6153fb8b80 1 -- start start 2023-11-16T16:42:08.630 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.627+0000 7f6153fb8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560191e29ff0 0x560191e2c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:08.630 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.627+0000 7f6153fb8b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560191e2a900 0x560191e3ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:08.631 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.627+0000 7f6153fb8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560191e2ed40 0x560191e36110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:08.631 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.627+0000 7f6153fb8b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614010f3b0 con 0x560191e2a900 2023-11-16T16:42:08.631 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.627+0000 7f6153fb8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560191e2b080 con 0x560191e2ed40 2023-11-16T16:42:08.631 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.627+0000 7f6153fb8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560191e40830 con 0x560191e29ff0 2023-11-16T16:42:08.631 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.627+0000 7f61453c7700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560191e2a900 0x560191e3ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:08.631 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.627+0000 7f61453c7700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560191e2a900 0x560191e3ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45038/0 (socket says 172.21.15.1:45038) 2023-11-16T16:42:08.631 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.627+0000 7f61453c7700 1 -- 172.21.15.1:0/2305317145 learned_addr learned my addr 172.21.15.1:0/2305317145 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:42:08.631 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.627+0000 7f6145bc8700 1 --2- 172.21.15.1:0/2305317145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560191e29ff0 0x560191e2c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:08.631 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.627+0000 7f61463c9700 1 --2- 172.21.15.1:0/2305317145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560191e2ed40 0x560191e36110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:08.631 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.627+0000 7f61453c7700 1 -- 172.21.15.1:0/2305317145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560191e29ff0 msgr2=0x560191e2c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:08.631 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.627+0000 7f61453c7700 1 --2- 172.21.15.1:0/2305317145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560191e29ff0 0x560191e2c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.631 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.627+0000 7f61453c7700 1 -- 172.21.15.1:0/2305317145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560191e2ed40 msgr2=0x560191e36110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:08.631 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.627+0000 7f61453c7700 1 --2- 172.21.15.1:0/2305317145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560191e2ed40 0x560191e36110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.632 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.627+0000 7f61453c7700 1 -- 172.21.15.1:0/2305317145 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560191e1cb80 con 0x560191e2a900 2023-11-16T16:42:08.632 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.627+0000 7f61453c7700 1 --2- 172.21.15.1:0/2305317145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560191e2a900 0x560191e3ae30 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f61340097d0 tx=0x7f6134014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a75443467ccb3eb5 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:08.632 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.627+0000 7f6130ff9700 1 -- 172.21.15.1:0/2305317145 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613402c070 con 0x560191e2a900 2023-11-16T16:42:08.632 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.628+0000 7f6130ff9700 1 -- 172.21.15.1:0/2305317145 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6134016070 con 0x560191e2a900 2023-11-16T16:42:08.632 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.628+0000 7f6130ff9700 1 -- 172.21.15.1:0/2305317145 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6134027070 con 0x560191e2a900 2023-11-16T16:42:08.632 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.628+0000 7f6153fb8b80 1 -- 172.21.15.1:0/2305317145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560191e2a900 msgr2=0x560191e3ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:08.632 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.628+0000 7f6153fb8b80 1 --2- 172.21.15.1:0/2305317145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560191e2a900 0x560191e3ae30 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f61340097d0 tx=0x7f6134014000 comp rx=0 tx=0).stop 2023-11-16T16:42:08.632 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.628+0000 7f6153fb8b80 1 -- 172.21.15.1:0/2305317145 shutdown_connections 2023-11-16T16:42:08.632 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.628+0000 7f6153fb8b80 1 --2- 172.21.15.1:0/2305317145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560191e2ed40 0x560191e36110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.632 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.628+0000 7f6153fb8b80 1 --2- 172.21.15.1:0/2305317145 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560191e29ff0 0x560191e2c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.632 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.628+0000 7f6153fb8b80 1 --2- 172.21.15.1:0/2305317145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560191e2a900 0x560191e3ae30 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.632 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.628+0000 7f6153fb8b80 1 -- 172.21.15.1:0/2305317145 >> 172.21.15.1:0/2305317145 conn(0x560191e1b3e0 msgr2=0x560191e2cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:08.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.628+0000 7f6153fb8b80 1 -- 172.21.15.1:0/2305317145 shutdown_connections 2023-11-16T16:42:08.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.628+0000 7f6153fb8b80 1 -- 172.21.15.1:0/2305317145 wait complete. 2023-11-16T16:42:08.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.628+0000 7f6153fb8b80 1 -- start start 2023-11-16T16:42:08.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.629+0000 7f6153fb8b80 1 Processor -- start 2023-11-16T16:42:08.636 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.629+0000 7f6153fb8b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560191e29ff0 0x560191e3b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:08.636 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.629+0000 7f6153fb8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560191e2a900 0x560191e321c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:08.636 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.629+0000 7f6153fb8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560191e2ed40 0x560191e32760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:08.636 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.629+0000 7f6153fb8b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560191e32e70 con 0x560191e29ff0 2023-11-16T16:42:08.636 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.629+0000 7f6153fb8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560191e32fe0 con 0x560191e2ed40 2023-11-16T16:42:08.636 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.629+0000 7f6153fb8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560191e97810 con 0x560191e2a900 2023-11-16T16:42:08.636 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.629+0000 7f6153fb8b80 10 client.0 _ll_register_callbacks cb 0x7f613c005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:42:08.636 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.629+0000 7f6145bc8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560191e29ff0 0x560191e3b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:08.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.629+0000 7f6145bc8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560191e29ff0 0x560191e3b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45042/0 (socket says 172.21.15.1:45042) 2023-11-16T16:42:08.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.629+0000 7f6145bc8700 1 -- 172.21.15.1:0/3287764455 learned_addr learned my addr 172.21.15.1:0/3287764455 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:42:08.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.629+0000 7f61453c7700 1 --2- 172.21.15.1:0/3287764455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560191e2a900 0x560191e321c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:08.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.629+0000 7f6145bc8700 1 -- 172.21.15.1:0/3287764455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560191e2a900 msgr2=0x560191e321c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:08.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.629+0000 7f6145bc8700 1 --2- 172.21.15.1:0/3287764455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560191e2a900 0x560191e321c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.629+0000 7f6145bc8700 1 -- 172.21.15.1:0/3287764455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560191e2ed40 msgr2=0x560191e32760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:08.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.629+0000 7f6145bc8700 1 --2- 172.21.15.1:0/3287764455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560191e2ed40 0x560191e32760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.629+0000 7f6145bc8700 1 -- 172.21.15.1:0/3287764455 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6134022450 con 0x560191e29ff0 2023-11-16T16:42:08.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.630+0000 7f6145bc8700 1 --2- 172.21.15.1:0/3287764455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560191e29ff0 0x560191e3b030 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f613c0a1fb0 tx=0x7f613c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e9041f82776a2ee server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:08.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.630+0000 7f61297fa700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:42:08.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.630+0000 7f61297fa700 1 -- 172.21.15.1:0/3287764455 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613c0b2ea0 con 0x560191e29ff0 2023-11-16T16:42:08.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.630+0000 7f61297fa700 1 -- 172.21.15.1:0/3287764455 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f613c0b8470 con 0x560191e29ff0 2023-11-16T16:42:08.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.630+0000 7f6153fb8b80 1 -- 172.21.15.1:0/3287764455 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f613c09a2f0 con 0x560191e29ff0 2023-11-16T16:42:08.638 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.630+0000 7f61297fa700 1 -- 172.21.15.1:0/3287764455 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f613c0b2c60 con 0x560191e29ff0 2023-11-16T16:42:08.638 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.630+0000 7f6153fb8b80 10 client.16071 fetch_fsmap learned FSMap version 25 2023-11-16T16:42:08.638 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.630+0000 7f6153fb8b80 1 -- 172.21.15.1:0/3287764455 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x560191d3cf60 con 0x560191e29ff0 2023-11-16T16:42:08.638 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.630+0000 7f61297fa700 1 -- 172.21.15.1:0/3287764455 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f613c09a2f0 con 0x560191e29ff0 2023-11-16T16:42:08.638 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.630+0000 7f61297fa700 1 -- 172.21.15.1:0/3287764455 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f613c0ca460 con 0x560191e29ff0 2023-11-16T16:42:08.638 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.630+0000 7f6153fb8b80 10 client.16071 fetch_fsmap finished waiting for FSMap version 25 2023-11-16T16:42:08.638 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.630+0000 7f6153fb8b80 10 client.16071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:42:08.638 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.630+0000 7f6153fb8b80 20 client.16071 populate_metadata read hostname 'smithi001' 2023-11-16T16:42:08.638 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.630+0000 7f6153fb8b80 1 --2- 172.21.15.1:0/3287764455 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x560191e98ac0 0x560191e9aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:08.638 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.630+0000 7f6153fb8b80 4 client.16071 mds_command: new command op to 24521 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-16T16:42:08.638 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.630+0000 7f6153fb8b80 1 -- 172.21.15.1:0/3287764455 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x560191d456e0 con 0x560191e98ac0 2023-11-16T16:42:08.638 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.631+0000 7f61453c7700 1 --2- 172.21.15.1:0/3287764455 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x560191e98ac0 0x560191e9aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:08.639 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.631+0000 7f61297fa700 10 client.16071 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:42:08.639 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.631+0000 7f61453c7700 1 --2- 172.21.15.1:0/3287764455 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x560191e98ac0 0x560191e9aed0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2634e9711ec0f827 server_cookie=173cf01c667c573a in_seq=0 out_seq=0 2023-11-16T16:42:08.639 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.633+0000 7f61297fa700 1 -- 172.21.15.1:0/3287764455 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560191d456e0 con 0x560191e98ac0 2023-11-16T16:42:08.639 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.633+0000 7f61297fa700 10 client.16071 handle_command_reply: tid=0 2023-11-16T16:42:08.639 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.633+0000 7f6153fb8b80 1 client.16071 shutdown 2023-11-16T16:42:08.639 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.633+0000 7f6153fb8b80 1 -- 172.21.15.1:0/3287764455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560191e29ff0 msgr2=0x560191e3b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:08.639 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.633+0000 7f6153fb8b80 1 --2- 172.21.15.1:0/3287764455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560191e29ff0 0x560191e3b030 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f613c0a1fb0 tx=0x7f613c01f000 comp rx=0 tx=0).stop 2023-11-16T16:42:08.639 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.633+0000 7f6153fb8b80 1 -- 172.21.15.1:0/3287764455 shutdown_connections 2023-11-16T16:42:08.639 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.633+0000 7f6153fb8b80 1 --2- 172.21.15.1:0/3287764455 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x560191e98ac0 0x560191e9aed0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.639 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.633+0000 7f6153fb8b80 1 --2- 172.21.15.1:0/3287764455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560191e2ed40 0x560191e32760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.639 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.633+0000 7f6153fb8b80 1 --2- 172.21.15.1:0/3287764455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560191e29ff0 0x560191e3b030 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.639 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.633+0000 7f6153fb8b80 1 --2- 172.21.15.1:0/3287764455 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560191e2a900 0x560191e321c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.639 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.633+0000 7f61297fa700 0 client.16071 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:42:08.640 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.633+0000 7f6153fb8b80 1 -- 172.21.15.1:0/3287764455 >> 172.21.15.1:0/3287764455 conn(0x560191e1b3e0 msgr2=0x560191e20670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:08.640 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.633+0000 7f6153fb8b80 1 -- 172.21.15.1:0/3287764455 shutdown_connections 2023-11-16T16:42:08.640 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.633+0000 7f6153fb8b80 1 -- 172.21.15.1:0/3287764455 wait complete. 2023-11-16T16:42:08.640 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.633+0000 7f6153fb8b80 20 client.16071 trim_cache size 0 max 16384 2023-11-16T16:42:08.640 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:42:08.640 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2023-11-16T16:42:08.640 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "4e68c117-e4ae-40d9-847d-0d6d5d2b1317", 2023-11-16T16:42:08.640 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2023-11-16T16:42:08.640 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:42:08.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.634+0000 7f6147e2c700 1 -- 172.21.15.1:0/1936599336 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f612c0a12c0 msgr2=0x7f612c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:08.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.634+0000 7f6147e2c700 1 --2- 172.21.15.1:0/1936599336 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f612c0a12c0 0x7f612c0a3770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f61401979b0 tx=0x7f6134009260 comp rx=0 tx=0).stop 2023-11-16T16:42:08.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.634+0000 7f6147e2c700 1 -- 172.21.15.1:0/1936599336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61400ff8f0 msgr2=0x7f61401963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:08.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.634+0000 7f6147e2c700 1 --2- 172.21.15.1:0/1936599336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61400ff8f0 0x7f61401963a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f613c00e9a0 tx=0x7f613c00ecb0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.634+0000 7f6147e2c700 1 -- 172.21.15.1:0/1936599336 shutdown_connections 2023-11-16T16:42:08.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.634+0000 7f6147e2c700 1 --2- 172.21.15.1:0/1936599336 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f612c0a12c0 0x7f612c0a3770 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.634+0000 7f6147e2c700 1 --2- 172.21.15.1:0/1936599336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6140104140 0x7f614019ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.634+0000 7f6147e2c700 1 --2- 172.21.15.1:0/1936599336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f61400ff8f0 0x7f61401963a0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.634+0000 7f6147e2c700 1 --2- 172.21.15.1:0/1936599336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61401002a0 0x7f61401968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.634+0000 7f6147e2c700 1 -- 172.21.15.1:0/1936599336 >> 172.21.15.1:0/1936599336 conn(0x7f6140075880 msgr2=0x7f61400fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:08.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.634+0000 7f6147e2c700 1 -- 172.21.15.1:0/1936599336 shutdown_connections 2023-11-16T16:42:08.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.634+0000 7f6147e2c700 1 -- 172.21.15.1:0/1936599336 wait complete. 2023-11-16T16:42:08.649 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-16T16:42:08.780 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.776+0000 7f1d9e12a700 1 -- 172.21.15.1:0/1098772879 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d98105250 msgr2=0x7f1d98105620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:08.780 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.776+0000 7f1d9e12a700 1 --2- 172.21.15.1:0/1098772879 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d98105250 0x7f1d98105620 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1d8800b0a0 tx=0x7f1d8800b3b0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.780 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.776+0000 7f1d9e12a700 1 -- 172.21.15.1:0/1098772879 shutdown_connections 2023-11-16T16:42:08.781 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.776+0000 7f1d9e12a700 1 --2- 172.21.15.1:0/1098772879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d98100420 0x7f1d9810bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.781 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.776+0000 7f1d9e12a700 1 --2- 172.21.15.1:0/1098772879 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d98105250 0x7f1d98105620 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.781 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.776+0000 7f1d9e12a700 1 --2- 172.21.15.1:0/1098772879 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d980ffa70 0x7f1d980ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.781 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.776+0000 7f1d9e12a700 1 -- 172.21.15.1:0/1098772879 >> 172.21.15.1:0/1098772879 conn(0x7f1d980763a0 msgr2=0x7f1d980767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:08.781 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.776+0000 7f1d9e12a700 1 -- 172.21.15.1:0/1098772879 shutdown_connections 2023-11-16T16:42:08.781 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.776+0000 7f1d9e12a700 1 -- 172.21.15.1:0/1098772879 wait complete. 2023-11-16T16:42:08.781 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.777+0000 7f1d9e12a700 1 Processor -- start 2023-11-16T16:42:08.781 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.777+0000 7f1d9e12a700 1 -- start start 2023-11-16T16:42:08.781 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.777+0000 7f1d9e12a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d980ffa70 0x7f1d98195340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:08.781 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.777+0000 7f1d9e12a700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d98100420 0x7f1d98195880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:08.781 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.777+0000 7f1d9e12a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d98105250 0x7f1d98199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:08.781 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.777+0000 7f1d9e12a700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9819a320 con 0x7f1d98100420 2023-11-16T16:42:08.782 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.777+0000 7f1d9e12a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9819a490 con 0x7f1d980ffa70 2023-11-16T16:42:08.782 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.777+0000 7f1d9e12a700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9819a650 con 0x7f1d98105250 2023-11-16T16:42:08.782 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.777+0000 7f1d96ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d98100420 0x7f1d98195880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:08.782 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.777+0000 7f1d977fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d980ffa70 0x7f1d98195340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:08.782 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.777+0000 7f1d977fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d980ffa70 0x7f1d98195340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.1:39242/0 (socket says 172.21.15.1:39242) 2023-11-16T16:42:08.782 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.777+0000 7f1d96ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d98100420 0x7f1d98195880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45058/0 (socket says 172.21.15.1:45058) 2023-11-16T16:42:08.782 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.777+0000 7f1d977fe700 1 -- 172.21.15.1:0/2062815478 learned_addr learned my addr 172.21.15.1:0/2062815478 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:42:08.782 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.777+0000 7f1d97fff700 1 --2- 172.21.15.1:0/2062815478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d98105250 0x7f1d98199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:08.782 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.778+0000 7f1d96ffd700 1 -- 172.21.15.1:0/2062815478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d98105250 msgr2=0x7f1d98199ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:08.782 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.778+0000 7f1d96ffd700 1 --2- 172.21.15.1:0/2062815478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d98105250 0x7f1d98199ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.782 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.778+0000 7f1d96ffd700 1 -- 172.21.15.1:0/2062815478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d980ffa70 msgr2=0x7f1d98195340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:08.783 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.778+0000 7f1d96ffd700 1 --2- 172.21.15.1:0/2062815478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d980ffa70 0x7f1d98195340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.783 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.778+0000 7f1d96ffd700 1 -- 172.21.15.1:0/2062815478 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d80009520 con 0x7f1d98100420 2023-11-16T16:42:08.783 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.778+0000 7f1d977fe700 1 --2- 172.21.15.1:0/2062815478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d980ffa70 0x7f1d98195340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:42:08.783 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.778+0000 7f1d96ffd700 1 --2- 172.21.15.1:0/2062815478 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d98100420 0x7f1d98195880 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f1d8000e9f0 tx=0x7f1d8000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:08.783 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.778+0000 7f1d94ff9700 1 -- 172.21.15.1:0/2062815478 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8000cb70 con 0x7f1d98100420 2023-11-16T16:42:08.783 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.778+0000 7f1d94ff9700 1 -- 172.21.15.1:0/2062815478 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d8000ee90 con 0x7f1d98100420 2023-11-16T16:42:08.783 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.778+0000 7f1d9e12a700 1 -- 172.21.15.1:0/2062815478 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d88009d00 con 0x7f1d98100420 2023-11-16T16:42:08.783 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.778+0000 7f1d94ff9700 1 -- 172.21.15.1:0/2062815478 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d80003cf0 con 0x7f1d98100420 2023-11-16T16:42:08.783 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.778+0000 7f1d9e12a700 1 -- 172.21.15.1:0/2062815478 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d9810f1b0 con 0x7f1d98100420 2023-11-16T16:42:08.786 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.781+0000 7f1d94ff9700 1 -- 172.21.15.1:0/2062815478 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f1d8000ccd0 con 0x7f1d98100420 2023-11-16T16:42:08.787 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.781+0000 7f1d94ff9700 1 --2- 172.21.15.1:0/2062815478 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f1d840a12c0 0x7f1d840a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:08.787 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.782+0000 7f1d977fe700 1 --2- 172.21.15.1:0/2062815478 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f1d840a12c0 0x7f1d840a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:08.787 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.782+0000 7f1d94ff9700 1 -- 172.21.15.1:0/2062815478 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f1d80014070 con 0x7f1d98100420 2023-11-16T16:42:08.787 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.782+0000 7f1d977fe700 1 --2- 172.21.15.1:0/2062815478 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f1d840a12c0 0x7f1d840a3770 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1d88009fd0 tx=0x7f1d88006210 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:08.787 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.782+0000 7f1daa2b6b80 1 Processor -- start 2023-11-16T16:42:08.787 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.782+0000 7f1daa2b6b80 1 -- start start 2023-11-16T16:42:08.787 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.783+0000 7f1daa2b6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f3c6a77b60 0x55f3c6a77f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:08.787 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.783+0000 7f1daa2b6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3c6a78470 0x55f3c6a7c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:08.787 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.783+0000 7f1daa2b6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f3c6a7c8b0 0x55f3c6a7cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:08.787 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.783+0000 7f1daa2b6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3c697d060 con 0x55f3c6a77b60 2023-11-16T16:42:08.787 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.783+0000 7f1daa2b6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3c69cf8a0 con 0x55f3c6a78470 2023-11-16T16:42:08.787 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.783+0000 7f1daa2b6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3c69c8d30 con 0x55f3c6a7c8b0 2023-11-16T16:42:08.787 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.783+0000 7f1d977fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f3c6a77b60 0x55f3c6a77f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:08.788 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.783+0000 7f1d96ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3c6a78470 0x55f3c6a7c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:08.788 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.783+0000 7f1d97fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f3c6a7c8b0 0x55f3c6a7cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:08.788 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.783+0000 7f1d96ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3c6a78470 0x55f3c6a7c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.1:39252/0 (socket says 172.21.15.1:39252) 2023-11-16T16:42:08.788 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.783+0000 7f1d96ffd700 1 -- 172.21.15.1:0/376338451 learned_addr learned my addr 172.21.15.1:0/376338451 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:42:08.788 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.783+0000 7f1d96ffd700 1 -- 172.21.15.1:0/376338451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f3c6a7c8b0 msgr2=0x55f3c6a7cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:08.788 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.783+0000 7f1d96ffd700 1 --2- 172.21.15.1:0/376338451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f3c6a7c8b0 0x55f3c6a7cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.788 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.783+0000 7f1d96ffd700 1 -- 172.21.15.1:0/376338451 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f3c6a77b60 msgr2=0x55f3c6a77f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:08.788 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.783+0000 7f1d96ffd700 1 --2- 172.21.15.1:0/376338451 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f3c6a77b60 0x55f3c6a77f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.788 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.783+0000 7f1d96ffd700 1 -- 172.21.15.1:0/376338451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3c684a630 con 0x55f3c6a78470 2023-11-16T16:42:08.788 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.783+0000 7f1d97fff700 1 --2- 172.21.15.1:0/376338451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f3c6a7c8b0 0x55f3c6a7cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:42:08.788 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.783+0000 7f1d96ffd700 1 --2- 172.21.15.1:0/376338451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3c6a78470 0x55f3c6a7c370 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1d800a1fc0 tx=0x7f1d80099a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2097654c6499d0b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:08.788 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.783+0000 7f1d8dffb700 1 -- 172.21.15.1:0/376338451 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8009e0d0 con 0x55f3c6a78470 2023-11-16T16:42:08.789 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.784+0000 7f1d8dffb700 1 -- 172.21.15.1:0/376338451 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d800a24f0 con 0x55f3c6a78470 2023-11-16T16:42:08.789 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.784+0000 7f1d8dffb700 1 -- 172.21.15.1:0/376338451 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d800a2060 con 0x55f3c6a78470 2023-11-16T16:42:08.789 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.784+0000 7f1daa2b6b80 1 -- 172.21.15.1:0/376338451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3c6a78470 msgr2=0x55f3c6a7c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:08.789 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.784+0000 7f1daa2b6b80 1 --2- 172.21.15.1:0/376338451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3c6a78470 0x55f3c6a7c370 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1d800a1fc0 tx=0x7f1d80099a80 comp rx=0 tx=0).stop 2023-11-16T16:42:08.789 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.784+0000 7f1daa2b6b80 1 -- 172.21.15.1:0/376338451 shutdown_connections 2023-11-16T16:42:08.789 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.784+0000 7f1daa2b6b80 1 --2- 172.21.15.1:0/376338451 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f3c6a7c8b0 0x55f3c6a7cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.789 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.784+0000 7f1daa2b6b80 1 --2- 172.21.15.1:0/376338451 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f3c6a77b60 0x55f3c6a77f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.789 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.784+0000 7f1daa2b6b80 1 --2- 172.21.15.1:0/376338451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3c6a78470 0x55f3c6a7c370 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.789 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.784+0000 7f1daa2b6b80 1 -- 172.21.15.1:0/376338451 >> 172.21.15.1:0/376338451 conn(0x55f3c6a6d360 msgr2=0x55f3c6a72b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:08.789 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.784+0000 7f1daa2b6b80 1 -- 172.21.15.1:0/376338451 shutdown_connections 2023-11-16T16:42:08.789 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.784+0000 7f1daa2b6b80 1 -- 172.21.15.1:0/376338451 wait complete. 2023-11-16T16:42:08.789 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.785+0000 7f1daa2b6b80 1 -- start start 2023-11-16T16:42:08.789 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.785+0000 7f1daa2b6b80 1 Processor -- start 2023-11-16T16:42:08.793 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.785+0000 7f1daa2b6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f3c6a77b60 0x55f3c6a840b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:08.793 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.785+0000 7f1daa2b6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3c6a78470 0x55f3c6a812c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:08.793 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.785+0000 7f1daa2b6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f3c6a7c8b0 0x55f3c6a81a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:08.793 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.785+0000 7f1daa2b6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3c696a8e0 con 0x55f3c6a7c8b0 2023-11-16T16:42:08.793 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.785+0000 7f1daa2b6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3c696a580 con 0x55f3c6a78470 2023-11-16T16:42:08.793 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.785+0000 7f1daa2b6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3c684c920 con 0x55f3c6a77b60 2023-11-16T16:42:08.793 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.785+0000 7f1daa2b6b80 10 client.0 _ll_register_callbacks cb 0x55f3c6988780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:42:08.793 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.786+0000 7f1d977fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f3c6a77b60 0x55f3c6a840b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:08.793 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.786+0000 7f1d97fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f3c6a7c8b0 0x55f3c6a81a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:08.793 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.786+0000 7f1d977fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f3c6a77b60 0x55f3c6a840b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:41752/0 (socket says 172.21.15.1:41752) 2023-11-16T16:42:08.794 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.786+0000 7f1d977fe700 1 -- 172.21.15.1:0/4222342640 learned_addr learned my addr 172.21.15.1:0/4222342640 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:42:08.794 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.786+0000 7f1d97fff700 1 --2- 172.21.15.1:0/4222342640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f3c6a7c8b0 0x55f3c6a81a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45074/0 (socket says 172.21.15.1:45074) 2023-11-16T16:42:08.794 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.786+0000 7f1d96ffd700 1 --2- 172.21.15.1:0/4222342640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3c6a78470 0x55f3c6a812c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:08.794 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.786+0000 7f1d97fff700 1 -- 172.21.15.1:0/4222342640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f3c6a77b60 msgr2=0x55f3c6a840b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:08.794 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.786+0000 7f1d97fff700 1 --2- 172.21.15.1:0/4222342640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f3c6a77b60 0x55f3c6a840b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.794 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.786+0000 7f1d97fff700 1 -- 172.21.15.1:0/4222342640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3c6a78470 msgr2=0x55f3c6a812c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:08.794 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.786+0000 7f1d97fff700 1 --2- 172.21.15.1:0/4222342640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3c6a78470 0x55f3c6a812c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.794 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.786+0000 7f1d97fff700 1 -- 172.21.15.1:0/4222342640 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d9810f1b0 con 0x55f3c6a7c8b0 2023-11-16T16:42:08.794 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.786+0000 7f1d977fe700 1 --2- 172.21.15.1:0/4222342640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f3c6a77b60 0x55f3c6a840b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:42:08.794 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.786+0000 7f1d96ffd700 1 --2- 172.21.15.1:0/4222342640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3c6a78470 0x55f3c6a812c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:42:08.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.786+0000 7f1d97fff700 1 --2- 172.21.15.1:0/4222342640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f3c6a7c8b0 0x55f3c6a81a00 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1d900126b0 tx=0x7f1d900129c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b1da430a1c1471 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:08.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.786+0000 7f1d76ffd700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:42:08.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.786+0000 7f1d76ffd700 1 -- 172.21.15.1:0/4222342640 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d90017070 con 0x55f3c6a7c8b0 2023-11-16T16:42:08.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.786+0000 7f1d76ffd700 1 -- 172.21.15.1:0/4222342640 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f1d90004050 con 0x55f3c6a7c8b0 2023-11-16T16:42:08.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.786+0000 7f1daa2b6b80 1 -- 172.21.15.1:0/4222342640 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f3c6a6ab20 con 0x55f3c6a7c8b0 2023-11-16T16:42:08.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.786+0000 7f1d76ffd700 1 -- 172.21.15.1:0/4222342640 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d90004250 con 0x55f3c6a7c8b0 2023-11-16T16:42:08.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.786+0000 7f1daa2b6b80 10 client.16083 fetch_fsmap learned FSMap version 25 2023-11-16T16:42:08.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.786+0000 7f1daa2b6b80 1 -- 172.21.15.1:0/4222342640 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f3c698f190 con 0x55f3c6a7c8b0 2023-11-16T16:42:08.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.787+0000 7f1d76ffd700 1 -- 172.21.15.1:0/4222342640 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f3c6a6ab20 con 0x55f3c6a7c8b0 2023-11-16T16:42:08.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.787+0000 7f1d76ffd700 1 -- 172.21.15.1:0/4222342640 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f1d90023070 con 0x55f3c6a7c8b0 2023-11-16T16:42:08.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.787+0000 7f1daa2b6b80 10 client.16083 fetch_fsmap finished waiting for FSMap version 25 2023-11-16T16:42:08.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.787+0000 7f1daa2b6b80 10 client.16083 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:42:08.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.787+0000 7f1daa2b6b80 20 client.16083 populate_metadata read hostname 'smithi001' 2023-11-16T16:42:08.796 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.787+0000 7f1daa2b6b80 1 --2- 172.21.15.1:0/4222342640 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55f3c6a919a0 0x55f3c6a93db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:08.796 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.787+0000 7f1daa2b6b80 4 client.16083 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:42:08.796 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.787+0000 7f1daa2b6b80 1 -- 172.21.15.1:0/4222342640 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f3c698b0a0 con 0x55f3c6a919a0 2023-11-16T16:42:08.796 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.787+0000 7f1d977fe700 1 --2- 172.21.15.1:0/4222342640 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55f3c6a919a0 0x55f3c6a93db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:08.796 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.788+0000 7f1d76ffd700 10 client.16083 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:42:08.796 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.788+0000 7f1d977fe700 1 --2- 172.21.15.1:0/4222342640 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55f3c6a919a0 0x55f3c6a93db0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5a19527100bcff4 server_cookie=12850fed719d77cd in_seq=0 out_seq=0 2023-11-16T16:42:08.796 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.789+0000 7f1d76ffd700 1 -- 172.21.15.1:0/4222342640 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f3c698b0a0 con 0x55f3c6a919a0 2023-11-16T16:42:08.796 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.789+0000 7f1d76ffd700 10 client.16083 handle_command_reply: tid=0 2023-11-16T16:42:08.796 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.789+0000 7f1daa2b6b80 1 client.16083 shutdown 2023-11-16T16:42:08.796 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.789+0000 7f1daa2b6b80 1 -- 172.21.15.1:0/4222342640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f3c6a7c8b0 msgr2=0x55f3c6a81a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:08.796 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.789+0000 7f1daa2b6b80 1 --2- 172.21.15.1:0/4222342640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f3c6a7c8b0 0x55f3c6a81a00 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1d900126b0 tx=0x7f1d900129c0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.789+0000 7f1daa2b6b80 1 -- 172.21.15.1:0/4222342640 shutdown_connections 2023-11-16T16:42:08.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.789+0000 7f1daa2b6b80 1 --2- 172.21.15.1:0/4222342640 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55f3c6a919a0 0x55f3c6a93db0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.789+0000 7f1daa2b6b80 1 --2- 172.21.15.1:0/4222342640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f3c6a7c8b0 0x55f3c6a81a00 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.789+0000 7f1daa2b6b80 1 --2- 172.21.15.1:0/4222342640 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f3c6a77b60 0x55f3c6a840b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.789+0000 7f1daa2b6b80 1 --2- 172.21.15.1:0/4222342640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3c6a78470 0x55f3c6a812c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.789+0000 7f1d76ffd700 0 client.16083 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:42:08.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.789+0000 7f1daa2b6b80 1 -- 172.21.15.1:0/4222342640 >> 172.21.15.1:0/4222342640 conn(0x55f3c6a6d360 msgr2=0x55f3c6a7dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:08.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.789+0000 7f1daa2b6b80 1 -- 172.21.15.1:0/4222342640 shutdown_connections 2023-11-16T16:42:08.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.789+0000 7f1daa2b6b80 1 -- 172.21.15.1:0/4222342640 wait complete. 2023-11-16T16:42:08.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.789+0000 7f1daa2b6b80 20 client.16083 trim_cache size 0 max 16384 2023-11-16T16:42:08.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.808+0000 7f1daa2b6b80 1 Processor -- start 2023-11-16T16:42:08.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.808+0000 7f1daa2b6b80 1 -- start start 2023-11-16T16:42:08.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.808+0000 7f1daa2b6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f3c6a77b60 0x55f3c6a889a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:08.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.808+0000 7f1daa2b6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3c6a78470 0x55f3c6a80870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:08.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.808+0000 7f1daa2b6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f3c6a7c8b0 0x55f3c6a83be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:08.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.808+0000 7f1daa2b6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3c6a7b450 con 0x55f3c6a7c8b0 2023-11-16T16:42:08.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.808+0000 7f1daa2b6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3c6a74110 con 0x55f3c6a78470 2023-11-16T16:42:08.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.808+0000 7f1daa2b6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3c6a6a960 con 0x55f3c6a77b60 2023-11-16T16:42:08.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.808+0000 7f1d977fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f3c6a77b60 0x55f3c6a889a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:08.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.808+0000 7f1d96ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3c6a78470 0x55f3c6a80870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:08.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.808+0000 7f1d97fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f3c6a7c8b0 0x55f3c6a83be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:08.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.808+0000 7f1d977fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f3c6a77b60 0x55f3c6a889a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:41760/0 (socket says 172.21.15.1:41760) 2023-11-16T16:42:08.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.808+0000 7f1d977fe700 1 -- 172.21.15.1:0/1772408532 learned_addr learned my addr 172.21.15.1:0/1772408532 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:42:08.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.808+0000 7f1d977fe700 1 -- 172.21.15.1:0/1772408532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3c6a78470 msgr2=0x55f3c6a80870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:08.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.808+0000 7f1d977fe700 1 --2- 172.21.15.1:0/1772408532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3c6a78470 0x55f3c6a80870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.808+0000 7f1d977fe700 1 -- 172.21.15.1:0/1772408532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f3c6a7c8b0 msgr2=0x55f3c6a83be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:08.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.808+0000 7f1d977fe700 1 --2- 172.21.15.1:0/1772408532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f3c6a7c8b0 0x55f3c6a83be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.809+0000 7f1d977fe700 1 -- 172.21.15.1:0/1772408532 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3c698eee0 con 0x55f3c6a77b60 2023-11-16T16:42:08.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.809+0000 7f1d96ffd700 1 --2- 172.21.15.1:0/1772408532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3c6a78470 0x55f3c6a80870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:42:08.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.809+0000 7f1d97fff700 1 --2- 172.21.15.1:0/1772408532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f3c6a7c8b0 0x55f3c6a83be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:42:08.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.809+0000 7f1d977fe700 1 --2- 172.21.15.1:0/1772408532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f3c6a77b60 0x55f3c6a889a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1d8800b070 tx=0x7f1d88005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=844287dfc6c4f850 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:08.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.809+0000 7f1d8e7fc700 1 -- 172.21.15.1:0/1772408532 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8802f040 con 0x55f3c6a77b60 2023-11-16T16:42:08.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.809+0000 7f1d8e7fc700 1 -- 172.21.15.1:0/1772408532 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d880317f0 con 0x55f3c6a77b60 2023-11-16T16:42:08.814 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.809+0000 7f1d8e7fc700 1 -- 172.21.15.1:0/1772408532 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8800e040 con 0x55f3c6a77b60 2023-11-16T16:42:08.814 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.810+0000 7f1daa2b6b80 1 -- 172.21.15.1:0/1772408532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f3c6a77b60 msgr2=0x55f3c6a889a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:08.814 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.810+0000 7f1daa2b6b80 1 --2- 172.21.15.1:0/1772408532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f3c6a77b60 0x55f3c6a889a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1d8800b070 tx=0x7f1d88005040 comp rx=0 tx=0).stop 2023-11-16T16:42:08.814 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.810+0000 7f1daa2b6b80 1 -- 172.21.15.1:0/1772408532 shutdown_connections 2023-11-16T16:42:08.814 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.810+0000 7f1daa2b6b80 1 --2- 172.21.15.1:0/1772408532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f3c6a7c8b0 0x55f3c6a83be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.814 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.810+0000 7f1daa2b6b80 1 --2- 172.21.15.1:0/1772408532 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f3c6a77b60 0x55f3c6a889a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.814 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.810+0000 7f1daa2b6b80 1 --2- 172.21.15.1:0/1772408532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3c6a78470 0x55f3c6a80870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.814 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.810+0000 7f1daa2b6b80 1 -- 172.21.15.1:0/1772408532 >> 172.21.15.1:0/1772408532 conn(0x55f3c6a6d360 msgr2=0x55f3c6a7a970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:08.814 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.810+0000 7f1daa2b6b80 1 -- 172.21.15.1:0/1772408532 shutdown_connections 2023-11-16T16:42:08.814 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.810+0000 7f1daa2b6b80 1 -- 172.21.15.1:0/1772408532 wait complete. 2023-11-16T16:42:08.814 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.810+0000 7f1daa2b6b80 1 -- start start 2023-11-16T16:42:08.815 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.810+0000 7f1daa2b6b80 1 Processor -- start 2023-11-16T16:42:08.818 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.811+0000 7f1daa2b6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3c6a77b60 0x55f3c6ae30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:08.818 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.811+0000 7f1daa2b6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f3c6a78470 0x55f3c6ae3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:08.818 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.811+0000 7f1daa2b6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f3c6a7c8b0 0x55f3c6a86990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:08.818 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.811+0000 7f1daa2b6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3c6a86f90 con 0x55f3c6a78470 2023-11-16T16:42:08.819 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.811+0000 7f1daa2b6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3c6a87100 con 0x55f3c6a77b60 2023-11-16T16:42:08.819 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.811+0000 7f1daa2b6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3c6a872c0 con 0x55f3c6a7c8b0 2023-11-16T16:42:08.819 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.811+0000 7f1daa2b6b80 10 client.0 _ll_register_callbacks cb 0x7f1d90009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:42:08.819 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.811+0000 7f1d96ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f3c6a78470 0x55f3c6ae3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:08.819 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.811+0000 7f1d96ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f3c6a78470 0x55f3c6ae3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45086/0 (socket says 172.21.15.1:45086) 2023-11-16T16:42:08.819 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.811+0000 7f1d96ffd700 1 -- 172.21.15.1:0/2897930247 learned_addr learned my addr 172.21.15.1:0/2897930247 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:42:08.819 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.811+0000 7f1d977fe700 1 --2- 172.21.15.1:0/2897930247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3c6a77b60 0x55f3c6ae30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:08.819 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.811+0000 7f1d97fff700 1 --2- 172.21.15.1:0/2897930247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f3c6a7c8b0 0x55f3c6a86990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:08.819 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.811+0000 7f1d96ffd700 1 -- 172.21.15.1:0/2897930247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f3c6a7c8b0 msgr2=0x55f3c6a86990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:08.819 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.811+0000 7f1d96ffd700 1 --2- 172.21.15.1:0/2897930247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f3c6a7c8b0 0x55f3c6a86990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.819 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.811+0000 7f1d96ffd700 1 -- 172.21.15.1:0/2897930247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3c6a77b60 msgr2=0x55f3c6ae30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:08.819 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.811+0000 7f1d96ffd700 1 --2- 172.21.15.1:0/2897930247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3c6a77b60 0x55f3c6ae30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.819 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.811+0000 7f1d96ffd700 1 -- 172.21.15.1:0/2897930247 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f3c684a630 con 0x55f3c6a78470 2023-11-16T16:42:08.820 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.811+0000 7f1d977fe700 1 --2- 172.21.15.1:0/2897930247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3c6a77b60 0x55f3c6ae30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:42:08.820 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.811+0000 7f1d96ffd700 1 --2- 172.21.15.1:0/2897930247 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f3c6a78470 0x55f3c6ae3620 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f1d800965a0 tx=0x7f1d80003780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ad22e990eb4696 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:08.820 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.811+0000 7f1d76ffd700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:42:08.820 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.811+0000 7f1d76ffd700 1 -- 172.21.15.1:0/2897930247 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8009e0d0 con 0x55f3c6a78470 2023-11-16T16:42:08.820 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.811+0000 7f1d76ffd700 1 -- 172.21.15.1:0/2897930247 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f1d800a2060 con 0x55f3c6a78470 2023-11-16T16:42:08.820 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.811+0000 7f1daa2b6b80 1 -- 172.21.15.1:0/2897930247 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1d800a24f0 con 0x55f3c6a78470 2023-11-16T16:42:08.820 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.811+0000 7f1d76ffd700 1 -- 172.21.15.1:0/2897930247 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d800a2220 con 0x55f3c6a78470 2023-11-16T16:42:08.820 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.812+0000 7f1daa2b6b80 10 client.16089 fetch_fsmap learned FSMap version 25 2023-11-16T16:42:08.820 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.812+0000 7f1daa2b6b80 1 -- 172.21.15.1:0/2897930247 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f3c698f190 con 0x55f3c6a78470 2023-11-16T16:42:08.820 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.812+0000 7f1d76ffd700 1 -- 172.21.15.1:0/2897930247 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d800a24f0 con 0x55f3c6a78470 2023-11-16T16:42:08.820 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.812+0000 7f1d76ffd700 1 -- 172.21.15.1:0/2897930247 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f1d800991c0 con 0x55f3c6a78470 2023-11-16T16:42:08.820 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.812+0000 7f1daa2b6b80 10 client.16089 fetch_fsmap finished waiting for FSMap version 25 2023-11-16T16:42:08.821 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.812+0000 7f1daa2b6b80 10 client.16089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:42:08.821 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.812+0000 7f1daa2b6b80 20 client.16089 populate_metadata read hostname 'smithi001' 2023-11-16T16:42:08.821 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.812+0000 7f1daa2b6b80 1 --2- 172.21.15.1:0/2897930247 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55f3c6aeaa00 0x55f3c6aece10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:08.821 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.812+0000 7f1daa2b6b80 4 client.16089 mds_command: new command op to 24521 tid=0[{"prefix": "scrub status"}] 2023-11-16T16:42:08.821 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.812+0000 7f1daa2b6b80 1 -- 172.21.15.1:0/2897930247 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f3c6997660 con 0x55f3c6aeaa00 2023-11-16T16:42:08.821 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.812+0000 7f1d977fe700 1 --2- 172.21.15.1:0/2897930247 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55f3c6aeaa00 0x55f3c6aece10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:08.821 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.812+0000 7f1d76ffd700 10 client.16089 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:42:08.821 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.812+0000 7f1d977fe700 1 --2- 172.21.15.1:0/2897930247 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55f3c6aeaa00 0x55f3c6aece10 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bf91a3d8e707337 server_cookie=a9ec7ef523564a1 in_seq=0 out_seq=0 2023-11-16T16:42:08.821 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.813+0000 7f1d76ffd700 1 -- 172.21.15.1:0/2897930247 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x55f3c6997660 con 0x55f3c6aeaa00 2023-11-16T16:42:08.821 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.813+0000 7f1d76ffd700 10 client.16089 handle_command_reply: tid=0 2023-11-16T16:42:08.821 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.813+0000 7f1daa2b6b80 1 client.16089 shutdown 2023-11-16T16:42:08.821 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.813+0000 7f1daa2b6b80 1 -- 172.21.15.1:0/2897930247 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f3c6a78470 msgr2=0x55f3c6ae3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:08.822 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.813+0000 7f1daa2b6b80 1 --2- 172.21.15.1:0/2897930247 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f3c6a78470 0x55f3c6ae3620 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f1d800965a0 tx=0x7f1d80003780 comp rx=0 tx=0).stop 2023-11-16T16:42:08.822 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.814+0000 7f1daa2b6b80 1 -- 172.21.15.1:0/2897930247 shutdown_connections 2023-11-16T16:42:08.822 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.814+0000 7f1daa2b6b80 1 --2- 172.21.15.1:0/2897930247 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55f3c6aeaa00 0x55f3c6aece10 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.822 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.814+0000 7f1daa2b6b80 1 --2- 172.21.15.1:0/2897930247 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55f3c6a7c8b0 0x55f3c6a86990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.822 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.814+0000 7f1daa2b6b80 1 --2- 172.21.15.1:0/2897930247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f3c6a77b60 0x55f3c6ae30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.822 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.814+0000 7f1daa2b6b80 1 --2- 172.21.15.1:0/2897930247 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f3c6a78470 0x55f3c6ae3620 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.822 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.814+0000 7f1d76ffd700 0 client.16089 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:42:08.822 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.814+0000 7f1daa2b6b80 1 -- 172.21.15.1:0/2897930247 >> 172.21.15.1:0/2897930247 conn(0x55f3c6a6d360 msgr2=0x55f3c6a76960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:08.822 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.814+0000 7f1daa2b6b80 1 -- 172.21.15.1:0/2897930247 shutdown_connections 2023-11-16T16:42:08.822 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.814+0000 7f1daa2b6b80 1 -- 172.21.15.1:0/2897930247 wait complete. 2023-11-16T16:42:08.822 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.814+0000 7f1daa2b6b80 20 client.16089 trim_cache size 0 max 16384 2023-11-16T16:42:08.823 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:42:08.823 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "scrub active (3381 inodes in the stack)", 2023-11-16T16:42:08.823 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2023-11-16T16:42:08.823 INFO:teuthology.orchestra.run.smithi001.stdout: "4e68c117-e4ae-40d9-847d-0d6d5d2b1317": { 2023-11-16T16:42:08.823 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2023-11-16T16:42:08.823 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "4e68c117-e4ae-40d9-847d-0d6d5d2b1317", 2023-11-16T16:42:08.823 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2023-11-16T16:42:08.823 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:42:08.823 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:42:08.823 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:42:08.824 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.814+0000 7f1d9e12a700 1 -- 172.21.15.1:0/2062815478 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f1d840a12c0 msgr2=0x7f1d840a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:08.824 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.814+0000 7f1d9e12a700 1 --2- 172.21.15.1:0/2062815478 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f1d840a12c0 0x7f1d840a3770 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1d88009fd0 tx=0x7f1d88006210 comp rx=0 tx=0).stop 2023-11-16T16:42:08.824 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.814+0000 7f1d9e12a700 1 -- 172.21.15.1:0/2062815478 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d98100420 msgr2=0x7f1d98195880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:08.824 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.814+0000 7f1d9e12a700 1 --2- 172.21.15.1:0/2062815478 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d98100420 0x7f1d98195880 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f1d8000e9f0 tx=0x7f1d8000ed00 comp rx=0 tx=0).stop 2023-11-16T16:42:08.824 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.814+0000 7f1d9e12a700 1 -- 172.21.15.1:0/2062815478 shutdown_connections 2023-11-16T16:42:08.824 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.814+0000 7f1d9e12a700 1 --2- 172.21.15.1:0/2062815478 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f1d840a12c0 0x7f1d840a3770 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.824 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.814+0000 7f1d9e12a700 1 --2- 172.21.15.1:0/2062815478 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1d98105250 0x7f1d98199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.814+0000 7f1d9e12a700 1 --2- 172.21.15.1:0/2062815478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d980ffa70 0x7f1d98195340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.814+0000 7f1d9e12a700 1 --2- 172.21.15.1:0/2062815478 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d98100420 0x7f1d98195880 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:08.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.814+0000 7f1d9e12a700 1 -- 172.21.15.1:0/2062815478 >> 172.21.15.1:0/2062815478 conn(0x7f1d980763a0 msgr2=0x7f1d980fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:08.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.815+0000 7f1d9e12a700 1 -- 172.21.15.1:0/2062815478 shutdown_connections 2023-11-16T16:42:08.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:08.815+0000 7f1d9e12a700 1 -- 172.21.15.1:0/2062815478 wait complete. 2023-11-16T16:42:08.829 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:08 smithi001 ceph-mon[98800]: pgmap v92: 97 pgs: 97 active+clean; 68 MiB data, 148 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 28 op/s 2023-11-16T16:42:08.830 INFO:tasks.cephfs.filesystem:scrub status for tag:4e68c117-e4ae-40d9-847d-0d6d5d2b1317 - {'path': '/', 'tag': '4e68c117-e4ae-40d9-847d-0d6d5d2b1317', 'options': 'recursive,force'} 2023-11-16T16:42:09.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:08 smithi026 ceph-mon[100112]: pgmap v92: 97 pgs: 97 active+clean; 68 MiB data, 148 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 28 op/s 2023-11-16T16:42:09.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:08 smithi053 ceph-mon[100638]: pgmap v92: 97 pgs: 97 active+clean; 68 MiB data, 148 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 28 op/s 2023-11-16T16:42:09.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:09 smithi001 ceph-mon[98800]: scrub queued for path: / 2023-11-16T16:42:09.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:09 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:09.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:09 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:09.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:09 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:09.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:09 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:09.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:09 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:10.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:09 smithi026 ceph-mon[100112]: scrub queued for path: / 2023-11-16T16:42:10.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:09 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:10.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:09 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:10.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:09 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:10.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:09 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:10.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:09 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:10.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:09 smithi053 ceph-mon[100638]: scrub queued for path: / 2023-11-16T16:42:10.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:09 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:10.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:09 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:10.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:09 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:10.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:09 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:10.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:09 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:10.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:10 smithi001 ceph-mon[98800]: pgmap v93: 97 pgs: 97 active+clean; 68 MiB data, 148 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 28 op/s 2023-11-16T16:42:11.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:10 smithi026 ceph-mon[100112]: pgmap v93: 97 pgs: 97 active+clean; 68 MiB data, 148 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 28 op/s 2023-11-16T16:42:11.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:10 smithi053 ceph-mon[100638]: pgmap v93: 97 pgs: 97 active+clean; 68 MiB data, 148 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 28 op/s 2023-11-16T16:42:12.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:12 smithi026 ceph-mon[100112]: pgmap v94: 97 pgs: 97 active+clean; 81 MiB data, 161 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.0 MiB/s wr, 37 op/s 2023-11-16T16:42:12.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:12 smithi053 ceph-mon[100638]: pgmap v94: 97 pgs: 97 active+clean; 81 MiB data, 161 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.0 MiB/s wr, 37 op/s 2023-11-16T16:42:12.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:12 smithi001 ceph-mon[98800]: pgmap v94: 97 pgs: 97 active+clean; 81 MiB data, 161 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.0 MiB/s wr, 37 op/s 2023-11-16T16:42:14.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:14 smithi001 ceph-mon[98800]: pgmap v95: 97 pgs: 97 active+clean; 81 MiB data, 161 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 27 op/s 2023-11-16T16:42:15.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:14 smithi026 ceph-mon[100112]: pgmap v95: 97 pgs: 97 active+clean; 81 MiB data, 161 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 27 op/s 2023-11-16T16:42:15.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:14 smithi053 ceph-mon[100638]: pgmap v95: 97 pgs: 97 active+clean; 81 MiB data, 161 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 27 op/s 2023-11-16T16:42:16.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:16 smithi001 ceph-mon[98800]: pgmap v96: 97 pgs: 97 active+clean; 97 MiB data, 172 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.6 MiB/s wr, 41 op/s 2023-11-16T16:42:17.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:16 smithi026 ceph-mon[100112]: pgmap v96: 97 pgs: 97 active+clean; 97 MiB data, 172 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.6 MiB/s wr, 41 op/s 2023-11-16T16:42:17.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:16 smithi053 ceph-mon[100638]: pgmap v96: 97 pgs: 97 active+clean; 97 MiB data, 172 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.6 MiB/s wr, 41 op/s 2023-11-16T16:42:18.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:18 smithi001 ceph-mon[98800]: pgmap v97: 97 pgs: 97 active+clean; 97 MiB data, 172 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 22 op/s 2023-11-16T16:42:18.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:18 smithi001 ceph-mon[98800]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:42:19.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:18 smithi026 ceph-mon[100112]: pgmap v97: 97 pgs: 97 active+clean; 97 MiB data, 172 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 22 op/s 2023-11-16T16:42:19.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:18 smithi026 ceph-mon[100112]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:42:19.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:18 smithi053 ceph-mon[100638]: pgmap v97: 97 pgs: 97 active+clean; 97 MiB data, 172 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 22 op/s 2023-11-16T16:42:19.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:18 smithi053 ceph-mon[100638]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:42:20.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:20 smithi001 ceph-mon[98800]: pgmap v98: 97 pgs: 97 active+clean; 97 MiB data, 172 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 22 op/s 2023-11-16T16:42:21.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:20 smithi026 ceph-mon[100112]: pgmap v98: 97 pgs: 97 active+clean; 97 MiB data, 172 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 22 op/s 2023-11-16T16:42:21.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:20 smithi053 ceph-mon[100638]: pgmap v98: 97 pgs: 97 active+clean; 97 MiB data, 172 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 22 op/s 2023-11-16T16:42:22.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:22 smithi026 ceph-mon[100112]: pgmap v99: 97 pgs: 97 active+clean; 117 MiB data, 184 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.2 MiB/s wr, 32 op/s 2023-11-16T16:42:22.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:22 smithi053 ceph-mon[100638]: pgmap v99: 97 pgs: 97 active+clean; 117 MiB data, 184 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.2 MiB/s wr, 32 op/s 2023-11-16T16:42:22.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:22 smithi001 ceph-mon[98800]: pgmap v99: 97 pgs: 97 active+clean; 117 MiB data, 184 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.2 MiB/s wr, 32 op/s 2023-11-16T16:42:24.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:24 smithi001 ceph-mon[98800]: pgmap v100: 97 pgs: 97 active+clean; 117 MiB data, 184 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 23 op/s 2023-11-16T16:42:25.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:24 smithi026 ceph-mon[100112]: pgmap v100: 97 pgs: 97 active+clean; 117 MiB data, 184 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 23 op/s 2023-11-16T16:42:25.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:24 smithi053 ceph-mon[100638]: pgmap v100: 97 pgs: 97 active+clean; 117 MiB data, 184 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 23 op/s 2023-11-16T16:42:26.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:26 smithi001 ceph-mon[98800]: pgmap v101: 97 pgs: 97 active+clean; 138 MiB data, 231 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.3 MiB/s wr, 144 op/s 2023-11-16T16:42:26.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:26 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:42:27.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:26 smithi026 ceph-mon[100112]: pgmap v101: 97 pgs: 97 active+clean; 138 MiB data, 231 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.3 MiB/s wr, 144 op/s 2023-11-16T16:42:27.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:26 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:42:27.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:26 smithi053 ceph-mon[100638]: pgmap v101: 97 pgs: 97 active+clean; 138 MiB data, 231 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.3 MiB/s wr, 144 op/s 2023-11-16T16:42:27.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:26 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:42:28.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:28 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:42:28.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:28 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:42:28.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:28 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:42:28.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:28 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:42:28.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:28 smithi001 ceph-mon[98800]: pgmap v102: 97 pgs: 97 active+clean; 138 MiB data, 231 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.9 MiB/s wr, 131 op/s 2023-11-16T16:42:28.507 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:28 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:42:28.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:28 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:42:28.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:28 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:42:28.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:28 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:42:28.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:28 smithi026 ceph-mon[100112]: pgmap v102: 97 pgs: 97 active+clean; 138 MiB data, 231 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.9 MiB/s wr, 131 op/s 2023-11-16T16:42:28.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:28 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:42:28.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:28 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:42:28.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:28 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:42:28.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:28 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:42:28.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:28 smithi053 ceph-mon[100638]: pgmap v102: 97 pgs: 97 active+clean; 138 MiB data, 231 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.9 MiB/s wr, 131 op/s 2023-11-16T16:42:29.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:29 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:42:29.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:29 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:42:29.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:29 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:42:29.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:29 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:42:29.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:29 smithi001 ceph-mon[98800]: Service mds.all has no related daemon already placed. 2023-11-16T16:42:29.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:29 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:42:29.433 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:29 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:42:29.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:29 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:42:29.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:29 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:42:29.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:29 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:42:29.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:29 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:42:29.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:29 smithi026 ceph-mon[100112]: Service mds.all has no related daemon already placed. 2023-11-16T16:42:29.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:29 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:42:29.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:29 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:42:29.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:29 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:42:29.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:29 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:42:29.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:29 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:42:29.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:29 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:42:29.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:29 smithi053 ceph-mon[100638]: Service mds.all has no related daemon already placed. 2023-11-16T16:42:29.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:29 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:42:29.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:29 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:42:30.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:30 smithi001 ceph-mon[98800]: pgmap v103: 97 pgs: 97 active+clean; 138 MiB data, 231 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.9 MiB/s wr, 131 op/s 2023-11-16T16:42:30.507 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:30 smithi026 ceph-mon[100112]: pgmap v103: 97 pgs: 97 active+clean; 138 MiB data, 231 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.9 MiB/s wr, 131 op/s 2023-11-16T16:42:30.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:30 smithi053 ceph-mon[100638]: pgmap v103: 97 pgs: 97 active+clean; 138 MiB data, 231 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.9 MiB/s wr, 131 op/s 2023-11-16T16:42:32.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:32 smithi026 ceph-mon[100112]: pgmap v104: 97 pgs: 97 active+clean; 152 MiB data, 288 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.7 MiB/s wr, 292 op/s 2023-11-16T16:42:32.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:32 smithi053 ceph-mon[100638]: pgmap v104: 97 pgs: 97 active+clean; 152 MiB data, 288 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.7 MiB/s wr, 292 op/s 2023-11-16T16:42:32.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:32 smithi001 ceph-mon[98800]: pgmap v104: 97 pgs: 97 active+clean; 152 MiB data, 288 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.7 MiB/s wr, 292 op/s 2023-11-16T16:42:34.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:34 smithi001 ceph-mon[98800]: pgmap v105: 97 pgs: 97 active+clean; 152 MiB data, 288 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 282 op/s 2023-11-16T16:42:35.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:34 smithi026 ceph-mon[100112]: pgmap v105: 97 pgs: 97 active+clean; 152 MiB data, 288 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 282 op/s 2023-11-16T16:42:35.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:34 smithi053 ceph-mon[100638]: pgmap v105: 97 pgs: 97 active+clean; 152 MiB data, 288 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.0 MiB/s wr, 282 op/s 2023-11-16T16:42:36.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:36 smithi001 ceph-mon[98800]: pgmap v106: 97 pgs: 97 active+clean; 167 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.9 MiB/s wr, 465 op/s 2023-11-16T16:42:37.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:36 smithi026 ceph-mon[100112]: pgmap v106: 97 pgs: 97 active+clean; 167 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.9 MiB/s wr, 465 op/s 2023-11-16T16:42:37.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:36 smithi053 ceph-mon[100638]: pgmap v106: 97 pgs: 97 active+clean; 167 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.9 MiB/s wr, 465 op/s 2023-11-16T16:42:38.831 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-16T16:42:38.859 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:38 smithi001 ceph-mon[98800]: pgmap v107: 97 pgs: 97 active+clean; 167 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.7 MiB/s wr, 344 op/s 2023-11-16T16:42:38.976 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.972+0000 7f7984503700 1 -- 172.21.15.1:0/4097701863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f797c101e60 msgr2=0x7f797c10a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:38.976 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.972+0000 7f7984503700 1 --2- 172.21.15.1:0/4097701863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f797c101e60 0x7f797c10a350 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f796c0098f0 tx=0x7f796c009c00 comp rx=0 tx=0).stop 2023-11-16T16:42:38.976 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.972+0000 7f7984503700 1 -- 172.21.15.1:0/4097701863 shutdown_connections 2023-11-16T16:42:38.976 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.972+0000 7f7984503700 1 --2- 172.21.15.1:0/4097701863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f797c10a890 0x7f797c10ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:38.976 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.972+0000 7f7984503700 1 --2- 172.21.15.1:0/4097701863 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f797c101550 0x7f797c101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:38.976 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.972+0000 7f7984503700 1 --2- 172.21.15.1:0/4097701863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f797c101e60 0x7f797c10a350 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:38.976 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.972+0000 7f7984503700 1 -- 172.21.15.1:0/4097701863 >> 172.21.15.1:0/4097701863 conn(0x7f797c0763a0 msgr2=0x7f797c0767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:38.977 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.972+0000 7f7984503700 1 -- 172.21.15.1:0/4097701863 shutdown_connections 2023-11-16T16:42:38.977 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.972+0000 7f7984503700 1 -- 172.21.15.1:0/4097701863 wait complete. 2023-11-16T16:42:38.977 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.972+0000 7f7984503700 1 Processor -- start 2023-11-16T16:42:38.977 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.972+0000 7f7984503700 1 -- start start 2023-11-16T16:42:38.977 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.973+0000 7f7984503700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f797c101550 0x7f797c1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:38.977 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.973+0000 7f7984503700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f797c101e60 0x7f797c198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:38.977 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.973+0000 7f7984503700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f797c10a890 0x7f797c19cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:38.977 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.973+0000 7f7984503700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797c19d5c0 con 0x7f797c101550 2023-11-16T16:42:38.977 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.973+0000 7f7984503700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797c19d730 con 0x7f797c101e60 2023-11-16T16:42:38.978 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.973+0000 7f7984503700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797c19d8f0 con 0x7f797c10a890 2023-11-16T16:42:38.978 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.973+0000 7f798229f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f797c101550 0x7f797c1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:38.978 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.973+0000 7f798229f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f797c101550 0x7f797c1985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45096/0 (socket says 172.21.15.1:45096) 2023-11-16T16:42:38.978 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.973+0000 7f798229f700 1 -- 172.21.15.1:0/348033336 learned_addr learned my addr 172.21.15.1:0/348033336 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:42:38.978 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.973+0000 7f7981a9e700 1 --2- 172.21.15.1:0/348033336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f797c101e60 0x7f797c198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:38.978 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.973+0000 7f7982aa0700 1 --2- 172.21.15.1:0/348033336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f797c10a890 0x7f797c19cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:38.978 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.973+0000 7f798229f700 1 -- 172.21.15.1:0/348033336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f797c10a890 msgr2=0x7f797c19cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:38.978 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.973+0000 7f798229f700 1 --2- 172.21.15.1:0/348033336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f797c10a890 0x7f797c19cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:38.978 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.973+0000 7f798229f700 1 -- 172.21.15.1:0/348033336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f797c101e60 msgr2=0x7f797c198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:38.979 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.973+0000 7f798229f700 1 --2- 172.21.15.1:0/348033336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f797c101e60 0x7f797c198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:38.979 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.973+0000 7f798229f700 1 -- 172.21.15.1:0/348033336 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79780095f0 con 0x7f797c101550 2023-11-16T16:42:38.979 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.973+0000 7f798229f700 1 --2- 172.21.15.1:0/348033336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f797c101550 0x7f797c1985e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f797800ec10 tx=0x7f797800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:38.979 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.974+0000 7f79737fe700 1 -- 172.21.15.1:0/348033336 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797800cc90 con 0x7f797c101550 2023-11-16T16:42:38.979 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.974+0000 7f7984503700 1 -- 172.21.15.1:0/348033336 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f796c009520 con 0x7f797c101550 2023-11-16T16:42:38.979 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.974+0000 7f79737fe700 1 -- 172.21.15.1:0/348033336 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79780044e0 con 0x7f797c101550 2023-11-16T16:42:38.979 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.974+0000 7f79737fe700 1 -- 172.21.15.1:0/348033336 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7978010430 con 0x7f797c101550 2023-11-16T16:42:38.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.974+0000 7f7984503700 1 -- 172.21.15.1:0/348033336 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f797c1125d0 con 0x7f797c101550 2023-11-16T16:42:38.986 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.976+0000 7f79737fe700 1 -- 172.21.15.1:0/348033336 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f7978003fb0 con 0x7f797c101550 2023-11-16T16:42:38.986 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.977+0000 7f79737fe700 1 --2- 172.21.15.1:0/348033336 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f79680a12c0 0x7f79680a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:38.986 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.977+0000 7f7981a9e700 1 --2- 172.21.15.1:0/348033336 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f79680a12c0 0x7f79680a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:38.986 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.977+0000 7f79737fe700 1 -- 172.21.15.1:0/348033336 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f7978014070 con 0x7f797c101550 2023-11-16T16:42:38.986 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.977+0000 7f7981a9e700 1 --2- 172.21.15.1:0/348033336 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f79680a12c0 0x7f79680a3770 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f796c009fd0 tx=0x7f796c019040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:38.986 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.978+0000 7f799068fb80 1 Processor -- start 2023-11-16T16:42:38.986 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.978+0000 7f799068fb80 1 -- start start 2023-11-16T16:42:38.986 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.978+0000 7f799068fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56247c8ecb60 0x56247c8ecf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:38.987 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.978+0000 7f799068fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56247c8ed470 0x56247c8f1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:38.987 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.978+0000 7f799068fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247c8f18b0 0x56247c8f1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:38.987 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.979+0000 7f799068fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247c7f2060 con 0x56247c8ecb60 2023-11-16T16:42:38.987 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.979+0000 7f799068fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247c8448a0 con 0x56247c8ed470 2023-11-16T16:42:38.987 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.979+0000 7f799068fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247c83dd30 con 0x56247c8f18b0 2023-11-16T16:42:38.987 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.979+0000 7f798229f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56247c8ecb60 0x56247c8ecf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:38.987 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.979+0000 7f798229f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56247c8ecb60 0x56247c8ecf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45104/0 (socket says 172.21.15.1:45104) 2023-11-16T16:42:38.987 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.979+0000 7f798229f700 1 -- 172.21.15.1:0/2803240360 learned_addr learned my addr 172.21.15.1:0/2803240360 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:42:38.987 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.979+0000 7f7982aa0700 1 --2- 172.21.15.1:0/2803240360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247c8f18b0 0x56247c8f1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:38.987 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.979+0000 7f7981a9e700 1 --2- 172.21.15.1:0/2803240360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56247c8ed470 0x56247c8f1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:38.988 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.979+0000 7f798229f700 1 -- 172.21.15.1:0/2803240360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247c8f18b0 msgr2=0x56247c8f1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:38.988 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.979+0000 7f798229f700 1 --2- 172.21.15.1:0/2803240360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247c8f18b0 0x56247c8f1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:38.988 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.979+0000 7f798229f700 1 -- 172.21.15.1:0/2803240360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56247c8ed470 msgr2=0x56247c8f1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:38.988 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.979+0000 7f798229f700 1 --2- 172.21.15.1:0/2803240360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56247c8ed470 0x56247c8f1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:38.988 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.979+0000 7f798229f700 1 -- 172.21.15.1:0/2803240360 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56247c6bf630 con 0x56247c8ecb60 2023-11-16T16:42:38.988 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.979+0000 7f7982aa0700 1 --2- 172.21.15.1:0/2803240360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247c8f18b0 0x56247c8f1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:42:38.988 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.979+0000 7f798229f700 1 --2- 172.21.15.1:0/2803240360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56247c8ecb60 0x56247c8ecf30 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f79780a2230 tx=0x7f797809a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e5e551eccf96a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:38.988 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.979+0000 7f7970ff9700 1 -- 172.21.15.1:0/2803240360 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797809e170 con 0x56247c8ecb60 2023-11-16T16:42:38.988 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.979+0000 7f7970ff9700 1 -- 172.21.15.1:0/2803240360 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f797809a310 con 0x56247c8ecb60 2023-11-16T16:42:38.989 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.979+0000 7f7970ff9700 1 -- 172.21.15.1:0/2803240360 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797809a570 con 0x56247c8ecb60 2023-11-16T16:42:38.989 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.980+0000 7f799068fb80 1 -- 172.21.15.1:0/2803240360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56247c8ecb60 msgr2=0x56247c8ecf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:38.989 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.980+0000 7f799068fb80 1 --2- 172.21.15.1:0/2803240360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56247c8ecb60 0x56247c8ecf30 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f79780a2230 tx=0x7f797809a160 comp rx=0 tx=0).stop 2023-11-16T16:42:38.989 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.980+0000 7f799068fb80 1 -- 172.21.15.1:0/2803240360 shutdown_connections 2023-11-16T16:42:38.989 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.980+0000 7f799068fb80 1 --2- 172.21.15.1:0/2803240360 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247c8f18b0 0x56247c8f1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:38.989 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.980+0000 7f799068fb80 1 --2- 172.21.15.1:0/2803240360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56247c8ecb60 0x56247c8ecf30 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:38.989 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.980+0000 7f799068fb80 1 --2- 172.21.15.1:0/2803240360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56247c8ed470 0x56247c8f1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:38.989 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.980+0000 7f799068fb80 1 -- 172.21.15.1:0/2803240360 >> 172.21.15.1:0/2803240360 conn(0x56247c8e2360 msgr2=0x56247c8e7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:38.989 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.980+0000 7f799068fb80 1 -- 172.21.15.1:0/2803240360 shutdown_connections 2023-11-16T16:42:38.990 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.980+0000 7f799068fb80 1 -- 172.21.15.1:0/2803240360 wait complete. 2023-11-16T16:42:38.990 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.981+0000 7f799068fb80 1 -- start start 2023-11-16T16:42:38.990 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.981+0000 7f799068fb80 1 Processor -- start 2023-11-16T16:42:38.990 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.981+0000 7f799068fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56247c8ecb60 0x56247c8f90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:38.990 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.981+0000 7f799068fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247c8ed470 0x56247c8f63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:38.990 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.981+0000 7f799068fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56247c8f18b0 0x56247c8f6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:38.990 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.981+0000 7f799068fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247c7df8e0 con 0x56247c8ecb60 2023-11-16T16:42:38.990 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.981+0000 7f799068fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247c7df580 con 0x56247c8f18b0 2023-11-16T16:42:38.990 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.981+0000 7f799068fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247c6c1920 con 0x56247c8ed470 2023-11-16T16:42:38.991 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.981+0000 7f799068fb80 10 client.0 _ll_register_callbacks cb 0x56247c7fd780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:42:38.991 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.981+0000 7f798229f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56247c8ecb60 0x56247c8f90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:38.991 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.981+0000 7f798229f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56247c8ecb60 0x56247c8f90f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45110/0 (socket says 172.21.15.1:45110) 2023-11-16T16:42:38.991 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.981+0000 7f798229f700 1 -- 172.21.15.1:0/2354205557 learned_addr learned my addr 172.21.15.1:0/2354205557 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:42:38.991 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.981+0000 7f7981a9e700 1 --2- 172.21.15.1:0/2354205557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247c8ed470 0x56247c8f63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:38.991 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.981+0000 7f7982aa0700 1 --2- 172.21.15.1:0/2354205557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56247c8f18b0 0x56247c8f6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:38.991 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.981+0000 7f798229f700 1 -- 172.21.15.1:0/2354205557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247c8ed470 msgr2=0x56247c8f63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:38.991 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.981+0000 7f798229f700 1 --2- 172.21.15.1:0/2354205557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247c8ed470 0x56247c8f63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:38.991 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.981+0000 7f798229f700 1 -- 172.21.15.1:0/2354205557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56247c8f18b0 msgr2=0x56247c8f6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:38.992 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.981+0000 7f798229f700 1 --2- 172.21.15.1:0/2354205557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56247c8f18b0 0x56247c8f6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:38.992 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.981+0000 7f798229f700 1 -- 172.21.15.1:0/2354205557 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f797c1125d0 con 0x56247c8ecb60 2023-11-16T16:42:38.992 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.982+0000 7f7981a9e700 1 --2- 172.21.15.1:0/2354205557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247c8ed470 0x56247c8f63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:42:38.992 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.982+0000 7f798229f700 1 --2- 172.21.15.1:0/2354205557 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56247c8ecb60 0x56247c8f90f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f79780caf90 tx=0x7f7978009a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e6ae4bffe303ed server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:38.992 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.982+0000 7f7965ffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:42:38.992 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.982+0000 7f7965ffb700 1 -- 172.21.15.1:0/2354205557 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797809e340 con 0x56247c8ecb60 2023-11-16T16:42:38.992 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.982+0000 7f7965ffb700 1 -- 172.21.15.1:0/2354205557 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f79780a2760 con 0x56247c8ecb60 2023-11-16T16:42:38.992 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.982+0000 7f799068fb80 1 -- 172.21.15.1:0/2354205557 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56247c8dfb20 con 0x56247c8ecb60 2023-11-16T16:42:38.992 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.982+0000 7f7965ffb700 1 -- 172.21.15.1:0/2354205557 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7978096a20 con 0x56247c8ecb60 2023-11-16T16:42:38.992 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.982+0000 7f799068fb80 10 client.16101 fetch_fsmap learned FSMap version 26 2023-11-16T16:42:38.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.982+0000 7f799068fb80 1 -- 172.21.15.1:0/2354205557 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56247c804190 con 0x56247c8ecb60 2023-11-16T16:42:38.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.983+0000 7f7965ffb700 1 -- 172.21.15.1:0/2354205557 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56247c8dfb20 con 0x56247c8ecb60 2023-11-16T16:42:38.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.983+0000 7f7965ffb700 1 -- 172.21.15.1:0/2354205557 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f79780a22d0 con 0x56247c8ecb60 2023-11-16T16:42:38.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.983+0000 7f799068fb80 10 client.16101 fetch_fsmap finished waiting for FSMap version 26 2023-11-16T16:42:38.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.983+0000 7f799068fb80 10 client.16101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:42:38.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.983+0000 7f799068fb80 20 client.16101 populate_metadata read hostname 'smithi001' 2023-11-16T16:42:38.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.983+0000 7f799068fb80 1 --2- 172.21.15.1:0/2354205557 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56247c906450 0x56247c908860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:38.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.983+0000 7f799068fb80 4 client.16101 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:42:38.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.983+0000 7f799068fb80 1 -- 172.21.15.1:0/2354205557 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56247c8000a0 con 0x56247c906450 2023-11-16T16:42:38.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.983+0000 7f7981a9e700 1 --2- 172.21.15.1:0/2354205557 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56247c906450 0x56247c908860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:38.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.983+0000 7f7965ffb700 10 client.16101 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:42:38.994 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.983+0000 7f7981a9e700 1 --2- 172.21.15.1:0/2354205557 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56247c906450 0x56247c908860 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d596ce7be9448f6c server_cookie=7327374cdaa4219f in_seq=0 out_seq=0 2023-11-16T16:42:38.995 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.985+0000 7f7965ffb700 1 -- 172.21.15.1:0/2354205557 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56247c8000a0 con 0x56247c906450 2023-11-16T16:42:38.995 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.985+0000 7f7965ffb700 10 client.16101 handle_command_reply: tid=0 2023-11-16T16:42:38.995 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.985+0000 7f799068fb80 1 client.16101 shutdown 2023-11-16T16:42:38.995 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.985+0000 7f799068fb80 1 -- 172.21.15.1:0/2354205557 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56247c8ecb60 msgr2=0x56247c8f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:38.995 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.985+0000 7f799068fb80 1 --2- 172.21.15.1:0/2354205557 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56247c8ecb60 0x56247c8f90f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f79780caf90 tx=0x7f7978009a80 comp rx=0 tx=0).stop 2023-11-16T16:42:38.995 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.985+0000 7f799068fb80 1 -- 172.21.15.1:0/2354205557 shutdown_connections 2023-11-16T16:42:38.995 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.985+0000 7f799068fb80 1 --2- 172.21.15.1:0/2354205557 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56247c906450 0x56247c908860 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:38.996 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.985+0000 7f799068fb80 1 --2- 172.21.15.1:0/2354205557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56247c8f18b0 0x56247c8f6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:38.996 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.985+0000 7f799068fb80 1 --2- 172.21.15.1:0/2354205557 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56247c8ecb60 0x56247c8f90f0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:38.996 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.985+0000 7f799068fb80 1 --2- 172.21.15.1:0/2354205557 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247c8ed470 0x56247c8f63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:38.996 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.985+0000 7f7965ffb700 0 client.16101 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:42:38.996 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.985+0000 7f799068fb80 1 -- 172.21.15.1:0/2354205557 >> 172.21.15.1:0/2354205557 conn(0x56247c8e2360 msgr2=0x56247c8f2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:38.996 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.985+0000 7f799068fb80 1 -- 172.21.15.1:0/2354205557 shutdown_connections 2023-11-16T16:42:38.996 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.985+0000 7f799068fb80 1 -- 172.21.15.1:0/2354205557 wait complete. 2023-11-16T16:42:38.996 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:38.986+0000 7f799068fb80 20 client.16101 trim_cache size 0 max 16384 2023-11-16T16:42:39.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:38 smithi026 ceph-mon[100112]: pgmap v107: 97 pgs: 97 active+clean; 167 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.7 MiB/s wr, 344 op/s 2023-11-16T16:42:39.010 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.006+0000 7f799068fb80 1 Processor -- start 2023-11-16T16:42:39.010 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.006+0000 7f799068fb80 1 -- start start 2023-11-16T16:42:39.010 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.007+0000 7f799068fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56247c8ecb60 0x56247c8f8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:39.010 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.007+0000 7f799068fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247c8ed470 0x56247c8efa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:39.010 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.007+0000 7f799068fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56247c8f18b0 0x56247c8eff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:39.010 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.007+0000 7f799068fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247c8f7930 con 0x56247c8ecb60 2023-11-16T16:42:39.010 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.007+0000 7f799068fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247c8e91c0 con 0x56247c8f18b0 2023-11-16T16:42:39.010 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.007+0000 7f799068fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247c8f4940 con 0x56247c8ed470 2023-11-16T16:42:39.011 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.007+0000 7f798229f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56247c8ecb60 0x56247c8f8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:39.011 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.007+0000 7f798229f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56247c8ecb60 0x56247c8f8c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45118/0 (socket says 172.21.15.1:45118) 2023-11-16T16:42:39.011 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.007+0000 7f798229f700 1 -- 172.21.15.1:0/1930828368 learned_addr learned my addr 172.21.15.1:0/1930828368 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:42:39.011 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.007+0000 7f7981a9e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247c8ed470 0x56247c8efa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:39.011 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.007+0000 7f7982aa0700 1 --2- 172.21.15.1:0/1930828368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56247c8f18b0 0x56247c8eff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:39.011 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.007+0000 7f798229f700 1 -- 172.21.15.1:0/1930828368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247c8ed470 msgr2=0x56247c8efa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:39.011 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.007+0000 7f798229f700 1 --2- 172.21.15.1:0/1930828368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247c8ed470 0x56247c8efa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:39.011 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.007+0000 7f798229f700 1 -- 172.21.15.1:0/1930828368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56247c8f18b0 msgr2=0x56247c8eff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:39.011 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.007+0000 7f798229f700 1 --2- 172.21.15.1:0/1930828368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56247c8f18b0 0x56247c8eff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:39.011 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.007+0000 7f798229f700 1 -- 172.21.15.1:0/1930828368 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56247c8e3b00 con 0x56247c8ecb60 2023-11-16T16:42:39.012 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.007+0000 7f7981a9e700 1 --2- 172.21.15.1:0/1930828368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247c8ed470 0x56247c8efa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:42:39.012 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.007+0000 7f798229f700 1 --2- 172.21.15.1:0/1930828368 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56247c8ecb60 0x56247c8f8c90 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f797809a010 tx=0x7f7978004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee95c8c99d3b8c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:39.012 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.007+0000 7f79717fa700 1 -- 172.21.15.1:0/1930828368 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7978099e50 con 0x56247c8ecb60 2023-11-16T16:42:39.012 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.008+0000 7f79717fa700 1 -- 172.21.15.1:0/1930828368 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f797809e170 con 0x56247c8ecb60 2023-11-16T16:42:39.012 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.008+0000 7f79717fa700 1 -- 172.21.15.1:0/1930828368 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7978099800 con 0x56247c8ecb60 2023-11-16T16:42:39.012 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.008+0000 7f799068fb80 1 -- 172.21.15.1:0/1930828368 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56247c8ecb60 msgr2=0x56247c8f8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:39.012 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.008+0000 7f799068fb80 1 --2- 172.21.15.1:0/1930828368 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56247c8ecb60 0x56247c8f8c90 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f797809a010 tx=0x7f7978004470 comp rx=0 tx=0).stop 2023-11-16T16:42:39.017 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.008+0000 7f799068fb80 1 -- 172.21.15.1:0/1930828368 shutdown_connections 2023-11-16T16:42:39.017 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.008+0000 7f799068fb80 1 --2- 172.21.15.1:0/1930828368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56247c8f18b0 0x56247c8eff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:39.017 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.008+0000 7f799068fb80 1 --2- 172.21.15.1:0/1930828368 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56247c8ecb60 0x56247c8f8c90 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:39.017 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.008+0000 7f799068fb80 1 --2- 172.21.15.1:0/1930828368 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247c8ed470 0x56247c8efa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:39.017 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.008+0000 7f799068fb80 1 -- 172.21.15.1:0/1930828368 >> 172.21.15.1:0/1930828368 conn(0x56247c8e2360 msgr2=0x56247c8f3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:39.017 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.008+0000 7f799068fb80 1 -- 172.21.15.1:0/1930828368 shutdown_connections 2023-11-16T16:42:39.017 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.008+0000 7f799068fb80 1 -- 172.21.15.1:0/1930828368 wait complete. 2023-11-16T16:42:39.017 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.009+0000 7f799068fb80 1 -- start start 2023-11-16T16:42:39.017 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.009+0000 7f799068fb80 1 Processor -- start 2023-11-16T16:42:39.018 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.009+0000 7f799068fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56247c8ecb60 0x56247c8fdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:39.018 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.009+0000 7f799068fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247c8ed470 0x56247c8fb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:39.018 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.009+0000 7f799068fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56247c8f18b0 0x56247c8fbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:39.018 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.009+0000 7f799068fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247c8fc5e0 con 0x56247c8ecb60 2023-11-16T16:42:39.018 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.009+0000 7f799068fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247c8fc750 con 0x56247c8f18b0 2023-11-16T16:42:39.018 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.009+0000 7f799068fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247c9592d0 con 0x56247c8ed470 2023-11-16T16:42:39.018 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.009+0000 7f799068fb80 10 client.0 _ll_register_callbacks cb 0x7f796c00bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:42:39.018 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.009+0000 7f798229f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56247c8ecb60 0x56247c8fdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:39.018 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.009+0000 7f7981a9e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247c8ed470 0x56247c8fb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:39.019 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.009+0000 7f798229f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56247c8ecb60 0x56247c8fdba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45124/0 (socket says 172.21.15.1:45124) 2023-11-16T16:42:39.019 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.009+0000 7f798229f700 1 -- 172.21.15.1:0/1170161015 learned_addr learned my addr 172.21.15.1:0/1170161015 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:42:39.019 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.010+0000 7f7982aa0700 1 --2- 172.21.15.1:0/1170161015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56247c8f18b0 0x56247c8fbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:39.019 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.010+0000 7f798229f700 1 -- 172.21.15.1:0/1170161015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247c8ed470 msgr2=0x56247c8fb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:39.019 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.010+0000 7f798229f700 1 --2- 172.21.15.1:0/1170161015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247c8ed470 0x56247c8fb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:39.019 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.010+0000 7f798229f700 1 -- 172.21.15.1:0/1170161015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56247c8f18b0 msgr2=0x56247c8fbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:42:39.019 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.010+0000 7f798229f700 1 --2- 172.21.15.1:0/1170161015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56247c8f18b0 0x56247c8fbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:39.019 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.010+0000 7f798229f700 1 -- 172.21.15.1:0/1170161015 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f797c10d990 con 0x56247c8ecb60 2023-11-16T16:42:39.019 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.010+0000 7f7981a9e700 1 --2- 172.21.15.1:0/1170161015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247c8ed470 0x56247c8fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:42:39.020 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.010+0000 7f7982aa0700 1 --2- 172.21.15.1:0/1170161015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56247c8f18b0 0x56247c8fbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:42:39.020 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.010+0000 7f798229f700 1 --2- 172.21.15.1:0/1170161015 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56247c8ecb60 0x56247c8fdba0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f797809a2a0 tx=0x7f7978099ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905710c346c1429d server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:42:39.020 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.010+0000 7f7965ffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:42:39.020 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.010+0000 7f7965ffb700 1 -- 172.21.15.1:0/1170161015 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79780a2760 con 0x56247c8ecb60 2023-11-16T16:42:39.020 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.010+0000 7f7965ffb700 1 -- 172.21.15.1:0/1170161015 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f797809e340 con 0x56247c8ecb60 2023-11-16T16:42:39.020 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.010+0000 7f799068fb80 1 -- 172.21.15.1:0/1170161015 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f797809a310 con 0x56247c8ecb60 2023-11-16T16:42:39.020 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.010+0000 7f7965ffb700 1 -- 172.21.15.1:0/1170161015 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7978099800 con 0x56247c8ecb60 2023-11-16T16:42:39.020 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.010+0000 7f799068fb80 10 client.16107 fetch_fsmap learned FSMap version 26 2023-11-16T16:42:39.020 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.010+0000 7f799068fb80 1 -- 172.21.15.1:0/1170161015 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56247c804190 con 0x56247c8ecb60 2023-11-16T16:42:39.020 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.010+0000 7f7965ffb700 1 -- 172.21.15.1:0/1170161015 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f797809a310 con 0x56247c8ecb60 2023-11-16T16:42:39.021 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.011+0000 7f7965ffb700 1 -- 172.21.15.1:0/1170161015 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f7978096220 con 0x56247c8ecb60 2023-11-16T16:42:39.021 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.011+0000 7f799068fb80 10 client.16107 fetch_fsmap finished waiting for FSMap version 26 2023-11-16T16:42:39.021 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.011+0000 7f799068fb80 10 client.16107 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:42:39.021 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.011+0000 7f799068fb80 20 client.16107 populate_metadata read hostname 'smithi001' 2023-11-16T16:42:39.021 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.011+0000 7f799068fb80 1 --2- 172.21.15.1:0/1170161015 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56247c95a4b0 0x56247c95c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:42:39.021 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.011+0000 7f799068fb80 4 client.16107 mds_command: new command op to 24521 tid=0[{"prefix": "scrub status"}] 2023-11-16T16:42:39.021 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.011+0000 7f799068fb80 1 -- 172.21.15.1:0/1170161015 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56247c80c660 con 0x56247c95a4b0 2023-11-16T16:42:39.021 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.011+0000 7f7981a9e700 1 --2- 172.21.15.1:0/1170161015 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56247c95a4b0 0x56247c95c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:42:39.021 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.011+0000 7f7965ffb700 10 client.16107 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:42:39.021 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.011+0000 7f7981a9e700 1 --2- 172.21.15.1:0/1170161015 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56247c95a4b0 0x56247c95c890 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41ebd8a9be0ca7b6 server_cookie=663e736d83201746 in_seq=0 out_seq=0 2023-11-16T16:42:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.013+0000 7f7965ffb700 1 -- 172.21.15.1:0/1170161015 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x56247c80c660 con 0x56247c95a4b0 2023-11-16T16:42:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.013+0000 7f7965ffb700 10 client.16107 handle_command_reply: tid=0 2023-11-16T16:42:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.013+0000 7f799068fb80 1 client.16107 shutdown 2023-11-16T16:42:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.013+0000 7f799068fb80 1 -- 172.21.15.1:0/1170161015 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56247c8ecb60 msgr2=0x56247c8fdba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.013+0000 7f799068fb80 1 --2- 172.21.15.1:0/1170161015 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56247c8ecb60 0x56247c8fdba0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f797809a2a0 tx=0x7f7978099ef0 comp rx=0 tx=0).stop 2023-11-16T16:42:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.013+0000 7f799068fb80 1 -- 172.21.15.1:0/1170161015 shutdown_connections 2023-11-16T16:42:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.013+0000 7f799068fb80 1 --2- 172.21.15.1:0/1170161015 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56247c95a4b0 0x56247c95c890 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.013+0000 7f799068fb80 1 --2- 172.21.15.1:0/1170161015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56247c8f18b0 0x56247c8fbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.013+0000 7f799068fb80 1 --2- 172.21.15.1:0/1170161015 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56247c8ecb60 0x56247c8fdba0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:39.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.013+0000 7f799068fb80 1 --2- 172.21.15.1:0/1170161015 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56247c8ed470 0x56247c8fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:39.023 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.013+0000 7f7965ffb700 0 client.16107 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:42:39.023 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.013+0000 7f799068fb80 1 -- 172.21.15.1:0/1170161015 >> 172.21.15.1:0/1170161015 conn(0x56247c8e2360 msgr2=0x56247c8efa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:39.023 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.013+0000 7f799068fb80 1 -- 172.21.15.1:0/1170161015 shutdown_connections 2023-11-16T16:42:39.023 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.013+0000 7f799068fb80 1 -- 172.21.15.1:0/1170161015 wait complete. 2023-11-16T16:42:39.023 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.014+0000 7f799068fb80 20 client.16107 trim_cache size 0 max 16384 2023-11-16T16:42:39.023 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:42:39.023 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "scrub active (1 inodes in the stack)", 2023-11-16T16:42:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2023-11-16T16:42:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: "4e68c117-e4ae-40d9-847d-0d6d5d2b1317": { 2023-11-16T16:42:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2023-11-16T16:42:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "4e68c117-e4ae-40d9-847d-0d6d5d2b1317", 2023-11-16T16:42:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2023-11-16T16:42:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:42:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:42:39.024 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:42:39.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.014+0000 7f7984503700 1 -- 172.21.15.1:0/348033336 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f79680a12c0 msgr2=0x7f79680a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:39.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.014+0000 7f7984503700 1 --2- 172.21.15.1:0/348033336 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f79680a12c0 0x7f79680a3770 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f796c009fd0 tx=0x7f796c019040 comp rx=0 tx=0).stop 2023-11-16T16:42:39.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.014+0000 7f7984503700 1 -- 172.21.15.1:0/348033336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f797c101550 msgr2=0x7f797c1985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:42:39.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.014+0000 7f7984503700 1 --2- 172.21.15.1:0/348033336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f797c101550 0x7f797c1985e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f797800ec10 tx=0x7f797800ef20 comp rx=0 tx=0).stop 2023-11-16T16:42:39.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.014+0000 7f7984503700 1 -- 172.21.15.1:0/348033336 shutdown_connections 2023-11-16T16:42:39.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.014+0000 7f7984503700 1 --2- 172.21.15.1:0/348033336 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f79680a12c0 0x7f79680a3770 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:39.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.014+0000 7f7984503700 1 --2- 172.21.15.1:0/348033336 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f797c10a890 0x7f797c19cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:39.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.014+0000 7f7984503700 1 --2- 172.21.15.1:0/348033336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f797c101550 0x7f797c1985e0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:39.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.014+0000 7f7984503700 1 --2- 172.21.15.1:0/348033336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f797c101e60 0x7f797c198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:42:39.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.014+0000 7f7984503700 1 -- 172.21.15.1:0/348033336 >> 172.21.15.1:0/348033336 conn(0x7f797c0763a0 msgr2=0x7f797c1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:42:39.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.014+0000 7f7984503700 1 -- 172.21.15.1:0/348033336 shutdown_connections 2023-11-16T16:42:39.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:42:39.014+0000 7f7984503700 1 -- 172.21.15.1:0/348033336 wait complete. 2023-11-16T16:42:39.030 INFO:tasks.cephfs.filesystem:scrub status for tag:4e68c117-e4ae-40d9-847d-0d6d5d2b1317 - {'path': '/', 'tag': '4e68c117-e4ae-40d9-847d-0d6d5d2b1317', 'options': 'recursive,force'} 2023-11-16T16:42:39.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:38 smithi053 ceph-mon[100638]: pgmap v107: 97 pgs: 97 active+clean; 167 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.7 MiB/s wr, 344 op/s 2023-11-16T16:42:40.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:40 smithi001 ceph-mon[98800]: pgmap v108: 97 pgs: 97 active+clean; 167 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.7 MiB/s wr, 344 op/s 2023-11-16T16:42:41.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:40 smithi026 ceph-mon[100112]: pgmap v108: 97 pgs: 97 active+clean; 167 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.7 MiB/s wr, 344 op/s 2023-11-16T16:42:41.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:40 smithi053 ceph-mon[100638]: pgmap v108: 97 pgs: 97 active+clean; 167 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.7 MiB/s wr, 344 op/s 2023-11-16T16:42:42.508 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:42 smithi026 ceph-mon[100112]: pgmap v109: 97 pgs: 97 active+clean; 183 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.2 MiB/s wr, 391 op/s 2023-11-16T16:42:42.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:42 smithi053 ceph-mon[100638]: pgmap v109: 97 pgs: 97 active+clean; 183 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.2 MiB/s wr, 391 op/s 2023-11-16T16:42:42.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:42 smithi001 ceph-mon[98800]: pgmap v109: 97 pgs: 97 active+clean; 183 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.2 MiB/s wr, 391 op/s 2023-11-16T16:42:44.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:44 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:44.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:44 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:44.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:44 smithi001 ceph-mon[98800]: pgmap v110: 97 pgs: 97 active+clean; 183 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 230 op/s 2023-11-16T16:42:45.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:44 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:45.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:44 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:45.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:44 smithi026 ceph-mon[100112]: pgmap v110: 97 pgs: 97 active+clean; 183 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 230 op/s 2023-11-16T16:42:45.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:44 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:45.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:44 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:45.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:44 smithi053 ceph-mon[100638]: pgmap v110: 97 pgs: 97 active+clean; 183 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 230 op/s 2023-11-16T16:42:46.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:46 smithi001 ceph-mon[98800]: pgmap v111: 97 pgs: 97 active+clean; 195 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.5 MiB/s wr, 266 op/s 2023-11-16T16:42:47.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:46 smithi026 ceph-mon[100112]: pgmap v111: 97 pgs: 97 active+clean; 195 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.5 MiB/s wr, 266 op/s 2023-11-16T16:42:47.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:46 smithi053 ceph-mon[100638]: pgmap v111: 97 pgs: 97 active+clean; 195 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.5 MiB/s wr, 266 op/s 2023-11-16T16:42:48.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:48 smithi001 ceph-mon[98800]: pgmap v112: 97 pgs: 97 active+clean; 195 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 82 op/s 2023-11-16T16:42:49.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:48 smithi026 ceph-mon[100112]: pgmap v112: 97 pgs: 97 active+clean; 195 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 82 op/s 2023-11-16T16:42:49.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:48 smithi053 ceph-mon[100638]: pgmap v112: 97 pgs: 97 active+clean; 195 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 82 op/s 2023-11-16T16:42:50.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:50 smithi001 ceph-mon[98800]: pgmap v113: 97 pgs: 97 active+clean; 195 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 82 op/s 2023-11-16T16:42:51.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:50 smithi026 ceph-mon[100112]: pgmap v113: 97 pgs: 97 active+clean; 195 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 82 op/s 2023-11-16T16:42:51.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:50 smithi053 ceph-mon[100638]: pgmap v113: 97 pgs: 97 active+clean; 195 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 82 op/s 2023-11-16T16:42:52.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:52 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:52.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:52 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:52.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:52 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:52.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:52 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:52.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:52 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:52.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:52 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:52.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:52 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:52.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:52 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:52.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:52 smithi053 ceph-mon[100638]: pgmap v114: 97 pgs: 97 active+clean; 208 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 102 op/s 2023-11-16T16:42:52.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:52 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:52.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:52 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:52.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:52 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:52.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:52 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:52.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:52 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:52.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:52 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:52.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:52 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:52.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:52 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:52.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:52 smithi001 ceph-mon[98800]: pgmap v114: 97 pgs: 97 active+clean; 208 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 102 op/s 2023-11-16T16:42:52.757 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:52 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:52.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:52 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:52.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:52 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:52.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:52 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:52.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:52 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:52.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:52 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:52.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:52 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:52.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:52 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:52.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:52 smithi026 ceph-mon[100112]: pgmap v114: 97 pgs: 97 active+clean; 208 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 102 op/s 2023-11-16T16:42:53.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:53 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:53.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:53 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:53.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:53 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:53.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:53 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:53.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:53 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:53.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:53 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:53.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:53 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:53.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:53 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:53.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:53 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:53.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:53 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:53.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:53 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:53.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:53 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:53.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:53 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:53.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:53 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:53.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:53 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:53.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:53 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:53.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:53 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:53.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:53 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:53.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:53 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:53.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:53 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:53.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:53 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:53.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:53 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:53.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:53 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:53.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:53 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:53.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:53 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:53.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:53 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:53.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:53 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:53.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:53 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:53.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:53 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:53.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:53 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:53.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:53 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:53.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:53 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:53.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:53 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:53.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:53 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:53.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:53 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:53.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:53 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:53.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:53 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:53.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:53 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:53.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:53 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:53.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:53 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:53.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:53 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:53.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:53 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:53.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:53 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:53.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:53 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:53.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:53 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:53.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:53 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:53.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:53 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:53.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:53 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:53.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:53 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:53.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:53 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:53.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:53 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:53.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:53 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:53.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:53 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:53.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:53 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:53.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:53 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:53.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:53 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:53.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:53 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:53.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:53 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:53.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:53 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:53.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:53 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: pgmap v115: 97 pgs: 97 active+clean; 208 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 55 op/s 2023-11-16T16:42:54.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.592 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.592 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.592 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.592 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.592 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.592 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.592 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.592 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.592 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.592 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.592 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.592 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.593 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.593 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.593 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.593 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.593 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.593 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.593 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.593 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.593 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.593 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.593 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.593 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.593 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.594 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.594 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.594 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.594 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.594 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.594 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.594 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.594 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.594 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.594 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.594 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.594 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.595 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.595 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.595 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.595 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.595 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.595 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.595 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.595 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.595 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.595 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.595 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.595 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.598 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.598 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.598 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.598 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.598 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.598 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.598 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.598 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.598 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.598 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.598 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.598 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.598 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.599 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.599 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.599 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.599 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.599 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.599 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.599 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.599 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.599 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.599 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.599 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.599 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.601 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.601 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.601 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.601 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.601 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.601 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.601 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.601 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.601 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.601 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.601 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.601 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.601 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.602 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.602 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.602 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.602 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.602 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.602 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.602 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.602 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:54.602 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: pgmap v115: 97 pgs: 97 active+clean; 208 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 55 op/s 2023-11-16T16:42:54.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.696 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.696 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.696 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.696 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.696 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.704 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.704 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.704 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.704 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.704 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.704 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.704 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.704 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.704 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.704 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.704 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:54.704 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: pgmap v115: 97 pgs: 97 active+clean; 208 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 55 op/s 2023-11-16T16:42:54.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.768 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.768 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.768 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.768 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.768 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.768 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.768 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.768 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.768 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.768 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.768 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.768 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.768 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.769 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.769 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.769 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.769 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.769 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.769 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.769 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.769 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.769 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.769 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.769 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.769 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.769 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.770 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.770 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.770 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.770 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.770 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.770 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.770 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.770 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.770 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.770 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.770 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.770 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.770 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.771 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.771 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.771 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.771 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.771 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.771 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.771 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.771 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.771 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.771 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.771 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.771 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.771 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.771 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.772 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.772 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.772 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.772 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.772 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.772 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.772 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.772 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.772 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.772 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.772 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.772 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.772 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.774 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.774 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.774 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.774 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.774 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.774 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.774 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.774 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.774 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.774 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.774 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.774 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.774 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.775 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.775 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.775 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.775 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.775 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.775 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.775 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.775 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.775 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.775 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.775 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.775 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.775 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.775 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.776 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.776 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.776 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.776 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.776 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.776 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.776 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.776 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.776 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.776 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.776 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.776 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.776 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.776 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.777 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.777 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.777 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.777 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.777 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.777 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.777 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.777 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.777 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.777 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.777 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.777 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.777 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.777 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.778 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.778 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.778 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.778 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:54.778 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:54.778 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.592 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.592 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.592 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.592 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.592 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.592 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.592 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.592 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.592 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.592 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.592 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.592 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.593 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.593 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.593 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.593 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.593 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.593 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.593 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.593 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.593 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.593 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.593 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.593 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.593 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.594 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.594 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.594 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.594 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.594 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.594 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.594 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.594 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.594 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.594 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.594 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.595 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.595 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.595 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.595 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.595 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.595 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.595 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.595 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.595 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.595 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.595 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.595 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.598 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.598 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.598 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.598 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.598 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.598 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.598 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.598 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.598 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.598 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.598 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.598 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.599 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.599 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.599 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.599 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.599 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.599 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.599 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.599 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.599 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.599 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.599 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.599 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.601 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.601 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.601 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.601 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.601 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.601 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.601 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.601 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.601 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.601 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.601 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.601 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.602 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.602 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.602 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.602 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.602 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.602 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.602 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.602 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.602 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.602 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.602 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.602 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.603 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.603 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.603 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.603 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.603 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.603 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.603 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.603 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.603 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.603 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.603 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.603 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.604 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.604 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.604 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.604 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.604 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.604 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.604 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.604 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.604 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.604 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.604 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.604 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.604 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.605 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.605 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.605 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.605 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.605 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.605 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.605 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.605 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.605 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.605 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.605 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.605 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.605 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.606 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.606 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.606 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.606 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.606 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.606 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.606 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.606 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.606 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.606 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.606 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:55 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:55.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.696 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.696 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.696 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.696 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.696 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.696 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.696 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.696 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.696 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.696 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.696 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.696 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.702 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.704 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.704 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.704 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.704 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.704 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.704 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.704 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.704 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.704 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.704 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.704 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.704 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.704 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.705 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.705 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.705 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.705 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.705 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.705 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.705 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.705 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.705 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.705 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.705 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.705 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.706 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.706 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.706 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.706 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.706 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.706 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.706 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.706 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.706 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.706 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.706 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.706 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.707 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.707 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.707 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.707 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.707 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.707 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.707 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.707 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.707 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.707 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.707 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:55 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:55.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.768 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.768 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.768 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.768 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.768 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.768 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.768 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.768 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.768 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.768 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.768 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.768 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.769 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.769 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.769 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.769 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.769 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.769 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.769 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.769 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.769 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.769 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.769 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.769 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.770 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.770 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.770 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.770 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.770 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.770 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.770 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.770 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.770 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.770 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.770 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.770 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.771 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.771 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.771 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.771 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.771 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.771 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.771 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.771 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.771 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.771 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.771 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.771 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.772 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.772 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.772 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.772 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.772 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.772 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.772 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.772 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.772 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.772 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.772 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.772 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.774 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.774 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.774 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.774 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.774 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.774 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.774 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.774 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.774 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.774 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.774 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.774 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.775 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.775 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.775 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.775 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.775 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.775 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.775 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.775 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.775 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.775 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.775 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.775 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.776 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.776 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.776 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.776 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.776 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.776 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.776 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.776 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.776 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.776 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.776 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.776 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.777 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.777 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.777 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.777 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.777 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.777 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.777 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.777 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.777 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.777 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.777 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.777 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.778 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.778 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.778 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.778 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.778 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.778 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.778 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.778 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.778 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.778 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.778 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.778 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.779 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.779 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.779 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.779 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.779 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.779 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.779 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.779 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.779 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.779 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.779 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.779 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.780 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.780 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.780 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.780 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.780 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.780 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.780 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.780 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.780 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.780 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.780 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.780 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.781 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.781 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.781 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.781 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.781 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.781 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.781 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.781 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.781 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.781 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.781 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.781 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.782 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.782 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.782 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.782 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.782 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.782 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.782 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.782 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.782 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.782 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.782 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.782 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.783 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.783 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.783 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.783 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:55.783 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:55.783 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:55 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: pgmap v116: 97 pgs: 97 active+clean; 234 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.5 MiB/s wr, 138 op/s 2023-11-16T16:42:56.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.592 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.592 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.592 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.592 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.592 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.592 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.592 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.592 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.592 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.592 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.592 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.592 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.592 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.593 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.593 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.593 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.593 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.593 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.593 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.593 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.593 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.593 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.593 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.593 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.593 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.594 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.594 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.594 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.594 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.594 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.594 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.594 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.594 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.594 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.594 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.594 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.594 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.595 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.595 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.595 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.595 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.595 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.595 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.595 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.595 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.595 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.595 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.595 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.595 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.596 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.597 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.598 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.598 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.598 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.598 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.598 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.598 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.598 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.598 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.598 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.598 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.598 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.599 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.599 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.599 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.599 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.599 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.599 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.599 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.599 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.599 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.599 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.599 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.599 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:56.600 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:56 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: pgmap v116: 97 pgs: 97 active+clean; 234 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.5 MiB/s wr, 138 op/s 2023-11-16T16:42:56.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.693 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.695 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.696 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.696 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.696 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.696 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.696 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.696 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.696 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.696 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.696 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.696 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.696 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.696 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.697 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.699 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.700 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:56.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:56 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: pgmap v116: 97 pgs: 97 active+clean; 234 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.5 MiB/s wr, 138 op/s 2023-11-16T16:42:56.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.768 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.768 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.768 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.768 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.768 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.768 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.768 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.768 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.768 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.768 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.768 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.769 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.769 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.769 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.769 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.769 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.769 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.769 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.769 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.769 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.769 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.769 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.769 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.769 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.770 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.770 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.770 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.770 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.770 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.770 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.770 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.770 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.770 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.770 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.770 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.771 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.771 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.771 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.771 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.771 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.771 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.771 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.771 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.771 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.771 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.771 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.771 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.772 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.772 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.772 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.772 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.772 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.772 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.772 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.772 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.772 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.772 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.772 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.772 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.772 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.773 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.774 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.774 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.774 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.774 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.774 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.774 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.774 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.774 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.774 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.774 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.774 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.774 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.775 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.775 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.775 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.775 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.775 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.775 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.775 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.775 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.775 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.775 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.775 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.775 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.776 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.776 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.776 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.776 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.776 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.776 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.776 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.776 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.776 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.776 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.776 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.776 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.776 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.777 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.777 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.777 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.777 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.777 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.777 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.777 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.777 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:56.777 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:56.777 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:56 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.583 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.584 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.585 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.586 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.587 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.588 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.589 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.590 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:42:57.591 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:57 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.685 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:42:57.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:57 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.761 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.762 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.763 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.764 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.765 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.766 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:57.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:42:57.767 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:57 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:42:58.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:42:58 smithi053 ceph-mon[100638]: pgmap v117: 97 pgs: 97 active+clean; 234 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 102 op/s 2023-11-16T16:42:58.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:42:58 smithi001 ceph-mon[98800]: pgmap v117: 97 pgs: 97 active+clean; 234 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 102 op/s 2023-11-16T16:42:58.757 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:42:58 smithi026 ceph-mon[100112]: pgmap v117: 97 pgs: 97 active+clean; 234 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 102 op/s 2023-11-16T16:43:00.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:00 smithi001 ceph-mon[98800]: pgmap v118: 97 pgs: 97 active+clean; 234 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 102 op/s 2023-11-16T16:43:01.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:00 smithi026 ceph-mon[100112]: pgmap v118: 97 pgs: 97 active+clean; 234 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 102 op/s 2023-11-16T16:43:01.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:00 smithi053 ceph-mon[100638]: pgmap v118: 97 pgs: 97 active+clean; 234 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 102 op/s 2023-11-16T16:43:02.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:02 smithi053 ceph-mon[100638]: pgmap v119: 97 pgs: 97 active+clean; 243 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.1 MiB/s wr, 219 op/s 2023-11-16T16:43:02.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:02 smithi001 ceph-mon[98800]: pgmap v119: 97 pgs: 97 active+clean; 243 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.1 MiB/s wr, 219 op/s 2023-11-16T16:43:02.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:02 smithi026 ceph-mon[100112]: pgmap v119: 97 pgs: 97 active+clean; 243 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.1 MiB/s wr, 219 op/s 2023-11-16T16:43:04.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:04 smithi001 ceph-mon[98800]: pgmap v120: 97 pgs: 97 active+clean; 243 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.0 MiB/s wr, 199 op/s 2023-11-16T16:43:05.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:04 smithi026 ceph-mon[100112]: pgmap v120: 97 pgs: 97 active+clean; 243 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.0 MiB/s wr, 199 op/s 2023-11-16T16:43:05.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:04 smithi053 ceph-mon[100638]: pgmap v120: 97 pgs: 97 active+clean; 243 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.0 MiB/s wr, 199 op/s 2023-11-16T16:43:06.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:06 smithi001 ceph-mon[98800]: pgmap v121: 97 pgs: 97 active+clean; 292 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.0 MiB/s wr, 221 op/s 2023-11-16T16:43:07.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:06 smithi026 ceph-mon[100112]: pgmap v121: 97 pgs: 97 active+clean; 292 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.0 MiB/s wr, 221 op/s 2023-11-16T16:43:07.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:06 smithi053 ceph-mon[100638]: pgmap v121: 97 pgs: 97 active+clean; 292 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.0 MiB/s wr, 221 op/s 2023-11-16T16:43:08.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:08 smithi001 ceph-mon[98800]: pgmap v122: 97 pgs: 97 active+clean; 292 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.8 MiB/s wr, 139 op/s 2023-11-16T16:43:09.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:08 smithi026 ceph-mon[100112]: pgmap v122: 97 pgs: 97 active+clean; 292 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.8 MiB/s wr, 139 op/s 2023-11-16T16:43:09.030 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-16T16:43:09.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:08 smithi053 ceph-mon[100638]: pgmap v122: 97 pgs: 97 active+clean; 292 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.8 MiB/s wr, 139 op/s 2023-11-16T16:43:09.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.166+0000 7fa85f306700 1 -- 172.21.15.1:0/2271904933 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa858100420 msgr2=0x7fa85810bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:43:09.172 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.166+0000 7fa85f306700 1 --2- 172.21.15.1:0/2271904933 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa858100420 0x7fa85810bfd0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fa850009f90 tx=0x7fa85000b330 comp rx=0 tx=0).stop 2023-11-16T16:43:09.172 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.166+0000 7fa85f306700 1 -- 172.21.15.1:0/2271904933 shutdown_connections 2023-11-16T16:43:09.172 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.166+0000 7fa85f306700 1 --2- 172.21.15.1:0/2271904933 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa858100420 0x7fa85810bfd0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:09.172 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.166+0000 7fa85f306700 1 --2- 172.21.15.1:0/2271904933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa858105250 0x7fa858105620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:09.172 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.166+0000 7fa85f306700 1 --2- 172.21.15.1:0/2271904933 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa8580ffa70 0x7fa8580ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:09.172 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.166+0000 7fa85f306700 1 -- 172.21.15.1:0/2271904933 >> 172.21.15.1:0/2271904933 conn(0x7fa8580763a0 msgr2=0x7fa8580767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:43:09.173 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.166+0000 7fa85f306700 1 -- 172.21.15.1:0/2271904933 shutdown_connections 2023-11-16T16:43:09.173 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.166+0000 7fa85f306700 1 -- 172.21.15.1:0/2271904933 wait complete. 2023-11-16T16:43:09.173 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.167+0000 7fa85f306700 1 Processor -- start 2023-11-16T16:43:09.173 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.167+0000 7fa85f306700 1 -- start start 2023-11-16T16:43:09.173 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.167+0000 7fa85f306700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8580ffa70 0x7fa8581972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:09.173 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.167+0000 7fa85f306700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa858100420 0x7fa858197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:09.173 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.167+0000 7fa85f306700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa858105250 0x7fa8581a2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:09.173 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.167+0000 7fa85f306700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa858198150 con 0x7fa858105250 2023-11-16T16:43:09.174 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.167+0000 7fa85f306700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8581a2890 con 0x7fa8580ffa70 2023-11-16T16:43:09.174 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.167+0000 7fa85f306700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8581a2a50 con 0x7fa858100420 2023-11-16T16:43:09.174 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.167+0000 7fa85d8a3700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa858105250 0x7fa8581a2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:09.174 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.168+0000 7fa85d8a3700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa858105250 0x7fa8581a2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45142/0 (socket says 172.21.15.1:45142) 2023-11-16T16:43:09.174 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.168+0000 7fa85d8a3700 1 -- 172.21.15.1:0/770405642 learned_addr learned my addr 172.21.15.1:0/770405642 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:43:09.174 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.168+0000 7fa85d0a2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8580ffa70 0x7fa8581972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:09.175 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.168+0000 7fa85c8a1700 1 --2- 172.21.15.1:0/770405642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa858100420 0x7fa858197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:09.175 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.168+0000 7fa85d8a3700 1 -- 172.21.15.1:0/770405642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa858100420 msgr2=0x7fa858197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:43:09.175 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.168+0000 7fa85d8a3700 1 --2- 172.21.15.1:0/770405642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa858100420 0x7fa858197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:09.175 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.168+0000 7fa85d8a3700 1 -- 172.21.15.1:0/770405642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8580ffa70 msgr2=0x7fa8581972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:43:09.175 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.168+0000 7fa85d8a3700 1 --2- 172.21.15.1:0/770405642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8580ffa70 0x7fa8581972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:09.175 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.168+0000 7fa85d8a3700 1 -- 172.21.15.1:0/770405642 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8540095f0 con 0x7fa858105250 2023-11-16T16:43:09.175 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.168+0000 7fa85d0a2700 1 --2- 172.21.15.1:0/770405642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8580ffa70 0x7fa8581972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:43:09.175 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.168+0000 7fa85d8a3700 1 --2- 172.21.15.1:0/770405642 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa858105250 0x7fa8581a2350 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fa85000b6c0 tx=0x7fa850004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:43:09.176 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.168+0000 7fa84e7fc700 1 -- 172.21.15.1:0/770405642 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85000e040 con 0x7fa858105250 2023-11-16T16:43:09.176 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.168+0000 7fa84e7fc700 1 -- 172.21.15.1:0/770405642 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa850008e80 con 0x7fa858105250 2023-11-16T16:43:09.176 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.168+0000 7fa85f306700 1 -- 172.21.15.1:0/770405642 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa850009c40 con 0x7fa858105250 2023-11-16T16:43:09.176 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.168+0000 7fa84e7fc700 1 -- 172.21.15.1:0/770405642 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa850005040 con 0x7fa858105250 2023-11-16T16:43:09.176 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.168+0000 7fa85f306700 1 -- 172.21.15.1:0/770405642 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa85810f1e0 con 0x7fa858105250 2023-11-16T16:43:09.185 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.170+0000 7fa84e7fc700 1 -- 172.21.15.1:0/770405642 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fa85001d470 con 0x7fa858105250 2023-11-16T16:43:09.185 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.171+0000 7fa84e7fc700 1 --2- 172.21.15.1:0/770405642 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fa8440a12c0 0x7fa8440a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:09.185 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.171+0000 7fa84e7fc700 1 -- 172.21.15.1:0/770405642 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fa8500ce080 con 0x7fa858105250 2023-11-16T16:43:09.185 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.171+0000 7fa85d0a2700 1 --2- 172.21.15.1:0/770405642 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fa8440a12c0 0x7fa8440a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:09.185 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.171+0000 7fa85d0a2700 1 --2- 172.21.15.1:0/770405642 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fa8440a12c0 0x7fa8440a3770 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa854005fd0 tx=0x7fa854009310 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:43:09.185 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.172+0000 7fa86b492b80 1 Processor -- start 2023-11-16T16:43:09.186 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.172+0000 7fa86b492b80 1 -- start start 2023-11-16T16:43:09.186 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.172+0000 7fa86b492b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5589ae716b60 0x5589ae716f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:09.186 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.172+0000 7fa86b492b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5589ae717470 0x5589ae71b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:09.186 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.172+0000 7fa86b492b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589ae71b8b0 0x5589ae71bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:09.186 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.172+0000 7fa86b492b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589ae61c060 con 0x5589ae717470 2023-11-16T16:43:09.186 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.172+0000 7fa86b492b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589ae66e8a0 con 0x5589ae71b8b0 2023-11-16T16:43:09.186 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.172+0000 7fa86b492b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589ae667d30 con 0x5589ae716b60 2023-11-16T16:43:09.186 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.172+0000 7fa85c8a1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5589ae717470 0x5589ae71b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:09.187 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.172+0000 7fa85d0a2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5589ae716b60 0x5589ae716f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:09.187 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.172+0000 7fa85d0a2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5589ae716b60 0x5589ae716f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:41828/0 (socket says 172.21.15.1:41828) 2023-11-16T16:43:09.187 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.172+0000 7fa85c8a1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5589ae717470 0x5589ae71b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45148/0 (socket says 172.21.15.1:45148) 2023-11-16T16:43:09.187 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.172+0000 7fa85d0a2700 1 -- 172.21.15.1:0/2130151964 learned_addr learned my addr 172.21.15.1:0/2130151964 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:43:09.187 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.172+0000 7fa85d8a3700 1 --2- 172.21.15.1:0/2130151964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589ae71b8b0 0x5589ae71bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:09.187 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.173+0000 7fa85d0a2700 1 -- 172.21.15.1:0/2130151964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589ae71b8b0 msgr2=0x5589ae71bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:43:09.187 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.173+0000 7fa85d0a2700 1 --2- 172.21.15.1:0/2130151964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589ae71b8b0 0x5589ae71bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:09.187 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.173+0000 7fa85d0a2700 1 -- 172.21.15.1:0/2130151964 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5589ae717470 msgr2=0x5589ae71b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:43:09.188 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.173+0000 7fa85d0a2700 1 --2- 172.21.15.1:0/2130151964 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5589ae717470 0x5589ae71b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:09.188 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.173+0000 7fa85d0a2700 1 -- 172.21.15.1:0/2130151964 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589ae4e9630 con 0x5589ae716b60 2023-11-16T16:43:09.188 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.173+0000 7fa85c8a1700 1 --2- 172.21.15.1:0/2130151964 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5589ae717470 0x5589ae71b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:43:09.188 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.173+0000 7fa85d0a2700 1 --2- 172.21.15.1:0/2130151964 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5589ae716b60 0x5589ae716f30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa85400f730 tx=0x7fa854014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98f4da0e025e9a7c server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:43:09.188 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.173+0000 7fa8437fe700 1 -- 172.21.15.1:0/2130151964 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa854004220 con 0x5589ae716b60 2023-11-16T16:43:09.188 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.173+0000 7fa8437fe700 1 -- 172.21.15.1:0/2130151964 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa854004380 con 0x5589ae716b60 2023-11-16T16:43:09.188 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.173+0000 7fa8437fe700 1 -- 172.21.15.1:0/2130151964 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8540267c0 con 0x5589ae716b60 2023-11-16T16:43:09.188 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.174+0000 7fa86b492b80 1 -- 172.21.15.1:0/2130151964 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5589ae716b60 msgr2=0x5589ae716f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:43:09.189 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.174+0000 7fa86b492b80 1 --2- 172.21.15.1:0/2130151964 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5589ae716b60 0x5589ae716f30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa85400f730 tx=0x7fa854014000 comp rx=0 tx=0).stop 2023-11-16T16:43:09.189 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.174+0000 7fa86b492b80 1 -- 172.21.15.1:0/2130151964 shutdown_connections 2023-11-16T16:43:09.189 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.174+0000 7fa86b492b80 1 --2- 172.21.15.1:0/2130151964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589ae71b8b0 0x5589ae71bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:09.189 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.174+0000 7fa86b492b80 1 --2- 172.21.15.1:0/2130151964 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5589ae716b60 0x5589ae716f30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:09.189 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.174+0000 7fa86b492b80 1 --2- 172.21.15.1:0/2130151964 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5589ae717470 0x5589ae71b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:09.189 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.174+0000 7fa86b492b80 1 -- 172.21.15.1:0/2130151964 >> 172.21.15.1:0/2130151964 conn(0x5589ae70c360 msgr2=0x5589ae711b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:43:09.189 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.174+0000 7fa86b492b80 1 -- 172.21.15.1:0/2130151964 shutdown_connections 2023-11-16T16:43:09.190 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.174+0000 7fa86b492b80 1 -- 172.21.15.1:0/2130151964 wait complete. 2023-11-16T16:43:09.190 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.174+0000 7fa86b492b80 1 -- start start 2023-11-16T16:43:09.190 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.174+0000 7fa86b492b80 1 Processor -- start 2023-11-16T16:43:09.190 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.175+0000 7fa86b492b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589ae716b60 0x5589ae723050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:09.190 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.175+0000 7fa86b492b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5589ae717470 0x5589ae7202d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:09.190 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.175+0000 7fa86b492b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5589ae71b8b0 0x5589ae720950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:09.190 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.175+0000 7fa86b492b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589ae6098e0 con 0x5589ae71b8b0 2023-11-16T16:43:09.191 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.175+0000 7fa86b492b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589ae609580 con 0x5589ae716b60 2023-11-16T16:43:09.191 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.175+0000 7fa86b492b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589ae4eb920 con 0x5589ae717470 2023-11-16T16:43:09.191 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.175+0000 7fa86b492b80 10 client.0 _ll_register_callbacks cb 0x5589ae627780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:43:09.191 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.175+0000 7fa85d8a3700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5589ae71b8b0 0x5589ae720950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:09.191 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.175+0000 7fa85c8a1700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5589ae717470 0x5589ae7202d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:09.191 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.175+0000 7fa85d8a3700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5589ae71b8b0 0x5589ae720950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45156/0 (socket says 172.21.15.1:45156) 2023-11-16T16:43:09.191 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.175+0000 7fa85d8a3700 1 -- 172.21.15.1:0/2722895315 learned_addr learned my addr 172.21.15.1:0/2722895315 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:43:09.191 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.175+0000 7fa85d0a2700 1 --2- 172.21.15.1:0/2722895315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589ae716b60 0x5589ae723050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:09.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.175+0000 7fa85d8a3700 1 -- 172.21.15.1:0/2722895315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5589ae717470 msgr2=0x5589ae7202d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:43:09.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.175+0000 7fa85d8a3700 1 --2- 172.21.15.1:0/2722895315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5589ae717470 0x5589ae7202d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:09.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.175+0000 7fa85d8a3700 1 -- 172.21.15.1:0/2722895315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589ae716b60 msgr2=0x5589ae723050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:43:09.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.175+0000 7fa85d8a3700 1 --2- 172.21.15.1:0/2722895315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589ae716b60 0x5589ae723050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:09.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.175+0000 7fa85d8a3700 1 -- 172.21.15.1:0/2722895315 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa854017050 con 0x5589ae71b8b0 2023-11-16T16:43:09.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.175+0000 7fa85d8a3700 1 --2- 172.21.15.1:0/2722895315 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5589ae71b8b0 0x5589ae720950 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa85009aef0 tx=0x7fa85009ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81eb8b7b1678935d server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:43:09.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.175+0000 7fa840ff9700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:43:09.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.175+0000 7fa840ff9700 1 -- 172.21.15.1:0/2722895315 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85009f030 con 0x5589ae71b8b0 2023-11-16T16:43:09.193 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.176+0000 7fa840ff9700 1 -- 172.21.15.1:0/2722895315 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fa85009f220 con 0x5589ae71b8b0 2023-11-16T16:43:09.193 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.176+0000 7fa86b492b80 1 -- 172.21.15.1:0/2722895315 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5589ae709b20 con 0x5589ae71b8b0 2023-11-16T16:43:09.193 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.176+0000 7fa840ff9700 1 -- 172.21.15.1:0/2722895315 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8500a35b0 con 0x5589ae71b8b0 2023-11-16T16:43:09.193 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.176+0000 7fa86b492b80 10 client.16119 fetch_fsmap learned FSMap version 26 2023-11-16T16:43:09.193 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.176+0000 7fa86b492b80 1 -- 172.21.15.1:0/2722895315 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5589ae62e190 con 0x5589ae71b8b0 2023-11-16T16:43:09.193 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.176+0000 7fa840ff9700 1 -- 172.21.15.1:0/2722895315 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5589ae709b20 con 0x5589ae71b8b0 2023-11-16T16:43:09.193 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.176+0000 7fa840ff9700 1 -- 172.21.15.1:0/2722895315 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fa85009b4e0 con 0x5589ae71b8b0 2023-11-16T16:43:09.193 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.176+0000 7fa86b492b80 10 client.16119 fetch_fsmap finished waiting for FSMap version 26 2023-11-16T16:43:09.193 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.176+0000 7fa86b492b80 10 client.16119 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:43:09.194 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.176+0000 7fa86b492b80 20 client.16119 populate_metadata read hostname 'smithi001' 2023-11-16T16:43:09.194 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.176+0000 7fa86b492b80 1 --2- 172.21.15.1:0/2722895315 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5589ae7307d0 0x5589ae732be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:09.194 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.176+0000 7fa86b492b80 4 client.16119 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:43:09.194 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.176+0000 7fa86b492b80 1 -- 172.21.15.1:0/2722895315 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5589ae62a0a0 con 0x5589ae7307d0 2023-11-16T16:43:09.194 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.176+0000 7fa85d0a2700 1 --2- 172.21.15.1:0/2722895315 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5589ae7307d0 0x5589ae732be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:09.194 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.177+0000 7fa840ff9700 10 client.16119 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:43:09.194 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.177+0000 7fa85d0a2700 1 --2- 172.21.15.1:0/2722895315 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5589ae7307d0 0x5589ae732be0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8e40543009e8f64 server_cookie=9f2492ac37a31a26 in_seq=0 out_seq=0 2023-11-16T16:43:09.194 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.177+0000 7fa840ff9700 1 -- 172.21.15.1:0/2722895315 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5589ae62a0a0 con 0x5589ae7307d0 2023-11-16T16:43:09.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.177+0000 7fa840ff9700 10 client.16119 handle_command_reply: tid=0 2023-11-16T16:43:09.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.177+0000 7fa86b492b80 1 client.16119 shutdown 2023-11-16T16:43:09.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.177+0000 7fa86b492b80 1 -- 172.21.15.1:0/2722895315 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5589ae71b8b0 msgr2=0x5589ae720950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:43:09.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.177+0000 7fa86b492b80 1 --2- 172.21.15.1:0/2722895315 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5589ae71b8b0 0x5589ae720950 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa85009aef0 tx=0x7fa85009ac40 comp rx=0 tx=0).stop 2023-11-16T16:43:09.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.178+0000 7fa86b492b80 1 -- 172.21.15.1:0/2722895315 shutdown_connections 2023-11-16T16:43:09.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.178+0000 7fa86b492b80 1 --2- 172.21.15.1:0/2722895315 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5589ae7307d0 0x5589ae732be0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:09.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.178+0000 7fa86b492b80 1 --2- 172.21.15.1:0/2722895315 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5589ae71b8b0 0x5589ae720950 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:09.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.178+0000 7fa86b492b80 1 --2- 172.21.15.1:0/2722895315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589ae716b60 0x5589ae723050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:09.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.178+0000 7fa86b492b80 1 --2- 172.21.15.1:0/2722895315 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5589ae717470 0x5589ae7202d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:09.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.178+0000 7fa86b492b80 1 -- 172.21.15.1:0/2722895315 >> 172.21.15.1:0/2722895315 conn(0x5589ae70c360 msgr2=0x5589ae710de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:43:09.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.178+0000 7fa840ff9700 0 client.16119 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:43:09.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.178+0000 7fa86b492b80 1 -- 172.21.15.1:0/2722895315 shutdown_connections 2023-11-16T16:43:09.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.178+0000 7fa86b492b80 1 -- 172.21.15.1:0/2722895315 wait complete. 2023-11-16T16:43:09.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.178+0000 7fa86b492b80 20 client.16119 trim_cache size 0 max 16384 2023-11-16T16:43:09.203 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.196+0000 7fa86b492b80 1 Processor -- start 2023-11-16T16:43:09.204 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.196+0000 7fa86b492b80 1 -- start start 2023-11-16T16:43:09.204 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.196+0000 7fa86b492b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589ae716b60 0x5589ae71aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:09.204 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.196+0000 7fa86b492b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5589ae717470 0x5589ae71af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:09.204 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.196+0000 7fa86b492b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5589ae71b8b0 0x5589ae711560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:09.204 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.196+0000 7fa86b492b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589ae717bc0 con 0x5589ae717470 2023-11-16T16:43:09.204 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.196+0000 7fa86b492b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589ae7099f0 con 0x5589ae716b60 2023-11-16T16:43:09.204 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.196+0000 7fa86b492b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589ae71a570 con 0x5589ae71b8b0 2023-11-16T16:43:09.204 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.196+0000 7fa85c8a1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5589ae717470 0x5589ae71af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:09.204 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.196+0000 7fa85c8a1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5589ae717470 0x5589ae71af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45160/0 (socket says 172.21.15.1:45160) 2023-11-16T16:43:09.205 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.196+0000 7fa85c8a1700 1 -- 172.21.15.1:0/3134986094 learned_addr learned my addr 172.21.15.1:0/3134986094 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:43:09.205 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.196+0000 7fa85d8a3700 1 --2- 172.21.15.1:0/3134986094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5589ae71b8b0 0x5589ae711560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:09.205 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.196+0000 7fa85d0a2700 1 --2- 172.21.15.1:0/3134986094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589ae716b60 0x5589ae71aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:09.205 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.196+0000 7fa85c8a1700 1 -- 172.21.15.1:0/3134986094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5589ae71b8b0 msgr2=0x5589ae711560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:43:09.205 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.196+0000 7fa85c8a1700 1 --2- 172.21.15.1:0/3134986094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5589ae71b8b0 0x5589ae711560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:09.205 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.196+0000 7fa85c8a1700 1 -- 172.21.15.1:0/3134986094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589ae716b60 msgr2=0x5589ae71aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:43:09.205 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.196+0000 7fa85c8a1700 1 --2- 172.21.15.1:0/3134986094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589ae716b60 0x5589ae71aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:09.205 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.196+0000 7fa85c8a1700 1 -- 172.21.15.1:0/3134986094 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589ae70db00 con 0x5589ae717470 2023-11-16T16:43:09.206 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.196+0000 7fa85d0a2700 1 --2- 172.21.15.1:0/3134986094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589ae716b60 0x5589ae71aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:43:09.206 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.196+0000 7fa85c8a1700 1 --2- 172.21.15.1:0/3134986094 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5589ae717470 0x5589ae71af70 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa848009e20 tx=0x7fa848009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccdfa9a6ac7aa044 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:43:09.206 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.197+0000 7fa843fff700 1 -- 172.21.15.1:0/3134986094 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa848017050 con 0x5589ae717470 2023-11-16T16:43:09.206 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.197+0000 7fa843fff700 1 -- 172.21.15.1:0/3134986094 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa848018040 con 0x5589ae717470 2023-11-16T16:43:09.206 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.197+0000 7fa843fff700 1 -- 172.21.15.1:0/3134986094 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa848003b60 con 0x5589ae717470 2023-11-16T16:43:09.206 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.197+0000 7fa86b492b80 1 -- 172.21.15.1:0/3134986094 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5589ae717470 msgr2=0x5589ae71af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:43:09.206 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.197+0000 7fa86b492b80 1 --2- 172.21.15.1:0/3134986094 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5589ae717470 0x5589ae71af70 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa848009e20 tx=0x7fa848009260 comp rx=0 tx=0).stop 2023-11-16T16:43:09.206 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.197+0000 7fa86b492b80 1 -- 172.21.15.1:0/3134986094 shutdown_connections 2023-11-16T16:43:09.207 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.197+0000 7fa86b492b80 1 --2- 172.21.15.1:0/3134986094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5589ae71b8b0 0x5589ae711560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:09.207 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.197+0000 7fa86b492b80 1 --2- 172.21.15.1:0/3134986094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589ae716b60 0x5589ae71aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:09.207 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.197+0000 7fa86b492b80 1 --2- 172.21.15.1:0/3134986094 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5589ae717470 0x5589ae71af70 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:09.207 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.197+0000 7fa86b492b80 1 -- 172.21.15.1:0/3134986094 >> 172.21.15.1:0/3134986094 conn(0x5589ae70c360 msgr2=0x5589ae72cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:43:09.207 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.198+0000 7fa86b492b80 1 -- 172.21.15.1:0/3134986094 shutdown_connections 2023-11-16T16:43:09.207 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.198+0000 7fa86b492b80 1 -- 172.21.15.1:0/3134986094 wait complete. 2023-11-16T16:43:09.207 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.198+0000 7fa86b492b80 1 -- start start 2023-11-16T16:43:09.207 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.198+0000 7fa86b492b80 1 Processor -- start 2023-11-16T16:43:09.207 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.198+0000 7fa86b492b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5589ae716b60 0x5589ae7259c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:09.208 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.198+0000 7fa86b492b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589ae717470 0x5589ae725f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:09.208 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.198+0000 7fa86b492b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5589ae71b8b0 0x5589ae726440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:09.208 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.198+0000 7fa86b492b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589ae727fe0 con 0x5589ae716b60 2023-11-16T16:43:09.208 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.198+0000 7fa86b492b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589ae783690 con 0x5589ae717470 2023-11-16T16:43:09.208 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.198+0000 7fa86b492b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589ae7837d0 con 0x5589ae71b8b0 2023-11-16T16:43:09.208 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.198+0000 7fa86b492b80 10 client.0 _ll_register_callbacks cb 0x7fa85009afc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:43:09.208 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.198+0000 7fa85d0a2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5589ae716b60 0x5589ae7259c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:09.208 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.198+0000 7fa85d0a2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5589ae716b60 0x5589ae7259c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45166/0 (socket says 172.21.15.1:45166) 2023-11-16T16:43:09.209 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.198+0000 7fa85d0a2700 1 -- 172.21.15.1:0/2887466728 learned_addr learned my addr 172.21.15.1:0/2887466728 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:43:09.209 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.199+0000 7fa85d8a3700 1 --2- 172.21.15.1:0/2887466728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5589ae71b8b0 0x5589ae726440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:09.209 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.199+0000 7fa85c8a1700 1 --2- 172.21.15.1:0/2887466728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589ae717470 0x5589ae725f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:09.209 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.199+0000 7fa85d0a2700 1 -- 172.21.15.1:0/2887466728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5589ae71b8b0 msgr2=0x5589ae726440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:43:09.209 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.199+0000 7fa85d0a2700 1 --2- 172.21.15.1:0/2887466728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5589ae71b8b0 0x5589ae726440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:09.209 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.199+0000 7fa85d0a2700 1 -- 172.21.15.1:0/2887466728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589ae717470 msgr2=0x5589ae725f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:43:09.209 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.199+0000 7fa85d0a2700 1 --2- 172.21.15.1:0/2887466728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589ae717470 0x5589ae725f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:09.209 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.199+0000 7fa85d0a2700 1 -- 172.21.15.1:0/2887466728 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa848009ad0 con 0x5589ae716b60 2023-11-16T16:43:09.209 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.199+0000 7fa85d0a2700 1 --2- 172.21.15.1:0/2887466728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5589ae716b60 0x5589ae7259c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa85400f6d0 tx=0x7fa85400f4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4b21b0fff6a505 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:43:09.210 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.199+0000 7fa840ff9700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:43:09.210 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.199+0000 7fa840ff9700 1 -- 172.21.15.1:0/2887466728 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85402c040 con 0x5589ae716b60 2023-11-16T16:43:09.210 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.199+0000 7fa840ff9700 1 -- 172.21.15.1:0/2887466728 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fa8540044f0 con 0x5589ae716b60 2023-11-16T16:43:09.210 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.199+0000 7fa86b492b80 1 -- 172.21.15.1:0/2887466728 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa854004380 con 0x5589ae716b60 2023-11-16T16:43:09.210 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.199+0000 7fa840ff9700 1 -- 172.21.15.1:0/2887466728 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8540046b0 con 0x5589ae716b60 2023-11-16T16:43:09.210 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.199+0000 7fa86b492b80 10 client.16125 fetch_fsmap learned FSMap version 26 2023-11-16T16:43:09.210 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.199+0000 7fa86b492b80 1 -- 172.21.15.1:0/2887466728 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5589ae495de0 con 0x5589ae716b60 2023-11-16T16:43:09.210 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.199+0000 7fa840ff9700 1 -- 172.21.15.1:0/2887466728 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa854004380 con 0x5589ae716b60 2023-11-16T16:43:09.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.200+0000 7fa840ff9700 1 -- 172.21.15.1:0/2887466728 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fa854003920 con 0x5589ae716b60 2023-11-16T16:43:09.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.200+0000 7fa86b492b80 10 client.16125 fetch_fsmap finished waiting for FSMap version 26 2023-11-16T16:43:09.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.200+0000 7fa86b492b80 10 client.16125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:43:09.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.200+0000 7fa86b492b80 20 client.16125 populate_metadata read hostname 'smithi001' 2023-11-16T16:43:09.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.200+0000 7fa86b492b80 1 --2- 172.21.15.1:0/2887466728 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5589ae784e70 0x5589ae787280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:09.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.200+0000 7fa86b492b80 4 client.16125 mds_command: new command op to 24521 tid=0[{"prefix": "scrub status"}] 2023-11-16T16:43:09.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.200+0000 7fa86b492b80 1 -- 172.21.15.1:0/2887466728 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5589ae636660 con 0x5589ae784e70 2023-11-16T16:43:09.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.200+0000 7fa85c8a1700 1 --2- 172.21.15.1:0/2887466728 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5589ae784e70 0x5589ae787280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:09.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.200+0000 7fa85c8a1700 1 --2- 172.21.15.1:0/2887466728 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5589ae784e70 0x5589ae787280 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=115fbdcc83ccd616 server_cookie=4c9e626332226ece in_seq=0 out_seq=0 2023-11-16T16:43:09.212 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.200+0000 7fa840ff9700 10 client.16125 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:43:09.214 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.211+0000 7fa840ff9700 1 -- 172.21.15.1:0/2887466728 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5589ae636660 con 0x5589ae784e70 2023-11-16T16:43:09.214 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.211+0000 7fa840ff9700 10 client.16125 handle_command_reply: tid=0 2023-11-16T16:43:09.214 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.211+0000 7fa86b492b80 1 client.16125 shutdown 2023-11-16T16:43:09.214 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.211+0000 7fa86b492b80 1 -- 172.21.15.1:0/2887466728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5589ae716b60 msgr2=0x5589ae7259c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:43:09.214 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.211+0000 7fa86b492b80 1 --2- 172.21.15.1:0/2887466728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5589ae716b60 0x5589ae7259c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa85400f6d0 tx=0x7fa85400f4d0 comp rx=0 tx=0).stop 2023-11-16T16:43:09.214 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.211+0000 7fa86b492b80 1 -- 172.21.15.1:0/2887466728 shutdown_connections 2023-11-16T16:43:09.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.211+0000 7fa86b492b80 1 --2- 172.21.15.1:0/2887466728 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5589ae784e70 0x5589ae787280 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:09.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.211+0000 7fa86b492b80 1 --2- 172.21.15.1:0/2887466728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5589ae71b8b0 0x5589ae726440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:09.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.211+0000 7fa840ff9700 0 client.16125 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:43:09.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.211+0000 7fa86b492b80 1 --2- 172.21.15.1:0/2887466728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5589ae716b60 0x5589ae7259c0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:09.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.211+0000 7fa86b492b80 1 --2- 172.21.15.1:0/2887466728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5589ae717470 0x5589ae725f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:09.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.211+0000 7fa86b492b80 1 -- 172.21.15.1:0/2887466728 >> 172.21.15.1:0/2887466728 conn(0x5589ae70c360 msgr2=0x5589ae71aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:43:09.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.212+0000 7fa86b492b80 1 -- 172.21.15.1:0/2887466728 shutdown_connections 2023-11-16T16:43:09.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.212+0000 7fa86b492b80 1 -- 172.21.15.1:0/2887466728 wait complete. 2023-11-16T16:43:09.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.212+0000 7fa86b492b80 20 client.16125 trim_cache size 0 max 16384 2023-11-16T16:43:09.216 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:43:09.216 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "scrub active (1 inodes in the stack)", 2023-11-16T16:43:09.216 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2023-11-16T16:43:09.216 INFO:teuthology.orchestra.run.smithi001.stdout: "4e68c117-e4ae-40d9-847d-0d6d5d2b1317": { 2023-11-16T16:43:09.216 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2023-11-16T16:43:09.216 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "4e68c117-e4ae-40d9-847d-0d6d5d2b1317", 2023-11-16T16:43:09.216 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2023-11-16T16:43:09.217 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:43:09.217 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:43:09.217 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:43:09.218 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.212+0000 7fa85f306700 1 -- 172.21.15.1:0/770405642 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fa8440a12c0 msgr2=0x7fa8440a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:43:09.218 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.212+0000 7fa85f306700 1 --2- 172.21.15.1:0/770405642 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fa8440a12c0 0x7fa8440a3770 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa854005fd0 tx=0x7fa854009310 comp rx=0 tx=0).stop 2023-11-16T16:43:09.218 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.212+0000 7fa85f306700 1 -- 172.21.15.1:0/770405642 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa858105250 msgr2=0x7fa8581a2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:43:09.218 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.212+0000 7fa85f306700 1 --2- 172.21.15.1:0/770405642 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa858105250 0x7fa8581a2350 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fa85000b6c0 tx=0x7fa850004ce0 comp rx=0 tx=0).stop 2023-11-16T16:43:09.218 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.212+0000 7fa85f306700 1 -- 172.21.15.1:0/770405642 shutdown_connections 2023-11-16T16:43:09.218 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.212+0000 7fa85f306700 1 --2- 172.21.15.1:0/770405642 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fa8440a12c0 0x7fa8440a3770 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:09.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.212+0000 7fa85f306700 1 --2- 172.21.15.1:0/770405642 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa858105250 0x7fa8581a2350 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:09.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.212+0000 7fa85f306700 1 --2- 172.21.15.1:0/770405642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8580ffa70 0x7fa8581972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:09.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.212+0000 7fa85f306700 1 --2- 172.21.15.1:0/770405642 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fa858100420 0x7fa858197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:09.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.212+0000 7fa85f306700 1 -- 172.21.15.1:0/770405642 >> 172.21.15.1:0/770405642 conn(0x7fa8580763a0 msgr2=0x7fa8580fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:43:09.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.212+0000 7fa85f306700 1 -- 172.21.15.1:0/770405642 shutdown_connections 2023-11-16T16:43:09.219 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:09.212+0000 7fa85f306700 1 -- 172.21.15.1:0/770405642 wait complete. 2023-11-16T16:43:09.226 INFO:tasks.cephfs.filesystem:scrub status for tag:4e68c117-e4ae-40d9-847d-0d6d5d2b1317 - {'path': '/', 'tag': '4e68c117-e4ae-40d9-847d-0d6d5d2b1317', 'options': 'recursive,force'} 2023-11-16T16:43:10.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:10 smithi001 ceph-mon[98800]: pgmap v123: 97 pgs: 97 active+clean; 292 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.8 MiB/s wr, 139 op/s 2023-11-16T16:43:11.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:10 smithi026 ceph-mon[100112]: pgmap v123: 97 pgs: 97 active+clean; 292 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.8 MiB/s wr, 139 op/s 2023-11-16T16:43:11.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:10 smithi053 ceph-mon[100638]: pgmap v123: 97 pgs: 97 active+clean; 292 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.8 MiB/s wr, 139 op/s 2023-11-16T16:43:12.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:12 smithi053 ceph-mon[100638]: pgmap v124: 97 pgs: 97 active+clean; 326 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.7 MiB/s wr, 164 op/s 2023-11-16T16:43:12.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:12 smithi001 ceph-mon[98800]: pgmap v124: 97 pgs: 97 active+clean; 326 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.7 MiB/s wr, 164 op/s 2023-11-16T16:43:12.757 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:12 smithi026 ceph-mon[100112]: pgmap v124: 97 pgs: 97 active+clean; 326 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.7 MiB/s wr, 164 op/s 2023-11-16T16:43:14.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:14 smithi001 ceph-mon[98800]: pgmap v125: 97 pgs: 97 active+clean; 326 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.9 MiB/s wr, 47 op/s 2023-11-16T16:43:15.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:14 smithi026 ceph-mon[100112]: pgmap v125: 97 pgs: 97 active+clean; 326 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.9 MiB/s wr, 47 op/s 2023-11-16T16:43:15.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:14 smithi053 ceph-mon[100638]: pgmap v125: 97 pgs: 97 active+clean; 326 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.9 MiB/s wr, 47 op/s 2023-11-16T16:43:16.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:16 smithi001 ceph-mon[98800]: pgmap v126: 97 pgs: 97 active+clean; 351 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.0 MiB/s wr, 78 op/s 2023-11-16T16:43:16.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:16 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:43:16.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:16 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:43:17.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:16 smithi026 ceph-mon[100112]: pgmap v126: 97 pgs: 97 active+clean; 351 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.0 MiB/s wr, 78 op/s 2023-11-16T16:43:17.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:16 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:43:17.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:16 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:43:17.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:16 smithi053 ceph-mon[100638]: pgmap v126: 97 pgs: 97 active+clean; 351 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 9.0 MiB/s wr, 78 op/s 2023-11-16T16:43:17.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:16 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:43:17.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:16 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:43:18.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:18 smithi001 ceph-mon[98800]: pgmap v127: 97 pgs: 97 active+clean; 351 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.0 MiB/s wr, 56 op/s 2023-11-16T16:43:19.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:18 smithi026 ceph-mon[100112]: pgmap v127: 97 pgs: 97 active+clean; 351 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.0 MiB/s wr, 56 op/s 2023-11-16T16:43:19.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:18 smithi053 ceph-mon[100638]: pgmap v127: 97 pgs: 97 active+clean; 351 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.0 MiB/s wr, 56 op/s 2023-11-16T16:43:20.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:20 smithi001 ceph-mon[98800]: pgmap v128: 97 pgs: 97 active+clean; 351 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.0 MiB/s wr, 56 op/s 2023-11-16T16:43:21.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:20 smithi026 ceph-mon[100112]: pgmap v128: 97 pgs: 97 active+clean; 351 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.0 MiB/s wr, 56 op/s 2023-11-16T16:43:21.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:20 smithi053 ceph-mon[100638]: pgmap v128: 97 pgs: 97 active+clean; 351 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.0 MiB/s wr, 56 op/s 2023-11-16T16:43:22.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:22 smithi053 ceph-mon[100638]: pgmap v129: 97 pgs: 97 active+clean; 371 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.7 MiB/s wr, 76 op/s 2023-11-16T16:43:22.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:22 smithi001 ceph-mon[98800]: pgmap v129: 97 pgs: 97 active+clean; 371 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.7 MiB/s wr, 76 op/s 2023-11-16T16:43:22.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:22 smithi026 ceph-mon[100112]: pgmap v129: 97 pgs: 97 active+clean; 371 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.7 MiB/s wr, 76 op/s 2023-11-16T16:43:24.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:24 smithi001 ceph-mon[98800]: pgmap v130: 97 pgs: 97 active+clean; 371 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 51 op/s 2023-11-16T16:43:25.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:24 smithi026 ceph-mon[100112]: pgmap v130: 97 pgs: 97 active+clean; 371 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 51 op/s 2023-11-16T16:43:25.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:24 smithi053 ceph-mon[100638]: pgmap v130: 97 pgs: 97 active+clean; 371 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 51 op/s 2023-11-16T16:43:26.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:26 smithi001 ceph-mon[98800]: pgmap v131: 97 pgs: 97 active+clean; 392 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.5 MiB/s wr, 64 op/s 2023-11-16T16:43:27.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:26 smithi026 ceph-mon[100112]: pgmap v131: 97 pgs: 97 active+clean; 392 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.5 MiB/s wr, 64 op/s 2023-11-16T16:43:27.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:26 smithi053 ceph-mon[100638]: pgmap v131: 97 pgs: 97 active+clean; 392 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.5 MiB/s wr, 64 op/s 2023-11-16T16:43:28.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:28 smithi001 ceph-mon[98800]: pgmap v132: 97 pgs: 97 active+clean; 392 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 33 op/s 2023-11-16T16:43:28.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:28 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:43:29.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:28 smithi026 ceph-mon[100112]: pgmap v132: 97 pgs: 97 active+clean; 392 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 33 op/s 2023-11-16T16:43:29.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:28 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:43:29.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:28 smithi053 ceph-mon[100638]: pgmap v132: 97 pgs: 97 active+clean; 392 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 33 op/s 2023-11-16T16:43:29.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:28 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:43:29.637 INFO:tasks.workunit.client.0.smithi001.stdout:created hierarchy, now cleaning up 2023-11-16T16:43:29.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:29 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:43:29.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:29 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:43:29.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:29 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:43:30.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:29 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:43:30.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:29 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:43:30.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:29 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:43:30.082 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:29 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:43:30.082 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:29 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:43:30.082 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:29 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:43:30.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:30 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:43:30.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:30 smithi001 ceph-mon[98800]: Service mds.all has no related daemon already placed. 2023-11-16T16:43:30.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:30 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:43:30.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:30 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:43:30.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:30 smithi001 ceph-mon[98800]: pgmap v133: 97 pgs: 97 active+clean; 392 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 33 op/s 2023-11-16T16:43:31.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:30 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:43:31.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:30 smithi026 ceph-mon[100112]: Service mds.all has no related daemon already placed. 2023-11-16T16:43:31.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:30 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:43:31.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:30 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:43:31.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:30 smithi026 ceph-mon[100112]: pgmap v133: 97 pgs: 97 active+clean; 392 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 33 op/s 2023-11-16T16:43:31.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:30 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:43:31.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:30 smithi053 ceph-mon[100638]: Service mds.all has no related daemon already placed. 2023-11-16T16:43:31.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:30 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:43:31.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:30 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:43:31.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:30 smithi053 ceph-mon[100638]: pgmap v133: 97 pgs: 97 active+clean; 392 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 33 op/s 2023-11-16T16:43:32.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:32 smithi053 ceph-mon[100638]: pgmap v134: 97 pgs: 97 active+clean; 402 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.3 MiB/s wr, 44 op/s 2023-11-16T16:43:32.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:32 smithi001 ceph-mon[98800]: pgmap v134: 97 pgs: 97 active+clean; 402 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.3 MiB/s wr, 44 op/s 2023-11-16T16:43:32.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:32 smithi026 ceph-mon[100112]: pgmap v134: 97 pgs: 97 active+clean; 402 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.3 MiB/s wr, 44 op/s 2023-11-16T16:43:34.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:34 smithi001 ceph-mon[98800]: pgmap v135: 97 pgs: 97 active+clean; 402 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 24 op/s 2023-11-16T16:43:35.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:34 smithi026 ceph-mon[100112]: pgmap v135: 97 pgs: 97 active+clean; 402 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 24 op/s 2023-11-16T16:43:35.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:34 smithi053 ceph-mon[100638]: pgmap v135: 97 pgs: 97 active+clean; 402 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 24 op/s 2023-11-16T16:43:36.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:36 smithi001 ceph-mon[98800]: pgmap v136: 97 pgs: 97 active+clean; 411 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 30 op/s 2023-11-16T16:43:36.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:36 smithi001 ceph-mon[98800]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:43:37.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:36 smithi026 ceph-mon[100112]: pgmap v136: 97 pgs: 97 active+clean; 411 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 30 op/s 2023-11-16T16:43:37.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:36 smithi026 ceph-mon[100112]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:43:37.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:36 smithi053 ceph-mon[100638]: pgmap v136: 97 pgs: 97 active+clean; 411 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 30 op/s 2023-11-16T16:43:37.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:36 smithi053 ceph-mon[100638]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:43:38.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:38 smithi001 ceph-mon[98800]: pgmap v137: 97 pgs: 97 active+clean; 411 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 16 op/s 2023-11-16T16:43:39.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:38 smithi026 ceph-mon[100112]: pgmap v137: 97 pgs: 97 active+clean; 411 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 16 op/s 2023-11-16T16:43:39.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:38 smithi053 ceph-mon[100638]: pgmap v137: 97 pgs: 97 active+clean; 411 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 16 op/s 2023-11-16T16:43:39.227 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-16T16:43:39.363 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.359+0000 7f7400091700 1 -- 172.21.15.1:0/4094551743 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73f81055c0 msgr2=0x7f73f810d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:43:39.364 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.359+0000 7f7400091700 1 --2- 172.21.15.1:0/4094551743 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73f81055c0 0x7f73f810d140 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f73f000b0d0 tx=0x7f73f000b3e0 comp rx=0 tx=0).stop 2023-11-16T16:43:39.364 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.359+0000 7f7400091700 1 -- 172.21.15.1:0/4094551743 shutdown_connections 2023-11-16T16:43:39.364 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.359+0000 7f7400091700 1 --2- 172.21.15.1:0/4094551743 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73f81055c0 0x7f73f810d140 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:39.364 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.359+0000 7f7400091700 1 --2- 172.21.15.1:0/4094551743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73f8100cc0 0x7f73f8101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:39.364 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.359+0000 7f7400091700 1 --2- 172.21.15.1:0/4094551743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73f8101660 0x7f73f8104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:39.364 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.359+0000 7f7400091700 1 -- 172.21.15.1:0/4094551743 >> 172.21.15.1:0/4094551743 conn(0x7f73f8077560 msgr2=0x7f73f8077960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:43:39.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.359+0000 7f7400091700 1 -- 172.21.15.1:0/4094551743 shutdown_connections 2023-11-16T16:43:39.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.359+0000 7f7400091700 1 -- 172.21.15.1:0/4094551743 wait complete. 2023-11-16T16:43:39.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.360+0000 7f7400091700 1 Processor -- start 2023-11-16T16:43:39.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.360+0000 7f7400091700 1 -- start start 2023-11-16T16:43:39.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.360+0000 7f7400091700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73f8100cc0 0x7f73f8072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:39.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.360+0000 7f7400091700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73f8101660 0x7f73f8073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:39.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.360+0000 7f7400091700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73f81055c0 0x7f73f81a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:39.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.360+0000 7f7400091700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f8073b60 con 0x7f73f8101660 2023-11-16T16:43:39.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.360+0000 7f7400091700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f81a4c50 con 0x7f73f8100cc0 2023-11-16T16:43:39.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.360+0000 7f7400091700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f81a4d90 con 0x7f73f81055c0 2023-11-16T16:43:39.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.360+0000 7f73fd62c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73f8101660 0x7f73f8073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:39.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.360+0000 7f73fde2d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73f8100cc0 0x7f73f8072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:39.367 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.360+0000 7f73fd62c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73f8101660 0x7f73f8073330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45182/0 (socket says 172.21.15.1:45182) 2023-11-16T16:43:39.367 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.360+0000 7f73fd62c700 1 -- 172.21.15.1:0/4228196263 learned_addr learned my addr 172.21.15.1:0/4228196263 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:43:39.367 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.360+0000 7f73fe62e700 1 --2- 172.21.15.1:0/4228196263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73f81055c0 0x7f73f81a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:39.367 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.360+0000 7f73fd62c700 1 -- 172.21.15.1:0/4228196263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73f81055c0 msgr2=0x7f73f81a4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:43:39.367 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.360+0000 7f73fd62c700 1 --2- 172.21.15.1:0/4228196263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73f81055c0 0x7f73f81a4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:39.367 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.360+0000 7f73fd62c700 1 -- 172.21.15.1:0/4228196263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73f8100cc0 msgr2=0x7f73f8072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:43:39.367 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.360+0000 7f73fd62c700 1 --2- 172.21.15.1:0/4228196263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73f8100cc0 0x7f73f8072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:39.367 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.360+0000 7f73fd62c700 1 -- 172.21.15.1:0/4228196263 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f0009d00 con 0x7f73f8101660 2023-11-16T16:43:39.367 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.361+0000 7f73fd62c700 1 --2- 172.21.15.1:0/4228196263 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73f8101660 0x7f73f8073330 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f73e800e910 tx=0x7f73e800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:43:39.368 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.361+0000 7f73eeffd700 1 -- 172.21.15.1:0/4228196263 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e8004a30 con 0x7f73f8101660 2023-11-16T16:43:39.368 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.361+0000 7f73eeffd700 1 -- 172.21.15.1:0/4228196263 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73e80054e0 con 0x7f73f8101660 2023-11-16T16:43:39.368 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.361+0000 7f7400091700 1 -- 172.21.15.1:0/4228196263 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73f8110340 con 0x7f73f8101660 2023-11-16T16:43:39.368 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.361+0000 7f73eeffd700 1 -- 172.21.15.1:0/4228196263 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e800cc40 con 0x7f73f8101660 2023-11-16T16:43:39.368 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.361+0000 7f7400091700 1 -- 172.21.15.1:0/4228196263 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73f81101b0 con 0x7f73f8101660 2023-11-16T16:43:39.378 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.364+0000 7f73eeffd700 1 -- 172.21.15.1:0/4228196263 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f73e8005000 con 0x7f73f8101660 2023-11-16T16:43:39.379 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.364+0000 7f73eeffd700 1 --2- 172.21.15.1:0/4228196263 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f73e40a1340 0x7f73e40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:39.379 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.364+0000 7f73eeffd700 1 -- 172.21.15.1:0/4228196263 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f73e8014070 con 0x7f73f8101660 2023-11-16T16:43:39.379 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.364+0000 7f73fde2d700 1 --2- 172.21.15.1:0/4228196263 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f73e40a1340 0x7f73e40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:39.379 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.365+0000 7f73fde2d700 1 --2- 172.21.15.1:0/4228196263 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f73e40a1340 0x7f73e40a37f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f73f4009c60 tx=0x7f73f4009260 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:43:39.379 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.365+0000 7f740c21db80 1 Processor -- start 2023-11-16T16:43:39.379 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.366+0000 7f740c21db80 1 -- start start 2023-11-16T16:43:39.380 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.366+0000 7f740c21db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa5533cb60 0x55fa5533cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:39.380 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.366+0000 7f740c21db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa5533d470 0x55fa55341370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:39.380 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.366+0000 7f740c21db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa553418b0 0x55fa55341d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:39.380 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.366+0000 7f740c21db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa55242060 con 0x55fa5533d470 2023-11-16T16:43:39.380 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.366+0000 7f740c21db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa552948a0 con 0x55fa5533cb60 2023-11-16T16:43:39.380 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.366+0000 7f740c21db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa5528dd30 con 0x55fa553418b0 2023-11-16T16:43:39.380 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.366+0000 7f73fd62c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa5533d470 0x55fa55341370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:39.380 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.366+0000 7f73fde2d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa5533cb60 0x55fa5533cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:39.381 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.366+0000 7f73fd62c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa5533d470 0x55fa55341370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45190/0 (socket says 172.21.15.1:45190) 2023-11-16T16:43:39.381 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.366+0000 7f73fd62c700 1 -- 172.21.15.1:0/3036826780 learned_addr learned my addr 172.21.15.1:0/3036826780 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:43:39.381 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.366+0000 7f73fde2d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa5533cb60 0x55fa5533cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.1:39374/0 (socket says 172.21.15.1:39374) 2023-11-16T16:43:39.381 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.366+0000 7f73fe62e700 1 --2- 172.21.15.1:0/3036826780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa553418b0 0x55fa55341d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:39.381 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.366+0000 7f73fd62c700 1 -- 172.21.15.1:0/3036826780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa553418b0 msgr2=0x55fa55341d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:43:39.381 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.366+0000 7f73fd62c700 1 --2- 172.21.15.1:0/3036826780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa553418b0 0x55fa55341d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:39.381 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.366+0000 7f73fd62c700 1 -- 172.21.15.1:0/3036826780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa5533cb60 msgr2=0x55fa5533cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:43:39.381 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.366+0000 7f73fd62c700 1 --2- 172.21.15.1:0/3036826780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa5533cb60 0x55fa5533cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:39.381 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.366+0000 7f73fd62c700 1 -- 172.21.15.1:0/3036826780 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa5510f630 con 0x55fa5533d470 2023-11-16T16:43:39.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.366+0000 7f73fde2d700 1 --2- 172.21.15.1:0/3036826780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa5533cb60 0x55fa5533cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:43:39.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.366+0000 7f73fd62c700 1 --2- 172.21.15.1:0/3036826780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa5533d470 0x55fa55341370 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f73e80969e0 tx=0x7f73e80998d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=977eae6f9de3e1ab server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:43:39.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.366+0000 7f73e3fff700 1 -- 172.21.15.1:0/3036826780 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e809dfa0 con 0x55fa5533d470 2023-11-16T16:43:39.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.367+0000 7f73e3fff700 1 -- 172.21.15.1:0/3036826780 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73e80a23c0 con 0x55fa5533d470 2023-11-16T16:43:39.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.367+0000 7f73e3fff700 1 -- 172.21.15.1:0/3036826780 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e809a1d0 con 0x55fa5533d470 2023-11-16T16:43:39.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.367+0000 7f740c21db80 1 -- 172.21.15.1:0/3036826780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa5533d470 msgr2=0x55fa55341370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:43:39.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.367+0000 7f740c21db80 1 --2- 172.21.15.1:0/3036826780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa5533d470 0x55fa55341370 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f73e80969e0 tx=0x7f73e80998d0 comp rx=0 tx=0).stop 2023-11-16T16:43:39.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.367+0000 7f740c21db80 1 -- 172.21.15.1:0/3036826780 shutdown_connections 2023-11-16T16:43:39.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.367+0000 7f740c21db80 1 --2- 172.21.15.1:0/3036826780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa553418b0 0x55fa55341d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:39.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.367+0000 7f740c21db80 1 --2- 172.21.15.1:0/3036826780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa5533cb60 0x55fa5533cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:39.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.367+0000 7f740c21db80 1 --2- 172.21.15.1:0/3036826780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa5533d470 0x55fa55341370 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:39.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.367+0000 7f740c21db80 1 -- 172.21.15.1:0/3036826780 >> 172.21.15.1:0/3036826780 conn(0x55fa55332360 msgr2=0x55fa55337b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:43:39.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.367+0000 7f740c21db80 1 -- 172.21.15.1:0/3036826780 shutdown_connections 2023-11-16T16:43:39.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.367+0000 7f740c21db80 1 -- 172.21.15.1:0/3036826780 wait complete. 2023-11-16T16:43:39.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.368+0000 7f740c21db80 1 -- start start 2023-11-16T16:43:39.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.368+0000 7f740c21db80 1 Processor -- start 2023-11-16T16:43:39.384 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.368+0000 7f740c21db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa5533cb60 0x55fa553490c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:39.384 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.369+0000 7f740c21db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa5533d470 0x55fa553462d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:39.384 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.369+0000 7f740c21db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa553418b0 0x55fa553469c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:39.384 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.369+0000 7f740c21db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa5522f8e0 con 0x55fa5533d470 2023-11-16T16:43:39.384 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.369+0000 7f740c21db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa5522f580 con 0x55fa5533cb60 2023-11-16T16:43:39.384 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.369+0000 7f740c21db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa55111920 con 0x55fa553418b0 2023-11-16T16:43:39.384 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.369+0000 7f740c21db80 10 client.0 _ll_register_callbacks cb 0x55fa5524d780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:43:39.384 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.369+0000 7f73fd62c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa5533d470 0x55fa553462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:39.385 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.369+0000 7f73fde2d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa5533cb60 0x55fa553490c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:39.385 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.369+0000 7f73fd62c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa5533d470 0x55fa553462d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45196/0 (socket says 172.21.15.1:45196) 2023-11-16T16:43:39.385 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.369+0000 7f73fd62c700 1 -- 172.21.15.1:0/468850348 learned_addr learned my addr 172.21.15.1:0/468850348 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:43:39.385 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.369+0000 7f73fe62e700 1 --2- 172.21.15.1:0/468850348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa553418b0 0x55fa553469c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:39.385 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.369+0000 7f73fde2d700 1 -- 172.21.15.1:0/468850348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa553418b0 msgr2=0x55fa553469c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:43:39.385 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.369+0000 7f73fde2d700 1 --2- 172.21.15.1:0/468850348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa553418b0 0x55fa553469c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:39.385 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.369+0000 7f73fde2d700 1 -- 172.21.15.1:0/468850348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa5533d470 msgr2=0x55fa553462d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:43:39.385 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.369+0000 7f73fde2d700 1 --2- 172.21.15.1:0/468850348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa5533d470 0x55fa553462d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:39.386 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.369+0000 7f73fde2d700 1 -- 172.21.15.1:0/468850348 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73f81101b0 con 0x55fa5533cb60 2023-11-16T16:43:39.386 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.369+0000 7f73fe62e700 1 --2- 172.21.15.1:0/468850348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa553418b0 0x55fa553469c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:43:39.386 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.369+0000 7f73fd62c700 1 --2- 172.21.15.1:0/468850348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa5533d470 0x55fa553462d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:43:39.386 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.369+0000 7f73fde2d700 1 --2- 172.21.15.1:0/468850348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa5533cb60 0x55fa553490c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f73f4011ab0 tx=0x7f73f40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59ac4ad416197fdc server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:43:39.386 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.369+0000 7f73e17fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-11-16T16:43:39.386 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.369+0000 7f73e17fa700 1 -- 172.21.15.1:0/468850348 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f4022070 con 0x55fa5533cb60 2023-11-16T16:43:39.386 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.370+0000 7f73e17fa700 1 -- 172.21.15.1:0/468850348 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f73f401d620 con 0x55fa5533cb60 2023-11-16T16:43:39.386 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.370+0000 7f740c21db80 1 -- 172.21.15.1:0/468850348 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fa5532fb20 con 0x55fa5533cb60 2023-11-16T16:43:39.387 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.370+0000 7f73e17fa700 1 -- 172.21.15.1:0/468850348 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73f401d7e0 con 0x55fa5533cb60 2023-11-16T16:43:39.387 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.370+0000 7f740c21db80 10 client.25700 fetch_fsmap learned FSMap version 27 2023-11-16T16:43:39.387 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.370+0000 7f740c21db80 1 -- 172.21.15.1:0/468850348 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55fa55254190 con 0x55fa5533cb60 2023-11-16T16:43:39.387 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.370+0000 7f73e17fa700 1 -- 172.21.15.1:0/468850348 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73f4033080 con 0x55fa5533cb60 2023-11-16T16:43:39.387 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.370+0000 7f73e17fa700 1 -- 172.21.15.1:0/468850348 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 27) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f73f4027a20 con 0x55fa5533cb60 2023-11-16T16:43:39.387 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.370+0000 7f740c21db80 10 client.25700 fetch_fsmap finished waiting for FSMap version 27 2023-11-16T16:43:39.387 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.370+0000 7f740c21db80 10 client.25700 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:43:39.387 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.370+0000 7f740c21db80 20 client.25700 populate_metadata read hostname 'smithi001' 2023-11-16T16:43:39.387 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.370+0000 7f740c21db80 1 --2- 172.21.15.1:0/468850348 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55fa553569d0 0x55fa55358de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:39.388 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.370+0000 7f740c21db80 4 client.25700 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:43:39.388 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.370+0000 7f740c21db80 1 -- 172.21.15.1:0/468850348 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fa552500a0 con 0x55fa553569d0 2023-11-16T16:43:39.388 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.370+0000 7f73fd62c700 1 --2- 172.21.15.1:0/468850348 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55fa553569d0 0x55fa55358de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:39.388 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.371+0000 7f73e17fa700 10 client.25700 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:43:39.388 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.371+0000 7f73fd62c700 1 --2- 172.21.15.1:0/468850348 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55fa553569d0 0x55fa55358de0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=101a2a2d0b258a2d server_cookie=8c74ff4bf8780b96 in_seq=0 out_seq=0 2023-11-16T16:43:39.388 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.374+0000 7f73e17fa700 1 -- 172.21.15.1:0/468850348 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fa552500a0 con 0x55fa553569d0 2023-11-16T16:43:39.388 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.374+0000 7f73e17fa700 10 client.25700 handle_command_reply: tid=0 2023-11-16T16:43:39.388 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.374+0000 7f740c21db80 1 client.25700 shutdown 2023-11-16T16:43:39.389 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.374+0000 7f740c21db80 1 -- 172.21.15.1:0/468850348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa5533cb60 msgr2=0x55fa553490c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:43:39.389 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.374+0000 7f740c21db80 1 --2- 172.21.15.1:0/468850348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa5533cb60 0x55fa553490c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f73f4011ab0 tx=0x7f73f40047c0 comp rx=0 tx=0).stop 2023-11-16T16:43:39.389 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.374+0000 7f740c21db80 1 -- 172.21.15.1:0/468850348 shutdown_connections 2023-11-16T16:43:39.389 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.374+0000 7f740c21db80 1 --2- 172.21.15.1:0/468850348 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55fa553569d0 0x55fa55358de0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:39.389 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.374+0000 7f740c21db80 1 --2- 172.21.15.1:0/468850348 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa553418b0 0x55fa553469c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:39.389 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.374+0000 7f73e17fa700 0 client.25700 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:43:39.389 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.374+0000 7f740c21db80 1 --2- 172.21.15.1:0/468850348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa5533cb60 0x55fa553490c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:39.389 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.374+0000 7f740c21db80 1 --2- 172.21.15.1:0/468850348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa5533d470 0x55fa553462d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:39.390 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.374+0000 7f740c21db80 1 -- 172.21.15.1:0/468850348 >> 172.21.15.1:0/468850348 conn(0x55fa55332360 msgr2=0x55fa55336de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:43:39.390 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.374+0000 7f740c21db80 1 -- 172.21.15.1:0/468850348 shutdown_connections 2023-11-16T16:43:39.390 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.374+0000 7f740c21db80 1 -- 172.21.15.1:0/468850348 wait complete. 2023-11-16T16:43:39.390 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.374+0000 7f740c21db80 20 client.25700 trim_cache size 0 max 16384 2023-11-16T16:43:39.400 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.392+0000 7f740c21db80 1 Processor -- start 2023-11-16T16:43:39.400 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.392+0000 7f740c21db80 1 -- start start 2023-11-16T16:43:39.400 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.392+0000 7f740c21db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa5533cb60 0x55fa55348bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:39.400 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.392+0000 7f740c21db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa5533d470 0x55fa5534d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:39.400 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.392+0000 7f740c21db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa553418b0 0x55fa55345b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:39.401 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.392+0000 7f740c21db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa55331f70 con 0x55fa5533cb60 2023-11-16T16:43:39.401 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.392+0000 7f740c21db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa55353fd0 con 0x55fa553418b0 2023-11-16T16:43:39.401 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.392+0000 7f740c21db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa55347790 con 0x55fa5533d470 2023-11-16T16:43:39.401 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.392+0000 7f73fde2d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa5533cb60 0x55fa55348bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:39.401 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.392+0000 7f73fd62c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa5533d470 0x55fa5534d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:39.401 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.392+0000 7f73fde2d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa5533cb60 0x55fa55348bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45202/0 (socket says 172.21.15.1:45202) 2023-11-16T16:43:39.401 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.392+0000 7f73fde2d700 1 -- 172.21.15.1:0/2147178646 learned_addr learned my addr 172.21.15.1:0/2147178646 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:43:39.401 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.392+0000 7f73fe62e700 1 --2- 172.21.15.1:0/2147178646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa553418b0 0x55fa55345b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:39.402 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.393+0000 7f73fde2d700 1 -- 172.21.15.1:0/2147178646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa5533d470 msgr2=0x55fa5534d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:43:39.402 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.393+0000 7f73fde2d700 1 --2- 172.21.15.1:0/2147178646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa5533d470 0x55fa5534d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:39.402 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.393+0000 7f73fde2d700 1 -- 172.21.15.1:0/2147178646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa553418b0 msgr2=0x55fa55345b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:43:39.402 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.393+0000 7f73fde2d700 1 --2- 172.21.15.1:0/2147178646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa553418b0 0x55fa55345b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:39.402 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.393+0000 7f73fde2d700 1 -- 172.21.15.1:0/2147178646 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa55253ee0 con 0x55fa5533cb60 2023-11-16T16:43:39.402 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.393+0000 7f73fd62c700 1 --2- 172.21.15.1:0/2147178646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa5533d470 0x55fa5534d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:43:39.403 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.393+0000 7f73fde2d700 1 --2- 172.21.15.1:0/2147178646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa5533cb60 0x55fa55348bf0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f73f4013010 tx=0x7f73f4011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42fb30e5e6a2de24 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:43:39.403 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.393+0000 7f73ecff9700 1 -- 172.21.15.1:0/2147178646 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f4022070 con 0x55fa5533cb60 2023-11-16T16:43:39.403 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.393+0000 7f73ecff9700 1 -- 172.21.15.1:0/2147178646 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fa5532fb20 con 0x55fa5533cb60 2023-11-16T16:43:39.403 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.393+0000 7f73ecff9700 1 -- 172.21.15.1:0/2147178646 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f402f930 con 0x55fa5533cb60 2023-11-16T16:43:39.403 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.394+0000 7f740c21db80 1 -- 172.21.15.1:0/2147178646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa5533cb60 msgr2=0x55fa55348bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:43:39.403 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.394+0000 7f740c21db80 1 --2- 172.21.15.1:0/2147178646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa5533cb60 0x55fa55348bf0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f73f4013010 tx=0x7f73f4011ae0 comp rx=0 tx=0).stop 2023-11-16T16:43:39.403 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.394+0000 7f740c21db80 1 -- 172.21.15.1:0/2147178646 shutdown_connections 2023-11-16T16:43:39.403 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.394+0000 7f740c21db80 1 --2- 172.21.15.1:0/2147178646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa553418b0 0x55fa55345b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:39.404 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.394+0000 7f740c21db80 1 --2- 172.21.15.1:0/2147178646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa5533cb60 0x55fa55348bf0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:39.404 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.394+0000 7f740c21db80 1 --2- 172.21.15.1:0/2147178646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa5533d470 0x55fa5534d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:39.404 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.394+0000 7f740c21db80 1 -- 172.21.15.1:0/2147178646 >> 172.21.15.1:0/2147178646 conn(0x55fa55332360 msgr2=0x55fa5533f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:43:39.404 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.394+0000 7f740c21db80 1 -- 172.21.15.1:0/2147178646 shutdown_connections 2023-11-16T16:43:39.404 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.394+0000 7f740c21db80 1 -- 172.21.15.1:0/2147178646 wait complete. 2023-11-16T16:43:39.404 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.394+0000 7f740c21db80 1 -- start start 2023-11-16T16:43:39.404 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.394+0000 7f740c21db80 1 Processor -- start 2023-11-16T16:43:39.404 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.394+0000 7f740c21db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa5533cb60 0x55fa55349b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:39.404 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.394+0000 7f740c21db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa5533d470 0x55fa5534a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:39.405 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.395+0000 7f740c21db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa553418b0 0x55fa553a9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:39.405 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.395+0000 7f740c21db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa553a9dc0 con 0x55fa553418b0 2023-11-16T16:43:39.405 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.395+0000 7f740c21db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa553a9f00 con 0x55fa5533cb60 2023-11-16T16:43:39.405 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.395+0000 7f740c21db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa553aa040 con 0x55fa5533d470 2023-11-16T16:43:39.405 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.395+0000 7f740c21db80 10 client.0 _ll_register_callbacks cb 0x55fa5524d780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:43:39.405 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.395+0000 7f73fe62e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa553418b0 0x55fa553a9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:39.405 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.395+0000 7f73fde2d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa5533cb60 0x55fa55349b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:39.405 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.395+0000 7f73fde2d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa5533cb60 0x55fa55349b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.1:39394/0 (socket says 172.21.15.1:39394) 2023-11-16T16:43:39.405 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.395+0000 7f73fde2d700 1 -- 172.21.15.1:0/2908356927 learned_addr learned my addr 172.21.15.1:0/2908356927 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:43:39.406 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.395+0000 7f73fd62c700 1 --2- 172.21.15.1:0/2908356927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa5533d470 0x55fa5534a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:39.406 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.395+0000 7f73fe62e700 1 -- 172.21.15.1:0/2908356927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa5533d470 msgr2=0x55fa5534a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:43:39.406 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.395+0000 7f73fe62e700 1 --2- 172.21.15.1:0/2908356927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa5533d470 0x55fa5534a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:39.406 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.395+0000 7f73fe62e700 1 -- 172.21.15.1:0/2908356927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa5533cb60 msgr2=0x55fa55349b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:43:39.406 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.395+0000 7f73fe62e700 1 --2- 172.21.15.1:0/2908356927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa5533cb60 0x55fa55349b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:39.406 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.395+0000 7f73fe62e700 1 -- 172.21.15.1:0/2908356927 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fa5510f630 con 0x55fa553418b0 2023-11-16T16:43:39.406 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.395+0000 7f73fe62e700 1 --2- 172.21.15.1:0/2908356927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa553418b0 0x55fa553a9880 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f73f8074f00 tx=0x7f73f00074e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d25002ce3c93e369 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:43:39.406 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.395+0000 7f73e17fa700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:43:39.406 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.395+0000 7f73e17fa700 1 -- 172.21.15.1:0/2908356927 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f0022070 con 0x55fa553418b0 2023-11-16T16:43:39.407 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.395+0000 7f73e17fa700 1 -- 172.21.15.1:0/2908356927 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f73f00077e0 con 0x55fa553418b0 2023-11-16T16:43:39.407 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.395+0000 7f740c21db80 1 -- 172.21.15.1:0/2908356927 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f73e80a23c0 con 0x55fa553418b0 2023-11-16T16:43:39.407 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.396+0000 7f73e17fa700 1 -- 172.21.15.1:0/2908356927 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73f000e040 con 0x55fa553418b0 2023-11-16T16:43:39.407 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.396+0000 7f740c21db80 10 client.16143 fetch_fsmap learned FSMap version 27 2023-11-16T16:43:39.407 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.396+0000 7f740c21db80 1 -- 172.21.15.1:0/2908356927 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55fa55254190 con 0x55fa553418b0 2023-11-16T16:43:39.407 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.396+0000 7f73e17fa700 1 -- 172.21.15.1:0/2908356927 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73e80a23c0 con 0x55fa553418b0 2023-11-16T16:43:39.407 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.396+0000 7f73e17fa700 1 -- 172.21.15.1:0/2908356927 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f73f000b100 con 0x55fa553418b0 2023-11-16T16:43:39.407 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.396+0000 7f740c21db80 10 client.16143 fetch_fsmap finished waiting for FSMap version 27 2023-11-16T16:43:39.408 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.396+0000 7f740c21db80 10 client.16143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:43:39.408 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.396+0000 7f740c21db80 20 client.16143 populate_metadata read hostname 'smithi001' 2023-11-16T16:43:39.408 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.396+0000 7f740c21db80 1 --2- 172.21.15.1:0/2908356927 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55fa553ab890 0x55fa553adca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:43:39.408 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.396+0000 7f740c21db80 4 client.16143 mds_command: new command op to 24521 tid=0[{"prefix": "scrub status"}] 2023-11-16T16:43:39.408 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.396+0000 7f740c21db80 1 -- 172.21.15.1:0/2908356927 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fa5525c660 con 0x55fa553ab890 2023-11-16T16:43:39.408 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.396+0000 7f73fde2d700 1 --2- 172.21.15.1:0/2908356927 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55fa553ab890 0x55fa553adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:43:39.408 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.397+0000 7f73e17fa700 10 client.16143 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:43:39.408 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.397+0000 7f73fde2d700 1 --2- 172.21.15.1:0/2908356927 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55fa553ab890 0x55fa553adca0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41fae0cf7390331c server_cookie=e6aba0c900e713d9 in_seq=0 out_seq=0 2023-11-16T16:43:39.410 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.400+0000 7f73e17fa700 1 -- 172.21.15.1:0/2908356927 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55fa5525c660 con 0x55fa553ab890 2023-11-16T16:43:39.410 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.400+0000 7f73e17fa700 10 client.16143 handle_command_reply: tid=0 2023-11-16T16:43:39.410 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.400+0000 7f740c21db80 1 client.16143 shutdown 2023-11-16T16:43:39.410 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.400+0000 7f740c21db80 1 -- 172.21.15.1:0/2908356927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa553418b0 msgr2=0x55fa553a9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:43:39.410 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.400+0000 7f740c21db80 1 --2- 172.21.15.1:0/2908356927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa553418b0 0x55fa553a9880 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f73f8074f00 tx=0x7f73f00074e0 comp rx=0 tx=0).stop 2023-11-16T16:43:39.410 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.401+0000 7f740c21db80 1 -- 172.21.15.1:0/2908356927 shutdown_connections 2023-11-16T16:43:39.410 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.401+0000 7f740c21db80 1 --2- 172.21.15.1:0/2908356927 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55fa553ab890 0x55fa553adca0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:39.411 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.401+0000 7f740c21db80 1 --2- 172.21.15.1:0/2908356927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa553418b0 0x55fa553a9880 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:39.411 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.401+0000 7f73e17fa700 0 client.16143 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:43:39.411 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.401+0000 7f740c21db80 1 --2- 172.21.15.1:0/2908356927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fa5533cb60 0x55fa55349b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:39.411 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.401+0000 7f740c21db80 1 --2- 172.21.15.1:0/2908356927 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55fa5533d470 0x55fa5534a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:39.411 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.401+0000 7f740c21db80 1 -- 172.21.15.1:0/2908356927 >> 172.21.15.1:0/2908356927 conn(0x55fa55332360 msgr2=0x55fa553458c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:43:39.411 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.401+0000 7f740c21db80 1 -- 172.21.15.1:0/2908356927 shutdown_connections 2023-11-16T16:43:39.411 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.401+0000 7f740c21db80 1 -- 172.21.15.1:0/2908356927 wait complete. 2023-11-16T16:43:39.411 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.401+0000 7f740c21db80 20 client.16143 trim_cache size 0 max 16384 2023-11-16T16:43:39.412 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:43:39.412 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "scrub active (1 inodes in the stack)", 2023-11-16T16:43:39.412 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2023-11-16T16:43:39.412 INFO:teuthology.orchestra.run.smithi001.stdout: "4e68c117-e4ae-40d9-847d-0d6d5d2b1317": { 2023-11-16T16:43:39.412 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2023-11-16T16:43:39.412 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "4e68c117-e4ae-40d9-847d-0d6d5d2b1317", 2023-11-16T16:43:39.412 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2023-11-16T16:43:39.412 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:43:39.413 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:43:39.413 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:43:39.413 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.401+0000 7f7400091700 1 -- 172.21.15.1:0/4228196263 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f73e40a1340 msgr2=0x7f73e40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:43:39.414 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.401+0000 7f7400091700 1 --2- 172.21.15.1:0/4228196263 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f73e40a1340 0x7f73e40a37f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f73f4009c60 tx=0x7f73f4009260 comp rx=0 tx=0).stop 2023-11-16T16:43:39.414 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.401+0000 7f7400091700 1 -- 172.21.15.1:0/4228196263 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73f8101660 msgr2=0x7f73f8073330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:43:39.414 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.401+0000 7f7400091700 1 --2- 172.21.15.1:0/4228196263 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73f8101660 0x7f73f8073330 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f73e800e910 tx=0x7f73e800ec20 comp rx=0 tx=0).stop 2023-11-16T16:43:39.414 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.401+0000 7f7400091700 1 -- 172.21.15.1:0/4228196263 shutdown_connections 2023-11-16T16:43:39.414 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.401+0000 7f7400091700 1 --2- 172.21.15.1:0/4228196263 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f73e40a1340 0x7f73e40a37f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:39.414 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.401+0000 7f7400091700 1 --2- 172.21.15.1:0/4228196263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73f81055c0 0x7f73f81a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:39.414 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.401+0000 7f7400091700 1 --2- 172.21.15.1:0/4228196263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73f8100cc0 0x7f73f8072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:39.414 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.401+0000 7f7400091700 1 --2- 172.21.15.1:0/4228196263 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73f8101660 0x7f73f8073330 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:43:39.415 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.401+0000 7f7400091700 1 -- 172.21.15.1:0/4228196263 >> 172.21.15.1:0/4228196263 conn(0x7f73f8077560 msgr2=0x7f73f80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:43:39.415 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.401+0000 7f7400091700 1 -- 172.21.15.1:0/4228196263 shutdown_connections 2023-11-16T16:43:39.415 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:43:39.401+0000 7f7400091700 1 -- 172.21.15.1:0/4228196263 wait complete. 2023-11-16T16:43:39.416 INFO:tasks.cephfs.filesystem:scrub status for tag:4e68c117-e4ae-40d9-847d-0d6d5d2b1317 - {'path': '/', 'tag': '4e68c117-e4ae-40d9-847d-0d6d5d2b1317', 'options': 'recursive,force'} 2023-11-16T16:43:40.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:40 smithi001 ceph-mon[98800]: pgmap v138: 97 pgs: 97 active+clean; 411 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 16 op/s 2023-11-16T16:43:41.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:40 smithi026 ceph-mon[100112]: pgmap v138: 97 pgs: 97 active+clean; 411 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 16 op/s 2023-11-16T16:43:41.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:40 smithi053 ceph-mon[100638]: pgmap v138: 97 pgs: 97 active+clean; 411 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 16 op/s 2023-11-16T16:43:41.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:41 smithi001 ceph-mon[98800]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:43:42.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:41 smithi026 ceph-mon[100112]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:43:42.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:41 smithi053 ceph-mon[100638]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:43:42.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:42 smithi001 ceph-mon[98800]: pgmap v139: 97 pgs: 97 active+clean; 426 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 30 op/s 2023-11-16T16:43:43.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:42 smithi026 ceph-mon[100112]: pgmap v139: 97 pgs: 97 active+clean; 426 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 30 op/s 2023-11-16T16:43:43.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:42 smithi053 ceph-mon[100638]: pgmap v139: 97 pgs: 97 active+clean; 426 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 30 op/s 2023-11-16T16:43:44.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:44 smithi001 ceph-mon[98800]: pgmap v140: 97 pgs: 97 active+clean; 426 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 19 op/s 2023-11-16T16:43:45.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:44 smithi026 ceph-mon[100112]: pgmap v140: 97 pgs: 97 active+clean; 426 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 19 op/s 2023-11-16T16:43:45.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:44 smithi053 ceph-mon[100638]: pgmap v140: 97 pgs: 97 active+clean; 426 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 19 op/s 2023-11-16T16:43:46.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:46 smithi001 ceph-mon[98800]: pgmap v141: 97 pgs: 97 active+clean; 449 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.9 MiB/s wr, 40 op/s 2023-11-16T16:43:47.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:46 smithi026 ceph-mon[100112]: pgmap v141: 97 pgs: 97 active+clean; 449 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.9 MiB/s wr, 40 op/s 2023-11-16T16:43:47.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:46 smithi053 ceph-mon[100638]: pgmap v141: 97 pgs: 97 active+clean; 449 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.9 MiB/s wr, 40 op/s 2023-11-16T16:43:48.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:48 smithi001 ceph-mon[98800]: pgmap v142: 97 pgs: 97 active+clean; 449 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 34 op/s 2023-11-16T16:43:49.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:48 smithi026 ceph-mon[100112]: pgmap v142: 97 pgs: 97 active+clean; 449 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 34 op/s 2023-11-16T16:43:49.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:48 smithi053 ceph-mon[100638]: pgmap v142: 97 pgs: 97 active+clean; 449 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 34 op/s 2023-11-16T16:43:50.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:50 smithi001 ceph-mon[98800]: pgmap v143: 97 pgs: 97 active+clean; 449 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 34 op/s 2023-11-16T16:43:51.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:50 smithi026 ceph-mon[100112]: pgmap v143: 97 pgs: 97 active+clean; 449 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 34 op/s 2023-11-16T16:43:51.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:50 smithi053 ceph-mon[100638]: pgmap v143: 97 pgs: 97 active+clean; 449 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 34 op/s 2023-11-16T16:43:52.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:52 smithi053 ceph-mon[100638]: pgmap v144: 97 pgs: 97 active+clean; 472 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.1 MiB/s wr, 59 op/s 2023-11-16T16:43:52.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:52 smithi001 ceph-mon[98800]: pgmap v144: 97 pgs: 97 active+clean; 472 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.1 MiB/s wr, 59 op/s 2023-11-16T16:43:52.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:52 smithi026 ceph-mon[100112]: pgmap v144: 97 pgs: 97 active+clean; 472 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.1 MiB/s wr, 59 op/s 2023-11-16T16:43:54.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:54 smithi001 ceph-mon[98800]: pgmap v145: 97 pgs: 97 active+clean; 472 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 45 op/s 2023-11-16T16:43:55.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:54 smithi026 ceph-mon[100112]: pgmap v145: 97 pgs: 97 active+clean; 472 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 45 op/s 2023-11-16T16:43:55.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:54 smithi053 ceph-mon[100638]: pgmap v145: 97 pgs: 97 active+clean; 472 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 45 op/s 2023-11-16T16:43:56.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:56 smithi001 ceph-mon[98800]: pgmap v146: 97 pgs: 97 active+clean; 494 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.7 MiB/s wr, 88 op/s 2023-11-16T16:43:57.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:56 smithi026 ceph-mon[100112]: pgmap v146: 97 pgs: 97 active+clean; 494 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.7 MiB/s wr, 88 op/s 2023-11-16T16:43:57.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:56 smithi053 ceph-mon[100638]: pgmap v146: 97 pgs: 97 active+clean; 494 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.7 MiB/s wr, 88 op/s 2023-11-16T16:43:58.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:43:58 smithi001 ceph-mon[98800]: pgmap v147: 97 pgs: 97 active+clean; 494 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 67 op/s 2023-11-16T16:43:59.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:43:58 smithi026 ceph-mon[100112]: pgmap v147: 97 pgs: 97 active+clean; 494 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 67 op/s 2023-11-16T16:43:59.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:43:58 smithi053 ceph-mon[100638]: pgmap v147: 97 pgs: 97 active+clean; 494 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 67 op/s 2023-11-16T16:44:00.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:00 smithi001 ceph-mon[98800]: pgmap v148: 97 pgs: 97 active+clean; 494 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 67 op/s 2023-11-16T16:44:01.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:00 smithi026 ceph-mon[100112]: pgmap v148: 97 pgs: 97 active+clean; 494 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 67 op/s 2023-11-16T16:44:01.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:00 smithi053 ceph-mon[100638]: pgmap v148: 97 pgs: 97 active+clean; 494 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 67 op/s 2023-11-16T16:44:02.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:02 smithi053 ceph-mon[100638]: pgmap v149: 97 pgs: 97 active+clean; 513 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.4 MiB/s wr, 105 op/s 2023-11-16T16:44:02.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:02 smithi001 ceph-mon[98800]: pgmap v149: 97 pgs: 97 active+clean; 513 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.4 MiB/s wr, 105 op/s 2023-11-16T16:44:02.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:02 smithi026 ceph-mon[100112]: pgmap v149: 97 pgs: 97 active+clean; 513 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.4 MiB/s wr, 105 op/s 2023-11-16T16:44:04.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:04 smithi001 ceph-mon[98800]: pgmap v150: 97 pgs: 97 active+clean; 513 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 79 op/s 2023-11-16T16:44:05.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:04 smithi026 ceph-mon[100112]: pgmap v150: 97 pgs: 97 active+clean; 513 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 79 op/s 2023-11-16T16:44:05.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:04 smithi053 ceph-mon[100638]: pgmap v150: 97 pgs: 97 active+clean; 513 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 79 op/s 2023-11-16T16:44:06.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:06 smithi001 ceph-mon[98800]: pgmap v151: 97 pgs: 97 active+clean; 539 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.7 MiB/s wr, 113 op/s 2023-11-16T16:44:07.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:06 smithi026 ceph-mon[100112]: pgmap v151: 97 pgs: 97 active+clean; 539 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.7 MiB/s wr, 113 op/s 2023-11-16T16:44:07.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:06 smithi053 ceph-mon[100638]: pgmap v151: 97 pgs: 97 active+clean; 539 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.7 MiB/s wr, 113 op/s 2023-11-16T16:44:08.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:08 smithi001 ceph-mon[98800]: pgmap v152: 97 pgs: 97 active+clean; 539 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 71 op/s 2023-11-16T16:44:09.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:08 smithi026 ceph-mon[100112]: pgmap v152: 97 pgs: 97 active+clean; 539 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 71 op/s 2023-11-16T16:44:09.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:08 smithi053 ceph-mon[100638]: pgmap v152: 97 pgs: 97 active+clean; 539 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 71 op/s 2023-11-16T16:44:09.417 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-16T16:44:09.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.549+0000 7f59162f3700 1 -- 172.21.15.1:0/3812417948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5910101660 msgr2=0x7f5910104e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:44:09.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.549+0000 7f59162f3700 1 --2- 172.21.15.1:0/3812417948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5910101660 0x7f5910104e90 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f59040099c0 tx=0x7f5904009cd0 comp rx=0 tx=0).stop 2023-11-16T16:44:09.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.549+0000 7f59162f3700 1 -- 172.21.15.1:0/3812417948 shutdown_connections 2023-11-16T16:44:09.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.549+0000 7f59162f3700 1 --2- 172.21.15.1:0/3812417948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f59101055c0 0x7f591010d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:09.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.549+0000 7f59162f3700 1 --2- 172.21.15.1:0/3812417948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5910100cc0 0x7f5910101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:09.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.549+0000 7f59162f3700 1 --2- 172.21.15.1:0/3812417948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5910101660 0x7f5910104e90 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:09.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.549+0000 7f59162f3700 1 -- 172.21.15.1:0/3812417948 >> 172.21.15.1:0/3812417948 conn(0x7f5910077560 msgr2=0x7f5910077960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:44:09.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.549+0000 7f59162f3700 1 -- 172.21.15.1:0/3812417948 shutdown_connections 2023-11-16T16:44:09.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.549+0000 7f59162f3700 1 -- 172.21.15.1:0/3812417948 wait complete. 2023-11-16T16:44:09.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.550+0000 7f59162f3700 1 Processor -- start 2023-11-16T16:44:09.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.550+0000 7f59162f3700 1 -- start start 2023-11-16T16:44:09.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.550+0000 7f59162f3700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5910100cc0 0x7f5910196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:09.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.550+0000 7f59162f3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5910101660 0x7f59101969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:09.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.550+0000 7f59162f3700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59101055c0 0x7f591019ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:09.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.550+0000 7f59162f3700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591019b470 con 0x7f59101055c0 2023-11-16T16:44:09.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.550+0000 7f59162f3700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591019b5e0 con 0x7f5910101660 2023-11-16T16:44:09.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.550+0000 7f59162f3700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591019b7a0 con 0x7f5910100cc0 2023-11-16T16:44:09.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.550+0000 7f590ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5910100cc0 0x7f5910196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:09.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.550+0000 7f590ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5910100cc0 0x7f5910196490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:41904/0 (socket says 172.21.15.1:41904) 2023-11-16T16:44:09.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.550+0000 7f5914890700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59101055c0 0x7f591019ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:09.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.550+0000 7f590ffff700 1 -- 172.21.15.1:0/3594568652 learned_addr learned my addr 172.21.15.1:0/3594568652 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:44:09.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.550+0000 7f590f7fe700 1 --2- 172.21.15.1:0/3594568652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5910101660 0x7f59101969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:09.557 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.550+0000 7f590ffff700 1 -- 172.21.15.1:0/3594568652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5910101660 msgr2=0x7f59101969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:44:09.557 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.550+0000 7f590ffff700 1 --2- 172.21.15.1:0/3594568652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5910101660 0x7f59101969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:09.557 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.550+0000 7f590ffff700 1 -- 172.21.15.1:0/3594568652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59101055c0 msgr2=0x7f591019ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:44:09.557 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.550+0000 7f590ffff700 1 --2- 172.21.15.1:0/3594568652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59101055c0 0x7f591019ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:09.557 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.551+0000 7f590ffff700 1 -- 172.21.15.1:0/3594568652 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59040095f0 con 0x7f5910100cc0 2023-11-16T16:44:09.557 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.551+0000 7f590ffff700 1 --2- 172.21.15.1:0/3594568652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5910100cc0 0x7f5910196490 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f590000e9a0 tx=0x7f590000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:44:09.557 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.551+0000 7f590d7fa700 1 -- 172.21.15.1:0/3594568652 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5900009790 con 0x7f5910100cc0 2023-11-16T16:44:09.557 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.551+0000 7f590d7fa700 1 -- 172.21.15.1:0/3594568652 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5900010430 con 0x7f5910100cc0 2023-11-16T16:44:09.558 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.551+0000 7f59162f3700 1 -- 172.21.15.1:0/3594568652 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59101102f0 con 0x7f5910100cc0 2023-11-16T16:44:09.558 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.551+0000 7f590d7fa700 1 -- 172.21.15.1:0/3594568652 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5900018680 con 0x7f5910100cc0 2023-11-16T16:44:09.558 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.551+0000 7f59162f3700 1 -- 172.21.15.1:0/3594568652 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5910110160 con 0x7f5910100cc0 2023-11-16T16:44:09.565 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.553+0000 7f590d7fa700 1 -- 172.21.15.1:0/3594568652 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f59000105a0 con 0x7f5910100cc0 2023-11-16T16:44:09.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.554+0000 7f590d7fa700 1 --2- 172.21.15.1:0/3594568652 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f58f80a12c0 0x7f58f80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:09.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.554+0000 7f590d7fa700 1 -- 172.21.15.1:0/3594568652 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f5900014070 con 0x7f5910100cc0 2023-11-16T16:44:09.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.554+0000 7f590f7fe700 1 --2- 172.21.15.1:0/3594568652 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f58f80a12c0 0x7f58f80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:09.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.554+0000 7f592247fb80 1 Processor -- start 2023-11-16T16:44:09.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.554+0000 7f590f7fe700 1 --2- 172.21.15.1:0/3594568652 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f58f80a12c0 0x7f58f80a3770 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f590400b580 tx=0x7f5904003680 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:44:09.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.555+0000 7f592247fb80 1 -- start start 2023-11-16T16:44:09.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.555+0000 7f592247fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573ad49bf70 0x5573ad49c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:09.566 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.555+0000 7f592247fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573ad49c880 0x5573ad4a0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:09.567 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.555+0000 7f592247fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573ad4a0cc0 0x5573ad4a1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:09.567 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.555+0000 7f592247fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573ad39d060 con 0x5573ad49bf70 2023-11-16T16:44:09.567 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.555+0000 7f592247fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573ad3ef8a0 con 0x5573ad4a0cc0 2023-11-16T16:44:09.567 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.555+0000 7f592247fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573ad3e8d30 con 0x5573ad49c880 2023-11-16T16:44:09.567 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.555+0000 7f590ffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573ad49bf70 0x5573ad49c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:09.567 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.555+0000 7f590ffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573ad49bf70 0x5573ad49c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45230/0 (socket says 172.21.15.1:45230) 2023-11-16T16:44:09.567 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.555+0000 7f590ffff700 1 -- 172.21.15.1:0/2602064319 learned_addr learned my addr 172.21.15.1:0/2602064319 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:44:09.567 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.555+0000 7f590f7fe700 1 --2- 172.21.15.1:0/2602064319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573ad49c880 0x5573ad4a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:09.567 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.555+0000 7f5914890700 1 --2- 172.21.15.1:0/2602064319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573ad4a0cc0 0x5573ad4a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:09.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.555+0000 7f590ffff700 1 -- 172.21.15.1:0/2602064319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573ad49c880 msgr2=0x5573ad4a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:44:09.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.555+0000 7f590ffff700 1 --2- 172.21.15.1:0/2602064319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573ad49c880 0x5573ad4a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:09.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.555+0000 7f590ffff700 1 -- 172.21.15.1:0/2602064319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573ad4a0cc0 msgr2=0x5573ad4a1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:44:09.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.555+0000 7f590ffff700 1 --2- 172.21.15.1:0/2602064319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573ad4a0cc0 0x5573ad4a1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:09.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.555+0000 7f590ffff700 1 -- 172.21.15.1:0/2602064319 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573ad26a630 con 0x5573ad49bf70 2023-11-16T16:44:09.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.555+0000 7f590ffff700 1 --2- 172.21.15.1:0/2602064319 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573ad49bf70 0x5573ad49c340 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f59000a1ed0 tx=0x7f5900099fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840b085dd07a9683 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:44:09.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.555+0000 7f58f67fc700 1 -- 172.21.15.1:0/2602064319 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5900010880 con 0x5573ad49bf70 2023-11-16T16:44:09.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.556+0000 7f58f67fc700 1 -- 172.21.15.1:0/2602064319 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59000109e0 con 0x5573ad49bf70 2023-11-16T16:44:09.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.556+0000 7f58f67fc700 1 -- 172.21.15.1:0/2602064319 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59000a1f70 con 0x5573ad49bf70 2023-11-16T16:44:09.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.556+0000 7f592247fb80 1 -- 172.21.15.1:0/2602064319 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573ad49bf70 msgr2=0x5573ad49c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:44:09.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.556+0000 7f592247fb80 1 --2- 172.21.15.1:0/2602064319 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573ad49bf70 0x5573ad49c340 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f59000a1ed0 tx=0x7f5900099fb0 comp rx=0 tx=0).stop 2023-11-16T16:44:09.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.556+0000 7f592247fb80 1 -- 172.21.15.1:0/2602064319 shutdown_connections 2023-11-16T16:44:09.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.556+0000 7f592247fb80 1 --2- 172.21.15.1:0/2602064319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573ad4a0cc0 0x5573ad4a1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:09.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.556+0000 7f592247fb80 1 --2- 172.21.15.1:0/2602064319 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573ad49bf70 0x5573ad49c340 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:09.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.556+0000 7f592247fb80 1 --2- 172.21.15.1:0/2602064319 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573ad49c880 0x5573ad4a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:09.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.556+0000 7f592247fb80 1 -- 172.21.15.1:0/2602064319 >> 172.21.15.1:0/2602064319 conn(0x5573ad48d360 msgr2=0x5573ad492b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:44:09.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.556+0000 7f592247fb80 1 -- 172.21.15.1:0/2602064319 shutdown_connections 2023-11-16T16:44:09.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.556+0000 7f592247fb80 1 -- 172.21.15.1:0/2602064319 wait complete. 2023-11-16T16:44:09.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.557+0000 7f592247fb80 1 -- start start 2023-11-16T16:44:09.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.557+0000 7f592247fb80 1 Processor -- start 2023-11-16T16:44:09.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.557+0000 7f592247fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573ad49bf70 0x5573ad4a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:09.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.557+0000 7f592247fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573ad49c880 0x5573ad4a5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:09.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.557+0000 7f592247fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573ad4a0cc0 0x5573ad4a5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:09.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.557+0000 7f592247fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573ad38a8e0 con 0x5573ad49c880 2023-11-16T16:44:09.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.557+0000 7f592247fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573ad38a580 con 0x5573ad4a0cc0 2023-11-16T16:44:09.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.557+0000 7f592247fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573ad26c920 con 0x5573ad49bf70 2023-11-16T16:44:09.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.557+0000 7f592247fb80 10 client.0 _ll_register_callbacks cb 0x5573ad3a8780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:44:09.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.557+0000 7f590f7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573ad49c880 0x5573ad4a5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:09.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.557+0000 7f590ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573ad49bf70 0x5573ad4a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:09.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.557+0000 7f590f7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573ad49c880 0x5573ad4a5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45238/0 (socket says 172.21.15.1:45238) 2023-11-16T16:44:09.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.557+0000 7f590f7fe700 1 -- 172.21.15.1:0/1300954200 learned_addr learned my addr 172.21.15.1:0/1300954200 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:44:09.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.557+0000 7f590ffff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573ad49bf70 0x5573ad4a84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:41918/0 (socket says 172.21.15.1:41918) 2023-11-16T16:44:09.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.557+0000 7f5914890700 1 --2- 172.21.15.1:0/1300954200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573ad4a0cc0 0x5573ad4a5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:09.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.558+0000 7f590ffff700 1 -- 172.21.15.1:0/1300954200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573ad4a0cc0 msgr2=0x5573ad4a5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:44:09.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.558+0000 7f590ffff700 1 --2- 172.21.15.1:0/1300954200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573ad4a0cc0 0x5573ad4a5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:09.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.558+0000 7f590ffff700 1 -- 172.21.15.1:0/1300954200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573ad49c880 msgr2=0x5573ad4a5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:44:09.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.558+0000 7f590ffff700 1 --2- 172.21.15.1:0/1300954200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573ad49c880 0x5573ad4a5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:09.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.558+0000 7f590ffff700 1 -- 172.21.15.1:0/1300954200 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5910110160 con 0x5573ad49bf70 2023-11-16T16:44:09.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.558+0000 7f590f7fe700 1 --2- 172.21.15.1:0/1300954200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573ad49c880 0x5573ad4a5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:44:09.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.558+0000 7f590ffff700 1 --2- 172.21.15.1:0/1300954200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573ad49bf70 0x5573ad4a84d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f59000a1ed0 tx=0x7f5900099750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78f0a1cf985db8c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:44:09.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.558+0000 7f58ef7fe700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-16T16:44:09.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.558+0000 7f58ef7fe700 1 -- 172.21.15.1:0/1300954200 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5900010880 con 0x5573ad49bf70 2023-11-16T16:44:09.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.558+0000 7f58ef7fe700 1 -- 172.21.15.1:0/1300954200 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f59000a2180 con 0x5573ad49bf70 2023-11-16T16:44:09.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.558+0000 7f592247fb80 1 -- 172.21.15.1:0/1300954200 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5573ad48ab20 con 0x5573ad49bf70 2023-11-16T16:44:09.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.558+0000 7f58ef7fe700 1 -- 172.21.15.1:0/1300954200 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5900094bc0 con 0x5573ad49bf70 2023-11-16T16:44:09.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.558+0000 7f592247fb80 10 client.25720 fetch_fsmap learned FSMap version 28 2023-11-16T16:44:09.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.558+0000 7f592247fb80 1 -- 172.21.15.1:0/1300954200 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5573ad3af190 con 0x5573ad49bf70 2023-11-16T16:44:09.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.558+0000 7f58ef7fe700 1 -- 172.21.15.1:0/1300954200 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5573ad48ab20 con 0x5573ad49bf70 2023-11-16T16:44:09.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.558+0000 7f58ef7fe700 1 -- 172.21.15.1:0/1300954200 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f59000b7fa0 con 0x5573ad49bf70 2023-11-16T16:44:09.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.559+0000 7f592247fb80 10 client.25720 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:44:09.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.559+0000 7f592247fb80 10 client.25720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:44:09.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.559+0000 7f592247fb80 20 client.25720 populate_metadata read hostname 'smithi001' 2023-11-16T16:44:09.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.559+0000 7f592247fb80 1 --2- 172.21.15.1:0/1300954200 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5573ad4b5880 0x5573ad4b7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:09.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.559+0000 7f592247fb80 4 client.25720 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:44:09.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.559+0000 7f592247fb80 1 -- 172.21.15.1:0/1300954200 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5573ad3ab0a0 con 0x5573ad4b5880 2023-11-16T16:44:09.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.559+0000 7f590f7fe700 1 --2- 172.21.15.1:0/1300954200 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5573ad4b5880 0x5573ad4b7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:09.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.559+0000 7f58ef7fe700 10 client.25720 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:44:09.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.559+0000 7f590f7fe700 1 --2- 172.21.15.1:0/1300954200 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5573ad4b5880 0x5573ad4b7c90 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ed02cc60f17fd25 server_cookie=2f868c3196a28edb in_seq=0 out_seq=0 2023-11-16T16:44:09.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.560+0000 7f58ef7fe700 1 -- 172.21.15.1:0/1300954200 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5573ad3ab0a0 con 0x5573ad4b5880 2023-11-16T16:44:09.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.560+0000 7f58ef7fe700 10 client.25720 handle_command_reply: tid=0 2023-11-16T16:44:09.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.560+0000 7f592247fb80 1 client.25720 shutdown 2023-11-16T16:44:09.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.560+0000 7f592247fb80 1 -- 172.21.15.1:0/1300954200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573ad49bf70 msgr2=0x5573ad4a84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:44:09.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.560+0000 7f592247fb80 1 --2- 172.21.15.1:0/1300954200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573ad49bf70 0x5573ad4a84d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f59000a1ed0 tx=0x7f5900099750 comp rx=0 tx=0).stop 2023-11-16T16:44:09.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.560+0000 7f592247fb80 1 -- 172.21.15.1:0/1300954200 shutdown_connections 2023-11-16T16:44:09.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.560+0000 7f592247fb80 1 --2- 172.21.15.1:0/1300954200 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5573ad4b5880 0x5573ad4b7c90 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:09.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.560+0000 7f592247fb80 1 --2- 172.21.15.1:0/1300954200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573ad4a0cc0 0x5573ad4a5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:09.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.560+0000 7f592247fb80 1 --2- 172.21.15.1:0/1300954200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573ad49bf70 0x5573ad4a84d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:09.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.560+0000 7f592247fb80 1 --2- 172.21.15.1:0/1300954200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573ad49c880 0x5573ad4a5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:09.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.560+0000 7f58ef7fe700 0 client.25720 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:44:09.576 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.560+0000 7f592247fb80 1 -- 172.21.15.1:0/1300954200 >> 172.21.15.1:0/1300954200 conn(0x5573ad48d360 msgr2=0x5573ad4a2170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:44:09.576 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.560+0000 7f592247fb80 1 -- 172.21.15.1:0/1300954200 shutdown_connections 2023-11-16T16:44:09.576 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.560+0000 7f592247fb80 1 -- 172.21.15.1:0/1300954200 wait complete. 2023-11-16T16:44:09.576 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.560+0000 7f592247fb80 20 client.25720 trim_cache size 0 max 16384 2023-11-16T16:44:09.587 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.578+0000 7f592247fb80 1 Processor -- start 2023-11-16T16:44:09.587 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.578+0000 7f592247fb80 1 -- start start 2023-11-16T16:44:09.587 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.578+0000 7f592247fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573ad49bf70 0x5573ad4acde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:09.587 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.578+0000 7f592247fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573ad49c880 0x5573ad4a0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:09.587 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.578+0000 7f592247fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573ad4a0cc0 0x5573ad4a20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:09.587 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.578+0000 7f592247fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573ad4a6d30 con 0x5573ad4a0cc0 2023-11-16T16:44:09.587 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.578+0000 7f592247fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573ad4b2760 con 0x5573ad49c880 2023-11-16T16:44:09.587 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.578+0000 7f592247fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573ad48a9c0 con 0x5573ad49bf70 2023-11-16T16:44:09.588 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.578+0000 7f5914890700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573ad4a0cc0 0x5573ad4a20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:09.588 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.578+0000 7f590f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573ad49c880 0x5573ad4a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:09.588 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.578+0000 7f5914890700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573ad4a0cc0 0x5573ad4a20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45248/0 (socket says 172.21.15.1:45248) 2023-11-16T16:44:09.588 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.578+0000 7f5914890700 1 -- 172.21.15.1:0/253192334 learned_addr learned my addr 172.21.15.1:0/253192334 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:44:09.588 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.578+0000 7f590ffff700 1 --2- 172.21.15.1:0/253192334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573ad49bf70 0x5573ad4acde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:09.588 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.579+0000 7f5914890700 1 -- 172.21.15.1:0/253192334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573ad49bf70 msgr2=0x5573ad4acde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:44:09.588 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.579+0000 7f5914890700 1 --2- 172.21.15.1:0/253192334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573ad49bf70 0x5573ad4acde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:09.588 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.579+0000 7f5914890700 1 -- 172.21.15.1:0/253192334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573ad49c880 msgr2=0x5573ad4a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:44:09.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.579+0000 7f5914890700 1 --2- 172.21.15.1:0/253192334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573ad49c880 0x5573ad4a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:09.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.579+0000 7f5914890700 1 -- 172.21.15.1:0/253192334 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573ad3aeee0 con 0x5573ad4a0cc0 2023-11-16T16:44:09.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.579+0000 7f590f7fe700 1 --2- 172.21.15.1:0/253192334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573ad49c880 0x5573ad4a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:44:09.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.579+0000 7f5914890700 1 --2- 172.21.15.1:0/253192334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573ad4a0cc0 0x5573ad4a20a0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f59080118f0 tx=0x7f5908009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa8ff1a296461545 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:44:09.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.579+0000 7f58f6ffd700 1 -- 172.21.15.1:0/253192334 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590800e040 con 0x5573ad4a0cc0 2023-11-16T16:44:09.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.579+0000 7f58f6ffd700 1 -- 172.21.15.1:0/253192334 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f590800f040 con 0x5573ad4a0cc0 2023-11-16T16:44:09.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.579+0000 7f58f6ffd700 1 -- 172.21.15.1:0/253192334 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59080079c0 con 0x5573ad4a0cc0 2023-11-16T16:44:09.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.580+0000 7f592247fb80 1 -- 172.21.15.1:0/253192334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573ad4a0cc0 msgr2=0x5573ad4a20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:44:09.589 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.580+0000 7f592247fb80 1 --2- 172.21.15.1:0/253192334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573ad4a0cc0 0x5573ad4a20a0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f59080118f0 tx=0x7f5908009920 comp rx=0 tx=0).stop 2023-11-16T16:44:09.590 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.580+0000 7f592247fb80 1 -- 172.21.15.1:0/253192334 shutdown_connections 2023-11-16T16:44:09.590 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.580+0000 7f592247fb80 1 --2- 172.21.15.1:0/253192334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573ad4a0cc0 0x5573ad4a20a0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:09.590 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.580+0000 7f592247fb80 1 --2- 172.21.15.1:0/253192334 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573ad49bf70 0x5573ad4acde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:09.590 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.580+0000 7f592247fb80 1 --2- 172.21.15.1:0/253192334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573ad49c880 0x5573ad4a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:09.590 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.580+0000 7f592247fb80 1 -- 172.21.15.1:0/253192334 >> 172.21.15.1:0/253192334 conn(0x5573ad48d360 msgr2=0x5573ad49fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:44:09.590 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.580+0000 7f592247fb80 1 -- 172.21.15.1:0/253192334 shutdown_connections 2023-11-16T16:44:09.590 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.580+0000 7f592247fb80 1 -- 172.21.15.1:0/253192334 wait complete. 2023-11-16T16:44:09.590 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.580+0000 7f592247fb80 1 -- start start 2023-11-16T16:44:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.581+0000 7f592247fb80 1 Processor -- start 2023-11-16T16:44:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.581+0000 7f592247fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573ad49bf70 0x5573ad4aca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.581+0000 7f592247fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573ad49c880 0x5573ad4acf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.581+0000 7f592247fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573ad4a0cc0 0x5573ad4a8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.581+0000 7f592247fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573ad4a9660 con 0x5573ad49c880 2023-11-16T16:44:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.581+0000 7f592247fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573ad4a97d0 con 0x5573ad49bf70 2023-11-16T16:44:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.581+0000 7f592247fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573ad4a9990 con 0x5573ad4a0cc0 2023-11-16T16:44:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.581+0000 7f592247fb80 10 client.0 _ll_register_callbacks cb 0x7f5908009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:44:09.591 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.581+0000 7f590f7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573ad49c880 0x5573ad4acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:09.592 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.581+0000 7f590f7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573ad49c880 0x5573ad4acf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45252/0 (socket says 172.21.15.1:45252) 2023-11-16T16:44:09.592 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.581+0000 7f590ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573ad49bf70 0x5573ad4aca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:09.592 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.581+0000 7f590f7fe700 1 -- 172.21.15.1:0/1713099091 learned_addr learned my addr 172.21.15.1:0/1713099091 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:44:09.592 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.581+0000 7f5914890700 1 --2- 172.21.15.1:0/1713099091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573ad4a0cc0 0x5573ad4a8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:09.592 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.581+0000 7f590f7fe700 1 -- 172.21.15.1:0/1713099091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573ad4a0cc0 msgr2=0x5573ad4a8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:44:09.592 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.581+0000 7f590f7fe700 1 --2- 172.21.15.1:0/1713099091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573ad4a0cc0 0x5573ad4a8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:09.592 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.581+0000 7f590f7fe700 1 -- 172.21.15.1:0/1713099091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573ad49bf70 msgr2=0x5573ad4aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:44:09.592 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.581+0000 7f590f7fe700 1 --2- 172.21.15.1:0/1713099091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573ad49bf70 0x5573ad4aca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:09.592 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.581+0000 7f590f7fe700 1 -- 172.21.15.1:0/1713099091 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f591010d980 con 0x5573ad49c880 2023-11-16T16:44:09.593 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.582+0000 7f590f7fe700 1 --2- 172.21.15.1:0/1713099091 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573ad49c880 0x5573ad4acf80 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f59040058e0 tx=0x7f5904009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a6be40f38e5f2dd server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:44:09.593 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.582+0000 7f58ef7fe700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:44:09.593 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.582+0000 7f58ef7fe700 1 -- 172.21.15.1:0/1713099091 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590402c040 con 0x5573ad49c880 2023-11-16T16:44:09.593 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.582+0000 7f58ef7fe700 1 -- 172.21.15.1:0/1713099091 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f5904033a30 con 0x5573ad49c880 2023-11-16T16:44:09.593 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.582+0000 7f592247fb80 1 -- 172.21.15.1:0/1713099091 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f59000109e0 con 0x5573ad49c880 2023-11-16T16:44:09.593 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.582+0000 7f58ef7fe700 1 -- 172.21.15.1:0/1713099091 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5904033bf0 con 0x5573ad49c880 2023-11-16T16:44:09.593 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.582+0000 7f592247fb80 10 client.16161 fetch_fsmap learned FSMap version 28 2023-11-16T16:44:09.593 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.582+0000 7f592247fb80 1 -- 172.21.15.1:0/1713099091 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f59080115d0 con 0x5573ad49c880 2023-11-16T16:44:09.593 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.582+0000 7f58ef7fe700 1 -- 172.21.15.1:0/1713099091 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59000109e0 con 0x5573ad49c880 2023-11-16T16:44:09.594 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.582+0000 7f58ef7fe700 1 -- 172.21.15.1:0/1713099091 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f59040458b0 con 0x5573ad49c880 2023-11-16T16:44:09.594 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.582+0000 7f592247fb80 10 client.16161 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:44:09.594 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.582+0000 7f592247fb80 10 client.16161 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:44:09.594 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.582+0000 7f592247fb80 20 client.16161 populate_metadata read hostname 'smithi001' 2023-11-16T16:44:09.594 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.582+0000 7f592247fb80 1 --2- 172.21.15.1:0/1713099091 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5573ad50aa80 0x5573ad50ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:09.594 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.582+0000 7f592247fb80 4 client.16161 mds_command: new command op to 24521 tid=0[{"prefix": "scrub status"}] 2023-11-16T16:44:09.594 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.582+0000 7f592247fb80 1 -- 172.21.15.1:0/1713099091 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5573ad3b7660 con 0x5573ad50aa80 2023-11-16T16:44:09.594 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.583+0000 7f590ffff700 1 --2- 172.21.15.1:0/1713099091 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5573ad50aa80 0x5573ad50ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:09.594 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.583+0000 7f58ef7fe700 10 client.16161 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:44:09.595 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.583+0000 7f590ffff700 1 --2- 172.21.15.1:0/1713099091 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5573ad50aa80 0x5573ad50ce90 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d2d5a28043fdd53 server_cookie=bb41607895469b3f in_seq=0 out_seq=0 2023-11-16T16:44:09.595 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.583+0000 7f58ef7fe700 1 -- 172.21.15.1:0/1713099091 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5573ad3b7660 con 0x5573ad50aa80 2023-11-16T16:44:09.595 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.583+0000 7f58ef7fe700 10 client.16161 handle_command_reply: tid=0 2023-11-16T16:44:09.595 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.583+0000 7f592247fb80 1 client.16161 shutdown 2023-11-16T16:44:09.595 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.583+0000 7f592247fb80 1 -- 172.21.15.1:0/1713099091 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573ad49c880 msgr2=0x5573ad4acf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:44:09.595 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.583+0000 7f592247fb80 1 --2- 172.21.15.1:0/1713099091 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573ad49c880 0x5573ad4acf80 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f59040058e0 tx=0x7f5904009310 comp rx=0 tx=0).stop 2023-11-16T16:44:09.595 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.583+0000 7f592247fb80 1 -- 172.21.15.1:0/1713099091 shutdown_connections 2023-11-16T16:44:09.595 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.583+0000 7f592247fb80 1 --2- 172.21.15.1:0/1713099091 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5573ad50aa80 0x5573ad50ce90 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:09.595 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.583+0000 7f592247fb80 1 --2- 172.21.15.1:0/1713099091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5573ad4a0cc0 0x5573ad4a8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:09.595 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.583+0000 7f58ef7fe700 0 client.16161 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:44:09.596 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.583+0000 7f592247fb80 1 --2- 172.21.15.1:0/1713099091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573ad49bf70 0x5573ad4aca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:09.596 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.583+0000 7f592247fb80 1 --2- 172.21.15.1:0/1713099091 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5573ad49c880 0x5573ad4acf80 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:09.596 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.583+0000 7f592247fb80 1 -- 172.21.15.1:0/1713099091 >> 172.21.15.1:0/1713099091 conn(0x5573ad48d360 msgr2=0x5573ad4a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:44:09.596 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.583+0000 7f592247fb80 1 -- 172.21.15.1:0/1713099091 shutdown_connections 2023-11-16T16:44:09.596 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.583+0000 7f592247fb80 1 -- 172.21.15.1:0/1713099091 wait complete. 2023-11-16T16:44:09.596 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.584+0000 7f592247fb80 20 client.16161 trim_cache size 0 max 16384 2023-11-16T16:44:09.596 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:44:09.597 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "scrub active (1 inodes in the stack)", 2023-11-16T16:44:09.597 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2023-11-16T16:44:09.597 INFO:teuthology.orchestra.run.smithi001.stdout: "4e68c117-e4ae-40d9-847d-0d6d5d2b1317": { 2023-11-16T16:44:09.597 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2023-11-16T16:44:09.597 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "4e68c117-e4ae-40d9-847d-0d6d5d2b1317", 2023-11-16T16:44:09.597 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2023-11-16T16:44:09.597 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:44:09.597 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:44:09.597 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:44:09.599 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.584+0000 7f59162f3700 1 -- 172.21.15.1:0/3594568652 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f58f80a12c0 msgr2=0x7f58f80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:44:09.599 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.584+0000 7f59162f3700 1 --2- 172.21.15.1:0/3594568652 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f58f80a12c0 0x7f58f80a3770 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f590400b580 tx=0x7f5904003680 comp rx=0 tx=0).stop 2023-11-16T16:44:09.600 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.584+0000 7f59162f3700 1 -- 172.21.15.1:0/3594568652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5910100cc0 msgr2=0x7f5910196490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:44:09.600 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.584+0000 7f59162f3700 1 --2- 172.21.15.1:0/3594568652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5910100cc0 0x7f5910196490 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f590000e9a0 tx=0x7f590000ecb0 comp rx=0 tx=0).stop 2023-11-16T16:44:09.600 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.584+0000 7f59162f3700 1 -- 172.21.15.1:0/3594568652 shutdown_connections 2023-11-16T16:44:09.600 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.584+0000 7f59162f3700 1 --2- 172.21.15.1:0/3594568652 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f58f80a12c0 0x7f58f80a3770 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:09.600 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.584+0000 7f59162f3700 1 --2- 172.21.15.1:0/3594568652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59101055c0 0x7f591019ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:09.600 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.584+0000 7f59162f3700 1 --2- 172.21.15.1:0/3594568652 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5910100cc0 0x7f5910196490 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:09.600 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.584+0000 7f59162f3700 1 --2- 172.21.15.1:0/3594568652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5910101660 0x7f59101969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:09.600 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.584+0000 7f59162f3700 1 -- 172.21.15.1:0/3594568652 >> 172.21.15.1:0/3594568652 conn(0x7f5910077560 msgr2=0x7f59100ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:44:09.600 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.584+0000 7f59162f3700 1 -- 172.21.15.1:0/3594568652 shutdown_connections 2023-11-16T16:44:09.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:09.584+0000 7f59162f3700 1 -- 172.21.15.1:0/3594568652 wait complete. 2023-11-16T16:44:09.601 INFO:tasks.cephfs.filesystem:scrub status for tag:4e68c117-e4ae-40d9-847d-0d6d5d2b1317 - {'path': '/', 'tag': '4e68c117-e4ae-40d9-847d-0d6d5d2b1317', 'options': 'recursive,force'} 2023-11-16T16:44:10.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:10 smithi001 ceph-mon[98800]: pgmap v153: 97 pgs: 97 active+clean; 539 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 71 op/s 2023-11-16T16:44:11.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:10 smithi026 ceph-mon[100112]: pgmap v153: 97 pgs: 97 active+clean; 539 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 71 op/s 2023-11-16T16:44:11.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:10 smithi053 ceph-mon[100638]: pgmap v153: 97 pgs: 97 active+clean; 539 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 71 op/s 2023-11-16T16:44:12.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:12 smithi053 ceph-mon[100638]: pgmap v154: 97 pgs: 97 active+clean; 557 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.2 MiB/s wr, 93 op/s 2023-11-16T16:44:12.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:12 smithi001 ceph-mon[98800]: pgmap v154: 97 pgs: 97 active+clean; 557 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.2 MiB/s wr, 93 op/s 2023-11-16T16:44:12.757 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:12 smithi026 ceph-mon[100112]: pgmap v154: 97 pgs: 97 active+clean; 557 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.2 MiB/s wr, 93 op/s 2023-11-16T16:44:14.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:14 smithi001 ceph-mon[98800]: pgmap v155: 97 pgs: 97 active+clean; 557 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.7 MiB/s wr, 55 op/s 2023-11-16T16:44:15.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:14 smithi026 ceph-mon[100112]: pgmap v155: 97 pgs: 97 active+clean; 557 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.7 MiB/s wr, 55 op/s 2023-11-16T16:44:15.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:14 smithi053 ceph-mon[100638]: pgmap v155: 97 pgs: 97 active+clean; 557 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.7 MiB/s wr, 55 op/s 2023-11-16T16:44:16.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:16 smithi001 ceph-mon[98800]: pgmap v156: 97 pgs: 97 active+clean; 576 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.3 MiB/s wr, 84 op/s 2023-11-16T16:44:17.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:16 smithi026 ceph-mon[100112]: pgmap v156: 97 pgs: 97 active+clean; 576 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.3 MiB/s wr, 84 op/s 2023-11-16T16:44:17.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:16 smithi053 ceph-mon[100638]: pgmap v156: 97 pgs: 97 active+clean; 576 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.3 MiB/s wr, 84 op/s 2023-11-16T16:44:18.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:18 smithi001 ceph-mon[98800]: pgmap v157: 97 pgs: 97 active+clean; 576 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 50 op/s 2023-11-16T16:44:19.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:18 smithi026 ceph-mon[100112]: pgmap v157: 97 pgs: 97 active+clean; 576 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 50 op/s 2023-11-16T16:44:19.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:18 smithi053 ceph-mon[100638]: pgmap v157: 97 pgs: 97 active+clean; 576 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 50 op/s 2023-11-16T16:44:19.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:19 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:44:20.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:19 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:44:20.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:19 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:44:20.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:20 smithi001 ceph-mon[98800]: pgmap v158: 97 pgs: 97 active+clean; 576 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 50 op/s 2023-11-16T16:44:21.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:20 smithi026 ceph-mon[100112]: pgmap v158: 97 pgs: 97 active+clean; 576 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 50 op/s 2023-11-16T16:44:21.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:20 smithi053 ceph-mon[100638]: pgmap v158: 97 pgs: 97 active+clean; 576 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 50 op/s 2023-11-16T16:44:22.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:22 smithi053 ceph-mon[100638]: pgmap v159: 97 pgs: 97 active+clean; 597 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.9 MiB/s wr, 85 op/s 2023-11-16T16:44:22.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:22 smithi001 ceph-mon[98800]: pgmap v159: 97 pgs: 97 active+clean; 597 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.9 MiB/s wr, 85 op/s 2023-11-16T16:44:22.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:22 smithi026 ceph-mon[100112]: pgmap v159: 97 pgs: 97 active+clean; 597 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.9 MiB/s wr, 85 op/s 2023-11-16T16:44:24.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:24 smithi001 ceph-mon[98800]: pgmap v160: 97 pgs: 97 active+clean; 597 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 63 op/s 2023-11-16T16:44:25.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:24 smithi026 ceph-mon[100112]: pgmap v160: 97 pgs: 97 active+clean; 597 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 63 op/s 2023-11-16T16:44:25.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:24 smithi053 ceph-mon[100638]: pgmap v160: 97 pgs: 97 active+clean; 597 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 63 op/s 2023-11-16T16:44:26.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:26 smithi001 ceph-mon[98800]: pgmap v161: 97 pgs: 97 active+clean; 616 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.0 MiB/s wr, 81 op/s 2023-11-16T16:44:27.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:26 smithi026 ceph-mon[100112]: pgmap v161: 97 pgs: 97 active+clean; 616 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.0 MiB/s wr, 81 op/s 2023-11-16T16:44:27.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:26 smithi053 ceph-mon[100638]: pgmap v161: 97 pgs: 97 active+clean; 616 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.0 MiB/s wr, 81 op/s 2023-11-16T16:44:28.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:28 smithi001 ceph-mon[98800]: pgmap v162: 97 pgs: 97 active+clean; 616 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 52 op/s 2023-11-16T16:44:29.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:28 smithi026 ceph-mon[100112]: pgmap v162: 97 pgs: 97 active+clean; 616 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 52 op/s 2023-11-16T16:44:29.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:28 smithi053 ceph-mon[100638]: pgmap v162: 97 pgs: 97 active+clean; 616 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 52 op/s 2023-11-16T16:44:29.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:29 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:44:30.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:29 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:44:30.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:29 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:44:30.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:30 smithi001 ceph-mon[98800]: pgmap v163: 97 pgs: 97 active+clean; 616 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 52 op/s 2023-11-16T16:44:30.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:30 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:44:30.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:30 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:44:30.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:30 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:44:31.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:30 smithi026 ceph-mon[100112]: pgmap v163: 97 pgs: 97 active+clean; 616 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 52 op/s 2023-11-16T16:44:31.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:30 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:44:31.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:30 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:44:31.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:30 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:44:31.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:30 smithi053 ceph-mon[100638]: pgmap v163: 97 pgs: 97 active+clean; 616 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 52 op/s 2023-11-16T16:44:31.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:30 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:44:31.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:30 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:44:31.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:30 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:44:31.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:31 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:44:31.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:31 smithi001 ceph-mon[98800]: Service mds.all has no related daemon already placed. 2023-11-16T16:44:31.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:31 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:44:31.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:31 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:44:32.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:31 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:44:32.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:31 smithi026 ceph-mon[100112]: Service mds.all has no related daemon already placed. 2023-11-16T16:44:32.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:31 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:44:32.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:31 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:44:32.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:31 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:44:32.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:31 smithi053 ceph-mon[100638]: Service mds.all has no related daemon already placed. 2023-11-16T16:44:32.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:31 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:44:32.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:31 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:44:32.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:32 smithi001 ceph-mon[98800]: pgmap v164: 97 pgs: 97 active+clean; 634 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.9 MiB/s wr, 82 op/s 2023-11-16T16:44:33.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:32 smithi026 ceph-mon[100112]: pgmap v164: 97 pgs: 97 active+clean; 634 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.9 MiB/s wr, 82 op/s 2023-11-16T16:44:33.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:32 smithi053 ceph-mon[100638]: pgmap v164: 97 pgs: 97 active+clean; 634 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.9 MiB/s wr, 82 op/s 2023-11-16T16:44:33.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:33 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:44:33.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:33 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:44:34.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:33 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:44:34.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:33 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:44:34.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:33 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:44:34.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:33 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:44:34.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:34 smithi001 ceph-mon[98800]: pgmap v165: 97 pgs: 97 active+clean; 634 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 47 op/s 2023-11-16T16:44:35.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:34 smithi026 ceph-mon[100112]: pgmap v165: 97 pgs: 97 active+clean; 634 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 47 op/s 2023-11-16T16:44:35.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:34 smithi053 ceph-mon[100638]: pgmap v165: 97 pgs: 97 active+clean; 634 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 47 op/s 2023-11-16T16:44:35.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:35 smithi001 ceph-mon[98800]: pgmap v166: 97 pgs: 97 active+clean; 653 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.6 MiB/s wr, 470 op/s 2023-11-16T16:44:36.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:35 smithi026 ceph-mon[100112]: pgmap v166: 97 pgs: 97 active+clean; 653 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.6 MiB/s wr, 470 op/s 2023-11-16T16:44:36.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:35 smithi053 ceph-mon[100638]: pgmap v166: 97 pgs: 97 active+clean; 653 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.6 MiB/s wr, 470 op/s 2023-11-16T16:44:38.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:38 smithi001 ceph-mon[98800]: pgmap v167: 97 pgs: 97 active+clean; 653 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 452 op/s 2023-11-16T16:44:39.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:38 smithi026 ceph-mon[100112]: pgmap v167: 97 pgs: 97 active+clean; 653 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 452 op/s 2023-11-16T16:44:39.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:38 smithi053 ceph-mon[100638]: pgmap v167: 97 pgs: 97 active+clean; 653 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 452 op/s 2023-11-16T16:44:39.602 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-16T16:44:39.877 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.874+0000 7fcebed50700 1 -- 172.21.15.1:0/1165865669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fceb8104eb0 msgr2=0x7fceb8105320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:44:39.877 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.874+0000 7fcebed50700 1 --2- 172.21.15.1:0/1165865669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fceb8104eb0 0x7fceb8105320 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fceac009990 tx=0x7fceac009ca0 comp rx=0 tx=0).stop 2023-11-16T16:44:39.877 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.874+0000 7fcebed50700 1 -- 172.21.15.1:0/1165865669 shutdown_connections 2023-11-16T16:44:39.877 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.874+0000 7fcebed50700 1 --2- 172.21.15.1:0/1165865669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fceb8105860 0x7fceb810c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:39.877 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.874+0000 7fcebed50700 1 --2- 172.21.15.1:0/1165865669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fceb81041e0 0x7fceb81045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:39.878 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.874+0000 7fcebed50700 1 --2- 172.21.15.1:0/1165865669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fceb8104eb0 0x7fceb8105320 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:39.878 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.874+0000 7fcebed50700 1 -- 172.21.15.1:0/1165865669 >> 172.21.15.1:0/1165865669 conn(0x7fceb8075210 msgr2=0x7fceb8075610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:44:39.878 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.874+0000 7fcebed50700 1 -- 172.21.15.1:0/1165865669 shutdown_connections 2023-11-16T16:44:39.878 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.874+0000 7fcebed50700 1 -- 172.21.15.1:0/1165865669 wait complete. 2023-11-16T16:44:39.878 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.875+0000 7fcebed50700 1 Processor -- start 2023-11-16T16:44:39.878 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.875+0000 7fcebed50700 1 -- start start 2023-11-16T16:44:39.878 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.875+0000 7fcebed50700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fceb81041e0 0x7fceb8196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:39.878 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.875+0000 7fcebed50700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fceb8104eb0 0x7fceb8196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:39.879 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.875+0000 7fcebed50700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fceb8105860 0x7fceb819acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:39.879 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.875+0000 7fcebed50700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb819b310 con 0x7fceb8104eb0 2023-11-16T16:44:39.879 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.875+0000 7fcebed50700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb819b480 con 0x7fceb8105860 2023-11-16T16:44:39.879 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.875+0000 7fcebed50700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb819b640 con 0x7fceb81041e0 2023-11-16T16:44:39.879 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.875+0000 7fceb7fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fceb8104eb0 0x7fceb8196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:39.879 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.875+0000 7fceb7fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fceb8104eb0 0x7fceb8196870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45266/0 (socket says 172.21.15.1:45266) 2023-11-16T16:44:39.879 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.875+0000 7fcebd2ed700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fceb8105860 0x7fceb819acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:39.879 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.875+0000 7fcebcaec700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fceb81041e0 0x7fceb8196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:39.880 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.875+0000 7fceb7fff700 1 -- 172.21.15.1:0/1991234949 learned_addr learned my addr 172.21.15.1:0/1991234949 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:44:39.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.875+0000 7fceb7fff700 1 -- 172.21.15.1:0/1991234949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fceb81041e0 msgr2=0x7fceb8196330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:44:39.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.876+0000 7fceb7fff700 1 --2- 172.21.15.1:0/1991234949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fceb81041e0 0x7fceb8196330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:39.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.876+0000 7fceb7fff700 1 -- 172.21.15.1:0/1991234949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fceb8105860 msgr2=0x7fceb819acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:44:39.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.876+0000 7fceb7fff700 1 --2- 172.21.15.1:0/1991234949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fceb8105860 0x7fceb819acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:39.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.876+0000 7fceb7fff700 1 -- 172.21.15.1:0/1991234949 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceac0095f0 con 0x7fceb8104eb0 2023-11-16T16:44:39.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.876+0000 7fcebcaec700 1 --2- 172.21.15.1:0/1991234949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fceb81041e0 0x7fceb8196330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:44:39.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.876+0000 7fceb7fff700 1 --2- 172.21.15.1:0/1991234949 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fceb8104eb0 0x7fceb8196870 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fceac000c00 tx=0x7fceac004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:44:39.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.876+0000 7fceb5ffb700 1 -- 172.21.15.1:0/1991234949 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceac01d070 con 0x7fceb8104eb0 2023-11-16T16:44:39.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.876+0000 7fceb5ffb700 1 -- 172.21.15.1:0/1991234949 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fceac021410 con 0x7fceb8104eb0 2023-11-16T16:44:39.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.876+0000 7fcebed50700 1 -- 172.21.15.1:0/1991234949 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fceb810f260 con 0x7fceb8104eb0 2023-11-16T16:44:39.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.876+0000 7fceb5ffb700 1 -- 172.21.15.1:0/1991234949 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceac017650 con 0x7fceb8104eb0 2023-11-16T16:44:39.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.876+0000 7fcebed50700 1 -- 172.21.15.1:0/1991234949 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fceb810f0d0 con 0x7fceb8104eb0 2023-11-16T16:44:39.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.879+0000 7fceb5ffb700 1 -- 172.21.15.1:0/1991234949 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fceac021580 con 0x7fceb8104eb0 2023-11-16T16:44:39.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.879+0000 7fceb5ffb700 1 --2- 172.21.15.1:0/1991234949 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fcea00a12c0 0x7fcea00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:39.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.879+0000 7fceb5ffb700 1 -- 172.21.15.1:0/1991234949 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fceac0cdf90 con 0x7fceb8104eb0 2023-11-16T16:44:39.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.879+0000 7fcebcaec700 1 --2- 172.21.15.1:0/1991234949 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fcea00a12c0 0x7fcea00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:39.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.880+0000 7fcebcaec700 1 --2- 172.21.15.1:0/1991234949 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fcea00a12c0 0x7fcea00a3770 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fceb0009480 tx=0x7fceb000b040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:44:39.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.880+0000 7fcecaedcb80 1 Processor -- start 2023-11-16T16:44:39.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.880+0000 7fcecaedcb80 1 -- start start 2023-11-16T16:44:39.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.881+0000 7fcecaedcb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7c054eb60 0x55a7c054ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:39.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.881+0000 7fcecaedcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7c054f470 0x55a7c0553370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:39.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.881+0000 7fcecaedcb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c05538b0 0x55a7c0553d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:39.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.881+0000 7fcecaedcb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c0454060 con 0x55a7c054eb60 2023-11-16T16:44:39.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.881+0000 7fcecaedcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c04a68a0 con 0x55a7c054f470 2023-11-16T16:44:39.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.881+0000 7fcecaedcb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c049fd30 con 0x55a7c05538b0 2023-11-16T16:44:39.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.881+0000 7fcebcaec700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7c054eb60 0x55a7c054ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:39.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.881+0000 7fceb7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7c054f470 0x55a7c0553370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:39.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.881+0000 7fcebcaec700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7c054eb60 0x55a7c054ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45272/0 (socket says 172.21.15.1:45272) 2023-11-16T16:44:39.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.881+0000 7fcebcaec700 1 -- 172.21.15.1:0/471742770 learned_addr learned my addr 172.21.15.1:0/471742770 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:44:39.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.881+0000 7fcebd2ed700 1 --2- 172.21.15.1:0/471742770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c05538b0 0x55a7c0553d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:39.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.881+0000 7fcebcaec700 1 -- 172.21.15.1:0/471742770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c05538b0 msgr2=0x55a7c0553d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:44:39.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.881+0000 7fcebcaec700 1 --2- 172.21.15.1:0/471742770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c05538b0 0x55a7c0553d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:39.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.881+0000 7fcebcaec700 1 -- 172.21.15.1:0/471742770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7c054f470 msgr2=0x55a7c0553370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:44:39.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.881+0000 7fcebcaec700 1 --2- 172.21.15.1:0/471742770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7c054f470 0x55a7c0553370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:39.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.881+0000 7fcebcaec700 1 -- 172.21.15.1:0/471742770 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7c0321630 con 0x55a7c054eb60 2023-11-16T16:44:39.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.881+0000 7fcebcaec700 1 --2- 172.21.15.1:0/471742770 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7c054eb60 0x55a7c054ef30 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fceb000e960 tx=0x7fceb0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd5c01fa519f97ee server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:44:39.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.881+0000 7fceb7fff700 1 --2- 172.21.15.1:0/471742770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7c054f470 0x55a7c0553370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:44:39.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.882+0000 7fce9effd700 1 -- 172.21.15.1:0/471742770 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb0025070 con 0x55a7c054eb60 2023-11-16T16:44:39.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.882+0000 7fce9effd700 1 -- 172.21.15.1:0/471742770 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fceb000f040 con 0x55a7c054eb60 2023-11-16T16:44:39.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.882+0000 7fce9effd700 1 -- 172.21.15.1:0/471742770 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb0021ac0 con 0x55a7c054eb60 2023-11-16T16:44:39.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.882+0000 7fcecaedcb80 1 -- 172.21.15.1:0/471742770 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7c054eb60 msgr2=0x55a7c054ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:44:39.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.882+0000 7fcecaedcb80 1 --2- 172.21.15.1:0/471742770 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7c054eb60 0x55a7c054ef30 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fceb000e960 tx=0x7fceb0014000 comp rx=0 tx=0).stop 2023-11-16T16:44:39.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.883+0000 7fcecaedcb80 1 -- 172.21.15.1:0/471742770 shutdown_connections 2023-11-16T16:44:39.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.883+0000 7fcecaedcb80 1 --2- 172.21.15.1:0/471742770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c05538b0 0x55a7c0553d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:39.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.883+0000 7fcecaedcb80 1 --2- 172.21.15.1:0/471742770 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7c054eb60 0x55a7c054ef30 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:39.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.883+0000 7fcecaedcb80 1 --2- 172.21.15.1:0/471742770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7c054f470 0x55a7c0553370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:39.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.883+0000 7fcecaedcb80 1 -- 172.21.15.1:0/471742770 >> 172.21.15.1:0/471742770 conn(0x55a7c0544360 msgr2=0x55a7c0549b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:44:39.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.883+0000 7fcecaedcb80 1 -- 172.21.15.1:0/471742770 shutdown_connections 2023-11-16T16:44:39.891 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.883+0000 7fcecaedcb80 1 -- 172.21.15.1:0/471742770 wait complete. 2023-11-16T16:44:39.891 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.883+0000 7fcecaedcb80 1 -- start start 2023-11-16T16:44:39.891 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.883+0000 7fcecaedcb80 1 Processor -- start 2023-11-16T16:44:39.895 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.884+0000 7fcecaedcb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c054eb60 0x55a7c055b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:39.896 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.884+0000 7fcecaedcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7c054f470 0x55a7c0558330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:39.896 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.884+0000 7fcecaedcb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7c05538b0 0x55a7c05589b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:39.896 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.884+0000 7fcecaedcb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c04418e0 con 0x55a7c05538b0 2023-11-16T16:44:39.896 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.884+0000 7fcecaedcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c0441580 con 0x55a7c054f470 2023-11-16T16:44:39.896 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.884+0000 7fcecaedcb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c0323920 con 0x55a7c054eb60 2023-11-16T16:44:39.896 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.884+0000 7fcecaedcb80 10 client.0 _ll_register_callbacks cb 0x55a7c045f780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:44:39.896 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.884+0000 7fcebd2ed700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7c05538b0 0x55a7c05589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:39.896 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.884+0000 7fcebd2ed700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7c05538b0 0x55a7c05589b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45282/0 (socket says 172.21.15.1:45282) 2023-11-16T16:44:39.896 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.884+0000 7fcebd2ed700 1 -- 172.21.15.1:0/2449318810 learned_addr learned my addr 172.21.15.1:0/2449318810 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:44:39.897 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.884+0000 7fceb7fff700 1 --2- 172.21.15.1:0/2449318810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7c054f470 0x55a7c0558330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:39.897 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.884+0000 7fcebcaec700 1 --2- 172.21.15.1:0/2449318810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c054eb60 0x55a7c055b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:39.897 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.884+0000 7fcebd2ed700 1 -- 172.21.15.1:0/2449318810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c054eb60 msgr2=0x55a7c055b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:44:39.897 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.884+0000 7fcebd2ed700 1 --2- 172.21.15.1:0/2449318810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c054eb60 0x55a7c055b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:39.897 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.884+0000 7fcebd2ed700 1 -- 172.21.15.1:0/2449318810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7c054f470 msgr2=0x55a7c0558330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:44:39.897 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.884+0000 7fcebd2ed700 1 --2- 172.21.15.1:0/2449318810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7c054f470 0x55a7c0558330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:39.897 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.884+0000 7fcebd2ed700 1 -- 172.21.15.1:0/2449318810 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fceb000e640 con 0x55a7c05538b0 2023-11-16T16:44:39.897 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.884+0000 7fceb7fff700 1 --2- 172.21.15.1:0/2449318810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7c054f470 0x55a7c0558330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:44:39.897 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.885+0000 7fcebd2ed700 1 --2- 172.21.15.1:0/2449318810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7c05538b0 0x55a7c05589b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fcea80127f0 tx=0x7fcea8012b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec0c433a820cb3c server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:44:39.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.885+0000 7fce97fff700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:44:39.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.885+0000 7fce97fff700 1 -- 172.21.15.1:0/2449318810 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea8017070 con 0x55a7c05538b0 2023-11-16T16:44:39.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.885+0000 7fcecaedcb80 1 -- 172.21.15.1:0/2449318810 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a7c0541b20 con 0x55a7c05538b0 2023-11-16T16:44:39.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.885+0000 7fce97fff700 1 -- 172.21.15.1:0/2449318810 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fcea8004020 con 0x55a7c05538b0 2023-11-16T16:44:39.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.885+0000 7fce97fff700 1 -- 172.21.15.1:0/2449318810 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcea80041e0 con 0x55a7c05538b0 2023-11-16T16:44:39.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.885+0000 7fcecaedcb80 10 client.16173 fetch_fsmap learned FSMap version 28 2023-11-16T16:44:39.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.885+0000 7fcecaedcb80 1 -- 172.21.15.1:0/2449318810 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a7c0466190 con 0x55a7c05538b0 2023-11-16T16:44:39.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.885+0000 7fce97fff700 1 -- 172.21.15.1:0/2449318810 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a7c0541b20 con 0x55a7c05538b0 2023-11-16T16:44:39.899 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.885+0000 7fce97fff700 1 -- 172.21.15.1:0/2449318810 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fcea8023070 con 0x55a7c05538b0 2023-11-16T16:44:39.899 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.885+0000 7fcecaedcb80 10 client.16173 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:44:39.899 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.885+0000 7fcecaedcb80 10 client.16173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:44:39.899 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.885+0000 7fcecaedcb80 20 client.16173 populate_metadata read hostname 'smithi001' 2023-11-16T16:44:39.899 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.885+0000 7fcecaedcb80 1 --2- 172.21.15.1:0/2449318810 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55a7c0568950 0x55a7c056ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:39.899 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.885+0000 7fcecaedcb80 4 client.16173 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:44:39.899 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.885+0000 7fcecaedcb80 1 -- 172.21.15.1:0/2449318810 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a7c04620a0 con 0x55a7c0568950 2023-11-16T16:44:39.899 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.886+0000 7fcebcaec700 1 --2- 172.21.15.1:0/2449318810 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55a7c0568950 0x55a7c056ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:39.899 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.886+0000 7fce97fff700 10 client.16173 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:44:39.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.886+0000 7fcebcaec700 1 --2- 172.21.15.1:0/2449318810 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55a7c0568950 0x55a7c056ad60 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d0f41ca9514f9ad server_cookie=ce1782a29aa84e5c in_seq=0 out_seq=0 2023-11-16T16:44:39.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.889+0000 7fce97fff700 1 -- 172.21.15.1:0/2449318810 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a7c04620a0 con 0x55a7c0568950 2023-11-16T16:44:39.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.889+0000 7fce97fff700 10 client.16173 handle_command_reply: tid=0 2023-11-16T16:44:39.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.889+0000 7fcecaedcb80 1 client.16173 shutdown 2023-11-16T16:44:39.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.889+0000 7fcecaedcb80 1 -- 172.21.15.1:0/2449318810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7c05538b0 msgr2=0x55a7c05589b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:44:39.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.889+0000 7fcecaedcb80 1 --2- 172.21.15.1:0/2449318810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7c05538b0 0x55a7c05589b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fcea80127f0 tx=0x7fcea8012b00 comp rx=0 tx=0).stop 2023-11-16T16:44:39.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.889+0000 7fcecaedcb80 1 -- 172.21.15.1:0/2449318810 shutdown_connections 2023-11-16T16:44:39.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.889+0000 7fcecaedcb80 1 --2- 172.21.15.1:0/2449318810 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55a7c0568950 0x55a7c056ad60 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:39.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.889+0000 7fcecaedcb80 1 --2- 172.21.15.1:0/2449318810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7c05538b0 0x55a7c05589b0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:39.901 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.889+0000 7fcecaedcb80 1 --2- 172.21.15.1:0/2449318810 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c054eb60 0x55a7c055b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:39.901 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.889+0000 7fce97fff700 0 client.16173 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:44:39.901 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.889+0000 7fcecaedcb80 1 --2- 172.21.15.1:0/2449318810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7c054f470 0x55a7c0558330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:39.901 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.890+0000 7fcecaedcb80 1 -- 172.21.15.1:0/2449318810 >> 172.21.15.1:0/2449318810 conn(0x55a7c0544360 msgr2=0x55a7c0554d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:44:39.901 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.890+0000 7fcecaedcb80 1 -- 172.21.15.1:0/2449318810 shutdown_connections 2023-11-16T16:44:39.901 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.890+0000 7fcecaedcb80 1 -- 172.21.15.1:0/2449318810 wait complete. 2023-11-16T16:44:39.901 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.890+0000 7fcecaedcb80 20 client.16173 trim_cache size 0 max 16384 2023-11-16T16:44:39.912 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.908+0000 7fcecaedcb80 1 Processor -- start 2023-11-16T16:44:39.912 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.908+0000 7fcecaedcb80 1 -- start start 2023-11-16T16:44:39.912 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.908+0000 7fcecaedcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7c054eb60 0x55a7c055f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:39.912 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.908+0000 7fcecaedcb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7c054f470 0x55a7c05519d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:39.912 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.908+0000 7fcecaedcb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c05538b0 0x55a7c0551f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:39.913 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.908+0000 7fcecaedcb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c0558d40 con 0x55a7c054f470 2023-11-16T16:44:39.913 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.908+0000 7fcecaedcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c0556890 con 0x55a7c054eb60 2023-11-16T16:44:39.913 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.908+0000 7fcecaedcb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c0552500 con 0x55a7c05538b0 2023-11-16T16:44:39.913 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.908+0000 7fceb7fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7c054f470 0x55a7c05519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:39.913 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.908+0000 7fceb7fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7c054f470 0x55a7c05519d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45288/0 (socket says 172.21.15.1:45288) 2023-11-16T16:44:39.913 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.908+0000 7fceb7fff700 1 -- 172.21.15.1:0/21326367 learned_addr learned my addr 172.21.15.1:0/21326367 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:44:39.913 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.908+0000 7fcebd2ed700 1 --2- 172.21.15.1:0/21326367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c05538b0 0x55a7c0551f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:39.913 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.908+0000 7fcebcaec700 1 --2- 172.21.15.1:0/21326367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7c054eb60 0x55a7c055f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:39.913 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.908+0000 7fceb7fff700 1 -- 172.21.15.1:0/21326367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c05538b0 msgr2=0x55a7c0551f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:44:39.914 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.908+0000 7fceb7fff700 1 --2- 172.21.15.1:0/21326367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c05538b0 0x55a7c0551f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:39.914 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.908+0000 7fceb7fff700 1 -- 172.21.15.1:0/21326367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7c054eb60 msgr2=0x55a7c055f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:44:39.914 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.908+0000 7fceb7fff700 1 --2- 172.21.15.1:0/21326367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7c054eb60 0x55a7c055f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:39.914 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.908+0000 7fceb7fff700 1 -- 172.21.15.1:0/21326367 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7c0465ee0 con 0x55a7c054f470 2023-11-16T16:44:39.914 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.909+0000 7fceb7fff700 1 --2- 172.21.15.1:0/21326367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7c054f470 0x55a7c05519d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fceac005910 tx=0x7fceac009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58dc109dcc1f096b server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:44:39.914 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.909+0000 7fce9f7fe700 1 -- 172.21.15.1:0/21326367 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceac09b100 con 0x55a7c054f470 2023-11-16T16:44:39.914 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.909+0000 7fce9f7fe700 1 -- 172.21.15.1:0/21326367 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fceac09f130 con 0x55a7c054f470 2023-11-16T16:44:39.914 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.909+0000 7fce9f7fe700 1 -- 172.21.15.1:0/21326367 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceac09a7f0 con 0x55a7c054f470 2023-11-16T16:44:39.914 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.910+0000 7fcecaedcb80 1 -- 172.21.15.1:0/21326367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7c054f470 msgr2=0x55a7c05519d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:44:39.915 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.910+0000 7fcecaedcb80 1 --2- 172.21.15.1:0/21326367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7c054f470 0x55a7c05519d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fceac005910 tx=0x7fceac009310 comp rx=0 tx=0).stop 2023-11-16T16:44:39.915 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.910+0000 7fcecaedcb80 1 -- 172.21.15.1:0/21326367 shutdown_connections 2023-11-16T16:44:39.915 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.910+0000 7fcecaedcb80 1 --2- 172.21.15.1:0/21326367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c05538b0 0x55a7c0551f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:39.915 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.910+0000 7fcecaedcb80 1 --2- 172.21.15.1:0/21326367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7c054eb60 0x55a7c055f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:39.915 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.910+0000 7fcecaedcb80 1 --2- 172.21.15.1:0/21326367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7c054f470 0x55a7c05519d0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:39.915 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.910+0000 7fcecaedcb80 1 -- 172.21.15.1:0/21326367 >> 172.21.15.1:0/21326367 conn(0x55a7c0544360 msgr2=0x55a7c0564be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:44:39.915 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.910+0000 7fcecaedcb80 1 -- 172.21.15.1:0/21326367 shutdown_connections 2023-11-16T16:44:39.915 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.910+0000 7fcecaedcb80 1 -- 172.21.15.1:0/21326367 wait complete. 2023-11-16T16:44:39.915 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.910+0000 7fcecaedcb80 1 -- start start 2023-11-16T16:44:39.915 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.910+0000 7fcecaedcb80 1 Processor -- start 2023-11-16T16:44:39.919 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.911+0000 7fcecaedcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7c054eb60 0x55a7c05b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:39.919 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.911+0000 7fcecaedcb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7c054f470 0x55a7c05b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:39.919 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.911+0000 7fcecaedcb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c05538b0 0x55a7c05bb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:39.919 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.911+0000 7fcecaedcb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c05b15b0 con 0x55a7c054f470 2023-11-16T16:44:39.919 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.911+0000 7fcecaedcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c05bbd60 con 0x55a7c054eb60 2023-11-16T16:44:39.919 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.911+0000 7fcecaedcb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c05bbea0 con 0x55a7c05538b0 2023-11-16T16:44:39.919 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.911+0000 7fcecaedcb80 10 client.0 _ll_register_callbacks cb 0x7fcea8009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:44:39.919 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.911+0000 7fceb7fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7c054f470 0x55a7c05b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:39.920 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.911+0000 7fceb7fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7c054f470 0x55a7c05b0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45294/0 (socket says 172.21.15.1:45294) 2023-11-16T16:44:39.920 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.911+0000 7fceb7fff700 1 -- 172.21.15.1:0/3500952213 learned_addr learned my addr 172.21.15.1:0/3500952213 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:44:39.920 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.911+0000 7fcebcaec700 1 --2- 172.21.15.1:0/3500952213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7c054eb60 0x55a7c05b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:39.920 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.911+0000 7fcebd2ed700 1 --2- 172.21.15.1:0/3500952213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c05538b0 0x55a7c05bb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:39.920 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.911+0000 7fceb7fff700 1 -- 172.21.15.1:0/3500952213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c05538b0 msgr2=0x55a7c05bb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:44:39.920 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.911+0000 7fceb7fff700 1 --2- 172.21.15.1:0/3500952213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c05538b0 0x55a7c05bb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:39.920 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.911+0000 7fceb7fff700 1 -- 172.21.15.1:0/3500952213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7c054eb60 msgr2=0x55a7c05b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:44:39.920 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.911+0000 7fceb7fff700 1 --2- 172.21.15.1:0/3500952213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7c054eb60 0x55a7c05b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:39.920 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.911+0000 7fceb7fff700 1 -- 172.21.15.1:0/3500952213 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fceb810f260 con 0x55a7c054f470 2023-11-16T16:44:39.921 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.911+0000 7fceb7fff700 1 --2- 172.21.15.1:0/3500952213 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7c054f470 0x55a7c05b0ce0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fceac0cb750 tx=0x7fceac005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8260fa8d5040e509 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:44:39.921 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.911+0000 7fce97fff700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:44:39.921 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.911+0000 7fce97fff700 1 -- 172.21.15.1:0/3500952213 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceac09b100 con 0x55a7c054f470 2023-11-16T16:44:39.921 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.912+0000 7fce97fff700 1 -- 172.21.15.1:0/3500952213 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fceac0cbe40 con 0x55a7c054f470 2023-11-16T16:44:39.921 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.912+0000 7fcecaedcb80 1 -- 172.21.15.1:0/3500952213 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fceb000f040 con 0x55a7c054f470 2023-11-16T16:44:39.921 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.912+0000 7fce97fff700 1 -- 172.21.15.1:0/3500952213 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fceac0a3030 con 0x55a7c054f470 2023-11-16T16:44:39.921 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.912+0000 7fcecaedcb80 10 client.16179 fetch_fsmap learned FSMap version 28 2023-11-16T16:44:39.921 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.912+0000 7fcecaedcb80 1 -- 172.21.15.1:0/3500952213 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a7c02b6c60 con 0x55a7c054f470 2023-11-16T16:44:39.921 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.912+0000 7fce97fff700 1 -- 172.21.15.1:0/3500952213 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fceb000f040 con 0x55a7c054f470 2023-11-16T16:44:39.922 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.912+0000 7fce97fff700 1 -- 172.21.15.1:0/3500952213 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fceac0b9180 con 0x55a7c054f470 2023-11-16T16:44:39.922 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.912+0000 7fcecaedcb80 10 client.16179 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:44:39.922 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.912+0000 7fcecaedcb80 10 client.16179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:44:39.922 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.912+0000 7fcecaedcb80 20 client.16179 populate_metadata read hostname 'smithi001' 2023-11-16T16:44:39.922 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.912+0000 7fcecaedcb80 1 --2- 172.21.15.1:0/3500952213 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55a7c05bd270 0x55a7c05bf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:44:39.922 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.912+0000 7fcecaedcb80 4 client.16179 mds_command: new command op to 24521 tid=0[{"prefix": "scrub status"}] 2023-11-16T16:44:39.922 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.912+0000 7fcecaedcb80 1 -- 172.21.15.1:0/3500952213 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a7c046e660 con 0x55a7c05bd270 2023-11-16T16:44:39.922 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.912+0000 7fcebcaec700 1 --2- 172.21.15.1:0/3500952213 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55a7c05bd270 0x55a7c05bf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:44:39.922 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.913+0000 7fcebcaec700 1 --2- 172.21.15.1:0/3500952213 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55a7c05bd270 0x55a7c05bf680 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1488eba5802a250e server_cookie=8f2c27df7b2d4382 in_seq=0 out_seq=0 2023-11-16T16:44:39.922 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.913+0000 7fce97fff700 10 client.16179 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:44:39.924 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.919+0000 7fce97fff700 1 -- 172.21.15.1:0/3500952213 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55a7c046e660 con 0x55a7c05bd270 2023-11-16T16:44:39.924 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.919+0000 7fce97fff700 10 client.16179 handle_command_reply: tid=0 2023-11-16T16:44:39.924 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.919+0000 7fcecaedcb80 1 client.16179 shutdown 2023-11-16T16:44:39.924 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.919+0000 7fcecaedcb80 1 -- 172.21.15.1:0/3500952213 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7c054f470 msgr2=0x55a7c05b0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:44:39.924 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.919+0000 7fcecaedcb80 1 --2- 172.21.15.1:0/3500952213 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7c054f470 0x55a7c05b0ce0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fceac0cb750 tx=0x7fceac005ad0 comp rx=0 tx=0).stop 2023-11-16T16:44:39.924 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.919+0000 7fcecaedcb80 1 -- 172.21.15.1:0/3500952213 shutdown_connections 2023-11-16T16:44:39.924 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.919+0000 7fcecaedcb80 1 --2- 172.21.15.1:0/3500952213 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55a7c05bd270 0x55a7c05bf680 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:39.924 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.919+0000 7fcecaedcb80 1 --2- 172.21.15.1:0/3500952213 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55a7c05538b0 0x55a7c05bb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:39.925 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.919+0000 7fcecaedcb80 1 --2- 172.21.15.1:0/3500952213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7c054eb60 0x55a7c05b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:39.925 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.919+0000 7fcecaedcb80 1 --2- 172.21.15.1:0/3500952213 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7c054f470 0x55a7c05b0ce0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:39.925 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.919+0000 7fce97fff700 0 client.16179 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:44:39.925 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.919+0000 7fcecaedcb80 1 -- 172.21.15.1:0/3500952213 >> 172.21.15.1:0/3500952213 conn(0x55a7c0544360 msgr2=0x55a7c05519d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:44:39.925 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.919+0000 7fcecaedcb80 1 -- 172.21.15.1:0/3500952213 shutdown_connections 2023-11-16T16:44:39.925 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.919+0000 7fcecaedcb80 1 -- 172.21.15.1:0/3500952213 wait complete. 2023-11-16T16:44:39.925 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.919+0000 7fcecaedcb80 20 client.16179 trim_cache size 0 max 16384 2023-11-16T16:44:39.926 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:44:39.926 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-16T16:44:39.926 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2023-11-16T16:44:39.926 INFO:teuthology.orchestra.run.smithi001.stdout: "4e68c117-e4ae-40d9-847d-0d6d5d2b1317": { 2023-11-16T16:44:39.926 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2023-11-16T16:44:39.926 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "4e68c117-e4ae-40d9-847d-0d6d5d2b1317", 2023-11-16T16:44:39.926 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2023-11-16T16:44:39.926 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:44:39.926 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:44:39.927 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:44:39.927 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.920+0000 7fcebed50700 1 -- 172.21.15.1:0/1991234949 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fcea00a12c0 msgr2=0x7fcea00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:44:39.928 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.920+0000 7fcebed50700 1 --2- 172.21.15.1:0/1991234949 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fcea00a12c0 0x7fcea00a3770 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fceb0009480 tx=0x7fceb000b040 comp rx=0 tx=0).stop 2023-11-16T16:44:39.928 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.920+0000 7fcebed50700 1 -- 172.21.15.1:0/1991234949 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fceb8104eb0 msgr2=0x7fceb8196870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:44:39.928 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.920+0000 7fcebed50700 1 --2- 172.21.15.1:0/1991234949 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fceb8104eb0 0x7fceb8196870 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fceac000c00 tx=0x7fceac004790 comp rx=0 tx=0).stop 2023-11-16T16:44:39.928 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.920+0000 7fcebed50700 1 -- 172.21.15.1:0/1991234949 shutdown_connections 2023-11-16T16:44:39.928 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.920+0000 7fcebed50700 1 --2- 172.21.15.1:0/1991234949 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fcea00a12c0 0x7fcea00a3770 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:39.928 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.920+0000 7fcebed50700 1 --2- 172.21.15.1:0/1991234949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fceb8105860 0x7fceb819acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:39.928 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.920+0000 7fcebed50700 1 --2- 172.21.15.1:0/1991234949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fceb81041e0 0x7fceb8196330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:39.928 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.920+0000 7fcebed50700 1 --2- 172.21.15.1:0/1991234949 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fceb8104eb0 0x7fceb8196870 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:44:39.929 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.920+0000 7fcebed50700 1 -- 172.21.15.1:0/1991234949 >> 172.21.15.1:0/1991234949 conn(0x7fceb8075210 msgr2=0x7fceb80fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:44:39.929 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.920+0000 7fcebed50700 1 -- 172.21.15.1:0/1991234949 shutdown_connections 2023-11-16T16:44:39.929 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:44:39.920+0000 7fcebed50700 1 -- 172.21.15.1:0/1991234949 wait complete. 2023-11-16T16:44:39.935 INFO:tasks.cephfs.filesystem:scrub status for tag:4e68c117-e4ae-40d9-847d-0d6d5d2b1317 - {'path': '/', 'tag': '4e68c117-e4ae-40d9-847d-0d6d5d2b1317', 'options': 'recursive,force'} 2023-11-16T16:44:40.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:40 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:44:40.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:40 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:44:40.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:40 smithi001 ceph-mon[98800]: pgmap v168: 97 pgs: 97 active+clean; 653 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 452 op/s 2023-11-16T16:44:41.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:40 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:44:41.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:40 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:44:41.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:40 smithi026 ceph-mon[100112]: pgmap v168: 97 pgs: 97 active+clean; 653 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 452 op/s 2023-11-16T16:44:41.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:40 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:44:41.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:40 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:44:41.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:40 smithi053 ceph-mon[100638]: pgmap v168: 97 pgs: 97 active+clean; 653 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 452 op/s 2023-11-16T16:44:42.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:42 smithi053 ceph-mon[100638]: pgmap v169: 97 pgs: 97 active+clean; 668 MiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.6 MiB/s wr, 542 op/s 2023-11-16T16:44:42.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:42 smithi001 ceph-mon[98800]: pgmap v169: 97 pgs: 97 active+clean; 668 MiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.6 MiB/s wr, 542 op/s 2023-11-16T16:44:42.757 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:42 smithi026 ceph-mon[100112]: pgmap v169: 97 pgs: 97 active+clean; 668 MiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.6 MiB/s wr, 542 op/s 2023-11-16T16:44:44.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:44 smithi001 ceph-mon[98800]: pgmap v170: 97 pgs: 97 active+clean; 668 MiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 512 op/s 2023-11-16T16:44:45.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:44 smithi026 ceph-mon[100112]: pgmap v170: 97 pgs: 97 active+clean; 668 MiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 512 op/s 2023-11-16T16:44:45.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:44 smithi053 ceph-mon[100638]: pgmap v170: 97 pgs: 97 active+clean; 668 MiB data, 817 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 512 op/s 2023-11-16T16:44:46.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:46 smithi001 ceph-mon[98800]: pgmap v171: 97 pgs: 97 active+clean; 682 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.7 MiB/s wr, 618 op/s 2023-11-16T16:44:47.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:46 smithi026 ceph-mon[100112]: pgmap v171: 97 pgs: 97 active+clean; 682 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.7 MiB/s wr, 618 op/s 2023-11-16T16:44:47.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:46 smithi053 ceph-mon[100638]: pgmap v171: 97 pgs: 97 active+clean; 682 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.7 MiB/s wr, 618 op/s 2023-11-16T16:44:48.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:48 smithi001 ceph-mon[98800]: pgmap v172: 97 pgs: 97 active+clean; 682 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 195 op/s 2023-11-16T16:44:49.010 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:48 smithi026 ceph-mon[100112]: pgmap v172: 97 pgs: 97 active+clean; 682 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 195 op/s 2023-11-16T16:44:49.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:48 smithi053 ceph-mon[100638]: pgmap v172: 97 pgs: 97 active+clean; 682 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 195 op/s 2023-11-16T16:44:50.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:50 smithi001 ceph-mon[98800]: pgmap v173: 97 pgs: 97 active+clean; 682 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 195 op/s 2023-11-16T16:44:51.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:50 smithi026 ceph-mon[100112]: pgmap v173: 97 pgs: 97 active+clean; 682 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 195 op/s 2023-11-16T16:44:51.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:50 smithi053 ceph-mon[100638]: pgmap v173: 97 pgs: 97 active+clean; 682 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 195 op/s 2023-11-16T16:44:52.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:52 smithi053 ceph-mon[100638]: pgmap v174: 97 pgs: 97 active+clean; 689 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.7 MiB/s wr, 254 op/s 2023-11-16T16:44:52.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:52 smithi001 ceph-mon[98800]: pgmap v174: 97 pgs: 97 active+clean; 689 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.7 MiB/s wr, 254 op/s 2023-11-16T16:44:52.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:52 smithi026 ceph-mon[100112]: pgmap v174: 97 pgs: 97 active+clean; 689 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.7 MiB/s wr, 254 op/s 2023-11-16T16:44:54.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:54 smithi001 ceph-mon[98800]: pgmap v175: 97 pgs: 97 active+clean; 689 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 165 op/s 2023-11-16T16:44:55.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:54 smithi026 ceph-mon[100112]: pgmap v175: 97 pgs: 97 active+clean; 689 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 165 op/s 2023-11-16T16:44:55.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:54 smithi053 ceph-mon[100638]: pgmap v175: 97 pgs: 97 active+clean; 689 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 165 op/s 2023-11-16T16:44:56.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:56 smithi001 ceph-mon[98800]: pgmap v176: 97 pgs: 97 active+clean; 700 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.7 MiB/s wr, 197 op/s 2023-11-16T16:44:57.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:56 smithi026 ceph-mon[100112]: pgmap v176: 97 pgs: 97 active+clean; 700 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.7 MiB/s wr, 197 op/s 2023-11-16T16:44:57.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:56 smithi053 ceph-mon[100638]: pgmap v176: 97 pgs: 97 active+clean; 700 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.7 MiB/s wr, 197 op/s 2023-11-16T16:44:58.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:44:58 smithi001 ceph-mon[98800]: pgmap v177: 97 pgs: 97 active+clean; 700 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 91 op/s 2023-11-16T16:44:59.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:44:58 smithi026 ceph-mon[100112]: pgmap v177: 97 pgs: 97 active+clean; 700 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 91 op/s 2023-11-16T16:44:59.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:44:58 smithi053 ceph-mon[100638]: pgmap v177: 97 pgs: 97 active+clean; 700 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 91 op/s 2023-11-16T16:45:00.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:00 smithi001 ceph-mon[98800]: pgmap v178: 97 pgs: 97 active+clean; 700 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 91 op/s 2023-11-16T16:45:00.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:00 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:45:00.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:00 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:45:01.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:00 smithi026 ceph-mon[100112]: pgmap v178: 97 pgs: 97 active+clean; 700 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 91 op/s 2023-11-16T16:45:01.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:00 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:45:01.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:00 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:45:01.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:00 smithi053 ceph-mon[100638]: pgmap v178: 97 pgs: 97 active+clean; 700 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 91 op/s 2023-11-16T16:45:01.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:00 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:45:01.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:00 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:45:02.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:02 smithi053 ceph-mon[100638]: pgmap v179: 97 pgs: 97 active+clean; 713 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.6 MiB/s wr, 157 op/s 2023-11-16T16:45:02.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:02 smithi001 ceph-mon[98800]: pgmap v179: 97 pgs: 97 active+clean; 713 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.6 MiB/s wr, 157 op/s 2023-11-16T16:45:02.757 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:02 smithi026 ceph-mon[100112]: pgmap v179: 97 pgs: 97 active+clean; 713 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.6 MiB/s wr, 157 op/s 2023-11-16T16:45:04.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:04 smithi001 ceph-mon[98800]: pgmap v180: 97 pgs: 97 active+clean; 713 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 98 op/s 2023-11-16T16:45:05.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:04 smithi026 ceph-mon[100112]: pgmap v180: 97 pgs: 97 active+clean; 713 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 98 op/s 2023-11-16T16:45:05.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:04 smithi053 ceph-mon[100638]: pgmap v180: 97 pgs: 97 active+clean; 713 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 98 op/s 2023-11-16T16:45:05.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:05 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:45:05.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:05 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:45:06.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:05 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:45:06.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:05 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:45:06.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:05 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:45:06.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:05 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:45:06.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:06 smithi001 ceph-mon[98800]: pgmap v181: 97 pgs: 97 active+clean; 728 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.7 MiB/s wr, 138 op/s 2023-11-16T16:45:07.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:06 smithi026 ceph-mon[100112]: pgmap v181: 97 pgs: 97 active+clean; 728 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.7 MiB/s wr, 138 op/s 2023-11-16T16:45:07.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:06 smithi053 ceph-mon[100638]: pgmap v181: 97 pgs: 97 active+clean; 728 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.7 MiB/s wr, 138 op/s 2023-11-16T16:45:08.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:08 smithi001 ceph-mon[98800]: pgmap v182: 97 pgs: 97 active+clean; 728 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 106 op/s 2023-11-16T16:45:09.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:08 smithi026 ceph-mon[100112]: pgmap v182: 97 pgs: 97 active+clean; 728 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 106 op/s 2023-11-16T16:45:09.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:08 smithi053 ceph-mon[100638]: pgmap v182: 97 pgs: 97 active+clean; 728 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 106 op/s 2023-11-16T16:45:09.936 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-16T16:45:10.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.084+0000 7f47facd6700 1 -- 172.21.15.1:0/3326902803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f47f4100420 msgr2=0x7f47f410bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:10.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.084+0000 7f47facd6700 1 --2- 172.21.15.1:0/3326902803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f47f4100420 0x7f47f410bfd0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f47ec00b0a0 tx=0x7f47ec00b3b0 comp rx=0 tx=0).stop 2023-11-16T16:45:10.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.084+0000 7f47facd6700 1 -- 172.21.15.1:0/3326902803 shutdown_connections 2023-11-16T16:45:10.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.084+0000 7f47facd6700 1 --2- 172.21.15.1:0/3326902803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f47f4100420 0x7f47f410bfd0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:10.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.084+0000 7f47facd6700 1 --2- 172.21.15.1:0/3326902803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47f4105250 0x7f47f4105620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:10.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.084+0000 7f47facd6700 1 --2- 172.21.15.1:0/3326902803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47f40ffa70 0x7f47f40ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:10.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.084+0000 7f47facd6700 1 -- 172.21.15.1:0/3326902803 >> 172.21.15.1:0/3326902803 conn(0x7f47f40763a0 msgr2=0x7f47f40767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:10.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.084+0000 7f47facd6700 1 -- 172.21.15.1:0/3326902803 shutdown_connections 2023-11-16T16:45:10.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.084+0000 7f47facd6700 1 -- 172.21.15.1:0/3326902803 wait complete. 2023-11-16T16:45:10.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.084+0000 7f47facd6700 1 Processor -- start 2023-11-16T16:45:10.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.084+0000 7f47facd6700 1 -- start start 2023-11-16T16:45:10.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.085+0000 7f47facd6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f47f40ffa70 0x7f47f4197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:10.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.085+0000 7f47facd6700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47f4100420 0x7f47f4197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:10.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.085+0000 7f47facd6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47f4105250 0x7f47f41a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:10.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.085+0000 7f47facd6700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f41981b0 con 0x7f47f40ffa70 2023-11-16T16:45:10.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.085+0000 7f47facd6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f41a28f0 con 0x7f47f4105250 2023-11-16T16:45:10.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.085+0000 7f47facd6700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f41a2ab0 con 0x7f47f4100420 2023-11-16T16:45:10.091 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.085+0000 7f47f8a72700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f47f40ffa70 0x7f47f4197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:10.091 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.085+0000 7f47f8a72700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f47f40ffa70 0x7f47f4197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45306/0 (socket says 172.21.15.1:45306) 2023-11-16T16:45:10.091 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.085+0000 7f47f8a72700 1 -- 172.21.15.1:0/1704886513 learned_addr learned my addr 172.21.15.1:0/1704886513 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:45:10.091 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.085+0000 7f47f3fff700 1 --2- 172.21.15.1:0/1704886513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47f4100420 0x7f47f4197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:10.091 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.085+0000 7f47f9273700 1 --2- 172.21.15.1:0/1704886513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47f4105250 0x7f47f41a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:10.091 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.085+0000 7f47f8a72700 1 -- 172.21.15.1:0/1704886513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47f4100420 msgr2=0x7f47f4197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:10.091 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.085+0000 7f47f8a72700 1 --2- 172.21.15.1:0/1704886513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47f4100420 0x7f47f4197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:10.091 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.085+0000 7f47f8a72700 1 -- 172.21.15.1:0/1704886513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47f4105250 msgr2=0x7f47f41a23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:10.091 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.085+0000 7f47f8a72700 1 --2- 172.21.15.1:0/1704886513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47f4105250 0x7f47f41a23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:10.091 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.085+0000 7f47f8a72700 1 -- 172.21.15.1:0/1704886513 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ec009d00 con 0x7f47f40ffa70 2023-11-16T16:45:10.091 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.085+0000 7f47f8a72700 1 --2- 172.21.15.1:0/1704886513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f47f40ffa70 0x7f47f4197330 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f47e800e9a0 tx=0x7f47e800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:10.091 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.085+0000 7f47f1ffb700 1 -- 172.21.15.1:0/1704886513 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e8009790 con 0x7f47f40ffa70 2023-11-16T16:45:10.092 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.085+0000 7f47facd6700 1 -- 172.21.15.1:0/1704886513 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47f410f230 con 0x7f47f40ffa70 2023-11-16T16:45:10.092 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.085+0000 7f47f1ffb700 1 -- 172.21.15.1:0/1704886513 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47e8010430 con 0x7f47f40ffa70 2023-11-16T16:45:10.092 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.085+0000 7f47f1ffb700 1 -- 172.21.15.1:0/1704886513 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e8018680 con 0x7f47f40ffa70 2023-11-16T16:45:10.092 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.086+0000 7f47facd6700 1 -- 172.21.15.1:0/1704886513 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47f410f0a0 con 0x7f47f40ffa70 2023-11-16T16:45:10.097 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.088+0000 7f47f1ffb700 1 -- 172.21.15.1:0/1704886513 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f47e80105a0 con 0x7f47f40ffa70 2023-11-16T16:45:10.097 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.089+0000 7f47f1ffb700 1 --2- 172.21.15.1:0/1704886513 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f47e40a12c0 0x7f47e40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:10.097 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.089+0000 7f47f3fff700 1 --2- 172.21.15.1:0/1704886513 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f47e40a12c0 0x7f47e40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:10.097 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.089+0000 7f47f1ffb700 1 -- 172.21.15.1:0/1704886513 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f47e8014070 con 0x7f47f40ffa70 2023-11-16T16:45:10.097 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.089+0000 7f47f3fff700 1 --2- 172.21.15.1:0/1704886513 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f47e40a12c0 0x7f47e40a3770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f47f4068690 tx=0x7f47e0009260 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:10.097 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.089+0000 7f4806e62b80 1 Processor -- start 2023-11-16T16:45:10.098 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.089+0000 7f4806e62b80 1 -- start start 2023-11-16T16:45:10.098 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.090+0000 7f4806e62b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597a9fd6b60 0x5597a9fd6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:10.098 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.090+0000 7f4806e62b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597a9fd7470 0x5597a9fdb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:10.098 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.090+0000 7f4806e62b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597a9fdb8b0 0x5597a9fdbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:10.098 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.090+0000 7f4806e62b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597a9edc060 con 0x5597a9fdb8b0 2023-11-16T16:45:10.098 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.090+0000 7f4806e62b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597a9f2e8a0 con 0x5597a9fd7470 2023-11-16T16:45:10.098 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.090+0000 7f4806e62b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597a9f27d30 con 0x5597a9fd6b60 2023-11-16T16:45:10.098 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.090+0000 7f47f8a72700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597a9fd6b60 0x5597a9fd6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:10.098 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.090+0000 7f47f9273700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597a9fdb8b0 0x5597a9fdbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:10.098 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.090+0000 7f47f3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597a9fd7470 0x5597a9fdb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:10.098 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.090+0000 7f47f8a72700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597a9fd6b60 0x5597a9fd6f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:41996/0 (socket says 172.21.15.1:41996) 2023-11-16T16:45:10.098 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.090+0000 7f47f8a72700 1 -- 172.21.15.1:0/1707281900 learned_addr learned my addr 172.21.15.1:0/1707281900 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:45:10.099 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.090+0000 7f47f9273700 1 -- 172.21.15.1:0/1707281900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597a9fd6b60 msgr2=0x5597a9fd6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:10.099 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.090+0000 7f47f9273700 1 --2- 172.21.15.1:0/1707281900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597a9fd6b60 0x5597a9fd6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:10.099 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.090+0000 7f47f9273700 1 -- 172.21.15.1:0/1707281900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597a9fd7470 msgr2=0x5597a9fdb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:10.099 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.090+0000 7f47f9273700 1 --2- 172.21.15.1:0/1707281900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597a9fd7470 0x5597a9fdb370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:10.099 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.090+0000 7f47f9273700 1 -- 172.21.15.1:0/1707281900 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597a9da9630 con 0x5597a9fdb8b0 2023-11-16T16:45:10.099 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.090+0000 7f47f8a72700 1 --2- 172.21.15.1:0/1707281900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597a9fd6b60 0x5597a9fd6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:45:10.099 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.090+0000 7f47f9273700 1 --2- 172.21.15.1:0/1707281900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597a9fdb8b0 0x5597a9fdbd00 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f47ec0062a0 tx=0x7f47ec009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c894dc5027e32d server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:10.099 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.090+0000 7f47deffd700 1 -- 172.21.15.1:0/1707281900 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec00e040 con 0x5597a9fdb8b0 2023-11-16T16:45:10.099 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.091+0000 7f47deffd700 1 -- 172.21.15.1:0/1707281900 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47ec00bcb0 con 0x5597a9fdb8b0 2023-11-16T16:45:10.099 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.091+0000 7f47deffd700 1 -- 172.21.15.1:0/1707281900 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec01c440 con 0x5597a9fdb8b0 2023-11-16T16:45:10.099 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.091+0000 7f4806e62b80 1 -- 172.21.15.1:0/1707281900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597a9fdb8b0 msgr2=0x5597a9fdbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:10.099 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.091+0000 7f4806e62b80 1 --2- 172.21.15.1:0/1707281900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597a9fdb8b0 0x5597a9fdbd00 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f47ec0062a0 tx=0x7f47ec009c20 comp rx=0 tx=0).stop 2023-11-16T16:45:10.099 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.091+0000 7f4806e62b80 1 -- 172.21.15.1:0/1707281900 shutdown_connections 2023-11-16T16:45:10.100 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.091+0000 7f4806e62b80 1 --2- 172.21.15.1:0/1707281900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597a9fdb8b0 0x5597a9fdbd00 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:10.100 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.091+0000 7f4806e62b80 1 --2- 172.21.15.1:0/1707281900 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597a9fd6b60 0x5597a9fd6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:10.100 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.091+0000 7f4806e62b80 1 --2- 172.21.15.1:0/1707281900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597a9fd7470 0x5597a9fdb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:10.100 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.091+0000 7f4806e62b80 1 -- 172.21.15.1:0/1707281900 >> 172.21.15.1:0/1707281900 conn(0x5597a9fcc360 msgr2=0x5597a9fd1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:10.100 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.092+0000 7f4806e62b80 1 -- 172.21.15.1:0/1707281900 shutdown_connections 2023-11-16T16:45:10.100 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.092+0000 7f4806e62b80 1 -- 172.21.15.1:0/1707281900 wait complete. 2023-11-16T16:45:10.100 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.092+0000 7f4806e62b80 1 -- start start 2023-11-16T16:45:10.100 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.092+0000 7f4806e62b80 1 Processor -- start 2023-11-16T16:45:10.100 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.092+0000 7f4806e62b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597a9fd6b60 0x5597a9fe3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:10.100 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.092+0000 7f4806e62b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597a9fd7470 0x5597a9fe0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:10.100 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.092+0000 7f4806e62b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597a9fdb8b0 0x5597a9fe07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:10.100 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.092+0000 7f4806e62b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597a9ec98e0 con 0x5597a9fd6b60 2023-11-16T16:45:10.101 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.092+0000 7f4806e62b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597a9ec9580 con 0x5597a9fdb8b0 2023-11-16T16:45:10.101 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.092+0000 7f4806e62b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597a9dab920 con 0x5597a9fd7470 2023-11-16T16:45:10.101 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.093+0000 7f4806e62b80 10 client.0 _ll_register_callbacks cb 0x5597a9ee7780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:45:10.101 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.093+0000 7f47f8a72700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597a9fd6b60 0x5597a9fe3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:10.101 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.093+0000 7f47f3fff700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597a9fd7470 0x5597a9fe0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:10.101 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.093+0000 7f47f9273700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597a9fdb8b0 0x5597a9fe07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:10.101 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.093+0000 7f47f8a72700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597a9fd6b60 0x5597a9fe3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45320/0 (socket says 172.21.15.1:45320) 2023-11-16T16:45:10.101 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.093+0000 7f47f8a72700 1 -- 172.21.15.1:0/322543297 learned_addr learned my addr 172.21.15.1:0/322543297 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:45:10.101 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.093+0000 7f47f8a72700 1 -- 172.21.15.1:0/322543297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597a9fd7470 msgr2=0x5597a9fe0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:10.101 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.093+0000 7f47f8a72700 1 --2- 172.21.15.1:0/322543297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597a9fd7470 0x5597a9fe0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:10.101 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.093+0000 7f47f8a72700 1 -- 172.21.15.1:0/322543297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597a9fdb8b0 msgr2=0x5597a9fe07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:10.101 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.093+0000 7f47f8a72700 1 --2- 172.21.15.1:0/322543297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597a9fdb8b0 0x5597a9fe07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:10.101 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.093+0000 7f47f8a72700 1 -- 172.21.15.1:0/322543297 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47f410c8d0 con 0x5597a9fd6b60 2023-11-16T16:45:10.102 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.093+0000 7f47f9273700 1 --2- 172.21.15.1:0/322543297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597a9fdb8b0 0x5597a9fe07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:45:10.102 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.093+0000 7f47f3fff700 1 --2- 172.21.15.1:0/322543297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597a9fd7470 0x5597a9fe0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:45:10.102 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.093+0000 7f47f8a72700 1 --2- 172.21.15.1:0/322543297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597a9fd6b60 0x5597a9fe3000 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f47e80963d0 tx=0x7f47e80984d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f00ea18ee3341d server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:10.102 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.093+0000 7f47d3fff700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:45:10.102 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.093+0000 7f47d3fff700 1 -- 172.21.15.1:0/322543297 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e809a380 con 0x5597a9fd6b60 2023-11-16T16:45:10.102 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.093+0000 7f47d3fff700 1 -- 172.21.15.1:0/322543297 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f47e8099410 con 0x5597a9fd6b60 2023-11-16T16:45:10.102 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.093+0000 7f4806e62b80 1 -- 172.21.15.1:0/322543297 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5597a9fc9b20 con 0x5597a9fd6b60 2023-11-16T16:45:10.102 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.094+0000 7f47d3fff700 1 -- 172.21.15.1:0/322543297 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47e80b82f0 con 0x5597a9fd6b60 2023-11-16T16:45:10.102 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.094+0000 7f4806e62b80 10 client.16191 fetch_fsmap learned FSMap version 28 2023-11-16T16:45:10.102 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.094+0000 7f4806e62b80 1 -- 172.21.15.1:0/322543297 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5597a9eee190 con 0x5597a9fd6b60 2023-11-16T16:45:10.102 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.094+0000 7f47d3fff700 1 -- 172.21.15.1:0/322543297 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47e80c8080 con 0x5597a9fd6b60 2023-11-16T16:45:10.102 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.094+0000 7f47d3fff700 1 -- 172.21.15.1:0/322543297 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f47e80c9f90 con 0x5597a9fd6b60 2023-11-16T16:45:10.102 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.094+0000 7f4806e62b80 10 client.16191 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:45:10.103 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.094+0000 7f4806e62b80 10 client.16191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:45:10.103 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.094+0000 7f4806e62b80 20 client.16191 populate_metadata read hostname 'smithi001' 2023-11-16T16:45:10.103 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.094+0000 7f4806e62b80 1 --2- 172.21.15.1:0/322543297 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5597a9ff0950 0x5597a9ff2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:10.103 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.094+0000 7f4806e62b80 4 client.16191 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:45:10.103 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.094+0000 7f4806e62b80 1 -- 172.21.15.1:0/322543297 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5597a9eea0a0 con 0x5597a9ff0950 2023-11-16T16:45:10.103 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.094+0000 7f47f3fff700 1 --2- 172.21.15.1:0/322543297 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5597a9ff0950 0x5597a9ff2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:10.103 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.095+0000 7f47d3fff700 10 client.16191 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:45:10.103 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.095+0000 7f47f3fff700 1 --2- 172.21.15.1:0/322543297 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5597a9ff0950 0x5597a9ff2d60 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e92d510a97958dda server_cookie=96ade3c889d5d78f in_seq=0 out_seq=0 2023-11-16T16:45:10.104 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.096+0000 7f47d3fff700 1 -- 172.21.15.1:0/322543297 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5597a9eea0a0 con 0x5597a9ff0950 2023-11-16T16:45:10.104 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.096+0000 7f47d3fff700 10 client.16191 handle_command_reply: tid=0 2023-11-16T16:45:10.104 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.097+0000 7f4806e62b80 1 client.16191 shutdown 2023-11-16T16:45:10.104 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.097+0000 7f4806e62b80 1 -- 172.21.15.1:0/322543297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597a9fd6b60 msgr2=0x5597a9fe3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:10.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.097+0000 7f4806e62b80 1 --2- 172.21.15.1:0/322543297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597a9fd6b60 0x5597a9fe3000 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f47e80963d0 tx=0x7f47e80984d0 comp rx=0 tx=0).stop 2023-11-16T16:45:10.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.097+0000 7f4806e62b80 1 -- 172.21.15.1:0/322543297 shutdown_connections 2023-11-16T16:45:10.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.097+0000 7f4806e62b80 1 --2- 172.21.15.1:0/322543297 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5597a9ff0950 0x5597a9ff2d60 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:10.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.097+0000 7f4806e62b80 1 --2- 172.21.15.1:0/322543297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597a9fdb8b0 0x5597a9fe07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:10.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.097+0000 7f47d3fff700 0 client.16191 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:45:10.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.097+0000 7f4806e62b80 1 --2- 172.21.15.1:0/322543297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597a9fd6b60 0x5597a9fe3000 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:10.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.097+0000 7f4806e62b80 1 --2- 172.21.15.1:0/322543297 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597a9fd7470 0x5597a9fe0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:10.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.097+0000 7f4806e62b80 1 -- 172.21.15.1:0/322543297 >> 172.21.15.1:0/322543297 conn(0x5597a9fcc360 msgr2=0x5597a9fd0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:10.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.097+0000 7f4806e62b80 1 -- 172.21.15.1:0/322543297 shutdown_connections 2023-11-16T16:45:10.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.097+0000 7f4806e62b80 1 -- 172.21.15.1:0/322543297 wait complete. 2023-11-16T16:45:10.105 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.097+0000 7f4806e62b80 20 client.16191 trim_cache size 0 max 16384 2023-11-16T16:45:10.124 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.122+0000 7f4806e62b80 1 Processor -- start 2023-11-16T16:45:10.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.122+0000 7f4806e62b80 1 -- start start 2023-11-16T16:45:10.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.122+0000 7f4806e62b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597a9fd6b60 0x5597a9fe79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:10.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.122+0000 7f4806e62b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597a9fd7470 0x5597a9fe2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:10.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.122+0000 7f4806e62b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597a9fdb8b0 0x5597a9fdfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:10.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.122+0000 7f4806e62b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597a9fda140 con 0x5597a9fdb8b0 2023-11-16T16:45:10.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.122+0000 7f4806e62b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597a9fe0dd0 con 0x5597a9fd7470 2023-11-16T16:45:10.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.122+0000 7f4806e62b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597a9fe3730 con 0x5597a9fd6b60 2023-11-16T16:45:10.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.122+0000 7f47f9273700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597a9fdb8b0 0x5597a9fdfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:10.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.122+0000 7f47f9273700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597a9fdb8b0 0x5597a9fdfab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45332/0 (socket says 172.21.15.1:45332) 2023-11-16T16:45:10.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.122+0000 7f47f3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597a9fd7470 0x5597a9fe2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:10.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.122+0000 7f47f9273700 1 -- 172.21.15.1:0/1255670913 learned_addr learned my addr 172.21.15.1:0/1255670913 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:45:10.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.123+0000 7f47f9273700 1 -- 172.21.15.1:0/1255670913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597a9fd6b60 msgr2=0x5597a9fe79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:10.126 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.123+0000 7f47f9273700 1 --2- 172.21.15.1:0/1255670913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597a9fd6b60 0x5597a9fe79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:10.126 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.123+0000 7f47f9273700 1 -- 172.21.15.1:0/1255670913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597a9fd7470 msgr2=0x5597a9fe2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:10.126 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.123+0000 7f47f9273700 1 --2- 172.21.15.1:0/1255670913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597a9fd7470 0x5597a9fe2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:10.126 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.123+0000 7f47f9273700 1 -- 172.21.15.1:0/1255670913 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597a9eedee0 con 0x5597a9fdb8b0 2023-11-16T16:45:10.126 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.123+0000 7f47f9273700 1 --2- 172.21.15.1:0/1255670913 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597a9fdb8b0 0x5597a9fdfab0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009bf0 tx=0x7f47ec0094e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b0eb22517696f38 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:10.127 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.123+0000 7f47df7fe700 1 -- 172.21.15.1:0/1255670913 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec00e040 con 0x5597a9fdb8b0 2023-11-16T16:45:10.128 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.123+0000 7f47df7fe700 1 -- 172.21.15.1:0/1255670913 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47ec0074a0 con 0x5597a9fdb8b0 2023-11-16T16:45:10.128 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.123+0000 7f47df7fe700 1 -- 172.21.15.1:0/1255670913 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec02dc70 con 0x5597a9fdb8b0 2023-11-16T16:45:10.128 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.124+0000 7f4806e62b80 1 -- 172.21.15.1:0/1255670913 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597a9fdb8b0 msgr2=0x5597a9fdfab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:10.128 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.124+0000 7f4806e62b80 1 --2- 172.21.15.1:0/1255670913 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597a9fdb8b0 0x5597a9fdfab0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009bf0 tx=0x7f47ec0094e0 comp rx=0 tx=0).stop 2023-11-16T16:45:10.128 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.124+0000 7f4806e62b80 1 -- 172.21.15.1:0/1255670913 shutdown_connections 2023-11-16T16:45:10.128 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.124+0000 7f4806e62b80 1 --2- 172.21.15.1:0/1255670913 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597a9fdb8b0 0x5597a9fdfab0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:10.128 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.124+0000 7f4806e62b80 1 --2- 172.21.15.1:0/1255670913 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597a9fd6b60 0x5597a9fe79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:10.128 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.124+0000 7f4806e62b80 1 --2- 172.21.15.1:0/1255670913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597a9fd7470 0x5597a9fe2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:10.128 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.124+0000 7f4806e62b80 1 -- 172.21.15.1:0/1255670913 >> 172.21.15.1:0/1255670913 conn(0x5597a9fcc360 msgr2=0x5597a9fecb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:10.128 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.124+0000 7f4806e62b80 1 -- 172.21.15.1:0/1255670913 shutdown_connections 2023-11-16T16:45:10.128 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.124+0000 7f4806e62b80 1 -- 172.21.15.1:0/1255670913 wait complete. 2023-11-16T16:45:10.128 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.125+0000 7f4806e62b80 1 -- start start 2023-11-16T16:45:10.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.125+0000 7f4806e62b80 1 Processor -- start 2023-11-16T16:45:10.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.125+0000 7f4806e62b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597a9fd6b60 0x5597a9fe3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:10.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.125+0000 7f4806e62b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597a9fd7470 0x5597a9fe3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:10.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.126+0000 7f4806e62b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597a9fdb8b0 0x5597a9fe4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:10.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.126+0000 7f4806e62b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597aa0439b0 con 0x5597a9fd7470 2023-11-16T16:45:10.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.126+0000 7f4806e62b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597aa043b20 con 0x5597a9fdb8b0 2023-11-16T16:45:10.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.126+0000 7f4806e62b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597aa043ce0 con 0x5597a9fd6b60 2023-11-16T16:45:10.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.126+0000 7f4806e62b80 10 client.0 _ll_register_callbacks cb 0x7f47e8099880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:45:10.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.126+0000 7f47f3fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597a9fd7470 0x5597a9fe3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:10.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.126+0000 7f47f3fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597a9fd7470 0x5597a9fe3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45336/0 (socket says 172.21.15.1:45336) 2023-11-16T16:45:10.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.126+0000 7f47f3fff700 1 -- 172.21.15.1:0/2598073022 learned_addr learned my addr 172.21.15.1:0/2598073022 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:45:10.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.126+0000 7f47f8a72700 1 --2- 172.21.15.1:0/2598073022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597a9fd6b60 0x5597a9fe3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:10.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.126+0000 7f47f3fff700 1 -- 172.21.15.1:0/2598073022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597a9fd6b60 msgr2=0x5597a9fe3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:10.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.126+0000 7f47f3fff700 1 --2- 172.21.15.1:0/2598073022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597a9fd6b60 0x5597a9fe3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:10.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.126+0000 7f47f3fff700 1 -- 172.21.15.1:0/2598073022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597a9fdb8b0 msgr2=0x5597a9fe4400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-16T16:45:10.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.126+0000 7f47f3fff700 1 --2- 172.21.15.1:0/2598073022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597a9fdb8b0 0x5597a9fe4400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:10.132 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.126+0000 7f47f3fff700 1 -- 172.21.15.1:0/2598073022 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5597a9da9630 con 0x5597a9fd7470 2023-11-16T16:45:10.132 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.127+0000 7f47f3fff700 1 --2- 172.21.15.1:0/2598073022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597a9fd7470 0x5597a9fe3ec0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f47e00097c0 tx=0x7f47e00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd62449e07a22a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:10.132 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.127+0000 7f47d3fff700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:45:10.132 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.127+0000 7f47d3fff700 1 -- 172.21.15.1:0/2598073022 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e0022070 con 0x5597a9fd7470 2023-11-16T16:45:10.132 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.127+0000 7f47d3fff700 1 -- 172.21.15.1:0/2598073022 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f47e0003890 con 0x5597a9fd7470 2023-11-16T16:45:10.132 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.127+0000 7f4806e62b80 1 -- 172.21.15.1:0/2598073022 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f47ec00bcb0 con 0x5597a9fd7470 2023-11-16T16:45:10.132 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.127+0000 7f47d3fff700 1 -- 172.21.15.1:0/2598073022 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47e0003a50 con 0x5597a9fd7470 2023-11-16T16:45:10.132 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.127+0000 7f4806e62b80 10 client.16197 fetch_fsmap learned FSMap version 28 2023-11-16T16:45:10.132 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.127+0000 7f4806e62b80 1 -- 172.21.15.1:0/2598073022 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5597a9d3ec60 con 0x5597a9fd7470 2023-11-16T16:45:10.132 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.128+0000 7f47d3fff700 1 -- 172.21.15.1:0/2598073022 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47ec00bcb0 con 0x5597a9fd7470 2023-11-16T16:45:10.132 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.128+0000 7f47d3fff700 1 -- 172.21.15.1:0/2598073022 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f47e0041d10 con 0x5597a9fd7470 2023-11-16T16:45:10.132 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.128+0000 7f4806e62b80 10 client.16197 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:45:10.132 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.128+0000 7f4806e62b80 10 client.16197 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:45:10.133 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.128+0000 7f4806e62b80 20 client.16197 populate_metadata read hostname 'smithi001' 2023-11-16T16:45:10.133 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.128+0000 7f4806e62b80 1 --2- 172.21.15.1:0/2598073022 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5597aa0454d0 0x5597aa0478e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:10.133 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.128+0000 7f4806e62b80 4 client.16197 mds_command: new command op to 24521 tid=0[{"prefix": "scrub status"}] 2023-11-16T16:45:10.133 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.128+0000 7f4806e62b80 1 -- 172.21.15.1:0/2598073022 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5597a9ef6660 con 0x5597aa0454d0 2023-11-16T16:45:10.133 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.128+0000 7f47f8a72700 1 --2- 172.21.15.1:0/2598073022 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5597aa0454d0 0x5597aa0478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:10.133 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.129+0000 7f47d3fff700 10 client.16197 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:45:10.133 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.129+0000 7f47f8a72700 1 --2- 172.21.15.1:0/2598073022 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5597aa0454d0 0x5597aa0478e0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1135a24a6b8d5c57 server_cookie=ec77692c7559f10e in_seq=0 out_seq=0 2023-11-16T16:45:10.136 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.134+0000 7f47d3fff700 1 -- 172.21.15.1:0/2598073022 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5597a9ef6660 con 0x5597aa0454d0 2023-11-16T16:45:10.136 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.134+0000 7f47d3fff700 10 client.16197 handle_command_reply: tid=0 2023-11-16T16:45:10.137 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.134+0000 7f4806e62b80 1 client.16197 shutdown 2023-11-16T16:45:10.137 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.134+0000 7f4806e62b80 1 -- 172.21.15.1:0/2598073022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597a9fd7470 msgr2=0x5597a9fe3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:10.137 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.134+0000 7f4806e62b80 1 --2- 172.21.15.1:0/2598073022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597a9fd7470 0x5597a9fe3ec0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f47e00097c0 tx=0x7f47e00047c0 comp rx=0 tx=0).stop 2023-11-16T16:45:10.137 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.135+0000 7f4806e62b80 1 -- 172.21.15.1:0/2598073022 shutdown_connections 2023-11-16T16:45:10.137 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.135+0000 7f4806e62b80 1 --2- 172.21.15.1:0/2598073022 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5597aa0454d0 0x5597aa0478e0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:10.137 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.135+0000 7f4806e62b80 1 --2- 172.21.15.1:0/2598073022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5597a9fdb8b0 0x5597a9fe4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:10.137 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.135+0000 7f4806e62b80 1 --2- 172.21.15.1:0/2598073022 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5597a9fd6b60 0x5597a9fe3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:10.137 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.135+0000 7f4806e62b80 1 --2- 172.21.15.1:0/2598073022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5597a9fd7470 0x5597a9fe3ec0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:10.137 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.135+0000 7f4806e62b80 1 -- 172.21.15.1:0/2598073022 >> 172.21.15.1:0/2598073022 conn(0x5597a9fcc360 msgr2=0x5597a9fdfab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:10.137 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.135+0000 7f47d3fff700 0 client.16197 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:45:10.137 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.135+0000 7f4806e62b80 1 -- 172.21.15.1:0/2598073022 shutdown_connections 2023-11-16T16:45:10.137 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.135+0000 7f4806e62b80 1 -- 172.21.15.1:0/2598073022 wait complete. 2023-11-16T16:45:10.138 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.135+0000 7f4806e62b80 20 client.16197 trim_cache size 0 max 16384 2023-11-16T16:45:10.138 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:45:10.138 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-16T16:45:10.138 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2023-11-16T16:45:10.138 INFO:teuthology.orchestra.run.smithi001.stdout: "4e68c117-e4ae-40d9-847d-0d6d5d2b1317": { 2023-11-16T16:45:10.139 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2023-11-16T16:45:10.139 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "4e68c117-e4ae-40d9-847d-0d6d5d2b1317", 2023-11-16T16:45:10.139 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2023-11-16T16:45:10.139 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:45:10.139 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:45:10.139 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:45:10.140 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.136+0000 7f47facd6700 1 -- 172.21.15.1:0/1704886513 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f47e40a12c0 msgr2=0x7f47e40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:10.140 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.136+0000 7f47facd6700 1 --2- 172.21.15.1:0/1704886513 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f47e40a12c0 0x7f47e40a3770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f47f4068690 tx=0x7f47e0009260 comp rx=0 tx=0).stop 2023-11-16T16:45:10.140 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.136+0000 7f47facd6700 1 -- 172.21.15.1:0/1704886513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f47f40ffa70 msgr2=0x7f47f4197330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:10.140 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.136+0000 7f47facd6700 1 --2- 172.21.15.1:0/1704886513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f47f40ffa70 0x7f47f4197330 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f47e800e9a0 tx=0x7f47e800ecb0 comp rx=0 tx=0).stop 2023-11-16T16:45:10.140 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.136+0000 7f47facd6700 1 -- 172.21.15.1:0/1704886513 shutdown_connections 2023-11-16T16:45:10.140 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.136+0000 7f47facd6700 1 --2- 172.21.15.1:0/1704886513 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f47e40a12c0 0x7f47e40a3770 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:10.140 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.136+0000 7f47facd6700 1 --2- 172.21.15.1:0/1704886513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47f4105250 0x7f47f41a23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:10.140 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.136+0000 7f47facd6700 1 --2- 172.21.15.1:0/1704886513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f47f40ffa70 0x7f47f4197330 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:10.140 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.136+0000 7f47facd6700 1 --2- 172.21.15.1:0/1704886513 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f47f4100420 0x7f47f4197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:10.140 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.136+0000 7f47facd6700 1 -- 172.21.15.1:0/1704886513 >> 172.21.15.1:0/1704886513 conn(0x7f47f40763a0 msgr2=0x7f47f40fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:10.140 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.136+0000 7f47facd6700 1 -- 172.21.15.1:0/1704886513 shutdown_connections 2023-11-16T16:45:10.141 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:10.136+0000 7f47facd6700 1 -- 172.21.15.1:0/1704886513 wait complete. 2023-11-16T16:45:10.156 INFO:tasks.cephfs.filesystem:scrub status for tag:4e68c117-e4ae-40d9-847d-0d6d5d2b1317 - {'path': '/', 'tag': '4e68c117-e4ae-40d9-847d-0d6d5d2b1317', 'options': 'recursive,force'} 2023-11-16T16:45:10.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:10 smithi001 ceph-mon[98800]: pgmap v183: 97 pgs: 97 active+clean; 728 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 106 op/s 2023-11-16T16:45:11.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:10 smithi026 ceph-mon[100112]: pgmap v183: 97 pgs: 97 active+clean; 728 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 106 op/s 2023-11-16T16:45:11.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:10 smithi053 ceph-mon[100638]: pgmap v183: 97 pgs: 97 active+clean; 728 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.1 MiB/s wr, 106 op/s 2023-11-16T16:45:11.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:11 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:45:11.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:11 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:45:12.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:11 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:45:12.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:11 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:45:12.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:11 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:45:12.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:11 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:45:12.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:12 smithi001 ceph-mon[98800]: pgmap v184: 97 pgs: 97 active+clean; 745 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.9 MiB/s wr, 160 op/s 2023-11-16T16:45:13.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:12 smithi026 ceph-mon[100112]: pgmap v184: 97 pgs: 97 active+clean; 745 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.9 MiB/s wr, 160 op/s 2023-11-16T16:45:13.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:12 smithi053 ceph-mon[100638]: pgmap v184: 97 pgs: 97 active+clean; 745 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.9 MiB/s wr, 160 op/s 2023-11-16T16:45:14.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:14 smithi001 ceph-mon[98800]: pgmap v185: 97 pgs: 97 active+clean; 745 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 94 op/s 2023-11-16T16:45:15.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:14 smithi026 ceph-mon[100112]: pgmap v185: 97 pgs: 97 active+clean; 745 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 94 op/s 2023-11-16T16:45:15.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:14 smithi053 ceph-mon[100638]: pgmap v185: 97 pgs: 97 active+clean; 745 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 94 op/s 2023-11-16T16:45:16.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:16 smithi001 ceph-mon[98800]: pgmap v186: 97 pgs: 97 active+clean; 766 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.1 MiB/s wr, 152 op/s 2023-11-16T16:45:17.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:16 smithi026 ceph-mon[100112]: pgmap v186: 97 pgs: 97 active+clean; 766 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.1 MiB/s wr, 152 op/s 2023-11-16T16:45:17.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:16 smithi053 ceph-mon[100638]: pgmap v186: 97 pgs: 97 active+clean; 766 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.1 MiB/s wr, 152 op/s 2023-11-16T16:45:18.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:18 smithi001 ceph-mon[98800]: pgmap v187: 97 pgs: 97 active+clean; 766 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 112 op/s 2023-11-16T16:45:19.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:18 smithi026 ceph-mon[100112]: pgmap v187: 97 pgs: 97 active+clean; 766 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 112 op/s 2023-11-16T16:45:19.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:18 smithi053 ceph-mon[100638]: pgmap v187: 97 pgs: 97 active+clean; 766 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 112 op/s 2023-11-16T16:45:20.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:20 smithi001 ceph-mon[98800]: pgmap v188: 97 pgs: 97 active+clean; 766 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 112 op/s 2023-11-16T16:45:21.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:20 smithi026 ceph-mon[100112]: pgmap v188: 97 pgs: 97 active+clean; 766 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 112 op/s 2023-11-16T16:45:21.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:20 smithi053 ceph-mon[100638]: pgmap v188: 97 pgs: 97 active+clean; 766 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 112 op/s 2023-11-16T16:45:22.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:22 smithi053 ceph-mon[100638]: pgmap v189: 97 pgs: 97 active+clean; 781 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.1 MiB/s wr, 176 op/s 2023-11-16T16:45:22.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:22 smithi001 ceph-mon[98800]: pgmap v189: 97 pgs: 97 active+clean; 781 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.1 MiB/s wr, 176 op/s 2023-11-16T16:45:22.757 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:22 smithi026 ceph-mon[100112]: pgmap v189: 97 pgs: 97 active+clean; 781 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.1 MiB/s wr, 176 op/s 2023-11-16T16:45:24.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:24 smithi001 ceph-mon[98800]: pgmap v190: 97 pgs: 97 active+clean; 781 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 123 op/s 2023-11-16T16:45:25.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:24 smithi026 ceph-mon[100112]: pgmap v190: 97 pgs: 97 active+clean; 781 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 123 op/s 2023-11-16T16:45:25.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:24 smithi053 ceph-mon[100638]: pgmap v190: 97 pgs: 97 active+clean; 781 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 123 op/s 2023-11-16T16:45:26.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:26 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:45:26.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:26 smithi001 ceph-mon[98800]: pgmap v191: 97 pgs: 97 active+clean; 788 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.6 MiB/s wr, 160 op/s 2023-11-16T16:45:27.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:26 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:45:27.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:26 smithi026 ceph-mon[100112]: pgmap v191: 97 pgs: 97 active+clean; 788 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.6 MiB/s wr, 160 op/s 2023-11-16T16:45:27.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:26 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:45:27.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:26 smithi053 ceph-mon[100638]: pgmap v191: 97 pgs: 97 active+clean; 788 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.6 MiB/s wr, 160 op/s 2023-11-16T16:45:28.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:28 smithi001 ceph-mon[98800]: pgmap v192: 97 pgs: 97 active+clean; 788 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 102 op/s 2023-11-16T16:45:29.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:28 smithi026 ceph-mon[100112]: pgmap v192: 97 pgs: 97 active+clean; 788 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 102 op/s 2023-11-16T16:45:29.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:28 smithi053 ceph-mon[100638]: pgmap v192: 97 pgs: 97 active+clean; 788 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 102 op/s 2023-11-16T16:45:30.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:30 smithi001 ceph-mon[98800]: pgmap v193: 97 pgs: 97 active+clean; 788 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 102 op/s 2023-11-16T16:45:30.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:30 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:45:31.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:30 smithi026 ceph-mon[100112]: pgmap v193: 97 pgs: 97 active+clean; 788 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 102 op/s 2023-11-16T16:45:31.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:30 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:45:31.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:30 smithi053 ceph-mon[100638]: pgmap v193: 97 pgs: 97 active+clean; 788 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 102 op/s 2023-11-16T16:45:31.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:30 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:45:31.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:31 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:45:31.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:31 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:45:31.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:31 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:45:32.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:31 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:45:32.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:31 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:45:32.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:31 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:45:32.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:31 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:45:32.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:31 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:45:32.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:31 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:45:32.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:32 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:45:32.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:32 smithi001 ceph-mon[98800]: Service mds.all has no related daemon already placed. 2023-11-16T16:45:32.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:32 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:45:32.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:32 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:45:32.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:32 smithi001 ceph-mon[98800]: pgmap v194: 97 pgs: 97 active+clean; 802 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 140 op/s 2023-11-16T16:45:33.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:32 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:45:33.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:32 smithi026 ceph-mon[100112]: Service mds.all has no related daemon already placed. 2023-11-16T16:45:33.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:32 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:45:33.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:32 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:45:33.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:32 smithi026 ceph-mon[100112]: pgmap v194: 97 pgs: 97 active+clean; 802 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 140 op/s 2023-11-16T16:45:33.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:32 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:45:33.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:32 smithi053 ceph-mon[100638]: Service mds.all has no related daemon already placed. 2023-11-16T16:45:33.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:32 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:45:33.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:32 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:45:33.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:32 smithi053 ceph-mon[100638]: pgmap v194: 97 pgs: 97 active+clean; 802 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 140 op/s 2023-11-16T16:45:33.597 INFO:tasks.workunit.client.0.smithi001.stdout:cleaned up hierarchy 2023-11-16T16:45:34.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:34 smithi001 ceph-mon[98800]: pgmap v195: 97 pgs: 97 active+clean; 802 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 76 op/s 2023-11-16T16:45:35.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:34 smithi026 ceph-mon[100112]: pgmap v195: 97 pgs: 97 active+clean; 802 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 76 op/s 2023-11-16T16:45:35.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:34 smithi053 ceph-mon[100638]: pgmap v195: 97 pgs: 97 active+clean; 802 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 76 op/s 2023-11-16T16:45:36.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:36 smithi001 ceph-mon[98800]: pgmap v196: 97 pgs: 97 active+clean; 809 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 221 op/s 2023-11-16T16:45:37.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:36 smithi026 ceph-mon[100112]: pgmap v196: 97 pgs: 97 active+clean; 809 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 221 op/s 2023-11-16T16:45:37.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:36 smithi053 ceph-mon[100638]: pgmap v196: 97 pgs: 97 active+clean; 809 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 221 op/s 2023-11-16T16:45:38.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:38 smithi001 ceph-mon[98800]: pgmap v197: 97 pgs: 97 active+clean; 809 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.8 MiB/s wr, 183 op/s 2023-11-16T16:45:39.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:38 smithi026 ceph-mon[100112]: pgmap v197: 97 pgs: 97 active+clean; 809 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.8 MiB/s wr, 183 op/s 2023-11-16T16:45:39.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:38 smithi053 ceph-mon[100638]: pgmap v197: 97 pgs: 97 active+clean; 809 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.8 MiB/s wr, 183 op/s 2023-11-16T16:45:39.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:39 smithi001 ceph-mon[98800]: scrub summary: idle 2023-11-16T16:45:39.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:39 smithi001 ceph-mon[98800]: scrub summary: idle 2023-11-16T16:45:40.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:39 smithi026 ceph-mon[100112]: scrub summary: idle 2023-11-16T16:45:40.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:39 smithi026 ceph-mon[100112]: scrub summary: idle 2023-11-16T16:45:40.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:39 smithi053 ceph-mon[100638]: scrub summary: idle 2023-11-16T16:45:40.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:39 smithi053 ceph-mon[100638]: scrub summary: idle 2023-11-16T16:45:40.157 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-16T16:45:40.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.300+0000 7f2ccfa8b700 1 -- 172.21.15.1:0/3699423399 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2cc80ff410 msgr2=0x7f2cc80ff7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:40.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.300+0000 7f2ccfa8b700 1 --2- 172.21.15.1:0/3699423399 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2cc80ff410 0x7f2cc80ff7e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40099c0 tx=0x7f2cc4009cd0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.300+0000 7f2ccfa8b700 1 -- 172.21.15.1:0/3699423399 shutdown_connections 2023-11-16T16:45:40.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.300+0000 7f2ccfa8b700 1 --2- 172.21.15.1:0/3699423399 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cc8104450 0x7f2cc810c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.300+0000 7f2ccfa8b700 1 --2- 172.21.15.1:0/3699423399 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2cc80ff410 0x7f2cc80ff7e0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.300+0000 7f2ccfa8b700 1 --2- 172.21.15.1:0/3699423399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cc80ffdb0 0x7f2cc8103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.300+0000 7f2ccfa8b700 1 -- 172.21.15.1:0/3699423399 >> 172.21.15.1:0/3699423399 conn(0x7f2cc80763a0 msgr2=0x7f2cc80767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:40.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.300+0000 7f2ccfa8b700 1 -- 172.21.15.1:0/3699423399 shutdown_connections 2023-11-16T16:45:40.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.300+0000 7f2ccfa8b700 1 -- 172.21.15.1:0/3699423399 wait complete. 2023-11-16T16:45:40.304 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.301+0000 7f2ccfa8b700 1 Processor -- start 2023-11-16T16:45:40.304 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.301+0000 7f2ccfa8b700 1 -- start start 2023-11-16T16:45:40.304 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.301+0000 7f2ccfa8b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cc80ff410 0x7f2cc81963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:40.304 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.301+0000 7f2ccfa8b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cc80ffdb0 0x7f2cc81968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:40.304 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.301+0000 7f2ccfa8b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2cc8104450 0x7f2cc819ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:40.304 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.301+0000 7f2ccfa8b700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc819b390 con 0x7f2cc8104450 2023-11-16T16:45:40.304 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.301+0000 7f2ccfa8b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc819b500 con 0x7f2cc80ffdb0 2023-11-16T16:45:40.304 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.301+0000 7f2ccfa8b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc819b6c0 con 0x7f2cc80ff410 2023-11-16T16:45:40.304 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.301+0000 7f2cce028700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2cc8104450 0x7f2cc819ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:40.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.301+0000 7f2ccd026700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cc80ffdb0 0x7f2cc81968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:40.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.301+0000 7f2ccd026700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cc80ffdb0 0x7f2cc81968f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.1:39534/0 (socket says 172.21.15.1:39534) 2023-11-16T16:45:40.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.301+0000 7f2cce028700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2cc8104450 0x7f2cc819ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45350/0 (socket says 172.21.15.1:45350) 2023-11-16T16:45:40.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.301+0000 7f2ccd026700 1 -- 172.21.15.1:0/2250568686 learned_addr learned my addr 172.21.15.1:0/2250568686 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:45:40.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.301+0000 7f2ccd827700 1 --2- 172.21.15.1:0/2250568686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cc80ff410 0x7f2cc81963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:40.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.302+0000 7f2ccd026700 1 -- 172.21.15.1:0/2250568686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cc80ff410 msgr2=0x7f2cc81963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:40.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.302+0000 7f2ccd026700 1 --2- 172.21.15.1:0/2250568686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cc80ff410 0x7f2cc81963b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.302+0000 7f2ccd026700 1 -- 172.21.15.1:0/2250568686 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2cc8104450 msgr2=0x7f2cc819ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:40.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.302+0000 7f2ccd026700 1 --2- 172.21.15.1:0/2250568686 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2cc8104450 0x7f2cc819ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.302+0000 7f2ccd026700 1 -- 172.21.15.1:0/2250568686 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc40095f0 con 0x7f2cc80ffdb0 2023-11-16T16:45:40.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.302+0000 7f2cce028700 1 --2- 172.21.15.1:0/2250568686 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2cc8104450 0x7f2cc819ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:45:40.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.302+0000 7f2ccd026700 1 --2- 172.21.15.1:0/2250568686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cc80ffdb0 0x7f2cc81968f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2cb800e940 tx=0x7f2cb800ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:40.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.302+0000 7f2cbeffd700 1 -- 172.21.15.1:0/2250568686 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8009730 con 0x7f2cc80ffdb0 2023-11-16T16:45:40.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.302+0000 7f2ccfa8b700 1 -- 172.21.15.1:0/2250568686 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cc810f260 con 0x7f2cc80ffdb0 2023-11-16T16:45:40.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.302+0000 7f2cbeffd700 1 -- 172.21.15.1:0/2250568686 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cb8010430 con 0x7f2cc80ffdb0 2023-11-16T16:45:40.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.302+0000 7f2cbeffd700 1 -- 172.21.15.1:0/2250568686 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb800cea0 con 0x7f2cc80ffdb0 2023-11-16T16:45:40.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.302+0000 7f2ccfa8b700 1 -- 172.21.15.1:0/2250568686 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cc810f0d0 con 0x7f2cc80ffdb0 2023-11-16T16:45:40.311 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.304+0000 7f2cbeffd700 1 -- 172.21.15.1:0/2250568686 <== mon.2 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f2cb8010810 con 0x7f2cc80ffdb0 2023-11-16T16:45:40.311 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.305+0000 7f2cbeffd700 1 --2- 172.21.15.1:0/2250568686 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f2cb40a12c0 0x7f2cb40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:40.311 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.305+0000 7f2ccd827700 1 --2- 172.21.15.1:0/2250568686 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f2cb40a12c0 0x7f2cb40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:40.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.305+0000 7f2cbeffd700 1 -- 172.21.15.1:0/2250568686 <== mon.2 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f2cb8014070 con 0x7f2cc80ffdb0 2023-11-16T16:45:40.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.305+0000 7f2cdbc17b80 1 Processor -- start 2023-11-16T16:45:40.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.306+0000 7f2cdbc17b80 1 -- start start 2023-11-16T16:45:40.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.306+0000 7f2ccd827700 1 --2- 172.21.15.1:0/2250568686 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f2cb40a12c0 0x7f2cb40a3770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4009990 tx=0x7f2cc4005fb0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:40.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.306+0000 7f2cdbc17b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55852b324f70 0x55852b325340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:40.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.306+0000 7f2cdbc17b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55852b325880 0x55852b329780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:40.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.306+0000 7f2cdbc17b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55852b329cc0 0x55852b32a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:40.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.306+0000 7f2cdbc17b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55852b226060 con 0x55852b324f70 2023-11-16T16:45:40.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.306+0000 7f2cdbc17b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55852b2788a0 con 0x55852b325880 2023-11-16T16:45:40.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.306+0000 7f2cdbc17b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55852b271d30 con 0x55852b329cc0 2023-11-16T16:45:40.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.306+0000 7f2ccd827700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55852b324f70 0x55852b325340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:40.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.306+0000 7f2ccd827700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55852b324f70 0x55852b325340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45354/0 (socket says 172.21.15.1:45354) 2023-11-16T16:45:40.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.306+0000 7f2ccd827700 1 -- 172.21.15.1:0/3446944588 learned_addr learned my addr 172.21.15.1:0/3446944588 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:45:40.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.306+0000 7f2ccd026700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55852b325880 0x55852b329780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:40.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.306+0000 7f2cce028700 1 --2- 172.21.15.1:0/3446944588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55852b329cc0 0x55852b32a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:40.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.306+0000 7f2ccd827700 1 -- 172.21.15.1:0/3446944588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55852b329cc0 msgr2=0x55852b32a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:40.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.306+0000 7f2ccd827700 1 --2- 172.21.15.1:0/3446944588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55852b329cc0 0x55852b32a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.306+0000 7f2ccd827700 1 -- 172.21.15.1:0/3446944588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55852b325880 msgr2=0x55852b329780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:40.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.306+0000 7f2ccd827700 1 --2- 172.21.15.1:0/3446944588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55852b325880 0x55852b329780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.306+0000 7f2ccd827700 1 -- 172.21.15.1:0/3446944588 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55852b0f3630 con 0x55852b324f70 2023-11-16T16:45:40.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.306+0000 7f2ccd026700 1 --2- 172.21.15.1:0/3446944588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55852b325880 0x55852b329780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:45:40.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.307+0000 7f2ccd827700 1 --2- 172.21.15.1:0/3446944588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55852b324f70 0x55852b325340 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40095a0 tx=0x7f2cc401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7890cbf5f23b9b06 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:40.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.307+0000 7f2cb3fff700 1 -- 172.21.15.1:0/3446944588 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4003880 con 0x55852b324f70 2023-11-16T16:45:40.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.307+0000 7f2cb3fff700 1 -- 172.21.15.1:0/3446944588 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cc401e040 con 0x55852b324f70 2023-11-16T16:45:40.314 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.307+0000 7f2cb3fff700 1 -- 172.21.15.1:0/3446944588 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc40306e0 con 0x55852b324f70 2023-11-16T16:45:40.314 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.307+0000 7f2cdbc17b80 1 -- 172.21.15.1:0/3446944588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55852b324f70 msgr2=0x55852b325340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:40.314 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.307+0000 7f2cdbc17b80 1 --2- 172.21.15.1:0/3446944588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55852b324f70 0x55852b325340 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40095a0 tx=0x7f2cc401f000 comp rx=0 tx=0).stop 2023-11-16T16:45:40.314 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.307+0000 7f2cdbc17b80 1 -- 172.21.15.1:0/3446944588 shutdown_connections 2023-11-16T16:45:40.314 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.307+0000 7f2cdbc17b80 1 --2- 172.21.15.1:0/3446944588 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55852b329cc0 0x55852b32a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.314 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.307+0000 7f2cdbc17b80 1 --2- 172.21.15.1:0/3446944588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55852b324f70 0x55852b325340 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.314 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.307+0000 7f2cdbc17b80 1 --2- 172.21.15.1:0/3446944588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55852b325880 0x55852b329780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.314 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.307+0000 7f2cdbc17b80 1 -- 172.21.15.1:0/3446944588 >> 172.21.15.1:0/3446944588 conn(0x55852b316360 msgr2=0x55852b31bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:40.314 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.308+0000 7f2cdbc17b80 1 -- 172.21.15.1:0/3446944588 shutdown_connections 2023-11-16T16:45:40.314 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.308+0000 7f2cdbc17b80 1 -- 172.21.15.1:0/3446944588 wait complete. 2023-11-16T16:45:40.314 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.308+0000 7f2cdbc17b80 1 -- start start 2023-11-16T16:45:40.314 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.308+0000 7f2cdbc17b80 1 Processor -- start 2023-11-16T16:45:40.314 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.308+0000 7f2cdbc17b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55852b324f70 0x55852b331450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:40.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.308+0000 7f2cdbc17b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55852b325880 0x55852b32e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:40.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.309+0000 7f2cdbc17b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55852b329cc0 0x55852b32edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:40.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.309+0000 7f2cdbc17b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55852b2138e0 con 0x55852b324f70 2023-11-16T16:45:40.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.309+0000 7f2cdbc17b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55852b213580 con 0x55852b329cc0 2023-11-16T16:45:40.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.309+0000 7f2cdbc17b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55852b0f5920 con 0x55852b325880 2023-11-16T16:45:40.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.309+0000 7f2cdbc17b80 10 client.0 _ll_register_callbacks cb 0x55852b231780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:45:40.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.309+0000 7f2ccd827700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55852b324f70 0x55852b331450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:40.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.309+0000 7f2cce028700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55852b329cc0 0x55852b32edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:40.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.309+0000 7f2ccd827700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55852b324f70 0x55852b331450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45360/0 (socket says 172.21.15.1:45360) 2023-11-16T16:45:40.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.309+0000 7f2ccd026700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55852b325880 0x55852b32e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:40.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.309+0000 7f2ccd827700 1 -- 172.21.15.1:0/1487148587 learned_addr learned my addr 172.21.15.1:0/1487148587 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:45:40.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.309+0000 7f2ccd827700 1 -- 172.21.15.1:0/1487148587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55852b325880 msgr2=0x55852b32e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:40.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.309+0000 7f2ccd827700 1 --2- 172.21.15.1:0/1487148587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55852b325880 0x55852b32e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.316 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.309+0000 7f2ccd827700 1 -- 172.21.15.1:0/1487148587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55852b329cc0 msgr2=0x55852b32edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:40.316 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.309+0000 7f2ccd827700 1 --2- 172.21.15.1:0/1487148587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55852b329cc0 0x55852b32edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.316 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.309+0000 7f2ccd827700 1 -- 172.21.15.1:0/1487148587 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cc810c860 con 0x55852b324f70 2023-11-16T16:45:40.316 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.309+0000 7f2ccd827700 1 --2- 172.21.15.1:0/1487148587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55852b324f70 0x55852b331450 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40098e0 tx=0x7f2cc4005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a189e83848a65 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:40.316 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.309+0000 7f2cb17fa700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:45:40.316 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.309+0000 7f2cb17fa700 1 -- 172.21.15.1:0/1487148587 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc40309a0 con 0x55852b324f70 2023-11-16T16:45:40.316 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.309+0000 7f2cb17fa700 1 -- 172.21.15.1:0/1487148587 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f2cc4030b00 con 0x55852b324f70 2023-11-16T16:45:40.316 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.309+0000 7f2cdbc17b80 1 -- 172.21.15.1:0/1487148587 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55852b313b20 con 0x55852b324f70 2023-11-16T16:45:40.316 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.310+0000 7f2cb17fa700 1 -- 172.21.15.1:0/1487148587 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2cc4030cc0 con 0x55852b324f70 2023-11-16T16:45:40.316 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.310+0000 7f2cdbc17b80 10 client.16209 fetch_fsmap learned FSMap version 28 2023-11-16T16:45:40.316 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.310+0000 7f2cdbc17b80 1 -- 172.21.15.1:0/1487148587 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55852b238190 con 0x55852b324f70 2023-11-16T16:45:40.316 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.310+0000 7f2cb17fa700 1 -- 172.21.15.1:0/1487148587 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55852b313b20 con 0x55852b324f70 2023-11-16T16:45:40.316 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.310+0000 7f2cb17fa700 1 -- 172.21.15.1:0/1487148587 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f2cc4043770 con 0x55852b324f70 2023-11-16T16:45:40.317 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.310+0000 7f2cdbc17b80 10 client.16209 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:45:40.317 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.310+0000 7f2cdbc17b80 10 client.16209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:45:40.317 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.310+0000 7f2cdbc17b80 20 client.16209 populate_metadata read hostname 'smithi001' 2023-11-16T16:45:40.317 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.310+0000 7f2cdbc17b80 1 --2- 172.21.15.1:0/1487148587 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55852b33e7f0 0x55852b340c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:40.317 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.310+0000 7f2cdbc17b80 4 client.16209 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:45:40.317 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.310+0000 7f2cdbc17b80 1 -- 172.21.15.1:0/1487148587 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55852b2340a0 con 0x55852b33e7f0 2023-11-16T16:45:40.317 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.310+0000 7f2ccd026700 1 --2- 172.21.15.1:0/1487148587 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55852b33e7f0 0x55852b340c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:40.317 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.311+0000 7f2cb17fa700 10 client.16209 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:45:40.317 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.311+0000 7f2ccd026700 1 --2- 172.21.15.1:0/1487148587 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55852b33e7f0 0x55852b340c00 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6dcbd0c508ed3e0 server_cookie=5150a44fc565c922 in_seq=0 out_seq=0 2023-11-16T16:45:40.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.504+0000 7f2cb17fa700 1 -- 172.21.15.1:0/1487148587 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55852b2340a0 con 0x55852b33e7f0 2023-11-16T16:45:40.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.504+0000 7f2cb17fa700 10 client.16209 handle_command_reply: tid=0 2023-11-16T16:45:40.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.504+0000 7f2cdbc17b80 1 client.16209 shutdown 2023-11-16T16:45:40.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.504+0000 7f2cdbc17b80 1 -- 172.21.15.1:0/1487148587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55852b324f70 msgr2=0x55852b331450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:40.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.504+0000 7f2cdbc17b80 1 --2- 172.21.15.1:0/1487148587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55852b324f70 0x55852b331450 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40098e0 tx=0x7f2cc4005a00 comp rx=0 tx=0).stop 2023-11-16T16:45:40.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.505+0000 7f2cdbc17b80 1 -- 172.21.15.1:0/1487148587 shutdown_connections 2023-11-16T16:45:40.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.505+0000 7f2cdbc17b80 1 --2- 172.21.15.1:0/1487148587 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55852b33e7f0 0x55852b340c00 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.505+0000 7f2cdbc17b80 1 --2- 172.21.15.1:0/1487148587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55852b329cc0 0x55852b32edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.505+0000 7f2cb17fa700 0 client.16209 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:45:40.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.505+0000 7f2cdbc17b80 1 --2- 172.21.15.1:0/1487148587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55852b324f70 0x55852b331450 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.505+0000 7f2cdbc17b80 1 --2- 172.21.15.1:0/1487148587 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55852b325880 0x55852b32e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.507 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.505+0000 7f2cdbc17b80 1 -- 172.21.15.1:0/1487148587 >> 172.21.15.1:0/1487148587 conn(0x55852b316360 msgr2=0x55852b31ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:40.507 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.505+0000 7f2cdbc17b80 1 -- 172.21.15.1:0/1487148587 shutdown_connections 2023-11-16T16:45:40.507 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.505+0000 7f2cdbc17b80 1 -- 172.21.15.1:0/1487148587 wait complete. 2023-11-16T16:45:40.507 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.505+0000 7f2cdbc17b80 20 client.16209 trim_cache size 0 max 16384 2023-11-16T16:45:40.524 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.522+0000 7f2cdbc17b80 1 Processor -- start 2023-11-16T16:45:40.525 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.523+0000 7f2cdbc17b80 1 -- start start 2023-11-16T16:45:40.525 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.523+0000 7f2cdbc17b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55852b324f70 0x55852b3285b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:40.525 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.523+0000 7f2cdbc17b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55852b325880 0x55852b335d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:40.525 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.523+0000 7f2cdbc17b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55852b329cc0 0x55852b330ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:40.525 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.523+0000 7f2cdbc17b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55852b33b6a0 con 0x55852b324f70 2023-11-16T16:45:40.525 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.523+0000 7f2cdbc17b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55852b32f6d0 con 0x55852b325880 2023-11-16T16:45:40.525 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.523+0000 7f2cdbc17b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55852b31c0a0 con 0x55852b329cc0 2023-11-16T16:45:40.525 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.523+0000 7f2ccd827700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55852b324f70 0x55852b3285b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:40.525 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.523+0000 7f2ccd827700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55852b324f70 0x55852b3285b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45368/0 (socket says 172.21.15.1:45368) 2023-11-16T16:45:40.525 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.523+0000 7f2ccd827700 1 -- 172.21.15.1:0/4075802790 learned_addr learned my addr 172.21.15.1:0/4075802790 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:45:40.525 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.523+0000 7f2cce028700 1 --2- 172.21.15.1:0/4075802790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55852b329cc0 0x55852b330ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:40.525 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.523+0000 7f2ccd026700 1 --2- 172.21.15.1:0/4075802790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55852b325880 0x55852b335d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:40.526 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.523+0000 7f2ccd827700 1 -- 172.21.15.1:0/4075802790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55852b329cc0 msgr2=0x55852b330ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:40.526 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.523+0000 7f2ccd827700 1 --2- 172.21.15.1:0/4075802790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55852b329cc0 0x55852b330ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.526 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.523+0000 7f2ccd827700 1 -- 172.21.15.1:0/4075802790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55852b325880 msgr2=0x55852b335d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:40.526 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.523+0000 7f2ccd827700 1 --2- 172.21.15.1:0/4075802790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55852b325880 0x55852b335d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.526 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.523+0000 7f2ccd827700 1 -- 172.21.15.1:0/4075802790 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55852b237ee0 con 0x55852b324f70 2023-11-16T16:45:40.526 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.523+0000 7f2cce028700 1 --2- 172.21.15.1:0/4075802790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55852b329cc0 0x55852b330ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:45:40.526 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.523+0000 7f2ccd026700 1 --2- 172.21.15.1:0/4075802790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55852b325880 0x55852b335d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:45:40.526 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.523+0000 7f2ccd827700 1 --2- 172.21.15.1:0/4075802790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55852b324f70 0x55852b3285b0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4004650 tx=0x7f2cc4003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab568c22d0bce669 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:40.526 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.524+0000 7f2cbcdac700 1 -- 172.21.15.1:0/4075802790 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4043da0 con 0x55852b324f70 2023-11-16T16:45:40.526 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.524+0000 7f2cbcdac700 1 -- 172.21.15.1:0/4075802790 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cc4043930 con 0x55852b324f70 2023-11-16T16:45:40.526 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.524+0000 7f2cbcdac700 1 -- 172.21.15.1:0/4075802790 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc400ba50 con 0x55852b324f70 2023-11-16T16:45:40.530 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.524+0000 7f2cdbc17b80 1 -- 172.21.15.1:0/4075802790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55852b324f70 msgr2=0x55852b3285b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:40.530 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.524+0000 7f2cdbc17b80 1 --2- 172.21.15.1:0/4075802790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55852b324f70 0x55852b3285b0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4004650 tx=0x7f2cc4003dc0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.530 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.524+0000 7f2cdbc17b80 1 -- 172.21.15.1:0/4075802790 shutdown_connections 2023-11-16T16:45:40.530 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.524+0000 7f2cdbc17b80 1 --2- 172.21.15.1:0/4075802790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55852b329cc0 0x55852b330ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.530 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.524+0000 7f2cdbc17b80 1 --2- 172.21.15.1:0/4075802790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55852b324f70 0x55852b3285b0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.530 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.524+0000 7f2cdbc17b80 1 --2- 172.21.15.1:0/4075802790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55852b325880 0x55852b335d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.531 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.524+0000 7f2cdbc17b80 1 -- 172.21.15.1:0/4075802790 >> 172.21.15.1:0/4075802790 conn(0x55852b316360 msgr2=0x55852b32a090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:40.531 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.524+0000 7f2cdbc17b80 1 -- 172.21.15.1:0/4075802790 shutdown_connections 2023-11-16T16:45:40.531 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.524+0000 7f2cdbc17b80 1 -- 172.21.15.1:0/4075802790 wait complete. 2023-11-16T16:45:40.531 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.525+0000 7f2cdbc17b80 1 -- start start 2023-11-16T16:45:40.531 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.525+0000 7f2cdbc17b80 1 Processor -- start 2023-11-16T16:45:40.531 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.525+0000 7f2cdbc17b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55852b324f70 0x55852b333a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:40.531 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.525+0000 7f2cdbc17b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55852b325880 0x55852b333f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:40.531 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.525+0000 7f2cdbc17b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55852b329cc0 0x55852b3916b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:40.531 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.526+0000 7f2cdbc17b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55852b334630 con 0x55852b325880 2023-11-16T16:45:40.531 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.526+0000 7f2cdbc17b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55852b391bf0 con 0x55852b324f70 2023-11-16T16:45:40.531 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.526+0000 7f2cdbc17b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55852b391d30 con 0x55852b329cc0 2023-11-16T16:45:40.531 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.526+0000 7f2cdbc17b80 10 client.0 _ll_register_callbacks cb 0x7f2cb8010f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:45:40.532 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.526+0000 7f2ccd026700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55852b325880 0x55852b333f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:40.532 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.526+0000 7f2ccd026700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55852b325880 0x55852b333f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45374/0 (socket says 172.21.15.1:45374) 2023-11-16T16:45:40.532 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.526+0000 7f2ccd026700 1 -- 172.21.15.1:0/588628462 learned_addr learned my addr 172.21.15.1:0/588628462 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:45:40.532 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.526+0000 7f2ccd827700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55852b324f70 0x55852b333a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:40.532 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.526+0000 7f2cce028700 1 --2- 172.21.15.1:0/588628462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55852b329cc0 0x55852b3916b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:40.532 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.526+0000 7f2ccd026700 1 -- 172.21.15.1:0/588628462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55852b329cc0 msgr2=0x55852b3916b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:40.532 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.526+0000 7f2ccd026700 1 --2- 172.21.15.1:0/588628462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55852b329cc0 0x55852b3916b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.532 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.526+0000 7f2ccd026700 1 -- 172.21.15.1:0/588628462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55852b324f70 msgr2=0x55852b333a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:40.532 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.526+0000 7f2ccd026700 1 --2- 172.21.15.1:0/588628462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55852b324f70 0x55852b333a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.532 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.526+0000 7f2ccd026700 1 -- 172.21.15.1:0/588628462 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55852b238190 con 0x55852b325880 2023-11-16T16:45:40.532 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.526+0000 7f2ccd827700 1 --2- 172.21.15.1:0/588628462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55852b324f70 0x55852b333a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:45:40.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.526+0000 7f2ccd026700 1 --2- 172.21.15.1:0/588628462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55852b325880 0x55852b333f70 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f2cb80ca630 tx=0x7f2cb8096310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e0d620c8939465 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:40.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.526+0000 7f2cb17fa700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:45:40.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.526+0000 7f2cb17fa700 1 -- 172.21.15.1:0/588628462 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8010660 con 0x55852b325880 2023-11-16T16:45:40.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.526+0000 7f2cb17fa700 1 -- 172.21.15.1:0/588628462 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f2cb80b6070 con 0x55852b325880 2023-11-16T16:45:40.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.526+0000 7f2cdbc17b80 1 -- 172.21.15.1:0/588628462 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2cc401e040 con 0x55852b325880 2023-11-16T16:45:40.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.526+0000 7f2cb17fa700 1 -- 172.21.15.1:0/588628462 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2cb80ca2f0 con 0x55852b325880 2023-11-16T16:45:40.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.527+0000 7f2cdbc17b80 10 client.16215 fetch_fsmap learned FSMap version 28 2023-11-16T16:45:40.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.527+0000 7f2cdbc17b80 1 -- 172.21.15.1:0/588628462 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55852b088c60 con 0x55852b325880 2023-11-16T16:45:40.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.527+0000 7f2cb17fa700 1 -- 172.21.15.1:0/588628462 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cb8063020 con 0x55852b325880 2023-11-16T16:45:40.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.527+0000 7f2cb17fa700 1 -- 172.21.15.1:0/588628462 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f2cb80a1f10 con 0x55852b325880 2023-11-16T16:45:40.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.527+0000 7f2cdbc17b80 10 client.16215 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:45:40.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.527+0000 7f2cdbc17b80 10 client.16215 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:45:40.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.527+0000 7f2cdbc17b80 20 client.16215 populate_metadata read hostname 'smithi001' 2023-11-16T16:45:40.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.527+0000 7f2cdbc17b80 1 --2- 172.21.15.1:0/588628462 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55852b393550 0x55852b395960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:40.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.527+0000 7f2cdbc17b80 4 client.16215 mds_command: new command op to 24521 tid=0[{"prefix": "scrub status"}] 2023-11-16T16:45:40.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.527+0000 7f2cdbc17b80 1 -- 172.21.15.1:0/588628462 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55852b240660 con 0x55852b393550 2023-11-16T16:45:40.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.527+0000 7f2ccd827700 1 --2- 172.21.15.1:0/588628462 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55852b393550 0x55852b395960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:40.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.528+0000 7f2cb17fa700 10 client.16215 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:45:40.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.528+0000 7f2ccd827700 1 --2- 172.21.15.1:0/588628462 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55852b393550 0x55852b395960 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=336162a52fa1aab2 server_cookie=4fd18bd64d2287fa in_seq=0 out_seq=0 2023-11-16T16:45:40.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.528+0000 7f2cb17fa700 1 -- 172.21.15.1:0/588628462 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55852b240660 con 0x55852b393550 2023-11-16T16:45:40.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.528+0000 7f2cb17fa700 10 client.16215 handle_command_reply: tid=0 2023-11-16T16:45:40.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.528+0000 7f2cdbc17b80 1 client.16215 shutdown 2023-11-16T16:45:40.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.528+0000 7f2cdbc17b80 1 -- 172.21.15.1:0/588628462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55852b325880 msgr2=0x55852b333f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:40.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.528+0000 7f2cdbc17b80 1 --2- 172.21.15.1:0/588628462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55852b325880 0x55852b333f70 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f2cb80ca630 tx=0x7f2cb8096310 comp rx=0 tx=0).stop 2023-11-16T16:45:40.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.528+0000 7f2cdbc17b80 1 -- 172.21.15.1:0/588628462 shutdown_connections 2023-11-16T16:45:40.535 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.528+0000 7f2cdbc17b80 1 --2- 172.21.15.1:0/588628462 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55852b393550 0x55852b395960 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.535 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.528+0000 7f2cdbc17b80 1 --2- 172.21.15.1:0/588628462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55852b329cc0 0x55852b3916b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.535 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.528+0000 7f2cb17fa700 0 client.16215 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:45:40.535 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.528+0000 7f2cdbc17b80 1 --2- 172.21.15.1:0/588628462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55852b324f70 0x55852b333a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.535 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.528+0000 7f2cdbc17b80 1 --2- 172.21.15.1:0/588628462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55852b325880 0x55852b333f70 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.535 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.528+0000 7f2cdbc17b80 1 -- 172.21.15.1:0/588628462 >> 172.21.15.1:0/588628462 conn(0x55852b316360 msgr2=0x55852b31f960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:40.535 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.528+0000 7f2cdbc17b80 1 -- 172.21.15.1:0/588628462 shutdown_connections 2023-11-16T16:45:40.535 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.528+0000 7f2cdbc17b80 1 -- 172.21.15.1:0/588628462 wait complete. 2023-11-16T16:45:40.535 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.528+0000 7f2cdbc17b80 20 client.16215 trim_cache size 0 max 16384 2023-11-16T16:45:40.536 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:45:40.536 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2023-11-16T16:45:40.536 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2023-11-16T16:45:40.536 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:45:40.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.529+0000 7f2ccfa8b700 1 -- 172.21.15.1:0/2250568686 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f2cb40a12c0 msgr2=0x7f2cb40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:40.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.529+0000 7f2ccfa8b700 1 --2- 172.21.15.1:0/2250568686 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f2cb40a12c0 0x7f2cb40a3770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4009990 tx=0x7f2cc4005fb0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.529+0000 7f2ccfa8b700 1 -- 172.21.15.1:0/2250568686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cc80ffdb0 msgr2=0x7f2cc81968f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:40.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.529+0000 7f2ccfa8b700 1 --2- 172.21.15.1:0/2250568686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cc80ffdb0 0x7f2cc81968f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2cb800e940 tx=0x7f2cb800ec50 comp rx=0 tx=0).stop 2023-11-16T16:45:40.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.529+0000 7f2ccfa8b700 1 -- 172.21.15.1:0/2250568686 shutdown_connections 2023-11-16T16:45:40.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.529+0000 7f2ccfa8b700 1 --2- 172.21.15.1:0/2250568686 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f2cb40a12c0 0x7f2cb40a3770 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.529+0000 7f2ccfa8b700 1 --2- 172.21.15.1:0/2250568686 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2cc8104450 0x7f2cc819ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.529+0000 7f2ccfa8b700 1 --2- 172.21.15.1:0/2250568686 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2cc80ff410 0x7f2cc81963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.529+0000 7f2ccfa8b700 1 --2- 172.21.15.1:0/2250568686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cc80ffdb0 0x7f2cc81968f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.537 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.529+0000 7f2ccfa8b700 1 -- 172.21.15.1:0/2250568686 >> 172.21.15.1:0/2250568686 conn(0x7f2cc80763a0 msgr2=0x7f2cc8102f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:40.538 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.529+0000 7f2ccfa8b700 1 -- 172.21.15.1:0/2250568686 shutdown_connections 2023-11-16T16:45:40.538 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.529+0000 7f2ccfa8b700 1 -- 172.21.15.1:0/2250568686 wait complete. 2023-11-16T16:45:40.542 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-16T16:45:40.542 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:45:40.677 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.675+0000 7f080382b700 1 -- 172.21.15.1:0/19428477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07fc100470 msgr2=0x7f07fc10c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:40.677 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.675+0000 7f080382b700 1 --2- 172.21.15.1:0/19428477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07fc100470 0x7f07fc10c040 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f07f400b0a0 tx=0x7f07f400b3b0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.677 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.676+0000 7f080382b700 1 -- 172.21.15.1:0/19428477 shutdown_connections 2023-11-16T16:45:40.677 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.676+0000 7f080382b700 1 --2- 172.21.15.1:0/19428477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07fc100470 0x7f07fc10c040 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.677 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.676+0000 7f080382b700 1 --2- 172.21.15.1:0/19428477 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07fc105210 0x7f07fc1055e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.677 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.676+0000 7f080382b700 1 --2- 172.21.15.1:0/19428477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07fc0ffac0 0x7f07fc0fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.677 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.676+0000 7f080382b700 1 -- 172.21.15.1:0/19428477 >> 172.21.15.1:0/19428477 conn(0x7f07fc076370 msgr2=0x7f07fc076770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:40.677 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.676+0000 7f080382b700 1 -- 172.21.15.1:0/19428477 shutdown_connections 2023-11-16T16:45:40.677 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.676+0000 7f080382b700 1 -- 172.21.15.1:0/19428477 wait complete. 2023-11-16T16:45:40.677 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.676+0000 7f080382b700 1 Processor -- start 2023-11-16T16:45:40.677 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.676+0000 7f080382b700 1 -- start start 2023-11-16T16:45:40.679 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.676+0000 7f080382b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07fc0ffac0 0x7f07fc1964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:40.680 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.677+0000 7f080382b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07fc100470 0x7f07fc196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:40.680 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.677+0000 7f080382b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07fc105210 0x7f07fc19ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:40.680 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.677+0000 7f080382b700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07fc19b4a0 con 0x7f07fc100470 2023-11-16T16:45:40.680 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.677+0000 7f080382b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07fc19b610 con 0x7f07fc0ffac0 2023-11-16T16:45:40.680 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.677+0000 7f080382b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07fc19b7d0 con 0x7f07fc105210 2023-11-16T16:45:40.680 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.677+0000 7f0800dc6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07fc100470 0x7f07fc196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:40.680 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.677+0000 7f08015c7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07fc0ffac0 0x7f07fc1964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:40.680 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.677+0000 7f08015c7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07fc0ffac0 0x7f07fc1964c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.1:39572/0 (socket says 172.21.15.1:39572) 2023-11-16T16:45:40.680 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.677+0000 7f08015c7700 1 -- 172.21.15.1:0/2340138405 learned_addr learned my addr 172.21.15.1:0/2340138405 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:45:40.680 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.677+0000 7f0800dc6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07fc100470 0x7f07fc196a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45388/0 (socket says 172.21.15.1:45388) 2023-11-16T16:45:40.681 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.677+0000 7f0801dc8700 1 --2- 172.21.15.1:0/2340138405 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07fc105210 0x7f07fc19ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:40.681 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.677+0000 7f0800dc6700 1 -- 172.21.15.1:0/2340138405 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07fc105210 msgr2=0x7f07fc19ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:40.681 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.677+0000 7f0800dc6700 1 --2- 172.21.15.1:0/2340138405 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07fc105210 0x7f07fc19ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.681 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.677+0000 7f0800dc6700 1 -- 172.21.15.1:0/2340138405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07fc0ffac0 msgr2=0x7f07fc1964c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:40.681 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.677+0000 7f0800dc6700 1 --2- 172.21.15.1:0/2340138405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07fc0ffac0 0x7f07fc1964c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.681 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.677+0000 7f0800dc6700 1 -- 172.21.15.1:0/2340138405 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f4009d00 con 0x7f07fc100470 2023-11-16T16:45:40.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.677+0000 7f0800dc6700 1 --2- 172.21.15.1:0/2340138405 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07fc100470 0x7f07fc196a00 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f07ec00e9f0 tx=0x7f07ec00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:40.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.677+0000 7f07f27fc700 1 -- 172.21.15.1:0/2340138405 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ec00c9c0 con 0x7f07fc100470 2023-11-16T16:45:40.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.677+0000 7f07f27fc700 1 -- 172.21.15.1:0/2340138405 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07ec010430 con 0x7f07fc100470 2023-11-16T16:45:40.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.677+0000 7f07f27fc700 1 -- 172.21.15.1:0/2340138405 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ec018680 con 0x7f07fc100470 2023-11-16T16:45:40.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.677+0000 7f080382b700 1 -- 172.21.15.1:0/2340138405 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07fc1103b0 con 0x7f07fc100470 2023-11-16T16:45:40.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.677+0000 7f080382b700 1 -- 172.21.15.1:0/2340138405 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07fc110220 con 0x7f07fc100470 2023-11-16T16:45:40.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.678+0000 7f080382b700 1 -- 172.21.15.1:0/2340138405 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07fc110540 con 0x7f07fc100470 2023-11-16T16:45:40.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.680+0000 7f07f27fc700 1 -- 172.21.15.1:0/2340138405 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f07ec010860 con 0x7f07fc100470 2023-11-16T16:45:40.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.680+0000 7f07f27fc700 1 --2- 172.21.15.1:0/2340138405 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f07e80a12c0 0x7f07e80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:40.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.680+0000 7f08015c7700 1 --2- 172.21.15.1:0/2340138405 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f07e80a12c0 0x7f07e80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:40.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.680+0000 7f07f27fc700 1 -- 172.21.15.1:0/2340138405 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f07ec014070 con 0x7f07fc100470 2023-11-16T16:45:40.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.681+0000 7f08015c7700 1 --2- 172.21.15.1:0/2340138405 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f07e80a12c0 0x7f07e80a3770 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f07f8005950 tx=0x7f07f8009260 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:40.684 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.684+0000 7f07f27fc700 1 -- 172.21.15.1:0/2340138405 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f07ec096b10 con 0x7f07fc100470 2023-11-16T16:45:40.930 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.930+0000 7f080382b700 1 -- 172.21.15.1:0/2340138405 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07fc067f40 con 0x7f07fc100470 2023-11-16T16:45:40.931 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.930+0000 7f07f27fc700 1 -- 172.21.15.1:0/2340138405 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10783 (secure 0 0 0) 0x7f07ec09a130 con 0x7f07fc100470 2023-11-16T16:45:40.931 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:45:40.931 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2587563684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2587563684},{"type":"v1","addr":"172.21.15.1:6835","nonce":2587563684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/277936168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":277936168},{"type":"v1","addr":"172.21.15.1:6837","nonce":277936168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/1172575045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1172575045},{"type":"v1","addr":"172.21.15.1:6839","nonce":1172575045}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/2631175899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2631175899},{"type":"v1","addr":"172.21.15.1:6841","nonce":2631175899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3693905267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3693905267},{"type":"v1","addr":"172.21.15.53:6833","nonce":3693905267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/164668033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":164668033},{"type":"v1","addr":"172.21.15.26:6833","nonce":164668033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-16T16:37:13.918943+0000","modified":"2023-11-16T16:43:40.628525+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2140542192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2140542192},{"type":"v1","addr":"172.21.15.53:6835","nonce":2140542192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1010760275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1010760275},{"type":"v1","addr":"172.21.15.26:6835","nonce":1010760275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3197060302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3197060302},{"type":"v1","addr":"172.21.15.53:6837","nonce":3197060302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6837/1453504863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1453504863},{"type":"v1","addr":"172.21.15.26:6837","nonce":1453504863}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.53:6839/1044753024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1044753024},{"type":"v1","addr":"172.21.15.53:6839","nonce":1044753024}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6839/2192205949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2192205949},{"type":"v1","addr":"172.21.15.26:6839","nonce":2192205949}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-16T16:45:40.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:40 smithi001 ceph-mon[98800]: scrub summary: idle 2023-11-16T16:45:40.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:40 smithi001 ceph-mon[98800]: pgmap v198: 97 pgs: 97 active+clean; 809 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.8 MiB/s wr, 183 op/s 2023-11-16T16:45:40.933 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.932+0000 7f080382b700 1 -- 172.21.15.1:0/2340138405 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f07e80a12c0 msgr2=0x7f07e80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:40.934 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.933+0000 7f080382b700 1 --2- 172.21.15.1:0/2340138405 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f07e80a12c0 0x7f07e80a3770 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f07f8005950 tx=0x7f07f8009260 comp rx=0 tx=0).stop 2023-11-16T16:45:40.934 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.933+0000 7f080382b700 1 -- 172.21.15.1:0/2340138405 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07fc100470 msgr2=0x7f07fc196a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:40.934 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.933+0000 7f080382b700 1 --2- 172.21.15.1:0/2340138405 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07fc100470 0x7f07fc196a00 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f07ec00e9f0 tx=0x7f07ec00ed00 comp rx=0 tx=0).stop 2023-11-16T16:45:40.934 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.933+0000 7f080382b700 1 -- 172.21.15.1:0/2340138405 shutdown_connections 2023-11-16T16:45:40.934 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.933+0000 7f080382b700 1 --2- 172.21.15.1:0/2340138405 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f07e80a12c0 0x7f07e80a3770 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.934 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.933+0000 7f080382b700 1 --2- 172.21.15.1:0/2340138405 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f07fc105210 0x7f07fc19ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.934 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.933+0000 7f080382b700 1 --2- 172.21.15.1:0/2340138405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07fc0ffac0 0x7f07fc1964c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.934 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.933+0000 7f080382b700 1 --2- 172.21.15.1:0/2340138405 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07fc100470 0x7f07fc196a00 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:40.934 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.933+0000 7f080382b700 1 -- 172.21.15.1:0/2340138405 >> 172.21.15.1:0/2340138405 conn(0x7f07fc076370 msgr2=0x7f07fc0fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:40.934 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.933+0000 7f080382b700 1 -- 172.21.15.1:0/2340138405 shutdown_connections 2023-11-16T16:45:40.934 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:40.933+0000 7f080382b700 1 -- 172.21.15.1:0/2340138405 wait complete. 2023-11-16T16:45:40.935 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 28 2023-11-16T16:45:40.946 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-11-16T16:45:41.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:40 smithi026 ceph-mon[100112]: scrub summary: idle 2023-11-16T16:45:41.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:40 smithi026 ceph-mon[100112]: pgmap v198: 97 pgs: 97 active+clean; 809 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.8 MiB/s wr, 183 op/s 2023-11-16T16:45:41.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.073+0000 7f0c63b88700 1 -- 172.21.15.1:0/4008040417 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c5c100420 msgr2=0x7f0c5c10bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:41.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.073+0000 7f0c63b88700 1 --2- 172.21.15.1:0/4008040417 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c5c100420 0x7f0c5c10bfd0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f0c5400b0a0 tx=0x7f0c5400b3b0 comp rx=0 tx=0).stop 2023-11-16T16:45:41.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.073+0000 7f0c63b88700 1 -- 172.21.15.1:0/4008040417 shutdown_connections 2023-11-16T16:45:41.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.073+0000 7f0c63b88700 1 --2- 172.21.15.1:0/4008040417 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c5c100420 0x7f0c5c10bfd0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:41.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.073+0000 7f0c63b88700 1 --2- 172.21.15.1:0/4008040417 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c5c105250 0x7f0c5c105620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:41.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.073+0000 7f0c63b88700 1 --2- 172.21.15.1:0/4008040417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c5c0ffa70 0x7f0c5c0ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:41.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.073+0000 7f0c63b88700 1 -- 172.21.15.1:0/4008040417 >> 172.21.15.1:0/4008040417 conn(0x7f0c5c0763a0 msgr2=0x7f0c5c0767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:41.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.073+0000 7f0c63b88700 1 -- 172.21.15.1:0/4008040417 shutdown_connections 2023-11-16T16:45:41.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.073+0000 7f0c63b88700 1 -- 172.21.15.1:0/4008040417 wait complete. 2023-11-16T16:45:41.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.074+0000 7f0c63b88700 1 Processor -- start 2023-11-16T16:45:41.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.074+0000 7f0c63b88700 1 -- start start 2023-11-16T16:45:41.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.074+0000 7f0c63b88700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c5c0ffa70 0x7f0c5c196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:41.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.074+0000 7f0c63b88700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c5c100420 0x7f0c5c1969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:41.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.074+0000 7f0c63b88700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c5c105250 0x7f0c5c19ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:41.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.074+0000 7f0c63b88700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c19b440 con 0x7f0c5c100420 2023-11-16T16:45:41.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.074+0000 7f0c63b88700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c19b5b0 con 0x7f0c5c105250 2023-11-16T16:45:41.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.074+0000 7f0c63b88700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c19b770 con 0x7f0c5c0ffa70 2023-11-16T16:45:41.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.074+0000 7f0c61123700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c5c100420 0x7f0c5c1969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:41.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.074+0000 7f0c61123700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c5c100420 0x7f0c5c1969a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45402/0 (socket says 172.21.15.1:45402) 2023-11-16T16:45:41.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.074+0000 7f0c61123700 1 -- 172.21.15.1:0/1220447549 learned_addr learned my addr 172.21.15.1:0/1220447549 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:45:41.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.074+0000 7f0c61924700 1 --2- 172.21.15.1:0/1220447549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c5c0ffa70 0x7f0c5c196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:41.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.074+0000 7f0c62125700 1 --2- 172.21.15.1:0/1220447549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c5c105250 0x7f0c5c19ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:41.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.074+0000 7f0c61123700 1 -- 172.21.15.1:0/1220447549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c5c0ffa70 msgr2=0x7f0c5c196460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:41.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.075+0000 7f0c61123700 1 --2- 172.21.15.1:0/1220447549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c5c0ffa70 0x7f0c5c196460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:41.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.075+0000 7f0c61123700 1 -- 172.21.15.1:0/1220447549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c5c105250 msgr2=0x7f0c5c19ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:41.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.075+0000 7f0c61123700 1 --2- 172.21.15.1:0/1220447549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c5c105250 0x7f0c5c19ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:41.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.075+0000 7f0c61123700 1 -- 172.21.15.1:0/1220447549 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c54009d00 con 0x7f0c5c100420 2023-11-16T16:45:41.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.075+0000 7f0c61123700 1 --2- 172.21.15.1:0/1220447549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c5c100420 0x7f0c5c1969a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f0c5800e9a0 tx=0x7f0c5800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:41.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.075+0000 7f0c52ffd700 1 -- 172.21.15.1:0/1220447549 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c58009790 con 0x7f0c5c100420 2023-11-16T16:45:41.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.075+0000 7f0c63b88700 1 -- 172.21.15.1:0/1220447549 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c5c10f230 con 0x7f0c5c100420 2023-11-16T16:45:41.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.075+0000 7f0c52ffd700 1 -- 172.21.15.1:0/1220447549 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c58010430 con 0x7f0c5c100420 2023-11-16T16:45:41.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.075+0000 7f0c52ffd700 1 -- 172.21.15.1:0/1220447549 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c58018680 con 0x7f0c5c100420 2023-11-16T16:45:41.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.075+0000 7f0c63b88700 1 -- 172.21.15.1:0/1220447549 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c5c10f0a0 con 0x7f0c5c100420 2023-11-16T16:45:41.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.078+0000 7f0c52ffd700 1 -- 172.21.15.1:0/1220447549 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f0c580105a0 con 0x7f0c5c100420 2023-11-16T16:45:41.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.078+0000 7f0c52ffd700 1 --2- 172.21.15.1:0/1220447549 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f0c480a12c0 0x7f0c480a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:41.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.078+0000 7f0c52ffd700 1 -- 172.21.15.1:0/1220447549 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f0c58014070 con 0x7f0c5c100420 2023-11-16T16:45:41.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.078+0000 7f0c61924700 1 --2- 172.21.15.1:0/1220447549 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f0c480a12c0 0x7f0c480a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:41.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.079+0000 7f0c61924700 1 --2- 172.21.15.1:0/1220447549 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f0c480a12c0 0x7f0c480a3770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c005950 tx=0x7f0c4c00f820 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:41.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.079+0000 7f0c6fd14b80 1 Processor -- start 2023-11-16T16:45:41.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.079+0000 7f0c6fd14b80 1 -- start start 2023-11-16T16:45:41.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.079+0000 7f0c6fd14b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc768b60 0x5615fc768f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:41.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.079+0000 7f0c6fd14b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fc769470 0x5615fc76d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:41.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.079+0000 7f0c6fd14b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fc76d8b0 0x5615fc76dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:41.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.079+0000 7f0c6fd14b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615fc66e060 con 0x5615fc768b60 2023-11-16T16:45:41.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.079+0000 7f0c6fd14b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615fc6c08a0 con 0x5615fc769470 2023-11-16T16:45:41.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.079+0000 7f0c6fd14b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615fc6b9d30 con 0x5615fc76d8b0 2023-11-16T16:45:41.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.079+0000 7f0c61924700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc768b60 0x5615fc768f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:41.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.079+0000 7f0c61924700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc768b60 0x5615fc768f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45408/0 (socket says 172.21.15.1:45408) 2023-11-16T16:45:41.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.079+0000 7f0c61924700 1 -- 172.21.15.1:0/2517919757 learned_addr learned my addr 172.21.15.1:0/2517919757 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:45:41.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.079+0000 7f0c61123700 1 --2- 172.21.15.1:0/2517919757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fc769470 0x5615fc76d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:41.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.080+0000 7f0c62125700 1 --2- 172.21.15.1:0/2517919757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fc76d8b0 0x5615fc76dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:41.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.080+0000 7f0c61924700 1 -- 172.21.15.1:0/2517919757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fc76d8b0 msgr2=0x5615fc76dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:41.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.080+0000 7f0c61924700 1 --2- 172.21.15.1:0/2517919757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fc76d8b0 0x5615fc76dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:41.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.080+0000 7f0c61924700 1 -- 172.21.15.1:0/2517919757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fc769470 msgr2=0x5615fc76d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:41.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.080+0000 7f0c61924700 1 --2- 172.21.15.1:0/2517919757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fc769470 0x5615fc76d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:41.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.080+0000 7f0c61924700 1 -- 172.21.15.1:0/2517919757 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615fc53b630 con 0x5615fc768b60 2023-11-16T16:45:41.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.080+0000 7f0c62125700 1 --2- 172.21.15.1:0/2517919757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fc76d8b0 0x5615fc76dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:45:41.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.080+0000 7f0c61924700 1 --2- 172.21.15.1:0/2517919757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc768b60 0x5615fc768f30 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c009fd0 tx=0x7f0c4c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b998ec7eea983f server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:41.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.080+0000 7f0c47fff700 1 -- 172.21.15.1:0/2517919757 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c01be80 con 0x5615fc768b60 2023-11-16T16:45:41.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.080+0000 7f0c47fff700 1 -- 172.21.15.1:0/2517919757 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c4c015040 con 0x5615fc768b60 2023-11-16T16:45:41.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.080+0000 7f0c47fff700 1 -- 172.21.15.1:0/2517919757 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c0196d0 con 0x5615fc768b60 2023-11-16T16:45:41.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.081+0000 7f0c6fd14b80 1 -- 172.21.15.1:0/2517919757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc768b60 msgr2=0x5615fc768f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:41.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.081+0000 7f0c6fd14b80 1 --2- 172.21.15.1:0/2517919757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc768b60 0x5615fc768f30 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c009fd0 tx=0x7f0c4c017000 comp rx=0 tx=0).stop 2023-11-16T16:45:41.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.081+0000 7f0c6fd14b80 1 -- 172.21.15.1:0/2517919757 shutdown_connections 2023-11-16T16:45:41.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.081+0000 7f0c6fd14b80 1 --2- 172.21.15.1:0/2517919757 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fc76d8b0 0x5615fc76dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:41.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.081+0000 7f0c6fd14b80 1 --2- 172.21.15.1:0/2517919757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc768b60 0x5615fc768f30 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:41.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.081+0000 7f0c6fd14b80 1 --2- 172.21.15.1:0/2517919757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fc769470 0x5615fc76d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:41.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.081+0000 7f0c6fd14b80 1 -- 172.21.15.1:0/2517919757 >> 172.21.15.1:0/2517919757 conn(0x5615fc75e360 msgr2=0x5615fc763b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:41.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.081+0000 7f0c6fd14b80 1 -- 172.21.15.1:0/2517919757 shutdown_connections 2023-11-16T16:45:41.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.081+0000 7f0c6fd14b80 1 -- 172.21.15.1:0/2517919757 wait complete. 2023-11-16T16:45:41.085 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.081+0000 7f0c6fd14b80 1 -- start start 2023-11-16T16:45:41.085 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.081+0000 7f0c6fd14b80 1 Processor -- start 2023-11-16T16:45:41.085 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:40 smithi053 ceph-mon[100638]: scrub summary: idle 2023-11-16T16:45:41.085 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:40 smithi053 ceph-mon[100638]: pgmap v198: 97 pgs: 97 active+clean; 809 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.8 MiB/s wr, 183 op/s 2023-11-16T16:45:41.088 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.082+0000 7f0c6fd14b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fc768b60 0x5615fc775030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:41.088 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.082+0000 7f0c6fd14b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fc769470 0x5615fc772320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:41.088 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.082+0000 7f0c6fd14b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc76d8b0 0x5615fc772980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:41.088 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.082+0000 7f0c6fd14b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615fc65b8e0 con 0x5615fc76d8b0 2023-11-16T16:45:41.088 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.082+0000 7f0c6fd14b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615fc65b580 con 0x5615fc768b60 2023-11-16T16:45:41.088 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.082+0000 7f0c6fd14b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615fc53d920 con 0x5615fc769470 2023-11-16T16:45:41.088 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.082+0000 7f0c6fd14b80 10 client.0 _ll_register_callbacks cb 0x5615fc679780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:45:41.088 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.082+0000 7f0c62125700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc76d8b0 0x5615fc772980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:41.088 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.082+0000 7f0c62125700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc76d8b0 0x5615fc772980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45418/0 (socket says 172.21.15.1:45418) 2023-11-16T16:45:41.088 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.082+0000 7f0c62125700 1 -- 172.21.15.1:0/4209041140 learned_addr learned my addr 172.21.15.1:0/4209041140 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:45:41.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.082+0000 7f0c61924700 1 --2- 172.21.15.1:0/4209041140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fc768b60 0x5615fc775030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:41.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.082+0000 7f0c61123700 1 --2- 172.21.15.1:0/4209041140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fc769470 0x5615fc772320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:41.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.082+0000 7f0c62125700 1 -- 172.21.15.1:0/4209041140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fc769470 msgr2=0x5615fc772320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:41.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.082+0000 7f0c62125700 1 --2- 172.21.15.1:0/4209041140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fc769470 0x5615fc772320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:41.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.082+0000 7f0c62125700 1 -- 172.21.15.1:0/4209041140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fc768b60 msgr2=0x5615fc775030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:41.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.082+0000 7f0c62125700 1 --2- 172.21.15.1:0/4209041140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fc768b60 0x5615fc775030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:41.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.082+0000 7f0c62125700 1 -- 172.21.15.1:0/4209041140 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c4c009d90 con 0x5615fc76d8b0 2023-11-16T16:45:41.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.082+0000 7f0c61123700 1 --2- 172.21.15.1:0/4209041140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fc769470 0x5615fc772320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:45:41.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.082+0000 7f0c62125700 1 --2- 172.21.15.1:0/4209041140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc76d8b0 0x5615fc772980 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f0c54015000 tx=0x7f0c54009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=622fe45ce3b50532 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:41.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.082+0000 7f0c457fa700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:45:41.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.083+0000 7f0c457fa700 1 -- 172.21.15.1:0/4209041140 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5400e040 con 0x5615fc76d8b0 2023-11-16T16:45:41.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.083+0000 7f0c6fd14b80 1 -- 172.21.15.1:0/4209041140 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5615fc75bb20 con 0x5615fc76d8b0 2023-11-16T16:45:41.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.083+0000 7f0c457fa700 1 -- 172.21.15.1:0/4209041140 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f0c540074e0 con 0x5615fc76d8b0 2023-11-16T16:45:41.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.083+0000 7f0c457fa700 1 -- 172.21.15.1:0/4209041140 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c540076a0 con 0x5615fc76d8b0 2023-11-16T16:45:41.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.083+0000 7f0c6fd14b80 10 client.16233 fetch_fsmap learned FSMap version 28 2023-11-16T16:45:41.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.083+0000 7f0c6fd14b80 1 -- 172.21.15.1:0/4209041140 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5615fc680190 con 0x5615fc76d8b0 2023-11-16T16:45:41.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.083+0000 7f0c457fa700 1 -- 172.21.15.1:0/4209041140 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5615fc75bb20 con 0x5615fc76d8b0 2023-11-16T16:45:41.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.083+0000 7f0c457fa700 1 -- 172.21.15.1:0/4209041140 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0c54022030 con 0x5615fc76d8b0 2023-11-16T16:45:41.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.083+0000 7f0c6fd14b80 10 client.16233 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:45:41.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.083+0000 7f0c6fd14b80 10 client.16233 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-11-16T16:45:41.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.083+0000 7f0c6fd14b80 20 client.16233 populate_metadata read hostname 'smithi001' 2023-11-16T16:45:41.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.083+0000 7f0c6fd14b80 1 --2- 172.21.15.1:0/4209041140 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x5615fc782870 0x5615fc784c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:41.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.083+0000 7f0c6fd14b80 4 client.16233 mds_command: new command op to 24482 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:45:41.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.083+0000 7f0c6fd14b80 1 -- 172.21.15.1:0/4209041140 --> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5615fc67c0a0 con 0x5615fc782870 2023-11-16T16:45:41.090 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.083+0000 7f0c61924700 1 --2- 172.21.15.1:0/4209041140 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x5615fc782870 0x5615fc784c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:41.091 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.084+0000 7f0c457fa700 10 client.16233 ms_handle_connect on v2:172.21.15.26:6836/1453504863 2023-11-16T16:45:41.091 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:41.084+0000 7f0c61924700 1 --2- 172.21.15.1:0/4209041140 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x5615fc782870 0x5615fc784c80 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c8d97ce3d9c9e2c5 server_cookie=84f1adfc99e6d466 in_seq=0 out_seq=0 2023-11-16T16:45:41.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:41 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/2340138405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:45:41.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:41 smithi001 ceph-mon[98800]: pgmap v199: 97 pgs: 97 active+clean; 805 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 244 op/s 2023-11-16T16:45:42.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:41 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/2340138405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:45:42.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:41 smithi026 ceph-mon[100112]: pgmap v199: 97 pgs: 97 active+clean; 805 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 244 op/s 2023-11-16T16:45:42.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:41 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/2340138405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:45:42.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:41 smithi053 ceph-mon[100638]: pgmap v199: 97 pgs: 97 active+clean; 805 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 244 op/s 2023-11-16T16:45:42.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.249+0000 7f0c457fa700 1 -- 172.21.15.1:0/4209041140 <== mds.2 v2:172.21.15.26:6836/1453504863 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5615fc67c0a0 con 0x5615fc782870 2023-11-16T16:45:42.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.249+0000 7f0c457fa700 10 client.16233 handle_command_reply: tid=0 2023-11-16T16:45:42.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.249+0000 7f0c6fd14b80 1 client.16233 shutdown 2023-11-16T16:45:42.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.250+0000 7f0c6fd14b80 1 -- 172.21.15.1:0/4209041140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc76d8b0 msgr2=0x5615fc772980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:42.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.250+0000 7f0c6fd14b80 1 --2- 172.21.15.1:0/4209041140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc76d8b0 0x5615fc772980 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f0c54015000 tx=0x7f0c54009590 comp rx=0 tx=0).stop 2023-11-16T16:45:42.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.250+0000 7f0c6fd14b80 1 -- 172.21.15.1:0/4209041140 shutdown_connections 2023-11-16T16:45:42.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.250+0000 7f0c6fd14b80 1 --2- 172.21.15.1:0/4209041140 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x5615fc782870 0x5615fc784c80 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.250+0000 7f0c6fd14b80 1 --2- 172.21.15.1:0/4209041140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc76d8b0 0x5615fc772980 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.250+0000 7f0c457fa700 0 client.16233 ms_handle_reset on v2:172.21.15.26:6836/1453504863 2023-11-16T16:45:42.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.250+0000 7f0c6fd14b80 1 --2- 172.21.15.1:0/4209041140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fc768b60 0x5615fc775030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.250+0000 7f0c6fd14b80 1 --2- 172.21.15.1:0/4209041140 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fc769470 0x5615fc772320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.250+0000 7f0c6fd14b80 1 -- 172.21.15.1:0/4209041140 >> 172.21.15.1:0/4209041140 conn(0x5615fc75e360 msgr2=0x5615fc762de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:42.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.250+0000 7f0c6fd14b80 1 -- 172.21.15.1:0/4209041140 shutdown_connections 2023-11-16T16:45:42.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.250+0000 7f0c6fd14b80 1 -- 172.21.15.1:0/4209041140 wait complete. 2023-11-16T16:45:42.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.250+0000 7f0c6fd14b80 20 client.16233 trim_cache size 0 max 16384 2023-11-16T16:45:42.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.264+0000 7f0c6fd14b80 1 Processor -- start 2023-11-16T16:45:42.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.265+0000 7f0c6fd14b80 1 -- start start 2023-11-16T16:45:42.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.265+0000 7f0c6fd14b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fc768b60 0x5615fc76c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.265+0000 7f0c6fd14b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc769470 0x5615fc7799e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.265+0000 7f0c6fd14b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fc76d8b0 0x5615fc76ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.265+0000 7f0c6fd14b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615fc775790 con 0x5615fc769470 2023-11-16T16:45:42.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.265+0000 7f0c6fd14b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615fc772d30 con 0x5615fc768b60 2023-11-16T16:45:42.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.265+0000 7f0c6fd14b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615fc769bc0 con 0x5615fc76d8b0 2023-11-16T16:45:42.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.265+0000 7f0c61123700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc769470 0x5615fc7799e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.265+0000 7f0c61924700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fc768b60 0x5615fc76c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.265+0000 7f0c61123700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc769470 0x5615fc7799e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45424/0 (socket says 172.21.15.1:45424) 2023-11-16T16:45:42.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.265+0000 7f0c61123700 1 -- 172.21.15.1:0/1502011641 learned_addr learned my addr 172.21.15.1:0/1502011641 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:45:42.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.265+0000 7f0c61924700 1 --2- 172.21.15.1:0/1502011641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fc768b60 0x5615fc76c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.1:39608/0 (socket says 172.21.15.1:39608) 2023-11-16T16:45:42.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.265+0000 7f0c62125700 1 --2- 172.21.15.1:0/1502011641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fc76d8b0 0x5615fc76ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.266+0000 7f0c61924700 1 -- 172.21.15.1:0/1502011641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fc76d8b0 msgr2=0x5615fc76ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:42.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.266+0000 7f0c61924700 1 --2- 172.21.15.1:0/1502011641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fc76d8b0 0x5615fc76ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.266+0000 7f0c61924700 1 -- 172.21.15.1:0/1502011641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc769470 msgr2=0x5615fc7799e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:42.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.266+0000 7f0c61924700 1 --2- 172.21.15.1:0/1502011641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc769470 0x5615fc7799e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.266+0000 7f0c61924700 1 -- 172.21.15.1:0/1502011641 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615fc76df50 con 0x5615fc768b60 2023-11-16T16:45:42.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.266+0000 7f0c61123700 1 --2- 172.21.15.1:0/1502011641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc769470 0x5615fc7799e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:45:42.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.266+0000 7f0c61924700 1 --2- 172.21.15.1:0/1502011641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fc768b60 0x5615fc76c1a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c009a10 tx=0x7f0c4c009a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25b4e773f8bbed82 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:42.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.266+0000 7f0c50dac700 1 -- 172.21.15.1:0/1502011641 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c026600 con 0x5615fc768b60 2023-11-16T16:45:42.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.266+0000 7f0c50dac700 1 -- 172.21.15.1:0/1502011641 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c4c026d10 con 0x5615fc768b60 2023-11-16T16:45:42.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.266+0000 7f0c50dac700 1 -- 172.21.15.1:0/1502011641 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c0048f0 con 0x5615fc768b60 2023-11-16T16:45:42.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.267+0000 7f0c6fd14b80 1 -- 172.21.15.1:0/1502011641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fc768b60 msgr2=0x5615fc76c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:42.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.267+0000 7f0c6fd14b80 1 --2- 172.21.15.1:0/1502011641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fc768b60 0x5615fc76c1a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c009a10 tx=0x7f0c4c009a40 comp rx=0 tx=0).stop 2023-11-16T16:45:42.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.267+0000 7f0c6fd14b80 1 -- 172.21.15.1:0/1502011641 shutdown_connections 2023-11-16T16:45:42.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.267+0000 7f0c6fd14b80 1 --2- 172.21.15.1:0/1502011641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fc76d8b0 0x5615fc76ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.267+0000 7f0c6fd14b80 1 --2- 172.21.15.1:0/1502011641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fc768b60 0x5615fc76c1a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.267+0000 7f0c6fd14b80 1 --2- 172.21.15.1:0/1502011641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc769470 0x5615fc7799e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.267+0000 7f0c6fd14b80 1 -- 172.21.15.1:0/1502011641 >> 172.21.15.1:0/1502011641 conn(0x5615fc75e360 msgr2=0x5615fc77ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:42.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.267+0000 7f0c6fd14b80 1 -- 172.21.15.1:0/1502011641 shutdown_connections 2023-11-16T16:45:42.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.267+0000 7f0c6fd14b80 1 -- 172.21.15.1:0/1502011641 wait complete. 2023-11-16T16:45:42.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.267+0000 7f0c6fd14b80 1 -- start start 2023-11-16T16:45:42.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.268+0000 7f0c6fd14b80 1 Processor -- start 2023-11-16T16:45:42.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.268+0000 7f0c6fd14b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fc768b60 0x5615fc7772b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.268+0000 7f0c6fd14b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc769470 0x5615fc7777f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.268+0000 7f0c6fd14b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fc76d8b0 0x5615fc777d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.268+0000 7f0c6fd14b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615fc7d58b0 con 0x5615fc769470 2023-11-16T16:45:42.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.268+0000 7f0c6fd14b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615fc7d5a20 con 0x5615fc76d8b0 2023-11-16T16:45:42.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.268+0000 7f0c6fd14b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615fc7d5be0 con 0x5615fc768b60 2023-11-16T16:45:42.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.268+0000 7f0c6fd14b80 10 client.0 _ll_register_callbacks cb 0x7f0c5400bdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:45:42.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.268+0000 7f0c61123700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc769470 0x5615fc7777f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.268+0000 7f0c62125700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fc76d8b0 0x5615fc777d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.268+0000 7f0c61123700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc769470 0x5615fc7777f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45430/0 (socket says 172.21.15.1:45430) 2023-11-16T16:45:42.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.268+0000 7f0c61123700 1 -- 172.21.15.1:0/166961616 learned_addr learned my addr 172.21.15.1:0/166961616 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:45:42.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.269+0000 7f0c61924700 1 --2- 172.21.15.1:0/166961616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fc768b60 0x5615fc7772b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.269+0000 7f0c62125700 1 -- 172.21.15.1:0/166961616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fc768b60 msgr2=0x5615fc7772b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:42.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.269+0000 7f0c62125700 1 --2- 172.21.15.1:0/166961616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fc768b60 0x5615fc7772b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.269+0000 7f0c62125700 1 -- 172.21.15.1:0/166961616 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc769470 msgr2=0x5615fc7777f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:42.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.269+0000 7f0c62125700 1 --2- 172.21.15.1:0/166961616 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc769470 0x5615fc7777f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.269+0000 7f0c62125700 1 -- 172.21.15.1:0/166961616 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c5c10f0a0 con 0x5615fc76d8b0 2023-11-16T16:45:42.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.269+0000 7f0c61123700 1 --2- 172.21.15.1:0/166961616 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc769470 0x5615fc7777f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:45:42.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.269+0000 7f0c62125700 1 --2- 172.21.15.1:0/166961616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fc76d8b0 0x5615fc777d30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0c54005b40 tx=0x7f0c54003a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0d9ab18017b7f88 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:42.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.269+0000 7f0c457fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-11-16T16:45:42.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.269+0000 7f0c457fa700 1 -- 172.21.15.1:0/166961616 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5400e040 con 0x5615fc76d8b0 2023-11-16T16:45:42.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.269+0000 7f0c457fa700 1 -- 172.21.15.1:0/166961616 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f0c540078b0 con 0x5615fc76d8b0 2023-11-16T16:45:42.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.269+0000 7f0c6fd14b80 1 -- 172.21.15.1:0/166961616 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0c4c015040 con 0x5615fc76d8b0 2023-11-16T16:45:42.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.270+0000 7f0c457fa700 1 -- 172.21.15.1:0/166961616 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c5400b7c0 con 0x5615fc76d8b0 2023-11-16T16:45:42.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.270+0000 7f0c6fd14b80 10 client.25799 fetch_fsmap learned FSMap version 28 2023-11-16T16:45:42.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.270+0000 7f0c6fd14b80 1 -- 172.21.15.1:0/166961616 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5615fc53b630 con 0x5615fc76d8b0 2023-11-16T16:45:42.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.270+0000 7f0c457fa700 1 -- 172.21.15.1:0/166961616 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c5403c080 con 0x5615fc76d8b0 2023-11-16T16:45:42.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.270+0000 7f0c457fa700 1 -- 172.21.15.1:0/166961616 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0c54019040 con 0x5615fc76d8b0 2023-11-16T16:45:42.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.270+0000 7f0c6fd14b80 10 client.25799 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:45:42.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.270+0000 7f0c6fd14b80 10 client.25799 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-11-16T16:45:42.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.270+0000 7f0c6fd14b80 20 client.25799 populate_metadata read hostname 'smithi001' 2023-11-16T16:45:42.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.270+0000 7f0c6fd14b80 1 --2- 172.21.15.1:0/166961616 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x5615fc7d6ff0 0x5615fc7d9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.270+0000 7f0c6fd14b80 4 client.25799 mds_command: new command op to 24482 tid=0[{"prefix": "damage ls"}] 2023-11-16T16:45:42.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.270+0000 7f0c6fd14b80 1 -- 172.21.15.1:0/166961616 --> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5615fc688660 con 0x5615fc7d6ff0 2023-11-16T16:45:42.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.270+0000 7f0c61924700 1 --2- 172.21.15.1:0/166961616 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x5615fc7d6ff0 0x5615fc7d9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.271+0000 7f0c457fa700 10 client.25799 ms_handle_connect on v2:172.21.15.26:6836/1453504863 2023-11-16T16:45:42.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.271+0000 7f0c61924700 1 --2- 172.21.15.1:0/166961616 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x5615fc7d6ff0 0x5615fc7d9400 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c36e0cf14e72699d server_cookie=d8388586ecae1c4d in_seq=0 out_seq=0 2023-11-16T16:45:42.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.282+0000 7f0c457fa700 1 -- 172.21.15.1:0/166961616 <== mds.2 v2:172.21.15.26:6836/1453504863 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5615fc688660 con 0x5615fc7d6ff0 2023-11-16T16:45:42.283 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.282+0000 7f0c457fa700 10 client.25799 handle_command_reply: tid=0 2023-11-16T16:45:42.283 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.282+0000 7f0c6fd14b80 1 client.25799 shutdown 2023-11-16T16:45:42.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.282+0000 7f0c6fd14b80 1 -- 172.21.15.1:0/166961616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fc76d8b0 msgr2=0x5615fc777d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:42.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.282+0000 7f0c6fd14b80 1 --2- 172.21.15.1:0/166961616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fc76d8b0 0x5615fc777d30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0c54005b40 tx=0x7f0c54003a70 comp rx=0 tx=0).stop 2023-11-16T16:45:42.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.282+0000 7f0c6fd14b80 1 -- 172.21.15.1:0/166961616 shutdown_connections 2023-11-16T16:45:42.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.282+0000 7f0c6fd14b80 1 --2- 172.21.15.1:0/166961616 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x5615fc7d6ff0 0x5615fc7d9400 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.282+0000 7f0c6fd14b80 1 --2- 172.21.15.1:0/166961616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fc76d8b0 0x5615fc777d30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.282+0000 7f0c6fd14b80 1 --2- 172.21.15.1:0/166961616 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fc768b60 0x5615fc7772b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.282+0000 7f0c6fd14b80 1 --2- 172.21.15.1:0/166961616 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fc769470 0x5615fc7777f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.282+0000 7f0c457fa700 0 client.25799 ms_handle_reset on v2:172.21.15.26:6836/1453504863 2023-11-16T16:45:42.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.282+0000 7f0c6fd14b80 1 -- 172.21.15.1:0/166961616 >> 172.21.15.1:0/166961616 conn(0x5615fc75e360 msgr2=0x5615fc76ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:42.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.283+0000 7f0c6fd14b80 1 -- 172.21.15.1:0/166961616 shutdown_connections 2023-11-16T16:45:42.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.283+0000 7f0c6fd14b80 1 -- 172.21.15.1:0/166961616 wait complete. 2023-11-16T16:45:42.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.283+0000 7f0c6fd14b80 20 client.25799 trim_cache size 0 max 16384 2023-11-16T16:45:42.285 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2023-11-16T16:45:42.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.283+0000 7f0c63b88700 1 -- 172.21.15.1:0/1220447549 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f0c480a12c0 msgr2=0x7f0c480a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:42.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.283+0000 7f0c63b88700 1 --2- 172.21.15.1:0/1220447549 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f0c480a12c0 0x7f0c480a3770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c005950 tx=0x7f0c4c00f820 comp rx=0 tx=0).stop 2023-11-16T16:45:42.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.283+0000 7f0c63b88700 1 -- 172.21.15.1:0/1220447549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c5c100420 msgr2=0x7f0c5c1969a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:42.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.283+0000 7f0c63b88700 1 --2- 172.21.15.1:0/1220447549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c5c100420 0x7f0c5c1969a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f0c5800e9a0 tx=0x7f0c5800ecb0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.283+0000 7f0c63b88700 1 -- 172.21.15.1:0/1220447549 shutdown_connections 2023-11-16T16:45:42.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.283+0000 7f0c63b88700 1 --2- 172.21.15.1:0/1220447549 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f0c480a12c0 0x7f0c480a3770 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.283+0000 7f0c63b88700 1 --2- 172.21.15.1:0/1220447549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c5c105250 0x7f0c5c19ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.283+0000 7f0c63b88700 1 --2- 172.21.15.1:0/1220447549 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0c5c0ffa70 0x7f0c5c196460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.283+0000 7f0c63b88700 1 --2- 172.21.15.1:0/1220447549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c5c100420 0x7f0c5c1969a0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.283+0000 7f0c63b88700 1 -- 172.21.15.1:0/1220447549 >> 172.21.15.1:0/1220447549 conn(0x7f0c5c0763a0 msgr2=0x7f0c5c0feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:42.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.284+0000 7f0c63b88700 1 -- 172.21.15.1:0/1220447549 shutdown_connections 2023-11-16T16:45:42.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.284+0000 7f0c63b88700 1 -- 172.21.15.1:0/1220447549 wait complete. 2023-11-16T16:45:42.300 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-11-16T16:45:42.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.436+0000 7f2592551700 1 -- 172.21.15.1:0/4059165438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f258c0ff410 msgr2=0x7f258c0ff7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:42.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.436+0000 7f2592551700 1 --2- 172.21.15.1:0/4059165438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f258c0ff410 0x7f258c0ff7e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f257c0099c0 tx=0x7f257c009cd0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.438 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.436+0000 7f2592551700 1 -- 172.21.15.1:0/4059165438 shutdown_connections 2023-11-16T16:45:42.438 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.436+0000 7f2592551700 1 --2- 172.21.15.1:0/4059165438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f258c104450 0x7f258c10c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.438 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.436+0000 7f2592551700 1 --2- 172.21.15.1:0/4059165438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f258c0ff410 0x7f258c0ff7e0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.438 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.436+0000 7f2592551700 1 --2- 172.21.15.1:0/4059165438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f258c0ffdb0 0x7f258c103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.438 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.436+0000 7f2592551700 1 -- 172.21.15.1:0/4059165438 >> 172.21.15.1:0/4059165438 conn(0x7f258c0763a0 msgr2=0x7f258c0767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:42.438 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.436+0000 7f2592551700 1 -- 172.21.15.1:0/4059165438 shutdown_connections 2023-11-16T16:45:42.438 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.436+0000 7f2592551700 1 -- 172.21.15.1:0/4059165438 wait complete. 2023-11-16T16:45:42.438 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.437+0000 7f2592551700 1 Processor -- start 2023-11-16T16:45:42.438 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.437+0000 7f2592551700 1 -- start start 2023-11-16T16:45:42.440 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.437+0000 7f2592551700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f258c0ff410 0x7f258c195250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.440 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.437+0000 7f2592551700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f258c0ffdb0 0x7f258c195790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.440 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.437+0000 7f2592551700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f258c104450 0x7f258c199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.440 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.437+0000 7f2592551700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258c19a230 con 0x7f258c104450 2023-11-16T16:45:42.440 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.437+0000 7f2592551700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258c19a3a0 con 0x7f258c0ffdb0 2023-11-16T16:45:42.440 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.437+0000 7f2592551700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258c19a560 con 0x7f258c0ff410 2023-11-16T16:45:42.440 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.437+0000 7f2590aee700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f258c104450 0x7f258c199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.440 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.437+0000 7f2590aee700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f258c104450 0x7f258c199bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45446/0 (socket says 172.21.15.1:45446) 2023-11-16T16:45:42.441 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.437+0000 7f2590aee700 1 -- 172.21.15.1:0/3292625044 learned_addr learned my addr 172.21.15.1:0/3292625044 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:45:42.441 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.437+0000 7f258bfff700 1 --2- 172.21.15.1:0/3292625044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f258c0ff410 0x7f258c195250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.441 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.437+0000 7f258b7fe700 1 --2- 172.21.15.1:0/3292625044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f258c0ffdb0 0x7f258c195790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.441 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.438+0000 7f2590aee700 1 -- 172.21.15.1:0/3292625044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f258c0ff410 msgr2=0x7f258c195250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:42.441 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.438+0000 7f2590aee700 1 --2- 172.21.15.1:0/3292625044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f258c0ff410 0x7f258c195250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.441 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.438+0000 7f2590aee700 1 -- 172.21.15.1:0/3292625044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f258c0ffdb0 msgr2=0x7f258c195790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:42.441 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.438+0000 7f2590aee700 1 --2- 172.21.15.1:0/3292625044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f258c0ffdb0 0x7f258c195790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.441 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.438+0000 7f2590aee700 1 -- 172.21.15.1:0/3292625044 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f257c0095f0 con 0x7f258c104450 2023-11-16T16:45:42.441 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.438+0000 7f2590aee700 1 --2- 172.21.15.1:0/3292625044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f258c104450 0x7f258c199bf0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f258400ef30 tx=0x7f258400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:42.441 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.438+0000 7f25897fa700 1 -- 172.21.15.1:0/3292625044 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2584018070 con 0x7f258c104450 2023-11-16T16:45:42.441 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.438+0000 7f25897fa700 1 -- 172.21.15.1:0/3292625044 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f258400f040 con 0x7f258c104450 2023-11-16T16:45:42.441 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.438+0000 7f2592551700 1 -- 172.21.15.1:0/3292625044 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f258c10f210 con 0x7f258c104450 2023-11-16T16:45:42.442 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.438+0000 7f25897fa700 1 -- 172.21.15.1:0/3292625044 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258400cd00 con 0x7f258c104450 2023-11-16T16:45:42.442 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.438+0000 7f2592551700 1 -- 172.21.15.1:0/3292625044 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f258c10f080 con 0x7f258c104450 2023-11-16T16:45:42.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.440+0000 7f25897fa700 1 -- 172.21.15.1:0/3292625044 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f2584003a00 con 0x7f258c104450 2023-11-16T16:45:42.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.441+0000 7f25897fa700 1 --2- 172.21.15.1:0/3292625044 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f25780a12c0 0x7f25780a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.441+0000 7f258bfff700 1 --2- 172.21.15.1:0/3292625044 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f25780a12c0 0x7f25780a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.441+0000 7f25897fa700 1 -- 172.21.15.1:0/3292625044 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f25840cc7e0 con 0x7f258c104450 2023-11-16T16:45:42.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.441+0000 7f258bfff700 1 --2- 172.21.15.1:0/3292625044 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f25780a12c0 0x7f25780a3770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f257c009990 tx=0x7f257c005fb0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:42.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.442+0000 7f259e6ddb80 1 Processor -- start 2023-11-16T16:45:42.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.442+0000 7f259e6ddb80 1 -- start start 2023-11-16T16:45:42.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.442+0000 7f259e6ddb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5f30b9f70 0x55e5f30ba340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.442+0000 7f259e6ddb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5f30ba880 0x55e5f30be780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.442+0000 7f259e6ddb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5f30becc0 0x55e5f30bf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.442+0000 7f259e6ddb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f2fbb060 con 0x55e5f30b9f70 2023-11-16T16:45:42.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.442+0000 7f259e6ddb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f300d8a0 con 0x55e5f30becc0 2023-11-16T16:45:42.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.442+0000 7f259e6ddb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f3006d30 con 0x55e5f30ba880 2023-11-16T16:45:42.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.442+0000 7f258bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5f30b9f70 0x55e5f30ba340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.442+0000 7f258bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5f30b9f70 0x55e5f30ba340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45450/0 (socket says 172.21.15.1:45450) 2023-11-16T16:45:42.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.442+0000 7f258bfff700 1 -- 172.21.15.1:0/3456179750 learned_addr learned my addr 172.21.15.1:0/3456179750 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:45:42.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.442+0000 7f2590aee700 1 --2- 172.21.15.1:0/3456179750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5f30becc0 0x55e5f30bf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.442+0000 7f258b7fe700 1 --2- 172.21.15.1:0/3456179750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5f30ba880 0x55e5f30be780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.443+0000 7f2590aee700 1 -- 172.21.15.1:0/3456179750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5f30ba880 msgr2=0x55e5f30be780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:42.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.443+0000 7f2590aee700 1 --2- 172.21.15.1:0/3456179750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5f30ba880 0x55e5f30be780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.443+0000 7f2590aee700 1 -- 172.21.15.1:0/3456179750 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5f30b9f70 msgr2=0x55e5f30ba340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:42.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.443+0000 7f2590aee700 1 --2- 172.21.15.1:0/3456179750 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5f30b9f70 0x55e5f30ba340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.443+0000 7f2590aee700 1 -- 172.21.15.1:0/3456179750 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5f2e88630 con 0x55e5f30becc0 2023-11-16T16:45:42.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.443+0000 7f258bfff700 1 --2- 172.21.15.1:0/3456179750 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5f30b9f70 0x55e5f30ba340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:45:42.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.443+0000 7f2590aee700 1 --2- 172.21.15.1:0/3456179750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5f30becc0 0x55e5f30bf110 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f25840c9ee0 tx=0x7f2584099430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5966a969605f574c server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:42.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.443+0000 7f25827fc700 1 -- 172.21.15.1:0/3456179750 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258400ce60 con 0x55e5f30becc0 2023-11-16T16:45:42.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.443+0000 7f25827fc700 1 -- 172.21.15.1:0/3456179750 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25840b7e90 con 0x55e5f30becc0 2023-11-16T16:45:42.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.443+0000 7f25827fc700 1 -- 172.21.15.1:0/3456179750 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2584099b50 con 0x55e5f30becc0 2023-11-16T16:45:42.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.444+0000 7f259e6ddb80 1 -- 172.21.15.1:0/3456179750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5f30becc0 msgr2=0x55e5f30bf110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:42.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.444+0000 7f259e6ddb80 1 --2- 172.21.15.1:0/3456179750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5f30becc0 0x55e5f30bf110 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f25840c9ee0 tx=0x7f2584099430 comp rx=0 tx=0).stop 2023-11-16T16:45:42.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.444+0000 7f259e6ddb80 1 -- 172.21.15.1:0/3456179750 shutdown_connections 2023-11-16T16:45:42.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.444+0000 7f259e6ddb80 1 --2- 172.21.15.1:0/3456179750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5f30becc0 0x55e5f30bf110 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.444+0000 7f259e6ddb80 1 --2- 172.21.15.1:0/3456179750 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5f30b9f70 0x55e5f30ba340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.444+0000 7f259e6ddb80 1 --2- 172.21.15.1:0/3456179750 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5f30ba880 0x55e5f30be780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.444+0000 7f259e6ddb80 1 -- 172.21.15.1:0/3456179750 >> 172.21.15.1:0/3456179750 conn(0x55e5f30ab360 msgr2=0x55e5f30b0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:42.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.444+0000 7f259e6ddb80 1 -- 172.21.15.1:0/3456179750 shutdown_connections 2023-11-16T16:45:42.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.444+0000 7f259e6ddb80 1 -- 172.21.15.1:0/3456179750 wait complete. 2023-11-16T16:45:42.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.444+0000 7f259e6ddb80 1 -- start start 2023-11-16T16:45:42.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.444+0000 7f259e6ddb80 1 Processor -- start 2023-11-16T16:45:42.451 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.445+0000 7f259e6ddb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5f30b9f70 0x55e5f30c3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.451 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.445+0000 7f259e6ddb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5f30ba880 0x55e5f30c3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.451 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.445+0000 7f259e6ddb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5f30becc0 0x55e5f30c4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.451 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.445+0000 7f259e6ddb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f2fa88e0 con 0x55e5f30b9f70 2023-11-16T16:45:42.451 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.445+0000 7f259e6ddb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f2fa8580 con 0x55e5f30becc0 2023-11-16T16:45:42.451 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.445+0000 7f259e6ddb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f2e8a920 con 0x55e5f30ba880 2023-11-16T16:45:42.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.445+0000 7f259e6ddb80 10 client.0 _ll_register_callbacks cb 0x55e5f2fc6780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:45:42.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.445+0000 7f258bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5f30b9f70 0x55e5f30c3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.445+0000 7f258bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5f30b9f70 0x55e5f30c3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45456/0 (socket says 172.21.15.1:45456) 2023-11-16T16:45:42.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.445+0000 7f258bfff700 1 -- 172.21.15.1:0/110571254 learned_addr learned my addr 172.21.15.1:0/110571254 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:45:42.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.445+0000 7f2590aee700 1 --2- 172.21.15.1:0/110571254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5f30becc0 0x55e5f30c4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.445+0000 7f258b7fe700 1 --2- 172.21.15.1:0/110571254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5f30ba880 0x55e5f30c3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.445+0000 7f258bfff700 1 -- 172.21.15.1:0/110571254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5f30ba880 msgr2=0x55e5f30c3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:42.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.445+0000 7f258bfff700 1 --2- 172.21.15.1:0/110571254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5f30ba880 0x55e5f30c3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.445+0000 7f258bfff700 1 -- 172.21.15.1:0/110571254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5f30becc0 msgr2=0x55e5f30c4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:42.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.445+0000 7f258bfff700 1 --2- 172.21.15.1:0/110571254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5f30becc0 0x55e5f30c4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.445+0000 7f258bfff700 1 -- 172.21.15.1:0/110571254 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f258c10f080 con 0x55e5f30b9f70 2023-11-16T16:45:42.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.445+0000 7f258bfff700 1 --2- 172.21.15.1:0/110571254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5f30b9f70 0x55e5f30c3790 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f257c00b580 tx=0x7f257c009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48b8162b3891449 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:42.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.445+0000 7f256b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:45:42.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.445+0000 7f256b7fe700 1 -- 172.21.15.1:0/110571254 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c02c040 con 0x55e5f30b9f70 2023-11-16T16:45:42.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.445+0000 7f256b7fe700 1 -- 172.21.15.1:0/110571254 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f257c004460 con 0x55e5f30b9f70 2023-11-16T16:45:42.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.445+0000 7f259e6ddb80 1 -- 172.21.15.1:0/110571254 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e5f30a8b20 con 0x55e5f30b9f70 2023-11-16T16:45:42.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.446+0000 7f256b7fe700 1 -- 172.21.15.1:0/110571254 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f257c004620 con 0x55e5f30b9f70 2023-11-16T16:45:42.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.446+0000 7f259e6ddb80 10 client.16251 fetch_fsmap learned FSMap version 28 2023-11-16T16:45:42.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.446+0000 7f259e6ddb80 1 -- 172.21.15.1:0/110571254 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e5f2fcd190 con 0x55e5f30b9f70 2023-11-16T16:45:42.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.446+0000 7f256b7fe700 1 -- 172.21.15.1:0/110571254 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f257c03d020 con 0x55e5f30b9f70 2023-11-16T16:45:42.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.446+0000 7f256b7fe700 1 -- 172.21.15.1:0/110571254 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f257c026430 con 0x55e5f30b9f70 2023-11-16T16:45:42.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.446+0000 7f259e6ddb80 10 client.16251 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:45:42.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.446+0000 7f259e6ddb80 10 client.16251 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-11-16T16:45:42.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.446+0000 7f259e6ddb80 20 client.16251 populate_metadata read hostname 'smithi001' 2023-11-16T16:45:42.453 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.446+0000 7f259e6ddb80 1 --2- 172.21.15.1:0/110571254 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55e5f30d3e00 0x55e5f30d6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.446+0000 7f259e6ddb80 4 client.16251 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:45:42.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.446+0000 7f259e6ddb80 1 -- 172.21.15.1:0/110571254 --> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e5f2fc90a0 con 0x55e5f30d3e00 2023-11-16T16:45:42.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.446+0000 7f258b7fe700 1 --2- 172.21.15.1:0/110571254 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55e5f30d3e00 0x55e5f30d6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.447+0000 7f258b7fe700 1 --2- 172.21.15.1:0/110571254 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55e5f30d3e00 0x55e5f30d6210 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1402806e1071bff9 server_cookie=5a9ee162ff44332b in_seq=0 out_seq=0 2023-11-16T16:45:42.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.447+0000 7f256b7fe700 10 client.16251 ms_handle_connect on v2:172.21.15.53:6838/1044753024 2023-11-16T16:45:42.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.447+0000 7f256b7fe700 1 -- 172.21.15.1:0/110571254 <== mds.1 v2:172.21.15.53:6838/1044753024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e5f2fc90a0 con 0x55e5f30d3e00 2023-11-16T16:45:42.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.447+0000 7f256b7fe700 10 client.16251 handle_command_reply: tid=0 2023-11-16T16:45:42.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.447+0000 7f259e6ddb80 1 client.16251 shutdown 2023-11-16T16:45:42.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.448+0000 7f259e6ddb80 1 -- 172.21.15.1:0/110571254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5f30b9f70 msgr2=0x55e5f30c3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:42.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.448+0000 7f259e6ddb80 1 --2- 172.21.15.1:0/110571254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5f30b9f70 0x55e5f30c3790 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f257c00b580 tx=0x7f257c009cd0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.448+0000 7f259e6ddb80 1 -- 172.21.15.1:0/110571254 shutdown_connections 2023-11-16T16:45:42.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.448+0000 7f259e6ddb80 1 --2- 172.21.15.1:0/110571254 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55e5f30d3e00 0x55e5f30d6210 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.448+0000 7f259e6ddb80 1 --2- 172.21.15.1:0/110571254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5f30becc0 0x55e5f30c4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.448+0000 7f259e6ddb80 1 --2- 172.21.15.1:0/110571254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5f30b9f70 0x55e5f30c3790 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.448+0000 7f256b7fe700 0 client.16251 ms_handle_reset on v2:172.21.15.53:6838/1044753024 2023-11-16T16:45:42.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.448+0000 7f259e6ddb80 1 --2- 172.21.15.1:0/110571254 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5f30ba880 0x55e5f30c3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.448+0000 7f259e6ddb80 1 -- 172.21.15.1:0/110571254 >> 172.21.15.1:0/110571254 conn(0x55e5f30ab360 msgr2=0x55e5f30c0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:42.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.448+0000 7f259e6ddb80 1 -- 172.21.15.1:0/110571254 shutdown_connections 2023-11-16T16:45:42.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.448+0000 7f259e6ddb80 1 -- 172.21.15.1:0/110571254 wait complete. 2023-11-16T16:45:42.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.448+0000 7f259e6ddb80 20 client.16251 trim_cache size 0 max 16384 2023-11-16T16:45:42.464 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.461+0000 7f259e6ddb80 1 Processor -- start 2023-11-16T16:45:42.464 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.461+0000 7f259e6ddb80 1 -- start start 2023-11-16T16:45:42.464 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.461+0000 7f259e6ddb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5f30ab360 0x55e5f30bc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.464 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.461+0000 7f259e6ddb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5f30b9f70 0x55e5f30cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.464 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.461+0000 7f259e6ddb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5f30ba880 0x55e5f30be450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.464 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.461+0000 7f259e6ddb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f30c1cf0 con 0x55e5f30ab360 2023-11-16T16:45:42.464 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.461+0000 7f259e6ddb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f30bd930 con 0x55e5f30b9f70 2023-11-16T16:45:42.464 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.461+0000 7f259e6ddb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f30bad10 con 0x55e5f30ba880 2023-11-16T16:45:42.464 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.461+0000 7f258bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5f30ab360 0x55e5f30bc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.464 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.461+0000 7f258bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5f30ab360 0x55e5f30bc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45464/0 (socket says 172.21.15.1:45464) 2023-11-16T16:45:42.464 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.461+0000 7f258bfff700 1 -- 172.21.15.1:0/3061676776 learned_addr learned my addr 172.21.15.1:0/3061676776 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:45:42.464 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.461+0000 7f258b7fe700 1 --2- 172.21.15.1:0/3061676776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5f30b9f70 0x55e5f30cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.461+0000 7f2590aee700 1 --2- 172.21.15.1:0/3061676776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5f30ba880 0x55e5f30be450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.462+0000 7f258bfff700 1 -- 172.21.15.1:0/3061676776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5f30ba880 msgr2=0x55e5f30be450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:42.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.462+0000 7f258bfff700 1 --2- 172.21.15.1:0/3061676776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5f30ba880 0x55e5f30be450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.462+0000 7f258bfff700 1 -- 172.21.15.1:0/3061676776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5f30b9f70 msgr2=0x55e5f30cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:42.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.462+0000 7f258bfff700 1 --2- 172.21.15.1:0/3061676776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5f30b9f70 0x55e5f30cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.462+0000 7f258bfff700 1 -- 172.21.15.1:0/3061676776 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5f2fccee0 con 0x55e5f30ab360 2023-11-16T16:45:42.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.462+0000 7f258bfff700 1 --2- 172.21.15.1:0/3061676776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5f30ab360 0x55e5f30bc5b0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f257c0098e0 tx=0x7f257c00bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8f581da786d1329 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:42.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.462+0000 7f2582ffd700 1 -- 172.21.15.1:0/3061676776 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c02c040 con 0x55e5f30ab360 2023-11-16T16:45:42.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.462+0000 7f2582ffd700 1 -- 172.21.15.1:0/3061676776 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f257c03d070 con 0x55e5f30ab360 2023-11-16T16:45:42.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.462+0000 7f2582ffd700 1 -- 172.21.15.1:0/3061676776 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c00f920 con 0x55e5f30ab360 2023-11-16T16:45:42.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.462+0000 7f259e6ddb80 1 -- 172.21.15.1:0/3061676776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5f30ab360 msgr2=0x55e5f30bc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:42.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.462+0000 7f259e6ddb80 1 --2- 172.21.15.1:0/3061676776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5f30ab360 0x55e5f30bc5b0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f257c0098e0 tx=0x7f257c00bf40 comp rx=0 tx=0).stop 2023-11-16T16:45:42.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.463+0000 7f259e6ddb80 1 -- 172.21.15.1:0/3061676776 shutdown_connections 2023-11-16T16:45:42.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.463+0000 7f259e6ddb80 1 --2- 172.21.15.1:0/3061676776 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5f30ba880 0x55e5f30be450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.466 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.463+0000 7f259e6ddb80 1 --2- 172.21.15.1:0/3061676776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5f30ab360 0x55e5f30bc5b0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.466 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.463+0000 7f259e6ddb80 1 --2- 172.21.15.1:0/3061676776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5f30b9f70 0x55e5f30cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.466 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.463+0000 7f259e6ddb80 1 -- 172.21.15.1:0/3061676776 >> 172.21.15.1:0/3061676776 conn(0x55e5f30d05b0 msgr2=0x55e5f30bdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:42.466 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.463+0000 7f259e6ddb80 1 -- 172.21.15.1:0/3061676776 shutdown_connections 2023-11-16T16:45:42.466 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.463+0000 7f259e6ddb80 1 -- 172.21.15.1:0/3061676776 wait complete. 2023-11-16T16:45:42.466 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.463+0000 7f259e6ddb80 1 -- start start 2023-11-16T16:45:42.466 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.463+0000 7f259e6ddb80 1 Processor -- start 2023-11-16T16:45:42.469 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.464+0000 7f259e6ddb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5f30ab360 0x55e5f30bffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.470 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.464+0000 7f259e6ddb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5f30b9f70 0x55e5f30c3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.470 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.464+0000 7f259e6ddb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5f30ba880 0x55e5f30c3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.470 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.464+0000 7f259e6ddb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f30cb0b0 con 0x55e5f30b9f70 2023-11-16T16:45:42.470 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.464+0000 7f259e6ddb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f30cb220 con 0x55e5f30ab360 2023-11-16T16:45:42.470 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.464+0000 7f259e6ddb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f30cb3e0 con 0x55e5f30ba880 2023-11-16T16:45:42.470 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.464+0000 7f259e6ddb80 10 client.0 _ll_register_callbacks cb 0x7f257c00b790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:45:42.470 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.464+0000 7f258b7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5f30b9f70 0x55e5f30c3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.470 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.464+0000 7f258b7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5f30b9f70 0x55e5f30c3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45472/0 (socket says 172.21.15.1:45472) 2023-11-16T16:45:42.470 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.464+0000 7f258b7fe700 1 -- 172.21.15.1:0/3740246493 learned_addr learned my addr 172.21.15.1:0/3740246493 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:45:42.470 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.464+0000 7f258bfff700 1 --2- 172.21.15.1:0/3740246493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5f30ab360 0x55e5f30bffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.470 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.464+0000 7f2590aee700 1 --2- 172.21.15.1:0/3740246493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5f30ba880 0x55e5f30c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.471 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.464+0000 7f258b7fe700 1 -- 172.21.15.1:0/3740246493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5f30ba880 msgr2=0x55e5f30c3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:42.471 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.464+0000 7f258b7fe700 1 --2- 172.21.15.1:0/3740246493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5f30ba880 0x55e5f30c3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.471 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.464+0000 7f258b7fe700 1 -- 172.21.15.1:0/3740246493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5f30ab360 msgr2=0x55e5f30bffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:42.471 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.464+0000 7f258b7fe700 1 --2- 172.21.15.1:0/3740246493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5f30ab360 0x55e5f30bffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.471 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.464+0000 7f258b7fe700 1 -- 172.21.15.1:0/3740246493 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e5f2fcd190 con 0x55e5f30b9f70 2023-11-16T16:45:42.471 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.464+0000 7f258bfff700 1 --2- 172.21.15.1:0/3740246493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5f30ab360 0x55e5f30bffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:45:42.471 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.464+0000 7f258b7fe700 1 --2- 172.21.15.1:0/3740246493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5f30b9f70 0x55e5f30c3030 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f258c196860 tx=0x7f257401c590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b718d86ca441a6 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:42.471 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.464+0000 7f256b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:45:42.471 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.464+0000 7f256b7fe700 1 -- 172.21.15.1:0/3740246493 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2574017040 con 0x55e5f30b9f70 2023-11-16T16:45:42.471 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.464+0000 7f256b7fe700 1 -- 172.21.15.1:0/3740246493 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f2574004300 con 0x55e5f30b9f70 2023-11-16T16:45:42.471 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.464+0000 7f259e6ddb80 1 -- 172.21.15.1:0/3740246493 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25840b7e90 con 0x55e5f30b9f70 2023-11-16T16:45:42.471 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.465+0000 7f256b7fe700 1 -- 172.21.15.1:0/3740246493 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25740044c0 con 0x55e5f30b9f70 2023-11-16T16:45:42.471 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.465+0000 7f259e6ddb80 10 client.16257 fetch_fsmap learned FSMap version 28 2023-11-16T16:45:42.472 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.465+0000 7f259e6ddb80 1 -- 172.21.15.1:0/3740246493 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e5f2e1dc60 con 0x55e5f30b9f70 2023-11-16T16:45:42.472 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.465+0000 7f256b7fe700 1 -- 172.21.15.1:0/3740246493 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25840b7e90 con 0x55e5f30b9f70 2023-11-16T16:45:42.472 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.465+0000 7f256b7fe700 1 -- 172.21.15.1:0/3740246493 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f257402c070 con 0x55e5f30b9f70 2023-11-16T16:45:42.472 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.465+0000 7f259e6ddb80 10 client.16257 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:45:42.472 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.465+0000 7f259e6ddb80 10 client.16257 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-11-16T16:45:42.472 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.465+0000 7f259e6ddb80 20 client.16257 populate_metadata read hostname 'smithi001' 2023-11-16T16:45:42.472 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.465+0000 7f259e6ddb80 1 --2- 172.21.15.1:0/3740246493 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55e5f30becc0 0x55e5f312ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.472 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.465+0000 7f259e6ddb80 4 client.16257 mds_command: new command op to 24493 tid=0[{"prefix": "damage ls"}] 2023-11-16T16:45:42.472 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.465+0000 7f259e6ddb80 1 -- 172.21.15.1:0/3740246493 --> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e5f2fd5660 con 0x55e5f30becc0 2023-11-16T16:45:42.472 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.465+0000 7f258bfff700 1 --2- 172.21.15.1:0/3740246493 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55e5f30becc0 0x55e5f312ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.472 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.466+0000 7f256b7fe700 10 client.16257 ms_handle_connect on v2:172.21.15.53:6838/1044753024 2023-11-16T16:45:42.472 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.466+0000 7f258bfff700 1 --2- 172.21.15.1:0/3740246493 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55e5f30becc0 0x55e5f312ad00 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=22e374eefa96c12c server_cookie=53c91d2c22114f42 in_seq=0 out_seq=0 2023-11-16T16:45:42.472 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.466+0000 7f256b7fe700 1 -- 172.21.15.1:0/3740246493 <== mds.1 v2:172.21.15.53:6838/1044753024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e5f2fd5660 con 0x55e5f30becc0 2023-11-16T16:45:42.473 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.466+0000 7f256b7fe700 10 client.16257 handle_command_reply: tid=0 2023-11-16T16:45:42.473 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.466+0000 7f259e6ddb80 1 client.16257 shutdown 2023-11-16T16:45:42.473 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.466+0000 7f259e6ddb80 1 -- 172.21.15.1:0/3740246493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5f30b9f70 msgr2=0x55e5f30c3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:42.473 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.466+0000 7f259e6ddb80 1 --2- 172.21.15.1:0/3740246493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5f30b9f70 0x55e5f30c3030 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f258c196860 tx=0x7f257401c590 comp rx=0 tx=0).stop 2023-11-16T16:45:42.473 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.466+0000 7f259e6ddb80 1 -- 172.21.15.1:0/3740246493 shutdown_connections 2023-11-16T16:45:42.473 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.466+0000 7f259e6ddb80 1 --2- 172.21.15.1:0/3740246493 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5f30ba880 0x55e5f30c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.473 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.466+0000 7f259e6ddb80 1 --2- 172.21.15.1:0/3740246493 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55e5f30becc0 0x55e5f312ad00 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.473 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.466+0000 7f259e6ddb80 1 --2- 172.21.15.1:0/3740246493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5f30ab360 0x55e5f30bffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.473 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.466+0000 7f256b7fe700 0 client.16257 ms_handle_reset on v2:172.21.15.53:6838/1044753024 2023-11-16T16:45:42.473 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.466+0000 7f259e6ddb80 1 --2- 172.21.15.1:0/3740246493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5f30b9f70 0x55e5f30c3030 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.473 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.466+0000 7f259e6ddb80 1 -- 172.21.15.1:0/3740246493 >> 172.21.15.1:0/3740246493 conn(0x55e5f30d05b0 msgr2=0x55e5f30b4960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:42.473 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.466+0000 7f259e6ddb80 1 -- 172.21.15.1:0/3740246493 shutdown_connections 2023-11-16T16:45:42.474 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.466+0000 7f259e6ddb80 1 -- 172.21.15.1:0/3740246493 wait complete. 2023-11-16T16:45:42.474 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.466+0000 7f259e6ddb80 20 client.16257 trim_cache size 0 max 16384 2023-11-16T16:45:42.474 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2023-11-16T16:45:42.475 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.467+0000 7f2592551700 1 -- 172.21.15.1:0/3292625044 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f25780a12c0 msgr2=0x7f25780a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:42.475 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.467+0000 7f2592551700 1 --2- 172.21.15.1:0/3292625044 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f25780a12c0 0x7f25780a3770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f257c009990 tx=0x7f257c005fb0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.475 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.467+0000 7f2592551700 1 -- 172.21.15.1:0/3292625044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f258c104450 msgr2=0x7f258c199bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:42.475 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.467+0000 7f2592551700 1 --2- 172.21.15.1:0/3292625044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f258c104450 0x7f258c199bf0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f258400ef30 tx=0x7f258400c3a0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.475 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.467+0000 7f2592551700 1 -- 172.21.15.1:0/3292625044 shutdown_connections 2023-11-16T16:45:42.475 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.467+0000 7f2592551700 1 --2- 172.21.15.1:0/3292625044 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f25780a12c0 0x7f25780a3770 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.475 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.467+0000 7f2592551700 1 --2- 172.21.15.1:0/3292625044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f258c104450 0x7f258c199bf0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.475 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.467+0000 7f2592551700 1 --2- 172.21.15.1:0/3292625044 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f258c0ff410 0x7f258c195250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.475 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.467+0000 7f2592551700 1 --2- 172.21.15.1:0/3292625044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f258c0ffdb0 0x7f258c195790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.475 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.467+0000 7f2592551700 1 -- 172.21.15.1:0/3292625044 >> 172.21.15.1:0/3292625044 conn(0x7f258c0763a0 msgr2=0x7f258c102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:42.475 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.467+0000 7f2592551700 1 -- 172.21.15.1:0/3292625044 shutdown_connections 2023-11-16T16:45:42.475 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.467+0000 7f2592551700 1 -- 172.21.15.1:0/3292625044 wait complete. 2023-11-16T16:45:42.480 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-16T16:45:42.611 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.610+0000 7f55e95de700 1 -- 172.21.15.1:0/3246345127 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55e4105250 msgr2=0x7f55e4105620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:42.612 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.610+0000 7f55e95de700 1 --2- 172.21.15.1:0/3246345127 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55e4105250 0x7f55e4105620 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f55d40099c0 tx=0x7f55d4009cd0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.612 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.610+0000 7f55e95de700 1 -- 172.21.15.1:0/3246345127 shutdown_connections 2023-11-16T16:45:42.612 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.610+0000 7f55e95de700 1 --2- 172.21.15.1:0/3246345127 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55e40699e0 0x7f55e410d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.612 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.610+0000 7f55e95de700 1 --2- 172.21.15.1:0/3246345127 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55e4105250 0x7f55e4105620 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.612 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.610+0000 7f55e95de700 1 --2- 172.21.15.1:0/3246345127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e4069030 0x7f55e40694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.612 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.610+0000 7f55e95de700 1 -- 172.21.15.1:0/3246345127 >> 172.21.15.1:0/3246345127 conn(0x7f55e4076370 msgr2=0x7f55e4076770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:42.612 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.610+0000 7f55e95de700 1 -- 172.21.15.1:0/3246345127 shutdown_connections 2023-11-16T16:45:42.612 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.610+0000 7f55e95de700 1 -- 172.21.15.1:0/3246345127 wait complete. 2023-11-16T16:45:42.612 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.611+0000 7f55e95de700 1 Processor -- start 2023-11-16T16:45:42.612 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.611+0000 7f55e95de700 1 -- start start 2023-11-16T16:45:42.614 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.611+0000 7f55e95de700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55e4069030 0x7f55e41962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.614 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.611+0000 7f55e95de700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55e40699e0 0x7f55e4196810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.614 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.611+0000 7f55e95de700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e4105250 0x7f55e419ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.614 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.611+0000 7f55e95de700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e419b2b0 con 0x7f55e40699e0 2023-11-16T16:45:42.614 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.611+0000 7f55e95de700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e419b420 con 0x7f55e4105250 2023-11-16T16:45:42.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.611+0000 7f55e95de700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e419b5e0 con 0x7f55e4069030 2023-11-16T16:45:42.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.611+0000 7f55e27fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55e40699e0 0x7f55e4196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.612+0000 7f55e2ffd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55e4069030 0x7f55e41962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.612+0000 7f55e27fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55e40699e0 0x7f55e4196810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45486/0 (socket says 172.21.15.1:45486) 2023-11-16T16:45:42.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.612+0000 7f55e27fc700 1 -- 172.21.15.1:0/1188473084 learned_addr learned my addr 172.21.15.1:0/1188473084 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:45:42.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.612+0000 7f55e37fe700 1 --2- 172.21.15.1:0/1188473084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e4105250 0x7f55e419ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.612+0000 7f55e27fc700 1 -- 172.21.15.1:0/1188473084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55e4069030 msgr2=0x7f55e41962d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:42.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.612+0000 7f55e27fc700 1 --2- 172.21.15.1:0/1188473084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55e4069030 0x7f55e41962d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.612+0000 7f55e27fc700 1 -- 172.21.15.1:0/1188473084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e4105250 msgr2=0x7f55e419ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:42.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.612+0000 7f55e27fc700 1 --2- 172.21.15.1:0/1188473084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e4105250 0x7f55e419ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.612+0000 7f55e27fc700 1 -- 172.21.15.1:0/1188473084 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d40095f0 con 0x7f55e40699e0 2023-11-16T16:45:42.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.612+0000 7f55e2ffd700 1 --2- 172.21.15.1:0/1188473084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55e4069030 0x7f55e41962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:45:42.616 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.612+0000 7f55e37fe700 1 --2- 172.21.15.1:0/1188473084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e4105250 0x7f55e419ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:45:42.616 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.612+0000 7f55e27fc700 1 --2- 172.21.15.1:0/1188473084 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55e40699e0 0x7f55e4196810 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f55d800e9a0 tx=0x7f55d800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:42.616 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.612+0000 7f55cbfff700 1 -- 172.21.15.1:0/1188473084 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d8009790 con 0x7f55e40699e0 2023-11-16T16:45:42.616 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.612+0000 7f55e95de700 1 -- 172.21.15.1:0/1188473084 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55e41102c0 con 0x7f55e40699e0 2023-11-16T16:45:42.616 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.612+0000 7f55cbfff700 1 -- 172.21.15.1:0/1188473084 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55d8010430 con 0x7f55e40699e0 2023-11-16T16:45:42.616 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.612+0000 7f55cbfff700 1 -- 172.21.15.1:0/1188473084 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d8018680 con 0x7f55e40699e0 2023-11-16T16:45:42.616 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.612+0000 7f55e95de700 1 -- 172.21.15.1:0/1188473084 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55e4110130 con 0x7f55e40699e0 2023-11-16T16:45:42.619 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.614+0000 7f55cbfff700 1 -- 172.21.15.1:0/1188473084 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f55d80105a0 con 0x7f55e40699e0 2023-11-16T16:45:42.619 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.615+0000 7f55cbfff700 1 --2- 172.21.15.1:0/1188473084 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f55cc0a12c0 0x7f55cc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.619 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.615+0000 7f55cbfff700 1 -- 172.21.15.1:0/1188473084 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f55d8014070 con 0x7f55e40699e0 2023-11-16T16:45:42.619 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.615+0000 7f55e2ffd700 1 --2- 172.21.15.1:0/1188473084 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f55cc0a12c0 0x7f55cc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.619 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.615+0000 7f55e2ffd700 1 --2- 172.21.15.1:0/1188473084 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f55cc0a12c0 0x7f55cc0a3770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f55d4009990 tx=0x7f55d401a040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:42.619 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.616+0000 7f55f576ab80 1 Processor -- start 2023-11-16T16:45:42.620 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.616+0000 7f55f576ab80 1 -- start start 2023-11-16T16:45:42.620 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.616+0000 7f55f576ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e87a140b60 0x55e87a140f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.620 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.616+0000 7f55f576ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e87a141470 0x55e87a145370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.620 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.616+0000 7f55f576ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e87a1458b0 0x55e87a145d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.620 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.616+0000 7f55f576ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e87a046060 con 0x55e87a140b60 2023-11-16T16:45:42.620 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.616+0000 7f55f576ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e87a0988a0 con 0x55e87a1458b0 2023-11-16T16:45:42.620 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.616+0000 7f55f576ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e87a091d30 con 0x55e87a141470 2023-11-16T16:45:42.620 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.616+0000 7f55e27fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e87a141470 0x55e87a145370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.620 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.616+0000 7f55e27fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e87a141470 0x55e87a145370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:42176/0 (socket says 172.21.15.1:42176) 2023-11-16T16:45:42.620 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.616+0000 7f55e27fc700 1 -- 172.21.15.1:0/1602317317 learned_addr learned my addr 172.21.15.1:0/1602317317 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:45:42.620 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.616+0000 7f55e37fe700 1 --2- 172.21.15.1:0/1602317317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e87a1458b0 0x55e87a145d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.620 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.616+0000 7f55e2ffd700 1 --2- 172.21.15.1:0/1602317317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e87a140b60 0x55e87a140f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.620 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.617+0000 7f55e27fc700 1 -- 172.21.15.1:0/1602317317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e87a1458b0 msgr2=0x55e87a145d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:42.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.617+0000 7f55e27fc700 1 --2- 172.21.15.1:0/1602317317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e87a1458b0 0x55e87a145d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.617+0000 7f55e27fc700 1 -- 172.21.15.1:0/1602317317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e87a140b60 msgr2=0x55e87a140f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:42.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.617+0000 7f55e27fc700 1 --2- 172.21.15.1:0/1602317317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e87a140b60 0x55e87a140f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.617+0000 7f55e27fc700 1 -- 172.21.15.1:0/1602317317 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e879f13630 con 0x55e87a141470 2023-11-16T16:45:42.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.617+0000 7f55e37fe700 1 --2- 172.21.15.1:0/1602317317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e87a1458b0 0x55e87a145d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:45:42.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.617+0000 7f55e27fc700 1 --2- 172.21.15.1:0/1602317317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e87a141470 0x55e87a145370 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f55d80a1ed0 tx=0x7f55d8099fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b2d70f3bbf0f711 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:42.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.617+0000 7f55c97fa700 1 -- 172.21.15.1:0/1602317317 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d8010880 con 0x55e87a141470 2023-11-16T16:45:42.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.617+0000 7f55c97fa700 1 -- 172.21.15.1:0/1602317317 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55d80109e0 con 0x55e87a141470 2023-11-16T16:45:42.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.617+0000 7f55c97fa700 1 -- 172.21.15.1:0/1602317317 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d80a1f70 con 0x55e87a141470 2023-11-16T16:45:42.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.618+0000 7f55f576ab80 1 -- 172.21.15.1:0/1602317317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e87a141470 msgr2=0x55e87a145370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:42.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.618+0000 7f55f576ab80 1 --2- 172.21.15.1:0/1602317317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e87a141470 0x55e87a145370 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f55d80a1ed0 tx=0x7f55d8099fb0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.618+0000 7f55f576ab80 1 -- 172.21.15.1:0/1602317317 shutdown_connections 2023-11-16T16:45:42.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.618+0000 7f55f576ab80 1 --2- 172.21.15.1:0/1602317317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e87a1458b0 0x55e87a145d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.622 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.618+0000 7f55f576ab80 1 --2- 172.21.15.1:0/1602317317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e87a140b60 0x55e87a140f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.622 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.618+0000 7f55f576ab80 1 --2- 172.21.15.1:0/1602317317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e87a141470 0x55e87a145370 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.622 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.618+0000 7f55f576ab80 1 -- 172.21.15.1:0/1602317317 >> 172.21.15.1:0/1602317317 conn(0x55e87a136360 msgr2=0x55e87a13bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:42.622 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.618+0000 7f55f576ab80 1 -- 172.21.15.1:0/1602317317 shutdown_connections 2023-11-16T16:45:42.622 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.618+0000 7f55f576ab80 1 -- 172.21.15.1:0/1602317317 wait complete. 2023-11-16T16:45:42.622 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.618+0000 7f55f576ab80 1 -- start start 2023-11-16T16:45:42.622 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.619+0000 7f55f576ab80 1 Processor -- start 2023-11-16T16:45:42.625 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.619+0000 7f55f576ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e87a140b60 0x55e87a14a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.625 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.619+0000 7f55f576ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e87a141470 0x55e87a14a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.625 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.619+0000 7f55f576ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e87a1458b0 0x55e87a14adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.626 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.619+0000 7f55f576ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e87a0338e0 con 0x55e87a141470 2023-11-16T16:45:42.626 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.619+0000 7f55f576ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e87a033580 con 0x55e87a140b60 2023-11-16T16:45:42.626 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.619+0000 7f55f576ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e879f15920 con 0x55e87a1458b0 2023-11-16T16:45:42.626 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.619+0000 7f55f576ab80 10 client.0 _ll_register_callbacks cb 0x55e87a051780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:45:42.626 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.619+0000 7f55e2ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e87a140b60 0x55e87a14a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.626 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.619+0000 7f55e2ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e87a140b60 0x55e87a14a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.1:39684/0 (socket says 172.21.15.1:39684) 2023-11-16T16:45:42.626 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.619+0000 7f55e2ffd700 1 -- 172.21.15.1:0/127250323 learned_addr learned my addr 172.21.15.1:0/127250323 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:45:42.626 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.619+0000 7f55e37fe700 1 --2- 172.21.15.1:0/127250323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e87a1458b0 0x55e87a14adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.626 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.619+0000 7f55e37fe700 1 -- 172.21.15.1:0/127250323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e87a140b60 msgr2=0x55e87a14a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:42.626 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.619+0000 7f55e37fe700 1 --2- 172.21.15.1:0/127250323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e87a140b60 0x55e87a14a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.626 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.620+0000 7f55e37fe700 1 -- 172.21.15.1:0/127250323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e87a141470 msgr2=0x55e87a14a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:42.626 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.620+0000 7f55e37fe700 1 --2- 172.21.15.1:0/127250323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e87a141470 0x55e87a14a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.627 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.620+0000 7f55e37fe700 1 -- 172.21.15.1:0/127250323 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55e4110130 con 0x55e87a1458b0 2023-11-16T16:45:42.627 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.620+0000 7f55e37fe700 1 --2- 172.21.15.1:0/127250323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e87a1458b0 0x55e87a14adc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f55dc0046a0 tx=0x7f55dc0049b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5affa6329ff387af server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:42.627 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.620+0000 7f55c6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.53:3300/0 2023-11-16T16:45:42.627 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.620+0000 7f55c6ff5700 1 -- 172.21.15.1:0/127250323 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc004db0 con 0x55e87a1458b0 2023-11-16T16:45:42.627 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.620+0000 7f55c6ff5700 1 -- 172.21.15.1:0/127250323 <== mon.1 v2:172.21.15.53:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f55dc0093a0 con 0x55e87a1458b0 2023-11-16T16:45:42.627 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.620+0000 7f55f576ab80 1 -- 172.21.15.1:0/127250323 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e87a133b20 con 0x55e87a1458b0 2023-11-16T16:45:42.627 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.620+0000 7f55c6ff5700 1 -- 172.21.15.1:0/127250323 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55dc00ead0 con 0x55e87a1458b0 2023-11-16T16:45:42.627 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.620+0000 7f55f576ab80 10 client.25828 fetch_fsmap learned FSMap version 28 2023-11-16T16:45:42.627 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.620+0000 7f55f576ab80 1 -- 172.21.15.1:0/127250323 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e87a058190 con 0x55e87a1458b0 2023-11-16T16:45:42.627 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.620+0000 7f55c6ff5700 1 -- 172.21.15.1:0/127250323 <== mon.1 v2:172.21.15.53:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e87a133b20 con 0x55e87a1458b0 2023-11-16T16:45:42.627 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.621+0000 7f55c6ff5700 1 -- 172.21.15.1:0/127250323 <== mon.1 v2:172.21.15.53:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f55dc01c030 con 0x55e87a1458b0 2023-11-16T16:45:42.627 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.621+0000 7f55f576ab80 10 client.25828 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:45:42.628 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.621+0000 7f55f576ab80 10 client.25828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:45:42.628 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.621+0000 7f55f576ab80 20 client.25828 populate_metadata read hostname 'smithi001' 2023-11-16T16:45:42.628 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.621+0000 7f55f576ab80 1 --2- 172.21.15.1:0/127250323 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55e87a15aa40 0x55e87a15ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.628 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.621+0000 7f55f576ab80 4 client.25828 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:45:42.628 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.621+0000 7f55f576ab80 1 -- 172.21.15.1:0/127250323 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e87a0540a0 con 0x55e87a15aa40 2023-11-16T16:45:42.628 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.621+0000 7f55e2ffd700 1 --2- 172.21.15.1:0/127250323 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55e87a15aa40 0x55e87a15ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.628 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.621+0000 7f55c6ff5700 10 client.25828 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:45:42.628 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.621+0000 7f55e2ffd700 1 --2- 172.21.15.1:0/127250323 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55e87a15aa40 0x55e87a15ce50 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78a0b5721b4f8cee server_cookie=56a832c39e9a167f in_seq=0 out_seq=0 2023-11-16T16:45:42.628 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.622+0000 7f55c6ff5700 1 -- 172.21.15.1:0/127250323 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e87a0540a0 con 0x55e87a15aa40 2023-11-16T16:45:42.628 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.622+0000 7f55c6ff5700 10 client.25828 handle_command_reply: tid=0 2023-11-16T16:45:42.628 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.622+0000 7f55f576ab80 1 client.25828 shutdown 2023-11-16T16:45:42.628 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.622+0000 7f55f576ab80 1 -- 172.21.15.1:0/127250323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e87a1458b0 msgr2=0x55e87a14adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:42.628 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.622+0000 7f55f576ab80 1 --2- 172.21.15.1:0/127250323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e87a1458b0 0x55e87a14adc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f55dc0046a0 tx=0x7f55dc0049b0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.629 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.622+0000 7f55f576ab80 1 -- 172.21.15.1:0/127250323 shutdown_connections 2023-11-16T16:45:42.629 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.622+0000 7f55f576ab80 1 --2- 172.21.15.1:0/127250323 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55e87a15aa40 0x55e87a15ce50 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.629 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.622+0000 7f55f576ab80 1 --2- 172.21.15.1:0/127250323 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e87a1458b0 0x55e87a14adc0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.629 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.622+0000 7f55f576ab80 1 --2- 172.21.15.1:0/127250323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e87a140b60 0x55e87a14a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.629 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.622+0000 7f55f576ab80 1 --2- 172.21.15.1:0/127250323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e87a141470 0x55e87a14a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.629 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.622+0000 7f55c6ff5700 0 client.25828 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:45:42.629 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.622+0000 7f55f576ab80 1 -- 172.21.15.1:0/127250323 >> 172.21.15.1:0/127250323 conn(0x55e87a136360 msgr2=0x55e87a146d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:42.629 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.622+0000 7f55f576ab80 1 -- 172.21.15.1:0/127250323 shutdown_connections 2023-11-16T16:45:42.629 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.622+0000 7f55f576ab80 1 -- 172.21.15.1:0/127250323 wait complete. 2023-11-16T16:45:42.629 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.622+0000 7f55f576ab80 20 client.25828 trim_cache size 0 max 16384 2023-11-16T16:45:42.640 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.635+0000 7f55f576ab80 1 Processor -- start 2023-11-16T16:45:42.640 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.635+0000 7f55f576ab80 1 -- start start 2023-11-16T16:45:42.640 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.635+0000 7f55f576ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e87a140b60 0x55e87a1519a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.640 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.635+0000 7f55f576ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e87a141470 0x55e87a1439d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.640 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.635+0000 7f55f576ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e87a1458b0 0x55e87a143f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.640 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.635+0000 7f55f576ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e87a135ee0 con 0x55e87a140b60 2023-11-16T16:45:42.640 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.635+0000 7f55f576ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e87a14d3d0 con 0x55e87a141470 2023-11-16T16:45:42.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.635+0000 7f55f576ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e87a144510 con 0x55e87a1458b0 2023-11-16T16:45:42.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.635+0000 7f55e2ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e87a140b60 0x55e87a1519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.635+0000 7f55e2ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e87a140b60 0x55e87a1519a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45506/0 (socket says 172.21.15.1:45506) 2023-11-16T16:45:42.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.635+0000 7f55e2ffd700 1 -- 172.21.15.1:0/2964987790 learned_addr learned my addr 172.21.15.1:0/2964987790 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:45:42.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.635+0000 7f55e27fc700 1 --2- 172.21.15.1:0/2964987790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e87a141470 0x55e87a1439d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.635+0000 7f55e37fe700 1 --2- 172.21.15.1:0/2964987790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e87a1458b0 0x55e87a143f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.636+0000 7f55e2ffd700 1 -- 172.21.15.1:0/2964987790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e87a1458b0 msgr2=0x55e87a143f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:42.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.636+0000 7f55e2ffd700 1 --2- 172.21.15.1:0/2964987790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e87a1458b0 0x55e87a143f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.636+0000 7f55e2ffd700 1 -- 172.21.15.1:0/2964987790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e87a141470 msgr2=0x55e87a1439d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:42.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.636+0000 7f55e2ffd700 1 --2- 172.21.15.1:0/2964987790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e87a141470 0x55e87a1439d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.636+0000 7f55e2ffd700 1 -- 172.21.15.1:0/2964987790 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e87a1461a0 con 0x55e87a140b60 2023-11-16T16:45:42.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.636+0000 7f55e27fc700 1 --2- 172.21.15.1:0/2964987790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e87a141470 0x55e87a1439d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:45:42.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.636+0000 7f55e2ffd700 1 --2- 172.21.15.1:0/2964987790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e87a140b60 0x55e87a1519a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f55d400b580 tx=0x7f55d401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=220e60b2b3a3f154 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:42.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.636+0000 7f55c9ffb700 1 -- 172.21.15.1:0/2964987790 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d402c070 con 0x55e87a140b60 2023-11-16T16:45:42.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.636+0000 7f55c9ffb700 1 -- 172.21.15.1:0/2964987790 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55d402d040 con 0x55e87a140b60 2023-11-16T16:45:42.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.636+0000 7f55c9ffb700 1 -- 172.21.15.1:0/2964987790 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d4024420 con 0x55e87a140b60 2023-11-16T16:45:42.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.637+0000 7f55f576ab80 1 -- 172.21.15.1:0/2964987790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e87a140b60 msgr2=0x55e87a1519a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:42.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.637+0000 7f55f576ab80 1 --2- 172.21.15.1:0/2964987790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e87a140b60 0x55e87a1519a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f55d400b580 tx=0x7f55d401f000 comp rx=0 tx=0).stop 2023-11-16T16:45:42.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.637+0000 7f55f576ab80 1 -- 172.21.15.1:0/2964987790 shutdown_connections 2023-11-16T16:45:42.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.637+0000 7f55f576ab80 1 --2- 172.21.15.1:0/2964987790 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e87a1458b0 0x55e87a143f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.637+0000 7f55f576ab80 1 --2- 172.21.15.1:0/2964987790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e87a140b60 0x55e87a1519a0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.637+0000 7f55f576ab80 1 --2- 172.21.15.1:0/2964987790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e87a141470 0x55e87a1439d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.637+0000 7f55f576ab80 1 -- 172.21.15.1:0/2964987790 >> 172.21.15.1:0/2964987790 conn(0x55e87a136360 msgr2=0x55e87a13a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:42.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.637+0000 7f55f576ab80 1 -- 172.21.15.1:0/2964987790 shutdown_connections 2023-11-16T16:45:42.643 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.637+0000 7f55f576ab80 1 -- 172.21.15.1:0/2964987790 wait complete. 2023-11-16T16:45:42.643 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.637+0000 7f55f576ab80 1 -- start start 2023-11-16T16:45:42.643 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.637+0000 7f55f576ab80 1 Processor -- start 2023-11-16T16:45:42.643 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.637+0000 7f55f576ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e87a140b60 0x55e87a1428c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.643 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.637+0000 7f55f576ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e87a141470 0x55e87a142e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.643 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.637+0000 7f55f576ab80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e87a1458b0 0x55e87a149be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.643 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.637+0000 7f55f576ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e87a14a120 con 0x55e87a140b60 2023-11-16T16:45:42.643 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.637+0000 7f55f576ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e87a14a260 con 0x55e87a141470 2023-11-16T16:45:42.643 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.637+0000 7f55f576ab80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e87a14a3f0 con 0x55e87a1458b0 2023-11-16T16:45:42.643 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.637+0000 7f55f576ab80 10 client.0 _ll_register_callbacks cb 0x55e87a051780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:45:42.643 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.637+0000 7f55e2ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e87a140b60 0x55e87a1428c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.643 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.637+0000 7f55e2ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e87a140b60 0x55e87a1428c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45512/0 (socket says 172.21.15.1:45512) 2023-11-16T16:45:42.643 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.637+0000 7f55e2ffd700 1 -- 172.21.15.1:0/215765316 learned_addr learned my addr 172.21.15.1:0/215765316 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:45:42.644 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.638+0000 7f55e37fe700 1 --2- 172.21.15.1:0/215765316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e87a1458b0 0x55e87a149be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.644 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.638+0000 7f55e27fc700 1 --2- 172.21.15.1:0/215765316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e87a141470 0x55e87a142e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.644 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.638+0000 7f55e2ffd700 1 -- 172.21.15.1:0/215765316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e87a1458b0 msgr2=0x55e87a149be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:42.644 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.638+0000 7f55e2ffd700 1 --2- 172.21.15.1:0/215765316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e87a1458b0 0x55e87a149be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.644 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.638+0000 7f55e2ffd700 1 -- 172.21.15.1:0/215765316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e87a141470 msgr2=0x55e87a142e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:42.644 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.638+0000 7f55e2ffd700 1 --2- 172.21.15.1:0/215765316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e87a141470 0x55e87a142e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.644 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.638+0000 7f55e2ffd700 1 -- 172.21.15.1:0/215765316 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e879f13630 con 0x55e87a140b60 2023-11-16T16:45:42.644 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.638+0000 7f55e27fc700 1 --2- 172.21.15.1:0/215765316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e87a141470 0x55e87a142e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:45:42.644 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.638+0000 7f55e2ffd700 1 --2- 172.21.15.1:0/215765316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e87a140b60 0x55e87a1428c0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f55d400c010 tx=0x7f55d4033500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=473e5f4302c8220b server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:42.644 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.638+0000 7f55c6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:45:42.644 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.638+0000 7f55c6ff5700 1 -- 172.21.15.1:0/215765316 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d402c070 con 0x55e87a140b60 2023-11-16T16:45:42.644 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.638+0000 7f55c6ff5700 1 -- 172.21.15.1:0/215765316 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f55d4004fc0 con 0x55e87a140b60 2023-11-16T16:45:42.645 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.638+0000 7f55f576ab80 1 -- 172.21.15.1:0/215765316 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f55d80109e0 con 0x55e87a140b60 2023-11-16T16:45:42.645 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.638+0000 7f55c6ff5700 1 -- 172.21.15.1:0/215765316 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55d4033e60 con 0x55e87a140b60 2023-11-16T16:45:42.645 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.638+0000 7f55f576ab80 10 client.16272 fetch_fsmap learned FSMap version 28 2023-11-16T16:45:42.645 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.638+0000 7f55f576ab80 1 -- 172.21.15.1:0/215765316 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f55e410d970 con 0x55e87a140b60 2023-11-16T16:45:42.645 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.638+0000 7f55c6ff5700 1 -- 172.21.15.1:0/215765316 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55d80109e0 con 0x55e87a140b60 2023-11-16T16:45:42.645 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.639+0000 7f55c6ff5700 1 -- 172.21.15.1:0/215765316 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f55d4045070 con 0x55e87a140b60 2023-11-16T16:45:42.645 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.639+0000 7f55f576ab80 10 client.16272 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:45:42.645 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.639+0000 7f55f576ab80 10 client.16272 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:45:42.645 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.639+0000 7f55f576ab80 20 client.16272 populate_metadata read hostname 'smithi001' 2023-11-16T16:45:42.645 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.639+0000 7f55f576ab80 1 --2- 172.21.15.1:0/215765316 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55e87a1af3d0 0x55e87a1b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:42.645 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.639+0000 7f55f576ab80 4 client.16272 mds_command: new command op to 24521 tid=0[{"prefix": "damage ls"}] 2023-11-16T16:45:42.645 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.639+0000 7f55f576ab80 1 -- 172.21.15.1:0/215765316 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e87a060660 con 0x55e87a1af3d0 2023-11-16T16:45:42.645 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.639+0000 7f55e27fc700 1 --2- 172.21.15.1:0/215765316 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55e87a1af3d0 0x55e87a1b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:42.646 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.639+0000 7f55e27fc700 1 --2- 172.21.15.1:0/215765316 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55e87a1af3d0 0x55e87a1b17b0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52ed6a3575fb6fa0 server_cookie=1358bafaff54d30d in_seq=0 out_seq=0 2023-11-16T16:45:42.646 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.639+0000 7f55c6ff5700 10 client.16272 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:45:42.647 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.642+0000 7f55c6ff5700 1 -- 172.21.15.1:0/215765316 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e87a060660 con 0x55e87a1af3d0 2023-11-16T16:45:42.647 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.642+0000 7f55c6ff5700 10 client.16272 handle_command_reply: tid=0 2023-11-16T16:45:42.647 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.642+0000 7f55f576ab80 1 client.16272 shutdown 2023-11-16T16:45:42.647 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.642+0000 7f55f576ab80 1 -- 172.21.15.1:0/215765316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e87a140b60 msgr2=0x55e87a1428c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:42.647 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.642+0000 7f55f576ab80 1 --2- 172.21.15.1:0/215765316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e87a140b60 0x55e87a1428c0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f55d400c010 tx=0x7f55d4033500 comp rx=0 tx=0).stop 2023-11-16T16:45:42.647 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.642+0000 7f55f576ab80 1 -- 172.21.15.1:0/215765316 shutdown_connections 2023-11-16T16:45:42.647 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.642+0000 7f55f576ab80 1 --2- 172.21.15.1:0/215765316 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55e87a1af3d0 0x55e87a1b17b0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.647 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.642+0000 7f55f576ab80 1 --2- 172.21.15.1:0/215765316 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e87a1458b0 0x55e87a149be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.647 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.642+0000 7f55c6ff5700 0 client.16272 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:45:42.648 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.642+0000 7f55f576ab80 1 --2- 172.21.15.1:0/215765316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e87a140b60 0x55e87a1428c0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.648 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.642+0000 7f55f576ab80 1 --2- 172.21.15.1:0/215765316 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e87a141470 0x55e87a142e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.648 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.642+0000 7f55f576ab80 1 -- 172.21.15.1:0/215765316 >> 172.21.15.1:0/215765316 conn(0x55e87a136360 msgr2=0x55e87a1439d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:42.648 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.642+0000 7f55f576ab80 1 -- 172.21.15.1:0/215765316 shutdown_connections 2023-11-16T16:45:42.648 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.642+0000 7f55f576ab80 1 -- 172.21.15.1:0/215765316 wait complete. 2023-11-16T16:45:42.648 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.642+0000 7f55f576ab80 20 client.16272 trim_cache size 0 max 16384 2023-11-16T16:45:42.648 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2023-11-16T16:45:42.649 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.642+0000 7f55e95de700 1 -- 172.21.15.1:0/1188473084 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f55cc0a12c0 msgr2=0x7f55cc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:42.649 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.642+0000 7f55e95de700 1 --2- 172.21.15.1:0/1188473084 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f55cc0a12c0 0x7f55cc0a3770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f55d4009990 tx=0x7f55d401a040 comp rx=0 tx=0).stop 2023-11-16T16:45:42.649 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.642+0000 7f55e95de700 1 -- 172.21.15.1:0/1188473084 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55e40699e0 msgr2=0x7f55e4196810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:42.649 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.642+0000 7f55e95de700 1 --2- 172.21.15.1:0/1188473084 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55e40699e0 0x7f55e4196810 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f55d800e9a0 tx=0x7f55d800ecb0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.649 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.643+0000 7f55e95de700 1 -- 172.21.15.1:0/1188473084 shutdown_connections 2023-11-16T16:45:42.650 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.643+0000 7f55e95de700 1 --2- 172.21.15.1:0/1188473084 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f55cc0a12c0 0x7f55cc0a3770 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.650 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.643+0000 7f55e95de700 1 --2- 172.21.15.1:0/1188473084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e4105250 0x7f55e419ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.650 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.643+0000 7f55e95de700 1 --2- 172.21.15.1:0/1188473084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f55e4069030 0x7f55e41962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.650 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.643+0000 7f55e95de700 1 --2- 172.21.15.1:0/1188473084 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55e40699e0 0x7f55e4196810 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:42.650 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.643+0000 7f55e95de700 1 -- 172.21.15.1:0/1188473084 >> 172.21.15.1:0/1188473084 conn(0x7f55e4076370 msgr2=0x7f55e40fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:42.650 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.643+0000 7f55e95de700 1 -- 172.21.15.1:0/1188473084 shutdown_connections 2023-11-16T16:45:42.650 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:42.643+0000 7f55e95de700 1 -- 172.21.15.1:0/1188473084 wait complete. 2023-11-16T16:45:43.656 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-16T16:45:43.657 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-16T16:45:43.805 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.802+0000 7fcccc3ed700 1 -- 172.21.15.1:0/942945331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc4105250 msgr2=0x7fccc4105620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:43.806 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.802+0000 7fcccc3ed700 1 --2- 172.21.15.1:0/942945331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc4105250 0x7fccc4105620 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fccc0009920 tx=0x7fccc0009c30 comp rx=0 tx=0).stop 2023-11-16T16:45:43.806 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.802+0000 7fcccc3ed700 1 -- 172.21.15.1:0/942945331 shutdown_connections 2023-11-16T16:45:43.806 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.802+0000 7fcccc3ed700 1 --2- 172.21.15.1:0/942945331 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fccc4100420 0x7fccc410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.806 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.802+0000 7fcccc3ed700 1 --2- 172.21.15.1:0/942945331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc4105250 0x7fccc4105620 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.806 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.802+0000 7fcccc3ed700 1 --2- 172.21.15.1:0/942945331 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fccc40ffa70 0x7fccc40ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.806 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.802+0000 7fcccc3ed700 1 -- 172.21.15.1:0/942945331 >> 172.21.15.1:0/942945331 conn(0x7fccc40763a0 msgr2=0x7fccc40767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:43.806 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.802+0000 7fcccc3ed700 1 -- 172.21.15.1:0/942945331 shutdown_connections 2023-11-16T16:45:43.807 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.802+0000 7fcccc3ed700 1 -- 172.21.15.1:0/942945331 wait complete. 2023-11-16T16:45:43.807 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.802+0000 7fcccc3ed700 1 Processor -- start 2023-11-16T16:45:43.807 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.802+0000 7fcccc3ed700 1 -- start start 2023-11-16T16:45:43.807 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.803+0000 7fcccc3ed700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fccc40ffa70 0x7fccc4195140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:43.807 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.803+0000 7fcccc3ed700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc4100420 0x7fccc4195680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:43.807 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.803+0000 7fcccc3ed700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fccc4105250 0x7fccc4195bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:43.807 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.803+0000 7fcccc3ed700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc41a49b0 con 0x7fccc4105250 2023-11-16T16:45:43.807 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.803+0000 7fcccc3ed700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc41a4b20 con 0x7fccc4100420 2023-11-16T16:45:43.807 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.803+0000 7fcccc3ed700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc41a4ce0 con 0x7fccc40ffa70 2023-11-16T16:45:43.808 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.803+0000 7fccca98a700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fccc4105250 0x7fccc4195bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:43.808 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.803+0000 7fccca189700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fccc40ffa70 0x7fccc4195140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:43.808 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.803+0000 7fccca98a700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fccc4105250 0x7fccc4195bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45530/0 (socket says 172.21.15.1:45530) 2023-11-16T16:45:43.808 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.803+0000 7fccca98a700 1 -- 172.21.15.1:0/806898269 learned_addr learned my addr 172.21.15.1:0/806898269 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:45:43.808 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.803+0000 7fccc9988700 1 --2- 172.21.15.1:0/806898269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc4100420 0x7fccc4195680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:43.808 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.803+0000 7fccca98a700 1 -- 172.21.15.1:0/806898269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fccc40ffa70 msgr2=0x7fccc4195140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:43.808 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.803+0000 7fccca98a700 1 --2- 172.21.15.1:0/806898269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fccc40ffa70 0x7fccc4195140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.808 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.803+0000 7fccca98a700 1 -- 172.21.15.1:0/806898269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc4100420 msgr2=0x7fccc4195680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:43.808 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.803+0000 7fccca98a700 1 --2- 172.21.15.1:0/806898269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc4100420 0x7fccc4195680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.809 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.803+0000 7fccca98a700 1 -- 172.21.15.1:0/806898269 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccbc009d00 con 0x7fccc4105250 2023-11-16T16:45:43.809 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.803+0000 7fccca189700 1 --2- 172.21.15.1:0/806898269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fccc40ffa70 0x7fccc4195140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:45:43.809 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.803+0000 7fccca98a700 1 --2- 172.21.15.1:0/806898269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fccc4105250 0x7fccc4195bc0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fccbc00e3c0 tx=0x7fccbc00e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:43.809 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.803+0000 7fccbb7fe700 1 -- 172.21.15.1:0/806898269 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc0090d0 con 0x7fccc4105250 2023-11-16T16:45:43.809 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.804+0000 7fcccc3ed700 1 -- 172.21.15.1:0/806898269 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc0009520 con 0x7fccc4105250 2023-11-16T16:45:43.809 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.804+0000 7fccbb7fe700 1 -- 172.21.15.1:0/806898269 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccbc00f040 con 0x7fccc4105250 2023-11-16T16:45:43.810 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.804+0000 7fccbb7fe700 1 -- 172.21.15.1:0/806898269 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc007c20 con 0x7fccc4105250 2023-11-16T16:45:43.810 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.804+0000 7fcccc3ed700 1 -- 172.21.15.1:0/806898269 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc410f200 con 0x7fccc4105250 2023-11-16T16:45:43.818 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.806+0000 7fccbb7fe700 1 -- 172.21.15.1:0/806898269 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fccbc003a00 con 0x7fccc4105250 2023-11-16T16:45:43.819 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.806+0000 7fccbb7fe700 1 --2- 172.21.15.1:0/806898269 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fccb00a12c0 0x7fccb00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:43.819 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.807+0000 7fccca189700 1 --2- 172.21.15.1:0/806898269 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fccb00a12c0 0x7fccb00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:43.819 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.807+0000 7fccbb7fe700 1 -- 172.21.15.1:0/806898269 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fccbc0ccc00 con 0x7fccc4105250 2023-11-16T16:45:43.819 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.807+0000 7fccca189700 1 --2- 172.21.15.1:0/806898269 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fccb00a12c0 0x7fccb00a3770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fccc000c010 tx=0x7fccc0019040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:43.819 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.807+0000 7fccd8579b80 1 Processor -- start 2023-11-16T16:45:43.819 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.808+0000 7fccd8579b80 1 -- start start 2023-11-16T16:45:43.820 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.808+0000 7fccd8579b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55683d0f6be0 0x55683d0f6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:43.820 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.808+0000 7fccd8579b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55683d0f74f0 0x55683d0fb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:43.820 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.808+0000 7fccd8579b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55683d0fb930 0x55683d0fbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:43.820 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.808+0000 7fccd8579b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55683cffc0e0 con 0x55683d0f6be0 2023-11-16T16:45:43.820 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.808+0000 7fccd8579b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55683d04e920 con 0x55683d0f74f0 2023-11-16T16:45:43.820 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.808+0000 7fccd8579b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55683d047db0 con 0x55683d0fb930 2023-11-16T16:45:43.820 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.808+0000 7fccca189700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55683d0f6be0 0x55683d0f6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:43.820 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.808+0000 7fccca189700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55683d0f6be0 0x55683d0f6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45534/0 (socket says 172.21.15.1:45534) 2023-11-16T16:45:43.821 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.808+0000 7fccca189700 1 -- 172.21.15.1:0/1378465951 learned_addr learned my addr 172.21.15.1:0/1378465951 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:45:43.821 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.808+0000 7fccc9988700 1 --2- 172.21.15.1:0/1378465951 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55683d0f74f0 0x55683d0fb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:43.821 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.808+0000 7fccca98a700 1 --2- 172.21.15.1:0/1378465951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55683d0fb930 0x55683d0fbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:43.821 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.808+0000 7fccc9988700 1 -- 172.21.15.1:0/1378465951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55683d0fb930 msgr2=0x55683d0fbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:43.821 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.808+0000 7fccc9988700 1 --2- 172.21.15.1:0/1378465951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55683d0fb930 0x55683d0fbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.821 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.808+0000 7fccc9988700 1 -- 172.21.15.1:0/1378465951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55683d0f6be0 msgr2=0x55683d0f6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:43.821 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.808+0000 7fccc9988700 1 --2- 172.21.15.1:0/1378465951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55683d0f6be0 0x55683d0f6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.821 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.808+0000 7fccc9988700 1 -- 172.21.15.1:0/1378465951 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55683cec96b0 con 0x55683d0f74f0 2023-11-16T16:45:43.821 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.808+0000 7fccca189700 1 --2- 172.21.15.1:0/1378465951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55683d0f6be0 0x55683d0f6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:45:43.821 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.809+0000 7fccc9988700 1 --2- 172.21.15.1:0/1378465951 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55683d0f74f0 0x55683d0fb3f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fccb4009c80 tx=0x7fccb4009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6a5d96d4a36fcc2 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:43.822 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.809+0000 7fccb8dac700 1 -- 172.21.15.1:0/1378465951 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb4011cb0 con 0x55683d0f74f0 2023-11-16T16:45:43.822 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.809+0000 7fccb8dac700 1 -- 172.21.15.1:0/1378465951 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccb4011e10 con 0x55683d0f74f0 2023-11-16T16:45:43.822 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.809+0000 7fccb8dac700 1 -- 172.21.15.1:0/1378465951 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb4005220 con 0x55683d0f74f0 2023-11-16T16:45:43.822 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.810+0000 7fccd8579b80 1 -- 172.21.15.1:0/1378465951 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55683d0f74f0 msgr2=0x55683d0fb3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:43.822 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.810+0000 7fccd8579b80 1 --2- 172.21.15.1:0/1378465951 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55683d0f74f0 0x55683d0fb3f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fccb4009c80 tx=0x7fccb4009f90 comp rx=0 tx=0).stop 2023-11-16T16:45:43.822 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.810+0000 7fccd8579b80 1 -- 172.21.15.1:0/1378465951 shutdown_connections 2023-11-16T16:45:43.822 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.810+0000 7fccd8579b80 1 --2- 172.21.15.1:0/1378465951 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55683d0fb930 0x55683d0fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.822 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.810+0000 7fccd8579b80 1 --2- 172.21.15.1:0/1378465951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55683d0f6be0 0x55683d0f6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.822 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.810+0000 7fccd8579b80 1 --2- 172.21.15.1:0/1378465951 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55683d0f74f0 0x55683d0fb3f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.823 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.810+0000 7fccd8579b80 1 -- 172.21.15.1:0/1378465951 >> 172.21.15.1:0/1378465951 conn(0x55683d0ec3e0 msgr2=0x55683d0f1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:43.823 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.810+0000 7fccd8579b80 1 -- 172.21.15.1:0/1378465951 shutdown_connections 2023-11-16T16:45:43.823 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.810+0000 7fccd8579b80 1 -- 172.21.15.1:0/1378465951 wait complete. 2023-11-16T16:45:43.823 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.810+0000 7fccd8579b80 1 -- start start 2023-11-16T16:45:43.823 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.810+0000 7fccd8579b80 1 Processor -- start 2023-11-16T16:45:43.823 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.811+0000 7fccd8579b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55683d0f6be0 0x55683d103040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:43.823 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.811+0000 7fccd8579b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55683d0f74f0 0x55683d1002c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:43.823 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.811+0000 7fccd8579b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55683d0fb930 0x55683d1008f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:43.823 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.811+0000 7fccd8579b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55683cfe9960 con 0x55683d0f6be0 2023-11-16T16:45:43.823 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.811+0000 7fccd8579b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55683cfe9600 con 0x55683d0fb930 2023-11-16T16:45:43.824 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.811+0000 7fccd8579b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55683cecb9a0 con 0x55683d0f74f0 2023-11-16T16:45:43.824 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.811+0000 7fccca189700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55683d0f6be0 0x55683d103040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:43.824 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.811+0000 7fccd8579b80 10 client.0 _ll_register_callbacks cb 0x55683d007800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:45:43.824 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.811+0000 7fccca189700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55683d0f6be0 0x55683d103040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45540/0 (socket says 172.21.15.1:45540) 2023-11-16T16:45:43.824 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.811+0000 7fccca189700 1 -- 172.21.15.1:0/3712095874 learned_addr learned my addr 172.21.15.1:0/3712095874 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:45:43.824 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.811+0000 7fccc9988700 1 --2- 172.21.15.1:0/3712095874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55683d0f74f0 0x55683d1002c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:43.824 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.811+0000 7fccca98a700 1 --2- 172.21.15.1:0/3712095874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55683d0fb930 0x55683d1008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:43.824 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.811+0000 7fccca189700 1 -- 172.21.15.1:0/3712095874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55683d0f74f0 msgr2=0x55683d1002c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:43.824 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.811+0000 7fccca189700 1 --2- 172.21.15.1:0/3712095874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55683d0f74f0 0x55683d1002c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.824 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.811+0000 7fccca189700 1 -- 172.21.15.1:0/3712095874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55683d0fb930 msgr2=0x55683d1008f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:43.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.811+0000 7fccca189700 1 --2- 172.21.15.1:0/3712095874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55683d0fb930 0x55683d1008f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.811+0000 7fccca189700 1 -- 172.21.15.1:0/3712095874 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccb40098e0 con 0x55683d0f6be0 2023-11-16T16:45:43.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.811+0000 7fccca189700 1 --2- 172.21.15.1:0/3712095874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55683d0f6be0 0x55683d103040 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fccc000fab0 tx=0x7fccc0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=931c98eaa88695ba server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:43.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.811+0000 7fccadffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:45:43.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.811+0000 7fccadffb700 1 -- 172.21.15.1:0/3712095874 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc0005450 con 0x55683d0f6be0 2023-11-16T16:45:43.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.811+0000 7fccd8579b80 1 -- 172.21.15.1:0/3712095874 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55683d0e9ba0 con 0x55683d0f6be0 2023-11-16T16:45:43.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.811+0000 7fccadffb700 1 -- 172.21.15.1:0/3712095874 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fccc00055b0 con 0x55683d0f6be0 2023-11-16T16:45:43.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.812+0000 7fccadffb700 1 -- 172.21.15.1:0/3712095874 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccc0003680 con 0x55683d0f6be0 2023-11-16T16:45:43.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.812+0000 7fccd8579b80 10 client.16284 fetch_fsmap learned FSMap version 28 2023-11-16T16:45:43.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.812+0000 7fccd8579b80 1 -- 172.21.15.1:0/3712095874 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55683d00e210 con 0x55683d0f6be0 2023-11-16T16:45:43.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.812+0000 7fccadffb700 1 -- 172.21.15.1:0/3712095874 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55683d0e9ba0 con 0x55683d0f6be0 2023-11-16T16:45:43.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.812+0000 7fccadffb700 1 -- 172.21.15.1:0/3712095874 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fccc002f680 con 0x55683d0f6be0 2023-11-16T16:45:43.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.812+0000 7fccd8579b80 10 client.16284 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:45:43.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.812+0000 7fccd8579b80 10 client.16284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:45:43.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.812+0000 7fccd8579b80 20 client.16284 populate_metadata read hostname 'smithi001' 2023-11-16T16:45:43.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.812+0000 7fccd8579b80 1 --2- 172.21.15.1:0/3712095874 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55683d110960 0x55683d112d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:43.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.812+0000 7fccd8579b80 4 client.16284 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:45:43.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.812+0000 7fccd8579b80 1 -- 172.21.15.1:0/3712095874 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55683d00a120 con 0x55683d110960 2023-11-16T16:45:43.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.812+0000 7fccc9988700 1 --2- 172.21.15.1:0/3712095874 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55683d110960 0x55683d112d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:43.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.813+0000 7fccadffb700 10 client.16284 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:45:43.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.813+0000 7fccc9988700 1 --2- 172.21.15.1:0/3712095874 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55683d110960 0x55683d112d70 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4307a36c48d6cc92 server_cookie=8c3e4862e90df104 in_seq=0 out_seq=0 2023-11-16T16:45:43.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.813+0000 7fccadffb700 1 -- 172.21.15.1:0/3712095874 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55683d00a120 con 0x55683d110960 2023-11-16T16:45:43.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.813+0000 7fccadffb700 10 client.16284 handle_command_reply: tid=0 2023-11-16T16:45:43.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.813+0000 7fccd8579b80 1 client.16284 shutdown 2023-11-16T16:45:43.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.813+0000 7fccd8579b80 1 -- 172.21.15.1:0/3712095874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55683d0f6be0 msgr2=0x55683d103040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:43.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.813+0000 7fccd8579b80 1 --2- 172.21.15.1:0/3712095874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55683d0f6be0 0x55683d103040 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fccc000fab0 tx=0x7fccc0009260 comp rx=0 tx=0).stop 2023-11-16T16:45:43.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.813+0000 7fccd8579b80 1 -- 172.21.15.1:0/3712095874 shutdown_connections 2023-11-16T16:45:43.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.813+0000 7fccd8579b80 1 --2- 172.21.15.1:0/3712095874 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55683d110960 0x55683d112d70 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.813+0000 7fccd8579b80 1 --2- 172.21.15.1:0/3712095874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55683d0fb930 0x55683d1008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.813+0000 7fccadffb700 0 client.16284 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:45:43.828 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.813+0000 7fccd8579b80 1 --2- 172.21.15.1:0/3712095874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55683d0f6be0 0x55683d103040 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.828 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.813+0000 7fccd8579b80 1 --2- 172.21.15.1:0/3712095874 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55683d0f74f0 0x55683d1002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.828 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.813+0000 7fccd8579b80 1 -- 172.21.15.1:0/3712095874 >> 172.21.15.1:0/3712095874 conn(0x55683d0ec3e0 msgr2=0x55683d0f59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:43.828 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.814+0000 7fccd8579b80 1 -- 172.21.15.1:0/3712095874 shutdown_connections 2023-11-16T16:45:43.828 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.814+0000 7fccd8579b80 1 -- 172.21.15.1:0/3712095874 wait complete. 2023-11-16T16:45:43.828 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.814+0000 7fccd8579b80 20 client.16284 trim_cache size 0 max 16384 2023-11-16T16:45:43.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.831+0000 7fccd8579b80 1 Processor -- start 2023-11-16T16:45:43.840 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.831+0000 7fccd8579b80 1 -- start start 2023-11-16T16:45:43.840 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.832+0000 7fccd8579b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55683d0f6be0 0x55683d0f60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:43.840 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.832+0000 7fccd8579b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55683d0f74f0 0x55683d102be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:43.840 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.832+0000 7fccd8579b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55683d0fb930 0x55683d0ffa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:43.840 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.832+0000 7fccd8579b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55683d0fa1c0 con 0x55683d0f6be0 2023-11-16T16:45:43.840 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.832+0000 7fccd8579b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55683d0f3240 con 0x55683d0fb930 2023-11-16T16:45:43.840 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.832+0000 7fccd8579b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55683d0fe820 con 0x55683d0f74f0 2023-11-16T16:45:43.840 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.832+0000 7fccca189700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55683d0f6be0 0x55683d0f60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:43.840 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.832+0000 7fccca189700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55683d0f6be0 0x55683d0f60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45548/0 (socket says 172.21.15.1:45548) 2023-11-16T16:45:43.840 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.832+0000 7fccca189700 1 -- 172.21.15.1:0/1295762034 learned_addr learned my addr 172.21.15.1:0/1295762034 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:45:43.840 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.832+0000 7fccc9988700 1 --2- 172.21.15.1:0/1295762034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55683d0f74f0 0x55683d102be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:43.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.832+0000 7fccca98a700 1 --2- 172.21.15.1:0/1295762034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55683d0fb930 0x55683d0ffa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:43.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.832+0000 7fccca189700 1 -- 172.21.15.1:0/1295762034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55683d0f74f0 msgr2=0x55683d102be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:43.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.832+0000 7fccca189700 1 --2- 172.21.15.1:0/1295762034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55683d0f74f0 0x55683d102be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.832+0000 7fccca189700 1 -- 172.21.15.1:0/1295762034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55683d0fb930 msgr2=0x55683d0ffa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:43.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.832+0000 7fccca189700 1 --2- 172.21.15.1:0/1295762034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55683d0fb930 0x55683d0ffa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.832+0000 7fccca189700 1 -- 172.21.15.1:0/1295762034 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55683d00df60 con 0x55683d0f6be0 2023-11-16T16:45:43.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.832+0000 7fccc9988700 1 --2- 172.21.15.1:0/1295762034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55683d0f74f0 0x55683d102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:45:43.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.832+0000 7fccca189700 1 --2- 172.21.15.1:0/1295762034 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55683d0f6be0 0x55683d0f60f0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fccc0005a80 tx=0x7fccc002fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c65c48278f9c6de0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:43.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.832+0000 7fccb95ad700 1 -- 172.21.15.1:0/1295762034 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc002f680 con 0x55683d0f6be0 2023-11-16T16:45:43.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.832+0000 7fccb95ad700 1 -- 172.21.15.1:0/1295762034 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccc001d040 con 0x55683d0f6be0 2023-11-16T16:45:43.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.832+0000 7fccb95ad700 1 -- 172.21.15.1:0/1295762034 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc00038e0 con 0x55683d0f6be0 2023-11-16T16:45:43.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.833+0000 7fccd8579b80 1 -- 172.21.15.1:0/1295762034 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55683d0f6be0 msgr2=0x55683d0f60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:43.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.833+0000 7fccd8579b80 1 --2- 172.21.15.1:0/1295762034 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55683d0f6be0 0x55683d0f60f0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fccc0005a80 tx=0x7fccc002fc00 comp rx=0 tx=0).stop 2023-11-16T16:45:43.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.833+0000 7fccd8579b80 1 -- 172.21.15.1:0/1295762034 shutdown_connections 2023-11-16T16:45:43.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.833+0000 7fccd8579b80 1 --2- 172.21.15.1:0/1295762034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55683d0fb930 0x55683d0ffa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.833+0000 7fccd8579b80 1 --2- 172.21.15.1:0/1295762034 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55683d0f6be0 0x55683d0f60f0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.833+0000 7fccd8579b80 1 --2- 172.21.15.1:0/1295762034 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55683d0f74f0 0x55683d102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.833+0000 7fccd8579b80 1 -- 172.21.15.1:0/1295762034 >> 172.21.15.1:0/1295762034 conn(0x55683d0ec3e0 msgr2=0x55683d0f59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:43.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.833+0000 7fccd8579b80 1 -- 172.21.15.1:0/1295762034 shutdown_connections 2023-11-16T16:45:43.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.833+0000 7fccd8579b80 1 -- 172.21.15.1:0/1295762034 wait complete. 2023-11-16T16:45:43.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.833+0000 7fccd8579b80 1 -- start start 2023-11-16T16:45:43.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.834+0000 7fccd8579b80 1 Processor -- start 2023-11-16T16:45:43.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.834+0000 7fccd8579b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55683d0f6be0 0x55683d106060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:43.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.834+0000 7fccd8579b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55683d0f74f0 0x55683d1065a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:43.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.834+0000 7fccd8579b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55683d0fb930 0x55683d106ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:43.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.834+0000 7fccd8579b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55683d107ff0 con 0x55683d0fb930 2023-11-16T16:45:43.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.834+0000 7fccd8579b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55683d164830 con 0x55683d0f6be0 2023-11-16T16:45:43.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.834+0000 7fccd8579b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55683d164970 con 0x55683d0f74f0 2023-11-16T16:45:43.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.834+0000 7fccd8579b80 10 client.0 _ll_register_callbacks cb 0x7fccc000b560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:45:43.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.834+0000 7fccca98a700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55683d0fb930 0x55683d106ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:43.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.834+0000 7fccca98a700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55683d0fb930 0x55683d106ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45558/0 (socket says 172.21.15.1:45558) 2023-11-16T16:45:43.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.834+0000 7fccca98a700 1 -- 172.21.15.1:0/1310366800 learned_addr learned my addr 172.21.15.1:0/1310366800 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:45:43.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.834+0000 7fccc9988700 1 --2- 172.21.15.1:0/1310366800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55683d0f74f0 0x55683d1065a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:43.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.834+0000 7fccca189700 1 --2- 172.21.15.1:0/1310366800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55683d0f6be0 0x55683d106060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:43.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.834+0000 7fccca98a700 1 -- 172.21.15.1:0/1310366800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55683d0f74f0 msgr2=0x55683d1065a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:43.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.834+0000 7fccca98a700 1 --2- 172.21.15.1:0/1310366800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55683d0f74f0 0x55683d1065a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.834+0000 7fccca98a700 1 -- 172.21.15.1:0/1310366800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55683d0f6be0 msgr2=0x55683d106060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:43.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.834+0000 7fccca98a700 1 --2- 172.21.15.1:0/1310366800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55683d0f6be0 0x55683d106060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.834+0000 7fccca98a700 1 -- 172.21.15.1:0/1310366800 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccb401d040 con 0x55683d0fb930 2023-11-16T16:45:43.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.834+0000 7fccca98a700 1 --2- 172.21.15.1:0/1310366800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55683d0fb930 0x55683d106ae0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fccbc009fd0 tx=0x7fccbc099960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39aa477fcdcfe973 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:43.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.834+0000 7fccadffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:45:43.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.834+0000 7fccadffb700 1 -- 172.21.15.1:0/1310366800 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc09dd40 con 0x55683d0fb930 2023-11-16T16:45:43.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.835+0000 7fccadffb700 1 -- 172.21.15.1:0/1310366800 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fccbc0a20d0 con 0x55683d0fb930 2023-11-16T16:45:43.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.835+0000 7fccd8579b80 1 -- 172.21.15.1:0/1310366800 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fccb4011e10 con 0x55683d0fb930 2023-11-16T16:45:43.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.835+0000 7fccadffb700 1 -- 172.21.15.1:0/1310366800 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccbc0092c0 con 0x55683d0fb930 2023-11-16T16:45:43.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.835+0000 7fccd8579b80 10 client.16290 fetch_fsmap learned FSMap version 28 2023-11-16T16:45:43.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.835+0000 7fccd8579b80 1 -- 172.21.15.1:0/1310366800 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55683d00e210 con 0x55683d0fb930 2023-11-16T16:45:43.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.835+0000 7fccadffb700 1 -- 172.21.15.1:0/1310366800 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccbc063080 con 0x55683d0fb930 2023-11-16T16:45:43.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.835+0000 7fccadffb700 1 -- 172.21.15.1:0/1310366800 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fccbc0b2270 con 0x55683d0fb930 2023-11-16T16:45:43.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.835+0000 7fccd8579b80 10 client.16290 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:45:43.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.835+0000 7fccd8579b80 10 client.16290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:45:43.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.835+0000 7fccd8579b80 20 client.16290 populate_metadata read hostname 'smithi001' 2023-11-16T16:45:43.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.835+0000 7fccd8579b80 1 --2- 172.21.15.1:0/1310366800 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55683d166090 0x55683d1684a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:43.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.835+0000 7fccd8579b80 4 client.16290 mds_command: new command op to 24521 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-16T16:45:43.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.835+0000 7fccd8579b80 1 -- 172.21.15.1:0/1310366800 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55683d0166e0 con 0x55683d166090 2023-11-16T16:45:43.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.835+0000 7fccca189700 1 --2- 172.21.15.1:0/1310366800 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55683d166090 0x55683d1684a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:43.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.836+0000 7fccadffb700 10 client.16290 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:45:43.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.836+0000 7fccca189700 1 --2- 172.21.15.1:0/1310366800 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55683d166090 0x55683d1684a0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17909dba5084613c server_cookie=94adb1985c18c3a2 in_seq=0 out_seq=0 2023-11-16T16:45:43.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.837+0000 7fccadffb700 1 -- 172.21.15.1:0/1310366800 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55683d0166e0 con 0x55683d166090 2023-11-16T16:45:43.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.837+0000 7fccadffb700 10 client.16290 handle_command_reply: tid=0 2023-11-16T16:45:43.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.837+0000 7fccd8579b80 1 client.16290 shutdown 2023-11-16T16:45:43.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.837+0000 7fccd8579b80 1 -- 172.21.15.1:0/1310366800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55683d0fb930 msgr2=0x55683d106ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:43.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.837+0000 7fccd8579b80 1 --2- 172.21.15.1:0/1310366800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55683d0fb930 0x55683d106ae0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fccbc009fd0 tx=0x7fccbc099960 comp rx=0 tx=0).stop 2023-11-16T16:45:43.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.837+0000 7fccd8579b80 1 -- 172.21.15.1:0/1310366800 shutdown_connections 2023-11-16T16:45:43.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.837+0000 7fccd8579b80 1 --2- 172.21.15.1:0/1310366800 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55683d166090 0x55683d1684a0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.837+0000 7fccd8579b80 1 --2- 172.21.15.1:0/1310366800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55683d0fb930 0x55683d106ae0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.837+0000 7fccd8579b80 1 --2- 172.21.15.1:0/1310366800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55683d0f6be0 0x55683d106060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.837+0000 7fccd8579b80 1 --2- 172.21.15.1:0/1310366800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55683d0f74f0 0x55683d1065a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.837+0000 7fccd8579b80 1 -- 172.21.15.1:0/1310366800 >> 172.21.15.1:0/1310366800 conn(0x55683d0ec3e0 msgr2=0x55683d0f6270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:43.847 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.837+0000 7fccadffb700 0 client.16290 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:45:43.847 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.837+0000 7fccd8579b80 1 -- 172.21.15.1:0/1310366800 shutdown_connections 2023-11-16T16:45:43.847 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.837+0000 7fccd8579b80 1 -- 172.21.15.1:0/1310366800 wait complete. 2023-11-16T16:45:43.847 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.837+0000 7fccd8579b80 20 client.16290 trim_cache size 0 max 16384 2023-11-16T16:45:43.847 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:45:43.847 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2023-11-16T16:45:43.847 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "0e3f0bf3-962e-4365-9a99-316127493e01", 2023-11-16T16:45:43.847 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2023-11-16T16:45:43.847 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:45:43.848 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.838+0000 7fcccc3ed700 1 -- 172.21.15.1:0/806898269 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fccb00a12c0 msgr2=0x7fccb00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:43.848 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.838+0000 7fcccc3ed700 1 --2- 172.21.15.1:0/806898269 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fccb00a12c0 0x7fccb00a3770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fccc000c010 tx=0x7fccc0019040 comp rx=0 tx=0).stop 2023-11-16T16:45:43.848 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.838+0000 7fcccc3ed700 1 -- 172.21.15.1:0/806898269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fccc4105250 msgr2=0x7fccc4195bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:43.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.838+0000 7fcccc3ed700 1 --2- 172.21.15.1:0/806898269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fccc4105250 0x7fccc4195bc0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fccbc00e3c0 tx=0x7fccbc00e6d0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.838+0000 7fcccc3ed700 1 -- 172.21.15.1:0/806898269 shutdown_connections 2023-11-16T16:45:43.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.838+0000 7fcccc3ed700 1 --2- 172.21.15.1:0/806898269 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fccb00a12c0 0x7fccb00a3770 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.838+0000 7fcccc3ed700 1 --2- 172.21.15.1:0/806898269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fccc4105250 0x7fccc4195bc0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.838+0000 7fcccc3ed700 1 --2- 172.21.15.1:0/806898269 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fccc40ffa70 0x7fccc4195140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.838+0000 7fcccc3ed700 1 --2- 172.21.15.1:0/806898269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fccc4100420 0x7fccc4195680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.838+0000 7fcccc3ed700 1 -- 172.21.15.1:0/806898269 >> 172.21.15.1:0/806898269 conn(0x7fccc40763a0 msgr2=0x7fccc40fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:43.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.838+0000 7fcccc3ed700 1 -- 172.21.15.1:0/806898269 shutdown_connections 2023-11-16T16:45:43.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.838+0000 7fcccc3ed700 1 -- 172.21.15.1:0/806898269 wait complete. 2023-11-16T16:45:43.851 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-16T16:45:43.981 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.980+0000 7f03c10e8700 1 -- 172.21.15.1:0/502436889 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03bc067dc0 msgr2=0x7f03bc068230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:43.981 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.980+0000 7f03c10e8700 1 --2- 172.21.15.1:0/502436889 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03bc067dc0 0x7f03bc068230 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f03b00099c0 tx=0x7f03b0009cd0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.981 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.980+0000 7f03c10e8700 1 -- 172.21.15.1:0/502436889 shutdown_connections 2023-11-16T16:45:43.982 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.980+0000 7f03c10e8700 1 --2- 172.21.15.1:0/502436889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03bc068770 0x7f03bc10d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.982 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.980+0000 7f03c10e8700 1 --2- 172.21.15.1:0/502436889 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03bc1052a0 0x7f03bc105670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.982 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.980+0000 7f03c10e8700 1 --2- 172.21.15.1:0/502436889 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03bc067dc0 0x7f03bc068230 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.982 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.980+0000 7f03c10e8700 1 -- 172.21.15.1:0/502436889 >> 172.21.15.1:0/502436889 conn(0x7f03bc0763a0 msgr2=0x7f03bc0767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:43.982 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.980+0000 7f03c10e8700 1 -- 172.21.15.1:0/502436889 shutdown_connections 2023-11-16T16:45:43.982 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.980+0000 7f03c10e8700 1 -- 172.21.15.1:0/502436889 wait complete. 2023-11-16T16:45:43.984 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.981+0000 7f03c10e8700 1 Processor -- start 2023-11-16T16:45:43.984 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.981+0000 7f03c10e8700 1 -- start start 2023-11-16T16:45:43.984 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.982+0000 7f03c10e8700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03bc067dc0 0x7f03bc196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:43.984 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.982+0000 7f03c10e8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03bc068770 0x7f03bc1968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:43.984 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.982+0000 7f03c10e8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03bc1052a0 0x7f03bc19ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:43.984 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.982+0000 7f03c10e8700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc19b370 con 0x7f03bc1052a0 2023-11-16T16:45:43.984 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.982+0000 7f03c10e8700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc19b4e0 con 0x7f03bc068770 2023-11-16T16:45:43.984 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.982+0000 7f03c10e8700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc19b6a0 con 0x7f03bc067dc0 2023-11-16T16:45:43.984 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.982+0000 7f03bb59e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03bc1052a0 0x7f03bc19ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:43.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.982+0000 7f03bad9d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03bc067dc0 0x7f03bc196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:43.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.982+0000 7f03bb59e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03bc1052a0 0x7f03bc19ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45572/0 (socket says 172.21.15.1:45572) 2023-11-16T16:45:43.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.982+0000 7f03bb59e700 1 -- 172.21.15.1:0/2279414991 learned_addr learned my addr 172.21.15.1:0/2279414991 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:45:43.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.982+0000 7f03ba59c700 1 --2- 172.21.15.1:0/2279414991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03bc068770 0x7f03bc1968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:43.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.982+0000 7f03bad9d700 1 -- 172.21.15.1:0/2279414991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03bc068770 msgr2=0x7f03bc1968d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:43.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.982+0000 7f03bad9d700 1 --2- 172.21.15.1:0/2279414991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03bc068770 0x7f03bc1968d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.982+0000 7f03bad9d700 1 -- 172.21.15.1:0/2279414991 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03bc1052a0 msgr2=0x7f03bc19ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:43.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.982+0000 7f03bad9d700 1 --2- 172.21.15.1:0/2279414991 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03bc1052a0 0x7f03bc19ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.982+0000 7f03bad9d700 1 -- 172.21.15.1:0/2279414991 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b00095f0 con 0x7f03bc067dc0 2023-11-16T16:45:43.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.982+0000 7f03bb59e700 1 --2- 172.21.15.1:0/2279414991 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03bc1052a0 0x7f03bc19ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:45:43.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.982+0000 7f03ba59c700 1 --2- 172.21.15.1:0/2279414991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03bc068770 0x7f03bc1968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:45:43.985 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.982+0000 7f03bad9d700 1 --2- 172.21.15.1:0/2279414991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03bc067dc0 0x7f03bc196390 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00e9f0 tx=0x7f03ac00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:43.986 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.982+0000 7f03a3fff700 1 -- 172.21.15.1:0/2279414991 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac00c9c0 con 0x7f03bc067dc0 2023-11-16T16:45:43.986 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.982+0000 7f03a3fff700 1 -- 172.21.15.1:0/2279414991 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03ac010430 con 0x7f03bc067dc0 2023-11-16T16:45:43.986 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.982+0000 7f03c10e8700 1 -- 172.21.15.1:0/2279414991 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03bc110390 con 0x7f03bc067dc0 2023-11-16T16:45:43.986 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.982+0000 7f03a3fff700 1 -- 172.21.15.1:0/2279414991 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac018680 con 0x7f03bc067dc0 2023-11-16T16:45:43.986 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.983+0000 7f03c10e8700 1 -- 172.21.15.1:0/2279414991 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03bc110200 con 0x7f03bc067dc0 2023-11-16T16:45:43.987 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.985+0000 7f03a3fff700 1 -- 172.21.15.1:0/2279414991 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f03ac010860 con 0x7f03bc067dc0 2023-11-16T16:45:43.987 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.985+0000 7f03a3fff700 1 --2- 172.21.15.1:0/2279414991 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f03a40a1340 0x7f03a40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:43.987 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.986+0000 7f03ba59c700 1 --2- 172.21.15.1:0/2279414991 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f03a40a1340 0x7f03a40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:43.987 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.986+0000 7f03a3fff700 1 -- 172.21.15.1:0/2279414991 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f03ac014070 con 0x7f03bc067dc0 2023-11-16T16:45:43.987 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.986+0000 7f03ba59c700 1 --2- 172.21.15.1:0/2279414991 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f03a40a1340 0x7f03a40a37f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f03b0009990 tx=0x7f03b001a040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:43.992 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.986+0000 7f03cd274b80 1 Processor -- start 2023-11-16T16:45:43.992 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.987+0000 7f03cd274b80 1 -- start start 2023-11-16T16:45:43.992 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.987+0000 7f03cd274b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56536b617b60 0x56536b617f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:43.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.987+0000 7f03cd274b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56536b618470 0x56536b61c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:43.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.987+0000 7f03cd274b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56536b61c8b0 0x56536b61cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:43.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.987+0000 7f03cd274b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56536b51d060 con 0x56536b61c8b0 2023-11-16T16:45:43.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.987+0000 7f03cd274b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56536b56f8a0 con 0x56536b617b60 2023-11-16T16:45:43.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.987+0000 7f03cd274b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56536b568d30 con 0x56536b618470 2023-11-16T16:45:43.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.987+0000 7f03bb59e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56536b61c8b0 0x56536b61cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:43.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.987+0000 7f03bad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56536b617b60 0x56536b617f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:43.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.987+0000 7f03bad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56536b617b60 0x56536b617f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.1:39762/0 (socket says 172.21.15.1:39762) 2023-11-16T16:45:43.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.987+0000 7f03bad9d700 1 -- 172.21.15.1:0/140991700 learned_addr learned my addr 172.21.15.1:0/140991700 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:45:43.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.987+0000 7f03ba59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56536b618470 0x56536b61c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:43.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.987+0000 7f03bad9d700 1 -- 172.21.15.1:0/140991700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56536b618470 msgr2=0x56536b61c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:43.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.987+0000 7f03bad9d700 1 --2- 172.21.15.1:0/140991700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56536b618470 0x56536b61c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.987+0000 7f03bad9d700 1 -- 172.21.15.1:0/140991700 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56536b61c8b0 msgr2=0x56536b61cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:43.994 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.987+0000 7f03bad9d700 1 --2- 172.21.15.1:0/140991700 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56536b61c8b0 0x56536b61cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.994 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.987+0000 7f03bad9d700 1 -- 172.21.15.1:0/140991700 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56536b3ea630 con 0x56536b617b60 2023-11-16T16:45:43.994 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.988+0000 7f03bad9d700 1 --2- 172.21.15.1:0/140991700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56536b617b60 0x56536b617f30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f03ac096910 tx=0x7f03ac099900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aabcdc93ff7a3b1b server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:43.994 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.988+0000 7f03a17fa700 1 -- 172.21.15.1:0/140991700 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac0a22f0 con 0x56536b617b60 2023-11-16T16:45:43.994 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.988+0000 7f03a17fa700 1 -- 172.21.15.1:0/140991700 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03ac0105a0 con 0x56536b617b60 2023-11-16T16:45:43.994 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.988+0000 7f03a17fa700 1 -- 172.21.15.1:0/140991700 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac0995d0 con 0x56536b617b60 2023-11-16T16:45:43.994 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.988+0000 7f03cd274b80 1 -- 172.21.15.1:0/140991700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56536b617b60 msgr2=0x56536b617f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:43.994 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.988+0000 7f03cd274b80 1 --2- 172.21.15.1:0/140991700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56536b617b60 0x56536b617f30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f03ac096910 tx=0x7f03ac099900 comp rx=0 tx=0).stop 2023-11-16T16:45:43.994 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.988+0000 7f03cd274b80 1 -- 172.21.15.1:0/140991700 shutdown_connections 2023-11-16T16:45:43.994 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.988+0000 7f03cd274b80 1 --2- 172.21.15.1:0/140991700 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56536b61c8b0 0x56536b61cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.994 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.988+0000 7f03cd274b80 1 --2- 172.21.15.1:0/140991700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56536b617b60 0x56536b617f30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.994 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.988+0000 7f03cd274b80 1 --2- 172.21.15.1:0/140991700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56536b618470 0x56536b61c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.994 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.988+0000 7f03cd274b80 1 -- 172.21.15.1:0/140991700 >> 172.21.15.1:0/140991700 conn(0x56536b60d360 msgr2=0x56536b612b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:43.995 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.988+0000 7f03cd274b80 1 -- 172.21.15.1:0/140991700 shutdown_connections 2023-11-16T16:45:43.995 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.988+0000 7f03cd274b80 1 -- 172.21.15.1:0/140991700 wait complete. 2023-11-16T16:45:43.995 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.989+0000 7f03cd274b80 1 -- start start 2023-11-16T16:45:43.995 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.989+0000 7f03cd274b80 1 Processor -- start 2023-11-16T16:45:43.995 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.989+0000 7f03cd274b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56536b617b60 0x56536b6240c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:43.995 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.989+0000 7f03cd274b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56536b618470 0x56536b6212d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:43.995 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.989+0000 7f03cd274b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56536b61c8b0 0x56536b621a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:43.995 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.989+0000 7f03cd274b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56536b50a8e0 con 0x56536b617b60 2023-11-16T16:45:43.995 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.989+0000 7f03cd274b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56536b50a580 con 0x56536b61c8b0 2023-11-16T16:45:43.995 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.989+0000 7f03cd274b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56536b3ec920 con 0x56536b618470 2023-11-16T16:45:43.995 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.989+0000 7f03cd274b80 10 client.0 _ll_register_callbacks cb 0x56536b528780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:45:43.995 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.989+0000 7f03bad9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56536b617b60 0x56536b6240c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:43.995 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.990+0000 7f03bad9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56536b617b60 0x56536b6240c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45582/0 (socket says 172.21.15.1:45582) 2023-11-16T16:45:43.996 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.990+0000 7f03ba59c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56536b618470 0x56536b6212d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:43.996 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.990+0000 7f03bad9d700 1 -- 172.21.15.1:0/2420855871 learned_addr learned my addr 172.21.15.1:0/2420855871 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:45:43.996 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.990+0000 7f03bb59e700 1 --2- 172.21.15.1:0/2420855871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56536b61c8b0 0x56536b621a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:43.996 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.990+0000 7f03bad9d700 1 -- 172.21.15.1:0/2420855871 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56536b618470 msgr2=0x56536b6212d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:43.996 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.990+0000 7f03bad9d700 1 --2- 172.21.15.1:0/2420855871 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56536b618470 0x56536b6212d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.996 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.990+0000 7f03bad9d700 1 -- 172.21.15.1:0/2420855871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56536b61c8b0 msgr2=0x56536b621a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:43.996 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.990+0000 7f03bad9d700 1 --2- 172.21.15.1:0/2420855871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56536b61c8b0 0x56536b621a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:43.996 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.990+0000 7f03bad9d700 1 -- 172.21.15.1:0/2420855871 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03bc110200 con 0x56536b617b60 2023-11-16T16:45:43.996 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.990+0000 7f03bad9d700 1 --2- 172.21.15.1:0/2420855871 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56536b617b60 0x56536b6240c0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f03ac011fd0 tx=0x7f03ac09a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8583ba9500c7a50 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:43.996 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.990+0000 7f039eff5700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:45:43.996 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.990+0000 7f039eff5700 1 -- 172.21.15.1:0/2420855871 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac0a22f0 con 0x56536b617b60 2023-11-16T16:45:43.996 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.990+0000 7f039eff5700 1 -- 172.21.15.1:0/2420855871 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f03ac0a2070 con 0x56536b617b60 2023-11-16T16:45:43.996 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.990+0000 7f03cd274b80 1 -- 172.21.15.1:0/2420855871 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56536b60ab20 con 0x56536b617b60 2023-11-16T16:45:43.996 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.990+0000 7f039eff5700 1 -- 172.21.15.1:0/2420855871 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03ac0b80d0 con 0x56536b617b60 2023-11-16T16:45:43.997 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.990+0000 7f03cd274b80 10 client.16302 fetch_fsmap learned FSMap version 28 2023-11-16T16:45:43.997 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.990+0000 7f03cd274b80 1 -- 172.21.15.1:0/2420855871 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56536b52f190 con 0x56536b617b60 2023-11-16T16:45:43.997 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.991+0000 7f039eff5700 1 -- 172.21.15.1:0/2420855871 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03ac07c080 con 0x56536b617b60 2023-11-16T16:45:43.997 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.991+0000 7f039eff5700 1 -- 172.21.15.1:0/2420855871 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f03ac0adc80 con 0x56536b617b60 2023-11-16T16:45:43.997 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.991+0000 7f03cd274b80 10 client.16302 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:45:43.997 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.991+0000 7f03cd274b80 10 client.16302 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:45:43.997 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.991+0000 7f03cd274b80 20 client.16302 populate_metadata read hostname 'smithi001' 2023-11-16T16:45:43.997 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.991+0000 7f03cd274b80 1 --2- 172.21.15.1:0/2420855871 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56536b631440 0x56536b633850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:43.997 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.991+0000 7f03cd274b80 4 client.16302 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:45:43.997 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.991+0000 7f03cd274b80 1 -- 172.21.15.1:0/2420855871 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56536b52b0a0 con 0x56536b631440 2023-11-16T16:45:43.997 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.991+0000 7f03ba59c700 1 --2- 172.21.15.1:0/2420855871 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56536b631440 0x56536b633850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:43.997 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.991+0000 7f039eff5700 10 client.16302 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:45:43.997 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:43.991+0000 7f03ba59c700 1 --2- 172.21.15.1:0/2420855871 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56536b631440 0x56536b633850 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86871b1705ecef2f server_cookie=a16bfb866aeef980 in_seq=0 out_seq=0 2023-11-16T16:45:44.060 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.059+0000 7f039eff5700 1 -- 172.21.15.1:0/2420855871 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56536b52b0a0 con 0x56536b631440 2023-11-16T16:45:44.061 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.059+0000 7f039eff5700 10 client.16302 handle_command_reply: tid=0 2023-11-16T16:45:44.061 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.059+0000 7f03cd274b80 1 client.16302 shutdown 2023-11-16T16:45:44.061 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.059+0000 7f03cd274b80 1 -- 172.21.15.1:0/2420855871 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56536b617b60 msgr2=0x56536b6240c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:44.061 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.059+0000 7f03cd274b80 1 --2- 172.21.15.1:0/2420855871 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56536b617b60 0x56536b6240c0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f03ac011fd0 tx=0x7f03ac09a330 comp rx=0 tx=0).stop 2023-11-16T16:45:44.061 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.059+0000 7f03cd274b80 1 -- 172.21.15.1:0/2420855871 shutdown_connections 2023-11-16T16:45:44.061 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.059+0000 7f03cd274b80 1 --2- 172.21.15.1:0/2420855871 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56536b631440 0x56536b633850 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:44.061 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.059+0000 7f03cd274b80 1 --2- 172.21.15.1:0/2420855871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56536b61c8b0 0x56536b621a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:44.061 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.059+0000 7f039eff5700 0 client.16302 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:45:44.061 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.059+0000 7f03cd274b80 1 --2- 172.21.15.1:0/2420855871 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56536b617b60 0x56536b6240c0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:44.061 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.059+0000 7f03cd274b80 1 --2- 172.21.15.1:0/2420855871 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56536b618470 0x56536b6212d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:44.061 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.059+0000 7f03cd274b80 1 -- 172.21.15.1:0/2420855871 >> 172.21.15.1:0/2420855871 conn(0x56536b60d360 msgr2=0x56536b611550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:44.062 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.059+0000 7f03cd274b80 1 -- 172.21.15.1:0/2420855871 shutdown_connections 2023-11-16T16:45:44.062 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.059+0000 7f03cd274b80 1 -- 172.21.15.1:0/2420855871 wait complete. 2023-11-16T16:45:44.062 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.060+0000 7f03cd274b80 20 client.16302 trim_cache size 0 max 16384 2023-11-16T16:45:44.081 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.076+0000 7f03cd274b80 1 Processor -- start 2023-11-16T16:45:44.081 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.077+0000 7f03cd274b80 1 -- start start 2023-11-16T16:45:44.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.077+0000 7f03cd274b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56536b617b60 0x56536b612670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:44.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.077+0000 7f03cd274b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56536b618470 0x56536b612bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:44.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.077+0000 7f03cd274b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56536b61c8b0 0x56536b620bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:44.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.077+0000 7f03cd274b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56536b60cee0 con 0x56536b617b60 2023-11-16T16:45:44.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.077+0000 7f03cd274b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56536b61b4a0 con 0x56536b618470 2023-11-16T16:45:44.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.077+0000 7f03cd274b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56536b617f30 con 0x56536b61c8b0 2023-11-16T16:45:44.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.077+0000 7f03bad9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56536b617b60 0x56536b612670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:44.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.077+0000 7f03bad9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56536b617b60 0x56536b612670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45590/0 (socket says 172.21.15.1:45590) 2023-11-16T16:45:44.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.077+0000 7f03bad9d700 1 -- 172.21.15.1:0/3900299067 learned_addr learned my addr 172.21.15.1:0/3900299067 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:45:44.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.077+0000 7f03ba59c700 1 --2- 172.21.15.1:0/3900299067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56536b618470 0x56536b612bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:44.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.077+0000 7f03bb59e700 1 --2- 172.21.15.1:0/3900299067 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56536b61c8b0 0x56536b620bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:44.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.077+0000 7f03bad9d700 1 -- 172.21.15.1:0/3900299067 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56536b61c8b0 msgr2=0x56536b620bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:44.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.077+0000 7f03bad9d700 1 --2- 172.21.15.1:0/3900299067 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56536b61c8b0 0x56536b620bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:44.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.077+0000 7f03bad9d700 1 -- 172.21.15.1:0/3900299067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56536b618470 msgr2=0x56536b612bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:44.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.077+0000 7f03bad9d700 1 --2- 172.21.15.1:0/3900299067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56536b618470 0x56536b612bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:44.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.077+0000 7f03bad9d700 1 -- 172.21.15.1:0/3900299067 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56536b60eb00 con 0x56536b617b60 2023-11-16T16:45:44.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.077+0000 7f03bad9d700 1 --2- 172.21.15.1:0/3900299067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56536b617b60 0x56536b612670 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00c5b0 tx=0x7f03ac010c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56b7f7a3b4e7e0f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:44.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.077+0000 7f03a1ffb700 1 -- 172.21.15.1:0/3900299067 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac005040 con 0x56536b617b60 2023-11-16T16:45:44.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.078+0000 7f03a1ffb700 1 -- 172.21.15.1:0/3900299067 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56536b60ab20 con 0x56536b617b60 2023-11-16T16:45:44.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.078+0000 7f03a1ffb700 1 -- 172.21.15.1:0/3900299067 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac0ca2b0 con 0x56536b617b60 2023-11-16T16:45:44.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.078+0000 7f03cd274b80 1 -- 172.21.15.1:0/3900299067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56536b617b60 msgr2=0x56536b612670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:44.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.078+0000 7f03cd274b80 1 --2- 172.21.15.1:0/3900299067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56536b617b60 0x56536b612670 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00c5b0 tx=0x7f03ac010c60 comp rx=0 tx=0).stop 2023-11-16T16:45:44.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.078+0000 7f03cd274b80 1 -- 172.21.15.1:0/3900299067 shutdown_connections 2023-11-16T16:45:44.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.078+0000 7f03cd274b80 1 --2- 172.21.15.1:0/3900299067 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56536b61c8b0 0x56536b620bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:44.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.078+0000 7f03cd274b80 1 --2- 172.21.15.1:0/3900299067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56536b617b60 0x56536b612670 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:44.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.078+0000 7f03cd274b80 1 --2- 172.21.15.1:0/3900299067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56536b618470 0x56536b612bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:44.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.078+0000 7f03cd274b80 1 -- 172.21.15.1:0/3900299067 >> 172.21.15.1:0/3900299067 conn(0x56536b60d360 msgr2=0x56536b61ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:44.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.078+0000 7f03cd274b80 1 -- 172.21.15.1:0/3900299067 shutdown_connections 2023-11-16T16:45:44.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.078+0000 7f03cd274b80 1 -- 172.21.15.1:0/3900299067 wait complete. 2023-11-16T16:45:44.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.079+0000 7f03cd274b80 1 -- start start 2023-11-16T16:45:44.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.079+0000 7f03cd274b80 1 Processor -- start 2023-11-16T16:45:44.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.079+0000 7f03cd274b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56536b617b60 0x56536b626990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:44.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.079+0000 7f03cd274b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56536b618470 0x56536b626ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:44.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.079+0000 7f03cd274b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56536b61c8b0 0x56536b627410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:44.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.079+0000 7f03cd274b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56536b6842c0 con 0x56536b618470 2023-11-16T16:45:44.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.079+0000 7f03cd274b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56536b684400 con 0x56536b617b60 2023-11-16T16:45:44.085 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.079+0000 7f03cd274b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56536b684570 con 0x56536b61c8b0 2023-11-16T16:45:44.085 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.079+0000 7f03cd274b80 10 client.0 _ll_register_callbacks cb 0x7f03b4009f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:45:44.085 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.079+0000 7f03ba59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56536b618470 0x56536b626ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:44.085 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.079+0000 7f03ba59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56536b618470 0x56536b626ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45598/0 (socket says 172.21.15.1:45598) 2023-11-16T16:45:44.085 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.079+0000 7f03ba59c700 1 -- 172.21.15.1:0/3578377438 learned_addr learned my addr 172.21.15.1:0/3578377438 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:45:44.085 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.079+0000 7f03bb59e700 1 --2- 172.21.15.1:0/3578377438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56536b61c8b0 0x56536b627410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:44.085 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.079+0000 7f03bad9d700 1 --2- 172.21.15.1:0/3578377438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56536b617b60 0x56536b626990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:44.085 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.079+0000 7f03bad9d700 1 -- 172.21.15.1:0/3578377438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56536b61c8b0 msgr2=0x56536b627410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:44.085 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.079+0000 7f03bad9d700 1 --2- 172.21.15.1:0/3578377438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56536b61c8b0 0x56536b627410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:44.085 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.079+0000 7f03bad9d700 1 -- 172.21.15.1:0/3578377438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56536b618470 msgr2=0x56536b626ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:45:44.085 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.079+0000 7f03bad9d700 1 --2- 172.21.15.1:0/3578377438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56536b618470 0x56536b626ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:44.085 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.079+0000 7f03bad9d700 1 -- 172.21.15.1:0/3578377438 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56536b52f190 con 0x56536b617b60 2023-11-16T16:45:44.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.079+0000 7f03ba59c700 1 --2- 172.21.15.1:0/3578377438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56536b618470 0x56536b626ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:45:44.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.079+0000 7f03bb59e700 1 --2- 172.21.15.1:0/3578377438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56536b61c8b0 0x56536b627410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:45:44.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.080+0000 7f03bad9d700 1 --2- 172.21.15.1:0/3578377438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56536b617b60 0x56536b626990 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f03ac099ba0 tx=0x7f03ac0ca620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=169a3bc9752dbd95 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:45:44.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.080+0000 7f039eff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-11-16T16:45:44.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.080+0000 7f039eff5700 1 -- 172.21.15.1:0/3578377438 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac095ea0 con 0x56536b617b60 2023-11-16T16:45:44.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.080+0000 7f03cd274b80 1 -- 172.21.15.1:0/3578377438 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f03ac0105a0 con 0x56536b617b60 2023-11-16T16:45:44.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.080+0000 7f039eff5700 1 -- 172.21.15.1:0/3578377438 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f03ac0a22f0 con 0x56536b617b60 2023-11-16T16:45:44.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.080+0000 7f039eff5700 1 -- 172.21.15.1:0/3578377438 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03ac099ea0 con 0x56536b617b60 2023-11-16T16:45:44.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.080+0000 7f03cd274b80 10 client.25871 fetch_fsmap learned FSMap version 28 2023-11-16T16:45:44.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.080+0000 7f03cd274b80 1 -- 172.21.15.1:0/3578377438 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56536b396de0 con 0x56536b617b60 2023-11-16T16:45:44.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.080+0000 7f039eff5700 1 -- 172.21.15.1:0/3578377438 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03ac0105a0 con 0x56536b617b60 2023-11-16T16:45:44.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.080+0000 7f039eff5700 1 -- 172.21.15.1:0/3578377438 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f03ac094bd0 con 0x56536b617b60 2023-11-16T16:45:44.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.080+0000 7f03cd274b80 10 client.25871 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:45:44.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.080+0000 7f03cd274b80 10 client.25871 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:45:44.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.080+0000 7f03cd274b80 20 client.25871 populate_metadata read hostname 'smithi001' 2023-11-16T16:45:44.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.080+0000 7f03cd274b80 1 --2- 172.21.15.1:0/3578377438 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56536b6857c0 0x56536b687bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:45:44.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.080+0000 7f03cd274b80 4 client.25871 mds_command: new command op to 24521 tid=0[{"prefix": "scrub status"}] 2023-11-16T16:45:44.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.080+0000 7f03cd274b80 1 -- 172.21.15.1:0/3578377438 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56536b537660 con 0x56536b6857c0 2023-11-16T16:45:44.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.081+0000 7f03ba59c700 1 --2- 172.21.15.1:0/3578377438 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56536b6857c0 0x56536b687bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:45:44.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.081+0000 7f039eff5700 10 client.25871 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:45:44.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.081+0000 7f03ba59c700 1 --2- 172.21.15.1:0/3578377438 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56536b6857c0 0x56536b687bd0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5334e3961dc0e74e server_cookie=eb278ef7b79bbd63 in_seq=0 out_seq=0 2023-11-16T16:45:44.092 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.091+0000 7f039eff5700 1 -- 172.21.15.1:0/3578377438 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56536b537660 con 0x56536b6857c0 2023-11-16T16:45:44.093 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.091+0000 7f039eff5700 10 client.25871 handle_command_reply: tid=0 2023-11-16T16:45:44.093 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.091+0000 7f03cd274b80 1 client.25871 shutdown 2023-11-16T16:45:44.093 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.091+0000 7f03cd274b80 1 -- 172.21.15.1:0/3578377438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56536b617b60 msgr2=0x56536b626990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:44.093 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.091+0000 7f03cd274b80 1 --2- 172.21.15.1:0/3578377438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56536b617b60 0x56536b626990 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f03ac099ba0 tx=0x7f03ac0ca620 comp rx=0 tx=0).stop 2023-11-16T16:45:44.093 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.091+0000 7f03cd274b80 1 -- 172.21.15.1:0/3578377438 shutdown_connections 2023-11-16T16:45:44.093 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.091+0000 7f03cd274b80 1 --2- 172.21.15.1:0/3578377438 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56536b6857c0 0x56536b687bd0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:44.093 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.091+0000 7f03cd274b80 1 --2- 172.21.15.1:0/3578377438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56536b61c8b0 0x56536b627410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:44.093 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.091+0000 7f03cd274b80 1 --2- 172.21.15.1:0/3578377438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56536b617b60 0x56536b626990 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:44.093 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.091+0000 7f03cd274b80 1 --2- 172.21.15.1:0/3578377438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56536b618470 0x56536b626ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:44.093 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.091+0000 7f039eff5700 0 client.25871 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:45:44.094 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.091+0000 7f03cd274b80 1 -- 172.21.15.1:0/3578377438 >> 172.21.15.1:0/3578377438 conn(0x56536b60d360 msgr2=0x56536b61b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:44.094 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.091+0000 7f03cd274b80 1 -- 172.21.15.1:0/3578377438 shutdown_connections 2023-11-16T16:45:44.094 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.091+0000 7f03cd274b80 1 -- 172.21.15.1:0/3578377438 wait complete. 2023-11-16T16:45:44.094 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.092+0000 7f03cd274b80 20 client.25871 trim_cache size 0 max 16384 2023-11-16T16:45:44.094 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:45:44.094 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-16T16:45:44.094 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2023-11-16T16:45:44.094 INFO:teuthology.orchestra.run.smithi001.stdout: "0e3f0bf3-962e-4365-9a99-316127493e01": { 2023-11-16T16:45:44.094 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2023-11-16T16:45:44.095 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "0e3f0bf3-962e-4365-9a99-316127493e01", 2023-11-16T16:45:44.095 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2023-11-16T16:45:44.095 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:45:44.095 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:45:44.095 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:45:44.096 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.092+0000 7f03c10e8700 1 -- 172.21.15.1:0/2279414991 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f03a40a1340 msgr2=0x7f03a40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:44.096 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.092+0000 7f03c10e8700 1 --2- 172.21.15.1:0/2279414991 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f03a40a1340 0x7f03a40a37f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f03b0009990 tx=0x7f03b001a040 comp rx=0 tx=0).stop 2023-11-16T16:45:44.096 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.092+0000 7f03c10e8700 1 -- 172.21.15.1:0/2279414991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03bc067dc0 msgr2=0x7f03bc196390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:45:44.096 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.092+0000 7f03c10e8700 1 --2- 172.21.15.1:0/2279414991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03bc067dc0 0x7f03bc196390 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00e9f0 tx=0x7f03ac00ed00 comp rx=0 tx=0).stop 2023-11-16T16:45:44.096 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.092+0000 7f03c10e8700 1 -- 172.21.15.1:0/2279414991 shutdown_connections 2023-11-16T16:45:44.096 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.092+0000 7f03c10e8700 1 --2- 172.21.15.1:0/2279414991 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f03a40a1340 0x7f03a40a37f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:44.096 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.092+0000 7f03c10e8700 1 --2- 172.21.15.1:0/2279414991 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03bc1052a0 0x7f03bc19ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:44.096 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.092+0000 7f03c10e8700 1 --2- 172.21.15.1:0/2279414991 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03bc067dc0 0x7f03bc196390 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:44.096 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.092+0000 7f03c10e8700 1 --2- 172.21.15.1:0/2279414991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03bc068770 0x7f03bc1968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:45:44.096 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.092+0000 7f03c10e8700 1 -- 172.21.15.1:0/2279414991 >> 172.21.15.1:0/2279414991 conn(0x7f03bc0763a0 msgr2=0x7f03bc0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:45:44.097 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.092+0000 7f03c10e8700 1 -- 172.21.15.1:0/2279414991 shutdown_connections 2023-11-16T16:45:44.097 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:45:44.092+0000 7f03c10e8700 1 -- 172.21.15.1:0/2279414991 wait complete. 2023-11-16T16:45:44.105 INFO:tasks.cephfs.filesystem:scrub status for tag:0e3f0bf3-962e-4365-9a99-316127493e01 - {'path': '/', 'tag': '0e3f0bf3-962e-4365-9a99-316127493e01', 'options': 'recursive,force'} 2023-11-16T16:45:44.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:44 smithi001 ceph-mon[98800]: pgmap v200: 97 pgs: 97 active+clean; 805 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 602 KiB/s wr, 206 op/s 2023-11-16T16:45:44.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:44 smithi001 ceph-mon[98800]: scrub queued for path: / 2023-11-16T16:45:44.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:44 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:45:44.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:44 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:45:44.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:44 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:45:44.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:44 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:45:45.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:44 smithi026 ceph-mon[100112]: pgmap v200: 97 pgs: 97 active+clean; 805 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 602 KiB/s wr, 206 op/s 2023-11-16T16:45:45.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:44 smithi026 ceph-mon[100112]: scrub queued for path: / 2023-11-16T16:45:45.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:44 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:45:45.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:44 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:45:45.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:44 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:45:45.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:44 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:45:45.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:44 smithi053 ceph-mon[100638]: pgmap v200: 97 pgs: 97 active+clean; 805 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 602 KiB/s wr, 206 op/s 2023-11-16T16:45:45.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:44 smithi053 ceph-mon[100638]: scrub queued for path: / 2023-11-16T16:45:45.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:44 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:45:45.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:44 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:45:45.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:44 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:45:45.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:44 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:45:45.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:45 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:45:45.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:45 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:45:45.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:45 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:45:45.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:45 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:45:46.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:45 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:45:46.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:45 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:45:46.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:45 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:45:46.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:45 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:45:46.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:45 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:45:46.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:45 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:45:46.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:45 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:45:46.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:45 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:45:46.780 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-16T16:45:46.780 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-16T16:45:46.820 INFO:tasks.workunit:Running workunit fs/misc/filelock_deadlock.py... 2023-11-16T16:45:46.821 DEBUG:teuthology.orchestra.run.smithi001:workunit test fs/misc/filelock_deadlock.py> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/filelock_deadlock.py 2023-11-16T16:45:46.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:46 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:45:46.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:46 smithi001 ceph-mon[98800]: pgmap v201: 97 pgs: 97 active+clean; 815 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.4 MiB/s wr, 565 op/s 2023-11-16T16:45:47.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:46 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:45:47.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:46 smithi026 ceph-mon[100112]: pgmap v201: 97 pgs: 97 active+clean; 815 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.4 MiB/s wr, 565 op/s 2023-11-16T16:45:47.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:46 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:45:47.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:46 smithi053 ceph-mon[100638]: pgmap v201: 97 pgs: 97 active+clean; 815 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.4 MiB/s wr, 565 op/s 2023-11-16T16:45:48.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:48 smithi001 ceph-mon[98800]: pgmap v202: 97 pgs: 97 active+clean; 815 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 897 KiB/s wr, 420 op/s 2023-11-16T16:45:49.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:48 smithi026 ceph-mon[100112]: pgmap v202: 97 pgs: 97 active+clean; 815 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 897 KiB/s wr, 420 op/s 2023-11-16T16:45:49.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:48 smithi053 ceph-mon[100638]: pgmap v202: 97 pgs: 97 active+clean; 815 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 897 KiB/s wr, 420 op/s 2023-11-16T16:45:50.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:50 smithi001 ceph-mon[98800]: pgmap v203: 97 pgs: 97 active+clean; 815 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 897 KiB/s wr, 420 op/s 2023-11-16T16:45:51.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:50 smithi026 ceph-mon[100112]: pgmap v203: 97 pgs: 97 active+clean; 815 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 897 KiB/s wr, 420 op/s 2023-11-16T16:45:51.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:50 smithi053 ceph-mon[100638]: pgmap v203: 97 pgs: 97 active+clean; 815 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 897 KiB/s wr, 420 op/s 2023-11-16T16:45:52.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:52 smithi053 ceph-mon[100638]: pgmap v204: 97 pgs: 97 active+clean; 816 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 958 KiB/s wr, 427 op/s 2023-11-16T16:45:52.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:52 smithi001 ceph-mon[98800]: pgmap v204: 97 pgs: 97 active+clean; 816 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 958 KiB/s wr, 427 op/s 2023-11-16T16:45:52.757 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:52 smithi026 ceph-mon[100112]: pgmap v204: 97 pgs: 97 active+clean; 816 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 958 KiB/s wr, 427 op/s 2023-11-16T16:45:54.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:54 smithi001 ceph-mon[98800]: pgmap v205: 97 pgs: 97 active+clean; 816 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 929 KiB/s wr, 366 op/s 2023-11-16T16:45:55.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:54 smithi026 ceph-mon[100112]: pgmap v205: 97 pgs: 97 active+clean; 816 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 929 KiB/s wr, 366 op/s 2023-11-16T16:45:55.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:54 smithi053 ceph-mon[100638]: pgmap v205: 97 pgs: 97 active+clean; 816 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 929 KiB/s wr, 366 op/s 2023-11-16T16:45:56.922 INFO:tasks.workunit.client.0.smithi001.stdout:ok 2023-11-16T16:45:56.927 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-16T16:45:56.928 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-16T16:45:56.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:56 smithi001 ceph-mon[98800]: pgmap v206: 97 pgs: 97 active+clean; 817 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 977 KiB/s wr, 371 op/s 2023-11-16T16:45:56.986 INFO:tasks.workunit:Running workunit fs/misc/filelock_interrupt.py... 2023-11-16T16:45:56.986 DEBUG:teuthology.orchestra.run.smithi001:workunit test fs/misc/filelock_interrupt.py> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/filelock_interrupt.py 2023-11-16T16:45:57.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:56 smithi026 ceph-mon[100112]: pgmap v206: 97 pgs: 97 active+clean; 817 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 977 KiB/s wr, 371 op/s 2023-11-16T16:45:57.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:56 smithi053 ceph-mon[100638]: pgmap v206: 97 pgs: 97 active+clean; 817 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 977 KiB/s wr, 371 op/s 2023-11-16T16:45:58.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:58 smithi001 ceph-mon[98800]: pgmap v207: 97 pgs: 97 active+clean; 817 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 109 KiB/s wr, 11 op/s 2023-11-16T16:45:59.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:58 smithi026 ceph-mon[100112]: pgmap v207: 97 pgs: 97 active+clean; 817 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 109 KiB/s wr, 11 op/s 2023-11-16T16:45:59.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:58 smithi053 ceph-mon[100638]: pgmap v207: 97 pgs: 97 active+clean; 817 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 109 KiB/s wr, 11 op/s 2023-11-16T16:45:59.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:59 smithi001 ceph-mon[98800]: scrub summary: idle 2023-11-16T16:45:59.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:45:59 smithi001 ceph-mon[98800]: scrub summary: idle 2023-11-16T16:46:00.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:59 smithi026 ceph-mon[100112]: scrub summary: idle 2023-11-16T16:46:00.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:45:59 smithi026 ceph-mon[100112]: scrub summary: idle 2023-11-16T16:46:00.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:59 smithi053 ceph-mon[100638]: scrub summary: idle 2023-11-16T16:46:00.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:45:59 smithi053 ceph-mon[100638]: scrub summary: idle 2023-11-16T16:46:00.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:00 smithi001 ceph-mon[98800]: scrub summary: idle 2023-11-16T16:46:00.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:00 smithi001 ceph-mon[98800]: pgmap v208: 97 pgs: 97 active+clean; 817 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 109 KiB/s wr, 11 op/s 2023-11-16T16:46:01.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:00 smithi026 ceph-mon[100112]: scrub summary: idle 2023-11-16T16:46:01.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:00 smithi026 ceph-mon[100112]: pgmap v208: 97 pgs: 97 active+clean; 817 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 109 KiB/s wr, 11 op/s 2023-11-16T16:46:01.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:00 smithi053 ceph-mon[100638]: scrub summary: idle 2023-11-16T16:46:01.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:00 smithi053 ceph-mon[100638]: pgmap v208: 97 pgs: 97 active+clean; 817 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 109 KiB/s wr, 11 op/s 2023-11-16T16:46:02.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:02 smithi053 ceph-mon[100638]: pgmap v209: 97 pgs: 97 active+clean; 817 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 162 KiB/s wr, 29 op/s 2023-11-16T16:46:02.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:02 smithi001 ceph-mon[98800]: pgmap v209: 97 pgs: 97 active+clean; 817 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 162 KiB/s wr, 29 op/s 2023-11-16T16:46:02.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:02 smithi026 ceph-mon[100112]: pgmap v209: 97 pgs: 97 active+clean; 817 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 162 KiB/s wr, 29 op/s 2023-11-16T16:46:04.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:04 smithi001 ceph-mon[98800]: pgmap v210: 97 pgs: 97 active+clean; 817 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 100 KiB/s wr, 22 op/s 2023-11-16T16:46:05.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:04 smithi026 ceph-mon[100112]: pgmap v210: 97 pgs: 97 active+clean; 817 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 100 KiB/s wr, 22 op/s 2023-11-16T16:46:05.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:04 smithi053 ceph-mon[100638]: pgmap v210: 97 pgs: 97 active+clean; 817 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 100 KiB/s wr, 22 op/s 2023-11-16T16:46:06.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:06 smithi001 ceph-mon[98800]: pgmap v211: 97 pgs: 97 active+clean; 817 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 105 KiB/s wr, 26 op/s 2023-11-16T16:46:07.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:06 smithi026 ceph-mon[100112]: pgmap v211: 97 pgs: 97 active+clean; 817 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 105 KiB/s wr, 26 op/s 2023-11-16T16:46:07.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:06 smithi053 ceph-mon[100638]: pgmap v211: 97 pgs: 97 active+clean; 817 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 105 KiB/s wr, 26 op/s 2023-11-16T16:46:07.084 INFO:tasks.workunit.client.0.smithi001.stdout:ok 2023-11-16T16:46:07.089 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-16T16:46:07.089 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-16T16:46:07.145 INFO:tasks.workunit:Running workunit fs/misc/general_vxattrs.sh... 2023-11-16T16:46:07.145 DEBUG:teuthology.orchestra.run.smithi001:workunit test fs/misc/general_vxattrs.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/general_vxattrs.sh 2023-11-16T16:46:07.190 INFO:tasks.workunit.client.0.smithi001.stderr:+ mkdir -p dir 2023-11-16T16:46:07.194 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.dir.pin dir 2023-11-16T16:46:07.194 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep 'ceph.dir.pin="-1"' 2023-11-16T16:46:07.195 INFO:tasks.workunit.client.0.smithi001.stdout:ceph.dir.pin="-1" 2023-11-16T16:46:07.196 INFO:tasks.workunit.client.0.smithi001.stderr:+ setfattr -n ceph.dir.pin dir 2023-11-16T16:46:07.196 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-11-16T16:46:07.197 INFO:tasks.workunit.client.0.smithi001.stdout:setfattr: dir: Invalid argument 2023-11-16T16:46:07.197 INFO:tasks.workunit.client.0.smithi001.stderr:+ setfattr -n ceph.dir.pin -v 1 dir 2023-11-16T16:46:07.199 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.dir.pin dir 2023-11-16T16:46:07.200 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep 'ceph.dir.pin="1"' 2023-11-16T16:46:07.201 INFO:tasks.workunit.client.0.smithi001.stdout:ceph.dir.pin="1" 2023-11-16T16:46:07.201 INFO:tasks.workunit.client.0.smithi001.stderr:+ setfattr -x ceph.dir.pin dir 2023-11-16T16:46:07.204 INFO:tasks.workunit.client.0.smithi001.stderr:+ getfattr -n ceph.dir.pin dir 2023-11-16T16:46:07.204 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep 'ceph.dir.pin="-1"' 2023-11-16T16:46:07.205 INFO:tasks.workunit.client.0.smithi001.stdout:ceph.dir.pin="-1" 2023-11-16T16:46:07.206 INFO:tasks.workunit.client.0.smithi001.stderr:+ setfattr -n ceph.dir.pin.distributed dir 2023-11-16T16:46:07.206 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-11-16T16:46:07.207 INFO:tasks.workunit.client.0.smithi001.stdout:setfattr: dir: Invalid argument 2023-11-16T16:46:07.207 INFO:tasks.workunit.client.0.smithi001.stderr:+ setfattr -n ceph.dir.pin.distributed -v 1 dir 2023-11-16T16:46:07.209 INFO:tasks.workunit.client.0.smithi001.stderr:+ setfattr -x ceph.dir.pin.distributed dir 2023-11-16T16:46:07.212 INFO:tasks.workunit.client.0.smithi001.stderr:+ setfattr -n ceph.dir.pin.random dir 2023-11-16T16:46:07.212 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-11-16T16:46:07.213 INFO:tasks.workunit.client.0.smithi001.stdout:setfattr: dir: Invalid argument 2023-11-16T16:46:07.213 INFO:tasks.workunit.client.0.smithi001.stderr:+ setfattr -n ceph.dir.pin.random -v 0.01 dir 2023-11-16T16:46:07.215 INFO:tasks.workunit.client.0.smithi001.stderr:+ setfattr -x ceph.dir.pin.random dir 2023-11-16T16:46:07.217 INFO:tasks.workunit.client.0.smithi001.stderr:+ setfattr -n ceph.dir.subvolume dir 2023-11-16T16:46:07.218 INFO:tasks.workunit.client.0.smithi001.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-11-16T16:46:07.218 INFO:tasks.workunit.client.0.smithi001.stdout:setfattr: dir: Invalid argument 2023-11-16T16:46:07.219 INFO:tasks.workunit.client.0.smithi001.stderr:+ setfattr -n ceph.dir.subvolume -v 1 dir 2023-11-16T16:46:07.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-16T16:46:07.231 INFO:tasks.workunit.client.0.smithi001.stderr:setfattr: dir: Invalid argument 2023-11-16T16:46:07.231 INFO:tasks.workunit:Stopping ['fs/misc'] on client.0... 2023-11-16T16:46:07.231 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-11-16T16:46:08.172 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/misc/general_vxattrs.sh) on smithi001 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/general_vxattrs.sh' 2023-11-16T16:46:08.267 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=e78be233e0634c3cbfd81992005a60b1 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/misc/general_vxattrs.sh) on smithi001 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/general_vxattrs.sh' 2023-11-16T16:46:08.270 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2023-11-16T16:46:08.280 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2023-11-16T16:46:08.280 INFO:teuthology.task.exec:Executing custom commands... 2023-11-16T16:46:08.280 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi001.front.sepia.ceph.com 2023-11-16T16:46:08.281 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2023-11-16T16:46:08.324 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2023-11-16T16:46:08.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:08 smithi001 ceph-mon[98800]: pgmap v212: 97 pgs: 97 active+clean; 817 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 57 KiB/s wr, 21 op/s 2023-11-16T16:46:09.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:08 smithi026 ceph-mon[100112]: pgmap v212: 97 pgs: 97 active+clean; 817 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 57 KiB/s wr, 21 op/s 2023-11-16T16:46:09.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:08 smithi053 ceph-mon[100638]: pgmap v212: 97 pgs: 97 active+clean; 817 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 57 KiB/s wr, 21 op/s 2023-11-16T16:46:10.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:10 smithi001 ceph-mon[98800]: pgmap v213: 97 pgs: 97 active+clean; 817 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 57 KiB/s wr, 21 op/s 2023-11-16T16:46:10.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:10 smithi001 ceph-mon[98800]: osdmap e88: 12 total, 12 up, 12 in 2023-11-16T16:46:11.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:10 smithi026 ceph-mon[100112]: pgmap v213: 97 pgs: 97 active+clean; 817 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 57 KiB/s wr, 21 op/s 2023-11-16T16:46:11.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:10 smithi026 ceph-mon[100112]: osdmap e88: 12 total, 12 up, 12 in 2023-11-16T16:46:11.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:10 smithi053 ceph-mon[100638]: pgmap v213: 97 pgs: 97 active+clean; 817 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 57 KiB/s wr, 21 op/s 2023-11-16T16:46:11.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:10 smithi053 ceph-mon[100638]: osdmap e88: 12 total, 12 up, 12 in 2023-11-16T16:46:12.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:12 smithi053 ceph-mon[100638]: pgmap v215: 97 pgs: 97 active+clean; 817 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 KiB/s wr, 8 op/s 2023-11-16T16:46:12.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:12 smithi001 ceph-mon[98800]: pgmap v215: 97 pgs: 97 active+clean; 817 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 KiB/s wr, 8 op/s 2023-11-16T16:46:12.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:12 smithi026 ceph-mon[100112]: pgmap v215: 97 pgs: 97 active+clean; 817 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 KiB/s wr, 8 op/s 2023-11-16T16:46:14.106 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-16T16:46:14.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.249+0000 7ff52dfa5700 1 -- 172.21.15.1:0/3241145770 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff528101660 msgr2=0x7ff528104e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:14.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.249+0000 7ff52dfa5700 1 --2- 172.21.15.1:0/3241145770 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff528101660 0x7ff528104e90 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7ff51c0099c0 tx=0x7ff51c009cd0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.249+0000 7ff52dfa5700 1 -- 172.21.15.1:0/3241145770 shutdown_connections 2023-11-16T16:46:14.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.249+0000 7ff52dfa5700 1 --2- 172.21.15.1:0/3241145770 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff5281055c0 0x7ff52810d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.249+0000 7ff52dfa5700 1 --2- 172.21.15.1:0/3241145770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff528100cc0 0x7ff528101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.249+0000 7ff52dfa5700 1 --2- 172.21.15.1:0/3241145770 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff528101660 0x7ff528104e90 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.249+0000 7ff52dfa5700 1 -- 172.21.15.1:0/3241145770 >> 172.21.15.1:0/3241145770 conn(0x7ff528077560 msgr2=0x7ff528077960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:14.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.249+0000 7ff52dfa5700 1 -- 172.21.15.1:0/3241145770 shutdown_connections 2023-11-16T16:46:14.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.249+0000 7ff52dfa5700 1 -- 172.21.15.1:0/3241145770 wait complete. 2023-11-16T16:46:14.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.249+0000 7ff52dfa5700 1 Processor -- start 2023-11-16T16:46:14.254 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.250+0000 7ff52dfa5700 1 -- start start 2023-11-16T16:46:14.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.250+0000 7ff52dfa5700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff528100cc0 0x7ff528196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.250+0000 7ff52dfa5700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff528101660 0x7ff5281968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.250+0000 7ff52dfa5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5281055c0 0x7ff52819ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.250+0000 7ff52dfa5700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52819b350 con 0x7ff528100cc0 2023-11-16T16:46:14.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.250+0000 7ff52dfa5700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52819b4c0 con 0x7ff5281055c0 2023-11-16T16:46:14.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.250+0000 7ff52dfa5700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52819b680 con 0x7ff528101660 2023-11-16T16:46:14.256 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.250+0000 7ff5277fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff528100cc0 0x7ff528196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.256 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.250+0000 7ff5277fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff528100cc0 0x7ff528196370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45610/0 (socket says 172.21.15.1:45610) 2023-11-16T16:46:14.256 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.250+0000 7ff5277fe700 1 -- 172.21.15.1:0/1047626766 learned_addr learned my addr 172.21.15.1:0/1047626766 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:14.256 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.250+0000 7ff526ffd700 1 --2- 172.21.15.1:0/1047626766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff528101660 0x7ff5281968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.256 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.250+0000 7ff527fff700 1 --2- 172.21.15.1:0/1047626766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5281055c0 0x7ff52819ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.256 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.250+0000 7ff5277fe700 1 -- 172.21.15.1:0/1047626766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff528101660 msgr2=0x7ff5281968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:14.257 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.250+0000 7ff5277fe700 1 --2- 172.21.15.1:0/1047626766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff528101660 0x7ff5281968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.257 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.250+0000 7ff5277fe700 1 -- 172.21.15.1:0/1047626766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5281055c0 msgr2=0x7ff52819ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:14.257 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.250+0000 7ff5277fe700 1 --2- 172.21.15.1:0/1047626766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5281055c0 0x7ff52819ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.257 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.250+0000 7ff5277fe700 1 -- 172.21.15.1:0/1047626766 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff51c0095f0 con 0x7ff528100cc0 2023-11-16T16:46:14.257 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.250+0000 7ff527fff700 1 --2- 172.21.15.1:0/1047626766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5281055c0 0x7ff52819ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:46:14.257 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.250+0000 7ff5277fe700 1 --2- 172.21.15.1:0/1047626766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff528100cc0 0x7ff528196370 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7ff51800e9a0 tx=0x7ff51800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:14.257 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.251+0000 7ff524ff9700 1 -- 172.21.15.1:0/1047626766 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff518009790 con 0x7ff528100cc0 2023-11-16T16:46:14.258 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.251+0000 7ff52dfa5700 1 -- 172.21.15.1:0/1047626766 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5281102f0 con 0x7ff528100cc0 2023-11-16T16:46:14.258 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.251+0000 7ff524ff9700 1 -- 172.21.15.1:0/1047626766 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff51800cbd0 con 0x7ff528100cc0 2023-11-16T16:46:14.258 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.251+0000 7ff524ff9700 1 -- 172.21.15.1:0/1047626766 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff518018680 con 0x7ff528100cc0 2023-11-16T16:46:14.258 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.251+0000 7ff52dfa5700 1 -- 172.21.15.1:0/1047626766 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff528110160 con 0x7ff528100cc0 2023-11-16T16:46:14.258 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.253+0000 7ff524ff9700 1 -- 172.21.15.1:0/1047626766 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7ff51800cd40 con 0x7ff528100cc0 2023-11-16T16:46:14.265 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.254+0000 7ff524ff9700 1 --2- 172.21.15.1:0/1047626766 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7ff5100a12c0 0x7ff5100a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.265 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.254+0000 7ff526ffd700 1 --2- 172.21.15.1:0/1047626766 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7ff5100a12c0 0x7ff5100a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.265 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.254+0000 7ff524ff9700 1 -- 172.21.15.1:0/1047626766 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8586+0+0 (secure 0 0 0) 0x7ff518014070 con 0x7ff528100cc0 2023-11-16T16:46:14.265 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.254+0000 7ff526ffd700 1 --2- 172.21.15.1:0/1047626766 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7ff5100a12c0 0x7ff5100a3770 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff51c00b580 tx=0x7ff51c01a040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:14.266 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.255+0000 7ff53a131b80 1 Processor -- start 2023-11-16T16:46:14.266 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.255+0000 7ff53a131b80 1 -- start start 2023-11-16T16:46:14.266 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.255+0000 7ff53a131b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fa4c5b60 0x5615fa4c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.266 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.255+0000 7ff53a131b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fa4c6470 0x5615fa4ca370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.266 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.255+0000 7ff53a131b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fa4ca8b0 0x5615fa4cad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.266 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.255+0000 7ff53a131b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615fa3cb060 con 0x5615fa4c5b60 2023-11-16T16:46:14.266 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.255+0000 7ff53a131b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615fa41d8a0 con 0x5615fa4c6470 2023-11-16T16:46:14.266 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.255+0000 7ff53a131b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615fa416d30 con 0x5615fa4ca8b0 2023-11-16T16:46:14.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.255+0000 7ff5277fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fa4c5b60 0x5615fa4c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.255+0000 7ff5277fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fa4c5b60 0x5615fa4c5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45618/0 (socket says 172.21.15.1:45618) 2023-11-16T16:46:14.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.255+0000 7ff5277fe700 1 -- 172.21.15.1:0/3308039274 learned_addr learned my addr 172.21.15.1:0/3308039274 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:14.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.255+0000 7ff526ffd700 1 --2- 172.21.15.1:0/3308039274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fa4c6470 0x5615fa4ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.255+0000 7ff527fff700 1 --2- 172.21.15.1:0/3308039274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fa4ca8b0 0x5615fa4cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.255+0000 7ff5277fe700 1 -- 172.21.15.1:0/3308039274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fa4ca8b0 msgr2=0x5615fa4cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:14.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.255+0000 7ff5277fe700 1 --2- 172.21.15.1:0/3308039274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fa4ca8b0 0x5615fa4cad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.255+0000 7ff5277fe700 1 -- 172.21.15.1:0/3308039274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fa4c6470 msgr2=0x5615fa4ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:14.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.255+0000 7ff5277fe700 1 --2- 172.21.15.1:0/3308039274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fa4c6470 0x5615fa4ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.255+0000 7ff5277fe700 1 -- 172.21.15.1:0/3308039274 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615fa298630 con 0x5615fa4c5b60 2023-11-16T16:46:14.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.255+0000 7ff527fff700 1 --2- 172.21.15.1:0/3308039274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fa4ca8b0 0x5615fa4cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:46:14.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.256+0000 7ff5277fe700 1 --2- 172.21.15.1:0/3308039274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fa4c5b60 0x5615fa4c5f30 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff5180a2090 tx=0x7ff51809a240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbfcb55b67ddb117 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:14.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.256+0000 7ff50dffb700 1 -- 172.21.15.1:0/3308039274 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51809a3d0 con 0x5615fa4c5b60 2023-11-16T16:46:14.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.256+0000 7ff50dffb700 1 -- 172.21.15.1:0/3308039274 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff51809a530 con 0x5615fa4c5b60 2023-11-16T16:46:14.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.256+0000 7ff50dffb700 1 -- 172.21.15.1:0/3308039274 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5180a2390 con 0x5615fa4c5b60 2023-11-16T16:46:14.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.256+0000 7ff53a131b80 1 -- 172.21.15.1:0/3308039274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fa4c5b60 msgr2=0x5615fa4c5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:14.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.256+0000 7ff53a131b80 1 --2- 172.21.15.1:0/3308039274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fa4c5b60 0x5615fa4c5f30 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff5180a2090 tx=0x7ff51809a240 comp rx=0 tx=0).stop 2023-11-16T16:46:14.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.257+0000 7ff53a131b80 1 -- 172.21.15.1:0/3308039274 shutdown_connections 2023-11-16T16:46:14.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.257+0000 7ff53a131b80 1 --2- 172.21.15.1:0/3308039274 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fa4ca8b0 0x5615fa4cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.257+0000 7ff53a131b80 1 --2- 172.21.15.1:0/3308039274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fa4c5b60 0x5615fa4c5f30 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.257+0000 7ff53a131b80 1 --2- 172.21.15.1:0/3308039274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fa4c6470 0x5615fa4ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.257+0000 7ff53a131b80 1 -- 172.21.15.1:0/3308039274 >> 172.21.15.1:0/3308039274 conn(0x5615fa4bb360 msgr2=0x5615fa4c0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:14.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.257+0000 7ff53a131b80 1 -- 172.21.15.1:0/3308039274 shutdown_connections 2023-11-16T16:46:14.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.257+0000 7ff53a131b80 1 -- 172.21.15.1:0/3308039274 wait complete. 2023-11-16T16:46:14.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.257+0000 7ff53a131b80 1 -- start start 2023-11-16T16:46:14.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.257+0000 7ff53a131b80 1 Processor -- start 2023-11-16T16:46:14.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.257+0000 7ff53a131b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fa4c5b60 0x5615fa4cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.257+0000 7ff53a131b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fa4c6470 0x5615fa4cf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.257+0000 7ff53a131b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fa4ca8b0 0x5615fa4cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.257+0000 7ff53a131b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615fa3b88e0 con 0x5615fa4c5b60 2023-11-16T16:46:14.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.257+0000 7ff53a131b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615fa3b8580 con 0x5615fa4ca8b0 2023-11-16T16:46:14.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.258+0000 7ff53a131b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615fa29a920 con 0x5615fa4c6470 2023-11-16T16:46:14.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.258+0000 7ff53a131b80 10 client.0 _ll_register_callbacks cb 0x5615fa3d6780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:46:14.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.258+0000 7ff5277fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fa4c5b60 0x5615fa4cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.258+0000 7ff5277fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fa4c5b60 0x5615fa4cf420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45624/0 (socket says 172.21.15.1:45624) 2023-11-16T16:46:14.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.258+0000 7ff5277fe700 1 -- 172.21.15.1:0/376575333 learned_addr learned my addr 172.21.15.1:0/376575333 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:14.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.258+0000 7ff526ffd700 1 --2- 172.21.15.1:0/376575333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fa4c6470 0x5615fa4cf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.258+0000 7ff527fff700 1 --2- 172.21.15.1:0/376575333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fa4ca8b0 0x5615fa4cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.258+0000 7ff5277fe700 1 -- 172.21.15.1:0/376575333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fa4c6470 msgr2=0x5615fa4cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:14.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.258+0000 7ff5277fe700 1 --2- 172.21.15.1:0/376575333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fa4c6470 0x5615fa4cf960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.258+0000 7ff5277fe700 1 -- 172.21.15.1:0/376575333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fa4ca8b0 msgr2=0x5615fa4cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:14.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.258+0000 7ff5277fe700 1 --2- 172.21.15.1:0/376575333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fa4ca8b0 0x5615fa4cfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.258+0000 7ff5277fe700 1 -- 172.21.15.1:0/376575333 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff528110160 con 0x5615fa4c5b60 2023-11-16T16:46:14.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.258+0000 7ff527fff700 1 --2- 172.21.15.1:0/376575333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fa4ca8b0 0x5615fa4cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:46:14.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.258+0000 7ff5277fe700 1 --2- 172.21.15.1:0/376575333 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fa4c5b60 0x5615fa4cf420 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff5180ca8a0 tx=0x7ff51809e320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce4fefe58ae12cf4 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:14.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.258+0000 7ff506ffd700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:46:14.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.258+0000 7ff506ffd700 1 -- 172.21.15.1:0/376575333 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51809a3d0 con 0x5615fa4c5b60 2023-11-16T16:46:14.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.258+0000 7ff506ffd700 1 -- 172.21.15.1:0/376575333 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8586+0+0 (secure 0 0 0) 0x7ff518096170 con 0x5615fa4c5b60 2023-11-16T16:46:14.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.258+0000 7ff53a131b80 1 -- 172.21.15.1:0/376575333 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5615fa4b8b20 con 0x5615fa4c5b60 2023-11-16T16:46:14.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.259+0000 7ff506ffd700 1 -- 172.21.15.1:0/376575333 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5180b2b50 con 0x5615fa4c5b60 2023-11-16T16:46:14.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.259+0000 7ff53a131b80 10 client.16320 fetch_fsmap learned FSMap version 28 2023-11-16T16:46:14.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.259+0000 7ff53a131b80 1 -- 172.21.15.1:0/376575333 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5615fa3dd190 con 0x5615fa4c5b60 2023-11-16T16:46:14.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.259+0000 7ff506ffd700 1 -- 172.21.15.1:0/376575333 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5615fa4b8b20 con 0x5615fa4c5b60 2023-11-16T16:46:14.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.259+0000 7ff506ffd700 1 -- 172.21.15.1:0/376575333 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff5180b83d0 con 0x5615fa4c5b60 2023-11-16T16:46:14.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.259+0000 7ff53a131b80 10 client.16320 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:46:14.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.259+0000 7ff53a131b80 10 client.16320 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:46:14.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.259+0000 7ff53a131b80 20 client.16320 populate_metadata read hostname 'smithi001' 2023-11-16T16:46:14.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.259+0000 7ff53a131b80 1 --2- 172.21.15.1:0/376575333 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5615fa4df4f0 0x5615fa4e1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.259+0000 7ff53a131b80 4 client.16320 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:46:14.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.259+0000 7ff53a131b80 1 -- 172.21.15.1:0/376575333 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5615fa3d90a0 con 0x5615fa4df4f0 2023-11-16T16:46:14.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.259+0000 7ff526ffd700 1 --2- 172.21.15.1:0/376575333 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5615fa4df4f0 0x5615fa4e1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.260+0000 7ff526ffd700 1 --2- 172.21.15.1:0/376575333 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5615fa4df4f0 0x5615fa4e1900 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c4b834852047852 server_cookie=71244bcac74885fb in_seq=0 out_seq=0 2023-11-16T16:46:14.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.260+0000 7ff506ffd700 10 client.16320 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:46:14.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.260+0000 7ff506ffd700 1 -- 172.21.15.1:0/376575333 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5615fa3d90a0 con 0x5615fa4df4f0 2023-11-16T16:46:14.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.260+0000 7ff506ffd700 10 client.16320 handle_command_reply: tid=0 2023-11-16T16:46:14.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.260+0000 7ff53a131b80 1 client.16320 shutdown 2023-11-16T16:46:14.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.261+0000 7ff53a131b80 1 -- 172.21.15.1:0/376575333 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fa4c5b60 msgr2=0x5615fa4cf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:14.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.261+0000 7ff53a131b80 1 --2- 172.21.15.1:0/376575333 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fa4c5b60 0x5615fa4cf420 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff5180ca8a0 tx=0x7ff51809e320 comp rx=0 tx=0).stop 2023-11-16T16:46:14.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.261+0000 7ff53a131b80 1 -- 172.21.15.1:0/376575333 shutdown_connections 2023-11-16T16:46:14.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.261+0000 7ff53a131b80 1 --2- 172.21.15.1:0/376575333 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5615fa4df4f0 0x5615fa4e1900 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.261+0000 7ff53a131b80 1 --2- 172.21.15.1:0/376575333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fa4ca8b0 0x5615fa4cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.261+0000 7ff53a131b80 1 --2- 172.21.15.1:0/376575333 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fa4c5b60 0x5615fa4cf420 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.261+0000 7ff506ffd700 0 client.16320 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:46:14.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.261+0000 7ff53a131b80 1 --2- 172.21.15.1:0/376575333 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fa4c6470 0x5615fa4cf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.261+0000 7ff53a131b80 1 -- 172.21.15.1:0/376575333 >> 172.21.15.1:0/376575333 conn(0x5615fa4bb360 msgr2=0x5615fa4cbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:14.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.261+0000 7ff53a131b80 1 -- 172.21.15.1:0/376575333 shutdown_connections 2023-11-16T16:46:14.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.261+0000 7ff53a131b80 1 -- 172.21.15.1:0/376575333 wait complete. 2023-11-16T16:46:14.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.261+0000 7ff53a131b80 20 client.16320 trim_cache size 0 max 16384 2023-11-16T16:46:14.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.278+0000 7ff53a131b80 1 Processor -- start 2023-11-16T16:46:14.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.278+0000 7ff53a131b80 1 -- start start 2023-11-16T16:46:14.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.278+0000 7ff53a131b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fa4c5b60 0x5615fa4c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.278+0000 7ff53a131b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fa4c6470 0x5615fa4d69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.278+0000 7ff53a131b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fa4ca8b0 0x5615fa4c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.278+0000 7ff53a131b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615fa4cd430 con 0x5615fa4ca8b0 2023-11-16T16:46:14.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.278+0000 7ff53a131b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615fa4baee0 con 0x5615fa4c5b60 2023-11-16T16:46:14.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.278+0000 7ff53a131b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615fa4dc330 con 0x5615fa4c6470 2023-11-16T16:46:14.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.278+0000 7ff527fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fa4ca8b0 0x5615fa4c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.278+0000 7ff527fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fa4ca8b0 0x5615fa4c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45636/0 (socket says 172.21.15.1:45636) 2023-11-16T16:46:14.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.278+0000 7ff527fff700 1 -- 172.21.15.1:0/523325495 learned_addr learned my addr 172.21.15.1:0/523325495 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:14.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.278+0000 7ff5277fe700 1 --2- 172.21.15.1:0/523325495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fa4c5b60 0x5615fa4c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.278+0000 7ff526ffd700 1 --2- 172.21.15.1:0/523325495 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fa4c6470 0x5615fa4d69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.279+0000 7ff527fff700 1 -- 172.21.15.1:0/523325495 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fa4c6470 msgr2=0x5615fa4d69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:14.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.279+0000 7ff527fff700 1 --2- 172.21.15.1:0/523325495 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fa4c6470 0x5615fa4d69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.279+0000 7ff527fff700 1 -- 172.21.15.1:0/523325495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fa4c5b60 msgr2=0x5615fa4c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:14.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.279+0000 7ff527fff700 1 --2- 172.21.15.1:0/523325495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fa4c5b60 0x5615fa4c81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.279+0000 7ff527fff700 1 -- 172.21.15.1:0/523325495 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615fa3dcee0 con 0x5615fa4ca8b0 2023-11-16T16:46:14.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.279+0000 7ff526ffd700 1 --2- 172.21.15.1:0/523325495 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fa4c6470 0x5615fa4d69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:46:14.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.279+0000 7ff527fff700 1 --2- 172.21.15.1:0/523325495 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fa4ca8b0 0x5615fa4c99b0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff5200094b0 tx=0x7ff52000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f5fdd249fe24ab8 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:14.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.279+0000 7ff50e7fc700 1 -- 172.21.15.1:0/523325495 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff520004db0 con 0x5615fa4ca8b0 2023-11-16T16:46:14.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.279+0000 7ff50e7fc700 1 -- 172.21.15.1:0/523325495 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5200168c0 con 0x5615fa4ca8b0 2023-11-16T16:46:14.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.279+0000 7ff50e7fc700 1 -- 172.21.15.1:0/523325495 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff520013970 con 0x5615fa4ca8b0 2023-11-16T16:46:14.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.279+0000 7ff53a131b80 1 -- 172.21.15.1:0/523325495 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fa4ca8b0 msgr2=0x5615fa4c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:14.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.279+0000 7ff53a131b80 1 --2- 172.21.15.1:0/523325495 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fa4ca8b0 0x5615fa4c99b0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff5200094b0 tx=0x7ff52000b040 comp rx=0 tx=0).stop 2023-11-16T16:46:14.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.280+0000 7ff53a131b80 1 -- 172.21.15.1:0/523325495 shutdown_connections 2023-11-16T16:46:14.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.280+0000 7ff53a131b80 1 --2- 172.21.15.1:0/523325495 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fa4ca8b0 0x5615fa4c99b0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.280+0000 7ff53a131b80 1 --2- 172.21.15.1:0/523325495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fa4c5b60 0x5615fa4c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.280+0000 7ff53a131b80 1 --2- 172.21.15.1:0/523325495 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fa4c6470 0x5615fa4d69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.280+0000 7ff53a131b80 1 -- 172.21.15.1:0/523325495 >> 172.21.15.1:0/523325495 conn(0x5615fa4bb360 msgr2=0x5615fa4c8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:14.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.280+0000 7ff53a131b80 1 -- 172.21.15.1:0/523325495 shutdown_connections 2023-11-16T16:46:14.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.280+0000 7ff53a131b80 1 -- 172.21.15.1:0/523325495 wait complete. 2023-11-16T16:46:14.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.280+0000 7ff53a131b80 1 -- start start 2023-11-16T16:46:14.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.280+0000 7ff53a131b80 1 Processor -- start 2023-11-16T16:46:14.290 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.280+0000 7ff53a131b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fa4c5b60 0x5615fa5274f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.290 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.280+0000 7ff53a131b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fa4c6470 0x5615fa527a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.290 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.280+0000 7ff53a131b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fa4ca8b0 0x5615fa52c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.290 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.280+0000 7ff53a131b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615fa52c9e0 con 0x5615fa4ca8b0 2023-11-16T16:46:14.290 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.280+0000 7ff53a131b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615fa52cb50 con 0x5615fa4c5b60 2023-11-16T16:46:14.290 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.280+0000 7ff53a131b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615fa52cd10 con 0x5615fa4c6470 2023-11-16T16:46:14.290 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.280+0000 7ff53a131b80 10 client.0 _ll_register_callbacks cb 0x5615fa3d6780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:46:14.291 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.281+0000 7ff527fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fa4ca8b0 0x5615fa52c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.291 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.281+0000 7ff527fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fa4ca8b0 0x5615fa52c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45642/0 (socket says 172.21.15.1:45642) 2023-11-16T16:46:14.291 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.281+0000 7ff527fff700 1 -- 172.21.15.1:0/3091262051 learned_addr learned my addr 172.21.15.1:0/3091262051 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:14.291 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.281+0000 7ff5277fe700 1 --2- 172.21.15.1:0/3091262051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fa4c5b60 0x5615fa5274f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.291 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.281+0000 7ff526ffd700 1 --2- 172.21.15.1:0/3091262051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fa4c6470 0x5615fa527a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.291 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.281+0000 7ff527fff700 1 -- 172.21.15.1:0/3091262051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fa4c6470 msgr2=0x5615fa527a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:14.291 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.281+0000 7ff527fff700 1 --2- 172.21.15.1:0/3091262051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fa4c6470 0x5615fa527a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.291 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.281+0000 7ff527fff700 1 -- 172.21.15.1:0/3091262051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fa4c5b60 msgr2=0x5615fa5274f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:14.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.281+0000 7ff527fff700 1 --2- 172.21.15.1:0/3091262051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fa4c5b60 0x5615fa5274f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.281+0000 7ff527fff700 1 -- 172.21.15.1:0/3091262051 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff520009190 con 0x5615fa4ca8b0 2023-11-16T16:46:14.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.281+0000 7ff527fff700 1 --2- 172.21.15.1:0/3091262051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fa4ca8b0 0x5615fa52c4a0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ff520009c10 tx=0x7ff5200166b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b37ef8ffdc667a server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:14.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.281+0000 7ff506ffd700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:46:14.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.281+0000 7ff506ffd700 1 -- 172.21.15.1:0/3091262051 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5200135b0 con 0x5615fa4ca8b0 2023-11-16T16:46:14.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.281+0000 7ff506ffd700 1 -- 172.21.15.1:0/3091262051 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8586+0+0 (secure 0 0 0) 0x7ff520013710 con 0x5615fa4ca8b0 2023-11-16T16:46:14.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.281+0000 7ff53a131b80 1 -- 172.21.15.1:0/3091262051 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff51809a530 con 0x5615fa4ca8b0 2023-11-16T16:46:14.293 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.281+0000 7ff506ffd700 1 -- 172.21.15.1:0/3091262051 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5200138f0 con 0x5615fa4ca8b0 2023-11-16T16:46:14.293 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.281+0000 7ff53a131b80 10 client.16326 fetch_fsmap learned FSMap version 28 2023-11-16T16:46:14.293 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.281+0000 7ff53a131b80 1 -- 172.21.15.1:0/3091262051 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5615fa22dc60 con 0x5615fa4ca8b0 2023-11-16T16:46:14.293 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.282+0000 7ff506ffd700 1 -- 172.21.15.1:0/3091262051 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff51809a530 con 0x5615fa4ca8b0 2023-11-16T16:46:14.293 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.282+0000 7ff506ffd700 1 -- 172.21.15.1:0/3091262051 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff52001e040 con 0x5615fa4ca8b0 2023-11-16T16:46:14.293 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.282+0000 7ff53a131b80 10 client.16326 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:46:14.293 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.282+0000 7ff53a131b80 10 client.16326 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:46:14.293 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.282+0000 7ff53a131b80 20 client.16326 populate_metadata read hostname 'smithi001' 2023-11-16T16:46:14.294 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.282+0000 7ff53a131b80 1 --2- 172.21.15.1:0/3091262051 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5615fa5343b0 0x5615fa5367c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.294 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.282+0000 7ff53a131b80 4 client.16326 mds_command: new command op to 24521 tid=0[{"prefix": "scrub status"}] 2023-11-16T16:46:14.294 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.282+0000 7ff53a131b80 1 -- 172.21.15.1:0/3091262051 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5615fa3e5660 con 0x5615fa5343b0 2023-11-16T16:46:14.294 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.282+0000 7ff5277fe700 1 --2- 172.21.15.1:0/3091262051 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5615fa5343b0 0x5615fa5367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.294 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.282+0000 7ff5277fe700 1 --2- 172.21.15.1:0/3091262051 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5615fa5343b0 0x5615fa5367c0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=736ca90df7ecc6b0 server_cookie=201666b83d0a7d66 in_seq=0 out_seq=0 2023-11-16T16:46:14.294 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.282+0000 7ff506ffd700 10 client.16326 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:46:14.294 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.283+0000 7ff506ffd700 1 -- 172.21.15.1:0/3091262051 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5615fa3e5660 con 0x5615fa5343b0 2023-11-16T16:46:14.295 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.283+0000 7ff506ffd700 10 client.16326 handle_command_reply: tid=0 2023-11-16T16:46:14.295 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.283+0000 7ff53a131b80 1 client.16326 shutdown 2023-11-16T16:46:14.295 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.283+0000 7ff53a131b80 1 -- 172.21.15.1:0/3091262051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fa4ca8b0 msgr2=0x5615fa52c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:14.295 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.283+0000 7ff53a131b80 1 --2- 172.21.15.1:0/3091262051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fa4ca8b0 0x5615fa52c4a0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ff520009c10 tx=0x7ff5200166b0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.295 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.283+0000 7ff53a131b80 1 -- 172.21.15.1:0/3091262051 shutdown_connections 2023-11-16T16:46:14.295 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.283+0000 7ff53a131b80 1 --2- 172.21.15.1:0/3091262051 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5615fa5343b0 0x5615fa5367c0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.295 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.283+0000 7ff53a131b80 1 --2- 172.21.15.1:0/3091262051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5615fa4ca8b0 0x5615fa52c4a0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.295 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.283+0000 7ff506ffd700 0 client.16326 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:46:14.296 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.283+0000 7ff53a131b80 1 --2- 172.21.15.1:0/3091262051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5615fa4c5b60 0x5615fa5274f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.296 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.283+0000 7ff53a131b80 1 --2- 172.21.15.1:0/3091262051 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5615fa4c6470 0x5615fa527a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.296 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.283+0000 7ff53a131b80 1 -- 172.21.15.1:0/3091262051 >> 172.21.15.1:0/3091262051 conn(0x5615fa4bb360 msgr2=0x5615fa4cf6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:14.296 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.283+0000 7ff53a131b80 1 -- 172.21.15.1:0/3091262051 shutdown_connections 2023-11-16T16:46:14.296 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.283+0000 7ff53a131b80 1 -- 172.21.15.1:0/3091262051 wait complete. 2023-11-16T16:46:14.296 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.283+0000 7ff53a131b80 20 client.16326 trim_cache size 0 max 16384 2023-11-16T16:46:14.296 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:46:14.297 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2023-11-16T16:46:14.297 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2023-11-16T16:46:14.297 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:46:14.298 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.284+0000 7ff52dfa5700 1 -- 172.21.15.1:0/1047626766 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7ff5100a12c0 msgr2=0x7ff5100a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:14.298 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.284+0000 7ff52dfa5700 1 --2- 172.21.15.1:0/1047626766 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7ff5100a12c0 0x7ff5100a3770 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff51c00b580 tx=0x7ff51c01a040 comp rx=0 tx=0).stop 2023-11-16T16:46:14.298 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.284+0000 7ff52dfa5700 1 -- 172.21.15.1:0/1047626766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff528100cc0 msgr2=0x7ff528196370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:14.299 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.284+0000 7ff52dfa5700 1 --2- 172.21.15.1:0/1047626766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff528100cc0 0x7ff528196370 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7ff51800e9a0 tx=0x7ff51800ed60 comp rx=0 tx=0).stop 2023-11-16T16:46:14.299 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.284+0000 7ff52dfa5700 1 -- 172.21.15.1:0/1047626766 shutdown_connections 2023-11-16T16:46:14.299 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.284+0000 7ff52dfa5700 1 --2- 172.21.15.1:0/1047626766 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7ff5100a12c0 0x7ff5100a3770 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.299 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.284+0000 7ff52dfa5700 1 --2- 172.21.15.1:0/1047626766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5281055c0 0x7ff52819ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.299 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.284+0000 7ff52dfa5700 1 --2- 172.21.15.1:0/1047626766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff528100cc0 0x7ff528196370 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.299 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.284+0000 7ff52dfa5700 1 --2- 172.21.15.1:0/1047626766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff528101660 0x7ff5281968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.299 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.284+0000 7ff52dfa5700 1 -- 172.21.15.1:0/1047626766 >> 172.21.15.1:0/1047626766 conn(0x7ff528077560 msgr2=0x7ff5280ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:14.300 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.284+0000 7ff52dfa5700 1 -- 172.21.15.1:0/1047626766 shutdown_connections 2023-11-16T16:46:14.300 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.284+0000 7ff52dfa5700 1 -- 172.21.15.1:0/1047626766 wait complete. 2023-11-16T16:46:14.300 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-16T16:46:14.300 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:46:14.426 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.424+0000 7f168a213700 1 -- 172.21.15.1:0/2967652240 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16840fe850 msgr2=0x7f16840fecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:14.426 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.424+0000 7f168a213700 1 --2- 172.21.15.1:0/2967652240 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16840fe850 0x7f16840fecc0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f1674009990 tx=0x7f1674009ca0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.426 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.424+0000 7f168a213700 1 -- 172.21.15.1:0/2967652240 shutdown_connections 2023-11-16T16:46:14.427 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.424+0000 7f168a213700 1 --2- 172.21.15.1:0/2967652240 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16840ff200 0x7f168410d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.427 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.424+0000 7f168a213700 1 --2- 172.21.15.1:0/2967652240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16841063f0 0x7f16841067c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.427 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.424+0000 7f168a213700 1 --2- 172.21.15.1:0/2967652240 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16840fe850 0x7f16840fecc0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.427 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.424+0000 7f168a213700 1 -- 172.21.15.1:0/2967652240 >> 172.21.15.1:0/2967652240 conn(0x7f16840758c0 msgr2=0x7f1684075cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:14.427 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.424+0000 7f168a213700 1 -- 172.21.15.1:0/2967652240 shutdown_connections 2023-11-16T16:46:14.427 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.424+0000 7f168a213700 1 -- 172.21.15.1:0/2967652240 wait complete. 2023-11-16T16:46:14.427 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.425+0000 7f168a213700 1 Processor -- start 2023-11-16T16:46:14.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.425+0000 7f168a213700 1 -- start start 2023-11-16T16:46:14.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.425+0000 7f168a213700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16840fe850 0x7f16841964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.425+0000 7f168a213700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16840ff200 0x7f1684196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.425+0000 7f168a213700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16841063f0 0x7f168419ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.430 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.425+0000 7f168a213700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168419b4a0 con 0x7f16840ff200 2023-11-16T16:46:14.431 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.425+0000 7f168a213700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168419b610 con 0x7f16840fe850 2023-11-16T16:46:14.431 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.425+0000 7f168a213700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168419b7d0 con 0x7f16841063f0 2023-11-16T16:46:14.431 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.425+0000 7f1682ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16840ff200 0x7f1684196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.431 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.425+0000 7f1682ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16840ff200 0x7f1684196a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45654/0 (socket says 172.21.15.1:45654) 2023-11-16T16:46:14.431 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.425+0000 7f1682ffd700 1 -- 172.21.15.1:0/4081825700 learned_addr learned my addr 172.21.15.1:0/4081825700 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:14.431 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.425+0000 7f16837fe700 1 --2- 172.21.15.1:0/4081825700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16840fe850 0x7f16841964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.432 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.425+0000 7f1683fff700 1 --2- 172.21.15.1:0/4081825700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16841063f0 0x7f168419ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.432 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.425+0000 7f1682ffd700 1 -- 172.21.15.1:0/4081825700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16841063f0 msgr2=0x7f168419ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:14.432 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.425+0000 7f1682ffd700 1 --2- 172.21.15.1:0/4081825700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16841063f0 0x7f168419ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.432 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.425+0000 7f1682ffd700 1 -- 172.21.15.1:0/4081825700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16840fe850 msgr2=0x7f16841964c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:14.432 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.425+0000 7f1682ffd700 1 --2- 172.21.15.1:0/4081825700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16840fe850 0x7f16841964c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.432 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.425+0000 7f1682ffd700 1 -- 172.21.15.1:0/4081825700 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16740095f0 con 0x7f16840ff200 2023-11-16T16:46:14.432 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.426+0000 7f1682ffd700 1 --2- 172.21.15.1:0/4081825700 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16840ff200 0x7f1684196a00 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f1674000c00 tx=0x7f1674004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:14.432 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.426+0000 7f1680ff9700 1 -- 172.21.15.1:0/4081825700 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167401d070 con 0x7f16840ff200 2023-11-16T16:46:14.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.426+0000 7f168a213700 1 -- 172.21.15.1:0/4081825700 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16841103f0 con 0x7f16840ff200 2023-11-16T16:46:14.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.426+0000 7f1680ff9700 1 -- 172.21.15.1:0/4081825700 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1674021410 con 0x7f16840ff200 2023-11-16T16:46:14.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.426+0000 7f1680ff9700 1 -- 172.21.15.1:0/4081825700 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1674017650 con 0x7f16840ff200 2023-11-16T16:46:14.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.426+0000 7f168a213700 1 -- 172.21.15.1:0/4081825700 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1684110260 con 0x7f16840ff200 2023-11-16T16:46:14.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.427+0000 7f168a213700 1 -- 172.21.15.1:0/4081825700 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1684110580 con 0x7f16840ff200 2023-11-16T16:46:14.433 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.428+0000 7f1680ff9700 1 -- 172.21.15.1:0/4081825700 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f1674021580 con 0x7f16840ff200 2023-11-16T16:46:14.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.428+0000 7f1680ff9700 1 --2- 172.21.15.1:0/4081825700 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f16700a12c0 0x7f16700a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.429+0000 7f1680ff9700 1 -- 172.21.15.1:0/4081825700 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8586+0+0 (secure 0 0 0) 0x7f16740ce0e0 con 0x7f16840ff200 2023-11-16T16:46:14.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.429+0000 7f16837fe700 1 --2- 172.21.15.1:0/4081825700 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f16700a12c0 0x7f16700a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.429+0000 7f16837fe700 1 --2- 172.21.15.1:0/4081825700 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f16700a12c0 0x7f16700a3770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f166c009be0 tx=0x7f166c009260 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:14.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.433+0000 7f1680ff9700 1 -- 172.21.15.1:0/4081825700 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f16740974e0 con 0x7f16840ff200 2023-11-16T16:46:14.678 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.676+0000 7f168a213700 1 -- 172.21.15.1:0/4081825700 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1684067260 con 0x7f16840ff200 2023-11-16T16:46:14.678 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.677+0000 7f1680ff9700 1 -- 172.21.15.1:0/4081825700 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10783 (secure 0 0 0) 0x7f167409ab00 con 0x7f16840ff200 2023-11-16T16:46:14.679 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:46:14.679 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2587563684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2587563684},{"type":"v1","addr":"172.21.15.1:6835","nonce":2587563684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/277936168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":277936168},{"type":"v1","addr":"172.21.15.1:6837","nonce":277936168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/1172575045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1172575045},{"type":"v1","addr":"172.21.15.1:6839","nonce":1172575045}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/2631175899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2631175899},{"type":"v1","addr":"172.21.15.1:6841","nonce":2631175899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3693905267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3693905267},{"type":"v1","addr":"172.21.15.53:6833","nonce":3693905267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/164668033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":164668033},{"type":"v1","addr":"172.21.15.26:6833","nonce":164668033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-16T16:37:13.918943+0000","modified":"2023-11-16T16:43:40.628525+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2140542192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2140542192},{"type":"v1","addr":"172.21.15.53:6835","nonce":2140542192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1010760275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1010760275},{"type":"v1","addr":"172.21.15.26:6835","nonce":1010760275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3197060302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3197060302},{"type":"v1","addr":"172.21.15.53:6837","nonce":3197060302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6837/1453504863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1453504863},{"type":"v1","addr":"172.21.15.26:6837","nonce":1453504863}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.53:6839/1044753024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1044753024},{"type":"v1","addr":"172.21.15.53:6839","nonce":1044753024}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6839/2192205949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2192205949},{"type":"v1","addr":"172.21.15.26:6839","nonce":2192205949}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-16T16:46:14.681 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.679+0000 7f168a213700 1 -- 172.21.15.1:0/4081825700 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f16700a12c0 msgr2=0x7f16700a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:14.681 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.679+0000 7f168a213700 1 --2- 172.21.15.1:0/4081825700 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f16700a12c0 0x7f16700a3770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f166c009be0 tx=0x7f166c009260 comp rx=0 tx=0).stop 2023-11-16T16:46:14.681 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.679+0000 7f168a213700 1 -- 172.21.15.1:0/4081825700 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16840ff200 msgr2=0x7f1684196a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:14.681 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.679+0000 7f168a213700 1 --2- 172.21.15.1:0/4081825700 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16840ff200 0x7f1684196a00 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f1674000c00 tx=0x7f1674004790 comp rx=0 tx=0).stop 2023-11-16T16:46:14.681 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.679+0000 7f168a213700 1 -- 172.21.15.1:0/4081825700 shutdown_connections 2023-11-16T16:46:14.681 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.679+0000 7f168a213700 1 --2- 172.21.15.1:0/4081825700 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f16700a12c0 0x7f16700a3770 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.679+0000 7f168a213700 1 --2- 172.21.15.1:0/4081825700 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f16841063f0 0x7f168419ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.679+0000 7f168a213700 1 --2- 172.21.15.1:0/4081825700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16840fe850 0x7f16841964c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.679+0000 7f168a213700 1 --2- 172.21.15.1:0/4081825700 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f16840ff200 0x7f1684196a00 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.679+0000 7f168a213700 1 -- 172.21.15.1:0/4081825700 >> 172.21.15.1:0/4081825700 conn(0x7f16840758c0 msgr2=0x7f16840fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:14.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.679+0000 7f168a213700 1 -- 172.21.15.1:0/4081825700 shutdown_connections 2023-11-16T16:46:14.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.679+0000 7f168a213700 1 -- 172.21.15.1:0/4081825700 wait complete. 2023-11-16T16:46:14.683 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 28 2023-11-16T16:46:14.693 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-11-16T16:46:14.823 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.819+0000 7ff6106b1700 1 -- 172.21.15.1:0/704288751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6080ff490 msgr2=0x7ff6080ff860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:14.823 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.819+0000 7ff6106b1700 1 --2- 172.21.15.1:0/704288751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6080ff490 0x7ff6080ff860 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff6040099c0 tx=0x7ff604009cd0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.823 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.819+0000 7ff6106b1700 1 -- 172.21.15.1:0/704288751 shutdown_connections 2023-11-16T16:46:14.823 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.819+0000 7ff6106b1700 1 --2- 172.21.15.1:0/704288751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff608104480 0x7ff60810c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.824 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.819+0000 7ff6106b1700 1 --2- 172.21.15.1:0/704288751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6080ff490 0x7ff6080ff860 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.824 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.819+0000 7ff6106b1700 1 --2- 172.21.15.1:0/704288751 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff6080ffe30 0x7ff608103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.824 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.819+0000 7ff6106b1700 1 -- 172.21.15.1:0/704288751 >> 172.21.15.1:0/704288751 conn(0x7ff6080763a0 msgr2=0x7ff6080767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:14.824 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.819+0000 7ff6106b1700 1 -- 172.21.15.1:0/704288751 shutdown_connections 2023-11-16T16:46:14.824 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.819+0000 7ff6106b1700 1 -- 172.21.15.1:0/704288751 wait complete. 2023-11-16T16:46:14.824 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.819+0000 7ff6106b1700 1 Processor -- start 2023-11-16T16:46:14.824 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.819+0000 7ff6106b1700 1 -- start start 2023-11-16T16:46:14.824 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.820+0000 7ff6106b1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6080ff490 0x7ff608100e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.820+0000 7ff6106b1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6080ffe30 0x7ff608101380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.820+0000 7ff6106b1700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff608104480 0x7ff6081a45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.820+0000 7ff6106b1700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff608101bb0 con 0x7ff6080ff490 2023-11-16T16:46:14.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.820+0000 7ff6106b1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6081a4b20 con 0x7ff6080ffe30 2023-11-16T16:46:14.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.820+0000 7ff6106b1700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6081a4c60 con 0x7ff608104480 2023-11-16T16:46:14.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.820+0000 7ff60e44d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6080ff490 0x7ff608100e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.820+0000 7ff60e44d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6080ff490 0x7ff608100e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45666/0 (socket says 172.21.15.1:45666) 2023-11-16T16:46:14.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.820+0000 7ff60e44d700 1 -- 172.21.15.1:0/2328002415 learned_addr learned my addr 172.21.15.1:0/2328002415 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:14.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.820+0000 7ff60ec4e700 1 --2- 172.21.15.1:0/2328002415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff608104480 0x7ff6081a45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.820+0000 7ff60dc4c700 1 --2- 172.21.15.1:0/2328002415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6080ffe30 0x7ff608101380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.820+0000 7ff60e44d700 1 -- 172.21.15.1:0/2328002415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff608104480 msgr2=0x7ff6081a45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:14.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.820+0000 7ff60e44d700 1 --2- 172.21.15.1:0/2328002415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff608104480 0x7ff6081a45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.820+0000 7ff60e44d700 1 -- 172.21.15.1:0/2328002415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6080ffe30 msgr2=0x7ff608101380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:14.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.820+0000 7ff60e44d700 1 --2- 172.21.15.1:0/2328002415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6080ffe30 0x7ff608101380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.820+0000 7ff60e44d700 1 -- 172.21.15.1:0/2328002415 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6040095f0 con 0x7ff6080ff490 2023-11-16T16:46:14.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.820+0000 7ff60e44d700 1 --2- 172.21.15.1:0/2328002415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6080ff490 0x7ff608100e40 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff60400b580 tx=0x7ff60400bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:14.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.820+0000 7ff5ff7fe700 1 -- 172.21.15.1:0/2328002415 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60401d070 con 0x7ff6080ff490 2023-11-16T16:46:14.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.820+0000 7ff5ff7fe700 1 -- 172.21.15.1:0/2328002415 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff604022470 con 0x7ff6080ff490 2023-11-16T16:46:14.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.820+0000 7ff5ff7fe700 1 -- 172.21.15.1:0/2328002415 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff604017650 con 0x7ff6080ff490 2023-11-16T16:46:14.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.820+0000 7ff6106b1700 1 -- 172.21.15.1:0/2328002415 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff60810f260 con 0x7ff6080ff490 2023-11-16T16:46:14.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.821+0000 7ff6106b1700 1 -- 172.21.15.1:0/2328002415 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff60810f0d0 con 0x7ff6080ff490 2023-11-16T16:46:14.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.823+0000 7ff5ff7fe700 1 -- 172.21.15.1:0/2328002415 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7ff6040225e0 con 0x7ff6080ff490 2023-11-16T16:46:14.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.823+0000 7ff5ff7fe700 1 --2- 172.21.15.1:0/2328002415 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7ff5f40a12c0 0x7ff5f40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.824+0000 7ff60dc4c700 1 --2- 172.21.15.1:0/2328002415 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7ff5f40a12c0 0x7ff5f40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.824+0000 7ff5ff7fe700 1 -- 172.21.15.1:0/2328002415 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8586+0+0 (secure 0 0 0) 0x7ff6040cdcb0 con 0x7ff6080ff490 2023-11-16T16:46:14.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.824+0000 7ff60dc4c700 1 --2- 172.21.15.1:0/2328002415 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7ff5f40a12c0 0x7ff5f40a3770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff608102630 tx=0x7ff5f8009260 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:14.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.824+0000 7ff61c83db80 1 Processor -- start 2023-11-16T16:46:14.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.825+0000 7ff61c83db80 1 -- start start 2023-11-16T16:46:14.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.825+0000 7ff61c83db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e258474b60 0x55e258474f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.825+0000 7ff61c83db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e258475470 0x55e258479370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.825+0000 7ff61c83db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2584798b0 0x55e258479d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.825+0000 7ff61c83db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e25837a060 con 0x55e258474b60 2023-11-16T16:46:14.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.825+0000 7ff61c83db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2583cc8a0 con 0x55e258475470 2023-11-16T16:46:14.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.825+0000 7ff61c83db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2583c5d30 con 0x55e2584798b0 2023-11-16T16:46:14.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.825+0000 7ff60e44d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e258474b60 0x55e258474f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.825+0000 7ff60e44d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e258474b60 0x55e258474f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45674/0 (socket says 172.21.15.1:45674) 2023-11-16T16:46:14.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.825+0000 7ff60e44d700 1 -- 172.21.15.1:0/2929600786 learned_addr learned my addr 172.21.15.1:0/2929600786 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:14.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.825+0000 7ff60ec4e700 1 --2- 172.21.15.1:0/2929600786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2584798b0 0x55e258479d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.825+0000 7ff60dc4c700 1 --2- 172.21.15.1:0/2929600786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e258475470 0x55e258479370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.825+0000 7ff60e44d700 1 -- 172.21.15.1:0/2929600786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2584798b0 msgr2=0x55e258479d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:14.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.825+0000 7ff60e44d700 1 --2- 172.21.15.1:0/2929600786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2584798b0 0x55e258479d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.825+0000 7ff60e44d700 1 -- 172.21.15.1:0/2929600786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e258475470 msgr2=0x55e258479370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:14.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.825+0000 7ff60e44d700 1 --2- 172.21.15.1:0/2929600786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e258475470 0x55e258479370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.825+0000 7ff60e44d700 1 -- 172.21.15.1:0/2929600786 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e258247630 con 0x55e258474b60 2023-11-16T16:46:14.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.826+0000 7ff60e44d700 1 --2- 172.21.15.1:0/2929600786 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e258474b60 0x55e258474f30 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff60409ad60 tx=0x7ff60409aa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23cc541a681a4588 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:14.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.826+0000 7ff5fcff9700 1 -- 172.21.15.1:0/2929600786 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60409ee00 con 0x55e258474b60 2023-11-16T16:46:14.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.826+0000 7ff5fcff9700 1 -- 172.21.15.1:0/2929600786 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6040a3190 con 0x55e258474b60 2023-11-16T16:46:14.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.826+0000 7ff5fcff9700 1 -- 172.21.15.1:0/2929600786 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6040a2d00 con 0x55e258474b60 2023-11-16T16:46:14.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.826+0000 7ff61c83db80 1 -- 172.21.15.1:0/2929600786 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e258474b60 msgr2=0x55e258474f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:14.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.826+0000 7ff61c83db80 1 --2- 172.21.15.1:0/2929600786 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e258474b60 0x55e258474f30 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff60409ad60 tx=0x7ff60409aa30 comp rx=0 tx=0).stop 2023-11-16T16:46:14.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.826+0000 7ff61c83db80 1 -- 172.21.15.1:0/2929600786 shutdown_connections 2023-11-16T16:46:14.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.827+0000 7ff61c83db80 1 --2- 172.21.15.1:0/2929600786 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2584798b0 0x55e258479d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.827+0000 7ff61c83db80 1 --2- 172.21.15.1:0/2929600786 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e258474b60 0x55e258474f30 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.827+0000 7ff61c83db80 1 --2- 172.21.15.1:0/2929600786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e258475470 0x55e258479370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.838 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.827+0000 7ff61c83db80 1 -- 172.21.15.1:0/2929600786 >> 172.21.15.1:0/2929600786 conn(0x55e25846a360 msgr2=0x55e25846fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:14.838 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.827+0000 7ff61c83db80 1 -- 172.21.15.1:0/2929600786 shutdown_connections 2023-11-16T16:46:14.838 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.827+0000 7ff61c83db80 1 -- 172.21.15.1:0/2929600786 wait complete. 2023-11-16T16:46:14.838 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.827+0000 7ff61c83db80 1 -- start start 2023-11-16T16:46:14.838 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.827+0000 7ff61c83db80 1 Processor -- start 2023-11-16T16:46:14.838 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.827+0000 7ff61c83db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e258474b60 0x55e258480fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.838 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.827+0000 7ff61c83db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e258475470 0x55e25847e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.827+0000 7ff61c83db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e2584798b0 0x55e25847e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.827+0000 7ff61c83db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2583678e0 con 0x55e2584798b0 2023-11-16T16:46:14.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.827+0000 7ff61c83db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e258367580 con 0x55e258475470 2023-11-16T16:46:14.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.827+0000 7ff61c83db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e258249920 con 0x55e258474b60 2023-11-16T16:46:14.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.827+0000 7ff61c83db80 10 client.0 _ll_register_callbacks cb 0x55e258385780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:46:14.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.828+0000 7ff60ec4e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e2584798b0 0x55e25847e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.828+0000 7ff60ec4e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e2584798b0 0x55e25847e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45684/0 (socket says 172.21.15.1:45684) 2023-11-16T16:46:14.840 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.828+0000 7ff60ec4e700 1 -- 172.21.15.1:0/2689292424 learned_addr learned my addr 172.21.15.1:0/2689292424 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:14.840 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.828+0000 7ff60e44d700 1 --2- 172.21.15.1:0/2689292424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e258474b60 0x55e258480fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.840 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.828+0000 7ff60dc4c700 1 --2- 172.21.15.1:0/2689292424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e258475470 0x55e25847e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.840 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.828+0000 7ff60ec4e700 1 -- 172.21.15.1:0/2689292424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e258474b60 msgr2=0x55e258480fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:14.840 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.828+0000 7ff60ec4e700 1 --2- 172.21.15.1:0/2689292424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e258474b60 0x55e258480fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.840 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.828+0000 7ff60ec4e700 1 -- 172.21.15.1:0/2689292424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e258475470 msgr2=0x55e25847e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:14.840 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.828+0000 7ff60ec4e700 1 --2- 172.21.15.1:0/2689292424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e258475470 0x55e25847e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.840 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.828+0000 7ff60ec4e700 1 -- 172.21.15.1:0/2689292424 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff60810f0d0 con 0x55e2584798b0 2023-11-16T16:46:14.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.828+0000 7ff60e44d700 1 --2- 172.21.15.1:0/2689292424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e258474b60 0x55e258480fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:46:14.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.828+0000 7ff60dc4c700 1 --2- 172.21.15.1:0/2689292424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e258475470 0x55e25847e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:46:14.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.828+0000 7ff60ec4e700 1 --2- 172.21.15.1:0/2689292424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e2584798b0 0x55e25847e870 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff600007ad0 tx=0x7ff600007e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa8c5b1d3f36422a server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:14.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.828+0000 7ff5f1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:46:14.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.828+0000 7ff5f1ffb700 1 -- 172.21.15.1:0/2689292424 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60000e040 con 0x55e2584798b0 2023-11-16T16:46:14.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.828+0000 7ff5f1ffb700 1 -- 172.21.15.1:0/2689292424 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8586+0+0 (secure 0 0 0) 0x7ff600012ca0 con 0x55e2584798b0 2023-11-16T16:46:14.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.828+0000 7ff61c83db80 1 -- 172.21.15.1:0/2689292424 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e258467b20 con 0x55e2584798b0 2023-11-16T16:46:14.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.828+0000 7ff5f1ffb700 1 -- 172.21.15.1:0/2689292424 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff600012e60 con 0x55e2584798b0 2023-11-16T16:46:14.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.828+0000 7ff61c83db80 10 client.16344 fetch_fsmap learned FSMap version 28 2023-11-16T16:46:14.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.828+0000 7ff61c83db80 1 -- 172.21.15.1:0/2689292424 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e25838c190 con 0x55e2584798b0 2023-11-16T16:46:14.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.829+0000 7ff5f1ffb700 1 -- 172.21.15.1:0/2689292424 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e258467b20 con 0x55e2584798b0 2023-11-16T16:46:14.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.829+0000 7ff5f1ffb700 1 -- 172.21.15.1:0/2689292424 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff600014040 con 0x55e2584798b0 2023-11-16T16:46:14.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.829+0000 7ff61c83db80 10 client.16344 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:46:14.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.829+0000 7ff61c83db80 10 client.16344 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-11-16T16:46:14.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.829+0000 7ff61c83db80 20 client.16344 populate_metadata read hostname 'smithi001' 2023-11-16T16:46:14.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.829+0000 7ff61c83db80 1 --2- 172.21.15.1:0/2689292424 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55e25848e620 0x55e258490a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.829+0000 7ff61c83db80 4 client.16344 mds_command: new command op to 24482 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:46:14.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.829+0000 7ff61c83db80 1 -- 172.21.15.1:0/2689292424 --> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e2583880a0 con 0x55e25848e620 2023-11-16T16:46:14.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.829+0000 7ff60e44d700 1 --2- 172.21.15.1:0/2689292424 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55e25848e620 0x55e258490a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.830+0000 7ff60e44d700 1 --2- 172.21.15.1:0/2689292424 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55e25848e620 0x55e258490a30 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3b0f0c6d45b66008 server_cookie=9c17ec9add002176 in_seq=0 out_seq=0 2023-11-16T16:46:14.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.830+0000 7ff5f1ffb700 10 client.16344 ms_handle_connect on v2:172.21.15.26:6836/1453504863 2023-11-16T16:46:14.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.830+0000 7ff5f1ffb700 1 -- 172.21.15.1:0/2689292424 <== mds.2 v2:172.21.15.26:6836/1453504863 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e2583880a0 con 0x55e25848e620 2023-11-16T16:46:14.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.830+0000 7ff5f1ffb700 10 client.16344 handle_command_reply: tid=0 2023-11-16T16:46:14.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.830+0000 7ff61c83db80 1 client.16344 shutdown 2023-11-16T16:46:14.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.830+0000 7ff61c83db80 1 -- 172.21.15.1:0/2689292424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e2584798b0 msgr2=0x55e25847e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:14.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.830+0000 7ff61c83db80 1 --2- 172.21.15.1:0/2689292424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e2584798b0 0x55e25847e870 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff600007ad0 tx=0x7ff600007e90 comp rx=0 tx=0).stop 2023-11-16T16:46:14.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.830+0000 7ff61c83db80 1 -- 172.21.15.1:0/2689292424 shutdown_connections 2023-11-16T16:46:14.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.830+0000 7ff61c83db80 1 --2- 172.21.15.1:0/2689292424 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55e25848e620 0x55e258490a30 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.830+0000 7ff61c83db80 1 --2- 172.21.15.1:0/2689292424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e2584798b0 0x55e25847e870 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.830+0000 7ff5f1ffb700 0 client.16344 ms_handle_reset on v2:172.21.15.26:6836/1453504863 2023-11-16T16:46:14.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.830+0000 7ff61c83db80 1 --2- 172.21.15.1:0/2689292424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e258474b60 0x55e258480fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.830+0000 7ff61c83db80 1 --2- 172.21.15.1:0/2689292424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e258475470 0x55e25847e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.831+0000 7ff61c83db80 1 -- 172.21.15.1:0/2689292424 >> 172.21.15.1:0/2689292424 conn(0x55e25846a360 msgr2=0x55e25847ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:14.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.831+0000 7ff61c83db80 1 -- 172.21.15.1:0/2689292424 shutdown_connections 2023-11-16T16:46:14.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.831+0000 7ff61c83db80 1 -- 172.21.15.1:0/2689292424 wait complete. 2023-11-16T16:46:14.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.831+0000 7ff61c83db80 20 client.16344 trim_cache size 0 max 16384 2023-11-16T16:46:14.850 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.844+0000 7ff61c83db80 1 Processor -- start 2023-11-16T16:46:14.850 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.844+0000 7ff61c83db80 1 -- start start 2023-11-16T16:46:14.851 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.844+0000 7ff61c83db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e258474b60 0x55e2584857c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.851 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.844+0000 7ff61c83db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e258475470 0x55e258480bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.851 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.844+0000 7ff61c83db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2584798b0 0x55e2584778b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.851 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.844+0000 7ff61c83db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2584759b0 con 0x55e258474b60 2023-11-16T16:46:14.851 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.844+0000 7ff61c83db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e258475bc0 con 0x55e258475470 2023-11-16T16:46:14.851 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.844+0000 7ff61c83db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e258469ee0 con 0x55e2584798b0 2023-11-16T16:46:14.851 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.844+0000 7ff60e44d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e258474b60 0x55e2584857c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.844+0000 7ff60e44d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e258474b60 0x55e2584857c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45688/0 (socket says 172.21.15.1:45688) 2023-11-16T16:46:14.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.844+0000 7ff60e44d700 1 -- 172.21.15.1:0/2532102440 learned_addr learned my addr 172.21.15.1:0/2532102440 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:14.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.844+0000 7ff60dc4c700 1 --2- 172.21.15.1:0/2532102440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e258475470 0x55e258480bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.844+0000 7ff60e44d700 1 -- 172.21.15.1:0/2532102440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2584798b0 msgr2=0x55e2584778b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:14.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.844+0000 7ff60e44d700 1 --2- 172.21.15.1:0/2532102440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2584798b0 0x55e2584778b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.844+0000 7ff60e44d700 1 -- 172.21.15.1:0/2532102440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e258475470 msgr2=0x55e258480bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:14.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.844+0000 7ff60e44d700 1 --2- 172.21.15.1:0/2532102440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e258475470 0x55e258480bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.844+0000 7ff60e44d700 1 -- 172.21.15.1:0/2532102440 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e25847a1a0 con 0x55e258474b60 2023-11-16T16:46:14.853 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.844+0000 7ff60dc4c700 1 --2- 172.21.15.1:0/2532102440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e258475470 0x55e258480bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:46:14.853 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.844+0000 7ff60e44d700 1 --2- 172.21.15.1:0/2532102440 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e258474b60 0x55e2584857c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff604009d70 tx=0x7ff60409a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1dd43ad839a0be0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:14.853 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.844+0000 7ff5fd7fa700 1 -- 172.21.15.1:0/2532102440 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6040b7070 con 0x55e258474b60 2023-11-16T16:46:14.853 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.845+0000 7ff5fd7fa700 1 -- 172.21.15.1:0/2532102440 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff604099170 con 0x55e258474b60 2023-11-16T16:46:14.853 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.845+0000 7ff5fd7fa700 1 -- 172.21.15.1:0/2532102440 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6040a2d00 con 0x55e258474b60 2023-11-16T16:46:14.853 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.845+0000 7ff61c83db80 1 -- 172.21.15.1:0/2532102440 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e258474b60 msgr2=0x55e2584857c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:14.853 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.845+0000 7ff61c83db80 1 --2- 172.21.15.1:0/2532102440 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e258474b60 0x55e2584857c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff604009d70 tx=0x7ff60409a860 comp rx=0 tx=0).stop 2023-11-16T16:46:14.853 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.845+0000 7ff61c83db80 1 -- 172.21.15.1:0/2532102440 shutdown_connections 2023-11-16T16:46:14.854 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.845+0000 7ff61c83db80 1 --2- 172.21.15.1:0/2532102440 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e2584798b0 0x55e2584778b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.854 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.845+0000 7ff61c83db80 1 --2- 172.21.15.1:0/2532102440 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e258474b60 0x55e2584857c0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.854 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.845+0000 7ff61c83db80 1 --2- 172.21.15.1:0/2532102440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e258475470 0x55e258480bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.854 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.845+0000 7ff61c83db80 1 -- 172.21.15.1:0/2532102440 >> 172.21.15.1:0/2532102440 conn(0x55e25846a360 msgr2=0x55e25848a970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:14.854 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.845+0000 7ff61c83db80 1 -- 172.21.15.1:0/2532102440 shutdown_connections 2023-11-16T16:46:14.854 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.845+0000 7ff61c83db80 1 -- 172.21.15.1:0/2532102440 wait complete. 2023-11-16T16:46:14.854 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.845+0000 7ff61c83db80 1 -- start start 2023-11-16T16:46:14.855 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.845+0000 7ff61c83db80 1 Processor -- start 2023-11-16T16:46:14.855 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.846+0000 7ff61c83db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e258474b60 0x55e2584850a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.855 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.846+0000 7ff61c83db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e258475470 0x55e2584855e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.855 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.846+0000 7ff61c83db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2584798b0 0x55e2584d9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.855 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.846+0000 7ff61c83db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2584da1c0 con 0x55e258474b60 2023-11-16T16:46:14.855 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.846+0000 7ff61c83db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2584da330 con 0x55e2584798b0 2023-11-16T16:46:14.855 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.846+0000 7ff61c83db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2584da4f0 con 0x55e258475470 2023-11-16T16:46:14.855 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.846+0000 7ff61c83db80 10 client.0 _ll_register_callbacks cb 0x7ff6000094b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:46:14.855 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.846+0000 7ff60e44d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e258474b60 0x55e2584850a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.856 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.846+0000 7ff60e44d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e258474b60 0x55e2584850a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45694/0 (socket says 172.21.15.1:45694) 2023-11-16T16:46:14.856 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.846+0000 7ff60e44d700 1 -- 172.21.15.1:0/1445149039 learned_addr learned my addr 172.21.15.1:0/1445149039 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:14.856 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.846+0000 7ff60dc4c700 1 --2- 172.21.15.1:0/1445149039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e258475470 0x55e2584855e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.856 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.846+0000 7ff60e44d700 1 -- 172.21.15.1:0/1445149039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e258475470 msgr2=0x55e2584855e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:14.856 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.846+0000 7ff60e44d700 1 --2- 172.21.15.1:0/1445149039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e258475470 0x55e2584855e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.856 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.846+0000 7ff60e44d700 1 -- 172.21.15.1:0/1445149039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2584798b0 msgr2=0x55e2584d9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:14.856 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.846+0000 7ff60e44d700 1 --2- 172.21.15.1:0/1445149039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2584798b0 0x55e2584d9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.857 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.846+0000 7ff60e44d700 1 -- 172.21.15.1:0/1445149039 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e258247630 con 0x55e258474b60 2023-11-16T16:46:14.857 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.846+0000 7ff60e44d700 1 --2- 172.21.15.1:0/1445149039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e258474b60 0x55e2584850a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff6040cb940 tx=0x7ff604022b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=441c1b8192348465 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:14.857 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.846+0000 7ff5f1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:46:14.857 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.846+0000 7ff5f1ffb700 1 -- 172.21.15.1:0/1445149039 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6040b7070 con 0x55e258474b60 2023-11-16T16:46:14.857 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.846+0000 7ff5f1ffb700 1 -- 172.21.15.1:0/1445149039 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8586+0+0 (secure 0 0 0) 0x7ff6040cb120 con 0x55e258474b60 2023-11-16T16:46:14.857 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.846+0000 7ff61c83db80 1 -- 172.21.15.1:0/1445149039 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff6040a3190 con 0x55e258474b60 2023-11-16T16:46:14.857 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.847+0000 7ff5f1ffb700 1 -- 172.21.15.1:0/1445149039 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6040cb2e0 con 0x55e258474b60 2023-11-16T16:46:14.857 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.847+0000 7ff61c83db80 10 client.16350 fetch_fsmap learned FSMap version 28 2023-11-16T16:46:14.858 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.847+0000 7ff61c83db80 1 -- 172.21.15.1:0/1445149039 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e2581f3de0 con 0x55e258474b60 2023-11-16T16:46:14.858 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.847+0000 7ff5f1ffb700 1 -- 172.21.15.1:0/1445149039 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6040a3190 con 0x55e258474b60 2023-11-16T16:46:14.858 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.847+0000 7ff5f1ffb700 1 -- 172.21.15.1:0/1445149039 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff6040b3250 con 0x55e258474b60 2023-11-16T16:46:14.858 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.847+0000 7ff61c83db80 10 client.16350 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:46:14.858 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.847+0000 7ff61c83db80 10 client.16350 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-11-16T16:46:14.858 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.847+0000 7ff61c83db80 20 client.16350 populate_metadata read hostname 'smithi001' 2023-11-16T16:46:14.858 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.847+0000 7ff61c83db80 1 --2- 172.21.15.1:0/1445149039 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55e2584e2a30 0x55e2584e4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.858 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.847+0000 7ff61c83db80 4 client.16350 mds_command: new command op to 24482 tid=0[{"prefix": "damage ls"}] 2023-11-16T16:46:14.859 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.847+0000 7ff61c83db80 1 -- 172.21.15.1:0/1445149039 --> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e258394660 con 0x55e2584e2a30 2023-11-16T16:46:14.859 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.847+0000 7ff60dc4c700 1 --2- 172.21.15.1:0/1445149039 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55e2584e2a30 0x55e2584e4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.859 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.848+0000 7ff60dc4c700 1 --2- 172.21.15.1:0/1445149039 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55e2584e2a30 0x55e2584e4e40 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cbb326020e3f6403 server_cookie=ee817ec83552c498 in_seq=0 out_seq=0 2023-11-16T16:46:14.859 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.848+0000 7ff5f1ffb700 10 client.16350 ms_handle_connect on v2:172.21.15.26:6836/1453504863 2023-11-16T16:46:14.859 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.848+0000 7ff5f1ffb700 1 -- 172.21.15.1:0/1445149039 <== mds.2 v2:172.21.15.26:6836/1453504863 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e258394660 con 0x55e2584e2a30 2023-11-16T16:46:14.859 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.848+0000 7ff5f1ffb700 10 client.16350 handle_command_reply: tid=0 2023-11-16T16:46:14.859 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.848+0000 7ff61c83db80 1 client.16350 shutdown 2023-11-16T16:46:14.859 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.848+0000 7ff61c83db80 1 -- 172.21.15.1:0/1445149039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e258474b60 msgr2=0x55e2584850a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:14.860 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.848+0000 7ff61c83db80 1 --2- 172.21.15.1:0/1445149039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e258474b60 0x55e2584850a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff6040cb940 tx=0x7ff604022b90 comp rx=0 tx=0).stop 2023-11-16T16:46:14.860 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.848+0000 7ff61c83db80 1 -- 172.21.15.1:0/1445149039 shutdown_connections 2023-11-16T16:46:14.860 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.848+0000 7ff61c83db80 1 --2- 172.21.15.1:0/1445149039 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55e2584e2a30 0x55e2584e4e40 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.860 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.848+0000 7ff61c83db80 1 --2- 172.21.15.1:0/1445149039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e2584798b0 0x55e2584d9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.860 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.848+0000 7ff5f1ffb700 0 client.16350 ms_handle_reset on v2:172.21.15.26:6836/1453504863 2023-11-16T16:46:14.860 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.848+0000 7ff61c83db80 1 --2- 172.21.15.1:0/1445149039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e258474b60 0x55e2584850a0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.860 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.848+0000 7ff61c83db80 1 --2- 172.21.15.1:0/1445149039 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e258475470 0x55e2584855e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.861 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.848+0000 7ff61c83db80 1 -- 172.21.15.1:0/1445149039 >> 172.21.15.1:0/1445149039 conn(0x55e25846a360 msgr2=0x55e258489f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:14.861 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.848+0000 7ff61c83db80 1 -- 172.21.15.1:0/1445149039 shutdown_connections 2023-11-16T16:46:14.861 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.848+0000 7ff61c83db80 1 -- 172.21.15.1:0/1445149039 wait complete. 2023-11-16T16:46:14.861 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.848+0000 7ff61c83db80 20 client.16350 trim_cache size 0 max 16384 2023-11-16T16:46:14.861 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2023-11-16T16:46:14.863 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.848+0000 7ff6106b1700 1 -- 172.21.15.1:0/2328002415 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7ff5f40a12c0 msgr2=0x7ff5f40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:14.863 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.848+0000 7ff6106b1700 1 --2- 172.21.15.1:0/2328002415 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7ff5f40a12c0 0x7ff5f40a3770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff608102630 tx=0x7ff5f8009260 comp rx=0 tx=0).stop 2023-11-16T16:46:14.863 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.849+0000 7ff6106b1700 1 -- 172.21.15.1:0/2328002415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6080ff490 msgr2=0x7ff608100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:14.863 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.849+0000 7ff6106b1700 1 --2- 172.21.15.1:0/2328002415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6080ff490 0x7ff608100e40 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff60400b580 tx=0x7ff60400bfd0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.863 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.849+0000 7ff6106b1700 1 -- 172.21.15.1:0/2328002415 shutdown_connections 2023-11-16T16:46:14.863 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.849+0000 7ff6106b1700 1 --2- 172.21.15.1:0/2328002415 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7ff5f40a12c0 0x7ff5f40a3770 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.863 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.849+0000 7ff6106b1700 1 --2- 172.21.15.1:0/2328002415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff608104480 0x7ff6081a45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.864 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.849+0000 7ff6106b1700 1 --2- 172.21.15.1:0/2328002415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6080ff490 0x7ff608100e40 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.864 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.849+0000 7ff6106b1700 1 --2- 172.21.15.1:0/2328002415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6080ffe30 0x7ff608101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.864 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.849+0000 7ff6106b1700 1 -- 172.21.15.1:0/2328002415 >> 172.21.15.1:0/2328002415 conn(0x7ff6080763a0 msgr2=0x7ff608068e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:14.864 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.849+0000 7ff6106b1700 1 -- 172.21.15.1:0/2328002415 shutdown_connections 2023-11-16T16:46:14.864 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.849+0000 7ff6106b1700 1 -- 172.21.15.1:0/2328002415 wait complete. 2023-11-16T16:46:14.864 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-11-16T16:46:14.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:14 smithi001 ceph-mon[98800]: pgmap v216: 97 pgs: 97 active+clean; 817 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 KiB/s wr, 8 op/s 2023-11-16T16:46:14.992 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.990+0000 7f984c704700 1 -- 172.21.15.1:0/1142249184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98440ff3f0 msgr2=0x7f9844104ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:14.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.990+0000 7f984c704700 1 --2- 172.21.15.1:0/1142249184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98440ff3f0 0x7f9844104ec0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f98340099c0 tx=0x7f9834009cd0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.991+0000 7f984c704700 1 -- 172.21.15.1:0/1142249184 shutdown_connections 2023-11-16T16:46:14.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.991+0000 7f984c704700 1 --2- 172.21.15.1:0/1142249184 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98441055f0 0x7f984410d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.991+0000 7f984c704700 1 --2- 172.21.15.1:0/1142249184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98440fea50 0x7f98440fee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.991+0000 7f984c704700 1 --2- 172.21.15.1:0/1142249184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98440ff3f0 0x7f9844104ec0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.991+0000 7f984c704700 1 -- 172.21.15.1:0/1142249184 >> 172.21.15.1:0/1142249184 conn(0x7f9844075880 msgr2=0x7f9844075c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:14.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.991+0000 7f984c704700 1 -- 172.21.15.1:0/1142249184 shutdown_connections 2023-11-16T16:46:14.994 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.991+0000 7f984c704700 1 -- 172.21.15.1:0/1142249184 wait complete. 2023-11-16T16:46:14.996 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.991+0000 7f984c704700 1 Processor -- start 2023-11-16T16:46:14.996 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.991+0000 7f984c704700 1 -- start start 2023-11-16T16:46:14.996 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.992+0000 7f984c704700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98440fea50 0x7f98440ff9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.996 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.992+0000 7f984c704700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98440ff3f0 0x7f98440fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.996 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.992+0000 7f984c704700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98441055f0 0x7f98441a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:14.996 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.992+0000 7f984c704700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9844100730 con 0x7f98441055f0 2023-11-16T16:46:14.996 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.992+0000 7f984c704700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98441a4b00 con 0x7f98440ff3f0 2023-11-16T16:46:14.997 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.992+0000 7f984c704700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98441a4c40 con 0x7f98440fea50 2023-11-16T16:46:14.997 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.992+0000 7f984aca1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98441055f0 0x7f98441a45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.997 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.992+0000 7f984aca1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98441055f0 0x7f98441a45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45710/0 (socket says 172.21.15.1:45710) 2023-11-16T16:46:14.997 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.992+0000 7f984aca1700 1 -- 172.21.15.1:0/4012169883 learned_addr learned my addr 172.21.15.1:0/4012169883 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:14.997 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.992+0000 7f984a4a0700 1 --2- 172.21.15.1:0/4012169883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98440fea50 0x7f98440ff9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.997 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.992+0000 7f9849c9f700 1 --2- 172.21.15.1:0/4012169883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98440ff3f0 0x7f98440fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:14.997 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.992+0000 7f984aca1700 1 -- 172.21.15.1:0/4012169883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98440fea50 msgr2=0x7f98440ff9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:14.997 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.992+0000 7f984aca1700 1 --2- 172.21.15.1:0/4012169883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98440fea50 0x7f98440ff9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.998 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.992+0000 7f984aca1700 1 -- 172.21.15.1:0/4012169883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98440ff3f0 msgr2=0x7f98440fff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:14.998 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.992+0000 7f984aca1700 1 --2- 172.21.15.1:0/4012169883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98440ff3f0 0x7f98440fff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:14.998 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.992+0000 7f984aca1700 1 -- 172.21.15.1:0/4012169883 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98340095f0 con 0x7f98441055f0 2023-11-16T16:46:14.998 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.992+0000 7f984aca1700 1 --2- 172.21.15.1:0/4012169883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98441055f0 0x7f98441a45c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f983c00ada0 tx=0x7f983c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:14.998 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.992+0000 7f983b7fe700 1 -- 172.21.15.1:0/4012169883 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c00f070 con 0x7f98441055f0 2023-11-16T16:46:14.998 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.993+0000 7f983b7fe700 1 -- 172.21.15.1:0/4012169883 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f983c007b30 con 0x7f98441055f0 2023-11-16T16:46:14.998 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.993+0000 7f984c704700 1 -- 172.21.15.1:0/4012169883 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9844110330 con 0x7f98441055f0 2023-11-16T16:46:14.999 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.993+0000 7f983b7fe700 1 -- 172.21.15.1:0/4012169883 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c0053e0 con 0x7f98441055f0 2023-11-16T16:46:14.999 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.993+0000 7f984c704700 1 -- 172.21.15.1:0/4012169883 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98441101a0 con 0x7f98441055f0 2023-11-16T16:46:15.005 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.995+0000 7f983b7fe700 1 -- 172.21.15.1:0/4012169883 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f983c00c980 con 0x7f98441055f0 2023-11-16T16:46:15.005 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.995+0000 7f983b7fe700 1 --2- 172.21.15.1:0/4012169883 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f98300a12c0 0x7f98300a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:15.005 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.996+0000 7f984a4a0700 1 --2- 172.21.15.1:0/4012169883 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f98300a12c0 0x7f98300a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:15.005 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.996+0000 7f983b7fe700 1 -- 172.21.15.1:0/4012169883 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8586+0+0 (secure 0 0 0) 0x7f983c0cc330 con 0x7f98441055f0 2023-11-16T16:46:15.005 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.996+0000 7f984a4a0700 1 --2- 172.21.15.1:0/4012169883 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f98300a12c0 0x7f98300a3770 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9840009a10 tx=0x7f9840009190 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:15.006 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.996+0000 7f9858890b80 1 Processor -- start 2023-11-16T16:46:15.006 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.997+0000 7f9858890b80 1 -- start start 2023-11-16T16:46:15.006 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.997+0000 7f9858890b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560103d35b60 0x560103d35f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:15.006 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.997+0000 7f9858890b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560103d36470 0x560103d3a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:15.006 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.997+0000 7f9858890b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560103d3a8b0 0x560103d3ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:15.006 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.997+0000 7f9858890b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560103c3b060 con 0x560103d35b60 2023-11-16T16:46:15.006 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.997+0000 7f9858890b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560103c8d8a0 con 0x560103d3a8b0 2023-11-16T16:46:15.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.997+0000 7f9858890b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560103c86d30 con 0x560103d36470 2023-11-16T16:46:15.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.997+0000 7f984a4a0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560103d35b60 0x560103d35f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:15.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.997+0000 7f984a4a0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560103d35b60 0x560103d35f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45714/0 (socket says 172.21.15.1:45714) 2023-11-16T16:46:15.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.997+0000 7f984a4a0700 1 -- 172.21.15.1:0/3084533063 learned_addr learned my addr 172.21.15.1:0/3084533063 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:15.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.997+0000 7f9849c9f700 1 --2- 172.21.15.1:0/3084533063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560103d36470 0x560103d3a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:15.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.997+0000 7f984aca1700 1 --2- 172.21.15.1:0/3084533063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560103d3a8b0 0x560103d3ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:15.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.998+0000 7f984a4a0700 1 -- 172.21.15.1:0/3084533063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560103d36470 msgr2=0x560103d3a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:15.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.998+0000 7f984a4a0700 1 --2- 172.21.15.1:0/3084533063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560103d36470 0x560103d3a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.998+0000 7f984a4a0700 1 -- 172.21.15.1:0/3084533063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560103d3a8b0 msgr2=0x560103d3ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:15.008 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.998+0000 7f984a4a0700 1 --2- 172.21.15.1:0/3084533063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560103d3a8b0 0x560103d3ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.008 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.998+0000 7f984a4a0700 1 -- 172.21.15.1:0/3084533063 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560103b08630 con 0x560103d35b60 2023-11-16T16:46:15.008 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.998+0000 7f984a4a0700 1 --2- 172.21.15.1:0/3084533063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560103d35b60 0x560103d35f30 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f984000ea00 tx=0x7f9840011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ced5855d0e534 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:15.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.998+0000 7f9838ff9700 1 -- 172.21.15.1:0/3084533063 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984000eec0 con 0x560103d35b60 2023-11-16T16:46:15.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.998+0000 7f9838ff9700 1 -- 172.21.15.1:0/3084533063 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9840015940 con 0x560103d35b60 2023-11-16T16:46:15.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.998+0000 7f9838ff9700 1 -- 172.21.15.1:0/3084533063 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9840013970 con 0x560103d35b60 2023-11-16T16:46:15.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.999+0000 7f9858890b80 1 -- 172.21.15.1:0/3084533063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560103d35b60 msgr2=0x560103d35f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:15.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.999+0000 7f9858890b80 1 --2- 172.21.15.1:0/3084533063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560103d35b60 0x560103d35f30 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f984000ea00 tx=0x7f9840011000 comp rx=0 tx=0).stop 2023-11-16T16:46:15.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.999+0000 7f9858890b80 1 -- 172.21.15.1:0/3084533063 shutdown_connections 2023-11-16T16:46:15.010 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.999+0000 7f9858890b80 1 --2- 172.21.15.1:0/3084533063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560103d3a8b0 0x560103d3ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.010 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.999+0000 7f9858890b80 1 --2- 172.21.15.1:0/3084533063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560103d35b60 0x560103d35f30 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.010 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.999+0000 7f9858890b80 1 --2- 172.21.15.1:0/3084533063 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560103d36470 0x560103d3a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.010 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.999+0000 7f9858890b80 1 -- 172.21.15.1:0/3084533063 >> 172.21.15.1:0/3084533063 conn(0x560103d2b360 msgr2=0x560103d30b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:15.010 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.999+0000 7f9858890b80 1 -- 172.21.15.1:0/3084533063 shutdown_connections 2023-11-16T16:46:15.010 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.999+0000 7f9858890b80 1 -- 172.21.15.1:0/3084533063 wait complete. 2023-11-16T16:46:15.010 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.999+0000 7f9858890b80 1 -- start start 2023-11-16T16:46:15.010 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.999+0000 7f9858890b80 1 Processor -- start 2023-11-16T16:46:15.011 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:14.999+0000 7f9858890b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560103d35b60 0x560103d420c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:15.011 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.000+0000 7f9858890b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560103d36470 0x560103d3f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:15.011 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.000+0000 7f9858890b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560103d3a8b0 0x560103d3fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:15.011 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.000+0000 7f9858890b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560103c288e0 con 0x560103d36470 2023-11-16T16:46:15.011 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.000+0000 7f9858890b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560103c28580 con 0x560103d35b60 2023-11-16T16:46:15.011 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.000+0000 7f9858890b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560103b0a920 con 0x560103d3a8b0 2023-11-16T16:46:15.011 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.000+0000 7f9858890b80 10 client.0 _ll_register_callbacks cb 0x560103c46780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:46:15.011 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.000+0000 7f9849c9f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560103d36470 0x560103d3f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:15.012 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.000+0000 7f9849c9f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560103d36470 0x560103d3f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45722/0 (socket says 172.21.15.1:45722) 2023-11-16T16:46:15.012 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.000+0000 7f9849c9f700 1 -- 172.21.15.1:0/3909637666 learned_addr learned my addr 172.21.15.1:0/3909637666 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:15.012 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.000+0000 7f984a4a0700 1 --2- 172.21.15.1:0/3909637666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560103d35b60 0x560103d420c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:15.012 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.000+0000 7f984aca1700 1 --2- 172.21.15.1:0/3909637666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560103d3a8b0 0x560103d3fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:15.012 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.000+0000 7f9849c9f700 1 -- 172.21.15.1:0/3909637666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560103d3a8b0 msgr2=0x560103d3fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:15.012 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.000+0000 7f9849c9f700 1 --2- 172.21.15.1:0/3909637666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560103d3a8b0 0x560103d3fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.012 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.000+0000 7f9849c9f700 1 -- 172.21.15.1:0/3909637666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560103d35b60 msgr2=0x560103d420c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:15.012 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.000+0000 7f9849c9f700 1 --2- 172.21.15.1:0/3909637666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560103d35b60 0x560103d420c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.013 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.000+0000 7f9849c9f700 1 -- 172.21.15.1:0/3909637666 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f984000e6e0 con 0x560103d36470 2023-11-16T16:46:15.013 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.000+0000 7f984a4a0700 1 --2- 172.21.15.1:0/3909637666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560103d35b60 0x560103d420c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:46:15.013 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.000+0000 7f984aca1700 1 --2- 172.21.15.1:0/3909637666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560103d3a8b0 0x560103d3fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:46:15.013 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.000+0000 7f9849c9f700 1 --2- 172.21.15.1:0/3909637666 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560103d36470 0x560103d3f340 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f98440ff7c0 tx=0x7f98340058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0acd419a281b1ff server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:15.013 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.000+0000 7f982dffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:46:15.013 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.000+0000 7f982dffb700 1 -- 172.21.15.1:0/3909637666 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983400bd20 con 0x560103d36470 2023-11-16T16:46:15.013 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.000+0000 7f982dffb700 1 -- 172.21.15.1:0/3909637666 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8586+0+0 (secure 0 0 0) 0x7f983400be80 con 0x560103d36470 2023-11-16T16:46:15.013 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.000+0000 7f9858890b80 1 -- 172.21.15.1:0/3909637666 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560103d28b20 con 0x560103d36470 2023-11-16T16:46:15.014 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.000+0000 7f982dffb700 1 -- 172.21.15.1:0/3909637666 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98340038c0 con 0x560103d36470 2023-11-16T16:46:15.014 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.000+0000 7f9858890b80 10 client.16362 fetch_fsmap learned FSMap version 28 2023-11-16T16:46:15.014 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.000+0000 7f9858890b80 1 -- 172.21.15.1:0/3909637666 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x560103c4d190 con 0x560103d36470 2023-11-16T16:46:15.014 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.001+0000 7f982dffb700 1 -- 172.21.15.1:0/3909637666 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560103d28b20 con 0x560103d36470 2023-11-16T16:46:15.014 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.001+0000 7f982dffb700 1 -- 172.21.15.1:0/3909637666 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9834004900 con 0x560103d36470 2023-11-16T16:46:15.014 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.001+0000 7f9858890b80 10 client.16362 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:46:15.014 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.001+0000 7f9858890b80 10 client.16362 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-11-16T16:46:15.014 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.001+0000 7f9858890b80 20 client.16362 populate_metadata read hostname 'smithi001' 2023-11-16T16:46:15.015 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.001+0000 7f9858890b80 1 --2- 172.21.15.1:0/3909637666 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x560103d4f9a0 0x560103d51db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:15.015 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.001+0000 7f9858890b80 4 client.16362 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:46:15.015 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.001+0000 7f9858890b80 1 -- 172.21.15.1:0/3909637666 --> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560103c490a0 con 0x560103d4f9a0 2023-11-16T16:46:15.015 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.001+0000 7f984a4a0700 1 --2- 172.21.15.1:0/3909637666 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x560103d4f9a0 0x560103d51db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:15.015 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.002+0000 7f982dffb700 10 client.16362 ms_handle_connect on v2:172.21.15.53:6838/1044753024 2023-11-16T16:46:15.015 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.002+0000 7f984a4a0700 1 --2- 172.21.15.1:0/3909637666 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x560103d4f9a0 0x560103d51db0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=625601866fa5afd8 server_cookie=45b48cbf8a580b in_seq=0 out_seq=0 2023-11-16T16:46:15.015 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.002+0000 7f982dffb700 1 -- 172.21.15.1:0/3909637666 <== mds.1 v2:172.21.15.53:6838/1044753024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560103c490a0 con 0x560103d4f9a0 2023-11-16T16:46:15.015 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.002+0000 7f982dffb700 10 client.16362 handle_command_reply: tid=0 2023-11-16T16:46:15.016 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.002+0000 7f9858890b80 1 client.16362 shutdown 2023-11-16T16:46:15.016 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.003+0000 7f9858890b80 1 -- 172.21.15.1:0/3909637666 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560103d36470 msgr2=0x560103d3f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:15.016 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.003+0000 7f9858890b80 1 --2- 172.21.15.1:0/3909637666 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560103d36470 0x560103d3f340 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f98440ff7c0 tx=0x7f98340058e0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.016 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.003+0000 7f9858890b80 1 -- 172.21.15.1:0/3909637666 shutdown_connections 2023-11-16T16:46:15.016 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.003+0000 7f9858890b80 1 --2- 172.21.15.1:0/3909637666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560103d3a8b0 0x560103d3fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.016 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.003+0000 7f9858890b80 1 --2- 172.21.15.1:0/3909637666 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x560103d4f9a0 0x560103d51db0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.016 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.003+0000 7f9858890b80 1 --2- 172.21.15.1:0/3909637666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560103d35b60 0x560103d420c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.017 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.003+0000 7f9858890b80 1 --2- 172.21.15.1:0/3909637666 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560103d36470 0x560103d3f340 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.017 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.003+0000 7f982dffb700 0 client.16362 ms_handle_reset on v2:172.21.15.53:6838/1044753024 2023-11-16T16:46:15.017 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.003+0000 7f9858890b80 1 -- 172.21.15.1:0/3909637666 >> 172.21.15.1:0/3909637666 conn(0x560103d2b360 msgr2=0x560103d3bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:15.017 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.003+0000 7f9858890b80 1 -- 172.21.15.1:0/3909637666 shutdown_connections 2023-11-16T16:46:15.017 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.003+0000 7f9858890b80 1 -- 172.21.15.1:0/3909637666 wait complete. 2023-11-16T16:46:15.017 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.003+0000 7f9858890b80 20 client.16362 trim_cache size 0 max 16384 2023-11-16T16:46:15.018 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:14 smithi026 ceph-mon[100112]: pgmap v216: 97 pgs: 97 active+clean; 817 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 KiB/s wr, 8 op/s 2023-11-16T16:46:15.024 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.016+0000 7f9858890b80 1 Processor -- start 2023-11-16T16:46:15.024 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.016+0000 7f9858890b80 1 -- start start 2023-11-16T16:46:15.024 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.016+0000 7f9858890b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560103d35b60 0x560103d381a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:15.024 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.016+0000 7f9858890b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560103d36470 0x560103d41bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:15.024 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.016+0000 7f9858890b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560103d3a8b0 0x560103d38a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:15.024 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.016+0000 7f9858890b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560103d3d8a0 con 0x560103d35b60 2023-11-16T16:46:15.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.016+0000 7f9858890b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560103d394c0 con 0x560103d3a8b0 2023-11-16T16:46:15.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.016+0000 7f9858890b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560103d3fc20 con 0x560103d36470 2023-11-16T16:46:15.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.016+0000 7f984a4a0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560103d35b60 0x560103d381a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:15.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.016+0000 7f984a4a0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560103d35b60 0x560103d381a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45728/0 (socket says 172.21.15.1:45728) 2023-11-16T16:46:15.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.016+0000 7f984a4a0700 1 -- 172.21.15.1:0/2183887230 learned_addr learned my addr 172.21.15.1:0/2183887230 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:15.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.016+0000 7f9849c9f700 1 --2- 172.21.15.1:0/2183887230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560103d36470 0x560103d41bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:15.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.016+0000 7f984a4a0700 1 -- 172.21.15.1:0/2183887230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560103d36470 msgr2=0x560103d41bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:15.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.016+0000 7f984a4a0700 1 --2- 172.21.15.1:0/2183887230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560103d36470 0x560103d41bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.016+0000 7f984a4a0700 1 -- 172.21.15.1:0/2183887230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560103d3a8b0 msgr2=0x560103d38a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:15.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.016+0000 7f984aca1700 1 --2- 172.21.15.1:0/2183887230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560103d3a8b0 0x560103d38a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:15.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.016+0000 7f984a4a0700 1 --2- 172.21.15.1:0/2183887230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560103d3a8b0 0x560103d38a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.016+0000 7f984a4a0700 1 -- 172.21.15.1:0/2183887230 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560103c4cee0 con 0x560103d35b60 2023-11-16T16:46:15.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.016+0000 7f9849c9f700 1 --2- 172.21.15.1:0/2183887230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560103d36470 0x560103d41bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:46:15.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.016+0000 7f984aca1700 1 --2- 172.21.15.1:0/2183887230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560103d3a8b0 0x560103d38a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:46:15.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.016+0000 7f984a4a0700 1 --2- 172.21.15.1:0/2183887230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560103d35b60 0x560103d381a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f98400134c0 tx=0x7f9840013450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3a840f64818e65 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:15.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.017+0000 7f98397fa700 1 -- 172.21.15.1:0/2183887230 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9840030580 con 0x560103d35b60 2023-11-16T16:46:15.027 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.017+0000 7f98397fa700 1 -- 172.21.15.1:0/2183887230 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9840015d90 con 0x560103d35b60 2023-11-16T16:46:15.027 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.017+0000 7f98397fa700 1 -- 172.21.15.1:0/2183887230 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9840030ea0 con 0x560103d35b60 2023-11-16T16:46:15.027 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.017+0000 7f9858890b80 1 -- 172.21.15.1:0/2183887230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560103d35b60 msgr2=0x560103d381a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:15.027 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.017+0000 7f9858890b80 1 --2- 172.21.15.1:0/2183887230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560103d35b60 0x560103d381a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f98400134c0 tx=0x7f9840013450 comp rx=0 tx=0).stop 2023-11-16T16:46:15.027 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.017+0000 7f9858890b80 1 -- 172.21.15.1:0/2183887230 shutdown_connections 2023-11-16T16:46:15.027 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.017+0000 7f9858890b80 1 --2- 172.21.15.1:0/2183887230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560103d3a8b0 0x560103d38a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.027 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.017+0000 7f9858890b80 1 --2- 172.21.15.1:0/2183887230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560103d35b60 0x560103d381a0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.028 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.017+0000 7f9858890b80 1 --2- 172.21.15.1:0/2183887230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560103d36470 0x560103d41bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.028 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.017+0000 7f9858890b80 1 -- 172.21.15.1:0/2183887230 >> 172.21.15.1:0/2183887230 conn(0x560103d2b360 msgr2=0x560103d4b150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:15.028 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.017+0000 7f9858890b80 1 -- 172.21.15.1:0/2183887230 shutdown_connections 2023-11-16T16:46:15.028 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.017+0000 7f9858890b80 1 -- 172.21.15.1:0/2183887230 wait complete. 2023-11-16T16:46:15.028 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.017+0000 7f9858890b80 1 -- start start 2023-11-16T16:46:15.028 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.018+0000 7f9858890b80 1 Processor -- start 2023-11-16T16:46:15.028 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.018+0000 7f9858890b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560103d35b60 0x560103d45bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:15.028 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.018+0000 7f9858890b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560103d36470 0x560103d46110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:15.029 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.018+0000 7f9858890b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560103d3a8b0 0x560103da2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:15.029 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.018+0000 7f9858890b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560103d46960 con 0x560103d35b60 2023-11-16T16:46:15.029 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.018+0000 7f9858890b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560103da2da0 con 0x560103d36470 2023-11-16T16:46:15.029 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.018+0000 7f9858890b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560103da2ee0 con 0x560103d3a8b0 2023-11-16T16:46:15.029 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.018+0000 7f9858890b80 10 client.0 _ll_register_callbacks cb 0x7f9834005350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:46:15.029 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.018+0000 7f984a4a0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560103d35b60 0x560103d45bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:15.029 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.018+0000 7f984a4a0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560103d35b60 0x560103d45bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45734/0 (socket says 172.21.15.1:45734) 2023-11-16T16:46:15.029 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.018+0000 7f984a4a0700 1 -- 172.21.15.1:0/1134164940 learned_addr learned my addr 172.21.15.1:0/1134164940 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:15.030 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.018+0000 7f9849c9f700 1 --2- 172.21.15.1:0/1134164940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560103d36470 0x560103d46110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:15.030 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.018+0000 7f984aca1700 1 --2- 172.21.15.1:0/1134164940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560103d3a8b0 0x560103da2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:15.030 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.018+0000 7f984a4a0700 1 -- 172.21.15.1:0/1134164940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560103d3a8b0 msgr2=0x560103da2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:15.030 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.018+0000 7f984a4a0700 1 --2- 172.21.15.1:0/1134164940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560103d3a8b0 0x560103da2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.030 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.018+0000 7f984a4a0700 1 -- 172.21.15.1:0/1134164940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560103d36470 msgr2=0x560103d46110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:15.030 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.018+0000 7f984a4a0700 1 --2- 172.21.15.1:0/1134164940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560103d36470 0x560103d46110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.030 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.018+0000 7f984a4a0700 1 -- 172.21.15.1:0/1134164940 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560103b08630 con 0x560103d35b60 2023-11-16T16:46:15.031 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.018+0000 7f984aca1700 1 --2- 172.21.15.1:0/1134164940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560103d3a8b0 0x560103da2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:46:15.031 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.018+0000 7f984a4a0700 1 --2- 172.21.15.1:0/1134164940 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560103d35b60 0x560103d45bd0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f9840010010 tx=0x7f984000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b11c4ac32c72283a server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:15.031 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.018+0000 7f982dffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:46:15.031 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.018+0000 7f982dffb700 1 -- 172.21.15.1:0/1134164940 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9840030580 con 0x560103d35b60 2023-11-16T16:46:15.031 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.019+0000 7f982dffb700 1 -- 172.21.15.1:0/1134164940 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8586+0+0 (secure 0 0 0) 0x7f984002d040 con 0x560103d35b60 2023-11-16T16:46:15.031 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.019+0000 7f9858890b80 1 -- 172.21.15.1:0/1134164940 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9840015940 con 0x560103d35b60 2023-11-16T16:46:15.031 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.019+0000 7f982dffb700 1 -- 172.21.15.1:0/1134164940 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9840030ae0 con 0x560103d35b60 2023-11-16T16:46:15.031 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.019+0000 7f9858890b80 10 client.16368 fetch_fsmap learned FSMap version 28 2023-11-16T16:46:15.032 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.019+0000 7f9858890b80 1 -- 172.21.15.1:0/1134164940 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x560103a9dc60 con 0x560103d35b60 2023-11-16T16:46:15.032 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.019+0000 7f982dffb700 1 -- 172.21.15.1:0/1134164940 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f984004b080 con 0x560103d35b60 2023-11-16T16:46:15.032 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.019+0000 7f982dffb700 1 -- 172.21.15.1:0/1134164940 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9840013be0 con 0x560103d35b60 2023-11-16T16:46:15.032 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.019+0000 7f9858890b80 10 client.16368 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:46:15.032 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.019+0000 7f9858890b80 10 client.16368 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-11-16T16:46:15.032 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.019+0000 7f9858890b80 20 client.16368 populate_metadata read hostname 'smithi001' 2023-11-16T16:46:15.032 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.019+0000 7f9858890b80 1 --2- 172.21.15.1:0/1134164940 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x560103da4000 0x560103da63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:15.032 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.019+0000 7f9858890b80 4 client.16368 mds_command: new command op to 24493 tid=0[{"prefix": "damage ls"}] 2023-11-16T16:46:15.033 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.019+0000 7f9858890b80 1 -- 172.21.15.1:0/1134164940 --> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560103c55660 con 0x560103da4000 2023-11-16T16:46:15.033 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.019+0000 7f9849c9f700 1 --2- 172.21.15.1:0/1134164940 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x560103da4000 0x560103da63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:15.033 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.020+0000 7f982dffb700 10 client.16368 ms_handle_connect on v2:172.21.15.53:6838/1044753024 2023-11-16T16:46:15.033 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.020+0000 7f9849c9f700 1 --2- 172.21.15.1:0/1134164940 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x560103da4000 0x560103da63e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9467e63410d78acd server_cookie=16472f4cddc9b41 in_seq=0 out_seq=0 2023-11-16T16:46:15.033 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.020+0000 7f982dffb700 1 -- 172.21.15.1:0/1134164940 <== mds.1 v2:172.21.15.53:6838/1044753024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560103c55660 con 0x560103da4000 2023-11-16T16:46:15.033 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.020+0000 7f982dffb700 10 client.16368 handle_command_reply: tid=0 2023-11-16T16:46:15.033 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.020+0000 7f9858890b80 1 client.16368 shutdown 2023-11-16T16:46:15.033 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.020+0000 7f9858890b80 1 -- 172.21.15.1:0/1134164940 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560103d35b60 msgr2=0x560103d45bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:15.034 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.020+0000 7f9858890b80 1 --2- 172.21.15.1:0/1134164940 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560103d35b60 0x560103d45bd0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f9840010010 tx=0x7f984000ee50 comp rx=0 tx=0).stop 2023-11-16T16:46:15.034 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.020+0000 7f9858890b80 1 -- 172.21.15.1:0/1134164940 shutdown_connections 2023-11-16T16:46:15.034 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.020+0000 7f9858890b80 1 --2- 172.21.15.1:0/1134164940 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560103d3a8b0 0x560103da2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.034 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.020+0000 7f9858890b80 1 --2- 172.21.15.1:0/1134164940 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560103d35b60 0x560103d45bd0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.034 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.020+0000 7f9858890b80 1 --2- 172.21.15.1:0/1134164940 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x560103da4000 0x560103da63e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.034 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.020+0000 7f9858890b80 1 --2- 172.21.15.1:0/1134164940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560103d36470 0x560103d46110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.034 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.020+0000 7f982dffb700 0 client.16368 ms_handle_reset on v2:172.21.15.53:6838/1044753024 2023-11-16T16:46:15.034 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.020+0000 7f9858890b80 1 -- 172.21.15.1:0/1134164940 >> 172.21.15.1:0/1134164940 conn(0x560103d2b360 msgr2=0x560103d38a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:15.035 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.020+0000 7f9858890b80 1 -- 172.21.15.1:0/1134164940 shutdown_connections 2023-11-16T16:46:15.035 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.020+0000 7f9858890b80 1 -- 172.21.15.1:0/1134164940 wait complete. 2023-11-16T16:46:15.035 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.020+0000 7f9858890b80 20 client.16368 trim_cache size 0 max 16384 2023-11-16T16:46:15.035 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2023-11-16T16:46:15.036 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.021+0000 7f984c704700 1 -- 172.21.15.1:0/4012169883 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f98300a12c0 msgr2=0x7f98300a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:15.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.021+0000 7f984c704700 1 --2- 172.21.15.1:0/4012169883 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f98300a12c0 0x7f98300a3770 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9840009a10 tx=0x7f9840009190 comp rx=0 tx=0).stop 2023-11-16T16:46:15.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.021+0000 7f984c704700 1 -- 172.21.15.1:0/4012169883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98441055f0 msgr2=0x7f98441a45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:15.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.021+0000 7f984c704700 1 --2- 172.21.15.1:0/4012169883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98441055f0 0x7f98441a45c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f983c00ada0 tx=0x7f983c00c330 comp rx=0 tx=0).stop 2023-11-16T16:46:15.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.021+0000 7f984c704700 1 -- 172.21.15.1:0/4012169883 shutdown_connections 2023-11-16T16:46:15.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.021+0000 7f984c704700 1 --2- 172.21.15.1:0/4012169883 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f98300a12c0 0x7f98300a3770 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.021+0000 7f984c704700 1 --2- 172.21.15.1:0/4012169883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98441055f0 0x7f98441a45c0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.021+0000 7f984c704700 1 --2- 172.21.15.1:0/4012169883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f98440fea50 0x7f98440ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.021+0000 7f984c704700 1 --2- 172.21.15.1:0/4012169883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98440ff3f0 0x7f98440fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.021+0000 7f984c704700 1 -- 172.21.15.1:0/4012169883 >> 172.21.15.1:0/4012169883 conn(0x7f9844075880 msgr2=0x7f9844103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:15.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.021+0000 7f984c704700 1 -- 172.21.15.1:0/4012169883 shutdown_connections 2023-11-16T16:46:15.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.021+0000 7f984c704700 1 -- 172.21.15.1:0/4012169883 wait complete. 2023-11-16T16:46:15.038 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-16T16:46:15.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:14 smithi053 ceph-mon[100638]: pgmap v216: 97 pgs: 97 active+clean; 817 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 KiB/s wr, 8 op/s 2023-11-16T16:46:15.166 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.164+0000 7f866f4db700 1 -- 172.21.15.1:0/166775509 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86680ff410 msgr2=0x7f86680ff7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:15.166 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.164+0000 7f866f4db700 1 --2- 172.21.15.1:0/166775509 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86680ff410 0x7f86680ff7e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f86580099c0 tx=0x7f8658009cd0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.166 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.164+0000 7f866f4db700 1 -- 172.21.15.1:0/166775509 shutdown_connections 2023-11-16T16:46:15.166 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.164+0000 7f866f4db700 1 --2- 172.21.15.1:0/166775509 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8668104450 0x7f866810c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.164+0000 7f866f4db700 1 --2- 172.21.15.1:0/166775509 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86680ff410 0x7f86680ff7e0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.164+0000 7f866f4db700 1 --2- 172.21.15.1:0/166775509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86680ffdb0 0x7f8668103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.164+0000 7f866f4db700 1 -- 172.21.15.1:0/166775509 >> 172.21.15.1:0/166775509 conn(0x7f8668076370 msgr2=0x7f8668076770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:15.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.164+0000 7f866f4db700 1 -- 172.21.15.1:0/166775509 shutdown_connections 2023-11-16T16:46:15.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.164+0000 7f866f4db700 1 -- 172.21.15.1:0/166775509 wait complete. 2023-11-16T16:46:15.169 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.165+0000 7f866f4db700 1 Processor -- start 2023-11-16T16:46:15.169 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.165+0000 7f866f4db700 1 -- start start 2023-11-16T16:46:15.169 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.165+0000 7f866f4db700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86680ff410 0x7f8668197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:15.170 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.165+0000 7f866f4db700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86680ffdb0 0x7f8668197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:15.170 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.165+0000 7f866f4db700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8668104450 0x7f86681a21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:15.170 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.165+0000 7f866f4db700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86681a2700 con 0x7f86680ffdb0 2023-11-16T16:46:15.170 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.165+0000 7f866f4db700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86681a2840 con 0x7f86680ff410 2023-11-16T16:46:15.170 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.165+0000 7f866f4db700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86681a2980 con 0x7f8668104450 2023-11-16T16:46:15.170 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.165+0000 7f866ca76700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86680ffdb0 0x7f8668197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:15.170 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.165+0000 7f866ca76700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86680ffdb0 0x7f8668197680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45750/0 (socket says 172.21.15.1:45750) 2023-11-16T16:46:15.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.165+0000 7f866ca76700 1 -- 172.21.15.1:0/840694823 learned_addr learned my addr 172.21.15.1:0/840694823 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:15.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.165+0000 7f866da78700 1 --2- 172.21.15.1:0/840694823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8668104450 0x7f86681a21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:15.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.165+0000 7f866d277700 1 --2- 172.21.15.1:0/840694823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86680ff410 0x7f8668197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:15.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.166+0000 7f866ca76700 1 -- 172.21.15.1:0/840694823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8668104450 msgr2=0x7f86681a21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:15.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.166+0000 7f866ca76700 1 --2- 172.21.15.1:0/840694823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8668104450 0x7f86681a21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.166+0000 7f866ca76700 1 -- 172.21.15.1:0/840694823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86680ff410 msgr2=0x7f8668197140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:15.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.166+0000 7f866ca76700 1 --2- 172.21.15.1:0/840694823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86680ff410 0x7f8668197140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.166+0000 7f866ca76700 1 -- 172.21.15.1:0/840694823 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86580095f0 con 0x7f86680ffdb0 2023-11-16T16:46:15.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.166+0000 7f866da78700 1 --2- 172.21.15.1:0/840694823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8668104450 0x7f86681a21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:46:15.172 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.166+0000 7f866d277700 1 --2- 172.21.15.1:0/840694823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86680ff410 0x7f8668197140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:46:15.172 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.166+0000 7f866ca76700 1 --2- 172.21.15.1:0/840694823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86680ffdb0 0x7f8668197680 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f866400d950 tx=0x7f866400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:15.172 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.166+0000 7f865e7fc700 1 -- 172.21.15.1:0/840694823 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8664009b40 con 0x7f86680ffdb0 2023-11-16T16:46:15.172 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.166+0000 7f865e7fc700 1 -- 172.21.15.1:0/840694823 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f866400de30 con 0x7f86680ffdb0 2023-11-16T16:46:15.172 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.166+0000 7f866f4db700 1 -- 172.21.15.1:0/840694823 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f866810f260 con 0x7f86680ffdb0 2023-11-16T16:46:15.172 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.166+0000 7f865e7fc700 1 -- 172.21.15.1:0/840694823 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866400f620 con 0x7f86680ffdb0 2023-11-16T16:46:15.172 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.166+0000 7f866f4db700 1 -- 172.21.15.1:0/840694823 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f866810f0d0 con 0x7f86680ffdb0 2023-11-16T16:46:15.178 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.168+0000 7f865e7fc700 1 -- 172.21.15.1:0/840694823 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f8664010460 con 0x7f86680ffdb0 2023-11-16T16:46:15.178 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.169+0000 7f865e7fc700 1 --2- 172.21.15.1:0/840694823 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f86540a12c0 0x7f86540a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:15.178 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.169+0000 7f865e7fc700 1 -- 172.21.15.1:0/840694823 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8586+0+0 (secure 0 0 0) 0x7f86640cc940 con 0x7f86680ffdb0 2023-11-16T16:46:15.179 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.169+0000 7f866d277700 1 --2- 172.21.15.1:0/840694823 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f86540a12c0 0x7f86540a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:15.179 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.169+0000 7f866d277700 1 --2- 172.21.15.1:0/840694823 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f86540a12c0 0x7f86540a3770 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8658000c00 tx=0x7f8658005fb0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:15.179 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.170+0000 7f867b667b80 1 Processor -- start 2023-11-16T16:46:15.179 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.170+0000 7f867b667b80 1 -- start start 2023-11-16T16:46:15.179 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.170+0000 7f867b667b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b673e6db60 0x55b673e6df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:15.179 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.170+0000 7f867b667b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b673e6e470 0x55b673e72370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:15.179 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.170+0000 7f867b667b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b673e728b0 0x55b673e72d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:15.180 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.170+0000 7f867b667b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b673d73060 con 0x55b673e6db60 2023-11-16T16:46:15.180 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.170+0000 7f867b667b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b673dc58a0 con 0x55b673e728b0 2023-11-16T16:46:15.180 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.170+0000 7f867b667b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b673dbed30 con 0x55b673e6e470 2023-11-16T16:46:15.180 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.170+0000 7f866d277700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b673e6db60 0x55b673e6df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:15.180 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.170+0000 7f866d277700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b673e6db60 0x55b673e6df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45756/0 (socket says 172.21.15.1:45756) 2023-11-16T16:46:15.180 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.170+0000 7f866d277700 1 -- 172.21.15.1:0/3495871646 learned_addr learned my addr 172.21.15.1:0/3495871646 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:15.180 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.170+0000 7f866ca76700 1 --2- 172.21.15.1:0/3495871646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b673e6e470 0x55b673e72370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:15.180 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.170+0000 7f866da78700 1 --2- 172.21.15.1:0/3495871646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b673e728b0 0x55b673e72d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:15.181 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.170+0000 7f866d277700 1 -- 172.21.15.1:0/3495871646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b673e6e470 msgr2=0x55b673e72370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:15.181 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.170+0000 7f866d277700 1 --2- 172.21.15.1:0/3495871646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b673e6e470 0x55b673e72370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.181 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.170+0000 7f866d277700 1 -- 172.21.15.1:0/3495871646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b673e728b0 msgr2=0x55b673e72d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:15.181 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.170+0000 7f866d277700 1 --2- 172.21.15.1:0/3495871646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b673e728b0 0x55b673e72d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.181 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.170+0000 7f866d277700 1 -- 172.21.15.1:0/3495871646 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b673c40630 con 0x55b673e6db60 2023-11-16T16:46:15.181 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.171+0000 7f866d277700 1 --2- 172.21.15.1:0/3495871646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b673e6db60 0x55b673e6df30 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f86580099c0 tx=0x7f865801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a97428f76e4274 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:15.181 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.171+0000 7f86537fe700 1 -- 172.21.15.1:0/3495871646 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865802b070 con 0x55b673e6db60 2023-11-16T16:46:15.181 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.171+0000 7f86537fe700 1 -- 172.21.15.1:0/3495871646 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f865801e040 con 0x55b673e6db60 2023-11-16T16:46:15.182 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.171+0000 7f86537fe700 1 -- 172.21.15.1:0/3495871646 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8658003fb0 con 0x55b673e6db60 2023-11-16T16:46:15.182 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.172+0000 7f867b667b80 1 -- 172.21.15.1:0/3495871646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b673e6db60 msgr2=0x55b673e6df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:15.182 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.172+0000 7f867b667b80 1 --2- 172.21.15.1:0/3495871646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b673e6db60 0x55b673e6df30 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f86580099c0 tx=0x7f865801f000 comp rx=0 tx=0).stop 2023-11-16T16:46:15.182 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.172+0000 7f867b667b80 1 -- 172.21.15.1:0/3495871646 shutdown_connections 2023-11-16T16:46:15.182 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.172+0000 7f867b667b80 1 --2- 172.21.15.1:0/3495871646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b673e728b0 0x55b673e72d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.182 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.172+0000 7f867b667b80 1 --2- 172.21.15.1:0/3495871646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b673e6db60 0x55b673e6df30 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.182 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.172+0000 7f867b667b80 1 --2- 172.21.15.1:0/3495871646 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b673e6e470 0x55b673e72370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.182 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.172+0000 7f867b667b80 1 -- 172.21.15.1:0/3495871646 >> 172.21.15.1:0/3495871646 conn(0x55b673e63360 msgr2=0x55b673e68b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:15.183 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.172+0000 7f867b667b80 1 -- 172.21.15.1:0/3495871646 shutdown_connections 2023-11-16T16:46:15.183 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.172+0000 7f867b667b80 1 -- 172.21.15.1:0/3495871646 wait complete. 2023-11-16T16:46:15.183 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.172+0000 7f867b667b80 1 -- start start 2023-11-16T16:46:15.183 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.172+0000 7f867b667b80 1 Processor -- start 2023-11-16T16:46:15.183 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.172+0000 7f867b667b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b673e6db60 0x55b673e79fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:15.183 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.172+0000 7f867b667b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b673e6e470 0x55b673e771e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:15.183 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.173+0000 7f867b667b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b673e728b0 0x55b673e77780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:15.183 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.173+0000 7f867b667b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b673d608e0 con 0x55b673e6db60 2023-11-16T16:46:15.184 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.173+0000 7f867b667b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b673d60580 con 0x55b673e6e470 2023-11-16T16:46:15.184 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.173+0000 7f867b667b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b673c42920 con 0x55b673e728b0 2023-11-16T16:46:15.184 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.173+0000 7f867b667b80 10 client.0 _ll_register_callbacks cb 0x55b673d7e780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:46:15.184 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.173+0000 7f866d277700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b673e6db60 0x55b673e79fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:15.184 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.173+0000 7f866d277700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b673e6db60 0x55b673e79fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45762/0 (socket says 172.21.15.1:45762) 2023-11-16T16:46:15.184 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.173+0000 7f866d277700 1 -- 172.21.15.1:0/202360194 learned_addr learned my addr 172.21.15.1:0/202360194 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:15.184 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.173+0000 7f866ca76700 1 --2- 172.21.15.1:0/202360194 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b673e6e470 0x55b673e771e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:15.184 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.173+0000 7f866d277700 1 -- 172.21.15.1:0/202360194 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b673e728b0 msgr2=0x55b673e77780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:15.185 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.173+0000 7f866d277700 1 --2- 172.21.15.1:0/202360194 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b673e728b0 0x55b673e77780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.185 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.173+0000 7f866d277700 1 -- 172.21.15.1:0/202360194 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b673e6e470 msgr2=0x55b673e771e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:15.185 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.173+0000 7f866d277700 1 --2- 172.21.15.1:0/202360194 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b673e6e470 0x55b673e771e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.185 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.173+0000 7f866d277700 1 -- 172.21.15.1:0/202360194 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f866810c860 con 0x55b673e6db60 2023-11-16T16:46:15.185 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.173+0000 7f866ca76700 1 --2- 172.21.15.1:0/202360194 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b673e6e470 0x55b673e771e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:46:15.185 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.173+0000 7f866d277700 1 --2- 172.21.15.1:0/202360194 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b673e6db60 0x55b673e79fd0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f8658031040 tx=0x7f865800ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=233f793a2b626038 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:15.185 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.173+0000 7f8650ff9700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:46:15.185 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.173+0000 7f8650ff9700 1 -- 172.21.15.1:0/202360194 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865802b070 con 0x55b673e6db60 2023-11-16T16:46:15.186 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.173+0000 7f8650ff9700 1 -- 172.21.15.1:0/202360194 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8586+0+0 (secure 0 0 0) 0x7f8658004500 con 0x55b673e6db60 2023-11-16T16:46:15.186 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.173+0000 7f867b667b80 1 -- 172.21.15.1:0/202360194 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b673e60b20 con 0x55b673e6db60 2023-11-16T16:46:15.186 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.173+0000 7f8650ff9700 1 -- 172.21.15.1:0/202360194 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8658004230 con 0x55b673e6db60 2023-11-16T16:46:15.186 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.173+0000 7f867b667b80 10 client.16380 fetch_fsmap learned FSMap version 28 2023-11-16T16:46:15.186 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.173+0000 7f867b667b80 1 -- 172.21.15.1:0/202360194 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b673d85190 con 0x55b673e6db60 2023-11-16T16:46:15.186 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.174+0000 7f8650ff9700 1 -- 172.21.15.1:0/202360194 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8658044080 con 0x55b673e6db60 2023-11-16T16:46:15.186 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.174+0000 7f8650ff9700 1 -- 172.21.15.1:0/202360194 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f865800f560 con 0x55b673e6db60 2023-11-16T16:46:15.186 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.174+0000 7f867b667b80 10 client.16380 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:46:15.187 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.174+0000 7f867b667b80 10 client.16380 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:46:15.187 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.174+0000 7f867b667b80 20 client.16380 populate_metadata read hostname 'smithi001' 2023-11-16T16:46:15.187 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.174+0000 7f867b667b80 1 --2- 172.21.15.1:0/202360194 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55b673e872a0 0x55b673e896b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:15.187 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.174+0000 7f867b667b80 4 client.16380 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:46:15.187 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.174+0000 7f867b667b80 1 -- 172.21.15.1:0/202360194 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b673d810a0 con 0x55b673e872a0 2023-11-16T16:46:15.187 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.174+0000 7f866ca76700 1 --2- 172.21.15.1:0/202360194 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55b673e872a0 0x55b673e896b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:15.187 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.175+0000 7f8650ff9700 10 client.16380 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:46:15.187 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.175+0000 7f866ca76700 1 --2- 172.21.15.1:0/202360194 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55b673e872a0 0x55b673e896b0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf5dce7051c40f81 server_cookie=ed4fc610eb6db979 in_seq=0 out_seq=0 2023-11-16T16:46:15.188 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.175+0000 7f8650ff9700 1 -- 172.21.15.1:0/202360194 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b673d810a0 con 0x55b673e872a0 2023-11-16T16:46:15.188 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.175+0000 7f8650ff9700 10 client.16380 handle_command_reply: tid=0 2023-11-16T16:46:15.188 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.175+0000 7f867b667b80 1 client.16380 shutdown 2023-11-16T16:46:15.188 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.176+0000 7f867b667b80 1 -- 172.21.15.1:0/202360194 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b673e6db60 msgr2=0x55b673e79fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:15.188 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.176+0000 7f867b667b80 1 --2- 172.21.15.1:0/202360194 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b673e6db60 0x55b673e79fd0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f8658031040 tx=0x7f865800ba40 comp rx=0 tx=0).stop 2023-11-16T16:46:15.188 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.176+0000 7f867b667b80 1 -- 172.21.15.1:0/202360194 shutdown_connections 2023-11-16T16:46:15.188 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.176+0000 7f867b667b80 1 --2- 172.21.15.1:0/202360194 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55b673e872a0 0x55b673e896b0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.188 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.176+0000 7f867b667b80 1 --2- 172.21.15.1:0/202360194 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b673e728b0 0x55b673e77780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.189 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.176+0000 7f867b667b80 1 --2- 172.21.15.1:0/202360194 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b673e6db60 0x55b673e79fd0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.189 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.176+0000 7f8650ff9700 0 client.16380 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:46:15.189 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.176+0000 7f867b667b80 1 --2- 172.21.15.1:0/202360194 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b673e6e470 0x55b673e771e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.189 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.176+0000 7f867b667b80 1 -- 172.21.15.1:0/202360194 >> 172.21.15.1:0/202360194 conn(0x55b673e63360 msgr2=0x55b673e73d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:15.189 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.176+0000 7f867b667b80 1 -- 172.21.15.1:0/202360194 shutdown_connections 2023-11-16T16:46:15.189 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.176+0000 7f867b667b80 1 -- 172.21.15.1:0/202360194 wait complete. 2023-11-16T16:46:15.189 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.176+0000 7f867b667b80 20 client.16380 trim_cache size 0 max 16384 2023-11-16T16:46:15.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.189+0000 7f867b667b80 1 Processor -- start 2023-11-16T16:46:15.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.189+0000 7f867b667b80 1 -- start start 2023-11-16T16:46:15.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.189+0000 7f867b667b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b673e6db60 0x55b673e7e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:15.193 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.189+0000 7f867b667b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b673e6e470 0x55b673e79b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:15.193 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.189+0000 7f867b667b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b673e728b0 0x55b673e68650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:15.193 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.189+0000 7f867b667b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b673e62ee0 con 0x55b673e728b0 2023-11-16T16:46:15.193 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.189+0000 7f867b667b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b673e7a650 con 0x55b673e6db60 2023-11-16T16:46:15.193 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.189+0000 7f867b667b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b673e75740 con 0x55b673e6e470 2023-11-16T16:46:15.193 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.189+0000 7f866da78700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b673e728b0 0x55b673e68650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:15.193 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.189+0000 7f866da78700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b673e728b0 0x55b673e68650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45774/0 (socket says 172.21.15.1:45774) 2023-11-16T16:46:15.193 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.189+0000 7f866da78700 1 -- 172.21.15.1:0/1785018706 learned_addr learned my addr 172.21.15.1:0/1785018706 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:15.194 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.189+0000 7f866ca76700 1 --2- 172.21.15.1:0/1785018706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b673e6e470 0x55b673e79b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:15.194 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.189+0000 7f866d277700 1 --2- 172.21.15.1:0/1785018706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b673e6db60 0x55b673e7e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:15.194 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.189+0000 7f866da78700 1 -- 172.21.15.1:0/1785018706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b673e6e470 msgr2=0x55b673e79b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:15.194 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.189+0000 7f866da78700 1 --2- 172.21.15.1:0/1785018706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b673e6e470 0x55b673e79b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.194 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.189+0000 7f866da78700 1 -- 172.21.15.1:0/1785018706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b673e6db60 msgr2=0x55b673e7e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:15.194 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.189+0000 7f866da78700 1 --2- 172.21.15.1:0/1785018706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b673e6db60 0x55b673e7e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.194 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.189+0000 7f866da78700 1 -- 172.21.15.1:0/1785018706 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b673d84ee0 con 0x55b673e728b0 2023-11-16T16:46:15.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.190+0000 7f866da78700 1 --2- 172.21.15.1:0/1785018706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b673e728b0 0x55b673e68650 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f8660011250 tx=0x7f8660011560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebfa0fc12c5c14fa server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:15.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.190+0000 7f8653fff700 1 -- 172.21.15.1:0/1785018706 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8660011d70 con 0x55b673e728b0 2023-11-16T16:46:15.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.190+0000 7f8653fff700 1 -- 172.21.15.1:0/1785018706 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8660011ed0 con 0x55b673e728b0 2023-11-16T16:46:15.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.190+0000 7f8653fff700 1 -- 172.21.15.1:0/1785018706 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8660005280 con 0x55b673e728b0 2023-11-16T16:46:15.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.190+0000 7f867b667b80 1 -- 172.21.15.1:0/1785018706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b673e728b0 msgr2=0x55b673e68650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:15.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.190+0000 7f867b667b80 1 --2- 172.21.15.1:0/1785018706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b673e728b0 0x55b673e68650 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f8660011250 tx=0x7f8660011560 comp rx=0 tx=0).stop 2023-11-16T16:46:15.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.190+0000 7f867b667b80 1 -- 172.21.15.1:0/1785018706 shutdown_connections 2023-11-16T16:46:15.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.190+0000 7f867b667b80 1 --2- 172.21.15.1:0/1785018706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b673e728b0 0x55b673e68650 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.190+0000 7f867b667b80 1 --2- 172.21.15.1:0/1785018706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b673e6db60 0x55b673e7e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.190+0000 7f867b667b80 1 --2- 172.21.15.1:0/1785018706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b673e6e470 0x55b673e79b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.190+0000 7f867b667b80 1 -- 172.21.15.1:0/1785018706 >> 172.21.15.1:0/1785018706 conn(0x55b673e63360 msgr2=0x55b673e83af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:15.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.190+0000 7f867b667b80 1 -- 172.21.15.1:0/1785018706 shutdown_connections 2023-11-16T16:46:15.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.190+0000 7f867b667b80 1 -- 172.21.15.1:0/1785018706 wait complete. 2023-11-16T16:46:15.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.191+0000 7f867b667b80 1 -- start start 2023-11-16T16:46:15.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.191+0000 7f867b667b80 1 Processor -- start 2023-11-16T16:46:15.199 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.191+0000 7f867b667b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b673e6db60 0x55b673e7caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:15.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.191+0000 7f867b667b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b673e6e470 0x55b673e7d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:15.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.191+0000 7f867b667b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b673e728b0 0x55b673ed8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:15.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.191+0000 7f867b667b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b673ed8a80 con 0x55b673e728b0 2023-11-16T16:46:15.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.191+0000 7f867b667b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b673ed8bf0 con 0x55b673e6db60 2023-11-16T16:46:15.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.191+0000 7f867b667b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b673ed8db0 con 0x55b673e6e470 2023-11-16T16:46:15.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.191+0000 7f867b667b80 10 client.0 _ll_register_callbacks cb 0x7f8660007970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:46:15.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.191+0000 7f866da78700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b673e728b0 0x55b673ed8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:15.201 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.191+0000 7f866da78700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b673e728b0 0x55b673ed8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45780/0 (socket says 172.21.15.1:45780) 2023-11-16T16:46:15.201 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.191+0000 7f866da78700 1 -- 172.21.15.1:0/2586591069 learned_addr learned my addr 172.21.15.1:0/2586591069 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:15.201 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.191+0000 7f866ca76700 1 --2- 172.21.15.1:0/2586591069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b673e6e470 0x55b673e7d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:15.201 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.191+0000 7f866d277700 1 --2- 172.21.15.1:0/2586591069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b673e6db60 0x55b673e7caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:15.201 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.192+0000 7f866da78700 1 -- 172.21.15.1:0/2586591069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b673e6e470 msgr2=0x55b673e7d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:15.201 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.192+0000 7f866da78700 1 --2- 172.21.15.1:0/2586591069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b673e6e470 0x55b673e7d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.201 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.192+0000 7f866da78700 1 -- 172.21.15.1:0/2586591069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b673e6db60 msgr2=0x55b673e7caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:15.202 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.192+0000 7f866da78700 1 --2- 172.21.15.1:0/2586591069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b673e6db60 0x55b673e7caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.202 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.192+0000 7f866da78700 1 -- 172.21.15.1:0/2586591069 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f866000f040 con 0x55b673e728b0 2023-11-16T16:46:15.202 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.192+0000 7f866da78700 1 --2- 172.21.15.1:0/2586591069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b673e728b0 0x55b673ed8540 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f8660004d00 tx=0x7f8660004de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=551db2944cf8fa55 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:15.202 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.192+0000 7f8650ff9700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:46:15.202 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.192+0000 7f8650ff9700 1 -- 172.21.15.1:0/2586591069 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8660015640 con 0x55b673e728b0 2023-11-16T16:46:15.202 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.192+0000 7f8650ff9700 1 -- 172.21.15.1:0/2586591069 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8586+0+0 (secure 0 0 0) 0x7f86600157a0 con 0x55b673e728b0 2023-11-16T16:46:15.202 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.192+0000 7f867b667b80 1 -- 172.21.15.1:0/2586591069 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f865801e040 con 0x55b673e728b0 2023-11-16T16:46:15.202 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.192+0000 7f8650ff9700 1 -- 172.21.15.1:0/2586591069 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8660003a00 con 0x55b673e728b0 2023-11-16T16:46:15.203 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.192+0000 7f867b667b80 10 client.16386 fetch_fsmap learned FSMap version 28 2023-11-16T16:46:15.203 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.192+0000 7f867b667b80 1 -- 172.21.15.1:0/2586591069 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b673bd5c60 con 0x55b673e728b0 2023-11-16T16:46:15.203 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.192+0000 7f8650ff9700 1 -- 172.21.15.1:0/2586591069 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f865801e040 con 0x55b673e728b0 2023-11-16T16:46:15.203 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.192+0000 7f8650ff9700 1 -- 172.21.15.1:0/2586591069 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f8660025430 con 0x55b673e728b0 2023-11-16T16:46:15.203 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.192+0000 7f867b667b80 10 client.16386 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:46:15.203 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.192+0000 7f867b667b80 10 client.16386 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:46:15.203 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.192+0000 7f867b667b80 20 client.16386 populate_metadata read hostname 'smithi001' 2023-11-16T16:46:15.203 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.192+0000 7f867b667b80 1 --2- 172.21.15.1:0/2586591069 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55b673ee0600 0x55b673ee2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:15.204 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.192+0000 7f867b667b80 4 client.16386 mds_command: new command op to 24521 tid=0[{"prefix": "damage ls"}] 2023-11-16T16:46:15.204 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.192+0000 7f867b667b80 1 -- 172.21.15.1:0/2586591069 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b673d8d660 con 0x55b673ee0600 2023-11-16T16:46:15.204 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.193+0000 7f866d277700 1 --2- 172.21.15.1:0/2586591069 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55b673ee0600 0x55b673ee2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:15.204 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.193+0000 7f8650ff9700 10 client.16386 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:46:15.204 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.193+0000 7f866d277700 1 --2- 172.21.15.1:0/2586591069 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55b673ee0600 0x55b673ee2a10 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa109642a5b2325b server_cookie=7f1be42383d02909 in_seq=0 out_seq=0 2023-11-16T16:46:15.204 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.193+0000 7f8650ff9700 1 -- 172.21.15.1:0/2586591069 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b673d8d660 con 0x55b673ee0600 2023-11-16T16:46:15.204 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.193+0000 7f8650ff9700 10 client.16386 handle_command_reply: tid=0 2023-11-16T16:46:15.204 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.193+0000 7f867b667b80 1 client.16386 shutdown 2023-11-16T16:46:15.205 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.194+0000 7f867b667b80 1 -- 172.21.15.1:0/2586591069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b673e728b0 msgr2=0x55b673ed8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:15.205 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.194+0000 7f867b667b80 1 --2- 172.21.15.1:0/2586591069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b673e728b0 0x55b673ed8540 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f8660004d00 tx=0x7f8660004de0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.205 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.194+0000 7f867b667b80 1 -- 172.21.15.1:0/2586591069 shutdown_connections 2023-11-16T16:46:15.205 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.194+0000 7f867b667b80 1 --2- 172.21.15.1:0/2586591069 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55b673ee0600 0x55b673ee2a10 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.205 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.194+0000 7f867b667b80 1 --2- 172.21.15.1:0/2586591069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b673e728b0 0x55b673ed8540 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.205 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.194+0000 7f867b667b80 1 --2- 172.21.15.1:0/2586591069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b673e6db60 0x55b673e7caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.205 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.194+0000 7f8650ff9700 0 client.16386 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:46:15.205 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.194+0000 7f867b667b80 1 --2- 172.21.15.1:0/2586591069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b673e6e470 0x55b673e7d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.206 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.194+0000 7f867b667b80 1 -- 172.21.15.1:0/2586591069 >> 172.21.15.1:0/2586591069 conn(0x55b673e63360 msgr2=0x55b673e68650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:15.206 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.194+0000 7f867b667b80 1 -- 172.21.15.1:0/2586591069 shutdown_connections 2023-11-16T16:46:15.206 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.194+0000 7f867b667b80 1 -- 172.21.15.1:0/2586591069 wait complete. 2023-11-16T16:46:15.206 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.194+0000 7f867b667b80 20 client.16386 trim_cache size 0 max 16384 2023-11-16T16:46:15.206 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2023-11-16T16:46:15.207 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.194+0000 7f866f4db700 1 -- 172.21.15.1:0/840694823 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f86540a12c0 msgr2=0x7f86540a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:15.207 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.194+0000 7f866f4db700 1 --2- 172.21.15.1:0/840694823 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f86540a12c0 0x7f86540a3770 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8658000c00 tx=0x7f8658005fb0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.207 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.194+0000 7f866f4db700 1 -- 172.21.15.1:0/840694823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86680ffdb0 msgr2=0x7f8668197680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:15.208 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.194+0000 7f866f4db700 1 --2- 172.21.15.1:0/840694823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86680ffdb0 0x7f8668197680 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f866400d950 tx=0x7f866400dd10 comp rx=0 tx=0).stop 2023-11-16T16:46:15.208 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.194+0000 7f866f4db700 1 -- 172.21.15.1:0/840694823 shutdown_connections 2023-11-16T16:46:15.208 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.194+0000 7f866f4db700 1 --2- 172.21.15.1:0/840694823 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f86540a12c0 0x7f86540a3770 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.208 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.194+0000 7f866f4db700 1 --2- 172.21.15.1:0/840694823 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8668104450 0x7f86681a21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.208 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.194+0000 7f866f4db700 1 --2- 172.21.15.1:0/840694823 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86680ff410 0x7f8668197140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.208 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.194+0000 7f866f4db700 1 --2- 172.21.15.1:0/840694823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86680ffdb0 0x7f8668197680 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:15.208 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.194+0000 7f866f4db700 1 -- 172.21.15.1:0/840694823 >> 172.21.15.1:0/840694823 conn(0x7f8668076370 msgr2=0x7f86680fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:15.208 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.194+0000 7f866f4db700 1 -- 172.21.15.1:0/840694823 shutdown_connections 2023-11-16T16:46:15.209 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:15.194+0000 7f866f4db700 1 -- 172.21.15.1:0/840694823 wait complete. 2023-11-16T16:46:15.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:15 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/4081825700' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:16.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:15 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/4081825700' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:16.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:15 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/4081825700' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:16.210 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-16T16:46:16.211 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-16T16:46:16.360 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.356+0000 7f28bf00b700 1 -- 172.21.15.1:0/342995522 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28b8101dc0 msgr2=0x7f28b810a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:16.360 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.356+0000 7f28bf00b700 1 --2- 172.21.15.1:0/342995522 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28b8101dc0 0x7f28b810a2b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f28a4009990 tx=0x7f28a4009ca0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.361 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.356+0000 7f28bf00b700 1 -- 172.21.15.1:0/342995522 shutdown_connections 2023-11-16T16:46:16.361 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.356+0000 7f28bf00b700 1 --2- 172.21.15.1:0/342995522 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28b810a7f0 0x7f28b810cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.361 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.356+0000 7f28bf00b700 1 --2- 172.21.15.1:0/342995522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28b81014b0 0x7f28b8101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.361 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.356+0000 7f28bf00b700 1 --2- 172.21.15.1:0/342995522 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28b8101dc0 0x7f28b810a2b0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.361 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.356+0000 7f28bf00b700 1 -- 172.21.15.1:0/342995522 >> 172.21.15.1:0/342995522 conn(0x7f28b8076340 msgr2=0x7f28b8076740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:16.361 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.356+0000 7f28bf00b700 1 -- 172.21.15.1:0/342995522 shutdown_connections 2023-11-16T16:46:16.362 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.356+0000 7f28bf00b700 1 -- 172.21.15.1:0/342995522 wait complete. 2023-11-16T16:46:16.362 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.356+0000 7f28bf00b700 1 Processor -- start 2023-11-16T16:46:16.362 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.356+0000 7f28bf00b700 1 -- start start 2023-11-16T16:46:16.362 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.357+0000 7f28bf00b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28b81014b0 0x7f28b81962f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.362 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.357+0000 7f28bf00b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28b8101dc0 0x7f28b8196830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.362 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.357+0000 7f28bf00b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28b810a7f0 0x7f28b819ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.363 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.357+0000 7f28bf00b700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b819b2d0 con 0x7f28b810a7f0 2023-11-16T16:46:16.363 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.357+0000 7f28bf00b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b819b440 con 0x7f28b8101dc0 2023-11-16T16:46:16.363 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.357+0000 7f28bf00b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b819b600 con 0x7f28b81014b0 2023-11-16T16:46:16.363 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.357+0000 7f28bd5a8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28b810a7f0 0x7f28b819ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.363 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.357+0000 7f28bd5a8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28b810a7f0 0x7f28b819ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45794/0 (socket says 172.21.15.1:45794) 2023-11-16T16:46:16.363 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.357+0000 7f28bd5a8700 1 -- 172.21.15.1:0/4022702281 learned_addr learned my addr 172.21.15.1:0/4022702281 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:16.364 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.357+0000 7f28b7fff700 1 --2- 172.21.15.1:0/4022702281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28b8101dc0 0x7f28b8196830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.364 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.357+0000 7f28bcda7700 1 --2- 172.21.15.1:0/4022702281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28b81014b0 0x7f28b81962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.364 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.357+0000 7f28bd5a8700 1 -- 172.21.15.1:0/4022702281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28b81014b0 msgr2=0x7f28b81962f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:16.364 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.357+0000 7f28bd5a8700 1 --2- 172.21.15.1:0/4022702281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28b81014b0 0x7f28b81962f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.364 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.357+0000 7f28bd5a8700 1 -- 172.21.15.1:0/4022702281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28b8101dc0 msgr2=0x7f28b8196830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:16.364 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.357+0000 7f28bd5a8700 1 --2- 172.21.15.1:0/4022702281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28b8101dc0 0x7f28b8196830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.364 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.357+0000 7f28bd5a8700 1 -- 172.21.15.1:0/4022702281 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a40095f0 con 0x7f28b810a7f0 2023-11-16T16:46:16.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.357+0000 7f28b7fff700 1 --2- 172.21.15.1:0/4022702281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28b8101dc0 0x7f28b8196830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:46:16.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.357+0000 7f28bd5a8700 1 --2- 172.21.15.1:0/4022702281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28b810a7f0 0x7f28b819ac90 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f28b0007ee0 tx=0x7f28b000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:16.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.357+0000 7f28b5ffb700 1 -- 172.21.15.1:0/4022702281 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b0017070 con 0x7f28b810a7f0 2023-11-16T16:46:16.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.358+0000 7f28b5ffb700 1 -- 172.21.15.1:0/4022702281 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28b000f040 con 0x7f28b810a7f0 2023-11-16T16:46:16.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.358+0000 7f28bf00b700 1 -- 172.21.15.1:0/4022702281 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28b81102c0 con 0x7f28b810a7f0 2023-11-16T16:46:16.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.358+0000 7f28b5ffb700 1 -- 172.21.15.1:0/4022702281 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b0005740 con 0x7f28b810a7f0 2023-11-16T16:46:16.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.358+0000 7f28bf00b700 1 -- 172.21.15.1:0/4022702281 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28b8110130 con 0x7f28b810a7f0 2023-11-16T16:46:16.373 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.360+0000 7f28b5ffb700 1 -- 172.21.15.1:0/4022702281 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f28b00058a0 con 0x7f28b810a7f0 2023-11-16T16:46:16.374 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.361+0000 7f28b5ffb700 1 --2- 172.21.15.1:0/4022702281 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f28a80a12c0 0x7f28a80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.374 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.361+0000 7f28bcda7700 1 --2- 172.21.15.1:0/4022702281 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f28a80a12c0 0x7f28a80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.374 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.361+0000 7f28b5ffb700 1 -- 172.21.15.1:0/4022702281 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f28b00cc570 con 0x7f28b810a7f0 2023-11-16T16:46:16.374 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.361+0000 7f28bcda7700 1 --2- 172.21.15.1:0/4022702281 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f28a80a12c0 0x7f28a80a3770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f28ac005fd0 tx=0x7f28ac005f60 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:16.374 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.362+0000 7f28cb197b80 1 Processor -- start 2023-11-16T16:46:16.374 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.362+0000 7f28cb197b80 1 -- start start 2023-11-16T16:46:16.375 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.362+0000 7f28cb197b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580e32acbe0 0x5580e32acfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.375 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.362+0000 7f28cb197b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580e32ad4f0 0x5580e32b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.375 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.362+0000 7f28cb197b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580e32b1930 0x5580e32b1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.375 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.362+0000 7f28cb197b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e31b20e0 con 0x5580e32ad4f0 2023-11-16T16:46:16.375 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.362+0000 7f28cb197b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e3204920 con 0x5580e32b1930 2023-11-16T16:46:16.375 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.362+0000 7f28cb197b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e31fddb0 con 0x5580e32acbe0 2023-11-16T16:46:16.375 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.362+0000 7f28b7fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580e32ad4f0 0x5580e32b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.376 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.362+0000 7f28b7fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580e32ad4f0 0x5580e32b13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45800/0 (socket says 172.21.15.1:45800) 2023-11-16T16:46:16.376 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.362+0000 7f28b7fff700 1 -- 172.21.15.1:0/1109956430 learned_addr learned my addr 172.21.15.1:0/1109956430 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:16.376 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.362+0000 7f28bd5a8700 1 --2- 172.21.15.1:0/1109956430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580e32b1930 0x5580e32b1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.376 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.362+0000 7f28bcda7700 1 --2- 172.21.15.1:0/1109956430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580e32acbe0 0x5580e32acfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.376 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.362+0000 7f28b7fff700 1 -- 172.21.15.1:0/1109956430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580e32acbe0 msgr2=0x5580e32acfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:16.376 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.362+0000 7f28b7fff700 1 --2- 172.21.15.1:0/1109956430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580e32acbe0 0x5580e32acfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.377 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.363+0000 7f28b7fff700 1 -- 172.21.15.1:0/1109956430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580e32b1930 msgr2=0x5580e32b1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:16.377 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.363+0000 7f28b7fff700 1 --2- 172.21.15.1:0/1109956430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580e32b1930 0x5580e32b1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.377 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.363+0000 7f28b7fff700 1 -- 172.21.15.1:0/1109956430 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580e307f6b0 con 0x5580e32ad4f0 2023-11-16T16:46:16.377 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.363+0000 7f28bd5a8700 1 --2- 172.21.15.1:0/1109956430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580e32b1930 0x5580e32b1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:46:16.377 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.363+0000 7f28b7fff700 1 --2- 172.21.15.1:0/1109956430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580e32ad4f0 0x5580e32b13f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f28a4006010 tx=0x7f28a4005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36bbe041f6115485 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:16.377 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.363+0000 7f28a2ffd700 1 -- 172.21.15.1:0/1109956430 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a4009ee0 con 0x5580e32ad4f0 2023-11-16T16:46:16.377 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.363+0000 7f28a2ffd700 1 -- 172.21.15.1:0/1109956430 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28a401e040 con 0x5580e32ad4f0 2023-11-16T16:46:16.378 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.363+0000 7f28a2ffd700 1 -- 172.21.15.1:0/1109956430 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a40054a0 con 0x5580e32ad4f0 2023-11-16T16:46:16.378 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.364+0000 7f28cb197b80 1 -- 172.21.15.1:0/1109956430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580e32ad4f0 msgr2=0x5580e32b13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:16.378 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.364+0000 7f28cb197b80 1 --2- 172.21.15.1:0/1109956430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580e32ad4f0 0x5580e32b13f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f28a4006010 tx=0x7f28a4005e80 comp rx=0 tx=0).stop 2023-11-16T16:46:16.378 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.364+0000 7f28cb197b80 1 -- 172.21.15.1:0/1109956430 shutdown_connections 2023-11-16T16:46:16.378 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.364+0000 7f28cb197b80 1 --2- 172.21.15.1:0/1109956430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580e32b1930 0x5580e32b1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.378 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.364+0000 7f28cb197b80 1 --2- 172.21.15.1:0/1109956430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580e32acbe0 0x5580e32acfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.378 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.364+0000 7f28cb197b80 1 --2- 172.21.15.1:0/1109956430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580e32ad4f0 0x5580e32b13f0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.379 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.364+0000 7f28cb197b80 1 -- 172.21.15.1:0/1109956430 >> 172.21.15.1:0/1109956430 conn(0x5580e32a23e0 msgr2=0x5580e32a7bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:16.379 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.364+0000 7f28cb197b80 1 -- 172.21.15.1:0/1109956430 shutdown_connections 2023-11-16T16:46:16.379 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.364+0000 7f28cb197b80 1 -- 172.21.15.1:0/1109956430 wait complete. 2023-11-16T16:46:16.379 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.364+0000 7f28cb197b80 1 -- start start 2023-11-16T16:46:16.379 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.364+0000 7f28cb197b80 1 Processor -- start 2023-11-16T16:46:16.380 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.365+0000 7f28cb197b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580e32acbe0 0x5580e32b90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.380 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.365+0000 7f28cb197b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580e32ad4f0 0x5580e32b62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.380 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.365+0000 7f28cb197b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580e32b1930 0x5580e32b68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.380 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.365+0000 7f28cb197b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e319f960 con 0x5580e32b1930 2023-11-16T16:46:16.380 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.365+0000 7f28cb197b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e319f600 con 0x5580e32ad4f0 2023-11-16T16:46:16.380 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.365+0000 7f28cb197b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e30819a0 con 0x5580e32acbe0 2023-11-16T16:46:16.380 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.365+0000 7f28cb197b80 10 client.0 _ll_register_callbacks cb 0x5580e31bd800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:46:16.381 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.365+0000 7f28bd5a8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580e32b1930 0x5580e32b68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.381 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.365+0000 7f28bd5a8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580e32b1930 0x5580e32b68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45808/0 (socket says 172.21.15.1:45808) 2023-11-16T16:46:16.381 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.365+0000 7f28bd5a8700 1 -- 172.21.15.1:0/3009458249 learned_addr learned my addr 172.21.15.1:0/3009458249 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:16.381 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.365+0000 7f28bcda7700 1 --2- 172.21.15.1:0/3009458249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580e32acbe0 0x5580e32b90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.381 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.365+0000 7f28b7fff700 1 --2- 172.21.15.1:0/3009458249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580e32ad4f0 0x5580e32b62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.381 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.365+0000 7f28bd5a8700 1 -- 172.21.15.1:0/3009458249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580e32acbe0 msgr2=0x5580e32b90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:16.381 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.365+0000 7f28bd5a8700 1 --2- 172.21.15.1:0/3009458249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580e32acbe0 0x5580e32b90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.365+0000 7f28bd5a8700 1 -- 172.21.15.1:0/3009458249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580e32ad4f0 msgr2=0x5580e32b62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:16.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.365+0000 7f28bd5a8700 1 --2- 172.21.15.1:0/3009458249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580e32ad4f0 0x5580e32b62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.365+0000 7f28bd5a8700 1 -- 172.21.15.1:0/3009458249 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28b810d940 con 0x5580e32b1930 2023-11-16T16:46:16.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.365+0000 7f28b7fff700 1 --2- 172.21.15.1:0/3009458249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580e32ad4f0 0x5580e32b62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:46:16.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.365+0000 7f28bd5a8700 1 --2- 172.21.15.1:0/3009458249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580e32b1930 0x5580e32b68b0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f28b0099310 tx=0x7f28b0099060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c89e49d0fd96f6d server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:16.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.365+0000 7f2897fff700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:46:16.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.365+0000 7f2897fff700 1 -- 172.21.15.1:0/3009458249 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b000ddd0 con 0x5580e32b1930 2023-11-16T16:46:16.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.365+0000 7f2897fff700 1 -- 172.21.15.1:0/3009458249 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f28b00a1a60 con 0x5580e32b1930 2023-11-16T16:46:16.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.366+0000 7f28cb197b80 1 -- 172.21.15.1:0/3009458249 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5580e329fba0 con 0x5580e32b1930 2023-11-16T16:46:16.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.366+0000 7f2897fff700 1 -- 172.21.15.1:0/3009458249 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28b00840b0 con 0x5580e32b1930 2023-11-16T16:46:16.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.366+0000 7f28cb197b80 10 client.16398 fetch_fsmap learned FSMap version 28 2023-11-16T16:46:16.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.366+0000 7f28cb197b80 1 -- 172.21.15.1:0/3009458249 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5580e31c4210 con 0x5580e32b1930 2023-11-16T16:46:16.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.366+0000 7f2897fff700 1 -- 172.21.15.1:0/3009458249 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28b00c7020 con 0x5580e32b1930 2023-11-16T16:46:16.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.366+0000 7f2897fff700 1 -- 172.21.15.1:0/3009458249 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f28b0098b00 con 0x5580e32b1930 2023-11-16T16:46:16.384 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.366+0000 7f28cb197b80 10 client.16398 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:46:16.384 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.366+0000 7f28cb197b80 10 client.16398 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:46:16.384 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.366+0000 7f28cb197b80 20 client.16398 populate_metadata read hostname 'smithi001' 2023-11-16T16:46:16.384 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.366+0000 7f28cb197b80 1 --2- 172.21.15.1:0/3009458249 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5580e32c6940 0x5580e32c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.384 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.366+0000 7f28cb197b80 4 client.16398 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:46:16.384 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.366+0000 7f28cb197b80 1 -- 172.21.15.1:0/3009458249 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5580e31c0120 con 0x5580e32c6940 2023-11-16T16:46:16.385 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.366+0000 7f28bcda7700 1 --2- 172.21.15.1:0/3009458249 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5580e32c6940 0x5580e32c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.385 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.367+0000 7f28bcda7700 1 --2- 172.21.15.1:0/3009458249 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5580e32c6940 0x5580e32c8d50 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc663beec63b6e87 server_cookie=e37f133e49980569 in_seq=0 out_seq=0 2023-11-16T16:46:16.385 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.367+0000 7f2897fff700 10 client.16398 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:46:16.385 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.368+0000 7f2897fff700 1 -- 172.21.15.1:0/3009458249 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5580e31c0120 con 0x5580e32c6940 2023-11-16T16:46:16.385 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.368+0000 7f2897fff700 10 client.16398 handle_command_reply: tid=0 2023-11-16T16:46:16.385 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.368+0000 7f28cb197b80 1 client.16398 shutdown 2023-11-16T16:46:16.385 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.368+0000 7f28cb197b80 1 -- 172.21.15.1:0/3009458249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580e32b1930 msgr2=0x5580e32b68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:16.386 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.368+0000 7f28cb197b80 1 --2- 172.21.15.1:0/3009458249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580e32b1930 0x5580e32b68b0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f28b0099310 tx=0x7f28b0099060 comp rx=0 tx=0).stop 2023-11-16T16:46:16.386 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.368+0000 7f28cb197b80 1 -- 172.21.15.1:0/3009458249 shutdown_connections 2023-11-16T16:46:16.386 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.368+0000 7f28cb197b80 1 --2- 172.21.15.1:0/3009458249 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5580e32c6940 0x5580e32c8d50 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.386 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.368+0000 7f28cb197b80 1 --2- 172.21.15.1:0/3009458249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580e32b1930 0x5580e32b68b0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.386 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.368+0000 7f28cb197b80 1 --2- 172.21.15.1:0/3009458249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580e32acbe0 0x5580e32b90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.386 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.368+0000 7f2897fff700 0 client.16398 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:46:16.387 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.368+0000 7f28cb197b80 1 --2- 172.21.15.1:0/3009458249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580e32ad4f0 0x5580e32b62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.387 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.368+0000 7f28cb197b80 1 -- 172.21.15.1:0/3009458249 >> 172.21.15.1:0/3009458249 conn(0x5580e32a23e0 msgr2=0x5580e32b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:16.387 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.368+0000 7f28cb197b80 1 -- 172.21.15.1:0/3009458249 shutdown_connections 2023-11-16T16:46:16.387 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.368+0000 7f28cb197b80 1 -- 172.21.15.1:0/3009458249 wait complete. 2023-11-16T16:46:16.387 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.368+0000 7f28cb197b80 20 client.16398 trim_cache size 0 max 16384 2023-11-16T16:46:16.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.385+0000 7f28cb197b80 1 Processor -- start 2023-11-16T16:46:16.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.385+0000 7f28cb197b80 1 -- start start 2023-11-16T16:46:16.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.386+0000 7f28cb197b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580e32acbe0 0x5580e32bd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.386+0000 7f28cb197b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580e32ad4f0 0x5580e32af930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.386+0000 7f28cb197b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580e32b1930 0x5580e32afe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.386+0000 7f28cb197b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e32b0450 con 0x5580e32b1930 2023-11-16T16:46:16.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.386+0000 7f28cb197b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e32b74e0 con 0x5580e32acbe0 2023-11-16T16:46:16.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.386+0000 7f28cb197b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e32bdf00 con 0x5580e32ad4f0 2023-11-16T16:46:16.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.386+0000 7f28bd5a8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580e32b1930 0x5580e32afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.386+0000 7f28bd5a8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580e32b1930 0x5580e32afe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45816/0 (socket says 172.21.15.1:45816) 2023-11-16T16:46:16.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.386+0000 7f28bd5a8700 1 -- 172.21.15.1:0/494760161 learned_addr learned my addr 172.21.15.1:0/494760161 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:16.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.386+0000 7f28b7fff700 1 --2- 172.21.15.1:0/494760161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580e32ad4f0 0x5580e32af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.386+0000 7f28bcda7700 1 --2- 172.21.15.1:0/494760161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580e32acbe0 0x5580e32bd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.386+0000 7f28bd5a8700 1 -- 172.21.15.1:0/494760161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580e32ad4f0 msgr2=0x5580e32af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:16.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.386+0000 7f28bd5a8700 1 --2- 172.21.15.1:0/494760161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580e32ad4f0 0x5580e32af930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.386+0000 7f28bd5a8700 1 -- 172.21.15.1:0/494760161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580e32acbe0 msgr2=0x5580e32bd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:16.397 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.386+0000 7f28bd5a8700 1 --2- 172.21.15.1:0/494760161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580e32acbe0 0x5580e32bd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.397 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.386+0000 7f28bd5a8700 1 -- 172.21.15.1:0/494760161 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580e32a3b80 con 0x5580e32b1930 2023-11-16T16:46:16.397 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.386+0000 7f28bd5a8700 1 --2- 172.21.15.1:0/494760161 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580e32b1930 0x5580e32afe70 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f28b0009510 tx=0x7f28b00ca220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7761fcc326acddb7 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:16.397 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.386+0000 7f28a37fe700 1 -- 172.21.15.1:0/494760161 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b00a1a60 con 0x5580e32b1930 2023-11-16T16:46:16.397 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.386+0000 7f28a37fe700 1 -- 172.21.15.1:0/494760161 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28b00c7070 con 0x5580e32b1930 2023-11-16T16:46:16.397 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.386+0000 7f28a37fe700 1 -- 172.21.15.1:0/494760161 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b00840b0 con 0x5580e32b1930 2023-11-16T16:46:16.397 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.387+0000 7f28cb197b80 1 -- 172.21.15.1:0/494760161 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580e32b1930 msgr2=0x5580e32afe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:16.398 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.387+0000 7f28cb197b80 1 --2- 172.21.15.1:0/494760161 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580e32b1930 0x5580e32afe70 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f28b0009510 tx=0x7f28b00ca220 comp rx=0 tx=0).stop 2023-11-16T16:46:16.398 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.387+0000 7f28cb197b80 1 -- 172.21.15.1:0/494760161 shutdown_connections 2023-11-16T16:46:16.398 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.387+0000 7f28cb197b80 1 --2- 172.21.15.1:0/494760161 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580e32b1930 0x5580e32afe70 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.398 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.387+0000 7f28cb197b80 1 --2- 172.21.15.1:0/494760161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580e32acbe0 0x5580e32bd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.398 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.387+0000 7f28cb197b80 1 --2- 172.21.15.1:0/494760161 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580e32ad4f0 0x5580e32af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.398 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.387+0000 7f28cb197b80 1 -- 172.21.15.1:0/494760161 >> 172.21.15.1:0/494760161 conn(0x5580e32a23e0 msgr2=0x5580e32a6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:16.399 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.387+0000 7f28cb197b80 1 -- 172.21.15.1:0/494760161 shutdown_connections 2023-11-16T16:46:16.399 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.387+0000 7f28cb197b80 1 -- 172.21.15.1:0/494760161 wait complete. 2023-11-16T16:46:16.399 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.387+0000 7f28cb197b80 1 -- start start 2023-11-16T16:46:16.399 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.387+0000 7f28cb197b80 1 Processor -- start 2023-11-16T16:46:16.399 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.388+0000 7f28cb197b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580e32acbe0 0x5580e32bd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.399 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.388+0000 7f28cb197b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580e32ad4f0 0x5580e32b9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.399 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.388+0000 7f28cb197b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580e32b1930 0x5580e32ba340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.400 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.388+0000 7f28cb197b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e32baa50 con 0x5580e32ad4f0 2023-11-16T16:46:16.400 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.388+0000 7f28cb197b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e331a800 con 0x5580e32b1930 2023-11-16T16:46:16.400 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.388+0000 7f28cb197b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580e331a940 con 0x5580e32acbe0 2023-11-16T16:46:16.400 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.388+0000 7f28cb197b80 10 client.0 _ll_register_callbacks cb 0x7f28b0099150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:46:16.400 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.388+0000 7f28b7fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580e32ad4f0 0x5580e32b9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.400 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.388+0000 7f28b7fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580e32ad4f0 0x5580e32b9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45820/0 (socket says 172.21.15.1:45820) 2023-11-16T16:46:16.400 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.388+0000 7f28b7fff700 1 -- 172.21.15.1:0/3154784099 learned_addr learned my addr 172.21.15.1:0/3154784099 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:16.401 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.388+0000 7f28bcda7700 1 --2- 172.21.15.1:0/3154784099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580e32acbe0 0x5580e32bd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.401 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.388+0000 7f28bd5a8700 1 --2- 172.21.15.1:0/3154784099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580e32b1930 0x5580e32ba340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.401 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.388+0000 7f28b7fff700 1 -- 172.21.15.1:0/3154784099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580e32acbe0 msgr2=0x5580e32bd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:16.401 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.388+0000 7f28b7fff700 1 --2- 172.21.15.1:0/3154784099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580e32acbe0 0x5580e32bd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.401 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.388+0000 7f28b7fff700 1 -- 172.21.15.1:0/3154784099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580e32b1930 msgr2=0x5580e32ba340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:16.401 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.388+0000 7f28b7fff700 1 --2- 172.21.15.1:0/3154784099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580e32b1930 0x5580e32ba340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.401 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.388+0000 7f28b7fff700 1 -- 172.21.15.1:0/3154784099 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5580e31c4210 con 0x5580e32ad4f0 2023-11-16T16:46:16.402 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.388+0000 7f28b7fff700 1 --2- 172.21.15.1:0/3154784099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580e32ad4f0 0x5580e32b9c00 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f28a40059d0 tx=0x7f28a400b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e14e49bbb05ffbf1 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:16.402 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.388+0000 7f2897fff700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:46:16.402 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.389+0000 7f2897fff700 1 -- 172.21.15.1:0/3154784099 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a4003da0 con 0x5580e32ad4f0 2023-11-16T16:46:16.402 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.389+0000 7f2897fff700 1 -- 172.21.15.1:0/3154784099 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f28a40050d0 con 0x5580e32ad4f0 2023-11-16T16:46:16.402 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.389+0000 7f28cb197b80 1 -- 172.21.15.1:0/3154784099 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f28a401e040 con 0x5580e32ad4f0 2023-11-16T16:46:16.402 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.389+0000 7f2897fff700 1 -- 172.21.15.1:0/3154784099 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28a4005290 con 0x5580e32ad4f0 2023-11-16T16:46:16.402 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.389+0000 7f28cb197b80 10 client.16404 fetch_fsmap learned FSMap version 28 2023-11-16T16:46:16.403 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.389+0000 7f28cb197b80 1 -- 172.21.15.1:0/3154784099 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5580e302be60 con 0x5580e32ad4f0 2023-11-16T16:46:16.403 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.389+0000 7f2897fff700 1 -- 172.21.15.1:0/3154784099 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28a401e040 con 0x5580e32ad4f0 2023-11-16T16:46:16.403 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.389+0000 7f2897fff700 1 -- 172.21.15.1:0/3154784099 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f28a4021070 con 0x5580e32ad4f0 2023-11-16T16:46:16.403 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.389+0000 7f28cb197b80 10 client.16404 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:46:16.403 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.389+0000 7f28cb197b80 10 client.16404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:46:16.403 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.389+0000 7f28cb197b80 20 client.16404 populate_metadata read hostname 'smithi001' 2023-11-16T16:46:16.403 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.389+0000 7f28cb197b80 1 --2- 172.21.15.1:0/3154784099 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5580e331bc70 0x5580e331e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.404 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.389+0000 7f28cb197b80 4 client.16404 mds_command: new command op to 24521 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-16T16:46:16.404 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.389+0000 7f28cb197b80 1 -- 172.21.15.1:0/3154784099 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5580e31cc6e0 con 0x5580e331bc70 2023-11-16T16:46:16.404 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.390+0000 7f28bcda7700 1 --2- 172.21.15.1:0/3154784099 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5580e331bc70 0x5580e331e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.404 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.390+0000 7f2897fff700 10 client.16404 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:46:16.404 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.390+0000 7f28bcda7700 1 --2- 172.21.15.1:0/3154784099 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5580e331bc70 0x5580e331e080 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3de37bd94c5d126 server_cookie=b8ceef87d066a012 in_seq=0 out_seq=0 2023-11-16T16:46:16.404 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.391+0000 7f2897fff700 1 -- 172.21.15.1:0/3154784099 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5580e31cc6e0 con 0x5580e331bc70 2023-11-16T16:46:16.404 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.391+0000 7f2897fff700 10 client.16404 handle_command_reply: tid=0 2023-11-16T16:46:16.405 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.392+0000 7f28cb197b80 1 client.16404 shutdown 2023-11-16T16:46:16.405 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.392+0000 7f28cb197b80 1 -- 172.21.15.1:0/3154784099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580e32ad4f0 msgr2=0x5580e32b9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:16.405 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.392+0000 7f28cb197b80 1 --2- 172.21.15.1:0/3154784099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580e32ad4f0 0x5580e32b9c00 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f28a40059d0 tx=0x7f28a400b5c0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.405 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.392+0000 7f28cb197b80 1 -- 172.21.15.1:0/3154784099 shutdown_connections 2023-11-16T16:46:16.405 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.392+0000 7f28cb197b80 1 --2- 172.21.15.1:0/3154784099 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5580e331bc70 0x5580e331e080 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.405 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.392+0000 7f28cb197b80 1 --2- 172.21.15.1:0/3154784099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580e32b1930 0x5580e32ba340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.405 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.392+0000 7f28cb197b80 1 --2- 172.21.15.1:0/3154784099 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5580e32acbe0 0x5580e32bd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.406 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.392+0000 7f28cb197b80 1 --2- 172.21.15.1:0/3154784099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580e32ad4f0 0x5580e32b9c00 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.406 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.392+0000 7f28cb197b80 1 -- 172.21.15.1:0/3154784099 >> 172.21.15.1:0/3154784099 conn(0x5580e32a23e0 msgr2=0x5580e32af930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:16.406 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.392+0000 7f2897fff700 0 client.16404 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:46:16.406 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.392+0000 7f28cb197b80 1 -- 172.21.15.1:0/3154784099 shutdown_connections 2023-11-16T16:46:16.406 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.392+0000 7f28cb197b80 1 -- 172.21.15.1:0/3154784099 wait complete. 2023-11-16T16:46:16.406 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.392+0000 7f28cb197b80 20 client.16404 trim_cache size 0 max 16384 2023-11-16T16:46:16.407 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:46:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2023-11-16T16:46:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "3683b1bd-e7c6-4fce-8e3e-103381fab906", 2023-11-16T16:46:16.407 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2023-11-16T16:46:16.407 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:46:16.409 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.393+0000 7f28bf00b700 1 -- 172.21.15.1:0/4022702281 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f28a80a12c0 msgr2=0x7f28a80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:16.409 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.393+0000 7f28bf00b700 1 --2- 172.21.15.1:0/4022702281 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f28a80a12c0 0x7f28a80a3770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f28ac005fd0 tx=0x7f28ac005f60 comp rx=0 tx=0).stop 2023-11-16T16:46:16.409 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.393+0000 7f28bf00b700 1 -- 172.21.15.1:0/4022702281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28b810a7f0 msgr2=0x7f28b819ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:16.409 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.393+0000 7f28bf00b700 1 --2- 172.21.15.1:0/4022702281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28b810a7f0 0x7f28b819ac90 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f28b0007ee0 tx=0x7f28b000d3b0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.409 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.393+0000 7f28bf00b700 1 -- 172.21.15.1:0/4022702281 shutdown_connections 2023-11-16T16:46:16.409 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.393+0000 7f28bf00b700 1 --2- 172.21.15.1:0/4022702281 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f28a80a12c0 0x7f28a80a3770 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.410 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.393+0000 7f28bf00b700 1 --2- 172.21.15.1:0/4022702281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f28b810a7f0 0x7f28b819ac90 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.410 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.393+0000 7f28bf00b700 1 --2- 172.21.15.1:0/4022702281 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f28b81014b0 0x7f28b81962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.410 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.393+0000 7f28bf00b700 1 --2- 172.21.15.1:0/4022702281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28b8101dc0 0x7f28b8196830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.410 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.393+0000 7f28bf00b700 1 -- 172.21.15.1:0/4022702281 >> 172.21.15.1:0/4022702281 conn(0x7f28b8076340 msgr2=0x7f28b80fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:16.410 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.393+0000 7f28bf00b700 1 -- 172.21.15.1:0/4022702281 shutdown_connections 2023-11-16T16:46:16.410 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.393+0000 7f28bf00b700 1 -- 172.21.15.1:0/4022702281 wait complete. 2023-11-16T16:46:16.411 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-16T16:46:16.541 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.537+0000 7f41d4269700 1 -- 172.21.15.1:0/782216168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f41cc105250 msgr2=0x7f41cc105620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:16.542 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.537+0000 7f41d4269700 1 --2- 172.21.15.1:0/782216168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f41cc105250 0x7f41cc105620 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f41c8009990 tx=0x7f41c8009ca0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.542 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.537+0000 7f41d4269700 1 -- 172.21.15.1:0/782216168 shutdown_connections 2023-11-16T16:46:16.542 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.537+0000 7f41d4269700 1 --2- 172.21.15.1:0/782216168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41cc069a60 0x7f41cc10d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.542 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.537+0000 7f41d4269700 1 --2- 172.21.15.1:0/782216168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f41cc105250 0x7f41cc105620 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.542 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.537+0000 7f41d4269700 1 --2- 172.21.15.1:0/782216168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41cc0690b0 0x7f41cc069520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.542 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.537+0000 7f41d4269700 1 -- 172.21.15.1:0/782216168 >> 172.21.15.1:0/782216168 conn(0x7f41cc076370 msgr2=0x7f41cc076770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:16.543 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.537+0000 7f41d4269700 1 -- 172.21.15.1:0/782216168 shutdown_connections 2023-11-16T16:46:16.543 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.538+0000 7f41d4269700 1 -- 172.21.15.1:0/782216168 wait complete. 2023-11-16T16:46:16.543 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.538+0000 7f41d4269700 1 Processor -- start 2023-11-16T16:46:16.543 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.538+0000 7f41d4269700 1 -- start start 2023-11-16T16:46:16.543 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.538+0000 7f41d4269700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f41cc0690b0 0x7f41cc1963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.543 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.538+0000 7f41d4269700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41cc069a60 0x7f41cc196910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.543 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.538+0000 7f41d4269700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41cc105250 0x7f41cc19ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.544 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.538+0000 7f41d4269700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41cc19b3b0 con 0x7f41cc0690b0 2023-11-16T16:46:16.544 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.538+0000 7f41d4269700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41cc19b520 con 0x7f41cc105250 2023-11-16T16:46:16.544 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.538+0000 7f41d4269700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41cc19b6e0 con 0x7f41cc069a60 2023-11-16T16:46:16.544 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.538+0000 7f41d2005700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f41cc0690b0 0x7f41cc1963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.544 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.539+0000 7f41d2005700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f41cc0690b0 0x7f41cc1963d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45832/0 (socket says 172.21.15.1:45832) 2023-11-16T16:46:16.544 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.539+0000 7f41d2005700 1 -- 172.21.15.1:0/293332518 learned_addr learned my addr 172.21.15.1:0/293332518 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:16.544 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.539+0000 7f41d2806700 1 --2- 172.21.15.1:0/293332518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41cc105250 0x7f41cc19ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.544 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.539+0000 7f41d1804700 1 --2- 172.21.15.1:0/293332518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41cc069a60 0x7f41cc196910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.545 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.539+0000 7f41d2005700 1 -- 172.21.15.1:0/293332518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41cc069a60 msgr2=0x7f41cc196910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:16.545 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.539+0000 7f41d2005700 1 --2- 172.21.15.1:0/293332518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41cc069a60 0x7f41cc196910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.545 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.539+0000 7f41d2005700 1 -- 172.21.15.1:0/293332518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41cc105250 msgr2=0x7f41cc19ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:16.545 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.539+0000 7f41d2005700 1 --2- 172.21.15.1:0/293332518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41cc105250 0x7f41cc19ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.545 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.539+0000 7f41d2005700 1 -- 172.21.15.1:0/293332518 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41c80095f0 con 0x7f41cc0690b0 2023-11-16T16:46:16.545 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.539+0000 7f41d2005700 1 --2- 172.21.15.1:0/293332518 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f41cc0690b0 0x7f41cc1963d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f41c80095c0 tx=0x7f41c800f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:16.545 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.539+0000 7f41c37fe700 1 -- 172.21.15.1:0/293332518 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c801d070 con 0x7f41cc0690b0 2023-11-16T16:46:16.545 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.539+0000 7f41c37fe700 1 -- 172.21.15.1:0/293332518 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41c800fb50 con 0x7f41cc0690b0 2023-11-16T16:46:16.546 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.539+0000 7f41d4269700 1 -- 172.21.15.1:0/293332518 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41cc110340 con 0x7f41cc0690b0 2023-11-16T16:46:16.546 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.539+0000 7f41c37fe700 1 -- 172.21.15.1:0/293332518 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c80173f0 con 0x7f41cc0690b0 2023-11-16T16:46:16.546 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.539+0000 7f41d4269700 1 -- 172.21.15.1:0/293332518 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41cc1101b0 con 0x7f41cc0690b0 2023-11-16T16:46:16.553 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.541+0000 7f41c37fe700 1 -- 172.21.15.1:0/293332518 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f41c800fcc0 con 0x7f41cc0690b0 2023-11-16T16:46:16.553 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.542+0000 7f41c37fe700 1 --2- 172.21.15.1:0/293332518 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f41b80a12c0 0x7f41b80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.553 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.542+0000 7f41c37fe700 1 -- 172.21.15.1:0/293332518 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f41c80cddd0 con 0x7f41cc0690b0 2023-11-16T16:46:16.553 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.542+0000 7f41d1804700 1 --2- 172.21.15.1:0/293332518 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f41b80a12c0 0x7f41b80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.553 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.542+0000 7f41d1804700 1 --2- 172.21.15.1:0/293332518 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f41b80a12c0 0x7f41b80a3770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f41bc005950 tx=0x7f41bc0058e0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:16.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.543+0000 7f41e03f5b80 1 Processor -- start 2023-11-16T16:46:16.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.543+0000 7f41e03f5b80 1 -- start start 2023-11-16T16:46:16.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.543+0000 7f41e03f5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5beb98b60 0x55e5beb98f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.543+0000 7f41e03f5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5beb99470 0x55e5beb9d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.543+0000 7f41e03f5b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5beb9d8b0 0x55e5beb9dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.543+0000 7f41e03f5b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5bea9e060 con 0x55e5beb9d8b0 2023-11-16T16:46:16.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.543+0000 7f41e03f5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5beaf08a0 con 0x55e5beb99470 2023-11-16T16:46:16.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.543+0000 7f41e03f5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5beae9d30 con 0x55e5beb98b60 2023-11-16T16:46:16.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.543+0000 7f41d2806700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5beb9d8b0 0x55e5beb9dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.544+0000 7f41d1804700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5beb99470 0x55e5beb9d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.544+0000 7f41d2806700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5beb9d8b0 0x55e5beb9dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45844/0 (socket says 172.21.15.1:45844) 2023-11-16T16:46:16.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.544+0000 7f41d2806700 1 -- 172.21.15.1:0/1058751759 learned_addr learned my addr 172.21.15.1:0/1058751759 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:16.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.544+0000 7f41d2005700 1 --2- 172.21.15.1:0/1058751759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5beb98b60 0x55e5beb98f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.544+0000 7f41d1804700 1 -- 172.21.15.1:0/1058751759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5beb98b60 msgr2=0x55e5beb98f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:16.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.544+0000 7f41d1804700 1 --2- 172.21.15.1:0/1058751759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5beb98b60 0x55e5beb98f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.544+0000 7f41d1804700 1 -- 172.21.15.1:0/1058751759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5beb9d8b0 msgr2=0x55e5beb9dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:16.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.544+0000 7f41d1804700 1 --2- 172.21.15.1:0/1058751759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5beb9d8b0 0x55e5beb9dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.544+0000 7f41d1804700 1 -- 172.21.15.1:0/1058751759 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5be96b630 con 0x55e5beb99470 2023-11-16T16:46:16.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.544+0000 7f41d2806700 1 --2- 172.21.15.1:0/1058751759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5beb9d8b0 0x55e5beb9dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:46:16.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.544+0000 7f41d1804700 1 --2- 172.21.15.1:0/1058751759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5beb99470 0x55e5beb9d370 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f41cc1979e0 tx=0x7f41bc012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d747371c2bccf10d server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:16.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.544+0000 7f41c0dac700 1 -- 172.21.15.1:0/1058751759 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41bc011ad0 con 0x55e5beb99470 2023-11-16T16:46:16.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.544+0000 7f41c0dac700 1 -- 172.21.15.1:0/1058751759 <== mon.2 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41bc011c30 con 0x55e5beb99470 2023-11-16T16:46:16.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.544+0000 7f41c0dac700 1 -- 172.21.15.1:0/1058751759 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41bc01d610 con 0x55e5beb99470 2023-11-16T16:46:16.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.545+0000 7f41e03f5b80 1 -- 172.21.15.1:0/1058751759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5beb99470 msgr2=0x55e5beb9d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:16.557 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.545+0000 7f41e03f5b80 1 --2- 172.21.15.1:0/1058751759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5beb99470 0x55e5beb9d370 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f41cc1979e0 tx=0x7f41bc012000 comp rx=0 tx=0).stop 2023-11-16T16:46:16.557 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.545+0000 7f41e03f5b80 1 -- 172.21.15.1:0/1058751759 shutdown_connections 2023-11-16T16:46:16.557 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.545+0000 7f41e03f5b80 1 --2- 172.21.15.1:0/1058751759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5beb9d8b0 0x55e5beb9dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.557 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.545+0000 7f41e03f5b80 1 --2- 172.21.15.1:0/1058751759 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5beb98b60 0x55e5beb98f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.557 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.545+0000 7f41e03f5b80 1 --2- 172.21.15.1:0/1058751759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5beb99470 0x55e5beb9d370 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.557 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.545+0000 7f41e03f5b80 1 -- 172.21.15.1:0/1058751759 >> 172.21.15.1:0/1058751759 conn(0x55e5beb8e360 msgr2=0x55e5beb93b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:16.557 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.545+0000 7f41e03f5b80 1 -- 172.21.15.1:0/1058751759 shutdown_connections 2023-11-16T16:46:16.557 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.545+0000 7f41e03f5b80 1 -- 172.21.15.1:0/1058751759 wait complete. 2023-11-16T16:46:16.557 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.545+0000 7f41e03f5b80 1 -- start start 2023-11-16T16:46:16.558 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.545+0000 7f41e03f5b80 1 Processor -- start 2023-11-16T16:46:16.558 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.546+0000 7f41e03f5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5beb98b60 0x55e5beba2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.558 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.546+0000 7f41e03f5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5beb9d8b0 0x55e5beba2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.558 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.546+0000 7f41e03f5b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5beba2fb0 0x55e5bebaea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.558 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.546+0000 7f41e03f5b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5bea8b8e0 con 0x55e5beba2fb0 2023-11-16T16:46:16.558 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.546+0000 7f41e03f5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5bea8b580 con 0x55e5beb98b60 2023-11-16T16:46:16.558 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.546+0000 7f41e03f5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5be96d920 con 0x55e5beb9d8b0 2023-11-16T16:46:16.559 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.546+0000 7f41e03f5b80 10 client.0 _ll_register_callbacks cb 0x55e5beaa9780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:46:16.559 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.546+0000 7f41d2806700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5beba2fb0 0x55e5bebaea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.559 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.546+0000 7f41d2806700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5beba2fb0 0x55e5bebaea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45850/0 (socket says 172.21.15.1:45850) 2023-11-16T16:46:16.559 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.546+0000 7f41d2806700 1 -- 172.21.15.1:0/870329307 learned_addr learned my addr 172.21.15.1:0/870329307 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:16.559 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.546+0000 7f41d2005700 1 --2- 172.21.15.1:0/870329307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5beb98b60 0x55e5beba2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.559 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.546+0000 7f41d1804700 1 --2- 172.21.15.1:0/870329307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5beb9d8b0 0x55e5beba2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.559 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.546+0000 7f41d2806700 1 -- 172.21.15.1:0/870329307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5beb9d8b0 msgr2=0x55e5beba2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:16.559 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.546+0000 7f41d2806700 1 --2- 172.21.15.1:0/870329307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5beb9d8b0 0x55e5beba2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.560 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.546+0000 7f41d2806700 1 -- 172.21.15.1:0/870329307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5beb98b60 msgr2=0x55e5beba2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:16.560 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.546+0000 7f41d2806700 1 --2- 172.21.15.1:0/870329307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5beb98b60 0x55e5beba2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.560 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.546+0000 7f41d2806700 1 -- 172.21.15.1:0/870329307 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41bc00f950 con 0x55e5beba2fb0 2023-11-16T16:46:16.560 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.546+0000 7f41d2806700 1 --2- 172.21.15.1:0/870329307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5beba2fb0 0x55e5bebaea60 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f41c400ae70 tx=0x7f41c4012510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=988b736af46185f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:16.560 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.546+0000 7f41b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:46:16.560 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.546+0000 7f41b5ffb700 1 -- 172.21.15.1:0/870329307 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c400f040 con 0x55e5beba2fb0 2023-11-16T16:46:16.560 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.547+0000 7f41b5ffb700 1 -- 172.21.15.1:0/870329307 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f41c4012c20 con 0x55e5beba2fb0 2023-11-16T16:46:16.560 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.547+0000 7f41e03f5b80 1 -- 172.21.15.1:0/870329307 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e5beb8bb20 con 0x55e5beba2fb0 2023-11-16T16:46:16.561 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.547+0000 7f41b5ffb700 1 -- 172.21.15.1:0/870329307 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41c4012de0 con 0x55e5beba2fb0 2023-11-16T16:46:16.561 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.547+0000 7f41e03f5b80 10 client.16416 fetch_fsmap learned FSMap version 28 2023-11-16T16:46:16.561 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.547+0000 7f41e03f5b80 1 -- 172.21.15.1:0/870329307 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e5beab0190 con 0x55e5beba2fb0 2023-11-16T16:46:16.561 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.547+0000 7f41b5ffb700 1 -- 172.21.15.1:0/870329307 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e5beb8bb20 con 0x55e5beba2fb0 2023-11-16T16:46:16.561 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.547+0000 7f41b5ffb700 1 -- 172.21.15.1:0/870329307 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f41c40083f0 con 0x55e5beba2fb0 2023-11-16T16:46:16.561 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.547+0000 7f41e03f5b80 10 client.16416 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:46:16.561 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.547+0000 7f41e03f5b80 10 client.16416 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:46:16.561 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.547+0000 7f41e03f5b80 20 client.16416 populate_metadata read hostname 'smithi001' 2023-11-16T16:46:16.562 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.547+0000 7f41e03f5b80 1 --2- 172.21.15.1:0/870329307 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55e5bebb2f80 0x55e5bebb5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.562 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.547+0000 7f41e03f5b80 4 client.16416 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:46:16.562 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.547+0000 7f41e03f5b80 1 -- 172.21.15.1:0/870329307 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e5beaac0a0 con 0x55e5bebb2f80 2023-11-16T16:46:16.562 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.547+0000 7f41d2005700 1 --2- 172.21.15.1:0/870329307 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55e5bebb2f80 0x55e5bebb5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.562 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.548+0000 7f41d2005700 1 --2- 172.21.15.1:0/870329307 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55e5bebb2f80 0x55e5bebb5390 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c397bcf12f95716 server_cookie=f294707f147d4cac in_seq=0 out_seq=0 2023-11-16T16:46:16.562 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.548+0000 7f41b5ffb700 10 client.16416 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:46:16.562 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.548+0000 7f41b5ffb700 1 -- 172.21.15.1:0/870329307 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e5beaac0a0 con 0x55e5bebb2f80 2023-11-16T16:46:16.563 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.548+0000 7f41b5ffb700 10 client.16416 handle_command_reply: tid=0 2023-11-16T16:46:16.563 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.548+0000 7f41e03f5b80 1 client.16416 shutdown 2023-11-16T16:46:16.563 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.549+0000 7f41e03f5b80 1 -- 172.21.15.1:0/870329307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5beba2fb0 msgr2=0x55e5bebaea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:16.563 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.549+0000 7f41e03f5b80 1 --2- 172.21.15.1:0/870329307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5beba2fb0 0x55e5bebaea60 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f41c400ae70 tx=0x7f41c4012510 comp rx=0 tx=0).stop 2023-11-16T16:46:16.563 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.549+0000 7f41e03f5b80 1 -- 172.21.15.1:0/870329307 shutdown_connections 2023-11-16T16:46:16.563 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.549+0000 7f41e03f5b80 1 --2- 172.21.15.1:0/870329307 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55e5bebb2f80 0x55e5bebb5390 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.563 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.549+0000 7f41e03f5b80 1 --2- 172.21.15.1:0/870329307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5beba2fb0 0x55e5bebaea60 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.563 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.549+0000 7f41e03f5b80 1 --2- 172.21.15.1:0/870329307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5beb98b60 0x55e5beba2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.563 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.549+0000 7f41e03f5b80 1 --2- 172.21.15.1:0/870329307 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5beb9d8b0 0x55e5beba2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.564 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.549+0000 7f41e03f5b80 1 -- 172.21.15.1:0/870329307 >> 172.21.15.1:0/870329307 conn(0x55e5beb8e360 msgr2=0x55e5beb97960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:16.564 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.549+0000 7f41b5ffb700 0 client.16416 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:46:16.564 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.549+0000 7f41e03f5b80 1 -- 172.21.15.1:0/870329307 shutdown_connections 2023-11-16T16:46:16.564 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.549+0000 7f41e03f5b80 1 -- 172.21.15.1:0/870329307 wait complete. 2023-11-16T16:46:16.564 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.549+0000 7f41e03f5b80 20 client.16416 trim_cache size 0 max 16384 2023-11-16T16:46:16.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.566+0000 7f41e03f5b80 1 Processor -- start 2023-11-16T16:46:16.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.566+0000 7f41e03f5b80 1 -- start start 2023-11-16T16:46:16.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.566+0000 7f41e03f5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5beb98b60 0x55e5bebae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.566+0000 7f41e03f5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5beb9d8b0 0x55e5beba99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.566+0000 7f41e03f5b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5beba2fb0 0x55e5beb9ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.566+0000 7f41e03f5b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5beba5370 con 0x55e5beba2fb0 2023-11-16T16:46:16.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.566+0000 7f41e03f5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5beba3b10 con 0x55e5beb9d8b0 2023-11-16T16:46:16.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.566+0000 7f41e03f5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5beb95110 con 0x55e5beb98b60 2023-11-16T16:46:16.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.566+0000 7f41d2806700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5beba2fb0 0x55e5beb9ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.566+0000 7f41d2806700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5beba2fb0 0x55e5beb9ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45858/0 (socket says 172.21.15.1:45858) 2023-11-16T16:46:16.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.566+0000 7f41d2806700 1 -- 172.21.15.1:0/3454540558 learned_addr learned my addr 172.21.15.1:0/3454540558 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:16.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.566+0000 7f41d2005700 1 --2- 172.21.15.1:0/3454540558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5beb98b60 0x55e5bebae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.566+0000 7f41d1804700 1 --2- 172.21.15.1:0/3454540558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5beb9d8b0 0x55e5beba99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.566+0000 7f41d2806700 1 -- 172.21.15.1:0/3454540558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5beb98b60 msgr2=0x55e5bebae9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:16.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.566+0000 7f41d2806700 1 --2- 172.21.15.1:0/3454540558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5beb98b60 0x55e5bebae9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.566+0000 7f41d2806700 1 -- 172.21.15.1:0/3454540558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5beb9d8b0 msgr2=0x55e5beba99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:16.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.566+0000 7f41d2806700 1 --2- 172.21.15.1:0/3454540558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5beb9d8b0 0x55e5beba99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.575 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.566+0000 7f41d2806700 1 -- 172.21.15.1:0/3454540558 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5beaafee0 con 0x55e5beba2fb0 2023-11-16T16:46:16.576 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.566+0000 7f41d2005700 1 --2- 172.21.15.1:0/3454540558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5beb98b60 0x55e5bebae9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:46:16.576 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.566+0000 7f41d1804700 1 --2- 172.21.15.1:0/3454540558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5beb9d8b0 0x55e5beba99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:46:16.576 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.567+0000 7f41d2806700 1 --2- 172.21.15.1:0/3454540558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5beba2fb0 0x55e5beb9ec90 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f41c400a390 tx=0x7f41c4007cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93bf3d2f91ce34f server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:16.576 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.567+0000 7f41c15ad700 1 -- 172.21.15.1:0/3454540558 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c400f070 con 0x55e5beba2fb0 2023-11-16T16:46:16.576 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.567+0000 7f41c15ad700 1 -- 172.21.15.1:0/3454540558 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41c4010040 con 0x55e5beba2fb0 2023-11-16T16:46:16.576 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.567+0000 7f41c15ad700 1 -- 172.21.15.1:0/3454540558 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c4012d90 con 0x55e5beba2fb0 2023-11-16T16:46:16.576 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.567+0000 7f41e03f5b80 1 -- 172.21.15.1:0/3454540558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5beba2fb0 msgr2=0x55e5beb9ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:16.576 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.567+0000 7f41e03f5b80 1 --2- 172.21.15.1:0/3454540558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5beba2fb0 0x55e5beb9ec90 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f41c400a390 tx=0x7f41c4007cc0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.577 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.567+0000 7f41e03f5b80 1 -- 172.21.15.1:0/3454540558 shutdown_connections 2023-11-16T16:46:16.577 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.567+0000 7f41e03f5b80 1 --2- 172.21.15.1:0/3454540558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5beba2fb0 0x55e5beb9ec90 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.577 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.567+0000 7f41e03f5b80 1 --2- 172.21.15.1:0/3454540558 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5beb98b60 0x55e5bebae9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.577 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.567+0000 7f41e03f5b80 1 --2- 172.21.15.1:0/3454540558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5beb9d8b0 0x55e5beba99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.577 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.567+0000 7f41e03f5b80 1 -- 172.21.15.1:0/3454540558 >> 172.21.15.1:0/3454540558 conn(0x55e5beb8e360 msgr2=0x55e5beb9d110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:16.577 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.567+0000 7f41e03f5b80 1 -- 172.21.15.1:0/3454540558 shutdown_connections 2023-11-16T16:46:16.577 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.567+0000 7f41e03f5b80 1 -- 172.21.15.1:0/3454540558 wait complete. 2023-11-16T16:46:16.577 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.568+0000 7f41e03f5b80 1 -- start start 2023-11-16T16:46:16.578 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.568+0000 7f41e03f5b80 1 Processor -- start 2023-11-16T16:46:16.578 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.568+0000 7f41e03f5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5beb98b60 0x55e5beba9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.578 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.568+0000 7f41e03f5b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5beb9d8b0 0x55e5beba98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.578 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.568+0000 7f41e03f5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5beba2fb0 0x55e5beba7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.578 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.568+0000 7f41e03f5b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5beba7e10 con 0x55e5beb9d8b0 2023-11-16T16:46:16.578 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.568+0000 7f41e03f5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5beba7f80 con 0x55e5beba2fb0 2023-11-16T16:46:16.578 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.568+0000 7f41e03f5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5beba8140 con 0x55e5beb98b60 2023-11-16T16:46:16.578 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.568+0000 7f41e03f5b80 10 client.0 _ll_register_callbacks cb 0x7f41c400a7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:46:16.579 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.568+0000 7f41d1804700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5beb9d8b0 0x55e5beba98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.579 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.568+0000 7f41d1804700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5beb9d8b0 0x55e5beba98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45862/0 (socket says 172.21.15.1:45862) 2023-11-16T16:46:16.579 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.568+0000 7f41d1804700 1 -- 172.21.15.1:0/496060084 learned_addr learned my addr 172.21.15.1:0/496060084 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:16.579 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.568+0000 7f41d2005700 1 --2- 172.21.15.1:0/496060084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5beb98b60 0x55e5beba9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.579 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.568+0000 7f41d2806700 1 --2- 172.21.15.1:0/496060084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5beba2fb0 0x55e5beba7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.579 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.568+0000 7f41d1804700 1 -- 172.21.15.1:0/496060084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5beb98b60 msgr2=0x55e5beba9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:16.579 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.568+0000 7f41d1804700 1 --2- 172.21.15.1:0/496060084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5beb98b60 0x55e5beba9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.579 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.568+0000 7f41d1804700 1 -- 172.21.15.1:0/496060084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5beba2fb0 msgr2=0x55e5beba7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:16.580 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.568+0000 7f41d1804700 1 --2- 172.21.15.1:0/496060084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5beba2fb0 0x55e5beba7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.580 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.568+0000 7f41d1804700 1 -- 172.21.15.1:0/496060084 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e5beab0190 con 0x55e5beb9d8b0 2023-11-16T16:46:16.580 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.568+0000 7f41d2806700 1 --2- 172.21.15.1:0/496060084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5beba2fb0 0x55e5beba7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:46:16.580 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.569+0000 7f41d1804700 1 --2- 172.21.15.1:0/496060084 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5beb9d8b0 0x55e5beba98a0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x55e5beb99980 tx=0x7f41bc0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b881b48234dfbfac server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:16.580 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.569+0000 7f41b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:46:16.580 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.569+0000 7f41b5ffb700 1 -- 172.21.15.1:0/496060084 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41bc005450 con 0x55e5beb9d8b0 2023-11-16T16:46:16.580 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.569+0000 7f41b5ffb700 1 -- 172.21.15.1:0/496060084 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f41bc0055b0 con 0x55e5beb9d8b0 2023-11-16T16:46:16.580 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.569+0000 7f41e03f5b80 1 -- 172.21.15.1:0/496060084 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f41bc011c30 con 0x55e5beb9d8b0 2023-11-16T16:46:16.581 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.569+0000 7f41b5ffb700 1 -- 172.21.15.1:0/496060084 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41bc01e450 con 0x55e5beb9d8b0 2023-11-16T16:46:16.581 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.569+0000 7f41e03f5b80 10 client.16422 fetch_fsmap learned FSMap version 28 2023-11-16T16:46:16.581 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.569+0000 7f41e03f5b80 1 -- 172.21.15.1:0/496060084 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e5be900c60 con 0x55e5beb9d8b0 2023-11-16T16:46:16.581 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.569+0000 7f41b5ffb700 1 -- 172.21.15.1:0/496060084 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41bc040020 con 0x55e5beb9d8b0 2023-11-16T16:46:16.581 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.569+0000 7f41b5ffb700 1 -- 172.21.15.1:0/496060084 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f41bc004b90 con 0x55e5beb9d8b0 2023-11-16T16:46:16.581 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.569+0000 7f41e03f5b80 10 client.16422 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:46:16.581 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.569+0000 7f41e03f5b80 10 client.16422 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:46:16.581 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.569+0000 7f41e03f5b80 20 client.16422 populate_metadata read hostname 'smithi001' 2023-11-16T16:46:16.582 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.569+0000 7f41e03f5b80 1 --2- 172.21.15.1:0/496060084 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55e5bec07ee0 0x55e5bec0a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:16.582 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.569+0000 7f41e03f5b80 4 client.16422 mds_command: new command op to 24521 tid=0[{"prefix": "scrub status"}] 2023-11-16T16:46:16.582 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.569+0000 7f41e03f5b80 1 -- 172.21.15.1:0/496060084 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e5beab8660 con 0x55e5bec07ee0 2023-11-16T16:46:16.582 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.569+0000 7f41d2005700 1 --2- 172.21.15.1:0/496060084 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55e5bec07ee0 0x55e5bec0a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:16.582 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.570+0000 7f41d2005700 1 --2- 172.21.15.1:0/496060084 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55e5bec07ee0 0x55e5bec0a2f0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed65f07e498bc411 server_cookie=8696f29625ca8c7 in_seq=0 out_seq=0 2023-11-16T16:46:16.582 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.570+0000 7f41b5ffb700 10 client.16422 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:46:16.582 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.570+0000 7f41b5ffb700 1 -- 172.21.15.1:0/496060084 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e5beab8660 con 0x55e5bec07ee0 2023-11-16T16:46:16.582 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.570+0000 7f41b5ffb700 10 client.16422 handle_command_reply: tid=0 2023-11-16T16:46:16.583 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.570+0000 7f41e03f5b80 1 client.16422 shutdown 2023-11-16T16:46:16.583 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.570+0000 7f41e03f5b80 1 -- 172.21.15.1:0/496060084 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5beb9d8b0 msgr2=0x55e5beba98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:16.583 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.570+0000 7f41e03f5b80 1 --2- 172.21.15.1:0/496060084 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5beb9d8b0 0x55e5beba98a0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x55e5beb99980 tx=0x7f41bc0118d0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.583 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.570+0000 7f41e03f5b80 1 -- 172.21.15.1:0/496060084 shutdown_connections 2023-11-16T16:46:16.583 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.570+0000 7f41e03f5b80 1 --2- 172.21.15.1:0/496060084 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55e5bec07ee0 0x55e5bec0a2f0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.583 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.570+0000 7f41e03f5b80 1 --2- 172.21.15.1:0/496060084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e5beba2fb0 0x55e5beba7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.583 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.570+0000 7f41e03f5b80 1 --2- 172.21.15.1:0/496060084 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55e5beb98b60 0x55e5beba9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.583 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.570+0000 7f41e03f5b80 1 --2- 172.21.15.1:0/496060084 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e5beb9d8b0 0x55e5beba98a0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.584 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.570+0000 7f41b5ffb700 0 client.16422 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:46:16.584 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.570+0000 7f41e03f5b80 1 -- 172.21.15.1:0/496060084 >> 172.21.15.1:0/496060084 conn(0x55e5beb8e360 msgr2=0x55e5beb9fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:16.584 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.570+0000 7f41e03f5b80 1 -- 172.21.15.1:0/496060084 shutdown_connections 2023-11-16T16:46:16.584 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.570+0000 7f41e03f5b80 1 -- 172.21.15.1:0/496060084 wait complete. 2023-11-16T16:46:16.584 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.570+0000 7f41e03f5b80 20 client.16422 trim_cache size 0 max 16384 2023-11-16T16:46:16.584 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:46:16.585 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-16T16:46:16.585 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2023-11-16T16:46:16.585 INFO:teuthology.orchestra.run.smithi001.stdout: "3683b1bd-e7c6-4fce-8e3e-103381fab906": { 2023-11-16T16:46:16.585 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2023-11-16T16:46:16.585 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "3683b1bd-e7c6-4fce-8e3e-103381fab906", 2023-11-16T16:46:16.585 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2023-11-16T16:46:16.585 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:46:16.585 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:46:16.586 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:46:16.587 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.571+0000 7f41d4269700 1 -- 172.21.15.1:0/293332518 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f41b80a12c0 msgr2=0x7f41b80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:16.587 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.571+0000 7f41d4269700 1 --2- 172.21.15.1:0/293332518 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f41b80a12c0 0x7f41b80a3770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f41bc005950 tx=0x7f41bc0058e0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.587 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.571+0000 7f41d4269700 1 -- 172.21.15.1:0/293332518 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f41cc0690b0 msgr2=0x7f41cc1963d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:16.587 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.571+0000 7f41d4269700 1 --2- 172.21.15.1:0/293332518 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f41cc0690b0 0x7f41cc1963d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f41c80095c0 tx=0x7f41c800f710 comp rx=0 tx=0).stop 2023-11-16T16:46:16.588 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.571+0000 7f41d4269700 1 -- 172.21.15.1:0/293332518 shutdown_connections 2023-11-16T16:46:16.588 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.571+0000 7f41d4269700 1 --2- 172.21.15.1:0/293332518 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f41b80a12c0 0x7f41b80a3770 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.588 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.571+0000 7f41d4269700 1 --2- 172.21.15.1:0/293332518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41cc105250 0x7f41cc19ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.588 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.571+0000 7f41d4269700 1 --2- 172.21.15.1:0/293332518 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f41cc0690b0 0x7f41cc1963d0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.588 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.571+0000 7f41d4269700 1 --2- 172.21.15.1:0/293332518 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f41cc069a60 0x7f41cc196910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:16.588 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.571+0000 7f41d4269700 1 -- 172.21.15.1:0/293332518 >> 172.21.15.1:0/293332518 conn(0x7f41cc076370 msgr2=0x7f41cc0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:16.588 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.571+0000 7f41d4269700 1 -- 172.21.15.1:0/293332518 shutdown_connections 2023-11-16T16:46:16.588 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:16.571+0000 7f41d4269700 1 -- 172.21.15.1:0/293332518 wait complete. 2023-11-16T16:46:16.589 INFO:tasks.cephfs.filesystem:scrub status for tag:3683b1bd-e7c6-4fce-8e3e-103381fab906 - {'path': '/', 'tag': '3683b1bd-e7c6-4fce-8e3e-103381fab906', 'options': 'recursive,force'} 2023-11-16T16:46:16.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:16 smithi001 ceph-mon[98800]: pgmap v217: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 7 op/s 2023-11-16T16:46:16.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:16 smithi001 ceph-mon[98800]: osdmap e89: 12 total, 12 up, 12 in 2023-11-16T16:46:17.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:16 smithi026 ceph-mon[100112]: pgmap v217: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 7 op/s 2023-11-16T16:46:17.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:16 smithi026 ceph-mon[100112]: osdmap e89: 12 total, 12 up, 12 in 2023-11-16T16:46:17.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:16 smithi053 ceph-mon[100638]: pgmap v217: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 7 op/s 2023-11-16T16:46:17.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:16 smithi053 ceph-mon[100638]: osdmap e89: 12 total, 12 up, 12 in 2023-11-16T16:46:17.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:17 smithi001 ceph-mon[98800]: scrub queued for path: / 2023-11-16T16:46:17.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:17 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:46:17.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:17 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:46:17.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:17 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:46:18.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:17 smithi026 ceph-mon[100112]: scrub queued for path: / 2023-11-16T16:46:18.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:17 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:46:18.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:17 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:46:18.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:17 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:46:18.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:17 smithi053 ceph-mon[100638]: scrub queued for path: / 2023-11-16T16:46:18.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:17 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:46:18.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:17 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:46:18.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:17 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:46:18.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:18 smithi001 ceph-mon[98800]: pgmap v219: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 9 op/s 2023-11-16T16:46:19.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:18 smithi026 ceph-mon[100112]: pgmap v219: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 9 op/s 2023-11-16T16:46:19.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:18 smithi053 ceph-mon[100638]: pgmap v219: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 9 op/s 2023-11-16T16:46:19.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:19 smithi001 ceph-mon[98800]: scrub summary: idle 2023-11-16T16:46:19.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:19 smithi001 ceph-mon[98800]: pgmap v220: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 7 op/s 2023-11-16T16:46:20.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:19 smithi026 ceph-mon[100112]: scrub summary: idle 2023-11-16T16:46:20.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:19 smithi026 ceph-mon[100112]: pgmap v220: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 7 op/s 2023-11-16T16:46:20.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:19 smithi053 ceph-mon[100638]: scrub summary: idle 2023-11-16T16:46:20.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:19 smithi053 ceph-mon[100638]: pgmap v220: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 7 op/s 2023-11-16T16:46:22.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:22 smithi053 ceph-mon[100638]: pgmap v221: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.9 KiB/s wr, 7 op/s 2023-11-16T16:46:30.442 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:24 smithi053 ceph-mon[100638]: pgmap v222: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.9 KiB/s wr, 7 op/s 2023-11-16T16:46:30.442 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:26 smithi053 ceph-mon[100638]: pgmap v223: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2023-11-16T16:46:30.442 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:28 smithi053 ceph-mon[100638]: pgmap v224: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2023-11-16T16:46:30.442 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:22 smithi026 ceph-mon[100112]: pgmap v221: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.9 KiB/s wr, 7 op/s 2023-11-16T16:46:30.442 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:24 smithi026 ceph-mon[100112]: pgmap v222: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.9 KiB/s wr, 7 op/s 2023-11-16T16:46:30.443 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:26 smithi026 ceph-mon[100112]: pgmap v223: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2023-11-16T16:46:30.443 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:28 smithi026 ceph-mon[100112]: pgmap v224: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2023-11-16T16:46:30.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:22 smithi001 ceph-mon[98800]: pgmap v221: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.9 KiB/s wr, 7 op/s 2023-11-16T16:46:30.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:24 smithi001 ceph-mon[98800]: pgmap v222: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.9 KiB/s wr, 7 op/s 2023-11-16T16:46:30.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:26 smithi001 ceph-mon[98800]: pgmap v223: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2023-11-16T16:46:30.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:28 smithi001 ceph-mon[98800]: pgmap v224: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2023-11-16T16:46:30.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:30 smithi001 ceph-mon[98800]: pgmap v225: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.6 KiB/s wr, 5 op/s 2023-11-16T16:46:31.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:30 smithi026 ceph-mon[100112]: pgmap v225: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.6 KiB/s wr, 5 op/s 2023-11-16T16:46:31.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:30 smithi053 ceph-mon[100638]: pgmap v225: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.6 KiB/s wr, 5 op/s 2023-11-16T16:46:31.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:31 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:46:32.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:31 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:46:32.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:31 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:46:32.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:32 smithi001 ceph-mon[98800]: pgmap v226: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.6 KiB/s wr, 8 op/s 2023-11-16T16:46:32.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:32 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:32.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:32 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:46:32.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:32 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:46:33.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:32 smithi026 ceph-mon[100112]: pgmap v226: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.6 KiB/s wr, 8 op/s 2023-11-16T16:46:33.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:32 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:33.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:32 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:46:33.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:32 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:46:33.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:32 smithi053 ceph-mon[100638]: pgmap v226: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.6 KiB/s wr, 8 op/s 2023-11-16T16:46:33.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:32 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:46:33.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:32 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:46:33.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:32 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:46:33.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:33 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:46:33.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:33 smithi001 ceph-mon[98800]: Service mds.all has no related daemon already placed. 2023-11-16T16:46:33.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:33 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:46:33.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:33 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:46:33.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:33 smithi001 ceph-mon[98800]: pgmap v227: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.3 KiB/s wr, 5 op/s 2023-11-16T16:46:34.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:33 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:46:34.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:33 smithi026 ceph-mon[100112]: Service mds.all has no related daemon already placed. 2023-11-16T16:46:34.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:33 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:46:34.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:33 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:46:34.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:33 smithi026 ceph-mon[100112]: pgmap v227: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.3 KiB/s wr, 5 op/s 2023-11-16T16:46:34.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:33 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:46:34.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:33 smithi053 ceph-mon[100638]: Service mds.all has no related daemon already placed. 2023-11-16T16:46:34.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:33 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:46:34.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:33 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:46:34.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:33 smithi053 ceph-mon[100638]: pgmap v227: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.3 KiB/s wr, 5 op/s 2023-11-16T16:46:36.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:36 smithi001 ceph-mon[98800]: pgmap v228: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2023-11-16T16:46:37.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:36 smithi026 ceph-mon[100112]: pgmap v228: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2023-11-16T16:46:37.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:36 smithi053 ceph-mon[100638]: pgmap v228: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2023-11-16T16:46:38.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:38 smithi001 ceph-mon[98800]: pgmap v229: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:46:39.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:38 smithi026 ceph-mon[100112]: pgmap v229: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:46:39.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:38 smithi053 ceph-mon[100638]: pgmap v229: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:46:40.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:40 smithi001 ceph-mon[98800]: pgmap v230: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:46:41.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:40 smithi026 ceph-mon[100112]: pgmap v230: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:46:41.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:40 smithi053 ceph-mon[100638]: pgmap v230: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:46:42.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:42 smithi053 ceph-mon[100638]: pgmap v231: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:46:44.306 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:42 smithi026 ceph-mon[100112]: pgmap v231: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:46:44.307 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:42 smithi001 ceph-mon[98800]: pgmap v231: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:46:44.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:44 smithi001 ceph-mon[98800]: pgmap v232: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:46:45.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:44 smithi026 ceph-mon[100112]: pgmap v232: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:46:45.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:44 smithi053 ceph-mon[100638]: pgmap v232: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:46:46.590 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-16T16:46:46.734 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.732+0000 7fd7e0854700 1 -- 172.21.15.1:0/2556969675 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7dc0ff410 msgr2=0x7fd7dc0ff7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:46.735 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.732+0000 7fd7e0854700 1 --2- 172.21.15.1:0/2556969675 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7dc0ff410 0x7fd7dc0ff7e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0099c0 tx=0x7fd7cc009cd0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.735 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.732+0000 7fd7e0854700 1 -- 172.21.15.1:0/2556969675 shutdown_connections 2023-11-16T16:46:46.735 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.732+0000 7fd7e0854700 1 --2- 172.21.15.1:0/2556969675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7dc104450 0x7fd7dc10c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.735 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.732+0000 7fd7e0854700 1 --2- 172.21.15.1:0/2556969675 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7dc0ff410 0x7fd7dc0ff7e0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.735 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.732+0000 7fd7e0854700 1 --2- 172.21.15.1:0/2556969675 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7dc0ffdb0 0x7fd7dc103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.735 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.732+0000 7fd7e0854700 1 -- 172.21.15.1:0/2556969675 >> 172.21.15.1:0/2556969675 conn(0x7fd7dc076370 msgr2=0x7fd7dc076770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:46.736 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.732+0000 7fd7e0854700 1 -- 172.21.15.1:0/2556969675 shutdown_connections 2023-11-16T16:46:46.736 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.732+0000 7fd7e0854700 1 -- 172.21.15.1:0/2556969675 wait complete. 2023-11-16T16:46:46.738 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.733+0000 7fd7e0854700 1 Processor -- start 2023-11-16T16:46:46.738 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.733+0000 7fd7e0854700 1 -- start start 2023-11-16T16:46:46.738 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.733+0000 7fd7e0854700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7dc0ff410 0x7fd7dc1971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.738 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.733+0000 7fd7e0854700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7dc0ffdb0 0x7fd7dc197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.738 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.733+0000 7fd7e0854700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7dc104450 0x7fd7dc1a2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.738 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.733+0000 7fd7e0854700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7dc198060 con 0x7fd7dc0ffdb0 2023-11-16T16:46:46.739 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.733+0000 7fd7e0854700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7dc1a27a0 con 0x7fd7dc0ff410 2023-11-16T16:46:46.739 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.733+0000 7fd7e0854700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7dc1a2960 con 0x7fd7dc104450 2023-11-16T16:46:46.739 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.734+0000 7fd7d9d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7dc0ffdb0 0x7fd7dc197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.739 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.734+0000 7fd7d9d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7dc0ffdb0 0x7fd7dc197720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45876/0 (socket says 172.21.15.1:45876) 2023-11-16T16:46:46.739 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.734+0000 7fd7d9d9b700 1 -- 172.21.15.1:0/3409500670 learned_addr learned my addr 172.21.15.1:0/3409500670 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:46.739 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.734+0000 7fd7da59c700 1 --2- 172.21.15.1:0/3409500670 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7dc0ff410 0x7fd7dc1971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.739 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.734+0000 7fd7dad9d700 1 --2- 172.21.15.1:0/3409500670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7dc104450 0x7fd7dc1a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.740 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.734+0000 7fd7d9d9b700 1 -- 172.21.15.1:0/3409500670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7dc104450 msgr2=0x7fd7dc1a2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:46.740 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.734+0000 7fd7d9d9b700 1 --2- 172.21.15.1:0/3409500670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7dc104450 0x7fd7dc1a2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.740 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.734+0000 7fd7d9d9b700 1 -- 172.21.15.1:0/3409500670 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7dc0ff410 msgr2=0x7fd7dc1971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:46.740 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.734+0000 7fd7d9d9b700 1 --2- 172.21.15.1:0/3409500670 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7dc0ff410 0x7fd7dc1971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.740 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.734+0000 7fd7d9d9b700 1 -- 172.21.15.1:0/3409500670 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7cc0095f0 con 0x7fd7dc0ffdb0 2023-11-16T16:46:46.740 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.734+0000 7fd7d9d9b700 1 --2- 172.21.15.1:0/3409500670 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7dc0ffdb0 0x7fd7dc197720 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd7c400e940 tx=0x7fd7c400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:46.740 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.734+0000 7fd7d37fe700 1 -- 172.21.15.1:0/3409500670 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4009730 con 0x7fd7dc0ffdb0 2023-11-16T16:46:46.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.734+0000 7fd7d37fe700 1 -- 172.21.15.1:0/3409500670 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7c400ee90 con 0x7fd7dc0ffdb0 2023-11-16T16:46:46.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.734+0000 7fd7e0854700 1 -- 172.21.15.1:0/3409500670 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7dc10f260 con 0x7fd7dc0ffdb0 2023-11-16T16:46:46.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.734+0000 7fd7d37fe700 1 -- 172.21.15.1:0/3409500670 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4010ea0 con 0x7fd7dc0ffdb0 2023-11-16T16:46:46.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.734+0000 7fd7e0854700 1 -- 172.21.15.1:0/3409500670 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7dc10f0d0 con 0x7fd7dc0ffdb0 2023-11-16T16:46:46.747 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.737+0000 7fd7d37fe700 1 -- 172.21.15.1:0/3409500670 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fd7c400cbd0 con 0x7fd7dc0ffdb0 2023-11-16T16:46:46.747 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.738+0000 7fd7d37fe700 1 --2- 172.21.15.1:0/3409500670 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fd7c80a12c0 0x7fd7c80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.748 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.738+0000 7fd7da59c700 1 --2- 172.21.15.1:0/3409500670 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fd7c80a12c0 0x7fd7c80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.748 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.738+0000 7fd7d37fe700 1 -- 172.21.15.1:0/3409500670 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fd7c4014070 con 0x7fd7dc0ffdb0 2023-11-16T16:46:46.748 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.738+0000 7fd7da59c700 1 --2- 172.21.15.1:0/3409500670 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fd7c80a12c0 0x7fd7c80a3770 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc009990 tx=0x7fd7cc005fb0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:46.748 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.738+0000 7fd7ec9e0b80 1 Processor -- start 2023-11-16T16:46:46.748 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.739+0000 7fd7ec9e0b80 1 -- start start 2023-11-16T16:46:46.748 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.739+0000 7fd7ec9e0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e0c362f70 0x557e0c363340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.748 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.739+0000 7fd7ec9e0b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e0c363880 0x557e0c367780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.749 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.739+0000 7fd7ec9e0b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e0c367cc0 0x557e0c368110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.749 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.739+0000 7fd7ec9e0b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e0c264060 con 0x557e0c367cc0 2023-11-16T16:46:46.749 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.739+0000 7fd7ec9e0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e0c2b68a0 con 0x557e0c362f70 2023-11-16T16:46:46.749 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.739+0000 7fd7ec9e0b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e0c2afd30 con 0x557e0c363880 2023-11-16T16:46:46.749 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.739+0000 7fd7dad9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e0c367cc0 0x557e0c368110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.749 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.739+0000 7fd7dad9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e0c367cc0 0x557e0c368110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45886/0 (socket says 172.21.15.1:45886) 2023-11-16T16:46:46.749 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.739+0000 7fd7dad9d700 1 -- 172.21.15.1:0/1543690426 learned_addr learned my addr 172.21.15.1:0/1543690426 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:46.749 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.739+0000 7fd7da59c700 1 --2- 172.21.15.1:0/1543690426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e0c362f70 0x557e0c363340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.750 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.739+0000 7fd7d9d9b700 1 --2- 172.21.15.1:0/1543690426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e0c363880 0x557e0c367780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.750 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.739+0000 7fd7dad9d700 1 -- 172.21.15.1:0/1543690426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e0c363880 msgr2=0x557e0c367780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:46.750 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.739+0000 7fd7dad9d700 1 --2- 172.21.15.1:0/1543690426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e0c363880 0x557e0c367780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.750 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.739+0000 7fd7dad9d700 1 -- 172.21.15.1:0/1543690426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e0c362f70 msgr2=0x557e0c363340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:46.750 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.739+0000 7fd7dad9d700 1 --2- 172.21.15.1:0/1543690426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e0c362f70 0x557e0c363340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.750 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.739+0000 7fd7dad9d700 1 -- 172.21.15.1:0/1543690426 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e0c131630 con 0x557e0c367cc0 2023-11-16T16:46:46.750 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.739+0000 7fd7da59c700 1 --2- 172.21.15.1:0/1543690426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e0c362f70 0x557e0c363340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:46:46.751 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.739+0000 7fd7dad9d700 1 --2- 172.21.15.1:0/1543690426 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e0c367cc0 0x557e0c368110 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fd7d40084f0 tx=0x7fd7d4008800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7acfa7c842e3dfad server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:46.751 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.740+0000 7fd7d0ff9700 1 -- 172.21.15.1:0/1543690426 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d4003c00 con 0x557e0c367cc0 2023-11-16T16:46:46.751 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.740+0000 7fd7d0ff9700 1 -- 172.21.15.1:0/1543690426 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7d4010040 con 0x557e0c367cc0 2023-11-16T16:46:46.751 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.740+0000 7fd7d0ff9700 1 -- 172.21.15.1:0/1543690426 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d4012760 con 0x557e0c367cc0 2023-11-16T16:46:46.751 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.740+0000 7fd7ec9e0b80 1 -- 172.21.15.1:0/1543690426 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e0c367cc0 msgr2=0x557e0c368110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:46.751 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.740+0000 7fd7ec9e0b80 1 --2- 172.21.15.1:0/1543690426 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e0c367cc0 0x557e0c368110 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fd7d40084f0 tx=0x7fd7d4008800 comp rx=0 tx=0).stop 2023-11-16T16:46:46.751 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.740+0000 7fd7ec9e0b80 1 -- 172.21.15.1:0/1543690426 shutdown_connections 2023-11-16T16:46:46.751 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.740+0000 7fd7ec9e0b80 1 --2- 172.21.15.1:0/1543690426 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e0c367cc0 0x557e0c368110 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.752 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.740+0000 7fd7ec9e0b80 1 --2- 172.21.15.1:0/1543690426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e0c362f70 0x557e0c363340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.752 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.740+0000 7fd7ec9e0b80 1 --2- 172.21.15.1:0/1543690426 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e0c363880 0x557e0c367780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.752 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.740+0000 7fd7ec9e0b80 1 -- 172.21.15.1:0/1543690426 >> 172.21.15.1:0/1543690426 conn(0x557e0c354360 msgr2=0x557e0c359b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:46.752 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.741+0000 7fd7ec9e0b80 1 -- 172.21.15.1:0/1543690426 shutdown_connections 2023-11-16T16:46:46.752 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.741+0000 7fd7ec9e0b80 1 -- 172.21.15.1:0/1543690426 wait complete. 2023-11-16T16:46:46.752 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.741+0000 7fd7ec9e0b80 1 -- start start 2023-11-16T16:46:46.752 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.741+0000 7fd7ec9e0b80 1 Processor -- start 2023-11-16T16:46:46.752 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.741+0000 7fd7ec9e0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e0c362f70 0x557e0c36f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.753 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.741+0000 7fd7ec9e0b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e0c363880 0x557e0c36c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.753 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.741+0000 7fd7ec9e0b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e0c367cc0 0x557e0c36cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.753 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.741+0000 7fd7ec9e0b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e0c2518e0 con 0x557e0c363880 2023-11-16T16:46:46.753 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.741+0000 7fd7ec9e0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e0c251580 con 0x557e0c362f70 2023-11-16T16:46:46.753 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.741+0000 7fd7ec9e0b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e0c133920 con 0x557e0c367cc0 2023-11-16T16:46:46.753 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.741+0000 7fd7ec9e0b80 10 client.0 _ll_register_callbacks cb 0x557e0c26f780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:46:46.753 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.741+0000 7fd7d9d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e0c363880 0x557e0c36c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.753 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.741+0000 7fd7d9d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e0c363880 0x557e0c36c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45890/0 (socket says 172.21.15.1:45890) 2023-11-16T16:46:46.754 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.741+0000 7fd7d9d9b700 1 -- 172.21.15.1:0/2948577230 learned_addr learned my addr 172.21.15.1:0/2948577230 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:46.754 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.741+0000 7fd7da59c700 1 --2- 172.21.15.1:0/2948577230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e0c362f70 0x557e0c36f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.754 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.741+0000 7fd7dad9d700 1 --2- 172.21.15.1:0/2948577230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e0c367cc0 0x557e0c36cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.754 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.742+0000 7fd7d9d9b700 1 -- 172.21.15.1:0/2948577230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e0c367cc0 msgr2=0x557e0c36cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:46.754 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.742+0000 7fd7d9d9b700 1 --2- 172.21.15.1:0/2948577230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e0c367cc0 0x557e0c36cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.754 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.742+0000 7fd7d9d9b700 1 -- 172.21.15.1:0/2948577230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e0c362f70 msgr2=0x557e0c36f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:46.754 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.742+0000 7fd7d9d9b700 1 --2- 172.21.15.1:0/2948577230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e0c362f70 0x557e0c36f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.755 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.742+0000 7fd7d9d9b700 1 -- 172.21.15.1:0/2948577230 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d4007d10 con 0x557e0c363880 2023-11-16T16:46:46.755 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.742+0000 7fd7dad9d700 1 --2- 172.21.15.1:0/2948577230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e0c367cc0 0x557e0c36cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:46:46.755 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.742+0000 7fd7da59c700 1 --2- 172.21.15.1:0/2948577230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e0c362f70 0x557e0c36f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:46:46.755 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.742+0000 7fd7d9d9b700 1 --2- 172.21.15.1:0/2948577230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e0c363880 0x557e0c36c610 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4099fc0 tx=0x7fd7c409a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a32b035430a731 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:46.755 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.742+0000 7fd7c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:46:46.755 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.742+0000 7fd7c1ffb700 1 -- 172.21.15.1:0/2948577230 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c40b1070 con 0x557e0c363880 2023-11-16T16:46:46.755 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.742+0000 7fd7c1ffb700 1 -- 172.21.15.1:0/2948577230 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fd7c40a2480 con 0x557e0c363880 2023-11-16T16:46:46.755 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.742+0000 7fd7ec9e0b80 1 -- 172.21.15.1:0/2948577230 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e0c351b20 con 0x557e0c363880 2023-11-16T16:46:46.756 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.742+0000 7fd7c1ffb700 1 -- 172.21.15.1:0/2948577230 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7c4099730 con 0x557e0c363880 2023-11-16T16:46:46.756 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.742+0000 7fd7ec9e0b80 10 client.16434 fetch_fsmap learned FSMap version 28 2023-11-16T16:46:46.756 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.742+0000 7fd7ec9e0b80 1 -- 172.21.15.1:0/2948577230 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557e0c276190 con 0x557e0c363880 2023-11-16T16:46:46.756 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.742+0000 7fd7c1ffb700 1 -- 172.21.15.1:0/2948577230 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7c40c8080 con 0x557e0c363880 2023-11-16T16:46:46.756 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.743+0000 7fd7c1ffb700 1 -- 172.21.15.1:0/2948577230 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd7c4098f90 con 0x557e0c363880 2023-11-16T16:46:46.756 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.743+0000 7fd7ec9e0b80 10 client.16434 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:46:46.756 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.743+0000 7fd7ec9e0b80 10 client.16434 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:46:46.756 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.743+0000 7fd7ec9e0b80 20 client.16434 populate_metadata read hostname 'smithi001' 2023-11-16T16:46:46.757 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.743+0000 7fd7ec9e0b80 1 --2- 172.21.15.1:0/2948577230 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x557e0c37cc30 0x557e0c37f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.757 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.743+0000 7fd7ec9e0b80 4 client.16434 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:46:46.757 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.743+0000 7fd7ec9e0b80 1 -- 172.21.15.1:0/2948577230 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e0c2720a0 con 0x557e0c37cc30 2023-11-16T16:46:46.757 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.743+0000 7fd7da59c700 1 --2- 172.21.15.1:0/2948577230 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x557e0c37cc30 0x557e0c37f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.757 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.743+0000 7fd7c1ffb700 10 client.16434 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:46:46.757 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.743+0000 7fd7da59c700 1 --2- 172.21.15.1:0/2948577230 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x557e0c37cc30 0x557e0c37f040 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=457fb71d71a054ea server_cookie=b570980d105d620f in_seq=0 out_seq=0 2023-11-16T16:46:46.757 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.744+0000 7fd7c1ffb700 1 -- 172.21.15.1:0/2948577230 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557e0c2720a0 con 0x557e0c37cc30 2023-11-16T16:46:46.758 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.744+0000 7fd7c1ffb700 10 client.16434 handle_command_reply: tid=0 2023-11-16T16:46:46.758 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.744+0000 7fd7ec9e0b80 1 client.16434 shutdown 2023-11-16T16:46:46.758 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.744+0000 7fd7ec9e0b80 1 -- 172.21.15.1:0/2948577230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e0c363880 msgr2=0x557e0c36c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:46.758 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.744+0000 7fd7ec9e0b80 1 --2- 172.21.15.1:0/2948577230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e0c363880 0x557e0c36c610 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4099fc0 tx=0x7fd7c409a160 comp rx=0 tx=0).stop 2023-11-16T16:46:46.758 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.744+0000 7fd7ec9e0b80 1 -- 172.21.15.1:0/2948577230 shutdown_connections 2023-11-16T16:46:46.758 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.744+0000 7fd7ec9e0b80 1 --2- 172.21.15.1:0/2948577230 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x557e0c37cc30 0x557e0c37f040 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.758 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.744+0000 7fd7ec9e0b80 1 --2- 172.21.15.1:0/2948577230 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e0c367cc0 0x557e0c36cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.758 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.744+0000 7fd7ec9e0b80 1 --2- 172.21.15.1:0/2948577230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e0c362f70 0x557e0c36f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.759 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.744+0000 7fd7c1ffb700 0 client.16434 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:46:46.759 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.744+0000 7fd7ec9e0b80 1 --2- 172.21.15.1:0/2948577230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e0c363880 0x557e0c36c610 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.759 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.744+0000 7fd7ec9e0b80 1 -- 172.21.15.1:0/2948577230 >> 172.21.15.1:0/2948577230 conn(0x557e0c354360 msgr2=0x557e0c369170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:46.759 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.744+0000 7fd7ec9e0b80 1 -- 172.21.15.1:0/2948577230 shutdown_connections 2023-11-16T16:46:46.759 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.744+0000 7fd7ec9e0b80 1 -- 172.21.15.1:0/2948577230 wait complete. 2023-11-16T16:46:46.759 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.745+0000 7fd7ec9e0b80 20 client.16434 trim_cache size 0 max 16384 2023-11-16T16:46:46.768 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.762+0000 7fd7ec9e0b80 1 Processor -- start 2023-11-16T16:46:46.769 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.762+0000 7fd7ec9e0b80 1 -- start start 2023-11-16T16:46:46.769 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.762+0000 7fd7ec9e0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e0c362f70 0x557e0c373de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.769 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.762+0000 7fd7ec9e0b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e0c367cc0 0x557e0c36ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.769 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.762+0000 7fd7ec9e0b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e0c3655b0 0x557e0c359560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.769 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.762+0000 7fd7ec9e0b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e0c36fba0 con 0x557e0c3655b0 2023-11-16T16:46:46.769 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.762+0000 7fd7ec9e0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e0c363fd0 con 0x557e0c362f70 2023-11-16T16:46:46.769 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.762+0000 7fd7ec9e0b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e0c366590 con 0x557e0c367cc0 2023-11-16T16:46:46.770 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.762+0000 7fd7dad9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e0c3655b0 0x557e0c359560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.770 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.762+0000 7fd7dad9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e0c3655b0 0x557e0c359560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45900/0 (socket says 172.21.15.1:45900) 2023-11-16T16:46:46.770 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.762+0000 7fd7dad9d700 1 -- 172.21.15.1:0/1709083278 learned_addr learned my addr 172.21.15.1:0/1709083278 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:46.770 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.762+0000 7fd7da59c700 1 --2- 172.21.15.1:0/1709083278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e0c362f70 0x557e0c373de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.770 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.762+0000 7fd7d9d9b700 1 --2- 172.21.15.1:0/1709083278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e0c367cc0 0x557e0c36ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.770 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.762+0000 7fd7dad9d700 1 -- 172.21.15.1:0/1709083278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e0c367cc0 msgr2=0x557e0c36ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:46.770 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.762+0000 7fd7dad9d700 1 --2- 172.21.15.1:0/1709083278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e0c367cc0 0x557e0c36ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.770 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.762+0000 7fd7dad9d700 1 -- 172.21.15.1:0/1709083278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e0c362f70 msgr2=0x557e0c373de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:46.771 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.762+0000 7fd7dad9d700 1 --2- 172.21.15.1:0/1709083278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e0c362f70 0x557e0c373de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.771 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.762+0000 7fd7dad9d700 1 -- 172.21.15.1:0/1709083278 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e0c275ee0 con 0x557e0c3655b0 2023-11-16T16:46:46.771 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.762+0000 7fd7dad9d700 1 --2- 172.21.15.1:0/1709083278 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e0c3655b0 0x557e0c359560 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4016040 tx=0x7fd7d40060c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c3f4eda6fca8e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:46.771 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.763+0000 7fd7d17fa700 1 -- 172.21.15.1:0/1709083278 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d40047e0 con 0x557e0c3655b0 2023-11-16T16:46:46.771 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.763+0000 7fd7d17fa700 1 -- 172.21.15.1:0/1709083278 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7d40129c0 con 0x557e0c3655b0 2023-11-16T16:46:46.771 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.763+0000 7fd7d17fa700 1 -- 172.21.15.1:0/1709083278 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d401f7e0 con 0x557e0c3655b0 2023-11-16T16:46:46.771 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.763+0000 7fd7ec9e0b80 1 -- 172.21.15.1:0/1709083278 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e0c3655b0 msgr2=0x557e0c359560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:46.771 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.763+0000 7fd7ec9e0b80 1 --2- 172.21.15.1:0/1709083278 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e0c3655b0 0x557e0c359560 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fd7d4016040 tx=0x7fd7d40060c0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.772 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.763+0000 7fd7ec9e0b80 1 -- 172.21.15.1:0/1709083278 shutdown_connections 2023-11-16T16:46:46.772 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.763+0000 7fd7ec9e0b80 1 --2- 172.21.15.1:0/1709083278 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e0c3655b0 0x557e0c359560 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.772 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.763+0000 7fd7ec9e0b80 1 --2- 172.21.15.1:0/1709083278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e0c362f70 0x557e0c373de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.772 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.763+0000 7fd7ec9e0b80 1 --2- 172.21.15.1:0/1709083278 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e0c367cc0 0x557e0c36ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.772 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.763+0000 7fd7ec9e0b80 1 -- 172.21.15.1:0/1709083278 >> 172.21.15.1:0/1709083278 conn(0x557e0c354360 msgr2=0x557e0c378fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:46.772 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.763+0000 7fd7ec9e0b80 1 -- 172.21.15.1:0/1709083278 shutdown_connections 2023-11-16T16:46:46.772 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.763+0000 7fd7ec9e0b80 1 -- 172.21.15.1:0/1709083278 wait complete. 2023-11-16T16:46:46.772 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.763+0000 7fd7ec9e0b80 1 -- start start 2023-11-16T16:46:46.773 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.764+0000 7fd7ec9e0b80 1 Processor -- start 2023-11-16T16:46:46.773 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.764+0000 7fd7ec9e0b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e0c362f70 0x557e0c3703c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.773 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.764+0000 7fd7ec9e0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e0c3655b0 0x557e0c370900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.773 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.764+0000 7fd7ec9e0b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e0c367cc0 0x557e0c370e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.773 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.764+0000 7fd7ec9e0b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e0c374430 con 0x557e0c362f70 2023-11-16T16:46:46.773 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.764+0000 7fd7ec9e0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e0c3d0ab0 con 0x557e0c3655b0 2023-11-16T16:46:46.773 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.764+0000 7fd7ec9e0b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e0c3d0c40 con 0x557e0c367cc0 2023-11-16T16:46:46.774 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.764+0000 7fd7ec9e0b80 10 client.0 _ll_register_callbacks cb 0x7fd7c4099b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:46:46.774 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.764+0000 7fd7da59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e0c362f70 0x557e0c3703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.774 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.764+0000 7fd7da59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e0c362f70 0x557e0c3703c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45902/0 (socket says 172.21.15.1:45902) 2023-11-16T16:46:46.774 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.764+0000 7fd7da59c700 1 -- 172.21.15.1:0/3478225586 learned_addr learned my addr 172.21.15.1:0/3478225586 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:46.774 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.764+0000 7fd7dad9d700 1 --2- 172.21.15.1:0/3478225586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e0c367cc0 0x557e0c370e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.774 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.764+0000 7fd7d9d9b700 1 --2- 172.21.15.1:0/3478225586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e0c3655b0 0x557e0c370900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.774 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.764+0000 7fd7da59c700 1 -- 172.21.15.1:0/3478225586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e0c367cc0 msgr2=0x557e0c370e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:46.774 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.764+0000 7fd7da59c700 1 --2- 172.21.15.1:0/3478225586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e0c367cc0 0x557e0c370e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.775 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.764+0000 7fd7da59c700 1 -- 172.21.15.1:0/3478225586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e0c3655b0 msgr2=0x557e0c370900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:46.775 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.764+0000 7fd7da59c700 1 --2- 172.21.15.1:0/3478225586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e0c3655b0 0x557e0c370900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.775 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.764+0000 7fd7da59c700 1 -- 172.21.15.1:0/3478225586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557e0c131630 con 0x557e0c362f70 2023-11-16T16:46:46.775 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.764+0000 7fd7da59c700 1 --2- 172.21.15.1:0/3478225586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e0c362f70 0x557e0c3703c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc000c00 tx=0x7fd7cc009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9ac8c6ab7bec7b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:46.775 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.764+0000 7fd7c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:46:46.775 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.765+0000 7fd7c1ffb700 1 -- 172.21.15.1:0/3478225586 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc02c040 con 0x557e0c362f70 2023-11-16T16:46:46.775 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.765+0000 7fd7c1ffb700 1 -- 172.21.15.1:0/3478225586 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fd7cc00baf0 con 0x557e0c362f70 2023-11-16T16:46:46.775 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.765+0000 7fd7ec9e0b80 1 -- 172.21.15.1:0/3478225586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7d4010040 con 0x557e0c362f70 2023-11-16T16:46:46.775 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.765+0000 7fd7c1ffb700 1 -- 172.21.15.1:0/3478225586 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7cc00bcb0 con 0x557e0c362f70 2023-11-16T16:46:46.776 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.765+0000 7fd7ec9e0b80 10 client.16440 fetch_fsmap learned FSMap version 28 2023-11-16T16:46:46.776 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.765+0000 7fd7ec9e0b80 1 -- 172.21.15.1:0/3478225586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557e0c0c6c60 con 0x557e0c362f70 2023-11-16T16:46:46.776 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.765+0000 7fd7c1ffb700 1 -- 172.21.15.1:0/3478225586 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7d4010040 con 0x557e0c362f70 2023-11-16T16:46:46.776 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.765+0000 7fd7c1ffb700 1 -- 172.21.15.1:0/3478225586 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd7cc037020 con 0x557e0c362f70 2023-11-16T16:46:46.776 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.765+0000 7fd7ec9e0b80 10 client.16440 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:46:46.776 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.765+0000 7fd7ec9e0b80 10 client.16440 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:46:46.776 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.765+0000 7fd7ec9e0b80 20 client.16440 populate_metadata read hostname 'smithi001' 2023-11-16T16:46:46.776 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.765+0000 7fd7ec9e0b80 1 --2- 172.21.15.1:0/3478225586 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x557e0c3d2210 0x557e0c3d4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.777 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.765+0000 7fd7ec9e0b80 4 client.16440 mds_command: new command op to 24521 tid=0[{"prefix": "scrub status"}] 2023-11-16T16:46:46.777 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.765+0000 7fd7ec9e0b80 1 -- 172.21.15.1:0/3478225586 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557e0c27e660 con 0x557e0c3d2210 2023-11-16T16:46:46.777 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.765+0000 7fd7d9d9b700 1 --2- 172.21.15.1:0/3478225586 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x557e0c3d2210 0x557e0c3d4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.777 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.766+0000 7fd7c1ffb700 10 client.16440 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:46:46.777 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.766+0000 7fd7d9d9b700 1 --2- 172.21.15.1:0/3478225586 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x557e0c3d2210 0x557e0c3d4620 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=498919446990ad54 server_cookie=4fdc2070ba3c4a88 in_seq=0 out_seq=0 2023-11-16T16:46:46.777 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.766+0000 7fd7c1ffb700 1 -- 172.21.15.1:0/3478225586 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557e0c27e660 con 0x557e0c3d2210 2023-11-16T16:46:46.777 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.766+0000 7fd7c1ffb700 10 client.16440 handle_command_reply: tid=0 2023-11-16T16:46:46.777 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.766+0000 7fd7ec9e0b80 1 client.16440 shutdown 2023-11-16T16:46:46.778 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.766+0000 7fd7ec9e0b80 1 -- 172.21.15.1:0/3478225586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e0c362f70 msgr2=0x557e0c3703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:46.778 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.766+0000 7fd7ec9e0b80 1 --2- 172.21.15.1:0/3478225586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e0c362f70 0x557e0c3703c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc000c00 tx=0x7fd7cc009cd0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.778 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.766+0000 7fd7ec9e0b80 1 -- 172.21.15.1:0/3478225586 shutdown_connections 2023-11-16T16:46:46.778 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.766+0000 7fd7ec9e0b80 1 --2- 172.21.15.1:0/3478225586 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x557e0c3d2210 0x557e0c3d4620 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.778 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.766+0000 7fd7ec9e0b80 1 --2- 172.21.15.1:0/3478225586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x557e0c367cc0 0x557e0c370e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.778 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.766+0000 7fd7c1ffb700 0 client.16440 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:46:46.778 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.766+0000 7fd7ec9e0b80 1 --2- 172.21.15.1:0/3478225586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e0c362f70 0x557e0c3703c0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.778 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.766+0000 7fd7ec9e0b80 1 --2- 172.21.15.1:0/3478225586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e0c3655b0 0x557e0c370900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.779 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.766+0000 7fd7ec9e0b80 1 -- 172.21.15.1:0/3478225586 >> 172.21.15.1:0/3478225586 conn(0x557e0c354360 msgr2=0x557e0c36ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:46.779 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.766+0000 7fd7ec9e0b80 1 -- 172.21.15.1:0/3478225586 shutdown_connections 2023-11-16T16:46:46.779 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.766+0000 7fd7ec9e0b80 1 -- 172.21.15.1:0/3478225586 wait complete. 2023-11-16T16:46:46.779 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.766+0000 7fd7ec9e0b80 20 client.16440 trim_cache size 0 max 16384 2023-11-16T16:46:46.779 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:46:46.780 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2023-11-16T16:46:46.780 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2023-11-16T16:46:46.780 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:46:46.781 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.767+0000 7fd7e0854700 1 -- 172.21.15.1:0/3409500670 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fd7c80a12c0 msgr2=0x7fd7c80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:46.781 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.767+0000 7fd7e0854700 1 --2- 172.21.15.1:0/3409500670 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fd7c80a12c0 0x7fd7c80a3770 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc009990 tx=0x7fd7cc005fb0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.781 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.767+0000 7fd7e0854700 1 -- 172.21.15.1:0/3409500670 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7dc0ffdb0 msgr2=0x7fd7dc197720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:46.782 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.767+0000 7fd7e0854700 1 --2- 172.21.15.1:0/3409500670 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7dc0ffdb0 0x7fd7dc197720 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd7c400e940 tx=0x7fd7c400ec50 comp rx=0 tx=0).stop 2023-11-16T16:46:46.782 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.767+0000 7fd7e0854700 1 -- 172.21.15.1:0/3409500670 shutdown_connections 2023-11-16T16:46:46.782 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.767+0000 7fd7e0854700 1 --2- 172.21.15.1:0/3409500670 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fd7c80a12c0 0x7fd7c80a3770 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.782 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.767+0000 7fd7e0854700 1 --2- 172.21.15.1:0/3409500670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd7dc104450 0x7fd7dc1a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.782 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.767+0000 7fd7e0854700 1 --2- 172.21.15.1:0/3409500670 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7dc0ff410 0x7fd7dc1971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.782 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.767+0000 7fd7e0854700 1 --2- 172.21.15.1:0/3409500670 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd7dc0ffdb0 0x7fd7dc197720 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.782 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.767+0000 7fd7e0854700 1 -- 172.21.15.1:0/3409500670 >> 172.21.15.1:0/3409500670 conn(0x7fd7dc076370 msgr2=0x7fd7dc0fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:46.782 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.767+0000 7fd7e0854700 1 -- 172.21.15.1:0/3409500670 shutdown_connections 2023-11-16T16:46:46.783 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.767+0000 7fd7e0854700 1 -- 172.21.15.1:0/3409500670 wait complete. 2023-11-16T16:46:46.783 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-16T16:46:46.783 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:46:46.909 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.907+0000 7f90f66e0700 1 -- 172.21.15.1:0/4259661466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90f01044a0 msgr2=0x7f90f010c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:46.909 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.907+0000 7f90f66e0700 1 --2- 172.21.15.1:0/4259661466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90f01044a0 0x7f90f010c070 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f90e800b0d0 tx=0x7f90e800b3e0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.909 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.907+0000 7f90f66e0700 1 -- 172.21.15.1:0/4259661466 shutdown_connections 2023-11-16T16:46:46.910 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.907+0000 7f90f66e0700 1 --2- 172.21.15.1:0/4259661466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90f01044a0 0x7f90f010c070 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.910 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.907+0000 7f90f66e0700 1 --2- 172.21.15.1:0/4259661466 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90f00ff460 0x7f90f00ff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.910 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.907+0000 7f90f66e0700 1 --2- 172.21.15.1:0/4259661466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90f00ffe00 0x7f90f0103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.910 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.907+0000 7f90f66e0700 1 -- 172.21.15.1:0/4259661466 >> 172.21.15.1:0/4259661466 conn(0x7f90f0076310 msgr2=0x7f90f0076710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:46.910 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.907+0000 7f90f66e0700 1 -- 172.21.15.1:0/4259661466 shutdown_connections 2023-11-16T16:46:46.910 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.907+0000 7f90f66e0700 1 -- 172.21.15.1:0/4259661466 wait complete. 2023-11-16T16:46:46.910 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.907+0000 7f90f66e0700 1 Processor -- start 2023-11-16T16:46:46.910 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.908+0000 7f90f66e0700 1 -- start start 2023-11-16T16:46:46.912 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.908+0000 7f90f66e0700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90f00ff460 0x7f90f01964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.913 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.908+0000 7f90f66e0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90f00ffe00 0x7f90f0196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.913 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.908+0000 7f90f66e0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90f01044a0 0x7f90f019ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.913 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.908+0000 7f90f66e0700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f019b4d0 con 0x7f90f01044a0 2023-11-16T16:46:46.913 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.908+0000 7f90f66e0700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f019b640 con 0x7f90f00ffe00 2023-11-16T16:46:46.913 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.908+0000 7f90f66e0700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f019b800 con 0x7f90f00ff460 2023-11-16T16:46:46.913 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.908+0000 7f90f4c7d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90f01044a0 0x7f90f019ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.914 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.908+0000 7f90f4c7d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90f01044a0 0x7f90f019ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45920/0 (socket says 172.21.15.1:45920) 2023-11-16T16:46:46.914 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.908+0000 7f90f4c7d700 1 -- 172.21.15.1:0/2402801168 learned_addr learned my addr 172.21.15.1:0/2402801168 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:46.914 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.908+0000 7f90ef7fe700 1 --2- 172.21.15.1:0/2402801168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90f00ffe00 0x7f90f0196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.914 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.908+0000 7f90effff700 1 --2- 172.21.15.1:0/2402801168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90f00ff460 0x7f90f01964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.914 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.908+0000 7f90f4c7d700 1 -- 172.21.15.1:0/2402801168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90f00ff460 msgr2=0x7f90f01964f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:46.914 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.908+0000 7f90f4c7d700 1 --2- 172.21.15.1:0/2402801168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90f00ff460 0x7f90f01964f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.914 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.908+0000 7f90f4c7d700 1 -- 172.21.15.1:0/2402801168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90f00ffe00 msgr2=0x7f90f0196a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:46.914 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.908+0000 7f90f4c7d700 1 --2- 172.21.15.1:0/2402801168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90f00ffe00 0x7f90f0196a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:46.915 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.908+0000 7f90f4c7d700 1 -- 172.21.15.1:0/2402801168 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e8009d00 con 0x7f90f01044a0 2023-11-16T16:46:46.915 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.908+0000 7f90ef7fe700 1 --2- 172.21.15.1:0/2402801168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90f00ffe00 0x7f90f0196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:46:46.915 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.909+0000 7f90f4c7d700 1 --2- 172.21.15.1:0/2402801168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90f01044a0 0x7f90f019ae90 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f90e8007870 tx=0x7f90e8007950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:46.915 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.909+0000 7f90ed7fa700 1 -- 172.21.15.1:0/2402801168 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e800e040 con 0x7f90f01044a0 2023-11-16T16:46:46.915 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.909+0000 7f90f66e0700 1 -- 172.21.15.1:0/2402801168 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90f010f270 con 0x7f90f01044a0 2023-11-16T16:46:46.915 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.909+0000 7f90ed7fa700 1 -- 172.21.15.1:0/2402801168 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90e800be30 con 0x7f90f01044a0 2023-11-16T16:46:46.915 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.909+0000 7f90ed7fa700 1 -- 172.21.15.1:0/2402801168 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e801b6a0 con 0x7f90f01044a0 2023-11-16T16:46:46.916 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.909+0000 7f90f66e0700 1 -- 172.21.15.1:0/2402801168 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90f010f0e0 con 0x7f90f01044a0 2023-11-16T16:46:46.916 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.910+0000 7f90f66e0700 1 -- 172.21.15.1:0/2402801168 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90f010f400 con 0x7f90f01044a0 2023-11-16T16:46:46.916 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.911+0000 7f90ed7fa700 1 -- 172.21.15.1:0/2402801168 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f90e8005080 con 0x7f90f01044a0 2023-11-16T16:46:46.917 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.911+0000 7f90ed7fa700 1 --2- 172.21.15.1:0/2402801168 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f90dc0a12c0 0x7f90dc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:46.917 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.912+0000 7f90effff700 1 --2- 172.21.15.1:0/2402801168 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f90dc0a12c0 0x7f90dc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:46.917 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.912+0000 7f90ed7fa700 1 -- 172.21.15.1:0/2402801168 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f90e80ce3d0 con 0x7f90f01044a0 2023-11-16T16:46:46.917 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.912+0000 7f90effff700 1 --2- 172.21.15.1:0/2402801168 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f90dc0a12c0 0x7f90dc0a3770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f90e0009bf0 tx=0x7f90e0009260 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:46.917 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:46.915+0000 7f90ed7fa700 1 -- 172.21.15.1:0/2402801168 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f90e8097800 con 0x7f90f01044a0 2023-11-16T16:46:46.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:46 smithi001 ceph-mon[98800]: pgmap v233: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:46:47.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:46 smithi026 ceph-mon[100112]: pgmap v233: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:46:47.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:46 smithi053 ceph-mon[100638]: pgmap v233: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:46:47.163 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.162+0000 7f90f66e0700 1 -- 172.21.15.1:0/2402801168 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f90f0067260 con 0x7f90f01044a0 2023-11-16T16:46:47.164 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.163+0000 7f90ed7fa700 1 -- 172.21.15.1:0/2402801168 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10783 (secure 0 0 0) 0x7f90e8011390 con 0x7f90f01044a0 2023-11-16T16:46:47.164 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:46:47.165 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2587563684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2587563684},{"type":"v1","addr":"172.21.15.1:6835","nonce":2587563684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/277936168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":277936168},{"type":"v1","addr":"172.21.15.1:6837","nonce":277936168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/1172575045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1172575045},{"type":"v1","addr":"172.21.15.1:6839","nonce":1172575045}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/2631175899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2631175899},{"type":"v1","addr":"172.21.15.1:6841","nonce":2631175899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3693905267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3693905267},{"type":"v1","addr":"172.21.15.53:6833","nonce":3693905267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/164668033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":164668033},{"type":"v1","addr":"172.21.15.26:6833","nonce":164668033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-16T16:37:13.918943+0000","modified":"2023-11-16T16:43:40.628525+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2140542192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2140542192},{"type":"v1","addr":"172.21.15.53:6835","nonce":2140542192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1010760275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1010760275},{"type":"v1","addr":"172.21.15.26:6835","nonce":1010760275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3197060302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3197060302},{"type":"v1","addr":"172.21.15.53:6837","nonce":3197060302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6837/1453504863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1453504863},{"type":"v1","addr":"172.21.15.26:6837","nonce":1453504863}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.53:6839/1044753024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1044753024},{"type":"v1","addr":"172.21.15.53:6839","nonce":1044753024}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6839/2192205949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2192205949},{"type":"v1","addr":"172.21.15.26:6839","nonce":2192205949}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-16T16:46:47.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.165+0000 7f90f66e0700 1 -- 172.21.15.1:0/2402801168 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f90dc0a12c0 msgr2=0x7f90dc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:47.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.165+0000 7f90f66e0700 1 --2- 172.21.15.1:0/2402801168 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f90dc0a12c0 0x7f90dc0a3770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f90e0009bf0 tx=0x7f90e0009260 comp rx=0 tx=0).stop 2023-11-16T16:46:47.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.165+0000 7f90f66e0700 1 -- 172.21.15.1:0/2402801168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90f01044a0 msgr2=0x7f90f019ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:47.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.165+0000 7f90f66e0700 1 --2- 172.21.15.1:0/2402801168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90f01044a0 0x7f90f019ae90 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f90e8007870 tx=0x7f90e8007950 comp rx=0 tx=0).stop 2023-11-16T16:46:47.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.165+0000 7f90f66e0700 1 -- 172.21.15.1:0/2402801168 shutdown_connections 2023-11-16T16:46:47.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.165+0000 7f90f66e0700 1 --2- 172.21.15.1:0/2402801168 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f90dc0a12c0 0x7f90dc0a3770 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.168 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.165+0000 7f90f66e0700 1 --2- 172.21.15.1:0/2402801168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90f01044a0 0x7f90f019ae90 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.168 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.165+0000 7f90f66e0700 1 --2- 172.21.15.1:0/2402801168 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f90f00ff460 0x7f90f01964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.168 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.165+0000 7f90f66e0700 1 --2- 172.21.15.1:0/2402801168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90f00ffe00 0x7f90f0196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.168 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.165+0000 7f90f66e0700 1 -- 172.21.15.1:0/2402801168 >> 172.21.15.1:0/2402801168 conn(0x7f90f0076310 msgr2=0x7f90f0068db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:47.168 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.165+0000 7f90f66e0700 1 -- 172.21.15.1:0/2402801168 shutdown_connections 2023-11-16T16:46:47.168 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.165+0000 7f90f66e0700 1 -- 172.21.15.1:0/2402801168 wait complete. 2023-11-16T16:46:47.169 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 28 2023-11-16T16:46:47.179 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-11-16T16:46:47.308 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.304+0000 7fb9a2fd1700 1 -- 172.21.15.1:0/3461563736 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb99c0ff8f0 msgr2=0x7fb99c0ffd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:47.308 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.304+0000 7fb9a2fd1700 1 --2- 172.21.15.1:0/3461563736 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb99c0ff8f0 0x7fb99c0ffd60 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb9880099c0 tx=0x7fb988009cd0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.309 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.304+0000 7fb9a2fd1700 1 -- 172.21.15.1:0/3461563736 shutdown_connections 2023-11-16T16:46:47.309 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.304+0000 7fb9a2fd1700 1 --2- 172.21.15.1:0/3461563736 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb99c1002a0 0x7fb99c10bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.309 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.304+0000 7fb9a2fd1700 1 --2- 172.21.15.1:0/3461563736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb99c104140 0x7fb99c104510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.309 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.304+0000 7fb9a2fd1700 1 --2- 172.21.15.1:0/3461563736 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb99c0ff8f0 0x7fb99c0ffd60 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.309 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.304+0000 7fb9a2fd1700 1 -- 172.21.15.1:0/3461563736 >> 172.21.15.1:0/3461563736 conn(0x7fb99c075880 msgr2=0x7fb99c075c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:47.309 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.304+0000 7fb9a2fd1700 1 -- 172.21.15.1:0/3461563736 shutdown_connections 2023-11-16T16:46:47.309 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.304+0000 7fb9a2fd1700 1 -- 172.21.15.1:0/3461563736 wait complete. 2023-11-16T16:46:47.310 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.305+0000 7fb9a2fd1700 1 Processor -- start 2023-11-16T16:46:47.310 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.305+0000 7fb9a2fd1700 1 -- start start 2023-11-16T16:46:47.310 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.305+0000 7fb9a2fd1700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb99c0ff8f0 0x7fb99c1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.310 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.305+0000 7fb9a2fd1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb99c1002a0 0x7fb99c198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.310 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.305+0000 7fb9a2fd1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb99c104140 0x7fb99c19ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.310 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.305+0000 7fb9a2fd1700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c19d4b0 con 0x7fb99c1002a0 2023-11-16T16:46:47.310 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.305+0000 7fb9a2fd1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c19d620 con 0x7fb99c104140 2023-11-16T16:46:47.310 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.305+0000 7fb9a2fd1700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c19d7e0 con 0x7fb99c0ff8f0 2023-11-16T16:46:47.311 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.305+0000 7fb99bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb99c1002a0 0x7fb99c198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.311 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.305+0000 7fb99bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb99c1002a0 0x7fb99c198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45932/0 (socket says 172.21.15.1:45932) 2023-11-16T16:46:47.311 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.305+0000 7fb99bfff700 1 -- 172.21.15.1:0/4234105630 learned_addr learned my addr 172.21.15.1:0/4234105630 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:47.311 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.305+0000 7fb9a0d6d700 1 --2- 172.21.15.1:0/4234105630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb99c0ff8f0 0x7fb99c1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.311 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.306+0000 7fb9a156e700 1 --2- 172.21.15.1:0/4234105630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb99c104140 0x7fb99c19ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.311 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.306+0000 7fb99bfff700 1 -- 172.21.15.1:0/4234105630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb99c0ff8f0 msgr2=0x7fb99c1984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:47.311 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.306+0000 7fb99bfff700 1 --2- 172.21.15.1:0/4234105630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb99c0ff8f0 0x7fb99c1984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.306+0000 7fb99bfff700 1 -- 172.21.15.1:0/4234105630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb99c104140 msgr2=0x7fb99c19ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:47.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.306+0000 7fb99bfff700 1 --2- 172.21.15.1:0/4234105630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb99c104140 0x7fb99c19ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.306+0000 7fb99bfff700 1 -- 172.21.15.1:0/4234105630 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9880095f0 con 0x7fb99c1002a0 2023-11-16T16:46:47.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.306+0000 7fb9a156e700 1 --2- 172.21.15.1:0/4234105630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb99c104140 0x7fb99c19ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:46:47.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.306+0000 7fb99bfff700 1 --2- 172.21.15.1:0/4234105630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb99c1002a0 0x7fb99c198a10 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fb988005910 tx=0x7fb988004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:47.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.306+0000 7fb999ffb700 1 -- 172.21.15.1:0/4234105630 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98801d070 con 0x7fb99c1002a0 2023-11-16T16:46:47.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.306+0000 7fb9a2fd1700 1 -- 172.21.15.1:0/4234105630 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb99c10f1d0 con 0x7fb99c1002a0 2023-11-16T16:46:47.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.306+0000 7fb999ffb700 1 -- 172.21.15.1:0/4234105630 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb988004200 con 0x7fb99c1002a0 2023-11-16T16:46:47.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.306+0000 7fb999ffb700 1 -- 172.21.15.1:0/4234105630 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb988022e60 con 0x7fb99c1002a0 2023-11-16T16:46:47.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.306+0000 7fb9a2fd1700 1 -- 172.21.15.1:0/4234105630 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb99c10f040 con 0x7fb99c1002a0 2023-11-16T16:46:47.319 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.308+0000 7fb999ffb700 1 -- 172.21.15.1:0/4234105630 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fb988003d40 con 0x7fb99c1002a0 2023-11-16T16:46:47.319 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.309+0000 7fb999ffb700 1 --2- 172.21.15.1:0/4234105630 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fb98c0a12c0 0x7fb98c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.319 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.309+0000 7fb999ffb700 1 -- 172.21.15.1:0/4234105630 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fb9880ce0c0 con 0x7fb99c1002a0 2023-11-16T16:46:47.319 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.309+0000 7fb9a0d6d700 1 --2- 172.21.15.1:0/4234105630 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fb98c0a12c0 0x7fb98c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.319 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.309+0000 7fb9a0d6d700 1 --2- 172.21.15.1:0/4234105630 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fb98c0a12c0 0x7fb98c0a3770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb990005950 tx=0x7fb99000a300 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:47.320 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.310+0000 7fb9af15db80 1 Processor -- start 2023-11-16T16:46:47.320 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.310+0000 7fb9af15db80 1 -- start start 2023-11-16T16:46:47.320 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.310+0000 7fb9af15db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b18d18b60 0x555b18d18f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.320 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.310+0000 7fb9af15db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b18d19470 0x555b18d1d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.320 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.310+0000 7fb9af15db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b18d1d8b0 0x555b18d1dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.320 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.310+0000 7fb9af15db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b18c1e060 con 0x555b18d19470 2023-11-16T16:46:47.320 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.310+0000 7fb9af15db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b18c708a0 con 0x555b18d1d8b0 2023-11-16T16:46:47.320 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.310+0000 7fb9af15db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b18c69d30 con 0x555b18d18b60 2023-11-16T16:46:47.321 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.310+0000 7fb99bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b18d19470 0x555b18d1d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.321 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.310+0000 7fb99bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b18d19470 0x555b18d1d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45940/0 (socket says 172.21.15.1:45940) 2023-11-16T16:46:47.321 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.310+0000 7fb99bfff700 1 -- 172.21.15.1:0/1149087462 learned_addr learned my addr 172.21.15.1:0/1149087462 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:47.321 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.310+0000 7fb9a0d6d700 1 --2- 172.21.15.1:0/1149087462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b18d18b60 0x555b18d18f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.321 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.310+0000 7fb9a156e700 1 --2- 172.21.15.1:0/1149087462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b18d1d8b0 0x555b18d1dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.321 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.311+0000 7fb99bfff700 1 -- 172.21.15.1:0/1149087462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b18d18b60 msgr2=0x555b18d18f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:47.322 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.311+0000 7fb99bfff700 1 --2- 172.21.15.1:0/1149087462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b18d18b60 0x555b18d18f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.322 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.311+0000 7fb99bfff700 1 -- 172.21.15.1:0/1149087462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b18d1d8b0 msgr2=0x555b18d1dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:47.322 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.311+0000 7fb99bfff700 1 --2- 172.21.15.1:0/1149087462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b18d1d8b0 0x555b18d1dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.322 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.311+0000 7fb99bfff700 1 -- 172.21.15.1:0/1149087462 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b18aeb630 con 0x555b18d19470 2023-11-16T16:46:47.322 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.311+0000 7fb99bfff700 1 --2- 172.21.15.1:0/1149087462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b18d19470 0x555b18d1d370 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fb988097c80 tx=0x7fb988009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b810c2efcb12cb server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:47.322 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.311+0000 7fb986ffd700 1 -- 172.21.15.1:0/1149087462 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9880a10e0 con 0x555b18d19470 2023-11-16T16:46:47.322 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.311+0000 7fb986ffd700 1 -- 172.21.15.1:0/1149087462 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9880a35d0 con 0x555b18d19470 2023-11-16T16:46:47.322 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.311+0000 7fb986ffd700 1 -- 172.21.15.1:0/1149087462 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9880a3140 con 0x555b18d19470 2023-11-16T16:46:47.323 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.312+0000 7fb9af15db80 1 -- 172.21.15.1:0/1149087462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b18d19470 msgr2=0x555b18d1d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:47.323 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.312+0000 7fb9af15db80 1 --2- 172.21.15.1:0/1149087462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b18d19470 0x555b18d1d370 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fb988097c80 tx=0x7fb988009cd0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.323 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.312+0000 7fb9af15db80 1 -- 172.21.15.1:0/1149087462 shutdown_connections 2023-11-16T16:46:47.323 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.312+0000 7fb9af15db80 1 --2- 172.21.15.1:0/1149087462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b18d1d8b0 0x555b18d1dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.323 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.312+0000 7fb9af15db80 1 --2- 172.21.15.1:0/1149087462 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b18d18b60 0x555b18d18f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.323 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.312+0000 7fb9af15db80 1 --2- 172.21.15.1:0/1149087462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b18d19470 0x555b18d1d370 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.323 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.312+0000 7fb9af15db80 1 -- 172.21.15.1:0/1149087462 >> 172.21.15.1:0/1149087462 conn(0x555b18d0e360 msgr2=0x555b18d13b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:47.324 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.312+0000 7fb9af15db80 1 -- 172.21.15.1:0/1149087462 shutdown_connections 2023-11-16T16:46:47.324 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.312+0000 7fb9af15db80 1 -- 172.21.15.1:0/1149087462 wait complete. 2023-11-16T16:46:47.324 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.312+0000 7fb9af15db80 1 -- start start 2023-11-16T16:46:47.324 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.312+0000 7fb9af15db80 1 Processor -- start 2023-11-16T16:46:47.324 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.313+0000 7fb9af15db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b18d18b60 0x555b18d24f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.324 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.313+0000 7fb9af15db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b18d19470 0x555b18d22180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.324 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.313+0000 7fb9af15db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b18d1d8b0 0x555b18d226c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.324 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.313+0000 7fb9af15db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b18c0b8e0 con 0x555b18d1d8b0 2023-11-16T16:46:47.325 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.313+0000 7fb9af15db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b18c0b580 con 0x555b18d18b60 2023-11-16T16:46:47.325 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.313+0000 7fb9af15db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b18aed920 con 0x555b18d19470 2023-11-16T16:46:47.325 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.313+0000 7fb9af15db80 10 client.0 _ll_register_callbacks cb 0x555b18c29780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:46:47.325 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.313+0000 7fb9a156e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b18d1d8b0 0x555b18d226c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.325 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.313+0000 7fb9a156e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b18d1d8b0 0x555b18d226c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45948/0 (socket says 172.21.15.1:45948) 2023-11-16T16:46:47.325 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.313+0000 7fb9a156e700 1 -- 172.21.15.1:0/3100048038 learned_addr learned my addr 172.21.15.1:0/3100048038 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:47.325 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.313+0000 7fb99bfff700 1 --2- 172.21.15.1:0/3100048038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b18d19470 0x555b18d22180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.325 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.313+0000 7fb9a0d6d700 1 --2- 172.21.15.1:0/3100048038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b18d18b60 0x555b18d24f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.326 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.313+0000 7fb9a156e700 1 -- 172.21.15.1:0/3100048038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b18d19470 msgr2=0x555b18d22180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:47.326 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.313+0000 7fb9a156e700 1 --2- 172.21.15.1:0/3100048038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b18d19470 0x555b18d22180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.326 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.313+0000 7fb9a156e700 1 -- 172.21.15.1:0/3100048038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b18d18b60 msgr2=0x555b18d24f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:47.326 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.313+0000 7fb9a156e700 1 --2- 172.21.15.1:0/3100048038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b18d18b60 0x555b18d24f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.326 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.313+0000 7fb9a156e700 1 -- 172.21.15.1:0/3100048038 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb99c10f040 con 0x555b18d1d8b0 2023-11-16T16:46:47.326 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.313+0000 7fb9a0d6d700 1 --2- 172.21.15.1:0/3100048038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b18d18b60 0x555b18d24f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:46:47.326 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.313+0000 7fb9a156e700 1 --2- 172.21.15.1:0/3100048038 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b18d1d8b0 0x555b18d226c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fb994008ed0 tx=0x7fb9940114a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c02cd24c7fb3a server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:47.327 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.313+0000 7fb97bfff700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:46:47.327 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.313+0000 7fb97bfff700 1 -- 172.21.15.1:0/3100048038 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99401a070 con 0x555b18d1d8b0 2023-11-16T16:46:47.327 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.313+0000 7fb97bfff700 1 -- 172.21.15.1:0/3100048038 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fb994011d80 con 0x555b18d1d8b0 2023-11-16T16:46:47.327 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.313+0000 7fb9af15db80 1 -- 172.21.15.1:0/3100048038 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555b18d0bb20 con 0x555b18d1d8b0 2023-11-16T16:46:47.327 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.313+0000 7fb97bfff700 1 -- 172.21.15.1:0/3100048038 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb994004bb0 con 0x555b18d1d8b0 2023-11-16T16:46:47.327 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.314+0000 7fb9af15db80 10 client.16458 fetch_fsmap learned FSMap version 28 2023-11-16T16:46:47.327 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.314+0000 7fb9af15db80 1 -- 172.21.15.1:0/3100048038 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555b18c30190 con 0x555b18d1d8b0 2023-11-16T16:46:47.327 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.314+0000 7fb97bfff700 1 -- 172.21.15.1:0/3100048038 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555b18d0bb20 con 0x555b18d1d8b0 2023-11-16T16:46:47.327 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.314+0000 7fb97bfff700 1 -- 172.21.15.1:0/3100048038 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fb99401d020 con 0x555b18d1d8b0 2023-11-16T16:46:47.328 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.314+0000 7fb9af15db80 10 client.16458 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:46:47.328 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.314+0000 7fb9af15db80 10 client.16458 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-11-16T16:46:47.328 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.314+0000 7fb9af15db80 20 client.16458 populate_metadata read hostname 'smithi001' 2023-11-16T16:46:47.328 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.314+0000 7fb9af15db80 1 --2- 172.21.15.1:0/3100048038 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x555b18d32600 0x555b18d34a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.328 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.314+0000 7fb9af15db80 4 client.16458 mds_command: new command op to 24482 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:46:47.328 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.314+0000 7fb9af15db80 1 -- 172.21.15.1:0/3100048038 --> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555b18c2c0a0 con 0x555b18d32600 2023-11-16T16:46:47.328 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.314+0000 7fb9a0d6d700 1 --2- 172.21.15.1:0/3100048038 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x555b18d32600 0x555b18d34a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.328 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.315+0000 7fb97bfff700 10 client.16458 ms_handle_connect on v2:172.21.15.26:6836/1453504863 2023-11-16T16:46:47.329 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.315+0000 7fb9a0d6d700 1 --2- 172.21.15.1:0/3100048038 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x555b18d32600 0x555b18d34a10 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a6c1fde078fffca3 server_cookie=56146c7fd56e3be5 in_seq=0 out_seq=0 2023-11-16T16:46:47.329 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.315+0000 7fb97bfff700 1 -- 172.21.15.1:0/3100048038 <== mds.2 v2:172.21.15.26:6836/1453504863 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555b18c2c0a0 con 0x555b18d32600 2023-11-16T16:46:47.329 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.315+0000 7fb97bfff700 10 client.16458 handle_command_reply: tid=0 2023-11-16T16:46:47.329 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.315+0000 7fb9af15db80 1 client.16458 shutdown 2023-11-16T16:46:47.329 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.316+0000 7fb9af15db80 1 -- 172.21.15.1:0/3100048038 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b18d1d8b0 msgr2=0x555b18d226c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:47.329 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.316+0000 7fb9af15db80 1 --2- 172.21.15.1:0/3100048038 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b18d1d8b0 0x555b18d226c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fb994008ed0 tx=0x7fb9940114a0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.329 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.316+0000 7fb9af15db80 1 -- 172.21.15.1:0/3100048038 shutdown_connections 2023-11-16T16:46:47.330 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.316+0000 7fb9af15db80 1 --2- 172.21.15.1:0/3100048038 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x555b18d32600 0x555b18d34a10 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.330 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.316+0000 7fb9af15db80 1 --2- 172.21.15.1:0/3100048038 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b18d1d8b0 0x555b18d226c0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.330 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.316+0000 7fb9af15db80 1 --2- 172.21.15.1:0/3100048038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b18d18b60 0x555b18d24f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.330 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.316+0000 7fb9af15db80 1 --2- 172.21.15.1:0/3100048038 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b18d19470 0x555b18d22180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.330 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.316+0000 7fb97bfff700 0 client.16458 ms_handle_reset on v2:172.21.15.26:6836/1453504863 2023-11-16T16:46:47.330 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.316+0000 7fb9af15db80 1 -- 172.21.15.1:0/3100048038 >> 172.21.15.1:0/3100048038 conn(0x555b18d0e360 msgr2=0x555b18d1ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:47.330 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.316+0000 7fb9af15db80 1 -- 172.21.15.1:0/3100048038 shutdown_connections 2023-11-16T16:46:47.330 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.316+0000 7fb9af15db80 1 -- 172.21.15.1:0/3100048038 wait complete. 2023-11-16T16:46:47.331 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.316+0000 7fb9af15db80 20 client.16458 trim_cache size 0 max 16384 2023-11-16T16:46:47.336 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.329+0000 7fb9af15db80 1 Processor -- start 2023-11-16T16:46:47.336 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.329+0000 7fb9af15db80 1 -- start start 2023-11-16T16:46:47.337 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.329+0000 7fb9af15db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b18d18b60 0x555b18d13ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.337 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.329+0000 7fb9af15db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b18d19470 0x555b18d1c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.337 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.329+0000 7fb9af15db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b18d1d8b0 0x555b18d1b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.337 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.329+0000 7fb9af15db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b18d149a0 con 0x555b18d19470 2023-11-16T16:46:47.337 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.329+0000 7fb9af15db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b18d14780 con 0x555b18d18b60 2023-11-16T16:46:47.337 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.329+0000 7fb9af15db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b18d0dee0 con 0x555b18d1d8b0 2023-11-16T16:46:47.337 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.329+0000 7fb99bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b18d19470 0x555b18d1c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.338 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.329+0000 7fb99bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b18d19470 0x555b18d1c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45954/0 (socket says 172.21.15.1:45954) 2023-11-16T16:46:47.338 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.329+0000 7fb99bfff700 1 -- 172.21.15.1:0/407669231 learned_addr learned my addr 172.21.15.1:0/407669231 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:47.338 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.329+0000 7fb9a0d6d700 1 --2- 172.21.15.1:0/407669231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b18d18b60 0x555b18d13ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.338 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.329+0000 7fb9a156e700 1 --2- 172.21.15.1:0/407669231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b18d1d8b0 0x555b18d1b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.338 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.329+0000 7fb99bfff700 1 -- 172.21.15.1:0/407669231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b18d1d8b0 msgr2=0x555b18d1b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:47.338 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.329+0000 7fb99bfff700 1 --2- 172.21.15.1:0/407669231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b18d1d8b0 0x555b18d1b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.338 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.329+0000 7fb99bfff700 1 -- 172.21.15.1:0/407669231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b18d18b60 msgr2=0x555b18d13ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:47.338 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.329+0000 7fb99bfff700 1 --2- 172.21.15.1:0/407669231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b18d18b60 0x555b18d13ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.339 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.329+0000 7fb99bfff700 1 -- 172.21.15.1:0/407669231 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b18c2fee0 con 0x555b18d19470 2023-11-16T16:46:47.339 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.329+0000 7fb9a0d6d700 1 --2- 172.21.15.1:0/407669231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b18d18b60 0x555b18d13ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:46:47.339 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.330+0000 7fb99bfff700 1 --2- 172.21.15.1:0/407669231 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b18d19470 0x555b18d1c6c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fb988097710 tx=0x7fb98809b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcd3d4b07b149b66 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:47.339 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.330+0000 7fb9877fe700 1 -- 172.21.15.1:0/407669231 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9880995b0 con 0x555b18d19470 2023-11-16T16:46:47.339 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.330+0000 7fb9877fe700 1 -- 172.21.15.1:0/407669231 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb98809b470 con 0x555b18d19470 2023-11-16T16:46:47.339 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.330+0000 7fb9877fe700 1 -- 172.21.15.1:0/407669231 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9880a33a0 con 0x555b18d19470 2023-11-16T16:46:47.339 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.330+0000 7fb9af15db80 1 -- 172.21.15.1:0/407669231 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b18d19470 msgr2=0x555b18d1c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:47.339 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.330+0000 7fb9af15db80 1 --2- 172.21.15.1:0/407669231 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b18d19470 0x555b18d1c6c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fb988097710 tx=0x7fb98809b390 comp rx=0 tx=0).stop 2023-11-16T16:46:47.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.330+0000 7fb9af15db80 1 -- 172.21.15.1:0/407669231 shutdown_connections 2023-11-16T16:46:47.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.330+0000 7fb9af15db80 1 --2- 172.21.15.1:0/407669231 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b18d1d8b0 0x555b18d1b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.330+0000 7fb9af15db80 1 --2- 172.21.15.1:0/407669231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b18d18b60 0x555b18d13ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.330+0000 7fb9af15db80 1 --2- 172.21.15.1:0/407669231 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b18d19470 0x555b18d1c6c0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.330+0000 7fb9af15db80 1 -- 172.21.15.1:0/407669231 >> 172.21.15.1:0/407669231 conn(0x555b18d0e360 msgr2=0x555b18d2e850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:47.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.330+0000 7fb9af15db80 1 -- 172.21.15.1:0/407669231 shutdown_connections 2023-11-16T16:46:47.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.330+0000 7fb9af15db80 1 -- 172.21.15.1:0/407669231 wait complete. 2023-11-16T16:46:47.341 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.331+0000 7fb9af15db80 1 -- start start 2023-11-16T16:46:47.341 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.331+0000 7fb9af15db80 1 Processor -- start 2023-11-16T16:46:47.341 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.331+0000 7fb9af15db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b18d18b60 0x555b18d27140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.341 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.331+0000 7fb9af15db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b18d19470 0x555b18d27680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.341 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.331+0000 7fb9af15db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b18d1d8b0 0x555b18d27c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.341 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.331+0000 7fb9af15db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b18d856a0 con 0x555b18d18b60 2023-11-16T16:46:47.341 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.331+0000 7fb9af15db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b18d85810 con 0x555b18d1d8b0 2023-11-16T16:46:47.341 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.331+0000 7fb9af15db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b18d859d0 con 0x555b18d19470 2023-11-16T16:46:47.341 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.331+0000 7fb9af15db80 10 client.0 _ll_register_callbacks cb 0x7fb994008790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:46:47.342 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.331+0000 7fb9a0d6d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b18d18b60 0x555b18d27140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.342 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.331+0000 7fb9a0d6d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b18d18b60 0x555b18d27140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45958/0 (socket says 172.21.15.1:45958) 2023-11-16T16:46:47.342 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.331+0000 7fb9a0d6d700 1 -- 172.21.15.1:0/273902494 learned_addr learned my addr 172.21.15.1:0/273902494 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:47.342 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.331+0000 7fb99bfff700 1 --2- 172.21.15.1:0/273902494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b18d19470 0x555b18d27680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.342 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.332+0000 7fb9a0d6d700 1 -- 172.21.15.1:0/273902494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b18d19470 msgr2=0x555b18d27680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:47.342 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.332+0000 7fb9a0d6d700 1 --2- 172.21.15.1:0/273902494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b18d19470 0x555b18d27680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.342 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.332+0000 7fb9a0d6d700 1 -- 172.21.15.1:0/273902494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b18d1d8b0 msgr2=0x555b18d27c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:47.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.332+0000 7fb9a0d6d700 1 --2- 172.21.15.1:0/273902494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b18d1d8b0 0x555b18d27c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.332+0000 7fb9a0d6d700 1 -- 172.21.15.1:0/273902494 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555b18aeb630 con 0x555b18d18b60 2023-11-16T16:46:47.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.332+0000 7fb9a0d6d700 1 --2- 172.21.15.1:0/273902494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b18d18b60 0x555b18d27140 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x555b18d352b0 tx=0x7fb9900047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8022bdb8d156e02 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:47.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.332+0000 7fb97bfff700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:46:47.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.332+0000 7fb97bfff700 1 -- 172.21.15.1:0/273902494 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb990021040 con 0x555b18d18b60 2023-11-16T16:46:47.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.332+0000 7fb97bfff700 1 -- 172.21.15.1:0/273902494 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fb9900049b0 con 0x555b18d18b60 2023-11-16T16:46:47.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.332+0000 7fb9af15db80 1 -- 172.21.15.1:0/273902494 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9880a35d0 con 0x555b18d18b60 2023-11-16T16:46:47.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.332+0000 7fb97bfff700 1 -- 172.21.15.1:0/273902494 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb990004b70 con 0x555b18d18b60 2023-11-16T16:46:47.344 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.332+0000 7fb9af15db80 10 client.16464 fetch_fsmap learned FSMap version 28 2023-11-16T16:46:47.344 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.332+0000 7fb9af15db80 1 -- 172.21.15.1:0/273902494 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555b18a80c60 con 0x555b18d18b60 2023-11-16T16:46:47.344 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.332+0000 7fb97bfff700 1 -- 172.21.15.1:0/273902494 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9880a35d0 con 0x555b18d18b60 2023-11-16T16:46:47.344 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.332+0000 7fb97bfff700 1 -- 172.21.15.1:0/273902494 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fb9900105f0 con 0x555b18d18b60 2023-11-16T16:46:47.344 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.333+0000 7fb9af15db80 10 client.16464 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:46:47.344 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.333+0000 7fb9af15db80 10 client.16464 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-11-16T16:46:47.344 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.333+0000 7fb9af15db80 20 client.16464 populate_metadata read hostname 'smithi001' 2023-11-16T16:46:47.344 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.333+0000 7fb9af15db80 1 --2- 172.21.15.1:0/273902494 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x555b18d87100 0x555b18d89510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.344 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.333+0000 7fb9af15db80 4 client.16464 mds_command: new command op to 24482 tid=0[{"prefix": "damage ls"}] 2023-11-16T16:46:47.345 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.333+0000 7fb9af15db80 1 -- 172.21.15.1:0/273902494 --> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555b18c38660 con 0x555b18d87100 2023-11-16T16:46:47.345 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.333+0000 7fb99bfff700 1 --2- 172.21.15.1:0/273902494 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x555b18d87100 0x555b18d89510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.345 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.333+0000 7fb97bfff700 10 client.16464 ms_handle_connect on v2:172.21.15.26:6836/1453504863 2023-11-16T16:46:47.345 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.333+0000 7fb99bfff700 1 --2- 172.21.15.1:0/273902494 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x555b18d87100 0x555b18d89510 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8c3355354b731bac server_cookie=230290446e92b0d in_seq=0 out_seq=0 2023-11-16T16:46:47.345 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.333+0000 7fb97bfff700 1 -- 172.21.15.1:0/273902494 <== mds.2 v2:172.21.15.26:6836/1453504863 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555b18c38660 con 0x555b18d87100 2023-11-16T16:46:47.345 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.333+0000 7fb97bfff700 10 client.16464 handle_command_reply: tid=0 2023-11-16T16:46:47.345 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.333+0000 7fb9af15db80 1 client.16464 shutdown 2023-11-16T16:46:47.345 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.334+0000 7fb9af15db80 1 -- 172.21.15.1:0/273902494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b18d18b60 msgr2=0x555b18d27140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:47.346 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.334+0000 7fb9af15db80 1 --2- 172.21.15.1:0/273902494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b18d18b60 0x555b18d27140 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x555b18d352b0 tx=0x7fb9900047c0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.346 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.334+0000 7fb9af15db80 1 -- 172.21.15.1:0/273902494 shutdown_connections 2023-11-16T16:46:47.346 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.334+0000 7fb9af15db80 1 --2- 172.21.15.1:0/273902494 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x555b18d87100 0x555b18d89510 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.346 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.334+0000 7fb9af15db80 1 --2- 172.21.15.1:0/273902494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b18d1d8b0 0x555b18d27c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.346 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.334+0000 7fb9af15db80 1 --2- 172.21.15.1:0/273902494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555b18d18b60 0x555b18d27140 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.346 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.334+0000 7fb9af15db80 1 --2- 172.21.15.1:0/273902494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555b18d19470 0x555b18d27680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.346 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.334+0000 7fb97bfff700 0 client.16464 ms_handle_reset on v2:172.21.15.26:6836/1453504863 2023-11-16T16:46:47.346 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.334+0000 7fb9af15db80 1 -- 172.21.15.1:0/273902494 >> 172.21.15.1:0/273902494 conn(0x555b18d0e360 msgr2=0x555b18d1c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:47.346 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.334+0000 7fb9af15db80 1 -- 172.21.15.1:0/273902494 shutdown_connections 2023-11-16T16:46:47.347 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.334+0000 7fb9af15db80 1 -- 172.21.15.1:0/273902494 wait complete. 2023-11-16T16:46:47.347 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.334+0000 7fb9af15db80 20 client.16464 trim_cache size 0 max 16384 2023-11-16T16:46:47.347 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2023-11-16T16:46:47.348 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.334+0000 7fb9a2fd1700 1 -- 172.21.15.1:0/4234105630 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fb98c0a12c0 msgr2=0x7fb98c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:47.348 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.334+0000 7fb9a2fd1700 1 --2- 172.21.15.1:0/4234105630 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fb98c0a12c0 0x7fb98c0a3770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb990005950 tx=0x7fb99000a300 comp rx=0 tx=0).stop 2023-11-16T16:46:47.348 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.334+0000 7fb9a2fd1700 1 -- 172.21.15.1:0/4234105630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb99c1002a0 msgr2=0x7fb99c198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:47.349 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.334+0000 7fb9a2fd1700 1 --2- 172.21.15.1:0/4234105630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb99c1002a0 0x7fb99c198a10 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fb988005910 tx=0x7fb988004730 comp rx=0 tx=0).stop 2023-11-16T16:46:47.349 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.334+0000 7fb9a2fd1700 1 -- 172.21.15.1:0/4234105630 shutdown_connections 2023-11-16T16:46:47.349 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.334+0000 7fb9a2fd1700 1 --2- 172.21.15.1:0/4234105630 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fb98c0a12c0 0x7fb98c0a3770 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.349 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.334+0000 7fb9a2fd1700 1 --2- 172.21.15.1:0/4234105630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb99c104140 0x7fb99c19ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.349 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.335+0000 7fb9a2fd1700 1 --2- 172.21.15.1:0/4234105630 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb99c0ff8f0 0x7fb99c1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.349 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.335+0000 7fb9a2fd1700 1 --2- 172.21.15.1:0/4234105630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb99c1002a0 0x7fb99c198a10 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.349 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.335+0000 7fb9a2fd1700 1 -- 172.21.15.1:0/4234105630 >> 172.21.15.1:0/4234105630 conn(0x7fb99c075880 msgr2=0x7fb99c0fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:47.350 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.335+0000 7fb9a2fd1700 1 -- 172.21.15.1:0/4234105630 shutdown_connections 2023-11-16T16:46:47.350 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.335+0000 7fb9a2fd1700 1 -- 172.21.15.1:0/4234105630 wait complete. 2023-11-16T16:46:47.350 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-11-16T16:46:47.480 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.476+0000 7fb76c515700 1 -- 172.21.15.1:0/193371746 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7640ffdb0 msgr2=0x7fb764103d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:47.480 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.476+0000 7fb76c515700 1 --2- 172.21.15.1:0/193371746 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7640ffdb0 0x7fb764103d70 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fb7540099c0 tx=0x7fb754009cd0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.480 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.476+0000 7fb76c515700 1 -- 172.21.15.1:0/193371746 shutdown_connections 2023-11-16T16:46:47.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.476+0000 7fb76c515700 1 --2- 172.21.15.1:0/193371746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb764104450 0x7fb76410c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.476+0000 7fb76c515700 1 --2- 172.21.15.1:0/193371746 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb7640ff410 0x7fb7640ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.476+0000 7fb76c515700 1 --2- 172.21.15.1:0/193371746 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7640ffdb0 0x7fb764103d70 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.476+0000 7fb76c515700 1 -- 172.21.15.1:0/193371746 >> 172.21.15.1:0/193371746 conn(0x7fb764076370 msgr2=0x7fb764076770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:47.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.476+0000 7fb76c515700 1 -- 172.21.15.1:0/193371746 shutdown_connections 2023-11-16T16:46:47.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.476+0000 7fb76c515700 1 -- 172.21.15.1:0/193371746 wait complete. 2023-11-16T16:46:47.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.477+0000 7fb76c515700 1 Processor -- start 2023-11-16T16:46:47.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.477+0000 7fb76c515700 1 -- start start 2023-11-16T16:46:47.482 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.477+0000 7fb76c515700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7640ff410 0x7fb764196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.482 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.477+0000 7fb76c515700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7640ffdb0 0x7fb7641969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.482 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.477+0000 7fb76c515700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb764104450 0x7fb76419ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.482 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.477+0000 7fb76c515700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76419b450 con 0x7fb7640ff410 2023-11-16T16:46:47.482 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.477+0000 7fb76c515700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76419b5c0 con 0x7fb7640ffdb0 2023-11-16T16:46:47.482 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.477+0000 7fb76c515700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76419b780 con 0x7fb764104450 2023-11-16T16:46:47.482 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.477+0000 7fb76a2b1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7640ff410 0x7fb764196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.483 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.477+0000 7fb76a2b1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7640ff410 0x7fb764196470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45972/0 (socket says 172.21.15.1:45972) 2023-11-16T16:46:47.483 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.477+0000 7fb76a2b1700 1 -- 172.21.15.1:0/2957558398 learned_addr learned my addr 172.21.15.1:0/2957558398 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:47.483 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.477+0000 7fb76aab2700 1 --2- 172.21.15.1:0/2957558398 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb764104450 0x7fb76419ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.483 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.477+0000 7fb769ab0700 1 --2- 172.21.15.1:0/2957558398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7640ffdb0 0x7fb7641969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.483 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.477+0000 7fb76a2b1700 1 -- 172.21.15.1:0/2957558398 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb764104450 msgr2=0x7fb76419ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:47.483 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.477+0000 7fb76a2b1700 1 --2- 172.21.15.1:0/2957558398 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb764104450 0x7fb76419ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.483 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.477+0000 7fb76a2b1700 1 -- 172.21.15.1:0/2957558398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7640ffdb0 msgr2=0x7fb7641969b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:47.483 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.477+0000 7fb76a2b1700 1 --2- 172.21.15.1:0/2957558398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7640ffdb0 0x7fb7641969b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.484 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.477+0000 7fb76a2b1700 1 -- 172.21.15.1:0/2957558398 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7540095f0 con 0x7fb7640ff410 2023-11-16T16:46:47.484 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.477+0000 7fb76aab2700 1 --2- 172.21.15.1:0/2957558398 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb764104450 0x7fb76419ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:46:47.484 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.478+0000 7fb76a2b1700 1 --2- 172.21.15.1:0/2957558398 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7640ff410 0x7fb764196470 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb76000e9a0 tx=0x7fb76000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:47.484 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.478+0000 7fb75b7fe700 1 -- 172.21.15.1:0/2957558398 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb760009790 con 0x7fb7640ff410 2023-11-16T16:46:47.484 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.478+0000 7fb75b7fe700 1 -- 172.21.15.1:0/2957558398 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb76000cbd0 con 0x7fb7640ff410 2023-11-16T16:46:47.484 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.478+0000 7fb76c515700 1 -- 172.21.15.1:0/2957558398 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb76410f220 con 0x7fb7640ff410 2023-11-16T16:46:47.484 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.478+0000 7fb75b7fe700 1 -- 172.21.15.1:0/2957558398 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb760018680 con 0x7fb7640ff410 2023-11-16T16:46:47.484 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.478+0000 7fb76c515700 1 -- 172.21.15.1:0/2957558398 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb76410f090 con 0x7fb7640ff410 2023-11-16T16:46:47.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.480+0000 7fb75b7fe700 1 -- 172.21.15.1:0/2957558398 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fb76000cd40 con 0x7fb7640ff410 2023-11-16T16:46:47.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.481+0000 7fb75b7fe700 1 --2- 172.21.15.1:0/2957558398 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fb7500a12c0 0x7fb7500a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.481+0000 7fb769ab0700 1 --2- 172.21.15.1:0/2957558398 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fb7500a12c0 0x7fb7500a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.481+0000 7fb75b7fe700 1 -- 172.21.15.1:0/2957558398 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fb760014070 con 0x7fb7640ff410 2023-11-16T16:46:47.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.481+0000 7fb769ab0700 1 --2- 172.21.15.1:0/2957558398 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fb7500a12c0 0x7fb7500a3770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb75400b580 tx=0x7fb75401a040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:47.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.482+0000 7fb7786a1b80 1 Processor -- start 2023-11-16T16:46:47.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.482+0000 7fb7786a1b80 1 -- start start 2023-11-16T16:46:47.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.482+0000 7fb7786a1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdcc6a2f70 0x55bdcc6a3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.482+0000 7fb7786a1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bdcc6a3880 0x55bdcc6a7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.482+0000 7fb7786a1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdcc6a7cc0 0x55bdcc6a8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.482+0000 7fb7786a1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdcc5a4060 con 0x55bdcc6a7cc0 2023-11-16T16:46:47.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.482+0000 7fb7786a1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdcc5f68a0 con 0x55bdcc6a2f70 2023-11-16T16:46:47.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.482+0000 7fb7786a1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdcc5efd30 con 0x55bdcc6a3880 2023-11-16T16:46:47.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.482+0000 7fb76aab2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdcc6a7cc0 0x55bdcc6a8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.482+0000 7fb76aab2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdcc6a7cc0 0x55bdcc6a8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45984/0 (socket says 172.21.15.1:45984) 2023-11-16T16:46:47.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.482+0000 7fb76aab2700 1 -- 172.21.15.1:0/2254066165 learned_addr learned my addr 172.21.15.1:0/2254066165 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:47.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.482+0000 7fb76a2b1700 1 --2- 172.21.15.1:0/2254066165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdcc6a2f70 0x55bdcc6a3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.482+0000 7fb769ab0700 1 --2- 172.21.15.1:0/2254066165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bdcc6a3880 0x55bdcc6a7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.482+0000 7fb76aab2700 1 -- 172.21.15.1:0/2254066165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bdcc6a3880 msgr2=0x55bdcc6a7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:47.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.482+0000 7fb76aab2700 1 --2- 172.21.15.1:0/2254066165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bdcc6a3880 0x55bdcc6a7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.482+0000 7fb76aab2700 1 -- 172.21.15.1:0/2254066165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdcc6a2f70 msgr2=0x55bdcc6a3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:47.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.482+0000 7fb76aab2700 1 --2- 172.21.15.1:0/2254066165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdcc6a2f70 0x55bdcc6a3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.482+0000 7fb76aab2700 1 -- 172.21.15.1:0/2254066165 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdcc471630 con 0x55bdcc6a7cc0 2023-11-16T16:46:47.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.482+0000 7fb76a2b1700 1 --2- 172.21.15.1:0/2254066165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdcc6a2f70 0x55bdcc6a3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:46:47.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.483+0000 7fb76aab2700 1 --2- 172.21.15.1:0/2254066165 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdcc6a7cc0 0x55bdcc6a8110 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb75c009dd0 tx=0x7fb75c0077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=347aeec993badbe0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:47.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.483+0000 7fb758ff9700 1 -- 172.21.15.1:0/2254066165 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75c018070 con 0x55bdcc6a7cc0 2023-11-16T16:46:47.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.483+0000 7fb758ff9700 1 -- 172.21.15.1:0/2254066165 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb75c00f050 con 0x55bdcc6a7cc0 2023-11-16T16:46:47.495 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.483+0000 7fb758ff9700 1 -- 172.21.15.1:0/2254066165 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75c014410 con 0x55bdcc6a7cc0 2023-11-16T16:46:47.495 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.483+0000 7fb7786a1b80 1 -- 172.21.15.1:0/2254066165 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdcc6a7cc0 msgr2=0x55bdcc6a8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:47.495 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.483+0000 7fb7786a1b80 1 --2- 172.21.15.1:0/2254066165 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdcc6a7cc0 0x55bdcc6a8110 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb75c009dd0 tx=0x7fb75c0077e0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.495 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.484+0000 7fb7786a1b80 1 -- 172.21.15.1:0/2254066165 shutdown_connections 2023-11-16T16:46:47.495 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.484+0000 7fb7786a1b80 1 --2- 172.21.15.1:0/2254066165 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdcc6a7cc0 0x55bdcc6a8110 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.495 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.484+0000 7fb7786a1b80 1 --2- 172.21.15.1:0/2254066165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdcc6a2f70 0x55bdcc6a3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.495 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.484+0000 7fb7786a1b80 1 --2- 172.21.15.1:0/2254066165 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bdcc6a3880 0x55bdcc6a7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.495 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.484+0000 7fb7786a1b80 1 -- 172.21.15.1:0/2254066165 >> 172.21.15.1:0/2254066165 conn(0x55bdcc694360 msgr2=0x55bdcc699b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:47.496 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.484+0000 7fb7786a1b80 1 -- 172.21.15.1:0/2254066165 shutdown_connections 2023-11-16T16:46:47.496 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.484+0000 7fb7786a1b80 1 -- 172.21.15.1:0/2254066165 wait complete. 2023-11-16T16:46:47.496 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.484+0000 7fb7786a1b80 1 -- start start 2023-11-16T16:46:47.496 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.484+0000 7fb7786a1b80 1 Processor -- start 2023-11-16T16:46:47.496 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.484+0000 7fb7786a1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bdcc6a2f70 0x55bdcc6af450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.496 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.484+0000 7fb7786a1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdcc6a3880 0x55bdcc6ac6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.496 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.484+0000 7fb7786a1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdcc6a7cc0 0x55bdcc6acd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.484+0000 7fb7786a1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdcc5918e0 con 0x55bdcc6a3880 2023-11-16T16:46:47.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.484+0000 7fb7786a1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdcc591580 con 0x55bdcc6a7cc0 2023-11-16T16:46:47.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.484+0000 7fb7786a1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdcc473920 con 0x55bdcc6a2f70 2023-11-16T16:46:47.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.484+0000 7fb7786a1b80 10 client.0 _ll_register_callbacks cb 0x55bdcc5af780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:46:47.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.484+0000 7fb769ab0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdcc6a3880 0x55bdcc6ac6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.485+0000 7fb769ab0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdcc6a3880 0x55bdcc6ac6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45988/0 (socket says 172.21.15.1:45988) 2023-11-16T16:46:47.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.485+0000 7fb769ab0700 1 -- 172.21.15.1:0/1260426800 learned_addr learned my addr 172.21.15.1:0/1260426800 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:47.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.485+0000 7fb76a2b1700 1 --2- 172.21.15.1:0/1260426800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bdcc6a2f70 0x55bdcc6af450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.485+0000 7fb76aab2700 1 --2- 172.21.15.1:0/1260426800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdcc6a7cc0 0x55bdcc6acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.485+0000 7fb769ab0700 1 -- 172.21.15.1:0/1260426800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bdcc6a2f70 msgr2=0x55bdcc6af450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:47.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.485+0000 7fb769ab0700 1 --2- 172.21.15.1:0/1260426800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bdcc6a2f70 0x55bdcc6af450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.485+0000 7fb769ab0700 1 -- 172.21.15.1:0/1260426800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdcc6a7cc0 msgr2=0x55bdcc6acd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:47.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.485+0000 7fb769ab0700 1 --2- 172.21.15.1:0/1260426800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdcc6a7cc0 0x55bdcc6acd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.485+0000 7fb769ab0700 1 -- 172.21.15.1:0/1260426800 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb75c009a00 con 0x55bdcc6a3880 2023-11-16T16:46:47.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.485+0000 7fb769ab0700 1 --2- 172.21.15.1:0/1260426800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdcc6a3880 0x55bdcc6ac6d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb7540098e0 tx=0x7fb754009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b449fb79f914e563 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:47.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.485+0000 7fb74dffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:46:47.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.485+0000 7fb74dffb700 1 -- 172.21.15.1:0/1260426800 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb754028050 con 0x55bdcc6a3880 2023-11-16T16:46:47.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.485+0000 7fb74dffb700 1 -- 172.21.15.1:0/1260426800 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fb754023820 con 0x55bdcc6a3880 2023-11-16T16:46:47.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.485+0000 7fb7786a1b80 1 -- 172.21.15.1:0/1260426800 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bdcc691b20 con 0x55bdcc6a3880 2023-11-16T16:46:47.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.485+0000 7fb74dffb700 1 -- 172.21.15.1:0/1260426800 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb75401d040 con 0x55bdcc6a3880 2023-11-16T16:46:47.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.485+0000 7fb7786a1b80 10 client.16476 fetch_fsmap learned FSMap version 28 2023-11-16T16:46:47.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.485+0000 7fb7786a1b80 1 -- 172.21.15.1:0/1260426800 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55bdcc5b6190 con 0x55bdcc6a3880 2023-11-16T16:46:47.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.486+0000 7fb74dffb700 1 -- 172.21.15.1:0/1260426800 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb75403b020 con 0x55bdcc6a3880 2023-11-16T16:46:47.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.486+0000 7fb74dffb700 1 -- 172.21.15.1:0/1260426800 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fb75402eb50 con 0x55bdcc6a3880 2023-11-16T16:46:47.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.486+0000 7fb7786a1b80 10 client.16476 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:46:47.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.486+0000 7fb7786a1b80 10 client.16476 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-11-16T16:46:47.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.486+0000 7fb7786a1b80 20 client.16476 populate_metadata read hostname 'smithi001' 2023-11-16T16:46:47.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.486+0000 7fb7786a1b80 1 --2- 172.21.15.1:0/1260426800 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55bdcc6bcc70 0x55bdcc6bf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.486+0000 7fb7786a1b80 4 client.16476 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:46:47.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.486+0000 7fb7786a1b80 1 -- 172.21.15.1:0/1260426800 --> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bdcc5b20a0 con 0x55bdcc6bcc70 2023-11-16T16:46:47.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.486+0000 7fb76a2b1700 1 --2- 172.21.15.1:0/1260426800 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55bdcc6bcc70 0x55bdcc6bf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.487+0000 7fb74dffb700 10 client.16476 ms_handle_connect on v2:172.21.15.53:6838/1044753024 2023-11-16T16:46:47.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.487+0000 7fb76a2b1700 1 --2- 172.21.15.1:0/1260426800 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55bdcc6bcc70 0x55bdcc6bf080 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=90e5ef7b0c058aba server_cookie=a15ea9909624a7fc in_seq=0 out_seq=0 2023-11-16T16:46:47.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.488+0000 7fb74dffb700 1 -- 172.21.15.1:0/1260426800 <== mds.1 v2:172.21.15.53:6838/1044753024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bdcc5b20a0 con 0x55bdcc6bcc70 2023-11-16T16:46:47.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.488+0000 7fb74dffb700 10 client.16476 handle_command_reply: tid=0 2023-11-16T16:46:47.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.488+0000 7fb7786a1b80 1 client.16476 shutdown 2023-11-16T16:46:47.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.488+0000 7fb7786a1b80 1 -- 172.21.15.1:0/1260426800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdcc6a3880 msgr2=0x55bdcc6ac6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:47.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.488+0000 7fb7786a1b80 1 --2- 172.21.15.1:0/1260426800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdcc6a3880 0x55bdcc6ac6d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb7540098e0 tx=0x7fb754009cd0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.488+0000 7fb7786a1b80 1 -- 172.21.15.1:0/1260426800 shutdown_connections 2023-11-16T16:46:47.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.488+0000 7fb7786a1b80 1 --2- 172.21.15.1:0/1260426800 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55bdcc6bcc70 0x55bdcc6bf080 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.502 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.488+0000 7fb7786a1b80 1 --2- 172.21.15.1:0/1260426800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdcc6a7cc0 0x55bdcc6acd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.502 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.488+0000 7fb74dffb700 0 client.16476 ms_handle_reset on v2:172.21.15.53:6838/1044753024 2023-11-16T16:46:47.502 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.488+0000 7fb7786a1b80 1 --2- 172.21.15.1:0/1260426800 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bdcc6a2f70 0x55bdcc6af450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.502 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.488+0000 7fb7786a1b80 1 --2- 172.21.15.1:0/1260426800 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdcc6a3880 0x55bdcc6ac6d0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.502 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.488+0000 7fb7786a1b80 1 -- 172.21.15.1:0/1260426800 >> 172.21.15.1:0/1260426800 conn(0x55bdcc694360 msgr2=0x55bdcc6a9170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:47.502 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.488+0000 7fb7786a1b80 1 -- 172.21.15.1:0/1260426800 shutdown_connections 2023-11-16T16:46:47.502 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.488+0000 7fb7786a1b80 1 -- 172.21.15.1:0/1260426800 wait complete. 2023-11-16T16:46:47.503 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.488+0000 7fb7786a1b80 20 client.16476 trim_cache size 0 max 16384 2023-11-16T16:46:47.509 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.501+0000 7fb7786a1b80 1 Processor -- start 2023-11-16T16:46:47.509 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.501+0000 7fb7786a1b80 1 -- start start 2023-11-16T16:46:47.509 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.501+0000 7fb7786a1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bdcc6a2f70 0x55bdcc6b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.509 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.501+0000 7fb7786a1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdcc6a3880 0x55bdcc6b9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.509 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.501+0000 7fb7786a1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdcc6a7cc0 0x55bdcc6a90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.509 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.501+0000 7fb7786a1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdcc6a3fd0 con 0x55bdcc6a3880 2023-11-16T16:46:47.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.501+0000 7fb7786a1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdcc6adcb0 con 0x55bdcc6a7cc0 2023-11-16T16:46:47.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.501+0000 7fb7786a1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdcc6a6800 con 0x55bdcc6a2f70 2023-11-16T16:46:47.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.501+0000 7fb769ab0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdcc6a3880 0x55bdcc6b9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.501+0000 7fb769ab0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdcc6a3880 0x55bdcc6b9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45996/0 (socket says 172.21.15.1:45996) 2023-11-16T16:46:47.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.501+0000 7fb769ab0700 1 -- 172.21.15.1:0/1634667742 learned_addr learned my addr 172.21.15.1:0/1634667742 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:47.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.501+0000 7fb76aab2700 1 --2- 172.21.15.1:0/1634667742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdcc6a7cc0 0x55bdcc6a90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.501+0000 7fb76a2b1700 1 --2- 172.21.15.1:0/1634667742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bdcc6a2f70 0x55bdcc6b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.510 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.502+0000 7fb769ab0700 1 -- 172.21.15.1:0/1634667742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bdcc6a2f70 msgr2=0x55bdcc6b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:47.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.502+0000 7fb769ab0700 1 --2- 172.21.15.1:0/1634667742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bdcc6a2f70 0x55bdcc6b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.502+0000 7fb769ab0700 1 -- 172.21.15.1:0/1634667742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdcc6a7cc0 msgr2=0x55bdcc6a90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:47.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.502+0000 7fb769ab0700 1 --2- 172.21.15.1:0/1634667742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdcc6a7cc0 0x55bdcc6a90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.502+0000 7fb769ab0700 1 -- 172.21.15.1:0/1634667742 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdcc5b5ee0 con 0x55bdcc6a3880 2023-11-16T16:46:47.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.502+0000 7fb769ab0700 1 --2- 172.21.15.1:0/1634667742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdcc6a3880 0x55bdcc6b9050 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb7540036c0 tx=0x7fb754009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee639d0be22c5022 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:47.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.502+0000 7fb7597fa700 1 -- 172.21.15.1:0/1634667742 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75401d070 con 0x55bdcc6a3880 2023-11-16T16:46:47.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.502+0000 7fb7597fa700 1 -- 172.21.15.1:0/1634667742 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb75403b030 con 0x55bdcc6a3880 2023-11-16T16:46:47.511 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.502+0000 7fb7597fa700 1 -- 172.21.15.1:0/1634667742 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb754028040 con 0x55bdcc6a3880 2023-11-16T16:46:47.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.502+0000 7fb7786a1b80 1 -- 172.21.15.1:0/1634667742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdcc6a3880 msgr2=0x55bdcc6b9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:47.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.502+0000 7fb7786a1b80 1 --2- 172.21.15.1:0/1634667742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdcc6a3880 0x55bdcc6b9050 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb7540036c0 tx=0x7fb754009310 comp rx=0 tx=0).stop 2023-11-16T16:46:47.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.502+0000 7fb7786a1b80 1 -- 172.21.15.1:0/1634667742 shutdown_connections 2023-11-16T16:46:47.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.502+0000 7fb7786a1b80 1 --2- 172.21.15.1:0/1634667742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdcc6a7cc0 0x55bdcc6a90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.502+0000 7fb7786a1b80 1 --2- 172.21.15.1:0/1634667742 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bdcc6a2f70 0x55bdcc6b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.502+0000 7fb7786a1b80 1 --2- 172.21.15.1:0/1634667742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdcc6a3880 0x55bdcc6b9050 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.502+0000 7fb7786a1b80 1 -- 172.21.15.1:0/1634667742 >> 172.21.15.1:0/1634667742 conn(0x55bdcc694360 msgr2=0x55bdcc698550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:47.512 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.502+0000 7fb7786a1b80 1 -- 172.21.15.1:0/1634667742 shutdown_connections 2023-11-16T16:46:47.513 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.503+0000 7fb7786a1b80 1 -- 172.21.15.1:0/1634667742 wait complete. 2023-11-16T16:46:47.513 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.503+0000 7fb7786a1b80 1 -- start start 2023-11-16T16:46:47.513 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.503+0000 7fb7786a1b80 1 Processor -- start 2023-11-16T16:46:47.513 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.503+0000 7fb7786a1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bdcc6a2f70 0x55bdcc6aae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.513 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.503+0000 7fb7786a1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdcc6a3880 0x55bdcc6ab3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.513 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.503+0000 7fb7786a1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdcc6a7cc0 0x55bdcc6ab900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.513 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.503+0000 7fb7786a1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdcc6b43a0 con 0x55bdcc6a7cc0 2023-11-16T16:46:47.514 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.503+0000 7fb7786a1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdcc6abe40 con 0x55bdcc6a3880 2023-11-16T16:46:47.514 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.503+0000 7fb7786a1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdcc6abfd0 con 0x55bdcc6a2f70 2023-11-16T16:46:47.514 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.503+0000 7fb7786a1b80 10 client.0 _ll_register_callbacks cb 0x7fb75400bd90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:46:47.514 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.503+0000 7fb76aab2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdcc6a7cc0 0x55bdcc6ab900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.514 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.503+0000 7fb769ab0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdcc6a3880 0x55bdcc6ab3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.514 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.503+0000 7fb76a2b1700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bdcc6a2f70 0x55bdcc6aae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.514 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.503+0000 7fb769ab0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdcc6a3880 0x55bdcc6ab3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.1:40188/0 (socket says 172.21.15.1:40188) 2023-11-16T16:46:47.514 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.503+0000 7fb769ab0700 1 -- 172.21.15.1:0/2930299013 learned_addr learned my addr 172.21.15.1:0/2930299013 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:47.515 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.503+0000 7fb76aab2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdcc6a7cc0 0x55bdcc6ab900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46004/0 (socket says 172.21.15.1:46004) 2023-11-16T16:46:47.515 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.504+0000 7fb769ab0700 1 -- 172.21.15.1:0/2930299013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bdcc6a2f70 msgr2=0x55bdcc6aae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:47.515 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.504+0000 7fb769ab0700 1 --2- 172.21.15.1:0/2930299013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bdcc6a2f70 0x55bdcc6aae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.515 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.504+0000 7fb769ab0700 1 -- 172.21.15.1:0/2930299013 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdcc6a7cc0 msgr2=0x55bdcc6ab900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:47.515 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.504+0000 7fb769ab0700 1 --2- 172.21.15.1:0/2930299013 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdcc6a7cc0 0x55bdcc6ab900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.515 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.504+0000 7fb769ab0700 1 -- 172.21.15.1:0/2930299013 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bdcc5b6190 con 0x55bdcc6a3880 2023-11-16T16:46:47.515 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.504+0000 7fb76aab2700 1 --2- 172.21.15.1:0/2930299013 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdcc6a7cc0 0x55bdcc6ab900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:46:47.515 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.504+0000 7fb769ab0700 1 --2- 172.21.15.1:0/2930299013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdcc6a3880 0x55bdcc6ab3c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb75400c010 tx=0x7fb7540049f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cad87332658784a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:47.516 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.504+0000 7fb74dffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-11-16T16:46:47.516 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.504+0000 7fb74dffb700 1 -- 172.21.15.1:0/2930299013 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb754028070 con 0x55bdcc6a3880 2023-11-16T16:46:47.516 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.504+0000 7fb7786a1b80 1 -- 172.21.15.1:0/2930299013 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb75c00f050 con 0x55bdcc6a3880 2023-11-16T16:46:47.516 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.504+0000 7fb74dffb700 1 -- 172.21.15.1:0/2930299013 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fb75403c080 con 0x55bdcc6a3880 2023-11-16T16:46:47.516 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.504+0000 7fb74dffb700 1 -- 172.21.15.1:0/2930299013 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb75401d040 con 0x55bdcc6a3880 2023-11-16T16:46:47.516 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.504+0000 7fb7786a1b80 10 client.25994 fetch_fsmap learned FSMap version 28 2023-11-16T16:46:47.516 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.504+0000 7fb7786a1b80 1 -- 172.21.15.1:0/2930299013 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55bdcc406c60 con 0x55bdcc6a3880 2023-11-16T16:46:47.517 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.505+0000 7fb74dffb700 1 -- 172.21.15.1:0/2930299013 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb75c00f050 con 0x55bdcc6a3880 2023-11-16T16:46:47.517 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.505+0000 7fb74dffb700 1 -- 172.21.15.1:0/2930299013 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fb7540055a0 con 0x55bdcc6a3880 2023-11-16T16:46:47.517 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.505+0000 7fb7786a1b80 10 client.25994 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:46:47.517 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.505+0000 7fb7786a1b80 10 client.25994 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-11-16T16:46:47.517 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.505+0000 7fb7786a1b80 20 client.25994 populate_metadata read hostname 'smithi001' 2023-11-16T16:46:47.517 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.505+0000 7fb7786a1b80 1 --2- 172.21.15.1:0/2930299013 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55bdcc711110 0x55bdcc713520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.517 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.505+0000 7fb7786a1b80 4 client.25994 mds_command: new command op to 24493 tid=0[{"prefix": "damage ls"}] 2023-11-16T16:46:47.517 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.505+0000 7fb7786a1b80 1 -- 172.21.15.1:0/2930299013 --> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bdcc5be660 con 0x55bdcc711110 2023-11-16T16:46:47.518 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.505+0000 7fb76a2b1700 1 --2- 172.21.15.1:0/2930299013 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55bdcc711110 0x55bdcc713520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.518 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.506+0000 7fb74dffb700 10 client.25994 ms_handle_connect on v2:172.21.15.53:6838/1044753024 2023-11-16T16:46:47.518 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.506+0000 7fb76a2b1700 1 --2- 172.21.15.1:0/2930299013 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55bdcc711110 0x55bdcc713520 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d445e2a29f3014b3 server_cookie=3c6d660572d50f5f in_seq=0 out_seq=0 2023-11-16T16:46:47.518 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.506+0000 7fb74dffb700 1 -- 172.21.15.1:0/2930299013 <== mds.1 v2:172.21.15.53:6838/1044753024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bdcc5be660 con 0x55bdcc711110 2023-11-16T16:46:47.518 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.506+0000 7fb74dffb700 10 client.25994 handle_command_reply: tid=0 2023-11-16T16:46:47.518 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.506+0000 7fb7786a1b80 1 client.25994 shutdown 2023-11-16T16:46:47.518 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.506+0000 7fb7786a1b80 1 -- 172.21.15.1:0/2930299013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdcc6a3880 msgr2=0x55bdcc6ab3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:47.519 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.506+0000 7fb7786a1b80 1 --2- 172.21.15.1:0/2930299013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdcc6a3880 0x55bdcc6ab3c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb75400c010 tx=0x7fb7540049f0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.519 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.506+0000 7fb7786a1b80 1 -- 172.21.15.1:0/2930299013 shutdown_connections 2023-11-16T16:46:47.519 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.506+0000 7fb7786a1b80 1 --2- 172.21.15.1:0/2930299013 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdcc6a7cc0 0x55bdcc6ab900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.519 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.506+0000 7fb7786a1b80 1 --2- 172.21.15.1:0/2930299013 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bdcc6a2f70 0x55bdcc6aae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.519 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.506+0000 7fb7786a1b80 1 --2- 172.21.15.1:0/2930299013 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55bdcc711110 0x55bdcc713520 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.519 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.506+0000 7fb7786a1b80 1 --2- 172.21.15.1:0/2930299013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bdcc6a3880 0x55bdcc6ab3c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.519 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.506+0000 7fb7786a1b80 1 -- 172.21.15.1:0/2930299013 >> 172.21.15.1:0/2930299013 conn(0x55bdcc694360 msgr2=0x55bdcc6b9050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:47.519 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.506+0000 7fb74dffb700 0 client.25994 ms_handle_reset on v2:172.21.15.53:6838/1044753024 2023-11-16T16:46:47.520 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.506+0000 7fb7786a1b80 1 -- 172.21.15.1:0/2930299013 shutdown_connections 2023-11-16T16:46:47.520 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.506+0000 7fb7786a1b80 1 -- 172.21.15.1:0/2930299013 wait complete. 2023-11-16T16:46:47.520 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.506+0000 7fb7786a1b80 20 client.25994 trim_cache size 0 max 16384 2023-11-16T16:46:47.520 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2023-11-16T16:46:47.521 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.506+0000 7fb76c515700 1 -- 172.21.15.1:0/2957558398 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fb7500a12c0 msgr2=0x7fb7500a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:47.521 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.506+0000 7fb76c515700 1 --2- 172.21.15.1:0/2957558398 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fb7500a12c0 0x7fb7500a3770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb75400b580 tx=0x7fb75401a040 comp rx=0 tx=0).stop 2023-11-16T16:46:47.522 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.506+0000 7fb76c515700 1 -- 172.21.15.1:0/2957558398 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7640ff410 msgr2=0x7fb764196470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:47.522 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.506+0000 7fb76c515700 1 --2- 172.21.15.1:0/2957558398 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7640ff410 0x7fb764196470 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb76000e9a0 tx=0x7fb76000ed60 comp rx=0 tx=0).stop 2023-11-16T16:46:47.522 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.507+0000 7fb76c515700 1 -- 172.21.15.1:0/2957558398 shutdown_connections 2023-11-16T16:46:47.522 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.507+0000 7fb76c515700 1 --2- 172.21.15.1:0/2957558398 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fb7500a12c0 0x7fb7500a3770 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.522 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.507+0000 7fb76c515700 1 --2- 172.21.15.1:0/2957558398 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb764104450 0x7fb76419ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.522 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.507+0000 7fb76c515700 1 --2- 172.21.15.1:0/2957558398 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7640ff410 0x7fb764196470 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.522 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.507+0000 7fb76c515700 1 --2- 172.21.15.1:0/2957558398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7640ffdb0 0x7fb7641969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.523 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.507+0000 7fb76c515700 1 -- 172.21.15.1:0/2957558398 >> 172.21.15.1:0/2957558398 conn(0x7fb764076370 msgr2=0x7fb764068fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:47.523 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.507+0000 7fb76c515700 1 -- 172.21.15.1:0/2957558398 shutdown_connections 2023-11-16T16:46:47.523 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.507+0000 7fb76c515700 1 -- 172.21.15.1:0/2957558398 wait complete. 2023-11-16T16:46:47.523 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-16T16:46:47.651 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.649+0000 7f4a5b819700 1 -- 172.21.15.1:0/766314047 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a54101660 msgr2=0x7f4a54104e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:47.651 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.649+0000 7f4a5b819700 1 --2- 172.21.15.1:0/766314047 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a54101660 0x7f4a54104e90 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f4a500099c0 tx=0x7f4a50009cd0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.651 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.649+0000 7f4a5b819700 1 -- 172.21.15.1:0/766314047 shutdown_connections 2023-11-16T16:46:47.652 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.649+0000 7f4a5b819700 1 --2- 172.21.15.1:0/766314047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a541055c0 0x7f4a5410d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.652 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.649+0000 7f4a5b819700 1 --2- 172.21.15.1:0/766314047 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a54100cc0 0x7f4a54101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.652 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.649+0000 7f4a5b819700 1 --2- 172.21.15.1:0/766314047 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a54101660 0x7f4a54104e90 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.652 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.649+0000 7f4a5b819700 1 -- 172.21.15.1:0/766314047 >> 172.21.15.1:0/766314047 conn(0x7f4a54077560 msgr2=0x7f4a54077960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:47.652 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.649+0000 7f4a5b819700 1 -- 172.21.15.1:0/766314047 shutdown_connections 2023-11-16T16:46:47.652 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.649+0000 7f4a5b819700 1 -- 172.21.15.1:0/766314047 wait complete. 2023-11-16T16:46:47.654 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.650+0000 7f4a5b819700 1 Processor -- start 2023-11-16T16:46:47.654 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.650+0000 7f4a5b819700 1 -- start start 2023-11-16T16:46:47.655 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.650+0000 7f4a5b819700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a54100cc0 0x7f4a54196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.655 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.650+0000 7f4a5b819700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a54101660 0x7f4a54196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.655 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.650+0000 7f4a5b819700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a541055c0 0x7f4a5419ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.655 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.650+0000 7f4a5b819700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5419b3e0 con 0x7f4a54100cc0 2023-11-16T16:46:47.655 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.650+0000 7f4a5b819700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5419b550 con 0x7f4a54101660 2023-11-16T16:46:47.655 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.650+0000 7f4a5b819700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5419b710 con 0x7f4a541055c0 2023-11-16T16:46:47.655 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.650+0000 7f4a595b5700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a54100cc0 0x7f4a54196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.656 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.650+0000 7f4a595b5700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a54100cc0 0x7f4a54196400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46014/0 (socket says 172.21.15.1:46014) 2023-11-16T16:46:47.656 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.650+0000 7f4a595b5700 1 -- 172.21.15.1:0/2307162570 learned_addr learned my addr 172.21.15.1:0/2307162570 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:47.656 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.650+0000 7f4a58db4700 1 --2- 172.21.15.1:0/2307162570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a54101660 0x7f4a54196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.656 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.650+0000 7f4a59db6700 1 --2- 172.21.15.1:0/2307162570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a541055c0 0x7f4a5419ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.656 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.651+0000 7f4a595b5700 1 -- 172.21.15.1:0/2307162570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a541055c0 msgr2=0x7f4a5419ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:47.656 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.651+0000 7f4a595b5700 1 --2- 172.21.15.1:0/2307162570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a541055c0 0x7f4a5419ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.657 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.651+0000 7f4a595b5700 1 -- 172.21.15.1:0/2307162570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a54101660 msgr2=0x7f4a54196940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:47.657 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.651+0000 7f4a595b5700 1 --2- 172.21.15.1:0/2307162570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a54101660 0x7f4a54196940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.657 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.651+0000 7f4a595b5700 1 -- 172.21.15.1:0/2307162570 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a500095f0 con 0x7f4a54100cc0 2023-11-16T16:46:47.657 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.651+0000 7f4a59db6700 1 --2- 172.21.15.1:0/2307162570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a541055c0 0x7f4a5419ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:46:47.657 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.651+0000 7f4a595b5700 1 --2- 172.21.15.1:0/2307162570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a54100cc0 0x7f4a54196400 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f4a4400e9a0 tx=0x7f4a4400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:47.657 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.651+0000 7f4a4a7fc700 1 -- 172.21.15.1:0/2307162570 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a44009790 con 0x7f4a54100cc0 2023-11-16T16:46:47.657 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.651+0000 7f4a4a7fc700 1 -- 172.21.15.1:0/2307162570 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a4400cbd0 con 0x7f4a54100cc0 2023-11-16T16:46:47.658 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.651+0000 7f4a5b819700 1 -- 172.21.15.1:0/2307162570 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a54110340 con 0x7f4a54100cc0 2023-11-16T16:46:47.658 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.651+0000 7f4a4a7fc700 1 -- 172.21.15.1:0/2307162570 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a44018680 con 0x7f4a54100cc0 2023-11-16T16:46:47.658 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.651+0000 7f4a5b819700 1 -- 172.21.15.1:0/2307162570 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a541101b0 con 0x7f4a54100cc0 2023-11-16T16:46:47.664 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.653+0000 7f4a4a7fc700 1 -- 172.21.15.1:0/2307162570 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f4a4400cd40 con 0x7f4a54100cc0 2023-11-16T16:46:47.665 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.654+0000 7f4a4a7fc700 1 --2- 172.21.15.1:0/2307162570 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f4a400a12c0 0x7f4a400a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.665 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.654+0000 7f4a4a7fc700 1 -- 172.21.15.1:0/2307162570 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f4a44014070 con 0x7f4a54100cc0 2023-11-16T16:46:47.665 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.654+0000 7f4a58db4700 1 --2- 172.21.15.1:0/2307162570 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f4a400a12c0 0x7f4a400a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.665 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.654+0000 7f4a58db4700 1 --2- 172.21.15.1:0/2307162570 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f4a400a12c0 0x7f4a400a3770 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4a500058e0 tx=0x7f4a50003680 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:47.665 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.655+0000 7f4a679a5b80 1 Processor -- start 2023-11-16T16:46:47.665 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.655+0000 7f4a679a5b80 1 -- start start 2023-11-16T16:46:47.665 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.655+0000 7f4a679a5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556891dfab60 0x556891dfaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.666 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.655+0000 7f4a679a5b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556891dfb470 0x556891dff370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.666 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.655+0000 7f4a679a5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556891dff8b0 0x556891dffd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.666 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.655+0000 7f4a679a5b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556891d00060 con 0x556891dfb470 2023-11-16T16:46:47.666 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.655+0000 7f4a679a5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556891d528a0 con 0x556891dfab60 2023-11-16T16:46:47.666 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.655+0000 7f4a679a5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556891d4bd30 con 0x556891dff8b0 2023-11-16T16:46:47.666 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.655+0000 7f4a58db4700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556891dfb470 0x556891dff370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.666 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.655+0000 7f4a58db4700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556891dfb470 0x556891dff370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46024/0 (socket says 172.21.15.1:46024) 2023-11-16T16:46:47.667 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.655+0000 7f4a58db4700 1 -- 172.21.15.1:0/2811667456 learned_addr learned my addr 172.21.15.1:0/2811667456 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:47.667 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.655+0000 7f4a59db6700 1 --2- 172.21.15.1:0/2811667456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556891dff8b0 0x556891dffd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.667 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.655+0000 7f4a595b5700 1 --2- 172.21.15.1:0/2811667456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556891dfab60 0x556891dfaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.667 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.655+0000 7f4a58db4700 1 -- 172.21.15.1:0/2811667456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556891dff8b0 msgr2=0x556891dffd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:47.667 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.655+0000 7f4a58db4700 1 --2- 172.21.15.1:0/2811667456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556891dff8b0 0x556891dffd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.667 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.655+0000 7f4a58db4700 1 -- 172.21.15.1:0/2811667456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556891dfab60 msgr2=0x556891dfaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:47.667 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.655+0000 7f4a58db4700 1 --2- 172.21.15.1:0/2811667456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556891dfab60 0x556891dfaf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.667 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.656+0000 7f4a58db4700 1 -- 172.21.15.1:0/2811667456 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556891bcd630 con 0x556891dfb470 2023-11-16T16:46:47.668 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.656+0000 7f4a59db6700 1 --2- 172.21.15.1:0/2811667456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556891dff8b0 0x556891dffd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:46:47.668 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.656+0000 7f4a58db4700 1 --2- 172.21.15.1:0/2811667456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556891dfb470 0x556891dff370 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f4a5000b580 tx=0x7f4a5001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1485ea1e7584226 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:47.668 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.656+0000 7f4a3f7fe700 1 -- 172.21.15.1:0/2811667456 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5002a070 con 0x556891dfb470 2023-11-16T16:46:47.668 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.656+0000 7f4a3f7fe700 1 -- 172.21.15.1:0/2811667456 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a5001e040 con 0x556891dfb470 2023-11-16T16:46:47.668 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.656+0000 7f4a3f7fe700 1 -- 172.21.15.1:0/2811667456 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5000f6a0 con 0x556891dfb470 2023-11-16T16:46:47.668 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.656+0000 7f4a679a5b80 1 -- 172.21.15.1:0/2811667456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556891dfb470 msgr2=0x556891dff370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:47.668 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.656+0000 7f4a679a5b80 1 --2- 172.21.15.1:0/2811667456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556891dfb470 0x556891dff370 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f4a5000b580 tx=0x7f4a5001f000 comp rx=0 tx=0).stop 2023-11-16T16:46:47.668 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.657+0000 7f4a679a5b80 1 -- 172.21.15.1:0/2811667456 shutdown_connections 2023-11-16T16:46:47.669 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.657+0000 7f4a679a5b80 1 --2- 172.21.15.1:0/2811667456 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556891dff8b0 0x556891dffd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.669 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.657+0000 7f4a679a5b80 1 --2- 172.21.15.1:0/2811667456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556891dfab60 0x556891dfaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.669 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.657+0000 7f4a679a5b80 1 --2- 172.21.15.1:0/2811667456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556891dfb470 0x556891dff370 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.669 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.657+0000 7f4a679a5b80 1 -- 172.21.15.1:0/2811667456 >> 172.21.15.1:0/2811667456 conn(0x556891df0360 msgr2=0x556891df5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:47.669 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.657+0000 7f4a679a5b80 1 -- 172.21.15.1:0/2811667456 shutdown_connections 2023-11-16T16:46:47.669 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.657+0000 7f4a679a5b80 1 -- 172.21.15.1:0/2811667456 wait complete. 2023-11-16T16:46:47.669 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.657+0000 7f4a679a5b80 1 -- start start 2023-11-16T16:46:47.670 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.657+0000 7f4a679a5b80 1 Processor -- start 2023-11-16T16:46:47.670 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.657+0000 7f4a679a5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556891dfab60 0x556891e07050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.670 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.657+0000 7f4a679a5b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556891dfb470 0x556891e042d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.670 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.657+0000 7f4a679a5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556891dff8b0 0x556891e04950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.670 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.657+0000 7f4a679a5b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556891ced8e0 con 0x556891dfb470 2023-11-16T16:46:47.670 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.657+0000 7f4a679a5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556891ced580 con 0x556891dff8b0 2023-11-16T16:46:47.670 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.657+0000 7f4a679a5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556891bcf920 con 0x556891dfab60 2023-11-16T16:46:47.670 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.658+0000 7f4a679a5b80 10 client.0 _ll_register_callbacks cb 0x556891d0b780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:46:47.671 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.658+0000 7f4a58db4700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556891dfb470 0x556891e042d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.671 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.658+0000 7f4a58db4700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556891dfb470 0x556891e042d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46030/0 (socket says 172.21.15.1:46030) 2023-11-16T16:46:47.671 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.658+0000 7f4a58db4700 1 -- 172.21.15.1:0/3140775317 learned_addr learned my addr 172.21.15.1:0/3140775317 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:47.671 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.658+0000 7f4a59db6700 1 --2- 172.21.15.1:0/3140775317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556891dff8b0 0x556891e04950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.671 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.658+0000 7f4a595b5700 1 --2- 172.21.15.1:0/3140775317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556891dfab60 0x556891e07050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.671 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.658+0000 7f4a58db4700 1 -- 172.21.15.1:0/3140775317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556891dfab60 msgr2=0x556891e07050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:47.671 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.658+0000 7f4a58db4700 1 --2- 172.21.15.1:0/3140775317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556891dfab60 0x556891e07050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.671 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.658+0000 7f4a58db4700 1 -- 172.21.15.1:0/3140775317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556891dff8b0 msgr2=0x556891e04950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:47.672 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.658+0000 7f4a58db4700 1 --2- 172.21.15.1:0/3140775317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556891dff8b0 0x556891e04950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.672 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.658+0000 7f4a58db4700 1 -- 172.21.15.1:0/3140775317 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a5410d980 con 0x556891dfb470 2023-11-16T16:46:47.672 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.658+0000 7f4a58db4700 1 --2- 172.21.15.1:0/3140775317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556891dfb470 0x556891e042d0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009d10 tx=0x7f4a5002fe70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1133c8d878bf2b61 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:47.672 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.658+0000 7f4a3cff9700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:46:47.672 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.658+0000 7f4a3cff9700 1 -- 172.21.15.1:0/3140775317 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5002a040 con 0x556891dfb470 2023-11-16T16:46:47.672 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.658+0000 7f4a3cff9700 1 -- 172.21.15.1:0/3140775317 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f4a5000f6e0 con 0x556891dfb470 2023-11-16T16:46:47.672 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.658+0000 7f4a679a5b80 1 -- 172.21.15.1:0/3140775317 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556891dedb20 con 0x556891dfb470 2023-11-16T16:46:47.672 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.658+0000 7f4a3cff9700 1 -- 172.21.15.1:0/3140775317 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a5000fd70 con 0x556891dfb470 2023-11-16T16:46:47.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.658+0000 7f4a679a5b80 10 client.16494 fetch_fsmap learned FSMap version 28 2023-11-16T16:46:47.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.658+0000 7f4a679a5b80 1 -- 172.21.15.1:0/3140775317 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556891d12190 con 0x556891dfb470 2023-11-16T16:46:47.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.659+0000 7f4a3cff9700 1 -- 172.21.15.1:0/3140775317 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556891dedb20 con 0x556891dfb470 2023-11-16T16:46:47.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.659+0000 7f4a3cff9700 1 -- 172.21.15.1:0/3140775317 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f4a50036030 con 0x556891dfb470 2023-11-16T16:46:47.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.659+0000 7f4a679a5b80 10 client.16494 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:46:47.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.659+0000 7f4a679a5b80 10 client.16494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:46:47.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.659+0000 7f4a679a5b80 20 client.16494 populate_metadata read hostname 'smithi001' 2023-11-16T16:46:47.674 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.659+0000 7f4a679a5b80 1 --2- 172.21.15.1:0/3140775317 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x556891e14400 0x556891e167e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.674 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.659+0000 7f4a679a5b80 4 client.16494 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:46:47.674 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.659+0000 7f4a679a5b80 1 -- 172.21.15.1:0/3140775317 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556891d0e0a0 con 0x556891e14400 2023-11-16T16:46:47.674 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.659+0000 7f4a595b5700 1 --2- 172.21.15.1:0/3140775317 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x556891e14400 0x556891e167e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.674 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.659+0000 7f4a3cff9700 10 client.16494 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:46:47.674 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.659+0000 7f4a595b5700 1 --2- 172.21.15.1:0/3140775317 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x556891e14400 0x556891e167e0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3c8351c2b1017d1 server_cookie=b5959fed0e1948f5 in_seq=0 out_seq=0 2023-11-16T16:46:47.674 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.660+0000 7f4a3cff9700 1 -- 172.21.15.1:0/3140775317 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556891d0e0a0 con 0x556891e14400 2023-11-16T16:46:47.674 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.660+0000 7f4a3cff9700 10 client.16494 handle_command_reply: tid=0 2023-11-16T16:46:47.675 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.660+0000 7f4a679a5b80 1 client.16494 shutdown 2023-11-16T16:46:47.675 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.660+0000 7f4a679a5b80 1 -- 172.21.15.1:0/3140775317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556891dfb470 msgr2=0x556891e042d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:47.675 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.660+0000 7f4a679a5b80 1 --2- 172.21.15.1:0/3140775317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556891dfb470 0x556891e042d0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009d10 tx=0x7f4a5002fe70 comp rx=0 tx=0).stop 2023-11-16T16:46:47.675 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.660+0000 7f4a679a5b80 1 -- 172.21.15.1:0/3140775317 shutdown_connections 2023-11-16T16:46:47.675 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.660+0000 7f4a679a5b80 1 --2- 172.21.15.1:0/3140775317 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x556891e14400 0x556891e167e0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.677 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.660+0000 7f4a679a5b80 1 --2- 172.21.15.1:0/3140775317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556891dff8b0 0x556891e04950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.677 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.660+0000 7f4a3cff9700 0 client.16494 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:46:47.677 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.660+0000 7f4a679a5b80 1 --2- 172.21.15.1:0/3140775317 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556891dfab60 0x556891e07050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.678 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.660+0000 7f4a679a5b80 1 --2- 172.21.15.1:0/3140775317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556891dfb470 0x556891e042d0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.678 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.660+0000 7f4a679a5b80 1 -- 172.21.15.1:0/3140775317 >> 172.21.15.1:0/3140775317 conn(0x556891df0360 msgr2=0x556891e00d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:47.678 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.660+0000 7f4a679a5b80 1 -- 172.21.15.1:0/3140775317 shutdown_connections 2023-11-16T16:46:47.678 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.660+0000 7f4a679a5b80 1 -- 172.21.15.1:0/3140775317 wait complete. 2023-11-16T16:46:47.679 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.661+0000 7f4a679a5b80 20 client.16494 trim_cache size 0 max 16384 2023-11-16T16:46:47.685 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.674+0000 7f4a679a5b80 1 Processor -- start 2023-11-16T16:46:47.685 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.674+0000 7f4a679a5b80 1 -- start start 2023-11-16T16:46:47.685 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.674+0000 7f4a679a5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556891dfab60 0x556891e10be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.674+0000 7f4a679a5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556891dfb470 0x556891dfd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.674+0000 7f4a679a5b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556891dff8b0 0x556891dfde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.674+0000 7f4a679a5b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556891e05440 con 0x556891dff8b0 2023-11-16T16:46:47.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.674+0000 7f4a679a5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556891d0e0a0 con 0x556891dfb470 2023-11-16T16:46:47.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.674+0000 7f4a679a5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556891dfbbc0 con 0x556891dfab60 2023-11-16T16:46:47.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.674+0000 7f4a59db6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556891dff8b0 0x556891dfde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.686 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.674+0000 7f4a59db6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556891dff8b0 0x556891dfde70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46040/0 (socket says 172.21.15.1:46040) 2023-11-16T16:46:47.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.674+0000 7f4a59db6700 1 -- 172.21.15.1:0/2946601680 learned_addr learned my addr 172.21.15.1:0/2946601680 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:47.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.674+0000 7f4a595b5700 1 --2- 172.21.15.1:0/2946601680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556891dfab60 0x556891e10be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.674+0000 7f4a58db4700 1 --2- 172.21.15.1:0/2946601680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556891dfb470 0x556891dfd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.674+0000 7f4a59db6700 1 -- 172.21.15.1:0/2946601680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556891dfab60 msgr2=0x556891e10be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:47.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.674+0000 7f4a59db6700 1 --2- 172.21.15.1:0/2946601680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556891dfab60 0x556891e10be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.674+0000 7f4a59db6700 1 -- 172.21.15.1:0/2946601680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556891dfb470 msgr2=0x556891dfd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:47.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.674+0000 7f4a59db6700 1 --2- 172.21.15.1:0/2946601680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556891dfb470 0x556891dfd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.687 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.674+0000 7f4a59db6700 1 -- 172.21.15.1:0/2946601680 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556891df1b00 con 0x556891dff8b0 2023-11-16T16:46:47.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.675+0000 7f4a59db6700 1 --2- 172.21.15.1:0/2946601680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556891dff8b0 0x556891dfde70 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c009f80 tx=0x7f4a4c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a15894b8f7baebb server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:47.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.675+0000 7f4a3ffff700 1 -- 172.21.15.1:0/2946601680 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c017440 con 0x556891dff8b0 2023-11-16T16:46:47.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.675+0000 7f4a3ffff700 1 -- 172.21.15.1:0/2946601680 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a4c00f070 con 0x556891dff8b0 2023-11-16T16:46:47.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.675+0000 7f4a3ffff700 1 -- 172.21.15.1:0/2946601680 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c016840 con 0x556891dff8b0 2023-11-16T16:46:47.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.675+0000 7f4a679a5b80 1 -- 172.21.15.1:0/2946601680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556891dff8b0 msgr2=0x556891dfde70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:47.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.675+0000 7f4a679a5b80 1 --2- 172.21.15.1:0/2946601680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556891dff8b0 0x556891dfde70 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c009f80 tx=0x7f4a4c009040 comp rx=0 tx=0).stop 2023-11-16T16:46:47.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.675+0000 7f4a679a5b80 1 -- 172.21.15.1:0/2946601680 shutdown_connections 2023-11-16T16:46:47.688 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.675+0000 7f4a679a5b80 1 --2- 172.21.15.1:0/2946601680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556891dff8b0 0x556891dfde70 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.675+0000 7f4a679a5b80 1 --2- 172.21.15.1:0/2946601680 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556891dfab60 0x556891e10be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.675+0000 7f4a679a5b80 1 --2- 172.21.15.1:0/2946601680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556891dfb470 0x556891dfd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.676+0000 7f4a679a5b80 1 -- 172.21.15.1:0/2946601680 >> 172.21.15.1:0/2946601680 conn(0x556891df0360 msgr2=0x556891e01c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:47.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.676+0000 7f4a679a5b80 1 -- 172.21.15.1:0/2946601680 shutdown_connections 2023-11-16T16:46:47.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.676+0000 7f4a679a5b80 1 -- 172.21.15.1:0/2946601680 wait complete. 2023-11-16T16:46:47.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.676+0000 7f4a679a5b80 1 -- start start 2023-11-16T16:46:47.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.676+0000 7f4a679a5b80 1 Processor -- start 2023-11-16T16:46:47.690 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.677+0000 7f4a679a5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556891dfab60 0x556891e5c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.690 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.677+0000 7f4a679a5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556891dfb470 0x556891e5c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.690 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.677+0000 7f4a679a5b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556891dff8b0 0x556891e672a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.690 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.677+0000 7f4a679a5b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556891e5ceb0 con 0x556891dff8b0 2023-11-16T16:46:47.690 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.677+0000 7f4a679a5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556891e5d020 con 0x556891dfab60 2023-11-16T16:46:47.690 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.677+0000 7f4a679a5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556891e677e0 con 0x556891dfb470 2023-11-16T16:46:47.690 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.677+0000 7f4a679a5b80 10 client.0 _ll_register_callbacks cb 0x7f4a5000b830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:46:47.691 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.677+0000 7f4a59db6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556891dff8b0 0x556891e672a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.691 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.677+0000 7f4a58db4700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556891dfb470 0x556891e5c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.691 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.677+0000 7f4a595b5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556891dfab60 0x556891e5c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.691 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.677+0000 7f4a58db4700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556891dfb470 0x556891e5c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:42726/0 (socket says 172.21.15.1:42726) 2023-11-16T16:46:47.691 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.677+0000 7f4a59db6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556891dff8b0 0x556891e672a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46046/0 (socket says 172.21.15.1:46046) 2023-11-16T16:46:47.691 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.677+0000 7f4a595b5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556891dfab60 0x556891e5c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.1:40228/0 (socket says 172.21.15.1:40228) 2023-11-16T16:46:47.691 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.677+0000 7f4a58db4700 1 -- 172.21.15.1:0/613354738 learned_addr learned my addr 172.21.15.1:0/613354738 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:47.691 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.677+0000 7f4a59db6700 1 -- 172.21.15.1:0/613354738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556891dfb470 msgr2=0x556891e5c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:47.692 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.677+0000 7f4a59db6700 1 --2- 172.21.15.1:0/613354738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556891dfb470 0x556891e5c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.692 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.677+0000 7f4a59db6700 1 -- 172.21.15.1:0/613354738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556891dfab60 msgr2=0x556891e5c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:47.692 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.677+0000 7f4a59db6700 1 --2- 172.21.15.1:0/613354738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556891dfab60 0x556891e5c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.692 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.677+0000 7f4a59db6700 1 -- 172.21.15.1:0/613354738 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a4c009c60 con 0x556891dff8b0 2023-11-16T16:46:47.692 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.677+0000 7f4a58db4700 1 --2- 172.21.15.1:0/613354738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556891dfb470 0x556891e5c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:46:47.692 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.677+0000 7f4a595b5700 1 --2- 172.21.15.1:0/613354738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556891dfab60 0x556891e5c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:46:47.692 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.677+0000 7f4a59db6700 1 --2- 172.21.15.1:0/613354738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556891dff8b0 0x556891e672a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c000f80 tx=0x7f4a4c017b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde30a31b1dbaa7c server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:47.693 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.677+0000 7f4a3cff9700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:46:47.693 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.679+0000 7f4a3cff9700 1 -- 172.21.15.1:0/613354738 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c020860 con 0x556891dff8b0 2023-11-16T16:46:47.693 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.679+0000 7f4a679a5b80 1 -- 172.21.15.1:0/613354738 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a5001e040 con 0x556891dff8b0 2023-11-16T16:46:47.693 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.679+0000 7f4a3cff9700 1 -- 172.21.15.1:0/613354738 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f4a4c017e40 con 0x556891dff8b0 2023-11-16T16:46:47.693 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.679+0000 7f4a3cff9700 1 -- 172.21.15.1:0/613354738 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a4c0209c0 con 0x556891dff8b0 2023-11-16T16:46:47.693 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.679+0000 7f4a679a5b80 10 client.16500 fetch_fsmap learned FSMap version 28 2023-11-16T16:46:47.693 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.679+0000 7f4a679a5b80 1 -- 172.21.15.1:0/613354738 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556891b79de0 con 0x556891dff8b0 2023-11-16T16:46:47.694 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.679+0000 7f4a3cff9700 1 -- 172.21.15.1:0/613354738 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a4c035020 con 0x556891dff8b0 2023-11-16T16:46:47.694 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.679+0000 7f4a3cff9700 1 -- 172.21.15.1:0/613354738 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f4a4c014050 con 0x556891dff8b0 2023-11-16T16:46:47.694 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.680+0000 7f4a679a5b80 10 client.16500 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:46:47.694 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.680+0000 7f4a679a5b80 10 client.16500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:46:47.694 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.680+0000 7f4a679a5b80 20 client.16500 populate_metadata read hostname 'smithi001' 2023-11-16T16:46:47.694 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.680+0000 7f4a679a5b80 1 --2- 172.21.15.1:0/613354738 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x556891e68e60 0x556891e6b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:47.694 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.680+0000 7f4a679a5b80 4 client.16500 mds_command: new command op to 24521 tid=0[{"prefix": "damage ls"}] 2023-11-16T16:46:47.694 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.680+0000 7f4a679a5b80 1 -- 172.21.15.1:0/613354738 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556891d1a660 con 0x556891e68e60 2023-11-16T16:46:47.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.680+0000 7f4a595b5700 1 --2- 172.21.15.1:0/613354738 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x556891e68e60 0x556891e6b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:47.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.680+0000 7f4a3cff9700 10 client.16500 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:46:47.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.680+0000 7f4a595b5700 1 --2- 172.21.15.1:0/613354738 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x556891e68e60 0x556891e6b270 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4356d69b66eca579 server_cookie=6d2f83f2e043357c in_seq=0 out_seq=0 2023-11-16T16:46:47.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.680+0000 7f4a3cff9700 1 -- 172.21.15.1:0/613354738 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556891d1a660 con 0x556891e68e60 2023-11-16T16:46:47.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.680+0000 7f4a3cff9700 10 client.16500 handle_command_reply: tid=0 2023-11-16T16:46:47.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.680+0000 7f4a679a5b80 1 client.16500 shutdown 2023-11-16T16:46:47.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.681+0000 7f4a679a5b80 1 -- 172.21.15.1:0/613354738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556891dff8b0 msgr2=0x556891e672a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:47.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.681+0000 7f4a679a5b80 1 --2- 172.21.15.1:0/613354738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556891dff8b0 0x556891e672a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c000f80 tx=0x7f4a4c017b10 comp rx=0 tx=0).stop 2023-11-16T16:46:47.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.681+0000 7f4a679a5b80 1 -- 172.21.15.1:0/613354738 shutdown_connections 2023-11-16T16:46:47.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.681+0000 7f4a679a5b80 1 --2- 172.21.15.1:0/613354738 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x556891e68e60 0x556891e6b270 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.681+0000 7f4a679a5b80 1 --2- 172.21.15.1:0/613354738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556891dff8b0 0x556891e672a0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.681+0000 7f4a679a5b80 1 --2- 172.21.15.1:0/613354738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556891dfab60 0x556891e5c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.681+0000 7f4a679a5b80 1 --2- 172.21.15.1:0/613354738 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x556891dfb470 0x556891e5c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.681+0000 7f4a3cff9700 0 client.16500 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:46:47.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.681+0000 7f4a679a5b80 1 -- 172.21.15.1:0/613354738 >> 172.21.15.1:0/613354738 conn(0x556891df0360 msgr2=0x556891dfd930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:47.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.681+0000 7f4a679a5b80 1 -- 172.21.15.1:0/613354738 shutdown_connections 2023-11-16T16:46:47.697 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.681+0000 7f4a679a5b80 1 -- 172.21.15.1:0/613354738 wait complete. 2023-11-16T16:46:47.697 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.681+0000 7f4a679a5b80 20 client.16500 trim_cache size 0 max 16384 2023-11-16T16:46:47.697 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2023-11-16T16:46:47.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.681+0000 7f4a5b819700 1 -- 172.21.15.1:0/2307162570 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f4a400a12c0 msgr2=0x7f4a400a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:47.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.681+0000 7f4a5b819700 1 --2- 172.21.15.1:0/2307162570 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f4a400a12c0 0x7f4a400a3770 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4a500058e0 tx=0x7f4a50003680 comp rx=0 tx=0).stop 2023-11-16T16:46:47.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.681+0000 7f4a5b819700 1 -- 172.21.15.1:0/2307162570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a54100cc0 msgr2=0x7f4a54196400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:47.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.681+0000 7f4a5b819700 1 --2- 172.21.15.1:0/2307162570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a54100cc0 0x7f4a54196400 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f4a4400e9a0 tx=0x7f4a4400ecb0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.681+0000 7f4a5b819700 1 -- 172.21.15.1:0/2307162570 shutdown_connections 2023-11-16T16:46:47.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.681+0000 7f4a5b819700 1 --2- 172.21.15.1:0/2307162570 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f4a400a12c0 0x7f4a400a3770 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.681+0000 7f4a5b819700 1 --2- 172.21.15.1:0/2307162570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f4a541055c0 0x7f4a5419ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.681+0000 7f4a5b819700 1 --2- 172.21.15.1:0/2307162570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a54100cc0 0x7f4a54196400 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.681+0000 7f4a5b819700 1 --2- 172.21.15.1:0/2307162570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a54101660 0x7f4a54196940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:47.700 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.681+0000 7f4a5b819700 1 -- 172.21.15.1:0/2307162570 >> 172.21.15.1:0/2307162570 conn(0x7f4a54077560 msgr2=0x7f4a54100250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:47.700 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.681+0000 7f4a5b819700 1 -- 172.21.15.1:0/2307162570 shutdown_connections 2023-11-16T16:46:47.700 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:47.681+0000 7f4a5b819700 1 -- 172.21.15.1:0/2307162570 wait complete. 2023-11-16T16:46:47.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:47 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/2402801168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:48.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:47 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/2402801168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:48.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:47 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/2402801168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:46:48.700 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-16T16:46:48.701 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-16T16:46:48.848 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.844+0000 7f5e076c1700 1 -- 172.21.15.1:0/3766882752 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e001002a0 msgr2=0x7f5e0010bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:48.848 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.844+0000 7f5e076c1700 1 --2- 172.21.15.1:0/3766882752 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e001002a0 0x7f5e0010bfd0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f5df800b0d0 tx=0x7f5df800b3e0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.848 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.844+0000 7f5e076c1700 1 -- 172.21.15.1:0/3766882752 shutdown_connections 2023-11-16T16:46:48.848 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.844+0000 7f5e076c1700 1 --2- 172.21.15.1:0/3766882752 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e001002a0 0x7f5e0010bfd0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.848 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.844+0000 7f5e076c1700 1 --2- 172.21.15.1:0/3766882752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e00104140 0x7f5e00104510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.844+0000 7f5e076c1700 1 --2- 172.21.15.1:0/3766882752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e000ff8f0 0x7f5e000ffd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.844+0000 7f5e076c1700 1 -- 172.21.15.1:0/3766882752 >> 172.21.15.1:0/3766882752 conn(0x7f5e00075880 msgr2=0x7f5e00075c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:48.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.844+0000 7f5e076c1700 1 -- 172.21.15.1:0/3766882752 shutdown_connections 2023-11-16T16:46:48.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.844+0000 7f5e076c1700 1 -- 172.21.15.1:0/3766882752 wait complete. 2023-11-16T16:46:48.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.845+0000 7f5e076c1700 1 Processor -- start 2023-11-16T16:46:48.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.845+0000 7f5e076c1700 1 -- start start 2023-11-16T16:46:48.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.845+0000 7f5e076c1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e000ff8f0 0x7f5e00197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:48.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.845+0000 7f5e076c1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e001002a0 0x7f5e001977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:48.850 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.845+0000 7f5e076c1700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e00104140 0x7f5e001a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:48.850 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.845+0000 7f5e076c1700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e00198110 con 0x7f5e001002a0 2023-11-16T16:46:48.850 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.845+0000 7f5e076c1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e001a2850 con 0x7f5e000ff8f0 2023-11-16T16:46:48.850 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.845+0000 7f5e076c1700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e001a2a10 con 0x7f5e00104140 2023-11-16T16:46:48.850 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.845+0000 7f5e04c5c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e001002a0 0x7f5e001977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:48.850 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.845+0000 7f5e04c5c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e001002a0 0x7f5e001977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46058/0 (socket says 172.21.15.1:46058) 2023-11-16T16:46:48.850 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.845+0000 7f5e04c5c700 1 -- 172.21.15.1:0/4165732215 learned_addr learned my addr 172.21.15.1:0/4165732215 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:48.850 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.845+0000 7f5e0545d700 1 --2- 172.21.15.1:0/4165732215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e000ff8f0 0x7f5e00197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:48.851 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.845+0000 7f5e05c5e700 1 --2- 172.21.15.1:0/4165732215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e00104140 0x7f5e001a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:48.851 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.845+0000 7f5e04c5c700 1 -- 172.21.15.1:0/4165732215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e00104140 msgr2=0x7f5e001a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:48.851 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.845+0000 7f5e04c5c700 1 --2- 172.21.15.1:0/4165732215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e00104140 0x7f5e001a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.851 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.845+0000 7f5e04c5c700 1 -- 172.21.15.1:0/4165732215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e000ff8f0 msgr2=0x7f5e00197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:48.851 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.845+0000 7f5e04c5c700 1 --2- 172.21.15.1:0/4165732215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e000ff8f0 0x7f5e00197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.851 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.845+0000 7f5e04c5c700 1 -- 172.21.15.1:0/4165732215 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df8009d00 con 0x7f5e001002a0 2023-11-16T16:46:48.851 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.845+0000 7f5e05c5e700 1 --2- 172.21.15.1:0/4165732215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e00104140 0x7f5e001a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:46:48.851 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.846+0000 7f5e04c5c700 1 --2- 172.21.15.1:0/4165732215 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e001002a0 0x7f5e001977d0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc00d980 tx=0x7f5dfc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:48.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.846+0000 7f5df67fc700 1 -- 172.21.15.1:0/4165732215 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc004a30 con 0x7f5e001002a0 2023-11-16T16:46:48.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.846+0000 7f5df67fc700 1 -- 172.21.15.1:0/4165732215 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5dfc0054e0 con 0x7f5e001002a0 2023-11-16T16:46:48.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.846+0000 7f5e076c1700 1 -- 172.21.15.1:0/4165732215 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e0010f240 con 0x7f5e001002a0 2023-11-16T16:46:48.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.846+0000 7f5df67fc700 1 -- 172.21.15.1:0/4165732215 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc009c90 con 0x7f5e001002a0 2023-11-16T16:46:48.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.846+0000 7f5e076c1700 1 -- 172.21.15.1:0/4165732215 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e0010f0b0 con 0x7f5e001002a0 2023-11-16T16:46:48.858 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.848+0000 7f5df67fc700 1 -- 172.21.15.1:0/4165732215 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f5dfc004c20 con 0x7f5e001002a0 2023-11-16T16:46:48.858 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.848+0000 7f5df67fc700 1 --2- 172.21.15.1:0/4165732215 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5dec0a12c0 0x7f5dec0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:48.859 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.849+0000 7f5df67fc700 1 -- 172.21.15.1:0/4165732215 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f5dfc0ccc40 con 0x7f5e001002a0 2023-11-16T16:46:48.859 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.849+0000 7f5e0545d700 1 --2- 172.21.15.1:0/4165732215 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5dec0a12c0 0x7f5dec0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:48.859 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.849+0000 7f5e0545d700 1 --2- 172.21.15.1:0/4165732215 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5dec0a12c0 0x7f5dec0a3770 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5df000abd0 tx=0x7f5df000a380 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:48.859 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.849+0000 7f5e1384db80 1 Processor -- start 2023-11-16T16:46:48.859 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.850+0000 7f5e1384db80 1 -- start start 2023-11-16T16:46:48.859 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.850+0000 7f5e1384db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592e1b0aff0 0x5592e1b0b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:48.860 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.850+0000 7f5e1384db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592e1b0b900 0x5592e1b0f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:48.860 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.850+0000 7f5e1384db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592e1b0fd40 0x5592e1b10190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:48.860 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.850+0000 7f5e1384db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e1a0c0e0 con 0x5592e1b0aff0 2023-11-16T16:46:48.860 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.850+0000 7f5e1384db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e1a5e920 con 0x5592e1b0b900 2023-11-16T16:46:48.860 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.850+0000 7f5e1384db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e1a57db0 con 0x5592e1b0fd40 2023-11-16T16:46:48.861 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.850+0000 7f5e0545d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592e1b0aff0 0x5592e1b0b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:48.861 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.850+0000 7f5e0545d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592e1b0aff0 0x5592e1b0b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46064/0 (socket says 172.21.15.1:46064) 2023-11-16T16:46:48.861 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.850+0000 7f5e0545d700 1 -- 172.21.15.1:0/1706376263 learned_addr learned my addr 172.21.15.1:0/1706376263 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:48.861 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.850+0000 7f5e05c5e700 1 --2- 172.21.15.1:0/1706376263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592e1b0fd40 0x5592e1b10190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:48.861 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.850+0000 7f5e04c5c700 1 --2- 172.21.15.1:0/1706376263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592e1b0b900 0x5592e1b0f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:48.861 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.850+0000 7f5e0545d700 1 -- 172.21.15.1:0/1706376263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592e1b0fd40 msgr2=0x5592e1b10190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:48.861 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.850+0000 7f5e0545d700 1 --2- 172.21.15.1:0/1706376263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592e1b0fd40 0x5592e1b10190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.861 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.850+0000 7f5e0545d700 1 -- 172.21.15.1:0/1706376263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592e1b0b900 msgr2=0x5592e1b0f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:48.862 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.850+0000 7f5e0545d700 1 --2- 172.21.15.1:0/1706376263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592e1b0b900 0x5592e1b0f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.862 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.850+0000 7f5e0545d700 1 -- 172.21.15.1:0/1706376263 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592e18d96b0 con 0x5592e1b0aff0 2023-11-16T16:46:48.862 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.850+0000 7f5e05c5e700 1 --2- 172.21.15.1:0/1706376263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592e1b0fd40 0x5592e1b10190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:46:48.862 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.850+0000 7f5e0545d700 1 --2- 172.21.15.1:0/1706376263 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592e1b0aff0 0x5592e1b0b3c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f5df0004530 tx=0x7f5df0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=655d1118eca86c72 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:48.862 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.851+0000 7f5deb7fe700 1 -- 172.21.15.1:0/1706376263 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df0003af0 con 0x5592e1b0aff0 2023-11-16T16:46:48.862 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.851+0000 7f5deb7fe700 1 -- 172.21.15.1:0/1706376263 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5df0003c50 con 0x5592e1b0aff0 2023-11-16T16:46:48.862 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.851+0000 7f5deb7fe700 1 -- 172.21.15.1:0/1706376263 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df001d660 con 0x5592e1b0aff0 2023-11-16T16:46:48.862 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.851+0000 7f5e1384db80 1 -- 172.21.15.1:0/1706376263 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592e1b0aff0 msgr2=0x5592e1b0b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:48.863 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.851+0000 7f5e1384db80 1 --2- 172.21.15.1:0/1706376263 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592e1b0aff0 0x5592e1b0b3c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f5df0004530 tx=0x7f5df0012000 comp rx=0 tx=0).stop 2023-11-16T16:46:48.863 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.851+0000 7f5e1384db80 1 -- 172.21.15.1:0/1706376263 shutdown_connections 2023-11-16T16:46:48.863 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.851+0000 7f5e1384db80 1 --2- 172.21.15.1:0/1706376263 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592e1b0fd40 0x5592e1b10190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.863 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.851+0000 7f5e1384db80 1 --2- 172.21.15.1:0/1706376263 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592e1b0aff0 0x5592e1b0b3c0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.863 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.851+0000 7f5e1384db80 1 --2- 172.21.15.1:0/1706376263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592e1b0b900 0x5592e1b0f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.863 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.851+0000 7f5e1384db80 1 -- 172.21.15.1:0/1706376263 >> 172.21.15.1:0/1706376263 conn(0x5592e1afc3e0 msgr2=0x5592e1b01bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:48.864 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.851+0000 7f5e1384db80 1 -- 172.21.15.1:0/1706376263 shutdown_connections 2023-11-16T16:46:48.864 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.851+0000 7f5e1384db80 1 -- 172.21.15.1:0/1706376263 wait complete. 2023-11-16T16:46:48.864 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.852+0000 7f5e1384db80 1 -- start start 2023-11-16T16:46:48.864 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.852+0000 7f5e1384db80 1 Processor -- start 2023-11-16T16:46:48.864 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.852+0000 7f5e1384db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592e1b0aff0 0x5592e1b174c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:48.864 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.852+0000 7f5e1384db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592e1b0b900 0x5592e1b147b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:48.864 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.852+0000 7f5e1384db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592e1b0fd40 0x5592e1b14e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:48.864 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.852+0000 7f5e1384db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e19f9960 con 0x5592e1b0fd40 2023-11-16T16:46:48.865 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.852+0000 7f5e1384db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e19f9600 con 0x5592e1b0aff0 2023-11-16T16:46:48.865 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.852+0000 7f5e1384db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e18db9a0 con 0x5592e1b0b900 2023-11-16T16:46:48.865 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.852+0000 7f5e1384db80 10 client.0 _ll_register_callbacks cb 0x5592e1a17800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:46:48.865 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.852+0000 7f5e05c5e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592e1b0fd40 0x5592e1b14e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:48.865 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.852+0000 7f5e05c5e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592e1b0fd40 0x5592e1b14e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46074/0 (socket says 172.21.15.1:46074) 2023-11-16T16:46:48.866 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.852+0000 7f5e05c5e700 1 -- 172.21.15.1:0/980197067 learned_addr learned my addr 172.21.15.1:0/980197067 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:48.866 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.853+0000 7f5e0545d700 1 --2- 172.21.15.1:0/980197067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592e1b0aff0 0x5592e1b174c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:48.866 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.853+0000 7f5e04c5c700 1 --2- 172.21.15.1:0/980197067 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592e1b0b900 0x5592e1b147b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:48.866 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.853+0000 7f5e05c5e700 1 -- 172.21.15.1:0/980197067 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592e1b0b900 msgr2=0x5592e1b147b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:48.866 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.853+0000 7f5e05c5e700 1 --2- 172.21.15.1:0/980197067 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592e1b0b900 0x5592e1b147b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.866 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.853+0000 7f5e05c5e700 1 -- 172.21.15.1:0/980197067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592e1b0aff0 msgr2=0x5592e1b174c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:48.866 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.853+0000 7f5e05c5e700 1 --2- 172.21.15.1:0/980197067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592e1b0aff0 0x5592e1b174c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.867 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.853+0000 7f5e05c5e700 1 -- 172.21.15.1:0/980197067 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5df00041e0 con 0x5592e1b0fd40 2023-11-16T16:46:48.867 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.853+0000 7f5e05c5e700 1 --2- 172.21.15.1:0/980197067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592e1b0fd40 0x5592e1b14e10 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f5df8004d70 tx=0x7f5df8004da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be833c9c39f629c server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:48.867 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.853+0000 7f5de8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:46:48.867 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.853+0000 7f5de8ff9700 1 -- 172.21.15.1:0/980197067 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df800e070 con 0x5592e1b0fd40 2023-11-16T16:46:48.867 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.853+0000 7f5de8ff9700 1 -- 172.21.15.1:0/980197067 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f5df8007970 con 0x5592e1b0fd40 2023-11-16T16:46:48.867 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.853+0000 7f5e1384db80 1 -- 172.21.15.1:0/980197067 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5592e1af9ba0 con 0x5592e1b0fd40 2023-11-16T16:46:48.867 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.853+0000 7f5de8ff9700 1 -- 172.21.15.1:0/980197067 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5df8005040 con 0x5592e1b0fd40 2023-11-16T16:46:48.867 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.853+0000 7f5e1384db80 10 client.16512 fetch_fsmap learned FSMap version 28 2023-11-16T16:46:48.868 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.853+0000 7f5e1384db80 1 -- 172.21.15.1:0/980197067 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5592e1a1e210 con 0x5592e1b0fd40 2023-11-16T16:46:48.868 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.854+0000 7f5de8ff9700 1 -- 172.21.15.1:0/980197067 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5592e1af9ba0 con 0x5592e1b0fd40 2023-11-16T16:46:48.868 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.854+0000 7f5de8ff9700 1 -- 172.21.15.1:0/980197067 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f5df8022020 con 0x5592e1b0fd40 2023-11-16T16:46:48.868 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.854+0000 7f5e1384db80 10 client.16512 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:46:48.868 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.854+0000 7f5e1384db80 10 client.16512 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:46:48.868 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.854+0000 7f5e1384db80 20 client.16512 populate_metadata read hostname 'smithi001' 2023-11-16T16:46:48.868 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.854+0000 7f5e1384db80 1 --2- 172.21.15.1:0/980197067 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5592e1b24d30 0x5592e1b27140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:48.868 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.854+0000 7f5e1384db80 4 client.16512 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:46:48.869 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.854+0000 7f5e1384db80 1 -- 172.21.15.1:0/980197067 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5592e1a1a120 con 0x5592e1b24d30 2023-11-16T16:46:48.869 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.854+0000 7f5e0545d700 1 --2- 172.21.15.1:0/980197067 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5592e1b24d30 0x5592e1b27140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:48.869 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.855+0000 7f5de8ff9700 10 client.16512 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:46:48.869 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.855+0000 7f5e0545d700 1 --2- 172.21.15.1:0/980197067 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5592e1b24d30 0x5592e1b27140 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=301d91d68a2da028 server_cookie=30ee9d703f8ac983 in_seq=0 out_seq=0 2023-11-16T16:46:48.869 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.855+0000 7f5de8ff9700 1 -- 172.21.15.1:0/980197067 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5592e1a1a120 con 0x5592e1b24d30 2023-11-16T16:46:48.869 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.855+0000 7f5de8ff9700 10 client.16512 handle_command_reply: tid=0 2023-11-16T16:46:48.869 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.855+0000 7f5e1384db80 1 client.16512 shutdown 2023-11-16T16:46:48.869 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.855+0000 7f5e1384db80 1 -- 172.21.15.1:0/980197067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592e1b0fd40 msgr2=0x5592e1b14e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:48.870 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.855+0000 7f5e1384db80 1 --2- 172.21.15.1:0/980197067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592e1b0fd40 0x5592e1b14e10 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f5df8004d70 tx=0x7f5df8004da0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.870 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.855+0000 7f5e1384db80 1 -- 172.21.15.1:0/980197067 shutdown_connections 2023-11-16T16:46:48.870 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.856+0000 7f5e1384db80 1 --2- 172.21.15.1:0/980197067 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5592e1b24d30 0x5592e1b27140 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.870 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.856+0000 7f5e1384db80 1 --2- 172.21.15.1:0/980197067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592e1b0fd40 0x5592e1b14e10 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.870 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.856+0000 7f5de8ff9700 0 client.16512 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:46:48.870 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.856+0000 7f5e1384db80 1 --2- 172.21.15.1:0/980197067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592e1b0aff0 0x5592e1b174c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.870 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.856+0000 7f5e1384db80 1 --2- 172.21.15.1:0/980197067 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592e1b0b900 0x5592e1b147b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.871 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.856+0000 7f5e1384db80 1 -- 172.21.15.1:0/980197067 >> 172.21.15.1:0/980197067 conn(0x5592e1afc3e0 msgr2=0x5592e1b00e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:48.871 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.856+0000 7f5e1384db80 1 -- 172.21.15.1:0/980197067 shutdown_connections 2023-11-16T16:46:48.871 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.856+0000 7f5e1384db80 1 -- 172.21.15.1:0/980197067 wait complete. 2023-11-16T16:46:48.871 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.856+0000 7f5e1384db80 20 client.16512 trim_cache size 0 max 16384 2023-11-16T16:46:48.877 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.874+0000 7f5e1384db80 1 Processor -- start 2023-11-16T16:46:48.878 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.874+0000 7f5e1384db80 1 -- start start 2023-11-16T16:46:48.878 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.874+0000 7f5e1384db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592e1b0aff0 0x5592e1b1be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:48.878 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.874+0000 7f5e1384db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592e1b0b900 0x5592e1b17060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:48.878 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.874+0000 7f5e1384db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592e1b0fd40 0x5592e1b0d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:48.878 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.874+0000 7f5e1384db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e1afbf60 con 0x5592e1b0b900 2023-11-16T16:46:48.878 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.874+0000 7f5e1384db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e1b22390 con 0x5592e1b0aff0 2023-11-16T16:46:48.878 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.874+0000 7f5e1384db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e1b153f0 con 0x5592e1b0fd40 2023-11-16T16:46:48.878 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.874+0000 7f5e04c5c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592e1b0b900 0x5592e1b17060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:48.879 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.874+0000 7f5e04c5c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592e1b0b900 0x5592e1b17060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46080/0 (socket says 172.21.15.1:46080) 2023-11-16T16:46:48.879 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.874+0000 7f5e04c5c700 1 -- 172.21.15.1:0/3870393430 learned_addr learned my addr 172.21.15.1:0/3870393430 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:48.879 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.874+0000 7f5e05c5e700 1 --2- 172.21.15.1:0/3870393430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592e1b0fd40 0x5592e1b0d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:48.879 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.874+0000 7f5e04c5c700 1 -- 172.21.15.1:0/3870393430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592e1b0fd40 msgr2=0x5592e1b0d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:48.879 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.874+0000 7f5e04c5c700 1 --2- 172.21.15.1:0/3870393430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592e1b0fd40 0x5592e1b0d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.879 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.874+0000 7f5e0545d700 1 --2- 172.21.15.1:0/3870393430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592e1b0aff0 0x5592e1b1be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:48.879 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.874+0000 7f5e04c5c700 1 -- 172.21.15.1:0/3870393430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592e1b0aff0 msgr2=0x5592e1b1be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:48.880 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.874+0000 7f5e04c5c700 1 --2- 172.21.15.1:0/3870393430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592e1b0aff0 0x5592e1b1be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.880 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.874+0000 7f5e04c5c700 1 -- 172.21.15.1:0/3870393430 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592e1a1df60 con 0x5592e1b0b900 2023-11-16T16:46:48.880 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.874+0000 7f5e0545d700 1 --2- 172.21.15.1:0/3870393430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592e1b0aff0 0x5592e1b1be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:46:48.880 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.875+0000 7f5e04c5c700 1 --2- 172.21.15.1:0/3870393430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592e1b0b900 0x5592e1b17060 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc096200 tx=0x7f5dfc0998c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c935d987b7e4968c server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:48.880 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.875+0000 7f5debfff700 1 -- 172.21.15.1:0/3870393430 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc099d00 con 0x5592e1b0b900 2023-11-16T16:46:48.880 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.875+0000 7f5debfff700 1 -- 172.21.15.1:0/3870393430 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5dfc09dd30 con 0x5592e1b0b900 2023-11-16T16:46:48.880 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.875+0000 7f5debfff700 1 -- 172.21.15.1:0/3870393430 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc0b2510 con 0x5592e1b0b900 2023-11-16T16:46:48.880 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.875+0000 7f5e1384db80 1 -- 172.21.15.1:0/3870393430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592e1b0b900 msgr2=0x5592e1b17060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:48.881 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.875+0000 7f5e1384db80 1 --2- 172.21.15.1:0/3870393430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592e1b0b900 0x5592e1b17060 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc096200 tx=0x7f5dfc0998c0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.881 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.875+0000 7f5e1384db80 1 -- 172.21.15.1:0/3870393430 shutdown_connections 2023-11-16T16:46:48.881 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.875+0000 7f5e1384db80 1 --2- 172.21.15.1:0/3870393430 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592e1b0fd40 0x5592e1b0d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.881 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.875+0000 7f5e1384db80 1 --2- 172.21.15.1:0/3870393430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592e1b0aff0 0x5592e1b1be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.881 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.875+0000 7f5e1384db80 1 --2- 172.21.15.1:0/3870393430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592e1b0b900 0x5592e1b17060 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.881 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.875+0000 7f5e1384db80 1 -- 172.21.15.1:0/3870393430 >> 172.21.15.1:0/3870393430 conn(0x5592e1afc3e0 msgr2=0x5592e1b06270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:48.881 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.875+0000 7f5e1384db80 1 -- 172.21.15.1:0/3870393430 shutdown_connections 2023-11-16T16:46:48.881 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.875+0000 7f5e1384db80 1 -- 172.21.15.1:0/3870393430 wait complete. 2023-11-16T16:46:48.882 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.876+0000 7f5e1384db80 1 -- start start 2023-11-16T16:46:48.882 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.876+0000 7f5e1384db80 1 Processor -- start 2023-11-16T16:46:48.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.876+0000 7f5e1384db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592e1b0aff0 0x5592e1b1ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:48.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.876+0000 7f5e1384db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592e1b0b900 0x5592e1b1b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:48.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.876+0000 7f5e1384db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592e1b0fd40 0x5592e1b78bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:48.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.876+0000 7f5e1384db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e1b79100 con 0x5592e1b0b900 2023-11-16T16:46:48.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.876+0000 7f5e1384db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e1b79240 con 0x5592e1b0aff0 2023-11-16T16:46:48.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.876+0000 7f5e1384db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e1b793d0 con 0x5592e1b0fd40 2023-11-16T16:46:48.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.876+0000 7f5e1384db80 10 client.0 _ll_register_callbacks cb 0x7f5df8009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:46:48.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.876+0000 7f5e04c5c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592e1b0b900 0x5592e1b1b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:48.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.876+0000 7f5e04c5c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592e1b0b900 0x5592e1b1b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46086/0 (socket says 172.21.15.1:46086) 2023-11-16T16:46:48.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.876+0000 7f5e04c5c700 1 -- 172.21.15.1:0/1157215249 learned_addr learned my addr 172.21.15.1:0/1157215249 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:48.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.876+0000 7f5e0545d700 1 --2- 172.21.15.1:0/1157215249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592e1b0aff0 0x5592e1b1ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:48.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.877+0000 7f5e05c5e700 1 --2- 172.21.15.1:0/1157215249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592e1b0fd40 0x5592e1b78bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:48.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.877+0000 7f5e04c5c700 1 -- 172.21.15.1:0/1157215249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592e1b0fd40 msgr2=0x5592e1b78bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:48.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.877+0000 7f5e04c5c700 1 --2- 172.21.15.1:0/1157215249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592e1b0fd40 0x5592e1b78bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.877+0000 7f5e04c5c700 1 -- 172.21.15.1:0/1157215249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592e1b0aff0 msgr2=0x5592e1b1ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:48.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.877+0000 7f5e04c5c700 1 --2- 172.21.15.1:0/1157215249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592e1b0aff0 0x5592e1b1ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.877+0000 7f5e04c5c700 1 -- 172.21.15.1:0/1157215249 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dfc09c040 con 0x5592e1b0b900 2023-11-16T16:46:48.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.877+0000 7f5e05c5e700 1 --2- 172.21.15.1:0/1157215249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592e1b0fd40 0x5592e1b78bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:46:48.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.877+0000 7f5e04c5c700 1 --2- 172.21.15.1:0/1157215249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592e1b0b900 0x5592e1b1b1b0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0ca980 tx=0x7f5dfc0057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70ad28c82deddda2 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:48.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.877+0000 7f5de8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:46:48.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.877+0000 7f5de8ff9700 1 -- 172.21.15.1:0/1157215249 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc099d00 con 0x5592e1b0b900 2023-11-16T16:46:48.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.877+0000 7f5de8ff9700 1 -- 172.21.15.1:0/1157215249 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f5dfc0b2470 con 0x5592e1b0b900 2023-11-16T16:46:48.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.877+0000 7f5e1384db80 1 -- 172.21.15.1:0/1157215249 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5df0003c50 con 0x5592e1b0b900 2023-11-16T16:46:48.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.877+0000 7f5de8ff9700 1 -- 172.21.15.1:0/1157215249 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5dfc099f60 con 0x5592e1b0b900 2023-11-16T16:46:48.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.877+0000 7f5e1384db80 10 client.16518 fetch_fsmap learned FSMap version 28 2023-11-16T16:46:48.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.877+0000 7f5e1384db80 1 -- 172.21.15.1:0/1157215249 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5592e186ece0 con 0x5592e1b0b900 2023-11-16T16:46:48.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.877+0000 7f5de8ff9700 1 -- 172.21.15.1:0/1157215249 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5dfc07c020 con 0x5592e1b0b900 2023-11-16T16:46:48.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.877+0000 7f5de8ff9700 1 -- 172.21.15.1:0/1157215249 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f5dfc0a1da0 con 0x5592e1b0b900 2023-11-16T16:46:48.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.878+0000 7f5e1384db80 10 client.16518 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:46:48.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.878+0000 7f5e1384db80 10 client.16518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:46:48.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.878+0000 7f5e1384db80 20 client.16518 populate_metadata read hostname 'smithi001' 2023-11-16T16:46:48.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.878+0000 7f5e1384db80 1 --2- 172.21.15.1:0/1157215249 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5592e1b7a770 0x5592e1b7cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:48.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.878+0000 7f5e1384db80 4 client.16518 mds_command: new command op to 24521 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-16T16:46:48.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.878+0000 7f5e1384db80 1 -- 172.21.15.1:0/1157215249 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5592e1a266e0 con 0x5592e1b7a770 2023-11-16T16:46:48.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.878+0000 7f5e0545d700 1 --2- 172.21.15.1:0/1157215249 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5592e1b7a770 0x5592e1b7cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:48.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.878+0000 7f5de8ff9700 10 client.16518 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:46:48.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.878+0000 7f5e0545d700 1 --2- 172.21.15.1:0/1157215249 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5592e1b7a770 0x5592e1b7cb80 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ce17ee11d1d257f server_cookie=8d27647ef5d2660b in_seq=0 out_seq=0 2023-11-16T16:46:48.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.880+0000 7f5de8ff9700 1 -- 172.21.15.1:0/1157215249 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5592e1a266e0 con 0x5592e1b7a770 2023-11-16T16:46:48.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.880+0000 7f5de8ff9700 10 client.16518 handle_command_reply: tid=0 2023-11-16T16:46:48.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.880+0000 7f5e1384db80 1 client.16518 shutdown 2023-11-16T16:46:48.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.880+0000 7f5e1384db80 1 -- 172.21.15.1:0/1157215249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592e1b0b900 msgr2=0x5592e1b1b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:48.891 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.880+0000 7f5e1384db80 1 --2- 172.21.15.1:0/1157215249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592e1b0b900 0x5592e1b1b1b0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc0ca980 tx=0x7f5dfc0057a0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.891 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.880+0000 7f5e1384db80 1 -- 172.21.15.1:0/1157215249 shutdown_connections 2023-11-16T16:46:48.891 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.880+0000 7f5e1384db80 1 --2- 172.21.15.1:0/1157215249 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5592e1b7a770 0x5592e1b7cb80 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.891 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.880+0000 7f5e1384db80 1 --2- 172.21.15.1:0/1157215249 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5592e1b0fd40 0x5592e1b78bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.891 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.880+0000 7f5e1384db80 1 --2- 172.21.15.1:0/1157215249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592e1b0aff0 0x5592e1b1ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.891 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.880+0000 7f5e1384db80 1 --2- 172.21.15.1:0/1157215249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592e1b0b900 0x5592e1b1b1b0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.891 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.880+0000 7f5de8ff9700 0 client.16518 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:46:48.891 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.880+0000 7f5e1384db80 1 -- 172.21.15.1:0/1157215249 >> 172.21.15.1:0/1157215249 conn(0x5592e1afc3e0 msgr2=0x5592e1b16050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:48.892 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.880+0000 7f5e1384db80 1 -- 172.21.15.1:0/1157215249 shutdown_connections 2023-11-16T16:46:48.892 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.880+0000 7f5e1384db80 1 -- 172.21.15.1:0/1157215249 wait complete. 2023-11-16T16:46:48.892 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.880+0000 7f5e1384db80 20 client.16518 trim_cache size 0 max 16384 2023-11-16T16:46:48.892 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:46:48.892 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2023-11-16T16:46:48.892 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "5bbb7780-b812-4543-b5e9-55b8fb65a2b0", 2023-11-16T16:46:48.893 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2023-11-16T16:46:48.893 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:46:48.893 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.881+0000 7f5e076c1700 1 -- 172.21.15.1:0/4165732215 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5dec0a12c0 msgr2=0x7f5dec0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:48.894 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.881+0000 7f5e076c1700 1 --2- 172.21.15.1:0/4165732215 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5dec0a12c0 0x7f5dec0a3770 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5df000abd0 tx=0x7f5df000a380 comp rx=0 tx=0).stop 2023-11-16T16:46:48.894 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.881+0000 7f5e076c1700 1 -- 172.21.15.1:0/4165732215 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e001002a0 msgr2=0x7f5e001977d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:48.894 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.881+0000 7f5e076c1700 1 --2- 172.21.15.1:0/4165732215 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e001002a0 0x7f5e001977d0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc00d980 tx=0x7f5dfc00dc90 comp rx=0 tx=0).stop 2023-11-16T16:46:48.894 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.881+0000 7f5e076c1700 1 -- 172.21.15.1:0/4165732215 shutdown_connections 2023-11-16T16:46:48.894 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.881+0000 7f5e076c1700 1 --2- 172.21.15.1:0/4165732215 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5dec0a12c0 0x7f5dec0a3770 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.894 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.881+0000 7f5e076c1700 1 --2- 172.21.15.1:0/4165732215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5e00104140 0x7f5e001a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.894 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.881+0000 7f5e076c1700 1 --2- 172.21.15.1:0/4165732215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e000ff8f0 0x7f5e00197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.895 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.881+0000 7f5e076c1700 1 --2- 172.21.15.1:0/4165732215 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e001002a0 0x7f5e001977d0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:48.895 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.881+0000 7f5e076c1700 1 -- 172.21.15.1:0/4165732215 >> 172.21.15.1:0/4165732215 conn(0x7f5e00075880 msgr2=0x7f5e000fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:48.895 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.881+0000 7f5e076c1700 1 -- 172.21.15.1:0/4165732215 shutdown_connections 2023-11-16T16:46:48.895 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:48.881+0000 7f5e076c1700 1 -- 172.21.15.1:0/4165732215 wait complete. 2023-11-16T16:46:48.896 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-16T16:46:48.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:48 smithi001 ceph-mon[98800]: pgmap v234: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:46:49.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:48 smithi026 ceph-mon[100112]: pgmap v234: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:46:49.024 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.022+0000 7fbf89976700 1 -- 172.21.15.1:0/3887231887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf84100420 msgr2=0x7fbf8410bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:49.024 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.022+0000 7fbf89976700 1 --2- 172.21.15.1:0/3887231887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf84100420 0x7fbf8410bfd0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c00b0a0 tx=0x7fbf7c00b3b0 comp rx=0 tx=0).stop 2023-11-16T16:46:49.024 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.022+0000 7fbf89976700 1 -- 172.21.15.1:0/3887231887 shutdown_connections 2023-11-16T16:46:49.024 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.022+0000 7fbf89976700 1 --2- 172.21.15.1:0/3887231887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf84100420 0x7fbf8410bfd0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:49.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.022+0000 7fbf89976700 1 --2- 172.21.15.1:0/3887231887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf84105250 0x7fbf84105620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:49.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.022+0000 7fbf89976700 1 --2- 172.21.15.1:0/3887231887 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf840ffa70 0x7fbf840ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:49.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.022+0000 7fbf89976700 1 -- 172.21.15.1:0/3887231887 >> 172.21.15.1:0/3887231887 conn(0x7fbf840763a0 msgr2=0x7fbf840767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:49.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.022+0000 7fbf89976700 1 -- 172.21.15.1:0/3887231887 shutdown_connections 2023-11-16T16:46:49.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.022+0000 7fbf89976700 1 -- 172.21.15.1:0/3887231887 wait complete. 2023-11-16T16:46:49.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.023+0000 7fbf89976700 1 Processor -- start 2023-11-16T16:46:49.027 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.023+0000 7fbf89976700 1 -- start start 2023-11-16T16:46:49.027 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.023+0000 7fbf89976700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf840ffa70 0x7fbf841953f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:49.028 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.023+0000 7fbf89976700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf84100420 0x7fbf84195930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:49.028 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.023+0000 7fbf89976700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf84105250 0x7fbf84199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:49.028 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.023+0000 7fbf89976700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8419a3d0 con 0x7fbf840ffa70 2023-11-16T16:46:49.028 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.023+0000 7fbf89976700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8419a540 con 0x7fbf84105250 2023-11-16T16:46:49.028 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.023+0000 7fbf89976700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8419a700 con 0x7fbf84100420 2023-11-16T16:46:49.028 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.023+0000 7fbf82ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf840ffa70 0x7fbf841953f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:49.028 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.023+0000 7fbf82ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf840ffa70 0x7fbf841953f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46098/0 (socket says 172.21.15.1:46098) 2023-11-16T16:46:49.029 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.023+0000 7fbf82ffd700 1 -- 172.21.15.1:0/2387536784 learned_addr learned my addr 172.21.15.1:0/2387536784 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:49.029 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.023+0000 7fbf837fe700 1 --2- 172.21.15.1:0/2387536784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf84105250 0x7fbf84199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:49.029 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.023+0000 7fbf827fc700 1 --2- 172.21.15.1:0/2387536784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf84100420 0x7fbf84195930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:49.029 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.023+0000 7fbf82ffd700 1 -- 172.21.15.1:0/2387536784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf84100420 msgr2=0x7fbf84195930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:49.029 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.023+0000 7fbf82ffd700 1 --2- 172.21.15.1:0/2387536784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf84100420 0x7fbf84195930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:49.029 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.023+0000 7fbf82ffd700 1 -- 172.21.15.1:0/2387536784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf84105250 msgr2=0x7fbf84199d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:49.029 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.023+0000 7fbf82ffd700 1 --2- 172.21.15.1:0/2387536784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf84105250 0x7fbf84199d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:49.029 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.023+0000 7fbf82ffd700 1 -- 172.21.15.1:0/2387536784 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7c009d00 con 0x7fbf840ffa70 2023-11-16T16:46:49.030 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.023+0000 7fbf837fe700 1 --2- 172.21.15.1:0/2387536784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf84105250 0x7fbf84199d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:46:49.030 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.024+0000 7fbf82ffd700 1 --2- 172.21.15.1:0/2387536784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf840ffa70 0x7fbf841953f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fbf7400eab0 tx=0x7fbf7400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:49.030 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.024+0000 7fbf88974700 1 -- 172.21.15.1:0/2387536784 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf74013070 con 0x7fbf840ffa70 2023-11-16T16:46:49.030 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.024+0000 7fbf88974700 1 -- 172.21.15.1:0/2387536784 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf74017410 con 0x7fbf840ffa70 2023-11-16T16:46:49.030 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.024+0000 7fbf89976700 1 -- 172.21.15.1:0/2387536784 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf8410f230 con 0x7fbf840ffa70 2023-11-16T16:46:49.030 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.024+0000 7fbf88974700 1 -- 172.21.15.1:0/2387536784 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf74016610 con 0x7fbf840ffa70 2023-11-16T16:46:49.030 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.024+0000 7fbf89976700 1 -- 172.21.15.1:0/2387536784 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf8410f0a0 con 0x7fbf840ffa70 2023-11-16T16:46:49.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.027+0000 7fbf88974700 1 -- 172.21.15.1:0/2387536784 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fbf74017880 con 0x7fbf840ffa70 2023-11-16T16:46:49.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.027+0000 7fbf88974700 1 --2- 172.21.15.1:0/2387536784 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fbf6c0a12c0 0x7fbf6c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:49.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.027+0000 7fbf88974700 1 -- 172.21.15.1:0/2387536784 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fbf740ce380 con 0x7fbf840ffa70 2023-11-16T16:46:49.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.027+0000 7fbf827fc700 1 --2- 172.21.15.1:0/2387536784 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fbf6c0a12c0 0x7fbf6c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:49.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.028+0000 7fbf827fc700 1 --2- 172.21.15.1:0/2387536784 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fbf6c0a12c0 0x7fbf6c0a3770 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbf84196a00 tx=0x7fbf780094f0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:49.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.028+0000 7fbf95b02b80 1 Processor -- start 2023-11-16T16:46:49.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.028+0000 7fbf95b02b80 1 -- start start 2023-11-16T16:46:49.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.028+0000 7fbf95b02b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a3ba8fb60 0x564a3ba8ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:49.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.028+0000 7fbf95b02b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a3ba90470 0x564a3ba94370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:49.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.028+0000 7fbf95b02b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a3ba948b0 0x564a3ba94d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:49.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.028+0000 7fbf95b02b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a3b995060 con 0x564a3ba948b0 2023-11-16T16:46:49.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.028+0000 7fbf95b02b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a3b9e78a0 con 0x564a3ba8fb60 2023-11-16T16:46:49.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.028+0000 7fbf95b02b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a3b9e0d30 con 0x564a3ba90470 2023-11-16T16:46:49.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.028+0000 7fbf837fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a3ba948b0 0x564a3ba94d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:49.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.028+0000 7fbf837fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a3ba948b0 0x564a3ba94d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46110/0 (socket says 172.21.15.1:46110) 2023-11-16T16:46:49.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.028+0000 7fbf837fe700 1 -- 172.21.15.1:0/3455416728 learned_addr learned my addr 172.21.15.1:0/3455416728 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:49.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.029+0000 7fbf82ffd700 1 --2- 172.21.15.1:0/3455416728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a3ba8fb60 0x564a3ba8ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:49.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.029+0000 7fbf827fc700 1 --2- 172.21.15.1:0/3455416728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a3ba90470 0x564a3ba94370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:49.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.029+0000 7fbf837fe700 1 -- 172.21.15.1:0/3455416728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a3ba90470 msgr2=0x564a3ba94370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:49.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.029+0000 7fbf837fe700 1 --2- 172.21.15.1:0/3455416728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a3ba90470 0x564a3ba94370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:49.040 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.029+0000 7fbf837fe700 1 -- 172.21.15.1:0/3455416728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a3ba8fb60 msgr2=0x564a3ba8ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:49.040 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.029+0000 7fbf837fe700 1 --2- 172.21.15.1:0/3455416728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a3ba8fb60 0x564a3ba8ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:49.040 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.029+0000 7fbf837fe700 1 -- 172.21.15.1:0/3455416728 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a3b862630 con 0x564a3ba948b0 2023-11-16T16:46:49.040 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.029+0000 7fbf837fe700 1 --2- 172.21.15.1:0/3455416728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a3ba948b0 0x564a3ba94d00 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c00b780 tx=0x7fbf7c009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba2103a6326c0dbe server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:49.040 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.029+0000 7fbf69ffb700 1 -- 172.21.15.1:0/3455416728 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c00e040 con 0x564a3ba948b0 2023-11-16T16:46:49.040 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.029+0000 7fbf69ffb700 1 -- 172.21.15.1:0/3455416728 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf7c009460 con 0x564a3ba948b0 2023-11-16T16:46:49.040 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.029+0000 7fbf69ffb700 1 -- 172.21.15.1:0/3455416728 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c01d470 con 0x564a3ba948b0 2023-11-16T16:46:49.040 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.030+0000 7fbf95b02b80 1 -- 172.21.15.1:0/3455416728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a3ba948b0 msgr2=0x564a3ba94d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:49.041 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.030+0000 7fbf95b02b80 1 --2- 172.21.15.1:0/3455416728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a3ba948b0 0x564a3ba94d00 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c00b780 tx=0x7fbf7c009c80 comp rx=0 tx=0).stop 2023-11-16T16:46:49.041 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.030+0000 7fbf95b02b80 1 -- 172.21.15.1:0/3455416728 shutdown_connections 2023-11-16T16:46:49.041 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.030+0000 7fbf95b02b80 1 --2- 172.21.15.1:0/3455416728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a3ba948b0 0x564a3ba94d00 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:49.041 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.030+0000 7fbf95b02b80 1 --2- 172.21.15.1:0/3455416728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a3ba8fb60 0x564a3ba8ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:49.041 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.030+0000 7fbf95b02b80 1 --2- 172.21.15.1:0/3455416728 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a3ba90470 0x564a3ba94370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:49.041 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.030+0000 7fbf95b02b80 1 -- 172.21.15.1:0/3455416728 >> 172.21.15.1:0/3455416728 conn(0x564a3ba85360 msgr2=0x564a3ba8ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:49.041 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.030+0000 7fbf95b02b80 1 -- 172.21.15.1:0/3455416728 shutdown_connections 2023-11-16T16:46:49.041 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.030+0000 7fbf95b02b80 1 -- 172.21.15.1:0/3455416728 wait complete. 2023-11-16T16:46:49.042 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.030+0000 7fbf95b02b80 1 -- start start 2023-11-16T16:46:49.042 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.030+0000 7fbf95b02b80 1 Processor -- start 2023-11-16T16:46:49.042 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.031+0000 7fbf95b02b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a3ba8fb60 0x564a3ba9c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:49.042 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.031+0000 7fbf95b02b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a3ba90470 0x564a3ba992c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:49.042 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.031+0000 7fbf95b02b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a3ba948b0 0x564a3ba99a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:49.042 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.031+0000 7fbf95b02b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a3b9828e0 con 0x564a3ba948b0 2023-11-16T16:46:49.042 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.031+0000 7fbf95b02b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a3b982580 con 0x564a3ba8fb60 2023-11-16T16:46:49.042 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.031+0000 7fbf95b02b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a3b864920 con 0x564a3ba90470 2023-11-16T16:46:49.043 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.031+0000 7fbf95b02b80 10 client.0 _ll_register_callbacks cb 0x564a3b9a0780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:46:49.043 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.031+0000 7fbf837fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a3ba948b0 0x564a3ba99a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:49.043 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.031+0000 7fbf827fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a3ba90470 0x564a3ba992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:49.043 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.031+0000 7fbf837fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a3ba948b0 0x564a3ba99a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46116/0 (socket says 172.21.15.1:46116) 2023-11-16T16:46:49.043 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.031+0000 7fbf837fe700 1 -- 172.21.15.1:0/47715062 learned_addr learned my addr 172.21.15.1:0/47715062 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:49.043 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.031+0000 7fbf82ffd700 1 --2- 172.21.15.1:0/47715062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a3ba8fb60 0x564a3ba9c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:49.043 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.031+0000 7fbf837fe700 1 -- 172.21.15.1:0/47715062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a3ba90470 msgr2=0x564a3ba992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:49.043 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.031+0000 7fbf837fe700 1 --2- 172.21.15.1:0/47715062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a3ba90470 0x564a3ba992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:49.044 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.031+0000 7fbf837fe700 1 -- 172.21.15.1:0/47715062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a3ba8fb60 msgr2=0x564a3ba9c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:49.044 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.031+0000 7fbf837fe700 1 --2- 172.21.15.1:0/47715062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a3ba8fb60 0x564a3ba9c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:49.044 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.031+0000 7fbf837fe700 1 -- 172.21.15.1:0/47715062 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf8410c8d0 con 0x564a3ba948b0 2023-11-16T16:46:49.044 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.031+0000 7fbf82ffd700 1 --2- 172.21.15.1:0/47715062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a3ba8fb60 0x564a3ba9c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:46:49.044 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.031+0000 7fbf837fe700 1 --2- 172.21.15.1:0/47715062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a3ba948b0 0x564a3ba99a00 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c006090 tx=0x7fbf7c004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aff63f8ddff76b server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:49.044 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.031+0000 7fbf62ffd700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:46:49.044 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.032+0000 7fbf62ffd700 1 -- 172.21.15.1:0/47715062 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c00e040 con 0x564a3ba948b0 2023-11-16T16:46:49.044 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.032+0000 7fbf62ffd700 1 -- 172.21.15.1:0/47715062 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fbf7c007850 con 0x564a3ba948b0 2023-11-16T16:46:49.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.032+0000 7fbf95b02b80 1 -- 172.21.15.1:0/47715062 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a3ba82b20 con 0x564a3ba948b0 2023-11-16T16:46:49.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.032+0000 7fbf62ffd700 1 -- 172.21.15.1:0/47715062 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf7c01d470 con 0x564a3ba948b0 2023-11-16T16:46:49.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.032+0000 7fbf95b02b80 10 client.16530 fetch_fsmap learned FSMap version 28 2023-11-16T16:46:49.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.032+0000 7fbf95b02b80 1 -- 172.21.15.1:0/47715062 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564a3b9a7190 con 0x564a3ba948b0 2023-11-16T16:46:49.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.032+0000 7fbf62ffd700 1 -- 172.21.15.1:0/47715062 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564a3ba82b20 con 0x564a3ba948b0 2023-11-16T16:46:49.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.032+0000 7fbf62ffd700 1 -- 172.21.15.1:0/47715062 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fbf7c019040 con 0x564a3ba948b0 2023-11-16T16:46:49.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.032+0000 7fbf95b02b80 10 client.16530 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:46:49.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.032+0000 7fbf95b02b80 10 client.16530 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:46:49.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.032+0000 7fbf95b02b80 20 client.16530 populate_metadata read hostname 'smithi001' 2023-11-16T16:46:49.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.032+0000 7fbf95b02b80 1 --2- 172.21.15.1:0/47715062 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x564a3baa95f0 0x564a3baaba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:49.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.032+0000 7fbf95b02b80 4 client.16530 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:46:49.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.032+0000 7fbf95b02b80 1 -- 172.21.15.1:0/47715062 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a3b9a30a0 con 0x564a3baa95f0 2023-11-16T16:46:49.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.032+0000 7fbf82ffd700 1 --2- 172.21.15.1:0/47715062 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x564a3baa95f0 0x564a3baaba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:49.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.033+0000 7fbf82ffd700 1 --2- 172.21.15.1:0/47715062 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x564a3baa95f0 0x564a3baaba00 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e94b71130aca9a5 server_cookie=51dd78ec7ee7546f in_seq=0 out_seq=0 2023-11-16T16:46:49.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.033+0000 7fbf62ffd700 10 client.16530 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:46:49.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.034+0000 7fbf62ffd700 1 -- 172.21.15.1:0/47715062 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564a3b9a30a0 con 0x564a3baa95f0 2023-11-16T16:46:49.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.034+0000 7fbf62ffd700 10 client.16530 handle_command_reply: tid=0 2023-11-16T16:46:49.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.034+0000 7fbf95b02b80 1 client.16530 shutdown 2023-11-16T16:46:49.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.034+0000 7fbf95b02b80 1 -- 172.21.15.1:0/47715062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a3ba948b0 msgr2=0x564a3ba99a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:49.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.034+0000 7fbf95b02b80 1 --2- 172.21.15.1:0/47715062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a3ba948b0 0x564a3ba99a00 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c006090 tx=0x7fbf7c004c80 comp rx=0 tx=0).stop 2023-11-16T16:46:49.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.034+0000 7fbf95b02b80 1 -- 172.21.15.1:0/47715062 shutdown_connections 2023-11-16T16:46:49.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.034+0000 7fbf95b02b80 1 --2- 172.21.15.1:0/47715062 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x564a3baa95f0 0x564a3baaba00 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:49.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.034+0000 7fbf95b02b80 1 --2- 172.21.15.1:0/47715062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a3ba948b0 0x564a3ba99a00 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:49.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.034+0000 7fbf95b02b80 1 --2- 172.21.15.1:0/47715062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a3ba8fb60 0x564a3ba9c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:49.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.034+0000 7fbf62ffd700 0 client.16530 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:46:49.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.034+0000 7fbf95b02b80 1 --2- 172.21.15.1:0/47715062 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a3ba90470 0x564a3ba992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:49.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.034+0000 7fbf95b02b80 1 -- 172.21.15.1:0/47715062 >> 172.21.15.1:0/47715062 conn(0x564a3ba85360 msgr2=0x564a3ba95d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:49.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.034+0000 7fbf95b02b80 1 -- 172.21.15.1:0/47715062 shutdown_connections 2023-11-16T16:46:49.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.034+0000 7fbf95b02b80 1 -- 172.21.15.1:0/47715062 wait complete. 2023-11-16T16:46:49.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.034+0000 7fbf95b02b80 20 client.16530 trim_cache size 0 max 16384 2023-11-16T16:46:49.054 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.051+0000 7fbf95b02b80 1 Processor -- start 2023-11-16T16:46:49.054 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.051+0000 7fbf95b02b80 1 -- start start 2023-11-16T16:46:49.055 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.051+0000 7fbf95b02b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a3ba8fb60 0x564a3baa0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:49.055 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.051+0000 7fbf95b02b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a3ba90470 0x564a3ba9bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:49.055 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.051+0000 7fbf95b02b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a3ba948b0 0x564a3ba98b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:49.055 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.051+0000 7fbf95b02b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a3ba829f0 con 0x564a3ba90470 2023-11-16T16:46:49.055 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.051+0000 7fbf95b02b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a3ba90970 con 0x564a3ba948b0 2023-11-16T16:46:49.055 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.051+0000 7fbf95b02b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a3ba933f0 con 0x564a3ba8fb60 2023-11-16T16:46:49.055 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.051+0000 7fbf827fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a3ba90470 0x564a3ba9bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:49.056 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.051+0000 7fbf827fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a3ba90470 0x564a3ba9bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46122/0 (socket says 172.21.15.1:46122) 2023-11-16T16:46:49.056 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.051+0000 7fbf827fc700 1 -- 172.21.15.1:0/3619264102 learned_addr learned my addr 172.21.15.1:0/3619264102 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:49.056 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.051+0000 7fbf82ffd700 1 --2- 172.21.15.1:0/3619264102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a3ba8fb60 0x564a3baa0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:49.056 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.051+0000 7fbf837fe700 1 --2- 172.21.15.1:0/3619264102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a3ba948b0 0x564a3ba98b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:49.056 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.051+0000 7fbf827fc700 1 -- 172.21.15.1:0/3619264102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a3ba8fb60 msgr2=0x564a3baa0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:49.056 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.051+0000 7fbf827fc700 1 --2- 172.21.15.1:0/3619264102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a3ba8fb60 0x564a3baa0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:49.056 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.051+0000 7fbf827fc700 1 -- 172.21.15.1:0/3619264102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a3ba948b0 msgr2=0x564a3ba98b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:49.056 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.051+0000 7fbf827fc700 1 --2- 172.21.15.1:0/3619264102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a3ba948b0 0x564a3ba98b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:49.057 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.051+0000 7fbf827fc700 1 -- 172.21.15.1:0/3619264102 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a3b9a6ee0 con 0x564a3ba90470 2023-11-16T16:46:49.057 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.051+0000 7fbf82ffd700 1 --2- 172.21.15.1:0/3619264102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a3ba8fb60 0x564a3baa0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:46:49.057 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.051+0000 7fbf837fe700 1 --2- 172.21.15.1:0/3619264102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a3ba948b0 0x564a3ba98b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:46:49.057 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.051+0000 7fbf827fc700 1 --2- 172.21.15.1:0/3619264102 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a3ba90470 0x564a3ba9bbe0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fbf7800ee20 tx=0x7fbf78011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d2422a64a2f524d server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:49.057 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.052+0000 7fbf6a7fc700 1 -- 172.21.15.1:0/3619264102 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf78011e90 con 0x564a3ba90470 2023-11-16T16:46:49.057 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.052+0000 7fbf6a7fc700 1 -- 172.21.15.1:0/3619264102 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf78029930 con 0x564a3ba90470 2023-11-16T16:46:49.057 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.052+0000 7fbf6a7fc700 1 -- 172.21.15.1:0/3619264102 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf78020960 con 0x564a3ba90470 2023-11-16T16:46:49.058 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.052+0000 7fbf95b02b80 1 -- 172.21.15.1:0/3619264102 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a3ba90470 msgr2=0x564a3ba9bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:49.058 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.052+0000 7fbf95b02b80 1 --2- 172.21.15.1:0/3619264102 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a3ba90470 0x564a3ba9bbe0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fbf7800ee20 tx=0x7fbf78011000 comp rx=0 tx=0).stop 2023-11-16T16:46:49.058 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.052+0000 7fbf95b02b80 1 -- 172.21.15.1:0/3619264102 shutdown_connections 2023-11-16T16:46:49.058 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.052+0000 7fbf95b02b80 1 --2- 172.21.15.1:0/3619264102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a3ba948b0 0x564a3ba98b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:49.058 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.052+0000 7fbf95b02b80 1 --2- 172.21.15.1:0/3619264102 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a3ba8fb60 0x564a3baa0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:49.058 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.052+0000 7fbf95b02b80 1 --2- 172.21.15.1:0/3619264102 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a3ba90470 0x564a3ba9bbe0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:49.058 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.052+0000 7fbf95b02b80 1 -- 172.21.15.1:0/3619264102 >> 172.21.15.1:0/3619264102 conn(0x564a3ba85360 msgr2=0x564a3ba928f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:49.058 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.052+0000 7fbf95b02b80 1 -- 172.21.15.1:0/3619264102 shutdown_connections 2023-11-16T16:46:49.059 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.052+0000 7fbf95b02b80 1 -- 172.21.15.1:0/3619264102 wait complete. 2023-11-16T16:46:49.059 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.053+0000 7fbf95b02b80 1 -- start start 2023-11-16T16:46:49.059 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.053+0000 7fbf95b02b80 1 Processor -- start 2023-11-16T16:46:49.062 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.053+0000 7fbf95b02b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a3ba8fb60 0x564a3baa0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:49.062 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.053+0000 7fbf95b02b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a3ba90470 0x564a3baa0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:49.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.053+0000 7fbf95b02b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a3ba948b0 0x564a3ba9e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:49.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.053+0000 7fbf95b02b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a3ba9f070 con 0x564a3ba8fb60 2023-11-16T16:46:49.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.053+0000 7fbf95b02b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a3ba9f1e0 con 0x564a3ba948b0 2023-11-16T16:46:49.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.053+0000 7fbf95b02b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a3ba9f3a0 con 0x564a3ba90470 2023-11-16T16:46:49.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.053+0000 7fbf95b02b80 10 client.0 _ll_register_callbacks cb 0x7fbf740970c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:46:49.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.053+0000 7fbf82ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a3ba8fb60 0x564a3baa0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:49.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.053+0000 7fbf827fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a3ba90470 0x564a3baa0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:49.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.053+0000 7fbf82ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a3ba8fb60 0x564a3baa0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46126/0 (socket says 172.21.15.1:46126) 2023-11-16T16:46:49.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.053+0000 7fbf82ffd700 1 -- 172.21.15.1:0/2916722550 learned_addr learned my addr 172.21.15.1:0/2916722550 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:46:49.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.054+0000 7fbf837fe700 1 --2- 172.21.15.1:0/2916722550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a3ba948b0 0x564a3ba9e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:49.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.054+0000 7fbf82ffd700 1 -- 172.21.15.1:0/2916722550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a3ba90470 msgr2=0x564a3baa0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:49.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.054+0000 7fbf82ffd700 1 --2- 172.21.15.1:0/2916722550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a3ba90470 0x564a3baa0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:49.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.054+0000 7fbf82ffd700 1 -- 172.21.15.1:0/2916722550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a3ba948b0 msgr2=0x564a3ba9e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:46:49.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.054+0000 7fbf82ffd700 1 --2- 172.21.15.1:0/2916722550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a3ba948b0 0x564a3ba9e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:49.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.054+0000 7fbf82ffd700 1 -- 172.21.15.1:0/2916722550 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf7800eaa0 con 0x564a3ba8fb60 2023-11-16T16:46:49.065 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.054+0000 7fbf827fc700 1 --2- 172.21.15.1:0/2916722550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a3ba90470 0x564a3baa0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:46:49.065 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.054+0000 7fbf82ffd700 1 --2- 172.21.15.1:0/2916722550 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a3ba8fb60 0x564a3baa0210 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fbf740cbb20 tx=0x7fbf74097170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4cfc4c5f5c2e97b server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:46:49.065 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.054+0000 7fbf62ffd700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:46:49.065 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.054+0000 7fbf62ffd700 1 -- 172.21.15.1:0/2916722550 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf740860b0 con 0x564a3ba8fb60 2023-11-16T16:46:49.065 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.054+0000 7fbf62ffd700 1 -- 172.21.15.1:0/2916722550 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fbf74097b50 con 0x564a3ba8fb60 2023-11-16T16:46:49.065 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.054+0000 7fbf95b02b80 1 -- 172.21.15.1:0/2916722550 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf7c009460 con 0x564a3ba8fb60 2023-11-16T16:46:49.065 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.054+0000 7fbf62ffd700 1 -- 172.21.15.1:0/2916722550 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf74097d10 con 0x564a3ba8fb60 2023-11-16T16:46:49.065 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.054+0000 7fbf95b02b80 10 client.16536 fetch_fsmap learned FSMap version 28 2023-11-16T16:46:49.066 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.054+0000 7fbf95b02b80 1 -- 172.21.15.1:0/2916722550 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564a3b7f7c60 con 0x564a3ba8fb60 2023-11-16T16:46:49.066 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.054+0000 7fbf62ffd700 1 -- 172.21.15.1:0/2916722550 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf7c009460 con 0x564a3ba8fb60 2023-11-16T16:46:49.066 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.055+0000 7fbf62ffd700 1 -- 172.21.15.1:0/2916722550 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fbf7409a930 con 0x564a3ba8fb60 2023-11-16T16:46:49.066 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.055+0000 7fbf95b02b80 10 client.16536 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:46:49.066 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.055+0000 7fbf95b02b80 10 client.16536 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:46:49.066 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.055+0000 7fbf95b02b80 20 client.16536 populate_metadata read hostname 'smithi001' 2023-11-16T16:46:49.066 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.055+0000 7fbf95b02b80 1 --2- 172.21.15.1:0/2916722550 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x564a3bafe5e0 0x564a3bb009c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:46:49.066 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.055+0000 7fbf95b02b80 4 client.16536 mds_command: new command op to 24521 tid=0[{"prefix": "scrub status"}] 2023-11-16T16:46:49.067 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.055+0000 7fbf95b02b80 1 -- 172.21.15.1:0/2916722550 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564a3b9af660 con 0x564a3bafe5e0 2023-11-16T16:46:49.067 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.055+0000 7fbf827fc700 1 --2- 172.21.15.1:0/2916722550 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x564a3bafe5e0 0x564a3bb009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:46:49.067 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.055+0000 7fbf827fc700 1 --2- 172.21.15.1:0/2916722550 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x564a3bafe5e0 0x564a3bb009c0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6182ae9f0f23833a server_cookie=1254d40cb5080591 in_seq=0 out_seq=0 2023-11-16T16:46:49.067 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.055+0000 7fbf62ffd700 10 client.16536 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:46:49.067 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.056+0000 7fbf62ffd700 1 -- 172.21.15.1:0/2916722550 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564a3b9af660 con 0x564a3bafe5e0 2023-11-16T16:46:49.067 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.056+0000 7fbf62ffd700 10 client.16536 handle_command_reply: tid=0 2023-11-16T16:46:49.067 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.056+0000 7fbf95b02b80 1 client.16536 shutdown 2023-11-16T16:46:49.068 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.056+0000 7fbf95b02b80 1 -- 172.21.15.1:0/2916722550 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a3ba8fb60 msgr2=0x564a3baa0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:49.068 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.056+0000 7fbf95b02b80 1 --2- 172.21.15.1:0/2916722550 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a3ba8fb60 0x564a3baa0210 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fbf740cbb20 tx=0x7fbf74097170 comp rx=0 tx=0).stop 2023-11-16T16:46:49.068 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.056+0000 7fbf95b02b80 1 -- 172.21.15.1:0/2916722550 shutdown_connections 2023-11-16T16:46:49.068 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.056+0000 7fbf95b02b80 1 --2- 172.21.15.1:0/2916722550 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x564a3bafe5e0 0x564a3bb009c0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:49.068 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.056+0000 7fbf95b02b80 1 --2- 172.21.15.1:0/2916722550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564a3ba948b0 0x564a3ba9e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:49.068 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.056+0000 7fbf95b02b80 1 --2- 172.21.15.1:0/2916722550 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564a3ba8fb60 0x564a3baa0210 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:49.068 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.056+0000 7fbf62ffd700 0 client.16536 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:46:49.068 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.056+0000 7fbf95b02b80 1 --2- 172.21.15.1:0/2916722550 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564a3ba90470 0x564a3baa0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:49.069 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.056+0000 7fbf95b02b80 1 -- 172.21.15.1:0/2916722550 >> 172.21.15.1:0/2916722550 conn(0x564a3ba85360 msgr2=0x564a3ba8e960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:49.069 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.056+0000 7fbf95b02b80 1 -- 172.21.15.1:0/2916722550 shutdown_connections 2023-11-16T16:46:49.069 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.056+0000 7fbf95b02b80 1 -- 172.21.15.1:0/2916722550 wait complete. 2023-11-16T16:46:49.069 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.056+0000 7fbf95b02b80 20 client.16536 trim_cache size 0 max 16384 2023-11-16T16:46:49.069 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:46:49.069 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-16T16:46:49.070 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2023-11-16T16:46:49.070 INFO:teuthology.orchestra.run.smithi001.stdout: "5bbb7780-b812-4543-b5e9-55b8fb65a2b0": { 2023-11-16T16:46:49.070 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2023-11-16T16:46:49.070 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "5bbb7780-b812-4543-b5e9-55b8fb65a2b0", 2023-11-16T16:46:49.070 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2023-11-16T16:46:49.070 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:46:49.070 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:46:49.070 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:46:49.072 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.056+0000 7fbf89976700 1 -- 172.21.15.1:0/2387536784 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fbf6c0a12c0 msgr2=0x7fbf6c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:49.072 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.056+0000 7fbf89976700 1 --2- 172.21.15.1:0/2387536784 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fbf6c0a12c0 0x7fbf6c0a3770 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbf84196a00 tx=0x7fbf780094f0 comp rx=0 tx=0).stop 2023-11-16T16:46:49.072 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.056+0000 7fbf89976700 1 -- 172.21.15.1:0/2387536784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf840ffa70 msgr2=0x7fbf841953f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:46:49.072 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.056+0000 7fbf89976700 1 --2- 172.21.15.1:0/2387536784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf840ffa70 0x7fbf841953f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fbf7400eab0 tx=0x7fbf7400ee70 comp rx=0 tx=0).stop 2023-11-16T16:46:49.072 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.057+0000 7fbf89976700 1 -- 172.21.15.1:0/2387536784 shutdown_connections 2023-11-16T16:46:49.072 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.057+0000 7fbf89976700 1 --2- 172.21.15.1:0/2387536784 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fbf6c0a12c0 0x7fbf6c0a3770 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:49.072 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.057+0000 7fbf89976700 1 --2- 172.21.15.1:0/2387536784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf84105250 0x7fbf84199d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:49.073 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.057+0000 7fbf89976700 1 --2- 172.21.15.1:0/2387536784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf840ffa70 0x7fbf841953f0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:49.073 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.057+0000 7fbf89976700 1 --2- 172.21.15.1:0/2387536784 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbf84100420 0x7fbf84195930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:46:49.073 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.057+0000 7fbf89976700 1 -- 172.21.15.1:0/2387536784 >> 172.21.15.1:0/2387536784 conn(0x7fbf840763a0 msgr2=0x7fbf840fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:46:49.073 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.057+0000 7fbf89976700 1 -- 172.21.15.1:0/2387536784 shutdown_connections 2023-11-16T16:46:49.073 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:46:49.057+0000 7fbf89976700 1 -- 172.21.15.1:0/2387536784 wait complete. 2023-11-16T16:46:49.073 INFO:tasks.cephfs.filesystem:scrub status for tag:5bbb7780-b812-4543-b5e9-55b8fb65a2b0 - {'path': '/', 'tag': '5bbb7780-b812-4543-b5e9-55b8fb65a2b0', 'options': 'recursive,force'} 2023-11-16T16:46:49.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:48 smithi053 ceph-mon[100638]: pgmap v234: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:46:50.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:49 smithi026 ceph-mon[100112]: scrub queued for path: / 2023-11-16T16:46:50.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:49 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:46:50.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:49 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:46:50.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:49 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:46:50.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:49 smithi026 ceph-mon[100112]: scrub summary: idle 2023-11-16T16:46:50.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:49 smithi026 ceph-mon[100112]: pgmap v235: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:46:50.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:49 smithi053 ceph-mon[100638]: scrub queued for path: / 2023-11-16T16:46:50.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:49 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:46:50.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:49 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:46:50.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:49 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:46:50.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:49 smithi053 ceph-mon[100638]: scrub summary: idle 2023-11-16T16:46:50.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:49 smithi053 ceph-mon[100638]: pgmap v235: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:46:50.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:49 smithi001 ceph-mon[98800]: scrub queued for path: / 2023-11-16T16:46:50.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:49 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:46:50.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:49 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:46:50.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:49 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:46:50.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:49 smithi001 ceph-mon[98800]: scrub summary: idle 2023-11-16T16:46:50.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:49 smithi001 ceph-mon[98800]: pgmap v235: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:46:52.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:52 smithi053 ceph-mon[100638]: pgmap v236: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-16T16:46:52.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:52 smithi001 ceph-mon[98800]: pgmap v236: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-16T16:46:52.757 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:52 smithi026 ceph-mon[100112]: pgmap v236: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-16T16:46:54.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:54 smithi001 ceph-mon[98800]: pgmap v237: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:46:55.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:54 smithi026 ceph-mon[100112]: pgmap v237: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:46:55.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:54 smithi053 ceph-mon[100638]: pgmap v237: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:46:56.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:56 smithi001 ceph-mon[98800]: pgmap v238: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-16T16:46:57.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:56 smithi026 ceph-mon[100112]: pgmap v238: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-16T16:46:57.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:56 smithi053 ceph-mon[100638]: pgmap v238: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-16T16:46:58.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:46:57 smithi026 ceph-mon[100112]: pgmap v239: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:46:58.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:46:57 smithi053 ceph-mon[100638]: pgmap v239: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:46:58.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:46:57 smithi001 ceph-mon[98800]: pgmap v239: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:47:00.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:00 smithi001 ceph-mon[98800]: pgmap v240: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:47:01.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:00 smithi026 ceph-mon[100112]: pgmap v240: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:47:01.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:00 smithi053 ceph-mon[100638]: pgmap v240: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:47:02.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:02 smithi053 ceph-mon[100638]: pgmap v241: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-11-16T16:47:02.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:02 smithi001 ceph-mon[98800]: pgmap v241: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-11-16T16:47:02.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:02 smithi026 ceph-mon[100112]: pgmap v241: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-11-16T16:47:04.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:04 smithi001 ceph-mon[98800]: pgmap v242: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-16T16:47:05.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:04 smithi026 ceph-mon[100112]: pgmap v242: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-16T16:47:05.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:04 smithi053 ceph-mon[100638]: pgmap v242: 97 pgs: 97 active+clean; 817 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-16T16:47:06.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:06 smithi001 ceph-mon[98800]: pgmap v243: 97 pgs: 97 active+clean; 817 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-11-16T16:47:07.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:06 smithi026 ceph-mon[100112]: pgmap v243: 97 pgs: 97 active+clean; 817 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-11-16T16:47:07.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:06 smithi053 ceph-mon[100638]: pgmap v243: 97 pgs: 97 active+clean; 817 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-11-16T16:47:08.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:07 smithi026 ceph-mon[100112]: pgmap v244: 97 pgs: 97 active+clean; 817 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-16T16:47:08.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:07 smithi053 ceph-mon[100638]: pgmap v244: 97 pgs: 97 active+clean; 817 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-16T16:47:08.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:07 smithi001 ceph-mon[98800]: pgmap v244: 97 pgs: 97 active+clean; 817 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-16T16:47:10.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:10 smithi053 ceph-mon[100638]: osdmap e90: 12 total, 12 up, 12 in 2023-11-16T16:47:10.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:10 smithi053 ceph-mon[100638]: pgmap v246: 97 pgs: 97 active+clean; 817 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-11-16T16:47:10.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:10 smithi001 ceph-mon[98800]: osdmap e90: 12 total, 12 up, 12 in 2023-11-16T16:47:10.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:10 smithi001 ceph-mon[98800]: pgmap v246: 97 pgs: 97 active+clean; 817 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-11-16T16:47:10.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:10 smithi026 ceph-mon[100112]: osdmap e90: 12 total, 12 up, 12 in 2023-11-16T16:47:10.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:10 smithi026 ceph-mon[100112]: pgmap v246: 97 pgs: 97 active+clean; 817 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-11-16T16:47:12.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:12 smithi053 ceph-mon[100638]: pgmap v247: 97 pgs: 97 active+clean; 817 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-16T16:47:12.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:12 smithi001 ceph-mon[98800]: pgmap v247: 97 pgs: 97 active+clean; 817 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-16T16:47:12.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:12 smithi026 ceph-mon[100112]: pgmap v247: 97 pgs: 97 active+clean; 817 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-16T16:47:14.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:14 smithi001 ceph-mon[98800]: pgmap v248: 97 pgs: 97 active+clean; 817 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-16T16:47:15.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:14 smithi026 ceph-mon[100112]: pgmap v248: 97 pgs: 97 active+clean; 817 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-16T16:47:15.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:14 smithi053 ceph-mon[100638]: pgmap v248: 97 pgs: 97 active+clean; 817 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-16T16:47:17.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:16 smithi026 ceph-mon[100112]: pgmap v249: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2023-11-16T16:47:17.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:16 smithi026 ceph-mon[100112]: osdmap e91: 12 total, 12 up, 12 in 2023-11-16T16:47:17.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:16 smithi053 ceph-mon[100638]: pgmap v249: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2023-11-16T16:47:17.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:16 smithi053 ceph-mon[100638]: osdmap e91: 12 total, 12 up, 12 in 2023-11-16T16:47:17.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:16 smithi001 ceph-mon[98800]: pgmap v249: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2023-11-16T16:47:17.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:16 smithi001 ceph-mon[98800]: osdmap e91: 12 total, 12 up, 12 in 2023-11-16T16:47:18.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:17 smithi026 ceph-mon[100112]: pgmap v251: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 368 B/s wr, 7 op/s 2023-11-16T16:47:18.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:17 smithi053 ceph-mon[100638]: pgmap v251: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 368 B/s wr, 7 op/s 2023-11-16T16:47:18.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:17 smithi001 ceph-mon[98800]: pgmap v251: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 368 B/s wr, 7 op/s 2023-11-16T16:47:19.075 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-16T16:47:19.221 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.216+0000 7f1812133700 1 -- 172.21.15.1:0/1595208160 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f180c101dc0 msgr2=0x7f180c10a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:19.221 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.216+0000 7f1812133700 1 --2- 172.21.15.1:0/1595208160 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f180c101dc0 0x7f180c10a2b0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f1800009990 tx=0x7f1800009ca0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.221 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.217+0000 7f1812133700 1 -- 172.21.15.1:0/1595208160 shutdown_connections 2023-11-16T16:47:19.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.217+0000 7f1812133700 1 --2- 172.21.15.1:0/1595208160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f180c10a880 0x7f180c10ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.217+0000 7f1812133700 1 --2- 172.21.15.1:0/1595208160 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f180c1014b0 0x7f180c101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.217+0000 7f1812133700 1 --2- 172.21.15.1:0/1595208160 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f180c101dc0 0x7f180c10a2b0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.217+0000 7f1812133700 1 -- 172.21.15.1:0/1595208160 >> 172.21.15.1:0/1595208160 conn(0x7f180c0763a0 msgr2=0x7f180c0767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:19.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.217+0000 7f1812133700 1 -- 172.21.15.1:0/1595208160 shutdown_connections 2023-11-16T16:47:19.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.217+0000 7f1812133700 1 -- 172.21.15.1:0/1595208160 wait complete. 2023-11-16T16:47:19.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.217+0000 7f1812133700 1 Processor -- start 2023-11-16T16:47:19.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.217+0000 7f1812133700 1 -- start start 2023-11-16T16:47:19.224 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.218+0000 7f1812133700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f180c1014b0 0x7f180c196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.224 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.218+0000 7f1812133700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f180c101dc0 0x7f180c1968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.224 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.218+0000 7f1812133700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f180c10a880 0x7f180c19acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.224 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.218+0000 7f1812133700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c19b300 con 0x7f180c10a880 2023-11-16T16:47:19.224 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.218+0000 7f1812133700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c19b470 con 0x7f180c1014b0 2023-11-16T16:47:19.225 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.218+0000 7f1812133700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c19b630 con 0x7f180c101dc0 2023-11-16T16:47:19.225 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.218+0000 7f180bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f180c10a880 0x7f180c19acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.225 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.218+0000 7f180bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f180c10a880 0x7f180c19acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46142/0 (socket says 172.21.15.1:46142) 2023-11-16T16:47:19.225 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.218+0000 7f180bfff700 1 -- 172.21.15.1:0/4041729434 learned_addr learned my addr 172.21.15.1:0/4041729434 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:19.226 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.218+0000 7f180affd700 1 --2- 172.21.15.1:0/4041729434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f180c101dc0 0x7f180c1968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.226 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.218+0000 7f180b7fe700 1 --2- 172.21.15.1:0/4041729434 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f180c1014b0 0x7f180c196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.226 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.218+0000 7f180bfff700 1 -- 172.21.15.1:0/4041729434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f180c101dc0 msgr2=0x7f180c1968d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:19.226 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.218+0000 7f180bfff700 1 --2- 172.21.15.1:0/4041729434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f180c101dc0 0x7f180c1968d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.227 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.218+0000 7f180bfff700 1 -- 172.21.15.1:0/4041729434 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f180c1014b0 msgr2=0x7f180c196390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:19.227 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.218+0000 7f180bfff700 1 --2- 172.21.15.1:0/4041729434 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f180c1014b0 0x7f180c196390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.227 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.218+0000 7f180bfff700 1 -- 172.21.15.1:0/4041729434 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18000095f0 con 0x7f180c10a880 2023-11-16T16:47:19.227 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.218+0000 7f180bfff700 1 --2- 172.21.15.1:0/4041729434 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f180c10a880 0x7f180c19acc0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1804007f10 tx=0x7f180400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:19.228 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.218+0000 7f1808ff9700 1 -- 172.21.15.1:0/4041729434 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1804017070 con 0x7f180c10a880 2023-11-16T16:47:19.228 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.218+0000 7f1808ff9700 1 -- 172.21.15.1:0/4041729434 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f180400f040 con 0x7f180c10a880 2023-11-16T16:47:19.228 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.218+0000 7f1812133700 1 -- 172.21.15.1:0/4041729434 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f180c110300 con 0x7f180c10a880 2023-11-16T16:47:19.228 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.219+0000 7f1808ff9700 1 -- 172.21.15.1:0/4041729434 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1804005740 con 0x7f180c10a880 2023-11-16T16:47:19.229 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.219+0000 7f1812133700 1 -- 172.21.15.1:0/4041729434 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f180c110170 con 0x7f180c10a880 2023-11-16T16:47:19.236 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.221+0000 7f1808ff9700 1 -- 172.21.15.1:0/4041729434 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f180400dc60 con 0x7f180c10a880 2023-11-16T16:47:19.236 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.221+0000 7f1808ff9700 1 --2- 172.21.15.1:0/4041729434 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f17f40a12c0 0x7f17f40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.236 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.221+0000 7f180b7fe700 1 --2- 172.21.15.1:0/4041729434 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f17f40a12c0 0x7f17f40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.221+0000 7f1808ff9700 1 -- 172.21.15.1:0/4041729434 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f18040cc4e0 con 0x7f180c10a880 2023-11-16T16:47:19.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.222+0000 7f180b7fe700 1 --2- 172.21.15.1:0/4041729434 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f17f40a12c0 0x7f17f40a3770 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f17fc005950 tx=0x7f17fc0058e0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:19.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.222+0000 7f181e2bfb80 1 Processor -- start 2023-11-16T16:47:19.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.222+0000 7f181e2bfb80 1 -- start start 2023-11-16T16:47:19.238 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.223+0000 7f181e2bfb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb6852bb60 0x55bb6852bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.238 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.223+0000 7f181e2bfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb6852c470 0x55bb68530370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.238 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.223+0000 7f181e2bfb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb685308b0 0x55bb68530d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.238 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.223+0000 7f181e2bfb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb68431060 con 0x55bb685308b0 2023-11-16T16:47:19.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.223+0000 7f181e2bfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb684838a0 con 0x55bb6852c470 2023-11-16T16:47:19.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.223+0000 7f181e2bfb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6847cd30 con 0x55bb6852bb60 2023-11-16T16:47:19.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.223+0000 7f180bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb685308b0 0x55bb68530d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.223+0000 7f180bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb685308b0 0x55bb68530d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46150/0 (socket says 172.21.15.1:46150) 2023-11-16T16:47:19.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.223+0000 7f180bfff700 1 -- 172.21.15.1:0/3170196571 learned_addr learned my addr 172.21.15.1:0/3170196571 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:19.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.223+0000 7f180affd700 1 --2- 172.21.15.1:0/3170196571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb6852c470 0x55bb68530370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.223+0000 7f180b7fe700 1 --2- 172.21.15.1:0/3170196571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb6852bb60 0x55bb6852bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.223+0000 7f180bfff700 1 -- 172.21.15.1:0/3170196571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb6852bb60 msgr2=0x55bb6852bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:19.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.223+0000 7f180bfff700 1 --2- 172.21.15.1:0/3170196571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb6852bb60 0x55bb6852bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.223+0000 7f180bfff700 1 -- 172.21.15.1:0/3170196571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb6852c470 msgr2=0x55bb68530370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:19.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.223+0000 7f180bfff700 1 --2- 172.21.15.1:0/3170196571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb6852c470 0x55bb68530370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.223+0000 7f180bfff700 1 -- 172.21.15.1:0/3170196571 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb682fe630 con 0x55bb685308b0 2023-11-16T16:47:19.242 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.223+0000 7f180b7fe700 1 --2- 172.21.15.1:0/3170196571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb6852bb60 0x55bb6852bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:47:19.242 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.223+0000 7f180bfff700 1 --2- 172.21.15.1:0/3170196571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb685308b0 0x55bb68530d00 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f1804095a00 tx=0x7f180400df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=995da631411fc7ab server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:19.242 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.223+0000 7f17f1ffb700 1 -- 172.21.15.1:0/3170196571 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18040a1950 con 0x55bb685308b0 2023-11-16T16:47:19.242 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.223+0000 7f17f1ffb700 1 -- 172.21.15.1:0/3170196571 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18040978a0 con 0x55bb685308b0 2023-11-16T16:47:19.243 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.223+0000 7f17f1ffb700 1 -- 172.21.15.1:0/3170196571 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180409d370 con 0x55bb685308b0 2023-11-16T16:47:19.243 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.224+0000 7f181e2bfb80 1 -- 172.21.15.1:0/3170196571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb685308b0 msgr2=0x55bb68530d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:19.243 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.224+0000 7f181e2bfb80 1 --2- 172.21.15.1:0/3170196571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb685308b0 0x55bb68530d00 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f1804095a00 tx=0x7f180400df60 comp rx=0 tx=0).stop 2023-11-16T16:47:19.243 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.224+0000 7f181e2bfb80 1 -- 172.21.15.1:0/3170196571 shutdown_connections 2023-11-16T16:47:19.243 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.224+0000 7f181e2bfb80 1 --2- 172.21.15.1:0/3170196571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb685308b0 0x55bb68530d00 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.244 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.224+0000 7f181e2bfb80 1 --2- 172.21.15.1:0/3170196571 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb6852bb60 0x55bb6852bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.244 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.224+0000 7f181e2bfb80 1 --2- 172.21.15.1:0/3170196571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb6852c470 0x55bb68530370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.244 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.224+0000 7f181e2bfb80 1 -- 172.21.15.1:0/3170196571 >> 172.21.15.1:0/3170196571 conn(0x55bb68521360 msgr2=0x55bb68526b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:19.244 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.224+0000 7f181e2bfb80 1 -- 172.21.15.1:0/3170196571 shutdown_connections 2023-11-16T16:47:19.245 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.224+0000 7f181e2bfb80 1 -- 172.21.15.1:0/3170196571 wait complete. 2023-11-16T16:47:19.245 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.225+0000 7f181e2bfb80 1 -- start start 2023-11-16T16:47:19.245 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.225+0000 7f181e2bfb80 1 Processor -- start 2023-11-16T16:47:19.245 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.225+0000 7f181e2bfb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb6852bb60 0x55bb68537fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.246 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.225+0000 7f181e2bfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb6852c470 0x55bb685352b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.246 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.225+0000 7f181e2bfb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb685308b0 0x55bb685358e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.246 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.225+0000 7f181e2bfb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6841e8e0 con 0x55bb685308b0 2023-11-16T16:47:19.246 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.225+0000 7f181e2bfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6841e580 con 0x55bb6852c470 2023-11-16T16:47:19.247 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.225+0000 7f181e2bfb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb68300920 con 0x55bb6852bb60 2023-11-16T16:47:19.247 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.225+0000 7f181e2bfb80 10 client.0 _ll_register_callbacks cb 0x55bb6843c780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:47:19.247 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.225+0000 7f180bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb685308b0 0x55bb685358e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.247 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.225+0000 7f180bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb685308b0 0x55bb685358e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46156/0 (socket says 172.21.15.1:46156) 2023-11-16T16:47:19.247 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.225+0000 7f180bfff700 1 -- 172.21.15.1:0/1972276043 learned_addr learned my addr 172.21.15.1:0/1972276043 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:19.248 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.225+0000 7f180b7fe700 1 --2- 172.21.15.1:0/1972276043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb6852bb60 0x55bb68537fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.248 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.225+0000 7f180affd700 1 --2- 172.21.15.1:0/1972276043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb6852c470 0x55bb685352b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.248 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.226+0000 7f180bfff700 1 -- 172.21.15.1:0/1972276043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb6852bb60 msgr2=0x55bb68537fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:19.248 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.226+0000 7f180bfff700 1 --2- 172.21.15.1:0/1972276043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb6852bb60 0x55bb68537fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.249 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.226+0000 7f180bfff700 1 -- 172.21.15.1:0/1972276043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb6852c470 msgr2=0x55bb685352b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:19.249 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.226+0000 7f180bfff700 1 --2- 172.21.15.1:0/1972276043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb6852c470 0x55bb685352b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.249 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.226+0000 7f180bfff700 1 -- 172.21.15.1:0/1972276043 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f180c110170 con 0x55bb685308b0 2023-11-16T16:47:19.249 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.226+0000 7f180bfff700 1 --2- 172.21.15.1:0/1972276043 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb685308b0 0x55bb685358e0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f1804099200 tx=0x7f18040c9b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2385ae49acd4a57b server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:19.250 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.226+0000 7f17eaffd700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:47:19.250 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.226+0000 7f17eaffd700 1 -- 172.21.15.1:0/1972276043 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180409d370 con 0x55bb685308b0 2023-11-16T16:47:19.250 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.226+0000 7f17eaffd700 1 -- 172.21.15.1:0/1972276043 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f18040058a0 con 0x55bb685308b0 2023-11-16T16:47:19.250 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.226+0000 7f181e2bfb80 1 -- 172.21.15.1:0/1972276043 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bb6851eb20 con 0x55bb685308b0 2023-11-16T16:47:19.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.226+0000 7f17eaffd700 1 -- 172.21.15.1:0/1972276043 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1804005a60 con 0x55bb685308b0 2023-11-16T16:47:19.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.226+0000 7f181e2bfb80 10 client.16548 fetch_fsmap learned FSMap version 28 2023-11-16T16:47:19.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.226+0000 7f181e2bfb80 1 -- 172.21.15.1:0/1972276043 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55bb68443190 con 0x55bb685308b0 2023-11-16T16:47:19.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.226+0000 7f17eaffd700 1 -- 172.21.15.1:0/1972276043 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bb6851eb20 con 0x55bb685308b0 2023-11-16T16:47:19.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.226+0000 7f17eaffd700 1 -- 172.21.15.1:0/1972276043 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f18040989f0 con 0x55bb685308b0 2023-11-16T16:47:19.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.226+0000 7f181e2bfb80 10 client.16548 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:47:19.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.226+0000 7f181e2bfb80 10 client.16548 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:47:19.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.226+0000 7f181e2bfb80 20 client.16548 populate_metadata read hostname 'smithi001' 2023-11-16T16:47:19.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.226+0000 7f181e2bfb80 1 --2- 172.21.15.1:0/1972276043 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55bb68545230 0x55bb68547640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.253 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.226+0000 7f181e2bfb80 4 client.16548 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:47:19.253 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.226+0000 7f181e2bfb80 1 -- 172.21.15.1:0/1972276043 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bb6843f0a0 con 0x55bb68545230 2023-11-16T16:47:19.253 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.227+0000 7f180b7fe700 1 --2- 172.21.15.1:0/1972276043 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55bb68545230 0x55bb68547640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.253 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.227+0000 7f180b7fe700 1 --2- 172.21.15.1:0/1972276043 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55bb68545230 0x55bb68547640 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77ff8973f335dadd server_cookie=4b85681498d3e77d in_seq=0 out_seq=0 2023-11-16T16:47:19.253 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.227+0000 7f17eaffd700 10 client.16548 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:47:19.254 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.228+0000 7f17eaffd700 1 -- 172.21.15.1:0/1972276043 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bb6843f0a0 con 0x55bb68545230 2023-11-16T16:47:19.254 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.228+0000 7f17eaffd700 10 client.16548 handle_command_reply: tid=0 2023-11-16T16:47:19.254 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.228+0000 7f181e2bfb80 1 client.16548 shutdown 2023-11-16T16:47:19.254 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.228+0000 7f181e2bfb80 1 -- 172.21.15.1:0/1972276043 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb685308b0 msgr2=0x55bb685358e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:19.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.228+0000 7f181e2bfb80 1 --2- 172.21.15.1:0/1972276043 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb685308b0 0x55bb685358e0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f1804099200 tx=0x7f18040c9b60 comp rx=0 tx=0).stop 2023-11-16T16:47:19.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.228+0000 7f181e2bfb80 1 -- 172.21.15.1:0/1972276043 shutdown_connections 2023-11-16T16:47:19.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.228+0000 7f181e2bfb80 1 --2- 172.21.15.1:0/1972276043 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55bb68545230 0x55bb68547640 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.228+0000 7f181e2bfb80 1 --2- 172.21.15.1:0/1972276043 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb685308b0 0x55bb685358e0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.228+0000 7f17eaffd700 0 client.16548 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:47:19.256 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.228+0000 7f181e2bfb80 1 --2- 172.21.15.1:0/1972276043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb6852bb60 0x55bb68537fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.256 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.228+0000 7f181e2bfb80 1 --2- 172.21.15.1:0/1972276043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb6852c470 0x55bb685352b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.256 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.228+0000 7f181e2bfb80 1 -- 172.21.15.1:0/1972276043 >> 172.21.15.1:0/1972276043 conn(0x55bb68521360 msgr2=0x55bb68525de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:19.256 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.228+0000 7f181e2bfb80 1 -- 172.21.15.1:0/1972276043 shutdown_connections 2023-11-16T16:47:19.257 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.228+0000 7f181e2bfb80 1 -- 172.21.15.1:0/1972276043 wait complete. 2023-11-16T16:47:19.257 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.228+0000 7f181e2bfb80 20 client.16548 trim_cache size 0 max 16384 2023-11-16T16:47:19.263 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.245+0000 7f181e2bfb80 1 Processor -- start 2023-11-16T16:47:19.263 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.245+0000 7f181e2bfb80 1 -- start start 2023-11-16T16:47:19.263 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.245+0000 7f181e2bfb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb6852bb60 0x55bb68537b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.263 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.245+0000 7f181e2bfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb6852c470 0x55bb68526560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.264 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.245+0000 7f181e2bfb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb685308b0 0x55bb68526aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.264 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.245+0000 7f181e2bfb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb68520ee0 con 0x55bb685308b0 2023-11-16T16:47:19.264 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.245+0000 7f181e2bfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb68542140 con 0x55bb6852c470 2023-11-16T16:47:19.264 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.245+0000 7f181e2bfb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6852c840 con 0x55bb6852bb60 2023-11-16T16:47:19.265 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.245+0000 7f180bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb685308b0 0x55bb68526aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.265 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.245+0000 7f180bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb685308b0 0x55bb68526aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46164/0 (socket says 172.21.15.1:46164) 2023-11-16T16:47:19.265 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.245+0000 7f180bfff700 1 -- 172.21.15.1:0/799952669 learned_addr learned my addr 172.21.15.1:0/799952669 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:19.265 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.245+0000 7f180affd700 1 --2- 172.21.15.1:0/799952669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb6852c470 0x55bb68526560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.266 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.246+0000 7f180b7fe700 1 --2- 172.21.15.1:0/799952669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb6852bb60 0x55bb68537b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.266 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.246+0000 7f180bfff700 1 -- 172.21.15.1:0/799952669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb6852bb60 msgr2=0x55bb68537b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:19.266 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.246+0000 7f180bfff700 1 --2- 172.21.15.1:0/799952669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb6852bb60 0x55bb68537b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.266 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.246+0000 7f180bfff700 1 -- 172.21.15.1:0/799952669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb6852c470 msgr2=0x55bb68526560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:19.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.246+0000 7f180bfff700 1 --2- 172.21.15.1:0/799952669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb6852c470 0x55bb68526560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.246+0000 7f180bfff700 1 -- 172.21.15.1:0/799952669 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb68522b00 con 0x55bb685308b0 2023-11-16T16:47:19.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.246+0000 7f180affd700 1 --2- 172.21.15.1:0/799952669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb6852c470 0x55bb68526560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:47:19.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.246+0000 7f180bfff700 1 --2- 172.21.15.1:0/799952669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb685308b0 0x55bb68526aa0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1804097850 tx=0x7f180400df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a61c04ca5aa6091 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:19.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.246+0000 7f17f27fc700 1 -- 172.21.15.1:0/799952669 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18040a15c0 con 0x55bb685308b0 2023-11-16T16:47:19.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.246+0000 7f17f27fc700 1 -- 172.21.15.1:0/799952669 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18040989f0 con 0x55bb685308b0 2023-11-16T16:47:19.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.246+0000 7f17f27fc700 1 -- 172.21.15.1:0/799952669 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180409d300 con 0x55bb685308b0 2023-11-16T16:47:19.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.246+0000 7f181e2bfb80 1 -- 172.21.15.1:0/799952669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb685308b0 msgr2=0x55bb68526aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:19.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.247+0000 7f181e2bfb80 1 --2- 172.21.15.1:0/799952669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb685308b0 0x55bb68526aa0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1804097850 tx=0x7f180400df60 comp rx=0 tx=0).stop 2023-11-16T16:47:19.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.247+0000 7f181e2bfb80 1 -- 172.21.15.1:0/799952669 shutdown_connections 2023-11-16T16:47:19.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.247+0000 7f181e2bfb80 1 --2- 172.21.15.1:0/799952669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb685308b0 0x55bb68526aa0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.247+0000 7f181e2bfb80 1 --2- 172.21.15.1:0/799952669 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb6852bb60 0x55bb68537b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.247+0000 7f181e2bfb80 1 --2- 172.21.15.1:0/799952669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb6852c470 0x55bb68526560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.247+0000 7f181e2bfb80 1 -- 172.21.15.1:0/799952669 >> 172.21.15.1:0/799952669 conn(0x55bb68521360 msgr2=0x55bb68532c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:19.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.247+0000 7f181e2bfb80 1 -- 172.21.15.1:0/799952669 shutdown_connections 2023-11-16T16:47:19.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.247+0000 7f181e2bfb80 1 -- 172.21.15.1:0/799952669 wait complete. 2023-11-16T16:47:19.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.247+0000 7f181e2bfb80 1 -- start start 2023-11-16T16:47:19.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.247+0000 7f181e2bfb80 1 Processor -- start 2023-11-16T16:47:19.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.247+0000 7f181e2bfb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb6852bb60 0x55bb6853a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.247+0000 7f181e2bfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb6852c470 0x55bb6853a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.247+0000 7f181e2bfb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb685308b0 0x55bb6853ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.247+0000 7f181e2bfb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb68598270 con 0x55bb685308b0 2023-11-16T16:47:19.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.247+0000 7f181e2bfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb685983e0 con 0x55bb6852c470 2023-11-16T16:47:19.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.247+0000 7f181e2bfb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb685985a0 con 0x55bb6852bb60 2023-11-16T16:47:19.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.247+0000 7f181e2bfb80 10 client.0 _ll_register_callbacks cb 0x7f1800009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:47:19.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.248+0000 7f180bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb685308b0 0x55bb6853ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.248+0000 7f180bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb685308b0 0x55bb6853ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46170/0 (socket says 172.21.15.1:46170) 2023-11-16T16:47:19.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.248+0000 7f180bfff700 1 -- 172.21.15.1:0/859152101 learned_addr learned my addr 172.21.15.1:0/859152101 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:19.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.248+0000 7f180affd700 1 --2- 172.21.15.1:0/859152101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb6852c470 0x55bb6853a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.248+0000 7f180b7fe700 1 --2- 172.21.15.1:0/859152101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb6852bb60 0x55bb6853a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.248+0000 7f180bfff700 1 -- 172.21.15.1:0/859152101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb6852bb60 msgr2=0x55bb6853a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:19.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.248+0000 7f180bfff700 1 --2- 172.21.15.1:0/859152101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb6852bb60 0x55bb6853a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.248+0000 7f180bfff700 1 -- 172.21.15.1:0/859152101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb6852c470 msgr2=0x55bb6853a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:19.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.248+0000 7f180bfff700 1 --2- 172.21.15.1:0/859152101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb6852c470 0x55bb6853a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.248+0000 7f180bfff700 1 -- 172.21.15.1:0/859152101 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb68443190 con 0x55bb685308b0 2023-11-16T16:47:19.274 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.248+0000 7f180b7fe700 1 --2- 172.21.15.1:0/859152101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb6852bb60 0x55bb6853a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:47:19.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.248+0000 7f180affd700 1 --2- 172.21.15.1:0/859152101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb6852c470 0x55bb6853a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:47:19.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.248+0000 7f180bfff700 1 --2- 172.21.15.1:0/859152101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb685308b0 0x55bb6853ac40 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f1804095f70 tx=0x7f1804095ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac5fc82ad748195 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:19.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.248+0000 7f17eaffd700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:47:19.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.248+0000 7f17eaffd700 1 -- 172.21.15.1:0/859152101 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1804099960 con 0x55bb685308b0 2023-11-16T16:47:19.275 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.248+0000 7f17eaffd700 1 -- 172.21.15.1:0/859152101 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f1804098b60 con 0x55bb685308b0 2023-11-16T16:47:19.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.248+0000 7f181e2bfb80 1 -- 172.21.15.1:0/859152101 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f18040978a0 con 0x55bb685308b0 2023-11-16T16:47:19.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.248+0000 7f17eaffd700 1 -- 172.21.15.1:0/859152101 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18040a15c0 con 0x55bb685308b0 2023-11-16T16:47:19.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.248+0000 7f181e2bfb80 10 client.16554 fetch_fsmap learned FSMap version 28 2023-11-16T16:47:19.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.248+0000 7f181e2bfb80 1 -- 172.21.15.1:0/859152101 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55bb682aade0 con 0x55bb685308b0 2023-11-16T16:47:19.276 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.249+0000 7f17eaffd700 1 -- 172.21.15.1:0/859152101 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18040978a0 con 0x55bb685308b0 2023-11-16T16:47:19.277 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.249+0000 7f17eaffd700 1 -- 172.21.15.1:0/859152101 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f1804095380 con 0x55bb685308b0 2023-11-16T16:47:19.277 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.249+0000 7f181e2bfb80 10 client.16554 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:47:19.277 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.249+0000 7f181e2bfb80 10 client.16554 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:47:19.277 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.249+0000 7f181e2bfb80 20 client.16554 populate_metadata read hostname 'smithi001' 2023-11-16T16:47:19.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.249+0000 7f181e2bfb80 1 --2- 172.21.15.1:0/859152101 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55bb685997e0 0x55bb6859bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.249+0000 7f181e2bfb80 4 client.16554 mds_command: new command op to 24521 tid=0[{"prefix": "scrub status"}] 2023-11-16T16:47:19.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.249+0000 7f181e2bfb80 1 -- 172.21.15.1:0/859152101 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bb6844b660 con 0x55bb685997e0 2023-11-16T16:47:19.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.249+0000 7f180b7fe700 1 --2- 172.21.15.1:0/859152101 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55bb685997e0 0x55bb6859bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.278 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.249+0000 7f17eaffd700 10 client.16554 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:47:19.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.249+0000 7f180b7fe700 1 --2- 172.21.15.1:0/859152101 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55bb685997e0 0x55bb6859bbc0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4a4608934edbf44 server_cookie=372a45d893f6b5f7 in_seq=0 out_seq=0 2023-11-16T16:47:19.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.250+0000 7f17eaffd700 1 -- 172.21.15.1:0/859152101 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55bb6844b660 con 0x55bb685997e0 2023-11-16T16:47:19.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.250+0000 7f17eaffd700 10 client.16554 handle_command_reply: tid=0 2023-11-16T16:47:19.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.250+0000 7f181e2bfb80 1 client.16554 shutdown 2023-11-16T16:47:19.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.250+0000 7f181e2bfb80 1 -- 172.21.15.1:0/859152101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb685308b0 msgr2=0x55bb6853ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:19.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.250+0000 7f181e2bfb80 1 --2- 172.21.15.1:0/859152101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb685308b0 0x55bb6853ac40 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f1804095f70 tx=0x7f1804095ed0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.250+0000 7f181e2bfb80 1 -- 172.21.15.1:0/859152101 shutdown_connections 2023-11-16T16:47:19.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.250+0000 7f181e2bfb80 1 --2- 172.21.15.1:0/859152101 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55bb685997e0 0x55bb6859bbc0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.250+0000 7f181e2bfb80 1 --2- 172.21.15.1:0/859152101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb685308b0 0x55bb6853ac40 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.250+0000 7f181e2bfb80 1 --2- 172.21.15.1:0/859152101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bb6852bb60 0x55bb6853a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.250+0000 7f17eaffd700 0 client.16554 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:47:19.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.250+0000 7f181e2bfb80 1 --2- 172.21.15.1:0/859152101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb6852c470 0x55bb6853a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.250+0000 7f181e2bfb80 1 -- 172.21.15.1:0/859152101 >> 172.21.15.1:0/859152101 conn(0x55bb68521360 msgr2=0x55bb68532c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:19.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.250+0000 7f181e2bfb80 1 -- 172.21.15.1:0/859152101 shutdown_connections 2023-11-16T16:47:19.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.250+0000 7f181e2bfb80 1 -- 172.21.15.1:0/859152101 wait complete. 2023-11-16T16:47:19.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.250+0000 7f181e2bfb80 20 client.16554 trim_cache size 0 max 16384 2023-11-16T16:47:19.283 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:47:19.283 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2023-11-16T16:47:19.283 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2023-11-16T16:47:19.283 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:47:19.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.250+0000 7f1812133700 1 -- 172.21.15.1:0/4041729434 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f17f40a12c0 msgr2=0x7f17f40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:19.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.250+0000 7f1812133700 1 --2- 172.21.15.1:0/4041729434 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f17f40a12c0 0x7f17f40a3770 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f17fc005950 tx=0x7f17fc0058e0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.250+0000 7f1812133700 1 -- 172.21.15.1:0/4041729434 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f180c10a880 msgr2=0x7f180c19acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:19.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.250+0000 7f1812133700 1 --2- 172.21.15.1:0/4041729434 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f180c10a880 0x7f180c19acc0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1804007f10 tx=0x7f180400d3b0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.250+0000 7f1812133700 1 -- 172.21.15.1:0/4041729434 shutdown_connections 2023-11-16T16:47:19.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.250+0000 7f1812133700 1 --2- 172.21.15.1:0/4041729434 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f17f40a12c0 0x7f17f40a3770 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.250+0000 7f1812133700 1 --2- 172.21.15.1:0/4041729434 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f180c10a880 0x7f180c19acc0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.250+0000 7f1812133700 1 --2- 172.21.15.1:0/4041729434 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f180c1014b0 0x7f180c196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.250+0000 7f1812133700 1 --2- 172.21.15.1:0/4041729434 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f180c101dc0 0x7f180c1968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.250+0000 7f1812133700 1 -- 172.21.15.1:0/4041729434 >> 172.21.15.1:0/4041729434 conn(0x7f180c0763a0 msgr2=0x7f180c102250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:19.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.251+0000 7f1812133700 1 -- 172.21.15.1:0/4041729434 shutdown_connections 2023-11-16T16:47:19.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.251+0000 7f1812133700 1 -- 172.21.15.1:0/4041729434 wait complete. 2023-11-16T16:47:19.287 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-16T16:47:19.287 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:47:19.416 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.412+0000 7f12ac633700 1 -- 172.21.15.1:0/2097880196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12a4069ad0 msgr2=0x7f12a410d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:19.416 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.412+0000 7f12ac633700 1 --2- 172.21.15.1:0/2097880196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12a4069ad0 0x7f12a410d210 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f129c00b0a0 tx=0x7f129c00b3b0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.416 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.412+0000 7f12ac633700 1 -- 172.21.15.1:0/2097880196 shutdown_connections 2023-11-16T16:47:19.416 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.412+0000 7f12ac633700 1 --2- 172.21.15.1:0/2097880196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12a4069ad0 0x7f12a410d210 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.417 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.412+0000 7f12ac633700 1 --2- 172.21.15.1:0/2097880196 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12a41064e0 0x7f12a41068b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.417 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.412+0000 7f12ac633700 1 --2- 172.21.15.1:0/2097880196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12a4069120 0x7f12a4069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.417 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.412+0000 7f12ac633700 1 -- 172.21.15.1:0/2097880196 >> 172.21.15.1:0/2097880196 conn(0x7f12a4075470 msgr2=0x7f12a4075870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:19.417 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.412+0000 7f12ac633700 1 -- 172.21.15.1:0/2097880196 shutdown_connections 2023-11-16T16:47:19.417 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.412+0000 7f12ac633700 1 -- 172.21.15.1:0/2097880196 wait complete. 2023-11-16T16:47:19.417 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.413+0000 7f12ac633700 1 Processor -- start 2023-11-16T16:47:19.417 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.413+0000 7f12ac633700 1 -- start start 2023-11-16T16:47:19.417 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.413+0000 7f12ac633700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12a4069120 0x7f12a4196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.417 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.413+0000 7f12ac633700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12a4069ad0 0x7f12a4196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.418 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.413+0000 7f12ac633700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12a41064e0 0x7f12a419af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.418 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.413+0000 7f12ac633700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a419b5c0 con 0x7f12a41064e0 2023-11-16T16:47:19.418 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.413+0000 7f12ac633700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a419b730 con 0x7f12a4069ad0 2023-11-16T16:47:19.418 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.413+0000 7f12ac633700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a419b8f0 con 0x7f12a4069120 2023-11-16T16:47:19.418 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.413+0000 7f12aabd0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12a41064e0 0x7f12a419af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.418 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.413+0000 7f12aabd0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12a41064e0 0x7f12a419af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46184/0 (socket says 172.21.15.1:46184) 2023-11-16T16:47:19.418 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.413+0000 7f12aabd0700 1 -- 172.21.15.1:0/291890706 learned_addr learned my addr 172.21.15.1:0/291890706 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:19.418 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.413+0000 7f12aa3cf700 1 --2- 172.21.15.1:0/291890706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12a4069120 0x7f12a4196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.419 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.413+0000 7f12a9bce700 1 --2- 172.21.15.1:0/291890706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12a4069ad0 0x7f12a4196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.419 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.414+0000 7f12aabd0700 1 -- 172.21.15.1:0/291890706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12a4069120 msgr2=0x7f12a4196630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:19.419 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.414+0000 7f12aabd0700 1 --2- 172.21.15.1:0/291890706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12a4069120 0x7f12a4196630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.419 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.414+0000 7f12aabd0700 1 -- 172.21.15.1:0/291890706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12a4069ad0 msgr2=0x7f12a4196b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:19.419 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.414+0000 7f12aabd0700 1 --2- 172.21.15.1:0/291890706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12a4069ad0 0x7f12a4196b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.419 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.414+0000 7f12aabd0700 1 -- 172.21.15.1:0/291890706 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f129c009d00 con 0x7f12a41064e0 2023-11-16T16:47:19.419 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.414+0000 7f12a9bce700 1 --2- 172.21.15.1:0/291890706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12a4069ad0 0x7f12a4196b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:47:19.419 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.414+0000 7f12aabd0700 1 --2- 172.21.15.1:0/291890706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12a41064e0 0x7f12a419af80 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f129c009fd0 tx=0x7f129c00bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:19.420 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.414+0000 7f129b7fe700 1 -- 172.21.15.1:0/291890706 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c00e040 con 0x7f12a41064e0 2023-11-16T16:47:19.420 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.414+0000 7f12ac633700 1 -- 172.21.15.1:0/291890706 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a4110490 con 0x7f12a41064e0 2023-11-16T16:47:19.420 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.414+0000 7f129b7fe700 1 -- 172.21.15.1:0/291890706 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f129c004e60 con 0x7f12a41064e0 2023-11-16T16:47:19.420 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.414+0000 7f129b7fe700 1 -- 172.21.15.1:0/291890706 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c0053c0 con 0x7f12a41064e0 2023-11-16T16:47:19.420 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.414+0000 7f12ac633700 1 -- 172.21.15.1:0/291890706 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a4110300 con 0x7f12a41064e0 2023-11-16T16:47:19.421 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.415+0000 7f12ac633700 1 -- 172.21.15.1:0/291890706 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12a4110620 con 0x7f12a41064e0 2023-11-16T16:47:19.421 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.416+0000 7f129b7fe700 1 -- 172.21.15.1:0/291890706 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f129c01da30 con 0x7f12a41064e0 2023-11-16T16:47:19.421 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.417+0000 7f129b7fe700 1 --2- 172.21.15.1:0/291890706 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f12900a12c0 0x7f12900a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.421 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.417+0000 7f12aa3cf700 1 --2- 172.21.15.1:0/291890706 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f12900a12c0 0x7f12900a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.422 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.417+0000 7f129b7fe700 1 -- 172.21.15.1:0/291890706 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f129c0ce690 con 0x7f12a41064e0 2023-11-16T16:47:19.422 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.417+0000 7f12aa3cf700 1 --2- 172.21.15.1:0/291890706 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f12900a12c0 0x7f12900a3770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f12a0009be0 tx=0x7f12a0009260 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:19.422 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.420+0000 7f129b7fe700 1 -- 172.21.15.1:0/291890706 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f129c097ac0 con 0x7f12a41064e0 2023-11-16T16:47:19.669 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.668+0000 7f12ac633700 1 -- 172.21.15.1:0/291890706 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f12a41979d0 con 0x7f12a41064e0 2023-11-16T16:47:19.670 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.668+0000 7f129b7fe700 1 -- 172.21.15.1:0/291890706 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10783 (secure 0 0 0) 0x7f129c0253b0 con 0x7f12a41064e0 2023-11-16T16:47:19.671 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:47:19.671 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2587563684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2587563684},{"type":"v1","addr":"172.21.15.1:6835","nonce":2587563684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/277936168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":277936168},{"type":"v1","addr":"172.21.15.1:6837","nonce":277936168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/1172575045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1172575045},{"type":"v1","addr":"172.21.15.1:6839","nonce":1172575045}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/2631175899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2631175899},{"type":"v1","addr":"172.21.15.1:6841","nonce":2631175899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3693905267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3693905267},{"type":"v1","addr":"172.21.15.53:6833","nonce":3693905267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/164668033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":164668033},{"type":"v1","addr":"172.21.15.26:6833","nonce":164668033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-16T16:37:13.918943+0000","modified":"2023-11-16T16:43:40.628525+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2140542192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2140542192},{"type":"v1","addr":"172.21.15.53:6835","nonce":2140542192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1010760275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1010760275},{"type":"v1","addr":"172.21.15.26:6835","nonce":1010760275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3197060302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3197060302},{"type":"v1","addr":"172.21.15.53:6837","nonce":3197060302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6837/1453504863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1453504863},{"type":"v1","addr":"172.21.15.26:6837","nonce":1453504863}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.53:6839/1044753024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1044753024},{"type":"v1","addr":"172.21.15.53:6839","nonce":1044753024}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6839/2192205949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2192205949},{"type":"v1","addr":"172.21.15.26:6839","nonce":2192205949}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-16T16:47:19.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.671+0000 7f12ac633700 1 -- 172.21.15.1:0/291890706 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f12900a12c0 msgr2=0x7f12900a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:19.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.671+0000 7f12ac633700 1 --2- 172.21.15.1:0/291890706 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f12900a12c0 0x7f12900a3770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f12a0009be0 tx=0x7f12a0009260 comp rx=0 tx=0).stop 2023-11-16T16:47:19.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.671+0000 7f12ac633700 1 -- 172.21.15.1:0/291890706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12a41064e0 msgr2=0x7f12a419af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:19.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.671+0000 7f12ac633700 1 --2- 172.21.15.1:0/291890706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12a41064e0 0x7f12a419af80 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f129c009fd0 tx=0x7f129c00bce0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.671+0000 7f12ac633700 1 -- 172.21.15.1:0/291890706 shutdown_connections 2023-11-16T16:47:19.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.671+0000 7f12ac633700 1 --2- 172.21.15.1:0/291890706 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f12900a12c0 0x7f12900a3770 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.671+0000 7f12ac633700 1 --2- 172.21.15.1:0/291890706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12a41064e0 0x7f12a419af80 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.674 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.671+0000 7f12ac633700 1 --2- 172.21.15.1:0/291890706 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f12a4069120 0x7f12a4196630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.674 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.671+0000 7f12ac633700 1 --2- 172.21.15.1:0/291890706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12a4069ad0 0x7f12a4196b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.674 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.671+0000 7f12ac633700 1 -- 172.21.15.1:0/291890706 >> 172.21.15.1:0/291890706 conn(0x7f12a4075470 msgr2=0x7f12a40fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:19.674 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.671+0000 7f12ac633700 1 -- 172.21.15.1:0/291890706 shutdown_connections 2023-11-16T16:47:19.674 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.671+0000 7f12ac633700 1 -- 172.21.15.1:0/291890706 wait complete. 2023-11-16T16:47:19.674 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 28 2023-11-16T16:47:19.685 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-11-16T16:47:19.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.811+0000 7f9e08acf700 1 -- 172.21.15.1:0/3003466780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e041007c0 msgr2=0x7f9e04100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:19.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.811+0000 7f9e08acf700 1 --2- 172.21.15.1:0/3003466780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e041007c0 0x7f9e04100b90 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f9df80099c0 tx=0x7f9df8009cd0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.811+0000 7f9e08acf700 1 -- 172.21.15.1:0/3003466780 shutdown_connections 2023-11-16T16:47:19.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.811+0000 7f9e08acf700 1 --2- 172.21.15.1:0/3003466780 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e04109d70 0x7f9e0410c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.811+0000 7f9e08acf700 1 --2- 172.21.15.1:0/3003466780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e041007c0 0x7f9e04100b90 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.811+0000 7f9e08acf700 1 --2- 172.21.15.1:0/3003466780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e041010d0 0x7f9e04109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.814 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.811+0000 7f9e08acf700 1 -- 172.21.15.1:0/3003466780 >> 172.21.15.1:0/3003466780 conn(0x7f9e04074c60 msgr2=0x7f9e04075060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:19.814 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.811+0000 7f9e08acf700 1 -- 172.21.15.1:0/3003466780 shutdown_connections 2023-11-16T16:47:19.814 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.811+0000 7f9e08acf700 1 -- 172.21.15.1:0/3003466780 wait complete. 2023-11-16T16:47:19.816 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.812+0000 7f9e08acf700 1 Processor -- start 2023-11-16T16:47:19.816 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.812+0000 7f9e08acf700 1 -- start start 2023-11-16T16:47:19.816 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.812+0000 7f9e08acf700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e041007c0 0x7f9e04196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.816 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.812+0000 7f9e08acf700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e041010d0 0x7f9e041969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.816 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.812+0000 7f9e08acf700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e04109d70 0x7f9e0419ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.816 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.812+0000 7f9e08acf700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0419b470 con 0x7f9e041007c0 2023-11-16T16:47:19.817 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.812+0000 7f9e08acf700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0419b5e0 con 0x7f9e04109d70 2023-11-16T16:47:19.817 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.812+0000 7f9e08acf700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0419b7a0 con 0x7f9e041010d0 2023-11-16T16:47:19.817 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.812+0000 7f9e0259c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e041007c0 0x7f9e04196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.817 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.812+0000 7f9e0259c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e041007c0 0x7f9e04196490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46194/0 (socket says 172.21.15.1:46194) 2023-11-16T16:47:19.817 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.812+0000 7f9e0259c700 1 -- 172.21.15.1:0/1951535379 learned_addr learned my addr 172.21.15.1:0/1951535379 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:19.817 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.812+0000 7f9e01d9b700 1 --2- 172.21.15.1:0/1951535379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e041010d0 0x7f9e041969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.817 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.812+0000 7f9e02d9d700 1 --2- 172.21.15.1:0/1951535379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e04109d70 0x7f9e0419ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.818 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.812+0000 7f9e0259c700 1 -- 172.21.15.1:0/1951535379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e041010d0 msgr2=0x7f9e041969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:19.818 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.812+0000 7f9e0259c700 1 --2- 172.21.15.1:0/1951535379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e041010d0 0x7f9e041969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.818 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.812+0000 7f9e0259c700 1 -- 172.21.15.1:0/1951535379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e04109d70 msgr2=0x7f9e0419ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:19.818 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.812+0000 7f9e0259c700 1 --2- 172.21.15.1:0/1951535379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e04109d70 0x7f9e0419ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.818 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.812+0000 7f9e0259c700 1 -- 172.21.15.1:0/1951535379 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df80095f0 con 0x7f9e041007c0 2023-11-16T16:47:19.818 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.813+0000 7f9e0259c700 1 --2- 172.21.15.1:0/1951535379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e041007c0 0x7f9e04196490 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f9df8004730 tx=0x7f9df8004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:19.818 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.813+0000 7f9df37fe700 1 -- 172.21.15.1:0/1951535379 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df801d070 con 0x7f9e041007c0 2023-11-16T16:47:19.818 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.813+0000 7f9e08acf700 1 -- 172.21.15.1:0/1951535379 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e0410f470 con 0x7f9e041007c0 2023-11-16T16:47:19.819 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.813+0000 7f9df37fe700 1 -- 172.21.15.1:0/1951535379 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9df8004c90 con 0x7f9e041007c0 2023-11-16T16:47:19.819 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.813+0000 7f9df37fe700 1 -- 172.21.15.1:0/1951535379 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df8003d30 con 0x7f9e041007c0 2023-11-16T16:47:19.819 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.813+0000 7f9e08acf700 1 -- 172.21.15.1:0/1951535379 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e0410f2e0 con 0x7f9e041007c0 2023-11-16T16:47:19.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.815+0000 7f9df37fe700 1 -- 172.21.15.1:0/1951535379 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f9df8005000 con 0x7f9e041007c0 2023-11-16T16:47:19.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.815+0000 7f9df37fe700 1 --2- 172.21.15.1:0/1951535379 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f9dec0a12c0 0x7f9dec0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.816+0000 7f9df37fe700 1 -- 172.21.15.1:0/1951535379 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f9df80cda10 con 0x7f9e041007c0 2023-11-16T16:47:19.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.816+0000 7f9e01d9b700 1 --2- 172.21.15.1:0/1951535379 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f9dec0a12c0 0x7f9dec0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.816+0000 7f9e01d9b700 1 --2- 172.21.15.1:0/1951535379 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f9dec0a12c0 0x7f9dec0a3770 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9e04197aa0 tx=0x7f9df400a380 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:19.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.816+0000 7f9e14c5bb80 1 Processor -- start 2023-11-16T16:47:19.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.816+0000 7f9e14c5bb80 1 -- start start 2023-11-16T16:47:19.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.817+0000 7f9e14c5bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562143f1ab60 0x562143f1af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.817+0000 7f9e14c5bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562143f1b470 0x562143f1f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.817+0000 7f9e14c5bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562143f1f8b0 0x562143f1fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.817+0000 7f9e14c5bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x562143e20060 con 0x562143f1f8b0 2023-11-16T16:47:19.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.817+0000 7f9e14c5bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562143e728a0 con 0x562143f1b470 2023-11-16T16:47:19.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.817+0000 7f9e14c5bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562143e6bd30 con 0x562143f1ab60 2023-11-16T16:47:19.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.817+0000 7f9e02d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562143f1f8b0 0x562143f1fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.817+0000 7f9e02d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562143f1f8b0 0x562143f1fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46206/0 (socket says 172.21.15.1:46206) 2023-11-16T16:47:19.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.817+0000 7f9e02d9d700 1 -- 172.21.15.1:0/457487797 learned_addr learned my addr 172.21.15.1:0/457487797 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:19.827 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.817+0000 7f9e0259c700 1 --2- 172.21.15.1:0/457487797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562143f1ab60 0x562143f1af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.828 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.817+0000 7f9e01d9b700 1 --2- 172.21.15.1:0/457487797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562143f1b470 0x562143f1f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.828 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.817+0000 7f9e02d9d700 1 -- 172.21.15.1:0/457487797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562143f1ab60 msgr2=0x562143f1af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:19.828 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.817+0000 7f9e02d9d700 1 --2- 172.21.15.1:0/457487797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562143f1ab60 0x562143f1af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.828 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.817+0000 7f9e02d9d700 1 -- 172.21.15.1:0/457487797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562143f1b470 msgr2=0x562143f1f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:19.828 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.817+0000 7f9e02d9d700 1 --2- 172.21.15.1:0/457487797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562143f1b470 0x562143f1f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.828 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.817+0000 7f9e02d9d700 1 -- 172.21.15.1:0/457487797 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562143ced630 con 0x562143f1f8b0 2023-11-16T16:47:19.828 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.817+0000 7f9e01d9b700 1 --2- 172.21.15.1:0/457487797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562143f1b470 0x562143f1f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:47:19.828 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.817+0000 7f9e02d9d700 1 --2- 172.21.15.1:0/457487797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562143f1f8b0 0x562143f1fd00 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc009dc0 tx=0x7f9dfc0077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ef4336908eaa6e server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:19.829 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.817+0000 7f9df0ff9700 1 -- 172.21.15.1:0/457487797 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc014610 con 0x562143f1f8b0 2023-11-16T16:47:19.829 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.818+0000 7f9df0ff9700 1 -- 172.21.15.1:0/457487797 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9dfc00f050 con 0x562143f1f8b0 2023-11-16T16:47:19.829 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.818+0000 7f9df0ff9700 1 -- 172.21.15.1:0/457487797 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc013900 con 0x562143f1f8b0 2023-11-16T16:47:19.829 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.818+0000 7f9e14c5bb80 1 -- 172.21.15.1:0/457487797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562143f1f8b0 msgr2=0x562143f1fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:19.829 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.818+0000 7f9e14c5bb80 1 --2- 172.21.15.1:0/457487797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562143f1f8b0 0x562143f1fd00 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc009dc0 tx=0x7f9dfc0077e0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.829 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.818+0000 7f9e14c5bb80 1 -- 172.21.15.1:0/457487797 shutdown_connections 2023-11-16T16:47:19.829 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.818+0000 7f9e14c5bb80 1 --2- 172.21.15.1:0/457487797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562143f1f8b0 0x562143f1fd00 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.830 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.818+0000 7f9e14c5bb80 1 --2- 172.21.15.1:0/457487797 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562143f1ab60 0x562143f1af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.830 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.818+0000 7f9e14c5bb80 1 --2- 172.21.15.1:0/457487797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562143f1b470 0x562143f1f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.830 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.818+0000 7f9e14c5bb80 1 -- 172.21.15.1:0/457487797 >> 172.21.15.1:0/457487797 conn(0x562143f10360 msgr2=0x562143f15b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:19.830 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.818+0000 7f9e14c5bb80 1 -- 172.21.15.1:0/457487797 shutdown_connections 2023-11-16T16:47:19.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.818+0000 7f9e14c5bb80 1 -- 172.21.15.1:0/457487797 wait complete. 2023-11-16T16:47:19.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.819+0000 7f9e14c5bb80 1 -- start start 2023-11-16T16:47:19.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.819+0000 7f9e14c5bb80 1 Processor -- start 2023-11-16T16:47:19.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.819+0000 7f9e14c5bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562143f1ab60 0x562143f27040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.819+0000 7f9e14c5bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562143f1b470 0x562143f24250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.819+0000 7f9e14c5bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562143f1f8b0 0x562143f24860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.819+0000 7f9e14c5bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x562143e0d8e0 con 0x562143f1b470 2023-11-16T16:47:19.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.819+0000 7f9e14c5bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562143e0d580 con 0x562143f1ab60 2023-11-16T16:47:19.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.819+0000 7f9e14c5bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562143cef920 con 0x562143f1f8b0 2023-11-16T16:47:19.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.819+0000 7f9e14c5bb80 10 client.0 _ll_register_callbacks cb 0x562143e2b780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:47:19.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.819+0000 7f9e01d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562143f1b470 0x562143f24250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.819+0000 7f9e01d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562143f1b470 0x562143f24250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46210/0 (socket says 172.21.15.1:46210) 2023-11-16T16:47:19.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.819+0000 7f9e01d9b700 1 -- 172.21.15.1:0/3643050827 learned_addr learned my addr 172.21.15.1:0/3643050827 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:19.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.819+0000 7f9e02d9d700 1 --2- 172.21.15.1:0/3643050827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562143f1f8b0 0x562143f24860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.819+0000 7f9e0259c700 1 --2- 172.21.15.1:0/3643050827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562143f1ab60 0x562143f27040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.819+0000 7f9e01d9b700 1 -- 172.21.15.1:0/3643050827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562143f1f8b0 msgr2=0x562143f24860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:19.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.819+0000 7f9e01d9b700 1 --2- 172.21.15.1:0/3643050827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562143f1f8b0 0x562143f24860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.819+0000 7f9e01d9b700 1 -- 172.21.15.1:0/3643050827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562143f1ab60 msgr2=0x562143f27040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:19.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.819+0000 7f9e01d9b700 1 --2- 172.21.15.1:0/3643050827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562143f1ab60 0x562143f27040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.819+0000 7f9e01d9b700 1 -- 172.21.15.1:0/3643050827 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dfc009a20 con 0x562143f1b470 2023-11-16T16:47:19.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.819+0000 7f9e02d9d700 1 --2- 172.21.15.1:0/3643050827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562143f1f8b0 0x562143f24860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:47:19.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.819+0000 7f9e0259c700 1 --2- 172.21.15.1:0/3643050827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562143f1ab60 0x562143f27040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:47:19.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.820+0000 7f9e01d9b700 1 --2- 172.21.15.1:0/3643050827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562143f1b470 0x562143f24250 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f9df4004640 tx=0x7f9df40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6832b7c712b5a7a6 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:19.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.820+0000 7f9de9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:47:19.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.820+0000 7f9de9ffb700 1 -- 172.21.15.1:0/3643050827 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df4010840 con 0x562143f1b470 2023-11-16T16:47:19.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.820+0000 7f9de9ffb700 1 -- 172.21.15.1:0/3643050827 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f9df40109a0 con 0x562143f1b470 2023-11-16T16:47:19.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.820+0000 7f9e14c5bb80 1 -- 172.21.15.1:0/3643050827 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562143f0db20 con 0x562143f1b470 2023-11-16T16:47:19.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.820+0000 7f9de9ffb700 1 -- 172.21.15.1:0/3643050827 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9df4010b60 con 0x562143f1b470 2023-11-16T16:47:19.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.820+0000 7f9e14c5bb80 10 client.16572 fetch_fsmap learned FSMap version 28 2023-11-16T16:47:19.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.820+0000 7f9e14c5bb80 1 -- 172.21.15.1:0/3643050827 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562143e32190 con 0x562143f1b470 2023-11-16T16:47:19.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.820+0000 7f9de9ffb700 1 -- 172.21.15.1:0/3643050827 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562143f0db20 con 0x562143f1b470 2023-11-16T16:47:19.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.820+0000 7f9de9ffb700 1 -- 172.21.15.1:0/3643050827 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9df4026d20 con 0x562143f1b470 2023-11-16T16:47:19.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.820+0000 7f9e14c5bb80 10 client.16572 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:47:19.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.820+0000 7f9e14c5bb80 10 client.16572 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-11-16T16:47:19.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.820+0000 7f9e14c5bb80 20 client.16572 populate_metadata read hostname 'smithi001' 2023-11-16T16:47:19.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.820+0000 7f9e14c5bb80 1 --2- 172.21.15.1:0/3643050827 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x562143f34820 0x562143f36c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.820+0000 7f9e14c5bb80 4 client.16572 mds_command: new command op to 24482 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:47:19.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.820+0000 7f9e14c5bb80 1 -- 172.21.15.1:0/3643050827 --> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562143e2e0a0 con 0x562143f34820 2023-11-16T16:47:19.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.821+0000 7f9e0259c700 1 --2- 172.21.15.1:0/3643050827 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x562143f34820 0x562143f36c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.821+0000 7f9de9ffb700 10 client.16572 ms_handle_connect on v2:172.21.15.26:6836/1453504863 2023-11-16T16:47:19.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.821+0000 7f9e0259c700 1 --2- 172.21.15.1:0/3643050827 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x562143f34820 0x562143f36c30 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b16ea31ee79a2ab server_cookie=3a408d615f0281bc in_seq=0 out_seq=0 2023-11-16T16:47:19.838 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.822+0000 7f9de9ffb700 1 -- 172.21.15.1:0/3643050827 <== mds.2 v2:172.21.15.26:6836/1453504863 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562143e2e0a0 con 0x562143f34820 2023-11-16T16:47:19.838 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.822+0000 7f9de9ffb700 10 client.16572 handle_command_reply: tid=0 2023-11-16T16:47:19.838 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.822+0000 7f9e14c5bb80 1 client.16572 shutdown 2023-11-16T16:47:19.838 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.822+0000 7f9e14c5bb80 1 -- 172.21.15.1:0/3643050827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562143f1b470 msgr2=0x562143f24250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:19.838 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.822+0000 7f9e14c5bb80 1 --2- 172.21.15.1:0/3643050827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562143f1b470 0x562143f24250 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f9df4004640 tx=0x7f9df40047c0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.838 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.822+0000 7f9e14c5bb80 1 -- 172.21.15.1:0/3643050827 shutdown_connections 2023-11-16T16:47:19.838 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.822+0000 7f9e14c5bb80 1 --2- 172.21.15.1:0/3643050827 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x562143f34820 0x562143f36c30 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.822+0000 7f9e14c5bb80 1 --2- 172.21.15.1:0/3643050827 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562143f1f8b0 0x562143f24860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.822+0000 7f9de9ffb700 0 client.16572 ms_handle_reset on v2:172.21.15.26:6836/1453504863 2023-11-16T16:47:19.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.822+0000 7f9e14c5bb80 1 --2- 172.21.15.1:0/3643050827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562143f1ab60 0x562143f27040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.822+0000 7f9e14c5bb80 1 --2- 172.21.15.1:0/3643050827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562143f1b470 0x562143f24250 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.822+0000 7f9e14c5bb80 1 -- 172.21.15.1:0/3643050827 >> 172.21.15.1:0/3643050827 conn(0x562143f10360 msgr2=0x562143f20d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:19.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.822+0000 7f9e14c5bb80 1 -- 172.21.15.1:0/3643050827 shutdown_connections 2023-11-16T16:47:19.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.822+0000 7f9e14c5bb80 1 -- 172.21.15.1:0/3643050827 wait complete. 2023-11-16T16:47:19.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.823+0000 7f9e14c5bb80 20 client.16572 trim_cache size 0 max 16384 2023-11-16T16:47:19.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.835+0000 7f9e14c5bb80 1 Processor -- start 2023-11-16T16:47:19.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.835+0000 7f9e14c5bb80 1 -- start start 2023-11-16T16:47:19.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.836+0000 7f9e14c5bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562143f1ab60 0x562143f2b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.836+0000 7f9e14c5bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562143f1b470 0x562143f26b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.836+0000 7f9e14c5bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562143f1f8b0 0x562143f1d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.836+0000 7f9e14c5bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x562143f227d0 con 0x562143f1f8b0 2023-11-16T16:47:19.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.836+0000 7f9e14c5bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562143f1e3c0 con 0x562143f1b470 2023-11-16T16:47:19.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.836+0000 7f9e14c5bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562143f1b900 con 0x562143f1ab60 2023-11-16T16:47:19.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.836+0000 7f9e02d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562143f1f8b0 0x562143f1d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.836+0000 7f9e02d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562143f1f8b0 0x562143f1d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46220/0 (socket says 172.21.15.1:46220) 2023-11-16T16:47:19.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.836+0000 7f9e02d9d700 1 -- 172.21.15.1:0/1281100088 learned_addr learned my addr 172.21.15.1:0/1281100088 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:19.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.836+0000 7f9e0259c700 1 --2- 172.21.15.1:0/1281100088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562143f1ab60 0x562143f2b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.836+0000 7f9e02d9d700 1 -- 172.21.15.1:0/1281100088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562143f1ab60 msgr2=0x562143f2b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:19.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.836+0000 7f9e02d9d700 1 --2- 172.21.15.1:0/1281100088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562143f1ab60 0x562143f2b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.836+0000 7f9e02d9d700 1 -- 172.21.15.1:0/1281100088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562143f1b470 msgr2=0x562143f26b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:19.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.836+0000 7f9e02d9d700 1 --2- 172.21.15.1:0/1281100088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562143f1b470 0x562143f26b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.847 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.836+0000 7f9e02d9d700 1 -- 172.21.15.1:0/1281100088 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562143e31ee0 con 0x562143f1f8b0 2023-11-16T16:47:19.847 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.836+0000 7f9e02d9d700 1 --2- 172.21.15.1:0/1281100088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562143f1f8b0 0x562143f1d8b0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc009dc0 tx=0x7f9dfc013430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4866a8cb7645e3f server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:19.847 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.836+0000 7f9df17fa700 1 -- 172.21.15.1:0/1281100088 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc004cf0 con 0x562143f1f8b0 2023-11-16T16:47:19.847 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.836+0000 7f9df17fa700 1 -- 172.21.15.1:0/1281100088 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9dfc004e50 con 0x562143f1f8b0 2023-11-16T16:47:19.847 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.836+0000 7f9df17fa700 1 -- 172.21.15.1:0/1281100088 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc0139b0 con 0x562143f1f8b0 2023-11-16T16:47:19.847 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.837+0000 7f9e14c5bb80 1 -- 172.21.15.1:0/1281100088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562143f1f8b0 msgr2=0x562143f1d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:19.847 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.837+0000 7f9e14c5bb80 1 --2- 172.21.15.1:0/1281100088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562143f1f8b0 0x562143f1d8b0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc009dc0 tx=0x7f9dfc013430 comp rx=0 tx=0).stop 2023-11-16T16:47:19.848 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.837+0000 7f9e14c5bb80 1 -- 172.21.15.1:0/1281100088 shutdown_connections 2023-11-16T16:47:19.848 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.837+0000 7f9e14c5bb80 1 --2- 172.21.15.1:0/1281100088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562143f1f8b0 0x562143f1d8b0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.848 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.837+0000 7f9e14c5bb80 1 --2- 172.21.15.1:0/1281100088 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562143f1ab60 0x562143f2b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.848 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.837+0000 7f9e14c5bb80 1 --2- 172.21.15.1:0/1281100088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562143f1b470 0x562143f26b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.848 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.837+0000 7f9e14c5bb80 1 -- 172.21.15.1:0/1281100088 >> 172.21.15.1:0/1281100088 conn(0x562143f10360 msgr2=0x562143f21c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:19.848 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.837+0000 7f9e14c5bb80 1 -- 172.21.15.1:0/1281100088 shutdown_connections 2023-11-16T16:47:19.848 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.837+0000 7f9e14c5bb80 1 -- 172.21.15.1:0/1281100088 wait complete. 2023-11-16T16:47:19.848 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.837+0000 7f9e14c5bb80 1 -- start start 2023-11-16T16:47:19.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.837+0000 7f9e14c5bb80 1 Processor -- start 2023-11-16T16:47:19.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.838+0000 7f9e14c5bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562143f1ab60 0x562143f2ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.838+0000 7f9e14c5bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562143f1b470 0x562143f2b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.838+0000 7f9e14c5bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562143f1f8b0 0x562143f87700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.838+0000 7f9e14c5bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x562143f2b880 con 0x562143f1ab60 2023-11-16T16:47:19.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.838+0000 7f9e14c5bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562143f2b9f0 con 0x562143f1b470 2023-11-16T16:47:19.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.838+0000 7f9e14c5bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562143f87c90 con 0x562143f1f8b0 2023-11-16T16:47:19.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.838+0000 7f9e14c5bb80 10 client.0 _ll_register_callbacks cb 0x7f9df4004030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:47:19.850 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.838+0000 7f9e0259c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562143f1ab60 0x562143f2ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.850 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.838+0000 7f9e0259c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562143f1ab60 0x562143f2ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46222/0 (socket says 172.21.15.1:46222) 2023-11-16T16:47:19.850 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.838+0000 7f9e0259c700 1 -- 172.21.15.1:0/2720078295 learned_addr learned my addr 172.21.15.1:0/2720078295 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:19.850 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.838+0000 7f9e01d9b700 1 --2- 172.21.15.1:0/2720078295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562143f1b470 0x562143f2b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.850 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.838+0000 7f9e02d9d700 1 --2- 172.21.15.1:0/2720078295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562143f1f8b0 0x562143f87700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.850 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.838+0000 7f9e0259c700 1 -- 172.21.15.1:0/2720078295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562143f1f8b0 msgr2=0x562143f87700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:19.850 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.838+0000 7f9e0259c700 1 --2- 172.21.15.1:0/2720078295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562143f1f8b0 0x562143f87700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.850 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.838+0000 7f9e0259c700 1 -- 172.21.15.1:0/2720078295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562143f1b470 msgr2=0x562143f2b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:19.851 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.838+0000 7f9e0259c700 1 --2- 172.21.15.1:0/2720078295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562143f1b470 0x562143f2b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.851 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.838+0000 7f9e0259c700 1 -- 172.21.15.1:0/2720078295 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562143ced630 con 0x562143f1ab60 2023-11-16T16:47:19.851 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.838+0000 7f9e0259c700 1 --2- 172.21.15.1:0/2720078295 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562143f1ab60 0x562143f2ab60 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f9df8097060 tx=0x7f9df801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630dfed44b73d321 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:19.851 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.838+0000 7f9de9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:47:19.851 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.838+0000 7f9de9ffb700 1 -- 172.21.15.1:0/2720078295 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df809ab60 con 0x562143f1ab60 2023-11-16T16:47:19.851 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.838+0000 7f9de9ffb700 1 -- 172.21.15.1:0/2720078295 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f9df809eb90 con 0x562143f1ab60 2023-11-16T16:47:19.851 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.838+0000 7f9e14c5bb80 1 -- 172.21.15.1:0/2720078295 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9dfc00f050 con 0x562143f1ab60 2023-11-16T16:47:19.851 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.839+0000 7f9de9ffb700 1 -- 172.21.15.1:0/2720078295 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9df80a2f20 con 0x562143f1ab60 2023-11-16T16:47:19.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.839+0000 7f9e14c5bb80 10 client.16578 fetch_fsmap learned FSMap version 28 2023-11-16T16:47:19.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.839+0000 7f9e14c5bb80 1 -- 172.21.15.1:0/2720078295 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562143c82c60 con 0x562143f1ab60 2023-11-16T16:47:19.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.839+0000 7f9de9ffb700 1 -- 172.21.15.1:0/2720078295 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9dfc00f050 con 0x562143f1ab60 2023-11-16T16:47:19.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.839+0000 7f9de9ffb700 1 -- 172.21.15.1:0/2720078295 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9df809a0e0 con 0x562143f1ab60 2023-11-16T16:47:19.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.839+0000 7f9e14c5bb80 10 client.16578 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:47:19.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.839+0000 7f9e14c5bb80 10 client.16578 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-11-16T16:47:19.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.839+0000 7f9e14c5bb80 20 client.16578 populate_metadata read hostname 'smithi001' 2023-11-16T16:47:19.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.839+0000 7f9e14c5bb80 1 --2- 172.21.15.1:0/2720078295 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x562143f89070 0x562143f8b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.853 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.839+0000 7f9e14c5bb80 4 client.16578 mds_command: new command op to 24482 tid=0[{"prefix": "damage ls"}] 2023-11-16T16:47:19.853 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.839+0000 7f9e14c5bb80 1 -- 172.21.15.1:0/2720078295 --> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562143e3a660 con 0x562143f89070 2023-11-16T16:47:19.853 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.839+0000 7f9e01d9b700 1 --2- 172.21.15.1:0/2720078295 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x562143f89070 0x562143f8b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.853 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.839+0000 7f9de9ffb700 10 client.16578 ms_handle_connect on v2:172.21.15.26:6836/1453504863 2023-11-16T16:47:19.853 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.839+0000 7f9e01d9b700 1 --2- 172.21.15.1:0/2720078295 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x562143f89070 0x562143f8b450 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=47aad917a47b1767 server_cookie=6d2387793a99cc2b in_seq=0 out_seq=0 2023-11-16T16:47:19.853 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.840+0000 7f9de9ffb700 1 -- 172.21.15.1:0/2720078295 <== mds.2 v2:172.21.15.26:6836/1453504863 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562143e3a660 con 0x562143f89070 2023-11-16T16:47:19.853 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.840+0000 7f9de9ffb700 10 client.16578 handle_command_reply: tid=0 2023-11-16T16:47:19.853 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.840+0000 7f9e14c5bb80 1 client.16578 shutdown 2023-11-16T16:47:19.854 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.840+0000 7f9e14c5bb80 1 -- 172.21.15.1:0/2720078295 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562143f1ab60 msgr2=0x562143f2ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:19.854 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.840+0000 7f9e14c5bb80 1 --2- 172.21.15.1:0/2720078295 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562143f1ab60 0x562143f2ab60 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f9df8097060 tx=0x7f9df801a040 comp rx=0 tx=0).stop 2023-11-16T16:47:19.854 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.840+0000 7f9e14c5bb80 1 -- 172.21.15.1:0/2720078295 shutdown_connections 2023-11-16T16:47:19.854 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.840+0000 7f9e14c5bb80 1 --2- 172.21.15.1:0/2720078295 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x562143f89070 0x562143f8b450 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.854 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.840+0000 7f9e14c5bb80 1 --2- 172.21.15.1:0/2720078295 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562143f1f8b0 0x562143f87700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.854 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.840+0000 7f9e14c5bb80 1 --2- 172.21.15.1:0/2720078295 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562143f1ab60 0x562143f2ab60 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.854 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.840+0000 7f9e14c5bb80 1 --2- 172.21.15.1:0/2720078295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562143f1b470 0x562143f2b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.854 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.840+0000 7f9de9ffb700 0 client.16578 ms_handle_reset on v2:172.21.15.26:6836/1453504863 2023-11-16T16:47:19.855 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.840+0000 7f9e14c5bb80 1 -- 172.21.15.1:0/2720078295 >> 172.21.15.1:0/2720078295 conn(0x562143f10360 msgr2=0x562143f1d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:19.855 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.840+0000 7f9e14c5bb80 1 -- 172.21.15.1:0/2720078295 shutdown_connections 2023-11-16T16:47:19.855 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.840+0000 7f9e14c5bb80 1 -- 172.21.15.1:0/2720078295 wait complete. 2023-11-16T16:47:19.855 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.840+0000 7f9e14c5bb80 20 client.16578 trim_cache size 0 max 16384 2023-11-16T16:47:19.855 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2023-11-16T16:47:19.856 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.840+0000 7f9e08acf700 1 -- 172.21.15.1:0/1951535379 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f9dec0a12c0 msgr2=0x7f9dec0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:19.857 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.840+0000 7f9e08acf700 1 --2- 172.21.15.1:0/1951535379 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f9dec0a12c0 0x7f9dec0a3770 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9e04197aa0 tx=0x7f9df400a380 comp rx=0 tx=0).stop 2023-11-16T16:47:19.857 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.840+0000 7f9e08acf700 1 -- 172.21.15.1:0/1951535379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e041007c0 msgr2=0x7f9e04196490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:19.857 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.840+0000 7f9e08acf700 1 --2- 172.21.15.1:0/1951535379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e041007c0 0x7f9e04196490 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f9df8004730 tx=0x7f9df8004760 comp rx=0 tx=0).stop 2023-11-16T16:47:19.857 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.841+0000 7f9e08acf700 1 -- 172.21.15.1:0/1951535379 shutdown_connections 2023-11-16T16:47:19.857 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.841+0000 7f9e08acf700 1 --2- 172.21.15.1:0/1951535379 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f9dec0a12c0 0x7f9dec0a3770 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.857 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.841+0000 7f9e08acf700 1 --2- 172.21.15.1:0/1951535379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e04109d70 0x7f9e0419ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.857 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.841+0000 7f9e08acf700 1 --2- 172.21.15.1:0/1951535379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e041007c0 0x7f9e04196490 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.858 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.841+0000 7f9e08acf700 1 --2- 172.21.15.1:0/1951535379 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9e041010d0 0x7f9e041969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.858 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.841+0000 7f9e08acf700 1 -- 172.21.15.1:0/1951535379 >> 172.21.15.1:0/1951535379 conn(0x7f9e04074c60 msgr2=0x7f9e040fc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:19.858 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.841+0000 7f9e08acf700 1 -- 172.21.15.1:0/1951535379 shutdown_connections 2023-11-16T16:47:19.858 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.841+0000 7f9e08acf700 1 -- 172.21.15.1:0/1951535379 wait complete. 2023-11-16T16:47:19.858 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-11-16T16:47:19.987 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.985+0000 7ffa07390700 1 -- 172.21.15.1:0/482028494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa000ffbb0 msgr2=0x7ffa00104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:19.987 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.985+0000 7ffa07390700 1 --2- 172.21.15.1:0/482028494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa000ffbb0 0x7ffa00104080 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc0099c0 tx=0x7ff9fc009cd0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.987 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.985+0000 7ffa07390700 1 -- 172.21.15.1:0/482028494 shutdown_connections 2023-11-16T16:47:19.987 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.985+0000 7ffa07390700 1 --2- 172.21.15.1:0/482028494 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa001046a0 0x7ffa00104b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.987 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.985+0000 7ffa07390700 1 --2- 172.21.15.1:0/482028494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa000ff2a0 0x7ffa000ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.988 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.985+0000 7ffa07390700 1 --2- 172.21.15.1:0/482028494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa000ffbb0 0x7ffa00104080 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.988 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.985+0000 7ffa07390700 1 -- 172.21.15.1:0/482028494 >> 172.21.15.1:0/482028494 conn(0x7ffa000763a0 msgr2=0x7ffa000767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:19.988 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.985+0000 7ffa07390700 1 -- 172.21.15.1:0/482028494 shutdown_connections 2023-11-16T16:47:19.988 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.985+0000 7ffa07390700 1 -- 172.21.15.1:0/482028494 wait complete. 2023-11-16T16:47:19.990 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.986+0000 7ffa07390700 1 Processor -- start 2023-11-16T16:47:19.990 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.986+0000 7ffa07390700 1 -- start start 2023-11-16T16:47:19.990 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.986+0000 7ffa07390700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa000ff2a0 0x7ffa001a2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.991 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.986+0000 7ffa07390700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa000ffbb0 0x7ffa001a2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.991 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.986+0000 7ffa07390700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa001046a0 0x7ffa0019c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:19.991 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.986+0000 7ffa07390700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0019cff0 con 0x7ffa000ff2a0 2023-11-16T16:47:19.991 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.986+0000 7ffa07390700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0019d160 con 0x7ffa000ffbb0 2023-11-16T16:47:19.991 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.986+0000 7ffa07390700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0019d320 con 0x7ffa001046a0 2023-11-16T16:47:19.991 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.986+0000 7ffa0512c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa000ff2a0 0x7ffa001a2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.991 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.986+0000 7ffa0512c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa000ff2a0 0x7ffa001a2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46236/0 (socket says 172.21.15.1:46236) 2023-11-16T16:47:19.992 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.986+0000 7ffa0512c700 1 -- 172.21.15.1:0/10406712 learned_addr learned my addr 172.21.15.1:0/10406712 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:19.992 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.986+0000 7ffa0592d700 1 --2- 172.21.15.1:0/10406712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa001046a0 0x7ffa0019c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.992 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.986+0000 7ffa0492b700 1 --2- 172.21.15.1:0/10406712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa000ffbb0 0x7ffa001a2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:19.992 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.986+0000 7ffa0512c700 1 -- 172.21.15.1:0/10406712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa001046a0 msgr2=0x7ffa0019c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:19.992 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.987+0000 7ffa0512c700 1 --2- 172.21.15.1:0/10406712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa001046a0 0x7ffa0019c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.992 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.987+0000 7ffa0512c700 1 -- 172.21.15.1:0/10406712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa000ffbb0 msgr2=0x7ffa001a2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:19.992 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.987+0000 7ffa0512c700 1 --2- 172.21.15.1:0/10406712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa000ffbb0 0x7ffa001a2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:19.992 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.987+0000 7ffa0512c700 1 -- 172.21.15.1:0/10406712 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9fc0095f0 con 0x7ffa000ff2a0 2023-11-16T16:47:19.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.987+0000 7ffa0592d700 1 --2- 172.21.15.1:0/10406712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa001046a0 0x7ffa0019c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:47:19.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.987+0000 7ffa0492b700 1 --2- 172.21.15.1:0/10406712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa000ffbb0 0x7ffa001a2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:47:19.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.987+0000 7ffa0512c700 1 --2- 172.21.15.1:0/10406712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa000ff2a0 0x7ffa001a2840 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000d980 tx=0x7ff9f000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:19.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.987+0000 7ff9f67fc700 1 -- 172.21.15.1:0/10406712 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f000bb60 con 0x7ffa000ff2a0 2023-11-16T16:47:19.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.987+0000 7ff9f67fc700 1 -- 172.21.15.1:0/10406712 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9f0007640 con 0x7ffa000ff2a0 2023-11-16T16:47:19.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.987+0000 7ffa07390700 1 -- 172.21.15.1:0/10406712 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa00116950 con 0x7ffa000ff2a0 2023-11-16T16:47:19.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.987+0000 7ff9f67fc700 1 -- 172.21.15.1:0/10406712 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0004e90 con 0x7ffa000ff2a0 2023-11-16T16:47:19.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.987+0000 7ffa07390700 1 -- 172.21.15.1:0/10406712 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa001167c0 con 0x7ffa000ff2a0 2023-11-16T16:47:20.000 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.989+0000 7ff9f67fc700 1 -- 172.21.15.1:0/10406712 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7ff9f000bcc0 con 0x7ffa000ff2a0 2023-11-16T16:47:20.000 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.990+0000 7ff9f67fc700 1 --2- 172.21.15.1:0/10406712 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7ff9ec0a12c0 0x7ff9ec0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:20.000 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.990+0000 7ffa0492b700 1 --2- 172.21.15.1:0/10406712 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7ff9ec0a12c0 0x7ff9ec0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:20.000 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.990+0000 7ff9f67fc700 1 -- 172.21.15.1:0/10406712 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7ff9f00ccc90 con 0x7ffa000ff2a0 2023-11-16T16:47:20.000 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.990+0000 7ffa0492b700 1 --2- 172.21.15.1:0/10406712 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7ff9ec0a12c0 0x7ff9ec0a3770 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc0098e0 tx=0x7ff9fc019040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:20.001 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.991+0000 7ffa1351cb80 1 Processor -- start 2023-11-16T16:47:20.001 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.991+0000 7ffa1351cb80 1 -- start start 2023-11-16T16:47:20.001 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.991+0000 7ffa1351cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593fe5d2b60 0x5593fe5d2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:20.001 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.991+0000 7ffa1351cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5593fe5d3470 0x5593fe5d7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:20.001 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.991+0000 7ffa1351cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593fe5d78b0 0x5593fe5d7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:20.001 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.991+0000 7ffa1351cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fe4d8060 con 0x5593fe5d3470 2023-11-16T16:47:20.001 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.991+0000 7ffa1351cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fe52a8a0 con 0x5593fe5d78b0 2023-11-16T16:47:20.001 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.991+0000 7ffa1351cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fe523d30 con 0x5593fe5d2b60 2023-11-16T16:47:20.002 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.991+0000 7ffa0492b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5593fe5d3470 0x5593fe5d7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:20.002 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.991+0000 7ffa0492b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5593fe5d3470 0x5593fe5d7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46246/0 (socket says 172.21.15.1:46246) 2023-11-16T16:47:20.002 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.991+0000 7ffa0492b700 1 -- 172.21.15.1:0/3254218092 learned_addr learned my addr 172.21.15.1:0/3254218092 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:20.002 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.991+0000 7ffa0512c700 1 --2- 172.21.15.1:0/3254218092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593fe5d2b60 0x5593fe5d2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:20.002 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.991+0000 7ffa0592d700 1 --2- 172.21.15.1:0/3254218092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593fe5d78b0 0x5593fe5d7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:20.002 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.991+0000 7ffa0492b700 1 -- 172.21.15.1:0/3254218092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593fe5d2b60 msgr2=0x5593fe5d2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:20.002 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.991+0000 7ffa0492b700 1 --2- 172.21.15.1:0/3254218092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593fe5d2b60 0x5593fe5d2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.002 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.991+0000 7ffa0492b700 1 -- 172.21.15.1:0/3254218092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593fe5d78b0 msgr2=0x5593fe5d7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:20.003 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.991+0000 7ffa0492b700 1 --2- 172.21.15.1:0/3254218092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593fe5d78b0 0x5593fe5d7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.003 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.991+0000 7ffa0492b700 1 -- 172.21.15.1:0/3254218092 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593fe3a5630 con 0x5593fe5d3470 2023-11-16T16:47:20.003 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.991+0000 7ffa0512c700 1 --2- 172.21.15.1:0/3254218092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593fe5d2b60 0x5593fe5d2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:47:20.003 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.991+0000 7ffa0592d700 1 --2- 172.21.15.1:0/3254218092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593fe5d78b0 0x5593fe5d7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:47:20.003 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.992+0000 7ffa0492b700 1 --2- 172.21.15.1:0/3254218092 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5593fe5d3470 0x5593fe5d7370 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc0099c0 tx=0x7ff9fc009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8eabc599246791 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:20.003 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.992+0000 7ff9eb7fe700 1 -- 172.21.15.1:0/3254218092 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc01c040 con 0x5593fe5d3470 2023-11-16T16:47:20.003 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.992+0000 7ff9eb7fe700 1 -- 172.21.15.1:0/3254218092 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9fc028040 con 0x5593fe5d3470 2023-11-16T16:47:20.003 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.992+0000 7ff9eb7fe700 1 -- 172.21.15.1:0/3254218092 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc022d80 con 0x5593fe5d3470 2023-11-16T16:47:20.004 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.992+0000 7ffa1351cb80 1 -- 172.21.15.1:0/3254218092 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5593fe5d3470 msgr2=0x5593fe5d7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:20.004 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.992+0000 7ffa1351cb80 1 --2- 172.21.15.1:0/3254218092 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5593fe5d3470 0x5593fe5d7370 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc0099c0 tx=0x7ff9fc009cd0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.004 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.993+0000 7ffa1351cb80 1 -- 172.21.15.1:0/3254218092 shutdown_connections 2023-11-16T16:47:20.004 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.993+0000 7ffa1351cb80 1 --2- 172.21.15.1:0/3254218092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593fe5d78b0 0x5593fe5d7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.004 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.993+0000 7ffa1351cb80 1 --2- 172.21.15.1:0/3254218092 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593fe5d2b60 0x5593fe5d2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.004 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.993+0000 7ffa1351cb80 1 --2- 172.21.15.1:0/3254218092 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5593fe5d3470 0x5593fe5d7370 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.004 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.993+0000 7ffa1351cb80 1 -- 172.21.15.1:0/3254218092 >> 172.21.15.1:0/3254218092 conn(0x5593fe5c8360 msgr2=0x5593fe5cdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:20.004 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.993+0000 7ffa1351cb80 1 -- 172.21.15.1:0/3254218092 shutdown_connections 2023-11-16T16:47:20.005 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.993+0000 7ffa1351cb80 1 -- 172.21.15.1:0/3254218092 wait complete. 2023-11-16T16:47:20.005 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.993+0000 7ffa1351cb80 1 -- start start 2023-11-16T16:47:20.005 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.993+0000 7ffa1351cb80 1 Processor -- start 2023-11-16T16:47:20.005 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.993+0000 7ffa1351cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593fe5d2b60 0x5593fe5df020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:20.005 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.993+0000 7ffa1351cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5593fe5d3470 0x5593fe5dc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:20.005 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.993+0000 7ffa1351cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593fe5d78b0 0x5593fe5dc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:20.005 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.994+0000 7ffa1351cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fe4c58e0 con 0x5593fe5d3470 2023-11-16T16:47:20.005 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.994+0000 7ffa1351cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fe4c5580 con 0x5593fe5d78b0 2023-11-16T16:47:20.006 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.994+0000 7ffa1351cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fe3a7920 con 0x5593fe5d2b60 2023-11-16T16:47:20.006 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.994+0000 7ffa1351cb80 10 client.0 _ll_register_callbacks cb 0x5593fe4e3780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:47:20.006 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.994+0000 7ffa0492b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5593fe5d3470 0x5593fe5dc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:20.006 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.994+0000 7ffa0492b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5593fe5d3470 0x5593fe5dc2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46252/0 (socket says 172.21.15.1:46252) 2023-11-16T16:47:20.006 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.994+0000 7ffa0492b700 1 -- 172.21.15.1:0/2905113766 learned_addr learned my addr 172.21.15.1:0/2905113766 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:20.006 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.994+0000 7ffa0512c700 1 --2- 172.21.15.1:0/2905113766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593fe5d2b60 0x5593fe5df020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:20.006 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.994+0000 7ffa0592d700 1 --2- 172.21.15.1:0/2905113766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593fe5d78b0 0x5593fe5dc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:20.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.994+0000 7ffa0492b700 1 -- 172.21.15.1:0/2905113766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593fe5d2b60 msgr2=0x5593fe5df020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:20.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.994+0000 7ffa0492b700 1 --2- 172.21.15.1:0/2905113766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593fe5d2b60 0x5593fe5df020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.994+0000 7ffa0492b700 1 -- 172.21.15.1:0/2905113766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593fe5d78b0 msgr2=0x5593fe5dc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:20.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.994+0000 7ffa0492b700 1 --2- 172.21.15.1:0/2905113766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593fe5d78b0 0x5593fe5dc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.994+0000 7ffa0492b700 1 -- 172.21.15.1:0/2905113766 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa00113fd0 con 0x5593fe5d3470 2023-11-16T16:47:20.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.994+0000 7ffa0592d700 1 --2- 172.21.15.1:0/2905113766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593fe5d78b0 0x5593fe5dc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:47:20.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.994+0000 7ffa0492b700 1 --2- 172.21.15.1:0/2905113766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5593fe5d3470 0x5593fe5dc2a0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc000c00 tx=0x7ff9fc022670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bedc6c81f75224b server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:20.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.994+0000 7ff9e8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:47:20.008 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.994+0000 7ff9e8ff9700 1 -- 172.21.15.1:0/2905113766 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc01c040 con 0x5593fe5d3470 2023-11-16T16:47:20.008 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.994+0000 7ff9e8ff9700 1 -- 172.21.15.1:0/2905113766 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7ff9fc00fc50 con 0x5593fe5d3470 2023-11-16T16:47:20.008 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.994+0000 7ffa1351cb80 1 -- 172.21.15.1:0/2905113766 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5593fe5c5b20 con 0x5593fe5d3470 2023-11-16T16:47:20.008 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.994+0000 7ff9e8ff9700 1 -- 172.21.15.1:0/2905113766 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9fc00fe10 con 0x5593fe5d3470 2023-11-16T16:47:20.008 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.994+0000 7ffa1351cb80 10 client.16590 fetch_fsmap learned FSMap version 28 2023-11-16T16:47:20.008 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.994+0000 7ffa1351cb80 1 -- 172.21.15.1:0/2905113766 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5593fe4ea190 con 0x5593fe5d3470 2023-11-16T16:47:20.008 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.995+0000 7ff9e8ff9700 1 -- 172.21.15.1:0/2905113766 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5593fe5c5b20 con 0x5593fe5d3470 2023-11-16T16:47:20.008 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.995+0000 7ff9e8ff9700 1 -- 172.21.15.1:0/2905113766 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff9fc02b070 con 0x5593fe5d3470 2023-11-16T16:47:20.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.995+0000 7ffa1351cb80 10 client.16590 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:47:20.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.995+0000 7ffa1351cb80 10 client.16590 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-11-16T16:47:20.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.995+0000 7ffa1351cb80 20 client.16590 populate_metadata read hostname 'smithi001' 2023-11-16T16:47:20.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.995+0000 7ffa1351cb80 1 --2- 172.21.15.1:0/2905113766 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x5593fe5ec490 0x5593fe5ee8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:20.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.995+0000 7ffa1351cb80 4 client.16590 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:47:20.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.995+0000 7ffa1351cb80 1 -- 172.21.15.1:0/2905113766 --> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5593fe4e60a0 con 0x5593fe5ec490 2023-11-16T16:47:20.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.995+0000 7ffa0512c700 1 --2- 172.21.15.1:0/2905113766 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x5593fe5ec490 0x5593fe5ee8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:20.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.996+0000 7ff9e8ff9700 10 client.16590 ms_handle_connect on v2:172.21.15.53:6838/1044753024 2023-11-16T16:47:20.010 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.996+0000 7ffa0512c700 1 --2- 172.21.15.1:0/2905113766 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x5593fe5ec490 0x5593fe5ee8a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1da421108e5da1f9 server_cookie=85961c046dbcbb5f in_seq=0 out_seq=0 2023-11-16T16:47:20.010 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.996+0000 7ff9e8ff9700 1 -- 172.21.15.1:0/2905113766 <== mds.1 v2:172.21.15.53:6838/1044753024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5593fe4e60a0 con 0x5593fe5ec490 2023-11-16T16:47:20.010 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.996+0000 7ff9e8ff9700 10 client.16590 handle_command_reply: tid=0 2023-11-16T16:47:20.010 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.997+0000 7ffa1351cb80 1 client.16590 shutdown 2023-11-16T16:47:20.010 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.997+0000 7ffa1351cb80 1 -- 172.21.15.1:0/2905113766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5593fe5d3470 msgr2=0x5593fe5dc2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:20.010 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.997+0000 7ffa1351cb80 1 --2- 172.21.15.1:0/2905113766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5593fe5d3470 0x5593fe5dc2a0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc000c00 tx=0x7ff9fc022670 comp rx=0 tx=0).stop 2023-11-16T16:47:20.010 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.997+0000 7ffa1351cb80 1 -- 172.21.15.1:0/2905113766 shutdown_connections 2023-11-16T16:47:20.010 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.997+0000 7ffa1351cb80 1 --2- 172.21.15.1:0/2905113766 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x5593fe5ec490 0x5593fe5ee8a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.011 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.997+0000 7ffa1351cb80 1 --2- 172.21.15.1:0/2905113766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593fe5d78b0 0x5593fe5dc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.011 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.997+0000 7ffa1351cb80 1 --2- 172.21.15.1:0/2905113766 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593fe5d2b60 0x5593fe5df020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.011 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.997+0000 7ffa1351cb80 1 --2- 172.21.15.1:0/2905113766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5593fe5d3470 0x5593fe5dc2a0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.011 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.997+0000 7ff9e8ff9700 0 client.16590 ms_handle_reset on v2:172.21.15.53:6838/1044753024 2023-11-16T16:47:20.011 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.997+0000 7ffa1351cb80 1 -- 172.21.15.1:0/2905113766 >> 172.21.15.1:0/2905113766 conn(0x5593fe5c8360 msgr2=0x5593fe5ccde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:20.011 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.997+0000 7ffa1351cb80 1 -- 172.21.15.1:0/2905113766 shutdown_connections 2023-11-16T16:47:20.011 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.997+0000 7ffa1351cb80 1 -- 172.21.15.1:0/2905113766 wait complete. 2023-11-16T16:47:20.012 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:19.997+0000 7ffa1351cb80 20 client.16590 trim_cache size 0 max 16384 2023-11-16T16:47:20.012 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:19 smithi026 ceph-mon[100112]: pgmap v252: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2023-11-16T16:47:20.012 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:19 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/291890706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:47:20.018 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.010+0000 7ffa1351cb80 1 Processor -- start 2023-11-16T16:47:20.018 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.010+0000 7ffa1351cb80 1 -- start start 2023-11-16T16:47:20.018 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.010+0000 7ffa1351cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5593fe5d2b60 0x5593fe5e3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:20.018 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.010+0000 7ffa1351cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593fe5d3470 0x5593fe5e8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:20.019 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.010+0000 7ffa1351cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593fe5d78b0 0x5593fe5e80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:20.019 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.010+0000 7ffa1351cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fe5d39a0 con 0x5593fe5d2b60 2023-11-16T16:47:20.019 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.010+0000 7ffa1351cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fe5c7ee0 con 0x5593fe5d78b0 2023-11-16T16:47:20.019 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.010+0000 7ffa1351cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fe5cf1c0 con 0x5593fe5d3470 2023-11-16T16:47:20.019 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.010+0000 7ffa0512c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5593fe5d2b60 0x5593fe5e3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:20.019 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.010+0000 7ffa0512c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5593fe5d2b60 0x5593fe5e3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46258/0 (socket says 172.21.15.1:46258) 2023-11-16T16:47:20.019 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.010+0000 7ffa0512c700 1 -- 172.21.15.1:0/2131832771 learned_addr learned my addr 172.21.15.1:0/2131832771 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:20.019 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.010+0000 7ffa0492b700 1 --2- 172.21.15.1:0/2131832771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593fe5d3470 0x5593fe5e8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:20.020 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.011+0000 7ffa0592d700 1 --2- 172.21.15.1:0/2131832771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593fe5d78b0 0x5593fe5e80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:20.020 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.011+0000 7ffa0512c700 1 -- 172.21.15.1:0/2131832771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593fe5d3470 msgr2=0x5593fe5e8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:20.020 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.011+0000 7ffa0512c700 1 --2- 172.21.15.1:0/2131832771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593fe5d3470 0x5593fe5e8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.020 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.011+0000 7ffa0512c700 1 -- 172.21.15.1:0/2131832771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593fe5d78b0 msgr2=0x5593fe5e80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:20.020 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.011+0000 7ffa0512c700 1 --2- 172.21.15.1:0/2131832771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593fe5d78b0 0x5593fe5e80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.020 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.011+0000 7ffa0512c700 1 -- 172.21.15.1:0/2131832771 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593fe5d7f50 con 0x5593fe5d2b60 2023-11-16T16:47:20.020 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.011+0000 7ffa0592d700 1 --2- 172.21.15.1:0/2131832771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593fe5d78b0 0x5593fe5e80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:47:20.020 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.011+0000 7ffa0512c700 1 --2- 172.21.15.1:0/2131832771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5593fe5d2b60 0x5593fe5e3940 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ff9f00999f0 tx=0x7ff9f0099b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3493a3c2bfb449e server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:20.021 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.011+0000 7ff9ebfff700 1 -- 172.21.15.1:0/2131832771 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0098090 con 0x5593fe5d2b60 2023-11-16T16:47:20.021 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.011+0000 7ff9ebfff700 1 -- 172.21.15.1:0/2131832771 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9f00b2270 con 0x5593fe5d2b60 2023-11-16T16:47:20.021 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.011+0000 7ff9ebfff700 1 -- 172.21.15.1:0/2131832771 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0099440 con 0x5593fe5d2b60 2023-11-16T16:47:20.021 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.011+0000 7ffa1351cb80 1 -- 172.21.15.1:0/2131832771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5593fe5d2b60 msgr2=0x5593fe5e3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:20.021 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.011+0000 7ffa1351cb80 1 --2- 172.21.15.1:0/2131832771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5593fe5d2b60 0x5593fe5e3940 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ff9f00999f0 tx=0x7ff9f0099b40 comp rx=0 tx=0).stop 2023-11-16T16:47:20.021 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.011+0000 7ffa1351cb80 1 -- 172.21.15.1:0/2131832771 shutdown_connections 2023-11-16T16:47:20.021 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.012+0000 7ffa1351cb80 1 --2- 172.21.15.1:0/2131832771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593fe5d78b0 0x5593fe5e80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.021 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.012+0000 7ffa1351cb80 1 --2- 172.21.15.1:0/2131832771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5593fe5d2b60 0x5593fe5e3940 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.012+0000 7ffa1351cb80 1 --2- 172.21.15.1:0/2131832771 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593fe5d3470 0x5593fe5e8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.012+0000 7ffa1351cb80 1 -- 172.21.15.1:0/2131832771 >> 172.21.15.1:0/2131832771 conn(0x5593fe5c8360 msgr2=0x5593fe5d8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:20.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.012+0000 7ffa1351cb80 1 -- 172.21.15.1:0/2131832771 shutdown_connections 2023-11-16T16:47:20.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.012+0000 7ffa1351cb80 1 -- 172.21.15.1:0/2131832771 wait complete. 2023-11-16T16:47:20.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.012+0000 7ffa1351cb80 1 -- start start 2023-11-16T16:47:20.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.012+0000 7ffa1351cb80 1 Processor -- start 2023-11-16T16:47:20.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.012+0000 7ffa1351cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593fe5d2b60 0x5593fe5daa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:20.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.012+0000 7ffa1351cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5593fe5d3470 0x5593fe5daf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:20.023 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.012+0000 7ffa1351cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593fe5d78b0 0x5593fe5db4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:20.023 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.012+0000 7ffa1351cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fe5dbba0 con 0x5593fe5d3470 2023-11-16T16:47:20.023 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.012+0000 7ffa1351cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fe5dbd10 con 0x5593fe5d78b0 2023-11-16T16:47:20.023 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.012+0000 7ffa1351cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fe5dbed0 con 0x5593fe5d2b60 2023-11-16T16:47:20.023 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.012+0000 7ffa1351cb80 10 client.0 _ll_register_callbacks cb 0x7ff9fc00bab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:47:20.023 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.012+0000 7ffa0492b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5593fe5d3470 0x5593fe5daf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:20.023 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.012+0000 7ffa0492b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5593fe5d3470 0x5593fe5daf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46266/0 (socket says 172.21.15.1:46266) 2023-11-16T16:47:20.024 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.012+0000 7ffa0492b700 1 -- 172.21.15.1:0/518257094 learned_addr learned my addr 172.21.15.1:0/518257094 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:20.024 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.013+0000 7ffa0592d700 1 --2- 172.21.15.1:0/518257094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593fe5d78b0 0x5593fe5db4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:20.024 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.013+0000 7ffa0512c700 1 --2- 172.21.15.1:0/518257094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593fe5d2b60 0x5593fe5daa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:20.024 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.013+0000 7ffa0492b700 1 -- 172.21.15.1:0/518257094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593fe5d2b60 msgr2=0x5593fe5daa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:20.024 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.013+0000 7ffa0492b700 1 --2- 172.21.15.1:0/518257094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593fe5d2b60 0x5593fe5daa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.024 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.013+0000 7ffa0492b700 1 -- 172.21.15.1:0/518257094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593fe5d78b0 msgr2=0x5593fe5db4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:20.024 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.013+0000 7ffa0492b700 1 --2- 172.21.15.1:0/518257094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593fe5d78b0 0x5593fe5db4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.024 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.013+0000 7ffa0492b700 1 -- 172.21.15.1:0/518257094 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa001167c0 con 0x5593fe5d3470 2023-11-16T16:47:20.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.013+0000 7ffa0492b700 1 --2- 172.21.15.1:0/518257094 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5593fe5d3470 0x5593fe5daf90 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc005f50 tx=0x7ff9fc00b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6766343332d6b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:20.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.013+0000 7ff9e8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:47:20.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.013+0000 7ff9e8ff9700 1 -- 172.21.15.1:0/518257094 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc01c050 con 0x5593fe5d3470 2023-11-16T16:47:20.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.013+0000 7ff9e8ff9700 1 -- 172.21.15.1:0/518257094 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7ff9fc02c7c0 con 0x5593fe5d3470 2023-11-16T16:47:20.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.013+0000 7ffa1351cb80 1 -- 172.21.15.1:0/518257094 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff9fc028040 con 0x5593fe5d3470 2023-11-16T16:47:20.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.013+0000 7ff9e8ff9700 1 -- 172.21.15.1:0/518257094 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9fc02ce10 con 0x5593fe5d3470 2023-11-16T16:47:20.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.013+0000 7ffa1351cb80 10 client.16596 fetch_fsmap learned FSMap version 28 2023-11-16T16:47:20.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.013+0000 7ffa1351cb80 1 -- 172.21.15.1:0/518257094 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5593fe351de0 con 0x5593fe5d3470 2023-11-16T16:47:20.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.013+0000 7ff9e8ff9700 1 -- 172.21.15.1:0/518257094 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9fc028040 con 0x5593fe5d3470 2023-11-16T16:47:20.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.014+0000 7ff9e8ff9700 1 -- 172.21.15.1:0/518257094 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff9fc00f460 con 0x5593fe5d3470 2023-11-16T16:47:20.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.014+0000 7ffa1351cb80 10 client.16596 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:47:20.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.014+0000 7ffa1351cb80 10 client.16596 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-11-16T16:47:20.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.014+0000 7ffa1351cb80 20 client.16596 populate_metadata read hostname 'smithi001' 2023-11-16T16:47:20.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.014+0000 7ffa1351cb80 1 --2- 172.21.15.1:0/518257094 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x5593fe640af0 0x5593fe642f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:20.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.014+0000 7ffa1351cb80 4 client.16596 mds_command: new command op to 24493 tid=0[{"prefix": "damage ls"}] 2023-11-16T16:47:20.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.014+0000 7ffa1351cb80 1 -- 172.21.15.1:0/518257094 --> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5593fe4f2660 con 0x5593fe640af0 2023-11-16T16:47:20.027 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.014+0000 7ffa0512c700 1 --2- 172.21.15.1:0/518257094 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x5593fe640af0 0x5593fe642f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:20.027 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.015+0000 7ff9e8ff9700 10 client.16596 ms_handle_connect on v2:172.21.15.53:6838/1044753024 2023-11-16T16:47:20.027 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.015+0000 7ffa0512c700 1 --2- 172.21.15.1:0/518257094 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x5593fe640af0 0x5593fe642f00 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8d97805df47d91f4 server_cookie=5bb68ce8ece3eb9 in_seq=0 out_seq=0 2023-11-16T16:47:20.027 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.015+0000 7ff9e8ff9700 1 -- 172.21.15.1:0/518257094 <== mds.1 v2:172.21.15.53:6838/1044753024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5593fe4f2660 con 0x5593fe640af0 2023-11-16T16:47:20.027 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.015+0000 7ff9e8ff9700 10 client.16596 handle_command_reply: tid=0 2023-11-16T16:47:20.027 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.015+0000 7ffa1351cb80 1 client.16596 shutdown 2023-11-16T16:47:20.027 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.015+0000 7ffa1351cb80 1 -- 172.21.15.1:0/518257094 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5593fe5d3470 msgr2=0x5593fe5daf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:20.027 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.015+0000 7ffa1351cb80 1 --2- 172.21.15.1:0/518257094 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5593fe5d3470 0x5593fe5daf90 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc005f50 tx=0x7ff9fc00b8c0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.028 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.015+0000 7ffa1351cb80 1 -- 172.21.15.1:0/518257094 shutdown_connections 2023-11-16T16:47:20.028 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.015+0000 7ffa1351cb80 1 --2- 172.21.15.1:0/518257094 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x5593fe640af0 0x5593fe642f00 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.028 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.015+0000 7ffa1351cb80 1 --2- 172.21.15.1:0/518257094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5593fe5d78b0 0x5593fe5db4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.028 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.015+0000 7ffa1351cb80 1 --2- 172.21.15.1:0/518257094 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5593fe5d2b60 0x5593fe5daa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.028 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.015+0000 7ffa1351cb80 1 --2- 172.21.15.1:0/518257094 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5593fe5d3470 0x5593fe5daf90 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.028 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.015+0000 7ffa1351cb80 1 -- 172.21.15.1:0/518257094 >> 172.21.15.1:0/518257094 conn(0x5593fe5c8360 msgr2=0x5593fe5dec30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:20.028 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.015+0000 7ff9e8ff9700 0 client.16596 ms_handle_reset on v2:172.21.15.53:6838/1044753024 2023-11-16T16:47:20.028 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.015+0000 7ffa1351cb80 1 -- 172.21.15.1:0/518257094 shutdown_connections 2023-11-16T16:47:20.029 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.015+0000 7ffa1351cb80 1 -- 172.21.15.1:0/518257094 wait complete. 2023-11-16T16:47:20.029 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.015+0000 7ffa1351cb80 20 client.16596 trim_cache size 0 max 16384 2023-11-16T16:47:20.029 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2023-11-16T16:47:20.030 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.016+0000 7ffa07390700 1 -- 172.21.15.1:0/10406712 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7ff9ec0a12c0 msgr2=0x7ff9ec0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:20.030 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.016+0000 7ffa07390700 1 --2- 172.21.15.1:0/10406712 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7ff9ec0a12c0 0x7ff9ec0a3770 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc0098e0 tx=0x7ff9fc019040 comp rx=0 tx=0).stop 2023-11-16T16:47:20.030 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.016+0000 7ffa07390700 1 -- 172.21.15.1:0/10406712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa000ff2a0 msgr2=0x7ffa001a2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:20.030 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.016+0000 7ffa07390700 1 --2- 172.21.15.1:0/10406712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa000ff2a0 0x7ffa001a2840 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000d980 tx=0x7ff9f000dd40 comp rx=0 tx=0).stop 2023-11-16T16:47:20.031 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.016+0000 7ffa07390700 1 -- 172.21.15.1:0/10406712 shutdown_connections 2023-11-16T16:47:20.031 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.016+0000 7ffa07390700 1 --2- 172.21.15.1:0/10406712 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7ff9ec0a12c0 0x7ff9ec0a3770 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.031 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.016+0000 7ffa07390700 1 --2- 172.21.15.1:0/10406712 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ffa001046a0 0x7ffa0019c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.031 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.016+0000 7ffa07390700 1 --2- 172.21.15.1:0/10406712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffa000ff2a0 0x7ffa001a2840 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.031 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.016+0000 7ffa07390700 1 --2- 172.21.15.1:0/10406712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa000ffbb0 0x7ffa001a2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.031 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.016+0000 7ffa07390700 1 -- 172.21.15.1:0/10406712 >> 172.21.15.1:0/10406712 conn(0x7ffa000763a0 msgr2=0x7ffa00102130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:20.031 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.016+0000 7ffa07390700 1 -- 172.21.15.1:0/10406712 shutdown_connections 2023-11-16T16:47:20.031 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.016+0000 7ffa07390700 1 -- 172.21.15.1:0/10406712 wait complete. 2023-11-16T16:47:20.032 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:19 smithi001 ceph-mon[98800]: pgmap v252: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2023-11-16T16:47:20.032 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:19 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/291890706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:47:20.033 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-16T16:47:20.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:19 smithi053 ceph-mon[100638]: pgmap v252: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2023-11-16T16:47:20.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:19 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/291890706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:47:20.163 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.159+0000 7fdc870e8700 1 -- 172.21.15.1:0/145784444 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc80100420 msgr2=0x7fdc8010bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:20.163 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.159+0000 7fdc870e8700 1 --2- 172.21.15.1:0/145784444 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc80100420 0x7fdc8010bfd0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fdc7800b0a0 tx=0x7fdc7800b3b0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.163 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.159+0000 7fdc870e8700 1 -- 172.21.15.1:0/145784444 shutdown_connections 2023-11-16T16:47:20.163 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.159+0000 7fdc870e8700 1 --2- 172.21.15.1:0/145784444 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc80100420 0x7fdc8010bfd0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.163 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.159+0000 7fdc870e8700 1 --2- 172.21.15.1:0/145784444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc80105250 0x7fdc80105620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.164 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.159+0000 7fdc870e8700 1 --2- 172.21.15.1:0/145784444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc800ffa70 0x7fdc800ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.164 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.159+0000 7fdc870e8700 1 -- 172.21.15.1:0/145784444 >> 172.21.15.1:0/145784444 conn(0x7fdc800763a0 msgr2=0x7fdc800767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:20.164 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.159+0000 7fdc870e8700 1 -- 172.21.15.1:0/145784444 shutdown_connections 2023-11-16T16:47:20.164 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.159+0000 7fdc870e8700 1 -- 172.21.15.1:0/145784444 wait complete. 2023-11-16T16:47:20.164 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.160+0000 7fdc870e8700 1 Processor -- start 2023-11-16T16:47:20.164 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.160+0000 7fdc870e8700 1 -- start start 2023-11-16T16:47:20.164 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.160+0000 7fdc870e8700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc800ffa70 0x7fdc80195110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:20.164 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.160+0000 7fdc870e8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc80100420 0x7fdc80195650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:20.165 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.160+0000 7fdc870e8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc80105250 0x7fdc80195b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:20.165 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.160+0000 7fdc870e8700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc801a4980 con 0x7fdc80105250 2023-11-16T16:47:20.165 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.160+0000 7fdc870e8700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc801a4af0 con 0x7fdc80100420 2023-11-16T16:47:20.165 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.160+0000 7fdc870e8700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc801a4cb0 con 0x7fdc800ffa70 2023-11-16T16:47:20.165 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.160+0000 7fdc85685700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc80105250 0x7fdc80195b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:20.165 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.160+0000 7fdc84e84700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc800ffa70 0x7fdc80195110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:20.165 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.160+0000 7fdc84e84700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc800ffa70 0x7fdc80195110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:42960/0 (socket says 172.21.15.1:42960) 2023-11-16T16:47:20.165 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.160+0000 7fdc85685700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc80105250 0x7fdc80195b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46282/0 (socket says 172.21.15.1:46282) 2023-11-16T16:47:20.166 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.160+0000 7fdc84e84700 1 -- 172.21.15.1:0/2579491366 learned_addr learned my addr 172.21.15.1:0/2579491366 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:20.166 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.160+0000 7fdc7ffff700 1 --2- 172.21.15.1:0/2579491366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc80100420 0x7fdc80195650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:20.166 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.160+0000 7fdc85685700 1 -- 172.21.15.1:0/2579491366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc800ffa70 msgr2=0x7fdc80195110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:20.166 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.160+0000 7fdc85685700 1 --2- 172.21.15.1:0/2579491366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc800ffa70 0x7fdc80195110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.166 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.160+0000 7fdc85685700 1 -- 172.21.15.1:0/2579491366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc80100420 msgr2=0x7fdc80195650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:20.166 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.160+0000 7fdc85685700 1 --2- 172.21.15.1:0/2579491366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc80100420 0x7fdc80195650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.166 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.160+0000 7fdc85685700 1 -- 172.21.15.1:0/2579491366 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc78009d00 con 0x7fdc80105250 2023-11-16T16:47:20.166 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.160+0000 7fdc84e84700 1 --2- 172.21.15.1:0/2579491366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc800ffa70 0x7fdc80195110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:47:20.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.161+0000 7fdc85685700 1 --2- 172.21.15.1:0/2579491366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc80105250 0x7fdc80195b90 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fdc7800b0a0 tx=0x7fdc7800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:20.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.161+0000 7fdc7dffb700 1 -- 172.21.15.1:0/2579491366 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7800e040 con 0x7fdc80105250 2023-11-16T16:47:20.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.161+0000 7fdc870e8700 1 -- 172.21.15.1:0/2579491366 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc8010f1e0 con 0x7fdc80105250 2023-11-16T16:47:20.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.161+0000 7fdc7dffb700 1 -- 172.21.15.1:0/2579491366 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc7801d470 con 0x7fdc80105250 2023-11-16T16:47:20.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.161+0000 7fdc7dffb700 1 -- 172.21.15.1:0/2579491366 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc780052a0 con 0x7fdc80105250 2023-11-16T16:47:20.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.161+0000 7fdc870e8700 1 -- 172.21.15.1:0/2579491366 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc8010f050 con 0x7fdc80105250 2023-11-16T16:47:20.173 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.163+0000 7fdc7dffb700 1 -- 172.21.15.1:0/2579491366 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fdc7801d5e0 con 0x7fdc80105250 2023-11-16T16:47:20.173 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.164+0000 7fdc7dffb700 1 --2- 172.21.15.1:0/2579491366 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fdc700a12c0 0x7fdc700a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:20.174 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.164+0000 7fdc84e84700 1 --2- 172.21.15.1:0/2579491366 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fdc700a12c0 0x7fdc700a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:20.174 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.164+0000 7fdc7dffb700 1 -- 172.21.15.1:0/2579491366 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fdc780cdff0 con 0x7fdc80105250 2023-11-16T16:47:20.174 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.164+0000 7fdc84e84700 1 --2- 172.21.15.1:0/2579491366 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fdc700a12c0 0x7fdc700a3770 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdc74011550 tx=0x7fdc74009260 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:20.174 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.164+0000 7fdc93274b80 1 Processor -- start 2023-11-16T16:47:20.174 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.164+0000 7fdc93274b80 1 -- start start 2023-11-16T16:47:20.174 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.165+0000 7fdc93274b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281b2eab60 0x56281b2eaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:20.174 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.165+0000 7fdc93274b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56281b2eb470 0x56281b2ef370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:20.175 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.165+0000 7fdc93274b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281b2ef8b0 0x56281b2efd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:20.175 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.165+0000 7fdc93274b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56281b1f0060 con 0x56281b2eb470 2023-11-16T16:47:20.175 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.165+0000 7fdc93274b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56281b2428a0 con 0x56281b2ef8b0 2023-11-16T16:47:20.175 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.165+0000 7fdc93274b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56281b23bd30 con 0x56281b2eab60 2023-11-16T16:47:20.175 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.165+0000 7fdc7ffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56281b2eb470 0x56281b2ef370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:20.175 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.165+0000 7fdc7ffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56281b2eb470 0x56281b2ef370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46288/0 (socket says 172.21.15.1:46288) 2023-11-16T16:47:20.175 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.165+0000 7fdc7ffff700 1 -- 172.21.15.1:0/2518726218 learned_addr learned my addr 172.21.15.1:0/2518726218 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:20.175 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.165+0000 7fdc84e84700 1 --2- 172.21.15.1:0/2518726218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281b2eab60 0x56281b2eaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:20.176 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.165+0000 7fdc85685700 1 --2- 172.21.15.1:0/2518726218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281b2ef8b0 0x56281b2efd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:20.176 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.165+0000 7fdc7ffff700 1 -- 172.21.15.1:0/2518726218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281b2eab60 msgr2=0x56281b2eaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:20.176 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.165+0000 7fdc7ffff700 1 --2- 172.21.15.1:0/2518726218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281b2eab60 0x56281b2eaf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.176 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.165+0000 7fdc7ffff700 1 -- 172.21.15.1:0/2518726218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281b2ef8b0 msgr2=0x56281b2efd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:20.176 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.165+0000 7fdc7ffff700 1 --2- 172.21.15.1:0/2518726218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281b2ef8b0 0x56281b2efd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.176 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.165+0000 7fdc7ffff700 1 -- 172.21.15.1:0/2518726218 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56281b0bd630 con 0x56281b2eb470 2023-11-16T16:47:20.176 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.165+0000 7fdc84e84700 1 --2- 172.21.15.1:0/2518726218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281b2eab60 0x56281b2eaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:47:20.176 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.165+0000 7fdc7ffff700 1 --2- 172.21.15.1:0/2518726218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56281b2eb470 0x56281b2ef370 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c009d20 tx=0x7fdc6c011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=889c0ca65796f1a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:20.177 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.165+0000 7fdc6affd700 1 -- 172.21.15.1:0/2518726218 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c019630 con 0x56281b2eb470 2023-11-16T16:47:20.177 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.165+0000 7fdc6affd700 1 -- 172.21.15.1:0/2518726218 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc6c019c50 con 0x56281b2eb470 2023-11-16T16:47:20.177 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.166+0000 7fdc6affd700 1 -- 172.21.15.1:0/2518726218 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c00ebe0 con 0x56281b2eb470 2023-11-16T16:47:20.177 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.166+0000 7fdc93274b80 1 -- 172.21.15.1:0/2518726218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56281b2eb470 msgr2=0x56281b2ef370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:20.177 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.166+0000 7fdc93274b80 1 --2- 172.21.15.1:0/2518726218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56281b2eb470 0x56281b2ef370 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c009d20 tx=0x7fdc6c011720 comp rx=0 tx=0).stop 2023-11-16T16:47:20.177 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.166+0000 7fdc93274b80 1 -- 172.21.15.1:0/2518726218 shutdown_connections 2023-11-16T16:47:20.177 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.166+0000 7fdc93274b80 1 --2- 172.21.15.1:0/2518726218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281b2ef8b0 0x56281b2efd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.177 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.166+0000 7fdc93274b80 1 --2- 172.21.15.1:0/2518726218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281b2eab60 0x56281b2eaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.178 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.166+0000 7fdc93274b80 1 --2- 172.21.15.1:0/2518726218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56281b2eb470 0x56281b2ef370 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.178 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.166+0000 7fdc93274b80 1 -- 172.21.15.1:0/2518726218 >> 172.21.15.1:0/2518726218 conn(0x56281b2e0360 msgr2=0x56281b2e5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:20.178 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.166+0000 7fdc93274b80 1 -- 172.21.15.1:0/2518726218 shutdown_connections 2023-11-16T16:47:20.178 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.166+0000 7fdc93274b80 1 -- 172.21.15.1:0/2518726218 wait complete. 2023-11-16T16:47:20.178 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.167+0000 7fdc93274b80 1 -- start start 2023-11-16T16:47:20.178 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.167+0000 7fdc93274b80 1 Processor -- start 2023-11-16T16:47:20.178 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.167+0000 7fdc93274b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56281b2eab60 0x56281b2f6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:20.179 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.167+0000 7fdc93274b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281b2eb470 0x56281b2f4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:20.179 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.167+0000 7fdc93274b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281b2ef8b0 0x56281b2f4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:20.179 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.167+0000 7fdc93274b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56281b1dd8e0 con 0x56281b2eab60 2023-11-16T16:47:20.179 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.167+0000 7fdc93274b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56281b1dd580 con 0x56281b2ef8b0 2023-11-16T16:47:20.179 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.167+0000 7fdc93274b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56281b0bf920 con 0x56281b2eb470 2023-11-16T16:47:20.179 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.167+0000 7fdc93274b80 10 client.0 _ll_register_callbacks cb 0x56281b1fb780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:47:20.180 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.167+0000 7fdc84e84700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56281b2eab60 0x56281b2f6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:20.180 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.167+0000 7fdc84e84700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56281b2eab60 0x56281b2f6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46292/0 (socket says 172.21.15.1:46292) 2023-11-16T16:47:20.180 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.167+0000 7fdc84e84700 1 -- 172.21.15.1:0/1923987765 learned_addr learned my addr 172.21.15.1:0/1923987765 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:20.180 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.167+0000 7fdc7ffff700 1 --2- 172.21.15.1:0/1923987765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281b2eb470 0x56281b2f4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:20.180 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.167+0000 7fdc85685700 1 --2- 172.21.15.1:0/1923987765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281b2ef8b0 0x56281b2f4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:20.180 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.167+0000 7fdc84e84700 1 -- 172.21.15.1:0/1923987765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281b2eb470 msgr2=0x56281b2f4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:20.180 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.167+0000 7fdc84e84700 1 --2- 172.21.15.1:0/1923987765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281b2eb470 0x56281b2f4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.180 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.167+0000 7fdc84e84700 1 -- 172.21.15.1:0/1923987765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281b2ef8b0 msgr2=0x56281b2f4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:20.181 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.167+0000 7fdc84e84700 1 --2- 172.21.15.1:0/1923987765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281b2ef8b0 0x56281b2f4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.181 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.167+0000 7fdc84e84700 1 -- 172.21.15.1:0/1923987765 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc6c009980 con 0x56281b2eab60 2023-11-16T16:47:20.181 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.167+0000 7fdc7ffff700 1 --2- 172.21.15.1:0/1923987765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281b2eb470 0x56281b2f4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:47:20.181 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.168+0000 7fdc84e84700 1 --2- 172.21.15.1:0/1923987765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56281b2eab60 0x56281b2f6fa0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fdc7400eec0 tx=0x7fdc740047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=648582ffe0a24bb5 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:20.181 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.168+0000 7fdc5ffff700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:47:20.181 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.168+0000 7fdc5ffff700 1 -- 172.21.15.1:0/1923987765 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7401a5b0 con 0x56281b2eab60 2023-11-16T16:47:20.181 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.168+0000 7fdc5ffff700 1 -- 172.21.15.1:0/1923987765 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fdc7401a710 con 0x56281b2eab60 2023-11-16T16:47:20.181 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.168+0000 7fdc93274b80 1 -- 172.21.15.1:0/1923987765 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56281b2ddb20 con 0x56281b2eab60 2023-11-16T16:47:20.182 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.168+0000 7fdc5ffff700 1 -- 172.21.15.1:0/1923987765 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc7401a8d0 con 0x56281b2eab60 2023-11-16T16:47:20.182 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.168+0000 7fdc93274b80 10 client.16608 fetch_fsmap learned FSMap version 28 2023-11-16T16:47:20.182 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.168+0000 7fdc93274b80 1 -- 172.21.15.1:0/1923987765 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56281b202190 con 0x56281b2eab60 2023-11-16T16:47:20.182 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.168+0000 7fdc5ffff700 1 -- 172.21.15.1:0/1923987765 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc74034080 con 0x56281b2eab60 2023-11-16T16:47:20.182 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.168+0000 7fdc5ffff700 1 -- 172.21.15.1:0/1923987765 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fdc74028b80 con 0x56281b2eab60 2023-11-16T16:47:20.182 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.168+0000 7fdc93274b80 10 client.16608 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:47:20.182 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.168+0000 7fdc93274b80 10 client.16608 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:47:20.183 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.168+0000 7fdc93274b80 20 client.16608 populate_metadata read hostname 'smithi001' 2023-11-16T16:47:20.183 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.168+0000 7fdc93274b80 1 --2- 172.21.15.1:0/1923987765 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56281b3048d0 0x56281b306ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:20.183 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.168+0000 7fdc93274b80 4 client.16608 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:47:20.183 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.168+0000 7fdc93274b80 1 -- 172.21.15.1:0/1923987765 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56281b1fe0a0 con 0x56281b3048d0 2023-11-16T16:47:20.183 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.169+0000 7fdc7ffff700 1 --2- 172.21.15.1:0/1923987765 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56281b3048d0 0x56281b306ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:20.183 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.169+0000 7fdc5ffff700 10 client.16608 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:47:20.183 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.169+0000 7fdc7ffff700 1 --2- 172.21.15.1:0/1923987765 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56281b3048d0 0x56281b306ce0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67850ee103152278 server_cookie=11ecb4ae4ee1d9fb in_seq=0 out_seq=0 2023-11-16T16:47:20.183 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.170+0000 7fdc5ffff700 1 -- 172.21.15.1:0/1923987765 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56281b1fe0a0 con 0x56281b3048d0 2023-11-16T16:47:20.184 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.170+0000 7fdc5ffff700 10 client.16608 handle_command_reply: tid=0 2023-11-16T16:47:20.184 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.170+0000 7fdc93274b80 1 client.16608 shutdown 2023-11-16T16:47:20.184 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.170+0000 7fdc93274b80 1 -- 172.21.15.1:0/1923987765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56281b2eab60 msgr2=0x56281b2f6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:20.184 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.170+0000 7fdc93274b80 1 --2- 172.21.15.1:0/1923987765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56281b2eab60 0x56281b2f6fa0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fdc7400eec0 tx=0x7fdc740047c0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.184 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.170+0000 7fdc93274b80 1 -- 172.21.15.1:0/1923987765 shutdown_connections 2023-11-16T16:47:20.184 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.170+0000 7fdc93274b80 1 --2- 172.21.15.1:0/1923987765 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56281b3048d0 0x56281b306ce0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.184 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.170+0000 7fdc5ffff700 0 client.16608 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:47:20.184 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.170+0000 7fdc93274b80 1 --2- 172.21.15.1:0/1923987765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281b2ef8b0 0x56281b2f4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.185 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.170+0000 7fdc93274b80 1 --2- 172.21.15.1:0/1923987765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56281b2eab60 0x56281b2f6fa0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.185 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.170+0000 7fdc93274b80 1 --2- 172.21.15.1:0/1923987765 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281b2eb470 0x56281b2f4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.185 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.170+0000 7fdc93274b80 1 -- 172.21.15.1:0/1923987765 >> 172.21.15.1:0/1923987765 conn(0x56281b2e0360 msgr2=0x56281b2e4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:20.185 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.170+0000 7fdc93274b80 1 -- 172.21.15.1:0/1923987765 shutdown_connections 2023-11-16T16:47:20.185 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.170+0000 7fdc93274b80 1 -- 172.21.15.1:0/1923987765 wait complete. 2023-11-16T16:47:20.185 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.170+0000 7fdc93274b80 20 client.16608 trim_cache size 0 max 16384 2023-11-16T16:47:20.191 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.183+0000 7fdc93274b80 1 Processor -- start 2023-11-16T16:47:20.191 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.183+0000 7fdc93274b80 1 -- start start 2023-11-16T16:47:20.191 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.184+0000 7fdc93274b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281b2eab60 0x56281b2fb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:20.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.184+0000 7fdc93274b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56281b2eb470 0x56281b2f6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:20.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.184+0000 7fdc93274b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281b2ef8b0 0x56281b2e5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:20.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.184+0000 7fdc93274b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56281b2ee180 con 0x56281b2eb470 2023-11-16T16:47:20.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.184+0000 7fdc93274b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56281b2eb970 con 0x56281b2ef8b0 2023-11-16T16:47:20.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.184+0000 7fdc93274b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56281b2dfee0 con 0x56281b2eab60 2023-11-16T16:47:20.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.184+0000 7fdc7ffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56281b2eb470 0x56281b2f6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:20.192 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.184+0000 7fdc7ffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56281b2eb470 0x56281b2f6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46302/0 (socket says 172.21.15.1:46302) 2023-11-16T16:47:20.193 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.184+0000 7fdc7ffff700 1 -- 172.21.15.1:0/1060110384 learned_addr learned my addr 172.21.15.1:0/1060110384 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:20.193 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.184+0000 7fdc85685700 1 --2- 172.21.15.1:0/1060110384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281b2ef8b0 0x56281b2e5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:20.193 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.184+0000 7fdc7ffff700 1 -- 172.21.15.1:0/1060110384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281b2eab60 msgr2=0x56281b2fb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:20.193 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.184+0000 7fdc84e84700 1 --2- 172.21.15.1:0/1060110384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281b2eab60 0x56281b2fb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:20.193 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.184+0000 7fdc7ffff700 1 --2- 172.21.15.1:0/1060110384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281b2eab60 0x56281b2fb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.193 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.184+0000 7fdc7ffff700 1 -- 172.21.15.1:0/1060110384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281b2ef8b0 msgr2=0x56281b2e5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:20.193 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.184+0000 7fdc7ffff700 1 --2- 172.21.15.1:0/1060110384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281b2ef8b0 0x56281b2e5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.193 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.184+0000 7fdc7ffff700 1 -- 172.21.15.1:0/1060110384 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56281b201ee0 con 0x56281b2eb470 2023-11-16T16:47:20.194 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.184+0000 7fdc85685700 1 --2- 172.21.15.1:0/1060110384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281b2ef8b0 0x56281b2e5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:47:20.194 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.184+0000 7fdc84e84700 1 --2- 172.21.15.1:0/1060110384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281b2eab60 0x56281b2fb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:47:20.194 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.184+0000 7fdc7ffff700 1 --2- 172.21.15.1:0/1060110384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56281b2eb470 0x56281b2f6b40 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c01b000 tx=0x7fdc6c0098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a103249329ec21 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:20.194 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.184+0000 7fdc6b7fe700 1 -- 172.21.15.1:0/1060110384 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c029070 con 0x56281b2eb470 2023-11-16T16:47:20.194 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.184+0000 7fdc6b7fe700 1 -- 172.21.15.1:0/1060110384 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc6c00e3b0 con 0x56281b2eb470 2023-11-16T16:47:20.194 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.184+0000 7fdc6b7fe700 1 -- 172.21.15.1:0/1060110384 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c02aa50 con 0x56281b2eb470 2023-11-16T16:47:20.194 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.185+0000 7fdc93274b80 1 -- 172.21.15.1:0/1060110384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56281b2eb470 msgr2=0x56281b2f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:20.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.185+0000 7fdc93274b80 1 --2- 172.21.15.1:0/1060110384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56281b2eb470 0x56281b2f6b40 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c01b000 tx=0x7fdc6c0098e0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.185+0000 7fdc93274b80 1 -- 172.21.15.1:0/1060110384 shutdown_connections 2023-11-16T16:47:20.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.185+0000 7fdc93274b80 1 --2- 172.21.15.1:0/1060110384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281b2ef8b0 0x56281b2e5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.185+0000 7fdc93274b80 1 --2- 172.21.15.1:0/1060110384 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281b2eab60 0x56281b2fb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.185+0000 7fdc93274b80 1 --2- 172.21.15.1:0/1060110384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56281b2eb470 0x56281b2f6b40 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.185+0000 7fdc93274b80 1 -- 172.21.15.1:0/1060110384 >> 172.21.15.1:0/1060110384 conn(0x56281b2e0360 msgr2=0x56281b300bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:20.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.185+0000 7fdc93274b80 1 -- 172.21.15.1:0/1060110384 shutdown_connections 2023-11-16T16:47:20.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.185+0000 7fdc93274b80 1 -- 172.21.15.1:0/1060110384 wait complete. 2023-11-16T16:47:20.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.185+0000 7fdc93274b80 1 -- start start 2023-11-16T16:47:20.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.185+0000 7fdc93274b80 1 Processor -- start 2023-11-16T16:47:20.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.186+0000 7fdc93274b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281b2eab60 0x56281b2f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:20.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.186+0000 7fdc93274b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281b2eb470 0x56281b2f80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:20.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.186+0000 7fdc93274b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56281b2ef8b0 0x56281b357740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:20.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.186+0000 7fdc93274b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56281b2f8840 con 0x56281b2ef8b0 2023-11-16T16:47:20.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.186+0000 7fdc93274b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56281b357c80 con 0x56281b2eb470 2023-11-16T16:47:20.197 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.186+0000 7fdc93274b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56281b357dc0 con 0x56281b2eab60 2023-11-16T16:47:20.197 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.186+0000 7fdc93274b80 10 client.0 _ll_register_callbacks cb 0x7fdc7400ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:47:20.197 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.186+0000 7fdc85685700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56281b2ef8b0 0x56281b357740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:20.197 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.186+0000 7fdc85685700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56281b2ef8b0 0x56281b357740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46310/0 (socket says 172.21.15.1:46310) 2023-11-16T16:47:20.197 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.186+0000 7fdc85685700 1 -- 172.21.15.1:0/3144286502 learned_addr learned my addr 172.21.15.1:0/3144286502 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:20.197 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.186+0000 7fdc7ffff700 1 --2- 172.21.15.1:0/3144286502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281b2eb470 0x56281b2f80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:20.197 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.186+0000 7fdc84e84700 1 --2- 172.21.15.1:0/3144286502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281b2eab60 0x56281b2f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:20.197 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.186+0000 7fdc85685700 1 -- 172.21.15.1:0/3144286502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281b2eab60 msgr2=0x56281b2f7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:20.198 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.186+0000 7fdc85685700 1 --2- 172.21.15.1:0/3144286502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281b2eab60 0x56281b2f7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.198 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.186+0000 7fdc85685700 1 -- 172.21.15.1:0/3144286502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281b2eb470 msgr2=0x56281b2f80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:20.198 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.186+0000 7fdc85685700 1 --2- 172.21.15.1:0/3144286502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281b2eb470 0x56281b2f80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.198 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.186+0000 7fdc85685700 1 -- 172.21.15.1:0/3144286502 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56281b0bd630 con 0x56281b2ef8b0 2023-11-16T16:47:20.198 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.186+0000 7fdc85685700 1 --2- 172.21.15.1:0/3144286502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56281b2ef8b0 0x56281b357740 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fdc780cb770 tx=0x7fdc7809ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1f8b0079422f71 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:20.198 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.186+0000 7fdc5ffff700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:47:20.198 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.187+0000 7fdc5ffff700 1 -- 172.21.15.1:0/3144286502 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7809b580 con 0x56281b2ef8b0 2023-11-16T16:47:20.199 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.187+0000 7fdc5ffff700 1 -- 172.21.15.1:0/3144286502 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fdc780a34e0 con 0x56281b2ef8b0 2023-11-16T16:47:20.199 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.187+0000 7fdc93274b80 1 -- 172.21.15.1:0/3144286502 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc6c019c50 con 0x56281b2ef8b0 2023-11-16T16:47:20.199 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.187+0000 7fdc5ffff700 1 -- 172.21.15.1:0/3144286502 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc780b9320 con 0x56281b2ef8b0 2023-11-16T16:47:20.199 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.187+0000 7fdc93274b80 10 client.16614 fetch_fsmap learned FSMap version 28 2023-11-16T16:47:20.199 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.187+0000 7fdc93274b80 1 -- 172.21.15.1:0/3144286502 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56281b052c60 con 0x56281b2ef8b0 2023-11-16T16:47:20.199 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.187+0000 7fdc5ffff700 1 -- 172.21.15.1:0/3144286502 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc6c019c50 con 0x56281b2ef8b0 2023-11-16T16:47:20.199 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.187+0000 7fdc5ffff700 1 -- 172.21.15.1:0/3144286502 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fdc780a3050 con 0x56281b2ef8b0 2023-11-16T16:47:20.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.187+0000 7fdc93274b80 10 client.16614 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:47:20.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.187+0000 7fdc93274b80 10 client.16614 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:47:20.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.187+0000 7fdc93274b80 20 client.16614 populate_metadata read hostname 'smithi001' 2023-11-16T16:47:20.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.187+0000 7fdc93274b80 1 --2- 172.21.15.1:0/3144286502 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56281b3048d0 0x56281b35b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:20.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.187+0000 7fdc93274b80 4 client.16614 mds_command: new command op to 24521 tid=0[{"prefix": "damage ls"}] 2023-11-16T16:47:20.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.187+0000 7fdc93274b80 1 -- 172.21.15.1:0/3144286502 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56281b20a660 con 0x56281b3048d0 2023-11-16T16:47:20.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.187+0000 7fdc84e84700 1 --2- 172.21.15.1:0/3144286502 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56281b3048d0 0x56281b35b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:20.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.188+0000 7fdc5ffff700 10 client.16614 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:47:20.201 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.188+0000 7fdc84e84700 1 --2- 172.21.15.1:0/3144286502 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56281b3048d0 0x56281b35b260 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b93c59016d171ce server_cookie=7e69822c081e3047 in_seq=0 out_seq=0 2023-11-16T16:47:20.201 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.188+0000 7fdc5ffff700 1 -- 172.21.15.1:0/3144286502 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56281b20a660 con 0x56281b3048d0 2023-11-16T16:47:20.201 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.188+0000 7fdc5ffff700 10 client.16614 handle_command_reply: tid=0 2023-11-16T16:47:20.201 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.188+0000 7fdc93274b80 1 client.16614 shutdown 2023-11-16T16:47:20.201 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.188+0000 7fdc93274b80 1 -- 172.21.15.1:0/3144286502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56281b2ef8b0 msgr2=0x56281b357740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:20.201 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.188+0000 7fdc93274b80 1 --2- 172.21.15.1:0/3144286502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56281b2ef8b0 0x56281b357740 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fdc780cb770 tx=0x7fdc7809ad50 comp rx=0 tx=0).stop 2023-11-16T16:47:20.201 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.188+0000 7fdc93274b80 1 -- 172.21.15.1:0/3144286502 shutdown_connections 2023-11-16T16:47:20.201 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.188+0000 7fdc93274b80 1 --2- 172.21.15.1:0/3144286502 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56281b3048d0 0x56281b35b260 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.202 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.188+0000 7fdc93274b80 1 --2- 172.21.15.1:0/3144286502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56281b2ef8b0 0x56281b357740 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.202 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.188+0000 7fdc5ffff700 0 client.16614 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:47:20.202 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.188+0000 7fdc93274b80 1 --2- 172.21.15.1:0/3144286502 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56281b2eab60 0x56281b2f7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.202 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.188+0000 7fdc93274b80 1 --2- 172.21.15.1:0/3144286502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56281b2eb470 0x56281b2f80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.202 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.188+0000 7fdc93274b80 1 -- 172.21.15.1:0/3144286502 >> 172.21.15.1:0/3144286502 conn(0x56281b2e0360 msgr2=0x56281b2e9960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:20.202 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.188+0000 7fdc93274b80 1 -- 172.21.15.1:0/3144286502 shutdown_connections 2023-11-16T16:47:20.202 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.188+0000 7fdc93274b80 1 -- 172.21.15.1:0/3144286502 wait complete. 2023-11-16T16:47:20.203 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.188+0000 7fdc93274b80 20 client.16614 trim_cache size 0 max 16384 2023-11-16T16:47:20.203 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2023-11-16T16:47:20.204 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.189+0000 7fdc870e8700 1 -- 172.21.15.1:0/2579491366 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fdc700a12c0 msgr2=0x7fdc700a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:20.204 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.189+0000 7fdc870e8700 1 --2- 172.21.15.1:0/2579491366 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fdc700a12c0 0x7fdc700a3770 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdc74011550 tx=0x7fdc74009260 comp rx=0 tx=0).stop 2023-11-16T16:47:20.204 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.189+0000 7fdc870e8700 1 -- 172.21.15.1:0/2579491366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc80105250 msgr2=0x7fdc80195b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:20.205 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.189+0000 7fdc870e8700 1 --2- 172.21.15.1:0/2579491366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc80105250 0x7fdc80195b90 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fdc7800b0a0 tx=0x7fdc7800bea0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.205 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.189+0000 7fdc870e8700 1 -- 172.21.15.1:0/2579491366 shutdown_connections 2023-11-16T16:47:20.205 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.189+0000 7fdc870e8700 1 --2- 172.21.15.1:0/2579491366 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fdc700a12c0 0x7fdc700a3770 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.205 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.189+0000 7fdc870e8700 1 --2- 172.21.15.1:0/2579491366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc80105250 0x7fdc80195b90 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.205 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.189+0000 7fdc870e8700 1 --2- 172.21.15.1:0/2579491366 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdc800ffa70 0x7fdc80195110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.205 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.189+0000 7fdc870e8700 1 --2- 172.21.15.1:0/2579491366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc80100420 0x7fdc80195650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:20.205 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.189+0000 7fdc870e8700 1 -- 172.21.15.1:0/2579491366 >> 172.21.15.1:0/2579491366 conn(0x7fdc800763a0 msgr2=0x7fdc800fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:20.206 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.189+0000 7fdc870e8700 1 -- 172.21.15.1:0/2579491366 shutdown_connections 2023-11-16T16:47:20.206 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:20.189+0000 7fdc870e8700 1 -- 172.21.15.1:0/2579491366 wait complete. 2023-11-16T16:47:21.207 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-16T16:47:21.207 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-16T16:47:21.352 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.350+0000 7f8171c75700 1 -- 172.21.15.1:0/1430288670 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f816c1014b0 msgr2=0x7f816c101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:21.352 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.350+0000 7f8171c75700 1 --2- 172.21.15.1:0/1430288670 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f816c1014b0 0x7f816c101880 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f815c009990 tx=0x7f815c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.352 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.350+0000 7f8171c75700 1 -- 172.21.15.1:0/1430288670 shutdown_connections 2023-11-16T16:47:21.353 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.350+0000 7f8171c75700 1 --2- 172.21.15.1:0/1430288670 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f816c10a880 0x7f816c10ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.353 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.350+0000 7f8171c75700 1 --2- 172.21.15.1:0/1430288670 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f816c1014b0 0x7f816c101880 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.353 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.350+0000 7f8171c75700 1 --2- 172.21.15.1:0/1430288670 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f816c101dc0 0x7f816c10a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.353 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.350+0000 7f8171c75700 1 -- 172.21.15.1:0/1430288670 >> 172.21.15.1:0/1430288670 conn(0x7f816c0763a0 msgr2=0x7f816c0767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:21.353 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.350+0000 7f8171c75700 1 -- 172.21.15.1:0/1430288670 shutdown_connections 2023-11-16T16:47:21.353 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.350+0000 7f8171c75700 1 -- 172.21.15.1:0/1430288670 wait complete. 2023-11-16T16:47:21.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.351+0000 7f8171c75700 1 Processor -- start 2023-11-16T16:47:21.355 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.351+0000 7f8171c75700 1 -- start start 2023-11-16T16:47:21.356 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.351+0000 7f8171c75700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f816c1014b0 0x7f816c196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:21.356 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.351+0000 7f8171c75700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f816c101dc0 0x7f816c196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:21.356 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.351+0000 7f8171c75700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f816c10a880 0x7f816c19aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:21.356 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.351+0000 7f8171c75700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816c19b2e0 con 0x7f816c10a880 2023-11-16T16:47:21.356 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.351+0000 7f8171c75700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816c19b450 con 0x7f816c1014b0 2023-11-16T16:47:21.356 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.352+0000 7f8171c75700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816c19b610 con 0x7f816c101dc0 2023-11-16T16:47:21.356 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.352+0000 7f816bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f816c10a880 0x7f816c19aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:21.356 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.352+0000 7f816bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f816c10a880 0x7f816c19aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46324/0 (socket says 172.21.15.1:46324) 2023-11-16T16:47:21.357 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.352+0000 7f816bfff700 1 -- 172.21.15.1:0/756539768 learned_addr learned my addr 172.21.15.1:0/756539768 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:21.357 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.352+0000 7f816b7fe700 1 --2- 172.21.15.1:0/756539768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f816c1014b0 0x7f816c196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:21.357 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.352+0000 7f816affd700 1 --2- 172.21.15.1:0/756539768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f816c101dc0 0x7f816c196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:21.357 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.352+0000 7f816bfff700 1 -- 172.21.15.1:0/756539768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f816c101dc0 msgr2=0x7f816c196840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:21.357 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.352+0000 7f816bfff700 1 --2- 172.21.15.1:0/756539768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f816c101dc0 0x7f816c196840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.357 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.352+0000 7f816bfff700 1 -- 172.21.15.1:0/756539768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f816c1014b0 msgr2=0x7f816c196300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:21.357 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.352+0000 7f816bfff700 1 --2- 172.21.15.1:0/756539768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f816c1014b0 0x7f816c196300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.357 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.352+0000 7f816bfff700 1 -- 172.21.15.1:0/756539768 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f815c0095f0 con 0x7f816c10a880 2023-11-16T16:47:21.358 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.352+0000 7f816b7fe700 1 --2- 172.21.15.1:0/756539768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f816c1014b0 0x7f816c196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:47:21.358 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.352+0000 7f816bfff700 1 --2- 172.21.15.1:0/756539768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f816c10a880 0x7f816c19aca0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f816400efc0 tx=0x7f816400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:21.358 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.352+0000 7f8168ff9700 1 -- 172.21.15.1:0/756539768 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8164018070 con 0x7f816c10a880 2023-11-16T16:47:21.358 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.352+0000 7f8171c75700 1 -- 172.21.15.1:0/756539768 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f816c1102f0 con 0x7f816c10a880 2023-11-16T16:47:21.358 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.352+0000 7f8168ff9700 1 -- 172.21.15.1:0/756539768 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f816400f040 con 0x7f816c10a880 2023-11-16T16:47:21.358 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.352+0000 7f8168ff9700 1 -- 172.21.15.1:0/756539768 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816400ccc0 con 0x7f816c10a880 2023-11-16T16:47:21.358 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.352+0000 7f8171c75700 1 -- 172.21.15.1:0/756539768 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f816c110160 con 0x7f816c10a880 2023-11-16T16:47:21.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.354+0000 7f8168ff9700 1 -- 172.21.15.1:0/756539768 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f8164003a00 con 0x7f816c10a880 2023-11-16T16:47:21.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.355+0000 7f8168ff9700 1 --2- 172.21.15.1:0/756539768 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f81580a12c0 0x7f81580a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:21.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.355+0000 7f8168ff9700 1 -- 172.21.15.1:0/756539768 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f81640cc800 con 0x7f816c10a880 2023-11-16T16:47:21.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.355+0000 7f816b7fe700 1 --2- 172.21.15.1:0/756539768 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f81580a12c0 0x7f81580a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:21.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.356+0000 7f816b7fe700 1 --2- 172.21.15.1:0/756539768 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f81580a12c0 0x7f81580a3770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f815c00b2e0 tx=0x7f815c005960 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:21.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.356+0000 7f817de01b80 1 Processor -- start 2023-11-16T16:47:21.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.356+0000 7f817de01b80 1 -- start start 2023-11-16T16:47:21.365 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.356+0000 7f817de01b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562356d9ff0 0x5562356da3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:21.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.356+0000 7f817de01b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562356da900 0x5562356de800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:21.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.356+0000 7f817de01b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562356ded40 0x5562356df190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:21.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.356+0000 7f817de01b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562355db0e0 con 0x5562356d9ff0 2023-11-16T16:47:21.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.356+0000 7f817de01b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55623562d920 con 0x5562356ded40 2023-11-16T16:47:21.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.356+0000 7f817de01b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556235626db0 con 0x5562356da900 2023-11-16T16:47:21.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.356+0000 7f816b7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562356d9ff0 0x5562356da3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:21.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.356+0000 7f816b7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562356d9ff0 0x5562356da3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46328/0 (socket says 172.21.15.1:46328) 2023-11-16T16:47:21.366 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.357+0000 7f816b7fe700 1 -- 172.21.15.1:0/2332945915 learned_addr learned my addr 172.21.15.1:0/2332945915 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:21.367 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.357+0000 7f816bfff700 1 --2- 172.21.15.1:0/2332945915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562356ded40 0x5562356df190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:21.367 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.357+0000 7f816affd700 1 --2- 172.21.15.1:0/2332945915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562356da900 0x5562356de800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:21.367 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.357+0000 7f816b7fe700 1 -- 172.21.15.1:0/2332945915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562356da900 msgr2=0x5562356de800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:21.367 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.357+0000 7f816b7fe700 1 --2- 172.21.15.1:0/2332945915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562356da900 0x5562356de800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.367 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.357+0000 7f816b7fe700 1 -- 172.21.15.1:0/2332945915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562356ded40 msgr2=0x5562356df190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:21.367 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.357+0000 7f816b7fe700 1 --2- 172.21.15.1:0/2332945915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562356ded40 0x5562356df190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.367 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.357+0000 7f816b7fe700 1 -- 172.21.15.1:0/2332945915 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562354a86b0 con 0x5562356d9ff0 2023-11-16T16:47:21.367 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.357+0000 7f816b7fe700 1 --2- 172.21.15.1:0/2332945915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562356d9ff0 0x5562356da3c0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f815c00b6c0 tx=0x7f815c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137f88e4e5eb16d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:21.368 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.357+0000 7f8161ffb700 1 -- 172.21.15.1:0/2332945915 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815c004b60 con 0x5562356d9ff0 2023-11-16T16:47:21.368 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.357+0000 7f8161ffb700 1 -- 172.21.15.1:0/2332945915 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f815c01e040 con 0x5562356d9ff0 2023-11-16T16:47:21.368 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.357+0000 7f8161ffb700 1 -- 172.21.15.1:0/2332945915 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815c02f440 con 0x5562356d9ff0 2023-11-16T16:47:21.368 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.358+0000 7f817de01b80 1 -- 172.21.15.1:0/2332945915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562356d9ff0 msgr2=0x5562356da3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:21.368 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.358+0000 7f817de01b80 1 --2- 172.21.15.1:0/2332945915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562356d9ff0 0x5562356da3c0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f815c00b6c0 tx=0x7f815c020000 comp rx=0 tx=0).stop 2023-11-16T16:47:21.368 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.358+0000 7f817de01b80 1 -- 172.21.15.1:0/2332945915 shutdown_connections 2023-11-16T16:47:21.368 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.358+0000 7f817de01b80 1 --2- 172.21.15.1:0/2332945915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562356ded40 0x5562356df190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.368 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.358+0000 7f817de01b80 1 --2- 172.21.15.1:0/2332945915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562356d9ff0 0x5562356da3c0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.368 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.358+0000 7f817de01b80 1 --2- 172.21.15.1:0/2332945915 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562356da900 0x5562356de800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.369 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.358+0000 7f817de01b80 1 -- 172.21.15.1:0/2332945915 >> 172.21.15.1:0/2332945915 conn(0x5562356cb3e0 msgr2=0x5562356d0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:21.369 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.358+0000 7f817de01b80 1 -- 172.21.15.1:0/2332945915 shutdown_connections 2023-11-16T16:47:21.369 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.358+0000 7f817de01b80 1 -- 172.21.15.1:0/2332945915 wait complete. 2023-11-16T16:47:21.369 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.358+0000 7f817de01b80 1 -- start start 2023-11-16T16:47:21.369 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.359+0000 7f817de01b80 1 Processor -- start 2023-11-16T16:47:21.369 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.359+0000 7f817de01b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562356d9ff0 0x5562356e64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:21.369 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.359+0000 7f817de01b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562356da900 0x5562356e3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:21.370 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.359+0000 7f817de01b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562356ded40 0x5562356e3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:21.370 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.359+0000 7f817de01b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562355c8960 con 0x5562356da900 2023-11-16T16:47:21.370 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.359+0000 7f817de01b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562355c8600 con 0x5562356d9ff0 2023-11-16T16:47:21.370 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.359+0000 7f817de01b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562354aa9a0 con 0x5562356ded40 2023-11-16T16:47:21.370 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.359+0000 7f817de01b80 10 client.0 _ll_register_callbacks cb 0x5562355e6800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:47:21.370 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.359+0000 7f816affd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562356da900 0x5562356e3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:21.370 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.359+0000 7f816affd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562356da900 0x5562356e3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46336/0 (socket says 172.21.15.1:46336) 2023-11-16T16:47:21.370 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.359+0000 7f816affd700 1 -- 172.21.15.1:0/3367799059 learned_addr learned my addr 172.21.15.1:0/3367799059 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:21.371 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.359+0000 7f816bfff700 1 --2- 172.21.15.1:0/3367799059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562356ded40 0x5562356e3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:21.371 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.359+0000 7f816b7fe700 1 --2- 172.21.15.1:0/3367799059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562356d9ff0 0x5562356e64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:21.371 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.359+0000 7f816affd700 1 -- 172.21.15.1:0/3367799059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562356ded40 msgr2=0x5562356e3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:21.371 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.359+0000 7f816affd700 1 --2- 172.21.15.1:0/3367799059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562356ded40 0x5562356e3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.371 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.359+0000 7f816affd700 1 -- 172.21.15.1:0/3367799059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562356d9ff0 msgr2=0x5562356e64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:21.371 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.359+0000 7f816affd700 1 --2- 172.21.15.1:0/3367799059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562356d9ff0 0x5562356e64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.371 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.359+0000 7f816affd700 1 -- 172.21.15.1:0/3367799059 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f816c10d9d0 con 0x5562356da900 2023-11-16T16:47:21.371 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.359+0000 7f816affd700 1 --2- 172.21.15.1:0/3367799059 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562356da900 0x5562356e3720 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f8154011a30 tx=0x7f8154011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=428cd391ccf9e76b server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:21.372 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.359+0000 7f814affd700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:47:21.372 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.360+0000 7f814affd700 1 -- 172.21.15.1:0/3367799059 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8154008830 con 0x5562356da900 2023-11-16T16:47:21.372 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.360+0000 7f814affd700 1 -- 172.21.15.1:0/3367799059 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f815400fcc0 con 0x5562356da900 2023-11-16T16:47:21.372 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.360+0000 7f817de01b80 1 -- 172.21.15.1:0/3367799059 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5562356c8ba0 con 0x5562356da900 2023-11-16T16:47:21.372 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.360+0000 7f814affd700 1 -- 172.21.15.1:0/3367799059 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f815400fe80 con 0x5562356da900 2023-11-16T16:47:21.372 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.360+0000 7f817de01b80 10 client.16626 fetch_fsmap learned FSMap version 28 2023-11-16T16:47:21.372 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.360+0000 7f817de01b80 1 -- 172.21.15.1:0/3367799059 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5562355ed210 con 0x5562356da900 2023-11-16T16:47:21.372 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.360+0000 7f814affd700 1 -- 172.21.15.1:0/3367799059 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5562356c8ba0 con 0x5562356da900 2023-11-16T16:47:21.372 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.360+0000 7f814affd700 1 -- 172.21.15.1:0/3367799059 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f815401a440 con 0x5562356da900 2023-11-16T16:47:21.373 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.360+0000 7f817de01b80 10 client.16626 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:47:21.373 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.360+0000 7f817de01b80 10 client.16626 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:47:21.373 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.360+0000 7f817de01b80 20 client.16626 populate_metadata read hostname 'smithi001' 2023-11-16T16:47:21.373 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.360+0000 7f817de01b80 1 --2- 172.21.15.1:0/3367799059 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5562356f3d70 0x5562356f6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:21.373 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.360+0000 7f817de01b80 4 client.16626 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:47:21.373 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.360+0000 7f817de01b80 1 -- 172.21.15.1:0/3367799059 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5562355e9120 con 0x5562356f3d70 2023-11-16T16:47:21.373 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.360+0000 7f816b7fe700 1 --2- 172.21.15.1:0/3367799059 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5562356f3d70 0x5562356f6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:21.373 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.361+0000 7f816b7fe700 1 --2- 172.21.15.1:0/3367799059 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5562356f3d70 0x5562356f6180 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31df5ddeb79ab445 server_cookie=87fbdba23033b9db in_seq=0 out_seq=0 2023-11-16T16:47:21.374 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.361+0000 7f814affd700 10 client.16626 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:47:21.374 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.362+0000 7f814affd700 1 -- 172.21.15.1:0/3367799059 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5562355e9120 con 0x5562356f3d70 2023-11-16T16:47:21.374 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.362+0000 7f814affd700 10 client.16626 handle_command_reply: tid=0 2023-11-16T16:47:21.374 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.362+0000 7f817de01b80 1 client.16626 shutdown 2023-11-16T16:47:21.374 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.362+0000 7f817de01b80 1 -- 172.21.15.1:0/3367799059 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562356da900 msgr2=0x5562356e3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:21.374 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.362+0000 7f817de01b80 1 --2- 172.21.15.1:0/3367799059 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562356da900 0x5562356e3720 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f8154011a30 tx=0x7f8154011df0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.374 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.362+0000 7f817de01b80 1 -- 172.21.15.1:0/3367799059 shutdown_connections 2023-11-16T16:47:21.374 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.362+0000 7f817de01b80 1 --2- 172.21.15.1:0/3367799059 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5562356f3d70 0x5562356f6180 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.375 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.362+0000 7f817de01b80 1 --2- 172.21.15.1:0/3367799059 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562356ded40 0x5562356e3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.375 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.362+0000 7f814affd700 0 client.16626 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:47:21.375 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.362+0000 7f817de01b80 1 --2- 172.21.15.1:0/3367799059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562356d9ff0 0x5562356e64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.375 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.362+0000 7f817de01b80 1 --2- 172.21.15.1:0/3367799059 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562356da900 0x5562356e3720 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.375 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.362+0000 7f817de01b80 1 -- 172.21.15.1:0/3367799059 >> 172.21.15.1:0/3367799059 conn(0x5562356cb3e0 msgr2=0x5562356e01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:21.375 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.362+0000 7f817de01b80 1 -- 172.21.15.1:0/3367799059 shutdown_connections 2023-11-16T16:47:21.375 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.362+0000 7f817de01b80 1 -- 172.21.15.1:0/3367799059 wait complete. 2023-11-16T16:47:21.375 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.362+0000 7f817de01b80 20 client.16626 trim_cache size 0 max 16384 2023-11-16T16:47:21.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.379+0000 7f817de01b80 1 Processor -- start 2023-11-16T16:47:21.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.379+0000 7f817de01b80 1 -- start start 2023-11-16T16:47:21.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.379+0000 7f817de01b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562356d9ff0 0x5562356e1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:21.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.379+0000 7f817de01b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562356da900 0x5562356eadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:21.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.379+0000 7f817de01b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562356ded40 0x5562356e0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:21.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.379+0000 7f817de01b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562356dd920 con 0x5562356da900 2023-11-16T16:47:21.382 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.379+0000 7f817de01b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562356c8a00 con 0x5562356d9ff0 2023-11-16T16:47:21.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.379+0000 7f817de01b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562356e43a0 con 0x5562356ded40 2023-11-16T16:47:21.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.379+0000 7f816affd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562356da900 0x5562356eadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:21.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.379+0000 7f816affd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562356da900 0x5562356eadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46344/0 (socket says 172.21.15.1:46344) 2023-11-16T16:47:21.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.379+0000 7f816affd700 1 -- 172.21.15.1:0/40763835 learned_addr learned my addr 172.21.15.1:0/40763835 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:21.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.379+0000 7f816bfff700 1 --2- 172.21.15.1:0/40763835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562356ded40 0x5562356e0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:21.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.379+0000 7f816b7fe700 1 --2- 172.21.15.1:0/40763835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562356d9ff0 0x5562356e1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:21.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.380+0000 7f816affd700 1 -- 172.21.15.1:0/40763835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562356ded40 msgr2=0x5562356e0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:21.383 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.380+0000 7f816affd700 1 --2- 172.21.15.1:0/40763835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562356ded40 0x5562356e0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.384 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.380+0000 7f816affd700 1 -- 172.21.15.1:0/40763835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562356d9ff0 msgr2=0x5562356e1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:21.384 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.380+0000 7f816affd700 1 --2- 172.21.15.1:0/40763835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562356d9ff0 0x5562356e1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.384 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.380+0000 7f816affd700 1 -- 172.21.15.1:0/40763835 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562355ecf60 con 0x5562356da900 2023-11-16T16:47:21.384 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.380+0000 7f816affd700 1 --2- 172.21.15.1:0/40763835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562356da900 0x5562356eadd0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f8154009010 tx=0x7f8154005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=103844dd7b8192d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:21.384 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.380+0000 7f81627fc700 1 -- 172.21.15.1:0/40763835 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81540193e0 con 0x5562356da900 2023-11-16T16:47:21.384 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.380+0000 7f81627fc700 1 -- 172.21.15.1:0/40763835 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8154019540 con 0x5562356da900 2023-11-16T16:47:21.384 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.380+0000 7f81627fc700 1 -- 172.21.15.1:0/40763835 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81540199b0 con 0x5562356da900 2023-11-16T16:47:21.388 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.381+0000 7f817de01b80 1 -- 172.21.15.1:0/40763835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562356da900 msgr2=0x5562356eadd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:21.388 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.381+0000 7f817de01b80 1 --2- 172.21.15.1:0/40763835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562356da900 0x5562356eadd0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f8154009010 tx=0x7f8154005df0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.388 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.381+0000 7f817de01b80 1 -- 172.21.15.1:0/40763835 shutdown_connections 2023-11-16T16:47:21.389 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.381+0000 7f817de01b80 1 --2- 172.21.15.1:0/40763835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562356ded40 0x5562356e0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.389 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.381+0000 7f817de01b80 1 --2- 172.21.15.1:0/40763835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562356d9ff0 0x5562356e1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.389 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.381+0000 7f817de01b80 1 --2- 172.21.15.1:0/40763835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562356da900 0x5562356eadd0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.389 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.381+0000 7f817de01b80 1 -- 172.21.15.1:0/40763835 >> 172.21.15.1:0/40763835 conn(0x5562356cb3e0 msgr2=0x5562356ddcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:21.389 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.381+0000 7f817de01b80 1 -- 172.21.15.1:0/40763835 shutdown_connections 2023-11-16T16:47:21.389 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.381+0000 7f817de01b80 1 -- 172.21.15.1:0/40763835 wait complete. 2023-11-16T16:47:21.389 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.381+0000 7f817de01b80 1 -- start start 2023-11-16T16:47:21.390 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.381+0000 7f817de01b80 1 Processor -- start 2023-11-16T16:47:21.390 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.381+0000 7f817de01b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562356d9ff0 0x5562356e2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:21.390 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.381+0000 7f817de01b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562356da900 0x5562356e2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:21.390 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.381+0000 7f817de01b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562356ded40 0x556235747c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:21.390 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.381+0000 7f817de01b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562356e36d0 con 0x5562356d9ff0 2023-11-16T16:47:21.390 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.381+0000 7f817de01b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556235748190 con 0x5562356ded40 2023-11-16T16:47:21.390 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.381+0000 7f817de01b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x556235748300 con 0x5562356da900 2023-11-16T16:47:21.390 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.381+0000 7f817de01b80 10 client.0 _ll_register_callbacks cb 0x7f8154008d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:47:21.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.381+0000 7f816b7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562356d9ff0 0x5562356e2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:21.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.382+0000 7f816b7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562356d9ff0 0x5562356e2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46348/0 (socket says 172.21.15.1:46348) 2023-11-16T16:47:21.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.382+0000 7f816b7fe700 1 -- 172.21.15.1:0/4130630793 learned_addr learned my addr 172.21.15.1:0/4130630793 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:21.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.382+0000 7f816affd700 1 --2- 172.21.15.1:0/4130630793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562356da900 0x5562356e2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:21.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.382+0000 7f816b7fe700 1 -- 172.21.15.1:0/4130630793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562356da900 msgr2=0x5562356e2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:21.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.382+0000 7f816b7fe700 1 --2- 172.21.15.1:0/4130630793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562356da900 0x5562356e2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.382+0000 7f816bfff700 1 --2- 172.21.15.1:0/4130630793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562356ded40 0x556235747c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:21.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.382+0000 7f816b7fe700 1 -- 172.21.15.1:0/4130630793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562356ded40 msgr2=0x556235747c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:21.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.382+0000 7f816b7fe700 1 --2- 172.21.15.1:0/4130630793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562356ded40 0x556235747c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.382+0000 7f816b7fe700 1 -- 172.21.15.1:0/4130630793 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5562355ed210 con 0x5562356d9ff0 2023-11-16T16:47:21.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.382+0000 7f816b7fe700 1 --2- 172.21.15.1:0/4130630793 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562356d9ff0 0x5562356e2a80 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f815c009ce0 tx=0x7f815c009330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b8ed866100e09f server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:21.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.382+0000 7f814affd700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:47:21.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.382+0000 7f814affd700 1 -- 172.21.15.1:0/4130630793 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815c029040 con 0x5562356d9ff0 2023-11-16T16:47:21.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.382+0000 7f814affd700 1 -- 172.21.15.1:0/4130630793 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f815c03dd20 con 0x5562356d9ff0 2023-11-16T16:47:21.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.382+0000 7f817de01b80 1 -- 172.21.15.1:0/4130630793 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f815c01e040 con 0x5562356d9ff0 2023-11-16T16:47:21.393 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.382+0000 7f814affd700 1 -- 172.21.15.1:0/4130630793 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f815c00f460 con 0x5562356d9ff0 2023-11-16T16:47:21.393 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.382+0000 7f817de01b80 10 client.16632 fetch_fsmap learned FSMap version 28 2023-11-16T16:47:21.393 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.382+0000 7f817de01b80 1 -- 172.21.15.1:0/4130630793 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55623543dce0 con 0x5562356d9ff0 2023-11-16T16:47:21.393 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.383+0000 7f814affd700 1 -- 172.21.15.1:0/4130630793 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f815c01e040 con 0x5562356d9ff0 2023-11-16T16:47:21.393 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.383+0000 7f814affd700 1 -- 172.21.15.1:0/4130630793 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f815c004a70 con 0x5562356d9ff0 2023-11-16T16:47:21.393 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.383+0000 7f817de01b80 10 client.16632 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:47:21.393 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.383+0000 7f817de01b80 10 client.16632 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:47:21.393 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.383+0000 7f817de01b80 20 client.16632 populate_metadata read hostname 'smithi001' 2023-11-16T16:47:21.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.383+0000 7f817de01b80 1 --2- 172.21.15.1:0/4130630793 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5562356f3d70 0x55623574b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:21.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.383+0000 7f817de01b80 4 client.16632 mds_command: new command op to 24521 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-16T16:47:21.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.383+0000 7f817de01b80 1 -- 172.21.15.1:0/4130630793 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5562355f56e0 con 0x5562356f3d70 2023-11-16T16:47:21.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.383+0000 7f816affd700 1 --2- 172.21.15.1:0/4130630793 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5562356f3d70 0x55623574b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:21.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.383+0000 7f814affd700 10 client.16632 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:47:21.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.383+0000 7f816affd700 1 --2- 172.21.15.1:0/4130630793 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5562356f3d70 0x55623574b680 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64efdc1a6af89d4b server_cookie=b61ee7b5f583e49c in_seq=0 out_seq=0 2023-11-16T16:47:21.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.385+0000 7f814affd700 1 -- 172.21.15.1:0/4130630793 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5562355f56e0 con 0x5562356f3d70 2023-11-16T16:47:21.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.385+0000 7f814affd700 10 client.16632 handle_command_reply: tid=0 2023-11-16T16:47:21.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.385+0000 7f817de01b80 1 client.16632 shutdown 2023-11-16T16:47:21.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.385+0000 7f817de01b80 1 -- 172.21.15.1:0/4130630793 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562356d9ff0 msgr2=0x5562356e2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:21.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.385+0000 7f817de01b80 1 --2- 172.21.15.1:0/4130630793 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562356d9ff0 0x5562356e2a80 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f815c009ce0 tx=0x7f815c009330 comp rx=0 tx=0).stop 2023-11-16T16:47:21.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.386+0000 7f817de01b80 1 -- 172.21.15.1:0/4130630793 shutdown_connections 2023-11-16T16:47:21.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.386+0000 7f817de01b80 1 --2- 172.21.15.1:0/4130630793 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5562356f3d70 0x55623574b680 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.386+0000 7f817de01b80 1 --2- 172.21.15.1:0/4130630793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5562356ded40 0x556235747c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.386+0000 7f817de01b80 1 --2- 172.21.15.1:0/4130630793 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5562356d9ff0 0x5562356e2a80 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.395 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.386+0000 7f817de01b80 1 --2- 172.21.15.1:0/4130630793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5562356da900 0x5562356e2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.386+0000 7f814affd700 0 client.16632 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:47:21.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.386+0000 7f817de01b80 1 -- 172.21.15.1:0/4130630793 >> 172.21.15.1:0/4130630793 conn(0x5562356cb3e0 msgr2=0x5562356d49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:21.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.386+0000 7f817de01b80 1 -- 172.21.15.1:0/4130630793 shutdown_connections 2023-11-16T16:47:21.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.386+0000 7f817de01b80 1 -- 172.21.15.1:0/4130630793 wait complete. 2023-11-16T16:47:21.396 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.386+0000 7f817de01b80 20 client.16632 trim_cache size 0 max 16384 2023-11-16T16:47:21.396 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:47:21.397 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2023-11-16T16:47:21.397 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "8e0fac62-c345-4b7b-8d07-d1ddd839adb3", 2023-11-16T16:47:21.397 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2023-11-16T16:47:21.397 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:47:21.398 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.386+0000 7f8171c75700 1 -- 172.21.15.1:0/756539768 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f81580a12c0 msgr2=0x7f81580a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:21.398 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.386+0000 7f8171c75700 1 --2- 172.21.15.1:0/756539768 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f81580a12c0 0x7f81580a3770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f815c00b2e0 tx=0x7f815c005960 comp rx=0 tx=0).stop 2023-11-16T16:47:21.398 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.386+0000 7f8171c75700 1 -- 172.21.15.1:0/756539768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f816c10a880 msgr2=0x7f816c19aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:21.398 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.386+0000 7f8171c75700 1 --2- 172.21.15.1:0/756539768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f816c10a880 0x7f816c19aca0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f816400efc0 tx=0x7f816400c3a0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.398 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.386+0000 7f8171c75700 1 -- 172.21.15.1:0/756539768 shutdown_connections 2023-11-16T16:47:21.399 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.386+0000 7f8171c75700 1 --2- 172.21.15.1:0/756539768 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f81580a12c0 0x7f81580a3770 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.399 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.386+0000 7f8171c75700 1 --2- 172.21.15.1:0/756539768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f816c10a880 0x7f816c19aca0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.399 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.386+0000 7f8171c75700 1 --2- 172.21.15.1:0/756539768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f816c1014b0 0x7f816c196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.399 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.386+0000 7f8171c75700 1 --2- 172.21.15.1:0/756539768 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f816c101dc0 0x7f816c196840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.399 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.386+0000 7f8171c75700 1 -- 172.21.15.1:0/756539768 >> 172.21.15.1:0/756539768 conn(0x7f816c0763a0 msgr2=0x7f816c102250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:21.399 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.386+0000 7f8171c75700 1 -- 172.21.15.1:0/756539768 shutdown_connections 2023-11-16T16:47:21.399 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.386+0000 7f8171c75700 1 -- 172.21.15.1:0/756539768 wait complete. 2023-11-16T16:47:21.400 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-16T16:47:21.529 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.527+0000 7fcef68dd700 1 -- 172.21.15.1:0/417286069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef01055d0 msgr2=0x7fcef010d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:21.529 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.527+0000 7fcef68dd700 1 --2- 172.21.15.1:0/417286069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef01055d0 0x7fcef010d150 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fcee800b0d0 tx=0x7fcee800b3e0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.529 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.527+0000 7fcef68dd700 1 -- 172.21.15.1:0/417286069 shutdown_connections 2023-11-16T16:47:21.529 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.527+0000 7fcef68dd700 1 --2- 172.21.15.1:0/417286069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef01055d0 0x7fcef010d150 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.529 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.527+0000 7fcef68dd700 1 --2- 172.21.15.1:0/417286069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcef0068030 0x7fcef0068400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.529 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.527+0000 7fcef68dd700 1 --2- 172.21.15.1:0/417286069 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcef00689d0 0x7fcef0104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.530 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.527+0000 7fcef68dd700 1 -- 172.21.15.1:0/417286069 >> 172.21.15.1:0/417286069 conn(0x7fcef00763a0 msgr2=0x7fcef00767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:21.530 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.527+0000 7fcef68dd700 1 -- 172.21.15.1:0/417286069 shutdown_connections 2023-11-16T16:47:21.530 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.527+0000 7fcef68dd700 1 -- 172.21.15.1:0/417286069 wait complete. 2023-11-16T16:47:21.532 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.528+0000 7fcef68dd700 1 Processor -- start 2023-11-16T16:47:21.532 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.528+0000 7fcef68dd700 1 -- start start 2023-11-16T16:47:21.532 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.528+0000 7fcef68dd700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcef0068030 0x7fcef0196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:21.532 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.528+0000 7fcef68dd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef00689d0 0x7fcef0196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:21.532 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.528+0000 7fcef68dd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcef01055d0 0x7fcef019aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:21.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.528+0000 7fcef68dd700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef019b500 con 0x7fcef00689d0 2023-11-16T16:47:21.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.528+0000 7fcef68dd700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef019b670 con 0x7fcef01055d0 2023-11-16T16:47:21.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.528+0000 7fcef68dd700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef019b830 con 0x7fcef0068030 2023-11-16T16:47:21.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.529+0000 7fceef7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef00689d0 0x7fcef0196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:21.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.529+0000 7fceef7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef00689d0 0x7fcef0196a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46364/0 (socket says 172.21.15.1:46364) 2023-11-16T16:47:21.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.529+0000 7fceef7fe700 1 -- 172.21.15.1:0/1843912248 learned_addr learned my addr 172.21.15.1:0/1843912248 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:21.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.529+0000 7fceeffff700 1 --2- 172.21.15.1:0/1843912248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcef0068030 0x7fcef0196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:21.533 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.529+0000 7fcef4e7a700 1 --2- 172.21.15.1:0/1843912248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcef01055d0 0x7fcef019aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:21.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.529+0000 7fceef7fe700 1 -- 172.21.15.1:0/1843912248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcef0068030 msgr2=0x7fcef0196520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:21.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.529+0000 7fceef7fe700 1 --2- 172.21.15.1:0/1843912248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcef0068030 0x7fcef0196520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.529+0000 7fceef7fe700 1 -- 172.21.15.1:0/1843912248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcef01055d0 msgr2=0x7fcef019aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:21.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.529+0000 7fceef7fe700 1 --2- 172.21.15.1:0/1843912248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcef01055d0 0x7fcef019aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.529+0000 7fceef7fe700 1 -- 172.21.15.1:0/1843912248 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee8009d00 con 0x7fcef00689d0 2023-11-16T16:47:21.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.529+0000 7fceef7fe700 1 --2- 172.21.15.1:0/1843912248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef00689d0 0x7fcef0196a60 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fced800e910 tx=0x7fced800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:21.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.529+0000 7fceed7fa700 1 -- 172.21.15.1:0/1843912248 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced8004a30 con 0x7fcef00689d0 2023-11-16T16:47:21.534 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.529+0000 7fcef68dd700 1 -- 172.21.15.1:0/1843912248 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef01103a0 con 0x7fcef00689d0 2023-11-16T16:47:21.535 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.529+0000 7fceed7fa700 1 -- 172.21.15.1:0/1843912248 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fced80054e0 con 0x7fcef00689d0 2023-11-16T16:47:21.535 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.529+0000 7fceed7fa700 1 -- 172.21.15.1:0/1843912248 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced800cc40 con 0x7fcef00689d0 2023-11-16T16:47:21.535 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.529+0000 7fcef68dd700 1 -- 172.21.15.1:0/1843912248 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef0110210 con 0x7fcef00689d0 2023-11-16T16:47:21.540 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.532+0000 7fceed7fa700 1 -- 172.21.15.1:0/1843912248 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fced8005000 con 0x7fcef00689d0 2023-11-16T16:47:21.540 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.533+0000 7fceed7fa700 1 --2- 172.21.15.1:0/1843912248 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fcedc0a12c0 0x7fcedc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:21.540 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.533+0000 7fceed7fa700 1 -- 172.21.15.1:0/1843912248 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fced8014070 con 0x7fcef00689d0 2023-11-16T16:47:21.541 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.533+0000 7fceeffff700 1 --2- 172.21.15.1:0/1843912248 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fcedc0a12c0 0x7fcedc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:21.541 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.533+0000 7fcf02a69b80 1 Processor -- start 2023-11-16T16:47:21.541 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.533+0000 7fceeffff700 1 --2- 172.21.15.1:0/1843912248 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fcedc0a12c0 0x7fcedc0a3770 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcee0009c60 tx=0x7fcee0009260 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:21.541 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.533+0000 7fcf02a69b80 1 -- start start 2023-11-16T16:47:21.541 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.534+0000 7fcf02a69b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b65acb1b60 0x55b65acb1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:21.541 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.534+0000 7fcf02a69b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b65acb2470 0x55b65acb6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:21.541 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.534+0000 7fcf02a69b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b65acb68b0 0x55b65acb6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:21.541 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.534+0000 7fcf02a69b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65abb7060 con 0x55b65acb2470 2023-11-16T16:47:21.542 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.534+0000 7fcf02a69b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65ac098a0 con 0x55b65acb1b60 2023-11-16T16:47:21.542 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.534+0000 7fcf02a69b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65ac02d30 con 0x55b65acb68b0 2023-11-16T16:47:21.542 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.534+0000 7fceef7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b65acb2470 0x55b65acb6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:21.542 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.534+0000 7fceef7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b65acb2470 0x55b65acb6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46372/0 (socket says 172.21.15.1:46372) 2023-11-16T16:47:21.542 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.534+0000 7fceef7fe700 1 -- 172.21.15.1:0/3068391625 learned_addr learned my addr 172.21.15.1:0/3068391625 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:21.542 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.534+0000 7fcef4e7a700 1 --2- 172.21.15.1:0/3068391625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b65acb68b0 0x55b65acb6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:21.542 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.534+0000 7fceeffff700 1 --2- 172.21.15.1:0/3068391625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b65acb1b60 0x55b65acb1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:21.542 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.534+0000 7fceef7fe700 1 -- 172.21.15.1:0/3068391625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b65acb68b0 msgr2=0x55b65acb6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:21.543 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.534+0000 7fceef7fe700 1 --2- 172.21.15.1:0/3068391625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b65acb68b0 0x55b65acb6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.543 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.534+0000 7fceef7fe700 1 -- 172.21.15.1:0/3068391625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b65acb1b60 msgr2=0x55b65acb1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:21.543 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.534+0000 7fceef7fe700 1 --2- 172.21.15.1:0/3068391625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b65acb1b60 0x55b65acb1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.543 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.534+0000 7fceef7fe700 1 -- 172.21.15.1:0/3068391625 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b65aa84630 con 0x55b65acb2470 2023-11-16T16:47:21.543 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.534+0000 7fceef7fe700 1 --2- 172.21.15.1:0/3068391625 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b65acb2470 0x55b65acb6370 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fced80a1f40 tx=0x7fced8099a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba0760438c9ecfc server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:21.543 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.535+0000 7fcee67fc700 1 -- 172.21.15.1:0/3068391625 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced80b1070 con 0x55b65acb2470 2023-11-16T16:47:21.543 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.535+0000 7fcee67fc700 1 -- 172.21.15.1:0/3068391625 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fced809e050 con 0x55b65acb2470 2023-11-16T16:47:21.543 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.535+0000 7fcee67fc700 1 -- 172.21.15.1:0/3068391625 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced80981d0 con 0x55b65acb2470 2023-11-16T16:47:21.544 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.535+0000 7fcf02a69b80 1 -- 172.21.15.1:0/3068391625 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b65acb2470 msgr2=0x55b65acb6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:21.544 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.535+0000 7fcf02a69b80 1 --2- 172.21.15.1:0/3068391625 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b65acb2470 0x55b65acb6370 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fced80a1f40 tx=0x7fced8099a00 comp rx=0 tx=0).stop 2023-11-16T16:47:21.544 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.535+0000 7fcf02a69b80 1 -- 172.21.15.1:0/3068391625 shutdown_connections 2023-11-16T16:47:21.544 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.535+0000 7fcf02a69b80 1 --2- 172.21.15.1:0/3068391625 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b65acb68b0 0x55b65acb6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.544 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.535+0000 7fcf02a69b80 1 --2- 172.21.15.1:0/3068391625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b65acb1b60 0x55b65acb1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.544 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.535+0000 7fcf02a69b80 1 --2- 172.21.15.1:0/3068391625 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b65acb2470 0x55b65acb6370 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.544 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.535+0000 7fcf02a69b80 1 -- 172.21.15.1:0/3068391625 >> 172.21.15.1:0/3068391625 conn(0x55b65aca7360 msgr2=0x55b65acacb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:21.544 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.535+0000 7fcf02a69b80 1 -- 172.21.15.1:0/3068391625 shutdown_connections 2023-11-16T16:47:21.545 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.535+0000 7fcf02a69b80 1 -- 172.21.15.1:0/3068391625 wait complete. 2023-11-16T16:47:21.545 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.536+0000 7fcf02a69b80 1 -- start start 2023-11-16T16:47:21.545 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.536+0000 7fcf02a69b80 1 Processor -- start 2023-11-16T16:47:21.545 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.536+0000 7fcf02a69b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b65acb1b60 0x55b65acbe100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:21.545 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.536+0000 7fcf02a69b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b65acb2470 0x55b65acbb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:21.545 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.536+0000 7fcf02a69b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b65acb68b0 0x55b65acbbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:21.545 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.536+0000 7fcf02a69b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65aba48e0 con 0x55b65acb1b60 2023-11-16T16:47:21.545 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.536+0000 7fcf02a69b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65aba4580 con 0x55b65acb68b0 2023-11-16T16:47:21.546 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.536+0000 7fcf02a69b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65aa86920 con 0x55b65acb2470 2023-11-16T16:47:21.546 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.536+0000 7fcf02a69b80 10 client.0 _ll_register_callbacks cb 0x55b65abc2780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:47:21.546 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.536+0000 7fceeffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b65acb1b60 0x55b65acbe100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:21.546 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.536+0000 7fceeffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b65acb1b60 0x55b65acbe100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46376/0 (socket says 172.21.15.1:46376) 2023-11-16T16:47:21.546 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.536+0000 7fceeffff700 1 -- 172.21.15.1:0/3973257200 learned_addr learned my addr 172.21.15.1:0/3973257200 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:21.546 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.536+0000 7fcef4e7a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b65acb68b0 0x55b65acbbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:21.546 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.536+0000 7fceef7fe700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b65acb2470 0x55b65acbb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:21.546 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.537+0000 7fceeffff700 1 -- 172.21.15.1:0/3973257200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b65acb2470 msgr2=0x55b65acbb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:21.547 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.537+0000 7fceeffff700 1 --2- 172.21.15.1:0/3973257200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b65acb2470 0x55b65acbb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.547 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.537+0000 7fceeffff700 1 -- 172.21.15.1:0/3973257200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b65acb68b0 msgr2=0x55b65acbbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:21.547 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.537+0000 7fceeffff700 1 --2- 172.21.15.1:0/3973257200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b65acb68b0 0x55b65acbbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.547 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.537+0000 7fceeffff700 1 -- 172.21.15.1:0/3973257200 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef0110210 con 0x55b65acb1b60 2023-11-16T16:47:21.547 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.537+0000 7fceeffff700 1 --2- 172.21.15.1:0/3973257200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b65acb1b60 0x55b65acbe100 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fcee0005910 tx=0x7fcee00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7500645c0579d6da server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:21.547 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.537+0000 7fcecf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:47:21.547 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.537+0000 7fcecf7fe700 1 -- 172.21.15.1:0/3973257200 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee0022070 con 0x55b65acb1b60 2023-11-16T16:47:21.547 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.537+0000 7fcecf7fe700 1 -- 172.21.15.1:0/3973257200 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fcee001d620 con 0x55b65acb1b60 2023-11-16T16:47:21.548 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.537+0000 7fcf02a69b80 1 -- 172.21.15.1:0/3973257200 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b65aca4b20 con 0x55b65acb1b60 2023-11-16T16:47:21.548 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.537+0000 7fcecf7fe700 1 -- 172.21.15.1:0/3973257200 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcee001d7e0 con 0x55b65acb1b60 2023-11-16T16:47:21.548 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.537+0000 7fcf02a69b80 10 client.16644 fetch_fsmap learned FSMap version 28 2023-11-16T16:47:21.548 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.537+0000 7fcf02a69b80 1 -- 172.21.15.1:0/3973257200 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b65abc9190 con 0x55b65acb1b60 2023-11-16T16:47:21.548 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.537+0000 7fcecf7fe700 1 -- 172.21.15.1:0/3973257200 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcee0033080 con 0x55b65acb1b60 2023-11-16T16:47:21.548 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.538+0000 7fcecf7fe700 1 -- 172.21.15.1:0/3973257200 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fcee0027a70 con 0x55b65acb1b60 2023-11-16T16:47:21.548 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.538+0000 7fcf02a69b80 10 client.16644 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:47:21.548 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.538+0000 7fcf02a69b80 10 client.16644 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:47:21.549 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.538+0000 7fcf02a69b80 20 client.16644 populate_metadata read hostname 'smithi001' 2023-11-16T16:47:21.549 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.538+0000 7fcf02a69b80 1 --2- 172.21.15.1:0/3973257200 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55b65accba90 0x55b65accdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:21.549 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.538+0000 7fcf02a69b80 4 client.16644 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:47:21.549 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.538+0000 7fcf02a69b80 1 -- 172.21.15.1:0/3973257200 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b65abc50a0 con 0x55b65accba90 2023-11-16T16:47:21.549 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.538+0000 7fceef7fe700 1 --2- 172.21.15.1:0/3973257200 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55b65accba90 0x55b65accdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:21.549 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.538+0000 7fcecf7fe700 10 client.16644 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:47:21.549 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.538+0000 7fceef7fe700 1 --2- 172.21.15.1:0/3973257200 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55b65accba90 0x55b65accdea0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94d061b0263bb110 server_cookie=9375e88f76190ac8 in_seq=0 out_seq=0 2023-11-16T16:47:21.550 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.539+0000 7fcecf7fe700 1 -- 172.21.15.1:0/3973257200 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b65abc50a0 con 0x55b65accba90 2023-11-16T16:47:21.551 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.539+0000 7fcecf7fe700 10 client.16644 handle_command_reply: tid=0 2023-11-16T16:47:21.551 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.539+0000 7fcf02a69b80 1 client.16644 shutdown 2023-11-16T16:47:21.551 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.539+0000 7fcf02a69b80 1 -- 172.21.15.1:0/3973257200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b65acb1b60 msgr2=0x55b65acbe100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:21.551 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.539+0000 7fcf02a69b80 1 --2- 172.21.15.1:0/3973257200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b65acb1b60 0x55b65acbe100 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fcee0005910 tx=0x7fcee00047c0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.551 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.539+0000 7fcf02a69b80 1 -- 172.21.15.1:0/3973257200 shutdown_connections 2023-11-16T16:47:21.551 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.539+0000 7fcf02a69b80 1 --2- 172.21.15.1:0/3973257200 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55b65accba90 0x55b65accdea0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.551 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.539+0000 7fcf02a69b80 1 --2- 172.21.15.1:0/3973257200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b65acb68b0 0x55b65acbbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.551 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.539+0000 7fcf02a69b80 1 --2- 172.21.15.1:0/3973257200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b65acb1b60 0x55b65acbe100 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.552 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.539+0000 7fcf02a69b80 1 --2- 172.21.15.1:0/3973257200 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b65acb2470 0x55b65acbb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.552 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.539+0000 7fcecf7fe700 0 client.16644 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:47:21.552 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.539+0000 7fcf02a69b80 1 -- 172.21.15.1:0/3973257200 >> 172.21.15.1:0/3973257200 conn(0x55b65aca7360 msgr2=0x55b65acabde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:21.552 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.539+0000 7fcf02a69b80 1 -- 172.21.15.1:0/3973257200 shutdown_connections 2023-11-16T16:47:21.552 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.539+0000 7fcf02a69b80 1 -- 172.21.15.1:0/3973257200 wait complete. 2023-11-16T16:47:21.552 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.540+0000 7fcf02a69b80 20 client.16644 trim_cache size 0 max 16384 2023-11-16T16:47:21.560 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.557+0000 7fcf02a69b80 1 Processor -- start 2023-11-16T16:47:21.560 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.557+0000 7fcf02a69b80 1 -- start start 2023-11-16T16:47:21.560 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.557+0000 7fcf02a69b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b65acb1b60 0x55b65acc29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:21.561 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.557+0000 7fcf02a69b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b65acb2470 0x55b65acbdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:21.561 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.557+0000 7fcf02a69b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b65acb68b0 0x55b65acb8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:21.561 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.557+0000 7fcf02a69b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65acbc350 con 0x55b65acb68b0 2023-11-16T16:47:21.561 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.557+0000 7fcf02a69b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65aca6ee0 con 0x55b65acb1b60 2023-11-16T16:47:21.561 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.557+0000 7fcf02a69b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65acb2970 con 0x55b65acb2470 2023-11-16T16:47:21.561 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.557+0000 7fcef4e7a700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b65acb68b0 0x55b65acb8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:21.561 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.557+0000 7fcef4e7a700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b65acb68b0 0x55b65acb8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46388/0 (socket says 172.21.15.1:46388) 2023-11-16T16:47:21.561 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.557+0000 7fcef4e7a700 1 -- 172.21.15.1:0/1031862285 learned_addr learned my addr 172.21.15.1:0/1031862285 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:21.562 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.557+0000 7fceeffff700 1 --2- 172.21.15.1:0/1031862285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b65acb1b60 0x55b65acc29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:21.562 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.557+0000 7fceef7fe700 1 --2- 172.21.15.1:0/1031862285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b65acb2470 0x55b65acbdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:21.562 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.557+0000 7fcef4e7a700 1 -- 172.21.15.1:0/1031862285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b65acb2470 msgr2=0x55b65acbdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:21.562 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.557+0000 7fcef4e7a700 1 --2- 172.21.15.1:0/1031862285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b65acb2470 0x55b65acbdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.562 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.557+0000 7fcef4e7a700 1 -- 172.21.15.1:0/1031862285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b65acb1b60 msgr2=0x55b65acc29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:21.562 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.557+0000 7fcef4e7a700 1 --2- 172.21.15.1:0/1031862285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b65acb1b60 0x55b65acc29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.562 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.557+0000 7fcef4e7a700 1 -- 172.21.15.1:0/1031862285 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b65aca8b00 con 0x55b65acb68b0 2023-11-16T16:47:21.563 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.557+0000 7fceeffff700 1 --2- 172.21.15.1:0/1031862285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b65acb1b60 0x55b65acc29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:47:21.563 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.557+0000 7fceef7fe700 1 --2- 172.21.15.1:0/1031862285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b65acb2470 0x55b65acbdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:47:21.563 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.558+0000 7fcef4e7a700 1 --2- 172.21.15.1:0/1031862285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b65acb68b0 0x55b65acb8d00 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fcee80093f0 tx=0x7fcee8009420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b01e215854e8ab server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:21.563 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.558+0000 7fcee6ffd700 1 -- 172.21.15.1:0/1031862285 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee8021070 con 0x55b65acb68b0 2023-11-16T16:47:21.563 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.558+0000 7fcee6ffd700 1 -- 172.21.15.1:0/1031862285 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcee80074e0 con 0x55b65acb68b0 2023-11-16T16:47:21.563 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.558+0000 7fcee6ffd700 1 -- 172.21.15.1:0/1031862285 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee800e040 con 0x55b65acb68b0 2023-11-16T16:47:21.563 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.558+0000 7fcf02a69b80 1 -- 172.21.15.1:0/1031862285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b65acb68b0 msgr2=0x55b65acb8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:21.563 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.558+0000 7fcf02a69b80 1 --2- 172.21.15.1:0/1031862285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b65acb68b0 0x55b65acb8d00 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fcee80093f0 tx=0x7fcee8009420 comp rx=0 tx=0).stop 2023-11-16T16:47:21.564 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.558+0000 7fcf02a69b80 1 -- 172.21.15.1:0/1031862285 shutdown_connections 2023-11-16T16:47:21.564 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.558+0000 7fcf02a69b80 1 --2- 172.21.15.1:0/1031862285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b65acb68b0 0x55b65acb8d00 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.564 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.558+0000 7fcf02a69b80 1 --2- 172.21.15.1:0/1031862285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b65acb1b60 0x55b65acc29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.564 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.558+0000 7fcf02a69b80 1 --2- 172.21.15.1:0/1031862285 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b65acb2470 0x55b65acbdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.564 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.558+0000 7fcf02a69b80 1 -- 172.21.15.1:0/1031862285 >> 172.21.15.1:0/1031862285 conn(0x55b65aca7360 msgr2=0x55b65acb4970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:21.564 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.558+0000 7fcf02a69b80 1 -- 172.21.15.1:0/1031862285 shutdown_connections 2023-11-16T16:47:21.564 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.558+0000 7fcf02a69b80 1 -- 172.21.15.1:0/1031862285 wait complete. 2023-11-16T16:47:21.564 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.559+0000 7fcf02a69b80 1 -- start start 2023-11-16T16:47:21.565 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.559+0000 7fcf02a69b80 1 Processor -- start 2023-11-16T16:47:21.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.559+0000 7fcf02a69b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b65acb1b60 0x55b65acbaea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:21.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.559+0000 7fcf02a69b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b65acb2470 0x55b65acb9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:21.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.559+0000 7fcf02a69b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b65acb68b0 0x55b65acba090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:21.568 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.559+0000 7fcf02a69b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65acbb4c0 con 0x55b65acb1b60 2023-11-16T16:47:21.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.559+0000 7fcf02a69b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65acbb630 con 0x55b65acb2470 2023-11-16T16:47:21.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.559+0000 7fcf02a69b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65acba5d0 con 0x55b65acb68b0 2023-11-16T16:47:21.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.559+0000 7fcf02a69b80 10 client.0 _ll_register_callbacks cb 0x7fcee00044e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:47:21.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.559+0000 7fceeffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b65acb1b60 0x55b65acbaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:21.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.559+0000 7fceeffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b65acb1b60 0x55b65acbaea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46390/0 (socket says 172.21.15.1:46390) 2023-11-16T16:47:21.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.559+0000 7fceeffff700 1 -- 172.21.15.1:0/110621691 learned_addr learned my addr 172.21.15.1:0/110621691 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:21.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.559+0000 7fcef4e7a700 1 --2- 172.21.15.1:0/110621691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b65acb68b0 0x55b65acba090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:21.569 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.559+0000 7fceef7fe700 1 --2- 172.21.15.1:0/110621691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b65acb2470 0x55b65acb9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:21.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.559+0000 7fceeffff700 1 -- 172.21.15.1:0/110621691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b65acb68b0 msgr2=0x55b65acba090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:21.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.559+0000 7fceeffff700 1 --2- 172.21.15.1:0/110621691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b65acb68b0 0x55b65acba090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.559+0000 7fceeffff700 1 -- 172.21.15.1:0/110621691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b65acb2470 msgr2=0x55b65acb9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:21.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.559+0000 7fceeffff700 1 --2- 172.21.15.1:0/110621691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b65acb2470 0x55b65acb9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.559+0000 7fceeffff700 1 -- 172.21.15.1:0/110621691 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef010d990 con 0x55b65acb1b60 2023-11-16T16:47:21.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.559+0000 7fceef7fe700 1 --2- 172.21.15.1:0/110621691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b65acb2470 0x55b65acb9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:47:21.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.560+0000 7fceeffff700 1 --2- 172.21.15.1:0/110621691 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b65acb1b60 0x55b65acbaea0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fcee0003f40 tx=0x7fcee0027a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79c3cc8a64474ea server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:21.570 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.560+0000 7fcecf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:47:21.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.560+0000 7fcecf7fe700 1 -- 172.21.15.1:0/110621691 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee0022030 con 0x55b65acb1b60 2023-11-16T16:47:21.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.560+0000 7fcecf7fe700 1 -- 172.21.15.1:0/110621691 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fcee0023070 con 0x55b65acb1b60 2023-11-16T16:47:21.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.560+0000 7fcf02a69b80 1 -- 172.21.15.1:0/110621691 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fced809e050 con 0x55b65acb1b60 2023-11-16T16:47:21.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.560+0000 7fcecf7fe700 1 -- 172.21.15.1:0/110621691 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcee00092d0 con 0x55b65acb1b60 2023-11-16T16:47:21.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.560+0000 7fcf02a69b80 10 client.16650 fetch_fsmap learned FSMap version 28 2023-11-16T16:47:21.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.560+0000 7fcf02a69b80 1 -- 172.21.15.1:0/110621691 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b65aa30de0 con 0x55b65acb1b60 2023-11-16T16:47:21.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.560+0000 7fcecf7fe700 1 -- 172.21.15.1:0/110621691 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fced809e050 con 0x55b65acb1b60 2023-11-16T16:47:21.571 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.560+0000 7fcecf7fe700 1 -- 172.21.15.1:0/110621691 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fcee001d800 con 0x55b65acb1b60 2023-11-16T16:47:21.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.560+0000 7fcf02a69b80 10 client.16650 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:47:21.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.560+0000 7fcf02a69b80 10 client.16650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:47:21.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.560+0000 7fcf02a69b80 20 client.16650 populate_metadata read hostname 'smithi001' 2023-11-16T16:47:21.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.560+0000 7fcf02a69b80 1 --2- 172.21.15.1:0/110621691 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55b65ad20990 0x55b65ad22da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:21.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.560+0000 7fcf02a69b80 4 client.16650 mds_command: new command op to 24521 tid=0[{"prefix": "scrub status"}] 2023-11-16T16:47:21.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.560+0000 7fcf02a69b80 1 -- 172.21.15.1:0/110621691 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b65abd1660 con 0x55b65ad20990 2023-11-16T16:47:21.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.561+0000 7fceef7fe700 1 --2- 172.21.15.1:0/110621691 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55b65ad20990 0x55b65ad22da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:21.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.561+0000 7fcecf7fe700 10 client.16650 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:47:21.572 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.561+0000 7fceef7fe700 1 --2- 172.21.15.1:0/110621691 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55b65ad20990 0x55b65ad22da0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e09bf4d6bb6b7d6 server_cookie=164cb418b25532c1 in_seq=0 out_seq=0 2023-11-16T16:47:21.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.561+0000 7fcecf7fe700 1 -- 172.21.15.1:0/110621691 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b65abd1660 con 0x55b65ad20990 2023-11-16T16:47:21.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.561+0000 7fcecf7fe700 10 client.16650 handle_command_reply: tid=0 2023-11-16T16:47:21.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.561+0000 7fcf02a69b80 1 client.16650 shutdown 2023-11-16T16:47:21.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.561+0000 7fcf02a69b80 1 -- 172.21.15.1:0/110621691 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b65acb1b60 msgr2=0x55b65acbaea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:21.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.561+0000 7fcf02a69b80 1 --2- 172.21.15.1:0/110621691 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b65acb1b60 0x55b65acbaea0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fcee0003f40 tx=0x7fcee0027a40 comp rx=0 tx=0).stop 2023-11-16T16:47:21.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.561+0000 7fcf02a69b80 1 -- 172.21.15.1:0/110621691 shutdown_connections 2023-11-16T16:47:21.573 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.561+0000 7fcf02a69b80 1 --2- 172.21.15.1:0/110621691 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55b65ad20990 0x55b65ad22da0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.561+0000 7fcf02a69b80 1 --2- 172.21.15.1:0/110621691 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b65acb68b0 0x55b65acba090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.561+0000 7fcf02a69b80 1 --2- 172.21.15.1:0/110621691 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b65acb1b60 0x55b65acbaea0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.561+0000 7fcf02a69b80 1 --2- 172.21.15.1:0/110621691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b65acb2470 0x55b65acb9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.561+0000 7fcecf7fe700 0 client.16650 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:47:21.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.561+0000 7fcf02a69b80 1 -- 172.21.15.1:0/110621691 >> 172.21.15.1:0/110621691 conn(0x55b65aca7360 msgr2=0x55b65acb58c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:21.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.562+0000 7fcf02a69b80 1 -- 172.21.15.1:0/110621691 shutdown_connections 2023-11-16T16:47:21.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.562+0000 7fcf02a69b80 1 -- 172.21.15.1:0/110621691 wait complete. 2023-11-16T16:47:21.574 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.562+0000 7fcf02a69b80 20 client.16650 trim_cache size 0 max 16384 2023-11-16T16:47:21.575 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:47:21.575 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-16T16:47:21.575 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2023-11-16T16:47:21.575 INFO:teuthology.orchestra.run.smithi001.stdout: "8e0fac62-c345-4b7b-8d07-d1ddd839adb3": { 2023-11-16T16:47:21.575 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2023-11-16T16:47:21.575 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "8e0fac62-c345-4b7b-8d07-d1ddd839adb3", 2023-11-16T16:47:21.576 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2023-11-16T16:47:21.576 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:47:21.576 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:47:21.576 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:47:21.577 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.562+0000 7fcef68dd700 1 -- 172.21.15.1:0/1843912248 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fcedc0a12c0 msgr2=0x7fcedc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:21.577 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.562+0000 7fcef68dd700 1 --2- 172.21.15.1:0/1843912248 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fcedc0a12c0 0x7fcedc0a3770 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcee0009c60 tx=0x7fcee0009260 comp rx=0 tx=0).stop 2023-11-16T16:47:21.577 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.562+0000 7fcef68dd700 1 -- 172.21.15.1:0/1843912248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef00689d0 msgr2=0x7fcef0196a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:21.578 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.562+0000 7fcef68dd700 1 --2- 172.21.15.1:0/1843912248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef00689d0 0x7fcef0196a60 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fced800e910 tx=0x7fced800ec20 comp rx=0 tx=0).stop 2023-11-16T16:47:21.578 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.562+0000 7fcef68dd700 1 -- 172.21.15.1:0/1843912248 shutdown_connections 2023-11-16T16:47:21.578 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.562+0000 7fcef68dd700 1 --2- 172.21.15.1:0/1843912248 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fcedc0a12c0 0x7fcedc0a3770 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.578 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.562+0000 7fcef68dd700 1 --2- 172.21.15.1:0/1843912248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcef01055d0 0x7fcef019aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.578 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.562+0000 7fcef68dd700 1 --2- 172.21.15.1:0/1843912248 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fcef0068030 0x7fcef0196520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.578 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.562+0000 7fcef68dd700 1 --2- 172.21.15.1:0/1843912248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcef00689d0 0x7fcef0196a60 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:21.578 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.562+0000 7fcef68dd700 1 -- 172.21.15.1:0/1843912248 >> 172.21.15.1:0/1843912248 conn(0x7fcef00763a0 msgr2=0x7fcef00fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:21.578 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.562+0000 7fcef68dd700 1 -- 172.21.15.1:0/1843912248 shutdown_connections 2023-11-16T16:47:21.579 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:21.562+0000 7fcef68dd700 1 -- 172.21.15.1:0/1843912248 wait complete. 2023-11-16T16:47:21.579 INFO:tasks.cephfs.filesystem:scrub status for tag:8e0fac62-c345-4b7b-8d07-d1ddd839adb3 - {'path': '/', 'tag': '8e0fac62-c345-4b7b-8d07-d1ddd839adb3', 'options': 'recursive,force'} 2023-11-16T16:47:22.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:22 smithi053 ceph-mon[100638]: scrub queued for path: / 2023-11-16T16:47:22.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:22 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:47:22.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:22 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:47:22.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:22 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:47:22.582 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:22 smithi053 ceph-mon[100638]: pgmap v253: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-16T16:47:22.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:22 smithi001 ceph-mon[98800]: scrub queued for path: / 2023-11-16T16:47:22.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:22 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:47:22.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:22 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:47:22.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:22 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:47:22.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:22 smithi001 ceph-mon[98800]: pgmap v253: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-16T16:47:22.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:22 smithi026 ceph-mon[100112]: scrub queued for path: / 2023-11-16T16:47:22.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:22 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:47:22.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:22 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:47:22.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:22 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:47:22.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:22 smithi026 ceph-mon[100112]: pgmap v253: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-16T16:47:25.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:24 smithi026 ceph-mon[100112]: pgmap v254: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-16T16:47:25.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:24 smithi026 ceph-mon[100112]: scrub summary: idle 2023-11-16T16:47:25.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:24 smithi053 ceph-mon[100638]: pgmap v254: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-16T16:47:25.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:24 smithi053 ceph-mon[100638]: scrub summary: idle 2023-11-16T16:47:25.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:24 smithi001 ceph-mon[98800]: pgmap v254: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-16T16:47:25.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:24 smithi001 ceph-mon[98800]: scrub summary: idle 2023-11-16T16:47:26.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:25 smithi026 ceph-mon[100112]: pgmap v255: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2023-11-16T16:47:26.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:25 smithi053 ceph-mon[100638]: pgmap v255: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2023-11-16T16:47:26.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:25 smithi001 ceph-mon[98800]: pgmap v255: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2023-11-16T16:47:29.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:28 smithi026 ceph-mon[100112]: pgmap v256: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2023-11-16T16:47:29.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:28 smithi053 ceph-mon[100638]: pgmap v256: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2023-11-16T16:47:29.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:28 smithi001 ceph-mon[98800]: pgmap v256: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2023-11-16T16:47:30.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:29 smithi026 ceph-mon[100112]: pgmap v257: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:47:30.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:29 smithi053 ceph-mon[100638]: pgmap v257: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:47:30.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:29 smithi001 ceph-mon[98800]: pgmap v257: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:47:32.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:32 smithi053 ceph-mon[100638]: pgmap v258: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-16T16:47:32.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:32 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:47:32.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:32 smithi001 ceph-mon[98800]: pgmap v258: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-16T16:47:32.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:32 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:47:32.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:32 smithi026 ceph-mon[100112]: pgmap v258: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-16T16:47:32.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:32 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:47:33.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:33 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:47:33.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:33 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:47:33.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:33 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:47:33.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:33 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:47:33.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:33 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:47:33.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:33 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:47:33.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:33 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:47:33.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:33 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:47:33.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:33 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:47:34.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:34 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:47:34.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:34 smithi001 ceph-mon[98800]: Service mds.all has no related daemon already placed. 2023-11-16T16:47:34.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:34 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:47:34.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:34 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:47:34.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:34 smithi001 ceph-mon[98800]: pgmap v259: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:47:34.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:34 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:47:34.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:34 smithi026 ceph-mon[100112]: Service mds.all has no related daemon already placed. 2023-11-16T16:47:34.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:34 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:47:34.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:34 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:47:34.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:34 smithi026 ceph-mon[100112]: pgmap v259: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:47:34.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:34 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:47:34.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:34 smithi053 ceph-mon[100638]: Service mds.all has no related daemon already placed. 2023-11-16T16:47:34.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:34 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:47:34.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:34 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:47:34.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:34 smithi053 ceph-mon[100638]: pgmap v259: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:47:36.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:35 smithi026 ceph-mon[100112]: pgmap v260: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-16T16:47:36.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:35 smithi053 ceph-mon[100638]: pgmap v260: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-16T16:47:36.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:35 smithi001 ceph-mon[98800]: pgmap v260: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-16T16:47:38.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:37 smithi026 ceph-mon[100112]: pgmap v261: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:47:38.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:37 smithi053 ceph-mon[100638]: pgmap v261: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:47:38.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:37 smithi001 ceph-mon[98800]: pgmap v261: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:47:40.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:39 smithi026 ceph-mon[100112]: pgmap v262: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:47:40.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:39 smithi053 ceph-mon[100638]: pgmap v262: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:47:40.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:39 smithi001 ceph-mon[98800]: pgmap v262: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:47:42.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:42 smithi001 ceph-mon[98800]: pgmap v263: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:47:42.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:42 smithi026 ceph-mon[100112]: pgmap v263: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:47:42.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:42 smithi053 ceph-mon[100638]: pgmap v263: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:47:44.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:43 smithi026 ceph-mon[100112]: pgmap v264: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:47:44.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:43 smithi053 ceph-mon[100638]: pgmap v264: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:47:44.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:43 smithi001 ceph-mon[98800]: pgmap v264: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:47:46.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:45 smithi026 ceph-mon[100112]: pgmap v265: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:47:46.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:45 smithi053 ceph-mon[100638]: pgmap v265: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:47:46.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:45 smithi001 ceph-mon[98800]: pgmap v265: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:47:48.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:47 smithi026 ceph-mon[100112]: pgmap v266: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:47:48.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:47 smithi053 ceph-mon[100638]: pgmap v266: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:47:48.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:47 smithi001 ceph-mon[98800]: pgmap v266: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:47:50.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:49 smithi026 ceph-mon[100112]: pgmap v267: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:47:50.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:49 smithi053 ceph-mon[100638]: pgmap v267: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:47:50.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:49 smithi001 ceph-mon[98800]: pgmap v267: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:47:51.580 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-16T16:47:51.723 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.721+0000 7f1879df2700 1 -- 172.21.15.1:0/3999774082 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1874100cc0 msgr2=0x7f1874101090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:51.724 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.721+0000 7f1879df2700 1 --2- 172.21.15.1:0/3999774082 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1874100cc0 0x7f1874101090 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f18640099c0 tx=0x7f1864009cd0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.724 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.721+0000 7f1879df2700 1 -- 172.21.15.1:0/3999774082 shutdown_connections 2023-11-16T16:47:51.724 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.721+0000 7f1879df2700 1 --2- 172.21.15.1:0/3999774082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18741055c0 0x7f187410d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.724 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.721+0000 7f1879df2700 1 --2- 172.21.15.1:0/3999774082 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1874100cc0 0x7f1874101090 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.724 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.721+0000 7f1879df2700 1 --2- 172.21.15.1:0/3999774082 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1874101660 0x7f1874104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.724 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.721+0000 7f1879df2700 1 -- 172.21.15.1:0/3999774082 >> 172.21.15.1:0/3999774082 conn(0x7f1874077560 msgr2=0x7f1874077960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:51.724 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.721+0000 7f1879df2700 1 -- 172.21.15.1:0/3999774082 shutdown_connections 2023-11-16T16:47:51.725 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.721+0000 7f1879df2700 1 -- 172.21.15.1:0/3999774082 wait complete. 2023-11-16T16:47:51.725 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.722+0000 7f1879df2700 1 Processor -- start 2023-11-16T16:47:51.727 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.722+0000 7f1879df2700 1 -- start start 2023-11-16T16:47:51.727 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.722+0000 7f1879df2700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1874100cc0 0x7f1874196440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:51.727 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.723+0000 7f1879df2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1874101660 0x7f1874196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:51.727 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.723+0000 7f1879df2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18741055c0 0x7f187419ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:51.727 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.723+0000 7f1879df2700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187419b420 con 0x7f1874101660 2023-11-16T16:47:51.727 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.723+0000 7f1879df2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187419b590 con 0x7f18741055c0 2023-11-16T16:47:51.728 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.723+0000 7f1879df2700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187419b750 con 0x7f1874100cc0 2023-11-16T16:47:51.728 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.723+0000 7f1872ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1874101660 0x7f1874196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:51.728 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.723+0000 7f1872ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1874101660 0x7f1874196980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46406/0 (socket says 172.21.15.1:46406) 2023-11-16T16:47:51.728 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.723+0000 7f1872ffd700 1 -- 172.21.15.1:0/1390959515 learned_addr learned my addr 172.21.15.1:0/1390959515 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:51.728 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.723+0000 7f18737fe700 1 --2- 172.21.15.1:0/1390959515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1874100cc0 0x7f1874196440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:51.728 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.723+0000 7f1873fff700 1 --2- 172.21.15.1:0/1390959515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18741055c0 0x7f187419ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:51.728 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.723+0000 7f1872ffd700 1 -- 172.21.15.1:0/1390959515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1874100cc0 msgr2=0x7f1874196440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:51.728 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.723+0000 7f1872ffd700 1 --2- 172.21.15.1:0/1390959515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1874100cc0 0x7f1874196440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.729 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.723+0000 7f1872ffd700 1 -- 172.21.15.1:0/1390959515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18741055c0 msgr2=0x7f187419ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:51.729 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.723+0000 7f1872ffd700 1 --2- 172.21.15.1:0/1390959515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18741055c0 0x7f187419ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.729 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.723+0000 7f1872ffd700 1 -- 172.21.15.1:0/1390959515 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18640095f0 con 0x7f1874101660 2023-11-16T16:47:51.729 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.723+0000 7f1873fff700 1 --2- 172.21.15.1:0/1390959515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18741055c0 0x7f187419ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:47:51.729 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.723+0000 7f1872ffd700 1 --2- 172.21.15.1:0/1390959515 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1874101660 0x7f1874196980 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f185c00e910 tx=0x7f185c00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:51.729 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.723+0000 7f1870ff9700 1 -- 172.21.15.1:0/1390959515 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c00cbd0 con 0x7f1874101660 2023-11-16T16:47:51.729 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.723+0000 7f1870ff9700 1 -- 172.21.15.1:0/1390959515 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f185c00ee60 con 0x7f1874101660 2023-11-16T16:47:51.729 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.723+0000 7f1879df2700 1 -- 172.21.15.1:0/1390959515 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1874110380 con 0x7f1874101660 2023-11-16T16:47:51.730 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.724+0000 7f1870ff9700 1 -- 172.21.15.1:0/1390959515 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c0055c0 con 0x7f1874101660 2023-11-16T16:47:51.730 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.724+0000 7f1879df2700 1 -- 172.21.15.1:0/1390959515 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18741101f0 con 0x7f1874101660 2023-11-16T16:47:51.736 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.726+0000 7f1870ff9700 1 -- 172.21.15.1:0/1390959515 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f185c004750 con 0x7f1874101660 2023-11-16T16:47:51.736 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.727+0000 7f1870ff9700 1 --2- 172.21.15.1:0/1390959515 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f18600a1340 0x7f18600a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:51.736 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.727+0000 7f18737fe700 1 --2- 172.21.15.1:0/1390959515 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f18600a1340 0x7f18600a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:51.736 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.727+0000 7f1870ff9700 1 -- 172.21.15.1:0/1390959515 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f185c014070 con 0x7f1874101660 2023-11-16T16:47:51.736 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.728+0000 7f1885f7eb80 1 Processor -- start 2023-11-16T16:47:51.736 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.728+0000 7f18737fe700 1 --2- 172.21.15.1:0/1390959515 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f18600a1340 0x7f18600a37f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1864009990 tx=0x7f1864005fb0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:51.737 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.728+0000 7f1885f7eb80 1 -- start start 2023-11-16T16:47:51.737 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.728+0000 7f1885f7eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560df04a2b60 0x560df04a2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:51.737 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.728+0000 7f1885f7eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560df04a3470 0x560df04a7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:51.737 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.728+0000 7f1885f7eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df04a78b0 0x560df04a7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:51.737 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.728+0000 7f1885f7eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df03a8060 con 0x560df04a78b0 2023-11-16T16:47:51.737 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.728+0000 7f1885f7eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df03fa8a0 con 0x560df04a3470 2023-11-16T16:47:51.737 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.728+0000 7f1885f7eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df03f3d30 con 0x560df04a2b60 2023-11-16T16:47:51.737 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.728+0000 7f1873fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df04a78b0 0x560df04a7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:51.738 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.728+0000 7f1873fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df04a78b0 0x560df04a7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46416/0 (socket says 172.21.15.1:46416) 2023-11-16T16:47:51.738 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.728+0000 7f1873fff700 1 -- 172.21.15.1:0/3299827795 learned_addr learned my addr 172.21.15.1:0/3299827795 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:51.738 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.728+0000 7f1872ffd700 1 --2- 172.21.15.1:0/3299827795 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560df04a3470 0x560df04a7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:51.738 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.728+0000 7f1873fff700 1 -- 172.21.15.1:0/3299827795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560df04a2b60 msgr2=0x560df04a2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:51.738 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.728+0000 7f18737fe700 1 --2- 172.21.15.1:0/3299827795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560df04a2b60 0x560df04a2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:51.738 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.728+0000 7f1873fff700 1 --2- 172.21.15.1:0/3299827795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560df04a2b60 0x560df04a2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.738 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.728+0000 7f1873fff700 1 -- 172.21.15.1:0/3299827795 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560df04a3470 msgr2=0x560df04a7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:51.738 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.728+0000 7f1873fff700 1 --2- 172.21.15.1:0/3299827795 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560df04a3470 0x560df04a7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.739 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.728+0000 7f1873fff700 1 -- 172.21.15.1:0/3299827795 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560df0275630 con 0x560df04a78b0 2023-11-16T16:47:51.739 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.728+0000 7f18737fe700 1 --2- 172.21.15.1:0/3299827795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560df04a2b60 0x560df04a2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:47:51.739 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.729+0000 7f1873fff700 1 --2- 172.21.15.1:0/3299827795 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df04a78b0 0x560df04a7d00 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f186c009440 tx=0x7f186c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=393e209456a22671 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:51.739 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.729+0000 7f1869ffb700 1 -- 172.21.15.1:0/3299827795 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186c00e070 con 0x560df04a78b0 2023-11-16T16:47:51.739 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.729+0000 7f1869ffb700 1 -- 172.21.15.1:0/3299827795 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f186c00f040 con 0x560df04a78b0 2023-11-16T16:47:51.739 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.729+0000 7f1869ffb700 1 -- 172.21.15.1:0/3299827795 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186c014a70 con 0x560df04a78b0 2023-11-16T16:47:51.739 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.729+0000 7f1885f7eb80 1 -- 172.21.15.1:0/3299827795 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df04a78b0 msgr2=0x560df04a7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:51.739 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.729+0000 7f1885f7eb80 1 --2- 172.21.15.1:0/3299827795 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df04a78b0 0x560df04a7d00 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f186c009440 tx=0x7f186c009920 comp rx=0 tx=0).stop 2023-11-16T16:47:51.740 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.729+0000 7f1885f7eb80 1 -- 172.21.15.1:0/3299827795 shutdown_connections 2023-11-16T16:47:51.740 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.729+0000 7f1885f7eb80 1 --2- 172.21.15.1:0/3299827795 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df04a78b0 0x560df04a7d00 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.740 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.729+0000 7f1885f7eb80 1 --2- 172.21.15.1:0/3299827795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560df04a2b60 0x560df04a2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.740 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.729+0000 7f1885f7eb80 1 --2- 172.21.15.1:0/3299827795 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560df04a3470 0x560df04a7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.740 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.730+0000 7f1885f7eb80 1 -- 172.21.15.1:0/3299827795 >> 172.21.15.1:0/3299827795 conn(0x560df0498360 msgr2=0x560df049db40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:51.740 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.730+0000 7f1885f7eb80 1 -- 172.21.15.1:0/3299827795 shutdown_connections 2023-11-16T16:47:51.740 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.730+0000 7f1885f7eb80 1 -- 172.21.15.1:0/3299827795 wait complete. 2023-11-16T16:47:51.740 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.730+0000 7f1885f7eb80 1 -- start start 2023-11-16T16:47:51.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.730+0000 7f1885f7eb80 1 Processor -- start 2023-11-16T16:47:51.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.730+0000 7f1885f7eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560df04a2b60 0x560df04af080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:51.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.730+0000 7f1885f7eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560df04a3470 0x560df04ac290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:51.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.730+0000 7f1885f7eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df04a78b0 0x560df04ac9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:51.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.730+0000 7f1885f7eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df03958e0 con 0x560df04a78b0 2023-11-16T16:47:51.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.730+0000 7f1885f7eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df0395580 con 0x560df04a2b60 2023-11-16T16:47:51.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.730+0000 7f1885f7eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df0277920 con 0x560df04a3470 2023-11-16T16:47:51.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.730+0000 7f1885f7eb80 10 client.0 _ll_register_callbacks cb 0x560df03b3780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:47:51.742 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.731+0000 7f1873fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df04a78b0 0x560df04ac9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:51.742 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.731+0000 7f1873fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df04a78b0 0x560df04ac9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46422/0 (socket says 172.21.15.1:46422) 2023-11-16T16:47:51.742 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.731+0000 7f1873fff700 1 -- 172.21.15.1:0/34149218 learned_addr learned my addr 172.21.15.1:0/34149218 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:51.742 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.731+0000 7f18737fe700 1 --2- 172.21.15.1:0/34149218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560df04a2b60 0x560df04af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:51.742 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.731+0000 7f1872ffd700 1 --2- 172.21.15.1:0/34149218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560df04a3470 0x560df04ac290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:51.742 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.731+0000 7f1873fff700 1 -- 172.21.15.1:0/34149218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560df04a3470 msgr2=0x560df04ac290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:51.742 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.731+0000 7f1873fff700 1 --2- 172.21.15.1:0/34149218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560df04a3470 0x560df04ac290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.742 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.731+0000 7f1873fff700 1 -- 172.21.15.1:0/34149218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560df04a2b60 msgr2=0x560df04af080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:51.743 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.731+0000 7f1873fff700 1 --2- 172.21.15.1:0/34149218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560df04a2b60 0x560df04af080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.743 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.731+0000 7f1873fff700 1 -- 172.21.15.1:0/34149218 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f186c0090f0 con 0x560df04a78b0 2023-11-16T16:47:51.743 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.731+0000 7f1872ffd700 1 --2- 172.21.15.1:0/34149218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560df04a3470 0x560df04ac290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:47:51.743 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.731+0000 7f1873fff700 1 --2- 172.21.15.1:0/34149218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df04a78b0 0x560df04ac9d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f186c007ac0 tx=0x7f186c014dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a526bad28ecf33f server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:51.743 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.731+0000 7f1852ffd700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:47:51.743 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.731+0000 7f1852ffd700 1 -- 172.21.15.1:0/34149218 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186c00e040 con 0x560df04a78b0 2023-11-16T16:47:51.743 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.731+0000 7f1885f7eb80 1 -- 172.21.15.1:0/34149218 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560df0495b20 con 0x560df04a78b0 2023-11-16T16:47:51.743 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.731+0000 7f1852ffd700 1 -- 172.21.15.1:0/34149218 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f186c004ce0 con 0x560df04a78b0 2023-11-16T16:47:51.743 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.731+0000 7f1852ffd700 1 -- 172.21.15.1:0/34149218 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f186c004ea0 con 0x560df04a78b0 2023-11-16T16:47:51.744 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.731+0000 7f1885f7eb80 10 client.16662 fetch_fsmap learned FSMap version 28 2023-11-16T16:47:51.744 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.731+0000 7f1885f7eb80 1 -- 172.21.15.1:0/34149218 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x560df03ba190 con 0x560df04a78b0 2023-11-16T16:47:51.744 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.732+0000 7f1852ffd700 1 -- 172.21.15.1:0/34149218 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560df0495b20 con 0x560df04a78b0 2023-11-16T16:47:51.744 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.732+0000 7f1852ffd700 1 -- 172.21.15.1:0/34149218 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f186c01c070 con 0x560df04a78b0 2023-11-16T16:47:51.744 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.732+0000 7f1885f7eb80 10 client.16662 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:47:51.744 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.732+0000 7f1885f7eb80 10 client.16662 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:47:51.744 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.732+0000 7f1885f7eb80 20 client.16662 populate_metadata read hostname 'smithi001' 2023-11-16T16:47:51.744 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.732+0000 7f1885f7eb80 1 --2- 172.21.15.1:0/34149218 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x560df04bc700 0x560df04beb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:51.745 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.732+0000 7f1885f7eb80 4 client.16662 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:47:51.745 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.732+0000 7f1885f7eb80 1 -- 172.21.15.1:0/34149218 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560df03b60a0 con 0x560df04bc700 2023-11-16T16:47:51.745 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.732+0000 7f18737fe700 1 --2- 172.21.15.1:0/34149218 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x560df04bc700 0x560df04beb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:51.745 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.733+0000 7f1852ffd700 10 client.16662 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:47:51.745 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.733+0000 7f18737fe700 1 --2- 172.21.15.1:0/34149218 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x560df04bc700 0x560df04beb10 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2709dfc1565b1c21 server_cookie=f19d2769429cae5 in_seq=0 out_seq=0 2023-11-16T16:47:51.745 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.733+0000 7f1852ffd700 1 -- 172.21.15.1:0/34149218 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560df03b60a0 con 0x560df04bc700 2023-11-16T16:47:51.745 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.733+0000 7f1852ffd700 10 client.16662 handle_command_reply: tid=0 2023-11-16T16:47:51.745 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.733+0000 7f1885f7eb80 1 client.16662 shutdown 2023-11-16T16:47:51.746 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.734+0000 7f1885f7eb80 1 -- 172.21.15.1:0/34149218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df04a78b0 msgr2=0x560df04ac9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:51.746 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.734+0000 7f1885f7eb80 1 --2- 172.21.15.1:0/34149218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df04a78b0 0x560df04ac9d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f186c007ac0 tx=0x7f186c014dc0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.746 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.734+0000 7f1885f7eb80 1 -- 172.21.15.1:0/34149218 shutdown_connections 2023-11-16T16:47:51.746 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.734+0000 7f1885f7eb80 1 --2- 172.21.15.1:0/34149218 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x560df04bc700 0x560df04beb10 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.746 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.734+0000 7f1885f7eb80 1 --2- 172.21.15.1:0/34149218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df04a78b0 0x560df04ac9d0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.746 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.734+0000 7f1885f7eb80 1 --2- 172.21.15.1:0/34149218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560df04a2b60 0x560df04af080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.746 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.734+0000 7f1852ffd700 0 client.16662 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:47:51.746 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.734+0000 7f1885f7eb80 1 --2- 172.21.15.1:0/34149218 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560df04a3470 0x560df04ac290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.747 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.734+0000 7f1885f7eb80 1 -- 172.21.15.1:0/34149218 >> 172.21.15.1:0/34149218 conn(0x560df0498360 msgr2=0x560df049cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:51.747 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.734+0000 7f1885f7eb80 1 -- 172.21.15.1:0/34149218 shutdown_connections 2023-11-16T16:47:51.747 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.734+0000 7f1885f7eb80 1 -- 172.21.15.1:0/34149218 wait complete. 2023-11-16T16:47:51.747 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.734+0000 7f1885f7eb80 20 client.16662 trim_cache size 0 max 16384 2023-11-16T16:47:51.754 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.751+0000 7f1885f7eb80 1 Processor -- start 2023-11-16T16:47:51.754 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.751+0000 7f1885f7eb80 1 -- start start 2023-11-16T16:47:51.754 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.751+0000 7f1885f7eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560df04a2b60 0x560df04b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:51.755 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.751+0000 7f1885f7eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df04a3470 0x560df04b39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:51.755 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.751+0000 7f1885f7eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560df04a78b0 0x560df04aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:51.755 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.751+0000 7f1885f7eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df04af730 con 0x560df04a3470 2023-11-16T16:47:51.755 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.751+0000 7f1885f7eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df0497ee0 con 0x560df04a2b60 2023-11-16T16:47:51.755 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.751+0000 7f1885f7eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df04a63c0 con 0x560df04a78b0 2023-11-16T16:47:51.755 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.751+0000 7f1872ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df04a3470 0x560df04b39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:51.755 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.751+0000 7f1872ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df04a3470 0x560df04b39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46428/0 (socket says 172.21.15.1:46428) 2023-11-16T16:47:51.755 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.751+0000 7f1872ffd700 1 -- 172.21.15.1:0/721482560 learned_addr learned my addr 172.21.15.1:0/721482560 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:51.756 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.751+0000 7f18737fe700 1 --2- 172.21.15.1:0/721482560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560df04a2b60 0x560df04b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:51.756 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.751+0000 7f1873fff700 1 --2- 172.21.15.1:0/721482560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560df04a78b0 0x560df04aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:51.756 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.751+0000 7f1872ffd700 1 -- 172.21.15.1:0/721482560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560df04a78b0 msgr2=0x560df04aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:51.756 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.751+0000 7f1872ffd700 1 --2- 172.21.15.1:0/721482560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560df04a78b0 0x560df04aebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.756 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.752+0000 7f1872ffd700 1 -- 172.21.15.1:0/721482560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560df04a2b60 msgr2=0x560df04b8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:51.756 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.752+0000 7f1872ffd700 1 --2- 172.21.15.1:0/721482560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560df04a2b60 0x560df04b8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.756 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.752+0000 7f1872ffd700 1 -- 172.21.15.1:0/721482560 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560df03b9ee0 con 0x560df04a3470 2023-11-16T16:47:51.756 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.752+0000 7f18737fe700 1 --2- 172.21.15.1:0/721482560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560df04a2b60 0x560df04b8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:47:51.757 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.752+0000 7f1872ffd700 1 --2- 172.21.15.1:0/721482560 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df04a3470 0x560df04b39a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f185c00c5b0 tx=0x7f185c09a010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5833853fdd985c server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:51.757 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.752+0000 7f186a7fc700 1 -- 172.21.15.1:0/721482560 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c09e140 con 0x560df04a3470 2023-11-16T16:47:51.757 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.752+0000 7f186a7fc700 1 -- 172.21.15.1:0/721482560 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f185c0a2560 con 0x560df04a3470 2023-11-16T16:47:51.757 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.752+0000 7f186a7fc700 1 -- 172.21.15.1:0/721482560 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c09a370 con 0x560df04a3470 2023-11-16T16:47:51.757 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.752+0000 7f1885f7eb80 1 -- 172.21.15.1:0/721482560 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df04a3470 msgr2=0x560df04b39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:51.757 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.752+0000 7f1885f7eb80 1 --2- 172.21.15.1:0/721482560 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df04a3470 0x560df04b39a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f185c00c5b0 tx=0x7f185c09a010 comp rx=0 tx=0).stop 2023-11-16T16:47:51.757 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.752+0000 7f1885f7eb80 1 -- 172.21.15.1:0/721482560 shutdown_connections 2023-11-16T16:47:51.757 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.752+0000 7f1885f7eb80 1 --2- 172.21.15.1:0/721482560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560df04a78b0 0x560df04aebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.758 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.752+0000 7f1885f7eb80 1 --2- 172.21.15.1:0/721482560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560df04a2b60 0x560df04b8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.758 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.752+0000 7f1885f7eb80 1 --2- 172.21.15.1:0/721482560 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df04a3470 0x560df04b39a0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.758 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.752+0000 7f1885f7eb80 1 -- 172.21.15.1:0/721482560 >> 172.21.15.1:0/721482560 conn(0x560df0498360 msgr2=0x560df04a9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:51.758 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.752+0000 7f1885f7eb80 1 -- 172.21.15.1:0/721482560 shutdown_connections 2023-11-16T16:47:51.758 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.752+0000 7f1885f7eb80 1 -- 172.21.15.1:0/721482560 wait complete. 2023-11-16T16:47:51.758 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.753+0000 7f1885f7eb80 1 -- start start 2023-11-16T16:47:51.762 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.753+0000 7f1885f7eb80 1 Processor -- start 2023-11-16T16:47:51.762 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.753+0000 7f1885f7eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df04a2b60 0x560df04b3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:51.762 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.753+0000 7f1885f7eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560df04a3470 0x560df04afa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:51.762 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.753+0000 7f1885f7eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560df04a78b0 0x560df04aff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:51.762 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.753+0000 7f1885f7eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df04b04c0 con 0x560df04a2b60 2023-11-16T16:47:51.762 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.753+0000 7f1885f7eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df04b0630 con 0x560df04a3470 2023-11-16T16:47:51.763 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.753+0000 7f1885f7eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df04b07f0 con 0x560df04a78b0 2023-11-16T16:47:51.763 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.754+0000 7f18737fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df04a2b60 0x560df04b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:51.763 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.754+0000 7f1885f7eb80 10 client.0 _ll_register_callbacks cb 0x7f186c009310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:47:51.763 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.754+0000 7f18737fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df04a2b60 0x560df04b3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46432/0 (socket says 172.21.15.1:46432) 2023-11-16T16:47:51.763 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.754+0000 7f18737fe700 1 -- 172.21.15.1:0/1445826573 learned_addr learned my addr 172.21.15.1:0/1445826573 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:51.763 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.754+0000 7f1872ffd700 1 --2- 172.21.15.1:0/1445826573 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560df04a3470 0x560df04afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:51.763 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.754+0000 7f1873fff700 1 --2- 172.21.15.1:0/1445826573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560df04a78b0 0x560df04aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:51.763 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.754+0000 7f18737fe700 1 -- 172.21.15.1:0/1445826573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560df04a78b0 msgr2=0x560df04aff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:51.764 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.754+0000 7f18737fe700 1 --2- 172.21.15.1:0/1445826573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560df04a78b0 0x560df04aff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.764 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.754+0000 7f18737fe700 1 -- 172.21.15.1:0/1445826573 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560df04a3470 msgr2=0x560df04afa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:51.764 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.754+0000 7f18737fe700 1 --2- 172.21.15.1:0/1445826573 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560df04a3470 0x560df04afa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.764 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.754+0000 7f18737fe700 1 -- 172.21.15.1:0/1445826573 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18741101f0 con 0x560df04a2b60 2023-11-16T16:47:51.764 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.754+0000 7f18737fe700 1 --2- 172.21.15.1:0/1445826573 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df04a2b60 0x560df04b3730 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f1864000c00 tx=0x7f1864009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=675a43af90bc117d server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:51.764 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.754+0000 7f1852ffd700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:47:51.764 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.754+0000 7f1852ffd700 1 -- 172.21.15.1:0/1445826573 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186402b070 con 0x560df04a2b60 2023-11-16T16:47:51.764 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.754+0000 7f1852ffd700 1 -- 172.21.15.1:0/1445826573 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f18640329d0 con 0x560df04a2b60 2023-11-16T16:47:51.764 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.754+0000 7f1885f7eb80 1 -- 172.21.15.1:0/1445826573 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f186c00f040 con 0x560df04a2b60 2023-11-16T16:47:51.765 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.754+0000 7f1852ffd700 1 -- 172.21.15.1:0/1445826573 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f186400bea0 con 0x560df04a2b60 2023-11-16T16:47:51.765 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.754+0000 7f1885f7eb80 10 client.16668 fetch_fsmap learned FSMap version 28 2023-11-16T16:47:51.765 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.754+0000 7f1885f7eb80 1 -- 172.21.15.1:0/1445826573 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x560df020ac60 con 0x560df04a2b60 2023-11-16T16:47:51.765 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.755+0000 7f1852ffd700 1 -- 172.21.15.1:0/1445826573 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f186c00f040 con 0x560df04a2b60 2023-11-16T16:47:51.765 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.755+0000 7f1852ffd700 1 -- 172.21.15.1:0/1445826573 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f1864004660 con 0x560df04a2b60 2023-11-16T16:47:51.765 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.755+0000 7f1885f7eb80 10 client.16668 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:47:51.765 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.755+0000 7f1885f7eb80 10 client.16668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:47:51.765 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.755+0000 7f1885f7eb80 20 client.16668 populate_metadata read hostname 'smithi001' 2023-11-16T16:47:51.766 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.755+0000 7f1885f7eb80 1 --2- 172.21.15.1:0/1445826573 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x560df0511830 0x560df0513c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:51.766 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.755+0000 7f1885f7eb80 4 client.16668 mds_command: new command op to 24521 tid=0[{"prefix": "scrub status"}] 2023-11-16T16:47:51.766 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.755+0000 7f1885f7eb80 1 -- 172.21.15.1:0/1445826573 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560df03c2660 con 0x560df0511830 2023-11-16T16:47:51.766 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.755+0000 7f1872ffd700 1 --2- 172.21.15.1:0/1445826573 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x560df0511830 0x560df0513c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:51.766 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.755+0000 7f1872ffd700 1 --2- 172.21.15.1:0/1445826573 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x560df0511830 0x560df0513c40 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97dd4f65259c4b29 server_cookie=c4c56401633ec59d in_seq=0 out_seq=0 2023-11-16T16:47:51.766 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.755+0000 7f1852ffd700 10 client.16668 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:47:51.766 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.756+0000 7f1852ffd700 1 -- 172.21.15.1:0/1445826573 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560df03c2660 con 0x560df0511830 2023-11-16T16:47:51.766 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.756+0000 7f1852ffd700 10 client.16668 handle_command_reply: tid=0 2023-11-16T16:47:51.767 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.756+0000 7f1885f7eb80 1 client.16668 shutdown 2023-11-16T16:47:51.767 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.756+0000 7f1885f7eb80 1 -- 172.21.15.1:0/1445826573 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df04a2b60 msgr2=0x560df04b3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:51.767 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.756+0000 7f1885f7eb80 1 --2- 172.21.15.1:0/1445826573 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df04a2b60 0x560df04b3730 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f1864000c00 tx=0x7f1864009cd0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.767 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.756+0000 7f1885f7eb80 1 -- 172.21.15.1:0/1445826573 shutdown_connections 2023-11-16T16:47:51.767 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.756+0000 7f1885f7eb80 1 --2- 172.21.15.1:0/1445826573 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x560df0511830 0x560df0513c40 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.767 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.756+0000 7f1885f7eb80 1 --2- 172.21.15.1:0/1445826573 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x560df04a78b0 0x560df04aff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.767 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.756+0000 7f1885f7eb80 1 --2- 172.21.15.1:0/1445826573 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df04a2b60 0x560df04b3730 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.767 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.756+0000 7f1885f7eb80 1 --2- 172.21.15.1:0/1445826573 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560df04a3470 0x560df04afa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.767 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.756+0000 7f1885f7eb80 1 -- 172.21.15.1:0/1445826573 >> 172.21.15.1:0/1445826573 conn(0x560df0498360 msgr2=0x560df04a9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:51.768 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.756+0000 7f1852ffd700 0 client.16668 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:47:51.768 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.756+0000 7f1885f7eb80 1 -- 172.21.15.1:0/1445826573 shutdown_connections 2023-11-16T16:47:51.768 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.756+0000 7f1885f7eb80 1 -- 172.21.15.1:0/1445826573 wait complete. 2023-11-16T16:47:51.768 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.756+0000 7f1885f7eb80 20 client.16668 trim_cache size 0 max 16384 2023-11-16T16:47:51.768 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:47:51.768 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2023-11-16T16:47:51.769 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2023-11-16T16:47:51.769 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:47:51.769 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.757+0000 7f1879df2700 1 -- 172.21.15.1:0/1390959515 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f18600a1340 msgr2=0x7f18600a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:51.770 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.757+0000 7f1879df2700 1 --2- 172.21.15.1:0/1390959515 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f18600a1340 0x7f18600a37f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1864009990 tx=0x7f1864005fb0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.770 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.757+0000 7f1879df2700 1 -- 172.21.15.1:0/1390959515 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1874101660 msgr2=0x7f1874196980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:51.770 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.757+0000 7f1879df2700 1 --2- 172.21.15.1:0/1390959515 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1874101660 0x7f1874196980 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f185c00e910 tx=0x7f185c00ec20 comp rx=0 tx=0).stop 2023-11-16T16:47:51.770 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.757+0000 7f1879df2700 1 -- 172.21.15.1:0/1390959515 shutdown_connections 2023-11-16T16:47:51.770 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.757+0000 7f1879df2700 1 --2- 172.21.15.1:0/1390959515 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f18600a1340 0x7f18600a37f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.770 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.757+0000 7f1879df2700 1 --2- 172.21.15.1:0/1390959515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18741055c0 0x7f187419ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.771 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.757+0000 7f1879df2700 1 --2- 172.21.15.1:0/1390959515 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1874100cc0 0x7f1874196440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.771 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.757+0000 7f1879df2700 1 --2- 172.21.15.1:0/1390959515 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1874101660 0x7f1874196980 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.771 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.757+0000 7f1879df2700 1 -- 172.21.15.1:0/1390959515 >> 172.21.15.1:0/1390959515 conn(0x7f1874077560 msgr2=0x7f18740ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:51.771 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.757+0000 7f1879df2700 1 -- 172.21.15.1:0/1390959515 shutdown_connections 2023-11-16T16:47:51.771 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.757+0000 7f1879df2700 1 -- 172.21.15.1:0/1390959515 wait complete. 2023-11-16T16:47:51.772 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-16T16:47:51.772 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:47:51.901 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.896+0000 7f76ae09b700 1 -- 172.21.15.1:0/798652496 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76a8101660 msgr2=0x7f76a8104ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:51.901 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.896+0000 7f76ae09b700 1 --2- 172.21.15.1:0/798652496 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76a8101660 0x7f76a8104ef0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f769c0099c0 tx=0x7f769c009cd0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.901 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.896+0000 7f76ae09b700 1 -- 172.21.15.1:0/798652496 shutdown_connections 2023-11-16T16:47:51.901 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.896+0000 7f76ae09b700 1 --2- 172.21.15.1:0/798652496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76a8105620 0x7f76a810d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.901 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.896+0000 7f76ae09b700 1 --2- 172.21.15.1:0/798652496 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76a8100cc0 0x7f76a8101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.902 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.896+0000 7f76ae09b700 1 --2- 172.21.15.1:0/798652496 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76a8101660 0x7f76a8104ef0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.902 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.896+0000 7f76ae09b700 1 -- 172.21.15.1:0/798652496 >> 172.21.15.1:0/798652496 conn(0x7f76a80774d0 msgr2=0x7f76a80778d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:51.902 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.896+0000 7f76ae09b700 1 -- 172.21.15.1:0/798652496 shutdown_connections 2023-11-16T16:47:51.902 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.896+0000 7f76ae09b700 1 -- 172.21.15.1:0/798652496 wait complete. 2023-11-16T16:47:51.902 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.896+0000 7f76ae09b700 1 Processor -- start 2023-11-16T16:47:51.902 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.896+0000 7f76ae09b700 1 -- start start 2023-11-16T16:47:51.902 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.897+0000 7f76ae09b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76a8100cc0 0x7f76a8196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:51.902 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.897+0000 7f76ae09b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76a8101660 0x7f76a8196970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:51.903 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.897+0000 7f76ae09b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76a8105620 0x7f76a819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:51.903 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.897+0000 7f76ae09b700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a819b3a0 con 0x7f76a8101660 2023-11-16T16:47:51.903 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.897+0000 7f76ae09b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a819b510 con 0x7f76a8105620 2023-11-16T16:47:51.903 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.897+0000 7f76ae09b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a819b6d0 con 0x7f76a8100cc0 2023-11-16T16:47:51.903 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.897+0000 7f76a6ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76a8101660 0x7f76a8196970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:51.903 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.897+0000 7f76a6ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76a8101660 0x7f76a8196970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46448/0 (socket says 172.21.15.1:46448) 2023-11-16T16:47:51.903 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.897+0000 7f76a6ffd700 1 -- 172.21.15.1:0/2208290835 learned_addr learned my addr 172.21.15.1:0/2208290835 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:51.903 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.897+0000 7f76a7fff700 1 --2- 172.21.15.1:0/2208290835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76a8105620 0x7f76a819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:51.903 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.897+0000 7f76a77fe700 1 --2- 172.21.15.1:0/2208290835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76a8100cc0 0x7f76a8196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:51.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.897+0000 7f76a6ffd700 1 -- 172.21.15.1:0/2208290835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76a8100cc0 msgr2=0x7f76a8196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:51.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.897+0000 7f76a6ffd700 1 --2- 172.21.15.1:0/2208290835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76a8100cc0 0x7f76a8196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.897+0000 7f76a6ffd700 1 -- 172.21.15.1:0/2208290835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76a8105620 msgr2=0x7f76a819ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:51.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.897+0000 7f76a6ffd700 1 --2- 172.21.15.1:0/2208290835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76a8105620 0x7f76a819ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:51.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.897+0000 7f76a6ffd700 1 -- 172.21.15.1:0/2208290835 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f769c0095f0 con 0x7f76a8101660 2023-11-16T16:47:51.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.897+0000 7f76a77fe700 1 --2- 172.21.15.1:0/2208290835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76a8100cc0 0x7f76a8196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:47:51.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.897+0000 7f76a6ffd700 1 --2- 172.21.15.1:0/2208290835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76a8101660 0x7f76a8196970 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f769c009fd0 tx=0x7f769c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:51.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.897+0000 7f76a4ff9700 1 -- 172.21.15.1:0/2208290835 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c01d070 con 0x7f76a8101660 2023-11-16T16:47:51.905 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.897+0000 7f76a4ff9700 1 -- 172.21.15.1:0/2208290835 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f769c005020 con 0x7f76a8101660 2023-11-16T16:47:51.905 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.897+0000 7f76ae09b700 1 -- 172.21.15.1:0/2208290835 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a8110320 con 0x7f76a8101660 2023-11-16T16:47:51.905 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.897+0000 7f76a4ff9700 1 -- 172.21.15.1:0/2208290835 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c004aa0 con 0x7f76a8101660 2023-11-16T16:47:51.905 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.898+0000 7f76ae09b700 1 -- 172.21.15.1:0/2208290835 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a8110190 con 0x7f76a8101660 2023-11-16T16:47:51.905 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.899+0000 7f76ae09b700 1 -- 172.21.15.1:0/2208290835 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a81104b0 con 0x7f76a8101660 2023-11-16T16:47:51.905 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.899+0000 7f76a4ff9700 1 -- 172.21.15.1:0/2208290835 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f769c0054b0 con 0x7f76a8101660 2023-11-16T16:47:51.906 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.900+0000 7f76a4ff9700 1 --2- 172.21.15.1:0/2208290835 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f76900a12c0 0x7f76900a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:51.906 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.900+0000 7f76a77fe700 1 --2- 172.21.15.1:0/2208290835 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f76900a12c0 0x7f76900a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:51.906 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.900+0000 7f76a4ff9700 1 -- 172.21.15.1:0/2208290835 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f769c0cd800 con 0x7f76a8101660 2023-11-16T16:47:51.906 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.900+0000 7f76a77fe700 1 --2- 172.21.15.1:0/2208290835 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f76900a12c0 0x7f76900a3770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7698009cb0 tx=0x7f7698009260 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:51.906 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:51.905+0000 7f76a4ff9700 1 -- 172.21.15.1:0/2208290835 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f769c096c90 con 0x7f76a8101660 2023-11-16T16:47:52.150 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.148+0000 7f76ae09b700 1 -- 172.21.15.1:0/2208290835 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76a8197780 con 0x7f76a8101660 2023-11-16T16:47:52.150 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.149+0000 7f76a4ff9700 1 -- 172.21.15.1:0/2208290835 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10783 (secure 0 0 0) 0x7f769c09a2b0 con 0x7f76a8101660 2023-11-16T16:47:52.151 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:47:52.151 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2587563684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2587563684},{"type":"v1","addr":"172.21.15.1:6835","nonce":2587563684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/277936168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":277936168},{"type":"v1","addr":"172.21.15.1:6837","nonce":277936168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/1172575045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1172575045},{"type":"v1","addr":"172.21.15.1:6839","nonce":1172575045}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/2631175899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2631175899},{"type":"v1","addr":"172.21.15.1:6841","nonce":2631175899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3693905267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3693905267},{"type":"v1","addr":"172.21.15.53:6833","nonce":3693905267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/164668033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":164668033},{"type":"v1","addr":"172.21.15.26:6833","nonce":164668033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-16T16:37:13.918943+0000","modified":"2023-11-16T16:43:40.628525+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2140542192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2140542192},{"type":"v1","addr":"172.21.15.53:6835","nonce":2140542192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1010760275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1010760275},{"type":"v1","addr":"172.21.15.26:6835","nonce":1010760275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3197060302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3197060302},{"type":"v1","addr":"172.21.15.53:6837","nonce":3197060302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6837/1453504863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1453504863},{"type":"v1","addr":"172.21.15.26:6837","nonce":1453504863}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.53:6839/1044753024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1044753024},{"type":"v1","addr":"172.21.15.53:6839","nonce":1044753024}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6839/2192205949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2192205949},{"type":"v1","addr":"172.21.15.26:6839","nonce":2192205949}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-16T16:47:52.153 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.151+0000 7f76ae09b700 1 -- 172.21.15.1:0/2208290835 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f76900a12c0 msgr2=0x7f76900a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:52.153 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.151+0000 7f76ae09b700 1 --2- 172.21.15.1:0/2208290835 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f76900a12c0 0x7f76900a3770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7698009cb0 tx=0x7f7698009260 comp rx=0 tx=0).stop 2023-11-16T16:47:52.153 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.151+0000 7f76ae09b700 1 -- 172.21.15.1:0/2208290835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76a8101660 msgr2=0x7f76a8196970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:52.153 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.151+0000 7f76ae09b700 1 --2- 172.21.15.1:0/2208290835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76a8101660 0x7f76a8196970 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f769c009fd0 tx=0x7f769c009580 comp rx=0 tx=0).stop 2023-11-16T16:47:52.154 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.151+0000 7f76ae09b700 1 -- 172.21.15.1:0/2208290835 shutdown_connections 2023-11-16T16:47:52.154 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.151+0000 7f76ae09b700 1 --2- 172.21.15.1:0/2208290835 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f76900a12c0 0x7f76900a3770 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.154 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.151+0000 7f76ae09b700 1 --2- 172.21.15.1:0/2208290835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76a8105620 0x7f76a819ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.154 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.151+0000 7f76ae09b700 1 --2- 172.21.15.1:0/2208290835 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f76a8100cc0 0x7f76a8196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.154 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.151+0000 7f76ae09b700 1 --2- 172.21.15.1:0/2208290835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76a8101660 0x7f76a8196970 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.154 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.151+0000 7f76ae09b700 1 -- 172.21.15.1:0/2208290835 >> 172.21.15.1:0/2208290835 conn(0x7f76a80774d0 msgr2=0x7f76a80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:52.154 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.151+0000 7f76ae09b700 1 -- 172.21.15.1:0/2208290835 shutdown_connections 2023-11-16T16:47:52.154 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.151+0000 7f76ae09b700 1 -- 172.21.15.1:0/2208290835 wait complete. 2023-11-16T16:47:52.155 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 28 2023-11-16T16:47:52.165 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-11-16T16:47:52.295 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.291+0000 7f7c3480b700 1 -- 172.21.15.1:0/349026979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c300ffda0 msgr2=0x7f7c3010a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:52.295 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.291+0000 7f7c3480b700 1 --2- 172.21.15.1:0/349026979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c300ffda0 0x7f7c3010a810 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f7c20009990 tx=0x7f7c20009ca0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.295 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.291+0000 7f7c3480b700 1 -- 172.21.15.1:0/349026979 shutdown_connections 2023-11-16T16:47:52.295 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.291+0000 7f7c3480b700 1 --2- 172.21.15.1:0/349026979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c3010ad50 0x7f7c3010d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.295 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.291+0000 7f7c3480b700 1 --2- 172.21.15.1:0/349026979 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c300ff490 0x7f7c300ff860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.295 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.291+0000 7f7c3480b700 1 --2- 172.21.15.1:0/349026979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c300ffda0 0x7f7c3010a810 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.295 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.291+0000 7f7c3480b700 1 -- 172.21.15.1:0/349026979 >> 172.21.15.1:0/349026979 conn(0x7f7c30074ac0 msgr2=0x7f7c30074ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:52.296 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.291+0000 7f7c3480b700 1 -- 172.21.15.1:0/349026979 shutdown_connections 2023-11-16T16:47:52.296 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.291+0000 7f7c3480b700 1 -- 172.21.15.1:0/349026979 wait complete. 2023-11-16T16:47:52.296 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.292+0000 7f7c3480b700 1 Processor -- start 2023-11-16T16:47:52.296 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.292+0000 7f7c3480b700 1 -- start start 2023-11-16T16:47:52.296 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.292+0000 7f7c3480b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c300ff490 0x7f7c30196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.296 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.292+0000 7f7c3480b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c300ffda0 0x7f7c301968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.296 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.292+0000 7f7c3480b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c3010ad50 0x7f7c3019ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.292+0000 7f7c3480b700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3019b360 con 0x7f7c300ff490 2023-11-16T16:47:52.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.292+0000 7f7c3480b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3019b4d0 con 0x7f7c300ffda0 2023-11-16T16:47:52.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.292+0000 7f7c3480b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3019b690 con 0x7f7c3010ad50 2023-11-16T16:47:52.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.292+0000 7f7c2e59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c300ff490 0x7f7c30196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.292+0000 7f7c2e59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c300ff490 0x7f7c30196380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46462/0 (socket says 172.21.15.1:46462) 2023-11-16T16:47:52.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.292+0000 7f7c2e59c700 1 -- 172.21.15.1:0/2456780318 learned_addr learned my addr 172.21.15.1:0/2456780318 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:52.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.292+0000 7f7c2dd9b700 1 --2- 172.21.15.1:0/2456780318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c300ffda0 0x7f7c301968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.292+0000 7f7c2ed9d700 1 --2- 172.21.15.1:0/2456780318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c3010ad50 0x7f7c3019ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.292+0000 7f7c2e59c700 1 -- 172.21.15.1:0/2456780318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c3010ad50 msgr2=0x7f7c3019ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:52.298 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.292+0000 7f7c2e59c700 1 --2- 172.21.15.1:0/2456780318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c3010ad50 0x7f7c3019ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.298 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.292+0000 7f7c2e59c700 1 -- 172.21.15.1:0/2456780318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c300ffda0 msgr2=0x7f7c301968c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:52.298 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.292+0000 7f7c2e59c700 1 --2- 172.21.15.1:0/2456780318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c300ffda0 0x7f7c301968c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.298 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.292+0000 7f7c2e59c700 1 -- 172.21.15.1:0/2456780318 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c200095f0 con 0x7f7c300ff490 2023-11-16T16:47:52.298 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.292+0000 7f7c2ed9d700 1 --2- 172.21.15.1:0/2456780318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c3010ad50 0x7f7c3019ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:47:52.298 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.293+0000 7f7c2e59c700 1 --2- 172.21.15.1:0/2456780318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c300ff490 0x7f7c30196380 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f7c2400d950 tx=0x7f7c2400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:52.298 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.293+0000 7f7c1f7fe700 1 -- 172.21.15.1:0/2456780318 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c24009940 con 0x7f7c300ff490 2023-11-16T16:47:52.298 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.293+0000 7f7c1f7fe700 1 -- 172.21.15.1:0/2456780318 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c24010460 con 0x7f7c300ff490 2023-11-16T16:47:52.299 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.293+0000 7f7c3480b700 1 -- 172.21.15.1:0/2456780318 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c301103a0 con 0x7f7c300ff490 2023-11-16T16:47:52.299 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.293+0000 7f7c1f7fe700 1 -- 172.21.15.1:0/2456780318 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2400f620 con 0x7f7c300ff490 2023-11-16T16:47:52.299 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.293+0000 7f7c3480b700 1 -- 172.21.15.1:0/2456780318 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c30110210 con 0x7f7c300ff490 2023-11-16T16:47:52.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.295+0000 7f7c1f7fe700 1 -- 172.21.15.1:0/2456780318 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f7c240108d0 con 0x7f7c300ff490 2023-11-16T16:47:52.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.295+0000 7f7c1f7fe700 1 --2- 172.21.15.1:0/2456780318 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f7c180a12c0 0x7f7c180a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.296+0000 7f7c2dd9b700 1 --2- 172.21.15.1:0/2456780318 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f7c180a12c0 0x7f7c180a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.296+0000 7f7c1f7fe700 1 -- 172.21.15.1:0/2456780318 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f7c240ccc10 con 0x7f7c300ff490 2023-11-16T16:47:52.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.296+0000 7f7c2dd9b700 1 --2- 172.21.15.1:0/2456780318 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f7c180a12c0 0x7f7c180a3770 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7c30197990 tx=0x7f7c20005fd0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:52.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.296+0000 7f7c40997b80 1 Processor -- start 2023-11-16T16:47:52.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.296+0000 7f7c40997b80 1 -- start start 2023-11-16T16:47:52.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.297+0000 7f7c40997b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e9c6b7b60 0x564e9c6b7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.297+0000 7f7c40997b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e9c6b8470 0x564e9c6bc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.297+0000 7f7c40997b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e9c6bc8b0 0x564e9c6bcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.297+0000 7f7c40997b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e9c5bd060 con 0x564e9c6bc8b0 2023-11-16T16:47:52.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.297+0000 7f7c40997b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e9c60f8a0 con 0x564e9c6b7b60 2023-11-16T16:47:52.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.297+0000 7f7c40997b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e9c608d30 con 0x564e9c6b8470 2023-11-16T16:47:52.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.297+0000 7f7c2ed9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e9c6bc8b0 0x564e9c6bcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.297+0000 7f7c2ed9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e9c6bc8b0 0x564e9c6bcd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46472/0 (socket says 172.21.15.1:46472) 2023-11-16T16:47:52.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.297+0000 7f7c2ed9d700 1 -- 172.21.15.1:0/3518421136 learned_addr learned my addr 172.21.15.1:0/3518421136 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:52.307 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.297+0000 7f7c2dd9b700 1 --2- 172.21.15.1:0/3518421136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e9c6b8470 0x564e9c6bc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.307 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.297+0000 7f7c2e59c700 1 --2- 172.21.15.1:0/3518421136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e9c6b7b60 0x564e9c6b7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.307 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.297+0000 7f7c2ed9d700 1 -- 172.21.15.1:0/3518421136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e9c6b8470 msgr2=0x564e9c6bc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:52.307 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.297+0000 7f7c2ed9d700 1 --2- 172.21.15.1:0/3518421136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e9c6b8470 0x564e9c6bc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.307 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.297+0000 7f7c2ed9d700 1 -- 172.21.15.1:0/3518421136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e9c6b7b60 msgr2=0x564e9c6b7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:52.307 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.297+0000 7f7c2ed9d700 1 --2- 172.21.15.1:0/3518421136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e9c6b7b60 0x564e9c6b7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.307 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.297+0000 7f7c2ed9d700 1 -- 172.21.15.1:0/3518421136 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e9c48a630 con 0x564e9c6bc8b0 2023-11-16T16:47:52.307 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.297+0000 7f7c2ed9d700 1 --2- 172.21.15.1:0/3518421136 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e9c6bc8b0 0x564e9c6bcd00 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f7c28009d20 tx=0x7f7c280077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95db23daf4b3d390 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:52.307 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.297+0000 7f7c1cff9700 1 -- 172.21.15.1:0/3518421136 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c28014600 con 0x564e9c6bc8b0 2023-11-16T16:47:52.308 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.297+0000 7f7c1cff9700 1 -- 172.21.15.1:0/3518421136 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c28014c20 con 0x564e9c6bc8b0 2023-11-16T16:47:52.308 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.297+0000 7f7c1cff9700 1 -- 172.21.15.1:0/3518421136 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c28013ad0 con 0x564e9c6bc8b0 2023-11-16T16:47:52.308 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.298+0000 7f7c40997b80 1 -- 172.21.15.1:0/3518421136 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e9c6bc8b0 msgr2=0x564e9c6bcd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:52.308 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.298+0000 7f7c40997b80 1 --2- 172.21.15.1:0/3518421136 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e9c6bc8b0 0x564e9c6bcd00 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f7c28009d20 tx=0x7f7c280077e0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.308 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.298+0000 7f7c40997b80 1 -- 172.21.15.1:0/3518421136 shutdown_connections 2023-11-16T16:47:52.308 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.298+0000 7f7c40997b80 1 --2- 172.21.15.1:0/3518421136 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e9c6bc8b0 0x564e9c6bcd00 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.308 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.298+0000 7f7c40997b80 1 --2- 172.21.15.1:0/3518421136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e9c6b7b60 0x564e9c6b7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.308 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.298+0000 7f7c40997b80 1 --2- 172.21.15.1:0/3518421136 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e9c6b8470 0x564e9c6bc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.309 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.298+0000 7f7c40997b80 1 -- 172.21.15.1:0/3518421136 >> 172.21.15.1:0/3518421136 conn(0x564e9c6ad360 msgr2=0x564e9c6b2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:52.309 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.298+0000 7f7c40997b80 1 -- 172.21.15.1:0/3518421136 shutdown_connections 2023-11-16T16:47:52.309 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.298+0000 7f7c40997b80 1 -- 172.21.15.1:0/3518421136 wait complete. 2023-11-16T16:47:52.309 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.298+0000 7f7c40997b80 1 -- start start 2023-11-16T16:47:52.309 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.299+0000 7f7c40997b80 1 Processor -- start 2023-11-16T16:47:52.309 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.299+0000 7f7c40997b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e9c6b7b60 0x564e9c6c4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.309 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.299+0000 7f7c40997b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e9c6b8470 0x564e9c6c12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.310 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.299+0000 7f7c40997b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e9c6bc8b0 0x564e9c6c1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.310 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.299+0000 7f7c40997b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e9c5aa8e0 con 0x564e9c6bc8b0 2023-11-16T16:47:52.310 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.299+0000 7f7c40997b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e9c5aa580 con 0x564e9c6b7b60 2023-11-16T16:47:52.310 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.299+0000 7f7c40997b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e9c48c920 con 0x564e9c6b8470 2023-11-16T16:47:52.310 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.299+0000 7f7c40997b80 10 client.0 _ll_register_callbacks cb 0x564e9c5c8780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:47:52.310 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.299+0000 7f7c2ed9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e9c6bc8b0 0x564e9c6c1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.310 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.299+0000 7f7c2ed9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e9c6bc8b0 0x564e9c6c1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46478/0 (socket says 172.21.15.1:46478) 2023-11-16T16:47:52.311 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.299+0000 7f7c2ed9d700 1 -- 172.21.15.1:0/4285109641 learned_addr learned my addr 172.21.15.1:0/4285109641 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:52.311 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.299+0000 7f7c2e59c700 1 --2- 172.21.15.1:0/4285109641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e9c6b7b60 0x564e9c6c4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.311 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.299+0000 7f7c2dd9b700 1 --2- 172.21.15.1:0/4285109641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e9c6b8470 0x564e9c6c12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.311 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.299+0000 7f7c2ed9d700 1 -- 172.21.15.1:0/4285109641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e9c6b8470 msgr2=0x564e9c6c12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:52.311 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.299+0000 7f7c2ed9d700 1 --2- 172.21.15.1:0/4285109641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e9c6b8470 0x564e9c6c12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.311 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.299+0000 7f7c2ed9d700 1 -- 172.21.15.1:0/4285109641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e9c6b7b60 msgr2=0x564e9c6c4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:52.311 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.299+0000 7f7c2ed9d700 1 --2- 172.21.15.1:0/4285109641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e9c6b7b60 0x564e9c6c4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.311 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.299+0000 7f7c2ed9d700 1 -- 172.21.15.1:0/4285109641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c28009980 con 0x564e9c6bc8b0 2023-11-16T16:47:52.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.299+0000 7f7c2ed9d700 1 --2- 172.21.15.1:0/4285109641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e9c6bc8b0 0x564e9c6c1940 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f7c28007be0 tx=0x7f7c28004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc6751469481817 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:52.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.299+0000 7f7c15ffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:47:52.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.300+0000 7f7c15ffb700 1 -- 172.21.15.1:0/4285109641 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c28014810 con 0x564e9c6bc8b0 2023-11-16T16:47:52.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.300+0000 7f7c15ffb700 1 -- 172.21.15.1:0/4285109641 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f7c28014970 con 0x564e9c6bc8b0 2023-11-16T16:47:52.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.300+0000 7f7c40997b80 1 -- 172.21.15.1:0/4285109641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564e9c6aab20 con 0x564e9c6bc8b0 2023-11-16T16:47:52.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.300+0000 7f7c15ffb700 1 -- 172.21.15.1:0/4285109641 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c280136d0 con 0x564e9c6bc8b0 2023-11-16T16:47:52.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.300+0000 7f7c40997b80 10 client.16686 fetch_fsmap learned FSMap version 28 2023-11-16T16:47:52.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.300+0000 7f7c40997b80 1 -- 172.21.15.1:0/4285109641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564e9c5cf190 con 0x564e9c6bc8b0 2023-11-16T16:47:52.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.300+0000 7f7c15ffb700 1 -- 172.21.15.1:0/4285109641 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564e9c6aab20 con 0x564e9c6bc8b0 2023-11-16T16:47:52.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.300+0000 7f7c15ffb700 1 -- 172.21.15.1:0/4285109641 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f7c28005500 con 0x564e9c6bc8b0 2023-11-16T16:47:52.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.300+0000 7f7c40997b80 10 client.16686 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:47:52.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.300+0000 7f7c40997b80 10 client.16686 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-11-16T16:47:52.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.300+0000 7f7c40997b80 20 client.16686 populate_metadata read hostname 'smithi001' 2023-11-16T16:47:52.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.300+0000 7f7c40997b80 1 --2- 172.21.15.1:0/4285109641 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x564e9c6d1830 0x564e9c6d3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.300+0000 7f7c40997b80 4 client.16686 mds_command: new command op to 24482 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:47:52.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.300+0000 7f7c40997b80 1 -- 172.21.15.1:0/4285109641 --> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e9c5cb0a0 con 0x564e9c6d1830 2023-11-16T16:47:52.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.300+0000 7f7c2e59c700 1 --2- 172.21.15.1:0/4285109641 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x564e9c6d1830 0x564e9c6d3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.301+0000 7f7c15ffb700 10 client.16686 ms_handle_connect on v2:172.21.15.26:6836/1453504863 2023-11-16T16:47:52.314 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.301+0000 7f7c2e59c700 1 --2- 172.21.15.1:0/4285109641 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x564e9c6d1830 0x564e9c6d3c40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=70ad3333b345e335 server_cookie=5de493761b89aaaa in_seq=0 out_seq=0 2023-11-16T16:47:52.314 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.302+0000 7f7c15ffb700 1 -- 172.21.15.1:0/4285109641 <== mds.2 v2:172.21.15.26:6836/1453504863 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564e9c5cb0a0 con 0x564e9c6d1830 2023-11-16T16:47:52.314 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.302+0000 7f7c15ffb700 10 client.16686 handle_command_reply: tid=0 2023-11-16T16:47:52.314 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.302+0000 7f7c40997b80 1 client.16686 shutdown 2023-11-16T16:47:52.314 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.302+0000 7f7c40997b80 1 -- 172.21.15.1:0/4285109641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e9c6bc8b0 msgr2=0x564e9c6c1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:52.314 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.302+0000 7f7c40997b80 1 --2- 172.21.15.1:0/4285109641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e9c6bc8b0 0x564e9c6c1940 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f7c28007be0 tx=0x7f7c28004f90 comp rx=0 tx=0).stop 2023-11-16T16:47:52.314 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.302+0000 7f7c40997b80 1 -- 172.21.15.1:0/4285109641 shutdown_connections 2023-11-16T16:47:52.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.302+0000 7f7c40997b80 1 --2- 172.21.15.1:0/4285109641 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x564e9c6d1830 0x564e9c6d3c40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.302+0000 7f7c40997b80 1 --2- 172.21.15.1:0/4285109641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e9c6bc8b0 0x564e9c6c1940 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.302+0000 7f7c40997b80 1 --2- 172.21.15.1:0/4285109641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e9c6b7b60 0x564e9c6c4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.302+0000 7f7c15ffb700 0 client.16686 ms_handle_reset on v2:172.21.15.26:6836/1453504863 2023-11-16T16:47:52.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.302+0000 7f7c40997b80 1 --2- 172.21.15.1:0/4285109641 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e9c6b8470 0x564e9c6c12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.302+0000 7f7c40997b80 1 -- 172.21.15.1:0/4285109641 >> 172.21.15.1:0/4285109641 conn(0x564e9c6ad360 msgr2=0x564e9c6bdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:52.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.302+0000 7f7c40997b80 1 -- 172.21.15.1:0/4285109641 shutdown_connections 2023-11-16T16:47:52.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.302+0000 7f7c40997b80 1 -- 172.21.15.1:0/4285109641 wait complete. 2023-11-16T16:47:52.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.302+0000 7f7c40997b80 20 client.16686 trim_cache size 0 max 16384 2023-11-16T16:47:52.318 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.315+0000 7f7c40997b80 1 Processor -- start 2023-11-16T16:47:52.318 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.315+0000 7f7c40997b80 1 -- start start 2023-11-16T16:47:52.318 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.315+0000 7f7c40997b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e9c6b7b60 0x564e9c6cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.318 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.315+0000 7f7c40997b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e9c6b8470 0x564e9c6c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.318 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.315+0000 7f7c40997b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e9c6bc8b0 0x564e9c6b2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.318 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.315+0000 7f7c40997b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e9c6c47b0 con 0x564e9c6b8470 2023-11-16T16:47:52.319 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.316+0000 7f7c40997b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e9c6acf10 con 0x564e9c6bc8b0 2023-11-16T16:47:52.319 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.316+0000 7f7c40997b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e9c6c26b0 con 0x564e9c6b7b60 2023-11-16T16:47:52.319 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.316+0000 7f7c2dd9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e9c6b8470 0x564e9c6c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.319 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.316+0000 7f7c2dd9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e9c6b8470 0x564e9c6c3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46484/0 (socket says 172.21.15.1:46484) 2023-11-16T16:47:52.319 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.316+0000 7f7c2dd9b700 1 -- 172.21.15.1:0/3424738924 learned_addr learned my addr 172.21.15.1:0/3424738924 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:52.319 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.316+0000 7f7c2e59c700 1 --2- 172.21.15.1:0/3424738924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e9c6b7b60 0x564e9c6cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.319 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.316+0000 7f7c2dd9b700 1 -- 172.21.15.1:0/3424738924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e9c6b7b60 msgr2=0x564e9c6cdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:52.320 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.316+0000 7f7c2dd9b700 1 --2- 172.21.15.1:0/3424738924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e9c6b7b60 0x564e9c6cdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.320 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.316+0000 7f7c2dd9b700 1 -- 172.21.15.1:0/3424738924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e9c6bc8b0 msgr2=0x564e9c6b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:52.320 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.316+0000 7f7c2dd9b700 1 --2- 172.21.15.1:0/3424738924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e9c6bc8b0 0x564e9c6b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.320 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.316+0000 7f7c2dd9b700 1 -- 172.21.15.1:0/3424738924 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e9c6aeb00 con 0x564e9c6b8470 2023-11-16T16:47:52.320 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.316+0000 7f7c2e59c700 1 --2- 172.21.15.1:0/3424738924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e9c6b7b60 0x564e9c6cdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:47:52.320 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.316+0000 7f7c2dd9b700 1 --2- 172.21.15.1:0/3424738924 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e9c6b8470 0x564e9c6c3be0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f7c20009990 tx=0x7f7c20005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59600d727ca6dcdf server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:52.320 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.316+0000 7f7c1d7fa700 1 -- 172.21.15.1:0/3424738924 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c20005270 con 0x564e9c6b8470 2023-11-16T16:47:52.320 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.316+0000 7f7c1d7fa700 1 -- 172.21.15.1:0/3424738924 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c2002b070 con 0x564e9c6b8470 2023-11-16T16:47:52.320 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.316+0000 7f7c1d7fa700 1 -- 172.21.15.1:0/3424738924 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c20023db0 con 0x564e9c6b8470 2023-11-16T16:47:52.324 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.317+0000 7f7c40997b80 1 -- 172.21.15.1:0/3424738924 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e9c6b8470 msgr2=0x564e9c6c3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:52.324 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.317+0000 7f7c40997b80 1 --2- 172.21.15.1:0/3424738924 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e9c6b8470 0x564e9c6c3be0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f7c20009990 tx=0x7f7c20005000 comp rx=0 tx=0).stop 2023-11-16T16:47:52.324 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.317+0000 7f7c40997b80 1 -- 172.21.15.1:0/3424738924 shutdown_connections 2023-11-16T16:47:52.324 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.317+0000 7f7c40997b80 1 --2- 172.21.15.1:0/3424738924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e9c6bc8b0 0x564e9c6b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.325 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.317+0000 7f7c40997b80 1 --2- 172.21.15.1:0/3424738924 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e9c6b7b60 0x564e9c6cdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.325 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.317+0000 7f7c40997b80 1 --2- 172.21.15.1:0/3424738924 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e9c6b8470 0x564e9c6c3be0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.325 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.317+0000 7f7c40997b80 1 -- 172.21.15.1:0/3424738924 >> 172.21.15.1:0/3424738924 conn(0x564e9c6ad360 msgr2=0x564e9c6bec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:52.325 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.317+0000 7f7c40997b80 1 -- 172.21.15.1:0/3424738924 shutdown_connections 2023-11-16T16:47:52.325 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.317+0000 7f7c40997b80 1 -- 172.21.15.1:0/3424738924 wait complete. 2023-11-16T16:47:52.325 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.317+0000 7f7c40997b80 1 -- start start 2023-11-16T16:47:52.325 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.317+0000 7f7c40997b80 1 Processor -- start 2023-11-16T16:47:52.325 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.317+0000 7f7c40997b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e9c6b7b60 0x564e9c721440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.326 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.317+0000 7f7c40997b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e9c6b8470 0x564e9c721980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.326 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.317+0000 7f7c40997b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e9c6bc8b0 0x564e9c6c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.326 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.318+0000 7f7c40997b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e9c6c8700 con 0x564e9c6bc8b0 2023-11-16T16:47:52.326 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.318+0000 7f7c40997b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e9c6c8870 con 0x564e9c6b8470 2023-11-16T16:47:52.326 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.318+0000 7f7c40997b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e9c6c8a30 con 0x564e9c6b7b60 2023-11-16T16:47:52.326 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.318+0000 7f7c40997b80 10 client.0 _ll_register_callbacks cb 0x7f7c28009ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:47:52.326 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.318+0000 7f7c2ed9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e9c6bc8b0 0x564e9c6c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.326 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.318+0000 7f7c2ed9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e9c6bc8b0 0x564e9c6c7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46492/0 (socket says 172.21.15.1:46492) 2023-11-16T16:47:52.326 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.318+0000 7f7c2ed9d700 1 -- 172.21.15.1:0/2223993226 learned_addr learned my addr 172.21.15.1:0/2223993226 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:52.327 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.318+0000 7f7c2dd9b700 1 --2- 172.21.15.1:0/2223993226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e9c6b8470 0x564e9c721980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.327 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.318+0000 7f7c2e59c700 1 --2- 172.21.15.1:0/2223993226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e9c6b7b60 0x564e9c721440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.327 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.318+0000 7f7c2ed9d700 1 -- 172.21.15.1:0/2223993226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e9c6b7b60 msgr2=0x564e9c721440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:52.327 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.318+0000 7f7c2ed9d700 1 --2- 172.21.15.1:0/2223993226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e9c6b7b60 0x564e9c721440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.327 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.318+0000 7f7c2ed9d700 1 -- 172.21.15.1:0/2223993226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e9c6b8470 msgr2=0x564e9c721980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:52.327 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.318+0000 7f7c2ed9d700 1 --2- 172.21.15.1:0/2223993226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e9c6b8470 0x564e9c721980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.327 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.318+0000 7f7c2ed9d700 1 -- 172.21.15.1:0/2223993226 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c3010da20 con 0x564e9c6bc8b0 2023-11-16T16:47:52.327 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.318+0000 7f7c2ed9d700 1 --2- 172.21.15.1:0/2223993226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e9c6bc8b0 0x564e9c6c7fc0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f7c28009220 tx=0x7f7c28014970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24314d5eb0d8613d server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:52.328 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.318+0000 7f7c15ffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:47:52.328 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.318+0000 7f7c15ffb700 1 -- 172.21.15.1:0/2223993226 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c28007c10 con 0x564e9c6bc8b0 2023-11-16T16:47:52.328 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.318+0000 7f7c15ffb700 1 -- 172.21.15.1:0/2223993226 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f7c28005130 con 0x564e9c6bc8b0 2023-11-16T16:47:52.328 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.318+0000 7f7c40997b80 1 -- 172.21.15.1:0/2223993226 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c28014c20 con 0x564e9c6bc8b0 2023-11-16T16:47:52.328 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.318+0000 7f7c15ffb700 1 -- 172.21.15.1:0/2223993226 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c280052f0 con 0x564e9c6bc8b0 2023-11-16T16:47:52.328 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.318+0000 7f7c40997b80 10 client.16692 fetch_fsmap learned FSMap version 28 2023-11-16T16:47:52.328 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.318+0000 7f7c40997b80 1 -- 172.21.15.1:0/2223993226 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564e9c436de0 con 0x564e9c6bc8b0 2023-11-16T16:47:52.328 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.319+0000 7f7c15ffb700 1 -- 172.21.15.1:0/2223993226 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c28014c20 con 0x564e9c6bc8b0 2023-11-16T16:47:52.329 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.319+0000 7f7c15ffb700 1 -- 172.21.15.1:0/2223993226 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f7c28013b40 con 0x564e9c6bc8b0 2023-11-16T16:47:52.329 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.319+0000 7f7c40997b80 10 client.16692 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:47:52.329 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.319+0000 7f7c40997b80 10 client.16692 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-11-16T16:47:52.329 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.319+0000 7f7c40997b80 20 client.16692 populate_metadata read hostname 'smithi001' 2023-11-16T16:47:52.329 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.319+0000 7f7c40997b80 1 --2- 172.21.15.1:0/2223993226 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x564e9c729e90 0x564e9c72c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.329 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.319+0000 7f7c40997b80 4 client.16692 mds_command: new command op to 24482 tid=0[{"prefix": "damage ls"}] 2023-11-16T16:47:52.329 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.319+0000 7f7c40997b80 1 -- 172.21.15.1:0/2223993226 --> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564e9c5d7660 con 0x564e9c729e90 2023-11-16T16:47:52.329 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.319+0000 7f7c2e59c700 1 --2- 172.21.15.1:0/2223993226 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x564e9c729e90 0x564e9c72c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.330 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.320+0000 7f7c15ffb700 10 client.16692 ms_handle_connect on v2:172.21.15.26:6836/1453504863 2023-11-16T16:47:52.330 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.320+0000 7f7c2e59c700 1 --2- 172.21.15.1:0/2223993226 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x564e9c729e90 0x564e9c72c270 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=51f646f3b780778e server_cookie=beba15405aaec29 in_seq=0 out_seq=0 2023-11-16T16:47:52.330 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.320+0000 7f7c15ffb700 1 -- 172.21.15.1:0/2223993226 <== mds.2 v2:172.21.15.26:6836/1453504863 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564e9c5d7660 con 0x564e9c729e90 2023-11-16T16:47:52.330 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.320+0000 7f7c15ffb700 10 client.16692 handle_command_reply: tid=0 2023-11-16T16:47:52.330 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.320+0000 7f7c40997b80 1 client.16692 shutdown 2023-11-16T16:47:52.330 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.320+0000 7f7c40997b80 1 -- 172.21.15.1:0/2223993226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e9c6bc8b0 msgr2=0x564e9c6c7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:52.330 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.320+0000 7f7c40997b80 1 --2- 172.21.15.1:0/2223993226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e9c6bc8b0 0x564e9c6c7fc0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f7c28009220 tx=0x7f7c28014970 comp rx=0 tx=0).stop 2023-11-16T16:47:52.330 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.320+0000 7f7c40997b80 1 -- 172.21.15.1:0/2223993226 shutdown_connections 2023-11-16T16:47:52.331 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.320+0000 7f7c40997b80 1 --2- 172.21.15.1:0/2223993226 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x564e9c729e90 0x564e9c72c270 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.331 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.320+0000 7f7c40997b80 1 --2- 172.21.15.1:0/2223993226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e9c6bc8b0 0x564e9c6c7fc0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.331 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.320+0000 7f7c15ffb700 0 client.16692 ms_handle_reset on v2:172.21.15.26:6836/1453504863 2023-11-16T16:47:52.331 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.320+0000 7f7c40997b80 1 --2- 172.21.15.1:0/2223993226 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e9c6b7b60 0x564e9c721440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.331 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.320+0000 7f7c40997b80 1 --2- 172.21.15.1:0/2223993226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e9c6b8470 0x564e9c721980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.331 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.320+0000 7f7c40997b80 1 -- 172.21.15.1:0/2223993226 >> 172.21.15.1:0/2223993226 conn(0x564e9c6ad360 msgr2=0x564e9c6ba8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:52.331 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.320+0000 7f7c40997b80 1 -- 172.21.15.1:0/2223993226 shutdown_connections 2023-11-16T16:47:52.331 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.320+0000 7f7c40997b80 1 -- 172.21.15.1:0/2223993226 wait complete. 2023-11-16T16:47:52.331 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.320+0000 7f7c40997b80 20 client.16692 trim_cache size 0 max 16384 2023-11-16T16:47:52.332 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2023-11-16T16:47:52.332 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.321+0000 7f7c3480b700 1 -- 172.21.15.1:0/2456780318 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f7c180a12c0 msgr2=0x7f7c180a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:52.333 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.321+0000 7f7c3480b700 1 --2- 172.21.15.1:0/2456780318 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f7c180a12c0 0x7f7c180a3770 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7c30197990 tx=0x7f7c20005fd0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.333 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.321+0000 7f7c3480b700 1 -- 172.21.15.1:0/2456780318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c300ff490 msgr2=0x7f7c30196380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:52.333 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.321+0000 7f7c3480b700 1 --2- 172.21.15.1:0/2456780318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c300ff490 0x7f7c30196380 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f7c2400d950 tx=0x7f7c2400dc60 comp rx=0 tx=0).stop 2023-11-16T16:47:52.333 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.321+0000 7f7c3480b700 1 -- 172.21.15.1:0/2456780318 shutdown_connections 2023-11-16T16:47:52.333 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.321+0000 7f7c3480b700 1 --2- 172.21.15.1:0/2456780318 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f7c180a12c0 0x7f7c180a3770 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.333 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.321+0000 7f7c3480b700 1 --2- 172.21.15.1:0/2456780318 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7c3010ad50 0x7f7c3019ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.333 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.321+0000 7f7c3480b700 1 --2- 172.21.15.1:0/2456780318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c300ff490 0x7f7c30196380 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.333 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.321+0000 7f7c3480b700 1 --2- 172.21.15.1:0/2456780318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c300ffda0 0x7f7c301968c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.334 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.321+0000 7f7c3480b700 1 -- 172.21.15.1:0/2456780318 >> 172.21.15.1:0/2456780318 conn(0x7f7c30074ac0 msgr2=0x7f7c300fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:52.334 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.321+0000 7f7c3480b700 1 -- 172.21.15.1:0/2456780318 shutdown_connections 2023-11-16T16:47:52.334 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.321+0000 7f7c3480b700 1 -- 172.21.15.1:0/2456780318 wait complete. 2023-11-16T16:47:52.335 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-11-16T16:47:52.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:52 smithi001 ceph-mon[98800]: pgmap v268: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:47:52.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:52 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/2208290835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:47:52.462 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.460+0000 7f0bc7705700 1 -- 172.21.15.1:0/677969547 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bc00ffa70 msgr2=0x7f0bc00ffee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:52.462 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.460+0000 7f0bc7705700 1 --2- 172.21.15.1:0/677969547 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bc00ffa70 0x7f0bc00ffee0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0009990 tx=0x7f0bb0009ca0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.462 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.460+0000 7f0bc7705700 1 -- 172.21.15.1:0/677969547 shutdown_connections 2023-11-16T16:47:52.462 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.460+0000 7f0bc7705700 1 --2- 172.21.15.1:0/677969547 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bc0100420 0x7f0bc010bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.462 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.460+0000 7f0bc7705700 1 --2- 172.21.15.1:0/677969547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bc0105250 0x7f0bc0105620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.463 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.460+0000 7f0bc7705700 1 --2- 172.21.15.1:0/677969547 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bc00ffa70 0x7f0bc00ffee0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.463 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.460+0000 7f0bc7705700 1 -- 172.21.15.1:0/677969547 >> 172.21.15.1:0/677969547 conn(0x7f0bc00763a0 msgr2=0x7f0bc00767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:52.463 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.460+0000 7f0bc7705700 1 -- 172.21.15.1:0/677969547 shutdown_connections 2023-11-16T16:47:52.463 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.460+0000 7f0bc7705700 1 -- 172.21.15.1:0/677969547 wait complete. 2023-11-16T16:47:52.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.461+0000 7f0bc7705700 1 Processor -- start 2023-11-16T16:47:52.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.461+0000 7f0bc7705700 1 -- start start 2023-11-16T16:47:52.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.461+0000 7f0bc7705700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bc00ffa70 0x7f0bc0197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.461+0000 7f0bc7705700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bc0100420 0x7f0bc0197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.461+0000 7f0bc7705700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bc0105250 0x7f0bc01a2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.461+0000 7f0bc7705700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc0198080 con 0x7f0bc0105250 2023-11-16T16:47:52.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.461+0000 7f0bc7705700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc01a27c0 con 0x7f0bc0100420 2023-11-16T16:47:52.466 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.461+0000 7f0bc7705700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc01a2980 con 0x7f0bc00ffa70 2023-11-16T16:47:52.466 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.461+0000 7f0bc5ca2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bc0105250 0x7f0bc01a2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.466 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.461+0000 7f0bc5ca2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bc0105250 0x7f0bc01a2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46506/0 (socket says 172.21.15.1:46506) 2023-11-16T16:47:52.466 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.461+0000 7f0bc5ca2700 1 -- 172.21.15.1:0/595062241 learned_addr learned my addr 172.21.15.1:0/595062241 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:52.466 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.461+0000 7f0bc4ca0700 1 --2- 172.21.15.1:0/595062241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bc0100420 0x7f0bc0197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.466 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.461+0000 7f0bc54a1700 1 --2- 172.21.15.1:0/595062241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bc00ffa70 0x7f0bc0197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.466 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.461+0000 7f0bc5ca2700 1 -- 172.21.15.1:0/595062241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bc00ffa70 msgr2=0x7f0bc0197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:52.467 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.462+0000 7f0bc5ca2700 1 --2- 172.21.15.1:0/595062241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bc00ffa70 0x7f0bc0197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.467 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.462+0000 7f0bc5ca2700 1 -- 172.21.15.1:0/595062241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bc0100420 msgr2=0x7f0bc0197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:52.467 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.462+0000 7f0bc5ca2700 1 --2- 172.21.15.1:0/595062241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bc0100420 0x7f0bc0197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.467 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.462+0000 7f0bc5ca2700 1 -- 172.21.15.1:0/595062241 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb00095f0 con 0x7f0bc0105250 2023-11-16T16:47:52.467 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.462+0000 7f0bc5ca2700 1 --2- 172.21.15.1:0/595062241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bc0105250 0x7f0bc01a2280 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f0bb800ef30 tx=0x7f0bb800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:52.467 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.462+0000 7f0bb67fc700 1 -- 172.21.15.1:0/595062241 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8018070 con 0x7f0bc0105250 2023-11-16T16:47:52.467 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.462+0000 7f0bb67fc700 1 -- 172.21.15.1:0/595062241 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bb800f040 con 0x7f0bc0105250 2023-11-16T16:47:52.467 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.462+0000 7f0bb67fc700 1 -- 172.21.15.1:0/595062241 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb800cd00 con 0x7f0bc0105250 2023-11-16T16:47:52.467 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.462+0000 7f0bc7705700 1 -- 172.21.15.1:0/595062241 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bc010f1e0 con 0x7f0bc0105250 2023-11-16T16:47:52.468 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.462+0000 7f0bc7705700 1 -- 172.21.15.1:0/595062241 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc010f050 con 0x7f0bc0105250 2023-11-16T16:47:52.473 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.464+0000 7f0bb67fc700 1 -- 172.21.15.1:0/595062241 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f0bb8003a00 con 0x7f0bc0105250 2023-11-16T16:47:52.473 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.465+0000 7f0bb67fc700 1 --2- 172.21.15.1:0/595062241 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f0bac0a1340 0x7f0bac0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.473 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.465+0000 7f0bb67fc700 1 -- 172.21.15.1:0/595062241 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f0bb80cc7f0 con 0x7f0bc0105250 2023-11-16T16:47:52.473 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.465+0000 7f0bc54a1700 1 --2- 172.21.15.1:0/595062241 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f0bac0a1340 0x7f0bac0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.473 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.465+0000 7f0bc54a1700 1 --2- 172.21.15.1:0/595062241 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f0bac0a1340 0x7f0bac0a37f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009be0 tx=0x7f0bbc009260 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:52.473 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.466+0000 7f0bd3891b80 1 Processor -- start 2023-11-16T16:47:52.474 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.466+0000 7f0bd3891b80 1 -- start start 2023-11-16T16:47:52.474 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.466+0000 7f0bd3891b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56413d631f70 0x56413d632340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.474 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.466+0000 7f0bd3891b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56413d632880 0x56413d636780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.474 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.466+0000 7f0bd3891b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56413d636cc0 0x56413d637110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.474 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.466+0000 7f0bd3891b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56413d533060 con 0x56413d636cc0 2023-11-16T16:47:52.474 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.466+0000 7f0bd3891b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56413d5858a0 con 0x56413d631f70 2023-11-16T16:47:52.474 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.466+0000 7f0bd3891b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56413d57ed30 con 0x56413d632880 2023-11-16T16:47:52.474 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.466+0000 7f0bc5ca2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56413d636cc0 0x56413d637110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.475 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.466+0000 7f0bc4ca0700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56413d632880 0x56413d636780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.475 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.466+0000 7f0bc5ca2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56413d636cc0 0x56413d637110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46514/0 (socket says 172.21.15.1:46514) 2023-11-16T16:47:52.475 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.466+0000 7f0bc5ca2700 1 -- 172.21.15.1:0/4265707626 learned_addr learned my addr 172.21.15.1:0/4265707626 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:52.475 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.466+0000 7f0bc54a1700 1 --2- 172.21.15.1:0/4265707626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56413d631f70 0x56413d632340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.475 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.466+0000 7f0bc5ca2700 1 -- 172.21.15.1:0/4265707626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56413d632880 msgr2=0x56413d636780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:52.475 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.466+0000 7f0bc5ca2700 1 --2- 172.21.15.1:0/4265707626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56413d632880 0x56413d636780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.475 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.466+0000 7f0bc5ca2700 1 -- 172.21.15.1:0/4265707626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56413d631f70 msgr2=0x56413d632340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:52.475 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.466+0000 7f0bc5ca2700 1 --2- 172.21.15.1:0/4265707626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56413d631f70 0x56413d632340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.466+0000 7f0bc5ca2700 1 -- 172.21.15.1:0/4265707626 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56413d400630 con 0x56413d636cc0 2023-11-16T16:47:52.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.467+0000 7f0bc5ca2700 1 --2- 172.21.15.1:0/4265707626 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56413d636cc0 0x56413d637110 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f0bb80090a0 tx=0x7f0bb8099500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da0428e52624b2f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:52.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.467+0000 7f0bab7fe700 1 -- 172.21.15.1:0/4265707626 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8004ad0 con 0x56413d636cc0 2023-11-16T16:47:52.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.467+0000 7f0bab7fe700 1 -- 172.21.15.1:0/4265707626 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bb8084040 con 0x56413d636cc0 2023-11-16T16:47:52.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.467+0000 7f0bab7fe700 1 -- 172.21.15.1:0/4265707626 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8099e00 con 0x56413d636cc0 2023-11-16T16:47:52.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.467+0000 7f0bd3891b80 1 -- 172.21.15.1:0/4265707626 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56413d636cc0 msgr2=0x56413d637110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:52.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.467+0000 7f0bd3891b80 1 --2- 172.21.15.1:0/4265707626 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56413d636cc0 0x56413d637110 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f0bb80090a0 tx=0x7f0bb8099500 comp rx=0 tx=0).stop 2023-11-16T16:47:52.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.467+0000 7f0bd3891b80 1 -- 172.21.15.1:0/4265707626 shutdown_connections 2023-11-16T16:47:52.476 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.467+0000 7f0bd3891b80 1 --2- 172.21.15.1:0/4265707626 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56413d636cc0 0x56413d637110 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.477 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.467+0000 7f0bd3891b80 1 --2- 172.21.15.1:0/4265707626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56413d631f70 0x56413d632340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.477 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.467+0000 7f0bd3891b80 1 --2- 172.21.15.1:0/4265707626 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56413d632880 0x56413d636780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.477 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.467+0000 7f0bd3891b80 1 -- 172.21.15.1:0/4265707626 >> 172.21.15.1:0/4265707626 conn(0x56413d623360 msgr2=0x56413d628b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:52.477 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.467+0000 7f0bd3891b80 1 -- 172.21.15.1:0/4265707626 shutdown_connections 2023-11-16T16:47:52.477 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.468+0000 7f0bd3891b80 1 -- 172.21.15.1:0/4265707626 wait complete. 2023-11-16T16:47:52.477 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.468+0000 7f0bd3891b80 1 -- start start 2023-11-16T16:47:52.477 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.468+0000 7f0bd3891b80 1 Processor -- start 2023-11-16T16:47:52.477 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.468+0000 7f0bd3891b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56413d631f70 0x56413d63e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.478 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.468+0000 7f0bd3891b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56413d632880 0x56413d63b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.478 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.468+0000 7f0bd3891b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56413d636cc0 0x56413d63be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.478 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.468+0000 7f0bd3891b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56413d5208e0 con 0x56413d632880 2023-11-16T16:47:52.478 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.468+0000 7f0bd3891b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56413d520580 con 0x56413d631f70 2023-11-16T16:47:52.478 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.468+0000 7f0bd3891b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56413d402920 con 0x56413d636cc0 2023-11-16T16:47:52.478 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.468+0000 7f0bd3891b80 10 client.0 _ll_register_callbacks cb 0x56413d53e780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:47:52.478 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.468+0000 7f0bc4ca0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56413d632880 0x56413d63b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.478 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.469+0000 7f0bc4ca0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56413d632880 0x56413d63b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46518/0 (socket says 172.21.15.1:46518) 2023-11-16T16:47:52.479 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.469+0000 7f0bc4ca0700 1 -- 172.21.15.1:0/94178212 learned_addr learned my addr 172.21.15.1:0/94178212 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:52.479 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.469+0000 7f0bc54a1700 1 --2- 172.21.15.1:0/94178212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56413d631f70 0x56413d63e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.479 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.469+0000 7f0bc5ca2700 1 --2- 172.21.15.1:0/94178212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56413d636cc0 0x56413d63be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.479 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.469+0000 7f0bc54a1700 1 -- 172.21.15.1:0/94178212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56413d636cc0 msgr2=0x56413d63be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:52.479 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.469+0000 7f0bc54a1700 1 --2- 172.21.15.1:0/94178212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56413d636cc0 0x56413d63be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.479 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.469+0000 7f0bc54a1700 1 -- 172.21.15.1:0/94178212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56413d632880 msgr2=0x56413d63b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:52.479 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.469+0000 7f0bc54a1700 1 --2- 172.21.15.1:0/94178212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56413d632880 0x56413d63b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.479 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.469+0000 7f0bc54a1700 1 -- 172.21.15.1:0/94178212 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc010f050 con 0x56413d631f70 2023-11-16T16:47:52.479 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.469+0000 7f0bc4ca0700 1 --2- 172.21.15.1:0/94178212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56413d632880 0x56413d63b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:47:52.480 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.469+0000 7f0bc5ca2700 1 --2- 172.21.15.1:0/94178212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56413d636cc0 0x56413d63be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:47:52.480 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.469+0000 7f0bc54a1700 1 --2- 172.21.15.1:0/94178212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56413d631f70 0x56413d63e490 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc0117d0 tx=0x7f0bbc0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99aed710e49c12b server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:52.480 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.469+0000 7f0ba8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-11-16T16:47:52.480 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.469+0000 7f0ba8ff9700 1 -- 172.21.15.1:0/94178212 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc004020 con 0x56413d631f70 2023-11-16T16:47:52.480 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.469+0000 7f0ba8ff9700 1 -- 172.21.15.1:0/94178212 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f0bbc004180 con 0x56413d631f70 2023-11-16T16:47:52.480 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.469+0000 7f0bd3891b80 1 -- 172.21.15.1:0/94178212 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56413d620b20 con 0x56413d631f70 2023-11-16T16:47:52.480 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.470+0000 7f0ba8ff9700 1 -- 172.21.15.1:0/94178212 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bbc004340 con 0x56413d631f70 2023-11-16T16:47:52.480 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.470+0000 7f0bd3891b80 10 client.26168 fetch_fsmap learned FSMap version 28 2023-11-16T16:47:52.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.470+0000 7f0bd3891b80 1 -- 172.21.15.1:0/94178212 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56413d545190 con 0x56413d631f70 2023-11-16T16:47:52.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.470+0000 7f0ba8ff9700 1 -- 172.21.15.1:0/94178212 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56413d620b20 con 0x56413d631f70 2023-11-16T16:47:52.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.470+0000 7f0ba8ff9700 1 -- 172.21.15.1:0/94178212 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0bbc028640 con 0x56413d631f70 2023-11-16T16:47:52.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.470+0000 7f0bd3891b80 10 client.26168 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:47:52.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.470+0000 7f0bd3891b80 10 client.26168 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-11-16T16:47:52.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.470+0000 7f0bd3891b80 20 client.26168 populate_metadata read hostname 'smithi001' 2023-11-16T16:47:52.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.470+0000 7f0bd3891b80 1 --2- 172.21.15.1:0/94178212 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x56413d64be10 0x56413d64e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.470+0000 7f0bd3891b80 4 client.26168 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:47:52.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.470+0000 7f0bd3891b80 1 -- 172.21.15.1:0/94178212 --> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56413d5410a0 con 0x56413d64be10 2023-11-16T16:47:52.482 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.470+0000 7f0bc4ca0700 1 --2- 172.21.15.1:0/94178212 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x56413d64be10 0x56413d64e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.482 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.471+0000 7f0bc4ca0700 1 --2- 172.21.15.1:0/94178212 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x56413d64be10 0x56413d64e220 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1bcce59062b3ab70 server_cookie=d694b3b91ef70b12 in_seq=0 out_seq=0 2023-11-16T16:47:52.482 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.471+0000 7f0ba8ff9700 10 client.26168 ms_handle_connect on v2:172.21.15.53:6838/1044753024 2023-11-16T16:47:52.483 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.472+0000 7f0ba8ff9700 1 -- 172.21.15.1:0/94178212 <== mds.1 v2:172.21.15.53:6838/1044753024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56413d5410a0 con 0x56413d64be10 2023-11-16T16:47:52.483 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.472+0000 7f0ba8ff9700 10 client.26168 handle_command_reply: tid=0 2023-11-16T16:47:52.483 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.472+0000 7f0bd3891b80 1 client.26168 shutdown 2023-11-16T16:47:52.483 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.472+0000 7f0bd3891b80 1 -- 172.21.15.1:0/94178212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56413d631f70 msgr2=0x56413d63e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:52.483 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.472+0000 7f0bd3891b80 1 --2- 172.21.15.1:0/94178212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56413d631f70 0x56413d63e490 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc0117d0 tx=0x7f0bbc0047c0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.484 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.472+0000 7f0bd3891b80 1 -- 172.21.15.1:0/94178212 shutdown_connections 2023-11-16T16:47:52.484 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.472+0000 7f0bd3891b80 1 --2- 172.21.15.1:0/94178212 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56413d636cc0 0x56413d63be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.484 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.472+0000 7f0bd3891b80 1 --2- 172.21.15.1:0/94178212 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x56413d64be10 0x56413d64e220 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.484 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.472+0000 7f0bd3891b80 1 --2- 172.21.15.1:0/94178212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56413d631f70 0x56413d63e490 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.484 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.472+0000 7f0bd3891b80 1 --2- 172.21.15.1:0/94178212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56413d632880 0x56413d63b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.484 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.472+0000 7f0ba8ff9700 0 client.26168 ms_handle_reset on v2:172.21.15.53:6838/1044753024 2023-11-16T16:47:52.484 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.472+0000 7f0bd3891b80 1 -- 172.21.15.1:0/94178212 >> 172.21.15.1:0/94178212 conn(0x56413d623360 msgr2=0x56413d638170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:52.485 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.472+0000 7f0bd3891b80 1 -- 172.21.15.1:0/94178212 shutdown_connections 2023-11-16T16:47:52.485 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.472+0000 7f0bd3891b80 1 -- 172.21.15.1:0/94178212 wait complete. 2023-11-16T16:47:52.485 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.472+0000 7f0bd3891b80 20 client.26168 trim_cache size 0 max 16384 2023-11-16T16:47:52.489 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.485+0000 7f0bd3891b80 1 Processor -- start 2023-11-16T16:47:52.489 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.485+0000 7f0bd3891b80 1 -- start start 2023-11-16T16:47:52.489 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.485+0000 7f0bd3891b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56413d631f70 0x56413d642db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.489 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.485+0000 7f0bd3891b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56413d632880 0x56413d63e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.489 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.485+0000 7f0bd3891b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56413d636cc0 0x56413d628580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.489 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.485+0000 7f0bd3891b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56413d639d00 con 0x56413d631f70 2023-11-16T16:47:52.490 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.485+0000 7f0bd3891b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56413d635840 con 0x56413d636cc0 2023-11-16T16:47:52.490 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.485+0000 7f0bd3891b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56413d632d10 con 0x56413d632880 2023-11-16T16:47:52.490 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.486+0000 7f0bc54a1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56413d631f70 0x56413d642db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.490 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.486+0000 7f0bc54a1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56413d631f70 0x56413d642db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46524/0 (socket says 172.21.15.1:46524) 2023-11-16T16:47:52.490 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.486+0000 7f0bc54a1700 1 -- 172.21.15.1:0/3004636632 learned_addr learned my addr 172.21.15.1:0/3004636632 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:52.490 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.486+0000 7f0bc5ca2700 1 --2- 172.21.15.1:0/3004636632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56413d636cc0 0x56413d628580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.490 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.486+0000 7f0bc4ca0700 1 --2- 172.21.15.1:0/3004636632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56413d632880 0x56413d63e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.486+0000 7f0bc54a1700 1 -- 172.21.15.1:0/3004636632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56413d632880 msgr2=0x56413d63e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:52.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.486+0000 7f0bc54a1700 1 --2- 172.21.15.1:0/3004636632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56413d632880 0x56413d63e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.486+0000 7f0bc54a1700 1 -- 172.21.15.1:0/3004636632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56413d636cc0 msgr2=0x56413d628580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:52.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.486+0000 7f0bc54a1700 1 --2- 172.21.15.1:0/3004636632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56413d636cc0 0x56413d628580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.486+0000 7f0bc54a1700 1 -- 172.21.15.1:0/3004636632 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56413d544ee0 con 0x56413d631f70 2023-11-16T16:47:52.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.486+0000 7f0bc54a1700 1 --2- 172.21.15.1:0/3004636632 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56413d631f70 0x56413d642db0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009340 tx=0x7f0bbc009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b0ea5b8b10aa46b server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:52.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.486+0000 7f0babfff700 1 -- 172.21.15.1:0/3004636632 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc028b20 con 0x56413d631f70 2023-11-16T16:47:52.491 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.486+0000 7f0babfff700 1 -- 172.21.15.1:0/3004636632 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bbc023070 con 0x56413d631f70 2023-11-16T16:47:52.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.486+0000 7f0babfff700 1 -- 172.21.15.1:0/3004636632 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc031ce0 con 0x56413d631f70 2023-11-16T16:47:52.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.487+0000 7f0bd3891b80 1 -- 172.21.15.1:0/3004636632 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56413d631f70 msgr2=0x56413d642db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:52.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.487+0000 7f0bd3891b80 1 --2- 172.21.15.1:0/3004636632 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56413d631f70 0x56413d642db0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009340 tx=0x7f0bbc009040 comp rx=0 tx=0).stop 2023-11-16T16:47:52.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.487+0000 7f0bd3891b80 1 -- 172.21.15.1:0/3004636632 shutdown_connections 2023-11-16T16:47:52.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.487+0000 7f0bd3891b80 1 --2- 172.21.15.1:0/3004636632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56413d636cc0 0x56413d628580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.487+0000 7f0bd3891b80 1 --2- 172.21.15.1:0/3004636632 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56413d631f70 0x56413d642db0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.487+0000 7f0bd3891b80 1 --2- 172.21.15.1:0/3004636632 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56413d632880 0x56413d63e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.487+0000 7f0bd3891b80 1 -- 172.21.15.1:0/3004636632 >> 172.21.15.1:0/3004636632 conn(0x56413d623360 msgr2=0x56413d634d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:52.492 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.487+0000 7f0bd3891b80 1 -- 172.21.15.1:0/3004636632 shutdown_connections 2023-11-16T16:47:52.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.487+0000 7f0bd3891b80 1 -- 172.21.15.1:0/3004636632 wait complete. 2023-11-16T16:47:52.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.487+0000 7f0bd3891b80 1 -- start start 2023-11-16T16:47:52.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.487+0000 7f0bd3891b80 1 Processor -- start 2023-11-16T16:47:52.496 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.488+0000 7f0bd3891b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56413d631f70 0x56413d639ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.496 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.488+0000 7f0bd3891b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56413d632880 0x56413d63a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.496 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.488+0000 7f0bd3891b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56413d636cc0 0x56413d63a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.488+0000 7f0bd3891b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56413d63b020 con 0x56413d636cc0 2023-11-16T16:47:52.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.488+0000 7f0bd3891b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56413d63b190 con 0x56413d632880 2023-11-16T16:47:52.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.488+0000 7f0bd3891b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56413d63b350 con 0x56413d631f70 2023-11-16T16:47:52.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.488+0000 7f0bd3891b80 10 client.0 _ll_register_callbacks cb 0x7f0bbc011ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:47:52.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.488+0000 7f0bc5ca2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56413d636cc0 0x56413d63a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.488+0000 7f0bc5ca2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56413d636cc0 0x56413d63a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46534/0 (socket says 172.21.15.1:46534) 2023-11-16T16:47:52.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.488+0000 7f0bc5ca2700 1 -- 172.21.15.1:0/3914615672 learned_addr learned my addr 172.21.15.1:0/3914615672 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:52.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.488+0000 7f0bc4ca0700 1 --2- 172.21.15.1:0/3914615672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56413d632880 0x56413d63a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.488+0000 7f0bc54a1700 1 --2- 172.21.15.1:0/3914615672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56413d631f70 0x56413d639ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.488+0000 7f0bc5ca2700 1 -- 172.21.15.1:0/3914615672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56413d631f70 msgr2=0x56413d639ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:52.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.488+0000 7f0bc5ca2700 1 --2- 172.21.15.1:0/3914615672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56413d631f70 0x56413d639ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.488+0000 7f0bc5ca2700 1 -- 172.21.15.1:0/3914615672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56413d632880 msgr2=0x56413d63a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:52.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.488+0000 7f0bc5ca2700 1 --2- 172.21.15.1:0/3914615672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56413d632880 0x56413d63a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.488+0000 7f0bc5ca2700 1 -- 172.21.15.1:0/3914615672 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56413d545190 con 0x56413d636cc0 2023-11-16T16:47:52.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.488+0000 7f0bc5ca2700 1 --2- 172.21.15.1:0/3914615672 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56413d636cc0 0x56413d63a940 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f0bb800c6f0 tx=0x7f0bb8004c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c44a42bd3da15b77 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:52.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.488+0000 7f0ba8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:47:52.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.489+0000 7f0ba8ff9700 1 -- 172.21.15.1:0/3914615672 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb809d970 con 0x56413d636cc0 2023-11-16T16:47:52.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.489+0000 7f0ba8ff9700 1 -- 172.21.15.1:0/3914615672 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f0bb80a1d00 con 0x56413d636cc0 2023-11-16T16:47:52.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.489+0000 7f0bd3891b80 1 -- 172.21.15.1:0/3914615672 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0bb8084040 con 0x56413d636cc0 2023-11-16T16:47:52.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.489+0000 7f0ba8ff9700 1 -- 172.21.15.1:0/3914615672 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bb8099040 con 0x56413d636cc0 2023-11-16T16:47:52.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.489+0000 7f0bd3891b80 10 client.16710 fetch_fsmap learned FSMap version 28 2023-11-16T16:47:52.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.489+0000 7f0bd3891b80 1 -- 172.21.15.1:0/3914615672 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56413d395c60 con 0x56413d636cc0 2023-11-16T16:47:52.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.489+0000 7f0ba8ff9700 1 -- 172.21.15.1:0/3914615672 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bb8084040 con 0x56413d636cc0 2023-11-16T16:47:52.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.489+0000 7f0ba8ff9700 1 -- 172.21.15.1:0/3914615672 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0bb80c9020 con 0x56413d636cc0 2023-11-16T16:47:52.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.489+0000 7f0bd3891b80 10 client.16710 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:47:52.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.489+0000 7f0bd3891b80 10 client.16710 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-11-16T16:47:52.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.489+0000 7f0bd3891b80 20 client.16710 populate_metadata read hostname 'smithi001' 2023-11-16T16:47:52.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.489+0000 7f0bd3891b80 1 --2- 172.21.15.1:0/3914615672 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x56413d6a0610 0x56413d6a29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.489+0000 7f0bd3891b80 4 client.16710 mds_command: new command op to 24493 tid=0[{"prefix": "damage ls"}] 2023-11-16T16:47:52.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.489+0000 7f0bd3891b80 1 -- 172.21.15.1:0/3914615672 --> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56413d54d660 con 0x56413d6a0610 2023-11-16T16:47:52.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.489+0000 7f0bc54a1700 1 --2- 172.21.15.1:0/3914615672 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x56413d6a0610 0x56413d6a29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.490+0000 7f0ba8ff9700 10 client.16710 ms_handle_connect on v2:172.21.15.53:6838/1044753024 2023-11-16T16:47:52.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.490+0000 7f0bc54a1700 1 --2- 172.21.15.1:0/3914615672 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x56413d6a0610 0x56413d6a29f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5624a2f2a5fa908e server_cookie=ebc575333210bdb8 in_seq=0 out_seq=0 2023-11-16T16:47:52.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.490+0000 7f0ba8ff9700 1 -- 172.21.15.1:0/3914615672 <== mds.1 v2:172.21.15.53:6838/1044753024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56413d54d660 con 0x56413d6a0610 2023-11-16T16:47:52.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.490+0000 7f0ba8ff9700 10 client.16710 handle_command_reply: tid=0 2023-11-16T16:47:52.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.490+0000 7f0bd3891b80 1 client.16710 shutdown 2023-11-16T16:47:52.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.490+0000 7f0bd3891b80 1 -- 172.21.15.1:0/3914615672 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56413d636cc0 msgr2=0x56413d63a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:52.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.490+0000 7f0bd3891b80 1 --2- 172.21.15.1:0/3914615672 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56413d636cc0 0x56413d63a940 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f0bb800c6f0 tx=0x7f0bb8004c10 comp rx=0 tx=0).stop 2023-11-16T16:47:52.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.490+0000 7f0bd3891b80 1 -- 172.21.15.1:0/3914615672 shutdown_connections 2023-11-16T16:47:52.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.490+0000 7f0bd3891b80 1 --2- 172.21.15.1:0/3914615672 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56413d636cc0 0x56413d63a940 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.502 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.490+0000 7f0bd3891b80 1 --2- 172.21.15.1:0/3914615672 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56413d631f70 0x56413d639ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.502 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.490+0000 7f0bd3891b80 1 --2- 172.21.15.1:0/3914615672 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x56413d6a0610 0x56413d6a29f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.502 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.490+0000 7f0bd3891b80 1 --2- 172.21.15.1:0/3914615672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56413d632880 0x56413d63a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.502 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.490+0000 7f0ba8ff9700 0 client.16710 ms_handle_reset on v2:172.21.15.53:6838/1044753024 2023-11-16T16:47:52.502 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.490+0000 7f0bd3891b80 1 -- 172.21.15.1:0/3914615672 >> 172.21.15.1:0/3914615672 conn(0x56413d623360 msgr2=0x56413d62c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:52.502 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.490+0000 7f0bd3891b80 1 -- 172.21.15.1:0/3914615672 shutdown_connections 2023-11-16T16:47:52.502 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.490+0000 7f0bd3891b80 1 -- 172.21.15.1:0/3914615672 wait complete. 2023-11-16T16:47:52.502 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.490+0000 7f0bd3891b80 20 client.16710 trim_cache size 0 max 16384 2023-11-16T16:47:52.503 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2023-11-16T16:47:52.503 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.491+0000 7f0bc7705700 1 -- 172.21.15.1:0/595062241 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f0bac0a1340 msgr2=0x7f0bac0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:52.504 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.491+0000 7f0bc7705700 1 --2- 172.21.15.1:0/595062241 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f0bac0a1340 0x7f0bac0a37f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009be0 tx=0x7f0bbc009260 comp rx=0 tx=0).stop 2023-11-16T16:47:52.504 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.491+0000 7f0bc7705700 1 -- 172.21.15.1:0/595062241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bc0105250 msgr2=0x7f0bc01a2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:52.504 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.491+0000 7f0bc7705700 1 --2- 172.21.15.1:0/595062241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bc0105250 0x7f0bc01a2280 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f0bb800ef30 tx=0x7f0bb800c3a0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.504 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.491+0000 7f0bc7705700 1 -- 172.21.15.1:0/595062241 shutdown_connections 2023-11-16T16:47:52.504 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.491+0000 7f0bc7705700 1 --2- 172.21.15.1:0/595062241 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f0bac0a1340 0x7f0bac0a37f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.504 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.491+0000 7f0bc7705700 1 --2- 172.21.15.1:0/595062241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bc0105250 0x7f0bc01a2280 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.504 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.491+0000 7f0bc7705700 1 --2- 172.21.15.1:0/595062241 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f0bc00ffa70 0x7f0bc0197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.504 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.491+0000 7f0bc7705700 1 --2- 172.21.15.1:0/595062241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bc0100420 0x7f0bc0197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.505 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.491+0000 7f0bc7705700 1 -- 172.21.15.1:0/595062241 >> 172.21.15.1:0/595062241 conn(0x7f0bc00763a0 msgr2=0x7f0bc00fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:52.505 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.491+0000 7f0bc7705700 1 -- 172.21.15.1:0/595062241 shutdown_connections 2023-11-16T16:47:52.505 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.491+0000 7f0bc7705700 1 -- 172.21.15.1:0/595062241 wait complete. 2023-11-16T16:47:52.506 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-16T16:47:52.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.631+0000 7fb435966700 1 -- 172.21.15.1:0/3944514326 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb430101730 msgr2=0x7fb43010c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:52.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.631+0000 7fb435966700 1 --2- 172.21.15.1:0/3944514326 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb430101730 0x7fb43010c120 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fb42800b0d0 tx=0x7fb42800b3e0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.631+0000 7fb435966700 1 -- 172.21.15.1:0/3944514326 shutdown_connections 2023-11-16T16:47:52.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.631+0000 7fb435966700 1 --2- 172.21.15.1:0/3944514326 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb430101730 0x7fb43010c120 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.631+0000 7fb435966700 1 --2- 172.21.15.1:0/3944514326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb4300ff530 0x7fb4300ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.631+0000 7fb435966700 1 --2- 172.21.15.1:0/3944514326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb430100d80 0x7fb4301011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.631+0000 7fb435966700 1 -- 172.21.15.1:0/3944514326 >> 172.21.15.1:0/3944514326 conn(0x7fb430074f10 msgr2=0x7fb430075310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:52.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.632+0000 7fb435966700 1 -- 172.21.15.1:0/3944514326 shutdown_connections 2023-11-16T16:47:52.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.632+0000 7fb435966700 1 -- 172.21.15.1:0/3944514326 wait complete. 2023-11-16T16:47:52.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.632+0000 7fb435966700 1 Processor -- start 2023-11-16T16:47:52.636 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.632+0000 7fb435966700 1 -- start start 2023-11-16T16:47:52.636 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.633+0000 7fb435966700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4300ff530 0x7fb4301964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.636 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.633+0000 7fb435966700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb430100d80 0x7fb430196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.633+0000 7fb435966700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb430101730 0x7fb43019ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.633+0000 7fb435966700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43019b4b0 con 0x7fb4300ff530 2023-11-16T16:47:52.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.633+0000 7fb435966700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43019b620 con 0x7fb430101730 2023-11-16T16:47:52.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.633+0000 7fb435966700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43019b7e0 con 0x7fb430100d80 2023-11-16T16:47:52.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.633+0000 7fb42effd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4300ff530 0x7fb4301964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.633+0000 7fb42effd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4300ff530 0x7fb4301964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46542/0 (socket says 172.21.15.1:46542) 2023-11-16T16:47:52.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.633+0000 7fb42effd700 1 -- 172.21.15.1:0/3297042482 learned_addr learned my addr 172.21.15.1:0/3297042482 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:52.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.633+0000 7fb42f7fe700 1 --2- 172.21.15.1:0/3297042482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb430101730 0x7fb43019ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.638 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.633+0000 7fb42e7fc700 1 --2- 172.21.15.1:0/3297042482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb430100d80 0x7fb430196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.638 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.633+0000 7fb42effd700 1 -- 172.21.15.1:0/3297042482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb430100d80 msgr2=0x7fb430196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:52.638 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.633+0000 7fb42effd700 1 --2- 172.21.15.1:0/3297042482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb430100d80 0x7fb430196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.638 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.633+0000 7fb42effd700 1 -- 172.21.15.1:0/3297042482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb430101730 msgr2=0x7fb43019ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:52.638 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.633+0000 7fb42effd700 1 --2- 172.21.15.1:0/3297042482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb430101730 0x7fb43019ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.638 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.633+0000 7fb42effd700 1 -- 172.21.15.1:0/3297042482 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb428009d00 con 0x7fb4300ff530 2023-11-16T16:47:52.638 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.633+0000 7fb42effd700 1 --2- 172.21.15.1:0/3297042482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4300ff530 0x7fb4301964d0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fb42000c9b0 tx=0x7fb42000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:52.638 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.633+0000 7fb434964700 1 -- 172.21.15.1:0/3297042482 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb420007b00 con 0x7fb4300ff530 2023-11-16T16:47:52.639 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.633+0000 7fb435966700 1 -- 172.21.15.1:0/3297042482 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb43010f340 con 0x7fb4300ff530 2023-11-16T16:47:52.639 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.633+0000 7fb434964700 1 -- 172.21.15.1:0/3297042482 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb42000ce50 con 0x7fb4300ff530 2023-11-16T16:47:52.639 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.633+0000 7fb434964700 1 -- 172.21.15.1:0/3297042482 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb420018660 con 0x7fb4300ff530 2023-11-16T16:47:52.639 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.633+0000 7fb435966700 1 -- 172.21.15.1:0/3297042482 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb43010f1b0 con 0x7fb4300ff530 2023-11-16T16:47:52.645 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.635+0000 7fb434964700 1 -- 172.21.15.1:0/3297042482 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fb42000f450 con 0x7fb4300ff530 2023-11-16T16:47:52.645 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.636+0000 7fb434964700 1 --2- 172.21.15.1:0/3297042482 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fb4180a12c0 0x7fb4180a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.645 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.636+0000 7fb42e7fc700 1 --2- 172.21.15.1:0/3297042482 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fb4180a12c0 0x7fb4180a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.645 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.636+0000 7fb434964700 1 -- 172.21.15.1:0/3297042482 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fb4200ccc90 con 0x7fb4300ff530 2023-11-16T16:47:52.645 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.636+0000 7fb42e7fc700 1 --2- 172.21.15.1:0/3297042482 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fb4180a12c0 0x7fb4180a3770 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb430197ae0 tx=0x7fb424009260 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:52.645 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.637+0000 7fb441af2b80 1 Processor -- start 2023-11-16T16:47:52.646 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.637+0000 7fb441af2b80 1 -- start start 2023-11-16T16:47:52.646 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.637+0000 7fb441af2b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d962a8b60 0x564d962a8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.646 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.637+0000 7fb441af2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d962a9470 0x564d962ad370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.646 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.637+0000 7fb441af2b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d962ad8b0 0x564d962add00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.646 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.637+0000 7fb441af2b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d961ae060 con 0x564d962ad8b0 2023-11-16T16:47:52.646 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.637+0000 7fb441af2b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d962008a0 con 0x564d962a8b60 2023-11-16T16:47:52.646 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.637+0000 7fb441af2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d961f9d30 con 0x564d962a9470 2023-11-16T16:47:52.647 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.638+0000 7fb42f7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d962ad8b0 0x564d962add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.647 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.638+0000 7fb42effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d962a8b60 0x564d962a8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.647 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.638+0000 7fb42f7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d962ad8b0 0x564d962add00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46554/0 (socket says 172.21.15.1:46554) 2023-11-16T16:47:52.647 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.638+0000 7fb42e7fc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d962a9470 0x564d962ad370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.647 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.638+0000 7fb42f7fe700 1 -- 172.21.15.1:0/3350759937 learned_addr learned my addr 172.21.15.1:0/3350759937 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:52.647 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.638+0000 7fb42f7fe700 1 -- 172.21.15.1:0/3350759937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d962a9470 msgr2=0x564d962ad370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:52.647 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.638+0000 7fb42f7fe700 1 --2- 172.21.15.1:0/3350759937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d962a9470 0x564d962ad370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.647 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.638+0000 7fb42f7fe700 1 -- 172.21.15.1:0/3350759937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d962a8b60 msgr2=0x564d962a8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:52.648 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.638+0000 7fb42f7fe700 1 --2- 172.21.15.1:0/3350759937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d962a8b60 0x564d962a8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.648 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.638+0000 7fb42f7fe700 1 -- 172.21.15.1:0/3350759937 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d9607b630 con 0x564d962ad8b0 2023-11-16T16:47:52.648 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.638+0000 7fb42f7fe700 1 --2- 172.21.15.1:0/3350759937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d962ad8b0 0x564d962add00 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fb428009c60 tx=0x7fb428009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0866d57d0d472f server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:52.648 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.638+0000 7fb415ffb700 1 -- 172.21.15.1:0/3350759937 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42800e040 con 0x564d962ad8b0 2023-11-16T16:47:52.648 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.638+0000 7fb415ffb700 1 -- 172.21.15.1:0/3350759937 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb428009490 con 0x564d962ad8b0 2023-11-16T16:47:52.648 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.638+0000 7fb415ffb700 1 -- 172.21.15.1:0/3350759937 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb428005290 con 0x564d962ad8b0 2023-11-16T16:47:52.648 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.639+0000 7fb441af2b80 1 -- 172.21.15.1:0/3350759937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d962ad8b0 msgr2=0x564d962add00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:52.648 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.639+0000 7fb441af2b80 1 --2- 172.21.15.1:0/3350759937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d962ad8b0 0x564d962add00 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fb428009c60 tx=0x7fb428009c90 comp rx=0 tx=0).stop 2023-11-16T16:47:52.648 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.639+0000 7fb441af2b80 1 -- 172.21.15.1:0/3350759937 shutdown_connections 2023-11-16T16:47:52.649 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.639+0000 7fb441af2b80 1 --2- 172.21.15.1:0/3350759937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d962ad8b0 0x564d962add00 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.649 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.639+0000 7fb441af2b80 1 --2- 172.21.15.1:0/3350759937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d962a8b60 0x564d962a8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.649 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.639+0000 7fb441af2b80 1 --2- 172.21.15.1:0/3350759937 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d962a9470 0x564d962ad370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.649 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.639+0000 7fb441af2b80 1 -- 172.21.15.1:0/3350759937 >> 172.21.15.1:0/3350759937 conn(0x564d9629e360 msgr2=0x564d962a3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:52.649 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.639+0000 7fb441af2b80 1 -- 172.21.15.1:0/3350759937 shutdown_connections 2023-11-16T16:47:52.649 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.639+0000 7fb441af2b80 1 -- 172.21.15.1:0/3350759937 wait complete. 2023-11-16T16:47:52.649 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.639+0000 7fb441af2b80 1 -- start start 2023-11-16T16:47:52.650 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.640+0000 7fb441af2b80 1 Processor -- start 2023-11-16T16:47:52.650 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.640+0000 7fb441af2b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d962a8b60 0x564d962b5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.650 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.640+0000 7fb441af2b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d962a9470 0x564d962b2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.650 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.640+0000 7fb441af2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d962ad8b0 0x564d962b2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.650 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.640+0000 7fb441af2b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d9619b8e0 con 0x564d962a9470 2023-11-16T16:47:52.650 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.640+0000 7fb441af2b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d9619b580 con 0x564d962a8b60 2023-11-16T16:47:52.650 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.640+0000 7fb441af2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d9607d920 con 0x564d962ad8b0 2023-11-16T16:47:52.650 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.640+0000 7fb441af2b80 10 client.0 _ll_register_callbacks cb 0x564d961b9780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:47:52.651 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.640+0000 7fb42e7fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d962a9470 0x564d962b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.651 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.640+0000 7fb42e7fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d962a9470 0x564d962b2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46558/0 (socket says 172.21.15.1:46558) 2023-11-16T16:47:52.651 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.640+0000 7fb42e7fc700 1 -- 172.21.15.1:0/4152650121 learned_addr learned my addr 172.21.15.1:0/4152650121 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:52.651 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.640+0000 7fb42effd700 1 --2- 172.21.15.1:0/4152650121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d962a8b60 0x564d962b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.651 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.640+0000 7fb42f7fe700 1 --2- 172.21.15.1:0/4152650121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d962ad8b0 0x564d962b2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.651 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.640+0000 7fb42e7fc700 1 -- 172.21.15.1:0/4152650121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d962ad8b0 msgr2=0x564d962b2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:52.651 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.640+0000 7fb42e7fc700 1 --2- 172.21.15.1:0/4152650121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d962ad8b0 0x564d962b2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.651 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.640+0000 7fb42e7fc700 1 -- 172.21.15.1:0/4152650121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d962a8b60 msgr2=0x564d962b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:52.652 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.640+0000 7fb42e7fc700 1 --2- 172.21.15.1:0/4152650121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d962a8b60 0x564d962b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.652 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.640+0000 7fb42e7fc700 1 -- 172.21.15.1:0/4152650121 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb43010ca20 con 0x564d962a9470 2023-11-16T16:47:52.652 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.640+0000 7fb42e7fc700 1 --2- 172.21.15.1:0/4152650121 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d962a9470 0x564d962b2320 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fb42401fbf0 tx=0x7fb4240047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c90e893f0e7247ef server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:52.652 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.640+0000 7fb40effd700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:47:52.652 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.641+0000 7fb40effd700 1 -- 172.21.15.1:0/4152650121 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42401d6c0 con 0x564d962a9470 2023-11-16T16:47:52.652 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.641+0000 7fb40effd700 1 -- 172.21.15.1:0/4152650121 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fb42401d820 con 0x564d962a9470 2023-11-16T16:47:52.652 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.641+0000 7fb441af2b80 1 -- 172.21.15.1:0/4152650121 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564d9629bb20 con 0x564d962a9470 2023-11-16T16:47:52.652 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.641+0000 7fb40effd700 1 -- 172.21.15.1:0/4152650121 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb42401d9e0 con 0x564d962a9470 2023-11-16T16:47:52.653 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.641+0000 7fb441af2b80 10 client.16722 fetch_fsmap learned FSMap version 28 2023-11-16T16:47:52.653 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.641+0000 7fb441af2b80 1 -- 172.21.15.1:0/4152650121 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564d961c0190 con 0x564d962a9470 2023-11-16T16:47:52.653 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.641+0000 7fb40effd700 1 -- 172.21.15.1:0/4152650121 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564d9629bb20 con 0x564d962a9470 2023-11-16T16:47:52.653 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.641+0000 7fb40effd700 1 -- 172.21.15.1:0/4152650121 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fb42400e3b0 con 0x564d962a9470 2023-11-16T16:47:52.653 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.641+0000 7fb441af2b80 10 client.16722 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:47:52.653 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.641+0000 7fb441af2b80 10 client.16722 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:47:52.653 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.641+0000 7fb441af2b80 20 client.16722 populate_metadata read hostname 'smithi001' 2023-11-16T16:47:52.653 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.641+0000 7fb441af2b80 1 --2- 172.21.15.1:0/4152650121 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x564d962c2920 0x564d962c4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.654 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.641+0000 7fb441af2b80 4 client.16722 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:47:52.654 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.641+0000 7fb441af2b80 1 -- 172.21.15.1:0/4152650121 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564d961bc0a0 con 0x564d962c2920 2023-11-16T16:47:52.654 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.641+0000 7fb42effd700 1 --2- 172.21.15.1:0/4152650121 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x564d962c2920 0x564d962c4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.654 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.642+0000 7fb40effd700 10 client.16722 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:47:52.654 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.642+0000 7fb42effd700 1 --2- 172.21.15.1:0/4152650121 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x564d962c2920 0x564d962c4d30 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=845e0c51e37f9760 server_cookie=7f7e3803f128082d in_seq=0 out_seq=0 2023-11-16T16:47:52.654 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.643+0000 7fb40effd700 1 -- 172.21.15.1:0/4152650121 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564d961bc0a0 con 0x564d962c2920 2023-11-16T16:47:52.654 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.643+0000 7fb40effd700 10 client.16722 handle_command_reply: tid=0 2023-11-16T16:47:52.655 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.643+0000 7fb441af2b80 1 client.16722 shutdown 2023-11-16T16:47:52.655 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.643+0000 7fb441af2b80 1 -- 172.21.15.1:0/4152650121 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d962a9470 msgr2=0x564d962b2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:52.655 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.643+0000 7fb441af2b80 1 --2- 172.21.15.1:0/4152650121 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d962a9470 0x564d962b2320 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fb42401fbf0 tx=0x7fb4240047c0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.655 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.643+0000 7fb441af2b80 1 -- 172.21.15.1:0/4152650121 shutdown_connections 2023-11-16T16:47:52.655 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.643+0000 7fb441af2b80 1 --2- 172.21.15.1:0/4152650121 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x564d962c2920 0x564d962c4d30 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.655 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.643+0000 7fb441af2b80 1 --2- 172.21.15.1:0/4152650121 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d962ad8b0 0x564d962b2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.655 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.643+0000 7fb40effd700 0 client.16722 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:47:52.655 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.643+0000 7fb441af2b80 1 --2- 172.21.15.1:0/4152650121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d962a8b60 0x564d962b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.656 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.643+0000 7fb441af2b80 1 --2- 172.21.15.1:0/4152650121 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d962a9470 0x564d962b2320 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.656 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.643+0000 7fb441af2b80 1 -- 172.21.15.1:0/4152650121 >> 172.21.15.1:0/4152650121 conn(0x564d9629e360 msgr2=0x564d962aed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:52.656 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.643+0000 7fb441af2b80 1 -- 172.21.15.1:0/4152650121 shutdown_connections 2023-11-16T16:47:52.656 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.643+0000 7fb441af2b80 1 -- 172.21.15.1:0/4152650121 wait complete. 2023-11-16T16:47:52.656 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.643+0000 7fb441af2b80 20 client.16722 trim_cache size 0 max 16384 2023-11-16T16:47:52.662 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.656+0000 7fb441af2b80 1 Processor -- start 2023-11-16T16:47:52.662 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.656+0000 7fb441af2b80 1 -- start start 2023-11-16T16:47:52.662 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.656+0000 7fb441af2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d962a8b60 0x564d962b9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.662 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.656+0000 7fb441af2b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d962a9470 0x564d962b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.662 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.656+0000 7fb441af2b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d962ad8b0 0x564d962a3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.663 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.656+0000 7fb441af2b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d9629b9a0 con 0x564d962ad8b0 2023-11-16T16:47:52.663 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.656+0000 7fb441af2b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d962b0880 con 0x564d962a9470 2023-11-16T16:47:52.663 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.656+0000 7fb441af2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d962ac450 con 0x564d962a8b60 2023-11-16T16:47:52.663 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.656+0000 7fb42f7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d962ad8b0 0x564d962a3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.663 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.656+0000 7fb42f7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d962ad8b0 0x564d962a3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46568/0 (socket says 172.21.15.1:46568) 2023-11-16T16:47:52.663 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.656+0000 7fb42f7fe700 1 -- 172.21.15.1:0/1233052583 learned_addr learned my addr 172.21.15.1:0/1233052583 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:52.663 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.656+0000 7fb42e7fc700 1 --2- 172.21.15.1:0/1233052583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d962a9470 0x564d962b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.663 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.657+0000 7fb42effd700 1 --2- 172.21.15.1:0/1233052583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d962a8b60 0x564d962b9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.664 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.657+0000 7fb42f7fe700 1 -- 172.21.15.1:0/1233052583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d962a8b60 msgr2=0x564d962b9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:52.664 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.657+0000 7fb42f7fe700 1 --2- 172.21.15.1:0/1233052583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d962a8b60 0x564d962b9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.664 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.657+0000 7fb42f7fe700 1 -- 172.21.15.1:0/1233052583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d962a9470 msgr2=0x564d962b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:52.664 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.657+0000 7fb42f7fe700 1 --2- 172.21.15.1:0/1233052583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d962a9470 0x564d962b4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.664 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.657+0000 7fb42f7fe700 1 -- 172.21.15.1:0/1233052583 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d9629fb00 con 0x564d962ad8b0 2023-11-16T16:47:52.664 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.657+0000 7fb42effd700 1 --2- 172.21.15.1:0/1233052583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d962a8b60 0x564d962b9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:47:52.664 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.657+0000 7fb42f7fe700 1 --2- 172.21.15.1:0/1233052583 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d962ad8b0 0x564d962a3560 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fb428009a20 tx=0x7fb428003c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b4cf18564135c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:52.664 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.657+0000 7fb4167fc700 1 -- 172.21.15.1:0/1233052583 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42800e040 con 0x564d962ad8b0 2023-11-16T16:47:52.665 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.657+0000 7fb4167fc700 1 -- 172.21.15.1:0/1233052583 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb428007960 con 0x564d962ad8b0 2023-11-16T16:47:52.665 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.657+0000 7fb4167fc700 1 -- 172.21.15.1:0/1233052583 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4280055b0 con 0x564d962ad8b0 2023-11-16T16:47:52.665 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.658+0000 7fb441af2b80 1 -- 172.21.15.1:0/1233052583 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d962ad8b0 msgr2=0x564d962a3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:52.665 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.658+0000 7fb441af2b80 1 --2- 172.21.15.1:0/1233052583 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d962ad8b0 0x564d962a3560 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fb428009a20 tx=0x7fb428003c40 comp rx=0 tx=0).stop 2023-11-16T16:47:52.665 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.658+0000 7fb441af2b80 1 -- 172.21.15.1:0/1233052583 shutdown_connections 2023-11-16T16:47:52.665 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.658+0000 7fb441af2b80 1 --2- 172.21.15.1:0/1233052583 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d962ad8b0 0x564d962a3560 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.665 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.658+0000 7fb441af2b80 1 --2- 172.21.15.1:0/1233052583 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d962a8b60 0x564d962b9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.665 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.658+0000 7fb441af2b80 1 --2- 172.21.15.1:0/1233052583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d962a9470 0x564d962b4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.666 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.658+0000 7fb441af2b80 1 -- 172.21.15.1:0/1233052583 >> 172.21.15.1:0/1233052583 conn(0x564d9629e360 msgr2=0x564d962ab8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:52.666 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.658+0000 7fb441af2b80 1 -- 172.21.15.1:0/1233052583 shutdown_connections 2023-11-16T16:47:52.666 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.658+0000 7fb441af2b80 1 -- 172.21.15.1:0/1233052583 wait complete. 2023-11-16T16:47:52.666 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.658+0000 7fb441af2b80 1 -- start start 2023-11-16T16:47:52.666 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.658+0000 7fb441af2b80 1 Processor -- start 2023-11-16T16:47:52.666 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.658+0000 7fb441af2b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d962a8b60 0x564d962b7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.666 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.658+0000 7fb441af2b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d962a9470 0x564d962b7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.666 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.658+0000 7fb441af2b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d962ad8b0 0x564d962b83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.667 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.658+0000 7fb441af2b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d96315840 con 0x564d962a9470 2023-11-16T16:47:52.667 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.658+0000 7fb441af2b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d963159b0 con 0x564d962ad8b0 2023-11-16T16:47:52.667 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.658+0000 7fb441af2b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d96315b70 con 0x564d962a8b60 2023-11-16T16:47:52.667 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.658+0000 7fb441af2b80 10 client.0 _ll_register_callbacks cb 0x7fb42401f7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:47:52.667 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.659+0000 7fb42e7fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d962a9470 0x564d962b7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.667 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.659+0000 7fb42e7fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d962a9470 0x564d962b7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46572/0 (socket says 172.21.15.1:46572) 2023-11-16T16:47:52.667 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.659+0000 7fb42e7fc700 1 -- 172.21.15.1:0/2087130123 learned_addr learned my addr 172.21.15.1:0/2087130123 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:52.668 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.659+0000 7fb42f7fe700 1 --2- 172.21.15.1:0/2087130123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d962ad8b0 0x564d962b83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.668 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.659+0000 7fb42effd700 1 --2- 172.21.15.1:0/2087130123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d962a8b60 0x564d962b7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.668 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.659+0000 7fb42e7fc700 1 -- 172.21.15.1:0/2087130123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d962a8b60 msgr2=0x564d962b7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:52.668 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.659+0000 7fb42e7fc700 1 --2- 172.21.15.1:0/2087130123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d962a8b60 0x564d962b7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.668 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.659+0000 7fb42e7fc700 1 -- 172.21.15.1:0/2087130123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d962ad8b0 msgr2=0x564d962b83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:52.668 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.659+0000 7fb42e7fc700 1 --2- 172.21.15.1:0/2087130123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d962ad8b0 0x564d962b83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.668 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.659+0000 7fb42e7fc700 1 -- 172.21.15.1:0/2087130123 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564d9607b630 con 0x564d962a9470 2023-11-16T16:47:52.668 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.659+0000 7fb42f7fe700 1 --2- 172.21.15.1:0/2087130123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d962ad8b0 0x564d962b83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:47:52.669 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.659+0000 7fb42e7fc700 1 --2- 172.21.15.1:0/2087130123 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d962a9470 0x564d962b7e70 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fb42400e7c0 tx=0x7fb42401d420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6660148c424b45b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:52.669 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.659+0000 7fb40effd700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:47:52.669 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.659+0000 7fb40effd700 1 -- 172.21.15.1:0/2087130123 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb424004250 con 0x564d962a9470 2023-11-16T16:47:52.669 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.659+0000 7fb40effd700 1 -- 172.21.15.1:0/2087130123 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fb4240043b0 con 0x564d962a9470 2023-11-16T16:47:52.669 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.659+0000 7fb441af2b80 1 -- 172.21.15.1:0/2087130123 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb428009490 con 0x564d962a9470 2023-11-16T16:47:52.669 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.659+0000 7fb40effd700 1 -- 172.21.15.1:0/2087130123 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb424004570 con 0x564d962a9470 2023-11-16T16:47:52.669 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.659+0000 7fb441af2b80 10 client.16728 fetch_fsmap learned FSMap version 28 2023-11-16T16:47:52.670 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.659+0000 7fb441af2b80 1 -- 172.21.15.1:0/2087130123 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564d96027de0 con 0x564d962a9470 2023-11-16T16:47:52.670 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.660+0000 7fb40effd700 1 -- 172.21.15.1:0/2087130123 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb424045080 con 0x564d962a9470 2023-11-16T16:47:52.670 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.660+0000 7fb40effd700 1 -- 172.21.15.1:0/2087130123 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fb42401fc20 con 0x564d962a9470 2023-11-16T16:47:52.670 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.660+0000 7fb441af2b80 10 client.16728 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:47:52.670 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.660+0000 7fb441af2b80 10 client.16728 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:47:52.670 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.660+0000 7fb441af2b80 20 client.16728 populate_metadata read hostname 'smithi001' 2023-11-16T16:47:52.670 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.660+0000 7fb441af2b80 1 --2- 172.21.15.1:0/2087130123 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x564d963172e0 0x564d963196f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:52.670 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.660+0000 7fb441af2b80 4 client.16728 mds_command: new command op to 24521 tid=0[{"prefix": "damage ls"}] 2023-11-16T16:47:52.671 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.660+0000 7fb441af2b80 1 -- 172.21.15.1:0/2087130123 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564d961c8660 con 0x564d963172e0 2023-11-16T16:47:52.671 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.660+0000 7fb42effd700 1 --2- 172.21.15.1:0/2087130123 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x564d963172e0 0x564d963196f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:52.671 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.660+0000 7fb40effd700 10 client.16728 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:47:52.671 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.660+0000 7fb42effd700 1 --2- 172.21.15.1:0/2087130123 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x564d963172e0 0x564d963196f0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8c90a7c4a7d8839 server_cookie=fc9cf81e585b2992 in_seq=0 out_seq=0 2023-11-16T16:47:52.672 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.661+0000 7fb40effd700 1 -- 172.21.15.1:0/2087130123 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564d961c8660 con 0x564d963172e0 2023-11-16T16:47:52.672 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.661+0000 7fb40effd700 10 client.16728 handle_command_reply: tid=0 2023-11-16T16:47:52.672 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.661+0000 7fb441af2b80 1 client.16728 shutdown 2023-11-16T16:47:52.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.661+0000 7fb441af2b80 1 -- 172.21.15.1:0/2087130123 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d962a9470 msgr2=0x564d962b7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:52.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.661+0000 7fb441af2b80 1 --2- 172.21.15.1:0/2087130123 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d962a9470 0x564d962b7e70 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fb42400e7c0 tx=0x7fb42401d420 comp rx=0 tx=0).stop 2023-11-16T16:47:52.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.661+0000 7fb441af2b80 1 -- 172.21.15.1:0/2087130123 shutdown_connections 2023-11-16T16:47:52.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.661+0000 7fb441af2b80 1 --2- 172.21.15.1:0/2087130123 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x564d963172e0 0x564d963196f0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.661+0000 7fb441af2b80 1 --2- 172.21.15.1:0/2087130123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564d962ad8b0 0x564d962b83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.661+0000 7fb441af2b80 1 --2- 172.21.15.1:0/2087130123 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564d962a8b60 0x564d962b7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.661+0000 7fb40effd700 0 client.16728 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:47:52.673 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.661+0000 7fb441af2b80 1 --2- 172.21.15.1:0/2087130123 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564d962a9470 0x564d962b7e70 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.674 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.661+0000 7fb441af2b80 1 -- 172.21.15.1:0/2087130123 >> 172.21.15.1:0/2087130123 conn(0x564d9629e360 msgr2=0x564d962a7960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:52.674 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.661+0000 7fb441af2b80 1 -- 172.21.15.1:0/2087130123 shutdown_connections 2023-11-16T16:47:52.674 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.661+0000 7fb441af2b80 1 -- 172.21.15.1:0/2087130123 wait complete. 2023-11-16T16:47:52.674 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.661+0000 7fb441af2b80 20 client.16728 trim_cache size 0 max 16384 2023-11-16T16:47:52.674 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2023-11-16T16:47:52.675 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.661+0000 7fb435966700 1 -- 172.21.15.1:0/3297042482 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fb4180a12c0 msgr2=0x7fb4180a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:52.676 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.661+0000 7fb435966700 1 --2- 172.21.15.1:0/3297042482 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fb4180a12c0 0x7fb4180a3770 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb430197ae0 tx=0x7fb424009260 comp rx=0 tx=0).stop 2023-11-16T16:47:52.676 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.661+0000 7fb435966700 1 -- 172.21.15.1:0/3297042482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4300ff530 msgr2=0x7fb4301964d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:52.676 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.661+0000 7fb435966700 1 --2- 172.21.15.1:0/3297042482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4300ff530 0x7fb4301964d0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fb42000c9b0 tx=0x7fb42000ccc0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.676 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.662+0000 7fb435966700 1 -- 172.21.15.1:0/3297042482 shutdown_connections 2023-11-16T16:47:52.676 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.662+0000 7fb435966700 1 --2- 172.21.15.1:0/3297042482 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fb4180a12c0 0x7fb4180a3770 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.676 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.662+0000 7fb435966700 1 --2- 172.21.15.1:0/3297042482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb430101730 0x7fb43019ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.676 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.662+0000 7fb435966700 1 --2- 172.21.15.1:0/3297042482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb4300ff530 0x7fb4301964d0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.676 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.662+0000 7fb435966700 1 --2- 172.21.15.1:0/3297042482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb430100d80 0x7fb430196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:52.677 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.662+0000 7fb435966700 1 -- 172.21.15.1:0/3297042482 >> 172.21.15.1:0/3297042482 conn(0x7fb430074f10 msgr2=0x7fb4300690d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:52.677 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.662+0000 7fb435966700 1 -- 172.21.15.1:0/3297042482 shutdown_connections 2023-11-16T16:47:52.677 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:52.662+0000 7fb435966700 1 -- 172.21.15.1:0/3297042482 wait complete. 2023-11-16T16:47:52.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:52 smithi026 ceph-mon[100112]: pgmap v268: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:47:52.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:52 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/2208290835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:47:52.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:52 smithi053 ceph-mon[100638]: pgmap v268: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:47:52.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:52 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/2208290835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:47:53.678 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-16T16:47:53.679 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-16T16:47:53.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.823+0000 7feb6082a700 1 -- 172.21.15.1:0/2641582969 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb5c0ffa70 msgr2=0x7feb5c0ffee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:53.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.823+0000 7feb6082a700 1 --2- 172.21.15.1:0/2641582969 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb5c0ffa70 0x7feb5c0ffee0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7feb4c009990 tx=0x7feb4c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.823+0000 7feb6082a700 1 -- 172.21.15.1:0/2641582969 shutdown_connections 2023-11-16T16:47:53.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.823+0000 7feb6082a700 1 --2- 172.21.15.1:0/2641582969 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb5c100420 0x7feb5c10bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.823+0000 7feb6082a700 1 --2- 172.21.15.1:0/2641582969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb5c1051c0 0x7feb5c105590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.823+0000 7feb6082a700 1 --2- 172.21.15.1:0/2641582969 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb5c0ffa70 0x7feb5c0ffee0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.823+0000 7feb6082a700 1 -- 172.21.15.1:0/2641582969 >> 172.21.15.1:0/2641582969 conn(0x7feb5c076370 msgr2=0x7feb5c076770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:53.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.823+0000 7feb6082a700 1 -- 172.21.15.1:0/2641582969 shutdown_connections 2023-11-16T16:47:53.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.823+0000 7feb6082a700 1 -- 172.21.15.1:0/2641582969 wait complete. 2023-11-16T16:47:53.828 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.824+0000 7feb6082a700 1 Processor -- start 2023-11-16T16:47:53.829 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.824+0000 7feb6082a700 1 -- start start 2023-11-16T16:47:53.829 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.824+0000 7feb6082a700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb5c0ffa70 0x7feb5c197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:53.829 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.824+0000 7feb6082a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb5c100420 0x7feb5c1976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:53.829 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.824+0000 7feb6082a700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb5c1051c0 0x7feb5c1a2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:53.829 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.824+0000 7feb6082a700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c198010 con 0x7feb5c1051c0 2023-11-16T16:47:53.829 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.824+0000 7feb6082a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c1a2750 con 0x7feb5c100420 2023-11-16T16:47:53.829 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.824+0000 7feb6082a700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5c1a2910 con 0x7feb5c0ffa70 2023-11-16T16:47:53.830 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.824+0000 7feb5ad9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb5c1051c0 0x7feb5c1a2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:53.830 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.824+0000 7feb5ad9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb5c1051c0 0x7feb5c1a2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46588/0 (socket says 172.21.15.1:46588) 2023-11-16T16:47:53.830 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.824+0000 7feb5ad9d700 1 -- 172.21.15.1:0/229609424 learned_addr learned my addr 172.21.15.1:0/229609424 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:53.830 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.824+0000 7feb59d9b700 1 --2- 172.21.15.1:0/229609424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb5c100420 0x7feb5c1976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:53.830 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.825+0000 7feb5a59c700 1 --2- 172.21.15.1:0/229609424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb5c0ffa70 0x7feb5c197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:53.830 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.825+0000 7feb5ad9d700 1 -- 172.21.15.1:0/229609424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb5c0ffa70 msgr2=0x7feb5c197190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:53.831 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.825+0000 7feb5ad9d700 1 --2- 172.21.15.1:0/229609424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb5c0ffa70 0x7feb5c197190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.831 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.825+0000 7feb5ad9d700 1 -- 172.21.15.1:0/229609424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb5c100420 msgr2=0x7feb5c1976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:53.831 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.825+0000 7feb5ad9d700 1 --2- 172.21.15.1:0/229609424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb5c100420 0x7feb5c1976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.831 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.825+0000 7feb5ad9d700 1 -- 172.21.15.1:0/229609424 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb4c0095f0 con 0x7feb5c1051c0 2023-11-16T16:47:53.831 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.825+0000 7feb5ad9d700 1 --2- 172.21.15.1:0/229609424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb5c1051c0 0x7feb5c1a2210 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7feb54007ee0 tx=0x7feb5400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:53.831 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.825+0000 7feb537fe700 1 -- 172.21.15.1:0/229609424 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb54017070 con 0x7feb5c1051c0 2023-11-16T16:47:53.831 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.825+0000 7feb537fe700 1 -- 172.21.15.1:0/229609424 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb5400f040 con 0x7feb5c1051c0 2023-11-16T16:47:53.831 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.825+0000 7feb6082a700 1 -- 172.21.15.1:0/229609424 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb5c10f1b0 con 0x7feb5c1051c0 2023-11-16T16:47:53.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.825+0000 7feb537fe700 1 -- 172.21.15.1:0/229609424 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb54005580 con 0x7feb5c1051c0 2023-11-16T16:47:53.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.825+0000 7feb6082a700 1 -- 172.21.15.1:0/229609424 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb5c10f020 con 0x7feb5c1051c0 2023-11-16T16:47:53.838 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.827+0000 7feb537fe700 1 -- 172.21.15.1:0/229609424 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7feb5400db20 con 0x7feb5c1051c0 2023-11-16T16:47:53.838 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.828+0000 7feb537fe700 1 --2- 172.21.15.1:0/229609424 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7feb480a12c0 0x7feb480a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:53.838 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.828+0000 7feb5a59c700 1 --2- 172.21.15.1:0/229609424 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7feb480a12c0 0x7feb480a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:53.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.828+0000 7feb537fe700 1 -- 172.21.15.1:0/229609424 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7feb540cc460 con 0x7feb5c1051c0 2023-11-16T16:47:53.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.829+0000 7feb5a59c700 1 --2- 172.21.15.1:0/229609424 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7feb480a12c0 0x7feb480a3770 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7feb5c068db0 tx=0x7feb44009260 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:53.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.829+0000 7feb6c9b6b80 1 Processor -- start 2023-11-16T16:47:53.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.829+0000 7feb6c9b6b80 1 -- start start 2023-11-16T16:47:53.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.830+0000 7feb6c9b6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636ad9dfbe0 0x5636ad9dffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:53.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.830+0000 7feb6c9b6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636ad9e04f0 0x5636ad9e43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:53.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.830+0000 7feb6c9b6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636ad9e4930 0x5636ad9e4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:53.840 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.830+0000 7feb6c9b6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ad8e50e0 con 0x5636ad9e4930 2023-11-16T16:47:53.840 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.830+0000 7feb6c9b6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ad937920 con 0x5636ad9e04f0 2023-11-16T16:47:53.840 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.830+0000 7feb6c9b6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ad930db0 con 0x5636ad9dfbe0 2023-11-16T16:47:53.840 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.830+0000 7feb5ad9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636ad9e4930 0x5636ad9e4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:53.840 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.830+0000 7feb5ad9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636ad9e4930 0x5636ad9e4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46596/0 (socket says 172.21.15.1:46596) 2023-11-16T16:47:53.840 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.830+0000 7feb5ad9d700 1 -- 172.21.15.1:0/2268196045 learned_addr learned my addr 172.21.15.1:0/2268196045 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:53.840 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.830+0000 7feb5a59c700 1 --2- 172.21.15.1:0/2268196045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636ad9dfbe0 0x5636ad9dffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:53.840 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.830+0000 7feb5ad9d700 1 -- 172.21.15.1:0/2268196045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636ad9dfbe0 msgr2=0x5636ad9dffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:53.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.830+0000 7feb5ad9d700 1 --2- 172.21.15.1:0/2268196045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636ad9dfbe0 0x5636ad9dffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.830+0000 7feb5ad9d700 1 -- 172.21.15.1:0/2268196045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636ad9e04f0 msgr2=0x5636ad9e43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:53.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.830+0000 7feb5ad9d700 1 --2- 172.21.15.1:0/2268196045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636ad9e04f0 0x5636ad9e43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.830+0000 7feb5ad9d700 1 -- 172.21.15.1:0/2268196045 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636ad7b26b0 con 0x5636ad9e4930 2023-11-16T16:47:53.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.830+0000 7feb5ad9d700 1 --2- 172.21.15.1:0/2268196045 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636ad9e4930 0x5636ad9e4d80 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7feb540a1440 tx=0x7feb54095260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72b937b1f10bed60 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:53.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.830+0000 7feb50ff9700 1 -- 172.21.15.1:0/2268196045 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb54095040 con 0x5636ad9e4930 2023-11-16T16:47:53.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.831+0000 7feb50ff9700 1 -- 172.21.15.1:0/2268196045 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb540978c0 con 0x5636ad9e4930 2023-11-16T16:47:53.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.831+0000 7feb50ff9700 1 -- 172.21.15.1:0/2268196045 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb54095c30 con 0x5636ad9e4930 2023-11-16T16:47:53.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.831+0000 7feb6c9b6b80 1 -- 172.21.15.1:0/2268196045 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636ad9e4930 msgr2=0x5636ad9e4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:53.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.831+0000 7feb6c9b6b80 1 --2- 172.21.15.1:0/2268196045 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636ad9e4930 0x5636ad9e4d80 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7feb540a1440 tx=0x7feb54095260 comp rx=0 tx=0).stop 2023-11-16T16:47:53.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.831+0000 7feb6c9b6b80 1 -- 172.21.15.1:0/2268196045 shutdown_connections 2023-11-16T16:47:53.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.831+0000 7feb6c9b6b80 1 --2- 172.21.15.1:0/2268196045 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636ad9e4930 0x5636ad9e4d80 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.831+0000 7feb6c9b6b80 1 --2- 172.21.15.1:0/2268196045 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636ad9dfbe0 0x5636ad9dffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.831+0000 7feb6c9b6b80 1 --2- 172.21.15.1:0/2268196045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636ad9e04f0 0x5636ad9e43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.831+0000 7feb6c9b6b80 1 -- 172.21.15.1:0/2268196045 >> 172.21.15.1:0/2268196045 conn(0x5636ad9d53e0 msgr2=0x5636ad9dabc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:53.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.831+0000 7feb6c9b6b80 1 -- 172.21.15.1:0/2268196045 shutdown_connections 2023-11-16T16:47:53.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.831+0000 7feb6c9b6b80 1 -- 172.21.15.1:0/2268196045 wait complete. 2023-11-16T16:47:53.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.832+0000 7feb6c9b6b80 1 -- start start 2023-11-16T16:47:53.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.832+0000 7feb6c9b6b80 1 Processor -- start 2023-11-16T16:47:53.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.832+0000 7feb6c9b6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636ad9dfbe0 0x5636ad9ec180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:53.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.832+0000 7feb6c9b6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636ad9e04f0 0x5636ad9e9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:53.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.832+0000 7feb6c9b6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636ad9e4930 0x5636ad9e9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:53.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.832+0000 7feb6c9b6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ad8d2960 con 0x5636ad9dfbe0 2023-11-16T16:47:53.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.832+0000 7feb6c9b6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ad8d2600 con 0x5636ad9e4930 2023-11-16T16:47:53.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.832+0000 7feb6c9b6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ad7b49a0 con 0x5636ad9e04f0 2023-11-16T16:47:53.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.832+0000 7feb6c9b6b80 10 client.0 _ll_register_callbacks cb 0x5636ad8f0800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:47:53.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.832+0000 7feb5a59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636ad9dfbe0 0x5636ad9ec180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:53.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.832+0000 7feb5a59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636ad9dfbe0 0x5636ad9ec180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46598/0 (socket says 172.21.15.1:46598) 2023-11-16T16:47:53.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.832+0000 7feb5a59c700 1 -- 172.21.15.1:0/1825561148 learned_addr learned my addr 172.21.15.1:0/1825561148 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:53.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.832+0000 7feb59d9b700 1 --2- 172.21.15.1:0/1825561148 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636ad9e04f0 0x5636ad9e9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:53.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.833+0000 7feb5ad9d700 1 --2- 172.21.15.1:0/1825561148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636ad9e4930 0x5636ad9e9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:53.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.833+0000 7feb5a59c700 1 -- 172.21.15.1:0/1825561148 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636ad9e04f0 msgr2=0x5636ad9e9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:53.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.833+0000 7feb5a59c700 1 --2- 172.21.15.1:0/1825561148 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636ad9e04f0 0x5636ad9e9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.833+0000 7feb5a59c700 1 -- 172.21.15.1:0/1825561148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636ad9e4930 msgr2=0x5636ad9e9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:53.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.833+0000 7feb5a59c700 1 --2- 172.21.15.1:0/1825561148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636ad9e4930 0x5636ad9e9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.833+0000 7feb5a59c700 1 -- 172.21.15.1:0/1825561148 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb5c10f020 con 0x5636ad9dfbe0 2023-11-16T16:47:53.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.833+0000 7feb5a59c700 1 --2- 172.21.15.1:0/1825561148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636ad9dfbe0 0x5636ad9ec180 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7feb4401d760 tx=0x7feb440047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab5700cb8c911a44 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:53.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.833+0000 7feb41ffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:47:53.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.833+0000 7feb41ffb700 1 -- 172.21.15.1:0/1825561148 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4401b830 con 0x5636ad9dfbe0 2023-11-16T16:47:53.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.833+0000 7feb41ffb700 1 -- 172.21.15.1:0/1825561148 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7feb4401b990 con 0x5636ad9dfbe0 2023-11-16T16:47:53.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.833+0000 7feb6c9b6b80 1 -- 172.21.15.1:0/1825561148 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5636ad9d2ba0 con 0x5636ad9dfbe0 2023-11-16T16:47:53.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.833+0000 7feb41ffb700 1 -- 172.21.15.1:0/1825561148 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb4401bb50 con 0x5636ad9dfbe0 2023-11-16T16:47:53.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.833+0000 7feb6c9b6b80 10 client.16740 fetch_fsmap learned FSMap version 28 2023-11-16T16:47:53.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.833+0000 7feb6c9b6b80 1 -- 172.21.15.1:0/1825561148 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5636ad8f7210 con 0x5636ad9dfbe0 2023-11-16T16:47:53.847 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.833+0000 7feb41ffb700 1 -- 172.21.15.1:0/1825561148 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb44032020 con 0x5636ad9dfbe0 2023-11-16T16:47:53.847 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.834+0000 7feb41ffb700 1 -- 172.21.15.1:0/1825561148 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7feb4402d3c0 con 0x5636ad9dfbe0 2023-11-16T16:47:53.847 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.834+0000 7feb6c9b6b80 10 client.16740 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:47:53.847 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.834+0000 7feb6c9b6b80 10 client.16740 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:47:53.847 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.834+0000 7feb6c9b6b80 20 client.16740 populate_metadata read hostname 'smithi001' 2023-11-16T16:47:53.847 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.834+0000 7feb6c9b6b80 1 --2- 172.21.15.1:0/1825561148 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5636ad9f9af0 0x5636ad9fbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:53.847 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.834+0000 7feb6c9b6b80 4 client.16740 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:47:53.847 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.834+0000 7feb6c9b6b80 1 -- 172.21.15.1:0/1825561148 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5636ad8f3120 con 0x5636ad9f9af0 2023-11-16T16:47:53.848 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.834+0000 7feb59d9b700 1 --2- 172.21.15.1:0/1825561148 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5636ad9f9af0 0x5636ad9fbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:53.848 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.834+0000 7feb59d9b700 1 --2- 172.21.15.1:0/1825561148 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5636ad9f9af0 0x5636ad9fbf00 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce849cf90eac2f4d server_cookie=f862be22d3dab009 in_seq=0 out_seq=0 2023-11-16T16:47:53.848 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.834+0000 7feb41ffb700 10 client.16740 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:47:53.848 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.835+0000 7feb41ffb700 1 -- 172.21.15.1:0/1825561148 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5636ad8f3120 con 0x5636ad9f9af0 2023-11-16T16:47:53.848 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.835+0000 7feb41ffb700 10 client.16740 handle_command_reply: tid=0 2023-11-16T16:47:53.848 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.835+0000 7feb6c9b6b80 1 client.16740 shutdown 2023-11-16T16:47:53.848 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.835+0000 7feb6c9b6b80 1 -- 172.21.15.1:0/1825561148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636ad9dfbe0 msgr2=0x5636ad9ec180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:53.848 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.835+0000 7feb6c9b6b80 1 --2- 172.21.15.1:0/1825561148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636ad9dfbe0 0x5636ad9ec180 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7feb4401d760 tx=0x7feb440047c0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.835+0000 7feb6c9b6b80 1 -- 172.21.15.1:0/1825561148 shutdown_connections 2023-11-16T16:47:53.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.835+0000 7feb6c9b6b80 1 --2- 172.21.15.1:0/1825561148 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5636ad9f9af0 0x5636ad9fbf00 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.835+0000 7feb6c9b6b80 1 --2- 172.21.15.1:0/1825561148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636ad9e4930 0x5636ad9e9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.835+0000 7feb41ffb700 0 client.16740 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:47:53.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.835+0000 7feb6c9b6b80 1 --2- 172.21.15.1:0/1825561148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636ad9dfbe0 0x5636ad9ec180 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.835+0000 7feb6c9b6b80 1 --2- 172.21.15.1:0/1825561148 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636ad9e04f0 0x5636ad9e9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.835+0000 7feb6c9b6b80 1 -- 172.21.15.1:0/1825561148 >> 172.21.15.1:0/1825561148 conn(0x5636ad9d53e0 msgr2=0x5636ad9d9e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:53.850 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.836+0000 7feb6c9b6b80 1 -- 172.21.15.1:0/1825561148 shutdown_connections 2023-11-16T16:47:53.850 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.836+0000 7feb6c9b6b80 1 -- 172.21.15.1:0/1825561148 wait complete. 2023-11-16T16:47:53.850 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.836+0000 7feb6c9b6b80 20 client.16740 trim_cache size 0 max 16384 2023-11-16T16:47:53.859 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.853+0000 7feb6c9b6b80 1 Processor -- start 2023-11-16T16:47:53.859 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.853+0000 7feb6c9b6b80 1 -- start start 2023-11-16T16:47:53.859 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.853+0000 7feb6c9b6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636ad9dfbe0 0x5636ad9e2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:53.859 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.853+0000 7feb6c9b6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636ad9e04f0 0x5636ad9f0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:53.859 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.853+0000 7feb6c9b6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636ad9e4930 0x5636ad9e4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:53.860 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.853+0000 7feb6c9b6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ad9d4fa0 con 0x5636ad9dfbe0 2023-11-16T16:47:53.860 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.853+0000 7feb6c9b6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ad9ea380 con 0x5636ad9e04f0 2023-11-16T16:47:53.860 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.853+0000 7feb6c9b6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ad9e3320 con 0x5636ad9e4930 2023-11-16T16:47:53.860 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.853+0000 7feb5a59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636ad9dfbe0 0x5636ad9e2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:53.860 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.853+0000 7feb5a59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636ad9dfbe0 0x5636ad9e2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46606/0 (socket says 172.21.15.1:46606) 2023-11-16T16:47:53.860 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.853+0000 7feb5a59c700 1 -- 172.21.15.1:0/2431031985 learned_addr learned my addr 172.21.15.1:0/2431031985 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:53.860 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.853+0000 7feb5ad9d700 1 --2- 172.21.15.1:0/2431031985 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636ad9e4930 0x5636ad9e4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:53.860 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.853+0000 7feb59d9b700 1 --2- 172.21.15.1:0/2431031985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636ad9e04f0 0x5636ad9f0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:53.860 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.854+0000 7feb5a59c700 1 -- 172.21.15.1:0/2431031985 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636ad9e4930 msgr2=0x5636ad9e4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:53.861 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.854+0000 7feb5a59c700 1 --2- 172.21.15.1:0/2431031985 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636ad9e4930 0x5636ad9e4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.861 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.854+0000 7feb5a59c700 1 -- 172.21.15.1:0/2431031985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636ad9e04f0 msgr2=0x5636ad9f0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:53.861 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.854+0000 7feb5a59c700 1 --2- 172.21.15.1:0/2431031985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636ad9e04f0 0x5636ad9f0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.861 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.854+0000 7feb5a59c700 1 -- 172.21.15.1:0/2431031985 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636ad9e6140 con 0x5636ad9dfbe0 2023-11-16T16:47:53.861 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.854+0000 7feb59d9b700 1 --2- 172.21.15.1:0/2431031985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636ad9e04f0 0x5636ad9f0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:47:53.861 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.854+0000 7feb5a59c700 1 --2- 172.21.15.1:0/2431031985 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636ad9dfbe0 0x5636ad9e2220 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7feb44016040 tx=0x7feb44009950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c49510561a7614fe server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:53.861 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.854+0000 7feb517fa700 1 -- 172.21.15.1:0/2431031985 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4402d3c0 con 0x5636ad9dfbe0 2023-11-16T16:47:53.861 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.854+0000 7feb517fa700 1 -- 172.21.15.1:0/2431031985 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb44032030 con 0x5636ad9dfbe0 2023-11-16T16:47:53.861 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.854+0000 7feb517fa700 1 -- 172.21.15.1:0/2431031985 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb44003680 con 0x5636ad9dfbe0 2023-11-16T16:47:53.862 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.854+0000 7feb6c9b6b80 1 -- 172.21.15.1:0/2431031985 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636ad9dfbe0 msgr2=0x5636ad9e2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:53.862 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.854+0000 7feb6c9b6b80 1 --2- 172.21.15.1:0/2431031985 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636ad9dfbe0 0x5636ad9e2220 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7feb44016040 tx=0x7feb44009950 comp rx=0 tx=0).stop 2023-11-16T16:47:53.862 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.854+0000 7feb6c9b6b80 1 -- 172.21.15.1:0/2431031985 shutdown_connections 2023-11-16T16:47:53.862 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.854+0000 7feb6c9b6b80 1 --2- 172.21.15.1:0/2431031985 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636ad9e4930 0x5636ad9e4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.862 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.854+0000 7feb6c9b6b80 1 --2- 172.21.15.1:0/2431031985 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636ad9dfbe0 0x5636ad9e2220 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.862 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.854+0000 7feb6c9b6b80 1 --2- 172.21.15.1:0/2431031985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636ad9e04f0 0x5636ad9f0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.862 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.854+0000 7feb6c9b6b80 1 -- 172.21.15.1:0/2431031985 >> 172.21.15.1:0/2431031985 conn(0x5636ad9d53e0 msgr2=0x5636ad9e3940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:53.862 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.855+0000 7feb6c9b6b80 1 -- 172.21.15.1:0/2431031985 shutdown_connections 2023-11-16T16:47:53.863 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.855+0000 7feb6c9b6b80 1 -- 172.21.15.1:0/2431031985 wait complete. 2023-11-16T16:47:53.863 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.855+0000 7feb6c9b6b80 1 -- start start 2023-11-16T16:47:53.863 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.855+0000 7feb6c9b6b80 1 Processor -- start 2023-11-16T16:47:53.863 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.855+0000 7feb6c9b6b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636ad9dfbe0 0x5636ad9e8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:53.863 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.855+0000 7feb6c9b6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636ad9e04f0 0x5636ad9e8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:53.863 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.855+0000 7feb6c9b6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636ad9e4930 0x5636ad9e8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:53.863 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.855+0000 7feb6c9b6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ada4d9a0 con 0x5636ad9e04f0 2023-11-16T16:47:53.863 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.855+0000 7feb6c9b6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ada4db10 con 0x5636ad9e4930 2023-11-16T16:47:53.864 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.855+0000 7feb6c9b6b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ada4dcd0 con 0x5636ad9dfbe0 2023-11-16T16:47:53.864 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.855+0000 7feb6c9b6b80 10 client.0 _ll_register_callbacks cb 0x7feb44011ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:47:53.864 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.855+0000 7feb59d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636ad9e04f0 0x5636ad9e8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:53.864 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.855+0000 7feb59d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636ad9e04f0 0x5636ad9e8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46614/0 (socket says 172.21.15.1:46614) 2023-11-16T16:47:53.864 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.855+0000 7feb59d9b700 1 -- 172.21.15.1:0/237877390 learned_addr learned my addr 172.21.15.1:0/237877390 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:53.864 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.855+0000 7feb5ad9d700 1 --2- 172.21.15.1:0/237877390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636ad9e4930 0x5636ad9e8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:53.864 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.855+0000 7feb5a59c700 1 --2- 172.21.15.1:0/237877390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636ad9dfbe0 0x5636ad9e8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:53.864 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.856+0000 7feb59d9b700 1 -- 172.21.15.1:0/237877390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636ad9dfbe0 msgr2=0x5636ad9e8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:53.864 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.856+0000 7feb59d9b700 1 --2- 172.21.15.1:0/237877390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636ad9dfbe0 0x5636ad9e8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.865 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.856+0000 7feb59d9b700 1 -- 172.21.15.1:0/237877390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636ad9e4930 msgr2=0x5636ad9e8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:53.865 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.856+0000 7feb59d9b700 1 --2- 172.21.15.1:0/237877390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636ad9e4930 0x5636ad9e8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.865 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.856+0000 7feb59d9b700 1 -- 172.21.15.1:0/237877390 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636ad8f7210 con 0x5636ad9e04f0 2023-11-16T16:47:53.865 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.856+0000 7feb5a59c700 1 --2- 172.21.15.1:0/237877390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636ad9dfbe0 0x5636ad9e8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:47:53.865 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.856+0000 7feb5ad9d700 1 --2- 172.21.15.1:0/237877390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636ad9e4930 0x5636ad9e8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:47:53.865 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.856+0000 7feb59d9b700 1 --2- 172.21.15.1:0/237877390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636ad9e04f0 0x5636ad9e8750 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7feb4c0098e0 tx=0x7feb4c00bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a87703f6997102bf server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:53.865 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.856+0000 7feb41ffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:47:53.865 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.856+0000 7feb41ffb700 1 -- 172.21.15.1:0/237877390 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4c01d070 con 0x5636ad9e04f0 2023-11-16T16:47:53.866 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.856+0000 7feb41ffb700 1 -- 172.21.15.1:0/237877390 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7feb4c0270f0 con 0x5636ad9e04f0 2023-11-16T16:47:53.866 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.856+0000 7feb6c9b6b80 1 -- 172.21.15.1:0/237877390 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feb540978c0 con 0x5636ad9e04f0 2023-11-16T16:47:53.866 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.856+0000 7feb41ffb700 1 -- 172.21.15.1:0/237877390 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb4c0272b0 con 0x5636ad9e04f0 2023-11-16T16:47:53.866 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.856+0000 7feb6c9b6b80 10 client.16746 fetch_fsmap learned FSMap version 28 2023-11-16T16:47:53.866 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.856+0000 7feb6c9b6b80 1 -- 172.21.15.1:0/237877390 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5636ad75ee60 con 0x5636ad9e04f0 2023-11-16T16:47:53.866 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.856+0000 7feb41ffb700 1 -- 172.21.15.1:0/237877390 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb4c036020 con 0x5636ad9e04f0 2023-11-16T16:47:53.866 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.856+0000 7feb41ffb700 1 -- 172.21.15.1:0/237877390 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7feb4c004600 con 0x5636ad9e04f0 2023-11-16T16:47:53.866 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.857+0000 7feb6c9b6b80 10 client.16746 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:47:53.867 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.857+0000 7feb6c9b6b80 10 client.16746 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:47:53.867 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.857+0000 7feb6c9b6b80 20 client.16746 populate_metadata read hostname 'smithi001' 2023-11-16T16:47:53.867 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.857+0000 7feb6c9b6b80 1 --2- 172.21.15.1:0/237877390 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5636ada4f480 0x5636ada51890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:53.867 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.857+0000 7feb6c9b6b80 4 client.16746 mds_command: new command op to 24521 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-16T16:47:53.867 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.857+0000 7feb6c9b6b80 1 -- 172.21.15.1:0/237877390 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5636ad8ff6e0 con 0x5636ada4f480 2023-11-16T16:47:53.867 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.857+0000 7feb5a59c700 1 --2- 172.21.15.1:0/237877390 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5636ada4f480 0x5636ada51890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:53.867 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.857+0000 7feb41ffb700 10 client.16746 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:47:53.867 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.857+0000 7feb5a59c700 1 --2- 172.21.15.1:0/237877390 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5636ada4f480 0x5636ada51890 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2eeee3f9cfb787b5 server_cookie=7e1861866cf5cdf7 in_seq=0 out_seq=0 2023-11-16T16:47:53.868 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.859+0000 7feb41ffb700 1 -- 172.21.15.1:0/237877390 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5636ad8ff6e0 con 0x5636ada4f480 2023-11-16T16:47:53.869 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.859+0000 7feb41ffb700 10 client.16746 handle_command_reply: tid=0 2023-11-16T16:47:53.869 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.859+0000 7feb6c9b6b80 1 client.16746 shutdown 2023-11-16T16:47:53.869 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.859+0000 7feb6c9b6b80 1 -- 172.21.15.1:0/237877390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636ad9e04f0 msgr2=0x5636ad9e8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:53.869 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.859+0000 7feb6c9b6b80 1 --2- 172.21.15.1:0/237877390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636ad9e04f0 0x5636ad9e8750 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7feb4c0098e0 tx=0x7feb4c00bb50 comp rx=0 tx=0).stop 2023-11-16T16:47:53.869 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.859+0000 7feb6c9b6b80 1 -- 172.21.15.1:0/237877390 shutdown_connections 2023-11-16T16:47:53.869 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.859+0000 7feb6c9b6b80 1 --2- 172.21.15.1:0/237877390 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5636ada4f480 0x5636ada51890 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.869 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.859+0000 7feb6c9b6b80 1 --2- 172.21.15.1:0/237877390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5636ad9e4930 0x5636ad9e8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.870 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.859+0000 7feb6c9b6b80 1 --2- 172.21.15.1:0/237877390 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5636ad9dfbe0 0x5636ad9e8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.870 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.859+0000 7feb41ffb700 0 client.16746 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:47:53.870 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.859+0000 7feb6c9b6b80 1 --2- 172.21.15.1:0/237877390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636ad9e04f0 0x5636ad9e8750 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.870 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.859+0000 7feb6c9b6b80 1 -- 172.21.15.1:0/237877390 >> 172.21.15.1:0/237877390 conn(0x5636ad9d53e0 msgr2=0x5636ad9e41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:53.870 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.859+0000 7feb6c9b6b80 1 -- 172.21.15.1:0/237877390 shutdown_connections 2023-11-16T16:47:53.870 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.859+0000 7feb6c9b6b80 1 -- 172.21.15.1:0/237877390 wait complete. 2023-11-16T16:47:53.870 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.859+0000 7feb6c9b6b80 20 client.16746 trim_cache size 0 max 16384 2023-11-16T16:47:53.871 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:47:53.871 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2023-11-16T16:47:53.871 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "bc40e857-edb0-4fed-8580-e1fd05b64e9f", 2023-11-16T16:47:53.871 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2023-11-16T16:47:53.872 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:47:53.872 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.860+0000 7feb6082a700 1 -- 172.21.15.1:0/229609424 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7feb480a12c0 msgr2=0x7feb480a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:53.872 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.860+0000 7feb6082a700 1 --2- 172.21.15.1:0/229609424 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7feb480a12c0 0x7feb480a3770 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7feb5c068db0 tx=0x7feb44009260 comp rx=0 tx=0).stop 2023-11-16T16:47:53.873 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.860+0000 7feb6082a700 1 -- 172.21.15.1:0/229609424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb5c1051c0 msgr2=0x7feb5c1a2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:53.873 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.860+0000 7feb6082a700 1 --2- 172.21.15.1:0/229609424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb5c1051c0 0x7feb5c1a2210 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7feb54007ee0 tx=0x7feb5400d3b0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.873 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.860+0000 7feb6082a700 1 -- 172.21.15.1:0/229609424 shutdown_connections 2023-11-16T16:47:53.873 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.860+0000 7feb6082a700 1 --2- 172.21.15.1:0/229609424 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7feb480a12c0 0x7feb480a3770 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.873 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.860+0000 7feb6082a700 1 --2- 172.21.15.1:0/229609424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb5c1051c0 0x7feb5c1a2210 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.873 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.860+0000 7feb6082a700 1 --2- 172.21.15.1:0/229609424 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb5c0ffa70 0x7feb5c197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.873 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.860+0000 7feb6082a700 1 --2- 172.21.15.1:0/229609424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb5c100420 0x7feb5c1976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:53.873 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.860+0000 7feb6082a700 1 -- 172.21.15.1:0/229609424 >> 172.21.15.1:0/229609424 conn(0x7feb5c076370 msgr2=0x7feb5c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:53.873 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.860+0000 7feb6082a700 1 -- 172.21.15.1:0/229609424 shutdown_connections 2023-11-16T16:47:53.874 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:53.860+0000 7feb6082a700 1 -- 172.21.15.1:0/229609424 wait complete. 2023-11-16T16:47:53.875 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-16T16:47:54.004 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.002+0000 7f03c0015700 1 -- 172.21.15.1:0/1908046460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03b81002a0 msgr2=0x7f03b810bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:54.004 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.002+0000 7f03c0015700 1 --2- 172.21.15.1:0/1908046460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03b81002a0 0x7f03b810bfd0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f03b000b0d0 tx=0x7f03b000b3e0 comp rx=0 tx=0).stop 2023-11-16T16:47:54.004 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.002+0000 7f03c0015700 1 -- 172.21.15.1:0/1908046460 shutdown_connections 2023-11-16T16:47:54.004 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.002+0000 7f03c0015700 1 --2- 172.21.15.1:0/1908046460 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03b81002a0 0x7f03b810bfd0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:54.004 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.002+0000 7f03c0015700 1 --2- 172.21.15.1:0/1908046460 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03b8104140 0x7f03b8104510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:54.004 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.002+0000 7f03c0015700 1 --2- 172.21.15.1:0/1908046460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b80ff8f0 0x7f03b80ffd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:54.005 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.002+0000 7f03c0015700 1 -- 172.21.15.1:0/1908046460 >> 172.21.15.1:0/1908046460 conn(0x7f03b8075880 msgr2=0x7f03b8075c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:54.005 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.002+0000 7f03c0015700 1 -- 172.21.15.1:0/1908046460 shutdown_connections 2023-11-16T16:47:54.005 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.002+0000 7f03c0015700 1 -- 172.21.15.1:0/1908046460 wait complete. 2023-11-16T16:47:54.005 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.002+0000 7f03c0015700 1 Processor -- start 2023-11-16T16:47:54.005 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.002+0000 7f03c0015700 1 -- start start 2023-11-16T16:47:54.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.003+0000 7f03c0015700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b80ff8f0 0x7f03b8195370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:54.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.003+0000 7f03c0015700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03b81002a0 0x7f03b81958b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:54.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.003+0000 7f03c0015700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03b8104140 0x7f03b8199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:54.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.003+0000 7f03c0015700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b819a350 con 0x7f03b8104140 2023-11-16T16:47:54.007 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.003+0000 7f03c0015700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b819a4c0 con 0x7f03b80ff8f0 2023-11-16T16:47:54.008 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.003+0000 7f03c0015700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b819a680 con 0x7f03b81002a0 2023-11-16T16:47:54.008 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.003+0000 7f03be5b2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03b8104140 0x7f03b8199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:54.008 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.003+0000 7f03be5b2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03b8104140 0x7f03b8199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46630/0 (socket says 172.21.15.1:46630) 2023-11-16T16:47:54.008 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.003+0000 7f03be5b2700 1 -- 172.21.15.1:0/1846264795 learned_addr learned my addr 172.21.15.1:0/1846264795 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:54.008 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.003+0000 7f03bddb1700 1 --2- 172.21.15.1:0/1846264795 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b80ff8f0 0x7f03b8195370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:54.008 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.003+0000 7f03bd5b0700 1 --2- 172.21.15.1:0/1846264795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03b81002a0 0x7f03b81958b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:54.008 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.003+0000 7f03be5b2700 1 -- 172.21.15.1:0/1846264795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03b81002a0 msgr2=0x7f03b81958b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:54.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.003+0000 7f03be5b2700 1 --2- 172.21.15.1:0/1846264795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03b81002a0 0x7f03b81958b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:54.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.003+0000 7f03be5b2700 1 -- 172.21.15.1:0/1846264795 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b80ff8f0 msgr2=0x7f03b8195370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:54.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.003+0000 7f03be5b2700 1 --2- 172.21.15.1:0/1846264795 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b80ff8f0 0x7f03b8195370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:54.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.003+0000 7f03be5b2700 1 -- 172.21.15.1:0/1846264795 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b0009d00 con 0x7f03b8104140 2023-11-16T16:47:54.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.003+0000 7f03be5b2700 1 --2- 172.21.15.1:0/1846264795 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03b8104140 0x7f03b8199d10 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f03b000d010 tx=0x7f03b0007920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:54.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.003+0000 7f03aeffd700 1 -- 172.21.15.1:0/1846264795 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b000e040 con 0x7f03b8104140 2023-11-16T16:47:54.009 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.003+0000 7f03c0015700 1 -- 172.21.15.1:0/1846264795 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03b810f270 con 0x7f03b8104140 2023-11-16T16:47:54.010 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.003+0000 7f03aeffd700 1 -- 172.21.15.1:0/1846264795 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03b0003e10 con 0x7f03b8104140 2023-11-16T16:47:54.010 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.003+0000 7f03aeffd700 1 -- 172.21.15.1:0/1846264795 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b0012ad0 con 0x7f03b8104140 2023-11-16T16:47:54.010 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.004+0000 7f03c0015700 1 -- 172.21.15.1:0/1846264795 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03b810f0e0 con 0x7f03b8104140 2023-11-16T16:47:54.010 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:53 smithi026 ceph-mon[100112]: pgmap v269: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:47:54.016 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.006+0000 7f03aeffd700 1 -- 172.21.15.1:0/1846264795 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f03b0012c30 con 0x7f03b8104140 2023-11-16T16:47:54.016 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.006+0000 7f03aeffd700 1 --2- 172.21.15.1:0/1846264795 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f03a40a12c0 0x7f03a40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:54.016 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.007+0000 7f03aeffd700 1 -- 172.21.15.1:0/1846264795 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f03b00ce460 con 0x7f03b8104140 2023-11-16T16:47:54.017 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.007+0000 7f03bddb1700 1 --2- 172.21.15.1:0/1846264795 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f03a40a12c0 0x7f03a40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:54.017 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.007+0000 7f03bddb1700 1 --2- 172.21.15.1:0/1846264795 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f03a40a12c0 0x7f03a40a3770 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f03b4005950 tx=0x7f03b40058e0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:54.017 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.007+0000 7f03cc1a1b80 1 Processor -- start 2023-11-16T16:47:54.017 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.008+0000 7f03cc1a1b80 1 -- start start 2023-11-16T16:47:54.017 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.008+0000 7f03cc1a1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558121b8ab60 0x558121b8af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:54.017 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.008+0000 7f03cc1a1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558121b8b470 0x558121b8f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:54.017 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.008+0000 7f03cc1a1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558121b8f8b0 0x558121b8fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:54.018 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.008+0000 7f03cc1a1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558121a90060 con 0x558121b8b470 2023-11-16T16:47:54.018 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.008+0000 7f03cc1a1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558121ae28a0 con 0x558121b8f8b0 2023-11-16T16:47:54.018 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.008+0000 7f03cc1a1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558121adbd30 con 0x558121b8ab60 2023-11-16T16:47:54.018 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.008+0000 7f03bd5b0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558121b8b470 0x558121b8f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:54.018 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.008+0000 7f03bd5b0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558121b8b470 0x558121b8f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46636/0 (socket says 172.21.15.1:46636) 2023-11-16T16:47:54.018 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.008+0000 7f03bddb1700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558121b8ab60 0x558121b8af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:54.018 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.008+0000 7f03bd5b0700 1 -- 172.21.15.1:0/2794616215 learned_addr learned my addr 172.21.15.1:0/2794616215 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:54.019 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.008+0000 7f03be5b2700 1 --2- 172.21.15.1:0/2794616215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558121b8f8b0 0x558121b8fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:54.019 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.008+0000 7f03bd5b0700 1 -- 172.21.15.1:0/2794616215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558121b8ab60 msgr2=0x558121b8af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:54.019 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.008+0000 7f03bd5b0700 1 --2- 172.21.15.1:0/2794616215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558121b8ab60 0x558121b8af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:54.019 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.008+0000 7f03bd5b0700 1 -- 172.21.15.1:0/2794616215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558121b8f8b0 msgr2=0x558121b8fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:54.019 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.008+0000 7f03bd5b0700 1 --2- 172.21.15.1:0/2794616215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558121b8f8b0 0x558121b8fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:54.019 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.008+0000 7f03bd5b0700 1 -- 172.21.15.1:0/2794616215 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55812195d630 con 0x558121b8b470 2023-11-16T16:47:54.019 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.008+0000 7f03bddb1700 1 --2- 172.21.15.1:0/2794616215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558121b8ab60 0x558121b8af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:47:54.019 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.008+0000 7f03bd5b0700 1 --2- 172.21.15.1:0/2794616215 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558121b8b470 0x558121b8f370 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f03a800db50 tx=0x7f03a800de60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dca09eb1f7bd7ca server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:54.020 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.009+0000 7f03a3fff700 1 -- 172.21.15.1:0/2794616215 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a800fd30 con 0x558121b8b470 2023-11-16T16:47:54.020 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.009+0000 7f03a3fff700 1 -- 172.21.15.1:0/2794616215 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03a800fe90 con 0x558121b8b470 2023-11-16T16:47:54.020 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.009+0000 7f03a3fff700 1 -- 172.21.15.1:0/2794616215 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a8011990 con 0x558121b8b470 2023-11-16T16:47:54.020 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.009+0000 7f03cc1a1b80 1 -- 172.21.15.1:0/2794616215 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558121b8b470 msgr2=0x558121b8f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:54.020 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.009+0000 7f03cc1a1b80 1 --2- 172.21.15.1:0/2794616215 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558121b8b470 0x558121b8f370 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f03a800db50 tx=0x7f03a800de60 comp rx=0 tx=0).stop 2023-11-16T16:47:54.020 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.009+0000 7f03cc1a1b80 1 -- 172.21.15.1:0/2794616215 shutdown_connections 2023-11-16T16:47:54.020 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.009+0000 7f03cc1a1b80 1 --2- 172.21.15.1:0/2794616215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558121b8f8b0 0x558121b8fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:54.020 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.009+0000 7f03cc1a1b80 1 --2- 172.21.15.1:0/2794616215 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558121b8ab60 0x558121b8af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:54.021 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.009+0000 7f03cc1a1b80 1 --2- 172.21.15.1:0/2794616215 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558121b8b470 0x558121b8f370 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:54.021 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.009+0000 7f03cc1a1b80 1 -- 172.21.15.1:0/2794616215 >> 172.21.15.1:0/2794616215 conn(0x558121b80360 msgr2=0x558121b85b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:54.021 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.009+0000 7f03cc1a1b80 1 -- 172.21.15.1:0/2794616215 shutdown_connections 2023-11-16T16:47:54.021 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.009+0000 7f03cc1a1b80 1 -- 172.21.15.1:0/2794616215 wait complete. 2023-11-16T16:47:54.021 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.010+0000 7f03cc1a1b80 1 -- start start 2023-11-16T16:47:54.021 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.010+0000 7f03cc1a1b80 1 Processor -- start 2023-11-16T16:47:54.021 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.010+0000 7f03cc1a1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558121b8ab60 0x558121b97020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:54.021 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.010+0000 7f03cc1a1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558121b8b470 0x558121b942a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:54.021 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.010+0000 7f03cc1a1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558121b8f8b0 0x558121b94920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:54.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.010+0000 7f03cc1a1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558121a7d8e0 con 0x558121b8b470 2023-11-16T16:47:54.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.010+0000 7f03cc1a1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558121a7d580 con 0x558121b8ab60 2023-11-16T16:47:54.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.010+0000 7f03cc1a1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55812195f920 con 0x558121b8f8b0 2023-11-16T16:47:54.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.010+0000 7f03cc1a1b80 10 client.0 _ll_register_callbacks cb 0x558121a9b780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:47:54.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.010+0000 7f03bd5b0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558121b8b470 0x558121b942a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:54.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.010+0000 7f03bd5b0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558121b8b470 0x558121b942a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46642/0 (socket says 172.21.15.1:46642) 2023-11-16T16:47:54.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.010+0000 7f03bd5b0700 1 -- 172.21.15.1:0/3273018411 learned_addr learned my addr 172.21.15.1:0/3273018411 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:54.023 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.010+0000 7f03bddb1700 1 --2- 172.21.15.1:0/3273018411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558121b8ab60 0x558121b97020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:54.023 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.011+0000 7f03be5b2700 1 --2- 172.21.15.1:0/3273018411 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558121b8f8b0 0x558121b94920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:54.023 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.011+0000 7f03bd5b0700 1 -- 172.21.15.1:0/3273018411 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558121b8f8b0 msgr2=0x558121b94920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:54.023 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.011+0000 7f03bd5b0700 1 --2- 172.21.15.1:0/3273018411 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558121b8f8b0 0x558121b94920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:54.023 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.011+0000 7f03bd5b0700 1 -- 172.21.15.1:0/3273018411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558121b8ab60 msgr2=0x558121b97020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:54.023 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.011+0000 7f03bd5b0700 1 --2- 172.21.15.1:0/3273018411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558121b8ab60 0x558121b97020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:54.023 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.011+0000 7f03bd5b0700 1 -- 172.21.15.1:0/3273018411 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a800d7b0 con 0x558121b8b470 2023-11-16T16:47:54.023 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.011+0000 7f03bddb1700 1 --2- 172.21.15.1:0/3273018411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558121b8ab60 0x558121b97020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:47:54.023 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.011+0000 7f03bd5b0700 1 --2- 172.21.15.1:0/3273018411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558121b8b470 0x558121b942a0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f03a8000c00 tx=0x7f03a801a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe713d377f36533 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:54.024 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.011+0000 7f03a17fa700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:47:54.024 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.011+0000 7f03a17fa700 1 -- 172.21.15.1:0/3273018411 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a8011590 con 0x558121b8b470 2023-11-16T16:47:54.024 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.011+0000 7f03a17fa700 1 -- 172.21.15.1:0/3273018411 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f03a8021070 con 0x558121b8b470 2023-11-16T16:47:54.024 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.011+0000 7f03cc1a1b80 1 -- 172.21.15.1:0/3273018411 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558121b7db20 con 0x558121b8b470 2023-11-16T16:47:54.024 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.011+0000 7f03a17fa700 1 -- 172.21.15.1:0/3273018411 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03a80116f0 con 0x558121b8b470 2023-11-16T16:47:54.024 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.011+0000 7f03cc1a1b80 10 client.16758 fetch_fsmap learned FSMap version 28 2023-11-16T16:47:54.024 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.011+0000 7f03cc1a1b80 1 -- 172.21.15.1:0/3273018411 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558121aa2190 con 0x558121b8b470 2023-11-16T16:47:54.024 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.012+0000 7f03a17fa700 1 -- 172.21.15.1:0/3273018411 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03a8032020 con 0x558121b8b470 2023-11-16T16:47:54.024 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.012+0000 7f03a17fa700 1 -- 172.21.15.1:0/3273018411 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f03a8025870 con 0x558121b8b470 2023-11-16T16:47:54.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.012+0000 7f03cc1a1b80 10 client.16758 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:47:54.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.012+0000 7f03cc1a1b80 10 client.16758 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:47:54.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.012+0000 7f03cc1a1b80 20 client.16758 populate_metadata read hostname 'smithi001' 2023-11-16T16:47:54.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.012+0000 7f03cc1a1b80 1 --2- 172.21.15.1:0/3273018411 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x558121ba4740 0x558121ba6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:54.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.012+0000 7f03cc1a1b80 4 client.16758 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:47:54.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.012+0000 7f03cc1a1b80 1 -- 172.21.15.1:0/3273018411 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558121a9e0a0 con 0x558121ba4740 2023-11-16T16:47:54.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.012+0000 7f03bddb1700 1 --2- 172.21.15.1:0/3273018411 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x558121ba4740 0x558121ba6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:54.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.012+0000 7f03a17fa700 10 client.16758 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:47:54.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.012+0000 7f03bddb1700 1 --2- 172.21.15.1:0/3273018411 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x558121ba4740 0x558121ba6b50 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=773deed72924273 server_cookie=bc68f364aae5d8a6 in_seq=0 out_seq=0 2023-11-16T16:47:54.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.013+0000 7f03a17fa700 1 -- 172.21.15.1:0/3273018411 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558121a9e0a0 con 0x558121ba4740 2023-11-16T16:47:54.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.013+0000 7f03a17fa700 10 client.16758 handle_command_reply: tid=0 2023-11-16T16:47:54.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.013+0000 7f03cc1a1b80 1 client.16758 shutdown 2023-11-16T16:47:54.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.013+0000 7f03cc1a1b80 1 -- 172.21.15.1:0/3273018411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558121b8b470 msgr2=0x558121b942a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:54.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.013+0000 7f03cc1a1b80 1 --2- 172.21.15.1:0/3273018411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558121b8b470 0x558121b942a0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f03a8000c00 tx=0x7f03a801a690 comp rx=0 tx=0).stop 2023-11-16T16:47:54.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.013+0000 7f03cc1a1b80 1 -- 172.21.15.1:0/3273018411 shutdown_connections 2023-11-16T16:47:54.026 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.013+0000 7f03cc1a1b80 1 --2- 172.21.15.1:0/3273018411 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x558121ba4740 0x558121ba6b50 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:54.027 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.013+0000 7f03cc1a1b80 1 --2- 172.21.15.1:0/3273018411 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558121b8f8b0 0x558121b94920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:54.027 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.013+0000 7f03a17fa700 0 client.16758 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:47:54.027 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.013+0000 7f03cc1a1b80 1 --2- 172.21.15.1:0/3273018411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558121b8ab60 0x558121b97020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:54.027 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.013+0000 7f03cc1a1b80 1 --2- 172.21.15.1:0/3273018411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558121b8b470 0x558121b942a0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:54.027 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.013+0000 7f03cc1a1b80 1 -- 172.21.15.1:0/3273018411 >> 172.21.15.1:0/3273018411 conn(0x558121b80360 msgr2=0x558121b84de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:54.027 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.013+0000 7f03cc1a1b80 1 -- 172.21.15.1:0/3273018411 shutdown_connections 2023-11-16T16:47:54.027 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.013+0000 7f03cc1a1b80 1 -- 172.21.15.1:0/3273018411 wait complete. 2023-11-16T16:47:54.027 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.014+0000 7f03cc1a1b80 20 client.16758 trim_cache size 0 max 16384 2023-11-16T16:47:54.034 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.030+0000 7f03cc1a1b80 1 Processor -- start 2023-11-16T16:47:54.034 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.030+0000 7f03cc1a1b80 1 -- start start 2023-11-16T16:47:54.034 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.031+0000 7f03cc1a1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558121b8ab60 0x558121b9b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:54.034 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.031+0000 7f03cc1a1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558121b8b470 0x558121b96bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:54.034 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.031+0000 7f03cc1a1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558121b8f8b0 0x558121b85560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:54.035 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.031+0000 7f03cc1a1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558121b8bbc0 con 0x558121b8ab60 2023-11-16T16:47:54.035 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.031+0000 7f03cc1a1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558121b97730 con 0x558121b8f8b0 2023-11-16T16:47:54.035 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.031+0000 7f03cc1a1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558121b8e1c0 con 0x558121b8b470 2023-11-16T16:47:54.035 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.031+0000 7f03bddb1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558121b8ab60 0x558121b9b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:54.035 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.031+0000 7f03bddb1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558121b8ab60 0x558121b9b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46648/0 (socket says 172.21.15.1:46648) 2023-11-16T16:47:54.035 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.031+0000 7f03bddb1700 1 -- 172.21.15.1:0/3564502570 learned_addr learned my addr 172.21.15.1:0/3564502570 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:54.035 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.031+0000 7f03bd5b0700 1 --2- 172.21.15.1:0/3564502570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558121b8b470 0x558121b96bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:54.035 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.031+0000 7f03bddb1700 1 -- 172.21.15.1:0/3564502570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558121b8b470 msgr2=0x558121b96bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:54.036 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.031+0000 7f03bddb1700 1 --2- 172.21.15.1:0/3564502570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558121b8b470 0x558121b96bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:54.036 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.031+0000 7f03bddb1700 1 -- 172.21.15.1:0/3564502570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558121b8f8b0 msgr2=0x558121b85560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:54.036 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.031+0000 7f03be5b2700 1 --2- 172.21.15.1:0/3564502570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558121b8f8b0 0x558121b85560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:54.036 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.031+0000 7f03bddb1700 1 --2- 172.21.15.1:0/3564502570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558121b8f8b0 0x558121b85560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:54.036 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.031+0000 7f03bddb1700 1 -- 172.21.15.1:0/3564502570 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558121b90fe0 con 0x558121b8ab60 2023-11-16T16:47:54.036 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.031+0000 7f03bddb1700 1 --2- 172.21.15.1:0/3564502570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558121b8ab60 0x558121b9b9a0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x558121ba7460 tx=0x7f03b4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19294df768d68b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:54.036 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.031+0000 7f03acff9700 1 -- 172.21.15.1:0/3564502570 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b4025040 con 0x558121b8ab60 2023-11-16T16:47:54.036 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.032+0000 7f03acff9700 1 -- 172.21.15.1:0/3564502570 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03b4011cc0 con 0x558121b8ab60 2023-11-16T16:47:54.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.032+0000 7f03acff9700 1 -- 172.21.15.1:0/3564502570 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b401f690 con 0x558121b8ab60 2023-11-16T16:47:54.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.032+0000 7f03cc1a1b80 1 -- 172.21.15.1:0/3564502570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558121b8ab60 msgr2=0x558121b9b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:54.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.032+0000 7f03cc1a1b80 1 --2- 172.21.15.1:0/3564502570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558121b8ab60 0x558121b9b9a0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x558121ba7460 tx=0x7f03b4015000 comp rx=0 tx=0).stop 2023-11-16T16:47:54.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.032+0000 7f03cc1a1b80 1 -- 172.21.15.1:0/3564502570 shutdown_connections 2023-11-16T16:47:54.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.032+0000 7f03cc1a1b80 1 --2- 172.21.15.1:0/3564502570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558121b8f8b0 0x558121b85560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:54.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.032+0000 7f03cc1a1b80 1 --2- 172.21.15.1:0/3564502570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558121b8ab60 0x558121b9b9a0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:54.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.032+0000 7f03cc1a1b80 1 --2- 172.21.15.1:0/3564502570 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558121b8b470 0x558121b96bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:54.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.032+0000 7f03cc1a1b80 1 -- 172.21.15.1:0/3564502570 >> 172.21.15.1:0/3564502570 conn(0x558121b80360 msgr2=0x558121ba0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:54.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.032+0000 7f03cc1a1b80 1 -- 172.21.15.1:0/3564502570 shutdown_connections 2023-11-16T16:47:54.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.032+0000 7f03cc1a1b80 1 -- 172.21.15.1:0/3564502570 wait complete. 2023-11-16T16:47:54.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.032+0000 7f03cc1a1b80 1 -- start start 2023-11-16T16:47:54.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.033+0000 7f03cc1a1b80 1 Processor -- start 2023-11-16T16:47:54.041 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.033+0000 7f03cc1a1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558121b8ab60 0x558121b93c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:54.041 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.033+0000 7f03cc1a1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558121b8b470 0x558121b941d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:54.042 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.033+0000 7f03cc1a1b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558121b8f8b0 0x558121b92a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:54.042 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.033+0000 7f03cc1a1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558121b92f90 con 0x558121b8b470 2023-11-16T16:47:54.042 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.033+0000 7f03cc1a1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558121b93100 con 0x558121b8ab60 2023-11-16T16:47:54.042 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.033+0000 7f03cc1a1b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x558121b932c0 con 0x558121b8f8b0 2023-11-16T16:47:54.042 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.033+0000 7f03cc1a1b80 10 client.0 _ll_register_callbacks cb 0x7f03b0005900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:47:54.042 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.033+0000 7f03bd5b0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558121b8b470 0x558121b941d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:54.042 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.033+0000 7f03bd5b0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558121b8b470 0x558121b941d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46656/0 (socket says 172.21.15.1:46656) 2023-11-16T16:47:54.042 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.033+0000 7f03bd5b0700 1 -- 172.21.15.1:0/3919225098 learned_addr learned my addr 172.21.15.1:0/3919225098 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:47:54.043 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.033+0000 7f03be5b2700 1 --2- 172.21.15.1:0/3919225098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558121b8f8b0 0x558121b92a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:54.043 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.033+0000 7f03bddb1700 1 --2- 172.21.15.1:0/3919225098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558121b8ab60 0x558121b93c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:54.043 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.033+0000 7f03bd5b0700 1 -- 172.21.15.1:0/3919225098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558121b8f8b0 msgr2=0x558121b92a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:54.043 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.033+0000 7f03bd5b0700 1 --2- 172.21.15.1:0/3919225098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558121b8f8b0 0x558121b92a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:54.043 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.033+0000 7f03bd5b0700 1 -- 172.21.15.1:0/3919225098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558121b8ab60 msgr2=0x558121b93c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:47:54.043 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.033+0000 7f03bd5b0700 1 --2- 172.21.15.1:0/3919225098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558121b8ab60 0x558121b93c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:54.043 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.033+0000 7f03bd5b0700 1 -- 172.21.15.1:0/3919225098 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03b400fbb0 con 0x558121b8b470 2023-11-16T16:47:54.043 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.033+0000 7f03bddb1700 1 --2- 172.21.15.1:0/3919225098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558121b8ab60 0x558121b93c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:47:54.044 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.034+0000 7f03bd5b0700 1 --2- 172.21.15.1:0/3919225098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558121b8b470 0x558121b941d0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x558121b80300 tx=0x7f03a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd89f8ad042f07b server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:47:54.044 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.034+0000 7f03a17fa700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:47:54.044 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.034+0000 7f03a17fa700 1 -- 172.21.15.1:0/3919225098 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a8025ea0 con 0x558121b8b470 2023-11-16T16:47:54.044 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.034+0000 7f03a17fa700 1 -- 172.21.15.1:0/3919225098 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f03a8021070 con 0x558121b8b470 2023-11-16T16:47:54.044 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.034+0000 7f03cc1a1b80 1 -- 172.21.15.1:0/3919225098 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f03a800fe90 con 0x558121b8b470 2023-11-16T16:47:54.044 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.034+0000 7f03a17fa700 1 -- 172.21.15.1:0/3919225098 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03a8005e60 con 0x558121b8b470 2023-11-16T16:47:54.044 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.034+0000 7f03cc1a1b80 10 client.16764 fetch_fsmap learned FSMap version 28 2023-11-16T16:47:54.044 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.034+0000 7f03cc1a1b80 1 -- 172.21.15.1:0/3919225098 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558121aa1ee0 con 0x558121b8b470 2023-11-16T16:47:54.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.034+0000 7f03a17fa700 1 -- 172.21.15.1:0/3919225098 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03a800fe90 con 0x558121b8b470 2023-11-16T16:47:54.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.034+0000 7f03a17fa700 1 -- 172.21.15.1:0/3919225098 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f03a8018b60 con 0x558121b8b470 2023-11-16T16:47:54.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.034+0000 7f03cc1a1b80 10 client.16764 fetch_fsmap finished waiting for FSMap version 28 2023-11-16T16:47:54.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.034+0000 7f03cc1a1b80 10 client.16764 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:47:54.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.034+0000 7f03cc1a1b80 20 client.16764 populate_metadata read hostname 'smithi001' 2023-11-16T16:47:54.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.034+0000 7f03cc1a1b80 1 --2- 172.21.15.1:0/3919225098 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x558121bf9150 0x558121bfb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:47:54.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.034+0000 7f03cc1a1b80 4 client.16764 mds_command: new command op to 24521 tid=0[{"prefix": "scrub status"}] 2023-11-16T16:47:54.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.034+0000 7f03cc1a1b80 1 -- 172.21.15.1:0/3919225098 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558121aaa660 con 0x558121bf9150 2023-11-16T16:47:54.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.035+0000 7f03bddb1700 1 --2- 172.21.15.1:0/3919225098 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x558121bf9150 0x558121bfb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:47:54.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.035+0000 7f03bddb1700 1 --2- 172.21.15.1:0/3919225098 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x558121bf9150 0x558121bfb560 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e3b0cc6d49de3f6 server_cookie=26ecf30800dc038a in_seq=0 out_seq=0 2023-11-16T16:47:54.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.035+0000 7f03a17fa700 10 client.16764 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:47:54.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.035+0000 7f03a17fa700 1 -- 172.21.15.1:0/3919225098 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x558121aaa660 con 0x558121bf9150 2023-11-16T16:47:54.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.035+0000 7f03a17fa700 10 client.16764 handle_command_reply: tid=0 2023-11-16T16:47:54.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.035+0000 7f03cc1a1b80 1 client.16764 shutdown 2023-11-16T16:47:54.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.035+0000 7f03cc1a1b80 1 -- 172.21.15.1:0/3919225098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558121b8b470 msgr2=0x558121b941d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:54.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.035+0000 7f03cc1a1b80 1 --2- 172.21.15.1:0/3919225098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558121b8b470 0x558121b941d0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x558121b80300 tx=0x7f03a8004750 comp rx=0 tx=0).stop 2023-11-16T16:47:54.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.035+0000 7f03cc1a1b80 1 -- 172.21.15.1:0/3919225098 shutdown_connections 2023-11-16T16:47:54.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.035+0000 7f03cc1a1b80 1 --2- 172.21.15.1:0/3919225098 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x558121bf9150 0x558121bfb560 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:54.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.035+0000 7f03cc1a1b80 1 --2- 172.21.15.1:0/3919225098 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x558121b8f8b0 0x558121b92a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:54.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.035+0000 7f03a17fa700 0 client.16764 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:47:54.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.035+0000 7f03cc1a1b80 1 --2- 172.21.15.1:0/3919225098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558121b8ab60 0x558121b93c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:54.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.036+0000 7f03cc1a1b80 1 --2- 172.21.15.1:0/3919225098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558121b8b470 0x558121b941d0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:54.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.036+0000 7f03cc1a1b80 1 -- 172.21.15.1:0/3919225098 >> 172.21.15.1:0/3919225098 conn(0x558121b80360 msgr2=0x558121b89960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:54.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.036+0000 7f03cc1a1b80 1 -- 172.21.15.1:0/3919225098 shutdown_connections 2023-11-16T16:47:54.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.036+0000 7f03cc1a1b80 1 -- 172.21.15.1:0/3919225098 wait complete. 2023-11-16T16:47:54.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.036+0000 7f03cc1a1b80 20 client.16764 trim_cache size 0 max 16384 2023-11-16T16:47:54.048 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:47:54.048 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-16T16:47:54.048 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2023-11-16T16:47:54.048 INFO:teuthology.orchestra.run.smithi001.stdout: "bc40e857-edb0-4fed-8580-e1fd05b64e9f": { 2023-11-16T16:47:54.049 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2023-11-16T16:47:54.049 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "bc40e857-edb0-4fed-8580-e1fd05b64e9f", 2023-11-16T16:47:54.049 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2023-11-16T16:47:54.049 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:47:54.049 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:47:54.049 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:47:54.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.036+0000 7f03c0015700 1 -- 172.21.15.1:0/1846264795 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f03a40a12c0 msgr2=0x7f03a40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:54.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.036+0000 7f03c0015700 1 --2- 172.21.15.1:0/1846264795 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f03a40a12c0 0x7f03a40a3770 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f03b4005950 tx=0x7f03b40058e0 comp rx=0 tx=0).stop 2023-11-16T16:47:54.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.036+0000 7f03c0015700 1 -- 172.21.15.1:0/1846264795 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03b8104140 msgr2=0x7f03b8199d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:47:54.050 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.036+0000 7f03c0015700 1 --2- 172.21.15.1:0/1846264795 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03b8104140 0x7f03b8199d10 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f03b000d010 tx=0x7f03b0007920 comp rx=0 tx=0).stop 2023-11-16T16:47:54.051 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.036+0000 7f03c0015700 1 -- 172.21.15.1:0/1846264795 shutdown_connections 2023-11-16T16:47:54.051 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.036+0000 7f03c0015700 1 --2- 172.21.15.1:0/1846264795 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f03a40a12c0 0x7f03a40a3770 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:54.051 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.036+0000 7f03c0015700 1 --2- 172.21.15.1:0/1846264795 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03b8104140 0x7f03b8199d10 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:54.051 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.036+0000 7f03c0015700 1 --2- 172.21.15.1:0/1846264795 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b80ff8f0 0x7f03b8195370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:54.051 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.036+0000 7f03c0015700 1 --2- 172.21.15.1:0/1846264795 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f03b81002a0 0x7f03b81958b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:47:54.051 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.036+0000 7f03c0015700 1 -- 172.21.15.1:0/1846264795 >> 172.21.15.1:0/1846264795 conn(0x7f03b8075880 msgr2=0x7f03b80fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:47:54.051 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.036+0000 7f03c0015700 1 -- 172.21.15.1:0/1846264795 shutdown_connections 2023-11-16T16:47:54.051 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:47:54.036+0000 7f03c0015700 1 -- 172.21.15.1:0/1846264795 wait complete. 2023-11-16T16:47:54.052 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:53 smithi001 ceph-mon[98800]: pgmap v269: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:47:54.052 INFO:tasks.cephfs.filesystem:scrub status for tag:bc40e857-edb0-4fed-8580-e1fd05b64e9f - {'path': '/', 'tag': 'bc40e857-edb0-4fed-8580-e1fd05b64e9f', 'options': 'recursive,force'} 2023-11-16T16:47:54.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:53 smithi053 ceph-mon[100638]: pgmap v269: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:47:55.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:54 smithi026 ceph-mon[100112]: scrub queued for path: / 2023-11-16T16:47:55.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:47:55.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:54 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:47:55.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:54 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:47:55.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:54 smithi026 ceph-mon[100112]: scrub summary: idle 2023-11-16T16:47:55.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:54 smithi053 ceph-mon[100638]: scrub queued for path: / 2023-11-16T16:47:55.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:47:55.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:54 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:47:55.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:54 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:47:55.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:54 smithi053 ceph-mon[100638]: scrub summary: idle 2023-11-16T16:47:55.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:54 smithi001 ceph-mon[98800]: scrub queued for path: / 2023-11-16T16:47:55.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:47:55.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:54 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:47:55.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:54 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:47:55.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:54 smithi001 ceph-mon[98800]: scrub summary: idle 2023-11-16T16:47:56.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:55 smithi026 ceph-mon[100112]: pgmap v270: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-16T16:47:56.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:55 smithi053 ceph-mon[100638]: pgmap v270: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-16T16:47:56.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:55 smithi001 ceph-mon[98800]: pgmap v270: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-16T16:47:58.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:57 smithi026 ceph-mon[100112]: pgmap v271: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:47:58.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:57 smithi053 ceph-mon[100638]: pgmap v271: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:47:58.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:57 smithi001 ceph-mon[98800]: pgmap v271: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:48:00.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:47:59 smithi026 ceph-mon[100112]: pgmap v272: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:48:00.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:47:59 smithi053 ceph-mon[100638]: pgmap v272: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:48:00.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:47:59 smithi001 ceph-mon[98800]: pgmap v272: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:48:02.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:02 smithi001 ceph-mon[98800]: pgmap v273: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 22 KiB/s wr, 12 op/s 2023-11-16T16:48:02.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:02 smithi026 ceph-mon[100112]: pgmap v273: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 22 KiB/s wr, 12 op/s 2023-11-16T16:48:02.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:02 smithi053 ceph-mon[100638]: pgmap v273: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 22 KiB/s wr, 12 op/s 2023-11-16T16:48:04.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:03 smithi026 ceph-mon[100112]: pgmap v274: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 22 KiB/s wr, 9 op/s 2023-11-16T16:48:04.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:03 smithi053 ceph-mon[100638]: pgmap v274: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 22 KiB/s wr, 9 op/s 2023-11-16T16:48:04.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:03 smithi001 ceph-mon[98800]: pgmap v274: 97 pgs: 97 active+clean; 817 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 22 KiB/s wr, 9 op/s 2023-11-16T16:48:06.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:05 smithi026 ceph-mon[100112]: pgmap v275: 97 pgs: 97 active+clean; 817 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-11-16T16:48:06.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:05 smithi053 ceph-mon[100638]: pgmap v275: 97 pgs: 97 active+clean; 817 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-11-16T16:48:06.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:05 smithi001 ceph-mon[98800]: pgmap v275: 97 pgs: 97 active+clean; 817 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-11-16T16:48:08.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:07 smithi026 ceph-mon[100112]: pgmap v276: 97 pgs: 97 active+clean; 817 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-16T16:48:08.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:07 smithi053 ceph-mon[100638]: pgmap v276: 97 pgs: 97 active+clean; 817 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-16T16:48:08.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:07 smithi001 ceph-mon[98800]: pgmap v276: 97 pgs: 97 active+clean; 817 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-16T16:48:10.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:10 smithi053 ceph-mon[100638]: osdmap e92: 12 total, 12 up, 12 in 2023-11-16T16:48:10.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:10 smithi053 ceph-mon[100638]: pgmap v278: 97 pgs: 97 active+clean; 817 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-11-16T16:48:10.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:10 smithi001 ceph-mon[98800]: osdmap e92: 12 total, 12 up, 12 in 2023-11-16T16:48:10.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:10 smithi001 ceph-mon[98800]: pgmap v278: 97 pgs: 97 active+clean; 817 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-11-16T16:48:10.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:10 smithi026 ceph-mon[100112]: osdmap e92: 12 total, 12 up, 12 in 2023-11-16T16:48:10.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:10 smithi026 ceph-mon[100112]: pgmap v278: 97 pgs: 97 active+clean; 817 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-11-16T16:48:12.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:12 smithi001 ceph-mon[98800]: pgmap v279: 97 pgs: 97 active+clean; 817 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 KiB/s wr, 18 op/s 2023-11-16T16:48:12.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:12 smithi026 ceph-mon[100112]: pgmap v279: 97 pgs: 97 active+clean; 817 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 KiB/s wr, 18 op/s 2023-11-16T16:48:12.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:12 smithi053 ceph-mon[100638]: pgmap v279: 97 pgs: 97 active+clean; 817 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 KiB/s wr, 18 op/s 2023-11-16T16:48:14.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:13 smithi026 ceph-mon[100112]: pgmap v280: 97 pgs: 97 active+clean; 817 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 KiB/s wr, 18 op/s 2023-11-16T16:48:14.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:13 smithi053 ceph-mon[100638]: pgmap v280: 97 pgs: 97 active+clean; 817 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 KiB/s wr, 18 op/s 2023-11-16T16:48:14.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:13 smithi001 ceph-mon[98800]: pgmap v280: 97 pgs: 97 active+clean; 817 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 KiB/s wr, 18 op/s 2023-11-16T16:48:16.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:15 smithi026 ceph-mon[100112]: pgmap v281: 97 pgs: 97 active+clean; 817 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-16T16:48:16.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:15 smithi053 ceph-mon[100638]: pgmap v281: 97 pgs: 97 active+clean; 817 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-16T16:48:16.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:15 smithi001 ceph-mon[98800]: pgmap v281: 97 pgs: 97 active+clean; 817 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-16T16:48:17.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:17 smithi001 ceph-mon[98800]: osdmap e93: 12 total, 12 up, 12 in 2023-11-16T16:48:17.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:17 smithi026 ceph-mon[100112]: osdmap e93: 12 total, 12 up, 12 in 2023-11-16T16:48:17.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:17 smithi053 ceph-mon[100638]: osdmap e93: 12 total, 12 up, 12 in 2023-11-16T16:48:18.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:18 smithi001 ceph-mon[98800]: pgmap v283: 97 pgs: 97 active+clean; 817 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 245 B/s wr, 7 op/s 2023-11-16T16:48:18.757 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:18 smithi026 ceph-mon[100112]: pgmap v283: 97 pgs: 97 active+clean; 817 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 245 B/s wr, 7 op/s 2023-11-16T16:48:18.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:18 smithi053 ceph-mon[100638]: pgmap v283: 97 pgs: 97 active+clean; 817 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 245 B/s wr, 7 op/s 2023-11-16T16:48:20.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:19 smithi026 ceph-mon[100112]: pgmap v284: 97 pgs: 97 active+clean; 817 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-16T16:48:20.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:19 smithi053 ceph-mon[100638]: pgmap v284: 97 pgs: 97 active+clean; 817 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-16T16:48:20.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:19 smithi001 ceph-mon[98800]: pgmap v284: 97 pgs: 97 active+clean; 817 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-11-16T16:48:22.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:22 smithi001 ceph-mon[98800]: pgmap v285: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-11-16T16:48:22.757 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:22 smithi026 ceph-mon[100112]: pgmap v285: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-11-16T16:48:22.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:22 smithi053 ceph-mon[100638]: pgmap v285: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-11-16T16:48:24.054 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-16T16:48:24.199 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.195+0000 7fbe9bcdb700 1 -- 172.21.15.1:0/1356902406 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe940fea90 msgr2=0x7fbe940fef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:24.199 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.195+0000 7fbe9bcdb700 1 --2- 172.21.15.1:0/1356902406 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe940fea90 0x7fbe940fef00 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fbe90009990 tx=0x7fbe90009ca0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.199 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.195+0000 7fbe9bcdb700 1 -- 172.21.15.1:0/1356902406 shutdown_connections 2023-11-16T16:48:24.199 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.195+0000 7fbe9bcdb700 1 --2- 172.21.15.1:0/1356902406 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe940ff440 0x7fbe9410d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.195+0000 7fbe9bcdb700 1 --2- 172.21.15.1:0/1356902406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe94106480 0x7fbe94106850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.195+0000 7fbe9bcdb700 1 --2- 172.21.15.1:0/1356902406 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe940fea90 0x7fbe940fef00 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.195+0000 7fbe9bcdb700 1 -- 172.21.15.1:0/1356902406 >> 172.21.15.1:0/1356902406 conn(0x7fbe940748b0 msgr2=0x7fbe94074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:24.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.195+0000 7fbe9bcdb700 1 -- 172.21.15.1:0/1356902406 shutdown_connections 2023-11-16T16:48:24.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.195+0000 7fbe9bcdb700 1 -- 172.21.15.1:0/1356902406 wait complete. 2023-11-16T16:48:24.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.196+0000 7fbe9bcdb700 1 Processor -- start 2023-11-16T16:48:24.200 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.196+0000 7fbe9bcdb700 1 -- start start 2023-11-16T16:48:24.201 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.196+0000 7fbe9bcdb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe940fea90 0x7fbe941985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.201 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.196+0000 7fbe9bcdb700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe940ff440 0x7fbe94198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.201 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.196+0000 7fbe9bcdb700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe94106480 0x7fbe9419cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.201 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.196+0000 7fbe9bcdb700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9419d5b0 con 0x7fbe940ff440 2023-11-16T16:48:24.201 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.196+0000 7fbe9bcdb700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9419d720 con 0x7fbe940fea90 2023-11-16T16:48:24.201 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.196+0000 7fbe9bcdb700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9419d8e0 con 0x7fbe94106480 2023-11-16T16:48:24.201 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.196+0000 7fbe99276700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe940ff440 0x7fbe94198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.201 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.196+0000 7fbe99276700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe940ff440 0x7fbe94198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46672/0 (socket says 172.21.15.1:46672) 2023-11-16T16:48:24.201 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.196+0000 7fbe99276700 1 -- 172.21.15.1:0/3616886277 learned_addr learned my addr 172.21.15.1:0/3616886277 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:24.202 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.196+0000 7fbe9a278700 1 --2- 172.21.15.1:0/3616886277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe94106480 0x7fbe9419cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.202 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.196+0000 7fbe99a77700 1 --2- 172.21.15.1:0/3616886277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe940fea90 0x7fbe941985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.202 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.196+0000 7fbe99276700 1 -- 172.21.15.1:0/3616886277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe94106480 msgr2=0x7fbe9419cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:24.202 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.196+0000 7fbe99276700 1 --2- 172.21.15.1:0/3616886277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe94106480 0x7fbe9419cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.202 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.196+0000 7fbe99276700 1 -- 172.21.15.1:0/3616886277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe940fea90 msgr2=0x7fbe941985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:24.202 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.196+0000 7fbe99276700 1 --2- 172.21.15.1:0/3616886277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe940fea90 0x7fbe941985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.202 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.196+0000 7fbe99276700 1 -- 172.21.15.1:0/3616886277 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe900095f0 con 0x7fbe940ff440 2023-11-16T16:48:24.203 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.197+0000 7fbe99276700 1 --2- 172.21.15.1:0/3616886277 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe940ff440 0x7fbe94198b10 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fbe90005910 tx=0x7fbe90004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:24.203 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.197+0000 7fbe8affd700 1 -- 172.21.15.1:0/3616886277 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9001d070 con 0x7fbe940ff440 2023-11-16T16:48:24.203 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.197+0000 7fbe8affd700 1 -- 172.21.15.1:0/3616886277 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe90022470 con 0x7fbe940ff440 2023-11-16T16:48:24.203 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.197+0000 7fbe8affd700 1 -- 172.21.15.1:0/3616886277 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe90017650 con 0x7fbe940ff440 2023-11-16T16:48:24.203 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.197+0000 7fbe9bcdb700 1 -- 172.21.15.1:0/3616886277 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe94110450 con 0x7fbe940ff440 2023-11-16T16:48:24.203 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.197+0000 7fbe9bcdb700 1 -- 172.21.15.1:0/3616886277 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe941102c0 con 0x7fbe940ff440 2023-11-16T16:48:24.209 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.199+0000 7fbe8affd700 1 -- 172.21.15.1:0/3616886277 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fbe9000fad0 con 0x7fbe940ff440 2023-11-16T16:48:24.210 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.200+0000 7fbe8affd700 1 --2- 172.21.15.1:0/3616886277 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fbe800a1340 0x7fbe800a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.210 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.200+0000 7fbe8affd700 1 -- 172.21.15.1:0/3616886277 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fbe900ce170 con 0x7fbe940ff440 2023-11-16T16:48:24.210 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.200+0000 7fbe99a77700 1 --2- 172.21.15.1:0/3616886277 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fbe800a1340 0x7fbe800a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.210 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.201+0000 7fbe99a77700 1 --2- 172.21.15.1:0/3616886277 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fbe800a1340 0x7fbe800a37f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbe84007be0 tx=0x7fbe840073d0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:24.210 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.201+0000 7fbea7e67b80 1 Processor -- start 2023-11-16T16:48:24.210 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.201+0000 7fbea7e67b80 1 -- start start 2023-11-16T16:48:24.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.201+0000 7fbea7e67b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c9f184b60 0x562c9f184f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.201+0000 7fbea7e67b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c9f185470 0x562c9f189370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.201+0000 7fbea7e67b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c9f1898b0 0x562c9f189d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.201+0000 7fbea7e67b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c9f08a060 con 0x562c9f185470 2023-11-16T16:48:24.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.201+0000 7fbea7e67b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c9f0dc8a0 con 0x562c9f184b60 2023-11-16T16:48:24.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.201+0000 7fbea7e67b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c9f0d5d30 con 0x562c9f1898b0 2023-11-16T16:48:24.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.201+0000 7fbe99276700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c9f185470 0x562c9f189370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.212 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.201+0000 7fbe99276700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c9f185470 0x562c9f189370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46680/0 (socket says 172.21.15.1:46680) 2023-11-16T16:48:24.212 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.201+0000 7fbe99276700 1 -- 172.21.15.1:0/1786409976 learned_addr learned my addr 172.21.15.1:0/1786409976 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:24.212 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.201+0000 7fbe9a278700 1 --2- 172.21.15.1:0/1786409976 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c9f1898b0 0x562c9f189d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.212 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.202+0000 7fbe99a77700 1 --2- 172.21.15.1:0/1786409976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c9f184b60 0x562c9f184f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.212 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.202+0000 7fbe99276700 1 -- 172.21.15.1:0/1786409976 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c9f1898b0 msgr2=0x562c9f189d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:24.212 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.202+0000 7fbe99276700 1 --2- 172.21.15.1:0/1786409976 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c9f1898b0 0x562c9f189d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.212 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.202+0000 7fbe99276700 1 -- 172.21.15.1:0/1786409976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c9f184b60 msgr2=0x562c9f184f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:24.213 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.202+0000 7fbe99276700 1 --2- 172.21.15.1:0/1786409976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c9f184b60 0x562c9f184f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.213 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.202+0000 7fbe99276700 1 -- 172.21.15.1:0/1786409976 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c9ef57630 con 0x562c9f185470 2023-11-16T16:48:24.213 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.202+0000 7fbe99a77700 1 --2- 172.21.15.1:0/1786409976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c9f184b60 0x562c9f184f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:48:24.213 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.202+0000 7fbe99276700 1 --2- 172.21.15.1:0/1786409976 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c9f185470 0x562c9f189370 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fbe900a30f0 tx=0x7fbe9009b100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b52f19b379e8242 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:24.213 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.202+0000 7fbe7ffff700 1 -- 172.21.15.1:0/1786409976 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9009f290 con 0x562c9f185470 2023-11-16T16:48:24.213 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.202+0000 7fbe7ffff700 1 -- 172.21.15.1:0/1786409976 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe900a3620 con 0x562c9f185470 2023-11-16T16:48:24.213 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.202+0000 7fbe7ffff700 1 -- 172.21.15.1:0/1786409976 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe90022890 con 0x562c9f185470 2023-11-16T16:48:24.213 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.203+0000 7fbea7e67b80 1 -- 172.21.15.1:0/1786409976 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c9f185470 msgr2=0x562c9f189370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:24.214 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.203+0000 7fbea7e67b80 1 --2- 172.21.15.1:0/1786409976 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c9f185470 0x562c9f189370 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fbe900a30f0 tx=0x7fbe9009b100 comp rx=0 tx=0).stop 2023-11-16T16:48:24.214 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.203+0000 7fbea7e67b80 1 -- 172.21.15.1:0/1786409976 shutdown_connections 2023-11-16T16:48:24.214 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.203+0000 7fbea7e67b80 1 --2- 172.21.15.1:0/1786409976 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c9f1898b0 0x562c9f189d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.214 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.203+0000 7fbea7e67b80 1 --2- 172.21.15.1:0/1786409976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c9f184b60 0x562c9f184f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.214 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.203+0000 7fbea7e67b80 1 --2- 172.21.15.1:0/1786409976 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c9f185470 0x562c9f189370 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.214 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.203+0000 7fbea7e67b80 1 -- 172.21.15.1:0/1786409976 >> 172.21.15.1:0/1786409976 conn(0x562c9f17a360 msgr2=0x562c9f17fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:24.214 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.203+0000 7fbea7e67b80 1 -- 172.21.15.1:0/1786409976 shutdown_connections 2023-11-16T16:48:24.214 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.203+0000 7fbea7e67b80 1 -- 172.21.15.1:0/1786409976 wait complete. 2023-11-16T16:48:24.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.203+0000 7fbea7e67b80 1 -- start start 2023-11-16T16:48:24.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.203+0000 7fbea7e67b80 1 Processor -- start 2023-11-16T16:48:24.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.204+0000 7fbea7e67b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c9f184b60 0x562c9f190f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.204+0000 7fbea7e67b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c9f185470 0x562c9f18e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.204+0000 7fbea7e67b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c9f1898b0 0x562c9f18e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.204+0000 7fbea7e67b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c9f0778e0 con 0x562c9f185470 2023-11-16T16:48:24.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.204+0000 7fbea7e67b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c9f077580 con 0x562c9f184b60 2023-11-16T16:48:24.215 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.204+0000 7fbea7e67b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c9ef59920 con 0x562c9f1898b0 2023-11-16T16:48:24.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.204+0000 7fbea7e67b80 10 client.0 _ll_register_callbacks cb 0x562c9f095780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:48:24.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.204+0000 7fbe99276700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c9f185470 0x562c9f18e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.204+0000 7fbe99276700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c9f185470 0x562c9f18e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46684/0 (socket says 172.21.15.1:46684) 2023-11-16T16:48:24.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.204+0000 7fbe99276700 1 -- 172.21.15.1:0/1479349538 learned_addr learned my addr 172.21.15.1:0/1479349538 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:24.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.204+0000 7fbe9a278700 1 --2- 172.21.15.1:0/1479349538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c9f1898b0 0x562c9f18e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.204+0000 7fbe99a77700 1 --2- 172.21.15.1:0/1479349538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c9f184b60 0x562c9f190f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.216 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.204+0000 7fbe99276700 1 -- 172.21.15.1:0/1479349538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c9f1898b0 msgr2=0x562c9f18e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:24.217 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.204+0000 7fbe99276700 1 --2- 172.21.15.1:0/1479349538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c9f1898b0 0x562c9f18e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.217 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.204+0000 7fbe99276700 1 -- 172.21.15.1:0/1479349538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c9f184b60 msgr2=0x562c9f190f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:24.217 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.204+0000 7fbe99276700 1 --2- 172.21.15.1:0/1479349538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c9f184b60 0x562c9f190f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.217 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.204+0000 7fbe99276700 1 -- 172.21.15.1:0/1479349538 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe941102c0 con 0x562c9f185470 2023-11-16T16:48:24.218 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.204+0000 7fbe9a278700 1 --2- 172.21.15.1:0/1479349538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c9f1898b0 0x562c9f18e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:48:24.218 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.204+0000 7fbe99276700 1 --2- 172.21.15.1:0/1479349538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c9f185470 0x562c9f18e190 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fbe90097cd0 tx=0x7fbe90005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c657ae283712ca server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:24.218 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.204+0000 7fbe7d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:48:24.221 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.204+0000 7fbe7d7fa700 1 -- 172.21.15.1:0/1479349538 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9009f310 con 0x562c9f185470 2023-11-16T16:48:24.221 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.205+0000 7fbe7d7fa700 1 -- 172.21.15.1:0/1479349538 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fbe9009f040 con 0x562c9f185470 2023-11-16T16:48:24.221 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.205+0000 7fbea7e67b80 1 -- 172.21.15.1:0/1479349538 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562c9f177b20 con 0x562c9f185470 2023-11-16T16:48:24.221 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.205+0000 7fbe7d7fa700 1 -- 172.21.15.1:0/1479349538 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe900225e0 con 0x562c9f185470 2023-11-16T16:48:24.221 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.205+0000 7fbea7e67b80 10 client.16776 fetch_fsmap learned FSMap version 29 2023-11-16T16:48:24.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.205+0000 7fbea7e67b80 1 -- 172.21.15.1:0/1479349538 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x562c9f09c190 con 0x562c9f185470 2023-11-16T16:48:24.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.205+0000 7fbe7d7fa700 1 -- 172.21.15.1:0/1479349538 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562c9f177b20 con 0x562c9f185470 2023-11-16T16:48:24.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.205+0000 7fbe7d7fa700 1 -- 172.21.15.1:0/1479349538 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 29) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fbe900a3220 con 0x562c9f185470 2023-11-16T16:48:24.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.205+0000 7fbea7e67b80 10 client.16776 fetch_fsmap finished waiting for FSMap version 29 2023-11-16T16:48:24.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.205+0000 7fbea7e67b80 10 client.16776 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:48:24.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.205+0000 7fbea7e67b80 20 client.16776 populate_metadata read hostname 'smithi001' 2023-11-16T16:48:24.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.205+0000 7fbea7e67b80 1 --2- 172.21.15.1:0/1479349538 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x562c9f19e0a0 0x562c9f1a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.222 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.205+0000 7fbea7e67b80 4 client.16776 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:48:24.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.205+0000 7fbea7e67b80 1 -- 172.21.15.1:0/1479349538 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562c9f0980a0 con 0x562c9f19e0a0 2023-11-16T16:48:24.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.205+0000 7fbe99a77700 1 --2- 172.21.15.1:0/1479349538 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x562c9f19e0a0 0x562c9f1a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.206+0000 7fbe99a77700 1 --2- 172.21.15.1:0/1479349538 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x562c9f19e0a0 0x562c9f1a04b0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8001e57ff933b7d server_cookie=6afca038b304efd1 in_seq=0 out_seq=0 2023-11-16T16:48:24.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.206+0000 7fbe7d7fa700 10 client.16776 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:48:24.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.206+0000 7fbe7d7fa700 1 -- 172.21.15.1:0/1479349538 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562c9f0980a0 con 0x562c9f19e0a0 2023-11-16T16:48:24.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.206+0000 7fbe7d7fa700 10 client.16776 handle_command_reply: tid=0 2023-11-16T16:48:24.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.206+0000 7fbea7e67b80 1 client.16776 shutdown 2023-11-16T16:48:24.223 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.207+0000 7fbea7e67b80 1 -- 172.21.15.1:0/1479349538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c9f185470 msgr2=0x562c9f18e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:24.224 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.207+0000 7fbea7e67b80 1 --2- 172.21.15.1:0/1479349538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c9f185470 0x562c9f18e190 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fbe90097cd0 tx=0x7fbe90005c70 comp rx=0 tx=0).stop 2023-11-16T16:48:24.224 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.207+0000 7fbea7e67b80 1 -- 172.21.15.1:0/1479349538 shutdown_connections 2023-11-16T16:48:24.224 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.207+0000 7fbea7e67b80 1 --2- 172.21.15.1:0/1479349538 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x562c9f19e0a0 0x562c9f1a04b0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.224 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.207+0000 7fbea7e67b80 1 --2- 172.21.15.1:0/1479349538 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c9f1898b0 0x562c9f18e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.224 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.207+0000 7fbea7e67b80 1 --2- 172.21.15.1:0/1479349538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c9f184b60 0x562c9f190f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.224 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.207+0000 7fbea7e67b80 1 --2- 172.21.15.1:0/1479349538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c9f185470 0x562c9f18e190 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.224 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.207+0000 7fbe7d7fa700 0 client.16776 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:48:24.225 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.207+0000 7fbea7e67b80 1 -- 172.21.15.1:0/1479349538 >> 172.21.15.1:0/1479349538 conn(0x562c9f17a360 msgr2=0x562c9f17ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:24.225 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.207+0000 7fbea7e67b80 1 -- 172.21.15.1:0/1479349538 shutdown_connections 2023-11-16T16:48:24.225 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.207+0000 7fbea7e67b80 1 -- 172.21.15.1:0/1479349538 wait complete. 2023-11-16T16:48:24.225 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.207+0000 7fbea7e67b80 20 client.16776 trim_cache size 0 max 16384 2023-11-16T16:48:24.229 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.224+0000 7fbea7e67b80 1 Processor -- start 2023-11-16T16:48:24.230 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.224+0000 7fbea7e67b80 1 -- start start 2023-11-16T16:48:24.230 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.224+0000 7fbea7e67b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c9f184b60 0x562c9f195750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.230 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.224+0000 7fbea7e67b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c9f185470 0x562c9f18bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.230 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.224+0000 7fbea7e67b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c9f1898b0 0x562c9f18c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.230 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.224+0000 7fbea7e67b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c9f1814a0 con 0x562c9f184b60 2023-11-16T16:48:24.230 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.224+0000 7fbea7e67b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c9f17a170 con 0x562c9f1898b0 2023-11-16T16:48:24.230 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.224+0000 7fbea7e67b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c9f185bc0 con 0x562c9f185470 2023-11-16T16:48:24.231 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.224+0000 7fbe99a77700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c9f184b60 0x562c9f195750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.231 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.224+0000 7fbe99a77700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c9f184b60 0x562c9f195750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46690/0 (socket says 172.21.15.1:46690) 2023-11-16T16:48:24.231 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.224+0000 7fbe99a77700 1 -- 172.21.15.1:0/1548526936 learned_addr learned my addr 172.21.15.1:0/1548526936 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:24.231 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.224+0000 7fbe9a278700 1 --2- 172.21.15.1:0/1548526936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c9f1898b0 0x562c9f18c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.231 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.224+0000 7fbe99276700 1 --2- 172.21.15.1:0/1548526936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c9f185470 0x562c9f18bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.231 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.225+0000 7fbe99a77700 1 -- 172.21.15.1:0/1548526936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c9f185470 msgr2=0x562c9f18bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:24.231 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.225+0000 7fbe99a77700 1 --2- 172.21.15.1:0/1548526936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c9f185470 0x562c9f18bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.231 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.225+0000 7fbe99a77700 1 -- 172.21.15.1:0/1548526936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c9f1898b0 msgr2=0x562c9f18c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:24.232 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.225+0000 7fbe99a77700 1 --2- 172.21.15.1:0/1548526936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c9f1898b0 0x562c9f18c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.232 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.225+0000 7fbe99a77700 1 -- 172.21.15.1:0/1548526936 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c9f18a710 con 0x562c9f184b60 2023-11-16T16:48:24.232 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.225+0000 7fbe9a278700 1 --2- 172.21.15.1:0/1548526936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c9f1898b0 0x562c9f18c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:48:24.232 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.225+0000 7fbe99a77700 1 --2- 172.21.15.1:0/1548526936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c9f184b60 0x562c9f195750 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x562c9f1a0d90 tx=0x7fbe84013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b02fabc352d087a server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:24.232 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.225+0000 7fbe88ff9700 1 -- 172.21.15.1:0/1548526936 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe84004590 con 0x562c9f184b60 2023-11-16T16:48:24.232 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.225+0000 7fbe88ff9700 1 -- 172.21.15.1:0/1548526936 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe840275f0 con 0x562c9f184b60 2023-11-16T16:48:24.232 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.225+0000 7fbe88ff9700 1 -- 172.21.15.1:0/1548526936 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe84003950 con 0x562c9f184b60 2023-11-16T16:48:24.232 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.225+0000 7fbea7e67b80 1 -- 172.21.15.1:0/1548526936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c9f184b60 msgr2=0x562c9f195750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:24.233 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.225+0000 7fbea7e67b80 1 --2- 172.21.15.1:0/1548526936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c9f184b60 0x562c9f195750 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x562c9f1a0d90 tx=0x7fbe84013000 comp rx=0 tx=0).stop 2023-11-16T16:48:24.233 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.226+0000 7fbea7e67b80 1 -- 172.21.15.1:0/1548526936 shutdown_connections 2023-11-16T16:48:24.233 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.226+0000 7fbea7e67b80 1 --2- 172.21.15.1:0/1548526936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c9f1898b0 0x562c9f18c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.233 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.226+0000 7fbea7e67b80 1 --2- 172.21.15.1:0/1548526936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c9f184b60 0x562c9f195750 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.233 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.226+0000 7fbea7e67b80 1 --2- 172.21.15.1:0/1548526936 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c9f185470 0x562c9f18bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.233 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.226+0000 7fbea7e67b80 1 -- 172.21.15.1:0/1548526936 >> 172.21.15.1:0/1548526936 conn(0x562c9f17a360 msgr2=0x562c9f19a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:24.233 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.226+0000 7fbea7e67b80 1 -- 172.21.15.1:0/1548526936 shutdown_connections 2023-11-16T16:48:24.233 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.226+0000 7fbea7e67b80 1 -- 172.21.15.1:0/1548526936 wait complete. 2023-11-16T16:48:24.234 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.226+0000 7fbea7e67b80 1 -- start start 2023-11-16T16:48:24.234 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.226+0000 7fbea7e67b80 1 Processor -- start 2023-11-16T16:48:24.234 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.226+0000 7fbea7e67b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c9f184b60 0x562c9f17fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.234 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.226+0000 7fbea7e67b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c9f185470 0x562c9f17e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.234 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.226+0000 7fbea7e67b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c9f1898b0 0x562c9f17ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.234 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.226+0000 7fbea7e67b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c9f17efd0 con 0x562c9f1898b0 2023-11-16T16:48:24.234 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.226+0000 7fbea7e67b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c9f17f140 con 0x562c9f184b60 2023-11-16T16:48:24.234 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.226+0000 7fbea7e67b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c9f17f300 con 0x562c9f185470 2023-11-16T16:48:24.235 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.227+0000 7fbea7e67b80 10 client.0 _ll_register_callbacks cb 0x7fbe8c009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:48:24.235 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.227+0000 7fbe9a278700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c9f1898b0 0x562c9f17ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.235 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.227+0000 7fbe9a278700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c9f1898b0 0x562c9f17ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46700/0 (socket says 172.21.15.1:46700) 2023-11-16T16:48:24.235 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.227+0000 7fbe9a278700 1 -- 172.21.15.1:0/2297566486 learned_addr learned my addr 172.21.15.1:0/2297566486 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:24.235 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.227+0000 7fbe99276700 1 --2- 172.21.15.1:0/2297566486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c9f185470 0x562c9f17e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.235 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.227+0000 7fbe99a77700 1 --2- 172.21.15.1:0/2297566486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c9f184b60 0x562c9f17fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.235 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.227+0000 7fbe9a278700 1 -- 172.21.15.1:0/2297566486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c9f185470 msgr2=0x562c9f17e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:24.235 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.227+0000 7fbe9a278700 1 --2- 172.21.15.1:0/2297566486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c9f185470 0x562c9f17e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.236 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.227+0000 7fbe9a278700 1 -- 172.21.15.1:0/2297566486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c9f184b60 msgr2=0x562c9f17fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:24.236 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.227+0000 7fbe9a278700 1 --2- 172.21.15.1:0/2297566486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c9f184b60 0x562c9f17fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.236 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.227+0000 7fbe9a278700 1 -- 172.21.15.1:0/2297566486 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe840109d0 con 0x562c9f1898b0 2023-11-16T16:48:24.236 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.227+0000 7fbe99a77700 1 --2- 172.21.15.1:0/2297566486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c9f184b60 0x562c9f17fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:48:24.236 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.227+0000 7fbe99276700 1 --2- 172.21.15.1:0/2297566486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c9f185470 0x562c9f17e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:48:24.236 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.227+0000 7fbe7d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:48:24.236 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.227+0000 7fbe9a278700 1 --2- 172.21.15.1:0/2297566486 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c9f1898b0 0x562c9f17ea90 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c007ca0 tx=0x7fbe8c018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a22b05dee5dd18 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:24.236 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.227+0000 7fbe7d7fa700 1 -- 172.21.15.1:0/2297566486 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c00e040 con 0x562c9f1898b0 2023-11-16T16:48:24.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.227+0000 7fbe7d7fa700 1 -- 172.21.15.1:0/2297566486 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fbe8c018e30 con 0x562c9f1898b0 2023-11-16T16:48:24.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.227+0000 7fbea7e67b80 1 -- 172.21.15.1:0/2297566486 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe900a3620 con 0x562c9f1898b0 2023-11-16T16:48:24.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.228+0000 7fbe7d7fa700 1 -- 172.21.15.1:0/2297566486 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe8c003a00 con 0x562c9f1898b0 2023-11-16T16:48:24.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.228+0000 7fbea7e67b80 10 client.16782 fetch_fsmap learned FSMap version 29 2023-11-16T16:48:24.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.228+0000 7fbea7e67b80 1 -- 172.21.15.1:0/2297566486 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x562c9ef03de0 con 0x562c9f1898b0 2023-11-16T16:48:24.237 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.228+0000 7fbe7d7fa700 1 -- 172.21.15.1:0/2297566486 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe900a3620 con 0x562c9f1898b0 2023-11-16T16:48:24.238 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.228+0000 7fbe7d7fa700 1 -- 172.21.15.1:0/2297566486 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 29) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fbe8c0203f0 con 0x562c9f1898b0 2023-11-16T16:48:24.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.228+0000 7fbea7e67b80 10 client.16782 fetch_fsmap finished waiting for FSMap version 29 2023-11-16T16:48:24.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.228+0000 7fbea7e67b80 10 client.16782 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:48:24.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.228+0000 7fbea7e67b80 20 client.16782 populate_metadata read hostname 'smithi001' 2023-11-16T16:48:24.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.228+0000 7fbea7e67b80 1 --2- 172.21.15.1:0/2297566486 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x562c9f1f33a0 0x562c9f1f57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.228+0000 7fbea7e67b80 4 client.16782 mds_command: new command op to 24521 tid=0[{"prefix": "scrub status"}] 2023-11-16T16:48:24.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.228+0000 7fbea7e67b80 1 -- 172.21.15.1:0/2297566486 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562c9f0a4660 con 0x562c9f1f33a0 2023-11-16T16:48:24.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.228+0000 7fbe99a77700 1 --2- 172.21.15.1:0/2297566486 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x562c9f1f33a0 0x562c9f1f57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.229+0000 7fbe7d7fa700 10 client.16782 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:48:24.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.229+0000 7fbe99a77700 1 --2- 172.21.15.1:0/2297566486 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x562c9f1f33a0 0x562c9f1f57b0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94c8112f62a2aa63 server_cookie=a6cf73d0e03de410 in_seq=0 out_seq=0 2023-11-16T16:48:24.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.229+0000 7fbe7d7fa700 1 -- 172.21.15.1:0/2297566486 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562c9f0a4660 con 0x562c9f1f33a0 2023-11-16T16:48:24.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.229+0000 7fbe7d7fa700 10 client.16782 handle_command_reply: tid=0 2023-11-16T16:48:24.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.229+0000 7fbea7e67b80 1 client.16782 shutdown 2023-11-16T16:48:24.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.229+0000 7fbea7e67b80 1 -- 172.21.15.1:0/2297566486 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c9f1898b0 msgr2=0x562c9f17ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:24.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.229+0000 7fbea7e67b80 1 --2- 172.21.15.1:0/2297566486 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c9f1898b0 0x562c9f17ea90 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c007ca0 tx=0x7fbe8c018420 comp rx=0 tx=0).stop 2023-11-16T16:48:24.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.229+0000 7fbea7e67b80 1 -- 172.21.15.1:0/2297566486 shutdown_connections 2023-11-16T16:48:24.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.229+0000 7fbea7e67b80 1 --2- 172.21.15.1:0/2297566486 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x562c9f1f33a0 0x562c9f1f57b0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.229+0000 7fbea7e67b80 1 --2- 172.21.15.1:0/2297566486 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c9f1898b0 0x562c9f17ea90 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.229+0000 7fbea7e67b80 1 --2- 172.21.15.1:0/2297566486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562c9f184b60 0x562c9f17fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.229+0000 7fbea7e67b80 1 --2- 172.21.15.1:0/2297566486 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562c9f185470 0x562c9f17e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.229+0000 7fbe7d7fa700 0 client.16782 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:48:24.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.229+0000 7fbea7e67b80 1 -- 172.21.15.1:0/2297566486 >> 172.21.15.1:0/2297566486 conn(0x562c9f17a360 msgr2=0x562c9f18c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:24.242 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.229+0000 7fbea7e67b80 1 -- 172.21.15.1:0/2297566486 shutdown_connections 2023-11-16T16:48:24.242 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.229+0000 7fbea7e67b80 1 -- 172.21.15.1:0/2297566486 wait complete. 2023-11-16T16:48:24.242 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.229+0000 7fbea7e67b80 20 client.16782 trim_cache size 0 max 16384 2023-11-16T16:48:24.242 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:48:24.242 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2023-11-16T16:48:24.242 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2023-11-16T16:48:24.243 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:48:24.244 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.230+0000 7fbe9bcdb700 1 -- 172.21.15.1:0/3616886277 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fbe800a1340 msgr2=0x7fbe800a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:24.244 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.230+0000 7fbe9bcdb700 1 --2- 172.21.15.1:0/3616886277 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fbe800a1340 0x7fbe800a37f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbe84007be0 tx=0x7fbe840073d0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.244 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.230+0000 7fbe9bcdb700 1 -- 172.21.15.1:0/3616886277 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe940ff440 msgr2=0x7fbe94198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:24.244 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.230+0000 7fbe9bcdb700 1 --2- 172.21.15.1:0/3616886277 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe940ff440 0x7fbe94198b10 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fbe90005910 tx=0x7fbe90004730 comp rx=0 tx=0).stop 2023-11-16T16:48:24.244 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.230+0000 7fbe9bcdb700 1 -- 172.21.15.1:0/3616886277 shutdown_connections 2023-11-16T16:48:24.244 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.230+0000 7fbe9bcdb700 1 --2- 172.21.15.1:0/3616886277 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fbe800a1340 0x7fbe800a37f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.245 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.230+0000 7fbe9bcdb700 1 --2- 172.21.15.1:0/3616886277 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbe94106480 0x7fbe9419cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.245 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.230+0000 7fbe9bcdb700 1 --2- 172.21.15.1:0/3616886277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe940fea90 0x7fbe941985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.245 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.230+0000 7fbe9bcdb700 1 --2- 172.21.15.1:0/3616886277 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe940ff440 0x7fbe94198b10 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.245 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.230+0000 7fbe9bcdb700 1 -- 172.21.15.1:0/3616886277 >> 172.21.15.1:0/3616886277 conn(0x7fbe940748b0 msgr2=0x7fbe940ffba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:24.245 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.230+0000 7fbe9bcdb700 1 -- 172.21.15.1:0/3616886277 shutdown_connections 2023-11-16T16:48:24.245 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.230+0000 7fbe9bcdb700 1 -- 172.21.15.1:0/3616886277 wait complete. 2023-11-16T16:48:24.246 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-16T16:48:24.246 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:48:24.373 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.371+0000 7f853d1af700 1 -- 172.21.15.1:0/3412686310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8538100cc0 msgr2=0x7f8538101090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:24.373 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.371+0000 7f853d1af700 1 --2- 172.21.15.1:0/3412686310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8538100cc0 0x7f8538101090 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f85200099c0 tx=0x7f8520009cd0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.373 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.371+0000 7f853d1af700 1 -- 172.21.15.1:0/3412686310 shutdown_connections 2023-11-16T16:48:24.373 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.371+0000 7f853d1af700 1 --2- 172.21.15.1:0/3412686310 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85381055d0 0x7f853810d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.373 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.371+0000 7f853d1af700 1 --2- 172.21.15.1:0/3412686310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8538100cc0 0x7f8538101090 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.374 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.371+0000 7f853d1af700 1 --2- 172.21.15.1:0/3412686310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8538101660 0x7f8538104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.374 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.371+0000 7f853d1af700 1 -- 172.21.15.1:0/3412686310 >> 172.21.15.1:0/3412686310 conn(0x7f85380774c0 msgr2=0x7f85380778c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:24.374 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.371+0000 7f853d1af700 1 -- 172.21.15.1:0/3412686310 shutdown_connections 2023-11-16T16:48:24.374 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.371+0000 7f853d1af700 1 -- 172.21.15.1:0/3412686310 wait complete. 2023-11-16T16:48:24.376 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.372+0000 7f853d1af700 1 Processor -- start 2023-11-16T16:48:24.376 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.372+0000 7f853d1af700 1 -- start start 2023-11-16T16:48:24.376 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.372+0000 7f853d1af700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8538100cc0 0x7f85381963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.377 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.372+0000 7f853d1af700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8538101660 0x7f8538196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.377 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.373+0000 7f853d1af700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85381055d0 0x7f853819ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.377 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.373+0000 7f853d1af700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853819b3c0 con 0x7f8538101660 2023-11-16T16:48:24.377 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.373+0000 7f853d1af700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853819b530 con 0x7f8538100cc0 2023-11-16T16:48:24.377 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.373+0000 7f853d1af700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853819b6f0 con 0x7f85381055d0 2023-11-16T16:48:24.377 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.373+0000 7f853659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8538101660 0x7f8538196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.377 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.373+0000 7f853659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8538101660 0x7f8538196920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46712/0 (socket says 172.21.15.1:46712) 2023-11-16T16:48:24.377 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.373+0000 7f853659c700 1 -- 172.21.15.1:0/1708769408 learned_addr learned my addr 172.21.15.1:0/1708769408 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:24.378 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.373+0000 7f8536d9d700 1 --2- 172.21.15.1:0/1708769408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8538100cc0 0x7f85381963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.378 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.373+0000 7f853759e700 1 --2- 172.21.15.1:0/1708769408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85381055d0 0x7f853819ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.378 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.373+0000 7f853659c700 1 -- 172.21.15.1:0/1708769408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85381055d0 msgr2=0x7f853819ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:24.378 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.373+0000 7f853659c700 1 --2- 172.21.15.1:0/1708769408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85381055d0 0x7f853819ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.378 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.373+0000 7f853659c700 1 -- 172.21.15.1:0/1708769408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8538100cc0 msgr2=0x7f85381963e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:24.378 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.373+0000 7f853659c700 1 --2- 172.21.15.1:0/1708769408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8538100cc0 0x7f85381963e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.379 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.373+0000 7f853659c700 1 -- 172.21.15.1:0/1708769408 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85200095f0 con 0x7f8538101660 2023-11-16T16:48:24.379 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.373+0000 7f853659c700 1 --2- 172.21.15.1:0/1708769408 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8538101660 0x7f8538196920 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f852800e940 tx=0x7f852800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:24.379 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.373+0000 7f852ffff700 1 -- 172.21.15.1:0/1708769408 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8528009730 con 0x7f8538101660 2023-11-16T16:48:24.379 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.373+0000 7f852ffff700 1 -- 172.21.15.1:0/1708769408 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f852800ee90 con 0x7f8538101660 2023-11-16T16:48:24.379 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.373+0000 7f853d1af700 1 -- 172.21.15.1:0/1708769408 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8538110390 con 0x7f8538101660 2023-11-16T16:48:24.379 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.373+0000 7f852ffff700 1 -- 172.21.15.1:0/1708769408 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8528010ea0 con 0x7f8538101660 2023-11-16T16:48:24.379 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.373+0000 7f853d1af700 1 -- 172.21.15.1:0/1708769408 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8538110200 con 0x7f8538101660 2023-11-16T16:48:24.379 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.374+0000 7f853d1af700 1 -- 172.21.15.1:0/1708769408 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8538110520 con 0x7f8538101660 2023-11-16T16:48:24.380 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.375+0000 7f852ffff700 1 -- 172.21.15.1:0/1708769408 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f852800cbd0 con 0x7f8538101660 2023-11-16T16:48:24.380 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.376+0000 7f852ffff700 1 --2- 172.21.15.1:0/1708769408 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f85240a12c0 0x7f85240a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.380 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.376+0000 7f852ffff700 1 -- 172.21.15.1:0/1708769408 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f8528014070 con 0x7f8538101660 2023-11-16T16:48:24.381 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.376+0000 7f8536d9d700 1 --2- 172.21.15.1:0/1708769408 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f85240a12c0 0x7f85240a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.381 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.377+0000 7f8536d9d700 1 --2- 172.21.15.1:0/1708769408 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f85240a12c0 0x7f85240a3770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8520009990 tx=0x7f8520005fb0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:24.381 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.380+0000 7f852ffff700 1 -- 172.21.15.1:0/1708769408 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f85280967b0 con 0x7f8538101660 2023-11-16T16:48:24.625 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.623+0000 7f853d1af700 1 -- 172.21.15.1:0/1708769408 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8538067af0 con 0x7f8538101660 2023-11-16T16:48:24.626 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.624+0000 7f852ffff700 1 -- 172.21.15.1:0/1708769408 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10781 (secure 0 0 0) 0x7f85280b1a30 con 0x7f8538101660 2023-11-16T16:48:24.626 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:48:24.626 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2587563684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2587563684},{"type":"v1","addr":"172.21.15.1:6835","nonce":2587563684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/277936168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":277936168},{"type":"v1","addr":"172.21.15.1:6837","nonce":277936168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/1172575045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1172575045},{"type":"v1","addr":"172.21.15.1:6839","nonce":1172575045}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/2631175899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2631175899},{"type":"v1","addr":"172.21.15.1:6841","nonce":2631175899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3693905267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3693905267},{"type":"v1","addr":"172.21.15.53:6833","nonce":3693905267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/164668033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":164668033},{"type":"v1","addr":"172.21.15.26:6833","nonce":164668033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-16T16:37:13.918943+0000","modified":"2023-11-16T16:48:23.343233+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2140542192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2140542192},{"type":"v1","addr":"172.21.15.53:6835","nonce":2140542192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1010760275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1010760275},{"type":"v1","addr":"172.21.15.26:6835","nonce":1010760275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3197060302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3197060302},{"type":"v1","addr":"172.21.15.53:6837","nonce":3197060302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6837/1453504863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1453504863},{"type":"v1","addr":"172.21.15.26:6837","nonce":1453504863}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.53:6839/1044753024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1044753024},{"type":"v1","addr":"172.21.15.53:6839","nonce":1044753024}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6839/2192205949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2192205949},{"type":"v1","addr":"172.21.15.26:6839","nonce":2192205949}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-16T16:48:24.628 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.626+0000 7f853d1af700 1 -- 172.21.15.1:0/1708769408 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f85240a12c0 msgr2=0x7f85240a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:24.629 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.626+0000 7f853d1af700 1 --2- 172.21.15.1:0/1708769408 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f85240a12c0 0x7f85240a3770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8520009990 tx=0x7f8520005fb0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.629 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.626+0000 7f853d1af700 1 -- 172.21.15.1:0/1708769408 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8538101660 msgr2=0x7f8538196920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:24.629 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.626+0000 7f853d1af700 1 --2- 172.21.15.1:0/1708769408 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8538101660 0x7f8538196920 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f852800e940 tx=0x7f852800ec50 comp rx=0 tx=0).stop 2023-11-16T16:48:24.629 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.626+0000 7f853d1af700 1 -- 172.21.15.1:0/1708769408 shutdown_connections 2023-11-16T16:48:24.629 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.626+0000 7f853d1af700 1 --2- 172.21.15.1:0/1708769408 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f85240a12c0 0x7f85240a3770 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.629 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.626+0000 7f853d1af700 1 --2- 172.21.15.1:0/1708769408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f85381055d0 0x7f853819ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.629 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.626+0000 7f853d1af700 1 --2- 172.21.15.1:0/1708769408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8538100cc0 0x7f85381963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.629 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.626+0000 7f853d1af700 1 --2- 172.21.15.1:0/1708769408 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8538101660 0x7f8538196920 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.630 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.626+0000 7f853d1af700 1 -- 172.21.15.1:0/1708769408 >> 172.21.15.1:0/1708769408 conn(0x7f85380774c0 msgr2=0x7f85380fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:24.630 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.626+0000 7f853d1af700 1 -- 172.21.15.1:0/1708769408 shutdown_connections 2023-11-16T16:48:24.630 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.626+0000 7f853d1af700 1 -- 172.21.15.1:0/1708769408 wait complete. 2023-11-16T16:48:24.630 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 29 2023-11-16T16:48:24.639 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:24 smithi001 ceph-mon[98800]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:48:24.639 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:24 smithi001 ceph-mon[98800]: pgmap v286: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-11-16T16:48:24.640 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-11-16T16:48:24.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:24 smithi026 ceph-mon[100112]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:48:24.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:24 smithi026 ceph-mon[100112]: pgmap v286: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-11-16T16:48:24.768 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.766+0000 7f6130cb8700 1 -- 172.21.15.1:0/3662834769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f612c100420 msgr2=0x7f612c10bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:24.768 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.766+0000 7f6130cb8700 1 --2- 172.21.15.1:0/3662834769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f612c100420 0x7f612c10bfd0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f612400b0a0 tx=0x7f612400b3b0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.768 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.766+0000 7f6130cb8700 1 -- 172.21.15.1:0/3662834769 shutdown_connections 2023-11-16T16:48:24.768 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.766+0000 7f6130cb8700 1 --2- 172.21.15.1:0/3662834769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f612c100420 0x7f612c10bfd0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.769 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.766+0000 7f6130cb8700 1 --2- 172.21.15.1:0/3662834769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f612c105250 0x7f612c105620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.769 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.766+0000 7f6130cb8700 1 --2- 172.21.15.1:0/3662834769 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f612c0ffa70 0x7f612c0ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.769 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.766+0000 7f6130cb8700 1 -- 172.21.15.1:0/3662834769 >> 172.21.15.1:0/3662834769 conn(0x7f612c0763a0 msgr2=0x7f612c0767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:24.769 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.766+0000 7f6130cb8700 1 -- 172.21.15.1:0/3662834769 shutdown_connections 2023-11-16T16:48:24.769 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.766+0000 7f6130cb8700 1 -- 172.21.15.1:0/3662834769 wait complete. 2023-11-16T16:48:24.772 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.767+0000 7f6130cb8700 1 Processor -- start 2023-11-16T16:48:24.772 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.767+0000 7f6130cb8700 1 -- start start 2023-11-16T16:48:24.772 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.767+0000 7f6130cb8700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f612c0ffa70 0x7f612c198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.772 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.767+0000 7f6130cb8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f612c100420 0x7f612c198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.772 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.767+0000 7f6130cb8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f612c105250 0x7f612c19cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.772 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.767+0000 7f6130cb8700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612c19d600 con 0x7f612c105250 2023-11-16T16:48:24.772 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.767+0000 7f6130cb8700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612c19d770 con 0x7f612c100420 2023-11-16T16:48:24.772 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.767+0000 7f6130cb8700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612c19d930 con 0x7f612c0ffa70 2023-11-16T16:48:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.767+0000 7f612ad9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f612c105250 0x7f612c19cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.767+0000 7f612ad9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f612c105250 0x7f612c19cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46728/0 (socket says 172.21.15.1:46728) 2023-11-16T16:48:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.767+0000 7f612ad9d700 1 -- 172.21.15.1:0/1621336873 learned_addr learned my addr 172.21.15.1:0/1621336873 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.768+0000 7f612a59c700 1 --2- 172.21.15.1:0/1621336873 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f612c0ffa70 0x7f612c198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.768+0000 7f6129d9b700 1 --2- 172.21.15.1:0/1621336873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f612c100420 0x7f612c198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.768+0000 7f612ad9d700 1 -- 172.21.15.1:0/1621336873 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f612c0ffa70 msgr2=0x7f612c198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.768+0000 7f612ad9d700 1 --2- 172.21.15.1:0/1621336873 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f612c0ffa70 0x7f612c198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.768+0000 7f612ad9d700 1 -- 172.21.15.1:0/1621336873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f612c100420 msgr2=0x7f612c198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.768+0000 7f612ad9d700 1 --2- 172.21.15.1:0/1621336873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f612c100420 0x7f612c198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.774 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.768+0000 7f612ad9d700 1 -- 172.21.15.1:0/1621336873 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6124009d00 con 0x7f612c105250 2023-11-16T16:48:24.774 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.768+0000 7f6129d9b700 1 --2- 172.21.15.1:0/1621336873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f612c100420 0x7f612c198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:48:24.774 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.768+0000 7f612ad9d700 1 --2- 172.21.15.1:0/1621336873 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f612c105250 0x7f612c19cfc0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f61240062a0 tx=0x7f612400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:24.774 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.768+0000 7f61237fe700 1 -- 172.21.15.1:0/1621336873 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612400e040 con 0x7f612c105250 2023-11-16T16:48:24.774 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.768+0000 7f61237fe700 1 -- 172.21.15.1:0/1621336873 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6124004020 con 0x7f612c105250 2023-11-16T16:48:24.774 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.768+0000 7f6130cb8700 1 -- 172.21.15.1:0/1621336873 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f612c10f1e0 con 0x7f612c105250 2023-11-16T16:48:24.774 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.768+0000 7f61237fe700 1 -- 172.21.15.1:0/1621336873 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612401b670 con 0x7f612c105250 2023-11-16T16:48:24.774 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.768+0000 7f6130cb8700 1 -- 172.21.15.1:0/1621336873 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f612c10f050 con 0x7f612c105250 2023-11-16T16:48:24.775 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.770+0000 7f61237fe700 1 -- 172.21.15.1:0/1621336873 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f6124004470 con 0x7f612c105250 2023-11-16T16:48:24.781 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.771+0000 7f61237fe700 1 --2- 172.21.15.1:0/1621336873 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f61180a1340 0x7f61180a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.781 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.771+0000 7f61237fe700 1 -- 172.21.15.1:0/1621336873 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f61240ce310 con 0x7f612c105250 2023-11-16T16:48:24.781 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.771+0000 7f612a59c700 1 --2- 172.21.15.1:0/1621336873 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f61180a1340 0x7f61180a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.781 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.771+0000 7f612a59c700 1 --2- 172.21.15.1:0/1621336873 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f61180a1340 0x7f61180a37f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f611c005950 tx=0x7f611c00a300 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:24.781 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.772+0000 7f613ce44b80 1 Processor -- start 2023-11-16T16:48:24.781 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.772+0000 7f613ce44b80 1 -- start start 2023-11-16T16:48:24.782 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.772+0000 7f613ce44b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd1251db60 0x55bd1251df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.782 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.772+0000 7f613ce44b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd1251e470 0x55bd12522370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.782 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.772+0000 7f613ce44b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bd125228b0 0x55bd12522d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.782 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.772+0000 7f613ce44b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd12423060 con 0x55bd1251db60 2023-11-16T16:48:24.782 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.772+0000 7f613ce44b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd124758a0 con 0x55bd1251e470 2023-11-16T16:48:24.782 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.772+0000 7f613ce44b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd1246ed30 con 0x55bd125228b0 2023-11-16T16:48:24.782 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.772+0000 7f612a59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd1251db60 0x55bd1251df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.782 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.772+0000 7f612a59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd1251db60 0x55bd1251df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46732/0 (socket says 172.21.15.1:46732) 2023-11-16T16:48:24.783 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.772+0000 7f612a59c700 1 -- 172.21.15.1:0/293844151 learned_addr learned my addr 172.21.15.1:0/293844151 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:24.783 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.772+0000 7f6129d9b700 1 --2- 172.21.15.1:0/293844151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd1251e470 0x55bd12522370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.783 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.772+0000 7f612ad9d700 1 --2- 172.21.15.1:0/293844151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bd125228b0 0x55bd12522d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.783 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.772+0000 7f612a59c700 1 -- 172.21.15.1:0/293844151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bd125228b0 msgr2=0x55bd12522d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:24.783 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.772+0000 7f612a59c700 1 --2- 172.21.15.1:0/293844151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bd125228b0 0x55bd12522d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.783 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.772+0000 7f612a59c700 1 -- 172.21.15.1:0/293844151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd1251e470 msgr2=0x55bd12522370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:24.783 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.772+0000 7f612a59c700 1 --2- 172.21.15.1:0/293844151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd1251e470 0x55bd12522370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.784 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.772+0000 7f612a59c700 1 -- 172.21.15.1:0/293844151 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd122f0630 con 0x55bd1251db60 2023-11-16T16:48:24.784 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.772+0000 7f612ad9d700 1 --2- 172.21.15.1:0/293844151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bd125228b0 0x55bd12522d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:48:24.784 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.772+0000 7f6129d9b700 1 --2- 172.21.15.1:0/293844151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd1251e470 0x55bd12522370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:48:24.784 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.773+0000 7f612a59c700 1 --2- 172.21.15.1:0/293844151 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd1251db60 0x55bd1251df30 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f611c00fd10 tx=0x7f611c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad5326e1a6fdf17 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:24.784 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.773+0000 7f6120ff9700 1 -- 172.21.15.1:0/293844151 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c0108c0 con 0x55bd1251db60 2023-11-16T16:48:24.784 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.773+0000 7f6120ff9700 1 -- 172.21.15.1:0/293844151 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f611c020070 con 0x55bd1251db60 2023-11-16T16:48:24.784 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.773+0000 7f6120ff9700 1 -- 172.21.15.1:0/293844151 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c025e30 con 0x55bd1251db60 2023-11-16T16:48:24.784 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.773+0000 7f613ce44b80 1 -- 172.21.15.1:0/293844151 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd1251db60 msgr2=0x55bd1251df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.773+0000 7f613ce44b80 1 --2- 172.21.15.1:0/293844151 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd1251db60 0x55bd1251df30 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f611c00fd10 tx=0x7f611c012000 comp rx=0 tx=0).stop 2023-11-16T16:48:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.774+0000 7f613ce44b80 1 -- 172.21.15.1:0/293844151 shutdown_connections 2023-11-16T16:48:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.774+0000 7f613ce44b80 1 --2- 172.21.15.1:0/293844151 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bd125228b0 0x55bd12522d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.774+0000 7f613ce44b80 1 --2- 172.21.15.1:0/293844151 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd1251db60 0x55bd1251df30 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.774+0000 7f613ce44b80 1 --2- 172.21.15.1:0/293844151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd1251e470 0x55bd12522370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.774+0000 7f613ce44b80 1 -- 172.21.15.1:0/293844151 >> 172.21.15.1:0/293844151 conn(0x55bd12513360 msgr2=0x55bd12518b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.774+0000 7f613ce44b80 1 -- 172.21.15.1:0/293844151 shutdown_connections 2023-11-16T16:48:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.774+0000 7f613ce44b80 1 -- 172.21.15.1:0/293844151 wait complete. 2023-11-16T16:48:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.774+0000 7f613ce44b80 1 -- start start 2023-11-16T16:48:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.774+0000 7f613ce44b80 1 Processor -- start 2023-11-16T16:48:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.774+0000 7f613ce44b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bd1251db60 0x55bd1252a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.774+0000 7f613ce44b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd1251e470 0x55bd125272d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.774+0000 7f613ce44b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd125228b0 0x55bd12527950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.774+0000 7f613ce44b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd124108e0 con 0x55bd1251e470 2023-11-16T16:48:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.774+0000 7f613ce44b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd12410580 con 0x55bd125228b0 2023-11-16T16:48:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.774+0000 7f613ce44b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd122f2920 con 0x55bd1251db60 2023-11-16T16:48:24.787 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.774+0000 7f613ce44b80 10 client.0 _ll_register_callbacks cb 0x55bd1242e780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:48:24.787 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.774+0000 7f6129d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd1251e470 0x55bd125272d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.787 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.775+0000 7f6129d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd1251e470 0x55bd125272d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46740/0 (socket says 172.21.15.1:46740) 2023-11-16T16:48:24.787 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.775+0000 7f6129d9b700 1 -- 172.21.15.1:0/3513016272 learned_addr learned my addr 172.21.15.1:0/3513016272 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:24.787 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.775+0000 7f612a59c700 1 --2- 172.21.15.1:0/3513016272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bd1251db60 0x55bd1252a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.787 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.775+0000 7f612ad9d700 1 --2- 172.21.15.1:0/3513016272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd125228b0 0x55bd12527950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.787 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.775+0000 7f6129d9b700 1 -- 172.21.15.1:0/3513016272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bd1251db60 msgr2=0x55bd1252a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:24.788 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.775+0000 7f6129d9b700 1 --2- 172.21.15.1:0/3513016272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bd1251db60 0x55bd1252a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.788 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.775+0000 7f6129d9b700 1 -- 172.21.15.1:0/3513016272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd125228b0 msgr2=0x55bd12527950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:24.788 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.775+0000 7f6129d9b700 1 --2- 172.21.15.1:0/3513016272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd125228b0 0x55bd12527950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.788 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.775+0000 7f6129d9b700 1 -- 172.21.15.1:0/3513016272 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f611c00f9f0 con 0x55bd1251e470 2023-11-16T16:48:24.788 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.775+0000 7f612ad9d700 1 --2- 172.21.15.1:0/3513016272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd125228b0 0x55bd12527950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:48:24.788 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.775+0000 7f612a59c700 1 --2- 172.21.15.1:0/3513016272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bd1251db60 0x55bd1252a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:48:24.788 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.775+0000 7f6129d9b700 1 --2- 172.21.15.1:0/3513016272 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd1251e470 0x55bd125272d0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f6114010840 tx=0x7f6114010b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1500bf1173fd86c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:24.788 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.775+0000 7f6111ffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:48:24.789 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.775+0000 7f6111ffb700 1 -- 172.21.15.1:0/3513016272 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6114004a30 con 0x55bd1251e470 2023-11-16T16:48:24.789 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.775+0000 7f6111ffb700 1 -- 172.21.15.1:0/3513016272 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f6114017070 con 0x55bd1251e470 2023-11-16T16:48:24.789 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.775+0000 7f613ce44b80 1 -- 172.21.15.1:0/3513016272 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bd12510b20 con 0x55bd1251e470 2023-11-16T16:48:24.789 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.775+0000 7f6111ffb700 1 -- 172.21.15.1:0/3513016272 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6114004b90 con 0x55bd1251e470 2023-11-16T16:48:24.789 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.775+0000 7f613ce44b80 10 client.16800 fetch_fsmap learned FSMap version 29 2023-11-16T16:48:24.789 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.775+0000 7f613ce44b80 1 -- 172.21.15.1:0/3513016272 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55bd12435190 con 0x55bd1251e470 2023-11-16T16:48:24.789 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.776+0000 7f6111ffb700 1 -- 172.21.15.1:0/3513016272 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bd12510b20 con 0x55bd1251e470 2023-11-16T16:48:24.789 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.776+0000 7f6111ffb700 1 -- 172.21.15.1:0/3513016272 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 29) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f61140052a0 con 0x55bd1251e470 2023-11-16T16:48:24.790 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.776+0000 7f613ce44b80 10 client.16800 fetch_fsmap finished waiting for FSMap version 29 2023-11-16T16:48:24.790 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.776+0000 7f613ce44b80 10 client.16800 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-11-16T16:48:24.790 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.776+0000 7f613ce44b80 20 client.16800 populate_metadata read hostname 'smithi001' 2023-11-16T16:48:24.790 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.776+0000 7f613ce44b80 1 --2- 172.21.15.1:0/3513016272 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55bd125378e0 0x55bd12539cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.790 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.776+0000 7f613ce44b80 4 client.16800 mds_command: new command op to 24482 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:48:24.790 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.776+0000 7f613ce44b80 1 -- 172.21.15.1:0/3513016272 --> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bd124310a0 con 0x55bd125378e0 2023-11-16T16:48:24.790 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.776+0000 7f612a59c700 1 --2- 172.21.15.1:0/3513016272 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55bd125378e0 0x55bd12539cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.791 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.777+0000 7f6111ffb700 10 client.16800 ms_handle_connect on v2:172.21.15.26:6836/1453504863 2023-11-16T16:48:24.791 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.777+0000 7f612a59c700 1 --2- 172.21.15.1:0/3513016272 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55bd125378e0 0x55bd12539cf0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f10123aa3f985822 server_cookie=3789da9818b5b434 in_seq=0 out_seq=0 2023-11-16T16:48:24.791 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.777+0000 7f6111ffb700 1 -- 172.21.15.1:0/3513016272 <== mds.2 v2:172.21.15.26:6836/1453504863 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bd124310a0 con 0x55bd125378e0 2023-11-16T16:48:24.791 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.777+0000 7f6111ffb700 10 client.16800 handle_command_reply: tid=0 2023-11-16T16:48:24.791 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.777+0000 7f613ce44b80 1 client.16800 shutdown 2023-11-16T16:48:24.791 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.778+0000 7f613ce44b80 1 -- 172.21.15.1:0/3513016272 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd1251e470 msgr2=0x55bd125272d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:24.791 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.778+0000 7f613ce44b80 1 --2- 172.21.15.1:0/3513016272 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd1251e470 0x55bd125272d0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f6114010840 tx=0x7f6114010b50 comp rx=0 tx=0).stop 2023-11-16T16:48:24.792 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.778+0000 7f613ce44b80 1 -- 172.21.15.1:0/3513016272 shutdown_connections 2023-11-16T16:48:24.792 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.778+0000 7f613ce44b80 1 --2- 172.21.15.1:0/3513016272 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55bd125378e0 0x55bd12539cf0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.792 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.778+0000 7f613ce44b80 1 --2- 172.21.15.1:0/3513016272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd125228b0 0x55bd12527950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.792 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.778+0000 7f613ce44b80 1 --2- 172.21.15.1:0/3513016272 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bd1251db60 0x55bd1252a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.792 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.778+0000 7f613ce44b80 1 --2- 172.21.15.1:0/3513016272 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd1251e470 0x55bd125272d0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.792 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.778+0000 7f6111ffb700 0 client.16800 ms_handle_reset on v2:172.21.15.26:6836/1453504863 2023-11-16T16:48:24.792 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.778+0000 7f613ce44b80 1 -- 172.21.15.1:0/3513016272 >> 172.21.15.1:0/3513016272 conn(0x55bd12513360 msgr2=0x55bd12517de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:24.792 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.778+0000 7f613ce44b80 1 -- 172.21.15.1:0/3513016272 shutdown_connections 2023-11-16T16:48:24.793 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.778+0000 7f613ce44b80 1 -- 172.21.15.1:0/3513016272 wait complete. 2023-11-16T16:48:24.793 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.778+0000 7f613ce44b80 20 client.16800 trim_cache size 0 max 16384 2023-11-16T16:48:24.798 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.791+0000 7f613ce44b80 1 Processor -- start 2023-11-16T16:48:24.798 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.791+0000 7f613ce44b80 1 -- start start 2023-11-16T16:48:24.798 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.791+0000 7f613ce44b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd1251db60 0x55bd12518560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.798 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.791+0000 7f613ce44b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd1251e470 0x55bd12518aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.791+0000 7f613ce44b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bd125228b0 0x55bd125246d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.791+0000 7f613ce44b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd1251ebc0 con 0x55bd1251db60 2023-11-16T16:48:24.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.791+0000 7f613ce44b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd125199a0 con 0x55bd1251e470 2023-11-16T16:48:24.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.791+0000 7f613ce44b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd12512ee0 con 0x55bd125228b0 2023-11-16T16:48:24.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.791+0000 7f612a59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd1251db60 0x55bd12518560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.791+0000 7f612a59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd1251db60 0x55bd12518560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46746/0 (socket says 172.21.15.1:46746) 2023-11-16T16:48:24.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.791+0000 7f612a59c700 1 -- 172.21.15.1:0/3636037378 learned_addr learned my addr 172.21.15.1:0/3636037378 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:24.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.791+0000 7f612ad9d700 1 --2- 172.21.15.1:0/3636037378 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bd125228b0 0x55bd125246d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.792+0000 7f6129d9b700 1 --2- 172.21.15.1:0/3636037378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd1251e470 0x55bd12518aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.792+0000 7f612a59c700 1 -- 172.21.15.1:0/3636037378 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bd125228b0 msgr2=0x55bd125246d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:24.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.792+0000 7f612a59c700 1 --2- 172.21.15.1:0/3636037378 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bd125228b0 0x55bd125246d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.792+0000 7f612a59c700 1 -- 172.21.15.1:0/3636037378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd1251e470 msgr2=0x55bd12518aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:24.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.792+0000 7f612a59c700 1 --2- 172.21.15.1:0/3636037378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd1251e470 0x55bd12518aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.792+0000 7f612a59c700 1 -- 172.21.15.1:0/3636037378 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd12522f50 con 0x55bd1251db60 2023-11-16T16:48:24.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.792+0000 7f6129d9b700 1 --2- 172.21.15.1:0/3636037378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd1251e470 0x55bd12518aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:48:24.800 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.792+0000 7f612a59c700 1 --2- 172.21.15.1:0/3636037378 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd1251db60 0x55bd12518560 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f611c00fd10 tx=0x7f611c028040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2ce94a805cdd2d server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:24.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.792+0000 7f61217fa700 1 -- 172.21.15.1:0/3636037378 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c02b040 con 0x55bd1251db60 2023-11-16T16:48:24.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.792+0000 7f61217fa700 1 -- 172.21.15.1:0/3636037378 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f611c014070 con 0x55bd1251db60 2023-11-16T16:48:24.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.792+0000 7f61217fa700 1 -- 172.21.15.1:0/3636037378 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c010ac0 con 0x55bd1251db60 2023-11-16T16:48:24.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.792+0000 7f613ce44b80 1 -- 172.21.15.1:0/3636037378 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd1251db60 msgr2=0x55bd12518560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:24.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.792+0000 7f613ce44b80 1 --2- 172.21.15.1:0/3636037378 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd1251db60 0x55bd12518560 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f611c00fd10 tx=0x7f611c028040 comp rx=0 tx=0).stop 2023-11-16T16:48:24.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.793+0000 7f613ce44b80 1 -- 172.21.15.1:0/3636037378 shutdown_connections 2023-11-16T16:48:24.801 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.793+0000 7f613ce44b80 1 --2- 172.21.15.1:0/3636037378 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bd125228b0 0x55bd125246d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.802 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.793+0000 7f613ce44b80 1 --2- 172.21.15.1:0/3636037378 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd1251db60 0x55bd12518560 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.802 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.793+0000 7f613ce44b80 1 --2- 172.21.15.1:0/3636037378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd1251e470 0x55bd12518aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.802 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.793+0000 7f613ce44b80 1 -- 172.21.15.1:0/3636037378 >> 172.21.15.1:0/3636037378 conn(0x55bd12513360 msgr2=0x55bd12533be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:24.802 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.793+0000 7f613ce44b80 1 -- 172.21.15.1:0/3636037378 shutdown_connections 2023-11-16T16:48:24.802 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.793+0000 7f613ce44b80 1 -- 172.21.15.1:0/3636037378 wait complete. 2023-11-16T16:48:24.802 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.793+0000 7f613ce44b80 1 -- start start 2023-11-16T16:48:24.802 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.793+0000 7f613ce44b80 1 Processor -- start 2023-11-16T16:48:24.802 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.793+0000 7f613ce44b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bd1251db60 0x55bd1252a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.803 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.793+0000 7f613ce44b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd1251e470 0x55bd1252aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.803 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.793+0000 7f613ce44b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd125228b0 0x55bd1252b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.803 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.793+0000 7f613ce44b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd1258a790 con 0x55bd1251e470 2023-11-16T16:48:24.803 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.793+0000 7f613ce44b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd1258a8d0 con 0x55bd125228b0 2023-11-16T16:48:24.803 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.793+0000 7f613ce44b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd1258aa10 con 0x55bd1251db60 2023-11-16T16:48:24.803 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.793+0000 7f613ce44b80 10 client.0 _ll_register_callbacks cb 0x7f6114008cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:48:24.803 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.793+0000 7f6129d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd1251e470 0x55bd1252aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.803 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.793+0000 7f6129d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd1251e470 0x55bd1252aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46754/0 (socket says 172.21.15.1:46754) 2023-11-16T16:48:24.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.793+0000 7f6129d9b700 1 -- 172.21.15.1:0/2111495883 learned_addr learned my addr 172.21.15.1:0/2111495883 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:24.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.793+0000 7f612a59c700 1 --2- 172.21.15.1:0/2111495883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bd1251db60 0x55bd1252a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.793+0000 7f612ad9d700 1 --2- 172.21.15.1:0/2111495883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd125228b0 0x55bd1252b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.794+0000 7f6129d9b700 1 -- 172.21.15.1:0/2111495883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bd1251db60 msgr2=0x55bd1252a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:24.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.794+0000 7f6129d9b700 1 --2- 172.21.15.1:0/2111495883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bd1251db60 0x55bd1252a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.794+0000 7f6129d9b700 1 -- 172.21.15.1:0/2111495883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd125228b0 msgr2=0x55bd1252b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:24.804 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.794+0000 7f6129d9b700 1 --2- 172.21.15.1:0/2111495883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd125228b0 0x55bd1252b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.805 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.794+0000 7f6129d9b700 1 -- 172.21.15.1:0/2111495883 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bd122f0630 con 0x55bd1251e470 2023-11-16T16:48:24.805 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.794+0000 7f6129d9b700 1 --2- 172.21.15.1:0/2111495883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd1251e470 0x55bd1252aec0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f6114005600 tx=0x7f6114008640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=424fe3c88ca8bc13 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:24.805 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.794+0000 7f6111ffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:48:24.805 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.794+0000 7f6111ffb700 1 -- 172.21.15.1:0/2111495883 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6114011e30 con 0x55bd1251e470 2023-11-16T16:48:24.805 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.794+0000 7f6111ffb700 1 -- 172.21.15.1:0/2111495883 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f6114000e40 con 0x55bd1251e470 2023-11-16T16:48:24.805 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.794+0000 7f613ce44b80 1 -- 172.21.15.1:0/2111495883 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f611c020070 con 0x55bd1251e470 2023-11-16T16:48:24.805 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.794+0000 7f6111ffb700 1 -- 172.21.15.1:0/2111495883 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f611400e7e0 con 0x55bd1251e470 2023-11-16T16:48:24.805 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.794+0000 7f613ce44b80 10 client.16806 fetch_fsmap learned FSMap version 29 2023-11-16T16:48:24.806 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.794+0000 7f613ce44b80 1 -- 172.21.15.1:0/2111495883 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55bd1229cde0 con 0x55bd1251e470 2023-11-16T16:48:24.806 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.794+0000 7f6111ffb700 1 -- 172.21.15.1:0/2111495883 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6114039020 con 0x55bd1251e470 2023-11-16T16:48:24.806 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.794+0000 7f6111ffb700 1 -- 172.21.15.1:0/2111495883 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 29) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f6114024880 con 0x55bd1251e470 2023-11-16T16:48:24.806 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.795+0000 7f613ce44b80 10 client.16806 fetch_fsmap finished waiting for FSMap version 29 2023-11-16T16:48:24.806 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.795+0000 7f613ce44b80 10 client.16806 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-11-16T16:48:24.806 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.795+0000 7f613ce44b80 20 client.16806 populate_metadata read hostname 'smithi001' 2023-11-16T16:48:24.806 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.795+0000 7f613ce44b80 1 --2- 172.21.15.1:0/2111495883 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55bd1258c020 0x55bd1258e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.806 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.795+0000 7f613ce44b80 4 client.16806 mds_command: new command op to 24482 tid=0[{"prefix": "damage ls"}] 2023-11-16T16:48:24.807 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.795+0000 7f613ce44b80 1 -- 172.21.15.1:0/2111495883 --> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bd1243d660 con 0x55bd1258c020 2023-11-16T16:48:24.807 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.795+0000 7f612a59c700 1 --2- 172.21.15.1:0/2111495883 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55bd1258c020 0x55bd1258e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.807 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.795+0000 7f612a59c700 1 --2- 172.21.15.1:0/2111495883 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55bd1258c020 0x55bd1258e430 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b02c9a853ff9bcfe server_cookie=400f2db7c8c965d9 in_seq=0 out_seq=0 2023-11-16T16:48:24.807 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.795+0000 7f6111ffb700 10 client.16806 ms_handle_connect on v2:172.21.15.26:6836/1453504863 2023-11-16T16:48:24.807 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.795+0000 7f6111ffb700 1 -- 172.21.15.1:0/2111495883 <== mds.2 v2:172.21.15.26:6836/1453504863 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bd1243d660 con 0x55bd1258c020 2023-11-16T16:48:24.807 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.795+0000 7f6111ffb700 10 client.16806 handle_command_reply: tid=0 2023-11-16T16:48:24.807 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.795+0000 7f613ce44b80 1 client.16806 shutdown 2023-11-16T16:48:24.807 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.795+0000 7f613ce44b80 1 -- 172.21.15.1:0/2111495883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd1251e470 msgr2=0x55bd1252aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:24.808 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.795+0000 7f613ce44b80 1 --2- 172.21.15.1:0/2111495883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd1251e470 0x55bd1252aec0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f6114005600 tx=0x7f6114008640 comp rx=0 tx=0).stop 2023-11-16T16:48:24.808 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.796+0000 7f613ce44b80 1 -- 172.21.15.1:0/2111495883 shutdown_connections 2023-11-16T16:48:24.808 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.796+0000 7f613ce44b80 1 --2- 172.21.15.1:0/2111495883 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x55bd1258c020 0x55bd1258e430 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.808 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.796+0000 7f613ce44b80 1 --2- 172.21.15.1:0/2111495883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bd125228b0 0x55bd1252b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.808 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.796+0000 7f613ce44b80 1 --2- 172.21.15.1:0/2111495883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55bd1251db60 0x55bd1252a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.808 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.796+0000 7f6111ffb700 0 client.16806 ms_handle_reset on v2:172.21.15.26:6836/1453504863 2023-11-16T16:48:24.808 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.796+0000 7f613ce44b80 1 --2- 172.21.15.1:0/2111495883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd1251e470 0x55bd1252aec0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.808 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.796+0000 7f613ce44b80 1 -- 172.21.15.1:0/2111495883 >> 172.21.15.1:0/2111495883 conn(0x55bd12513360 msgr2=0x55bd1251c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:24.809 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.796+0000 7f613ce44b80 1 -- 172.21.15.1:0/2111495883 shutdown_connections 2023-11-16T16:48:24.809 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.796+0000 7f613ce44b80 1 -- 172.21.15.1:0/2111495883 wait complete. 2023-11-16T16:48:24.809 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.796+0000 7f613ce44b80 20 client.16806 trim_cache size 0 max 16384 2023-11-16T16:48:24.809 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2023-11-16T16:48:24.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.796+0000 7f6130cb8700 1 -- 172.21.15.1:0/1621336873 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f61180a1340 msgr2=0x7f61180a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:24.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.796+0000 7f6130cb8700 1 --2- 172.21.15.1:0/1621336873 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f61180a1340 0x7f61180a37f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f611c005950 tx=0x7f611c00a300 comp rx=0 tx=0).stop 2023-11-16T16:48:24.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.796+0000 7f6130cb8700 1 -- 172.21.15.1:0/1621336873 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f612c105250 msgr2=0x7f612c19cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:24.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.796+0000 7f6130cb8700 1 --2- 172.21.15.1:0/1621336873 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f612c105250 0x7f612c19cfc0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f61240062a0 tx=0x7f612400cf70 comp rx=0 tx=0).stop 2023-11-16T16:48:24.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.796+0000 7f6130cb8700 1 -- 172.21.15.1:0/1621336873 shutdown_connections 2023-11-16T16:48:24.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.796+0000 7f6130cb8700 1 --2- 172.21.15.1:0/1621336873 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f61180a1340 0x7f61180a37f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.796+0000 7f6130cb8700 1 --2- 172.21.15.1:0/1621336873 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f612c105250 0x7f612c19cfc0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.796+0000 7f6130cb8700 1 --2- 172.21.15.1:0/1621336873 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f612c0ffa70 0x7f612c198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.796+0000 7f6130cb8700 1 --2- 172.21.15.1:0/1621336873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f612c100420 0x7f612c198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.796+0000 7f6130cb8700 1 -- 172.21.15.1:0/1621336873 >> 172.21.15.1:0/1621336873 conn(0x7f612c0763a0 msgr2=0x7f612c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:24.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.796+0000 7f6130cb8700 1 -- 172.21.15.1:0/1621336873 shutdown_connections 2023-11-16T16:48:24.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.796+0000 7f6130cb8700 1 -- 172.21.15.1:0/1621336873 wait complete. 2023-11-16T16:48:24.812 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-11-16T16:48:24.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:24 smithi053 ceph-mon[100638]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:48:24.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:24 smithi053 ceph-mon[100638]: pgmap v286: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-11-16T16:48:24.942 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.938+0000 7f7d774a4700 1 -- 172.21.15.1:0/627954743 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d701002a0 msgr2=0x7f7d7010bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:24.942 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.938+0000 7f7d774a4700 1 --2- 172.21.15.1:0/627954743 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d701002a0 0x7f7d7010bfd0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f7d6800b0d0 tx=0x7f7d6800b3e0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.942 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.938+0000 7f7d774a4700 1 -- 172.21.15.1:0/627954743 shutdown_connections 2023-11-16T16:48:24.942 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.938+0000 7f7d774a4700 1 --2- 172.21.15.1:0/627954743 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d701002a0 0x7f7d7010bfd0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.942 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.938+0000 7f7d774a4700 1 --2- 172.21.15.1:0/627954743 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d70104140 0x7f7d70104510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.942 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.938+0000 7f7d774a4700 1 --2- 172.21.15.1:0/627954743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d700ff8f0 0x7f7d700ffd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.943 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.938+0000 7f7d774a4700 1 -- 172.21.15.1:0/627954743 >> 172.21.15.1:0/627954743 conn(0x7f7d70075880 msgr2=0x7f7d70075c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:24.943 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.938+0000 7f7d774a4700 1 -- 172.21.15.1:0/627954743 shutdown_connections 2023-11-16T16:48:24.943 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.938+0000 7f7d774a4700 1 -- 172.21.15.1:0/627954743 wait complete. 2023-11-16T16:48:24.943 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.939+0000 7f7d774a4700 1 Processor -- start 2023-11-16T16:48:24.943 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.939+0000 7f7d774a4700 1 -- start start 2023-11-16T16:48:24.943 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.939+0000 7f7d774a4700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d700ff8f0 0x7f7d7019e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.943 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.939+0000 7f7d774a4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d701002a0 0x7f7d7019e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.943 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.939+0000 7f7d774a4700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d70104140 0x7f7d70198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.943 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.939+0000 7f7d774a4700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d70198b60 con 0x7f7d70104140 2023-11-16T16:48:24.944 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.939+0000 7f7d774a4700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d70198cd0 con 0x7f7d701002a0 2023-11-16T16:48:24.944 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.939+0000 7f7d774a4700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d70198e90 con 0x7f7d700ff8f0 2023-11-16T16:48:24.944 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.939+0000 7f7d75a41700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d70104140 0x7f7d70198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.944 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.939+0000 7f7d75a41700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d70104140 0x7f7d70198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46770/0 (socket says 172.21.15.1:46770) 2023-11-16T16:48:24.944 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.939+0000 7f7d75a41700 1 -- 172.21.15.1:0/328154764 learned_addr learned my addr 172.21.15.1:0/328154764 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:24.944 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.939+0000 7f7d75240700 1 --2- 172.21.15.1:0/328154764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d700ff8f0 0x7f7d7019e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.944 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.939+0000 7f7d74a3f700 1 --2- 172.21.15.1:0/328154764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d701002a0 0x7f7d7019e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.944 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.939+0000 7f7d75a41700 1 -- 172.21.15.1:0/328154764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d700ff8f0 msgr2=0x7f7d7019e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:24.945 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.939+0000 7f7d75a41700 1 --2- 172.21.15.1:0/328154764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d700ff8f0 0x7f7d7019e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.945 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.939+0000 7f7d75a41700 1 -- 172.21.15.1:0/328154764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d701002a0 msgr2=0x7f7d7019e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:24.945 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.939+0000 7f7d75a41700 1 --2- 172.21.15.1:0/328154764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d701002a0 0x7f7d7019e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.945 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.939+0000 7f7d75a41700 1 -- 172.21.15.1:0/328154764 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d68009d00 con 0x7f7d70104140 2023-11-16T16:48:24.945 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.940+0000 7f7d75a41700 1 --2- 172.21.15.1:0/328154764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d70104140 0x7f7d70198500 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f7d68007860 tx=0x7f7d68007890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:24.945 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.940+0000 7f7d667fc700 1 -- 172.21.15.1:0/328154764 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6800e040 con 0x7f7d70104140 2023-11-16T16:48:24.945 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.940+0000 7f7d774a4700 1 -- 172.21.15.1:0/328154764 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d7010f220 con 0x7f7d70104140 2023-11-16T16:48:24.945 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.940+0000 7f7d667fc700 1 -- 172.21.15.1:0/328154764 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d6800bdb0 con 0x7f7d70104140 2023-11-16T16:48:24.946 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.940+0000 7f7d667fc700 1 -- 172.21.15.1:0/328154764 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6801d470 con 0x7f7d70104140 2023-11-16T16:48:24.946 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.940+0000 7f7d774a4700 1 -- 172.21.15.1:0/328154764 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d7010f090 con 0x7f7d70104140 2023-11-16T16:48:24.951 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.943+0000 7f7d667fc700 1 -- 172.21.15.1:0/328154764 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f7d68005020 con 0x7f7d70104140 2023-11-16T16:48:24.951 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.943+0000 7f7d667fc700 1 --2- 172.21.15.1:0/328154764 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f7d5c0a12c0 0x7f7d5c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.951 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.944+0000 7f7d75240700 1 --2- 172.21.15.1:0/328154764 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f7d5c0a12c0 0x7f7d5c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.951 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.944+0000 7f7d667fc700 1 -- 172.21.15.1:0/328154764 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f7d680cdd20 con 0x7f7d70104140 2023-11-16T16:48:24.951 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.944+0000 7f7d75240700 1 --2- 172.21.15.1:0/328154764 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f7d5c0a12c0 0x7f7d5c0a3770 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7d60009be0 tx=0x7f7d60009260 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:24.952 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.944+0000 7f7d83630b80 1 Processor -- start 2023-11-16T16:48:24.952 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.944+0000 7f7d83630b80 1 -- start start 2023-11-16T16:48:24.952 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.944+0000 7f7d83630b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c893df4b60 0x55c893df4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.952 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.944+0000 7f7d83630b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c893df5470 0x55c893df9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.952 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.944+0000 7f7d83630b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c893df98b0 0x55c893df9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.952 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.944+0000 7f7d83630b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c893cfa060 con 0x55c893df5470 2023-11-16T16:48:24.952 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.944+0000 7f7d83630b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c893d4c8a0 con 0x55c893df98b0 2023-11-16T16:48:24.952 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.944+0000 7f7d83630b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c893d45d30 con 0x55c893df4b60 2023-11-16T16:48:24.953 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.945+0000 7f7d74a3f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c893df5470 0x55c893df9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.953 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.945+0000 7f7d74a3f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c893df5470 0x55c893df9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46776/0 (socket says 172.21.15.1:46776) 2023-11-16T16:48:24.953 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.945+0000 7f7d74a3f700 1 -- 172.21.15.1:0/2318438404 learned_addr learned my addr 172.21.15.1:0/2318438404 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:24.953 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.945+0000 7f7d75a41700 1 --2- 172.21.15.1:0/2318438404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c893df98b0 0x55c893df9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.953 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.945+0000 7f7d75240700 1 --2- 172.21.15.1:0/2318438404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c893df4b60 0x55c893df4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.953 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.945+0000 7f7d74a3f700 1 -- 172.21.15.1:0/2318438404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c893df4b60 msgr2=0x55c893df4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:24.953 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.945+0000 7f7d74a3f700 1 --2- 172.21.15.1:0/2318438404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c893df4b60 0x55c893df4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.953 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.945+0000 7f7d74a3f700 1 -- 172.21.15.1:0/2318438404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c893df98b0 msgr2=0x55c893df9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:24.954 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.945+0000 7f7d74a3f700 1 --2- 172.21.15.1:0/2318438404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c893df98b0 0x55c893df9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.954 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.945+0000 7f7d74a3f700 1 -- 172.21.15.1:0/2318438404 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c893bc7630 con 0x55c893df5470 2023-11-16T16:48:24.954 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.945+0000 7f7d74a3f700 1 --2- 172.21.15.1:0/2318438404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c893df5470 0x55c893df9370 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c00dc40 tx=0x7f7d6c00df50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f9fa7ad36f65ddf server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:24.954 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.945+0000 7f7d5b7fe700 1 -- 172.21.15.1:0/2318438404 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c00fb70 con 0x55c893df5470 2023-11-16T16:48:24.954 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.945+0000 7f7d5b7fe700 1 -- 172.21.15.1:0/2318438404 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d6c004cf0 con 0x55c893df5470 2023-11-16T16:48:24.954 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.945+0000 7f7d5b7fe700 1 -- 172.21.15.1:0/2318438404 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c0056f0 con 0x55c893df5470 2023-11-16T16:48:24.954 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.946+0000 7f7d83630b80 1 -- 172.21.15.1:0/2318438404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c893df5470 msgr2=0x55c893df9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:24.955 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.946+0000 7f7d83630b80 1 --2- 172.21.15.1:0/2318438404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c893df5470 0x55c893df9370 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c00dc40 tx=0x7f7d6c00df50 comp rx=0 tx=0).stop 2023-11-16T16:48:24.955 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.946+0000 7f7d83630b80 1 -- 172.21.15.1:0/2318438404 shutdown_connections 2023-11-16T16:48:24.955 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.946+0000 7f7d83630b80 1 --2- 172.21.15.1:0/2318438404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c893df98b0 0x55c893df9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.955 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.946+0000 7f7d83630b80 1 --2- 172.21.15.1:0/2318438404 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c893df4b60 0x55c893df4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.955 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.946+0000 7f7d83630b80 1 --2- 172.21.15.1:0/2318438404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c893df5470 0x55c893df9370 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.955 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.946+0000 7f7d83630b80 1 -- 172.21.15.1:0/2318438404 >> 172.21.15.1:0/2318438404 conn(0x55c893dea360 msgr2=0x55c893defb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:24.955 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.946+0000 7f7d83630b80 1 -- 172.21.15.1:0/2318438404 shutdown_connections 2023-11-16T16:48:24.955 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.946+0000 7f7d83630b80 1 -- 172.21.15.1:0/2318438404 wait complete. 2023-11-16T16:48:24.955 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.946+0000 7f7d83630b80 1 -- start start 2023-11-16T16:48:24.956 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.947+0000 7f7d83630b80 1 Processor -- start 2023-11-16T16:48:24.956 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.947+0000 7f7d83630b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c893df4b60 0x55c893e01060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.956 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.947+0000 7f7d83630b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c893df5470 0x55c893dfe270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.956 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.947+0000 7f7d83630b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c893df98b0 0x55c893dfe8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.956 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.947+0000 7f7d83630b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c893ce78e0 con 0x55c893df4b60 2023-11-16T16:48:24.956 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.947+0000 7f7d83630b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c893ce7580 con 0x55c893df5470 2023-11-16T16:48:24.956 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.947+0000 7f7d83630b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c893bc9920 con 0x55c893df98b0 2023-11-16T16:48:24.956 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.947+0000 7f7d83630b80 10 client.0 _ll_register_callbacks cb 0x55c893d05780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:48:24.957 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.947+0000 7f7d75240700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c893df4b60 0x55c893e01060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.957 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.947+0000 7f7d75240700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c893df4b60 0x55c893e01060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46780/0 (socket says 172.21.15.1:46780) 2023-11-16T16:48:24.957 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.947+0000 7f7d75240700 1 -- 172.21.15.1:0/3755775152 learned_addr learned my addr 172.21.15.1:0/3755775152 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:24.957 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.947+0000 7f7d74a3f700 1 --2- 172.21.15.1:0/3755775152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c893df5470 0x55c893dfe270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.957 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.947+0000 7f7d75a41700 1 --2- 172.21.15.1:0/3755775152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c893df98b0 0x55c893dfe8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.957 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.948+0000 7f7d75240700 1 -- 172.21.15.1:0/3755775152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c893df98b0 msgr2=0x55c893dfe8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:24.957 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.948+0000 7f7d75240700 1 --2- 172.21.15.1:0/3755775152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c893df98b0 0x55c893dfe8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.957 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.948+0000 7f7d75240700 1 -- 172.21.15.1:0/3755775152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c893df5470 msgr2=0x55c893dfe270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:24.958 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.948+0000 7f7d75240700 1 --2- 172.21.15.1:0/3755775152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c893df5470 0x55c893dfe270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.958 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.948+0000 7f7d75240700 1 -- 172.21.15.1:0/3755775152 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d6c00d8a0 con 0x55c893df4b60 2023-11-16T16:48:24.958 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.948+0000 7f7d75240700 1 --2- 172.21.15.1:0/3755775152 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c893df4b60 0x55c893e01060 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f7d6001e8a0 tx=0x7f7d600047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b673b403f2b8d18 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:24.958 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.948+0000 7f7d58ff9700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:48:24.958 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.948+0000 7f7d58ff9700 1 -- 172.21.15.1:0/3755775152 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d60004020 con 0x55c893df4b60 2023-11-16T16:48:24.958 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.948+0000 7f7d58ff9700 1 -- 172.21.15.1:0/3755775152 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f7d60004180 con 0x55c893df4b60 2023-11-16T16:48:24.958 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.948+0000 7f7d83630b80 1 -- 172.21.15.1:0/3755775152 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c893de7b20 con 0x55c893df4b60 2023-11-16T16:48:24.958 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.948+0000 7f7d58ff9700 1 -- 172.21.15.1:0/3755775152 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d60004340 con 0x55c893df4b60 2023-11-16T16:48:24.959 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.948+0000 7f7d83630b80 10 client.16818 fetch_fsmap learned FSMap version 29 2023-11-16T16:48:24.959 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.948+0000 7f7d83630b80 1 -- 172.21.15.1:0/3755775152 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c893d0c190 con 0x55c893df4b60 2023-11-16T16:48:24.959 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.948+0000 7f7d58ff9700 1 -- 172.21.15.1:0/3755775152 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c893de7b20 con 0x55c893df4b60 2023-11-16T16:48:24.959 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.949+0000 7f7d58ff9700 1 -- 172.21.15.1:0/3755775152 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 29) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f7d60031670 con 0x55c893df4b60 2023-11-16T16:48:24.959 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.949+0000 7f7d83630b80 10 client.16818 fetch_fsmap finished waiting for FSMap version 29 2023-11-16T16:48:24.959 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.949+0000 7f7d83630b80 10 client.16818 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-11-16T16:48:24.959 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.949+0000 7f7d83630b80 20 client.16818 populate_metadata read hostname 'smithi001' 2023-11-16T16:48:24.959 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.949+0000 7f7d83630b80 1 --2- 172.21.15.1:0/3755775152 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55c893e0e840 0x55c893e10c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.960 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.949+0000 7f7d83630b80 4 client.16818 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:48:24.960 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.949+0000 7f7d83630b80 1 -- 172.21.15.1:0/3755775152 --> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c893d080a0 con 0x55c893e0e840 2023-11-16T16:48:24.960 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.949+0000 7f7d74a3f700 1 --2- 172.21.15.1:0/3755775152 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55c893e0e840 0x55c893e10c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.960 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.949+0000 7f7d74a3f700 1 --2- 172.21.15.1:0/3755775152 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55c893e0e840 0x55c893e10c50 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=451cec88c7678672 server_cookie=72ac3d7b4941b612 in_seq=0 out_seq=0 2023-11-16T16:48:24.960 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.949+0000 7f7d58ff9700 10 client.16818 ms_handle_connect on v2:172.21.15.53:6838/1044753024 2023-11-16T16:48:24.961 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.950+0000 7f7d58ff9700 1 -- 172.21.15.1:0/3755775152 <== mds.1 v2:172.21.15.53:6838/1044753024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c893d080a0 con 0x55c893e0e840 2023-11-16T16:48:24.961 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.950+0000 7f7d58ff9700 10 client.16818 handle_command_reply: tid=0 2023-11-16T16:48:24.961 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.950+0000 7f7d83630b80 1 client.16818 shutdown 2023-11-16T16:48:24.962 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.950+0000 7f7d83630b80 1 -- 172.21.15.1:0/3755775152 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c893df4b60 msgr2=0x55c893e01060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:24.962 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.950+0000 7f7d83630b80 1 --2- 172.21.15.1:0/3755775152 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c893df4b60 0x55c893e01060 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f7d6001e8a0 tx=0x7f7d600047c0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.962 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.950+0000 7f7d83630b80 1 -- 172.21.15.1:0/3755775152 shutdown_connections 2023-11-16T16:48:24.962 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.950+0000 7f7d83630b80 1 --2- 172.21.15.1:0/3755775152 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c893df98b0 0x55c893dfe8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.962 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.950+0000 7f7d83630b80 1 --2- 172.21.15.1:0/3755775152 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c893df4b60 0x55c893e01060 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.962 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.950+0000 7f7d83630b80 1 --2- 172.21.15.1:0/3755775152 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55c893e0e840 0x55c893e10c50 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.962 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.950+0000 7f7d83630b80 1 --2- 172.21.15.1:0/3755775152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c893df5470 0x55c893dfe270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.963 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.950+0000 7f7d58ff9700 0 client.16818 ms_handle_reset on v2:172.21.15.53:6838/1044753024 2023-11-16T16:48:24.963 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.950+0000 7f7d83630b80 1 -- 172.21.15.1:0/3755775152 >> 172.21.15.1:0/3755775152 conn(0x55c893dea360 msgr2=0x55c893dfad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:24.963 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.951+0000 7f7d83630b80 1 -- 172.21.15.1:0/3755775152 shutdown_connections 2023-11-16T16:48:24.963 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.951+0000 7f7d83630b80 1 -- 172.21.15.1:0/3755775152 wait complete. 2023-11-16T16:48:24.963 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.951+0000 7f7d83630b80 20 client.16818 trim_cache size 0 max 16384 2023-11-16T16:48:24.967 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.964+0000 7f7d83630b80 1 Processor -- start 2023-11-16T16:48:24.967 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.964+0000 7f7d83630b80 1 -- start start 2023-11-16T16:48:24.967 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.964+0000 7f7d83630b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c893df4b60 0x55c893e059a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.967 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.964+0000 7f7d83630b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c893df5470 0x55c893e00b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.968 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.964+0000 7f7d83630b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c893df98b0 0x55c893df78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.968 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.964+0000 7f7d83630b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c893e0bfa0 con 0x55c893df4b60 2023-11-16T16:48:24.968 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.964+0000 7f7d83630b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c893dfeea0 con 0x55c893df5470 2023-11-16T16:48:24.968 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.964+0000 7f7d83630b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c893dfc7d0 con 0x55c893df98b0 2023-11-16T16:48:24.968 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.964+0000 7f7d75240700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c893df4b60 0x55c893e059a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.968 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.964+0000 7f7d75240700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c893df4b60 0x55c893e059a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46788/0 (socket says 172.21.15.1:46788) 2023-11-16T16:48:24.968 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.964+0000 7f7d75240700 1 -- 172.21.15.1:0/516653188 learned_addr learned my addr 172.21.15.1:0/516653188 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:24.969 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.964+0000 7f7d75a41700 1 --2- 172.21.15.1:0/516653188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c893df98b0 0x55c893df78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.969 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.964+0000 7f7d75240700 1 -- 172.21.15.1:0/516653188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c893df98b0 msgr2=0x55c893df78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:24.969 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.964+0000 7f7d75240700 1 --2- 172.21.15.1:0/516653188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c893df98b0 0x55c893df78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.969 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.964+0000 7f7d75240700 1 -- 172.21.15.1:0/516653188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c893df5470 msgr2=0x55c893e00b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:24.969 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.964+0000 7f7d75240700 1 --2- 172.21.15.1:0/516653188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c893df5470 0x55c893e00b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.969 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.964+0000 7f7d75240700 1 -- 172.21.15.1:0/516653188 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c893debb00 con 0x55c893df4b60 2023-11-16T16:48:24.969 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.964+0000 7f7d75240700 1 --2- 172.21.15.1:0/516653188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c893df4b60 0x55c893e059a0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f7d60009340 tx=0x7f7d60031bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6bd1c8638462d14 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:24.969 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.965+0000 7f7d5bfff700 1 -- 172.21.15.1:0/516653188 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6001ee90 con 0x55c893df4b60 2023-11-16T16:48:24.970 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.965+0000 7f7d5bfff700 1 -- 172.21.15.1:0/516653188 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d60023070 con 0x55c893df4b60 2023-11-16T16:48:24.970 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.965+0000 7f7d5bfff700 1 -- 172.21.15.1:0/516653188 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d60028c50 con 0x55c893df4b60 2023-11-16T16:48:24.970 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.965+0000 7f7d83630b80 1 -- 172.21.15.1:0/516653188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c893df4b60 msgr2=0x55c893e059a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:24.970 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.965+0000 7f7d83630b80 1 --2- 172.21.15.1:0/516653188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c893df4b60 0x55c893e059a0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f7d60009340 tx=0x7f7d60031bf0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.970 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.965+0000 7f7d83630b80 1 -- 172.21.15.1:0/516653188 shutdown_connections 2023-11-16T16:48:24.970 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.965+0000 7f7d83630b80 1 --2- 172.21.15.1:0/516653188 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c893df98b0 0x55c893df78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.970 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.965+0000 7f7d83630b80 1 --2- 172.21.15.1:0/516653188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c893df4b60 0x55c893e059a0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.970 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.965+0000 7f7d83630b80 1 --2- 172.21.15.1:0/516653188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c893df5470 0x55c893e00b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.971 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.965+0000 7f7d83630b80 1 -- 172.21.15.1:0/516653188 >> 172.21.15.1:0/516653188 conn(0x55c893dea360 msgr2=0x55c893dfbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:24.971 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.965+0000 7f7d83630b80 1 -- 172.21.15.1:0/516653188 shutdown_connections 2023-11-16T16:48:24.971 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.965+0000 7f7d83630b80 1 -- 172.21.15.1:0/516653188 wait complete. 2023-11-16T16:48:24.971 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.965+0000 7f7d83630b80 1 -- start start 2023-11-16T16:48:24.971 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.966+0000 7f7d83630b80 1 Processor -- start 2023-11-16T16:48:24.974 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.966+0000 7f7d83630b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c893df4b60 0x55c893e03990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.974 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.966+0000 7f7d83630b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c893df5470 0x55c893e03ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.975 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.966+0000 7f7d83630b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c893df98b0 0x55c893e04410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.975 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.966+0000 7f7d83630b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c893e616c0 con 0x55c893df4b60 2023-11-16T16:48:24.975 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.966+0000 7f7d83630b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c893e61800 con 0x55c893df5470 2023-11-16T16:48:24.975 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.966+0000 7f7d83630b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c893e61990 con 0x55c893df98b0 2023-11-16T16:48:24.975 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.966+0000 7f7d83630b80 10 client.0 _ll_register_callbacks cb 0x7f7d6001e3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:48:24.975 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.966+0000 7f7d75240700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c893df4b60 0x55c893e03990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.975 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.966+0000 7f7d75240700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c893df4b60 0x55c893e03990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46794/0 (socket says 172.21.15.1:46794) 2023-11-16T16:48:24.976 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.966+0000 7f7d75240700 1 -- 172.21.15.1:0/2679525401 learned_addr learned my addr 172.21.15.1:0/2679525401 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:24.976 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.966+0000 7f7d75a41700 1 --2- 172.21.15.1:0/2679525401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c893df98b0 0x55c893e04410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.976 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.966+0000 7f7d74a3f700 1 --2- 172.21.15.1:0/2679525401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c893df5470 0x55c893e03ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.976 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.967+0000 7f7d75240700 1 -- 172.21.15.1:0/2679525401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c893df98b0 msgr2=0x55c893e04410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:24.976 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.967+0000 7f7d75240700 1 --2- 172.21.15.1:0/2679525401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c893df98b0 0x55c893e04410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.976 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.967+0000 7f7d75240700 1 -- 172.21.15.1:0/2679525401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c893df5470 msgr2=0x55c893e03ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:24.976 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.967+0000 7f7d75240700 1 --2- 172.21.15.1:0/2679525401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c893df5470 0x55c893e03ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.976 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.967+0000 7f7d75240700 1 -- 172.21.15.1:0/2679525401 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c893d0c190 con 0x55c893df4b60 2023-11-16T16:48:24.977 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.967+0000 7f7d75a41700 1 --2- 172.21.15.1:0/2679525401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c893df98b0 0x55c893e04410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:48:24.977 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.967+0000 7f7d75240700 1 --2- 172.21.15.1:0/2679525401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c893df4b60 0x55c893e03990 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f7d60011850 tx=0x7f7d60031d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d2ad5dfba0a611 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:24.977 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.967+0000 7f7d58ff9700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:48:24.977 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.967+0000 7f7d58ff9700 1 -- 172.21.15.1:0/2679525401 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d600285c0 con 0x55c893df4b60 2023-11-16T16:48:24.977 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.967+0000 7f7d58ff9700 1 -- 172.21.15.1:0/2679525401 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f7d600287f0 con 0x55c893df4b60 2023-11-16T16:48:24.977 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.967+0000 7f7d83630b80 1 -- 172.21.15.1:0/2679525401 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d6c004cf0 con 0x55c893df4b60 2023-11-16T16:48:24.977 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.967+0000 7f7d58ff9700 1 -- 172.21.15.1:0/2679525401 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d600289f0 con 0x55c893df4b60 2023-11-16T16:48:24.977 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.967+0000 7f7d83630b80 10 client.16824 fetch_fsmap learned FSMap version 29 2023-11-16T16:48:24.977 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.967+0000 7f7d83630b80 1 -- 172.21.15.1:0/2679525401 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c893b73de0 con 0x55c893df4b60 2023-11-16T16:48:24.978 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.967+0000 7f7d58ff9700 1 -- 172.21.15.1:0/2679525401 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d6c004cf0 con 0x55c893df4b60 2023-11-16T16:48:24.978 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.967+0000 7f7d58ff9700 1 -- 172.21.15.1:0/2679525401 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 29) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f7d600046f0 con 0x55c893df4b60 2023-11-16T16:48:24.978 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.967+0000 7f7d83630b80 10 client.16824 fetch_fsmap finished waiting for FSMap version 29 2023-11-16T16:48:24.978 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.967+0000 7f7d83630b80 10 client.16824 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-11-16T16:48:24.978 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.967+0000 7f7d83630b80 20 client.16824 populate_metadata read hostname 'smithi001' 2023-11-16T16:48:24.978 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.967+0000 7f7d83630b80 1 --2- 172.21.15.1:0/2679525401 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55c893e62c60 0x55c893e65070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:24.978 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.967+0000 7f7d83630b80 4 client.16824 mds_command: new command op to 24493 tid=0[{"prefix": "damage ls"}] 2023-11-16T16:48:24.978 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.967+0000 7f7d83630b80 1 -- 172.21.15.1:0/2679525401 --> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c893d14660 con 0x55c893e62c60 2023-11-16T16:48:24.979 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.968+0000 7f7d74a3f700 1 --2- 172.21.15.1:0/2679525401 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55c893e62c60 0x55c893e65070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:24.979 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.968+0000 7f7d74a3f700 1 --2- 172.21.15.1:0/2679525401 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55c893e62c60 0x55c893e65070 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3fabed7b62399ccf server_cookie=7908830357826de8 in_seq=0 out_seq=0 2023-11-16T16:48:24.979 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.968+0000 7f7d58ff9700 10 client.16824 ms_handle_connect on v2:172.21.15.53:6838/1044753024 2023-11-16T16:48:24.979 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.969+0000 7f7d58ff9700 1 -- 172.21.15.1:0/2679525401 <== mds.1 v2:172.21.15.53:6838/1044753024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c893d14660 con 0x55c893e62c60 2023-11-16T16:48:24.979 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.969+0000 7f7d58ff9700 10 client.16824 handle_command_reply: tid=0 2023-11-16T16:48:24.979 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.969+0000 7f7d83630b80 1 client.16824 shutdown 2023-11-16T16:48:24.979 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.969+0000 7f7d83630b80 1 -- 172.21.15.1:0/2679525401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c893df4b60 msgr2=0x55c893e03990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:24.979 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.969+0000 7f7d83630b80 1 --2- 172.21.15.1:0/2679525401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c893df4b60 0x55c893e03990 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f7d60011850 tx=0x7f7d60031d20 comp rx=0 tx=0).stop 2023-11-16T16:48:24.980 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.969+0000 7f7d83630b80 1 -- 172.21.15.1:0/2679525401 shutdown_connections 2023-11-16T16:48:24.980 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.969+0000 7f7d83630b80 1 --2- 172.21.15.1:0/2679525401 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c893df98b0 0x55c893e04410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.980 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.969+0000 7f7d83630b80 1 --2- 172.21.15.1:0/2679525401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c893df4b60 0x55c893e03990 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.980 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.969+0000 7f7d83630b80 1 --2- 172.21.15.1:0/2679525401 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x55c893e62c60 0x55c893e65070 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.980 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.969+0000 7f7d83630b80 1 --2- 172.21.15.1:0/2679525401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c893df5470 0x55c893e03ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.980 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.969+0000 7f7d58ff9700 0 client.16824 ms_handle_reset on v2:172.21.15.53:6838/1044753024 2023-11-16T16:48:24.980 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.969+0000 7f7d83630b80 1 -- 172.21.15.1:0/2679525401 >> 172.21.15.1:0/2679525401 conn(0x55c893dea360 msgr2=0x55c893df78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:24.980 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.969+0000 7f7d83630b80 1 -- 172.21.15.1:0/2679525401 shutdown_connections 2023-11-16T16:48:24.981 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.969+0000 7f7d83630b80 1 -- 172.21.15.1:0/2679525401 wait complete. 2023-11-16T16:48:24.981 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.969+0000 7f7d83630b80 20 client.16824 trim_cache size 0 max 16384 2023-11-16T16:48:24.981 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2023-11-16T16:48:24.982 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.969+0000 7f7d774a4700 1 -- 172.21.15.1:0/328154764 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f7d5c0a12c0 msgr2=0x7f7d5c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:24.982 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.969+0000 7f7d774a4700 1 --2- 172.21.15.1:0/328154764 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f7d5c0a12c0 0x7f7d5c0a3770 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7d60009be0 tx=0x7f7d60009260 comp rx=0 tx=0).stop 2023-11-16T16:48:24.982 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.969+0000 7f7d774a4700 1 -- 172.21.15.1:0/328154764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d70104140 msgr2=0x7f7d70198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:24.982 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.969+0000 7f7d774a4700 1 --2- 172.21.15.1:0/328154764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d70104140 0x7f7d70198500 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f7d68007860 tx=0x7f7d68007890 comp rx=0 tx=0).stop 2023-11-16T16:48:24.982 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.969+0000 7f7d774a4700 1 -- 172.21.15.1:0/328154764 shutdown_connections 2023-11-16T16:48:24.983 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.969+0000 7f7d774a4700 1 --2- 172.21.15.1:0/328154764 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f7d5c0a12c0 0x7f7d5c0a3770 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.983 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.969+0000 7f7d774a4700 1 --2- 172.21.15.1:0/328154764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d70104140 0x7f7d70198500 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.983 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.969+0000 7f7d774a4700 1 --2- 172.21.15.1:0/328154764 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7d700ff8f0 0x7f7d7019e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.983 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.969+0000 7f7d774a4700 1 --2- 172.21.15.1:0/328154764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d701002a0 0x7f7d7019e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:24.983 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.969+0000 7f7d774a4700 1 -- 172.21.15.1:0/328154764 >> 172.21.15.1:0/328154764 conn(0x7f7d70075880 msgr2=0x7f7d700fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:24.983 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.970+0000 7f7d774a4700 1 -- 172.21.15.1:0/328154764 shutdown_connections 2023-11-16T16:48:24.983 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:24.970+0000 7f7d774a4700 1 -- 172.21.15.1:0/328154764 wait complete. 2023-11-16T16:48:24.984 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-16T16:48:25.111 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.110+0000 7f9fdacc9700 1 -- 172.21.15.1:0/3442836363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fd4105310 msgr2=0x7f9fd41056e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:25.112 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.110+0000 7f9fdacc9700 1 --2- 172.21.15.1:0/3442836363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fd4105310 0x7f9fd41056e0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8009990 tx=0x7f9fc8009ca0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.112 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.110+0000 7f9fdacc9700 1 -- 172.21.15.1:0/3442836363 shutdown_connections 2023-11-16T16:48:25.112 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.110+0000 7f9fdacc9700 1 --2- 172.21.15.1:0/3442836363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fd4100370 0x7f9fd410bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.112 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.110+0000 7f9fdacc9700 1 --2- 172.21.15.1:0/3442836363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fd4105310 0x7f9fd41056e0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.112 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.110+0000 7f9fdacc9700 1 --2- 172.21.15.1:0/3442836363 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fd40ff9c0 0x7f9fd40ffe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.112 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.110+0000 7f9fdacc9700 1 -- 172.21.15.1:0/3442836363 >> 172.21.15.1:0/3442836363 conn(0x7f9fd4075470 msgr2=0x7f9fd4075870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:25.112 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.110+0000 7f9fdacc9700 1 -- 172.21.15.1:0/3442836363 shutdown_connections 2023-11-16T16:48:25.113 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.110+0000 7f9fdacc9700 1 -- 172.21.15.1:0/3442836363 wait complete. 2023-11-16T16:48:25.115 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.110+0000 7f9fdacc9700 1 Processor -- start 2023-11-16T16:48:25.115 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.110+0000 7f9fdacc9700 1 -- start start 2023-11-16T16:48:25.115 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.111+0000 7f9fdacc9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fd40ff9c0 0x7f9fd41972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:25.115 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.111+0000 7f9fdacc9700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fd4100370 0x7f9fd4197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:25.115 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.111+0000 7f9fdacc9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fd4105310 0x7f9fd41a2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:25.115 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.111+0000 7f9fdacc9700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd41980d0 con 0x7f9fd40ff9c0 2023-11-16T16:48:25.115 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.111+0000 7f9fdacc9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd41a2890 con 0x7f9fd4105310 2023-11-16T16:48:25.116 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.111+0000 7f9fdacc9700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd41a29d0 con 0x7f9fd4100370 2023-11-16T16:48:25.116 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.111+0000 7f9fd8a65700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fd40ff9c0 0x7f9fd41972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:25.116 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.111+0000 7f9fd8a65700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fd40ff9c0 0x7f9fd41972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46808/0 (socket says 172.21.15.1:46808) 2023-11-16T16:48:25.116 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.111+0000 7f9fd8a65700 1 -- 172.21.15.1:0/3402609164 learned_addr learned my addr 172.21.15.1:0/3402609164 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:25.116 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.111+0000 7f9fd3fff700 1 --2- 172.21.15.1:0/3402609164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fd4100370 0x7f9fd4197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:25.116 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.111+0000 7f9fd9266700 1 --2- 172.21.15.1:0/3402609164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fd4105310 0x7f9fd41a2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:25.116 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.111+0000 7f9fd8a65700 1 -- 172.21.15.1:0/3402609164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fd4100370 msgr2=0x7f9fd4197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:25.116 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.111+0000 7f9fd8a65700 1 --2- 172.21.15.1:0/3402609164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fd4100370 0x7f9fd4197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.117 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.111+0000 7f9fd8a65700 1 -- 172.21.15.1:0/3402609164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fd4105310 msgr2=0x7f9fd41a2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:25.117 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.111+0000 7f9fd8a65700 1 --2- 172.21.15.1:0/3402609164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fd4105310 0x7f9fd41a2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.117 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.111+0000 7f9fd8a65700 1 -- 172.21.15.1:0/3402609164 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc80095f0 con 0x7f9fd40ff9c0 2023-11-16T16:48:25.117 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.111+0000 7f9fd3fff700 1 --2- 172.21.15.1:0/3402609164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fd4100370 0x7f9fd4197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:48:25.117 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.111+0000 7f9fd8a65700 1 --2- 172.21.15.1:0/3402609164 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fd40ff9c0 0x7f9fd41972d0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f9fc80095c0 tx=0x7f9fc800f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:25.117 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.111+0000 7f9fd1ffb700 1 -- 172.21.15.1:0/3402609164 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc801d070 con 0x7f9fd40ff9c0 2023-11-16T16:48:25.117 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.112+0000 7f9fd1ffb700 1 -- 172.21.15.1:0/3402609164 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fc800fc00 con 0x7f9fd40ff9c0 2023-11-16T16:48:25.117 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.112+0000 7f9fd1ffb700 1 -- 172.21.15.1:0/3402609164 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc80173f0 con 0x7f9fd40ff9c0 2023-11-16T16:48:25.118 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.112+0000 7f9fdacc9700 1 -- 172.21.15.1:0/3402609164 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fd410f2f0 con 0x7f9fd40ff9c0 2023-11-16T16:48:25.118 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.112+0000 7f9fdacc9700 1 -- 172.21.15.1:0/3402609164 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fd410f160 con 0x7f9fd40ff9c0 2023-11-16T16:48:25.124 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.114+0000 7f9fd1ffb700 1 -- 172.21.15.1:0/3402609164 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f9fc8022470 con 0x7f9fd40ff9c0 2023-11-16T16:48:25.124 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.114+0000 7f9fd1ffb700 1 --2- 172.21.15.1:0/3402609164 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f9fc40a1340 0x7f9fc40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:25.124 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.115+0000 7f9fd1ffb700 1 -- 172.21.15.1:0/3402609164 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f9fc80cdf10 con 0x7f9fd40ff9c0 2023-11-16T16:48:25.124 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.115+0000 7f9fd3fff700 1 --2- 172.21.15.1:0/3402609164 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f9fc40a1340 0x7f9fc40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:25.124 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.115+0000 7f9fd3fff700 1 --2- 172.21.15.1:0/3402609164 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f9fc40a1340 0x7f9fc40a37f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0005950 tx=0x7f9fc00058e0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:25.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.115+0000 7f9fe6e55b80 1 Processor -- start 2023-11-16T16:48:25.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.115+0000 7f9fe6e55b80 1 -- start start 2023-11-16T16:48:25.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.116+0000 7f9fe6e55b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c91bb2cb60 0x55c91bb2cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:25.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.116+0000 7f9fe6e55b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c91bb2d470 0x55c91bb31370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:25.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.116+0000 7f9fe6e55b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c91bb318b0 0x55c91bb31d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:25.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.116+0000 7f9fe6e55b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c91ba32060 con 0x55c91bb2cb60 2023-11-16T16:48:25.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.116+0000 7f9fe6e55b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c91ba848a0 con 0x55c91bb318b0 2023-11-16T16:48:25.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.116+0000 7f9fe6e55b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c91ba7dd30 con 0x55c91bb2d470 2023-11-16T16:48:25.126 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.116+0000 7f9fd8a65700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c91bb2cb60 0x55c91bb2cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:25.126 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.116+0000 7f9fd8a65700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c91bb2cb60 0x55c91bb2cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46816/0 (socket says 172.21.15.1:46816) 2023-11-16T16:48:25.126 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.116+0000 7f9fd8a65700 1 -- 172.21.15.1:0/2415172090 learned_addr learned my addr 172.21.15.1:0/2415172090 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:25.126 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.116+0000 7f9fd8a65700 1 -- 172.21.15.1:0/2415172090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c91bb2d470 msgr2=0x55c91bb31370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:25.126 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.116+0000 7f9fd8a65700 1 --2- 172.21.15.1:0/2415172090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c91bb2d470 0x55c91bb31370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.126 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.116+0000 7f9fd9266700 1 --2- 172.21.15.1:0/2415172090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c91bb318b0 0x55c91bb31d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:25.126 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.116+0000 7f9fd8a65700 1 -- 172.21.15.1:0/2415172090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c91bb318b0 msgr2=0x55c91bb31d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:25.127 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.116+0000 7f9fd8a65700 1 --2- 172.21.15.1:0/2415172090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c91bb318b0 0x55c91bb31d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.127 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.116+0000 7f9fd8a65700 1 -- 172.21.15.1:0/2415172090 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c91b8ff630 con 0x55c91bb2cb60 2023-11-16T16:48:25.127 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.116+0000 7f9fd9266700 1 --2- 172.21.15.1:0/2415172090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c91bb318b0 0x55c91bb31d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:48:25.127 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.116+0000 7f9fd8a65700 1 --2- 172.21.15.1:0/2415172090 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c91bb2cb60 0x55c91bb2cf30 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f9fc80a2ef0 tx=0x7f9fc8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e9704a91f0a7687 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:25.127 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.116+0000 7f9fbeffd700 1 -- 172.21.15.1:0/2415172090 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc809f090 con 0x55c91bb2cb60 2023-11-16T16:48:25.127 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.116+0000 7f9fbeffd700 1 -- 172.21.15.1:0/2415172090 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fc80a3420 con 0x55c91bb2cb60 2023-11-16T16:48:25.127 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.116+0000 7f9fbeffd700 1 -- 172.21.15.1:0/2415172090 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc809a550 con 0x55c91bb2cb60 2023-11-16T16:48:25.127 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.117+0000 7f9fe6e55b80 1 -- 172.21.15.1:0/2415172090 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c91bb2cb60 msgr2=0x55c91bb2cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:25.127 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.117+0000 7f9fe6e55b80 1 --2- 172.21.15.1:0/2415172090 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c91bb2cb60 0x55c91bb2cf30 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f9fc80a2ef0 tx=0x7f9fc8009ca0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.128 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.117+0000 7f9fe6e55b80 1 -- 172.21.15.1:0/2415172090 shutdown_connections 2023-11-16T16:48:25.128 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.117+0000 7f9fe6e55b80 1 --2- 172.21.15.1:0/2415172090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c91bb318b0 0x55c91bb31d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.128 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.117+0000 7f9fe6e55b80 1 --2- 172.21.15.1:0/2415172090 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c91bb2cb60 0x55c91bb2cf30 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.128 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.117+0000 7f9fe6e55b80 1 --2- 172.21.15.1:0/2415172090 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c91bb2d470 0x55c91bb31370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.128 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.117+0000 7f9fe6e55b80 1 -- 172.21.15.1:0/2415172090 >> 172.21.15.1:0/2415172090 conn(0x55c91bb22360 msgr2=0x55c91bb27b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:25.128 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.117+0000 7f9fe6e55b80 1 -- 172.21.15.1:0/2415172090 shutdown_connections 2023-11-16T16:48:25.128 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.117+0000 7f9fe6e55b80 1 -- 172.21.15.1:0/2415172090 wait complete. 2023-11-16T16:48:25.128 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.117+0000 7f9fe6e55b80 1 -- start start 2023-11-16T16:48:25.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.117+0000 7f9fe6e55b80 1 Processor -- start 2023-11-16T16:48:25.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.118+0000 7f9fe6e55b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c91bb2cb60 0x55c91bb38fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:25.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.118+0000 7f9fe6e55b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c91bb2d470 0x55c91bb36220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:25.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.118+0000 7f9fe6e55b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c91bb318b0 0x55c91bb36800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:25.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.118+0000 7f9fe6e55b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c91ba1f8e0 con 0x55c91bb318b0 2023-11-16T16:48:25.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.118+0000 7f9fe6e55b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c91ba1f580 con 0x55c91bb2cb60 2023-11-16T16:48:25.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.118+0000 7f9fe6e55b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c91b901920 con 0x55c91bb2d470 2023-11-16T16:48:25.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.118+0000 7f9fe6e55b80 10 client.0 _ll_register_callbacks cb 0x55c91ba3d780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:48:25.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.118+0000 7f9fd9266700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c91bb318b0 0x55c91bb36800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:25.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.118+0000 7f9fd9266700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c91bb318b0 0x55c91bb36800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46826/0 (socket says 172.21.15.1:46826) 2023-11-16T16:48:25.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.118+0000 7f9fd9266700 1 -- 172.21.15.1:0/2048883704 learned_addr learned my addr 172.21.15.1:0/2048883704 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:25.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.118+0000 7f9fd8a65700 1 --2- 172.21.15.1:0/2048883704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c91bb2cb60 0x55c91bb38fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:25.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.118+0000 7f9fd3fff700 1 --2- 172.21.15.1:0/2048883704 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c91bb2d470 0x55c91bb36220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:25.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.118+0000 7f9fd9266700 1 -- 172.21.15.1:0/2048883704 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c91bb2d470 msgr2=0x55c91bb36220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:25.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.118+0000 7f9fd9266700 1 --2- 172.21.15.1:0/2048883704 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c91bb2d470 0x55c91bb36220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.118+0000 7f9fd9266700 1 -- 172.21.15.1:0/2048883704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c91bb2cb60 msgr2=0x55c91bb38fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:25.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.118+0000 7f9fd9266700 1 --2- 172.21.15.1:0/2048883704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c91bb2cb60 0x55c91bb38fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.118+0000 7f9fd9266700 1 -- 172.21.15.1:0/2048883704 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fd410f160 con 0x55c91bb318b0 2023-11-16T16:48:25.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.118+0000 7f9fd8a65700 1 --2- 172.21.15.1:0/2048883704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c91bb2cb60 0x55c91bb38fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:48:25.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.119+0000 7f9fd9266700 1 --2- 172.21.15.1:0/2048883704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c91bb318b0 0x55c91bb36800 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc007ba0 tx=0x7f9fcc007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f0bb6bfd823e9a server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:25.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.119+0000 7f9fb3fff700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:48:25.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.119+0000 7f9fb3fff700 1 -- 172.21.15.1:0/2048883704 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc010830 con 0x55c91bb318b0 2023-11-16T16:48:25.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.119+0000 7f9fb3fff700 1 -- 172.21.15.1:0/2048883704 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f9fcc010990 con 0x55c91bb318b0 2023-11-16T16:48:25.131 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.119+0000 7f9fe6e55b80 1 -- 172.21.15.1:0/2048883704 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c91bb1fb20 con 0x55c91bb318b0 2023-11-16T16:48:25.132 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.119+0000 7f9fb3fff700 1 -- 172.21.15.1:0/2048883704 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fcc010b50 con 0x55c91bb318b0 2023-11-16T16:48:25.132 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.119+0000 7f9fe6e55b80 10 client.16836 fetch_fsmap learned FSMap version 29 2023-11-16T16:48:25.132 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.119+0000 7f9fe6e55b80 1 -- 172.21.15.1:0/2048883704 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c91ba44190 con 0x55c91bb318b0 2023-11-16T16:48:25.132 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.119+0000 7f9fb3fff700 1 -- 172.21.15.1:0/2048883704 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c91bb1fb20 con 0x55c91bb318b0 2023-11-16T16:48:25.132 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.119+0000 7f9fb3fff700 1 -- 172.21.15.1:0/2048883704 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 29) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f9fcc01d3a0 con 0x55c91bb318b0 2023-11-16T16:48:25.132 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.119+0000 7f9fe6e55b80 10 client.16836 fetch_fsmap finished waiting for FSMap version 29 2023-11-16T16:48:25.132 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.119+0000 7f9fe6e55b80 10 client.16836 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:48:25.132 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.119+0000 7f9fe6e55b80 20 client.16836 populate_metadata read hostname 'smithi001' 2023-11-16T16:48:25.133 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.119+0000 7f9fe6e55b80 1 --2- 172.21.15.1:0/2048883704 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55c91bb46690 0x55c91bb48aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:25.133 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.119+0000 7f9fe6e55b80 4 client.16836 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:48:25.133 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.119+0000 7f9fe6e55b80 1 -- 172.21.15.1:0/2048883704 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c91ba400a0 con 0x55c91bb46690 2023-11-16T16:48:25.133 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.120+0000 7f9fd8a65700 1 --2- 172.21.15.1:0/2048883704 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55c91bb46690 0x55c91bb48aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:25.133 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.120+0000 7f9fb3fff700 10 client.16836 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:48:25.133 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.120+0000 7f9fd8a65700 1 --2- 172.21.15.1:0/2048883704 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55c91bb46690 0x55c91bb48aa0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51c854cea74f89ad server_cookie=f7b1397b49a8a068 in_seq=0 out_seq=0 2023-11-16T16:48:25.133 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.121+0000 7f9fb3fff700 1 -- 172.21.15.1:0/2048883704 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c91ba400a0 con 0x55c91bb46690 2023-11-16T16:48:25.134 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.121+0000 7f9fb3fff700 10 client.16836 handle_command_reply: tid=0 2023-11-16T16:48:25.134 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.121+0000 7f9fe6e55b80 1 client.16836 shutdown 2023-11-16T16:48:25.134 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.121+0000 7f9fe6e55b80 1 -- 172.21.15.1:0/2048883704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c91bb318b0 msgr2=0x55c91bb36800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:25.134 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.121+0000 7f9fe6e55b80 1 --2- 172.21.15.1:0/2048883704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c91bb318b0 0x55c91bb36800 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc007ba0 tx=0x7f9fcc007f60 comp rx=0 tx=0).stop 2023-11-16T16:48:25.134 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.121+0000 7f9fe6e55b80 1 -- 172.21.15.1:0/2048883704 shutdown_connections 2023-11-16T16:48:25.134 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.121+0000 7f9fe6e55b80 1 --2- 172.21.15.1:0/2048883704 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55c91bb46690 0x55c91bb48aa0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.134 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.121+0000 7f9fe6e55b80 1 --2- 172.21.15.1:0/2048883704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c91bb318b0 0x55c91bb36800 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.134 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.121+0000 7f9fb3fff700 0 client.16836 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:48:25.135 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.121+0000 7f9fe6e55b80 1 --2- 172.21.15.1:0/2048883704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c91bb2cb60 0x55c91bb38fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.135 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.121+0000 7f9fe6e55b80 1 --2- 172.21.15.1:0/2048883704 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c91bb2d470 0x55c91bb36220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.135 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.121+0000 7f9fe6e55b80 1 -- 172.21.15.1:0/2048883704 >> 172.21.15.1:0/2048883704 conn(0x55c91bb22360 msgr2=0x55c91bb32d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:25.135 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.121+0000 7f9fe6e55b80 1 -- 172.21.15.1:0/2048883704 shutdown_connections 2023-11-16T16:48:25.135 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.121+0000 7f9fe6e55b80 1 -- 172.21.15.1:0/2048883704 wait complete. 2023-11-16T16:48:25.135 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.121+0000 7f9fe6e55b80 20 client.16836 trim_cache size 0 max 16384 2023-11-16T16:48:25.141 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.134+0000 7f9fe6e55b80 1 Processor -- start 2023-11-16T16:48:25.141 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.134+0000 7f9fe6e55b80 1 -- start start 2023-11-16T16:48:25.141 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.134+0000 7f9fe6e55b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c91bb2cb60 0x55c91bb3d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:25.141 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.134+0000 7f9fe6e55b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c91bb2d470 0x55c91bb2f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:25.141 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.134+0000 7f9fe6e55b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c91bb318b0 0x55c91bb276f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:25.141 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.134+0000 7f9fe6e55b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c91bb29280 con 0x55c91bb2cb60 2023-11-16T16:48:25.142 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.134+0000 7f9fe6e55b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c91bb21ee0 con 0x55c91bb2d470 2023-11-16T16:48:25.142 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.134+0000 7f9fe6e55b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c91bb36de0 con 0x55c91bb318b0 2023-11-16T16:48:25.142 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.134+0000 7f9fd8a65700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c91bb2cb60 0x55c91bb3d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:25.142 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.134+0000 7f9fd8a65700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c91bb2cb60 0x55c91bb3d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46830/0 (socket says 172.21.15.1:46830) 2023-11-16T16:48:25.142 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.134+0000 7f9fd8a65700 1 -- 172.21.15.1:0/25417264 learned_addr learned my addr 172.21.15.1:0/25417264 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:25.142 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.134+0000 7f9fd3fff700 1 --2- 172.21.15.1:0/25417264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c91bb2d470 0x55c91bb2f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:25.142 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.135+0000 7f9fd9266700 1 --2- 172.21.15.1:0/25417264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c91bb318b0 0x55c91bb276f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:25.142 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.135+0000 7f9fd8a65700 1 -- 172.21.15.1:0/25417264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c91bb318b0 msgr2=0x55c91bb276f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:25.143 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.135+0000 7f9fd8a65700 1 --2- 172.21.15.1:0/25417264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c91bb318b0 0x55c91bb276f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.143 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.135+0000 7f9fd8a65700 1 -- 172.21.15.1:0/25417264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c91bb2d470 msgr2=0x55c91bb2f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:25.143 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.135+0000 7f9fd8a65700 1 --2- 172.21.15.1:0/25417264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c91bb2d470 0x55c91bb2f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.143 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.135+0000 7f9fd8a65700 1 -- 172.21.15.1:0/25417264 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c91bb321a0 con 0x55c91bb2cb60 2023-11-16T16:48:25.143 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.135+0000 7f9fd8a65700 1 --2- 172.21.15.1:0/25417264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c91bb2cb60 0x55c91bb3d7b0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f9fc80993b0 tx=0x7f9fc809acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1987039df7e14182 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:25.143 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.135+0000 7f9fbf7fe700 1 -- 172.21.15.1:0/25417264 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc800fe70 con 0x55c91bb2cb60 2023-11-16T16:48:25.143 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.135+0000 7f9fbf7fe700 1 -- 172.21.15.1:0/25417264 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fc8022720 con 0x55c91bb2cb60 2023-11-16T16:48:25.143 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.135+0000 7f9fbf7fe700 1 -- 172.21.15.1:0/25417264 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc809b2c0 con 0x55c91bb2cb60 2023-11-16T16:48:25.144 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.135+0000 7f9fe6e55b80 1 -- 172.21.15.1:0/25417264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c91bb2cb60 msgr2=0x55c91bb3d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:25.144 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.135+0000 7f9fe6e55b80 1 --2- 172.21.15.1:0/25417264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c91bb2cb60 0x55c91bb3d7b0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f9fc80993b0 tx=0x7f9fc809acd0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.144 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.136+0000 7f9fe6e55b80 1 -- 172.21.15.1:0/25417264 shutdown_connections 2023-11-16T16:48:25.144 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.136+0000 7f9fe6e55b80 1 --2- 172.21.15.1:0/25417264 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c91bb318b0 0x55c91bb276f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.144 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.136+0000 7f9fe6e55b80 1 --2- 172.21.15.1:0/25417264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c91bb2cb60 0x55c91bb3d7b0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.144 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.136+0000 7f9fe6e55b80 1 --2- 172.21.15.1:0/25417264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c91bb2d470 0x55c91bb2f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.144 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.136+0000 7f9fe6e55b80 1 -- 172.21.15.1:0/25417264 >> 172.21.15.1:0/25417264 conn(0x55c91bb22360 msgr2=0x55c91bb428b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:25.144 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.136+0000 7f9fe6e55b80 1 -- 172.21.15.1:0/25417264 shutdown_connections 2023-11-16T16:48:25.145 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.136+0000 7f9fe6e55b80 1 -- 172.21.15.1:0/25417264 wait complete. 2023-11-16T16:48:25.145 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.136+0000 7f9fe6e55b80 1 -- start start 2023-11-16T16:48:25.145 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.136+0000 7f9fe6e55b80 1 Processor -- start 2023-11-16T16:48:25.145 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.136+0000 7f9fe6e55b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c91bb2cb60 0x55c91bb3b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:25.145 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.136+0000 7f9fe6e55b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c91bb2d470 0x55c91bb3bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:25.145 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.136+0000 7f9fe6e55b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c91bb318b0 0x55c91bb3c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:25.145 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.136+0000 7f9fe6e55b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c91bb99580 con 0x55c91bb2d470 2023-11-16T16:48:25.145 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.136+0000 7f9fe6e55b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c91bb996f0 con 0x55c91bb2cb60 2023-11-16T16:48:25.146 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.136+0000 7f9fe6e55b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c91bb998b0 con 0x55c91bb318b0 2023-11-16T16:48:25.146 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.136+0000 7f9fe6e55b80 10 client.0 _ll_register_callbacks cb 0x55c91ba3d780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:48:25.146 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.136+0000 7f9fd3fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c91bb2d470 0x55c91bb3bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:25.146 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.137+0000 7f9fd3fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c91bb2d470 0x55c91bb3bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46838/0 (socket says 172.21.15.1:46838) 2023-11-16T16:48:25.146 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.137+0000 7f9fd3fff700 1 -- 172.21.15.1:0/418499883 learned_addr learned my addr 172.21.15.1:0/418499883 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:25.146 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.137+0000 7f9fd9266700 1 --2- 172.21.15.1:0/418499883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c91bb318b0 0x55c91bb3c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:25.146 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.137+0000 7f9fd8a65700 1 --2- 172.21.15.1:0/418499883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c91bb2cb60 0x55c91bb3b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:25.146 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.137+0000 7f9fd3fff700 1 -- 172.21.15.1:0/418499883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c91bb318b0 msgr2=0x55c91bb3c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:25.147 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.137+0000 7f9fd3fff700 1 --2- 172.21.15.1:0/418499883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c91bb318b0 0x55c91bb3c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.147 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.137+0000 7f9fd3fff700 1 -- 172.21.15.1:0/418499883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c91bb2cb60 msgr2=0x55c91bb3b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:25.147 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.137+0000 7f9fd3fff700 1 --2- 172.21.15.1:0/418499883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c91bb2cb60 0x55c91bb3b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.147 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.137+0000 7f9fd3fff700 1 -- 172.21.15.1:0/418499883 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c91b8ff630 con 0x55c91bb2d470 2023-11-16T16:48:25.147 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.137+0000 7f9fd8a65700 1 --2- 172.21.15.1:0/418499883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c91bb2cb60 0x55c91bb3b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:48:25.147 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.137+0000 7f9fd3fff700 1 --2- 172.21.15.1:0/418499883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c91bb2d470 0x55c91bb3bce0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4069930 tx=0x7f9fc000ffd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6fa995ee0ef5f77 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:25.147 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.137+0000 7f9fb3fff700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:48:25.147 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.137+0000 7f9fb3fff700 1 -- 172.21.15.1:0/418499883 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0017040 con 0x55c91bb2d470 2023-11-16T16:48:25.147 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.137+0000 7f9fe6e55b80 1 -- 172.21.15.1:0/418499883 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9fc80a3420 con 0x55c91bb2d470 2023-11-16T16:48:25.148 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.137+0000 7f9fb3fff700 1 -- 172.21.15.1:0/418499883 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f9fc0003c90 con 0x55c91bb2d470 2023-11-16T16:48:25.148 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.137+0000 7f9fb3fff700 1 -- 172.21.15.1:0/418499883 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fc0003e50 con 0x55c91bb2d470 2023-11-16T16:48:25.148 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.137+0000 7f9fe6e55b80 10 client.16842 fetch_fsmap learned FSMap version 29 2023-11-16T16:48:25.148 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.137+0000 7f9fe6e55b80 1 -- 172.21.15.1:0/418499883 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c91b8abde0 con 0x55c91bb2d470 2023-11-16T16:48:25.148 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.137+0000 7f9fb3fff700 1 -- 172.21.15.1:0/418499883 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fc80a3420 con 0x55c91bb2d470 2023-11-16T16:48:25.148 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.138+0000 7f9fb3fff700 1 -- 172.21.15.1:0/418499883 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 29) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f9fc001d070 con 0x55c91bb2d470 2023-11-16T16:48:25.148 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.138+0000 7f9fe6e55b80 10 client.16842 fetch_fsmap finished waiting for FSMap version 29 2023-11-16T16:48:25.148 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.138+0000 7f9fe6e55b80 10 client.16842 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:48:25.149 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.138+0000 7f9fe6e55b80 20 client.16842 populate_metadata read hostname 'smithi001' 2023-11-16T16:48:25.149 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.138+0000 7f9fe6e55b80 1 --2- 172.21.15.1:0/418499883 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55c91bb9b180 0x55c91bb9d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:25.149 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.138+0000 7f9fe6e55b80 4 client.16842 mds_command: new command op to 24521 tid=0[{"prefix": "damage ls"}] 2023-11-16T16:48:25.149 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.138+0000 7f9fe6e55b80 1 -- 172.21.15.1:0/418499883 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c91ba4c660 con 0x55c91bb9b180 2023-11-16T16:48:25.149 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.138+0000 7f9fd8a65700 1 --2- 172.21.15.1:0/418499883 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55c91bb9b180 0x55c91bb9d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:25.149 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.138+0000 7f9fd8a65700 1 --2- 172.21.15.1:0/418499883 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55c91bb9b180 0x55c91bb9d590 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7377f38dc24fcfbc server_cookie=7e3bc351923b356f in_seq=0 out_seq=0 2023-11-16T16:48:25.149 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.138+0000 7f9fb3fff700 10 client.16842 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:48:25.149 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.139+0000 7f9fb3fff700 1 -- 172.21.15.1:0/418499883 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c91ba4c660 con 0x55c91bb9b180 2023-11-16T16:48:25.150 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.139+0000 7f9fb3fff700 10 client.16842 handle_command_reply: tid=0 2023-11-16T16:48:25.150 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.139+0000 7f9fe6e55b80 1 client.16842 shutdown 2023-11-16T16:48:25.150 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.139+0000 7f9fe6e55b80 1 -- 172.21.15.1:0/418499883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c91bb2d470 msgr2=0x55c91bb3bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:25.150 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.139+0000 7f9fe6e55b80 1 --2- 172.21.15.1:0/418499883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c91bb2d470 0x55c91bb3bce0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4069930 tx=0x7f9fc000ffd0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.150 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.139+0000 7f9fe6e55b80 1 -- 172.21.15.1:0/418499883 shutdown_connections 2023-11-16T16:48:25.150 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.139+0000 7f9fe6e55b80 1 --2- 172.21.15.1:0/418499883 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55c91bb9b180 0x55c91bb9d590 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.150 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.139+0000 7f9fe6e55b80 1 --2- 172.21.15.1:0/418499883 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c91bb318b0 0x55c91bb3c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.150 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.139+0000 7f9fb3fff700 0 client.16842 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:48:25.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.139+0000 7f9fe6e55b80 1 --2- 172.21.15.1:0/418499883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c91bb2cb60 0x55c91bb3b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.139+0000 7f9fe6e55b80 1 --2- 172.21.15.1:0/418499883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c91bb2d470 0x55c91bb3bce0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.139+0000 7f9fe6e55b80 1 -- 172.21.15.1:0/418499883 >> 172.21.15.1:0/418499883 conn(0x55c91bb22360 msgr2=0x55c91bb2f930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:25.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.139+0000 7f9fe6e55b80 1 -- 172.21.15.1:0/418499883 shutdown_connections 2023-11-16T16:48:25.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.139+0000 7f9fe6e55b80 1 -- 172.21.15.1:0/418499883 wait complete. 2023-11-16T16:48:25.151 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.139+0000 7f9fe6e55b80 20 client.16842 trim_cache size 0 max 16384 2023-11-16T16:48:25.151 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2023-11-16T16:48:25.152 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.140+0000 7f9fdacc9700 1 -- 172.21.15.1:0/3402609164 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f9fc40a1340 msgr2=0x7f9fc40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:25.152 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.140+0000 7f9fdacc9700 1 --2- 172.21.15.1:0/3402609164 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f9fc40a1340 0x7f9fc40a37f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0005950 tx=0x7f9fc00058e0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.152 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.140+0000 7f9fdacc9700 1 -- 172.21.15.1:0/3402609164 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fd40ff9c0 msgr2=0x7f9fd41972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:25.153 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.140+0000 7f9fdacc9700 1 --2- 172.21.15.1:0/3402609164 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fd40ff9c0 0x7f9fd41972d0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f9fc80095c0 tx=0x7f9fc800f710 comp rx=0 tx=0).stop 2023-11-16T16:48:25.153 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.140+0000 7f9fdacc9700 1 -- 172.21.15.1:0/3402609164 shutdown_connections 2023-11-16T16:48:25.153 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.140+0000 7f9fdacc9700 1 --2- 172.21.15.1:0/3402609164 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f9fc40a1340 0x7f9fc40a37f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.153 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.140+0000 7f9fdacc9700 1 --2- 172.21.15.1:0/3402609164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fd4105310 0x7f9fd41a2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.153 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.140+0000 7f9fdacc9700 1 --2- 172.21.15.1:0/3402609164 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9fd40ff9c0 0x7f9fd41972d0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.153 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.140+0000 7f9fdacc9700 1 --2- 172.21.15.1:0/3402609164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f9fd4100370 0x7f9fd4197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:25.153 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.140+0000 7f9fdacc9700 1 -- 172.21.15.1:0/3402609164 >> 172.21.15.1:0/3402609164 conn(0x7f9fd4075470 msgr2=0x7f9fd40fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:25.153 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.140+0000 7f9fdacc9700 1 -- 172.21.15.1:0/3402609164 shutdown_connections 2023-11-16T16:48:25.154 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:25.140+0000 7f9fdacc9700 1 -- 172.21.15.1:0/3402609164 wait complete. 2023-11-16T16:48:25.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:25 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1708769408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:48:25.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:25 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1708769408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:48:25.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:25 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1708769408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:48:26.155 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-16T16:48:26.156 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-16T16:48:26.303 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.300+0000 7fbaa4503700 1 -- 172.21.15.1:0/78713020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba9c0ff490 msgr2=0x7fba9c0ff860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:26.303 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.300+0000 7fbaa4503700 1 --2- 172.21.15.1:0/78713020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba9c0ff490 0x7fba9c0ff860 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fba980099c0 tx=0x7fba98009cd0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.304 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.300+0000 7fbaa4503700 1 -- 172.21.15.1:0/78713020 shutdown_connections 2023-11-16T16:48:26.304 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.300+0000 7fbaa4503700 1 --2- 172.21.15.1:0/78713020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba9c104480 0x7fba9c10c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.304 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.300+0000 7fbaa4503700 1 --2- 172.21.15.1:0/78713020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba9c0ff490 0x7fba9c0ff860 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.304 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.300+0000 7fbaa4503700 1 --2- 172.21.15.1:0/78713020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba9c0ffe30 0x7fba9c103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.304 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.300+0000 7fbaa4503700 1 -- 172.21.15.1:0/78713020 >> 172.21.15.1:0/78713020 conn(0x7fba9c0763a0 msgr2=0x7fba9c0767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:26.304 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.300+0000 7fbaa4503700 1 -- 172.21.15.1:0/78713020 shutdown_connections 2023-11-16T16:48:26.304 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.300+0000 7fbaa4503700 1 -- 172.21.15.1:0/78713020 wait complete. 2023-11-16T16:48:26.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.300+0000 7fbaa4503700 1 Processor -- start 2023-11-16T16:48:26.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.300+0000 7fbaa4503700 1 -- start start 2023-11-16T16:48:26.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.301+0000 7fbaa4503700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba9c0ff490 0x7fba9c196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:26.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.301+0000 7fbaa4503700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba9c0ffe30 0x7fba9c196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:26.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.301+0000 7fbaa4503700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba9c104480 0x7fba9c19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:26.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.301+0000 7fbaa4503700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9c19b3f0 con 0x7fba9c0ffe30 2023-11-16T16:48:26.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.301+0000 7fbaa4503700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9c19b560 con 0x7fba9c0ff490 2023-11-16T16:48:26.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.301+0000 7fbaa4503700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9c19b720 con 0x7fba9c104480 2023-11-16T16:48:26.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.301+0000 7fbaa1a9e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba9c0ffe30 0x7fba9c196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:26.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.301+0000 7fbaa1a9e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba9c0ffe30 0x7fba9c196950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46852/0 (socket says 172.21.15.1:46852) 2023-11-16T16:48:26.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.301+0000 7fbaa1a9e700 1 -- 172.21.15.1:0/2753054444 learned_addr learned my addr 172.21.15.1:0/2753054444 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:26.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.301+0000 7fbaa229f700 1 --2- 172.21.15.1:0/2753054444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba9c0ff490 0x7fba9c196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:26.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.301+0000 7fbaa2aa0700 1 --2- 172.21.15.1:0/2753054444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba9c104480 0x7fba9c19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:26.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.301+0000 7fbaa1a9e700 1 -- 172.21.15.1:0/2753054444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba9c104480 msgr2=0x7fba9c19adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:26.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.301+0000 7fbaa1a9e700 1 --2- 172.21.15.1:0/2753054444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba9c104480 0x7fba9c19adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.307 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.301+0000 7fbaa1a9e700 1 -- 172.21.15.1:0/2753054444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba9c0ff490 msgr2=0x7fba9c196410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:26.307 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.301+0000 7fbaa1a9e700 1 --2- 172.21.15.1:0/2753054444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba9c0ff490 0x7fba9c196410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.307 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.301+0000 7fbaa1a9e700 1 -- 172.21.15.1:0/2753054444 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba980095f0 con 0x7fba9c0ffe30 2023-11-16T16:48:26.307 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.301+0000 7fbaa1a9e700 1 --2- 172.21.15.1:0/2753054444 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba9c0ffe30 0x7fba9c196950 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fba8c00d950 tx=0x7fba8c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:26.307 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.301+0000 7fba937fe700 1 -- 172.21.15.1:0/2753054444 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c00bb60 con 0x7fba9c0ffe30 2023-11-16T16:48:26.307 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.301+0000 7fba937fe700 1 -- 172.21.15.1:0/2753054444 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba8c00de30 con 0x7fba9c0ffe30 2023-11-16T16:48:26.307 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.301+0000 7fbaa4503700 1 -- 172.21.15.1:0/2753054444 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba9c10f290 con 0x7fba9c0ffe30 2023-11-16T16:48:26.308 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.302+0000 7fba937fe700 1 -- 172.21.15.1:0/2753054444 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c0055a0 con 0x7fba9c0ffe30 2023-11-16T16:48:26.308 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.302+0000 7fbaa4503700 1 -- 172.21.15.1:0/2753054444 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba9c10f100 con 0x7fba9c0ffe30 2023-11-16T16:48:26.314 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.304+0000 7fba937fe700 1 -- 172.21.15.1:0/2753054444 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fba8c004750 con 0x7fba9c0ffe30 2023-11-16T16:48:26.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.304+0000 7fba937fe700 1 --2- 172.21.15.1:0/2753054444 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fba880a12c0 0x7fba880a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:26.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.305+0000 7fbaa229f700 1 --2- 172.21.15.1:0/2753054444 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fba880a12c0 0x7fba880a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:26.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.305+0000 7fba937fe700 1 -- 172.21.15.1:0/2753054444 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fba8c0cc970 con 0x7fba9c0ffe30 2023-11-16T16:48:26.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.305+0000 7fbab068fb80 1 Processor -- start 2023-11-16T16:48:26.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.305+0000 7fbab068fb80 1 -- start start 2023-11-16T16:48:26.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.305+0000 7fbaa229f700 1 --2- 172.21.15.1:0/2753054444 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fba880a12c0 0x7fba880a3770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fba98009990 tx=0x7fba9801a040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:26.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.305+0000 7fbab068fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563782155be0 0x563782155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:26.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.305+0000 7fbab068fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637821564f0 0x56378215a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:26.316 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.305+0000 7fbab068fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56378215a930 0x56378215ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:26.316 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.305+0000 7fbab068fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56378205b0e0 con 0x563782155be0 2023-11-16T16:48:26.316 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.305+0000 7fbab068fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637820ad920 con 0x56378215a930 2023-11-16T16:48:26.316 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.305+0000 7fbab068fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637820a6db0 con 0x5637821564f0 2023-11-16T16:48:26.316 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.306+0000 7fbaa229f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563782155be0 0x563782155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:26.316 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.306+0000 7fbaa229f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563782155be0 0x563782155fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46858/0 (socket says 172.21.15.1:46858) 2023-11-16T16:48:26.316 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.306+0000 7fbaa229f700 1 -- 172.21.15.1:0/1898611352 learned_addr learned my addr 172.21.15.1:0/1898611352 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:26.316 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.306+0000 7fbaa1a9e700 1 --2- 172.21.15.1:0/1898611352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637821564f0 0x56378215a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:26.317 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.306+0000 7fbaa2aa0700 1 --2- 172.21.15.1:0/1898611352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56378215a930 0x56378215ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:26.317 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.306+0000 7fbaa229f700 1 -- 172.21.15.1:0/1898611352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637821564f0 msgr2=0x56378215a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:26.317 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.306+0000 7fbaa229f700 1 --2- 172.21.15.1:0/1898611352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637821564f0 0x56378215a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.317 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.306+0000 7fbaa229f700 1 -- 172.21.15.1:0/1898611352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56378215a930 msgr2=0x56378215ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:26.317 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.306+0000 7fbaa229f700 1 --2- 172.21.15.1:0/1898611352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56378215a930 0x56378215ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.317 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.306+0000 7fbaa229f700 1 -- 172.21.15.1:0/1898611352 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563781f286b0 con 0x563782155be0 2023-11-16T16:48:26.317 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.306+0000 7fbaa1a9e700 1 --2- 172.21.15.1:0/1898611352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637821564f0 0x56378215a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:48:26.317 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.306+0000 7fbaa229f700 1 --2- 172.21.15.1:0/1898611352 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563782155be0 0x563782155fb0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fba980099c0 tx=0x7fba9801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b913d954aa6a64 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:26.317 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.306+0000 7fba90ff9700 1 -- 172.21.15.1:0/1898611352 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba98031070 con 0x563782155be0 2023-11-16T16:48:26.318 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.306+0000 7fba90ff9700 1 -- 172.21.15.1:0/1898611352 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba98029040 con 0x563782155be0 2023-11-16T16:48:26.318 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.306+0000 7fba90ff9700 1 -- 172.21.15.1:0/1898611352 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9801d040 con 0x563782155be0 2023-11-16T16:48:26.318 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.307+0000 7fbab068fb80 1 -- 172.21.15.1:0/1898611352 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563782155be0 msgr2=0x563782155fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:26.318 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.307+0000 7fbab068fb80 1 --2- 172.21.15.1:0/1898611352 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563782155be0 0x563782155fb0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fba980099c0 tx=0x7fba9801e000 comp rx=0 tx=0).stop 2023-11-16T16:48:26.318 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.307+0000 7fbab068fb80 1 -- 172.21.15.1:0/1898611352 shutdown_connections 2023-11-16T16:48:26.318 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.307+0000 7fbab068fb80 1 --2- 172.21.15.1:0/1898611352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56378215a930 0x56378215ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.318 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.307+0000 7fbab068fb80 1 --2- 172.21.15.1:0/1898611352 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563782155be0 0x563782155fb0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.318 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.307+0000 7fbab068fb80 1 --2- 172.21.15.1:0/1898611352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5637821564f0 0x56378215a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.319 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.307+0000 7fbab068fb80 1 -- 172.21.15.1:0/1898611352 >> 172.21.15.1:0/1898611352 conn(0x56378214b3e0 msgr2=0x563782150bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:26.319 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.307+0000 7fbab068fb80 1 -- 172.21.15.1:0/1898611352 shutdown_connections 2023-11-16T16:48:26.319 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.307+0000 7fbab068fb80 1 -- 172.21.15.1:0/1898611352 wait complete. 2023-11-16T16:48:26.319 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.308+0000 7fbab068fb80 1 -- start start 2023-11-16T16:48:26.319 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.308+0000 7fbab068fb80 1 Processor -- start 2023-11-16T16:48:26.319 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.308+0000 7fbab068fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563782155be0 0x563782162140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:26.319 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.308+0000 7fbab068fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637821564f0 0x56378215f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:26.319 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.308+0000 7fbab068fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56378215a930 0x56378215fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:26.320 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.308+0000 7fbab068fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x563782048960 con 0x5637821564f0 2023-11-16T16:48:26.320 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.308+0000 7fbab068fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563782048600 con 0x56378215a930 2023-11-16T16:48:26.320 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.308+0000 7fbab068fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563781f2a9a0 con 0x563782155be0 2023-11-16T16:48:26.320 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.308+0000 7fbab068fb80 10 client.0 _ll_register_callbacks cb 0x563782066800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:48:26.320 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.308+0000 7fbaa1a9e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637821564f0 0x56378215f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:26.320 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.308+0000 7fbaa1a9e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637821564f0 0x56378215f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46866/0 (socket says 172.21.15.1:46866) 2023-11-16T16:48:26.320 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.308+0000 7fbaa1a9e700 1 -- 172.21.15.1:0/36566945 learned_addr learned my addr 172.21.15.1:0/36566945 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:26.320 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.308+0000 7fbaa229f700 1 --2- 172.21.15.1:0/36566945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563782155be0 0x563782162140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:26.320 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.308+0000 7fbaa2aa0700 1 --2- 172.21.15.1:0/36566945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56378215a930 0x56378215fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:26.321 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.309+0000 7fbaa1a9e700 1 -- 172.21.15.1:0/36566945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563782155be0 msgr2=0x563782162140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:26.321 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.309+0000 7fbaa1a9e700 1 --2- 172.21.15.1:0/36566945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563782155be0 0x563782162140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.321 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.309+0000 7fbaa1a9e700 1 -- 172.21.15.1:0/36566945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56378215a930 msgr2=0x56378215fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:26.321 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.309+0000 7fbaa1a9e700 1 --2- 172.21.15.1:0/36566945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56378215a930 0x56378215fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.321 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.309+0000 7fbaa1a9e700 1 -- 172.21.15.1:0/36566945 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba9c10c890 con 0x5637821564f0 2023-11-16T16:48:26.321 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.309+0000 7fbaa2aa0700 1 --2- 172.21.15.1:0/36566945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56378215a930 0x56378215fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:48:26.321 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.309+0000 7fbaa1a9e700 1 --2- 172.21.15.1:0/36566945 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637821564f0 0x56378215f3c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fba8c0996d0 tx=0x7fba8c0999d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=786df72015409306 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:26.321 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.309+0000 7fba85ffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:48:26.322 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.309+0000 7fba85ffb700 1 -- 172.21.15.1:0/36566945 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c0a1e20 con 0x5637821564f0 2023-11-16T16:48:26.322 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.309+0000 7fba85ffb700 1 -- 172.21.15.1:0/36566945 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fba8c00bcc0 con 0x5637821564f0 2023-11-16T16:48:26.322 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.309+0000 7fbab068fb80 1 -- 172.21.15.1:0/36566945 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563782148ba0 con 0x5637821564f0 2023-11-16T16:48:26.322 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.309+0000 7fba85ffb700 1 -- 172.21.15.1:0/36566945 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba8c099c30 con 0x5637821564f0 2023-11-16T16:48:26.322 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.309+0000 7fbab068fb80 10 client.16854 fetch_fsmap learned FSMap version 30 2023-11-16T16:48:26.322 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.309+0000 7fbab068fb80 1 -- 172.21.15.1:0/36566945 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56378206d210 con 0x5637821564f0 2023-11-16T16:48:26.322 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.309+0000 7fba85ffb700 1 -- 172.21.15.1:0/36566945 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563782148ba0 con 0x5637821564f0 2023-11-16T16:48:26.322 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.309+0000 7fba85ffb700 1 -- 172.21.15.1:0/36566945 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fba8c0a1990 con 0x5637821564f0 2023-11-16T16:48:26.322 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.309+0000 7fbab068fb80 10 client.16854 fetch_fsmap finished waiting for FSMap version 30 2023-11-16T16:48:26.323 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.309+0000 7fbab068fb80 10 client.16854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:48:26.323 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.309+0000 7fbab068fb80 20 client.16854 populate_metadata read hostname 'smithi001' 2023-11-16T16:48:26.323 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.310+0000 7fbab068fb80 1 --2- 172.21.15.1:0/36566945 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56378216f8b0 0x563782171cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:26.323 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.310+0000 7fbab068fb80 4 client.16854 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:48:26.323 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.310+0000 7fbab068fb80 1 -- 172.21.15.1:0/36566945 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563782069120 con 0x56378216f8b0 2023-11-16T16:48:26.323 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.310+0000 7fbaa229f700 1 --2- 172.21.15.1:0/36566945 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56378216f8b0 0x563782171cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:26.323 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.310+0000 7fba85ffb700 10 client.16854 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:48:26.323 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.310+0000 7fbaa229f700 1 --2- 172.21.15.1:0/36566945 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56378216f8b0 0x563782171cc0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b69800bb1052af8a server_cookie=18aecf07967909a3 in_seq=0 out_seq=0 2023-11-16T16:48:26.323 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.311+0000 7fba85ffb700 1 -- 172.21.15.1:0/36566945 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563782069120 con 0x56378216f8b0 2023-11-16T16:48:26.324 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.311+0000 7fba85ffb700 10 client.16854 handle_command_reply: tid=0 2023-11-16T16:48:26.324 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.311+0000 7fbab068fb80 1 client.16854 shutdown 2023-11-16T16:48:26.324 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.311+0000 7fbab068fb80 1 -- 172.21.15.1:0/36566945 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637821564f0 msgr2=0x56378215f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:26.324 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.311+0000 7fbab068fb80 1 --2- 172.21.15.1:0/36566945 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637821564f0 0x56378215f3c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fba8c0996d0 tx=0x7fba8c0999d0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.324 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.311+0000 7fbab068fb80 1 -- 172.21.15.1:0/36566945 shutdown_connections 2023-11-16T16:48:26.324 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.311+0000 7fbab068fb80 1 --2- 172.21.15.1:0/36566945 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56378216f8b0 0x563782171cc0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.324 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.311+0000 7fbab068fb80 1 --2- 172.21.15.1:0/36566945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56378215a930 0x56378215fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.324 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.311+0000 7fba85ffb700 0 client.16854 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:48:26.324 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.311+0000 7fbab068fb80 1 --2- 172.21.15.1:0/36566945 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x563782155be0 0x563782162140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.325 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.311+0000 7fbab068fb80 1 --2- 172.21.15.1:0/36566945 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637821564f0 0x56378215f3c0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.325 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.311+0000 7fbab068fb80 1 -- 172.21.15.1:0/36566945 >> 172.21.15.1:0/36566945 conn(0x56378214b3e0 msgr2=0x56378215bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:26.325 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.311+0000 7fbab068fb80 1 -- 172.21.15.1:0/36566945 shutdown_connections 2023-11-16T16:48:26.325 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.311+0000 7fbab068fb80 1 -- 172.21.15.1:0/36566945 wait complete. 2023-11-16T16:48:26.325 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.311+0000 7fbab068fb80 20 client.16854 trim_cache size 0 max 16384 2023-11-16T16:48:26.331 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.329+0000 7fbab068fb80 1 Processor -- start 2023-11-16T16:48:26.332 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.329+0000 7fbab068fb80 1 -- start start 2023-11-16T16:48:26.332 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.329+0000 7fbab068fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56378214b3e0 0x563782158220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:26.332 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.329+0000 7fbab068fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563782155be0 0x56378215f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:26.332 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.329+0000 7fbab068fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637821564f0 0x563782161c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:26.332 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.329+0000 7fbab068fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x563782160850 con 0x5637821564f0 2023-11-16T16:48:26.332 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.329+0000 7fbab068fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56378215d920 con 0x563782155be0 2023-11-16T16:48:26.333 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.329+0000 7fbab068fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x563782159540 con 0x56378214b3e0 2023-11-16T16:48:26.333 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.329+0000 7fbaa2aa0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637821564f0 0x563782161c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:26.333 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.329+0000 7fbaa2aa0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637821564f0 0x563782161c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46876/0 (socket says 172.21.15.1:46876) 2023-11-16T16:48:26.333 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.329+0000 7fbaa2aa0700 1 -- 172.21.15.1:0/1079771407 learned_addr learned my addr 172.21.15.1:0/1079771407 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:26.333 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.329+0000 7fbaa1a9e700 1 --2- 172.21.15.1:0/1079771407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563782155be0 0x56378215f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:26.333 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.329+0000 7fbaa229f700 1 --2- 172.21.15.1:0/1079771407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56378214b3e0 0x563782158220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:26.333 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.329+0000 7fbaa2aa0700 1 -- 172.21.15.1:0/1079771407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56378214b3e0 msgr2=0x563782158220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:26.333 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.329+0000 7fbaa2aa0700 1 --2- 172.21.15.1:0/1079771407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56378214b3e0 0x563782158220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.334 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.329+0000 7fbaa2aa0700 1 -- 172.21.15.1:0/1079771407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563782155be0 msgr2=0x56378215f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:26.334 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.329+0000 7fbaa2aa0700 1 --2- 172.21.15.1:0/1079771407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563782155be0 0x56378215f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.334 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.329+0000 7fbaa2aa0700 1 -- 172.21.15.1:0/1079771407 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56378214cb80 con 0x5637821564f0 2023-11-16T16:48:26.334 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.329+0000 7fbaa2aa0700 1 --2- 172.21.15.1:0/1079771407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637821564f0 0x563782161c70 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fba94007530 tx=0x7fba9400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f33b26b9968511 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:26.334 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.330+0000 7fba917fa700 1 -- 172.21.15.1:0/1079771407 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9400e070 con 0x5637821564f0 2023-11-16T16:48:26.334 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.330+0000 7fba917fa700 1 -- 172.21.15.1:0/1079771407 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba9400f040 con 0x5637821564f0 2023-11-16T16:48:26.334 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.330+0000 7fba917fa700 1 -- 172.21.15.1:0/1079771407 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba94015bb0 con 0x5637821564f0 2023-11-16T16:48:26.339 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.330+0000 7fbab068fb80 1 -- 172.21.15.1:0/1079771407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637821564f0 msgr2=0x563782161c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:26.339 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.330+0000 7fbab068fb80 1 --2- 172.21.15.1:0/1079771407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637821564f0 0x563782161c70 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fba94007530 tx=0x7fba9400b040 comp rx=0 tx=0).stop 2023-11-16T16:48:26.339 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.330+0000 7fbab068fb80 1 -- 172.21.15.1:0/1079771407 shutdown_connections 2023-11-16T16:48:26.339 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.330+0000 7fbab068fb80 1 --2- 172.21.15.1:0/1079771407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637821564f0 0x563782161c70 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.339 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.330+0000 7fbab068fb80 1 --2- 172.21.15.1:0/1079771407 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56378214b3e0 0x563782158220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.330+0000 7fbab068fb80 1 --2- 172.21.15.1:0/1079771407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563782155be0 0x56378215f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.330+0000 7fbab068fb80 1 -- 172.21.15.1:0/1079771407 >> 172.21.15.1:0/1079771407 conn(0x56378216c120 msgr2=0x56378215ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:26.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.330+0000 7fbab068fb80 1 -- 172.21.15.1:0/1079771407 shutdown_connections 2023-11-16T16:48:26.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.330+0000 7fbab068fb80 1 -- 172.21.15.1:0/1079771407 wait complete. 2023-11-16T16:48:26.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.331+0000 7fbab068fb80 1 -- start start 2023-11-16T16:48:26.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.331+0000 7fbab068fb80 1 Processor -- start 2023-11-16T16:48:26.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.331+0000 7fbab068fb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56378214b3e0 0x563782165220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:26.341 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.331+0000 7fbab068fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563782155be0 0x563782165760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:26.341 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.331+0000 7fbab068fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637821564f0 0x563782165ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:26.341 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.331+0000 7fbab068fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637821c38b0 con 0x5637821564f0 2023-11-16T16:48:26.341 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.331+0000 7fbab068fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637821c3a20 con 0x563782155be0 2023-11-16T16:48:26.341 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.331+0000 7fbab068fb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637821c3be0 con 0x56378214b3e0 2023-11-16T16:48:26.341 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.331+0000 7fbab068fb80 10 client.0 _ll_register_callbacks cb 0x7fba8c004f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:48:26.341 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.331+0000 7fbaa2aa0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637821564f0 0x563782165ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:26.342 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.331+0000 7fbaa2aa0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637821564f0 0x563782165ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46882/0 (socket says 172.21.15.1:46882) 2023-11-16T16:48:26.342 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.331+0000 7fbaa2aa0700 1 -- 172.21.15.1:0/654434578 learned_addr learned my addr 172.21.15.1:0/654434578 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:26.342 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.331+0000 7fbaa229f700 1 --2- 172.21.15.1:0/654434578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56378214b3e0 0x563782165220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:26.342 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.331+0000 7fbaa1a9e700 1 --2- 172.21.15.1:0/654434578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563782155be0 0x563782165760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:26.342 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.332+0000 7fbaa2aa0700 1 -- 172.21.15.1:0/654434578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56378214b3e0 msgr2=0x563782165220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:26.342 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.332+0000 7fbaa2aa0700 1 --2- 172.21.15.1:0/654434578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56378214b3e0 0x563782165220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.342 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.332+0000 7fbaa2aa0700 1 -- 172.21.15.1:0/654434578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563782155be0 msgr2=0x563782165760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:26.342 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.332+0000 7fbaa2aa0700 1 --2- 172.21.15.1:0/654434578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563782155be0 0x563782165760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.332+0000 7fbaa2aa0700 1 -- 172.21.15.1:0/654434578 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba94009240 con 0x5637821564f0 2023-11-16T16:48:26.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.332+0000 7fbaa2aa0700 1 --2- 172.21.15.1:0/654434578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637821564f0 0x563782165ca0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fba9400b010 tx=0x7fba94015ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ee0feffef9148e server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:26.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.332+0000 7fba85ffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:48:26.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.332+0000 7fba85ffb700 1 -- 172.21.15.1:0/654434578 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9400e040 con 0x5637821564f0 2023-11-16T16:48:26.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.332+0000 7fba85ffb700 1 -- 172.21.15.1:0/654434578 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fba94004cd0 con 0x5637821564f0 2023-11-16T16:48:26.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.332+0000 7fbab068fb80 1 -- 172.21.15.1:0/654434578 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba98029040 con 0x5637821564f0 2023-11-16T16:48:26.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.332+0000 7fba85ffb700 1 -- 172.21.15.1:0/654434578 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba94015ee0 con 0x5637821564f0 2023-11-16T16:48:26.344 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.332+0000 7fbab068fb80 10 client.16860 fetch_fsmap learned FSMap version 30 2023-11-16T16:48:26.344 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.332+0000 7fbab068fb80 1 -- 172.21.15.1:0/654434578 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563781ed4e60 con 0x5637821564f0 2023-11-16T16:48:26.344 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.332+0000 7fba85ffb700 1 -- 172.21.15.1:0/654434578 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba98029040 con 0x5637821564f0 2023-11-16T16:48:26.344 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.332+0000 7fba85ffb700 1 -- 172.21.15.1:0/654434578 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fba9401f7e0 con 0x5637821564f0 2023-11-16T16:48:26.344 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.332+0000 7fbab068fb80 10 client.16860 fetch_fsmap finished waiting for FSMap version 30 2023-11-16T16:48:26.344 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.332+0000 7fbab068fb80 10 client.16860 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:48:26.344 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.333+0000 7fbab068fb80 20 client.16860 populate_metadata read hostname 'smithi001' 2023-11-16T16:48:26.344 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.333+0000 7fbab068fb80 1 --2- 172.21.15.1:0/654434578 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56378215a930 0x5637821c7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:26.345 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.333+0000 7fbab068fb80 4 client.16860 mds_command: new command op to 24521 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-16T16:48:26.345 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.333+0000 7fbab068fb80 1 -- 172.21.15.1:0/654434578 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5637820756e0 con 0x56378215a930 2023-11-16T16:48:26.345 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.333+0000 7fbaa229f700 1 --2- 172.21.15.1:0/654434578 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56378215a930 0x5637821c7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:26.345 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.333+0000 7fba85ffb700 10 client.16860 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:48:26.345 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.333+0000 7fbaa229f700 1 --2- 172.21.15.1:0/654434578 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56378215a930 0x5637821c7370 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7112b44ec1ae4811 server_cookie=f0a56447b2235c25 in_seq=0 out_seq=0 2023-11-16T16:48:26.345 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.335+0000 7fba85ffb700 1 -- 172.21.15.1:0/654434578 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5637820756e0 con 0x56378215a930 2023-11-16T16:48:26.345 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.335+0000 7fba85ffb700 10 client.16860 handle_command_reply: tid=0 2023-11-16T16:48:26.345 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.335+0000 7fbab068fb80 1 client.16860 shutdown 2023-11-16T16:48:26.346 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.335+0000 7fbab068fb80 1 -- 172.21.15.1:0/654434578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637821564f0 msgr2=0x563782165ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:26.346 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.335+0000 7fbab068fb80 1 --2- 172.21.15.1:0/654434578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637821564f0 0x563782165ca0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fba9400b010 tx=0x7fba94015ae0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.346 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.335+0000 7fbab068fb80 1 -- 172.21.15.1:0/654434578 shutdown_connections 2023-11-16T16:48:26.346 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.335+0000 7fbab068fb80 1 --2- 172.21.15.1:0/654434578 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x56378215a930 0x5637821c7370 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.346 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.335+0000 7fbab068fb80 1 --2- 172.21.15.1:0/654434578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637821564f0 0x563782165ca0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.346 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.335+0000 7fbab068fb80 1 --2- 172.21.15.1:0/654434578 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56378214b3e0 0x563782165220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.346 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.335+0000 7fba85ffb700 0 client.16860 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:48:26.347 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.335+0000 7fbab068fb80 1 --2- 172.21.15.1:0/654434578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563782155be0 0x563782165760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.347 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.335+0000 7fbab068fb80 1 -- 172.21.15.1:0/654434578 >> 172.21.15.1:0/654434578 conn(0x56378216c120 msgr2=0x563782159950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:26.347 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.335+0000 7fbab068fb80 1 -- 172.21.15.1:0/654434578 shutdown_connections 2023-11-16T16:48:26.347 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.335+0000 7fbab068fb80 1 -- 172.21.15.1:0/654434578 wait complete. 2023-11-16T16:48:26.347 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.335+0000 7fbab068fb80 20 client.16860 trim_cache size 0 max 16384 2023-11-16T16:48:26.347 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:48:26.347 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2023-11-16T16:48:26.348 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "abc51d85-2c9f-4f6b-a32f-94db8c500ccb", 2023-11-16T16:48:26.348 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2023-11-16T16:48:26.348 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:48:26.349 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.336+0000 7fbaa4503700 1 -- 172.21.15.1:0/2753054444 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fba880a12c0 msgr2=0x7fba880a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:26.349 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.336+0000 7fbaa4503700 1 --2- 172.21.15.1:0/2753054444 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fba880a12c0 0x7fba880a3770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fba98009990 tx=0x7fba9801a040 comp rx=0 tx=0).stop 2023-11-16T16:48:26.349 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.336+0000 7fbaa4503700 1 -- 172.21.15.1:0/2753054444 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba9c0ffe30 msgr2=0x7fba9c196950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:26.349 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.336+0000 7fbaa4503700 1 --2- 172.21.15.1:0/2753054444 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba9c0ffe30 0x7fba9c196950 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fba8c00d950 tx=0x7fba8c00dd10 comp rx=0 tx=0).stop 2023-11-16T16:48:26.349 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.336+0000 7fbaa4503700 1 -- 172.21.15.1:0/2753054444 shutdown_connections 2023-11-16T16:48:26.350 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.336+0000 7fbaa4503700 1 --2- 172.21.15.1:0/2753054444 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fba880a12c0 0x7fba880a3770 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.350 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.336+0000 7fbaa4503700 1 --2- 172.21.15.1:0/2753054444 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fba9c104480 0x7fba9c19adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.350 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.336+0000 7fbaa4503700 1 --2- 172.21.15.1:0/2753054444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba9c0ff490 0x7fba9c196410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.350 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.336+0000 7fbaa4503700 1 --2- 172.21.15.1:0/2753054444 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba9c0ffe30 0x7fba9c196950 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.350 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.336+0000 7fbaa4503700 1 -- 172.21.15.1:0/2753054444 >> 172.21.15.1:0/2753054444 conn(0x7fba9c0763a0 msgr2=0x7fba9c068ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:26.350 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.336+0000 7fbaa4503700 1 -- 172.21.15.1:0/2753054444 shutdown_connections 2023-11-16T16:48:26.350 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.336+0000 7fbaa4503700 1 -- 172.21.15.1:0/2753054444 wait complete. 2023-11-16T16:48:26.352 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-16T16:48:26.480 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.478+0000 7f99bb497700 1 -- 172.21.15.1:0/2363438967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99b41009e0 msgr2=0x7f99b4100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:26.480 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.478+0000 7f99bb497700 1 --2- 172.21.15.1:0/2363438967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99b41009e0 0x7f99b4100e50 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f99a4009990 tx=0x7f99a4009ca0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.480 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.478+0000 7f99bb497700 1 -- 172.21.15.1:0/2363438967 shutdown_connections 2023-11-16T16:48:26.480 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.478+0000 7f99bb497700 1 --2- 172.21.15.1:0/2363438967 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99b4101390 0x7f99b410d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.480 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.478+0000 7f99bb497700 1 --2- 172.21.15.1:0/2363438967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99b41064d0 0x7f99b41068a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.478+0000 7f99bb497700 1 --2- 172.21.15.1:0/2363438967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99b41009e0 0x7f99b4100e50 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.478+0000 7f99bb497700 1 -- 172.21.15.1:0/2363438967 >> 172.21.15.1:0/2363438967 conn(0x7f99b4077530 msgr2=0x7f99b4077930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:26.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.478+0000 7f99bb497700 1 -- 172.21.15.1:0/2363438967 shutdown_connections 2023-11-16T16:48:26.481 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.478+0000 7f99bb497700 1 -- 172.21.15.1:0/2363438967 wait complete. 2023-11-16T16:48:26.483 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.480+0000 7f99bb497700 1 Processor -- start 2023-11-16T16:48:26.484 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.480+0000 7f99bb497700 1 -- start start 2023-11-16T16:48:26.484 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.480+0000 7f99bb497700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99b41009e0 0x7f99b41965a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:26.484 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.480+0000 7f99bb497700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99b4101390 0x7f99b4196ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:26.484 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.480+0000 7f99bb497700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99b41064d0 0x7f99b419af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:26.484 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.480+0000 7f99bb497700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b419b580 con 0x7f99b41009e0 2023-11-16T16:48:26.485 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.480+0000 7f99bb497700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b419b6f0 con 0x7f99b4101390 2023-11-16T16:48:26.485 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.480+0000 7f99bb497700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b419b8b0 con 0x7f99b41064d0 2023-11-16T16:48:26.485 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.480+0000 7f99b9233700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99b41009e0 0x7f99b41965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:26.485 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.480+0000 7f99b9233700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99b41009e0 0x7f99b41965a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46890/0 (socket says 172.21.15.1:46890) 2023-11-16T16:48:26.485 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.480+0000 7f99b9233700 1 -- 172.21.15.1:0/3992374498 learned_addr learned my addr 172.21.15.1:0/3992374498 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:26.485 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.480+0000 7f99b8a32700 1 --2- 172.21.15.1:0/3992374498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99b4101390 0x7f99b4196ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:26.485 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.480+0000 7f99b9a34700 1 --2- 172.21.15.1:0/3992374498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99b41064d0 0x7f99b419af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:26.486 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.480+0000 7f99b9233700 1 -- 172.21.15.1:0/3992374498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99b41064d0 msgr2=0x7f99b419af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:26.486 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.480+0000 7f99b9233700 1 --2- 172.21.15.1:0/3992374498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99b41064d0 0x7f99b419af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.486 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.480+0000 7f99b9233700 1 -- 172.21.15.1:0/3992374498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99b4101390 msgr2=0x7f99b4196ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:26.486 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.480+0000 7f99b9233700 1 --2- 172.21.15.1:0/3992374498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99b4101390 0x7f99b4196ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.486 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.480+0000 7f99b9233700 1 -- 172.21.15.1:0/3992374498 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a40095f0 con 0x7f99b41009e0 2023-11-16T16:48:26.486 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.480+0000 7f99b9233700 1 --2- 172.21.15.1:0/3992374498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99b41009e0 0x7f99b41965a0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f99b000d950 tx=0x7f99b000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:26.487 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.480+0000 7f99aa7fc700 1 -- 172.21.15.1:0/3992374498 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b0009940 con 0x7f99b41009e0 2023-11-16T16:48:26.487 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.480+0000 7f99aa7fc700 1 -- 172.21.15.1:0/3992374498 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99b0010460 con 0x7f99b41009e0 2023-11-16T16:48:26.487 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.480+0000 7f99bb497700 1 -- 172.21.15.1:0/3992374498 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99b4110510 con 0x7f99b41009e0 2023-11-16T16:48:26.487 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.480+0000 7f99aa7fc700 1 -- 172.21.15.1:0/3992374498 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b000f620 con 0x7f99b41009e0 2023-11-16T16:48:26.487 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.480+0000 7f99bb497700 1 -- 172.21.15.1:0/3992374498 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99b4110380 con 0x7f99b41009e0 2023-11-16T16:48:26.487 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.482+0000 7f99aa7fc700 1 -- 172.21.15.1:0/3992374498 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f99b00108d0 con 0x7f99b41009e0 2023-11-16T16:48:26.495 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.483+0000 7f99aa7fc700 1 --2- 172.21.15.1:0/3992374498 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f99a00a1340 0x7f99a00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:26.495 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.483+0000 7f99b8a32700 1 --2- 172.21.15.1:0/3992374498 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f99a00a1340 0x7f99a00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:26.495 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.483+0000 7f99aa7fc700 1 -- 172.21.15.1:0/3992374498 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f99b00ccaa0 con 0x7f99b41009e0 2023-11-16T16:48:26.495 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.483+0000 7f99b8a32700 1 --2- 172.21.15.1:0/3992374498 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f99a00a1340 0x7f99a00a37f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f99b4197bb0 tx=0x7f99a4005fb0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:26.495 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.483+0000 7f99c7623b80 1 Processor -- start 2023-11-16T16:48:26.496 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.484+0000 7f99c7623b80 1 -- start start 2023-11-16T16:48:26.496 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.484+0000 7f99c7623b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56529010cf70 0x56529010d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:26.496 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.484+0000 7f99c7623b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56529010d880 0x565290111780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:26.496 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.484+0000 7f99c7623b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x565290111cc0 0x565290112110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:26.496 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.484+0000 7f99c7623b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56529000e060 con 0x56529010d880 2023-11-16T16:48:26.496 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.484+0000 7f99c7623b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652900608a0 con 0x56529010cf70 2023-11-16T16:48:26.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.484+0000 7f99c7623b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x565290059d30 con 0x565290111cc0 2023-11-16T16:48:26.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.484+0000 7f99b8a32700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56529010d880 0x565290111780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:26.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.484+0000 7f99b8a32700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56529010d880 0x565290111780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46900/0 (socket says 172.21.15.1:46900) 2023-11-16T16:48:26.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.484+0000 7f99b8a32700 1 -- 172.21.15.1:0/613740644 learned_addr learned my addr 172.21.15.1:0/613740644 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:26.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.484+0000 7f99b9233700 1 --2- 172.21.15.1:0/613740644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56529010cf70 0x56529010d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:26.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.484+0000 7f99b9a34700 1 --2- 172.21.15.1:0/613740644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x565290111cc0 0x565290112110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:26.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.484+0000 7f99b8a32700 1 -- 172.21.15.1:0/613740644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x565290111cc0 msgr2=0x565290112110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:26.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.484+0000 7f99b8a32700 1 --2- 172.21.15.1:0/613740644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x565290111cc0 0x565290112110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.484+0000 7f99b8a32700 1 -- 172.21.15.1:0/613740644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56529010cf70 msgr2=0x56529010d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:26.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.484+0000 7f99b8a32700 1 --2- 172.21.15.1:0/613740644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56529010cf70 0x56529010d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.484+0000 7f99b8a32700 1 -- 172.21.15.1:0/613740644 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56528fedb630 con 0x56529010d880 2023-11-16T16:48:26.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.484+0000 7f99b9233700 1 --2- 172.21.15.1:0/613740644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56529010cf70 0x56529010d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:48:26.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.485+0000 7f99b8a32700 1 --2- 172.21.15.1:0/613740644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56529010d880 0x565290111780 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f99a400b7d0 tx=0x7f99a4020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac2e736d8ab5b5f server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:26.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.485+0000 7f999f7fe700 1 -- 172.21.15.1:0/613740644 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a402a070 con 0x56529010d880 2023-11-16T16:48:26.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.485+0000 7f999f7fe700 1 -- 172.21.15.1:0/613740644 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99a401e040 con 0x56529010d880 2023-11-16T16:48:26.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.485+0000 7f999f7fe700 1 -- 172.21.15.1:0/613740644 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a4023b60 con 0x56529010d880 2023-11-16T16:48:26.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.485+0000 7f99c7623b80 1 -- 172.21.15.1:0/613740644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56529010d880 msgr2=0x565290111780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:26.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.485+0000 7f99c7623b80 1 --2- 172.21.15.1:0/613740644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56529010d880 0x565290111780 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f99a400b7d0 tx=0x7f99a4020000 comp rx=0 tx=0).stop 2023-11-16T16:48:26.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.485+0000 7f99c7623b80 1 -- 172.21.15.1:0/613740644 shutdown_connections 2023-11-16T16:48:26.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.485+0000 7f99c7623b80 1 --2- 172.21.15.1:0/613740644 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x565290111cc0 0x565290112110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.485+0000 7f99c7623b80 1 --2- 172.21.15.1:0/613740644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56529010cf70 0x56529010d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.485+0000 7f99c7623b80 1 --2- 172.21.15.1:0/613740644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56529010d880 0x565290111780 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.485+0000 7f99c7623b80 1 -- 172.21.15.1:0/613740644 >> 172.21.15.1:0/613740644 conn(0x5652900fe360 msgr2=0x565290103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:26.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.486+0000 7f99c7623b80 1 -- 172.21.15.1:0/613740644 shutdown_connections 2023-11-16T16:48:26.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.486+0000 7f99c7623b80 1 -- 172.21.15.1:0/613740644 wait complete. 2023-11-16T16:48:26.500 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.486+0000 7f99c7623b80 1 -- start start 2023-11-16T16:48:26.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.486+0000 7f99c7623b80 1 Processor -- start 2023-11-16T16:48:26.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.486+0000 7f99c7623b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56529010cf70 0x565290119470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:26.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.486+0000 7f99c7623b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56529010d880 0x5652901166f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:26.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.486+0000 7f99c7623b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x565290111cc0 0x565290116d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:26.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.486+0000 7f99c7623b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56528fffb8e0 con 0x56529010d880 2023-11-16T16:48:26.501 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.486+0000 7f99c7623b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56528fffb580 con 0x56529010cf70 2023-11-16T16:48:26.502 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.486+0000 7f99c7623b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56528fedd920 con 0x565290111cc0 2023-11-16T16:48:26.502 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.486+0000 7f99c7623b80 10 client.0 _ll_register_callbacks cb 0x565290019780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:48:26.502 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.487+0000 7f99b8a32700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56529010d880 0x5652901166f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:26.502 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.487+0000 7f99b8a32700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56529010d880 0x5652901166f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46906/0 (socket says 172.21.15.1:46906) 2023-11-16T16:48:26.502 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.487+0000 7f99b8a32700 1 -- 172.21.15.1:0/1641901438 learned_addr learned my addr 172.21.15.1:0/1641901438 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:26.502 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.487+0000 7f99b9233700 1 --2- 172.21.15.1:0/1641901438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56529010cf70 0x565290119470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:26.502 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.487+0000 7f99b9a34700 1 --2- 172.21.15.1:0/1641901438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x565290111cc0 0x565290116d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:26.503 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.487+0000 7f99b8a32700 1 -- 172.21.15.1:0/1641901438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x565290111cc0 msgr2=0x565290116d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:26.503 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.487+0000 7f99b8a32700 1 --2- 172.21.15.1:0/1641901438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x565290111cc0 0x565290116d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.503 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.487+0000 7f99b8a32700 1 -- 172.21.15.1:0/1641901438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56529010cf70 msgr2=0x565290119470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:26.503 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.487+0000 7f99b8a32700 1 --2- 172.21.15.1:0/1641901438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56529010cf70 0x565290119470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.503 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.487+0000 7f99b8a32700 1 -- 172.21.15.1:0/1641901438 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99b410dc00 con 0x56529010d880 2023-11-16T16:48:26.503 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.487+0000 7f99b9233700 1 --2- 172.21.15.1:0/1641901438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56529010cf70 0x565290119470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:48:26.503 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.487+0000 7f99b8a32700 1 --2- 172.21.15.1:0/1641901438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56529010d880 0x5652901166f0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f99a40234a0 tx=0x7f99a4005370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8fb3d22a8ebb21e server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:26.504 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.487+0000 7f999cff9700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:48:26.504 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.487+0000 7f999cff9700 1 -- 172.21.15.1:0/1641901438 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a402a040 con 0x56529010d880 2023-11-16T16:48:26.504 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.487+0000 7f999cff9700 1 -- 172.21.15.1:0/1641901438 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f99a4005610 con 0x56529010d880 2023-11-16T16:48:26.504 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.487+0000 7f99c7623b80 1 -- 172.21.15.1:0/1641901438 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5652900fbb20 con 0x56529010d880 2023-11-16T16:48:26.504 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.487+0000 7f999cff9700 1 -- 172.21.15.1:0/1641901438 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99a4023500 con 0x56529010d880 2023-11-16T16:48:26.504 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.487+0000 7f99c7623b80 10 client.16872 fetch_fsmap learned FSMap version 30 2023-11-16T16:48:26.504 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.488+0000 7f99c7623b80 1 -- 172.21.15.1:0/1641901438 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x565290020190 con 0x56529010d880 2023-11-16T16:48:26.505 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.488+0000 7f999cff9700 1 -- 172.21.15.1:0/1641901438 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99a4044080 con 0x56529010d880 2023-11-16T16:48:26.505 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.488+0000 7f999cff9700 1 -- 172.21.15.1:0/1641901438 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f99a4040490 con 0x56529010d880 2023-11-16T16:48:26.505 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.488+0000 7f99c7623b80 10 client.16872 fetch_fsmap finished waiting for FSMap version 30 2023-11-16T16:48:26.505 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.488+0000 7f99c7623b80 10 client.16872 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:48:26.505 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.488+0000 7f99c7623b80 20 client.16872 populate_metadata read hostname 'smithi001' 2023-11-16T16:48:26.505 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.488+0000 7f99c7623b80 1 --2- 172.21.15.1:0/1641901438 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x565290126780 0x565290128b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:26.505 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.488+0000 7f99c7623b80 4 client.16872 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:48:26.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.488+0000 7f99c7623b80 1 -- 172.21.15.1:0/1641901438 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56529001c0a0 con 0x565290126780 2023-11-16T16:48:26.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.488+0000 7f99b9233700 1 --2- 172.21.15.1:0/1641901438 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x565290126780 0x565290128b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:26.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.489+0000 7f999cff9700 10 client.16872 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:48:26.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.489+0000 7f99b9233700 1 --2- 172.21.15.1:0/1641901438 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x565290126780 0x565290128b90 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df397b2df0a99ac2 server_cookie=24761dea7b744dfc in_seq=0 out_seq=0 2023-11-16T16:48:26.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.489+0000 7f999cff9700 1 -- 172.21.15.1:0/1641901438 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56529001c0a0 con 0x565290126780 2023-11-16T16:48:26.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.489+0000 7f999cff9700 10 client.16872 handle_command_reply: tid=0 2023-11-16T16:48:26.506 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.489+0000 7f99c7623b80 1 client.16872 shutdown 2023-11-16T16:48:26.507 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.489+0000 7f99c7623b80 1 -- 172.21.15.1:0/1641901438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56529010d880 msgr2=0x5652901166f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:26.507 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.489+0000 7f99c7623b80 1 --2- 172.21.15.1:0/1641901438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56529010d880 0x5652901166f0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f99a40234a0 tx=0x7f99a4005370 comp rx=0 tx=0).stop 2023-11-16T16:48:26.507 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.489+0000 7f99c7623b80 1 -- 172.21.15.1:0/1641901438 shutdown_connections 2023-11-16T16:48:26.507 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.489+0000 7f99c7623b80 1 --2- 172.21.15.1:0/1641901438 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x565290126780 0x565290128b90 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.507 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.490+0000 7f99c7623b80 1 --2- 172.21.15.1:0/1641901438 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x565290111cc0 0x565290116d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.507 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.490+0000 7f999cff9700 0 client.16872 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:48:26.507 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.490+0000 7f99c7623b80 1 --2- 172.21.15.1:0/1641901438 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56529010cf70 0x565290119470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.508 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.490+0000 7f99c7623b80 1 --2- 172.21.15.1:0/1641901438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56529010d880 0x5652901166f0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.508 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.490+0000 7f99c7623b80 1 -- 172.21.15.1:0/1641901438 >> 172.21.15.1:0/1641901438 conn(0x5652900fe360 msgr2=0x565290102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:26.508 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.490+0000 7f99c7623b80 1 -- 172.21.15.1:0/1641901438 shutdown_connections 2023-11-16T16:48:26.508 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.490+0000 7f99c7623b80 1 -- 172.21.15.1:0/1641901438 wait complete. 2023-11-16T16:48:26.508 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.490+0000 7f99c7623b80 20 client.16872 trim_cache size 0 max 16384 2023-11-16T16:48:26.515 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.507+0000 7f99c7623b80 1 Processor -- start 2023-11-16T16:48:26.515 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.507+0000 7f99c7623b80 1 -- start start 2023-11-16T16:48:26.515 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.507+0000 7f99c7623b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5652900fe360 0x565290119010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:26.515 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.507+0000 7f99c7623b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56529010cf70 0x56529011dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:26.515 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.507+0000 7f99c7623b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56529010d880 0x565290115c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:26.515 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.507+0000 7f99c7623b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x565290110640 con 0x56529010d880 2023-11-16T16:48:26.516 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.507+0000 7f99c7623b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56529010dfd0 con 0x5652900fe360 2023-11-16T16:48:26.516 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.507+0000 7f99c7623b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x565290117cd0 con 0x56529010cf70 2023-11-16T16:48:26.516 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.507+0000 7f99b9a34700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56529010d880 0x565290115c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:26.516 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.507+0000 7f99b9a34700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56529010d880 0x565290115c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46916/0 (socket says 172.21.15.1:46916) 2023-11-16T16:48:26.516 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.507+0000 7f99b9a34700 1 -- 172.21.15.1:0/1060865224 learned_addr learned my addr 172.21.15.1:0/1060865224 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:26.516 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.507+0000 7f99b8a32700 1 --2- 172.21.15.1:0/1060865224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56529010cf70 0x56529011dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:26.516 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.507+0000 7f99b9233700 1 --2- 172.21.15.1:0/1060865224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5652900fe360 0x565290119010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:26.517 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.507+0000 7f99b9a34700 1 -- 172.21.15.1:0/1060865224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56529010cf70 msgr2=0x56529011dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:26.517 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.507+0000 7f99b9a34700 1 --2- 172.21.15.1:0/1060865224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56529010cf70 0x56529011dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.517 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.507+0000 7f99b9a34700 1 -- 172.21.15.1:0/1060865224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5652900fe360 msgr2=0x565290119010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:26.517 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.507+0000 7f99b9a34700 1 --2- 172.21.15.1:0/1060865224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5652900fe360 0x565290119010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.517 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.507+0000 7f99b9a34700 1 -- 172.21.15.1:0/1060865224 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56529001fee0 con 0x56529010d880 2023-11-16T16:48:26.517 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.507+0000 7f99b9233700 1 --2- 172.21.15.1:0/1060865224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5652900fe360 0x565290119010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:48:26.517 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.507+0000 7f99b8a32700 1 --2- 172.21.15.1:0/1060865224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56529010cf70 0x56529011dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:48:26.518 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.508+0000 7f99b9a34700 1 --2- 172.21.15.1:0/1060865224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56529010d880 0x565290115c60 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f99ac009c80 tx=0x7f99ac009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9231ff33d84461b server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:26.518 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.508+0000 7f999ffff700 1 -- 172.21.15.1:0/1060865224 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ac007d10 con 0x56529010d880 2023-11-16T16:48:26.518 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.508+0000 7f999ffff700 1 -- 172.21.15.1:0/1060865224 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99ac007e70 con 0x56529010d880 2023-11-16T16:48:26.518 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.508+0000 7f999ffff700 1 -- 172.21.15.1:0/1060865224 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ac016860 con 0x56529010d880 2023-11-16T16:48:26.518 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.508+0000 7f99c7623b80 1 -- 172.21.15.1:0/1060865224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56529010d880 msgr2=0x565290115c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:26.518 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.508+0000 7f99c7623b80 1 --2- 172.21.15.1:0/1060865224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56529010d880 0x565290115c60 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f99ac009c80 tx=0x7f99ac009040 comp rx=0 tx=0).stop 2023-11-16T16:48:26.518 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.508+0000 7f99c7623b80 1 -- 172.21.15.1:0/1060865224 shutdown_connections 2023-11-16T16:48:26.519 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.508+0000 7f99c7623b80 1 --2- 172.21.15.1:0/1060865224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56529010d880 0x565290115c60 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.519 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.508+0000 7f99c7623b80 1 --2- 172.21.15.1:0/1060865224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5652900fe360 0x565290119010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.519 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.508+0000 7f99c7623b80 1 --2- 172.21.15.1:0/1060865224 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56529010cf70 0x56529011dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.519 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.508+0000 7f99c7623b80 1 -- 172.21.15.1:0/1060865224 >> 172.21.15.1:0/1060865224 conn(0x565290105250 msgr2=0x565290122f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:26.519 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.508+0000 7f99c7623b80 1 -- 172.21.15.1:0/1060865224 shutdown_connections 2023-11-16T16:48:26.519 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.508+0000 7f99c7623b80 1 -- 172.21.15.1:0/1060865224 wait complete. 2023-11-16T16:48:26.519 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.509+0000 7f99c7623b80 1 -- start start 2023-11-16T16:48:26.520 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.509+0000 7f99c7623b80 1 Processor -- start 2023-11-16T16:48:26.520 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.509+0000 7f99c7623b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652900fe360 0x56529011dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:26.520 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.509+0000 7f99c7623b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56529010cf70 0x56529011a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:26.520 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.509+0000 7f99c7623b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56529010d880 0x56529011a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:26.520 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.509+0000 7f99c7623b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56529011ad50 con 0x56529010cf70 2023-11-16T16:48:26.520 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.509+0000 7f99c7623b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56529011aec0 con 0x56529010d880 2023-11-16T16:48:26.520 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.509+0000 7f99c7623b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56529017a640 con 0x5652900fe360 2023-11-16T16:48:26.521 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.509+0000 7f99c7623b80 10 client.0 _ll_register_callbacks cb 0x565290019780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:48:26.521 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.509+0000 7f99b8a32700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56529010cf70 0x56529011a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:26.521 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.509+0000 7f99b8a32700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56529010cf70 0x56529011a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46920/0 (socket says 172.21.15.1:46920) 2023-11-16T16:48:26.521 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.509+0000 7f99b8a32700 1 -- 172.21.15.1:0/1782353181 learned_addr learned my addr 172.21.15.1:0/1782353181 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:26.521 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.509+0000 7f99b9a34700 1 --2- 172.21.15.1:0/1782353181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56529010d880 0x56529011a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:26.521 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.509+0000 7f99b9233700 1 --2- 172.21.15.1:0/1782353181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652900fe360 0x56529011dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:26.521 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.510+0000 7f99b8a32700 1 -- 172.21.15.1:0/1782353181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652900fe360 msgr2=0x56529011dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:26.521 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.510+0000 7f99b8a32700 1 --2- 172.21.15.1:0/1782353181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652900fe360 0x56529011dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.522 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.510+0000 7f99b8a32700 1 -- 172.21.15.1:0/1782353181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56529010d880 msgr2=0x56529011a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:26.522 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.510+0000 7f99b8a32700 1 --2- 172.21.15.1:0/1782353181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56529010d880 0x56529011a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.522 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.510+0000 7f99b8a32700 1 -- 172.21.15.1:0/1782353181 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99ac011040 con 0x56529010cf70 2023-11-16T16:48:26.522 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.510+0000 7f99b8a32700 1 --2- 172.21.15.1:0/1782353181 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56529010cf70 0x56529011a100 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f99a4040840 tx=0x7f99a4003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=234a2288a31a7c74 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:26.522 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.510+0000 7f999cff9700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:48:26.522 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.510+0000 7f999cff9700 1 -- 172.21.15.1:0/1782353181 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a402a070 con 0x56529010cf70 2023-11-16T16:48:26.522 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.510+0000 7f999cff9700 1 -- 172.21.15.1:0/1782353181 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f99a4005720 con 0x56529010cf70 2023-11-16T16:48:26.523 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.510+0000 7f99c7623b80 1 -- 172.21.15.1:0/1782353181 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f99a401e040 con 0x56529010cf70 2023-11-16T16:48:26.523 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.510+0000 7f999cff9700 1 -- 172.21.15.1:0/1782353181 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99a40405e0 con 0x56529010cf70 2023-11-16T16:48:26.523 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.510+0000 7f99c7623b80 10 client.16878 fetch_fsmap learned FSMap version 30 2023-11-16T16:48:26.523 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.510+0000 7f99c7623b80 1 -- 172.21.15.1:0/1782353181 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56528fe70c60 con 0x56529010cf70 2023-11-16T16:48:26.523 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.510+0000 7f999cff9700 1 -- 172.21.15.1:0/1782353181 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99a401e040 con 0x56529010cf70 2023-11-16T16:48:26.523 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.510+0000 7f999cff9700 1 -- 172.21.15.1:0/1782353181 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 30) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f99a4038930 con 0x56529010cf70 2023-11-16T16:48:26.523 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.510+0000 7f99c7623b80 10 client.16878 fetch_fsmap finished waiting for FSMap version 30 2023-11-16T16:48:26.523 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.510+0000 7f99c7623b80 10 client.16878 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:48:26.524 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.510+0000 7f99c7623b80 20 client.16878 populate_metadata read hostname 'smithi001' 2023-11-16T16:48:26.524 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.510+0000 7f99c7623b80 1 --2- 172.21.15.1:0/1782353181 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x565290111cc0 0x56529017d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:26.524 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.511+0000 7f99c7623b80 4 client.16878 mds_command: new command op to 24521 tid=0[{"prefix": "scrub status"}] 2023-11-16T16:48:26.524 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.511+0000 7f99c7623b80 1 -- 172.21.15.1:0/1782353181 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x565290028660 con 0x565290111cc0 2023-11-16T16:48:26.524 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.511+0000 7f99b9233700 1 --2- 172.21.15.1:0/1782353181 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x565290111cc0 0x56529017d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:26.524 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.511+0000 7f999cff9700 10 client.16878 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:48:26.525 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.511+0000 7f99b9233700 1 --2- 172.21.15.1:0/1782353181 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x565290111cc0 0x56529017d970 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89f2425dd7ff6091 server_cookie=531eff28e38918b6 in_seq=0 out_seq=0 2023-11-16T16:48:26.525 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.511+0000 7f999cff9700 1 -- 172.21.15.1:0/1782353181 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x565290028660 con 0x565290111cc0 2023-11-16T16:48:26.525 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.511+0000 7f999cff9700 10 client.16878 handle_command_reply: tid=0 2023-11-16T16:48:26.525 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.511+0000 7f99c7623b80 1 client.16878 shutdown 2023-11-16T16:48:26.525 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.511+0000 7f99c7623b80 1 -- 172.21.15.1:0/1782353181 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56529010cf70 msgr2=0x56529011a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:26.525 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.511+0000 7f99c7623b80 1 --2- 172.21.15.1:0/1782353181 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56529010cf70 0x56529011a100 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f99a4040840 tx=0x7f99a4003c30 comp rx=0 tx=0).stop 2023-11-16T16:48:26.525 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.511+0000 7f99c7623b80 1 -- 172.21.15.1:0/1782353181 shutdown_connections 2023-11-16T16:48:26.526 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.511+0000 7f99c7623b80 1 --2- 172.21.15.1:0/1782353181 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x565290111cc0 0x56529017d970 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.526 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.511+0000 7f99c7623b80 1 --2- 172.21.15.1:0/1782353181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56529010d880 0x56529011a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.526 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.511+0000 7f99c7623b80 1 --2- 172.21.15.1:0/1782353181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5652900fe360 0x56529011dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.526 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.511+0000 7f99c7623b80 1 --2- 172.21.15.1:0/1782353181 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56529010cf70 0x56529011a100 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.526 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.511+0000 7f999cff9700 0 client.16878 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:48:26.526 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.511+0000 7f99c7623b80 1 -- 172.21.15.1:0/1782353181 >> 172.21.15.1:0/1782353181 conn(0x565290105250 msgr2=0x56529010bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:26.526 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.512+0000 7f99c7623b80 1 -- 172.21.15.1:0/1782353181 shutdown_connections 2023-11-16T16:48:26.526 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.512+0000 7f99c7623b80 1 -- 172.21.15.1:0/1782353181 wait complete. 2023-11-16T16:48:26.527 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.512+0000 7f99c7623b80 20 client.16878 trim_cache size 0 max 16384 2023-11-16T16:48:26.527 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:48:26.527 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-16T16:48:26.527 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2023-11-16T16:48:26.527 INFO:teuthology.orchestra.run.smithi001.stdout: "abc51d85-2c9f-4f6b-a32f-94db8c500ccb": { 2023-11-16T16:48:26.528 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2023-11-16T16:48:26.528 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "abc51d85-2c9f-4f6b-a32f-94db8c500ccb", 2023-11-16T16:48:26.528 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2023-11-16T16:48:26.528 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:48:26.528 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:48:26.528 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:48:26.529 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.512+0000 7f99bb497700 1 -- 172.21.15.1:0/3992374498 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f99a00a1340 msgr2=0x7f99a00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:26.529 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.512+0000 7f99bb497700 1 --2- 172.21.15.1:0/3992374498 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f99a00a1340 0x7f99a00a37f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f99b4197bb0 tx=0x7f99a4005fb0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.529 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.512+0000 7f99bb497700 1 -- 172.21.15.1:0/3992374498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99b41009e0 msgr2=0x7f99b41965a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:26.530 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.512+0000 7f99bb497700 1 --2- 172.21.15.1:0/3992374498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99b41009e0 0x7f99b41965a0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f99b000d950 tx=0x7f99b000dc60 comp rx=0 tx=0).stop 2023-11-16T16:48:26.530 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.512+0000 7f99bb497700 1 -- 172.21.15.1:0/3992374498 shutdown_connections 2023-11-16T16:48:26.530 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.512+0000 7f99bb497700 1 --2- 172.21.15.1:0/3992374498 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f99a00a1340 0x7f99a00a37f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.530 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.512+0000 7f99bb497700 1 --2- 172.21.15.1:0/3992374498 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f99b41064d0 0x7f99b419af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.530 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.512+0000 7f99bb497700 1 --2- 172.21.15.1:0/3992374498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f99b41009e0 0x7f99b41965a0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.530 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.512+0000 7f99bb497700 1 --2- 172.21.15.1:0/3992374498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99b4101390 0x7f99b4196ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:26.530 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.512+0000 7f99bb497700 1 -- 172.21.15.1:0/3992374498 >> 172.21.15.1:0/3992374498 conn(0x7f99b4077530 msgr2=0x7f99b40fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:26.531 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.512+0000 7f99bb497700 1 -- 172.21.15.1:0/3992374498 shutdown_connections 2023-11-16T16:48:26.531 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:26.512+0000 7f99bb497700 1 -- 172.21.15.1:0/3992374498 wait complete. 2023-11-16T16:48:26.531 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:26 smithi001 ceph-mon[98800]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:48:26.531 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:26 smithi001 ceph-mon[98800]: pgmap v287: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-11-16T16:48:26.532 INFO:tasks.cephfs.filesystem:scrub status for tag:abc51d85-2c9f-4f6b-a32f-94db8c500ccb - {'path': '/', 'tag': 'abc51d85-2c9f-4f6b-a32f-94db8c500ccb', 'options': 'recursive,force'} 2023-11-16T16:48:26.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:26 smithi026 ceph-mon[100112]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:48:26.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:26 smithi026 ceph-mon[100112]: pgmap v287: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-11-16T16:48:26.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:26 smithi053 ceph-mon[100638]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:48:26.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:26 smithi053 ceph-mon[100638]: pgmap v287: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-11-16T16:48:27.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:27 smithi001 ceph-mon[98800]: scrub queued for path: / 2023-11-16T16:48:27.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:27 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:48:27.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:27 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:48:27.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:27 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:48:27.757 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:27 smithi026 ceph-mon[100112]: scrub queued for path: / 2023-11-16T16:48:27.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:27 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:48:27.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:27 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:48:27.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:27 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:48:27.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:27 smithi053 ceph-mon[100638]: scrub queued for path: / 2023-11-16T16:48:27.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:27 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:48:27.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:27 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:48:27.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:27 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:48:28.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:28 smithi001 ceph-mon[98800]: pgmap v288: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:48:28.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:28 smithi026 ceph-mon[100112]: pgmap v288: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:48:28.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:28 smithi053 ceph-mon[100638]: pgmap v288: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:48:29.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:29 smithi001 ceph-mon[98800]: scrub summary: idle 2023-11-16T16:48:29.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:29 smithi026 ceph-mon[100112]: scrub summary: idle 2023-11-16T16:48:29.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:29 smithi053 ceph-mon[100638]: scrub summary: idle 2023-11-16T16:48:30.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:30 smithi001 ceph-mon[98800]: pgmap v289: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:48:30.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:30 smithi026 ceph-mon[100112]: pgmap v289: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:48:30.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:30 smithi053 ceph-mon[100638]: pgmap v289: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:48:32.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:32 smithi001 ceph-mon[98800]: pgmap v290: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-16T16:48:32.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:32 smithi026 ceph-mon[100112]: pgmap v290: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-16T16:48:32.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:32 smithi053 ceph-mon[100638]: pgmap v290: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-16T16:48:33.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:33 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:48:33.757 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:33 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:48:33.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:33 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:48:34.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:34 smithi001 ceph-mon[98800]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:48:34.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:34 smithi001 ceph-mon[98800]: pgmap v291: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:48:34.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:34 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:48:34.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:34 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:48:34.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:34 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:48:34.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:34 smithi026 ceph-mon[100112]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:48:34.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:34 smithi026 ceph-mon[100112]: pgmap v291: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:48:34.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:34 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:48:34.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:34 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:48:34.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:34 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:48:34.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:34 smithi053 ceph-mon[100638]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:48:34.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:34 smithi053 ceph-mon[100638]: pgmap v291: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:48:34.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:34 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:48:34.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:34 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:48:34.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:34 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:48:35.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:35 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:48:35.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:35 smithi001 ceph-mon[98800]: Service mds.all has no related daemon already placed. 2023-11-16T16:48:35.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:35 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:48:35.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:35 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:48:35.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:35 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:48:35.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:35 smithi026 ceph-mon[100112]: Service mds.all has no related daemon already placed. 2023-11-16T16:48:35.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:35 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:48:35.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:35 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:48:35.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:35 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:48:35.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:35 smithi053 ceph-mon[100638]: Service mds.all has no related daemon already placed. 2023-11-16T16:48:35.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:35 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:48:35.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:35 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:48:36.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:36 smithi001 ceph-mon[98800]: pgmap v292: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-16T16:48:36.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:36 smithi026 ceph-mon[100112]: pgmap v292: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-16T16:48:36.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:36 smithi053 ceph-mon[100638]: pgmap v292: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-16T16:48:38.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:37 smithi026 ceph-mon[100112]: pgmap v293: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:48:38.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:37 smithi053 ceph-mon[100638]: pgmap v293: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:48:38.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:37 smithi001 ceph-mon[98800]: pgmap v293: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:48:40.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:40 smithi053 ceph-mon[100638]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:48:40.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:40 smithi053 ceph-mon[100638]: pgmap v294: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:48:40.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:40 smithi001 ceph-mon[98800]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:48:40.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:40 smithi001 ceph-mon[98800]: pgmap v294: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:48:40.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:40 smithi026 ceph-mon[100112]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:48:40.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:40 smithi026 ceph-mon[100112]: pgmap v294: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:48:42.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:42 smithi001 ceph-mon[98800]: pgmap v295: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-16T16:48:42.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:42 smithi026 ceph-mon[100112]: pgmap v295: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-16T16:48:42.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:42 smithi053 ceph-mon[100638]: pgmap v295: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-16T16:48:44.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:43 smithi026 ceph-mon[100112]: pgmap v296: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:48:44.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:43 smithi053 ceph-mon[100638]: pgmap v296: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:48:44.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:43 smithi001 ceph-mon[98800]: pgmap v296: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:48:46.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:45 smithi026 ceph-mon[100112]: pgmap v297: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:48:46.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:45 smithi053 ceph-mon[100638]: pgmap v297: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:48:46.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:45 smithi001 ceph-mon[98800]: pgmap v297: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:48:48.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:47 smithi026 ceph-mon[100112]: pgmap v298: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:48:48.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:47 smithi053 ceph-mon[100638]: pgmap v298: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:48:48.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:47 smithi001 ceph-mon[98800]: pgmap v298: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:48:50.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:49 smithi026 ceph-mon[100112]: pgmap v299: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:48:50.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:49 smithi053 ceph-mon[100638]: pgmap v299: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:48:50.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:49 smithi001 ceph-mon[98800]: pgmap v299: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:48:52.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:52 smithi001 ceph-mon[98800]: pgmap v300: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:48:52.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:52 smithi026 ceph-mon[100112]: pgmap v300: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:48:52.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:52 smithi053 ceph-mon[100638]: pgmap v300: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:48:54.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:53 smithi026 ceph-mon[100112]: pgmap v301: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:48:54.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:53 smithi053 ceph-mon[100638]: pgmap v301: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:48:54.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:53 smithi001 ceph-mon[98800]: pgmap v301: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:48:56.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:55 smithi026 ceph-mon[100112]: pgmap v302: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:48:56.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:55 smithi053 ceph-mon[100638]: pgmap v302: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:48:56.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:55 smithi001 ceph-mon[98800]: pgmap v302: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:48:56.533 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-16T16:48:56.677 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.675+0000 7fdf048dc700 1 -- 172.21.15.1:0/2718683283 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf00104550 msgr2=0x7fdf0010c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:56.677 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.675+0000 7fdf048dc700 1 --2- 172.21.15.1:0/2718683283 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf00104550 0x7fdf0010c120 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fdef800b0d0 tx=0x7fdef800b3e0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.677 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.675+0000 7fdf048dc700 1 -- 172.21.15.1:0/2718683283 shutdown_connections 2023-11-16T16:48:56.677 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.675+0000 7fdf048dc700 1 --2- 172.21.15.1:0/2718683283 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf00104550 0x7fdf0010c120 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.678 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.675+0000 7fdf048dc700 1 --2- 172.21.15.1:0/2718683283 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf000ffd10 0x7fdf001000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.678 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.675+0000 7fdf048dc700 1 --2- 172.21.15.1:0/2718683283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf001006b0 0x7fdf00103e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.678 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.675+0000 7fdf048dc700 1 -- 172.21.15.1:0/2718683283 >> 172.21.15.1:0/2718683283 conn(0x7fdf000747b0 msgr2=0x7fdf00074bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:56.678 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.675+0000 7fdf048dc700 1 -- 172.21.15.1:0/2718683283 shutdown_connections 2023-11-16T16:48:56.678 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.675+0000 7fdf048dc700 1 -- 172.21.15.1:0/2718683283 wait complete. 2023-11-16T16:48:56.680 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.676+0000 7fdf048dc700 1 Processor -- start 2023-11-16T16:48:56.680 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.676+0000 7fdf048dc700 1 -- start start 2023-11-16T16:48:56.680 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.676+0000 7fdf048dc700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf000ffd10 0x7fdf0010ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:56.680 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.676+0000 7fdf048dc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf001006b0 0x7fdf0010cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:56.681 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.676+0000 7fdf048dc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf00104550 0x7fdf001a4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:56.681 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.676+0000 7fdf048dc700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0010d7a0 con 0x7fdf001006b0 2023-11-16T16:48:56.681 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.676+0000 7fdf048dc700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf001a4d60 con 0x7fdf00104550 2023-11-16T16:48:56.681 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.676+0000 7fdf048dc700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf001a4ea0 con 0x7fdf000ffd10 2023-11-16T16:48:56.681 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.677+0000 7fdefdd9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf001006b0 0x7fdf0010cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:56.681 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.677+0000 7fdefdd9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf001006b0 0x7fdf0010cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46934/0 (socket says 172.21.15.1:46934) 2023-11-16T16:48:56.681 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.677+0000 7fdefdd9b700 1 -- 172.21.15.1:0/1949443890 learned_addr learned my addr 172.21.15.1:0/1949443890 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:56.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.677+0000 7fdefed9d700 1 --2- 172.21.15.1:0/1949443890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf00104550 0x7fdf001a4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:56.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.677+0000 7fdefe59c700 1 --2- 172.21.15.1:0/1949443890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf000ffd10 0x7fdf0010ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:56.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.677+0000 7fdefdd9b700 1 -- 172.21.15.1:0/1949443890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf000ffd10 msgr2=0x7fdf0010ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:56.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.677+0000 7fdefdd9b700 1 --2- 172.21.15.1:0/1949443890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf000ffd10 0x7fdf0010ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.677+0000 7fdefdd9b700 1 -- 172.21.15.1:0/1949443890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf00104550 msgr2=0x7fdf001a4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:56.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.677+0000 7fdefdd9b700 1 --2- 172.21.15.1:0/1949443890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf00104550 0x7fdf001a4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.677+0000 7fdefdd9b700 1 -- 172.21.15.1:0/1949443890 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef8009d00 con 0x7fdf001006b0 2023-11-16T16:48:56.682 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.677+0000 7fdefdd9b700 1 --2- 172.21.15.1:0/1949443890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf001006b0 0x7fdf0010cf70 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fdee800c9b0 tx=0x7fdee800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:56.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.677+0000 7fdef77fe700 1 -- 172.21.15.1:0/1949443890 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee8007aa0 con 0x7fdf001006b0 2023-11-16T16:48:56.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.677+0000 7fdf048dc700 1 -- 172.21.15.1:0/1949443890 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf001104e0 con 0x7fdf001006b0 2023-11-16T16:48:56.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.677+0000 7fdef77fe700 1 -- 172.21.15.1:0/1949443890 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdee8007c00 con 0x7fdf001006b0 2023-11-16T16:48:56.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.677+0000 7fdef77fe700 1 -- 172.21.15.1:0/1949443890 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee8018660 con 0x7fdf001006b0 2023-11-16T16:48:56.683 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.677+0000 7fdf048dc700 1 -- 172.21.15.1:0/1949443890 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf00110350 con 0x7fdf001006b0 2023-11-16T16:48:56.689 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.679+0000 7fdef77fe700 1 -- 172.21.15.1:0/1949443890 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fdee800f450 con 0x7fdf001006b0 2023-11-16T16:48:56.690 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.680+0000 7fdef77fe700 1 --2- 172.21.15.1:0/1949443890 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fdeec0a12c0 0x7fdeec0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:56.690 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.680+0000 7fdefe59c700 1 --2- 172.21.15.1:0/1949443890 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fdeec0a12c0 0x7fdeec0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:56.690 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.680+0000 7fdef77fe700 1 -- 172.21.15.1:0/1949443890 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fdee80ccb70 con 0x7fdf001006b0 2023-11-16T16:48:56.690 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.681+0000 7fdefe59c700 1 --2- 172.21.15.1:0/1949443890 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fdeec0a12c0 0x7fdeec0a3770 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdef0009bf0 tx=0x7fdef0009260 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:56.690 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.681+0000 7fdf10a68b80 1 Processor -- start 2023-11-16T16:48:56.690 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.682+0000 7fdf10a68b80 1 -- start start 2023-11-16T16:48:56.691 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.682+0000 7fdf10a68b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e03658b60 0x564e03658f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:56.691 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.682+0000 7fdf10a68b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e03659470 0x564e0365d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:56.691 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.682+0000 7fdf10a68b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e0365d8b0 0x564e0365dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:56.691 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.682+0000 7fdf10a68b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e0355e060 con 0x564e03659470 2023-11-16T16:48:56.691 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.682+0000 7fdf10a68b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e035b08a0 con 0x564e03658b60 2023-11-16T16:48:56.691 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.682+0000 7fdf10a68b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e035a9d30 con 0x564e0365d8b0 2023-11-16T16:48:56.691 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.682+0000 7fdefdd9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e03659470 0x564e0365d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:56.691 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.682+0000 7fdefdd9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e03659470 0x564e0365d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46942/0 (socket says 172.21.15.1:46942) 2023-11-16T16:48:56.692 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.682+0000 7fdefdd9b700 1 -- 172.21.15.1:0/379997437 learned_addr learned my addr 172.21.15.1:0/379997437 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:56.692 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.682+0000 7fdefed9d700 1 --2- 172.21.15.1:0/379997437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e0365d8b0 0x564e0365dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:56.692 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.682+0000 7fdefe59c700 1 --2- 172.21.15.1:0/379997437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e03658b60 0x564e03658f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:56.692 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.682+0000 7fdefdd9b700 1 -- 172.21.15.1:0/379997437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e0365d8b0 msgr2=0x564e0365dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:56.692 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.682+0000 7fdefdd9b700 1 --2- 172.21.15.1:0/379997437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e0365d8b0 0x564e0365dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.692 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.682+0000 7fdefdd9b700 1 -- 172.21.15.1:0/379997437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e03658b60 msgr2=0x564e03658f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:56.692 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.682+0000 7fdefdd9b700 1 --2- 172.21.15.1:0/379997437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e03658b60 0x564e03658f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.692 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.682+0000 7fdefdd9b700 1 -- 172.21.15.1:0/379997437 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e0342b630 con 0x564e03659470 2023-11-16T16:48:56.693 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.682+0000 7fdefdd9b700 1 --2- 172.21.15.1:0/379997437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e03659470 0x564e0365d370 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fdee8099bc0 tx=0x7fdee8099840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd96d854fe5574f server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:56.693 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.683+0000 7fdef4ff9700 1 -- 172.21.15.1:0/379997437 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee809dc60 con 0x564e03659470 2023-11-16T16:48:56.693 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.683+0000 7fdef4ff9700 1 -- 172.21.15.1:0/379997437 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdee80a2080 con 0x564e03659470 2023-11-16T16:48:56.693 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.683+0000 7fdef4ff9700 1 -- 172.21.15.1:0/379997437 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee80b2420 con 0x564e03659470 2023-11-16T16:48:56.693 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.683+0000 7fdf10a68b80 1 -- 172.21.15.1:0/379997437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e03659470 msgr2=0x564e0365d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:56.693 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.683+0000 7fdf10a68b80 1 --2- 172.21.15.1:0/379997437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e03659470 0x564e0365d370 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fdee8099bc0 tx=0x7fdee8099840 comp rx=0 tx=0).stop 2023-11-16T16:48:56.693 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.683+0000 7fdf10a68b80 1 -- 172.21.15.1:0/379997437 shutdown_connections 2023-11-16T16:48:56.693 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.683+0000 7fdf10a68b80 1 --2- 172.21.15.1:0/379997437 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e0365d8b0 0x564e0365dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.694 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.683+0000 7fdf10a68b80 1 --2- 172.21.15.1:0/379997437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e03658b60 0x564e03658f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.694 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.683+0000 7fdf10a68b80 1 --2- 172.21.15.1:0/379997437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e03659470 0x564e0365d370 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.694 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.683+0000 7fdf10a68b80 1 -- 172.21.15.1:0/379997437 >> 172.21.15.1:0/379997437 conn(0x564e0364e360 msgr2=0x564e03653b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:56.694 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.684+0000 7fdf10a68b80 1 -- 172.21.15.1:0/379997437 shutdown_connections 2023-11-16T16:48:56.694 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.684+0000 7fdf10a68b80 1 -- 172.21.15.1:0/379997437 wait complete. 2023-11-16T16:48:56.694 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.684+0000 7fdf10a68b80 1 -- start start 2023-11-16T16:48:56.694 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.684+0000 7fdf10a68b80 1 Processor -- start 2023-11-16T16:48:56.694 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.684+0000 7fdf10a68b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e03658b60 0x564e03665070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:56.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.684+0000 7fdf10a68b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e03659470 0x564e03662360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:56.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.684+0000 7fdf10a68b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e0365d8b0 0x564e03662a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:56.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.684+0000 7fdf10a68b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e0354b8e0 con 0x564e0365d8b0 2023-11-16T16:48:56.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.684+0000 7fdf10a68b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e0354b580 con 0x564e03658b60 2023-11-16T16:48:56.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.684+0000 7fdf10a68b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e0342d920 con 0x564e03659470 2023-11-16T16:48:56.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.684+0000 7fdf10a68b80 10 client.0 _ll_register_callbacks cb 0x564e03569780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:48:56.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.684+0000 7fdefe59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e03658b60 0x564e03665070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:56.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.684+0000 7fdefed9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e0365d8b0 0x564e03662a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:56.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.684+0000 7fdefe59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e03658b60 0x564e03665070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.1:41132/0 (socket says 172.21.15.1:41132) 2023-11-16T16:48:56.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.684+0000 7fdefe59c700 1 -- 172.21.15.1:0/658120408 learned_addr learned my addr 172.21.15.1:0/658120408 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:56.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.685+0000 7fdefdd9b700 1 --2- 172.21.15.1:0/658120408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e03659470 0x564e03662360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:56.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.685+0000 7fdefe59c700 1 -- 172.21.15.1:0/658120408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e03659470 msgr2=0x564e03662360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:56.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.685+0000 7fdefe59c700 1 --2- 172.21.15.1:0/658120408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e03659470 0x564e03662360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.685+0000 7fdefe59c700 1 -- 172.21.15.1:0/658120408 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e0365d8b0 msgr2=0x564e03662a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:56.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.685+0000 7fdefe59c700 1 --2- 172.21.15.1:0/658120408 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e0365d8b0 0x564e03662a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.685+0000 7fdefe59c700 1 -- 172.21.15.1:0/658120408 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf00110350 con 0x564e03658b60 2023-11-16T16:48:56.697 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.685+0000 7fdefed9d700 1 --2- 172.21.15.1:0/658120408 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e0365d8b0 0x564e03662a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:48:56.697 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.685+0000 7fdefe59c700 1 --2- 172.21.15.1:0/658120408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e03658b60 0x564e03665070 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdef00127f0 tx=0x7fdef00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1be217fdc1a247ad server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:56.697 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.685+0000 7fdee5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-11-16T16:48:56.697 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.685+0000 7fdee5ffb700 1 -- 172.21.15.1:0/658120408 <== mon.2 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef0011a90 con 0x564e03658b60 2023-11-16T16:48:56.697 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.685+0000 7fdf10a68b80 1 -- 172.21.15.1:0/658120408 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564e0364bb20 con 0x564e03658b60 2023-11-16T16:48:56.697 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.685+0000 7fdee5ffb700 1 -- 172.21.15.1:0/658120408 <== mon.2 v2:172.21.15.26:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fdef0018070 con 0x564e03658b60 2023-11-16T16:48:56.697 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.685+0000 7fdee5ffb700 1 -- 172.21.15.1:0/658120408 <== mon.2 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdef0020840 con 0x564e03658b60 2023-11-16T16:48:56.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.685+0000 7fdf10a68b80 10 client.26315 fetch_fsmap learned FSMap version 32 2023-11-16T16:48:56.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.685+0000 7fdf10a68b80 1 -- 172.21.15.1:0/658120408 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564e03570190 con 0x564e03658b60 2023-11-16T16:48:56.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.686+0000 7fdee5ffb700 1 -- 172.21.15.1:0/658120408 <== mon.2 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564e0364bb20 con 0x564e03658b60 2023-11-16T16:48:56.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.686+0000 7fdee5ffb700 1 -- 172.21.15.1:0/658120408 <== mon.2 v2:172.21.15.26:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fdef0032bf0 con 0x564e03658b60 2023-11-16T16:48:56.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.686+0000 7fdf10a68b80 10 client.26315 fetch_fsmap finished waiting for FSMap version 32 2023-11-16T16:48:56.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.686+0000 7fdf10a68b80 10 client.26315 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:48:56.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.686+0000 7fdf10a68b80 20 client.26315 populate_metadata read hostname 'smithi001' 2023-11-16T16:48:56.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.686+0000 7fdf10a68b80 1 --2- 172.21.15.1:0/658120408 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x564e03672870 0x564e03674c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:56.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.686+0000 7fdf10a68b80 4 client.26315 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:48:56.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.686+0000 7fdf10a68b80 1 -- 172.21.15.1:0/658120408 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e0356c0a0 con 0x564e03672870 2023-11-16T16:48:56.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.686+0000 7fdefdd9b700 1 --2- 172.21.15.1:0/658120408 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x564e03672870 0x564e03674c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:56.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.687+0000 7fdee5ffb700 10 client.26315 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:48:56.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.687+0000 7fdefdd9b700 1 --2- 172.21.15.1:0/658120408 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x564e03672870 0x564e03674c80 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e85eba6924c95dd7 server_cookie=4f42b5608c1a274b in_seq=0 out_seq=0 2023-11-16T16:48:56.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.687+0000 7fdee5ffb700 1 -- 172.21.15.1:0/658120408 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564e0356c0a0 con 0x564e03672870 2023-11-16T16:48:56.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.687+0000 7fdee5ffb700 10 client.26315 handle_command_reply: tid=0 2023-11-16T16:48:56.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.687+0000 7fdf10a68b80 1 client.26315 shutdown 2023-11-16T16:48:56.700 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.688+0000 7fdf10a68b80 1 -- 172.21.15.1:0/658120408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e03658b60 msgr2=0x564e03665070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:56.700 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.688+0000 7fdf10a68b80 1 --2- 172.21.15.1:0/658120408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e03658b60 0x564e03665070 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdef00127f0 tx=0x7fdef00047c0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.700 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.688+0000 7fdf10a68b80 1 -- 172.21.15.1:0/658120408 shutdown_connections 2023-11-16T16:48:56.700 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.688+0000 7fdf10a68b80 1 --2- 172.21.15.1:0/658120408 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x564e03672870 0x564e03674c80 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.700 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.688+0000 7fdf10a68b80 1 --2- 172.21.15.1:0/658120408 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e0365d8b0 0x564e03662a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.700 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.688+0000 7fdf10a68b80 1 --2- 172.21.15.1:0/658120408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e03658b60 0x564e03665070 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.700 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.688+0000 7fdf10a68b80 1 --2- 172.21.15.1:0/658120408 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e03659470 0x564e03662360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.700 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.688+0000 7fdee5ffb700 0 client.26315 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:48:56.701 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.688+0000 7fdf10a68b80 1 -- 172.21.15.1:0/658120408 >> 172.21.15.1:0/658120408 conn(0x564e0364e360 msgr2=0x564e03652de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:56.701 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.688+0000 7fdf10a68b80 1 -- 172.21.15.1:0/658120408 shutdown_connections 2023-11-16T16:48:56.701 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.688+0000 7fdf10a68b80 1 -- 172.21.15.1:0/658120408 wait complete. 2023-11-16T16:48:56.701 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.688+0000 7fdf10a68b80 20 client.26315 trim_cache size 0 max 16384 2023-11-16T16:48:56.711 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.705+0000 7fdf10a68b80 1 Processor -- start 2023-11-16T16:48:56.711 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.705+0000 7fdf10a68b80 1 -- start start 2023-11-16T16:48:56.712 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.705+0000 7fdf10a68b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e03658b60 0x564e03660510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:56.712 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.705+0000 7fdf10a68b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e03659470 0x564e036699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:56.712 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.705+0000 7fdf10a68b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e0365d8b0 0x564e03664c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:56.712 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.705+0000 7fdf10a68b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e0364dee0 con 0x564e03658b60 2023-11-16T16:48:56.712 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.705+0000 7fdf10a68b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e0365c490 con 0x564e0365d8b0 2023-11-16T16:48:56.712 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.705+0000 7fdf10a68b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e0365eb50 con 0x564e03659470 2023-11-16T16:48:56.712 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.705+0000 7fdefe59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e03658b60 0x564e03660510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:56.713 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.705+0000 7fdefe59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e03658b60 0x564e03660510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46954/0 (socket says 172.21.15.1:46954) 2023-11-16T16:48:56.713 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.705+0000 7fdefe59c700 1 -- 172.21.15.1:0/374438260 learned_addr learned my addr 172.21.15.1:0/374438260 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:56.713 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.705+0000 7fdefdd9b700 1 --2- 172.21.15.1:0/374438260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e03659470 0x564e036699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:56.713 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.705+0000 7fdefed9d700 1 --2- 172.21.15.1:0/374438260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e0365d8b0 0x564e03664c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:56.713 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.705+0000 7fdefe59c700 1 -- 172.21.15.1:0/374438260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e03659470 msgr2=0x564e036699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:56.713 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.705+0000 7fdefe59c700 1 --2- 172.21.15.1:0/374438260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e03659470 0x564e036699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.713 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.705+0000 7fdefe59c700 1 -- 172.21.15.1:0/374438260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e0365d8b0 msgr2=0x564e03664c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:56.713 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.705+0000 7fdefe59c700 1 --2- 172.21.15.1:0/374438260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e0365d8b0 0x564e03664c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.705+0000 7fdefe59c700 1 -- 172.21.15.1:0/374438260 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e0365f0c0 con 0x564e03658b60 2023-11-16T16:48:56.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.705+0000 7fdefed9d700 1 --2- 172.21.15.1:0/374438260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e0365d8b0 0x564e03664c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:48:56.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.706+0000 7fdefe59c700 1 --2- 172.21.15.1:0/374438260 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e03658b60 0x564e03660510 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fdef0000c00 tx=0x7fdef0012de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5c20dc937288e51 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:56.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.706+0000 7fdef57fa700 1 -- 172.21.15.1:0/374438260 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef0004220 con 0x564e03658b60 2023-11-16T16:48:56.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.706+0000 7fdef57fa700 1 -- 172.21.15.1:0/374438260 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdef0016070 con 0x564e03658b60 2023-11-16T16:48:56.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.706+0000 7fdef57fa700 1 -- 172.21.15.1:0/374438260 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef00038e0 con 0x564e03658b60 2023-11-16T16:48:56.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.706+0000 7fdf10a68b80 1 -- 172.21.15.1:0/374438260 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e03658b60 msgr2=0x564e03660510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:56.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.706+0000 7fdf10a68b80 1 --2- 172.21.15.1:0/374438260 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e03658b60 0x564e03660510 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fdef0000c00 tx=0x7fdef0012de0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.706+0000 7fdf10a68b80 1 -- 172.21.15.1:0/374438260 shutdown_connections 2023-11-16T16:48:56.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.706+0000 7fdf10a68b80 1 --2- 172.21.15.1:0/374438260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e0365d8b0 0x564e03664c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.706+0000 7fdf10a68b80 1 --2- 172.21.15.1:0/374438260 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e03658b60 0x564e03660510 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.706+0000 7fdf10a68b80 1 --2- 172.21.15.1:0/374438260 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e03659470 0x564e036699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.706+0000 7fdf10a68b80 1 -- 172.21.15.1:0/374438260 >> 172.21.15.1:0/374438260 conn(0x564e0364e360 msgr2=0x564e0365e440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:56.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.706+0000 7fdf10a68b80 1 -- 172.21.15.1:0/374438260 shutdown_connections 2023-11-16T16:48:56.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.706+0000 7fdf10a68b80 1 -- 172.21.15.1:0/374438260 wait complete. 2023-11-16T16:48:56.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.707+0000 7fdf10a68b80 1 -- start start 2023-11-16T16:48:56.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.707+0000 7fdf10a68b80 1 Processor -- start 2023-11-16T16:48:56.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.707+0000 7fdf10a68b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e03658b60 0x564e03661e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:56.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.707+0000 7fdf10a68b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e03659470 0x564e036623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:56.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.707+0000 7fdf10a68b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e0365d8b0 0x564e03660b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:56.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.707+0000 7fdf10a68b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e03661050 con 0x564e03659470 2023-11-16T16:48:56.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.707+0000 7fdf10a68b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e036611c0 con 0x564e0365d8b0 2023-11-16T16:48:56.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.707+0000 7fdf10a68b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e03661380 con 0x564e03658b60 2023-11-16T16:48:56.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.707+0000 7fdf10a68b80 10 client.0 _ll_register_callbacks cb 0x7fdef0011f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:48:56.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.707+0000 7fdefdd9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e03659470 0x564e036623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:56.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.707+0000 7fdefdd9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e03659470 0x564e036623a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46962/0 (socket says 172.21.15.1:46962) 2023-11-16T16:48:56.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.707+0000 7fdefdd9b700 1 -- 172.21.15.1:0/1766588042 learned_addr learned my addr 172.21.15.1:0/1766588042 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:56.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.707+0000 7fdefe59c700 1 --2- 172.21.15.1:0/1766588042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e03658b60 0x564e03661e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:56.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.707+0000 7fdefed9d700 1 --2- 172.21.15.1:0/1766588042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e0365d8b0 0x564e03660b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:56.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.708+0000 7fdefdd9b700 1 -- 172.21.15.1:0/1766588042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e03658b60 msgr2=0x564e03661e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:56.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.708+0000 7fdefdd9b700 1 --2- 172.21.15.1:0/1766588042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e03658b60 0x564e03661e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.708+0000 7fdefdd9b700 1 -- 172.21.15.1:0/1766588042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e0365d8b0 msgr2=0x564e03660b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:56.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.708+0000 7fdefdd9b700 1 --2- 172.21.15.1:0/1766588042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e0365d8b0 0x564e03660b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.708+0000 7fdefdd9b700 1 -- 172.21.15.1:0/1766588042 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e03570190 con 0x564e03659470 2023-11-16T16:48:56.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.708+0000 7fdefdd9b700 1 --2- 172.21.15.1:0/1766588042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e03659470 0x564e036623a0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fdee8005fd0 tx=0x7fdee8099c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c0c9645cc8a0321 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:56.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.708+0000 7fdee5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:48:56.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.708+0000 7fdee5ffb700 1 -- 172.21.15.1:0/1766588042 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee8063070 con 0x564e03659470 2023-11-16T16:48:56.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.708+0000 7fdee5ffb700 1 -- 172.21.15.1:0/1766588042 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fdee80ca030 con 0x564e03659470 2023-11-16T16:48:56.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.708+0000 7fdf10a68b80 1 -- 172.21.15.1:0/1766588042 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdee80a2080 con 0x564e03659470 2023-11-16T16:48:56.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.708+0000 7fdee5ffb700 1 -- 172.21.15.1:0/1766588042 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdee8097fd0 con 0x564e03659470 2023-11-16T16:48:56.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.708+0000 7fdf10a68b80 10 client.16896 fetch_fsmap learned FSMap version 32 2023-11-16T16:48:56.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.708+0000 7fdf10a68b80 1 -- 172.21.15.1:0/1766588042 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564e033d7de0 con 0x564e03659470 2023-11-16T16:48:56.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.708+0000 7fdee5ffb700 1 -- 172.21.15.1:0/1766588042 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdee80a2080 con 0x564e03659470 2023-11-16T16:48:56.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.708+0000 7fdee5ffb700 1 -- 172.21.15.1:0/1766588042 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fdee800f700 con 0x564e03659470 2023-11-16T16:48:56.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.708+0000 7fdf10a68b80 10 client.16896 fetch_fsmap finished waiting for FSMap version 32 2023-11-16T16:48:56.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.708+0000 7fdf10a68b80 10 client.16896 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:48:56.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.708+0000 7fdf10a68b80 20 client.16896 populate_metadata read hostname 'smithi001' 2023-11-16T16:48:56.720 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.708+0000 7fdf10a68b80 1 --2- 172.21.15.1:0/1766588042 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x564e036c7200 0x564e036c9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:56.720 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.708+0000 7fdf10a68b80 4 client.16896 mds_command: new command op to 24521 tid=0[{"prefix": "scrub status"}] 2023-11-16T16:48:56.720 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.709+0000 7fdf10a68b80 1 -- 172.21.15.1:0/1766588042 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564e03578660 con 0x564e036c7200 2023-11-16T16:48:56.720 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.709+0000 7fdefe59c700 1 --2- 172.21.15.1:0/1766588042 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x564e036c7200 0x564e036c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:56.720 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.709+0000 7fdee5ffb700 10 client.16896 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:48:56.720 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.709+0000 7fdefe59c700 1 --2- 172.21.15.1:0/1766588042 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x564e036c7200 0x564e036c9610 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6274ad942a8e9604 server_cookie=56b732ed5005a0b3 in_seq=0 out_seq=0 2023-11-16T16:48:56.720 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.709+0000 7fdee5ffb700 1 -- 172.21.15.1:0/1766588042 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x564e03578660 con 0x564e036c7200 2023-11-16T16:48:56.720 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.709+0000 7fdee5ffb700 10 client.16896 handle_command_reply: tid=0 2023-11-16T16:48:56.721 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.709+0000 7fdf10a68b80 1 client.16896 shutdown 2023-11-16T16:48:56.721 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.709+0000 7fdf10a68b80 1 -- 172.21.15.1:0/1766588042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e03659470 msgr2=0x564e036623a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:56.721 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.709+0000 7fdf10a68b80 1 --2- 172.21.15.1:0/1766588042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e03659470 0x564e036623a0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fdee8005fd0 tx=0x7fdee8099c30 comp rx=0 tx=0).stop 2023-11-16T16:48:56.721 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.709+0000 7fdf10a68b80 1 -- 172.21.15.1:0/1766588042 shutdown_connections 2023-11-16T16:48:56.721 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.709+0000 7fdf10a68b80 1 --2- 172.21.15.1:0/1766588042 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x564e036c7200 0x564e036c9610 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.721 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.709+0000 7fdf10a68b80 1 --2- 172.21.15.1:0/1766588042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564e0365d8b0 0x564e03660b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.721 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.709+0000 7fdf10a68b80 1 --2- 172.21.15.1:0/1766588042 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564e03658b60 0x564e03661e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.721 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.709+0000 7fdee5ffb700 0 client.16896 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:48:56.722 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.709+0000 7fdf10a68b80 1 --2- 172.21.15.1:0/1766588042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e03659470 0x564e036623a0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.722 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.709+0000 7fdf10a68b80 1 -- 172.21.15.1:0/1766588042 >> 172.21.15.1:0/1766588042 conn(0x564e0364e360 msgr2=0x564e0365ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:56.722 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.710+0000 7fdf10a68b80 1 -- 172.21.15.1:0/1766588042 shutdown_connections 2023-11-16T16:48:56.722 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.710+0000 7fdf10a68b80 1 -- 172.21.15.1:0/1766588042 wait complete. 2023-11-16T16:48:56.722 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.710+0000 7fdf10a68b80 20 client.16896 trim_cache size 0 max 16384 2023-11-16T16:48:56.722 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:48:56.723 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2023-11-16T16:48:56.723 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2023-11-16T16:48:56.723 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:48:56.724 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.710+0000 7fdf048dc700 1 -- 172.21.15.1:0/1949443890 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fdeec0a12c0 msgr2=0x7fdeec0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:56.724 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.710+0000 7fdf048dc700 1 --2- 172.21.15.1:0/1949443890 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fdeec0a12c0 0x7fdeec0a3770 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdef0009bf0 tx=0x7fdef0009260 comp rx=0 tx=0).stop 2023-11-16T16:48:56.724 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.710+0000 7fdf048dc700 1 -- 172.21.15.1:0/1949443890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf001006b0 msgr2=0x7fdf0010cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:56.724 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.710+0000 7fdf048dc700 1 --2- 172.21.15.1:0/1949443890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf001006b0 0x7fdf0010cf70 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fdee800c9b0 tx=0x7fdee800ccc0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.724 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.710+0000 7fdf048dc700 1 -- 172.21.15.1:0/1949443890 shutdown_connections 2023-11-16T16:48:56.725 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.710+0000 7fdf048dc700 1 --2- 172.21.15.1:0/1949443890 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fdeec0a12c0 0x7fdeec0a3770 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.725 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.710+0000 7fdf048dc700 1 --2- 172.21.15.1:0/1949443890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf00104550 0x7fdf001a4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.725 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.710+0000 7fdf048dc700 1 --2- 172.21.15.1:0/1949443890 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fdf000ffd10 0x7fdf0010ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.725 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.710+0000 7fdf048dc700 1 --2- 172.21.15.1:0/1949443890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf001006b0 0x7fdf0010cf70 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.725 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.710+0000 7fdf048dc700 1 -- 172.21.15.1:0/1949443890 >> 172.21.15.1:0/1949443890 conn(0x7fdf000747b0 msgr2=0x7fdf000fc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:56.725 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.710+0000 7fdf048dc700 1 -- 172.21.15.1:0/1949443890 shutdown_connections 2023-11-16T16:48:56.725 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.710+0000 7fdf048dc700 1 -- 172.21.15.1:0/1949443890 wait complete. 2023-11-16T16:48:56.726 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-16T16:48:56.726 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:48:56.851 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.850+0000 7f6b18c1d700 1 -- 172.21.15.1:0/1892957164 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b140ffdd0 msgr2=0x7f6b14103d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:56.851 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.850+0000 7f6b18c1d700 1 --2- 172.21.15.1:0/1892957164 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b140ffdd0 0x7f6b14103d90 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f6b080099c0 tx=0x7f6b08009cd0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.850+0000 7f6b18c1d700 1 -- 172.21.15.1:0/1892957164 shutdown_connections 2023-11-16T16:48:56.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.850+0000 7f6b18c1d700 1 --2- 172.21.15.1:0/1892957164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b14104470 0x7f6b1410c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.850+0000 7f6b18c1d700 1 --2- 172.21.15.1:0/1892957164 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b140ff430 0x7f6b140ff800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.850+0000 7f6b18c1d700 1 --2- 172.21.15.1:0/1892957164 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b140ffdd0 0x7f6b14103d90 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.850+0000 7f6b18c1d700 1 -- 172.21.15.1:0/1892957164 >> 172.21.15.1:0/1892957164 conn(0x7f6b14077060 msgr2=0x7f6b14077460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:56.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.850+0000 7f6b18c1d700 1 -- 172.21.15.1:0/1892957164 shutdown_connections 2023-11-16T16:48:56.852 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.850+0000 7f6b18c1d700 1 -- 172.21.15.1:0/1892957164 wait complete. 2023-11-16T16:48:56.854 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.850+0000 7f6b18c1d700 1 Processor -- start 2023-11-16T16:48:56.855 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.851+0000 7f6b18c1d700 1 -- start start 2023-11-16T16:48:56.855 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.851+0000 7f6b18c1d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b140ff430 0x7f6b14195320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:56.855 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.851+0000 7f6b18c1d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b140ffdd0 0x7f6b14195860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:56.855 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.851+0000 7f6b18c1d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b14104470 0x7f6b14199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:56.855 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.851+0000 7f6b18c1d700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1419a300 con 0x7f6b140ff430 2023-11-16T16:48:56.855 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.851+0000 7f6b18c1d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1419a470 con 0x7f6b140ffdd0 2023-11-16T16:48:56.855 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.851+0000 7f6b18c1d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1419a630 con 0x7f6b14104470 2023-11-16T16:48:56.856 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.851+0000 7f6b1259c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b140ff430 0x7f6b14195320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:56.856 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.851+0000 7f6b1259c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b140ff430 0x7f6b14195320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46974/0 (socket says 172.21.15.1:46974) 2023-11-16T16:48:56.856 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.851+0000 7f6b1259c700 1 -- 172.21.15.1:0/4257917126 learned_addr learned my addr 172.21.15.1:0/4257917126 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:56.856 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.851+0000 7f6b12d9d700 1 --2- 172.21.15.1:0/4257917126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b14104470 0x7f6b14199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:56.856 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.851+0000 7f6b11d9b700 1 --2- 172.21.15.1:0/4257917126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b140ffdd0 0x7f6b14195860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:56.856 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.851+0000 7f6b1259c700 1 -- 172.21.15.1:0/4257917126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b14104470 msgr2=0x7f6b14199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:56.856 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.851+0000 7f6b1259c700 1 --2- 172.21.15.1:0/4257917126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b14104470 0x7f6b14199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.856 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.851+0000 7f6b1259c700 1 -- 172.21.15.1:0/4257917126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b140ffdd0 msgr2=0x7f6b14195860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:56.857 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.851+0000 7f6b1259c700 1 --2- 172.21.15.1:0/4257917126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b140ffdd0 0x7f6b14195860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:56.857 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.851+0000 7f6b1259c700 1 -- 172.21.15.1:0/4257917126 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b080095f0 con 0x7f6b140ff430 2023-11-16T16:48:56.857 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.851+0000 7f6b12d9d700 1 --2- 172.21.15.1:0/4257917126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b14104470 0x7f6b14199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:48:56.857 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.851+0000 7f6b1259c700 1 --2- 172.21.15.1:0/4257917126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b140ff430 0x7f6b14195320 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f6b0400e9a0 tx=0x7f6b0400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:56.857 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.851+0000 7f6b037fe700 1 -- 172.21.15.1:0/4257917126 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b04009790 con 0x7f6b140ff430 2023-11-16T16:48:56.857 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.852+0000 7f6b18c1d700 1 -- 172.21.15.1:0/4257917126 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b1410f1f0 con 0x7f6b140ff430 2023-11-16T16:48:56.857 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.852+0000 7f6b037fe700 1 -- 172.21.15.1:0/4257917126 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b0400cbd0 con 0x7f6b140ff430 2023-11-16T16:48:56.857 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.852+0000 7f6b037fe700 1 -- 172.21.15.1:0/4257917126 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b04018680 con 0x7f6b140ff430 2023-11-16T16:48:56.858 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.852+0000 7f6b18c1d700 1 -- 172.21.15.1:0/4257917126 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b1410f060 con 0x7f6b140ff430 2023-11-16T16:48:56.858 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.852+0000 7f6b18c1d700 1 -- 172.21.15.1:0/4257917126 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b1410f380 con 0x7f6b140ff430 2023-11-16T16:48:56.858 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.853+0000 7f6b037fe700 1 -- 172.21.15.1:0/4257917126 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f6b0400cd40 con 0x7f6b140ff430 2023-11-16T16:48:56.859 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.854+0000 7f6b037fe700 1 --2- 172.21.15.1:0/4257917126 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f6afc0a12c0 0x7f6afc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:56.859 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.854+0000 7f6b11d9b700 1 --2- 172.21.15.1:0/4257917126 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f6afc0a12c0 0x7f6afc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:56.859 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.854+0000 7f6b037fe700 1 -- 172.21.15.1:0/4257917126 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f6b04014070 con 0x7f6b140ff430 2023-11-16T16:48:56.859 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.855+0000 7f6b11d9b700 1 --2- 172.21.15.1:0/4257917126 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f6afc0a12c0 0x7f6afc0a3770 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6b08004800 tx=0x7f6b08019040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:56.861 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:56.859+0000 7f6b037fe700 1 -- 172.21.15.1:0/4257917126 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6b04051730 con 0x7f6b140ff430 2023-11-16T16:48:57.106 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.104+0000 7f6b18c1d700 1 -- 172.21.15.1:0/4257917126 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b1419ab00 con 0x7f6b140ff430 2023-11-16T16:48:57.106 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.105+0000 7f6b037fe700 1 -- 172.21.15.1:0/4257917126 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+10775 (secure 0 0 0) 0x7f6b1419ab00 con 0x7f6b140ff430 2023-11-16T16:48:57.107 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:48:57.107 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2587563684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2587563684},{"type":"v1","addr":"172.21.15.1:6835","nonce":2587563684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/277936168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":277936168},{"type":"v1","addr":"172.21.15.1:6837","nonce":277936168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/1172575045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1172575045},{"type":"v1","addr":"172.21.15.1:6839","nonce":1172575045}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/2631175899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2631175899},{"type":"v1","addr":"172.21.15.1:6841","nonce":2631175899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3693905267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3693905267},{"type":"v1","addr":"172.21.15.53:6833","nonce":3693905267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/164668033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":164668033},{"type":"v1","addr":"172.21.15.26:6833","nonce":164668033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-16T16:37:13.918943+0000","modified":"2023-11-16T16:48:39.294587+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2140542192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2140542192},{"type":"v1","addr":"172.21.15.53:6835","nonce":2140542192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1010760275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1010760275},{"type":"v1","addr":"172.21.15.26:6835","nonce":1010760275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3197060302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3197060302},{"type":"v1","addr":"172.21.15.53:6837","nonce":3197060302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6837/1453504863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1453504863},{"type":"v1","addr":"172.21.15.26:6837","nonce":1453504863}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.53:6839/1044753024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1044753024},{"type":"v1","addr":"172.21.15.53:6839","nonce":1044753024}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6839/2192205949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2192205949},{"type":"v1","addr":"172.21.15.26:6839","nonce":2192205949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-16T16:48:57.109 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.107+0000 7f6b18c1d700 1 -- 172.21.15.1:0/4257917126 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f6afc0a12c0 msgr2=0x7f6afc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:57.109 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.107+0000 7f6b18c1d700 1 --2- 172.21.15.1:0/4257917126 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f6afc0a12c0 0x7f6afc0a3770 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6b08004800 tx=0x7f6b08019040 comp rx=0 tx=0).stop 2023-11-16T16:48:57.109 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.107+0000 7f6b18c1d700 1 -- 172.21.15.1:0/4257917126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b140ff430 msgr2=0x7f6b14195320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:57.109 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.107+0000 7f6b18c1d700 1 --2- 172.21.15.1:0/4257917126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b140ff430 0x7f6b14195320 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f6b0400e9a0 tx=0x7f6b0400ed60 comp rx=0 tx=0).stop 2023-11-16T16:48:57.109 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.107+0000 7f6b18c1d700 1 -- 172.21.15.1:0/4257917126 shutdown_connections 2023-11-16T16:48:57.109 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.107+0000 7f6b18c1d700 1 --2- 172.21.15.1:0/4257917126 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f6afc0a12c0 0x7f6afc0a3770 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.110 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.107+0000 7f6b18c1d700 1 --2- 172.21.15.1:0/4257917126 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6b14104470 0x7f6b14199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.110 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.107+0000 7f6b18c1d700 1 --2- 172.21.15.1:0/4257917126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b140ff430 0x7f6b14195320 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.110 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.107+0000 7f6b18c1d700 1 --2- 172.21.15.1:0/4257917126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b140ffdd0 0x7f6b14195860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.110 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.107+0000 7f6b18c1d700 1 -- 172.21.15.1:0/4257917126 >> 172.21.15.1:0/4257917126 conn(0x7f6b14077060 msgr2=0x7f6b140fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:57.110 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.107+0000 7f6b18c1d700 1 -- 172.21.15.1:0/4257917126 shutdown_connections 2023-11-16T16:48:57.110 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.107+0000 7f6b18c1d700 1 -- 172.21.15.1:0/4257917126 wait complete. 2023-11-16T16:48:57.111 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 32 2023-11-16T16:48:57.121 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-11-16T16:48:57.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.247+0000 7f5d2c58f700 1 -- 172.21.15.1:0/3602791729 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d24104140 msgr2=0x7f5d24104510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:57.251 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.247+0000 7f5d2c58f700 1 --2- 172.21.15.1:0/3602791729 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d24104140 0x7f5d24104510 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f5d200099c0 tx=0x7f5d20009cd0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.248+0000 7f5d2c58f700 1 -- 172.21.15.1:0/3602791729 shutdown_connections 2023-11-16T16:48:57.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.248+0000 7f5d2c58f700 1 --2- 172.21.15.1:0/3602791729 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d241002a0 0x7f5d2410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.248+0000 7f5d2c58f700 1 --2- 172.21.15.1:0/3602791729 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d24104140 0x7f5d24104510 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.248+0000 7f5d2c58f700 1 --2- 172.21.15.1:0/3602791729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d240ff8f0 0x7f5d240ffd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.248+0000 7f5d2c58f700 1 -- 172.21.15.1:0/3602791729 >> 172.21.15.1:0/3602791729 conn(0x7f5d24075880 msgr2=0x7f5d24075c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:57.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.248+0000 7f5d2c58f700 1 -- 172.21.15.1:0/3602791729 shutdown_connections 2023-11-16T16:48:57.252 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.248+0000 7f5d2c58f700 1 -- 172.21.15.1:0/3602791729 wait complete. 2023-11-16T16:48:57.253 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.248+0000 7f5d2c58f700 1 Processor -- start 2023-11-16T16:48:57.253 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.248+0000 7f5d2c58f700 1 -- start start 2023-11-16T16:48:57.253 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.248+0000 7f5d2c58f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d240ff8f0 0x7f5d24196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.253 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.248+0000 7f5d2c58f700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d241002a0 0x7f5d241968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.253 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.248+0000 7f5d2c58f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d24104140 0x7f5d2419ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.253 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.248+0000 7f5d2c58f700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2419b350 con 0x7f5d240ff8f0 2023-11-16T16:48:57.253 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.248+0000 7f5d2c58f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2419b4c0 con 0x7f5d24104140 2023-11-16T16:48:57.253 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.248+0000 7f5d2c58f700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2419b680 con 0x7f5d241002a0 2023-11-16T16:48:57.254 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.249+0000 7f5d2a32b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d240ff8f0 0x7f5d24196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.254 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.249+0000 7f5d2a32b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d240ff8f0 0x7f5d24196370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46988/0 (socket says 172.21.15.1:46988) 2023-11-16T16:48:57.254 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.249+0000 7f5d2a32b700 1 -- 172.21.15.1:0/1750570639 learned_addr learned my addr 172.21.15.1:0/1750570639 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:57.254 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.249+0000 7f5d2ab2c700 1 --2- 172.21.15.1:0/1750570639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d24104140 0x7f5d2419ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.254 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.249+0000 7f5d29b2a700 1 --2- 172.21.15.1:0/1750570639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d241002a0 0x7f5d241968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.254 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.249+0000 7f5d2a32b700 1 -- 172.21.15.1:0/1750570639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d241002a0 msgr2=0x7f5d241968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:57.254 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.249+0000 7f5d2a32b700 1 --2- 172.21.15.1:0/1750570639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d241002a0 0x7f5d241968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.254 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.249+0000 7f5d2a32b700 1 -- 172.21.15.1:0/1750570639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d24104140 msgr2=0x7f5d2419ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:57.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.249+0000 7f5d2a32b700 1 --2- 172.21.15.1:0/1750570639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d24104140 0x7f5d2419ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.249+0000 7f5d2a32b700 1 -- 172.21.15.1:0/1750570639 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d200095f0 con 0x7f5d240ff8f0 2023-11-16T16:48:57.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.249+0000 7f5d2a32b700 1 --2- 172.21.15.1:0/1750570639 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d240ff8f0 0x7f5d24196370 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f5d20009990 tx=0x7f5d2000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:57.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.249+0000 7f5d1b7fe700 1 -- 172.21.15.1:0/1750570639 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2001d070 con 0x7f5d240ff8f0 2023-11-16T16:48:57.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.249+0000 7f5d1b7fe700 1 -- 172.21.15.1:0/1750570639 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d200054f0 con 0x7f5d240ff8f0 2023-11-16T16:48:57.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.249+0000 7f5d2c58f700 1 -- 172.21.15.1:0/1750570639 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d2410f200 con 0x7f5d240ff8f0 2023-11-16T16:48:57.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.249+0000 7f5d1b7fe700 1 -- 172.21.15.1:0/1750570639 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d20004ae0 con 0x7f5d240ff8f0 2023-11-16T16:48:57.255 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.249+0000 7f5d2c58f700 1 -- 172.21.15.1:0/1750570639 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d2410f070 con 0x7f5d240ff8f0 2023-11-16T16:48:57.262 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.252+0000 7f5d1b7fe700 1 -- 172.21.15.1:0/1750570639 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f5d2000fb80 con 0x7f5d240ff8f0 2023-11-16T16:48:57.262 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.252+0000 7f5d1b7fe700 1 --2- 172.21.15.1:0/1750570639 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5d100a12c0 0x7f5d100a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.262 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.252+0000 7f5d29b2a700 1 --2- 172.21.15.1:0/1750570639 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5d100a12c0 0x7f5d100a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.262 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.252+0000 7f5d1b7fe700 1 -- 172.21.15.1:0/1750570639 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f5d200cdac0 con 0x7f5d240ff8f0 2023-11-16T16:48:57.262 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.253+0000 7f5d29b2a700 1 --2- 172.21.15.1:0/1750570639 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5d100a12c0 0x7f5d100a3770 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5d14005fd0 tx=0x7f5d1400a560 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:57.263 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.253+0000 7f5d3871bb80 1 Processor -- start 2023-11-16T16:48:57.263 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.253+0000 7f5d3871bb80 1 -- start start 2023-11-16T16:48:57.263 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.253+0000 7f5d3871bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56093a7cbb60 0x56093a7cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.263 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.253+0000 7f5d3871bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56093a7cc470 0x56093a7d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.263 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.253+0000 7f5d3871bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56093a7d08b0 0x56093a7d0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.263 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.253+0000 7f5d3871bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56093a6d1060 con 0x56093a7d08b0 2023-11-16T16:48:57.263 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.253+0000 7f5d3871bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56093a7238a0 con 0x56093a7cc470 2023-11-16T16:48:57.263 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.253+0000 7f5d3871bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56093a71cd30 con 0x56093a7cbb60 2023-11-16T16:48:57.264 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.254+0000 7f5d2ab2c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56093a7d08b0 0x56093a7d0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.264 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.254+0000 7f5d2ab2c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56093a7d08b0 0x56093a7d0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47000/0 (socket says 172.21.15.1:47000) 2023-11-16T16:48:57.264 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.254+0000 7f5d2ab2c700 1 -- 172.21.15.1:0/3278124748 learned_addr learned my addr 172.21.15.1:0/3278124748 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:57.264 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.254+0000 7f5d2a32b700 1 --2- 172.21.15.1:0/3278124748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56093a7cbb60 0x56093a7cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.264 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.254+0000 7f5d29b2a700 1 --2- 172.21.15.1:0/3278124748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56093a7cc470 0x56093a7d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.264 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.254+0000 7f5d2ab2c700 1 -- 172.21.15.1:0/3278124748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56093a7cbb60 msgr2=0x56093a7cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:57.264 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.254+0000 7f5d2ab2c700 1 --2- 172.21.15.1:0/3278124748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56093a7cbb60 0x56093a7cbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.264 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.254+0000 7f5d2ab2c700 1 -- 172.21.15.1:0/3278124748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56093a7cc470 msgr2=0x56093a7d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:57.265 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.254+0000 7f5d2ab2c700 1 --2- 172.21.15.1:0/3278124748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56093a7cc470 0x56093a7d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.265 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.254+0000 7f5d2ab2c700 1 -- 172.21.15.1:0/3278124748 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56093a59e630 con 0x56093a7d08b0 2023-11-16T16:48:57.265 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.254+0000 7f5d2a32b700 1 --2- 172.21.15.1:0/3278124748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56093a7cbb60 0x56093a7cbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:48:57.265 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.254+0000 7f5d2ab2c700 1 --2- 172.21.15.1:0/3278124748 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56093a7d08b0 0x56093a7d0d00 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c011600 tx=0x7f5d1c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5f3aa34288a8672 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:57.265 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.254+0000 7f5d18ff9700 1 -- 172.21.15.1:0/3278124748 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c00e040 con 0x56093a7d08b0 2023-11-16T16:48:57.265 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.254+0000 7f5d18ff9700 1 -- 172.21.15.1:0/3278124748 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d1c00f040 con 0x56093a7d08b0 2023-11-16T16:48:57.265 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.254+0000 7f5d18ff9700 1 -- 172.21.15.1:0/3278124748 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c004760 con 0x56093a7d08b0 2023-11-16T16:48:57.265 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.255+0000 7f5d3871bb80 1 -- 172.21.15.1:0/3278124748 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56093a7d08b0 msgr2=0x56093a7d0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:57.266 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.255+0000 7f5d3871bb80 1 --2- 172.21.15.1:0/3278124748 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56093a7d08b0 0x56093a7d0d00 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c011600 tx=0x7f5d1c009920 comp rx=0 tx=0).stop 2023-11-16T16:48:57.266 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.255+0000 7f5d3871bb80 1 -- 172.21.15.1:0/3278124748 shutdown_connections 2023-11-16T16:48:57.266 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.255+0000 7f5d3871bb80 1 --2- 172.21.15.1:0/3278124748 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56093a7d08b0 0x56093a7d0d00 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.266 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.255+0000 7f5d3871bb80 1 --2- 172.21.15.1:0/3278124748 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56093a7cbb60 0x56093a7cbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.266 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.255+0000 7f5d3871bb80 1 --2- 172.21.15.1:0/3278124748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56093a7cc470 0x56093a7d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.266 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.255+0000 7f5d3871bb80 1 -- 172.21.15.1:0/3278124748 >> 172.21.15.1:0/3278124748 conn(0x56093a7c1360 msgr2=0x56093a7c6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:57.266 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.255+0000 7f5d3871bb80 1 -- 172.21.15.1:0/3278124748 shutdown_connections 2023-11-16T16:48:57.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.255+0000 7f5d3871bb80 1 -- 172.21.15.1:0/3278124748 wait complete. 2023-11-16T16:48:57.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.255+0000 7f5d3871bb80 1 -- start start 2023-11-16T16:48:57.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.256+0000 7f5d3871bb80 1 Processor -- start 2023-11-16T16:48:57.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.256+0000 7f5d3871bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56093a7cbb60 0x56093a7d8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.256+0000 7f5d3871bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56093a7cc470 0x56093a7d5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.256+0000 7f5d3871bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56093a7d08b0 0x56093a7d59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.256+0000 7f5d3871bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56093a6be8e0 con 0x56093a7d08b0 2023-11-16T16:48:57.267 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.256+0000 7f5d3871bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56093a6be580 con 0x56093a7cc470 2023-11-16T16:48:57.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.256+0000 7f5d3871bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56093a5a0920 con 0x56093a7cbb60 2023-11-16T16:48:57.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.256+0000 7f5d3871bb80 10 client.0 _ll_register_callbacks cb 0x56093a6dc780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:48:57.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.256+0000 7f5d2ab2c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56093a7d08b0 0x56093a7d59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.256+0000 7f5d2ab2c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56093a7d08b0 0x56093a7d59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47006/0 (socket says 172.21.15.1:47006) 2023-11-16T16:48:57.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.256+0000 7f5d2ab2c700 1 -- 172.21.15.1:0/255152948 learned_addr learned my addr 172.21.15.1:0/255152948 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:57.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.256+0000 7f5d29b2a700 1 --2- 172.21.15.1:0/255152948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56093a7cc470 0x56093a7d5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.256+0000 7f5d2a32b700 1 --2- 172.21.15.1:0/255152948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56093a7cbb60 0x56093a7d8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.268 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.256+0000 7f5d2ab2c700 1 -- 172.21.15.1:0/255152948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56093a7cbb60 msgr2=0x56093a7d8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:57.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.256+0000 7f5d2ab2c700 1 --2- 172.21.15.1:0/255152948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56093a7cbb60 0x56093a7d8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.256+0000 7f5d2ab2c700 1 -- 172.21.15.1:0/255152948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56093a7cc470 msgr2=0x56093a7d5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:57.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.256+0000 7f5d2ab2c700 1 --2- 172.21.15.1:0/255152948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56093a7cc470 0x56093a7d5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.256+0000 7f5d2ab2c700 1 -- 172.21.15.1:0/255152948 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d1c0112b0 con 0x56093a7d08b0 2023-11-16T16:48:57.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.257+0000 7f5d2ab2c700 1 --2- 172.21.15.1:0/255152948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56093a7d08b0 0x56093a7d59d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c0042a0 tx=0x7f5d1c004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26f6dc0d95f72fab server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:57.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.257+0000 7f5d0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:48:57.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.257+0000 7f5d0dffb700 1 -- 172.21.15.1:0/255152948 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c00e070 con 0x56093a7d08b0 2023-11-16T16:48:57.269 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.257+0000 7f5d3871bb80 1 -- 172.21.15.1:0/255152948 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56093a7beb20 con 0x56093a7d08b0 2023-11-16T16:48:57.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.257+0000 7f5d0dffb700 1 -- 172.21.15.1:0/255152948 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f5d1c004510 con 0x56093a7d08b0 2023-11-16T16:48:57.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.257+0000 7f5d0dffb700 1 -- 172.21.15.1:0/255152948 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d1c0046d0 con 0x56093a7d08b0 2023-11-16T16:48:57.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.257+0000 7f5d3871bb80 10 client.16914 fetch_fsmap learned FSMap version 32 2023-11-16T16:48:57.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.257+0000 7f5d3871bb80 1 -- 172.21.15.1:0/255152948 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56093a6e3190 con 0x56093a7d08b0 2023-11-16T16:48:57.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.257+0000 7f5d0dffb700 1 -- 172.21.15.1:0/255152948 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56093a7beb20 con 0x56093a7d08b0 2023-11-16T16:48:57.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.257+0000 7f5d0dffb700 1 -- 172.21.15.1:0/255152948 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f5d1c027460 con 0x56093a7d08b0 2023-11-16T16:48:57.270 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.257+0000 7f5d3871bb80 10 client.16914 fetch_fsmap finished waiting for FSMap version 32 2023-11-16T16:48:57.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.257+0000 7f5d3871bb80 10 client.16914 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-11-16T16:48:57.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.257+0000 7f5d3871bb80 20 client.16914 populate_metadata read hostname 'smithi001' 2023-11-16T16:48:57.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.257+0000 7f5d3871bb80 1 --2- 172.21.15.1:0/255152948 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x56093a7e58e0 0x56093a7e7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.257+0000 7f5d3871bb80 4 client.16914 mds_command: new command op to 24482 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:48:57.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.257+0000 7f5d3871bb80 1 -- 172.21.15.1:0/255152948 --> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56093a6df0a0 con 0x56093a7e58e0 2023-11-16T16:48:57.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.258+0000 7f5d2a32b700 1 --2- 172.21.15.1:0/255152948 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x56093a7e58e0 0x56093a7e7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.258+0000 7f5d0dffb700 10 client.16914 ms_handle_connect on v2:172.21.15.26:6836/1453504863 2023-11-16T16:48:57.271 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.258+0000 7f5d2a32b700 1 --2- 172.21.15.1:0/255152948 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x56093a7e58e0 0x56093a7e7cf0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6c1e95cde7816eb2 server_cookie=6f9832ba23f5927a in_seq=0 out_seq=0 2023-11-16T16:48:57.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.259+0000 7f5d0dffb700 1 -- 172.21.15.1:0/255152948 <== mds.2 v2:172.21.15.26:6836/1453504863 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56093a6df0a0 con 0x56093a7e58e0 2023-11-16T16:48:57.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.259+0000 7f5d0dffb700 10 client.16914 handle_command_reply: tid=0 2023-11-16T16:48:57.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.259+0000 7f5d3871bb80 1 client.16914 shutdown 2023-11-16T16:48:57.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.259+0000 7f5d3871bb80 1 -- 172.21.15.1:0/255152948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56093a7d08b0 msgr2=0x56093a7d59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:57.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.259+0000 7f5d3871bb80 1 --2- 172.21.15.1:0/255152948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56093a7d08b0 0x56093a7d59d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c0042a0 tx=0x7f5d1c004170 comp rx=0 tx=0).stop 2023-11-16T16:48:57.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.259+0000 7f5d3871bb80 1 -- 172.21.15.1:0/255152948 shutdown_connections 2023-11-16T16:48:57.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.259+0000 7f5d3871bb80 1 --2- 172.21.15.1:0/255152948 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x56093a7e58e0 0x56093a7e7cf0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.272 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.259+0000 7f5d3871bb80 1 --2- 172.21.15.1:0/255152948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56093a7d08b0 0x56093a7d59d0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.259+0000 7f5d0dffb700 0 client.16914 ms_handle_reset on v2:172.21.15.26:6836/1453504863 2023-11-16T16:48:57.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.259+0000 7f5d3871bb80 1 --2- 172.21.15.1:0/255152948 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56093a7cbb60 0x56093a7d8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.259+0000 7f5d3871bb80 1 --2- 172.21.15.1:0/255152948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56093a7cc470 0x56093a7d5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.259+0000 7f5d3871bb80 1 -- 172.21.15.1:0/255152948 >> 172.21.15.1:0/255152948 conn(0x56093a7c1360 msgr2=0x56093a7c5de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:57.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.259+0000 7f5d3871bb80 1 -- 172.21.15.1:0/255152948 shutdown_connections 2023-11-16T16:48:57.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.259+0000 7f5d3871bb80 1 -- 172.21.15.1:0/255152948 wait complete. 2023-11-16T16:48:57.273 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.259+0000 7f5d3871bb80 20 client.16914 trim_cache size 0 max 16384 2023-11-16T16:48:57.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.272+0000 7f5d3871bb80 1 Processor -- start 2023-11-16T16:48:57.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.272+0000 7f5d3871bb80 1 -- start start 2023-11-16T16:48:57.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.272+0000 7f5d3871bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56093a7cbb60 0x56093a7e1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.279 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.272+0000 7f5d3871bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56093a7cc470 0x56093a7dc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.272+0000 7f5d3871bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56093a7d08b0 0x56093a7d7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.272+0000 7f5d3871bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56093a7c81c0 con 0x56093a7cc470 2023-11-16T16:48:57.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.272+0000 7f5d3871bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56093a7cf3c0 con 0x56093a7cbb60 2023-11-16T16:48:57.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.272+0000 7f5d3871bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56093a7cf140 con 0x56093a7d08b0 2023-11-16T16:48:57.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.272+0000 7f5d29b2a700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56093a7cc470 0x56093a7dc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.272+0000 7f5d29b2a700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56093a7cc470 0x56093a7dc9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47012/0 (socket says 172.21.15.1:47012) 2023-11-16T16:48:57.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.272+0000 7f5d29b2a700 1 -- 172.21.15.1:0/2085940892 learned_addr learned my addr 172.21.15.1:0/2085940892 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:57.280 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.273+0000 7f5d2ab2c700 1 --2- 172.21.15.1:0/2085940892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56093a7d08b0 0x56093a7d7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.273+0000 7f5d2a32b700 1 --2- 172.21.15.1:0/2085940892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56093a7cbb60 0x56093a7e1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.273+0000 7f5d29b2a700 1 -- 172.21.15.1:0/2085940892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56093a7d08b0 msgr2=0x56093a7d7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:57.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.273+0000 7f5d29b2a700 1 --2- 172.21.15.1:0/2085940892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56093a7d08b0 0x56093a7d7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.273+0000 7f5d29b2a700 1 -- 172.21.15.1:0/2085940892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56093a7cbb60 msgr2=0x56093a7e1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:57.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.273+0000 7f5d29b2a700 1 --2- 172.21.15.1:0/2085940892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56093a7cbb60 0x56093a7e1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.273+0000 7f5d29b2a700 1 -- 172.21.15.1:0/2085940892 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56093a6e2ee0 con 0x56093a7cc470 2023-11-16T16:48:57.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.273+0000 7f5d29b2a700 1 --2- 172.21.15.1:0/2085940892 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56093a7cc470 0x56093a7dc9a0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f5d14005950 tx=0x7f5d14011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788545eb894f7c71 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:57.281 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.273+0000 7f5d197fa700 1 -- 172.21.15.1:0/2085940892 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d14023070 con 0x56093a7cc470 2023-11-16T16:48:57.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.273+0000 7f5d197fa700 1 -- 172.21.15.1:0/2085940892 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d1400ee70 con 0x56093a7cc470 2023-11-16T16:48:57.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.273+0000 7f5d197fa700 1 -- 172.21.15.1:0/2085940892 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d14005680 con 0x56093a7cc470 2023-11-16T16:48:57.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.274+0000 7f5d3871bb80 1 -- 172.21.15.1:0/2085940892 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56093a7cc470 msgr2=0x56093a7dc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:57.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.274+0000 7f5d3871bb80 1 --2- 172.21.15.1:0/2085940892 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56093a7cc470 0x56093a7dc9a0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f5d14005950 tx=0x7f5d14011000 comp rx=0 tx=0).stop 2023-11-16T16:48:57.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.274+0000 7f5d3871bb80 1 -- 172.21.15.1:0/2085940892 shutdown_connections 2023-11-16T16:48:57.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.274+0000 7f5d3871bb80 1 --2- 172.21.15.1:0/2085940892 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56093a7d08b0 0x56093a7d7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.274+0000 7f5d3871bb80 1 --2- 172.21.15.1:0/2085940892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56093a7cbb60 0x56093a7e1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.282 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.274+0000 7f5d3871bb80 1 --2- 172.21.15.1:0/2085940892 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56093a7cc470 0x56093a7dc9a0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.283 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.274+0000 7f5d3871bb80 1 -- 172.21.15.1:0/2085940892 >> 172.21.15.1:0/2085940892 conn(0x56093a7c1360 msgr2=0x56093a7d2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:57.283 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.274+0000 7f5d3871bb80 1 -- 172.21.15.1:0/2085940892 shutdown_connections 2023-11-16T16:48:57.283 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.274+0000 7f5d3871bb80 1 -- 172.21.15.1:0/2085940892 wait complete. 2023-11-16T16:48:57.283 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.274+0000 7f5d3871bb80 1 -- start start 2023-11-16T16:48:57.283 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.274+0000 7f5d3871bb80 1 Processor -- start 2023-11-16T16:48:57.283 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.274+0000 7f5d3871bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56093a7cbb60 0x56093a7d8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.283 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.274+0000 7f5d3871bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56093a7cc470 0x56093a7d90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.283 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.274+0000 7f5d3871bb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56093a7d08b0 0x56093a838750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.274+0000 7f5d3871bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56093a838c90 con 0x56093a7cbb60 2023-11-16T16:48:57.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.274+0000 7f5d3871bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56093a838dd0 con 0x56093a7cc470 2023-11-16T16:48:57.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.274+0000 7f5d3871bb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x56093a838f90 con 0x56093a7d08b0 2023-11-16T16:48:57.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.274+0000 7f5d3871bb80 10 client.0 _ll_register_callbacks cb 0x7f5d14014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:48:57.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.274+0000 7f5d2a32b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56093a7cbb60 0x56093a7d8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.274+0000 7f5d2a32b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56093a7cbb60 0x56093a7d8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47016/0 (socket says 172.21.15.1:47016) 2023-11-16T16:48:57.284 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.274+0000 7f5d2a32b700 1 -- 172.21.15.1:0/3101841435 learned_addr learned my addr 172.21.15.1:0/3101841435 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:57.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.274+0000 7f5d29b2a700 1 --2- 172.21.15.1:0/3101841435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56093a7cc470 0x56093a7d90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.275+0000 7f5d2ab2c700 1 --2- 172.21.15.1:0/3101841435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56093a7d08b0 0x56093a838750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.275+0000 7f5d2a32b700 1 -- 172.21.15.1:0/3101841435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56093a7d08b0 msgr2=0x56093a838750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:57.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.275+0000 7f5d2a32b700 1 --2- 172.21.15.1:0/3101841435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56093a7d08b0 0x56093a838750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.275+0000 7f5d2a32b700 1 -- 172.21.15.1:0/3101841435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56093a7cc470 msgr2=0x56093a7d90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:57.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.275+0000 7f5d2a32b700 1 --2- 172.21.15.1:0/3101841435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56093a7cc470 0x56093a7d90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.275+0000 7f5d2a32b700 1 -- 172.21.15.1:0/3101841435 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d1400ea90 con 0x56093a7cbb60 2023-11-16T16:48:57.285 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.275+0000 7f5d2a32b700 1 --2- 172.21.15.1:0/3101841435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56093a7cbb60 0x56093a7d8b70 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f5d2000c010 tx=0x7f5d2009a870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c0812e25c011408 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:57.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.275+0000 7f5d0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:48:57.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.275+0000 7f5d0dffb700 1 -- 172.21.15.1:0/3101841435 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2009eea0 con 0x56093a7cbb60 2023-11-16T16:48:57.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.275+0000 7f5d0dffb700 1 -- 172.21.15.1:0/3101841435 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f5d200a2fd0 con 0x56093a7cbb60 2023-11-16T16:48:57.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.275+0000 7f5d3871bb80 1 -- 172.21.15.1:0/3101841435 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d1c00f040 con 0x56093a7cbb60 2023-11-16T16:48:57.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.275+0000 7f5d0dffb700 1 -- 172.21.15.1:0/3101841435 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d2009ae70 con 0x56093a7cbb60 2023-11-16T16:48:57.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.275+0000 7f5d3871bb80 10 client.16920 fetch_fsmap learned FSMap version 32 2023-11-16T16:48:57.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.275+0000 7f5d3871bb80 1 -- 172.21.15.1:0/3101841435 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56093a533c60 con 0x56093a7cbb60 2023-11-16T16:48:57.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.275+0000 7f5d0dffb700 1 -- 172.21.15.1:0/3101841435 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d1c00f040 con 0x56093a7cbb60 2023-11-16T16:48:57.286 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.276+0000 7f5d0dffb700 1 -- 172.21.15.1:0/3101841435 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f5d200a2b40 con 0x56093a7cbb60 2023-11-16T16:48:57.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.276+0000 7f5d3871bb80 10 client.16920 fetch_fsmap finished waiting for FSMap version 32 2023-11-16T16:48:57.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.276+0000 7f5d3871bb80 10 client.16920 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-11-16T16:48:57.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.276+0000 7f5d3871bb80 20 client.16920 populate_metadata read hostname 'smithi001' 2023-11-16T16:48:57.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.276+0000 7f5d3871bb80 1 --2- 172.21.15.1:0/3101841435 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x56093a83a2d0 0x56093a83c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.276+0000 7f5d3871bb80 4 client.16920 mds_command: new command op to 24482 tid=0[{"prefix": "damage ls"}] 2023-11-16T16:48:57.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.276+0000 7f5d3871bb80 1 -- 172.21.15.1:0/3101841435 --> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56093a6eb660 con 0x56093a83a2d0 2023-11-16T16:48:57.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.276+0000 7f5d29b2a700 1 --2- 172.21.15.1:0/3101841435 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x56093a83a2d0 0x56093a83c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.287 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.276+0000 7f5d29b2a700 1 --2- 172.21.15.1:0/3101841435 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x56093a83a2d0 0x56093a83c6e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6ba2e9777a9159b2 server_cookie=232b701fe4ef2efa in_seq=0 out_seq=0 2023-11-16T16:48:57.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.276+0000 7f5d0dffb700 10 client.16920 ms_handle_connect on v2:172.21.15.26:6836/1453504863 2023-11-16T16:48:57.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.276+0000 7f5d0dffb700 1 -- 172.21.15.1:0/3101841435 <== mds.2 v2:172.21.15.26:6836/1453504863 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56093a6eb660 con 0x56093a83a2d0 2023-11-16T16:48:57.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.276+0000 7f5d0dffb700 10 client.16920 handle_command_reply: tid=0 2023-11-16T16:48:57.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.276+0000 7f5d3871bb80 1 client.16920 shutdown 2023-11-16T16:48:57.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.277+0000 7f5d3871bb80 1 -- 172.21.15.1:0/3101841435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56093a7cbb60 msgr2=0x56093a7d8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:57.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.277+0000 7f5d3871bb80 1 --2- 172.21.15.1:0/3101841435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56093a7cbb60 0x56093a7d8b70 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f5d2000c010 tx=0x7f5d2009a870 comp rx=0 tx=0).stop 2023-11-16T16:48:57.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.277+0000 7f5d3871bb80 1 -- 172.21.15.1:0/3101841435 shutdown_connections 2023-11-16T16:48:57.288 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.277+0000 7f5d3871bb80 1 --2- 172.21.15.1:0/3101841435 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x56093a83a2d0 0x56093a83c6e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.277+0000 7f5d3871bb80 1 --2- 172.21.15.1:0/3101841435 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x56093a7d08b0 0x56093a838750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.277+0000 7f5d3871bb80 1 --2- 172.21.15.1:0/3101841435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56093a7cbb60 0x56093a7d8b70 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.277+0000 7f5d0dffb700 0 client.16920 ms_handle_reset on v2:172.21.15.26:6836/1453504863 2023-11-16T16:48:57.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.277+0000 7f5d3871bb80 1 --2- 172.21.15.1:0/3101841435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56093a7cc470 0x56093a7d90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.277+0000 7f5d3871bb80 1 -- 172.21.15.1:0/3101841435 >> 172.21.15.1:0/3101841435 conn(0x56093a7c1360 msgr2=0x56093a7d2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:57.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.277+0000 7f5d3871bb80 1 -- 172.21.15.1:0/3101841435 shutdown_connections 2023-11-16T16:48:57.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.277+0000 7f5d3871bb80 1 -- 172.21.15.1:0/3101841435 wait complete. 2023-11-16T16:48:57.289 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.277+0000 7f5d3871bb80 20 client.16920 trim_cache size 0 max 16384 2023-11-16T16:48:57.290 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2023-11-16T16:48:57.291 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.277+0000 7f5d2c58f700 1 -- 172.21.15.1:0/1750570639 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5d100a12c0 msgr2=0x7f5d100a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:57.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.277+0000 7f5d2c58f700 1 --2- 172.21.15.1:0/1750570639 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5d100a12c0 0x7f5d100a3770 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5d14005fd0 tx=0x7f5d1400a560 comp rx=0 tx=0).stop 2023-11-16T16:48:57.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.277+0000 7f5d2c58f700 1 -- 172.21.15.1:0/1750570639 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d240ff8f0 msgr2=0x7f5d24196370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:57.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.277+0000 7f5d2c58f700 1 --2- 172.21.15.1:0/1750570639 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d240ff8f0 0x7f5d24196370 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f5d20009990 tx=0x7f5d2000f690 comp rx=0 tx=0).stop 2023-11-16T16:48:57.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.277+0000 7f5d2c58f700 1 -- 172.21.15.1:0/1750570639 shutdown_connections 2023-11-16T16:48:57.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.277+0000 7f5d2c58f700 1 --2- 172.21.15.1:0/1750570639 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5d100a12c0 0x7f5d100a3770 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.292 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.277+0000 7f5d2c58f700 1 --2- 172.21.15.1:0/1750570639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d24104140 0x7f5d2419ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.293 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.277+0000 7f5d2c58f700 1 --2- 172.21.15.1:0/1750570639 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d240ff8f0 0x7f5d24196370 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.293 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.277+0000 7f5d2c58f700 1 --2- 172.21.15.1:0/1750570639 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5d241002a0 0x7f5d241968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.293 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.277+0000 7f5d2c58f700 1 -- 172.21.15.1:0/1750570639 >> 172.21.15.1:0/1750570639 conn(0x7f5d24075880 msgr2=0x7f5d240fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:57.293 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.277+0000 7f5d2c58f700 1 -- 172.21.15.1:0/1750570639 shutdown_connections 2023-11-16T16:48:57.293 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.277+0000 7f5d2c58f700 1 -- 172.21.15.1:0/1750570639 wait complete. 2023-11-16T16:48:57.293 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-11-16T16:48:57.425 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.420+0000 7f171feb0700 1 -- 172.21.15.1:0/890051175 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1718101200 msgr2=0x7f17180fe7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:57.425 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.420+0000 7f171feb0700 1 --2- 172.21.15.1:0/890051175 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1718101200 0x7f17180fe7e0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f1714009990 tx=0x7f1714009ca0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.425 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.421+0000 7f171feb0700 1 -- 172.21.15.1:0/890051175 shutdown_connections 2023-11-16T16:48:57.425 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.421+0000 7f171feb0700 1 --2- 172.21.15.1:0/890051175 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f171810cf60 0x7f171810f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.425 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.421+0000 7f171feb0700 1 --2- 172.21.15.1:0/890051175 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1718101200 0x7f17180fe7e0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.425 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.421+0000 7f171feb0700 1 --2- 172.21.15.1:0/890051175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1718101660 0x7f17180fed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.425 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.421+0000 7f171feb0700 1 -- 172.21.15.1:0/890051175 >> 172.21.15.1:0/890051175 conn(0x7f1718075880 msgr2=0x7f1718075c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:57.426 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.421+0000 7f171feb0700 1 -- 172.21.15.1:0/890051175 shutdown_connections 2023-11-16T16:48:57.426 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.421+0000 7f171feb0700 1 -- 172.21.15.1:0/890051175 wait complete. 2023-11-16T16:48:57.426 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.421+0000 7f171feb0700 1 Processor -- start 2023-11-16T16:48:57.426 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.421+0000 7f171feb0700 1 -- start start 2023-11-16T16:48:57.426 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.421+0000 7f171feb0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1718101200 0x7f1718198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.426 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.421+0000 7f171feb0700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1718101660 0x7f17181989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.426 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.421+0000 7f171feb0700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f171810cf60 0x7f171819ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.426 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.421+0000 7f171feb0700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171819d450 con 0x7f1718101200 2023-11-16T16:48:57.427 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.421+0000 7f171feb0700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171819d5c0 con 0x7f171810cf60 2023-11-16T16:48:57.427 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.421+0000 7f171feb0700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171819d780 con 0x7f1718101660 2023-11-16T16:48:57.427 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.422+0000 7f171dc4c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1718101200 0x7f1718198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.427 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.422+0000 7f171dc4c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1718101200 0x7f1718198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47030/0 (socket says 172.21.15.1:47030) 2023-11-16T16:48:57.427 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.422+0000 7f171dc4c700 1 -- 172.21.15.1:0/595858841 learned_addr learned my addr 172.21.15.1:0/595858841 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:57.427 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.422+0000 7f171e44d700 1 --2- 172.21.15.1:0/595858841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f171810cf60 0x7f171819ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.427 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.422+0000 7f171d44b700 1 --2- 172.21.15.1:0/595858841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1718101660 0x7f17181989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.427 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.422+0000 7f171dc4c700 1 -- 172.21.15.1:0/595858841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1718101660 msgr2=0x7f17181989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:57.428 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.422+0000 7f171dc4c700 1 --2- 172.21.15.1:0/595858841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1718101660 0x7f17181989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.428 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.422+0000 7f171dc4c700 1 -- 172.21.15.1:0/595858841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f171810cf60 msgr2=0x7f171819ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:57.428 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.422+0000 7f171dc4c700 1 --2- 172.21.15.1:0/595858841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f171810cf60 0x7f171819ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.428 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.422+0000 7f171dc4c700 1 -- 172.21.15.1:0/595858841 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17140095f0 con 0x7f1718101200 2023-11-16T16:48:57.428 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.422+0000 7f171d44b700 1 --2- 172.21.15.1:0/595858841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1718101660 0x7f17181989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:48:57.428 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.422+0000 7f171e44d700 1 --2- 172.21.15.1:0/595858841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f171810cf60 0x7f171819ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:48:57.428 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.422+0000 7f171dc4c700 1 --2- 172.21.15.1:0/595858841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1718101200 0x7f1718198470 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f171400b2e0 tx=0x7f1714004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:57.428 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.422+0000 7f170affd700 1 -- 172.21.15.1:0/595858841 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171401d070 con 0x7f1718101200 2023-11-16T16:48:57.428 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.422+0000 7f171feb0700 1 -- 172.21.15.1:0/595858841 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1718112490 con 0x7f1718101200 2023-11-16T16:48:57.429 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.422+0000 7f170affd700 1 -- 172.21.15.1:0/595858841 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f171400fbc0 con 0x7f1718101200 2023-11-16T16:48:57.429 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.422+0000 7f170affd700 1 -- 172.21.15.1:0/595858841 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1714017650 con 0x7f1718101200 2023-11-16T16:48:57.429 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.422+0000 7f171feb0700 1 -- 172.21.15.1:0/595858841 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1718112300 con 0x7f1718101200 2023-11-16T16:48:57.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.425+0000 7f170affd700 1 -- 172.21.15.1:0/595858841 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f171400fd30 con 0x7f1718101200 2023-11-16T16:48:57.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.425+0000 7f170affd700 1 --2- 172.21.15.1:0/595858841 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f17040a12c0 0x7f17040a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.426+0000 7f171d44b700 1 --2- 172.21.15.1:0/595858841 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f17040a12c0 0x7f17040a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.426+0000 7f170affd700 1 -- 172.21.15.1:0/595858841 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f17140ce100 con 0x7f1718101200 2023-11-16T16:48:57.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.426+0000 7f171d44b700 1 --2- 172.21.15.1:0/595858841 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f17040a12c0 0x7f17040a3770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f170c005fd0 tx=0x7f170c005ee0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:57.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.426+0000 7f172c03cb80 1 Processor -- start 2023-11-16T16:48:57.434 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.426+0000 7f172c03cb80 1 -- start start 2023-11-16T16:48:57.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.427+0000 7f172c03cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ab5bdcb60 0x564ab5bdcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.427+0000 7f172c03cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ab5bdd470 0x564ab5be1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.427+0000 7f172c03cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ab5be18b0 0x564ab5be1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.427+0000 7f172c03cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ab5ae2060 con 0x564ab5be18b0 2023-11-16T16:48:57.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.427+0000 7f172c03cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ab5b348a0 con 0x564ab5bdd470 2023-11-16T16:48:57.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.427+0000 7f172c03cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ab5b2dd30 con 0x564ab5bdcb60 2023-11-16T16:48:57.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.427+0000 7f171e44d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ab5be18b0 0x564ab5be1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.427+0000 7f171e44d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ab5be18b0 0x564ab5be1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47042/0 (socket says 172.21.15.1:47042) 2023-11-16T16:48:57.435 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.427+0000 7f171e44d700 1 -- 172.21.15.1:0/1066189529 learned_addr learned my addr 172.21.15.1:0/1066189529 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:57.436 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.427+0000 7f171dc4c700 1 --2- 172.21.15.1:0/1066189529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ab5bdcb60 0x564ab5bdcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.436 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.427+0000 7f171d44b700 1 --2- 172.21.15.1:0/1066189529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ab5bdd470 0x564ab5be1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.436 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.427+0000 7f171e44d700 1 -- 172.21.15.1:0/1066189529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ab5bdcb60 msgr2=0x564ab5bdcf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:57.436 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.427+0000 7f171e44d700 1 --2- 172.21.15.1:0/1066189529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ab5bdcb60 0x564ab5bdcf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.436 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.427+0000 7f171e44d700 1 -- 172.21.15.1:0/1066189529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ab5bdd470 msgr2=0x564ab5be1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:57.436 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.427+0000 7f171e44d700 1 --2- 172.21.15.1:0/1066189529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ab5bdd470 0x564ab5be1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.436 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.427+0000 7f171e44d700 1 -- 172.21.15.1:0/1066189529 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ab59af630 con 0x564ab5be18b0 2023-11-16T16:48:57.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.427+0000 7f171d44b700 1 --2- 172.21.15.1:0/1066189529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ab5bdd470 0x564ab5be1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:48:57.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.427+0000 7f171e44d700 1 --2- 172.21.15.1:0/1066189529 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ab5be18b0 0x564ab5be1d00 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f1710009440 tx=0x7f1710009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65691efd9b405e0b server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:57.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.427+0000 7f1703fff700 1 -- 172.21.15.1:0/1066189529 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171000e070 con 0x564ab5be18b0 2023-11-16T16:48:57.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.428+0000 7f1703fff700 1 -- 172.21.15.1:0/1066189529 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f171000f040 con 0x564ab5be18b0 2023-11-16T16:48:57.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.428+0000 7f1703fff700 1 -- 172.21.15.1:0/1066189529 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1710014a70 con 0x564ab5be18b0 2023-11-16T16:48:57.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.428+0000 7f172c03cb80 1 -- 172.21.15.1:0/1066189529 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ab5be18b0 msgr2=0x564ab5be1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:57.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.428+0000 7f172c03cb80 1 --2- 172.21.15.1:0/1066189529 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ab5be18b0 0x564ab5be1d00 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f1710009440 tx=0x7f1710009920 comp rx=0 tx=0).stop 2023-11-16T16:48:57.437 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.428+0000 7f172c03cb80 1 -- 172.21.15.1:0/1066189529 shutdown_connections 2023-11-16T16:48:57.438 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.428+0000 7f172c03cb80 1 --2- 172.21.15.1:0/1066189529 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ab5be18b0 0x564ab5be1d00 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.438 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.428+0000 7f172c03cb80 1 --2- 172.21.15.1:0/1066189529 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ab5bdcb60 0x564ab5bdcf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.438 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.428+0000 7f172c03cb80 1 --2- 172.21.15.1:0/1066189529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ab5bdd470 0x564ab5be1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.438 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.428+0000 7f172c03cb80 1 -- 172.21.15.1:0/1066189529 >> 172.21.15.1:0/1066189529 conn(0x564ab5bd2360 msgr2=0x564ab5bd7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:57.438 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.428+0000 7f172c03cb80 1 -- 172.21.15.1:0/1066189529 shutdown_connections 2023-11-16T16:48:57.438 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.428+0000 7f172c03cb80 1 -- 172.21.15.1:0/1066189529 wait complete. 2023-11-16T16:48:57.438 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.429+0000 7f172c03cb80 1 -- start start 2023-11-16T16:48:57.439 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.429+0000 7f172c03cb80 1 Processor -- start 2023-11-16T16:48:57.439 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.429+0000 7f172c03cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ab5bdcb60 0x564ab5be9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.439 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.429+0000 7f172c03cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ab5bdd470 0x564ab5be6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.439 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.429+0000 7f172c03cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ab5be18b0 0x564ab5be6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.439 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.429+0000 7f172c03cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ab5acf8e0 con 0x564ab5bdd470 2023-11-16T16:48:57.439 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.429+0000 7f172c03cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ab5acf580 con 0x564ab5bdcb60 2023-11-16T16:48:57.439 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.429+0000 7f172c03cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ab59b1920 con 0x564ab5be18b0 2023-11-16T16:48:57.440 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.429+0000 7f172c03cb80 10 client.0 _ll_register_callbacks cb 0x564ab5aed780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:48:57.440 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.429+0000 7f171d44b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ab5bdd470 0x564ab5be6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.440 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.429+0000 7f171d44b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ab5bdd470 0x564ab5be6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47046/0 (socket says 172.21.15.1:47046) 2023-11-16T16:48:57.440 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.429+0000 7f171dc4c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ab5bdcb60 0x564ab5be9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.440 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.429+0000 7f171d44b700 1 -- 172.21.15.1:0/674738844 learned_addr learned my addr 172.21.15.1:0/674738844 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:57.440 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.430+0000 7f171e44d700 1 --2- 172.21.15.1:0/674738844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ab5be18b0 0x564ab5be6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.440 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.430+0000 7f171d44b700 1 -- 172.21.15.1:0/674738844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ab5be18b0 msgr2=0x564ab5be6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:57.440 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.430+0000 7f171d44b700 1 --2- 172.21.15.1:0/674738844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ab5be18b0 0x564ab5be6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.440 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.430+0000 7f171d44b700 1 -- 172.21.15.1:0/674738844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ab5bdcb60 msgr2=0x564ab5be9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:57.440 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.430+0000 7f171d44b700 1 --2- 172.21.15.1:0/674738844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ab5bdcb60 0x564ab5be9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.441 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.430+0000 7f171d44b700 1 -- 172.21.15.1:0/674738844 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17100090f0 con 0x564ab5bdd470 2023-11-16T16:48:57.441 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.430+0000 7f171dc4c700 1 --2- 172.21.15.1:0/674738844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ab5bdcb60 0x564ab5be9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:48:57.441 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.430+0000 7f171d44b700 1 --2- 172.21.15.1:0/674738844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ab5bdd470 0x564ab5be6250 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f170c011720 tx=0x7f170c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b6f96ca8875942 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:57.441 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.430+0000 7f17017fa700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:48:57.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:57 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/4257917126' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:48:57.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.430+0000 7f17017fa700 1 -- 172.21.15.1:0/674738844 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170c004a30 con 0x564ab5bdd470 2023-11-16T16:48:57.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.430+0000 7f17017fa700 1 -- 172.21.15.1:0/674738844 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f170c004b90 con 0x564ab5bdd470 2023-11-16T16:48:57.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.430+0000 7f172c03cb80 1 -- 172.21.15.1:0/674738844 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564ab5bcfb20 con 0x564ab5bdd470 2023-11-16T16:48:57.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.430+0000 7f17017fa700 1 -- 172.21.15.1:0/674738844 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f170c004d50 con 0x564ab5bdd470 2023-11-16T16:48:57.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.430+0000 7f172c03cb80 10 client.16932 fetch_fsmap learned FSMap version 32 2023-11-16T16:48:57.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.430+0000 7f172c03cb80 1 -- 172.21.15.1:0/674738844 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564ab5af4190 con 0x564ab5bdd470 2023-11-16T16:48:57.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.431+0000 7f17017fa700 1 -- 172.21.15.1:0/674738844 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564ab5bcfb20 con 0x564ab5bdd470 2023-11-16T16:48:57.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.431+0000 7f17017fa700 1 -- 172.21.15.1:0/674738844 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f170c01e8f0 con 0x564ab5bdd470 2023-11-16T16:48:57.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.431+0000 7f172c03cb80 10 client.16932 fetch_fsmap finished waiting for FSMap version 32 2023-11-16T16:48:57.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.431+0000 7f172c03cb80 10 client.16932 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-11-16T16:48:57.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.431+0000 7f172c03cb80 20 client.16932 populate_metadata read hostname 'smithi001' 2023-11-16T16:48:57.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.431+0000 7f172c03cb80 1 --2- 172.21.15.1:0/674738844 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x564ab5bf6760 0x564ab5bf8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.431+0000 7f172c03cb80 4 client.16932 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:48:57.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.431+0000 7f172c03cb80 1 -- 172.21.15.1:0/674738844 --> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564ab5af00a0 con 0x564ab5bf6760 2023-11-16T16:48:57.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.431+0000 7f171dc4c700 1 --2- 172.21.15.1:0/674738844 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x564ab5bf6760 0x564ab5bf8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.432+0000 7f17017fa700 10 client.16932 ms_handle_connect on v2:172.21.15.53:6838/1044753024 2023-11-16T16:48:57.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.432+0000 7f171dc4c700 1 --2- 172.21.15.1:0/674738844 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x564ab5bf6760 0x564ab5bf8b70 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=52d9148308405272 server_cookie=54fd27f49ca107f3 in_seq=0 out_seq=0 2023-11-16T16:48:57.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.432+0000 7f17017fa700 1 -- 172.21.15.1:0/674738844 <== mds.1 v2:172.21.15.53:6838/1044753024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564ab5af00a0 con 0x564ab5bf6760 2023-11-16T16:48:57.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.432+0000 7f17017fa700 10 client.16932 handle_command_reply: tid=0 2023-11-16T16:48:57.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.432+0000 7f172c03cb80 1 client.16932 shutdown 2023-11-16T16:48:57.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.433+0000 7f172c03cb80 1 -- 172.21.15.1:0/674738844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ab5bdd470 msgr2=0x564ab5be6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:57.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.433+0000 7f172c03cb80 1 --2- 172.21.15.1:0/674738844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ab5bdd470 0x564ab5be6250 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f170c011720 tx=0x7f170c0047c0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.433+0000 7f172c03cb80 1 -- 172.21.15.1:0/674738844 shutdown_connections 2023-11-16T16:48:57.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.433+0000 7f172c03cb80 1 --2- 172.21.15.1:0/674738844 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ab5be18b0 0x564ab5be6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.433+0000 7f172c03cb80 1 --2- 172.21.15.1:0/674738844 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x564ab5bf6760 0x564ab5bf8b70 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.433+0000 7f172c03cb80 1 --2- 172.21.15.1:0/674738844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ab5bdcb60 0x564ab5be9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.433+0000 7f17017fa700 0 client.16932 ms_handle_reset on v2:172.21.15.53:6838/1044753024 2023-11-16T16:48:57.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.433+0000 7f172c03cb80 1 --2- 172.21.15.1:0/674738844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ab5bdd470 0x564ab5be6250 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.433+0000 7f172c03cb80 1 -- 172.21.15.1:0/674738844 >> 172.21.15.1:0/674738844 conn(0x564ab5bd2360 msgr2=0x564ab5be2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:57.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.433+0000 7f172c03cb80 1 -- 172.21.15.1:0/674738844 shutdown_connections 2023-11-16T16:48:57.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.433+0000 7f172c03cb80 1 -- 172.21.15.1:0/674738844 wait complete. 2023-11-16T16:48:57.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.433+0000 7f172c03cb80 20 client.16932 trim_cache size 0 max 16384 2023-11-16T16:48:57.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.446+0000 7f172c03cb80 1 Processor -- start 2023-11-16T16:48:57.454 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.446+0000 7f172c03cb80 1 -- start start 2023-11-16T16:48:57.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.446+0000 7f172c03cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ab5bdcb60 0x564ab5bed9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.446+0000 7f172c03cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ab5bdd470 0x564ab5be8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.446+0000 7f172c03cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ab5be18b0 0x564ab5bdf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.446+0000 7f172c03cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ab5bd1f10 con 0x564ab5bdd470 2023-11-16T16:48:57.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.446+0000 7f172c03cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ab5be47d0 con 0x564ab5be18b0 2023-11-16T16:48:57.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.446+0000 7f172c03cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ab5be03c0 con 0x564ab5bdcb60 2023-11-16T16:48:57.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.446+0000 7f171d44b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ab5bdd470 0x564ab5be8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.456 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.446+0000 7f171d44b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ab5bdd470 0x564ab5be8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47054/0 (socket says 172.21.15.1:47054) 2023-11-16T16:48:57.456 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.446+0000 7f171d44b700 1 -- 172.21.15.1:0/181445901 learned_addr learned my addr 172.21.15.1:0/181445901 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:57.456 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.446+0000 7f171dc4c700 1 --2- 172.21.15.1:0/181445901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ab5bdcb60 0x564ab5bed9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.456 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.446+0000 7f171e44d700 1 --2- 172.21.15.1:0/181445901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ab5be18b0 0x564ab5bdf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.456 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.447+0000 7f171d44b700 1 -- 172.21.15.1:0/181445901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ab5bdcb60 msgr2=0x564ab5bed9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:57.456 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.447+0000 7f171d44b700 1 --2- 172.21.15.1:0/181445901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ab5bdcb60 0x564ab5bed9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.456 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.447+0000 7f171d44b700 1 -- 172.21.15.1:0/181445901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ab5be18b0 msgr2=0x564ab5bdf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:57.457 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.447+0000 7f171d44b700 1 --2- 172.21.15.1:0/181445901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ab5be18b0 0x564ab5bdf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.457 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.447+0000 7f171d44b700 1 -- 172.21.15.1:0/181445901 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ab5bd3b00 con 0x564ab5bdd470 2023-11-16T16:48:57.457 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.447+0000 7f171dc4c700 1 --2- 172.21.15.1:0/181445901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ab5bdcb60 0x564ab5bed9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:48:57.457 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.447+0000 7f171d44b700 1 --2- 172.21.15.1:0/181445901 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ab5bdd470 0x564ab5be8b70 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f170c000c00 tx=0x7f170c0103f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25ae037df017265f server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:57.457 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.447+0000 7f1708ff9700 1 -- 172.21.15.1:0/181445901 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170c009950 con 0x564ab5bdd470 2023-11-16T16:48:57.457 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.447+0000 7f1708ff9700 1 -- 172.21.15.1:0/181445901 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f170c011d10 con 0x564ab5bdd470 2023-11-16T16:48:57.457 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.447+0000 7f1708ff9700 1 -- 172.21.15.1:0/181445901 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170c004a90 con 0x564ab5bdd470 2023-11-16T16:48:57.457 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.447+0000 7f172c03cb80 1 -- 172.21.15.1:0/181445901 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ab5bdd470 msgr2=0x564ab5be8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:57.458 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.447+0000 7f172c03cb80 1 --2- 172.21.15.1:0/181445901 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ab5bdd470 0x564ab5be8b70 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f170c000c00 tx=0x7f170c0103f0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.458 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.448+0000 7f172c03cb80 1 -- 172.21.15.1:0/181445901 shutdown_connections 2023-11-16T16:48:57.458 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.448+0000 7f172c03cb80 1 --2- 172.21.15.1:0/181445901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ab5be18b0 0x564ab5bdf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.458 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.448+0000 7f172c03cb80 1 --2- 172.21.15.1:0/181445901 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ab5bdcb60 0x564ab5bed9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.458 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.448+0000 7f172c03cb80 1 --2- 172.21.15.1:0/181445901 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ab5bdd470 0x564ab5be8b70 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.458 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.448+0000 7f172c03cb80 1 -- 172.21.15.1:0/181445901 >> 172.21.15.1:0/181445901 conn(0x564ab5bd2360 msgr2=0x564ab5be3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:57.458 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.448+0000 7f172c03cb80 1 -- 172.21.15.1:0/181445901 shutdown_connections 2023-11-16T16:48:57.459 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.448+0000 7f172c03cb80 1 -- 172.21.15.1:0/181445901 wait complete. 2023-11-16T16:48:57.459 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.448+0000 7f172c03cb80 1 -- start start 2023-11-16T16:48:57.459 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.448+0000 7f172c03cb80 1 Processor -- start 2023-11-16T16:48:57.459 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.448+0000 7f172c03cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ab5bdcb60 0x564ab5beb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.459 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.448+0000 7f172c03cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ab5bdd470 0x564ab5bebed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.459 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.448+0000 7f172c03cb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ab5be18b0 0x564ab5bec410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.459 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.448+0000 7f172c03cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ab5bedda0 con 0x564ab5bdcb60 2023-11-16T16:48:57.459 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.448+0000 7f172c03cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ab5c49650 con 0x564ab5bdd470 2023-11-16T16:48:57.460 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.448+0000 7f172c03cb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ab5c49810 con 0x564ab5be18b0 2023-11-16T16:48:57.460 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.448+0000 7f172c03cb80 10 client.0 _ll_register_callbacks cb 0x7f170c00ee00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:48:57.460 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.448+0000 7f171dc4c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ab5bdcb60 0x564ab5beb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.460 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.448+0000 7f171dc4c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ab5bdcb60 0x564ab5beb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47058/0 (socket says 172.21.15.1:47058) 2023-11-16T16:48:57.460 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.448+0000 7f171dc4c700 1 -- 172.21.15.1:0/2045351845 learned_addr learned my addr 172.21.15.1:0/2045351845 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:57.460 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.449+0000 7f171d44b700 1 --2- 172.21.15.1:0/2045351845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ab5bdd470 0x564ab5bebed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.460 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.449+0000 7f171e44d700 1 --2- 172.21.15.1:0/2045351845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ab5be18b0 0x564ab5bec410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.460 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.449+0000 7f171dc4c700 1 -- 172.21.15.1:0/2045351845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ab5be18b0 msgr2=0x564ab5bec410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:57.461 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.449+0000 7f171dc4c700 1 --2- 172.21.15.1:0/2045351845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ab5be18b0 0x564ab5bec410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.461 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.449+0000 7f171dc4c700 1 -- 172.21.15.1:0/2045351845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ab5bdd470 msgr2=0x564ab5bebed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:57.461 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.449+0000 7f171dc4c700 1 --2- 172.21.15.1:0/2045351845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ab5bdd470 0x564ab5bebed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.461 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.449+0000 7f171dc4c700 1 -- 172.21.15.1:0/2045351845 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564ab5af4190 con 0x564ab5bdcb60 2023-11-16T16:48:57.461 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.449+0000 7f171dc4c700 1 --2- 172.21.15.1:0/2045351845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ab5bdcb60 0x564ab5beb990 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f1714005b40 tx=0x7f1714021c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee533625f8c6df95 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:57.461 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.449+0000 7f17017fa700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:48:57.461 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.449+0000 7f17017fa700 1 -- 172.21.15.1:0/2045351845 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17140a36a0 con 0x564ab5bdcb60 2023-11-16T16:48:57.461 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.449+0000 7f17017fa700 1 -- 172.21.15.1:0/2045351845 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f171409a7d0 con 0x564ab5bdcb60 2023-11-16T16:48:57.462 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.449+0000 7f172c03cb80 1 -- 172.21.15.1:0/2045351845 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f171000f040 con 0x564ab5bdcb60 2023-11-16T16:48:57.462 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.449+0000 7f17017fa700 1 -- 172.21.15.1:0/2045351845 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f171409a990 con 0x564ab5bdcb60 2023-11-16T16:48:57.462 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.449+0000 7f172c03cb80 10 client.16938 fetch_fsmap learned FSMap version 32 2023-11-16T16:48:57.462 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.449+0000 7f172c03cb80 1 -- 172.21.15.1:0/2045351845 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564ab595bde0 con 0x564ab5bdcb60 2023-11-16T16:48:57.462 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.449+0000 7f17017fa700 1 -- 172.21.15.1:0/2045351845 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f171000f040 con 0x564ab5bdcb60 2023-11-16T16:48:57.462 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.450+0000 7f17017fa700 1 -- 172.21.15.1:0/2045351845 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f1714097240 con 0x564ab5bdcb60 2023-11-16T16:48:57.462 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.450+0000 7f172c03cb80 10 client.16938 fetch_fsmap finished waiting for FSMap version 32 2023-11-16T16:48:57.463 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.450+0000 7f172c03cb80 10 client.16938 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-11-16T16:48:57.463 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.450+0000 7f172c03cb80 20 client.16938 populate_metadata read hostname 'smithi001' 2023-11-16T16:48:57.463 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.450+0000 7f172c03cb80 1 --2- 172.21.15.1:0/2045351845 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x564ab5c4ab60 0x564ab5c4cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.463 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.450+0000 7f172c03cb80 4 client.16938 mds_command: new command op to 24493 tid=0[{"prefix": "damage ls"}] 2023-11-16T16:48:57.463 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.450+0000 7f172c03cb80 1 -- 172.21.15.1:0/2045351845 --> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564ab5afc660 con 0x564ab5c4ab60 2023-11-16T16:48:57.463 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.450+0000 7f171d44b700 1 --2- 172.21.15.1:0/2045351845 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x564ab5c4ab60 0x564ab5c4cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.463 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.450+0000 7f17017fa700 10 client.16938 ms_handle_connect on v2:172.21.15.53:6838/1044753024 2023-11-16T16:48:57.463 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.450+0000 7f171d44b700 1 --2- 172.21.15.1:0/2045351845 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x564ab5c4ab60 0x564ab5c4cf40 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2026f68a3b6c862b server_cookie=6d9580a6ae0e1c47 in_seq=0 out_seq=0 2023-11-16T16:48:57.464 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.451+0000 7f17017fa700 1 -- 172.21.15.1:0/2045351845 <== mds.1 v2:172.21.15.53:6838/1044753024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564ab5afc660 con 0x564ab5c4ab60 2023-11-16T16:48:57.464 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.451+0000 7f17017fa700 10 client.16938 handle_command_reply: tid=0 2023-11-16T16:48:57.464 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.451+0000 7f172c03cb80 1 client.16938 shutdown 2023-11-16T16:48:57.464 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.451+0000 7f172c03cb80 1 -- 172.21.15.1:0/2045351845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ab5bdcb60 msgr2=0x564ab5beb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:57.464 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.451+0000 7f172c03cb80 1 --2- 172.21.15.1:0/2045351845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ab5bdcb60 0x564ab5beb990 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f1714005b40 tx=0x7f1714021c70 comp rx=0 tx=0).stop 2023-11-16T16:48:57.464 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.451+0000 7f172c03cb80 1 -- 172.21.15.1:0/2045351845 shutdown_connections 2023-11-16T16:48:57.464 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.451+0000 7f172c03cb80 1 --2- 172.21.15.1:0/2045351845 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x564ab5be18b0 0x564ab5bec410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.451+0000 7f172c03cb80 1 --2- 172.21.15.1:0/2045351845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ab5bdcb60 0x564ab5beb990 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.451+0000 7f172c03cb80 1 --2- 172.21.15.1:0/2045351845 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x564ab5c4ab60 0x564ab5c4cf40 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.451+0000 7f172c03cb80 1 --2- 172.21.15.1:0/2045351845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564ab5bdd470 0x564ab5bebed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.451+0000 7f17017fa700 0 client.16938 ms_handle_reset on v2:172.21.15.53:6838/1044753024 2023-11-16T16:48:57.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.451+0000 7f172c03cb80 1 -- 172.21.15.1:0/2045351845 >> 172.21.15.1:0/2045351845 conn(0x564ab5bd2360 msgr2=0x564ab5bdf8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:57.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.451+0000 7f172c03cb80 1 -- 172.21.15.1:0/2045351845 shutdown_connections 2023-11-16T16:48:57.465 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.451+0000 7f172c03cb80 1 -- 172.21.15.1:0/2045351845 wait complete. 2023-11-16T16:48:57.466 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.451+0000 7f172c03cb80 20 client.16938 trim_cache size 0 max 16384 2023-11-16T16:48:57.466 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2023-11-16T16:48:57.467 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.452+0000 7f171feb0700 1 -- 172.21.15.1:0/595858841 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f17040a12c0 msgr2=0x7f17040a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:57.467 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.452+0000 7f171feb0700 1 --2- 172.21.15.1:0/595858841 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f17040a12c0 0x7f17040a3770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f170c005fd0 tx=0x7f170c005ee0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.468 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.452+0000 7f171feb0700 1 -- 172.21.15.1:0/595858841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1718101200 msgr2=0x7f1718198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:57.468 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.452+0000 7f171feb0700 1 --2- 172.21.15.1:0/595858841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1718101200 0x7f1718198470 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f171400b2e0 tx=0x7f1714004730 comp rx=0 tx=0).stop 2023-11-16T16:48:57.468 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.452+0000 7f171feb0700 1 -- 172.21.15.1:0/595858841 shutdown_connections 2023-11-16T16:48:57.468 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.452+0000 7f171feb0700 1 --2- 172.21.15.1:0/595858841 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f17040a12c0 0x7f17040a3770 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.468 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.452+0000 7f171feb0700 1 --2- 172.21.15.1:0/595858841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f171810cf60 0x7f171819ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.468 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.452+0000 7f171feb0700 1 --2- 172.21.15.1:0/595858841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1718101200 0x7f1718198470 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.468 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.452+0000 7f171feb0700 1 --2- 172.21.15.1:0/595858841 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f1718101660 0x7f17181989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.469 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.452+0000 7f171feb0700 1 -- 172.21.15.1:0/595858841 >> 172.21.15.1:0/595858841 conn(0x7f1718075880 msgr2=0x7f17180fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:57.469 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.452+0000 7f171feb0700 1 -- 172.21.15.1:0/595858841 shutdown_connections 2023-11-16T16:48:57.469 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.452+0000 7f171feb0700 1 -- 172.21.15.1:0/595858841 wait complete. 2023-11-16T16:48:57.469 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-16T16:48:57.598 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.596+0000 7fb01cd82700 1 -- 172.21.15.1:0/989257131 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb0181014d0 msgr2=0x7fb0181018a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:57.598 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.596+0000 7fb01cd82700 1 --2- 172.21.15.1:0/989257131 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb0181014d0 0x7fb0181018a0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fb00c009990 tx=0x7fb00c009ca0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.598 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.596+0000 7fb01cd82700 1 -- 172.21.15.1:0/989257131 shutdown_connections 2023-11-16T16:48:57.598 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.596+0000 7fb01cd82700 1 --2- 172.21.15.1:0/989257131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb01810a810 0x7fb01810cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.599 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.596+0000 7fb01cd82700 1 --2- 172.21.15.1:0/989257131 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb0181014d0 0x7fb0181018a0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.599 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.596+0000 7fb01cd82700 1 --2- 172.21.15.1:0/989257131 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb018101de0 0x7fb01810a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.599 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.596+0000 7fb01cd82700 1 -- 172.21.15.1:0/989257131 >> 172.21.15.1:0/989257131 conn(0x7fb018076370 msgr2=0x7fb018076770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:57.599 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.596+0000 7fb01cd82700 1 -- 172.21.15.1:0/989257131 shutdown_connections 2023-11-16T16:48:57.599 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.596+0000 7fb01cd82700 1 -- 172.21.15.1:0/989257131 wait complete. 2023-11-16T16:48:57.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.597+0000 7fb01cd82700 1 Processor -- start 2023-11-16T16:48:57.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.597+0000 7fb01cd82700 1 -- start start 2023-11-16T16:48:57.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.597+0000 7fb01cd82700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0181014d0 0x7fb0180ff120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.602 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.597+0000 7fb01cd82700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb018101de0 0x7fb0180ff660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.602 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.597+0000 7fb01cd82700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb01810a810 0x7fb0181a4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.602 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.597+0000 7fb01cd82700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0180ffe90 con 0x7fb018101de0 2023-11-16T16:48:57.602 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.597+0000 7fb01cd82700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0181a4990 con 0x7fb01810a810 2023-11-16T16:48:57.602 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.597+0000 7fb01cd82700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0181a4ad0 con 0x7fb0181014d0 2023-11-16T16:48:57.602 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.597+0000 7fb015d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb018101de0 0x7fb0180ff660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.602 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.597+0000 7fb015d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb018101de0 0x7fb0180ff660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47072/0 (socket says 172.21.15.1:47072) 2023-11-16T16:48:57.602 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.597+0000 7fb015d9b700 1 -- 172.21.15.1:0/4061601655 learned_addr learned my addr 172.21.15.1:0/4061601655 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:57.603 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.598+0000 7fb01659c700 1 --2- 172.21.15.1:0/4061601655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0181014d0 0x7fb0180ff120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.603 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.598+0000 7fb016d9d700 1 --2- 172.21.15.1:0/4061601655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb01810a810 0x7fb0181a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.603 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.598+0000 7fb015d9b700 1 -- 172.21.15.1:0/4061601655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0181014d0 msgr2=0x7fb0180ff120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:57.603 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.598+0000 7fb015d9b700 1 --2- 172.21.15.1:0/4061601655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0181014d0 0x7fb0180ff120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.603 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.598+0000 7fb015d9b700 1 -- 172.21.15.1:0/4061601655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb01810a810 msgr2=0x7fb0181a4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:57.603 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.598+0000 7fb015d9b700 1 --2- 172.21.15.1:0/4061601655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb01810a810 0x7fb0181a4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.603 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.598+0000 7fb015d9b700 1 -- 172.21.15.1:0/4061601655 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb00c0095f0 con 0x7fb018101de0 2023-11-16T16:48:57.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.598+0000 7fb016d9d700 1 --2- 172.21.15.1:0/4061601655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb01810a810 0x7fb0181a4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:48:57.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.598+0000 7fb015d9b700 1 --2- 172.21.15.1:0/4061601655 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb018101de0 0x7fb0180ff660 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fb00800c9b0 tx=0x7fb00800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:57.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.598+0000 7fb0077fe700 1 -- 172.21.15.1:0/4061601655 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb008007b00 con 0x7fb018101de0 2023-11-16T16:48:57.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.598+0000 7fb01cd82700 1 -- 172.21.15.1:0/4061601655 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0181102c0 con 0x7fb018101de0 2023-11-16T16:48:57.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.598+0000 7fb0077fe700 1 -- 172.21.15.1:0/4061601655 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb008007c60 con 0x7fb018101de0 2023-11-16T16:48:57.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.598+0000 7fb0077fe700 1 -- 172.21.15.1:0/4061601655 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb008018660 con 0x7fb018101de0 2023-11-16T16:48:57.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.598+0000 7fb01cd82700 1 -- 172.21.15.1:0/4061601655 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb018110130 con 0x7fb018101de0 2023-11-16T16:48:57.610 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.600+0000 7fb0077fe700 1 -- 172.21.15.1:0/4061601655 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fb008018880 con 0x7fb018101de0 2023-11-16T16:48:57.610 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.601+0000 7fb0077fe700 1 --2- 172.21.15.1:0/4061601655 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fb0000a12c0 0x7fb0000a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.611 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.601+0000 7fb0077fe700 1 -- 172.21.15.1:0/4061601655 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fb0080ccd90 con 0x7fb018101de0 2023-11-16T16:48:57.611 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.601+0000 7fb01659c700 1 --2- 172.21.15.1:0/4061601655 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fb0000a12c0 0x7fb0000a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.611 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.602+0000 7fb01659c700 1 --2- 172.21.15.1:0/4061601655 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fb0000a12c0 0x7fb0000a3770 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb00c006010 tx=0x7fb00c00b260 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:57.611 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.602+0000 7fb028f0eb80 1 Processor -- start 2023-11-16T16:48:57.611 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.602+0000 7fb028f0eb80 1 -- start start 2023-11-16T16:48:57.611 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.602+0000 7fb028f0eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5c26e0b60 0x55c5c26e0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.611 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.602+0000 7fb028f0eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5c26e1470 0x55c5c26e5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.611 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.602+0000 7fb028f0eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5c26e58b0 0x55c5c26e5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.612 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.602+0000 7fb028f0eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c25e6060 con 0x55c5c26e1470 2023-11-16T16:48:57.612 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.602+0000 7fb028f0eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c26388a0 con 0x55c5c26e0b60 2023-11-16T16:48:57.612 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.603+0000 7fb028f0eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c2631d30 con 0x55c5c26e58b0 2023-11-16T16:48:57.612 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.603+0000 7fb015d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5c26e1470 0x55c5c26e5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.612 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.603+0000 7fb015d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5c26e1470 0x55c5c26e5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47080/0 (socket says 172.21.15.1:47080) 2023-11-16T16:48:57.612 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.603+0000 7fb015d9b700 1 -- 172.21.15.1:0/3998067020 learned_addr learned my addr 172.21.15.1:0/3998067020 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:57.612 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.603+0000 7fb01659c700 1 --2- 172.21.15.1:0/3998067020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5c26e0b60 0x55c5c26e0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.612 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.603+0000 7fb016d9d700 1 --2- 172.21.15.1:0/3998067020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5c26e58b0 0x55c5c26e5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.613 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.603+0000 7fb015d9b700 1 -- 172.21.15.1:0/3998067020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5c26e58b0 msgr2=0x55c5c26e5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:57.613 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.603+0000 7fb015d9b700 1 --2- 172.21.15.1:0/3998067020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5c26e58b0 0x55c5c26e5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.613 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.603+0000 7fb015d9b700 1 -- 172.21.15.1:0/3998067020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5c26e0b60 msgr2=0x55c5c26e0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:57.613 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.603+0000 7fb015d9b700 1 --2- 172.21.15.1:0/3998067020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5c26e0b60 0x55c5c26e0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.613 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.603+0000 7fb015d9b700 1 -- 172.21.15.1:0/3998067020 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5c24b3630 con 0x55c5c26e1470 2023-11-16T16:48:57.613 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.603+0000 7fb016d9d700 1 --2- 172.21.15.1:0/3998067020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5c26e58b0 0x55c5c26e5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:48:57.613 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.603+0000 7fb015d9b700 1 --2- 172.21.15.1:0/3998067020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5c26e1470 0x55c5c26e5370 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fb008099de0 tx=0x7fb008007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f998ab1545ca687 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:57.613 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.603+0000 7fb004ff9700 1 -- 172.21.15.1:0/3998067020 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0080a22a0 con 0x55c5c26e1470 2023-11-16T16:48:57.614 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.603+0000 7fb004ff9700 1 -- 172.21.15.1:0/3998067020 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb008096560 con 0x55c5c26e1470 2023-11-16T16:48:57.614 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.603+0000 7fb004ff9700 1 -- 172.21.15.1:0/3998067020 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0080995a0 con 0x55c5c26e1470 2023-11-16T16:48:57.614 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.604+0000 7fb028f0eb80 1 -- 172.21.15.1:0/3998067020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5c26e1470 msgr2=0x55c5c26e5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:57.614 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.604+0000 7fb028f0eb80 1 --2- 172.21.15.1:0/3998067020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5c26e1470 0x55c5c26e5370 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fb008099de0 tx=0x7fb008007ea0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.614 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.604+0000 7fb028f0eb80 1 -- 172.21.15.1:0/3998067020 shutdown_connections 2023-11-16T16:48:57.614 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.604+0000 7fb028f0eb80 1 --2- 172.21.15.1:0/3998067020 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5c26e58b0 0x55c5c26e5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.614 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.604+0000 7fb028f0eb80 1 --2- 172.21.15.1:0/3998067020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5c26e0b60 0x55c5c26e0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.614 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.604+0000 7fb028f0eb80 1 --2- 172.21.15.1:0/3998067020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5c26e1470 0x55c5c26e5370 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.604+0000 7fb028f0eb80 1 -- 172.21.15.1:0/3998067020 >> 172.21.15.1:0/3998067020 conn(0x55c5c26d6360 msgr2=0x55c5c26dbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:57.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.604+0000 7fb028f0eb80 1 -- 172.21.15.1:0/3998067020 shutdown_connections 2023-11-16T16:48:57.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.604+0000 7fb028f0eb80 1 -- 172.21.15.1:0/3998067020 wait complete. 2023-11-16T16:48:57.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.604+0000 7fb028f0eb80 1 -- start start 2023-11-16T16:48:57.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.604+0000 7fb028f0eb80 1 Processor -- start 2023-11-16T16:48:57.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.605+0000 7fb028f0eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5c26e0b60 0x55c5c26ed100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.605+0000 7fb028f0eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5c26e1470 0x55c5c26ea310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.615 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.605+0000 7fb028f0eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5c26e58b0 0x55c5c26eaa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.616 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.605+0000 7fb028f0eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c25d38e0 con 0x55c5c26e58b0 2023-11-16T16:48:57.616 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.605+0000 7fb028f0eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c25d3580 con 0x55c5c26e0b60 2023-11-16T16:48:57.616 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.605+0000 7fb028f0eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c24b5920 con 0x55c5c26e1470 2023-11-16T16:48:57.616 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.605+0000 7fb028f0eb80 10 client.0 _ll_register_callbacks cb 0x55c5c25f1780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:48:57.616 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.605+0000 7fb016d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5c26e58b0 0x55c5c26eaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.616 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.605+0000 7fb016d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5c26e58b0 0x55c5c26eaa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47088/0 (socket says 172.21.15.1:47088) 2023-11-16T16:48:57.616 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.605+0000 7fb016d9d700 1 -- 172.21.15.1:0/1154530999 learned_addr learned my addr 172.21.15.1:0/1154530999 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:57.616 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.605+0000 7fb01659c700 1 --2- 172.21.15.1:0/1154530999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5c26e0b60 0x55c5c26ed100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.617 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.605+0000 7fb015d9b700 1 --2- 172.21.15.1:0/1154530999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5c26e1470 0x55c5c26ea310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.617 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.605+0000 7fb016d9d700 1 -- 172.21.15.1:0/1154530999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5c26e1470 msgr2=0x55c5c26ea310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:57.617 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.605+0000 7fb016d9d700 1 --2- 172.21.15.1:0/1154530999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5c26e1470 0x55c5c26ea310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.617 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.605+0000 7fb016d9d700 1 -- 172.21.15.1:0/1154530999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5c26e0b60 msgr2=0x55c5c26ed100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:57.617 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.605+0000 7fb016d9d700 1 --2- 172.21.15.1:0/1154530999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5c26e0b60 0x55c5c26ed100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.617 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.605+0000 7fb016d9d700 1 -- 172.21.15.1:0/1154530999 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb018110130 con 0x55c5c26e58b0 2023-11-16T16:48:57.617 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.605+0000 7fb016d9d700 1 --2- 172.21.15.1:0/1154530999 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5c26e58b0 0x55c5c26eaa50 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb010007b30 tx=0x7fb010007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c245cfebe1ac3d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:57.617 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.605+0000 7faffdffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:48:57.618 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.605+0000 7faffdffb700 1 -- 172.21.15.1:0/1154530999 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01000e030 con 0x55c5c26e58b0 2023-11-16T16:48:57.618 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.605+0000 7faffdffb700 1 -- 172.21.15.1:0/1154530999 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fb01000f070 con 0x55c5c26e58b0 2023-11-16T16:48:57.618 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.605+0000 7fb028f0eb80 1 -- 172.21.15.1:0/1154530999 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5c26d3b20 con 0x55c5c26e58b0 2023-11-16T16:48:57.618 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.606+0000 7faffdffb700 1 -- 172.21.15.1:0/1154530999 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb010011910 con 0x55c5c26e58b0 2023-11-16T16:48:57.618 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.606+0000 7fb028f0eb80 10 client.16950 fetch_fsmap learned FSMap version 32 2023-11-16T16:48:57.618 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.606+0000 7fb028f0eb80 1 -- 172.21.15.1:0/1154530999 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c5c25f8190 con 0x55c5c26e58b0 2023-11-16T16:48:57.618 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.606+0000 7faffdffb700 1 -- 172.21.15.1:0/1154530999 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c5c26d3b20 con 0x55c5c26e58b0 2023-11-16T16:48:57.618 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.606+0000 7faffdffb700 1 -- 172.21.15.1:0/1154530999 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fb010014070 con 0x55c5c26e58b0 2023-11-16T16:48:57.619 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.606+0000 7fb028f0eb80 10 client.16950 fetch_fsmap finished waiting for FSMap version 32 2023-11-16T16:48:57.619 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.606+0000 7fb028f0eb80 10 client.16950 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:48:57.619 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.606+0000 7fb028f0eb80 20 client.16950 populate_metadata read hostname 'smithi001' 2023-11-16T16:48:57.619 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.606+0000 7fb028f0eb80 1 --2- 172.21.15.1:0/1154530999 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55c5c26fa990 0x55c5c26fcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.619 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.606+0000 7fb028f0eb80 4 client.16950 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:48:57.619 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.606+0000 7fb028f0eb80 1 -- 172.21.15.1:0/1154530999 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5c25f40a0 con 0x55c5c26fa990 2023-11-16T16:48:57.619 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.606+0000 7fb01659c700 1 --2- 172.21.15.1:0/1154530999 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55c5c26fa990 0x55c5c26fcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.619 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.607+0000 7fb01659c700 1 --2- 172.21.15.1:0/1154530999 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55c5c26fa990 0x55c5c26fcda0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=574eca0a0e2ae50b server_cookie=41d4f5d2015aced4 in_seq=0 out_seq=0 2023-11-16T16:48:57.620 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.607+0000 7faffdffb700 10 client.16950 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:48:57.620 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.607+0000 7faffdffb700 1 -- 172.21.15.1:0/1154530999 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c5c25f40a0 con 0x55c5c26fa990 2023-11-16T16:48:57.620 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.607+0000 7faffdffb700 10 client.16950 handle_command_reply: tid=0 2023-11-16T16:48:57.620 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.608+0000 7fb028f0eb80 1 client.16950 shutdown 2023-11-16T16:48:57.620 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.608+0000 7fb028f0eb80 1 -- 172.21.15.1:0/1154530999 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5c26e58b0 msgr2=0x55c5c26eaa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:57.620 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.608+0000 7fb028f0eb80 1 --2- 172.21.15.1:0/1154530999 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5c26e58b0 0x55c5c26eaa50 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb010007b30 tx=0x7fb010007e40 comp rx=0 tx=0).stop 2023-11-16T16:48:57.620 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.608+0000 7fb028f0eb80 1 -- 172.21.15.1:0/1154530999 shutdown_connections 2023-11-16T16:48:57.620 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.608+0000 7fb028f0eb80 1 --2- 172.21.15.1:0/1154530999 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55c5c26fa990 0x55c5c26fcda0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.608+0000 7fb028f0eb80 1 --2- 172.21.15.1:0/1154530999 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5c26e58b0 0x55c5c26eaa50 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.608+0000 7fb028f0eb80 1 --2- 172.21.15.1:0/1154530999 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5c26e0b60 0x55c5c26ed100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.608+0000 7fb028f0eb80 1 --2- 172.21.15.1:0/1154530999 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5c26e1470 0x55c5c26ea310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.608+0000 7faffdffb700 0 client.16950 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:48:57.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.608+0000 7fb028f0eb80 1 -- 172.21.15.1:0/1154530999 >> 172.21.15.1:0/1154530999 conn(0x55c5c26d6360 msgr2=0x55c5c26e6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:57.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.608+0000 7fb028f0eb80 1 -- 172.21.15.1:0/1154530999 shutdown_connections 2023-11-16T16:48:57.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.608+0000 7fb028f0eb80 1 -- 172.21.15.1:0/1154530999 wait complete. 2023-11-16T16:48:57.621 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.608+0000 7fb028f0eb80 20 client.16950 trim_cache size 0 max 16384 2023-11-16T16:48:57.625 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.621+0000 7fb028f0eb80 1 Processor -- start 2023-11-16T16:48:57.625 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.621+0000 7fb028f0eb80 1 -- start start 2023-11-16T16:48:57.625 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.622+0000 7fb028f0eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5c26e0b60 0x55c5c26ecc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.625 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.622+0000 7fb028f0eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5c26e1470 0x55c5c26e38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.625 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.622+0000 7fb028f0eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5c26e58b0 0x55c5c26e3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.625 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.622+0000 7fb028f0eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c26e1bc0 con 0x55c5c26e1470 2023-11-16T16:48:57.626 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.622+0000 7fb028f0eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c26d39f0 con 0x55c5c26e0b60 2023-11-16T16:48:57.626 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.622+0000 7fb028f0eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c26d5ee0 con 0x55c5c26e58b0 2023-11-16T16:48:57.626 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.622+0000 7fb015d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5c26e1470 0x55c5c26e38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.626 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.622+0000 7fb015d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5c26e1470 0x55c5c26e38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47094/0 (socket says 172.21.15.1:47094) 2023-11-16T16:48:57.626 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.622+0000 7fb015d9b700 1 -- 172.21.15.1:0/1991781130 learned_addr learned my addr 172.21.15.1:0/1991781130 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:57.626 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.622+0000 7fb01659c700 1 --2- 172.21.15.1:0/1991781130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5c26e0b60 0x55c5c26ecc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.626 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.622+0000 7fb015d9b700 1 -- 172.21.15.1:0/1991781130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5c26e58b0 msgr2=0x55c5c26e3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:57.627 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.622+0000 7fb015d9b700 1 --2- 172.21.15.1:0/1991781130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5c26e58b0 0x55c5c26e3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.627 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.622+0000 7fb015d9b700 1 -- 172.21.15.1:0/1991781130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5c26e0b60 msgr2=0x55c5c26ecc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:57.627 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.622+0000 7fb015d9b700 1 --2- 172.21.15.1:0/1991781130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5c26e0b60 0x55c5c26ecc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.627 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.622+0000 7fb015d9b700 1 -- 172.21.15.1:0/1991781130 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5c25f7ee0 con 0x55c5c26e1470 2023-11-16T16:48:57.627 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.622+0000 7fb015d9b700 1 --2- 172.21.15.1:0/1991781130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5c26e1470 0x55c5c26e38f0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fb008099b50 tx=0x7fb008099ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b934e82b9d0231 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:57.627 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.622+0000 7fb0057fa700 1 -- 172.21.15.1:0/1991781130 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0080a22a0 con 0x55c5c26e1470 2023-11-16T16:48:57.627 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.622+0000 7fb0057fa700 1 -- 172.21.15.1:0/1991781130 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb008099340 con 0x55c5c26e1470 2023-11-16T16:48:57.627 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.622+0000 7fb0057fa700 1 -- 172.21.15.1:0/1991781130 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0080a2070 con 0x55c5c26e1470 2023-11-16T16:48:57.628 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.623+0000 7fb028f0eb80 1 -- 172.21.15.1:0/1991781130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5c26e1470 msgr2=0x55c5c26e38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:57.628 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.623+0000 7fb028f0eb80 1 --2- 172.21.15.1:0/1991781130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5c26e1470 0x55c5c26e38f0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fb008099b50 tx=0x7fb008099ac0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.628 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.623+0000 7fb028f0eb80 1 -- 172.21.15.1:0/1991781130 shutdown_connections 2023-11-16T16:48:57.628 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.623+0000 7fb028f0eb80 1 --2- 172.21.15.1:0/1991781130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5c26e58b0 0x55c5c26e3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.628 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.623+0000 7fb028f0eb80 1 --2- 172.21.15.1:0/1991781130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5c26e0b60 0x55c5c26ecc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.628 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.623+0000 7fb028f0eb80 1 --2- 172.21.15.1:0/1991781130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5c26e1470 0x55c5c26e38f0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.628 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.623+0000 7fb028f0eb80 1 -- 172.21.15.1:0/1991781130 >> 172.21.15.1:0/1991781130 conn(0x55c5c26d6360 msgr2=0x55c5c26db730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:57.629 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.623+0000 7fb028f0eb80 1 -- 172.21.15.1:0/1991781130 shutdown_connections 2023-11-16T16:48:57.629 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.623+0000 7fb028f0eb80 1 -- 172.21.15.1:0/1991781130 wait complete. 2023-11-16T16:48:57.629 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.623+0000 7fb028f0eb80 1 -- start start 2023-11-16T16:48:57.629 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.623+0000 7fb028f0eb80 1 Processor -- start 2023-11-16T16:48:57.632 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.624+0000 7fb028f0eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5c26e0b60 0x55c5c26ef990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.624+0000 7fb028f0eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5c26e1470 0x55c5c26efed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.624+0000 7fb028f0eb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5c26e58b0 0x55c5c26f0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.624+0000 7fb028f0eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c274d810 con 0x55c5c26e1470 2023-11-16T16:48:57.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.624+0000 7fb028f0eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c274d950 con 0x55c5c26e0b60 2023-11-16T16:48:57.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.624+0000 7fb028f0eb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5c274dac0 con 0x55c5c26e58b0 2023-11-16T16:48:57.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.624+0000 7fb028f0eb80 10 client.0 _ll_register_callbacks cb 0x7fb010007500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:48:57.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.624+0000 7fb015d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5c26e1470 0x55c5c26efed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.624+0000 7fb015d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5c26e1470 0x55c5c26efed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47100/0 (socket says 172.21.15.1:47100) 2023-11-16T16:48:57.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.624+0000 7fb015d9b700 1 -- 172.21.15.1:0/4170424912 learned_addr learned my addr 172.21.15.1:0/4170424912 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:57.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.624+0000 7fb016d9d700 1 --2- 172.21.15.1:0/4170424912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5c26e58b0 0x55c5c26f0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.624+0000 7fb01659c700 1 --2- 172.21.15.1:0/4170424912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5c26e0b60 0x55c5c26ef990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.624+0000 7fb015d9b700 1 -- 172.21.15.1:0/4170424912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5c26e58b0 msgr2=0x55c5c26f0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:57.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.624+0000 7fb015d9b700 1 --2- 172.21.15.1:0/4170424912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5c26e58b0 0x55c5c26f0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.624+0000 7fb015d9b700 1 -- 172.21.15.1:0/4170424912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5c26e0b60 msgr2=0x55c5c26ef990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:57.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.624+0000 7fb015d9b700 1 --2- 172.21.15.1:0/4170424912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5c26e0b60 0x55c5c26ef990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.624+0000 7fb015d9b700 1 -- 172.21.15.1:0/4170424912 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5c24b3630 con 0x55c5c26e1470 2023-11-16T16:48:57.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.624+0000 7fb016d9d700 1 --2- 172.21.15.1:0/4170424912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5c26e58b0 0x55c5c26f0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:48:57.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.624+0000 7fb015d9b700 1 --2- 172.21.15.1:0/4170424912 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5c26e1470 0x55c5c26efed0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fb008096350 tx=0x7fb0080b8380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b700247af8b84575 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:57.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.624+0000 7faffdffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:48:57.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.624+0000 7faffdffb700 1 -- 172.21.15.1:0/4170424912 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb008099570 con 0x55c5c26e1470 2023-11-16T16:48:57.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.625+0000 7faffdffb700 1 -- 172.21.15.1:0/4170424912 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fb0080ca300 con 0x55c5c26e1470 2023-11-16T16:48:57.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.625+0000 7fb028f0eb80 1 -- 172.21.15.1:0/4170424912 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb008096560 con 0x55c5c26e1470 2023-11-16T16:48:57.636 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.625+0000 7faffdffb700 1 -- 172.21.15.1:0/4170424912 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0080adb50 con 0x55c5c26e1470 2023-11-16T16:48:57.636 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.625+0000 7fb028f0eb80 10 client.16956 fetch_fsmap learned FSMap version 32 2023-11-16T16:48:57.636 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.625+0000 7fb028f0eb80 1 -- 172.21.15.1:0/4170424912 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c5c2448c60 con 0x55c5c26e1470 2023-11-16T16:48:57.636 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.625+0000 7faffdffb700 1 -- 172.21.15.1:0/4170424912 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb00807c020 con 0x55c5c26e1470 2023-11-16T16:48:57.636 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.625+0000 7faffdffb700 1 -- 172.21.15.1:0/4170424912 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fb008078d20 con 0x55c5c26e1470 2023-11-16T16:48:57.636 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.625+0000 7fb028f0eb80 10 client.16956 fetch_fsmap finished waiting for FSMap version 32 2023-11-16T16:48:57.636 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.625+0000 7fb028f0eb80 10 client.16956 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:48:57.636 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.625+0000 7fb028f0eb80 20 client.16956 populate_metadata read hostname 'smithi001' 2023-11-16T16:48:57.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.625+0000 7fb028f0eb80 1 --2- 172.21.15.1:0/4170424912 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55c5c274ecf0 0x55c5c27510d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:57.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.625+0000 7fb028f0eb80 4 client.16956 mds_command: new command op to 24521 tid=0[{"prefix": "damage ls"}] 2023-11-16T16:48:57.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.625+0000 7fb028f0eb80 1 -- 172.21.15.1:0/4170424912 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c5c2600660 con 0x55c5c274ecf0 2023-11-16T16:48:57.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.625+0000 7fb01659c700 1 --2- 172.21.15.1:0/4170424912 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55c5c274ecf0 0x55c5c27510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:57.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.626+0000 7faffdffb700 10 client.16956 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:48:57.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.626+0000 7fb01659c700 1 --2- 172.21.15.1:0/4170424912 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55c5c274ecf0 0x55c5c27510d0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dce49e1456fda27f server_cookie=ef900703f33c162 in_seq=0 out_seq=0 2023-11-16T16:48:57.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.626+0000 7faffdffb700 1 -- 172.21.15.1:0/4170424912 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c5c2600660 con 0x55c5c274ecf0 2023-11-16T16:48:57.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.626+0000 7faffdffb700 10 client.16956 handle_command_reply: tid=0 2023-11-16T16:48:57.638 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.626+0000 7fb028f0eb80 1 client.16956 shutdown 2023-11-16T16:48:57.638 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.626+0000 7fb028f0eb80 1 -- 172.21.15.1:0/4170424912 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5c26e1470 msgr2=0x55c5c26efed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:57.638 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.626+0000 7fb028f0eb80 1 --2- 172.21.15.1:0/4170424912 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5c26e1470 0x55c5c26efed0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fb008096350 tx=0x7fb0080b8380 comp rx=0 tx=0).stop 2023-11-16T16:48:57.638 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.626+0000 7fb028f0eb80 1 -- 172.21.15.1:0/4170424912 shutdown_connections 2023-11-16T16:48:57.638 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.626+0000 7fb028f0eb80 1 --2- 172.21.15.1:0/4170424912 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55c5c274ecf0 0x55c5c27510d0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.638 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.626+0000 7fb028f0eb80 1 --2- 172.21.15.1:0/4170424912 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55c5c26e58b0 0x55c5c26f0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.638 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.626+0000 7fb028f0eb80 1 --2- 172.21.15.1:0/4170424912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5c26e0b60 0x55c5c26ef990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.638 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.626+0000 7fb028f0eb80 1 --2- 172.21.15.1:0/4170424912 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5c26e1470 0x55c5c26efed0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.639 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.626+0000 7faffdffb700 0 client.16956 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:48:57.639 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.626+0000 7fb028f0eb80 1 -- 172.21.15.1:0/4170424912 >> 172.21.15.1:0/4170424912 conn(0x55c5c26d6360 msgr2=0x55c5c26df960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:57.639 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.626+0000 7fb028f0eb80 1 -- 172.21.15.1:0/4170424912 shutdown_connections 2023-11-16T16:48:57.639 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.626+0000 7fb028f0eb80 1 -- 172.21.15.1:0/4170424912 wait complete. 2023-11-16T16:48:57.639 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.626+0000 7fb028f0eb80 20 client.16956 trim_cache size 0 max 16384 2023-11-16T16:48:57.639 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2023-11-16T16:48:57.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.627+0000 7fb01cd82700 1 -- 172.21.15.1:0/4061601655 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fb0000a12c0 msgr2=0x7fb0000a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:57.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.627+0000 7fb01cd82700 1 --2- 172.21.15.1:0/4061601655 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fb0000a12c0 0x7fb0000a3770 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb00c006010 tx=0x7fb00c00b260 comp rx=0 tx=0).stop 2023-11-16T16:48:57.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.627+0000 7fb01cd82700 1 -- 172.21.15.1:0/4061601655 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb018101de0 msgr2=0x7fb0180ff660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:57.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.627+0000 7fb01cd82700 1 --2- 172.21.15.1:0/4061601655 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb018101de0 0x7fb0180ff660 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fb00800c9b0 tx=0x7fb00800cd70 comp rx=0 tx=0).stop 2023-11-16T16:48:57.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.627+0000 7fb01cd82700 1 -- 172.21.15.1:0/4061601655 shutdown_connections 2023-11-16T16:48:57.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.627+0000 7fb01cd82700 1 --2- 172.21.15.1:0/4061601655 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fb0000a12c0 0x7fb0000a3770 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.627+0000 7fb01cd82700 1 --2- 172.21.15.1:0/4061601655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb01810a810 0x7fb0181a4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.627+0000 7fb01cd82700 1 --2- 172.21.15.1:0/4061601655 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fb0181014d0 0x7fb0180ff120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.627+0000 7fb01cd82700 1 --2- 172.21.15.1:0/4061601655 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb018101de0 0x7fb0180ff660 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:57.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.627+0000 7fb01cd82700 1 -- 172.21.15.1:0/4061601655 >> 172.21.15.1:0/4061601655 conn(0x7fb018076370 msgr2=0x7fb0180fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:57.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.627+0000 7fb01cd82700 1 -- 172.21.15.1:0/4061601655 shutdown_connections 2023-11-16T16:48:57.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:57.627+0000 7fb01cd82700 1 -- 172.21.15.1:0/4061601655 wait complete. 2023-11-16T16:48:57.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:57 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/4257917126' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:48:57.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:57 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/4257917126' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:48:58.643 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-16T16:48:58.643 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-16T16:48:58.673 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:58 smithi001 ceph-mon[98800]: pgmap v303: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:48:58.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:58 smithi026 ceph-mon[100112]: pgmap v303: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:48:58.792 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.787+0000 7f811843c700 1 -- 172.21.15.1:0/2182163181 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81101007b0 msgr2=0x7f8110103dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:58.792 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.787+0000 7f811843c700 1 --2- 172.21.15.1:0/2182163181 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81101007b0 0x7f8110103dd0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f81000099c0 tx=0x7f8100009cd0 comp rx=0 tx=0).stop 2023-11-16T16:48:58.792 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.787+0000 7f811843c700 1 -- 172.21.15.1:0/2182163181 shutdown_connections 2023-11-16T16:48:58.792 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.787+0000 7f811843c700 1 --2- 172.21.15.1:0/2182163181 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81101044b0 0x7f811010c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:58.793 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.787+0000 7f811843c700 1 --2- 172.21.15.1:0/2182163181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81100ffe10 0x7f81101001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:58.793 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.787+0000 7f811843c700 1 --2- 172.21.15.1:0/2182163181 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81101007b0 0x7f8110103dd0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:58.793 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.787+0000 7f811843c700 1 -- 172.21.15.1:0/2182163181 >> 172.21.15.1:0/2182163181 conn(0x7f81100748b0 msgr2=0x7f8110074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:58.793 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.787+0000 7f811843c700 1 -- 172.21.15.1:0/2182163181 shutdown_connections 2023-11-16T16:48:58.793 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.787+0000 7f811843c700 1 -- 172.21.15.1:0/2182163181 wait complete. 2023-11-16T16:48:58.794 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.788+0000 7f811843c700 1 Processor -- start 2023-11-16T16:48:58.794 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.788+0000 7f811843c700 1 -- start start 2023-11-16T16:48:58.794 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.788+0000 7f811843c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81100ffe10 0x7f8110100e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:58.794 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.788+0000 7f811843c700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81101007b0 0x7f81101013b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:58.794 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.788+0000 7f811843c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81101044b0 0x7f81101a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:58.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.788+0000 7f811843c700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8110101be0 con 0x7f81101044b0 2023-11-16T16:48:58.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.788+0000 7f811843c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81101a4b30 con 0x7f81100ffe10 2023-11-16T16:48:58.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.788+0000 7f811843c700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81101a4c70 con 0x7f81101007b0 2023-11-16T16:48:58.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.788+0000 7f81169d9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81101044b0 0x7f81101a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:58.795 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.788+0000 7f81169d9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81101044b0 0x7f81101a45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47116/0 (socket says 172.21.15.1:47116) 2023-11-16T16:48:58.796 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.788+0000 7f81169d9700 1 -- 172.21.15.1:0/2421424773 learned_addr learned my addr 172.21.15.1:0/2421424773 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:58.796 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.788+0000 7f81161d8700 1 --2- 172.21.15.1:0/2421424773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81100ffe10 0x7f8110100e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:58.796 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.788+0000 7f81159d7700 1 --2- 172.21.15.1:0/2421424773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81101007b0 0x7f81101013b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:58.796 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.788+0000 7f81169d9700 1 -- 172.21.15.1:0/2421424773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81101007b0 msgr2=0x7f81101013b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:58.796 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.788+0000 7f81169d9700 1 --2- 172.21.15.1:0/2421424773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81101007b0 0x7f81101013b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:58.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.788+0000 7f81169d9700 1 -- 172.21.15.1:0/2421424773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81100ffe10 msgr2=0x7f8110100e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:58.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.788+0000 7f81169d9700 1 --2- 172.21.15.1:0/2421424773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81100ffe10 0x7f8110100e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:58.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.788+0000 7f81169d9700 1 -- 172.21.15.1:0/2421424773 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81000095f0 con 0x7f81101044b0 2023-11-16T16:48:58.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.788+0000 7f81159d7700 1 --2- 172.21.15.1:0/2421424773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81101007b0 0x7f81101013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:48:58.797 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.789+0000 7f81169d9700 1 --2- 172.21.15.1:0/2421424773 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81101044b0 0x7f81101a45f0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f8108007ee0 tx=0x7f810800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:58.798 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.789+0000 7f81077fe700 1 -- 172.21.15.1:0/2421424773 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8108017070 con 0x7f81101044b0 2023-11-16T16:48:58.798 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.789+0000 7f811843c700 1 -- 172.21.15.1:0/2421424773 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f811010f270 con 0x7f81101044b0 2023-11-16T16:48:58.798 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.789+0000 7f81077fe700 1 -- 172.21.15.1:0/2421424773 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f810800f040 con 0x7f81101044b0 2023-11-16T16:48:58.798 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.789+0000 7f81077fe700 1 -- 172.21.15.1:0/2421424773 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8108005740 con 0x7f81101044b0 2023-11-16T16:48:58.798 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.789+0000 7f811843c700 1 -- 172.21.15.1:0/2421424773 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f811010f0e0 con 0x7f81101044b0 2023-11-16T16:48:58.807 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.792+0000 7f81077fe700 1 -- 172.21.15.1:0/2421424773 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f810800dc60 con 0x7f81101044b0 2023-11-16T16:48:58.807 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.792+0000 7f81077fe700 1 --2- 172.21.15.1:0/2421424773 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f80fc0a12c0 0x7f80fc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:58.808 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.792+0000 7f81077fe700 1 -- 172.21.15.1:0/2421424773 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f81080cc4e0 con 0x7f81101044b0 2023-11-16T16:48:58.808 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.792+0000 7f81161d8700 1 --2- 172.21.15.1:0/2421424773 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f80fc0a12c0 0x7f80fc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:58.808 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.793+0000 7f81161d8700 1 --2- 172.21.15.1:0/2421424773 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f80fc0a12c0 0x7f80fc0a3770 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f810c005950 tx=0x7f810c00b410 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:58.808 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.793+0000 7f81245c8b80 1 Processor -- start 2023-11-16T16:48:58.808 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.793+0000 7f81245c8b80 1 -- start start 2023-11-16T16:48:58.809 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.794+0000 7f81245c8b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3ad3e8be0 0x55b3ad3e8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:58.809 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.794+0000 7f81245c8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ad3e94f0 0x55b3ad3ed3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:58.809 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.794+0000 7f81245c8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad3ed930 0x55b3ad3edd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:58.809 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.794+0000 7f81245c8b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3ad2ee0e0 con 0x55b3ad3e8be0 2023-11-16T16:48:58.809 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.794+0000 7f81245c8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3ad340920 con 0x55b3ad3e94f0 2023-11-16T16:48:58.809 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.794+0000 7f81245c8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3ad339db0 con 0x55b3ad3ed930 2023-11-16T16:48:58.810 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.794+0000 7f81161d8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3ad3e8be0 0x55b3ad3e8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:58.810 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.794+0000 7f81161d8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3ad3e8be0 0x55b3ad3e8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47120/0 (socket says 172.21.15.1:47120) 2023-11-16T16:48:58.810 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.794+0000 7f81161d8700 1 -- 172.21.15.1:0/534615909 learned_addr learned my addr 172.21.15.1:0/534615909 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:58.810 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.794+0000 7f81159d7700 1 --2- 172.21.15.1:0/534615909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ad3e94f0 0x55b3ad3ed3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:58.810 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.794+0000 7f81169d9700 1 --2- 172.21.15.1:0/534615909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad3ed930 0x55b3ad3edd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:58.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.794+0000 7f81161d8700 1 -- 172.21.15.1:0/534615909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad3ed930 msgr2=0x55b3ad3edd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:58.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.794+0000 7f81161d8700 1 --2- 172.21.15.1:0/534615909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad3ed930 0x55b3ad3edd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:58.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.794+0000 7f81161d8700 1 -- 172.21.15.1:0/534615909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ad3e94f0 msgr2=0x55b3ad3ed3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:58.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.794+0000 7f81161d8700 1 --2- 172.21.15.1:0/534615909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ad3e94f0 0x55b3ad3ed3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:58.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.794+0000 7f81161d8700 1 -- 172.21.15.1:0/534615909 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3ad1bb6b0 con 0x55b3ad3e8be0 2023-11-16T16:48:58.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.794+0000 7f81159d7700 1 --2- 172.21.15.1:0/534615909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ad3e94f0 0x55b3ad3ed3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:48:58.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.794+0000 7f81169d9700 1 --2- 172.21.15.1:0/534615909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad3ed930 0x55b3ad3edd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:48:58.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.794+0000 7f81161d8700 1 --2- 172.21.15.1:0/534615909 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3ad3e8be0 0x55b3ad3e8fb0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f810c010fa0 tx=0x7f810c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2416f5be2e8b692 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:58.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.794+0000 7f8104dac700 1 -- 172.21.15.1:0/534615909 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810c01e620 con 0x55b3ad3e8be0 2023-11-16T16:48:58.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.794+0000 7f8104dac700 1 -- 172.21.15.1:0/534615909 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f810c015070 con 0x55b3ad3e8be0 2023-11-16T16:48:58.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.794+0000 7f8104dac700 1 -- 172.21.15.1:0/534615909 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810c01c9e0 con 0x55b3ad3e8be0 2023-11-16T16:48:58.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.795+0000 7f81245c8b80 1 -- 172.21.15.1:0/534615909 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3ad3e8be0 msgr2=0x55b3ad3e8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:58.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.795+0000 7f81245c8b80 1 --2- 172.21.15.1:0/534615909 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3ad3e8be0 0x55b3ad3e8fb0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f810c010fa0 tx=0x7f810c013000 comp rx=0 tx=0).stop 2023-11-16T16:48:58.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.795+0000 7f81245c8b80 1 -- 172.21.15.1:0/534615909 shutdown_connections 2023-11-16T16:48:58.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.795+0000 7f81245c8b80 1 --2- 172.21.15.1:0/534615909 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad3ed930 0x55b3ad3edd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:58.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.795+0000 7f81245c8b80 1 --2- 172.21.15.1:0/534615909 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3ad3e8be0 0x55b3ad3e8fb0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:58.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.795+0000 7f81245c8b80 1 --2- 172.21.15.1:0/534615909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ad3e94f0 0x55b3ad3ed3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:58.814 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.795+0000 7f81245c8b80 1 -- 172.21.15.1:0/534615909 >> 172.21.15.1:0/534615909 conn(0x55b3ad3de3e0 msgr2=0x55b3ad3e3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:58.814 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.795+0000 7f81245c8b80 1 -- 172.21.15.1:0/534615909 shutdown_connections 2023-11-16T16:48:58.814 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.795+0000 7f81245c8b80 1 -- 172.21.15.1:0/534615909 wait complete. 2023-11-16T16:48:58.814 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.796+0000 7f81245c8b80 1 -- start start 2023-11-16T16:48:58.814 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.796+0000 7f81245c8b80 1 Processor -- start 2023-11-16T16:48:58.815 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.796+0000 7f81245c8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ad3e8be0 0x55b3ad3f4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:58.815 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.796+0000 7f81245c8b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3ad3e94f0 0x55b3ad3f2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:58.815 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.796+0000 7f81245c8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad3ed930 0x55b3ad3f27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:58.815 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.796+0000 7f81245c8b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3ad2db960 con 0x55b3ad3e94f0 2023-11-16T16:48:58.815 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.796+0000 7f81245c8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3ad2db600 con 0x55b3ad3e8be0 2023-11-16T16:48:58.815 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.796+0000 7f81245c8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3ad1bd9a0 con 0x55b3ad3ed930 2023-11-16T16:48:58.816 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.796+0000 7f81245c8b80 10 client.0 _ll_register_callbacks cb 0x55b3ad2f9800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:48:58.816 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.796+0000 7f81159d7700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3ad3e94f0 0x55b3ad3f2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:58.816 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.796+0000 7f81159d7700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3ad3e94f0 0x55b3ad3f2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47128/0 (socket says 172.21.15.1:47128) 2023-11-16T16:48:58.816 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.796+0000 7f81159d7700 1 -- 172.21.15.1:0/3467074443 learned_addr learned my addr 172.21.15.1:0/3467074443 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:58.816 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.796+0000 7f81161d8700 1 --2- 172.21.15.1:0/3467074443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ad3e8be0 0x55b3ad3f4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:58.816 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.796+0000 7f81169d9700 1 --2- 172.21.15.1:0/3467074443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad3ed930 0x55b3ad3f27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:58.817 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.796+0000 7f81159d7700 1 -- 172.21.15.1:0/3467074443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad3ed930 msgr2=0x55b3ad3f27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:58.817 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.796+0000 7f81159d7700 1 --2- 172.21.15.1:0/3467074443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad3ed930 0x55b3ad3f27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:58.817 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.796+0000 7f81159d7700 1 -- 172.21.15.1:0/3467074443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ad3e8be0 msgr2=0x55b3ad3f4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:58.817 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.796+0000 7f81159d7700 1 --2- 172.21.15.1:0/3467074443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ad3e8be0 0x55b3ad3f4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:58.817 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.796+0000 7f81159d7700 1 -- 172.21.15.1:0/3467074443 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f810c010c50 con 0x55b3ad3e94f0 2023-11-16T16:48:58.817 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.796+0000 7f81169d9700 1 --2- 172.21.15.1:0/3467074443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad3ed930 0x55b3ad3f27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:48:58.818 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.797+0000 7f81159d7700 1 --2- 172.21.15.1:0/3467074443 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3ad3e94f0 0x55b3ad3f2250 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f8100000c00 tx=0x7f8100009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=124091a65132a201 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:58.818 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.797+0000 7f80f9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:48:58.818 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.797+0000 7f80f9ffb700 1 -- 172.21.15.1:0/3467074443 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810000bba0 con 0x55b3ad3e94f0 2023-11-16T16:48:58.818 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.797+0000 7f80f9ffb700 1 -- 172.21.15.1:0/3467074443 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f810000bd00 con 0x55b3ad3e94f0 2023-11-16T16:48:58.818 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.797+0000 7f81245c8b80 1 -- 172.21.15.1:0/3467074443 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b3ad3dbba0 con 0x55b3ad3e94f0 2023-11-16T16:48:58.819 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.797+0000 7f80f9ffb700 1 -- 172.21.15.1:0/3467074443 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f810000bec0 con 0x55b3ad3e94f0 2023-11-16T16:48:58.819 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.797+0000 7f81245c8b80 10 client.16968 fetch_fsmap learned FSMap version 32 2023-11-16T16:48:58.819 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.797+0000 7f81245c8b80 1 -- 172.21.15.1:0/3467074443 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55b3ad300210 con 0x55b3ad3e94f0 2023-11-16T16:48:58.819 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.797+0000 7f80f9ffb700 1 -- 172.21.15.1:0/3467074443 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8100030020 con 0x55b3ad3e94f0 2023-11-16T16:48:58.819 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.797+0000 7f80f9ffb700 1 -- 172.21.15.1:0/3467074443 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f81000047d0 con 0x55b3ad3e94f0 2023-11-16T16:48:58.819 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.797+0000 7f81245c8b80 10 client.16968 fetch_fsmap finished waiting for FSMap version 32 2023-11-16T16:48:58.819 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.797+0000 7f81245c8b80 10 client.16968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:48:58.820 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.798+0000 7f81245c8b80 20 client.16968 populate_metadata read hostname 'smithi001' 2023-11-16T16:48:58.820 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.798+0000 7f81245c8b80 1 --2- 172.21.15.1:0/3467074443 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55b3ad402830 0x55b3ad404c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:58.820 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.798+0000 7f81245c8b80 4 client.16968 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:48:58.820 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.798+0000 7f81245c8b80 1 -- 172.21.15.1:0/3467074443 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b3ad2fc120 con 0x55b3ad402830 2023-11-16T16:48:58.820 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.798+0000 7f81161d8700 1 --2- 172.21.15.1:0/3467074443 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55b3ad402830 0x55b3ad404c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:58.821 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.798+0000 7f80f9ffb700 10 client.16968 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:48:58.821 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.798+0000 7f81161d8700 1 --2- 172.21.15.1:0/3467074443 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55b3ad402830 0x55b3ad404c40 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad6aa8ccb5382faf server_cookie=c918b1e1442ceccd in_seq=0 out_seq=0 2023-11-16T16:48:58.821 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.799+0000 7f80f9ffb700 1 -- 172.21.15.1:0/3467074443 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b3ad2fc120 con 0x55b3ad402830 2023-11-16T16:48:58.821 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.799+0000 7f80f9ffb700 10 client.16968 handle_command_reply: tid=0 2023-11-16T16:48:58.821 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.799+0000 7f81245c8b80 1 client.16968 shutdown 2023-11-16T16:48:58.821 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.799+0000 7f81245c8b80 1 -- 172.21.15.1:0/3467074443 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3ad3e94f0 msgr2=0x55b3ad3f2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:58.822 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.799+0000 7f81245c8b80 1 --2- 172.21.15.1:0/3467074443 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3ad3e94f0 0x55b3ad3f2250 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f8100000c00 tx=0x7f8100009f70 comp rx=0 tx=0).stop 2023-11-16T16:48:58.822 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.799+0000 7f81245c8b80 1 -- 172.21.15.1:0/3467074443 shutdown_connections 2023-11-16T16:48:58.822 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.799+0000 7f81245c8b80 1 --2- 172.21.15.1:0/3467074443 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55b3ad402830 0x55b3ad404c40 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:58.822 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.799+0000 7f81245c8b80 1 --2- 172.21.15.1:0/3467074443 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad3ed930 0x55b3ad3f27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:58.822 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.799+0000 7f80f9ffb700 0 client.16968 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:48:58.822 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.799+0000 7f81245c8b80 1 --2- 172.21.15.1:0/3467074443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ad3e8be0 0x55b3ad3f4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:58.823 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.799+0000 7f81245c8b80 1 --2- 172.21.15.1:0/3467074443 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3ad3e94f0 0x55b3ad3f2250 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:58.823 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.799+0000 7f81245c8b80 1 -- 172.21.15.1:0/3467074443 >> 172.21.15.1:0/3467074443 conn(0x55b3ad3de3e0 msgr2=0x55b3ad3e2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:58.823 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.799+0000 7f81245c8b80 1 -- 172.21.15.1:0/3467074443 shutdown_connections 2023-11-16T16:48:58.823 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.799+0000 7f81245c8b80 1 -- 172.21.15.1:0/3467074443 wait complete. 2023-11-16T16:48:58.823 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.799+0000 7f81245c8b80 20 client.16968 trim_cache size 0 max 16384 2023-11-16T16:48:58.831 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.817+0000 7f81245c8b80 1 Processor -- start 2023-11-16T16:48:58.831 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.817+0000 7f81245c8b80 1 -- start start 2023-11-16T16:48:58.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.817+0000 7f81245c8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ad3e8be0 0x55b3ad3f9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:58.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.817+0000 7f81245c8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad3e94f0 0x55b3ad3f4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:58.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.817+0000 7f81245c8b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3ad3ed930 0x55b3ad3e35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:58.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.817+0000 7f81245c8b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3ad3e5400 con 0x55b3ad3ed930 2023-11-16T16:48:58.832 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.817+0000 7f81245c8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3ad3ddf60 con 0x55b3ad3e8be0 2023-11-16T16:48:58.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.817+0000 7f81245c8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3ad3ed480 con 0x55b3ad3e94f0 2023-11-16T16:48:58.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.817+0000 7f81169d9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3ad3ed930 0x55b3ad3e35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:58.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.817+0000 7f81169d9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3ad3ed930 0x55b3ad3e35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47138/0 (socket says 172.21.15.1:47138) 2023-11-16T16:48:58.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.817+0000 7f81169d9700 1 -- 172.21.15.1:0/2669538064 learned_addr learned my addr 172.21.15.1:0/2669538064 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:58.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.817+0000 7f81159d7700 1 --2- 172.21.15.1:0/2669538064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad3e94f0 0x55b3ad3f4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:58.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.817+0000 7f81161d8700 1 --2- 172.21.15.1:0/2669538064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ad3e8be0 0x55b3ad3f9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:58.833 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.817+0000 7f81169d9700 1 -- 172.21.15.1:0/2669538064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad3e94f0 msgr2=0x55b3ad3f4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:58.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.817+0000 7f81169d9700 1 --2- 172.21.15.1:0/2669538064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad3e94f0 0x55b3ad3f4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:58.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.817+0000 7f81169d9700 1 -- 172.21.15.1:0/2669538064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ad3e8be0 msgr2=0x55b3ad3f9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:58.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.817+0000 7f81169d9700 1 --2- 172.21.15.1:0/2669538064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ad3e8be0 0x55b3ad3f9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:58.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.817+0000 7f81169d9700 1 -- 172.21.15.1:0/2669538064 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3ad2fff60 con 0x55b3ad3ed930 2023-11-16T16:48:58.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.817+0000 7f81159d7700 1 --2- 172.21.15.1:0/2669538064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad3e94f0 0x55b3ad3f4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:48:58.834 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.817+0000 7f81161d8700 1 --2- 172.21.15.1:0/2669538064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ad3e8be0 0x55b3ad3f9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:48:58.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.817+0000 7f81169d9700 1 --2- 172.21.15.1:0/2669538064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3ad3ed930 0x55b3ad3e35e0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f810800d7b0 tx=0x7f81080b9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ed81b0de0525a1d server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:58.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.818+0000 7f81055ad700 1 -- 172.21.15.1:0/2669538064 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81080995a0 con 0x55b3ad3ed930 2023-11-16T16:48:58.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.818+0000 7f81055ad700 1 -- 172.21.15.1:0/2669538064 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f810809d5d0 con 0x55b3ad3ed930 2023-11-16T16:48:58.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.818+0000 7f81055ad700 1 -- 172.21.15.1:0/2669538064 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81080058a0 con 0x55b3ad3ed930 2023-11-16T16:48:58.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.818+0000 7f81245c8b80 1 -- 172.21.15.1:0/2669538064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3ad3ed930 msgr2=0x55b3ad3e35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:58.835 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.818+0000 7f81245c8b80 1 --2- 172.21.15.1:0/2669538064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3ad3ed930 0x55b3ad3e35e0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f810800d7b0 tx=0x7f81080b9040 comp rx=0 tx=0).stop 2023-11-16T16:48:58.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.818+0000 7f81245c8b80 1 -- 172.21.15.1:0/2669538064 shutdown_connections 2023-11-16T16:48:58.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.818+0000 7f81245c8b80 1 --2- 172.21.15.1:0/2669538064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3ad3ed930 0x55b3ad3e35e0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:58.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.818+0000 7f81245c8b80 1 --2- 172.21.15.1:0/2669538064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ad3e8be0 0x55b3ad3f9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:58.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.818+0000 7f81245c8b80 1 --2- 172.21.15.1:0/2669538064 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad3e94f0 0x55b3ad3f4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:58.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.818+0000 7f81245c8b80 1 -- 172.21.15.1:0/2669538064 >> 172.21.15.1:0/2669538064 conn(0x55b3ad3de3e0 msgr2=0x55b3ad3f3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:58.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.818+0000 7f81245c8b80 1 -- 172.21.15.1:0/2669538064 shutdown_connections 2023-11-16T16:48:58.836 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.818+0000 7f81245c8b80 1 -- 172.21.15.1:0/2669538064 wait complete. 2023-11-16T16:48:58.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.819+0000 7f81245c8b80 1 -- start start 2023-11-16T16:48:58.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.819+0000 7f81245c8b80 1 Processor -- start 2023-11-16T16:48:58.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.819+0000 7f81245c8b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad3e8be0 0x55b3ad450d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:58.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.819+0000 7f81245c8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ad3e94f0 0x55b3ad451270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:58.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.819+0000 7f81245c8b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3ad3ed930 0x55b3ad44b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:58.837 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.819+0000 7f81245c8b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3ad44b670 con 0x55b3ad3ed930 2023-11-16T16:48:58.838 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.819+0000 7f81245c8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3ad44b7e0 con 0x55b3ad3e94f0 2023-11-16T16:48:58.838 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.819+0000 7f81245c8b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3ad44b9a0 con 0x55b3ad3e8be0 2023-11-16T16:48:58.838 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.819+0000 7f81245c8b80 10 client.0 _ll_register_callbacks cb 0x7f8100009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:48:58.838 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.819+0000 7f81169d9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3ad3ed930 0x55b3ad44b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:58.838 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.819+0000 7f81169d9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3ad3ed930 0x55b3ad44b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47144/0 (socket says 172.21.15.1:47144) 2023-11-16T16:48:58.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.819+0000 7f81169d9700 1 -- 172.21.15.1:0/4135213586 learned_addr learned my addr 172.21.15.1:0/4135213586 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:58.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.819+0000 7f81161d8700 1 --2- 172.21.15.1:0/4135213586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad3e8be0 0x55b3ad450d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:58.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.819+0000 7f81159d7700 1 --2- 172.21.15.1:0/4135213586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ad3e94f0 0x55b3ad451270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:58.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.819+0000 7f81169d9700 1 -- 172.21.15.1:0/4135213586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad3e8be0 msgr2=0x55b3ad450d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:58.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.819+0000 7f81169d9700 1 --2- 172.21.15.1:0/4135213586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad3e8be0 0x55b3ad450d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:58.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.819+0000 7f81169d9700 1 -- 172.21.15.1:0/4135213586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ad3e94f0 msgr2=0x55b3ad451270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:58.839 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.819+0000 7f81169d9700 1 --2- 172.21.15.1:0/4135213586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ad3e94f0 0x55b3ad451270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:58.840 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.819+0000 7f81169d9700 1 -- 172.21.15.1:0/4135213586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f811010f0e0 con 0x55b3ad3ed930 2023-11-16T16:48:58.840 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.819+0000 7f81159d7700 1 --2- 172.21.15.1:0/4135213586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ad3e94f0 0x55b3ad451270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:48:58.840 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.820+0000 7f81169d9700 1 --2- 172.21.15.1:0/4135213586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3ad3ed930 0x55b3ad44b090 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f81080074d0 tx=0x7f810800dbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bfec4cfd65daf45 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:58.840 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.820+0000 7f80f9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:48:58.840 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.820+0000 7f80f9ffb700 1 -- 172.21.15.1:0/4135213586 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81080b7070 con 0x55b3ad3ed930 2023-11-16T16:48:58.840 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.820+0000 7f80f9ffb700 1 -- 172.21.15.1:0/4135213586 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f8108005aa0 con 0x55b3ad3ed930 2023-11-16T16:48:58.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.820+0000 7f81245c8b80 1 -- 172.21.15.1:0/4135213586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f810c015070 con 0x55b3ad3ed930 2023-11-16T16:48:58.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.820+0000 7f80f9ffb700 1 -- 172.21.15.1:0/4135213586 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8108084260 con 0x55b3ad3ed930 2023-11-16T16:48:58.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.820+0000 7f81245c8b80 10 client.16974 fetch_fsmap learned FSMap version 32 2023-11-16T16:48:58.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.820+0000 7f81245c8b80 1 -- 172.21.15.1:0/4135213586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55b3ad150ce0 con 0x55b3ad3ed930 2023-11-16T16:48:58.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.820+0000 7f80f9ffb700 1 -- 172.21.15.1:0/4135213586 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8108063080 con 0x55b3ad3ed930 2023-11-16T16:48:58.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.820+0000 7f80f9ffb700 1 -- 172.21.15.1:0/4135213586 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f81080998e0 con 0x55b3ad3ed930 2023-11-16T16:48:58.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.820+0000 7f81245c8b80 10 client.16974 fetch_fsmap finished waiting for FSMap version 32 2023-11-16T16:48:58.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.820+0000 7f81245c8b80 10 client.16974 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:48:58.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.820+0000 7f81245c8b80 20 client.16974 populate_metadata read hostname 'smithi001' 2023-11-16T16:48:58.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.820+0000 7f81245c8b80 1 --2- 172.21.15.1:0/4135213586 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55b3ad458710 0x55b3ad45aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:58.842 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.820+0000 7f81245c8b80 4 client.16974 mds_command: new command op to 24521 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-16T16:48:58.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.820+0000 7f81245c8b80 1 -- 172.21.15.1:0/4135213586 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b3ad3086e0 con 0x55b3ad458710 2023-11-16T16:48:58.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.820+0000 7f81161d8700 1 --2- 172.21.15.1:0/4135213586 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55b3ad458710 0x55b3ad45aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:58.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.821+0000 7f81161d8700 1 --2- 172.21.15.1:0/4135213586 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55b3ad458710 0x55b3ad45aaf0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8e985fe131513ff server_cookie=50d26a3d893187c7 in_seq=0 out_seq=0 2023-11-16T16:48:58.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.821+0000 7f80f9ffb700 10 client.16974 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:48:58.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.822+0000 7f80f9ffb700 1 -- 172.21.15.1:0/4135213586 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b3ad3086e0 con 0x55b3ad458710 2023-11-16T16:48:58.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.822+0000 7f80f9ffb700 10 client.16974 handle_command_reply: tid=0 2023-11-16T16:48:58.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.822+0000 7f81245c8b80 1 client.16974 shutdown 2023-11-16T16:48:58.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.822+0000 7f81245c8b80 1 -- 172.21.15.1:0/4135213586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3ad3ed930 msgr2=0x55b3ad44b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:58.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.822+0000 7f81245c8b80 1 --2- 172.21.15.1:0/4135213586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3ad3ed930 0x55b3ad44b090 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f81080074d0 tx=0x7f810800dbd0 comp rx=0 tx=0).stop 2023-11-16T16:48:58.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.822+0000 7f81245c8b80 1 -- 172.21.15.1:0/4135213586 shutdown_connections 2023-11-16T16:48:58.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.822+0000 7f81245c8b80 1 --2- 172.21.15.1:0/4135213586 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55b3ad458710 0x55b3ad45aaf0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:58.844 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.822+0000 7f81245c8b80 1 --2- 172.21.15.1:0/4135213586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3ad3ed930 0x55b3ad44b090 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:58.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.822+0000 7f80f9ffb700 0 client.16974 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:48:58.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.822+0000 7f81245c8b80 1 --2- 172.21.15.1:0/4135213586 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55b3ad3e8be0 0x55b3ad450d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:58.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.822+0000 7f81245c8b80 1 --2- 172.21.15.1:0/4135213586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b3ad3e94f0 0x55b3ad451270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:58.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.822+0000 7f81245c8b80 1 -- 172.21.15.1:0/4135213586 >> 172.21.15.1:0/4135213586 conn(0x55b3ad3de3e0 msgr2=0x55b3ad3e8270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:58.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.823+0000 7f81245c8b80 1 -- 172.21.15.1:0/4135213586 shutdown_connections 2023-11-16T16:48:58.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.823+0000 7f81245c8b80 1 -- 172.21.15.1:0/4135213586 wait complete. 2023-11-16T16:48:58.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.823+0000 7f81245c8b80 20 client.16974 trim_cache size 0 max 16384 2023-11-16T16:48:58.846 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:48:58.846 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2023-11-16T16:48:58.846 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "e8cd662f-4bd3-401d-9c70-1bc1f2776848", 2023-11-16T16:48:58.847 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2023-11-16T16:48:58.847 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:48:58.848 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.823+0000 7f811843c700 1 -- 172.21.15.1:0/2421424773 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f80fc0a12c0 msgr2=0x7f80fc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:58.848 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.823+0000 7f811843c700 1 --2- 172.21.15.1:0/2421424773 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f80fc0a12c0 0x7f80fc0a3770 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f810c005950 tx=0x7f810c00b410 comp rx=0 tx=0).stop 2023-11-16T16:48:58.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.823+0000 7f811843c700 1 -- 172.21.15.1:0/2421424773 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81101044b0 msgr2=0x7f81101a45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:58.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.823+0000 7f811843c700 1 --2- 172.21.15.1:0/2421424773 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81101044b0 0x7f81101a45f0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f8108007ee0 tx=0x7f810800d3b0 comp rx=0 tx=0).stop 2023-11-16T16:48:58.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.823+0000 7f811843c700 1 -- 172.21.15.1:0/2421424773 shutdown_connections 2023-11-16T16:48:58.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.823+0000 7f811843c700 1 --2- 172.21.15.1:0/2421424773 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f80fc0a12c0 0x7f80fc0a3770 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:58.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.823+0000 7f811843c700 1 --2- 172.21.15.1:0/2421424773 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81101044b0 0x7f81101a45f0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:58.849 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.823+0000 7f811843c700 1 --2- 172.21.15.1:0/2421424773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81100ffe10 0x7f8110100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:58.850 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.823+0000 7f811843c700 1 --2- 172.21.15.1:0/2421424773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f81101007b0 0x7f81101013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:58.850 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.823+0000 7f811843c700 1 -- 172.21.15.1:0/2421424773 >> 172.21.15.1:0/2421424773 conn(0x7f81100748b0 msgr2=0x7f81100fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:58.850 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.823+0000 7f811843c700 1 -- 172.21.15.1:0/2421424773 shutdown_connections 2023-11-16T16:48:58.850 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:58.823+0000 7f811843c700 1 -- 172.21.15.1:0/2421424773 wait complete. 2023-11-16T16:48:58.851 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-16T16:48:58.851 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:58 smithi053 ceph-mon[100638]: pgmap v303: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:48:59.019 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.015+0000 7fe6d1fb1700 1 -- 172.21.15.1:0/2406119752 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6cc1055f0 msgr2=0x7fe6cc10d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:59.020 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.015+0000 7fe6d1fb1700 1 --2- 172.21.15.1:0/2406119752 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6cc1055f0 0x7fe6cc10d150 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fe6c400b0d0 tx=0x7fe6c400b3e0 comp rx=0 tx=0).stop 2023-11-16T16:48:59.020 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.015+0000 7fe6d1fb1700 1 -- 172.21.15.1:0/2406119752 shutdown_connections 2023-11-16T16:48:59.020 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.015+0000 7fe6d1fb1700 1 --2- 172.21.15.1:0/2406119752 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6cc1055f0 0x7fe6cc10d150 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:59.020 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.015+0000 7fe6d1fb1700 1 --2- 172.21.15.1:0/2406119752 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6cc0fea50 0x7fe6cc0fee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:59.020 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.015+0000 7fe6d1fb1700 1 --2- 172.21.15.1:0/2406119752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6cc0ff3f0 0x7fe6cc104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:59.021 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.015+0000 7fe6d1fb1700 1 -- 172.21.15.1:0/2406119752 >> 172.21.15.1:0/2406119752 conn(0x7fe6cc075880 msgr2=0x7fe6cc075c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:59.021 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.015+0000 7fe6d1fb1700 1 -- 172.21.15.1:0/2406119752 shutdown_connections 2023-11-16T16:48:59.021 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.015+0000 7fe6d1fb1700 1 -- 172.21.15.1:0/2406119752 wait complete. 2023-11-16T16:48:59.021 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.016+0000 7fe6d1fb1700 1 Processor -- start 2023-11-16T16:48:59.021 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.016+0000 7fe6d1fb1700 1 -- start start 2023-11-16T16:48:59.021 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.016+0000 7fe6d1fb1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6cc0fea50 0x7fe6cc1963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:59.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.016+0000 7fe6d1fb1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6cc0ff3f0 0x7fe6cc1968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:59.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.016+0000 7fe6d1fb1700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6cc1055f0 0x7fe6cc19ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:59.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.016+0000 7fe6d1fb1700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc19b380 con 0x7fe6cc0fea50 2023-11-16T16:48:59.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.016+0000 7fe6d1fb1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc19b4f0 con 0x7fe6cc0ff3f0 2023-11-16T16:48:59.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.016+0000 7fe6d1fb1700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc19b6b0 con 0x7fe6cc1055f0 2023-11-16T16:48:59.022 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.017+0000 7fe6cb7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6cc0fea50 0x7fe6cc1963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:59.023 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.017+0000 7fe6cb7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6cc0fea50 0x7fe6cc1963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47154/0 (socket says 172.21.15.1:47154) 2023-11-16T16:48:59.023 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.017+0000 7fe6cb7fe700 1 -- 172.21.15.1:0/821476018 learned_addr learned my addr 172.21.15.1:0/821476018 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:59.023 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.017+0000 7fe6caffd700 1 --2- 172.21.15.1:0/821476018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6cc0ff3f0 0x7fe6cc1968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:59.023 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.017+0000 7fe6cbfff700 1 --2- 172.21.15.1:0/821476018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6cc1055f0 0x7fe6cc19ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:59.023 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.017+0000 7fe6cb7fe700 1 -- 172.21.15.1:0/821476018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6cc1055f0 msgr2=0x7fe6cc19ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:59.023 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.017+0000 7fe6cb7fe700 1 --2- 172.21.15.1:0/821476018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6cc1055f0 0x7fe6cc19ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:59.024 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.017+0000 7fe6cb7fe700 1 -- 172.21.15.1:0/821476018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6cc0ff3f0 msgr2=0x7fe6cc1968e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:59.024 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.017+0000 7fe6cb7fe700 1 --2- 172.21.15.1:0/821476018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6cc0ff3f0 0x7fe6cc1968e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:59.024 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.017+0000 7fe6cb7fe700 1 -- 172.21.15.1:0/821476018 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c4009d00 con 0x7fe6cc0fea50 2023-11-16T16:48:59.024 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.017+0000 7fe6cb7fe700 1 --2- 172.21.15.1:0/821476018 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6cc0fea50 0x7fe6cc1963a0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc00d980 tx=0x7fe6bc00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:59.024 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.017+0000 7fe6c8ff9700 1 -- 172.21.15.1:0/821476018 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc009b40 con 0x7fe6cc0fea50 2023-11-16T16:48:59.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.017+0000 7fe6c8ff9700 1 -- 172.21.15.1:0/821476018 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6bc00de60 con 0x7fe6cc0fea50 2023-11-16T16:48:59.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.017+0000 7fe6d1fb1700 1 -- 172.21.15.1:0/821476018 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6cc110350 con 0x7fe6cc0fea50 2023-11-16T16:48:59.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.017+0000 7fe6c8ff9700 1 -- 172.21.15.1:0/821476018 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc005580 con 0x7fe6cc0fea50 2023-11-16T16:48:59.025 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.017+0000 7fe6d1fb1700 1 -- 172.21.15.1:0/821476018 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6cc1101c0 con 0x7fe6cc0fea50 2023-11-16T16:48:59.032 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.019+0000 7fe6c8ff9700 1 -- 172.21.15.1:0/821476018 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fe6bc004750 con 0x7fe6cc0fea50 2023-11-16T16:48:59.032 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.020+0000 7fe6c8ff9700 1 --2- 172.21.15.1:0/821476018 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fe6b40a12c0 0x7fe6b40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:59.032 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.020+0000 7fe6caffd700 1 --2- 172.21.15.1:0/821476018 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fe6b40a12c0 0x7fe6b40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:59.033 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.020+0000 7fe6c8ff9700 1 -- 172.21.15.1:0/821476018 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fe6bc0ccd50 con 0x7fe6cc0fea50 2023-11-16T16:48:59.033 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.021+0000 7fe6caffd700 1 --2- 172.21.15.1:0/821476018 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fe6b40a12c0 0x7fe6b40a3770 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0005ea0 tx=0x7fe6c0005e30 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:59.033 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.021+0000 7fe6de13db80 1 Processor -- start 2023-11-16T16:48:59.033 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.021+0000 7fe6de13db80 1 -- start start 2023-11-16T16:48:59.033 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.021+0000 7fe6de13db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c3d2cf70 0x5640c3d2d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:59.034 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.021+0000 7fe6de13db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640c3d2d880 0x5640c3d31780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:59.034 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.021+0000 7fe6de13db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640c3d31cc0 0x5640c3d32110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:59.034 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.021+0000 7fe6de13db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c3c2e060 con 0x5640c3d31cc0 2023-11-16T16:48:59.034 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.021+0000 7fe6de13db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c3c808a0 con 0x5640c3d2d880 2023-11-16T16:48:59.034 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.021+0000 7fe6de13db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c3c79d30 con 0x5640c3d2cf70 2023-11-16T16:48:59.034 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.021+0000 7fe6cbfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640c3d31cc0 0x5640c3d32110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:59.034 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.021+0000 7fe6cbfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640c3d31cc0 0x5640c3d32110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47166/0 (socket says 172.21.15.1:47166) 2023-11-16T16:48:59.035 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.021+0000 7fe6cbfff700 1 -- 172.21.15.1:0/1599955715 learned_addr learned my addr 172.21.15.1:0/1599955715 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:59.035 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.021+0000 7fe6cb7fe700 1 --2- 172.21.15.1:0/1599955715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c3d2cf70 0x5640c3d2d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:59.035 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.021+0000 7fe6caffd700 1 --2- 172.21.15.1:0/1599955715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640c3d2d880 0x5640c3d31780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:59.035 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.021+0000 7fe6cbfff700 1 -- 172.21.15.1:0/1599955715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c3d2cf70 msgr2=0x5640c3d2d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:59.035 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.021+0000 7fe6cbfff700 1 --2- 172.21.15.1:0/1599955715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c3d2cf70 0x5640c3d2d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:59.035 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.021+0000 7fe6cbfff700 1 -- 172.21.15.1:0/1599955715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640c3d2d880 msgr2=0x5640c3d31780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:59.036 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.021+0000 7fe6cbfff700 1 --2- 172.21.15.1:0/1599955715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640c3d2d880 0x5640c3d31780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:59.036 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.021+0000 7fe6cbfff700 1 -- 172.21.15.1:0/1599955715 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640c3afb630 con 0x5640c3d31cc0 2023-11-16T16:48:59.036 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.021+0000 7fe6caffd700 1 --2- 172.21.15.1:0/1599955715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640c3d2d880 0x5640c3d31780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:48:59.036 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.022+0000 7fe6cbfff700 1 --2- 172.21.15.1:0/1599955715 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640c3d31cc0 0x5640c3d32110 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fe6c400cda0 tx=0x7fe6c400cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d06e648cbd9d24 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:59.036 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.022+0000 7fe6b1ffb700 1 -- 172.21.15.1:0/1599955715 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c400e050 con 0x5640c3d31cc0 2023-11-16T16:48:59.036 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.022+0000 7fe6b1ffb700 1 -- 172.21.15.1:0/1599955715 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6c4009b80 con 0x5640c3d31cc0 2023-11-16T16:48:59.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.022+0000 7fe6b1ffb700 1 -- 172.21.15.1:0/1599955715 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4005280 con 0x5640c3d31cc0 2023-11-16T16:48:59.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.023+0000 7fe6de13db80 1 -- 172.21.15.1:0/1599955715 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640c3d31cc0 msgr2=0x5640c3d32110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:59.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.023+0000 7fe6de13db80 1 --2- 172.21.15.1:0/1599955715 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640c3d31cc0 0x5640c3d32110 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fe6c400cda0 tx=0x7fe6c400cdd0 comp rx=0 tx=0).stop 2023-11-16T16:48:59.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.023+0000 7fe6de13db80 1 -- 172.21.15.1:0/1599955715 shutdown_connections 2023-11-16T16:48:59.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.023+0000 7fe6de13db80 1 --2- 172.21.15.1:0/1599955715 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640c3d31cc0 0x5640c3d32110 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:59.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.023+0000 7fe6de13db80 1 --2- 172.21.15.1:0/1599955715 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c3d2cf70 0x5640c3d2d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:59.037 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.023+0000 7fe6de13db80 1 --2- 172.21.15.1:0/1599955715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640c3d2d880 0x5640c3d31780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:59.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.023+0000 7fe6de13db80 1 -- 172.21.15.1:0/1599955715 >> 172.21.15.1:0/1599955715 conn(0x5640c3d1e360 msgr2=0x5640c3d23b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:59.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.023+0000 7fe6de13db80 1 -- 172.21.15.1:0/1599955715 shutdown_connections 2023-11-16T16:48:59.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.023+0000 7fe6de13db80 1 -- 172.21.15.1:0/1599955715 wait complete. 2023-11-16T16:48:59.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.023+0000 7fe6de13db80 1 -- start start 2023-11-16T16:48:59.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.023+0000 7fe6de13db80 1 Processor -- start 2023-11-16T16:48:59.038 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.024+0000 7fe6de13db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c3d2cf70 0x5640c3d39450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:59.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.024+0000 7fe6de13db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640c3d2d880 0x5640c3d36660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:59.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.024+0000 7fe6de13db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640c3d31cc0 0x5640c3d36c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:59.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.024+0000 7fe6de13db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c3c1b8e0 con 0x5640c3d31cc0 2023-11-16T16:48:59.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.024+0000 7fe6de13db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c3c1b580 con 0x5640c3d2d880 2023-11-16T16:48:59.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.024+0000 7fe6de13db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c3afd920 con 0x5640c3d2cf70 2023-11-16T16:48:59.039 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.024+0000 7fe6de13db80 10 client.0 _ll_register_callbacks cb 0x5640c3c39780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:48:59.040 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.024+0000 7fe6cbfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640c3d31cc0 0x5640c3d36c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:59.040 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.024+0000 7fe6cbfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640c3d31cc0 0x5640c3d36c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47172/0 (socket says 172.21.15.1:47172) 2023-11-16T16:48:59.040 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.024+0000 7fe6cbfff700 1 -- 172.21.15.1:0/863121671 learned_addr learned my addr 172.21.15.1:0/863121671 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:59.040 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.024+0000 7fe6cb7fe700 1 --2- 172.21.15.1:0/863121671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c3d2cf70 0x5640c3d39450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:59.040 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.024+0000 7fe6caffd700 1 --2- 172.21.15.1:0/863121671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640c3d2d880 0x5640c3d36660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:59.040 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.024+0000 7fe6cbfff700 1 -- 172.21.15.1:0/863121671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c3d2cf70 msgr2=0x5640c3d39450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:59.041 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.024+0000 7fe6cbfff700 1 --2- 172.21.15.1:0/863121671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c3d2cf70 0x5640c3d39450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:59.041 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.024+0000 7fe6cbfff700 1 -- 172.21.15.1:0/863121671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640c3d2d880 msgr2=0x5640c3d36660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:59.041 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.024+0000 7fe6cbfff700 1 --2- 172.21.15.1:0/863121671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640c3d2d880 0x5640c3d36660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:59.041 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.024+0000 7fe6cbfff700 1 -- 172.21.15.1:0/863121671 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6cc10d960 con 0x5640c3d31cc0 2023-11-16T16:48:59.041 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.024+0000 7fe6cbfff700 1 --2- 172.21.15.1:0/863121671 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640c3d31cc0 0x5640c3d36c70 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fe6c40078c0 tx=0x7fe6c4007510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4441b06d587dfa8 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:59.041 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.024+0000 7fe6aaffd700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:48:59.042 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.024+0000 7fe6aaffd700 1 -- 172.21.15.1:0/863121671 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4026450 con 0x5640c3d31cc0 2023-11-16T16:48:59.042 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.024+0000 7fe6aaffd700 1 -- 172.21.15.1:0/863121671 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fe6c40265b0 con 0x5640c3d31cc0 2023-11-16T16:48:59.042 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.024+0000 7fe6de13db80 1 -- 172.21.15.1:0/863121671 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5640c3c3c0a0 con 0x5640c3d31cc0 2023-11-16T16:48:59.042 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.025+0000 7fe6aaffd700 1 -- 172.21.15.1:0/863121671 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6c4026770 con 0x5640c3d31cc0 2023-11-16T16:48:59.042 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.025+0000 7fe6de13db80 10 client.16986 fetch_fsmap learned FSMap version 32 2023-11-16T16:48:59.042 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.025+0000 7fe6de13db80 1 -- 172.21.15.1:0/863121671 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5640c3c40190 con 0x5640c3d31cc0 2023-11-16T16:48:59.042 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.025+0000 7fe6aaffd700 1 -- 172.21.15.1:0/863121671 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5640c3c3c0a0 con 0x5640c3d31cc0 2023-11-16T16:48:59.043 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.025+0000 7fe6aaffd700 1 -- 172.21.15.1:0/863121671 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fe6c400b100 con 0x5640c3d31cc0 2023-11-16T16:48:59.043 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.025+0000 7fe6de13db80 10 client.16986 fetch_fsmap finished waiting for FSMap version 32 2023-11-16T16:48:59.043 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.025+0000 7fe6de13db80 10 client.16986 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:48:59.043 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.025+0000 7fe6de13db80 20 client.16986 populate_metadata read hostname 'smithi001' 2023-11-16T16:48:59.043 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.025+0000 7fe6de13db80 1 --2- 172.21.15.1:0/863121671 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5640c3d46750 0x5640c3d48b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:59.043 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.025+0000 7fe6de13db80 4 client.16986 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:48:59.044 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.025+0000 7fe6de13db80 1 -- 172.21.15.1:0/863121671 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5640c3c48660 con 0x5640c3d46750 2023-11-16T16:48:59.044 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.025+0000 7fe6cb7fe700 1 --2- 172.21.15.1:0/863121671 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5640c3d46750 0x5640c3d48b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:59.044 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.026+0000 7fe6aaffd700 10 client.16986 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:48:59.044 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.026+0000 7fe6cb7fe700 1 --2- 172.21.15.1:0/863121671 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5640c3d46750 0x5640c3d48b60 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1416428e327e7ef3 server_cookie=62350378041b15ab in_seq=0 out_seq=0 2023-11-16T16:48:59.044 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.026+0000 7fe6aaffd700 1 -- 172.21.15.1:0/863121671 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5640c3c48660 con 0x5640c3d46750 2023-11-16T16:48:59.044 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.026+0000 7fe6aaffd700 10 client.16986 handle_command_reply: tid=0 2023-11-16T16:48:59.044 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.026+0000 7fe6de13db80 1 client.16986 shutdown 2023-11-16T16:48:59.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.026+0000 7fe6de13db80 1 -- 172.21.15.1:0/863121671 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640c3d31cc0 msgr2=0x5640c3d36c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:59.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.026+0000 7fe6de13db80 1 --2- 172.21.15.1:0/863121671 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640c3d31cc0 0x5640c3d36c70 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fe6c40078c0 tx=0x7fe6c4007510 comp rx=0 tx=0).stop 2023-11-16T16:48:59.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.026+0000 7fe6de13db80 1 -- 172.21.15.1:0/863121671 shutdown_connections 2023-11-16T16:48:59.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.026+0000 7fe6de13db80 1 --2- 172.21.15.1:0/863121671 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5640c3d46750 0x5640c3d48b60 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:59.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.026+0000 7fe6de13db80 1 --2- 172.21.15.1:0/863121671 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640c3d31cc0 0x5640c3d36c70 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:59.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.026+0000 7fe6aaffd700 0 client.16986 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:48:59.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.026+0000 7fe6de13db80 1 --2- 172.21.15.1:0/863121671 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c3d2cf70 0x5640c3d39450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:59.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.026+0000 7fe6de13db80 1 --2- 172.21.15.1:0/863121671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640c3d2d880 0x5640c3d36660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:59.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.027+0000 7fe6de13db80 1 -- 172.21.15.1:0/863121671 >> 172.21.15.1:0/863121671 conn(0x5640c3d1e360 msgr2=0x5640c3d33170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:59.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.027+0000 7fe6de13db80 1 -- 172.21.15.1:0/863121671 shutdown_connections 2023-11-16T16:48:59.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.027+0000 7fe6de13db80 1 -- 172.21.15.1:0/863121671 wait complete. 2023-11-16T16:48:59.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.027+0000 7fe6de13db80 20 client.16986 trim_cache size 0 max 16384 2023-11-16T16:48:59.052 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.044+0000 7fe6de13db80 1 Processor -- start 2023-11-16T16:48:59.052 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.044+0000 7fe6de13db80 1 -- start start 2023-11-16T16:48:59.053 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.044+0000 7fe6de13db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640c3d2cf70 0x5640c3d23c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:59.053 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.044+0000 7fe6de13db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c3d2d880 0x5640c3d3dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:59.053 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.044+0000 7fe6de13db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640c3d31cc0 0x5640c3d38f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:59.053 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.044+0000 7fe6de13db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c3d34c10 con 0x5640c3d31cc0 2023-11-16T16:48:59.053 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.044+0000 7fe6de13db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c3d307d0 con 0x5640c3d2cf70 2023-11-16T16:48:59.053 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.044+0000 7fe6de13db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c3d39a60 con 0x5640c3d2d880 2023-11-16T16:48:59.054 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.044+0000 7fe6cbfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640c3d31cc0 0x5640c3d38f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:59.054 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.044+0000 7fe6cbfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640c3d31cc0 0x5640c3d38f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47180/0 (socket says 172.21.15.1:47180) 2023-11-16T16:48:59.054 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.044+0000 7fe6cbfff700 1 -- 172.21.15.1:0/4290268442 learned_addr learned my addr 172.21.15.1:0/4290268442 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:59.054 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.044+0000 7fe6cbfff700 1 -- 172.21.15.1:0/4290268442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c3d2d880 msgr2=0x5640c3d3dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:59.054 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.044+0000 7fe6cbfff700 1 --2- 172.21.15.1:0/4290268442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c3d2d880 0x5640c3d3dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:59.054 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.044+0000 7fe6cbfff700 1 -- 172.21.15.1:0/4290268442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640c3d2cf70 msgr2=0x5640c3d23c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:59.055 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.044+0000 7fe6cb7fe700 1 --2- 172.21.15.1:0/4290268442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640c3d2cf70 0x5640c3d23c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:59.055 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.044+0000 7fe6cbfff700 1 --2- 172.21.15.1:0/4290268442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640c3d2cf70 0x5640c3d23c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:59.055 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.044+0000 7fe6cbfff700 1 -- 172.21.15.1:0/4290268442 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640c3d361d0 con 0x5640c3d31cc0 2023-11-16T16:48:59.055 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.044+0000 7fe6cb7fe700 1 --2- 172.21.15.1:0/4290268442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640c3d2cf70 0x5640c3d23c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:48:59.055 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.045+0000 7fe6cbfff700 1 --2- 172.21.15.1:0/4290268442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640c3d31cc0 0x5640c3d38f80 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4002230 tx=0x7fe6c4007600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f880789d92df4710 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:59.055 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.045+0000 7fe6b27fc700 1 -- 172.21.15.1:0/4290268442 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c400e030 con 0x5640c3d31cc0 2023-11-16T16:48:59.055 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.045+0000 7fe6b27fc700 1 -- 172.21.15.1:0/4290268442 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6c4006080 con 0x5640c3d31cc0 2023-11-16T16:48:59.056 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.045+0000 7fe6b27fc700 1 -- 172.21.15.1:0/4290268442 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c40336b0 con 0x5640c3d31cc0 2023-11-16T16:48:59.056 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.045+0000 7fe6de13db80 1 -- 172.21.15.1:0/4290268442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640c3d31cc0 msgr2=0x5640c3d38f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:59.056 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.045+0000 7fe6de13db80 1 --2- 172.21.15.1:0/4290268442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640c3d31cc0 0x5640c3d38f80 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4002230 tx=0x7fe6c4007600 comp rx=0 tx=0).stop 2023-11-16T16:48:59.056 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.045+0000 7fe6de13db80 1 -- 172.21.15.1:0/4290268442 shutdown_connections 2023-11-16T16:48:59.056 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.045+0000 7fe6de13db80 1 --2- 172.21.15.1:0/4290268442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640c3d31cc0 0x5640c3d38f80 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:59.056 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.045+0000 7fe6de13db80 1 --2- 172.21.15.1:0/4290268442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640c3d2cf70 0x5640c3d23c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:59.057 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.045+0000 7fe6de13db80 1 --2- 172.21.15.1:0/4290268442 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c3d2d880 0x5640c3d3dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:59.057 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.045+0000 7fe6de13db80 1 -- 172.21.15.1:0/4290268442 >> 172.21.15.1:0/4290268442 conn(0x5640c3d1e360 msgr2=0x5640c3d340a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:59.057 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.045+0000 7fe6de13db80 1 -- 172.21.15.1:0/4290268442 shutdown_connections 2023-11-16T16:48:59.057 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.045+0000 7fe6de13db80 1 -- 172.21.15.1:0/4290268442 wait complete. 2023-11-16T16:48:59.057 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.046+0000 7fe6de13db80 1 -- start start 2023-11-16T16:48:59.057 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.046+0000 7fe6de13db80 1 Processor -- start 2023-11-16T16:48:59.057 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.046+0000 7fe6de13db80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c3d2cf70 0x5640c3d3bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:59.058 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.046+0000 7fe6de13db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640c3d2d880 0x5640c3d3c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:59.058 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.046+0000 7fe6de13db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640c3d31cc0 0x5640c3d935e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:59.058 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.046+0000 7fe6de13db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c3d93b20 con 0x5640c3d2d880 2023-11-16T16:48:59.058 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.046+0000 7fe6de13db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c3d93c90 con 0x5640c3d31cc0 2023-11-16T16:48:59.058 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.046+0000 7fe6de13db80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c3d93e50 con 0x5640c3d2cf70 2023-11-16T16:48:59.058 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.046+0000 7fe6de13db80 10 client.0 _ll_register_callbacks cb 0x7fe6c4009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:48:59.058 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.046+0000 7fe6caffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640c3d2d880 0x5640c3d3c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:59.059 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.046+0000 7fe6caffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640c3d2d880 0x5640c3d3c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47184/0 (socket says 172.21.15.1:47184) 2023-11-16T16:48:59.059 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.046+0000 7fe6caffd700 1 -- 172.21.15.1:0/806918370 learned_addr learned my addr 172.21.15.1:0/806918370 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:48:59.059 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.046+0000 7fe6cbfff700 1 --2- 172.21.15.1:0/806918370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640c3d31cc0 0x5640c3d935e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:59.059 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.046+0000 7fe6cb7fe700 1 --2- 172.21.15.1:0/806918370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c3d2cf70 0x5640c3d3bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:59.059 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.047+0000 7fe6caffd700 1 -- 172.21.15.1:0/806918370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c3d2cf70 msgr2=0x5640c3d3bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:59.059 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.047+0000 7fe6caffd700 1 --2- 172.21.15.1:0/806918370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c3d2cf70 0x5640c3d3bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:59.060 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.047+0000 7fe6caffd700 1 -- 172.21.15.1:0/806918370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640c3d31cc0 msgr2=0x5640c3d935e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:48:59.060 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.047+0000 7fe6caffd700 1 --2- 172.21.15.1:0/806918370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640c3d31cc0 0x5640c3d935e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:59.060 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.047+0000 7fe6caffd700 1 -- 172.21.15.1:0/806918370 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5640c3c40190 con 0x5640c3d2d880 2023-11-16T16:48:59.060 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.047+0000 7fe6caffd700 1 --2- 172.21.15.1:0/806918370 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640c3d2d880 0x5640c3d3c470 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc1979b0 tx=0x7fe6c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a06490f3d367684 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:48:59.060 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.047+0000 7fe6aaffd700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:48:59.060 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.047+0000 7fe6aaffd700 1 -- 172.21.15.1:0/806918370 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c001e840 con 0x5640c3d2d880 2023-11-16T16:48:59.061 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.047+0000 7fe6de13db80 1 -- 172.21.15.1:0/806918370 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5640c3d1bb20 con 0x5640c3d2d880 2023-11-16T16:48:59.061 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.047+0000 7fe6aaffd700 1 -- 172.21.15.1:0/806918370 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fe6c001e9a0 con 0x5640c3d2d880 2023-11-16T16:48:59.061 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.047+0000 7fe6aaffd700 1 -- 172.21.15.1:0/806918370 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6c001eb60 con 0x5640c3d2d880 2023-11-16T16:48:59.061 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.047+0000 7fe6de13db80 10 client.16992 fetch_fsmap learned FSMap version 32 2023-11-16T16:48:59.061 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.047+0000 7fe6de13db80 1 -- 172.21.15.1:0/806918370 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5640c3aa7de0 con 0x5640c3d2d880 2023-11-16T16:48:59.061 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.047+0000 7fe6aaffd700 1 -- 172.21.15.1:0/806918370 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5640c3d1bb20 con 0x5640c3d2d880 2023-11-16T16:48:59.061 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.047+0000 7fe6aaffd700 1 -- 172.21.15.1:0/806918370 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fe6c001d430 con 0x5640c3d2d880 2023-11-16T16:48:59.062 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.048+0000 7fe6de13db80 10 client.16992 fetch_fsmap finished waiting for FSMap version 32 2023-11-16T16:48:59.062 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.048+0000 7fe6de13db80 10 client.16992 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:48:59.062 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.048+0000 7fe6de13db80 20 client.16992 populate_metadata read hostname 'smithi001' 2023-11-16T16:48:59.062 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.048+0000 7fe6de13db80 1 --2- 172.21.15.1:0/806918370 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5640c3d9b610 0x5640c3d9da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:48:59.062 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.048+0000 7fe6de13db80 4 client.16992 mds_command: new command op to 24521 tid=0[{"prefix": "scrub status"}] 2023-11-16T16:48:59.062 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.048+0000 7fe6de13db80 1 -- 172.21.15.1:0/806918370 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5640c3ac8f70 con 0x5640c3d9b610 2023-11-16T16:48:59.062 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.048+0000 7fe6cb7fe700 1 --2- 172.21.15.1:0/806918370 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5640c3d9b610 0x5640c3d9da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:48:59.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.048+0000 7fe6aaffd700 10 client.16992 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:48:59.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.048+0000 7fe6cb7fe700 1 --2- 172.21.15.1:0/806918370 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5640c3d9b610 0x5640c3d9da20 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dd1b7a2ef99eb8b server_cookie=540c6fd928d5bc89 in_seq=0 out_seq=0 2023-11-16T16:48:59.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.048+0000 7fe6aaffd700 1 -- 172.21.15.1:0/806918370 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5640c3ac8f70 con 0x5640c3d9b610 2023-11-16T16:48:59.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.048+0000 7fe6aaffd700 10 client.16992 handle_command_reply: tid=0 2023-11-16T16:48:59.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.048+0000 7fe6de13db80 1 client.16992 shutdown 2023-11-16T16:48:59.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.049+0000 7fe6de13db80 1 -- 172.21.15.1:0/806918370 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640c3d2d880 msgr2=0x5640c3d3c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:59.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.049+0000 7fe6de13db80 1 --2- 172.21.15.1:0/806918370 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640c3d2d880 0x5640c3d3c470 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc1979b0 tx=0x7fe6c0004750 comp rx=0 tx=0).stop 2023-11-16T16:48:59.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.049+0000 7fe6de13db80 1 -- 172.21.15.1:0/806918370 shutdown_connections 2023-11-16T16:48:59.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.049+0000 7fe6de13db80 1 --2- 172.21.15.1:0/806918370 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x5640c3d9b610 0x5640c3d9da20 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:59.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.049+0000 7fe6de13db80 1 --2- 172.21.15.1:0/806918370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640c3d31cc0 0x5640c3d935e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:59.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.049+0000 7fe6de13db80 1 --2- 172.21.15.1:0/806918370 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x5640c3d2cf70 0x5640c3d3bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:59.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.049+0000 7fe6de13db80 1 --2- 172.21.15.1:0/806918370 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5640c3d2d880 0x5640c3d3c470 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:59.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.049+0000 7fe6aaffd700 0 client.16992 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:48:59.065 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.049+0000 7fe6de13db80 1 -- 172.21.15.1:0/806918370 >> 172.21.15.1:0/806918370 conn(0x5640c3d1e360 msgr2=0x5640c3d2fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:59.065 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.049+0000 7fe6de13db80 1 -- 172.21.15.1:0/806918370 shutdown_connections 2023-11-16T16:48:59.065 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.049+0000 7fe6de13db80 1 -- 172.21.15.1:0/806918370 wait complete. 2023-11-16T16:48:59.065 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.049+0000 7fe6de13db80 20 client.16992 trim_cache size 0 max 16384 2023-11-16T16:48:59.065 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:48:59.066 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-16T16:48:59.066 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2023-11-16T16:48:59.066 INFO:teuthology.orchestra.run.smithi001.stdout: "e8cd662f-4bd3-401d-9c70-1bc1f2776848": { 2023-11-16T16:48:59.066 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2023-11-16T16:48:59.066 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "e8cd662f-4bd3-401d-9c70-1bc1f2776848", 2023-11-16T16:48:59.066 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2023-11-16T16:48:59.067 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:48:59.067 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:48:59.067 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:48:59.068 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.049+0000 7fe6d1fb1700 1 -- 172.21.15.1:0/821476018 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fe6b40a12c0 msgr2=0x7fe6b40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:59.068 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.049+0000 7fe6d1fb1700 1 --2- 172.21.15.1:0/821476018 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fe6b40a12c0 0x7fe6b40a3770 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0005ea0 tx=0x7fe6c0005e30 comp rx=0 tx=0).stop 2023-11-16T16:48:59.069 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.049+0000 7fe6d1fb1700 1 -- 172.21.15.1:0/821476018 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6cc0fea50 msgr2=0x7fe6cc1963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:48:59.069 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.049+0000 7fe6d1fb1700 1 --2- 172.21.15.1:0/821476018 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6cc0fea50 0x7fe6cc1963a0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc00d980 tx=0x7fe6bc00dd40 comp rx=0 tx=0).stop 2023-11-16T16:48:59.069 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.049+0000 7fe6d1fb1700 1 -- 172.21.15.1:0/821476018 shutdown_connections 2023-11-16T16:48:59.069 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.049+0000 7fe6d1fb1700 1 --2- 172.21.15.1:0/821476018 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fe6b40a12c0 0x7fe6b40a3770 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:59.069 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.049+0000 7fe6d1fb1700 1 --2- 172.21.15.1:0/821476018 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe6cc1055f0 0x7fe6cc19ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:59.069 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.049+0000 7fe6d1fb1700 1 --2- 172.21.15.1:0/821476018 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6cc0fea50 0x7fe6cc1963a0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:59.070 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.049+0000 7fe6d1fb1700 1 --2- 172.21.15.1:0/821476018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6cc0ff3f0 0x7fe6cc1968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:48:59.070 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.049+0000 7fe6d1fb1700 1 -- 172.21.15.1:0/821476018 >> 172.21.15.1:0/821476018 conn(0x7fe6cc075880 msgr2=0x7fe6cc103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:48:59.070 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.049+0000 7fe6d1fb1700 1 -- 172.21.15.1:0/821476018 shutdown_connections 2023-11-16T16:48:59.070 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:48:59.049+0000 7fe6d1fb1700 1 -- 172.21.15.1:0/821476018 wait complete. 2023-11-16T16:48:59.070 INFO:tasks.cephfs.filesystem:scrub status for tag:e8cd662f-4bd3-401d-9c70-1bc1f2776848 - {'path': '/', 'tag': 'e8cd662f-4bd3-401d-9c70-1bc1f2776848', 'options': 'recursive,force'} 2023-11-16T16:48:59.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:59 smithi001 ceph-mon[98800]: scrub queued for path: / 2023-11-16T16:48:59.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:59 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:48:59.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:59 smithi001 ceph-mon[98800]: scrub summary: active paths [/] 2023-11-16T16:48:59.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:59 smithi001 ceph-mon[98800]: scrub summary: idle+waiting paths [/] 2023-11-16T16:48:59.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:48:59 smithi001 ceph-mon[98800]: scrub summary: idle 2023-11-16T16:48:59.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:59 smithi026 ceph-mon[100112]: scrub queued for path: / 2023-11-16T16:48:59.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:59 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:48:59.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:59 smithi026 ceph-mon[100112]: scrub summary: active paths [/] 2023-11-16T16:48:59.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:59 smithi026 ceph-mon[100112]: scrub summary: idle+waiting paths [/] 2023-11-16T16:48:59.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:48:59 smithi026 ceph-mon[100112]: scrub summary: idle 2023-11-16T16:48:59.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:59 smithi053 ceph-mon[100638]: scrub queued for path: / 2023-11-16T16:48:59.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:59 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:48:59.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:59 smithi053 ceph-mon[100638]: scrub summary: active paths [/] 2023-11-16T16:48:59.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:59 smithi053 ceph-mon[100638]: scrub summary: idle+waiting paths [/] 2023-11-16T16:48:59.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:48:59 smithi053 ceph-mon[100638]: scrub summary: idle 2023-11-16T16:49:00.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:00 smithi001 ceph-mon[98800]: pgmap v304: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:49:00.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:00 smithi026 ceph-mon[100112]: pgmap v304: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:49:00.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:00 smithi053 ceph-mon[100638]: pgmap v304: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:49:02.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:02 smithi001 ceph-mon[98800]: pgmap v305: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-16T16:49:02.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:02 smithi026 ceph-mon[100112]: pgmap v305: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-16T16:49:02.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:02 smithi053 ceph-mon[100638]: pgmap v305: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-11-16T16:49:04.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:03 smithi026 ceph-mon[100112]: pgmap v306: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:49:04.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:03 smithi053 ceph-mon[100638]: pgmap v306: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:49:04.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:03 smithi001 ceph-mon[98800]: pgmap v306: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:49:06.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:05 smithi026 ceph-mon[100112]: pgmap v307: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-11-16T16:49:06.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:05 smithi053 ceph-mon[100638]: pgmap v307: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-11-16T16:49:06.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:05 smithi001 ceph-mon[98800]: pgmap v307: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-11-16T16:49:08.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:07 smithi026 ceph-mon[100112]: pgmap v308: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-16T16:49:08.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:07 smithi053 ceph-mon[100638]: pgmap v308: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-16T16:49:08.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:07 smithi001 ceph-mon[98800]: pgmap v308: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-11-16T16:49:10.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:10 smithi053 ceph-mon[100638]: osdmap e94: 12 total, 12 up, 12 in 2023-11-16T16:49:10.581 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:10 smithi053 ceph-mon[100638]: pgmap v310: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-11-16T16:49:10.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:10 smithi001 ceph-mon[98800]: osdmap e94: 12 total, 12 up, 12 in 2023-11-16T16:49:10.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:10 smithi001 ceph-mon[98800]: pgmap v310: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-11-16T16:49:10.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:10 smithi026 ceph-mon[100112]: osdmap e94: 12 total, 12 up, 12 in 2023-11-16T16:49:10.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:10 smithi026 ceph-mon[100112]: pgmap v310: 97 pgs: 97 active+clean; 817 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-11-16T16:49:11.436 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-11-16T16:49:11.578 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.575+0000 7fe187080700 1 -- 172.21.15.1:0/217326043 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe180104460 msgr2=0x7fe18010c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:11.579 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.575+0000 7fe187080700 1 --2- 172.21.15.1:0/217326043 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe180104460 0x7fe18010c030 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fe17800b0d0 tx=0x7fe17800b3e0 comp rx=0 tx=0).stop 2023-11-16T16:49:11.579 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.575+0000 7fe187080700 1 -- 172.21.15.1:0/217326043 shutdown_connections 2023-11-16T16:49:11.579 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.575+0000 7fe187080700 1 --2- 172.21.15.1:0/217326043 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe180104460 0x7fe18010c030 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:11.579 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.575+0000 7fe187080700 1 --2- 172.21.15.1:0/217326043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1800ff420 0x7fe1800ff7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:11.579 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.575+0000 7fe187080700 1 --2- 172.21.15.1:0/217326043 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe1800ffdc0 0x7fe180103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:11.579 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.575+0000 7fe187080700 1 -- 172.21.15.1:0/217326043 >> 172.21.15.1:0/217326043 conn(0x7fe180075fd0 msgr2=0x7fe1800763d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:11.580 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.575+0000 7fe187080700 1 -- 172.21.15.1:0/217326043 shutdown_connections 2023-11-16T16:49:11.580 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.575+0000 7fe187080700 1 -- 172.21.15.1:0/217326043 wait complete. 2023-11-16T16:49:11.580 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.575+0000 7fe187080700 1 Processor -- start 2023-11-16T16:49:11.580 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.576+0000 7fe187080700 1 -- start start 2023-11-16T16:49:11.580 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.576+0000 7fe187080700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1800ff420 0x7fe180197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:11.580 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.576+0000 7fe187080700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1800ffdc0 0x7fe180197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:11.580 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.576+0000 7fe187080700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe180104460 0x7fe1801a2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:11.580 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.576+0000 7fe187080700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1801a28d0 con 0x7fe1800ff420 2023-11-16T16:49:11.581 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.576+0000 7fe187080700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1801a2a10 con 0x7fe1800ffdc0 2023-11-16T16:49:11.581 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.576+0000 7fe187080700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1801a2b50 con 0x7fe180104460 2023-11-16T16:49:11.581 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.576+0000 7fe184e1c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1800ff420 0x7fe180197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:11.581 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.576+0000 7fe184e1c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1800ff420 0x7fe180197310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47196/0 (socket says 172.21.15.1:47196) 2023-11-16T16:49:11.581 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.576+0000 7fe184e1c700 1 -- 172.21.15.1:0/4170281708 learned_addr learned my addr 172.21.15.1:0/4170281708 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:49:11.581 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.576+0000 7fe17ffff700 1 --2- 172.21.15.1:0/4170281708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1800ffdc0 0x7fe180197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:11.581 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.576+0000 7fe18561d700 1 --2- 172.21.15.1:0/4170281708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe180104460 0x7fe1801a2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:11.581 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.576+0000 7fe184e1c700 1 -- 172.21.15.1:0/4170281708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe180104460 msgr2=0x7fe1801a2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:11.581 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.576+0000 7fe184e1c700 1 --2- 172.21.15.1:0/4170281708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe180104460 0x7fe1801a2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:11.582 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.576+0000 7fe184e1c700 1 -- 172.21.15.1:0/4170281708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1800ffdc0 msgr2=0x7fe180197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:11.582 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.576+0000 7fe184e1c700 1 --2- 172.21.15.1:0/4170281708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1800ffdc0 0x7fe180197850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:11.582 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.576+0000 7fe184e1c700 1 -- 172.21.15.1:0/4170281708 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe178009d00 con 0x7fe1800ff420 2023-11-16T16:49:11.582 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.576+0000 7fe184e1c700 1 --2- 172.21.15.1:0/4170281708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1800ff420 0x7fe180197310 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fe17400c9b0 tx=0x7fe17400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:11.582 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.577+0000 7fe17dffb700 1 -- 172.21.15.1:0/4170281708 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe174007b00 con 0x7fe1800ff420 2023-11-16T16:49:11.582 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.577+0000 7fe187080700 1 -- 172.21.15.1:0/4170281708 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe18010f230 con 0x7fe1800ff420 2023-11-16T16:49:11.582 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.577+0000 7fe17dffb700 1 -- 172.21.15.1:0/4170281708 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe174007c60 con 0x7fe1800ff420 2023-11-16T16:49:11.582 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.577+0000 7fe17dffb700 1 -- 172.21.15.1:0/4170281708 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe174018660 con 0x7fe1800ff420 2023-11-16T16:49:11.583 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.577+0000 7fe187080700 1 -- 172.21.15.1:0/4170281708 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe18010f0a0 con 0x7fe1800ff420 2023-11-16T16:49:11.583 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.578+0000 7fe187080700 1 -- 172.21.15.1:0/4170281708 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe18010f3c0 con 0x7fe1800ff420 2023-11-16T16:49:11.584 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.579+0000 7fe17dffb700 1 -- 172.21.15.1:0/4170281708 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fe174018880 con 0x7fe1800ff420 2023-11-16T16:49:11.584 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.579+0000 7fe17dffb700 1 --2- 172.21.15.1:0/4170281708 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fe1700a12c0 0x7fe1700a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:11.584 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.580+0000 7fe17dffb700 1 -- 172.21.15.1:0/4170281708 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8586+0+0 (secure 0 0 0) 0x7fe1740cce30 con 0x7fe1800ff420 2023-11-16T16:49:11.584 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.580+0000 7fe17ffff700 1 --2- 172.21.15.1:0/4170281708 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fe1700a12c0 0x7fe1700a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:11.584 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.580+0000 7fe17ffff700 1 --2- 172.21.15.1:0/4170281708 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fe1700a12c0 0x7fe1700a3770 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe16c005fd0 tx=0x7fe16c005ee0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:11.586 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.584+0000 7fe17dffb700 1 -- 172.21.15.1:0/4170281708 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe1740961a0 con 0x7fe1800ff420 2023-11-16T16:49:11.813 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.812+0000 7fe187080700 1 -- 172.21.15.1:0/4170281708 --> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7fe1800fd3c0 con 0x7fe1700a12c0 2023-11-16T16:49:11.822 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.820+0000 7fe17dffb700 1 -- 172.21.15.1:0/4170281708 <== mgr.15432 v2:172.21.15.1:6800/4199419371 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+335 (secure 0 0 0) 0x7fe1800fd3c0 con 0x7fe1700a12c0 2023-11-16T16:49:11.822 INFO:teuthology.orchestra.run.smithi001.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1M", "retention": {"h": 3, "M": 6}, "start": "2023-11-16T00:00:00", "created": "2023-11-16T16:39:10", "first": "2023-11-16T16:40:00", "last": "2023-11-16T16:49:00", "last_pruned": "2023-11-16T16:49:00", "created_count": 10, "pruned_count": 4, "active": true} 2023-11-16T16:49:11.824 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.822+0000 7fe187080700 1 -- 172.21.15.1:0/4170281708 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fe1700a12c0 msgr2=0x7fe1700a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:11.824 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.822+0000 7fe187080700 1 --2- 172.21.15.1:0/4170281708 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fe1700a12c0 0x7fe1700a3770 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe16c005fd0 tx=0x7fe16c005ee0 comp rx=0 tx=0).stop 2023-11-16T16:49:11.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.822+0000 7fe187080700 1 -- 172.21.15.1:0/4170281708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1800ff420 msgr2=0x7fe180197310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:11.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.822+0000 7fe187080700 1 --2- 172.21.15.1:0/4170281708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1800ff420 0x7fe180197310 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fe17400c9b0 tx=0x7fe17400cd70 comp rx=0 tx=0).stop 2023-11-16T16:49:11.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.822+0000 7fe187080700 1 -- 172.21.15.1:0/4170281708 shutdown_connections 2023-11-16T16:49:11.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.822+0000 7fe187080700 1 --2- 172.21.15.1:0/4170281708 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fe1700a12c0 0x7fe1700a3770 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:11.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.822+0000 7fe187080700 1 --2- 172.21.15.1:0/4170281708 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe180104460 0x7fe1801a2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:11.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.822+0000 7fe187080700 1 --2- 172.21.15.1:0/4170281708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1800ff420 0x7fe180197310 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:11.825 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.822+0000 7fe187080700 1 --2- 172.21.15.1:0/4170281708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1800ffdc0 0x7fe180197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:11.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.822+0000 7fe187080700 1 -- 172.21.15.1:0/4170281708 >> 172.21.15.1:0/4170281708 conn(0x7fe180075fd0 msgr2=0x7fe1800fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:11.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.823+0000 7fe187080700 1 -- 172.21.15.1:0/4170281708 shutdown_connections 2023-11-16T16:49:11.826 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.823+0000 7fe187080700 1 -- 172.21.15.1:0/4170281708 wait complete. 2023-11-16T16:49:11.836 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-11-16T16:49:11.964 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.962+0000 7f5b5c9ac700 1 -- 172.21.15.1:0/1706142511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b580ff3c0 msgr2=0x7f5b580ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:11.964 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.962+0000 7f5b5c9ac700 1 --2- 172.21.15.1:0/1706142511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b580ff3c0 0x7f5b580ff790 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c0099c0 tx=0x7f5b4c009cd0 comp rx=0 tx=0).stop 2023-11-16T16:49:11.964 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.962+0000 7f5b5c9ac700 1 -- 172.21.15.1:0/1706142511 shutdown_connections 2023-11-16T16:49:11.964 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.962+0000 7f5b5c9ac700 1 --2- 172.21.15.1:0/1706142511 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b58104400 0x7f5b5810bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:11.964 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.962+0000 7f5b5c9ac700 1 --2- 172.21.15.1:0/1706142511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b580ff3c0 0x7f5b580ff790 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:11.965 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.962+0000 7f5b5c9ac700 1 --2- 172.21.15.1:0/1706142511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b580ffd60 0x7f5b58103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:11.965 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.962+0000 7f5b5c9ac700 1 -- 172.21.15.1:0/1706142511 >> 172.21.15.1:0/1706142511 conn(0x7f5b580763a0 msgr2=0x7f5b580767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:11.965 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.962+0000 7f5b5c9ac700 1 -- 172.21.15.1:0/1706142511 shutdown_connections 2023-11-16T16:49:11.965 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.962+0000 7f5b5c9ac700 1 -- 172.21.15.1:0/1706142511 wait complete. 2023-11-16T16:49:11.967 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.963+0000 7f5b5c9ac700 1 Processor -- start 2023-11-16T16:49:11.967 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.963+0000 7f5b5c9ac700 1 -- start start 2023-11-16T16:49:11.967 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.963+0000 7f5b5c9ac700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b580ff3c0 0x7f5b581952f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:11.967 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.963+0000 7f5b5c9ac700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b580ffd60 0x7f5b58195830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:11.968 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.963+0000 7f5b5c9ac700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b58104400 0x7f5b58199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:11.968 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.963+0000 7f5b5c9ac700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5819a2d0 con 0x7f5b580ff3c0 2023-11-16T16:49:11.968 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.963+0000 7f5b5c9ac700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5819a440 con 0x7f5b580ffd60 2023-11-16T16:49:11.968 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.963+0000 7f5b5c9ac700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5819a600 con 0x7f5b58104400 2023-11-16T16:49:11.968 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.963+0000 7f5b5659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b580ff3c0 0x7f5b581952f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:11.968 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.963+0000 7f5b5659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b580ff3c0 0x7f5b581952f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47210/0 (socket says 172.21.15.1:47210) 2023-11-16T16:49:11.968 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.963+0000 7f5b5659c700 1 -- 172.21.15.1:0/432000888 learned_addr learned my addr 172.21.15.1:0/432000888 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:49:11.968 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.963+0000 7f5b56d9d700 1 --2- 172.21.15.1:0/432000888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b58104400 0x7f5b58199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:11.969 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.963+0000 7f5b55d9b700 1 --2- 172.21.15.1:0/432000888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b580ffd60 0x7f5b58195830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:11.969 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.963+0000 7f5b5659c700 1 -- 172.21.15.1:0/432000888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b58104400 msgr2=0x7f5b58199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:11.969 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.963+0000 7f5b5659c700 1 --2- 172.21.15.1:0/432000888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b58104400 0x7f5b58199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:11.969 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.963+0000 7f5b5659c700 1 -- 172.21.15.1:0/432000888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b580ffd60 msgr2=0x7f5b58195830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:11.969 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.963+0000 7f5b5659c700 1 --2- 172.21.15.1:0/432000888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b580ffd60 0x7f5b58195830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:11.969 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.963+0000 7f5b5659c700 1 -- 172.21.15.1:0/432000888 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b4c0095f0 con 0x7f5b580ff3c0 2023-11-16T16:49:11.969 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.964+0000 7f5b5659c700 1 --2- 172.21.15.1:0/432000888 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b580ff3c0 0x7f5b581952f0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c009990 tx=0x7f5b4c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:11.969 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.964+0000 7f5b477fe700 1 -- 172.21.15.1:0/432000888 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c01d070 con 0x7f5b580ff3c0 2023-11-16T16:49:11.970 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.964+0000 7f5b477fe700 1 -- 172.21.15.1:0/432000888 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b4c003f90 con 0x7f5b580ff3c0 2023-11-16T16:49:11.970 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.964+0000 7f5b5c9ac700 1 -- 172.21.15.1:0/432000888 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b5810f210 con 0x7f5b580ff3c0 2023-11-16T16:49:11.970 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.964+0000 7f5b477fe700 1 -- 172.21.15.1:0/432000888 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c022c50 con 0x7f5b580ff3c0 2023-11-16T16:49:11.970 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.964+0000 7f5b5c9ac700 1 -- 172.21.15.1:0/432000888 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b5810f080 con 0x7f5b580ff3c0 2023-11-16T16:49:11.970 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.965+0000 7f5b5c9ac700 1 -- 172.21.15.1:0/432000888 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b5810f3a0 con 0x7f5b580ff3c0 2023-11-16T16:49:11.971 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.966+0000 7f5b477fe700 1 -- 172.21.15.1:0/432000888 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f5b4c022470 con 0x7f5b580ff3c0 2023-11-16T16:49:11.971 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.967+0000 7f5b477fe700 1 --2- 172.21.15.1:0/432000888 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5b400a12c0 0x7f5b400a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:11.971 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.967+0000 7f5b55d9b700 1 --2- 172.21.15.1:0/432000888 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5b400a12c0 0x7f5b400a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:11.971 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.967+0000 7f5b477fe700 1 -- 172.21.15.1:0/432000888 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8586+0+0 (secure 0 0 0) 0x7f5b4c0ce260 con 0x7f5b580ff3c0 2023-11-16T16:49:11.971 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.967+0000 7f5b55d9b700 1 --2- 172.21.15.1:0/432000888 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5b400a12c0 0x7f5b400a3770 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5b58196900 tx=0x7f5b4800a300 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:11.973 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:11.971+0000 7f5b477fe700 1 -- 172.21.15.1:0/432000888 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5b4c0976c0 con 0x7f5b580ff3c0 2023-11-16T16:49:12.198 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:12.197+0000 7f5b5c9ac700 1 -- 172.21.15.1:0/432000888 --> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f5b580fd0f0 con 0x7f5b400a12c0 2023-11-16T16:49:12.206 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:12.204+0000 7f5b477fe700 1 -- 172.21.15.1:0/432000888 <== mgr.15432 v2:172.21.15.1:6800/4199419371 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f5b580fd0f0 con 0x7f5b400a12c0 2023-11-16T16:49:12.206 INFO:teuthology.orchestra.run.smithi001.stdout:/ 1M 6M3h 2023-11-16T16:49:12.210 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:12.206+0000 7f5b5c9ac700 1 -- 172.21.15.1:0/432000888 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5b400a12c0 msgr2=0x7f5b400a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:12.210 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:12.206+0000 7f5b5c9ac700 1 --2- 172.21.15.1:0/432000888 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5b400a12c0 0x7f5b400a3770 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5b58196900 tx=0x7f5b4800a300 comp rx=0 tx=0).stop 2023-11-16T16:49:12.210 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:12.206+0000 7f5b5c9ac700 1 -- 172.21.15.1:0/432000888 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b580ff3c0 msgr2=0x7f5b581952f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:12.210 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:12.206+0000 7f5b5c9ac700 1 --2- 172.21.15.1:0/432000888 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b580ff3c0 0x7f5b581952f0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c009990 tx=0x7f5b4c005740 comp rx=0 tx=0).stop 2023-11-16T16:49:12.210 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:12.207+0000 7f5b5c9ac700 1 -- 172.21.15.1:0/432000888 shutdown_connections 2023-11-16T16:49:12.210 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:12.207+0000 7f5b5c9ac700 1 --2- 172.21.15.1:0/432000888 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f5b400a12c0 0x7f5b400a3770 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:12.210 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:12.207+0000 7f5b5c9ac700 1 --2- 172.21.15.1:0/432000888 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f5b58104400 0x7f5b58199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:12.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:12.207+0000 7f5b5c9ac700 1 --2- 172.21.15.1:0/432000888 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b580ff3c0 0x7f5b581952f0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:12.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:12.207+0000 7f5b5c9ac700 1 --2- 172.21.15.1:0/432000888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b580ffd60 0x7f5b58195830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:12.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:12.207+0000 7f5b5c9ac700 1 -- 172.21.15.1:0/432000888 >> 172.21.15.1:0/432000888 conn(0x7f5b580763a0 msgr2=0x7f5b580fc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:12.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:12.207+0000 7f5b5c9ac700 1 -- 172.21.15.1:0/432000888 shutdown_connections 2023-11-16T16:49:12.211 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:12.207+0000 7f5b5c9ac700 1 -- 172.21.15.1:0/432000888 wait complete. 2023-11-16T16:49:12.220 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-11-16T16:49:12.230 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-11-16T16:49:12.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:12 smithi001 ceph-mon[98800]: pgmap v311: 97 pgs: 97 active+clean; 817 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-11-16T16:49:12.757 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:12 smithi026 ceph-mon[100112]: pgmap v311: 97 pgs: 97 active+clean; 817 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-11-16T16:49:12.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:12 smithi053 ceph-mon[100638]: pgmap v311: 97 pgs: 97 active+clean; 817 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-11-16T16:49:13.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:13 smithi001 ceph-mon[98800]: from='client.16998 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:49:13.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:13 smithi001 ceph-mon[98800]: from='client.17004 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:49:13.757 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:13 smithi026 ceph-mon[100112]: from='client.16998 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:49:13.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:13 smithi026 ceph-mon[100112]: from='client.17004 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:49:13.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:13 smithi053 ceph-mon[100638]: from='client.16998 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:49:13.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:13 smithi053 ceph-mon[100638]: from='client.17004 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-11-16T16:49:14.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:14 smithi001 ceph-mon[98800]: pgmap v312: 97 pgs: 97 active+clean; 817 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-11-16T16:49:14.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:14 smithi026 ceph-mon[100112]: pgmap v312: 97 pgs: 97 active+clean; 817 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-11-16T16:49:14.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:14 smithi053 ceph-mon[100638]: pgmap v312: 97 pgs: 97 active+clean; 817 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-11-16T16:49:16.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:16 smithi001 ceph-mon[98800]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:49:16.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:16 smithi001 ceph-mon[98800]: pgmap v313: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 9 op/s 2023-11-16T16:49:16.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:16 smithi001 ceph-mon[98800]: osdmap e95: 12 total, 12 up, 12 in 2023-11-16T16:49:16.757 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:16 smithi026 ceph-mon[100112]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:49:16.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:16 smithi026 ceph-mon[100112]: pgmap v313: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 9 op/s 2023-11-16T16:49:16.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:16 smithi026 ceph-mon[100112]: osdmap e95: 12 total, 12 up, 12 in 2023-11-16T16:49:16.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:16 smithi053 ceph-mon[100638]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-16T16:49:16.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:16 smithi053 ceph-mon[100638]: pgmap v313: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 9 op/s 2023-11-16T16:49:16.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:16 smithi053 ceph-mon[100638]: osdmap e95: 12 total, 12 up, 12 in 2023-11-16T16:49:18.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:17 smithi026 ceph-mon[100112]: pgmap v315: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 244 B/s wr, 10 op/s 2023-11-16T16:49:18.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:17 smithi053 ceph-mon[100638]: pgmap v315: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 244 B/s wr, 10 op/s 2023-11-16T16:49:18.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:17 smithi001 ceph-mon[98800]: pgmap v315: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 244 B/s wr, 10 op/s 2023-11-16T16:49:20.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:19 smithi026 ceph-mon[100112]: pgmap v316: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 9 op/s 2023-11-16T16:49:20.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:19 smithi053 ceph-mon[100638]: pgmap v316: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 9 op/s 2023-11-16T16:49:20.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:19 smithi001 ceph-mon[98800]: pgmap v316: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 9 op/s 2023-11-16T16:49:22.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:22 smithi001 ceph-mon[98800]: pgmap v317: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 8 op/s 2023-11-16T16:49:22.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:22 smithi026 ceph-mon[100112]: pgmap v317: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 8 op/s 2023-11-16T16:49:22.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:22 smithi053 ceph-mon[100638]: pgmap v317: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 8 op/s 2023-11-16T16:49:24.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:23 smithi026 ceph-mon[100112]: pgmap v318: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 8 op/s 2023-11-16T16:49:32.978 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-16T16:49:32.980 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:25 smithi026 ceph-mon[100112]: pgmap v319: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-11-16T16:49:32.981 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:27 smithi026 ceph-mon[100112]: pgmap v320: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:49:32.981 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:29 smithi026 ceph-mon[100112]: pgmap v321: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:49:32.981 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:32 smithi026 ceph-mon[100112]: pgmap v322: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:49:32.981 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:23 smithi001 ceph-mon[98800]: pgmap v318: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 8 op/s 2023-11-16T16:49:32.981 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:25 smithi001 ceph-mon[98800]: pgmap v319: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-11-16T16:49:32.981 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:27 smithi001 ceph-mon[98800]: pgmap v320: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:49:32.981 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:29 smithi001 ceph-mon[98800]: pgmap v321: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:49:32.981 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:32 smithi001 ceph-mon[98800]: pgmap v322: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:49:32.982 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:23 smithi053 ceph-mon[100638]: pgmap v318: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 8 op/s 2023-11-16T16:49:32.982 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:25 smithi053 ceph-mon[100638]: pgmap v319: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-11-16T16:49:32.982 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:27 smithi053 ceph-mon[100638]: pgmap v320: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-11-16T16:49:32.982 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:29 smithi053 ceph-mon[100638]: pgmap v321: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:49:32.982 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:32 smithi053 ceph-mon[100638]: pgmap v322: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:49:33.123 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.121+0000 7ff7d7c6b700 1 -- 172.21.15.1:0/975446533 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7d00ff420 msgr2=0x7ff7d00ff7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.124 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.121+0000 7ff7d7c6b700 1 --2- 172.21.15.1:0/975446533 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7d00ff420 0x7ff7d00ff7f0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc0099c0 tx=0x7ff7cc009cd0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.124 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.121+0000 7ff7d7c6b700 1 -- 172.21.15.1:0/975446533 shutdown_connections 2023-11-16T16:49:33.124 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.121+0000 7ff7d7c6b700 1 --2- 172.21.15.1:0/975446533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7d0104460 0x7ff7d010c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.124 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.121+0000 7ff7d7c6b700 1 --2- 172.21.15.1:0/975446533 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7d00ff420 0x7ff7d00ff7f0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.124 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.121+0000 7ff7d7c6b700 1 --2- 172.21.15.1:0/975446533 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7d00ffdc0 0x7ff7d0103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.124 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.121+0000 7ff7d7c6b700 1 -- 172.21.15.1:0/975446533 >> 172.21.15.1:0/975446533 conn(0x7ff7d0076230 msgr2=0x7ff7d0076630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:33.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.121+0000 7ff7d7c6b700 1 -- 172.21.15.1:0/975446533 shutdown_connections 2023-11-16T16:49:33.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.121+0000 7ff7d7c6b700 1 -- 172.21.15.1:0/975446533 wait complete. 2023-11-16T16:49:33.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.122+0000 7ff7d7c6b700 1 Processor -- start 2023-11-16T16:49:33.125 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.122+0000 7ff7d7c6b700 1 -- start start 2023-11-16T16:49:33.127 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.122+0000 7ff7d7c6b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7d00ff420 0x7ff7d0197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.127 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.122+0000 7ff7d7c6b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7d00ffdc0 0x7ff7d0197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.128 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.122+0000 7ff7d7c6b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7d0104460 0x7ff7d01a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.128 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.122+0000 7ff7d7c6b700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d01980a0 con 0x7ff7d00ff420 2023-11-16T16:49:33.128 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.122+0000 7ff7d7c6b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d01a27e0 con 0x7ff7d00ffdc0 2023-11-16T16:49:33.128 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.122+0000 7ff7d7c6b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d01a29a0 con 0x7ff7d0104460 2023-11-16T16:49:33.128 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.122+0000 7ff7d5a07700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7d00ff420 0x7ff7d0197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.128 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.122+0000 7ff7d5a07700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7d00ff420 0x7ff7d0197220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47222/0 (socket says 172.21.15.1:47222) 2023-11-16T16:49:33.128 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.122+0000 7ff7d5a07700 1 -- 172.21.15.1:0/2878234695 learned_addr learned my addr 172.21.15.1:0/2878234695 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:49:33.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.123+0000 7ff7d5206700 1 --2- 172.21.15.1:0/2878234695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7d00ffdc0 0x7ff7d0197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.123+0000 7ff7d6208700 1 --2- 172.21.15.1:0/2878234695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7d0104460 0x7ff7d01a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.123+0000 7ff7d5a07700 1 -- 172.21.15.1:0/2878234695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7d0104460 msgr2=0x7ff7d01a22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.123+0000 7ff7d5a07700 1 --2- 172.21.15.1:0/2878234695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7d0104460 0x7ff7d01a22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.123+0000 7ff7d5a07700 1 -- 172.21.15.1:0/2878234695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7d00ffdc0 msgr2=0x7ff7d0197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.123+0000 7ff7d5a07700 1 --2- 172.21.15.1:0/2878234695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7d00ffdc0 0x7ff7d0197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.123+0000 7ff7d5a07700 1 -- 172.21.15.1:0/2878234695 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7cc0095f0 con 0x7ff7d00ff420 2023-11-16T16:49:33.129 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.123+0000 7ff7d5a07700 1 --2- 172.21.15.1:0/2878234695 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7d00ff420 0x7ff7d0197220 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc004710 tx=0x7ff7cc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:33.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.123+0000 7ff7c2ffd700 1 -- 172.21.15.1:0/2878234695 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc01d070 con 0x7ff7d00ff420 2023-11-16T16:49:33.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.123+0000 7ff7d7c6b700 1 -- 172.21.15.1:0/2878234695 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7d010f270 con 0x7ff7d00ff420 2023-11-16T16:49:33.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.123+0000 7ff7c2ffd700 1 -- 172.21.15.1:0/2878234695 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7cc004090 con 0x7ff7d00ff420 2023-11-16T16:49:33.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.123+0000 7ff7c2ffd700 1 -- 172.21.15.1:0/2878234695 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc00f460 con 0x7ff7d00ff420 2023-11-16T16:49:33.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.123+0000 7ff7d7c6b700 1 -- 172.21.15.1:0/2878234695 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d010f0e0 con 0x7ff7d00ff420 2023-11-16T16:49:33.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.125+0000 7ff7c2ffd700 1 -- 172.21.15.1:0/2878234695 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7ff7cc022600 con 0x7ff7d00ff420 2023-11-16T16:49:33.130 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.126+0000 7ff7c2ffd700 1 --2- 172.21.15.1:0/2878234695 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7ff7bc0a12c0 0x7ff7bc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.137 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.126+0000 7ff7c2ffd700 1 -- 172.21.15.1:0/2878234695 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7ff7cc0ce180 con 0x7ff7d00ff420 2023-11-16T16:49:33.137 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.126+0000 7ff7d5206700 1 --2- 172.21.15.1:0/2878234695 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7ff7bc0a12c0 0x7ff7bc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.137 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.127+0000 7ff7d5206700 1 --2- 172.21.15.1:0/2878234695 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7ff7bc0a12c0 0x7ff7bc0a3770 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0069900 tx=0x7ff7c400a300 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:33.137 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.127+0000 7ff7e3df7b80 1 Processor -- start 2023-11-16T16:49:33.137 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.127+0000 7ff7e3df7b80 1 -- start start 2023-11-16T16:49:33.137 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.127+0000 7ff7e3df7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562bd250fb60 0x562bd250ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.138 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.128+0000 7ff7e3df7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562bd2510470 0x562bd2514370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.138 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.128+0000 7ff7e3df7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bd25148b0 0x562bd2514d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.138 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.128+0000 7ff7e3df7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bd2415060 con 0x562bd25148b0 2023-11-16T16:49:33.138 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.128+0000 7ff7e3df7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bd24678a0 con 0x562bd2510470 2023-11-16T16:49:33.138 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.128+0000 7ff7e3df7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bd2460d30 con 0x562bd250fb60 2023-11-16T16:49:33.138 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.128+0000 7ff7d6208700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bd25148b0 0x562bd2514d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.138 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.128+0000 7ff7d6208700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bd25148b0 0x562bd2514d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47234/0 (socket says 172.21.15.1:47234) 2023-11-16T16:49:33.138 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.128+0000 7ff7d6208700 1 -- 172.21.15.1:0/3364758818 learned_addr learned my addr 172.21.15.1:0/3364758818 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:49:33.139 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.128+0000 7ff7d5a07700 1 --2- 172.21.15.1:0/3364758818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562bd250fb60 0x562bd250ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.139 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.128+0000 7ff7d5206700 1 --2- 172.21.15.1:0/3364758818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562bd2510470 0x562bd2514370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.139 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.128+0000 7ff7d6208700 1 -- 172.21.15.1:0/3364758818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562bd250fb60 msgr2=0x562bd250ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:33.139 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.128+0000 7ff7d6208700 1 --2- 172.21.15.1:0/3364758818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562bd250fb60 0x562bd250ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.139 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.128+0000 7ff7d6208700 1 -- 172.21.15.1:0/3364758818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562bd2510470 msgr2=0x562bd2514370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:33.139 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.128+0000 7ff7d6208700 1 --2- 172.21.15.1:0/3364758818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562bd2510470 0x562bd2514370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.139 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.128+0000 7ff7d6208700 1 -- 172.21.15.1:0/3364758818 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bd22e2630 con 0x562bd25148b0 2023-11-16T16:49:33.139 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.128+0000 7ff7d6208700 1 --2- 172.21.15.1:0/3364758818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bd25148b0 0x562bd2514d00 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7ff7c800f410 tx=0x7ff7c800f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a5044e94e82170 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:33.140 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.128+0000 7ff7bbfff700 1 -- 172.21.15.1:0/3364758818 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8004bb0 con 0x562bd25148b0 2023-11-16T16:49:33.140 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.129+0000 7ff7bbfff700 1 -- 172.21.15.1:0/3364758818 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7c8004d10 con 0x562bd25148b0 2023-11-16T16:49:33.140 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.129+0000 7ff7bbfff700 1 -- 172.21.15.1:0/3364758818 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8005020 con 0x562bd25148b0 2023-11-16T16:49:33.140 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.129+0000 7ff7e3df7b80 1 -- 172.21.15.1:0/3364758818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bd25148b0 msgr2=0x562bd2514d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.140 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.129+0000 7ff7e3df7b80 1 --2- 172.21.15.1:0/3364758818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bd25148b0 0x562bd2514d00 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7ff7c800f410 tx=0x7ff7c800f720 comp rx=0 tx=0).stop 2023-11-16T16:49:33.140 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.129+0000 7ff7e3df7b80 1 -- 172.21.15.1:0/3364758818 shutdown_connections 2023-11-16T16:49:33.140 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.129+0000 7ff7e3df7b80 1 --2- 172.21.15.1:0/3364758818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bd25148b0 0x562bd2514d00 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.140 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.129+0000 7ff7e3df7b80 1 --2- 172.21.15.1:0/3364758818 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562bd250fb60 0x562bd250ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.140 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.129+0000 7ff7e3df7b80 1 --2- 172.21.15.1:0/3364758818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562bd2510470 0x562bd2514370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.141 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.129+0000 7ff7e3df7b80 1 -- 172.21.15.1:0/3364758818 >> 172.21.15.1:0/3364758818 conn(0x562bd2505360 msgr2=0x562bd250ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:33.141 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.129+0000 7ff7e3df7b80 1 -- 172.21.15.1:0/3364758818 shutdown_connections 2023-11-16T16:49:33.141 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.129+0000 7ff7e3df7b80 1 -- 172.21.15.1:0/3364758818 wait complete. 2023-11-16T16:49:33.141 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.130+0000 7ff7e3df7b80 1 -- start start 2023-11-16T16:49:33.141 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.130+0000 7ff7e3df7b80 1 Processor -- start 2023-11-16T16:49:33.141 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.130+0000 7ff7e3df7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562bd250fb60 0x562bd251bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.141 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.130+0000 7ff7e3df7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bd2510470 0x562bd2519250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.141 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.130+0000 7ff7e3df7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562bd25148b0 0x562bd2519880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.142 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.130+0000 7ff7e3df7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bd24028e0 con 0x562bd2510470 2023-11-16T16:49:33.142 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.130+0000 7ff7e3df7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bd2402580 con 0x562bd250fb60 2023-11-16T16:49:33.142 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.130+0000 7ff7e3df7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bd22e4920 con 0x562bd25148b0 2023-11-16T16:49:33.142 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.130+0000 7ff7e3df7b80 10 client.0 _ll_register_callbacks cb 0x562bd2420780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:49:33.142 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.130+0000 7ff7d5206700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bd2510470 0x562bd2519250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.142 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.130+0000 7ff7d5206700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bd2510470 0x562bd2519250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47238/0 (socket says 172.21.15.1:47238) 2023-11-16T16:49:33.142 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.130+0000 7ff7d5206700 1 -- 172.21.15.1:0/3160689259 learned_addr learned my addr 172.21.15.1:0/3160689259 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:49:33.142 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.130+0000 7ff7d5a07700 1 --2- 172.21.15.1:0/3160689259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562bd250fb60 0x562bd251bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.143 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.130+0000 7ff7d6208700 1 --2- 172.21.15.1:0/3160689259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562bd25148b0 0x562bd2519880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.143 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.131+0000 7ff7d5206700 1 -- 172.21.15.1:0/3160689259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562bd25148b0 msgr2=0x562bd2519880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:33.143 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.131+0000 7ff7d5206700 1 --2- 172.21.15.1:0/3160689259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562bd25148b0 0x562bd2519880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.143 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.131+0000 7ff7d5206700 1 -- 172.21.15.1:0/3160689259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562bd250fb60 msgr2=0x562bd251bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:33.143 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.131+0000 7ff7d5206700 1 --2- 172.21.15.1:0/3160689259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562bd250fb60 0x562bd251bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.143 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.131+0000 7ff7d5206700 1 -- 172.21.15.1:0/3160689259 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c8007e50 con 0x562bd2510470 2023-11-16T16:49:33.143 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.131+0000 7ff7d6208700 1 --2- 172.21.15.1:0/3160689259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562bd25148b0 0x562bd2519880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:49:33.143 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.131+0000 7ff7d5206700 1 --2- 172.21.15.1:0/3160689259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bd2510470 0x562bd2519250 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7ff7c401da60 tx=0x7ff7c40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f1e49215e2fa688 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:33.144 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.131+0000 7ff7b97fa700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:49:33.144 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.131+0000 7ff7b97fa700 1 -- 172.21.15.1:0/3160689259 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c4010900 con 0x562bd2510470 2023-11-16T16:49:33.144 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.131+0000 7ff7b97fa700 1 -- 172.21.15.1:0/3160689259 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7ff7c4010a60 con 0x562bd2510470 2023-11-16T16:49:33.144 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.131+0000 7ff7e3df7b80 1 -- 172.21.15.1:0/3160689259 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562bd2502b20 con 0x562bd2510470 2023-11-16T16:49:33.144 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.131+0000 7ff7b97fa700 1 -- 172.21.15.1:0/3160689259 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7c4010c20 con 0x562bd2510470 2023-11-16T16:49:33.144 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.131+0000 7ff7e3df7b80 10 client.17016 fetch_fsmap learned FSMap version 33 2023-11-16T16:49:33.144 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.131+0000 7ff7e3df7b80 1 -- 172.21.15.1:0/3160689259 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x562bd2427190 con 0x562bd2510470 2023-11-16T16:49:33.144 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.131+0000 7ff7b97fa700 1 -- 172.21.15.1:0/3160689259 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562bd2502b20 con 0x562bd2510470 2023-11-16T16:49:33.145 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.132+0000 7ff7b97fa700 1 -- 172.21.15.1:0/3160689259 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 33) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff7c40053d0 con 0x562bd2510470 2023-11-16T16:49:33.145 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.132+0000 7ff7e3df7b80 10 client.17016 fetch_fsmap finished waiting for FSMap version 33 2023-11-16T16:49:33.145 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.132+0000 7ff7e3df7b80 10 client.17016 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:49:33.145 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.132+0000 7ff7e3df7b80 20 client.17016 populate_metadata read hostname 'smithi001' 2023-11-16T16:49:33.145 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.132+0000 7ff7e3df7b80 1 --2- 172.21.15.1:0/3160689259 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x562bd2529950 0x562bd252bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.145 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.132+0000 7ff7e3df7b80 4 client.17016 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:49:33.145 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.132+0000 7ff7e3df7b80 1 -- 172.21.15.1:0/3160689259 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562bd24230a0 con 0x562bd2529950 2023-11-16T16:49:33.145 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.132+0000 7ff7d5a07700 1 --2- 172.21.15.1:0/3160689259 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x562bd2529950 0x562bd252bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.146 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.132+0000 7ff7b97fa700 10 client.17016 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:49:33.146 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.132+0000 7ff7d5a07700 1 --2- 172.21.15.1:0/3160689259 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x562bd2529950 0x562bd252bd60 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23864faf7f9a7ef8 server_cookie=eec0e3c7b854b534 in_seq=0 out_seq=0 2023-11-16T16:49:33.146 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.133+0000 7ff7b97fa700 1 -- 172.21.15.1:0/3160689259 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562bd24230a0 con 0x562bd2529950 2023-11-16T16:49:33.146 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.133+0000 7ff7b97fa700 10 client.17016 handle_command_reply: tid=0 2023-11-16T16:49:33.146 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.133+0000 7ff7e3df7b80 1 client.17016 shutdown 2023-11-16T16:49:33.146 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.133+0000 7ff7e3df7b80 1 -- 172.21.15.1:0/3160689259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bd2510470 msgr2=0x562bd2519250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.146 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.133+0000 7ff7e3df7b80 1 --2- 172.21.15.1:0/3160689259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bd2510470 0x562bd2519250 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7ff7c401da60 tx=0x7ff7c40047c0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.146 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.133+0000 7ff7e3df7b80 1 -- 172.21.15.1:0/3160689259 shutdown_connections 2023-11-16T16:49:33.147 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.133+0000 7ff7e3df7b80 1 --2- 172.21.15.1:0/3160689259 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x562bd2529950 0x562bd252bd60 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.147 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.133+0000 7ff7e3df7b80 1 --2- 172.21.15.1:0/3160689259 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562bd25148b0 0x562bd2519880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.147 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.133+0000 7ff7b97fa700 0 client.17016 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:49:33.147 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.133+0000 7ff7e3df7b80 1 --2- 172.21.15.1:0/3160689259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562bd250fb60 0x562bd251bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.147 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.133+0000 7ff7e3df7b80 1 --2- 172.21.15.1:0/3160689259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bd2510470 0x562bd2519250 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.147 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.134+0000 7ff7e3df7b80 1 -- 172.21.15.1:0/3160689259 >> 172.21.15.1:0/3160689259 conn(0x562bd2505360 msgr2=0x562bd2515d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:33.147 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.134+0000 7ff7e3df7b80 1 -- 172.21.15.1:0/3160689259 shutdown_connections 2023-11-16T16:49:33.147 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.134+0000 7ff7e3df7b80 1 -- 172.21.15.1:0/3160689259 wait complete. 2023-11-16T16:49:33.147 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.134+0000 7ff7e3df7b80 20 client.17016 trim_cache size 0 max 16384 2023-11-16T16:49:33.154 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.151+0000 7ff7e3df7b80 1 Processor -- start 2023-11-16T16:49:33.154 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.151+0000 7ff7e3df7b80 1 -- start start 2023-11-16T16:49:33.154 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.151+0000 7ff7e3df7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bd250fb60 0x562bd25209d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.155 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.151+0000 7ff7e3df7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562bd2510470 0x562bd2509c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.155 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.151+0000 7ff7e3df7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562bd25148b0 0x562bd25128b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.155 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.151+0000 7ff7e3df7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bd250c1c0 con 0x562bd250fb60 2023-11-16T16:49:33.155 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.151+0000 7ff7e3df7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bd251c310 con 0x562bd2510470 2023-11-16T16:49:33.155 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.151+0000 7ff7e3df7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bd25177d0 con 0x562bd25148b0 2023-11-16T16:49:33.155 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.151+0000 7ff7d5a07700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bd250fb60 0x562bd25209d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.155 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.151+0000 7ff7d5a07700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bd250fb60 0x562bd25209d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47244/0 (socket says 172.21.15.1:47244) 2023-11-16T16:49:33.156 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.151+0000 7ff7d5a07700 1 -- 172.21.15.1:0/1812986559 learned_addr learned my addr 172.21.15.1:0/1812986559 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:49:33.156 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.151+0000 7ff7d5206700 1 --2- 172.21.15.1:0/1812986559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562bd2510470 0x562bd2509c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.156 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.151+0000 7ff7d5a07700 1 -- 172.21.15.1:0/1812986559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562bd25148b0 msgr2=0x562bd25128b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:33.156 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.151+0000 7ff7d5a07700 1 --2- 172.21.15.1:0/1812986559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562bd25148b0 0x562bd25128b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.156 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.151+0000 7ff7d5a07700 1 -- 172.21.15.1:0/1812986559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562bd2510470 msgr2=0x562bd2509c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:33.156 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.151+0000 7ff7d5a07700 1 --2- 172.21.15.1:0/1812986559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562bd2510470 0x562bd2509c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.156 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.151+0000 7ff7d5a07700 1 -- 172.21.15.1:0/1812986559 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bd2509820 con 0x562bd250fb60 2023-11-16T16:49:33.156 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.151+0000 7ff7d5206700 1 --2- 172.21.15.1:0/1812986559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562bd2510470 0x562bd2509c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:49:33.156 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.151+0000 7ff7d5a07700 1 --2- 172.21.15.1:0/1812986559 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bd250fb60 0x562bd25209d0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc000c00 tx=0x7ff7cc009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b4d756067797c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:33.157 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.152+0000 7ff7c0dac700 1 -- 172.21.15.1:0/1812986559 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc09f2a0 con 0x562bd250fb60 2023-11-16T16:49:33.157 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.152+0000 7ff7c0dac700 1 -- 172.21.15.1:0/1812986559 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7cc0a3630 con 0x562bd250fb60 2023-11-16T16:49:33.157 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.152+0000 7ff7c0dac700 1 -- 172.21.15.1:0/1812986559 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc09a760 con 0x562bd250fb60 2023-11-16T16:49:33.157 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.152+0000 7ff7e3df7b80 1 -- 172.21.15.1:0/1812986559 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bd250fb60 msgr2=0x562bd25209d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.157 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.152+0000 7ff7e3df7b80 1 --2- 172.21.15.1:0/1812986559 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bd250fb60 0x562bd25209d0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc000c00 tx=0x7ff7cc009cd0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.157 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.152+0000 7ff7e3df7b80 1 -- 172.21.15.1:0/1812986559 shutdown_connections 2023-11-16T16:49:33.157 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.152+0000 7ff7e3df7b80 1 --2- 172.21.15.1:0/1812986559 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562bd25148b0 0x562bd25128b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.157 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.152+0000 7ff7e3df7b80 1 --2- 172.21.15.1:0/1812986559 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bd250fb60 0x562bd25209d0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.158 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.152+0000 7ff7e3df7b80 1 --2- 172.21.15.1:0/1812986559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562bd2510470 0x562bd2509c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.158 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.152+0000 7ff7e3df7b80 1 -- 172.21.15.1:0/1812986559 >> 172.21.15.1:0/1812986559 conn(0x562bd2505360 msgr2=0x562bd251bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:33.158 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.152+0000 7ff7e3df7b80 1 -- 172.21.15.1:0/1812986559 shutdown_connections 2023-11-16T16:49:33.158 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.152+0000 7ff7e3df7b80 1 -- 172.21.15.1:0/1812986559 wait complete. 2023-11-16T16:49:33.158 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.153+0000 7ff7e3df7b80 1 -- start start 2023-11-16T16:49:33.158 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.153+0000 7ff7e3df7b80 1 Processor -- start 2023-11-16T16:49:33.162 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.153+0000 7ff7e3df7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bd250fb60 0x562bd2520810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.162 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.153+0000 7ff7e3df7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562bd2510470 0x562bd251cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.162 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.153+0000 7ff7e3df7b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562bd25148b0 0x562bd251d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.162 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.153+0000 7ff7e3df7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bd251d5d0 con 0x562bd250fb60 2023-11-16T16:49:33.162 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.153+0000 7ff7e3df7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bd251d740 con 0x562bd2510470 2023-11-16T16:49:33.162 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.153+0000 7ff7e3df7b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bd251d900 con 0x562bd25148b0 2023-11-16T16:49:33.162 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.153+0000 7ff7e3df7b80 10 client.0 _ll_register_callbacks cb 0x7ff7c401d430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:49:33.162 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.153+0000 7ff7d5a07700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bd250fb60 0x562bd2520810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.163 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.153+0000 7ff7d5a07700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bd250fb60 0x562bd2520810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47250/0 (socket says 172.21.15.1:47250) 2023-11-16T16:49:33.163 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.153+0000 7ff7d5a07700 1 -- 172.21.15.1:0/2904871130 learned_addr learned my addr 172.21.15.1:0/2904871130 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:49:33.163 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.154+0000 7ff7d5206700 1 --2- 172.21.15.1:0/2904871130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562bd2510470 0x562bd251cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.163 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.154+0000 7ff7d6208700 1 --2- 172.21.15.1:0/2904871130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562bd25148b0 0x562bd251d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.163 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.154+0000 7ff7d5a07700 1 -- 172.21.15.1:0/2904871130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562bd25148b0 msgr2=0x562bd251d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:33.163 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.154+0000 7ff7d5a07700 1 --2- 172.21.15.1:0/2904871130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562bd25148b0 0x562bd251d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.163 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.154+0000 7ff7d5a07700 1 -- 172.21.15.1:0/2904871130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562bd2510470 msgr2=0x562bd251cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:33.163 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.154+0000 7ff7d5a07700 1 --2- 172.21.15.1:0/2904871130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562bd2510470 0x562bd251cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.164 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.154+0000 7ff7d5a07700 1 -- 172.21.15.1:0/2904871130 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d010f0e0 con 0x562bd250fb60 2023-11-16T16:49:33.164 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.154+0000 7ff7d6208700 1 --2- 172.21.15.1:0/2904871130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562bd25148b0 0x562bd251d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:49:33.164 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.154+0000 7ff7d5a07700 1 --2- 172.21.15.1:0/2904871130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bd250fb60 0x562bd2520810 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc02e040 tx=0x7ff7cc0cbdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33378f8bbc84da4e server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:33.164 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.154+0000 7ff7b97fa700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:49:33.164 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.154+0000 7ff7b97fa700 1 -- 172.21.15.1:0/2904871130 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc09a960 con 0x562bd250fb60 2023-11-16T16:49:33.164 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.154+0000 7ff7b97fa700 1 -- 172.21.15.1:0/2904871130 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7ff7cc09f2a0 con 0x562bd250fb60 2023-11-16T16:49:33.164 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.154+0000 7ff7e3df7b80 1 -- 172.21.15.1:0/2904871130 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff7c8004d10 con 0x562bd250fb60 2023-11-16T16:49:33.164 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.154+0000 7ff7b97fa700 1 -- 172.21.15.1:0/2904871130 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7cc09b4d0 con 0x562bd250fb60 2023-11-16T16:49:33.165 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.154+0000 7ff7e3df7b80 10 client.17022 fetch_fsmap learned FSMap version 33 2023-11-16T16:49:33.165 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.154+0000 7ff7e3df7b80 1 -- 172.21.15.1:0/2904871130 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x562bd228ede0 con 0x562bd250fb60 2023-11-16T16:49:33.165 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.154+0000 7ff7b97fa700 1 -- 172.21.15.1:0/2904871130 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7c8004d10 con 0x562bd250fb60 2023-11-16T16:49:33.165 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.155+0000 7ff7b97fa700 1 -- 172.21.15.1:0/2904871130 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 33) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff7cc0b3bc0 con 0x562bd250fb60 2023-11-16T16:49:33.165 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.155+0000 7ff7e3df7b80 10 client.17022 fetch_fsmap finished waiting for FSMap version 33 2023-11-16T16:49:33.165 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.155+0000 7ff7e3df7b80 10 client.17022 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:49:33.165 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.155+0000 7ff7e3df7b80 20 client.17022 populate_metadata read hostname 'smithi001' 2023-11-16T16:49:33.165 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.155+0000 7ff7e3df7b80 1 --2- 172.21.15.1:0/2904871130 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x562bd257e780 0x562bd2580b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.166 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.155+0000 7ff7e3df7b80 4 client.17022 mds_command: new command op to 24521 tid=0[{"prefix": "scrub status"}] 2023-11-16T16:49:33.166 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.155+0000 7ff7e3df7b80 1 -- 172.21.15.1:0/2904871130 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562bd242f660 con 0x562bd257e780 2023-11-16T16:49:33.166 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.155+0000 7ff7d5206700 1 --2- 172.21.15.1:0/2904871130 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x562bd257e780 0x562bd2580b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.166 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.155+0000 7ff7d5206700 1 --2- 172.21.15.1:0/2904871130 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x562bd257e780 0x562bd2580b60 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8de05031cbe32187 server_cookie=668119cab185c56c in_seq=0 out_seq=0 2023-11-16T16:49:33.166 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.155+0000 7ff7b97fa700 10 client.17022 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:49:33.166 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.155+0000 7ff7b97fa700 1 -- 172.21.15.1:0/2904871130 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562bd242f660 con 0x562bd257e780 2023-11-16T16:49:33.166 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.155+0000 7ff7b97fa700 10 client.17022 handle_command_reply: tid=0 2023-11-16T16:49:33.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.155+0000 7ff7e3df7b80 1 client.17022 shutdown 2023-11-16T16:49:33.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.155+0000 7ff7e3df7b80 1 -- 172.21.15.1:0/2904871130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bd250fb60 msgr2=0x562bd2520810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.155+0000 7ff7e3df7b80 1 --2- 172.21.15.1:0/2904871130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bd250fb60 0x562bd2520810 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc02e040 tx=0x7ff7cc0cbdc0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.156+0000 7ff7e3df7b80 1 -- 172.21.15.1:0/2904871130 shutdown_connections 2023-11-16T16:49:33.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.156+0000 7ff7e3df7b80 1 --2- 172.21.15.1:0/2904871130 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x562bd257e780 0x562bd2580b60 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.156+0000 7ff7e3df7b80 1 --2- 172.21.15.1:0/2904871130 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x562bd25148b0 0x562bd251d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.156+0000 7ff7e3df7b80 1 --2- 172.21.15.1:0/2904871130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562bd250fb60 0x562bd2520810 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.156+0000 7ff7e3df7b80 1 --2- 172.21.15.1:0/2904871130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562bd2510470 0x562bd251cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.168 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.156+0000 7ff7b97fa700 0 client.17022 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:49:33.168 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.156+0000 7ff7e3df7b80 1 -- 172.21.15.1:0/2904871130 >> 172.21.15.1:0/2904871130 conn(0x562bd2505360 msgr2=0x562bd2509c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:33.168 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.156+0000 7ff7e3df7b80 1 -- 172.21.15.1:0/2904871130 shutdown_connections 2023-11-16T16:49:33.168 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.156+0000 7ff7e3df7b80 1 -- 172.21.15.1:0/2904871130 wait complete. 2023-11-16T16:49:33.168 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.156+0000 7ff7e3df7b80 20 client.17022 trim_cache size 0 max 16384 2023-11-16T16:49:33.169 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:49:33.169 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2023-11-16T16:49:33.169 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2023-11-16T16:49:33.169 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:49:33.170 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.156+0000 7ff7d7c6b700 1 -- 172.21.15.1:0/2878234695 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7ff7bc0a12c0 msgr2=0x7ff7bc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.170 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.156+0000 7ff7d7c6b700 1 --2- 172.21.15.1:0/2878234695 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7ff7bc0a12c0 0x7ff7bc0a3770 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0069900 tx=0x7ff7c400a300 comp rx=0 tx=0).stop 2023-11-16T16:49:33.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.156+0000 7ff7d7c6b700 1 -- 172.21.15.1:0/2878234695 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7d00ff420 msgr2=0x7ff7d0197220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.156+0000 7ff7d7c6b700 1 --2- 172.21.15.1:0/2878234695 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7d00ff420 0x7ff7d0197220 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc004710 tx=0x7ff7cc004740 comp rx=0 tx=0).stop 2023-11-16T16:49:33.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.156+0000 7ff7d7c6b700 1 -- 172.21.15.1:0/2878234695 shutdown_connections 2023-11-16T16:49:33.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.156+0000 7ff7d7c6b700 1 --2- 172.21.15.1:0/2878234695 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7ff7bc0a12c0 0x7ff7bc0a3770 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.156+0000 7ff7d7c6b700 1 --2- 172.21.15.1:0/2878234695 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7ff7d0104460 0x7ff7d01a22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.156+0000 7ff7d7c6b700 1 --2- 172.21.15.1:0/2878234695 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7d00ff420 0x7ff7d0197220 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.156+0000 7ff7d7c6b700 1 --2- 172.21.15.1:0/2878234695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7d00ffdc0 0x7ff7d0197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.171 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.156+0000 7ff7d7c6b700 1 -- 172.21.15.1:0/2878234695 >> 172.21.15.1:0/2878234695 conn(0x7ff7d0076230 msgr2=0x7ff7d00fca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:33.172 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.156+0000 7ff7d7c6b700 1 -- 172.21.15.1:0/2878234695 shutdown_connections 2023-11-16T16:49:33.172 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.156+0000 7ff7d7c6b700 1 -- 172.21.15.1:0/2878234695 wait complete. 2023-11-16T16:49:33.172 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-16T16:49:33.172 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:49:33.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.297+0000 7fbc6c066700 1 -- 172.21.15.1:0/2774521436 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc640ff430 msgr2=0x7fbc640ff800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.297+0000 7fbc6c066700 1 --2- 172.21.15.1:0/2774521436 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc640ff430 0x7fbc640ff800 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbc600099c0 tx=0x7fbc60009cd0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.297+0000 7fbc6c066700 1 -- 172.21.15.1:0/2774521436 shutdown_connections 2023-11-16T16:49:33.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.297+0000 7fbc6c066700 1 --2- 172.21.15.1:0/2774521436 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc64104470 0x7fbc6410c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.302 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.297+0000 7fbc6c066700 1 --2- 172.21.15.1:0/2774521436 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc640ff430 0x7fbc640ff800 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.302 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.297+0000 7fbc6c066700 1 --2- 172.21.15.1:0/2774521436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc640ffdd0 0x7fbc64103d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.302 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.297+0000 7fbc6c066700 1 -- 172.21.15.1:0/2774521436 >> 172.21.15.1:0/2774521436 conn(0x7fbc64077060 msgr2=0x7fbc64077460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:33.302 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.297+0000 7fbc6c066700 1 -- 172.21.15.1:0/2774521436 shutdown_connections 2023-11-16T16:49:33.302 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.297+0000 7fbc6c066700 1 -- 172.21.15.1:0/2774521436 wait complete. 2023-11-16T16:49:33.302 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.298+0000 7fbc6c066700 1 Processor -- start 2023-11-16T16:49:33.302 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.298+0000 7fbc6c066700 1 -- start start 2023-11-16T16:49:33.302 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.298+0000 7fbc6c066700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc640ff430 0x7fbc64195300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.303 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.298+0000 7fbc6c066700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc640ffdd0 0x7fbc64195840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.303 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.298+0000 7fbc6c066700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc64104470 0x7fbc64199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.303 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.298+0000 7fbc6c066700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6419a2e0 con 0x7fbc640ff430 2023-11-16T16:49:33.303 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.298+0000 7fbc6c066700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6419a450 con 0x7fbc64104470 2023-11-16T16:49:33.303 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.298+0000 7fbc6c066700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6419a610 con 0x7fbc640ffdd0 2023-11-16T16:49:33.303 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.298+0000 7fbc69e02700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc640ff430 0x7fbc64195300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.303 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.298+0000 7fbc69e02700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc640ff430 0x7fbc64195300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47264/0 (socket says 172.21.15.1:47264) 2023-11-16T16:49:33.304 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.298+0000 7fbc69e02700 1 -- 172.21.15.1:0/2784360745 learned_addr learned my addr 172.21.15.1:0/2784360745 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:49:33.304 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.298+0000 7fbc69601700 1 --2- 172.21.15.1:0/2784360745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc640ffdd0 0x7fbc64195840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.304 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.298+0000 7fbc6a603700 1 --2- 172.21.15.1:0/2784360745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc64104470 0x7fbc64199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.304 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.298+0000 7fbc69e02700 1 -- 172.21.15.1:0/2784360745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc640ffdd0 msgr2=0x7fbc64195840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.304 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.298+0000 7fbc69e02700 1 --2- 172.21.15.1:0/2784360745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc640ffdd0 0x7fbc64195840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.304 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.298+0000 7fbc69e02700 1 -- 172.21.15.1:0/2784360745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc64104470 msgr2=0x7fbc64199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.304 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.299+0000 7fbc69e02700 1 --2- 172.21.15.1:0/2784360745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc64104470 0x7fbc64199ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.304 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.299+0000 7fbc69e02700 1 -- 172.21.15.1:0/2784360745 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc5c009c40 con 0x7fbc640ff430 2023-11-16T16:49:33.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.299+0000 7fbc69e02700 1 --2- 172.21.15.1:0/2784360745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc640ff430 0x7fbc64195300 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fbc6000b580 tx=0x7fbc6000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:33.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.299+0000 7fbc56ffd700 1 -- 172.21.15.1:0/2784360745 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6001d070 con 0x7fbc640ff430 2023-11-16T16:49:33.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.299+0000 7fbc56ffd700 1 -- 172.21.15.1:0/2784360745 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc60022470 con 0x7fbc640ff430 2023-11-16T16:49:33.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.299+0000 7fbc6c066700 1 -- 172.21.15.1:0/2784360745 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc600095f0 con 0x7fbc640ff430 2023-11-16T16:49:33.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.299+0000 7fbc56ffd700 1 -- 172.21.15.1:0/2784360745 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc60017650 con 0x7fbc640ff430 2023-11-16T16:49:33.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.299+0000 7fbc6c066700 1 -- 172.21.15.1:0/2784360745 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc6410f230 con 0x7fbc640ff430 2023-11-16T16:49:33.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.300+0000 7fbc6c066700 1 -- 172.21.15.1:0/2784360745 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc6410f0a0 con 0x7fbc640ff430 2023-11-16T16:49:33.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.301+0000 7fbc56ffd700 1 -- 172.21.15.1:0/2784360745 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fbc600225e0 con 0x7fbc640ff430 2023-11-16T16:49:33.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.302+0000 7fbc56ffd700 1 --2- 172.21.15.1:0/2784360745 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fbc500a12c0 0x7fbc500a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.302+0000 7fbc56ffd700 1 -- 172.21.15.1:0/2784360745 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fbc600cde30 con 0x7fbc640ff430 2023-11-16T16:49:33.307 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.302+0000 7fbc69601700 1 --2- 172.21.15.1:0/2784360745 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fbc500a12c0 0x7fbc500a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.307 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.302+0000 7fbc69601700 1 --2- 172.21.15.1:0/2784360745 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fbc500a12c0 0x7fbc500a3770 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbc64196910 tx=0x7fbc58009260 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:33.307 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.305+0000 7fbc56ffd700 1 -- 172.21.15.1:0/2784360745 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbc600972c0 con 0x7fbc640ff430 2023-11-16T16:49:33.551 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.550+0000 7fbc6c066700 1 -- 172.21.15.1:0/2784360745 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc6410f3c0 con 0x7fbc640ff430 2023-11-16T16:49:33.552 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.551+0000 7fbc56ffd700 1 -- 172.21.15.1:0/2784360745 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+10774 (secure 0 0 0) 0x7fbc6009a8e0 con 0x7fbc640ff430 2023-11-16T16:49:33.552 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:49:33.553 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2587563684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2587563684},{"type":"v1","addr":"172.21.15.1:6835","nonce":2587563684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/277936168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":277936168},{"type":"v1","addr":"172.21.15.1:6837","nonce":277936168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/1172575045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1172575045},{"type":"v1","addr":"172.21.15.1:6839","nonce":1172575045}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/2631175899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2631175899},{"type":"v1","addr":"172.21.15.1:6841","nonce":2631175899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3693905267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3693905267},{"type":"v1","addr":"172.21.15.53:6833","nonce":3693905267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/164668033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":164668033},{"type":"v1","addr":"172.21.15.26:6833","nonce":164668033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-16T16:37:13.918943+0000","modified":"2023-11-16T16:49:15.365148+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2140542192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2140542192},{"type":"v1","addr":"172.21.15.53:6835","nonce":2140542192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1010760275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1010760275},{"type":"v1","addr":"172.21.15.26:6835","nonce":1010760275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3197060302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3197060302},{"type":"v1","addr":"172.21.15.53:6837","nonce":3197060302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6837/1453504863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1453504863},{"type":"v1","addr":"172.21.15.26:6837","nonce":1453504863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.53:6839/1044753024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1044753024},{"type":"v1","addr":"172.21.15.53:6839","nonce":1044753024}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6839/2192205949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2192205949},{"type":"v1","addr":"172.21.15.26:6839","nonce":2192205949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-16T16:49:33.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.553+0000 7fbc6c066700 1 -- 172.21.15.1:0/2784360745 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fbc500a12c0 msgr2=0x7fbc500a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.553+0000 7fbc6c066700 1 --2- 172.21.15.1:0/2784360745 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fbc500a12c0 0x7fbc500a3770 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbc64196910 tx=0x7fbc58009260 comp rx=0 tx=0).stop 2023-11-16T16:49:33.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.553+0000 7fbc6c066700 1 -- 172.21.15.1:0/2784360745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc640ff430 msgr2=0x7fbc64195300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.553+0000 7fbc6c066700 1 --2- 172.21.15.1:0/2784360745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc640ff430 0x7fbc64195300 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fbc6000b580 tx=0x7fbc6000bfd0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.553+0000 7fbc6c066700 1 -- 172.21.15.1:0/2784360745 shutdown_connections 2023-11-16T16:49:33.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.553+0000 7fbc6c066700 1 --2- 172.21.15.1:0/2784360745 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fbc500a12c0 0x7fbc500a3770 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.553+0000 7fbc6c066700 1 --2- 172.21.15.1:0/2784360745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc64104470 0x7fbc64199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.553+0000 7fbc6c066700 1 --2- 172.21.15.1:0/2784360745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc640ff430 0x7fbc64195300 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.553+0000 7fbc6c066700 1 --2- 172.21.15.1:0/2784360745 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fbc640ffdd0 0x7fbc64195840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.553+0000 7fbc6c066700 1 -- 172.21.15.1:0/2784360745 >> 172.21.15.1:0/2784360745 conn(0x7fbc64077060 msgr2=0x7fbc640fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:33.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.553+0000 7fbc6c066700 1 -- 172.21.15.1:0/2784360745 shutdown_connections 2023-11-16T16:49:33.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.553+0000 7fbc6c066700 1 -- 172.21.15.1:0/2784360745 wait complete. 2023-11-16T16:49:33.557 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 33 2023-11-16T16:49:33.567 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-11-16T16:49:33.697 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.695+0000 7f708117b700 1 -- 172.21.15.1:0/3958103867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f707c0ffdb0 msgr2=0x7f707c103d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.697 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.695+0000 7f708117b700 1 --2- 172.21.15.1:0/3958103867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f707c0ffdb0 0x7f707c103d70 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f70640099c0 tx=0x7f7064009cd0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.695+0000 7f708117b700 1 -- 172.21.15.1:0/3958103867 shutdown_connections 2023-11-16T16:49:33.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.695+0000 7f708117b700 1 --2- 172.21.15.1:0/3958103867 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f707c104450 0x7f707c10c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.695+0000 7f708117b700 1 --2- 172.21.15.1:0/3958103867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f707c0ff410 0x7f707c0ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.695+0000 7f708117b700 1 --2- 172.21.15.1:0/3958103867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f707c0ffdb0 0x7f707c103d70 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.695+0000 7f708117b700 1 -- 172.21.15.1:0/3958103867 >> 172.21.15.1:0/3958103867 conn(0x7f707c0763a0 msgr2=0x7f707c0767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:33.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.695+0000 7f708117b700 1 -- 172.21.15.1:0/3958103867 shutdown_connections 2023-11-16T16:49:33.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.695+0000 7f708117b700 1 -- 172.21.15.1:0/3958103867 wait complete. 2023-11-16T16:49:33.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.696+0000 7f708117b700 1 Processor -- start 2023-11-16T16:49:33.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.696+0000 7f708117b700 1 -- start start 2023-11-16T16:49:33.701 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.696+0000 7f708117b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f707c0ff410 0x7f707c1971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.701 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.696+0000 7f708117b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f707c0ffdb0 0x7f707c197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.701 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.696+0000 7f708117b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f707c104450 0x7f707c1a2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.701 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.696+0000 7f708117b700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707c198020 con 0x7f707c0ffdb0 2023-11-16T16:49:33.701 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.696+0000 7f708117b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707c1a2790 con 0x7f707c104450 2023-11-16T16:49:33.702 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.696+0000 7f708117b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707c1a2900 con 0x7f707c0ff410 2023-11-16T16:49:33.702 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.696+0000 7f707a59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f707c0ffdb0 0x7f707c197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.702 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.696+0000 7f707a59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f707c0ffdb0 0x7f707c197710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47280/0 (socket says 172.21.15.1:47280) 2023-11-16T16:49:33.702 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.696+0000 7f707a59c700 1 -- 172.21.15.1:0/3090499510 learned_addr learned my addr 172.21.15.1:0/3090499510 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:49:33.702 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.696+0000 7f707b59e700 1 --2- 172.21.15.1:0/3090499510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f707c104450 0x7f707c1a2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.702 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.696+0000 7f707ad9d700 1 --2- 172.21.15.1:0/3090499510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f707c0ff410 0x7f707c1971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.702 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.697+0000 7f707a59c700 1 -- 172.21.15.1:0/3090499510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f707c0ff410 msgr2=0x7f707c1971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.703 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.697+0000 7f707a59c700 1 --2- 172.21.15.1:0/3090499510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f707c0ff410 0x7f707c1971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.703 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.697+0000 7f707a59c700 1 -- 172.21.15.1:0/3090499510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f707c104450 msgr2=0x7f707c1a2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.703 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.697+0000 7f707a59c700 1 --2- 172.21.15.1:0/3090499510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f707c104450 0x7f707c1a2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.703 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.697+0000 7f707a59c700 1 -- 172.21.15.1:0/3090499510 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70640095f0 con 0x7f707c0ffdb0 2023-11-16T16:49:33.703 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.697+0000 7f707a59c700 1 --2- 172.21.15.1:0/3090499510 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f707c0ffdb0 0x7f707c197710 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f7064009990 tx=0x7f7064009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:33.703 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.697+0000 7f7073fff700 1 -- 172.21.15.1:0/3090499510 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706401d070 con 0x7f707c0ffdb0 2023-11-16T16:49:33.703 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.697+0000 7f7073fff700 1 -- 172.21.15.1:0/3090499510 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7064003f90 con 0x7f707c0ffdb0 2023-11-16T16:49:33.703 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.697+0000 7f708117b700 1 -- 172.21.15.1:0/3090499510 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f707c10f220 con 0x7f707c0ffdb0 2023-11-16T16:49:33.704 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.697+0000 7f7073fff700 1 -- 172.21.15.1:0/3090499510 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7064022d00 con 0x7f707c0ffdb0 2023-11-16T16:49:33.704 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.697+0000 7f708117b700 1 -- 172.21.15.1:0/3090499510 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f707c10f090 con 0x7f707c0ffdb0 2023-11-16T16:49:33.704 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.699+0000 7f7073fff700 1 -- 172.21.15.1:0/3090499510 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f7064022470 con 0x7f707c0ffdb0 2023-11-16T16:49:33.710 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.700+0000 7f7073fff700 1 --2- 172.21.15.1:0/3090499510 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f70680a12c0 0x7f70680a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.710 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.700+0000 7f707ad9d700 1 --2- 172.21.15.1:0/3090499510 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f70680a12c0 0x7f70680a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.710 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.700+0000 7f7073fff700 1 -- 172.21.15.1:0/3090499510 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f70640cdf30 con 0x7f707c0ffdb0 2023-11-16T16:49:33.710 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.700+0000 7f707ad9d700 1 --2- 172.21.15.1:0/3090499510 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f70680a12c0 0x7f70680a3770 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f706c005950 tx=0x7f706c00b410 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:33.710 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.701+0000 7f708d307b80 1 Processor -- start 2023-11-16T16:49:33.711 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.701+0000 7f708d307b80 1 -- start start 2023-11-16T16:49:33.711 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.701+0000 7f708d307b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e3b593b60 0x561e3b593f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.711 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.701+0000 7f708d307b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e3b594470 0x561e3b598370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.711 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.701+0000 7f708d307b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e3b5988b0 0x561e3b598d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.711 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.701+0000 7f708d307b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e3b499060 con 0x561e3b593b60 2023-11-16T16:49:33.711 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.701+0000 7f708d307b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e3b4eb8a0 con 0x561e3b5988b0 2023-11-16T16:49:33.711 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.701+0000 7f708d307b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e3b4e4d30 con 0x561e3b594470 2023-11-16T16:49:33.711 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.701+0000 7f707ad9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e3b593b60 0x561e3b593f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.712 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.701+0000 7f707ad9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e3b593b60 0x561e3b593f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47286/0 (socket says 172.21.15.1:47286) 2023-11-16T16:49:33.712 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.701+0000 7f707ad9d700 1 -- 172.21.15.1:0/2999744054 learned_addr learned my addr 172.21.15.1:0/2999744054 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:49:33.712 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.701+0000 7f707b59e700 1 --2- 172.21.15.1:0/2999744054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e3b5988b0 0x561e3b598d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.712 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.701+0000 7f707a59c700 1 --2- 172.21.15.1:0/2999744054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e3b594470 0x561e3b598370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.712 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.701+0000 7f707ad9d700 1 -- 172.21.15.1:0/2999744054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e3b594470 msgr2=0x561e3b598370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:33.712 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.701+0000 7f707ad9d700 1 --2- 172.21.15.1:0/2999744054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e3b594470 0x561e3b598370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.712 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.701+0000 7f707ad9d700 1 -- 172.21.15.1:0/2999744054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e3b5988b0 msgr2=0x561e3b598d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:33.712 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.701+0000 7f707ad9d700 1 --2- 172.21.15.1:0/2999744054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e3b5988b0 0x561e3b598d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.713 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.701+0000 7f707ad9d700 1 -- 172.21.15.1:0/2999744054 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e3b366630 con 0x561e3b593b60 2023-11-16T16:49:33.713 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.701+0000 7f707b59e700 1 --2- 172.21.15.1:0/2999744054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e3b5988b0 0x561e3b598d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:49:33.713 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.702+0000 7f707ad9d700 1 --2- 172.21.15.1:0/2999744054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e3b593b60 0x561e3b593f30 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f706c010fa0 tx=0x7f706c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13fb2d76cedf5f8c server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:33.713 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.702+0000 7f70717fa700 1 -- 172.21.15.1:0/2999744054 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c01e620 con 0x561e3b593b60 2023-11-16T16:49:33.713 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.702+0000 7f70717fa700 1 -- 172.21.15.1:0/2999744054 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f706c015070 con 0x561e3b593b60 2023-11-16T16:49:33.713 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.702+0000 7f70717fa700 1 -- 172.21.15.1:0/2999744054 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c01c9e0 con 0x561e3b593b60 2023-11-16T16:49:33.713 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.702+0000 7f708d307b80 1 -- 172.21.15.1:0/2999744054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e3b593b60 msgr2=0x561e3b593f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.713 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.702+0000 7f708d307b80 1 --2- 172.21.15.1:0/2999744054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e3b593b60 0x561e3b593f30 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f706c010fa0 tx=0x7f706c013000 comp rx=0 tx=0).stop 2023-11-16T16:49:33.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.703+0000 7f708d307b80 1 -- 172.21.15.1:0/2999744054 shutdown_connections 2023-11-16T16:49:33.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.703+0000 7f708d307b80 1 --2- 172.21.15.1:0/2999744054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e3b5988b0 0x561e3b598d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.703+0000 7f708d307b80 1 --2- 172.21.15.1:0/2999744054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e3b593b60 0x561e3b593f30 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.703+0000 7f708d307b80 1 --2- 172.21.15.1:0/2999744054 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e3b594470 0x561e3b598370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.703+0000 7f708d307b80 1 -- 172.21.15.1:0/2999744054 >> 172.21.15.1:0/2999744054 conn(0x561e3b589360 msgr2=0x561e3b58eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:33.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.703+0000 7f708d307b80 1 -- 172.21.15.1:0/2999744054 shutdown_connections 2023-11-16T16:49:33.714 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.703+0000 7f708d307b80 1 -- 172.21.15.1:0/2999744054 wait complete. 2023-11-16T16:49:33.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.703+0000 7f708d307b80 1 -- start start 2023-11-16T16:49:33.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.703+0000 7f708d307b80 1 Processor -- start 2023-11-16T16:49:33.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.703+0000 7f708d307b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e3b593b60 0x561e3b59fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.703+0000 7f708d307b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e3b594470 0x561e3b59d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.703+0000 7f708d307b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e3b5988b0 0x561e3b59d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.703+0000 7f708d307b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e3b4868e0 con 0x561e3b594470 2023-11-16T16:49:33.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.703+0000 7f708d307b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e3b486580 con 0x561e3b593b60 2023-11-16T16:49:33.715 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.703+0000 7f708d307b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e3b368920 con 0x561e3b5988b0 2023-11-16T16:49:33.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.703+0000 7f707a59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e3b594470 0x561e3b59d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.703+0000 7f708d307b80 10 client.0 _ll_register_callbacks cb 0x561e3b4a4780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:49:33.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.704+0000 7f707a59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e3b594470 0x561e3b59d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47294/0 (socket says 172.21.15.1:47294) 2023-11-16T16:49:33.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.704+0000 7f707a59c700 1 -- 172.21.15.1:0/1818554959 learned_addr learned my addr 172.21.15.1:0/1818554959 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:49:33.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.704+0000 7f707ad9d700 1 --2- 172.21.15.1:0/1818554959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e3b593b60 0x561e3b59fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.704+0000 7f707a59c700 1 -- 172.21.15.1:0/1818554959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e3b5988b0 msgr2=0x561e3b59d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:33.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.704+0000 7f707b59e700 1 --2- 172.21.15.1:0/1818554959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e3b5988b0 0x561e3b59d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.716 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.704+0000 7f707a59c700 1 --2- 172.21.15.1:0/1818554959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e3b5988b0 0x561e3b59d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.704+0000 7f707a59c700 1 -- 172.21.15.1:0/1818554959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e3b593b60 msgr2=0x561e3b59fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:33.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.704+0000 7f707a59c700 1 --2- 172.21.15.1:0/1818554959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e3b593b60 0x561e3b59fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.704+0000 7f707a59c700 1 -- 172.21.15.1:0/1818554959 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f706c010c50 con 0x561e3b594470 2023-11-16T16:49:33.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.704+0000 7f707ad9d700 1 --2- 172.21.15.1:0/1818554959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e3b593b60 0x561e3b59fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:49:33.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.704+0000 7f707a59c700 1 --2- 172.21.15.1:0/1818554959 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e3b594470 0x561e3b59d270 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f7064000c00 tx=0x7f7064023f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=343f4b727de81f7d server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:33.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.704+0000 7f70627fc700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:49:33.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.704+0000 7f70627fc700 1 -- 172.21.15.1:0/1818554959 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706409b480 con 0x561e3b594470 2023-11-16T16:49:33.717 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.704+0000 7f70627fc700 1 -- 172.21.15.1:0/1818554959 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f70640a33e0 con 0x561e3b594470 2023-11-16T16:49:33.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.704+0000 7f708d307b80 1 -- 172.21.15.1:0/1818554959 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561e3b586b20 con 0x561e3b594470 2023-11-16T16:49:33.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.704+0000 7f70627fc700 1 -- 172.21.15.1:0/1818554959 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f706409a510 con 0x561e3b594470 2023-11-16T16:49:33.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.704+0000 7f708d307b80 10 client.17040 fetch_fsmap learned FSMap version 33 2023-11-16T16:49:33.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.704+0000 7f708d307b80 1 -- 172.21.15.1:0/1818554959 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x561e3b4ab190 con 0x561e3b594470 2023-11-16T16:49:33.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.705+0000 7f70627fc700 1 -- 172.21.15.1:0/1818554959 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561e3b586b20 con 0x561e3b594470 2023-11-16T16:49:33.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.705+0000 7f70627fc700 1 -- 172.21.15.1:0/1818554959 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 33) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7064096e10 con 0x561e3b594470 2023-11-16T16:49:33.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.705+0000 7f708d307b80 10 client.17040 fetch_fsmap finished waiting for FSMap version 33 2023-11-16T16:49:33.718 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.705+0000 7f708d307b80 10 client.17040 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-11-16T16:49:33.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.705+0000 7f708d307b80 20 client.17040 populate_metadata read hostname 'smithi001' 2023-11-16T16:49:33.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.705+0000 7f708d307b80 1 --2- 172.21.15.1:0/1818554959 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x561e3b5ad2f0 0x561e3b5af700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.705+0000 7f708d307b80 4 client.17040 mds_command: new command op to 24482 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:49:33.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.705+0000 7f708d307b80 1 -- 172.21.15.1:0/1818554959 --> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561e3b4a70a0 con 0x561e3b5ad2f0 2023-11-16T16:49:33.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.705+0000 7f707ad9d700 1 --2- 172.21.15.1:0/1818554959 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x561e3b5ad2f0 0x561e3b5af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.705+0000 7f70627fc700 10 client.17040 ms_handle_connect on v2:172.21.15.26:6836/1453504863 2023-11-16T16:49:33.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.705+0000 7f707ad9d700 1 --2- 172.21.15.1:0/1818554959 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x561e3b5ad2f0 0x561e3b5af700 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6f92362011f7bff0 server_cookie=142b689e8b15afa8 in_seq=0 out_seq=0 2023-11-16T16:49:33.719 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.706+0000 7f70627fc700 1 -- 172.21.15.1:0/1818554959 <== mds.2 v2:172.21.15.26:6836/1453504863 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561e3b4a70a0 con 0x561e3b5ad2f0 2023-11-16T16:49:33.720 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.706+0000 7f70627fc700 10 client.17040 handle_command_reply: tid=0 2023-11-16T16:49:33.720 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.706+0000 7f708d307b80 1 client.17040 shutdown 2023-11-16T16:49:33.720 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.706+0000 7f708d307b80 1 -- 172.21.15.1:0/1818554959 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e3b594470 msgr2=0x561e3b59d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.720 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.706+0000 7f708d307b80 1 --2- 172.21.15.1:0/1818554959 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e3b594470 0x561e3b59d270 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f7064000c00 tx=0x7f7064023f20 comp rx=0 tx=0).stop 2023-11-16T16:49:33.720 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.706+0000 7f708d307b80 1 -- 172.21.15.1:0/1818554959 shutdown_connections 2023-11-16T16:49:33.720 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.706+0000 7f708d307b80 1 --2- 172.21.15.1:0/1818554959 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x561e3b5ad2f0 0x561e3b5af700 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.720 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.706+0000 7f708d307b80 1 --2- 172.21.15.1:0/1818554959 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e3b5988b0 0x561e3b59d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.720 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.706+0000 7f708d307b80 1 --2- 172.21.15.1:0/1818554959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e3b593b60 0x561e3b59fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.721 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.706+0000 7f708d307b80 1 --2- 172.21.15.1:0/1818554959 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e3b594470 0x561e3b59d270 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.721 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.706+0000 7f70627fc700 0 client.17040 ms_handle_reset on v2:172.21.15.26:6836/1453504863 2023-11-16T16:49:33.721 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.706+0000 7f708d307b80 1 -- 172.21.15.1:0/1818554959 >> 172.21.15.1:0/1818554959 conn(0x561e3b589360 msgr2=0x561e3b599d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:33.721 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.706+0000 7f708d307b80 1 -- 172.21.15.1:0/1818554959 shutdown_connections 2023-11-16T16:49:33.721 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.707+0000 7f708d307b80 1 -- 172.21.15.1:0/1818554959 wait complete. 2023-11-16T16:49:33.721 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.707+0000 7f708d307b80 20 client.17040 trim_cache size 0 max 16384 2023-11-16T16:49:33.727 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.720+0000 7f708d307b80 1 Processor -- start 2023-11-16T16:49:33.728 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.720+0000 7f708d307b80 1 -- start start 2023-11-16T16:49:33.728 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.720+0000 7f708d307b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e3b593b60 0x561e3b5a9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.728 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.720+0000 7f708d307b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e3b594470 0x561e3b59fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.728 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.720+0000 7f708d307b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e3b5988b0 0x561e3b5968b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.728 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.720+0000 7f708d307b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e3b58f900 con 0x561e3b594470 2023-11-16T16:49:33.728 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.720+0000 7f708d307b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e3b588f10 con 0x561e3b593b60 2023-11-16T16:49:33.728 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.720+0000 7f708d307b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e3b597450 con 0x561e3b5988b0 2023-11-16T16:49:33.729 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.720+0000 7f707a59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e3b594470 0x561e3b59fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.729 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.720+0000 7f707a59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e3b594470 0x561e3b59fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47302/0 (socket says 172.21.15.1:47302) 2023-11-16T16:49:33.729 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.720+0000 7f707a59c700 1 -- 172.21.15.1:0/2557520976 learned_addr learned my addr 172.21.15.1:0/2557520976 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:49:33.729 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.720+0000 7f707ad9d700 1 --2- 172.21.15.1:0/2557520976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e3b593b60 0x561e3b5a9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.729 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.720+0000 7f707b59e700 1 --2- 172.21.15.1:0/2557520976 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e3b5988b0 0x561e3b5968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.729 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.720+0000 7f707a59c700 1 -- 172.21.15.1:0/2557520976 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e3b5988b0 msgr2=0x561e3b5968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:33.729 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.720+0000 7f707a59c700 1 --2- 172.21.15.1:0/2557520976 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e3b5988b0 0x561e3b5968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.729 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.720+0000 7f707a59c700 1 -- 172.21.15.1:0/2557520976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e3b593b60 msgr2=0x561e3b5a9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:33.730 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.720+0000 7f707a59c700 1 --2- 172.21.15.1:0/2557520976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e3b593b60 0x561e3b5a9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.730 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.720+0000 7f707a59c700 1 -- 172.21.15.1:0/2557520976 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e3b58d820 con 0x561e3b594470 2023-11-16T16:49:33.730 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.720+0000 7f707ad9d700 1 --2- 172.21.15.1:0/2557520976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e3b593b60 0x561e3b5a9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:49:33.730 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.721+0000 7f707a59c700 1 --2- 172.21.15.1:0/2557520976 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e3b594470 0x561e3b59fb90 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f7064004b10 tx=0x7f70640995d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d8f2011c8d2583 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:33.730 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.721+0000 7f7071ffb700 1 -- 172.21.15.1:0/2557520976 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70640a33e0 con 0x561e3b594470 2023-11-16T16:49:33.730 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.721+0000 7f7071ffb700 1 -- 172.21.15.1:0/2557520976 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7064099f80 con 0x561e3b594470 2023-11-16T16:49:33.730 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.721+0000 7f7071ffb700 1 -- 172.21.15.1:0/2557520976 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70640b93d0 con 0x561e3b594470 2023-11-16T16:49:33.730 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.721+0000 7f708d307b80 1 -- 172.21.15.1:0/2557520976 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e3b594470 msgr2=0x561e3b59fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.731 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.721+0000 7f708d307b80 1 --2- 172.21.15.1:0/2557520976 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e3b594470 0x561e3b59fb90 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f7064004b10 tx=0x7f70640995d0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.731 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.721+0000 7f708d307b80 1 -- 172.21.15.1:0/2557520976 shutdown_connections 2023-11-16T16:49:33.731 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.721+0000 7f708d307b80 1 --2- 172.21.15.1:0/2557520976 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e3b5988b0 0x561e3b5968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.731 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.721+0000 7f708d307b80 1 --2- 172.21.15.1:0/2557520976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e3b593b60 0x561e3b5a9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.731 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.721+0000 7f708d307b80 1 --2- 172.21.15.1:0/2557520976 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e3b594470 0x561e3b59fb90 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.731 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.721+0000 7f708d307b80 1 -- 172.21.15.1:0/2557520976 >> 172.21.15.1:0/2557520976 conn(0x561e3b589360 msgr2=0x561e3b59ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:33.731 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.721+0000 7f708d307b80 1 -- 172.21.15.1:0/2557520976 shutdown_connections 2023-11-16T16:49:33.731 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.721+0000 7f708d307b80 1 -- 172.21.15.1:0/2557520976 wait complete. 2023-11-16T16:49:33.732 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.722+0000 7f708d307b80 1 -- start start 2023-11-16T16:49:33.732 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.722+0000 7f708d307b80 1 Processor -- start 2023-11-16T16:49:33.732 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.722+0000 7f708d307b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e3b593b60 0x561e3b5fd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.732 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.722+0000 7f708d307b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e3b594470 0x561e3b5fdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.732 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.722+0000 7f708d307b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e3b5988b0 0x561e3b5f9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.732 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.722+0000 7f708d307b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e3b5f9870 con 0x561e3b594470 2023-11-16T16:49:33.732 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.722+0000 7f708d307b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e3b5f99e0 con 0x561e3b593b60 2023-11-16T16:49:33.732 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.722+0000 7f708d307b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e3b5f9ba0 con 0x561e3b5988b0 2023-11-16T16:49:33.733 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.722+0000 7f708d307b80 10 client.0 _ll_register_callbacks cb 0x7f706409ab60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:49:33.733 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.722+0000 7f707a59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e3b594470 0x561e3b5fdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.733 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.722+0000 7f707a59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e3b594470 0x561e3b5fdc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47308/0 (socket says 172.21.15.1:47308) 2023-11-16T16:49:33.733 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.722+0000 7f707a59c700 1 -- 172.21.15.1:0/1721849367 learned_addr learned my addr 172.21.15.1:0/1721849367 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:49:33.733 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.722+0000 7f707ad9d700 1 --2- 172.21.15.1:0/1721849367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e3b593b60 0x561e3b5fd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.733 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.722+0000 7f707b59e700 1 --2- 172.21.15.1:0/1721849367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e3b5988b0 0x561e3b5f9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.733 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.722+0000 7f707a59c700 1 -- 172.21.15.1:0/1721849367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e3b5988b0 msgr2=0x561e3b5f9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:33.734 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.722+0000 7f707a59c700 1 --2- 172.21.15.1:0/1721849367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e3b5988b0 0x561e3b5f9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.734 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.723+0000 7f707a59c700 1 -- 172.21.15.1:0/1721849367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e3b593b60 msgr2=0x561e3b5fd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:33.734 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.723+0000 7f707a59c700 1 --2- 172.21.15.1:0/1721849367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e3b593b60 0x561e3b5fd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.734 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.723+0000 7f707a59c700 1 -- 172.21.15.1:0/1721849367 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e3b4ab190 con 0x561e3b594470 2023-11-16T16:49:33.734 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.723+0000 7f70627fc700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:49:33.734 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.723+0000 7f707a59c700 1 --2- 172.21.15.1:0/1721849367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e3b594470 0x561e3b5fdc80 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f70640a2eb0 tx=0x7f7064023f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f4d3f3693ea101 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:33.734 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.723+0000 7f70627fc700 1 -- 172.21.15.1:0/1721849367 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706409a710 con 0x561e3b594470 2023-11-16T16:49:33.734 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.723+0000 7f70627fc700 1 -- 172.21.15.1:0/1721849367 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f706409a2e0 con 0x561e3b594470 2023-11-16T16:49:33.735 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.723+0000 7f708d307b80 1 -- 172.21.15.1:0/1721849367 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f706c015070 con 0x561e3b594470 2023-11-16T16:49:33.735 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.723+0000 7f70627fc700 1 -- 172.21.15.1:0/1721849367 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7064062080 con 0x561e3b594470 2023-11-16T16:49:33.735 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.723+0000 7f708d307b80 10 client.17046 fetch_fsmap learned FSMap version 33 2023-11-16T16:49:33.735 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.723+0000 7f708d307b80 1 -- 172.21.15.1:0/1721849367 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x561e3b312de0 con 0x561e3b594470 2023-11-16T16:49:33.735 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.723+0000 7f70627fc700 1 -- 172.21.15.1:0/1721849367 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f706407d070 con 0x561e3b594470 2023-11-16T16:49:33.735 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.723+0000 7f70627fc700 1 -- 172.21.15.1:0/1721849367 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 33) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f70640cb120 con 0x561e3b594470 2023-11-16T16:49:33.735 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.723+0000 7f708d307b80 10 client.17046 fetch_fsmap finished waiting for FSMap version 33 2023-11-16T16:49:33.735 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.723+0000 7f708d307b80 10 client.17046 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-11-16T16:49:33.736 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.723+0000 7f708d307b80 20 client.17046 populate_metadata read hostname 'smithi001' 2023-11-16T16:49:33.736 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.723+0000 7f708d307b80 1 --2- 172.21.15.1:0/1721849367 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x561e3b6061b0 0x561e3b608590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.736 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.723+0000 7f708d307b80 4 client.17046 mds_command: new command op to 24482 tid=0[{"prefix": "damage ls"}] 2023-11-16T16:49:33.736 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.723+0000 7f708d307b80 1 -- 172.21.15.1:0/1721849367 --> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561e3b4b3660 con 0x561e3b6061b0 2023-11-16T16:49:33.736 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.724+0000 7f707ad9d700 1 --2- 172.21.15.1:0/1721849367 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x561e3b6061b0 0x561e3b608590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.736 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.724+0000 7f70627fc700 10 client.17046 ms_handle_connect on v2:172.21.15.26:6836/1453504863 2023-11-16T16:49:33.736 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.724+0000 7f707ad9d700 1 --2- 172.21.15.1:0/1721849367 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x561e3b6061b0 0x561e3b608590 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cf90a153025f8064 server_cookie=9302aed65e40afd8 in_seq=0 out_seq=0 2023-11-16T16:49:33.736 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.724+0000 7f70627fc700 1 -- 172.21.15.1:0/1721849367 <== mds.2 v2:172.21.15.26:6836/1453504863 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561e3b4b3660 con 0x561e3b6061b0 2023-11-16T16:49:33.737 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.724+0000 7f70627fc700 10 client.17046 handle_command_reply: tid=0 2023-11-16T16:49:33.737 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.724+0000 7f708d307b80 1 client.17046 shutdown 2023-11-16T16:49:33.737 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.724+0000 7f708d307b80 1 -- 172.21.15.1:0/1721849367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e3b594470 msgr2=0x561e3b5fdc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.737 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.724+0000 7f708d307b80 1 --2- 172.21.15.1:0/1721849367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e3b594470 0x561e3b5fdc80 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f70640a2eb0 tx=0x7f7064023f20 comp rx=0 tx=0).stop 2023-11-16T16:49:33.737 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.724+0000 7f708d307b80 1 -- 172.21.15.1:0/1721849367 shutdown_connections 2023-11-16T16:49:33.737 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.724+0000 7f708d307b80 1 --2- 172.21.15.1:0/1721849367 >> [v2:172.21.15.26:6836/1453504863,v1:172.21.15.26:6837/1453504863] conn(0x561e3b6061b0 0x561e3b608590 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.737 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.724+0000 7f708d307b80 1 --2- 172.21.15.1:0/1721849367 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x561e3b5988b0 0x561e3b5f9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.737 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.724+0000 7f70627fc700 0 client.17046 ms_handle_reset on v2:172.21.15.26:6836/1453504863 2023-11-16T16:49:33.738 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.724+0000 7f708d307b80 1 --2- 172.21.15.1:0/1721849367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e3b593b60 0x561e3b5fd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.738 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.724+0000 7f708d307b80 1 --2- 172.21.15.1:0/1721849367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e3b594470 0x561e3b5fdc80 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.738 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.724+0000 7f708d307b80 1 -- 172.21.15.1:0/1721849367 >> 172.21.15.1:0/1721849367 conn(0x561e3b589360 msgr2=0x561e3b5a9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:33.738 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.724+0000 7f708d307b80 1 -- 172.21.15.1:0/1721849367 shutdown_connections 2023-11-16T16:49:33.738 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.725+0000 7f708d307b80 1 -- 172.21.15.1:0/1721849367 wait complete. 2023-11-16T16:49:33.738 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.725+0000 7f708d307b80 20 client.17046 trim_cache size 0 max 16384 2023-11-16T16:49:33.739 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2023-11-16T16:49:33.740 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.725+0000 7f708117b700 1 -- 172.21.15.1:0/3090499510 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f70680a12c0 msgr2=0x7f70680a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.740 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.725+0000 7f708117b700 1 --2- 172.21.15.1:0/3090499510 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f70680a12c0 0x7f70680a3770 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f706c005950 tx=0x7f706c00b410 comp rx=0 tx=0).stop 2023-11-16T16:49:33.740 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.725+0000 7f708117b700 1 -- 172.21.15.1:0/3090499510 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f707c0ffdb0 msgr2=0x7f707c197710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.740 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.725+0000 7f708117b700 1 --2- 172.21.15.1:0/3090499510 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f707c0ffdb0 0x7f707c197710 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f7064009990 tx=0x7f7064009580 comp rx=0 tx=0).stop 2023-11-16T16:49:33.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.725+0000 7f708117b700 1 -- 172.21.15.1:0/3090499510 shutdown_connections 2023-11-16T16:49:33.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.725+0000 7f708117b700 1 --2- 172.21.15.1:0/3090499510 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f70680a12c0 0x7f70680a3770 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.725+0000 7f708117b700 1 --2- 172.21.15.1:0/3090499510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f707c104450 0x7f707c1a2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.725+0000 7f708117b700 1 --2- 172.21.15.1:0/3090499510 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f707c0ff410 0x7f707c1971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.725+0000 7f708117b700 1 --2- 172.21.15.1:0/3090499510 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f707c0ffdb0 0x7f707c197710 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.725+0000 7f708117b700 1 -- 172.21.15.1:0/3090499510 >> 172.21.15.1:0/3090499510 conn(0x7f707c0763a0 msgr2=0x7f707c0fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:33.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.725+0000 7f708117b700 1 -- 172.21.15.1:0/3090499510 shutdown_connections 2023-11-16T16:49:33.741 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.725+0000 7f708117b700 1 -- 172.21.15.1:0/3090499510 wait complete. 2023-11-16T16:49:33.742 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-11-16T16:49:33.869 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.867+0000 7f20a5b52700 1 -- 172.21.15.1:0/3686487322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20a00ff410 msgr2=0x7f20a00ff7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.869 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.867+0000 7f20a5b52700 1 --2- 172.21.15.1:0/3686487322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20a00ff410 0x7f20a00ff7e0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f20900099c0 tx=0x7f2090009cd0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.869 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.867+0000 7f20a5b52700 1 -- 172.21.15.1:0/3686487322 shutdown_connections 2023-11-16T16:49:33.870 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.867+0000 7f20a5b52700 1 --2- 172.21.15.1:0/3686487322 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20a0104450 0x7f20a010c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.870 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.867+0000 7f20a5b52700 1 --2- 172.21.15.1:0/3686487322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20a00ff410 0x7f20a00ff7e0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.870 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.867+0000 7f20a5b52700 1 --2- 172.21.15.1:0/3686487322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20a00ffdb0 0x7f20a0103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.870 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.867+0000 7f20a5b52700 1 -- 172.21.15.1:0/3686487322 >> 172.21.15.1:0/3686487322 conn(0x7f20a0076370 msgr2=0x7f20a0076770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:33.870 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.867+0000 7f20a5b52700 1 -- 172.21.15.1:0/3686487322 shutdown_connections 2023-11-16T16:49:33.870 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.867+0000 7f20a5b52700 1 -- 172.21.15.1:0/3686487322 wait complete. 2023-11-16T16:49:33.870 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.868+0000 7f20a5b52700 1 Processor -- start 2023-11-16T16:49:33.872 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.868+0000 7f20a5b52700 1 -- start start 2023-11-16T16:49:33.872 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.868+0000 7f20a5b52700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20a00ff410 0x7f20a0197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.873 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.868+0000 7f20a5b52700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20a00ffdb0 0x7f20a0197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.873 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.868+0000 7f20a5b52700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20a0104450 0x7f20a01a21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.873 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.868+0000 7f20a5b52700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a0197fb0 con 0x7f20a00ffdb0 2023-11-16T16:49:33.873 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.868+0000 7f20a5b52700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a01a26f0 con 0x7f20a0104450 2023-11-16T16:49:33.873 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.868+0000 7f20a5b52700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a01a28b0 con 0x7f20a00ff410 2023-11-16T16:49:33.873 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.868+0000 7f209effd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20a00ffdb0 0x7f20a0197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.873 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.868+0000 7f209effd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20a00ffdb0 0x7f20a0197670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47322/0 (socket says 172.21.15.1:47322) 2023-11-16T16:49:33.874 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.868+0000 7f209effd700 1 -- 172.21.15.1:0/1375358774 learned_addr learned my addr 172.21.15.1:0/1375358774 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:49:33.874 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.868+0000 7f209f7fe700 1 --2- 172.21.15.1:0/1375358774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20a00ff410 0x7f20a0197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.874 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.869+0000 7f209ffff700 1 --2- 172.21.15.1:0/1375358774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20a0104450 0x7f20a01a21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.874 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.869+0000 7f209effd700 1 -- 172.21.15.1:0/1375358774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20a00ff410 msgr2=0x7f20a0197130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.874 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.869+0000 7f209effd700 1 --2- 172.21.15.1:0/1375358774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20a00ff410 0x7f20a0197130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.874 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.869+0000 7f209effd700 1 -- 172.21.15.1:0/1375358774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20a0104450 msgr2=0x7f20a01a21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.874 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.869+0000 7f209effd700 1 --2- 172.21.15.1:0/1375358774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20a0104450 0x7f20a01a21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.874 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.869+0000 7f209effd700 1 -- 172.21.15.1:0/1375358774 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20900095f0 con 0x7f20a00ffdb0 2023-11-16T16:49:33.875 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.869+0000 7f209effd700 1 --2- 172.21.15.1:0/1375358774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20a00ffdb0 0x7f20a0197670 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f208800c9b0 tx=0x7f208800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:33.875 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.869+0000 7f209cff9700 1 -- 172.21.15.1:0/1375358774 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2088007aa0 con 0x7f20a00ffdb0 2023-11-16T16:49:33.875 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.869+0000 7f20a5b52700 1 -- 172.21.15.1:0/1375358774 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20a010f210 con 0x7f20a00ffdb0 2023-11-16T16:49:33.875 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.869+0000 7f209cff9700 1 -- 172.21.15.1:0/1375358774 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2088007c00 con 0x7f20a00ffdb0 2023-11-16T16:49:33.875 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.869+0000 7f209cff9700 1 -- 172.21.15.1:0/1375358774 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2088018660 con 0x7f20a00ffdb0 2023-11-16T16:49:33.875 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.869+0000 7f20a5b52700 1 -- 172.21.15.1:0/1375358774 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a010f080 con 0x7f20a00ffdb0 2023-11-16T16:49:33.881 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.871+0000 7f209cff9700 1 -- 172.21.15.1:0/1375358774 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f208800f450 con 0x7f20a00ffdb0 2023-11-16T16:49:33.882 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.872+0000 7f209cff9700 1 --2- 172.21.15.1:0/1375358774 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f208c0a12c0 0x7f208c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.882 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.872+0000 7f209f7fe700 1 --2- 172.21.15.1:0/1375358774 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f208c0a12c0 0x7f208c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.882 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.872+0000 7f209cff9700 1 -- 172.21.15.1:0/1375358774 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f20880ccca0 con 0x7f20a00ffdb0 2023-11-16T16:49:33.882 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.872+0000 7f209f7fe700 1 --2- 172.21.15.1:0/1375358774 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f208c0a12c0 0x7f208c0a3770 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2090009990 tx=0x7f2090003680 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:33.882 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.873+0000 7f20b1cdeb80 1 Processor -- start 2023-11-16T16:49:33.882 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.873+0000 7f20b1cdeb80 1 -- start start 2023-11-16T16:49:33.882 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.873+0000 7f20b1cdeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555712b87b60 0x555712b87f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.883 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.873+0000 7f20b1cdeb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555712b88470 0x555712b8c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.883 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.873+0000 7f20b1cdeb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555712b8c8b0 0x555712b8cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.883 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.873+0000 7f20b1cdeb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x555712a8d060 con 0x555712b88470 2023-11-16T16:49:33.883 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.873+0000 7f20b1cdeb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555712adf8a0 con 0x555712b8c8b0 2023-11-16T16:49:33.883 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.873+0000 7f20b1cdeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555712ad8d30 con 0x555712b87b60 2023-11-16T16:49:33.883 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.873+0000 7f209effd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555712b88470 0x555712b8c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.883 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.873+0000 7f209effd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555712b88470 0x555712b8c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47330/0 (socket says 172.21.15.1:47330) 2023-11-16T16:49:33.883 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.873+0000 7f209effd700 1 -- 172.21.15.1:0/2866199666 learned_addr learned my addr 172.21.15.1:0/2866199666 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:49:33.884 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.873+0000 7f209effd700 1 -- 172.21.15.1:0/2866199666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555712b87b60 msgr2=0x555712b87f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:33.884 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.873+0000 7f209effd700 1 --2- 172.21.15.1:0/2866199666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555712b87b60 0x555712b87f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.884 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.873+0000 7f209effd700 1 -- 172.21.15.1:0/2866199666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555712b8c8b0 msgr2=0x555712b8cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:33.884 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.873+0000 7f209effd700 1 --2- 172.21.15.1:0/2866199666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555712b8c8b0 0x555712b8cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.884 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.873+0000 7f209effd700 1 -- 172.21.15.1:0/2866199666 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55571295a630 con 0x555712b88470 2023-11-16T16:49:33.884 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.873+0000 7f209effd700 1 --2- 172.21.15.1:0/2866199666 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555712b88470 0x555712b8c370 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f2088096770 tx=0x7f2088099940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6374e4b79111304 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:33.884 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.874+0000 7f2095ffb700 1 -- 172.21.15.1:0/2866199666 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20880a2150 con 0x555712b88470 2023-11-16T16:49:33.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.874+0000 7f2095ffb700 1 -- 172.21.15.1:0/2866199666 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2088007d70 con 0x555712b88470 2023-11-16T16:49:33.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.874+0000 7f2095ffb700 1 -- 172.21.15.1:0/2866199666 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2088099f60 con 0x555712b88470 2023-11-16T16:49:33.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.874+0000 7f20b1cdeb80 1 -- 172.21.15.1:0/2866199666 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555712b88470 msgr2=0x555712b8c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.874+0000 7f20b1cdeb80 1 --2- 172.21.15.1:0/2866199666 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555712b88470 0x555712b8c370 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f2088096770 tx=0x7f2088099940 comp rx=0 tx=0).stop 2023-11-16T16:49:33.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.874+0000 7f20b1cdeb80 1 -- 172.21.15.1:0/2866199666 shutdown_connections 2023-11-16T16:49:33.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.874+0000 7f20b1cdeb80 1 --2- 172.21.15.1:0/2866199666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555712b8c8b0 0x555712b8cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.874+0000 7f20b1cdeb80 1 --2- 172.21.15.1:0/2866199666 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555712b87b60 0x555712b87f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.885 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.874+0000 7f20b1cdeb80 1 --2- 172.21.15.1:0/2866199666 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555712b88470 0x555712b8c370 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.874+0000 7f20b1cdeb80 1 -- 172.21.15.1:0/2866199666 >> 172.21.15.1:0/2866199666 conn(0x555712b7d360 msgr2=0x555712b82b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:33.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.874+0000 7f20b1cdeb80 1 -- 172.21.15.1:0/2866199666 shutdown_connections 2023-11-16T16:49:33.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.874+0000 7f20b1cdeb80 1 -- 172.21.15.1:0/2866199666 wait complete. 2023-11-16T16:49:33.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.875+0000 7f20b1cdeb80 1 -- start start 2023-11-16T16:49:33.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.875+0000 7f20b1cdeb80 1 Processor -- start 2023-11-16T16:49:33.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.875+0000 7f20b1cdeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555712b87b60 0x555712b91360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.875+0000 7f20b1cdeb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555712b88470 0x555712b918a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.886 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.875+0000 7f20b1cdeb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555712b8c8b0 0x555712b91de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.875+0000 7f20b1cdeb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x555712a7a8e0 con 0x555712b88470 2023-11-16T16:49:33.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.875+0000 7f20b1cdeb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555712a7a580 con 0x555712b8c8b0 2023-11-16T16:49:33.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.875+0000 7f20b1cdeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55571295c920 con 0x555712b87b60 2023-11-16T16:49:33.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.875+0000 7f20b1cdeb80 10 client.0 _ll_register_callbacks cb 0x555712a98780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:49:33.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.875+0000 7f209effd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555712b88470 0x555712b918a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.875+0000 7f209effd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555712b88470 0x555712b918a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47336/0 (socket says 172.21.15.1:47336) 2023-11-16T16:49:33.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.875+0000 7f209effd700 1 -- 172.21.15.1:0/3799213803 learned_addr learned my addr 172.21.15.1:0/3799213803 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:49:33.887 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.875+0000 7f209f7fe700 1 --2- 172.21.15.1:0/3799213803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555712b87b60 0x555712b91360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.876+0000 7f209ffff700 1 --2- 172.21.15.1:0/3799213803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555712b8c8b0 0x555712b91de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.876+0000 7f209effd700 1 -- 172.21.15.1:0/3799213803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555712b87b60 msgr2=0x555712b91360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:33.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.876+0000 7f209effd700 1 --2- 172.21.15.1:0/3799213803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555712b87b60 0x555712b91360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.876+0000 7f209effd700 1 -- 172.21.15.1:0/3799213803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555712b8c8b0 msgr2=0x555712b91de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:33.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.876+0000 7f209effd700 1 --2- 172.21.15.1:0/3799213803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555712b8c8b0 0x555712b91de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.876+0000 7f209effd700 1 -- 172.21.15.1:0/3799213803 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a010f080 con 0x555712b88470 2023-11-16T16:49:33.888 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.876+0000 7f209effd700 1 --2- 172.21.15.1:0/3799213803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555712b88470 0x555712b918a0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f2088099a00 tx=0x7f208800f930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a83b4822f70f7cdf server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:33.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.876+0000 7f207effd700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:49:33.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.876+0000 7f207effd700 1 -- 172.21.15.1:0/3799213803 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20880980a0 con 0x555712b88470 2023-11-16T16:49:33.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.876+0000 7f207effd700 1 -- 172.21.15.1:0/3799213803 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f2088099400 con 0x555712b88470 2023-11-16T16:49:33.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.876+0000 7f20b1cdeb80 1 -- 172.21.15.1:0/3799213803 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555712b7ab20 con 0x555712b88470 2023-11-16T16:49:33.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.876+0000 7f207effd700 1 -- 172.21.15.1:0/3799213803 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2088099f60 con 0x555712b88470 2023-11-16T16:49:33.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.876+0000 7f20b1cdeb80 10 client.17058 fetch_fsmap learned FSMap version 33 2023-11-16T16:49:33.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.876+0000 7f20b1cdeb80 1 -- 172.21.15.1:0/3799213803 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x555712a9f190 con 0x555712b88470 2023-11-16T16:49:33.889 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.876+0000 7f207effd700 1 -- 172.21.15.1:0/3799213803 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f208807c020 con 0x555712b88470 2023-11-16T16:49:33.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.876+0000 7f207effd700 1 -- 172.21.15.1:0/3799213803 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 33) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f20880a1d50 con 0x555712b88470 2023-11-16T16:49:33.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.877+0000 7f20b1cdeb80 10 client.17058 fetch_fsmap finished waiting for FSMap version 33 2023-11-16T16:49:33.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.877+0000 7f20b1cdeb80 10 client.17058 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-11-16T16:49:33.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.877+0000 7f20b1cdeb80 20 client.17058 populate_metadata read hostname 'smithi001' 2023-11-16T16:49:33.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.877+0000 7f20b1cdeb80 1 --2- 172.21.15.1:0/3799213803 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x555712ba1440 0x555712ba3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.877+0000 7f20b1cdeb80 4 client.17058 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:49:33.890 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.877+0000 7f20b1cdeb80 1 -- 172.21.15.1:0/3799213803 --> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555712a9b0a0 con 0x555712ba1440 2023-11-16T16:49:33.891 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.877+0000 7f209f7fe700 1 --2- 172.21.15.1:0/3799213803 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x555712ba1440 0x555712ba3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.891 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.877+0000 7f207effd700 10 client.17058 ms_handle_connect on v2:172.21.15.53:6838/1044753024 2023-11-16T16:49:33.891 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.877+0000 7f209f7fe700 1 --2- 172.21.15.1:0/3799213803 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x555712ba1440 0x555712ba3850 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=106b982ecdfa5d15 server_cookie=c39f217bfa257afe in_seq=0 out_seq=0 2023-11-16T16:49:33.891 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.878+0000 7f207effd700 1 -- 172.21.15.1:0/3799213803 <== mds.1 v2:172.21.15.53:6838/1044753024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555712a9b0a0 con 0x555712ba1440 2023-11-16T16:49:33.891 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.878+0000 7f207effd700 10 client.17058 handle_command_reply: tid=0 2023-11-16T16:49:33.891 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.878+0000 7f20b1cdeb80 1 client.17058 shutdown 2023-11-16T16:49:33.891 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.878+0000 7f20b1cdeb80 1 -- 172.21.15.1:0/3799213803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555712b88470 msgr2=0x555712b918a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.891 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.878+0000 7f20b1cdeb80 1 --2- 172.21.15.1:0/3799213803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555712b88470 0x555712b918a0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f2088099a00 tx=0x7f208800f930 comp rx=0 tx=0).stop 2023-11-16T16:49:33.892 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.878+0000 7f20b1cdeb80 1 -- 172.21.15.1:0/3799213803 shutdown_connections 2023-11-16T16:49:33.892 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.878+0000 7f20b1cdeb80 1 --2- 172.21.15.1:0/3799213803 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x555712ba1440 0x555712ba3850 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.892 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.878+0000 7f20b1cdeb80 1 --2- 172.21.15.1:0/3799213803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555712b8c8b0 0x555712b91de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.892 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.878+0000 7f207effd700 0 client.17058 ms_handle_reset on v2:172.21.15.53:6838/1044753024 2023-11-16T16:49:33.892 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.878+0000 7f20b1cdeb80 1 --2- 172.21.15.1:0/3799213803 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555712b87b60 0x555712b91360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.892 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.878+0000 7f20b1cdeb80 1 --2- 172.21.15.1:0/3799213803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555712b88470 0x555712b918a0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.892 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.878+0000 7f20b1cdeb80 1 -- 172.21.15.1:0/3799213803 >> 172.21.15.1:0/3799213803 conn(0x555712b7d360 msgr2=0x555712b81de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:33.893 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.879+0000 7f20b1cdeb80 1 -- 172.21.15.1:0/3799213803 shutdown_connections 2023-11-16T16:49:33.893 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.879+0000 7f20b1cdeb80 1 -- 172.21.15.1:0/3799213803 wait complete. 2023-11-16T16:49:33.893 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.879+0000 7f20b1cdeb80 20 client.17058 trim_cache size 0 max 16384 2023-11-16T16:49:33.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.892+0000 7f20b1cdeb80 1 Processor -- start 2023-11-16T16:49:33.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.892+0000 7f20b1cdeb80 1 -- start start 2023-11-16T16:49:33.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.892+0000 7f20b1cdeb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555712b87b60 0x555712b8a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.892+0000 7f20b1cdeb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555712b88470 0x555712b8aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.892+0000 7f20b1cdeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555712b8c8b0 0x555712b82560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.892+0000 7f20b1cdeb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x555712b943f0 con 0x555712b88470 2023-11-16T16:49:33.898 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.892+0000 7f20b1cdeb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555712b88bc0 con 0x555712b87b60 2023-11-16T16:49:33.899 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.892+0000 7f20b1cdeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555712b7cee0 con 0x555712b8c8b0 2023-11-16T16:49:33.899 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.892+0000 7f209effd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555712b88470 0x555712b8aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.899 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.892+0000 7f209effd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555712b88470 0x555712b8aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47344/0 (socket says 172.21.15.1:47344) 2023-11-16T16:49:33.899 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.892+0000 7f209effd700 1 -- 172.21.15.1:0/2443362773 learned_addr learned my addr 172.21.15.1:0/2443362773 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:49:33.899 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.892+0000 7f209ffff700 1 --2- 172.21.15.1:0/2443362773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555712b8c8b0 0x555712b82560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.899 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.892+0000 7f209f7fe700 1 --2- 172.21.15.1:0/2443362773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555712b87b60 0x555712b8a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.899 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.892+0000 7f209effd700 1 -- 172.21.15.1:0/2443362773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555712b8c8b0 msgr2=0x555712b82560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:33.899 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.892+0000 7f209effd700 1 --2- 172.21.15.1:0/2443362773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555712b8c8b0 0x555712b82560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.892+0000 7f209effd700 1 -- 172.21.15.1:0/2443362773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555712b87b60 msgr2=0x555712b8a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:33.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.892+0000 7f209effd700 1 --2- 172.21.15.1:0/2443362773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555712b87b60 0x555712b8a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.892+0000 7f209effd700 1 -- 172.21.15.1:0/2443362773 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555712b8cf50 con 0x555712b88470 2023-11-16T16:49:33.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.892+0000 7f209f7fe700 1 --2- 172.21.15.1:0/2443362773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555712b87b60 0x555712b8a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:49:33.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.893+0000 7f209effd700 1 --2- 172.21.15.1:0/2443362773 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555712b88470 0x555712b8aeb0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f2088098050 tx=0x7f2088000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43818e91cf68f666 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:33.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.893+0000 7f20967fc700 1 -- 172.21.15.1:0/2443362773 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208800f9a0 con 0x555712b88470 2023-11-16T16:49:33.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.893+0000 7f20967fc700 1 -- 172.21.15.1:0/2443362773 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f208807c030 con 0x555712b88470 2023-11-16T16:49:33.900 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.893+0000 7f20967fc700 1 -- 172.21.15.1:0/2443362773 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20880caad0 con 0x555712b88470 2023-11-16T16:49:33.901 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.893+0000 7f20b1cdeb80 1 -- 172.21.15.1:0/2443362773 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555712b88470 msgr2=0x555712b8aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.901 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.893+0000 7f20b1cdeb80 1 --2- 172.21.15.1:0/2443362773 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555712b88470 0x555712b8aeb0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f2088098050 tx=0x7f2088000bc0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.901 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.893+0000 7f20b1cdeb80 1 -- 172.21.15.1:0/2443362773 shutdown_connections 2023-11-16T16:49:33.901 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.893+0000 7f20b1cdeb80 1 --2- 172.21.15.1:0/2443362773 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555712b8c8b0 0x555712b82560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.901 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.893+0000 7f20b1cdeb80 1 --2- 172.21.15.1:0/2443362773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555712b87b60 0x555712b8a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.901 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.893+0000 7f20b1cdeb80 1 --2- 172.21.15.1:0/2443362773 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555712b88470 0x555712b8aeb0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.901 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.893+0000 7f20b1cdeb80 1 -- 172.21.15.1:0/2443362773 >> 172.21.15.1:0/2443362773 conn(0x555712b7d360 msgr2=0x555712b9dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:33.901 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.893+0000 7f20b1cdeb80 1 -- 172.21.15.1:0/2443362773 shutdown_connections 2023-11-16T16:49:33.902 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.894+0000 7f20b1cdeb80 1 -- 172.21.15.1:0/2443362773 wait complete. 2023-11-16T16:49:33.902 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.894+0000 7f20b1cdeb80 1 -- start start 2023-11-16T16:49:33.902 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.894+0000 7f20b1cdeb80 1 Processor -- start 2023-11-16T16:49:33.902 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.894+0000 7f20b1cdeb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555712b87b60 0x555712b98ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.902 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.894+0000 7f20b1cdeb80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555712b88470 0x555712b96990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.902 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.894+0000 7f20b1cdeb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555712b8c8b0 0x555712b97080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.902 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.894+0000 7f20b1cdeb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x555712b97750 con 0x555712b87b60 2023-11-16T16:49:33.902 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.894+0000 7f20b1cdeb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555712bf42c0 con 0x555712b8c8b0 2023-11-16T16:49:33.903 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.894+0000 7f20b1cdeb80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x555712bf4400 con 0x555712b88470 2023-11-16T16:49:33.903 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.894+0000 7f20b1cdeb80 10 client.0 _ll_register_callbacks cb 0x7f2088099840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:49:33.903 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.894+0000 7f209f7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555712b87b60 0x555712b98ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.903 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.894+0000 7f209f7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555712b87b60 0x555712b98ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47348/0 (socket says 172.21.15.1:47348) 2023-11-16T16:49:33.903 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.894+0000 7f209f7fe700 1 -- 172.21.15.1:0/1517213358 learned_addr learned my addr 172.21.15.1:0/1517213358 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:49:33.903 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.894+0000 7f209ffff700 1 --2- 172.21.15.1:0/1517213358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555712b8c8b0 0x555712b97080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.903 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.895+0000 7f209effd700 1 --2- 172.21.15.1:0/1517213358 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555712b88470 0x555712b96990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.895+0000 7f209f7fe700 1 -- 172.21.15.1:0/1517213358 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555712b88470 msgr2=0x555712b96990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:33.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.895+0000 7f209f7fe700 1 --2- 172.21.15.1:0/1517213358 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555712b88470 0x555712b96990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.895+0000 7f209f7fe700 1 -- 172.21.15.1:0/1517213358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555712b8c8b0 msgr2=0x555712b97080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:33.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.895+0000 7f209f7fe700 1 --2- 172.21.15.1:0/1517213358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555712b8c8b0 0x555712b97080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.895+0000 7f209f7fe700 1 -- 172.21.15.1:0/1517213358 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555712a9f190 con 0x555712b87b60 2023-11-16T16:49:33.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.895+0000 7f209f7fe700 1 --2- 172.21.15.1:0/1517213358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555712b87b60 0x555712b98ba0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f2090000c00 tx=0x7f2090009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a5cc280d658fb74 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:33.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.895+0000 7f207effd700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:49:33.904 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.895+0000 7f207effd700 1 -- 172.21.15.1:0/1517213358 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2090033a60 con 0x555712b87b60 2023-11-16T16:49:33.905 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.895+0000 7f20b1cdeb80 1 -- 172.21.15.1:0/1517213358 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2088007d70 con 0x555712b87b60 2023-11-16T16:49:33.905 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.895+0000 7f207effd700 1 -- 172.21.15.1:0/1517213358 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f2090033bc0 con 0x555712b87b60 2023-11-16T16:49:33.905 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.895+0000 7f207effd700 1 -- 172.21.15.1:0/1517213358 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2090033d80 con 0x555712b87b60 2023-11-16T16:49:33.905 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.895+0000 7f20b1cdeb80 10 client.17064 fetch_fsmap learned FSMap version 33 2023-11-16T16:49:33.905 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.895+0000 7f20b1cdeb80 1 -- 172.21.15.1:0/1517213358 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x555712906de0 con 0x555712b87b60 2023-11-16T16:49:33.905 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.895+0000 7f207effd700 1 -- 172.21.15.1:0/1517213358 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2090048080 con 0x555712b87b60 2023-11-16T16:49:33.905 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.895+0000 7f207effd700 1 -- 172.21.15.1:0/1517213358 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 33) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2090003b90 con 0x555712b87b60 2023-11-16T16:49:33.905 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.896+0000 7f20b1cdeb80 10 client.17064 fetch_fsmap finished waiting for FSMap version 33 2023-11-16T16:49:33.906 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.896+0000 7f20b1cdeb80 10 client.17064 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-11-16T16:49:33.906 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.896+0000 7f20b1cdeb80 20 client.17064 populate_metadata read hostname 'smithi001' 2023-11-16T16:49:33.906 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.896+0000 7f20b1cdeb80 1 --2- 172.21.15.1:0/1517213358 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x555712bf5b00 0x555712bf7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:33.906 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.896+0000 7f20b1cdeb80 4 client.17064 mds_command: new command op to 24493 tid=0[{"prefix": "damage ls"}] 2023-11-16T16:49:33.906 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.896+0000 7f20b1cdeb80 1 -- 172.21.15.1:0/1517213358 --> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555712aa7660 con 0x555712bf5b00 2023-11-16T16:49:33.906 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.896+0000 7f209effd700 1 --2- 172.21.15.1:0/1517213358 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x555712bf5b00 0x555712bf7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:33.907 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.897+0000 7f209effd700 1 --2- 172.21.15.1:0/1517213358 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x555712bf5b00 0x555712bf7f10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=730f7e55b022037b server_cookie=a36d212115822b5d in_seq=0 out_seq=0 2023-11-16T16:49:33.908 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.897+0000 7f207effd700 10 client.17064 ms_handle_connect on v2:172.21.15.53:6838/1044753024 2023-11-16T16:49:33.908 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.897+0000 7f207effd700 1 -- 172.21.15.1:0/1517213358 <== mds.1 v2:172.21.15.53:6838/1044753024 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555712aa7660 con 0x555712bf5b00 2023-11-16T16:49:33.908 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.897+0000 7f207effd700 10 client.17064 handle_command_reply: tid=0 2023-11-16T16:49:33.908 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.897+0000 7f20b1cdeb80 1 client.17064 shutdown 2023-11-16T16:49:33.908 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.897+0000 7f20b1cdeb80 1 -- 172.21.15.1:0/1517213358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555712b87b60 msgr2=0x555712b98ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.908 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.897+0000 7f20b1cdeb80 1 --2- 172.21.15.1:0/1517213358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555712b87b60 0x555712b98ba0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f2090000c00 tx=0x7f2090009cd0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.908 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.897+0000 7f20b1cdeb80 1 -- 172.21.15.1:0/1517213358 shutdown_connections 2023-11-16T16:49:33.909 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.897+0000 7f20b1cdeb80 1 --2- 172.21.15.1:0/1517213358 >> [v2:172.21.15.53:6838/1044753024,v1:172.21.15.53:6839/1044753024] conn(0x555712bf5b00 0x555712bf7f10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.909 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.897+0000 7f20b1cdeb80 1 --2- 172.21.15.1:0/1517213358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555712b8c8b0 0x555712b97080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.909 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.897+0000 7f20b1cdeb80 1 --2- 172.21.15.1:0/1517213358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555712b87b60 0x555712b98ba0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.909 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.897+0000 7f207effd700 0 client.17064 ms_handle_reset on v2:172.21.15.53:6838/1044753024 2023-11-16T16:49:33.909 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.897+0000 7f20b1cdeb80 1 --2- 172.21.15.1:0/1517213358 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x555712b88470 0x555712b96990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.909 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.897+0000 7f20b1cdeb80 1 -- 172.21.15.1:0/1517213358 >> 172.21.15.1:0/1517213358 conn(0x555712b7d360 msgr2=0x555712b8a970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:33.909 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.897+0000 7f20b1cdeb80 1 -- 172.21.15.1:0/1517213358 shutdown_connections 2023-11-16T16:49:33.910 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.897+0000 7f20b1cdeb80 1 -- 172.21.15.1:0/1517213358 wait complete. 2023-11-16T16:49:33.910 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.897+0000 7f20b1cdeb80 20 client.17064 trim_cache size 0 max 16384 2023-11-16T16:49:33.910 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2023-11-16T16:49:33.911 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.898+0000 7f20a5b52700 1 -- 172.21.15.1:0/1375358774 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f208c0a12c0 msgr2=0x7f208c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.911 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.898+0000 7f20a5b52700 1 --2- 172.21.15.1:0/1375358774 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f208c0a12c0 0x7f208c0a3770 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2090009990 tx=0x7f2090003680 comp rx=0 tx=0).stop 2023-11-16T16:49:33.911 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.898+0000 7f20a5b52700 1 -- 172.21.15.1:0/1375358774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20a00ffdb0 msgr2=0x7f20a0197670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:33.912 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.898+0000 7f20a5b52700 1 --2- 172.21.15.1:0/1375358774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20a00ffdb0 0x7f20a0197670 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f208800c9b0 tx=0x7f208800ccc0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.912 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.898+0000 7f20a5b52700 1 -- 172.21.15.1:0/1375358774 shutdown_connections 2023-11-16T16:49:33.912 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.898+0000 7f20a5b52700 1 --2- 172.21.15.1:0/1375358774 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f208c0a12c0 0x7f208c0a3770 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.912 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.898+0000 7f20a5b52700 1 --2- 172.21.15.1:0/1375358774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20a0104450 0x7f20a01a21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.912 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.898+0000 7f20a5b52700 1 --2- 172.21.15.1:0/1375358774 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f20a00ff410 0x7f20a0197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.912 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.898+0000 7f20a5b52700 1 --2- 172.21.15.1:0/1375358774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20a00ffdb0 0x7f20a0197670 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:33.912 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.898+0000 7f20a5b52700 1 -- 172.21.15.1:0/1375358774 >> 172.21.15.1:0/1375358774 conn(0x7f20a0076370 msgr2=0x7f20a00fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:33.912 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.898+0000 7f20a5b52700 1 -- 172.21.15.1:0/1375358774 shutdown_connections 2023-11-16T16:49:33.913 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:33.898+0000 7f20a5b52700 1 -- 172.21.15.1:0/1375358774 wait complete. 2023-11-16T16:49:33.913 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:33 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/2784360745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:49:33.914 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-16T16:49:34.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:33 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/2784360745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:49:34.044 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.040+0000 7f730b729700 1 -- 172.21.15.1:0/441097419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7304104140 msgr2=0x7f7304104510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:34.044 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.040+0000 7f730b729700 1 --2- 172.21.15.1:0/441097419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7304104140 0x7f7304104510 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f72f40099c0 tx=0x7f72f4009cd0 comp rx=0 tx=0).stop 2023-11-16T16:49:34.044 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.040+0000 7f730b729700 1 -- 172.21.15.1:0/441097419 shutdown_connections 2023-11-16T16:49:34.044 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.040+0000 7f730b729700 1 --2- 172.21.15.1:0/441097419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73041002a0 0x7f730410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:34.044 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.040+0000 7f730b729700 1 --2- 172.21.15.1:0/441097419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7304104140 0x7f7304104510 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:34.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.040+0000 7f730b729700 1 --2- 172.21.15.1:0/441097419 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73040ff8f0 0x7f73040ffd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:34.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.040+0000 7f730b729700 1 -- 172.21.15.1:0/441097419 >> 172.21.15.1:0/441097419 conn(0x7f7304075880 msgr2=0x7f7304075c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:34.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.040+0000 7f730b729700 1 -- 172.21.15.1:0/441097419 shutdown_connections 2023-11-16T16:49:34.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.040+0000 7f730b729700 1 -- 172.21.15.1:0/441097419 wait complete. 2023-11-16T16:49:34.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.041+0000 7f730b729700 1 Processor -- start 2023-11-16T16:49:34.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.041+0000 7f730b729700 1 -- start start 2023-11-16T16:49:34.045 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.041+0000 7f730b729700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73040ff8f0 0x7f73041970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:34.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.041+0000 7f730b729700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73041002a0 0x7f7304197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:34.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.041+0000 7f730b729700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7304104140 0x7f73041a2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:34.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.041+0000 7f730b729700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7304197f40 con 0x7f73041002a0 2023-11-16T16:49:34.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.041+0000 7f730b729700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73041a2680 con 0x7f7304104140 2023-11-16T16:49:34.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.041+0000 7f730b729700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73041a2840 con 0x7f73040ff8f0 2023-11-16T16:49:34.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.041+0000 7f7308cc4700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73041002a0 0x7f7304197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:34.046 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.041+0000 7f7308cc4700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73041002a0 0x7f7304197600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47364/0 (socket says 172.21.15.1:47364) 2023-11-16T16:49:34.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.041+0000 7f7308cc4700 1 -- 172.21.15.1:0/3966057308 learned_addr learned my addr 172.21.15.1:0/3966057308 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:49:34.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.041+0000 7f73094c5700 1 --2- 172.21.15.1:0/3966057308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73040ff8f0 0x7f73041970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:34.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.041+0000 7f7309cc6700 1 --2- 172.21.15.1:0/3966057308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7304104140 0x7f73041a2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:34.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.041+0000 7f7308cc4700 1 -- 172.21.15.1:0/3966057308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73040ff8f0 msgr2=0x7f73041970c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:34.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.041+0000 7f7308cc4700 1 --2- 172.21.15.1:0/3966057308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73040ff8f0 0x7f73041970c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:34.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.041+0000 7f7308cc4700 1 -- 172.21.15.1:0/3966057308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7304104140 msgr2=0x7f73041a2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:34.047 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.041+0000 7f7308cc4700 1 --2- 172.21.15.1:0/3966057308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7304104140 0x7f73041a2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:34.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.041+0000 7f7308cc4700 1 -- 172.21.15.1:0/3966057308 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f40095f0 con 0x7f73041002a0 2023-11-16T16:49:34.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.041+0000 7f7308cc4700 1 --2- 172.21.15.1:0/3966057308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73041002a0 0x7f7304197600 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f730000e9a0 tx=0x7f730000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:34.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.042+0000 7f72fa7fc700 1 -- 172.21.15.1:0/3966057308 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7300009790 con 0x7f73041002a0 2023-11-16T16:49:34.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.042+0000 7f730b729700 1 -- 172.21.15.1:0/3966057308 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f730410f1b0 con 0x7f73041002a0 2023-11-16T16:49:34.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.042+0000 7f72fa7fc700 1 -- 172.21.15.1:0/3966057308 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f730000cbd0 con 0x7f73041002a0 2023-11-16T16:49:34.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.042+0000 7f72fa7fc700 1 -- 172.21.15.1:0/3966057308 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7300018680 con 0x7f73041002a0 2023-11-16T16:49:34.048 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.042+0000 7f730b729700 1 -- 172.21.15.1:0/3966057308 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f730410f020 con 0x7f73041002a0 2023-11-16T16:49:34.055 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.044+0000 7f72fa7fc700 1 -- 172.21.15.1:0/3966057308 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f730000cd40 con 0x7f73041002a0 2023-11-16T16:49:34.055 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.044+0000 7f72fa7fc700 1 --2- 172.21.15.1:0/3966057308 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f72f00a12c0 0x7f72f00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:34.055 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.044+0000 7f72fa7fc700 1 -- 172.21.15.1:0/3966057308 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f7300014070 con 0x7f73041002a0 2023-11-16T16:49:34.055 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.044+0000 7f73094c5700 1 --2- 172.21.15.1:0/3966057308 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f72f00a12c0 0x7f72f00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:34.055 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.045+0000 7f73094c5700 1 --2- 172.21.15.1:0/3966057308 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f72f00a12c0 0x7f72f00a3770 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f72f4009990 tx=0x7f72f401a040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:34.055 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.045+0000 7f73178b5b80 1 Processor -- start 2023-11-16T16:49:34.056 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.045+0000 7f73178b5b80 1 -- start start 2023-11-16T16:49:34.056 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.046+0000 7f73178b5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca5b101f70 0x55ca5b102340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:34.056 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.046+0000 7f73178b5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca5b102880 0x55ca5b106780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:34.056 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.046+0000 7f73178b5b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca5b106cc0 0x55ca5b107110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:34.056 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.046+0000 7f73178b5b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca5b003060 con 0x55ca5b106cc0 2023-11-16T16:49:34.056 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.046+0000 7f73178b5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca5b0558a0 con 0x55ca5b102880 2023-11-16T16:49:34.056 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.046+0000 7f73178b5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca5b04ed30 con 0x55ca5b101f70 2023-11-16T16:49:34.057 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.046+0000 7f7309cc6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca5b106cc0 0x55ca5b107110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:34.057 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.046+0000 7f7309cc6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca5b106cc0 0x55ca5b107110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47374/0 (socket says 172.21.15.1:47374) 2023-11-16T16:49:34.057 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.046+0000 7f7309cc6700 1 -- 172.21.15.1:0/215146128 learned_addr learned my addr 172.21.15.1:0/215146128 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:49:34.057 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.046+0000 7f7308cc4700 1 --2- 172.21.15.1:0/215146128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca5b102880 0x55ca5b106780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:34.057 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.046+0000 7f73094c5700 1 --2- 172.21.15.1:0/215146128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca5b101f70 0x55ca5b102340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:34.057 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.046+0000 7f7309cc6700 1 -- 172.21.15.1:0/215146128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca5b101f70 msgr2=0x55ca5b102340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:34.057 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.046+0000 7f7309cc6700 1 --2- 172.21.15.1:0/215146128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca5b101f70 0x55ca5b102340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:34.057 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.046+0000 7f7309cc6700 1 -- 172.21.15.1:0/215146128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca5b102880 msgr2=0x55ca5b106780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:34.058 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.046+0000 7f7309cc6700 1 --2- 172.21.15.1:0/215146128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca5b102880 0x55ca5b106780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:34.058 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.046+0000 7f7309cc6700 1 -- 172.21.15.1:0/215146128 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca5aed0630 con 0x55ca5b106cc0 2023-11-16T16:49:34.058 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.046+0000 7f7308cc4700 1 --2- 172.21.15.1:0/215146128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca5b102880 0x55ca5b106780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:49:34.058 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.046+0000 7f7309cc6700 1 --2- 172.21.15.1:0/215146128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca5b106cc0 0x55ca5b107110 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f72fc009d80 tx=0x7f72fc0077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a0b36c50451f39 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:34.058 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.046+0000 7f72ef7fe700 1 -- 172.21.15.1:0/215146128 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72fc014610 con 0x55ca5b106cc0 2023-11-16T16:49:34.058 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.046+0000 7f72ef7fe700 1 -- 172.21.15.1:0/215146128 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72fc00f050 con 0x55ca5b106cc0 2023-11-16T16:49:34.058 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.046+0000 7f72ef7fe700 1 -- 172.21.15.1:0/215146128 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72fc013960 con 0x55ca5b106cc0 2023-11-16T16:49:34.059 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.047+0000 7f73178b5b80 1 -- 172.21.15.1:0/215146128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca5b106cc0 msgr2=0x55ca5b107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:34.059 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.047+0000 7f73178b5b80 1 --2- 172.21.15.1:0/215146128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca5b106cc0 0x55ca5b107110 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f72fc009d80 tx=0x7f72fc0077e0 comp rx=0 tx=0).stop 2023-11-16T16:49:34.059 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.047+0000 7f73178b5b80 1 -- 172.21.15.1:0/215146128 shutdown_connections 2023-11-16T16:49:34.059 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.047+0000 7f73178b5b80 1 --2- 172.21.15.1:0/215146128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca5b106cc0 0x55ca5b107110 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:34.059 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.047+0000 7f73178b5b80 1 --2- 172.21.15.1:0/215146128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca5b101f70 0x55ca5b102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:34.059 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.047+0000 7f73178b5b80 1 --2- 172.21.15.1:0/215146128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca5b102880 0x55ca5b106780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:34.059 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.047+0000 7f73178b5b80 1 -- 172.21.15.1:0/215146128 >> 172.21.15.1:0/215146128 conn(0x55ca5b0f3360 msgr2=0x55ca5b0f8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:34.060 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.047+0000 7f73178b5b80 1 -- 172.21.15.1:0/215146128 shutdown_connections 2023-11-16T16:49:34.060 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.047+0000 7f73178b5b80 1 -- 172.21.15.1:0/215146128 wait complete. 2023-11-16T16:49:34.060 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.048+0000 7f73178b5b80 1 -- start start 2023-11-16T16:49:34.060 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.048+0000 7f73178b5b80 1 Processor -- start 2023-11-16T16:49:34.060 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.048+0000 7f73178b5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca5b101f70 0x55ca5b10e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:34.060 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.048+0000 7f73178b5b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca5b102880 0x55ca5b10b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:34.060 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.048+0000 7f73178b5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca5b106cc0 0x55ca5b10bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:34.060 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.048+0000 7f73178b5b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca5aff08e0 con 0x55ca5b102880 2023-11-16T16:49:34.061 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.048+0000 7f73178b5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca5aff0580 con 0x55ca5b101f70 2023-11-16T16:49:34.061 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.048+0000 7f73178b5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca5aed2920 con 0x55ca5b106cc0 2023-11-16T16:49:34.061 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.048+0000 7f73178b5b80 10 client.0 _ll_register_callbacks cb 0x55ca5b00e780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:49:34.061 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.048+0000 7f7308cc4700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca5b102880 0x55ca5b10b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:34.061 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.048+0000 7f7308cc4700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca5b102880 0x55ca5b10b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47378/0 (socket says 172.21.15.1:47378) 2023-11-16T16:49:34.061 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.048+0000 7f7308cc4700 1 -- 172.21.15.1:0/1788471482 learned_addr learned my addr 172.21.15.1:0/1788471482 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:49:34.061 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.048+0000 7f73094c5700 1 --2- 172.21.15.1:0/1788471482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca5b101f70 0x55ca5b10e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:34.062 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.048+0000 7f7309cc6700 1 --2- 172.21.15.1:0/1788471482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca5b106cc0 0x55ca5b10bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:34.062 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.048+0000 7f7308cc4700 1 -- 172.21.15.1:0/1788471482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca5b106cc0 msgr2=0x55ca5b10bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:34.062 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.048+0000 7f7308cc4700 1 --2- 172.21.15.1:0/1788471482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca5b106cc0 0x55ca5b10bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:34.062 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.048+0000 7f7308cc4700 1 -- 172.21.15.1:0/1788471482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca5b101f70 msgr2=0x55ca5b10e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:34.062 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.048+0000 7f7308cc4700 1 --2- 172.21.15.1:0/1788471482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca5b101f70 0x55ca5b10e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:34.062 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.048+0000 7f7308cc4700 1 -- 172.21.15.1:0/1788471482 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72fc0099e0 con 0x55ca5b102880 2023-11-16T16:49:34.062 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.049+0000 7f7309cc6700 1 --2- 172.21.15.1:0/1788471482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca5b106cc0 0x55ca5b10bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:49:34.062 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.049+0000 7f7308cc4700 1 --2- 172.21.15.1:0/1788471482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca5b102880 0x55ca5b10b660 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f730009a100 tx=0x7f730009e2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be73b9b04583c23 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:34.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.049+0000 7f72ecff9700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:49:34.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.049+0000 7f72ecff9700 1 -- 172.21.15.1:0/1788471482 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73000b1070 con 0x55ca5b102880 2023-11-16T16:49:34.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.049+0000 7f72ecff9700 1 -- 172.21.15.1:0/1788471482 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f7300099870 con 0x55ca5b102880 2023-11-16T16:49:34.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.049+0000 7f73178b5b80 1 -- 172.21.15.1:0/1788471482 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ca5b0f0b20 con 0x55ca5b102880 2023-11-16T16:49:34.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.049+0000 7f72ecff9700 1 -- 172.21.15.1:0/1788471482 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73000a2130 con 0x55ca5b102880 2023-11-16T16:49:34.063 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.049+0000 7f73178b5b80 10 client.17076 fetch_fsmap learned FSMap version 33 2023-11-16T16:49:34.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.049+0000 7f73178b5b80 1 -- 172.21.15.1:0/1788471482 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55ca5b015190 con 0x55ca5b102880 2023-11-16T16:49:34.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.049+0000 7f72ecff9700 1 -- 172.21.15.1:0/1788471482 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73000c8080 con 0x55ca5b102880 2023-11-16T16:49:34.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.049+0000 7f72ecff9700 1 -- 172.21.15.1:0/1788471482 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 33) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f73000b8170 con 0x55ca5b102880 2023-11-16T16:49:34.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.049+0000 7f73178b5b80 10 client.17076 fetch_fsmap finished waiting for FSMap version 33 2023-11-16T16:49:34.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.049+0000 7f73178b5b80 10 client.17076 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:49:34.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.049+0000 7f73178b5b80 20 client.17076 populate_metadata read hostname 'smithi001' 2023-11-16T16:49:34.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.049+0000 7f73178b5b80 1 --2- 172.21.15.1:0/1788471482 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55ca5b11bbf0 0x55ca5b11e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:34.064 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.049+0000 7f73178b5b80 4 client.17076 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-16T16:49:34.065 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.049+0000 7f73178b5b80 1 -- 172.21.15.1:0/1788471482 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ca5b0110a0 con 0x55ca5b11bbf0 2023-11-16T16:49:34.065 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.050+0000 7f73094c5700 1 --2- 172.21.15.1:0/1788471482 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55ca5b11bbf0 0x55ca5b11e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:34.065 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.050+0000 7f73094c5700 1 --2- 172.21.15.1:0/1788471482 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55ca5b11bbf0 0x55ca5b11e000 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3fd06dde1ab1085 server_cookie=ffc38515fba292d8 in_seq=0 out_seq=0 2023-11-16T16:49:34.065 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.050+0000 7f72ecff9700 10 client.17076 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:49:34.065 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.051+0000 7f72ecff9700 1 -- 172.21.15.1:0/1788471482 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ca5b0110a0 con 0x55ca5b11bbf0 2023-11-16T16:49:34.065 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.051+0000 7f72ecff9700 10 client.17076 handle_command_reply: tid=0 2023-11-16T16:49:34.065 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.051+0000 7f73178b5b80 1 client.17076 shutdown 2023-11-16T16:49:34.066 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.051+0000 7f73178b5b80 1 -- 172.21.15.1:0/1788471482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca5b102880 msgr2=0x55ca5b10b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:34.066 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.051+0000 7f73178b5b80 1 --2- 172.21.15.1:0/1788471482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca5b102880 0x55ca5b10b660 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f730009a100 tx=0x7f730009e2d0 comp rx=0 tx=0).stop 2023-11-16T16:49:34.066 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.051+0000 7f73178b5b80 1 -- 172.21.15.1:0/1788471482 shutdown_connections 2023-11-16T16:49:34.066 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.051+0000 7f73178b5b80 1 --2- 172.21.15.1:0/1788471482 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55ca5b11bbf0 0x55ca5b11e000 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:34.066 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.051+0000 7f73178b5b80 1 --2- 172.21.15.1:0/1788471482 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca5b106cc0 0x55ca5b10bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:34.066 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.051+0000 7f72ecff9700 0 client.17076 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:49:34.066 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.051+0000 7f73178b5b80 1 --2- 172.21.15.1:0/1788471482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca5b101f70 0x55ca5b10e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:34.067 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.051+0000 7f73178b5b80 1 --2- 172.21.15.1:0/1788471482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca5b102880 0x55ca5b10b660 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:34.067 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.051+0000 7f73178b5b80 1 -- 172.21.15.1:0/1788471482 >> 172.21.15.1:0/1788471482 conn(0x55ca5b0f3360 msgr2=0x55ca5b108170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:34.067 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.051+0000 7f73178b5b80 1 -- 172.21.15.1:0/1788471482 shutdown_connections 2023-11-16T16:49:34.067 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.051+0000 7f73178b5b80 1 -- 172.21.15.1:0/1788471482 wait complete. 2023-11-16T16:49:34.067 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.051+0000 7f73178b5b80 20 client.17076 trim_cache size 0 max 16384 2023-11-16T16:49:34.073 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.064+0000 7f73178b5b80 1 Processor -- start 2023-11-16T16:49:34.073 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.064+0000 7f73178b5b80 1 -- start start 2023-11-16T16:49:34.073 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.065+0000 7f73178b5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca5b101f70 0x55ca5b112db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:34.073 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.065+0000 7f73178b5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca5b102880 0x55ca5b117f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:34.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.065+0000 7f73178b5b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca5b106cc0 0x55ca5b10df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:34.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.065+0000 7f73178b5b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca5b0fa270 con 0x55ca5b106cc0 2023-11-16T16:49:34.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.065+0000 7f73178b5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca5b103000 con 0x55ca5b101f70 2023-11-16T16:49:34.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.065+0000 7f73178b5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca5b109be0 con 0x55ca5b102880 2023-11-16T16:49:34.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.065+0000 7f7309cc6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca5b106cc0 0x55ca5b10df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:34.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.065+0000 7f7309cc6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca5b106cc0 0x55ca5b10df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47388/0 (socket says 172.21.15.1:47388) 2023-11-16T16:49:34.074 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.065+0000 7f7309cc6700 1 -- 172.21.15.1:0/1749692505 learned_addr learned my addr 172.21.15.1:0/1749692505 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:49:34.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.065+0000 7f73094c5700 1 --2- 172.21.15.1:0/1749692505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca5b101f70 0x55ca5b112db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:34.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.065+0000 7f7308cc4700 1 --2- 172.21.15.1:0/1749692505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca5b102880 0x55ca5b117f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:34.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.065+0000 7f7309cc6700 1 -- 172.21.15.1:0/1749692505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca5b102880 msgr2=0x55ca5b117f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:34.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.065+0000 7f7309cc6700 1 --2- 172.21.15.1:0/1749692505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca5b102880 0x55ca5b117f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:34.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.065+0000 7f7309cc6700 1 -- 172.21.15.1:0/1749692505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca5b101f70 msgr2=0x55ca5b112db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:34.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.065+0000 7f7309cc6700 1 --2- 172.21.15.1:0/1749692505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca5b101f70 0x55ca5b112db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:34.075 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.065+0000 7f7309cc6700 1 -- 172.21.15.1:0/1749692505 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca5b014ee0 con 0x55ca5b106cc0 2023-11-16T16:49:34.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.065+0000 7f7308cc4700 1 --2- 172.21.15.1:0/1749692505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca5b102880 0x55ca5b117f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:49:34.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.065+0000 7f7309cc6700 1 --2- 172.21.15.1:0/1749692505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca5b106cc0 0x55ca5b10df80 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f72fc009670 tx=0x7f72fc007f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9168e62ab64eeba server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:34.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.065+0000 7f72effff700 1 -- 172.21.15.1:0/1749692505 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72fc004c90 con 0x55ca5b106cc0 2023-11-16T16:49:34.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.065+0000 7f72effff700 1 -- 172.21.15.1:0/1749692505 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72fc004df0 con 0x55ca5b106cc0 2023-11-16T16:49:34.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.065+0000 7f72effff700 1 -- 172.21.15.1:0/1749692505 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72fc014410 con 0x55ca5b106cc0 2023-11-16T16:49:34.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.066+0000 7f73178b5b80 1 -- 172.21.15.1:0/1749692505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca5b106cc0 msgr2=0x55ca5b10df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:34.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.066+0000 7f73178b5b80 1 --2- 172.21.15.1:0/1749692505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca5b106cc0 0x55ca5b10df80 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f72fc009670 tx=0x7f72fc007f10 comp rx=0 tx=0).stop 2023-11-16T16:49:34.076 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.066+0000 7f73178b5b80 1 -- 172.21.15.1:0/1749692505 shutdown_connections 2023-11-16T16:49:34.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.066+0000 7f73178b5b80 1 --2- 172.21.15.1:0/1749692505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca5b106cc0 0x55ca5b10df80 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:34.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.066+0000 7f73178b5b80 1 --2- 172.21.15.1:0/1749692505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca5b101f70 0x55ca5b112db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:34.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.066+0000 7f73178b5b80 1 --2- 172.21.15.1:0/1749692505 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca5b102880 0x55ca5b117f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:34.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.066+0000 7f73178b5b80 1 -- 172.21.15.1:0/1749692505 >> 172.21.15.1:0/1749692505 conn(0x55ca5b0f3360 msgr2=0x55ca5b1090a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:34.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.066+0000 7f73178b5b80 1 -- 172.21.15.1:0/1749692505 shutdown_connections 2023-11-16T16:49:34.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.066+0000 7f73178b5b80 1 -- 172.21.15.1:0/1749692505 wait complete. 2023-11-16T16:49:34.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.066+0000 7f73178b5b80 1 -- start start 2023-11-16T16:49:34.077 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.067+0000 7f73178b5b80 1 Processor -- start 2023-11-16T16:49:34.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.067+0000 7f73178b5b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca5b101f70 0x55ca5b112750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:34.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.067+0000 7f73178b5b80 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca5b102880 0x55ca5b112c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:34.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.067+0000 7f73178b5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca5b106cc0 0x55ca5b110da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:34.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.067+0000 7f73178b5b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca5b1113b0 con 0x55ca5b101f70 2023-11-16T16:49:34.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.067+0000 7f73178b5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca5b111520 con 0x55ca5b106cc0 2023-11-16T16:49:34.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.067+0000 7f73178b5b80 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca5b1116e0 con 0x55ca5b102880 2023-11-16T16:49:34.078 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.067+0000 7f73178b5b80 10 client.0 _ll_register_callbacks cb 0x7f730009a170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-16T16:49:34.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.067+0000 7f73094c5700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca5b101f70 0x55ca5b112750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:34.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.067+0000 7f73094c5700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca5b101f70 0x55ca5b112750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47390/0 (socket says 172.21.15.1:47390) 2023-11-16T16:49:34.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.067+0000 7f73094c5700 1 -- 172.21.15.1:0/3015679667 learned_addr learned my addr 172.21.15.1:0/3015679667 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:49:34.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.067+0000 7f7309cc6700 1 --2- 172.21.15.1:0/3015679667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca5b106cc0 0x55ca5b110da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:34.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.067+0000 7f7308cc4700 1 --2- 172.21.15.1:0/3015679667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca5b102880 0x55ca5b112c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:34.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.068+0000 7f73094c5700 1 -- 172.21.15.1:0/3015679667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca5b102880 msgr2=0x55ca5b112c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:34.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.068+0000 7f73094c5700 1 --2- 172.21.15.1:0/3015679667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca5b102880 0x55ca5b112c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:34.079 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.068+0000 7f73094c5700 1 -- 172.21.15.1:0/3015679667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca5b106cc0 msgr2=0x55ca5b110da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-16T16:49:34.080 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.068+0000 7f73094c5700 1 --2- 172.21.15.1:0/3015679667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca5b106cc0 0x55ca5b110da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:34.080 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.068+0000 7f73094c5700 1 -- 172.21.15.1:0/3015679667 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca5aed0630 con 0x55ca5b101f70 2023-11-16T16:49:34.080 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.068+0000 7f7309cc6700 1 --2- 172.21.15.1:0/3015679667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca5b106cc0 0x55ca5b110da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:49:34.080 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.068+0000 7f73094c5700 1 --2- 172.21.15.1:0/3015679667 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca5b101f70 0x55ca5b112750 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f72f4005850 tx=0x7f72f4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74c60ab618bbfaa2 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:34.080 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.068+0000 7f72ecff9700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2023-11-16T16:49:34.080 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.068+0000 7f72ecff9700 1 -- 172.21.15.1:0/3015679667 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f401d040 con 0x55ca5b101f70 2023-11-16T16:49:34.080 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.068+0000 7f72ecff9700 1 -- 172.21.15.1:0/3015679667 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f72f400fe30 con 0x55ca5b101f70 2023-11-16T16:49:34.081 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.068+0000 7f73178b5b80 1 -- 172.21.15.1:0/3015679667 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f72fc00f050 con 0x55ca5b101f70 2023-11-16T16:49:34.081 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.068+0000 7f72ecff9700 1 -- 172.21.15.1:0/3015679667 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72f4033e60 con 0x55ca5b101f70 2023-11-16T16:49:34.081 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.068+0000 7f73178b5b80 10 client.17082 fetch_fsmap learned FSMap version 33 2023-11-16T16:49:34.081 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.068+0000 7f73178b5b80 1 -- 172.21.15.1:0/3015679667 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55ca5ae65c60 con 0x55ca5b101f70 2023-11-16T16:49:34.081 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.068+0000 7f72ecff9700 1 -- 172.21.15.1:0/3015679667 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72fc00f050 con 0x55ca5b101f70 2023-11-16T16:49:34.081 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.068+0000 7f72ecff9700 1 -- 172.21.15.1:0/3015679667 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 33) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f72f403c9c0 con 0x55ca5b101f70 2023-11-16T16:49:34.081 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.068+0000 7f73178b5b80 10 client.17082 fetch_fsmap finished waiting for FSMap version 33 2023-11-16T16:49:34.081 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.068+0000 7f73178b5b80 10 client.17082 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-16T16:49:34.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.068+0000 7f73178b5b80 20 client.17082 populate_metadata read hostname 'smithi001' 2023-11-16T16:49:34.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.068+0000 7f73178b5b80 1 --2- 172.21.15.1:0/3015679667 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55ca5b170c20 0x55ca5b173030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:34.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.068+0000 7f73178b5b80 4 client.17082 mds_command: new command op to 24521 tid=0[{"prefix": "damage ls"}] 2023-11-16T16:49:34.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.068+0000 7f73178b5b80 1 -- 172.21.15.1:0/3015679667 --> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ca5b01d660 con 0x55ca5b170c20 2023-11-16T16:49:34.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.069+0000 7f7308cc4700 1 --2- 172.21.15.1:0/3015679667 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55ca5b170c20 0x55ca5b173030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:34.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.069+0000 7f7308cc4700 1 --2- 172.21.15.1:0/3015679667 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55ca5b170c20 0x55ca5b173030 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50bec3cef558e042 server_cookie=e214750751d0230f in_seq=0 out_seq=0 2023-11-16T16:49:34.082 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.069+0000 7f72ecff9700 10 client.17082 ms_handle_connect on v2:172.21.15.26:6838/2192205949 2023-11-16T16:49:34.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.069+0000 7f72ecff9700 1 -- 172.21.15.1:0/3015679667 <== mds.0 v2:172.21.15.26:6838/2192205949 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ca5b01d660 con 0x55ca5b170c20 2023-11-16T16:49:34.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.069+0000 7f72ecff9700 10 client.17082 handle_command_reply: tid=0 2023-11-16T16:49:34.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.069+0000 7f73178b5b80 1 client.17082 shutdown 2023-11-16T16:49:34.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.069+0000 7f73178b5b80 1 -- 172.21.15.1:0/3015679667 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca5b101f70 msgr2=0x55ca5b112750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:34.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.069+0000 7f73178b5b80 1 --2- 172.21.15.1:0/3015679667 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca5b101f70 0x55ca5b112750 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f72f4005850 tx=0x7f72f4009310 comp rx=0 tx=0).stop 2023-11-16T16:49:34.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.069+0000 7f73178b5b80 1 -- 172.21.15.1:0/3015679667 shutdown_connections 2023-11-16T16:49:34.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.069+0000 7f73178b5b80 1 --2- 172.21.15.1:0/3015679667 >> [v2:172.21.15.26:6838/2192205949,v1:172.21.15.26:6839/2192205949] conn(0x55ca5b170c20 0x55ca5b173030 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:34.083 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.069+0000 7f73178b5b80 1 --2- 172.21.15.1:0/3015679667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca5b106cc0 0x55ca5b110da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:34.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.069+0000 7f72ecff9700 0 client.17082 ms_handle_reset on v2:172.21.15.26:6838/2192205949 2023-11-16T16:49:34.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.069+0000 7f73178b5b80 1 --2- 172.21.15.1:0/3015679667 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca5b101f70 0x55ca5b112750 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:34.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.069+0000 7f73178b5b80 1 --2- 172.21.15.1:0/3015679667 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x55ca5b102880 0x55ca5b112c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:34.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.069+0000 7f73178b5b80 1 -- 172.21.15.1:0/3015679667 >> 172.21.15.1:0/3015679667 conn(0x55ca5b0f3360 msgr2=0x55ca5b104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:34.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.070+0000 7f73178b5b80 1 -- 172.21.15.1:0/3015679667 shutdown_connections 2023-11-16T16:49:34.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.070+0000 7f73178b5b80 1 -- 172.21.15.1:0/3015679667 wait complete. 2023-11-16T16:49:34.084 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.070+0000 7f73178b5b80 20 client.17082 trim_cache size 0 max 16384 2023-11-16T16:49:34.085 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2023-11-16T16:49:34.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.070+0000 7f730b729700 1 -- 172.21.15.1:0/3966057308 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f72f00a12c0 msgr2=0x7f72f00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:34.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.070+0000 7f730b729700 1 --2- 172.21.15.1:0/3966057308 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f72f00a12c0 0x7f72f00a3770 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f72f4009990 tx=0x7f72f401a040 comp rx=0 tx=0).stop 2023-11-16T16:49:34.086 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.070+0000 7f730b729700 1 -- 172.21.15.1:0/3966057308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73041002a0 msgr2=0x7f7304197600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:34.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.070+0000 7f730b729700 1 --2- 172.21.15.1:0/3966057308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73041002a0 0x7f7304197600 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f730000e9a0 tx=0x7f730000ed60 comp rx=0 tx=0).stop 2023-11-16T16:49:34.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.070+0000 7f730b729700 1 -- 172.21.15.1:0/3966057308 shutdown_connections 2023-11-16T16:49:34.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.070+0000 7f730b729700 1 --2- 172.21.15.1:0/3966057308 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f72f00a12c0 0x7f72f00a3770 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:34.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.070+0000 7f730b729700 1 --2- 172.21.15.1:0/3966057308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7304104140 0x7f73041a2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:34.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.070+0000 7f730b729700 1 --2- 172.21.15.1:0/3966057308 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f73040ff8f0 0x7f73041970c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:34.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.070+0000 7f730b729700 1 --2- 172.21.15.1:0/3966057308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73041002a0 0x7f7304197600 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:34.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.070+0000 7f730b729700 1 -- 172.21.15.1:0/3966057308 >> 172.21.15.1:0/3966057308 conn(0x7f7304075880 msgr2=0x7f73040fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:34.087 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.070+0000 7f730b729700 1 -- 172.21.15.1:0/3966057308 shutdown_connections 2023-11-16T16:49:34.088 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:34.070+0000 7f730b729700 1 -- 172.21.15.1:0/3966057308 wait complete. 2023-11-16T16:49:34.088 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:33 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/2784360745' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:49:34.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:34 smithi001 ceph-mon[98800]: pgmap v323: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:49:34.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:34 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:49:35.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:34 smithi026 ceph-mon[100112]: pgmap v323: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:49:35.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:34 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:49:35.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:34 smithi053 ceph-mon[100638]: pgmap v323: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:49:35.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:34 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-16T16:49:35.089 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2023-11-16T16:49:35.089 INFO:tasks.fwd_scrub:done joining 2023-11-16T16:49:35.090 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-11-16T16:49:35.100 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-16T16:49:35.234 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.232+0000 7fac02145700 1 -- 172.21.15.1:0/1617574946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabfc10cce0 msgr2=0x7fabfc10f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:35.234 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.232+0000 7fac02145700 1 --2- 172.21.15.1:0/1617574946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabfc10cce0 0x7fabfc10f0c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fabf4009f90 tx=0x7fabf400b330 comp rx=0 tx=0).stop 2023-11-16T16:49:35.234 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.232+0000 7fac02145700 1 -- 172.21.15.1:0/1617574946 shutdown_connections 2023-11-16T16:49:35.235 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.232+0000 7fac02145700 1 --2- 172.21.15.1:0/1617574946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabfc10cce0 0x7fabfc10f0c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:35.235 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.232+0000 7fac02145700 1 --2- 172.21.15.1:0/1617574946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabfc0ff130 0x7fabfc0ff500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:35.235 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.232+0000 7fac02145700 1 --2- 172.21.15.1:0/1617574946 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabfc0ffa40 0x7fabfc10c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:35.235 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.232+0000 7fac02145700 1 -- 172.21.15.1:0/1617574946 >> 172.21.15.1:0/1617574946 conn(0x7fabfc0760d0 msgr2=0x7fabfc0764d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:35.235 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.232+0000 7fac02145700 1 -- 172.21.15.1:0/1617574946 shutdown_connections 2023-11-16T16:49:35.235 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.232+0000 7fac02145700 1 -- 172.21.15.1:0/1617574946 wait complete. 2023-11-16T16:49:35.238 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.233+0000 7fac02145700 1 Processor -- start 2023-11-16T16:49:35.238 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.233+0000 7fac02145700 1 -- start start 2023-11-16T16:49:35.238 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.233+0000 7fac02145700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabfc0ff130 0x7fabfc19e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:35.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.234+0000 7fac02145700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabfc0ffa40 0x7fabfc19e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:35.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.234+0000 7fac02145700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabfc10cce0 0x7fabfc198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:35.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.234+0000 7fac02145700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabfc198b90 con 0x7fabfc0ff130 2023-11-16T16:49:35.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.234+0000 7fac02145700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabfc198d00 con 0x7fabfc0ffa40 2023-11-16T16:49:35.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.234+0000 7fac02145700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabfc198ec0 con 0x7fabfc10cce0 2023-11-16T16:49:35.239 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.234+0000 7fabfb7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabfc0ff130 0x7fabfc19e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:35.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.234+0000 7fabfb7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabfc0ff130 0x7fabfc19e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47402/0 (socket says 172.21.15.1:47402) 2023-11-16T16:49:35.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.234+0000 7fabfb7fe700 1 -- 172.21.15.1:0/958841949 learned_addr learned my addr 172.21.15.1:0/958841949 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:49:35.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.234+0000 7fabfaffd700 1 --2- 172.21.15.1:0/958841949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabfc0ffa40 0x7fabfc19e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:35.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.234+0000 7fabfbfff700 1 --2- 172.21.15.1:0/958841949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabfc10cce0 0x7fabfc198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:35.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.234+0000 7fabfb7fe700 1 -- 172.21.15.1:0/958841949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabfc10cce0 msgr2=0x7fabfc198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:35.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.234+0000 7fabfb7fe700 1 --2- 172.21.15.1:0/958841949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabfc10cce0 0x7fabfc198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:35.240 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.234+0000 7fabfb7fe700 1 -- 172.21.15.1:0/958841949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabfc0ffa40 msgr2=0x7fabfc19e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:35.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.234+0000 7fabfb7fe700 1 --2- 172.21.15.1:0/958841949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabfc0ffa40 0x7fabfc19e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:35.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.234+0000 7fabfb7fe700 1 -- 172.21.15.1:0/958841949 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf4009c40 con 0x7fabfc0ff130 2023-11-16T16:49:35.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.234+0000 7fabfb7fe700 1 --2- 172.21.15.1:0/958841949 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabfc0ff130 0x7fabfc19e460 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fabec00e9c0 tx=0x7fabec00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:35.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.234+0000 7fabf8ff9700 1 -- 172.21.15.1:0/958841949 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabec004a30 con 0x7fabfc0ff130 2023-11-16T16:49:35.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.234+0000 7fac02145700 1 -- 172.21.15.1:0/958841949 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabfc1125f0 con 0x7fabfc0ff130 2023-11-16T16:49:35.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.235+0000 7fabf8ff9700 1 -- 172.21.15.1:0/958841949 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fabec00cc80 con 0x7fabfc0ff130 2023-11-16T16:49:35.241 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.235+0000 7fabf8ff9700 1 -- 172.21.15.1:0/958841949 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabec0054e0 con 0x7fabfc0ff130 2023-11-16T16:49:35.242 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.235+0000 7fac02145700 1 -- 172.21.15.1:0/958841949 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabfc112460 con 0x7fabfc0ff130 2023-11-16T16:49:35.242 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.236+0000 7fac02145700 1 -- 172.21.15.1:0/958841949 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabfc112780 con 0x7fabfc0ff130 2023-11-16T16:49:35.242 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.237+0000 7fabf8ff9700 1 -- 172.21.15.1:0/958841949 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fabec004b90 con 0x7fabfc0ff130 2023-11-16T16:49:35.243 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.237+0000 7fabf8ff9700 1 --2- 172.21.15.1:0/958841949 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fabe40a12c0 0x7fabe40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:35.243 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.237+0000 7fabf8ff9700 1 -- 172.21.15.1:0/958841949 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fabec014070 con 0x7fabfc0ff130 2023-11-16T16:49:35.243 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.237+0000 7fabfaffd700 1 --2- 172.21.15.1:0/958841949 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fabe40a12c0 0x7fabe40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:35.243 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.238+0000 7fabfaffd700 1 --2- 172.21.15.1:0/958841949 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fabe40a12c0 0x7fabe40a3770 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fabfc199d20 tx=0x7fabf0009190 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:35.244 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.242+0000 7fabf8ff9700 1 -- 172.21.15.1:0/958841949 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fabec096200 con 0x7fabfc0ff130 2023-11-16T16:49:35.451 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.449+0000 7fac02145700 1 -- 172.21.15.1:0/958841949 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fabfc067260 con 0x7fabfc0ff130 2023-11-16T16:49:35.452 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.451+0000 7fabf8ff9700 1 -- 172.21.15.1:0/958841949 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+22931 (secure 0 0 0) 0x7fabec099820 con 0x7fabfc0ff130 2023-11-16T16:49:35.453 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:49:35.453 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":95,"fsid":"35d9b010-849d-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:30:06.217778+0000","modified":"2023-11-16T16:49:16.352495+0000","last_up_change":"2023-11-16T16:36:31.735403+0000","last_in_change":"2023-11-16T16:36:14.083021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:33:15.916723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-16T16:37:12.158031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-16T16:37:13.272343+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":94,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a8480507-5275-4e52-953b-626110beb436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6803","nonce":2469671609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6805","nonce":2469671609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6809","nonce":2469671609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6807","nonce":2469671609}]},"public_addr":"172.21.15.1:6803/2469671609","cluster_addr":"172.21.15.1:6805/2469671609","heartbeat_back_addr":"172.21.15.1:6809/2469671609","heartbeat_front_addr":"172.21.15.1:6807/2469671609","state":["exists","up"]},{"osd":1,"uuid":"1d011db2-99d1-4575-bdc5-51f88a308fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6801","nonce":247994796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6803","nonce":247994796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6807","nonce":247994796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6805","nonce":247994796}]},"public_addr":"172.21.15.26:6801/247994796","cluster_addr":"172.21.15.26:6803/247994796","heartbeat_back_addr":"172.21.15.26:6807/247994796","heartbeat_front_addr":"172.21.15.26:6805/247994796","state":["exists","up"]},{"osd":2,"uuid":"4d7a2bd1-3c8e-4546-a243-f0354473b031","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6801","nonce":1659316290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6803","nonce":1659316290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6807","nonce":1659316290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6805","nonce":1659316290}]},"public_addr":"172.21.15.53:6801/1659316290","cluster_addr":"172.21.15.53:6803/1659316290","heartbeat_back_addr":"172.21.15.53:6807/1659316290","heartbeat_front_addr":"172.21.15.53:6805/1659316290","state":["exists","up"]},{"osd":3,"uuid":"9c290fd9-e121-4a81-9687-b0eb93debd86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6811","nonce":2728987281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6813","nonce":2728987281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6817","nonce":2728987281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6815","nonce":2728987281}]},"public_addr":"172.21.15.1:6811/2728987281","cluster_addr":"172.21.15.1:6813/2728987281","heartbeat_back_addr":"172.21.15.1:6817/2728987281","heartbeat_front_addr":"172.21.15.1:6815/2728987281","state":["exists","up"]},{"osd":4,"uuid":"533226bc-cc57-4f15-9e29-820294d57e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6809","nonce":2954740024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6811","nonce":2954740024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6815","nonce":2954740024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6813","nonce":2954740024}]},"public_addr":"172.21.15.26:6809/2954740024","cluster_addr":"172.21.15.26:6811/2954740024","heartbeat_back_addr":"172.21.15.26:6815/2954740024","heartbeat_front_addr":"172.21.15.26:6813/2954740024","state":["exists","up"]},{"osd":5,"uuid":"d3ab25eb-e1d4-4557-bc89-efc9072401ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6809","nonce":3966666318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6811","nonce":3966666318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6815","nonce":3966666318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6813","nonce":3966666318}]},"public_addr":"172.21.15.53:6809/3966666318","cluster_addr":"172.21.15.53:6811/3966666318","heartbeat_back_addr":"172.21.15.53:6815/3966666318","heartbeat_front_addr":"172.21.15.53:6813/3966666318","state":["exists","up"]},{"osd":6,"uuid":"180f506e-afb8-4043-b0b4-2dc86f6d4bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6819","nonce":3364032646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6821","nonce":3364032646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6825","nonce":3364032646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6823","nonce":3364032646}]},"public_addr":"172.21.15.1:6819/3364032646","cluster_addr":"172.21.15.1:6821/3364032646","heartbeat_back_addr":"172.21.15.1:6825/3364032646","heartbeat_front_addr":"172.21.15.1:6823/3364032646","state":["exists","up"]},{"osd":7,"uuid":"0d837bac-194f-498d-b7fb-b3147e992329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6817","nonce":3766901391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6819","nonce":3766901391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6823","nonce":3766901391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6821","nonce":3766901391}]},"public_addr":"172.21.15.26:6817/3766901391","cluster_addr":"172.21.15.26:6819/3766901391","heartbeat_back_addr":"172.21.15.26:6823/3766901391","heartbeat_front_addr":"172.21.15.26:6821/3766901391","state":["exists","up"]},{"osd":8,"uuid":"933f231a-7454-47b0-9291-f1fa64f0baec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6817","nonce":3224335871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6819","nonce":3224335871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6823","nonce":3224335871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6821","nonce":3224335871}]},"public_addr":"172.21.15.53:6817/3224335871","cluster_addr":"172.21.15.53:6819/3224335871","heartbeat_back_addr":"172.21.15.53:6823/3224335871","heartbeat_front_addr":"172.21.15.53:6821/3224335871","state":["exists","up"]},{"osd":9,"uuid":"245fcbb3-c651-4703-ab80-309dc98bd983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6827","nonce":1391199844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6829","nonce":1391199844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6833","nonce":1391199844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6831","nonce":1391199844}]},"public_addr":"172.21.15.1:6827/1391199844","cluster_addr":"172.21.15.1:6829/1391199844","heartbeat_back_addr":"172.21.15.1:6833/1391199844","heartbeat_front_addr":"172.21.15.1:6831/1391199844","state":["exists","up"]},{"osd":10,"uuid":"2c090e7a-5240-4d52-ae4f-e94fe05383d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6825","nonce":1773545447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6827","nonce":1773545447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6831","nonce":1773545447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6829","nonce":1773545447}]},"public_addr":"172.21.15.26:6825/1773545447","cluster_addr":"172.21.15.26:6827/1773545447","heartbeat_back_addr":"172.21.15.26:6831/1773545447","heartbeat_front_addr":"172.21.15.26:6829/1773545447","state":["exists","up"]},{"osd":11,"uuid":"4511e32d-e469-45e1-8310-0ff400cb7b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6825","nonce":2564669546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6827","nonce":2564669546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6831","nonce":2564669546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6829","nonce":2564669546}]},"public_addr":"172.21.15.53:6825/2564669546","cluster_addr":"172.21.15.53:6827/2564669546","heartbeat_back_addr":"172.21.15.53:6831/2564669546","heartbeat_front_addr":"172.21.15.53:6829/2564669546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:28.140250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:49.885461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:12.521487+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:34.926197+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:56.815594+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:18.387751+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:40.587106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:02.319370+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:23.832298+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:46.338040+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:08.730414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:30.050070+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1210074029":"2023-11-17T16:39:09.360183+0000","172.21.15.1:0/2308375878":"2023-11-17T16:39:09.360183+0000","172.21.15.1:0/3270932790":"2023-11-17T16:39:09.360183+0000","172.21.15.1:6801/2490729488":"2023-11-17T16:39:09.360183+0000","172.21.15.1:6800/2490729488":"2023-11-17T16:39:09.360183+0000","172.21.15.1:0/1494336948":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6801/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6800/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/1768923877":"2023-11-17T16:39:09.360183+0000","172.21.15.1:6800/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/1641976669":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/2895923638":"2023-11-17T16:30:35.064289+0000","172.21.15.1:6801/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/78811711":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/4131957708":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/344403698":"2023-11-17T16:30:35.064289+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:49:35.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.453+0000 7fac02145700 1 -- 172.21.15.1:0/958841949 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fabe40a12c0 msgr2=0x7fabe40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:35.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.453+0000 7fac02145700 1 --2- 172.21.15.1:0/958841949 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fabe40a12c0 0x7fabe40a3770 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fabfc199d20 tx=0x7fabf0009190 comp rx=0 tx=0).stop 2023-11-16T16:49:35.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.453+0000 7fac02145700 1 -- 172.21.15.1:0/958841949 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabfc0ff130 msgr2=0x7fabfc19e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:35.455 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.453+0000 7fac02145700 1 --2- 172.21.15.1:0/958841949 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabfc0ff130 0x7fabfc19e460 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fabec00e9c0 tx=0x7fabec00ecd0 comp rx=0 tx=0).stop 2023-11-16T16:49:35.456 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.453+0000 7fac02145700 1 -- 172.21.15.1:0/958841949 shutdown_connections 2023-11-16T16:49:35.456 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.453+0000 7fac02145700 1 --2- 172.21.15.1:0/958841949 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fabe40a12c0 0x7fabe40a3770 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:35.456 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.453+0000 7fac02145700 1 --2- 172.21.15.1:0/958841949 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fabfc10cce0 0x7fabfc198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:35.456 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.453+0000 7fac02145700 1 --2- 172.21.15.1:0/958841949 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabfc0ff130 0x7fabfc19e460 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:35.456 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.453+0000 7fac02145700 1 --2- 172.21.15.1:0/958841949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabfc0ffa40 0x7fabfc19e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:35.456 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.453+0000 7fac02145700 1 -- 172.21.15.1:0/958841949 >> 172.21.15.1:0/958841949 conn(0x7fabfc0760d0 msgr2=0x7fabfc0fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:35.457 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.453+0000 7fac02145700 1 -- 172.21.15.1:0/958841949 shutdown_connections 2023-11-16T16:49:35.457 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.453+0000 7fac02145700 1 -- 172.21.15.1:0/958841949 wait complete. 2023-11-16T16:49:35.467 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-16T16:33:15.916723+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-16T16:37:12.158031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-16T16:37:13.272343+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 94, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-16T16:49:35.468 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-16T16:49:35.600 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.596+0000 7fc51c5d5700 1 -- 172.21.15.1:0/106218974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc514100cc0 msgr2=0x7fc514101090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:35.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.596+0000 7fc51c5d5700 1 --2- 172.21.15.1:0/106218974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc514100cc0 0x7fc514101090 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fc5100099c0 tx=0x7fc510009cd0 comp rx=0 tx=0).stop 2023-11-16T16:49:35.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.596+0000 7fc51c5d5700 1 -- 172.21.15.1:0/106218974 shutdown_connections 2023-11-16T16:49:35.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.596+0000 7fc51c5d5700 1 --2- 172.21.15.1:0/106218974 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc5141055c0 0x7fc51410d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:35.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.596+0000 7fc51c5d5700 1 --2- 172.21.15.1:0/106218974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc514100cc0 0x7fc514101090 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:35.601 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.596+0000 7fc51c5d5700 1 --2- 172.21.15.1:0/106218974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc514101660 0x7fc514104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:35.602 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.596+0000 7fc51c5d5700 1 -- 172.21.15.1:0/106218974 >> 172.21.15.1:0/106218974 conn(0x7fc514077560 msgr2=0x7fc514077960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:35.602 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.596+0000 7fc51c5d5700 1 -- 172.21.15.1:0/106218974 shutdown_connections 2023-11-16T16:49:35.602 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.596+0000 7fc51c5d5700 1 -- 172.21.15.1:0/106218974 wait complete. 2023-11-16T16:49:35.602 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.596+0000 7fc51c5d5700 1 Processor -- start 2023-11-16T16:49:35.602 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.597+0000 7fc51c5d5700 1 -- start start 2023-11-16T16:49:35.602 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.597+0000 7fc51c5d5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc514100cc0 0x7fc5141964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:35.603 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.597+0000 7fc51c5d5700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc514101660 0x7fc5141969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:35.603 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.597+0000 7fc51c5d5700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5141055c0 0x7fc51419ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:35.603 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.597+0000 7fc51c5d5700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51419b480 con 0x7fc5141055c0 2023-11-16T16:49:35.603 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.597+0000 7fc51c5d5700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51419b5f0 con 0x7fc514100cc0 2023-11-16T16:49:35.603 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.597+0000 7fc51c5d5700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51419b7b0 con 0x7fc514101660 2023-11-16T16:49:35.603 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.597+0000 7fc51ab72700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5141055c0 0x7fc51419ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:35.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.597+0000 7fc51ab72700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5141055c0 0x7fc51419ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47420/0 (socket says 172.21.15.1:47420) 2023-11-16T16:49:35.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.597+0000 7fc51ab72700 1 -- 172.21.15.1:0/886397378 learned_addr learned my addr 172.21.15.1:0/886397378 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:49:35.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.597+0000 7fc519b70700 1 --2- 172.21.15.1:0/886397378 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc514101660 0x7fc5141969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:35.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.597+0000 7fc51a371700 1 --2- 172.21.15.1:0/886397378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc514100cc0 0x7fc5141964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:35.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.597+0000 7fc51ab72700 1 -- 172.21.15.1:0/886397378 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc514101660 msgr2=0x7fc5141969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:35.604 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.597+0000 7fc51ab72700 1 --2- 172.21.15.1:0/886397378 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc514101660 0x7fc5141969e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:35.605 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.597+0000 7fc51ab72700 1 -- 172.21.15.1:0/886397378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc514100cc0 msgr2=0x7fc5141964a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:35.605 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.597+0000 7fc51ab72700 1 --2- 172.21.15.1:0/886397378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc514100cc0 0x7fc5141964a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:35.605 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.597+0000 7fc51ab72700 1 -- 172.21.15.1:0/886397378 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5100095f0 con 0x7fc5141055c0 2023-11-16T16:49:35.605 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.597+0000 7fc51ab72700 1 --2- 172.21.15.1:0/886397378 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5141055c0 0x7fc51419ae40 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fc50c007ee0 tx=0x7fc50c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:35.605 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.598+0000 7fc50b7fe700 1 -- 172.21.15.1:0/886397378 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c00dd20 con 0x7fc5141055c0 2023-11-16T16:49:35.605 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.598+0000 7fc50b7fe700 1 -- 172.21.15.1:0/886397378 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc50c00f040 con 0x7fc5141055c0 2023-11-16T16:49:35.606 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.598+0000 7fc51c5d5700 1 -- 172.21.15.1:0/886397378 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc514110380 con 0x7fc5141055c0 2023-11-16T16:49:35.606 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.598+0000 7fc50b7fe700 1 -- 172.21.15.1:0/886397378 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c0048a0 con 0x7fc5141055c0 2023-11-16T16:49:35.606 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.598+0000 7fc51c5d5700 1 -- 172.21.15.1:0/886397378 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5141101f0 con 0x7fc5141055c0 2023-11-16T16:49:35.606 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.599+0000 7fc51c5d5700 1 -- 172.21.15.1:0/886397378 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc514110510 con 0x7fc5141055c0 2023-11-16T16:49:35.607 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.600+0000 7fc50b7fe700 1 -- 172.21.15.1:0/886397378 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fc50c003a00 con 0x7fc5141055c0 2023-11-16T16:49:35.607 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.600+0000 7fc50b7fe700 1 --2- 172.21.15.1:0/886397378 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fc5000a12c0 0x7fc5000a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:35.607 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.601+0000 7fc50b7fe700 1 -- 172.21.15.1:0/886397378 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fc50c0cccf0 con 0x7fc5141055c0 2023-11-16T16:49:35.608 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.601+0000 7fc51a371700 1 --2- 172.21.15.1:0/886397378 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fc5000a12c0 0x7fc5000a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:35.608 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.601+0000 7fc51a371700 1 --2- 172.21.15.1:0/886397378 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fc5000a12c0 0x7fc5000a3770 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc51000b580 tx=0x7fc5100058e0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:35.608 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.605+0000 7fc50b7fe700 1 -- 172.21.15.1:0/886397378 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc50c0960f0 con 0x7fc5141055c0 2023-11-16T16:49:35.811 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.809+0000 7fc51c5d5700 1 -- 172.21.15.1:0/886397378 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc5141a3510 con 0x7fc5141055c0 2023-11-16T16:49:35.812 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.810+0000 7fc50b7fe700 1 -- 172.21.15.1:0/886397378 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc50c099710 con 0x7fc5141055c0 2023-11-16T16:49:35.812 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2023-11-16T16:49:35.814 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.812+0000 7fc51c5d5700 1 -- 172.21.15.1:0/886397378 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fc5000a12c0 msgr2=0x7fc5000a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:35.814 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.812+0000 7fc51c5d5700 1 --2- 172.21.15.1:0/886397378 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fc5000a12c0 0x7fc5000a3770 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc51000b580 tx=0x7fc5100058e0 comp rx=0 tx=0).stop 2023-11-16T16:49:35.814 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.812+0000 7fc51c5d5700 1 -- 172.21.15.1:0/886397378 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5141055c0 msgr2=0x7fc51419ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:35.815 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.812+0000 7fc51c5d5700 1 --2- 172.21.15.1:0/886397378 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5141055c0 0x7fc51419ae40 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fc50c007ee0 tx=0x7fc50c00d3b0 comp rx=0 tx=0).stop 2023-11-16T16:49:35.815 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.812+0000 7fc51c5d5700 1 -- 172.21.15.1:0/886397378 shutdown_connections 2023-11-16T16:49:35.815 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.812+0000 7fc51c5d5700 1 --2- 172.21.15.1:0/886397378 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fc5000a12c0 0x7fc5000a3770 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:35.815 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.812+0000 7fc51c5d5700 1 --2- 172.21.15.1:0/886397378 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc5141055c0 0x7fc51419ae40 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:35.815 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.812+0000 7fc51c5d5700 1 --2- 172.21.15.1:0/886397378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc514100cc0 0x7fc5141964a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:35.815 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.812+0000 7fc51c5d5700 1 --2- 172.21.15.1:0/886397378 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc514101660 0x7fc5141969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:35.815 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.812+0000 7fc51c5d5700 1 -- 172.21.15.1:0/886397378 >> 172.21.15.1:0/886397378 conn(0x7fc514077560 msgr2=0x7fc5140ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:35.816 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.812+0000 7fc51c5d5700 1 -- 172.21.15.1:0/886397378 shutdown_connections 2023-11-16T16:49:35.816 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.812+0000 7fc51c5d5700 1 -- 172.21.15.1:0/886397378 wait complete. 2023-11-16T16:49:35.825 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:35 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/958841949' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:49:35.826 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-16T16:49:35.958 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.954+0000 7f6ea5747700 1 -- 172.21.15.1:0/4265227560 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ea0068770 msgr2=0x7f6ea010d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:35.959 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.954+0000 7f6ea5747700 1 --2- 172.21.15.1:0/4265227560 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ea0068770 0x7f6ea010d0e0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f6e9800b0a0 tx=0x7f6e9800b3b0 comp rx=0 tx=0).stop 2023-11-16T16:49:35.959 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.954+0000 7f6ea5747700 1 -- 172.21.15.1:0/4265227560 shutdown_connections 2023-11-16T16:49:35.959 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.954+0000 7f6ea5747700 1 --2- 172.21.15.1:0/4265227560 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ea0068770 0x7f6ea010d0e0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:35.959 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.954+0000 7f6ea5747700 1 --2- 172.21.15.1:0/4265227560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ea0105250 0x7f6ea0105620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:35.959 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.954+0000 7f6ea5747700 1 --2- 172.21.15.1:0/4265227560 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ea0067dc0 0x7f6ea0068230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:35.960 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.954+0000 7f6ea5747700 1 -- 172.21.15.1:0/4265227560 >> 172.21.15.1:0/4265227560 conn(0x7f6ea00763a0 msgr2=0x7f6ea00767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:35.960 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.954+0000 7f6ea5747700 1 -- 172.21.15.1:0/4265227560 shutdown_connections 2023-11-16T16:49:35.960 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.954+0000 7f6ea5747700 1 -- 172.21.15.1:0/4265227560 wait complete. 2023-11-16T16:49:35.960 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.955+0000 7f6ea5747700 1 Processor -- start 2023-11-16T16:49:35.960 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.955+0000 7f6ea5747700 1 -- start start 2023-11-16T16:49:35.960 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.955+0000 7f6ea5747700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ea0067dc0 0x7f6ea00ffb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:35.960 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.955+0000 7f6ea5747700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ea0068770 0x7f6ea0100080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:35.961 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.955+0000 7f6ea5747700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ea0105250 0x7f6ea01a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:35.961 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.955+0000 7f6ea5747700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea01008b0 con 0x7f6ea0068770 2023-11-16T16:49:35.961 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.955+0000 7f6ea5747700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea01a4bc0 con 0x7f6ea0105250 2023-11-16T16:49:35.961 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.955+0000 7f6ea5747700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea01a4d00 con 0x7f6ea0067dc0 2023-11-16T16:49:35.961 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.956+0000 7f6e9e7fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ea0068770 0x7f6ea0100080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:35.961 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.956+0000 7f6e9e7fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ea0068770 0x7f6ea0100080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47432/0 (socket says 172.21.15.1:47432) 2023-11-16T16:49:35.961 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.956+0000 7f6e9e7fc700 1 -- 172.21.15.1:0/1183631661 learned_addr learned my addr 172.21.15.1:0/1183631661 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:49:35.962 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.956+0000 7f6e9effd700 1 --2- 172.21.15.1:0/1183631661 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ea0067dc0 0x7f6ea00ffb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:35.962 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.956+0000 7f6e9f7fe700 1 --2- 172.21.15.1:0/1183631661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ea0105250 0x7f6ea01a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:35.962 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.956+0000 7f6e9e7fc700 1 -- 172.21.15.1:0/1183631661 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ea0067dc0 msgr2=0x7f6ea00ffb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:35.962 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.956+0000 7f6e9e7fc700 1 --2- 172.21.15.1:0/1183631661 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ea0067dc0 0x7f6ea00ffb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:35.962 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.956+0000 7f6e9e7fc700 1 -- 172.21.15.1:0/1183631661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ea0105250 msgr2=0x7f6ea01a4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:35.962 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.956+0000 7f6e9e7fc700 1 --2- 172.21.15.1:0/1183631661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ea0105250 0x7f6ea01a4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:35.962 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.956+0000 7f6e9e7fc700 1 -- 172.21.15.1:0/1183631661 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e98009d00 con 0x7f6ea0068770 2023-11-16T16:49:35.963 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.956+0000 7f6e9e7fc700 1 --2- 172.21.15.1:0/1183631661 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ea0068770 0x7f6ea0100080 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400e9a0 tx=0x7f6e9400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:35.963 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.956+0000 7f6e87fff700 1 -- 172.21.15.1:0/1183631661 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e94009790 con 0x7f6ea0068770 2023-11-16T16:49:35.963 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.956+0000 7f6ea5747700 1 -- 172.21.15.1:0/1183631661 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ea01103d0 con 0x7f6ea0068770 2023-11-16T16:49:35.963 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.956+0000 7f6e87fff700 1 -- 172.21.15.1:0/1183631661 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e94010430 con 0x7f6ea0068770 2023-11-16T16:49:35.963 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.956+0000 7f6e87fff700 1 -- 172.21.15.1:0/1183631661 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e94018680 con 0x7f6ea0068770 2023-11-16T16:49:35.963 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.956+0000 7f6ea5747700 1 -- 172.21.15.1:0/1183631661 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ea0110240 con 0x7f6ea0068770 2023-11-16T16:49:35.963 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.957+0000 7f6ea5747700 1 -- 172.21.15.1:0/1183631661 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ea0110560 con 0x7f6ea0068770 2023-11-16T16:49:35.964 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.958+0000 7f6e87fff700 1 -- 172.21.15.1:0/1183631661 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f6e940105a0 con 0x7f6ea0068770 2023-11-16T16:49:35.965 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.959+0000 7f6e87fff700 1 --2- 172.21.15.1:0/1183631661 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f6e880a12c0 0x7f6e880a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:35.965 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.959+0000 7f6e87fff700 1 -- 172.21.15.1:0/1183631661 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f6e94014070 con 0x7f6ea0068770 2023-11-16T16:49:35.965 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.959+0000 7f6e9effd700 1 --2- 172.21.15.1:0/1183631661 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f6e880a12c0 0x7f6e880a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:35.965 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.959+0000 7f6e9effd700 1 --2- 172.21.15.1:0/1183631661 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f6e880a12c0 0x7f6e880a3770 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6e90009c60 tx=0x7f6e90009260 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:35.965 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:35.962+0000 7f6e87fff700 1 -- 172.21.15.1:0/1183631661 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6e940964a0 con 0x7f6ea0068770 2023-11-16T16:49:36.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:35 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/958841949' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:49:36.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:35 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/958841949' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:49:36.163 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.161+0000 7f6ea5747700 1 -- 172.21.15.1:0/1183631661 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6ea01a56d0 con 0x7f6ea0068770 2023-11-16T16:49:36.163 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.162+0000 7f6e87fff700 1 -- 172.21.15.1:0/1183631661 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6e94099ac0 con 0x7f6ea0068770 2023-11-16T16:49:36.164 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2023-11-16T16:49:36.166 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.164+0000 7f6ea5747700 1 -- 172.21.15.1:0/1183631661 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f6e880a12c0 msgr2=0x7f6e880a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:36.166 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.164+0000 7f6ea5747700 1 --2- 172.21.15.1:0/1183631661 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f6e880a12c0 0x7f6e880a3770 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6e90009c60 tx=0x7f6e90009260 comp rx=0 tx=0).stop 2023-11-16T16:49:36.166 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.164+0000 7f6ea5747700 1 -- 172.21.15.1:0/1183631661 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ea0068770 msgr2=0x7f6ea0100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:36.166 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.164+0000 7f6ea5747700 1 --2- 172.21.15.1:0/1183631661 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ea0068770 0x7f6ea0100080 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400e9a0 tx=0x7f6e9400ecb0 comp rx=0 tx=0).stop 2023-11-16T16:49:36.166 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.164+0000 7f6ea5747700 1 -- 172.21.15.1:0/1183631661 shutdown_connections 2023-11-16T16:49:36.166 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.164+0000 7f6ea5747700 1 --2- 172.21.15.1:0/1183631661 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f6e880a12c0 0x7f6e880a3770 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:36.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.164+0000 7f6ea5747700 1 --2- 172.21.15.1:0/1183631661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ea0105250 0x7f6ea01a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:36.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.164+0000 7f6ea5747700 1 --2- 172.21.15.1:0/1183631661 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6ea0067dc0 0x7f6ea00ffb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:36.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.164+0000 7f6ea5747700 1 --2- 172.21.15.1:0/1183631661 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ea0068770 0x7f6ea0100080 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:36.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.164+0000 7f6ea5747700 1 -- 172.21.15.1:0/1183631661 >> 172.21.15.1:0/1183631661 conn(0x7f6ea00763a0 msgr2=0x7f6ea00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:36.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.164+0000 7f6ea5747700 1 -- 172.21.15.1:0/1183631661 shutdown_connections 2023-11-16T16:49:36.167 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.164+0000 7f6ea5747700 1 -- 172.21.15.1:0/1183631661 wait complete. 2023-11-16T16:49:36.178 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-16T16:49:36.308 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.306+0000 7fc1eb47d700 1 -- 172.21.15.1:0/836035530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1e41052a0 msgr2=0x7fc1e4105670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:36.308 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.306+0000 7fc1eb47d700 1 --2- 172.21.15.1:0/836035530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1e41052a0 0x7fc1e4105670 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fc1e00099c0 tx=0x7fc1e0009cd0 comp rx=0 tx=0).stop 2023-11-16T16:49:36.308 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.306+0000 7fc1eb47d700 1 -- 172.21.15.1:0/836035530 shutdown_connections 2023-11-16T16:49:36.308 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.306+0000 7fc1eb47d700 1 --2- 172.21.15.1:0/836035530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1e4068770 0x7fc1e410d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:36.309 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.306+0000 7fc1eb47d700 1 --2- 172.21.15.1:0/836035530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1e41052a0 0x7fc1e4105670 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:36.309 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.306+0000 7fc1eb47d700 1 --2- 172.21.15.1:0/836035530 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1e4067dc0 0x7fc1e4068230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:36.309 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.306+0000 7fc1eb47d700 1 -- 172.21.15.1:0/836035530 >> 172.21.15.1:0/836035530 conn(0x7fc1e40763a0 msgr2=0x7fc1e40767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:36.309 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.306+0000 7fc1eb47d700 1 -- 172.21.15.1:0/836035530 shutdown_connections 2023-11-16T16:49:36.309 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.306+0000 7fc1eb47d700 1 -- 172.21.15.1:0/836035530 wait complete. 2023-11-16T16:49:36.311 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.307+0000 7fc1eb47d700 1 Processor -- start 2023-11-16T16:49:36.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.307+0000 7fc1eb47d700 1 -- start start 2023-11-16T16:49:36.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.307+0000 7fc1eb47d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1e4067dc0 0x7fc1e41963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:36.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.307+0000 7fc1eb47d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1e4068770 0x7fc1e41968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:36.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.307+0000 7fc1eb47d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1e41052a0 0x7fc1e419ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:36.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.307+0000 7fc1eb47d700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e419b390 con 0x7fc1e4067dc0 2023-11-16T16:49:36.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.307+0000 7fc1eb47d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e419b500 con 0x7fc1e4068770 2023-11-16T16:49:36.312 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.307+0000 7fc1eb47d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e419b6c0 con 0x7fc1e41052a0 2023-11-16T16:49:36.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.307+0000 7fc1e9219700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1e4067dc0 0x7fc1e41963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:36.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.307+0000 7fc1e9219700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1e4067dc0 0x7fc1e41963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47444/0 (socket says 172.21.15.1:47444) 2023-11-16T16:49:36.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.307+0000 7fc1e9219700 1 -- 172.21.15.1:0/4083684091 learned_addr learned my addr 172.21.15.1:0/4083684091 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:49:36.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.307+0000 7fc1e9a1a700 1 --2- 172.21.15.1:0/4083684091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1e41052a0 0x7fc1e419ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:36.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.308+0000 7fc1e8a18700 1 --2- 172.21.15.1:0/4083684091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1e4068770 0x7fc1e41968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:36.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.308+0000 7fc1e9219700 1 -- 172.21.15.1:0/4083684091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1e41052a0 msgr2=0x7fc1e419ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:36.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.308+0000 7fc1e9219700 1 --2- 172.21.15.1:0/4083684091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1e41052a0 0x7fc1e419ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:36.313 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.308+0000 7fc1e9219700 1 -- 172.21.15.1:0/4083684091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1e4068770 msgr2=0x7fc1e41968f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:36.314 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.308+0000 7fc1e9219700 1 --2- 172.21.15.1:0/4083684091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1e4068770 0x7fc1e41968f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:36.314 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.308+0000 7fc1e9219700 1 -- 172.21.15.1:0/4083684091 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e00095f0 con 0x7fc1e4067dc0 2023-11-16T16:49:36.314 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.308+0000 7fc1e9219700 1 --2- 172.21.15.1:0/4083684091 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1e4067dc0 0x7fc1e41963b0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0009990 tx=0x7fc1e000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:36.314 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.308+0000 7fc1da7fc700 1 -- 172.21.15.1:0/4083684091 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e001d070 con 0x7fc1e4067dc0 2023-11-16T16:49:36.314 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.308+0000 7fc1da7fc700 1 -- 172.21.15.1:0/4083684091 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1e0005030 con 0x7fc1e4067dc0 2023-11-16T16:49:36.314 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.308+0000 7fc1da7fc700 1 -- 172.21.15.1:0/4083684091 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e0004a50 con 0x7fc1e4067dc0 2023-11-16T16:49:36.314 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.308+0000 7fc1eb47d700 1 -- 172.21.15.1:0/4083684091 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1e4110370 con 0x7fc1e4067dc0 2023-11-16T16:49:36.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.308+0000 7fc1eb47d700 1 -- 172.21.15.1:0/4083684091 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1e41101e0 con 0x7fc1e4067dc0 2023-11-16T16:49:36.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.309+0000 7fc1eb47d700 1 -- 172.21.15.1:0/4083684091 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1e4110500 con 0x7fc1e4067dc0 2023-11-16T16:49:36.315 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.310+0000 7fc1da7fc700 1 -- 172.21.15.1:0/4083684091 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fc1e0005430 con 0x7fc1e4067dc0 2023-11-16T16:49:36.316 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.311+0000 7fc1da7fc700 1 --2- 172.21.15.1:0/4083684091 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fc1d00a1340 0x7fc1d00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:36.316 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.311+0000 7fc1da7fc700 1 -- 172.21.15.1:0/4083684091 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fc1e00cda10 con 0x7fc1e4067dc0 2023-11-16T16:49:36.316 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.311+0000 7fc1e8a18700 1 --2- 172.21.15.1:0/4083684091 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fc1d00a1340 0x7fc1d00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:36.316 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.311+0000 7fc1e8a18700 1 --2- 172.21.15.1:0/4083684091 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fc1d00a1340 0x7fc1d00a37f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc1e41979c0 tx=0x7fc1d4009260 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:36.317 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.315+0000 7fc1da7fc700 1 -- 172.21.15.1:0/4083684091 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc1e0096e40 con 0x7fc1e4067dc0 2023-11-16T16:49:36.519 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.517+0000 7fc1eb47d700 1 -- 172.21.15.1:0/4083684091 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc1e4067a50 con 0x7fc1e4067dc0 2023-11-16T16:49:36.519 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.518+0000 7fc1da7fc700 1 -- 172.21.15.1:0/4083684091 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc1e009a460 con 0x7fc1e4067dc0 2023-11-16T16:49:36.519 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2023-11-16T16:49:36.522 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.519+0000 7fc1eb47d700 1 -- 172.21.15.1:0/4083684091 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fc1d00a1340 msgr2=0x7fc1d00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:36.522 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.519+0000 7fc1eb47d700 1 --2- 172.21.15.1:0/4083684091 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fc1d00a1340 0x7fc1d00a37f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc1e41979c0 tx=0x7fc1d4009260 comp rx=0 tx=0).stop 2023-11-16T16:49:36.522 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.519+0000 7fc1eb47d700 1 -- 172.21.15.1:0/4083684091 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1e4067dc0 msgr2=0x7fc1e41963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:36.522 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.519+0000 7fc1eb47d700 1 --2- 172.21.15.1:0/4083684091 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1e4067dc0 0x7fc1e41963b0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0009990 tx=0x7fc1e000f690 comp rx=0 tx=0).stop 2023-11-16T16:49:36.522 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.520+0000 7fc1eb47d700 1 -- 172.21.15.1:0/4083684091 shutdown_connections 2023-11-16T16:49:36.522 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.520+0000 7fc1eb47d700 1 --2- 172.21.15.1:0/4083684091 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fc1d00a1340 0x7fc1d00a37f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:36.523 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.520+0000 7fc1eb47d700 1 --2- 172.21.15.1:0/4083684091 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc1e41052a0 0x7fc1e419ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:36.523 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.520+0000 7fc1eb47d700 1 --2- 172.21.15.1:0/4083684091 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc1e4067dc0 0x7fc1e41963b0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:36.523 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.520+0000 7fc1eb47d700 1 --2- 172.21.15.1:0/4083684091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1e4068770 0x7fc1e41968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:36.523 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.520+0000 7fc1eb47d700 1 -- 172.21.15.1:0/4083684091 >> 172.21.15.1:0/4083684091 conn(0x7fc1e40763a0 msgr2=0x7fc1e40fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:36.523 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.520+0000 7fc1eb47d700 1 -- 172.21.15.1:0/4083684091 shutdown_connections 2023-11-16T16:49:36.523 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.520+0000 7fc1eb47d700 1 -- 172.21.15.1:0/4083684091 wait complete. 2023-11-16T16:49:36.534 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-11-16T16:49:36.662 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.660+0000 7f9214e8e700 1 -- 172.21.15.1:0/2240834415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f92100ffdd0 msgr2=0x7f9210103d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:36.663 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.660+0000 7f9214e8e700 1 --2- 172.21.15.1:0/2240834415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f92100ffdd0 0x7f9210103d90 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f92040099c0 tx=0x7f9204009cd0 comp rx=0 tx=0).stop 2023-11-16T16:49:36.663 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.661+0000 7f9214e8e700 1 -- 172.21.15.1:0/2240834415 shutdown_connections 2023-11-16T16:49:36.663 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.661+0000 7f9214e8e700 1 --2- 172.21.15.1:0/2240834415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9210104470 0x7f921010c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:36.663 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.661+0000 7f9214e8e700 1 --2- 172.21.15.1:0/2240834415 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92100ff430 0x7f92100ff800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:36.663 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.661+0000 7f9214e8e700 1 --2- 172.21.15.1:0/2240834415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f92100ffdd0 0x7f9210103d90 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:36.663 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.661+0000 7f9214e8e700 1 -- 172.21.15.1:0/2240834415 >> 172.21.15.1:0/2240834415 conn(0x7f9210077020 msgr2=0x7f9210077420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:36.663 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.661+0000 7f9214e8e700 1 -- 172.21.15.1:0/2240834415 shutdown_connections 2023-11-16T16:49:36.664 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.661+0000 7f9214e8e700 1 -- 172.21.15.1:0/2240834415 wait complete. 2023-11-16T16:49:36.666 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.661+0000 7f9214e8e700 1 Processor -- start 2023-11-16T16:49:36.666 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.662+0000 7f9214e8e700 1 -- start start 2023-11-16T16:49:36.666 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.662+0000 7f9214e8e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92100ff430 0x7f92101972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:36.666 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.662+0000 7f9214e8e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92100ffdd0 0x7f92101977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:36.666 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.662+0000 7f9214e8e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9210104470 0x7f92101a2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:36.667 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.662+0000 7f9214e8e700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9210198100 con 0x7f9210104470 2023-11-16T16:49:36.667 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.662+0000 7f9214e8e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92101a2870 con 0x7f92100ffdd0 2023-11-16T16:49:36.667 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.662+0000 7f9214e8e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92101a29e0 con 0x7f92100ff430 2023-11-16T16:49:36.667 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.662+0000 7f920ed9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9210104470 0x7f92101a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:36.667 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.662+0000 7f920ed9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9210104470 0x7f92101a2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47460/0 (socket says 172.21.15.1:47460) 2023-11-16T16:49:36.667 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.662+0000 7f920ed9d700 1 -- 172.21.15.1:0/3860561352 learned_addr learned my addr 172.21.15.1:0/3860561352 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:49:36.667 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.662+0000 7f920e59c700 1 --2- 172.21.15.1:0/3860561352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92100ff430 0x7f92101972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:36.667 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.662+0000 7f920dd9b700 1 --2- 172.21.15.1:0/3860561352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92100ffdd0 0x7f92101977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:36.668 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.662+0000 7f920ed9d700 1 -- 172.21.15.1:0/3860561352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92100ff430 msgr2=0x7f92101972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:36.668 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.662+0000 7f920ed9d700 1 --2- 172.21.15.1:0/3860561352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92100ff430 0x7f92101972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:36.668 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.662+0000 7f920ed9d700 1 -- 172.21.15.1:0/3860561352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92100ffdd0 msgr2=0x7f92101977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:36.668 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.662+0000 7f920ed9d700 1 --2- 172.21.15.1:0/3860561352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92100ffdd0 0x7f92101977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:36.668 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.662+0000 7f920ed9d700 1 -- 172.21.15.1:0/3860561352 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92040095f0 con 0x7f9210104470 2023-11-16T16:49:36.668 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.662+0000 7f920e59c700 1 --2- 172.21.15.1:0/3860561352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92100ff430 0x7f92101972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:49:36.668 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.663+0000 7f920ed9d700 1 --2- 172.21.15.1:0/3860561352 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9210104470 0x7f92101a2330 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f9208007ee0 tx=0x7f920800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:36.669 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.663+0000 7f91ff7fe700 1 -- 172.21.15.1:0/3860561352 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9208017070 con 0x7f9210104470 2023-11-16T16:49:36.669 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.663+0000 7f91ff7fe700 1 -- 172.21.15.1:0/3860561352 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f920800f040 con 0x7f9210104470 2023-11-16T16:49:36.669 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.663+0000 7f9214e8e700 1 -- 172.21.15.1:0/3860561352 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f921010f240 con 0x7f9210104470 2023-11-16T16:49:36.669 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.663+0000 7f91ff7fe700 1 -- 172.21.15.1:0/3860561352 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9208005740 con 0x7f9210104470 2023-11-16T16:49:36.669 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.663+0000 7f9214e8e700 1 -- 172.21.15.1:0/3860561352 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f921010f0b0 con 0x7f9210104470 2023-11-16T16:49:36.669 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.664+0000 7f9214e8e700 1 -- 172.21.15.1:0/3860561352 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f921010f3d0 con 0x7f9210104470 2023-11-16T16:49:36.670 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.665+0000 7f91ff7fe700 1 -- 172.21.15.1:0/3860561352 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f92080058a0 con 0x7f9210104470 2023-11-16T16:49:36.670 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.666+0000 7f91ff7fe700 1 --2- 172.21.15.1:0/3860561352 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f91f80a1340 0x7f91f80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:49:36.670 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.666+0000 7f920e59c700 1 --2- 172.21.15.1:0/3860561352 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f91f80a1340 0x7f91f80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:49:36.670 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.666+0000 7f91ff7fe700 1 -- 172.21.15.1:0/3860561352 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f92080ccad0 con 0x7f9210104470 2023-11-16T16:49:36.671 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.666+0000 7f920e59c700 1 --2- 172.21.15.1:0/3860561352 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f91f80a1340 0x7f91f80a37f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f920000bc10 tx=0x7f920000b5f0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:49:36.671 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.669+0000 7f91ff7fe700 1 -- 172.21.15.1:0/3860561352 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9208095ed0 con 0x7f9210104470 2023-11-16T16:49:36.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:36 smithi001 ceph-mon[98800]: pgmap v324: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:49:36.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:36 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/886397378' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:49:36.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:36 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1183631661' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:49:36.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:36 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/4083684091' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:49:36.939 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.938+0000 7f9214e8e700 1 -- 172.21.15.1:0/3860561352 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f9210067f40 con 0x7f9210104470 2023-11-16T16:49:36.948 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.947+0000 7f91ff7fe700 1 -- 172.21.15.1:0/3860561352 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) v1 ==== 81+0+733766 (secure 0 0 0) 0x7f92080994f0 con 0x7f9210104470 2023-11-16T16:49:36.958 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:49:36.958 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:49:36.958 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 25, 2023-11-16T16:49:36.958 INFO:teuthology.orchestra.run.smithi001.stdout: "active_gid": 15432, 2023-11-16T16:49:36.958 INFO:teuthology.orchestra.run.smithi001.stdout: "active_name": "x", 2023-11-16T16:49:36.958 INFO:teuthology.orchestra.run.smithi001.stdout: "active_addrs": { 2023-11-16T16:49:36.959 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2023-11-16T16:49:36.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:36.959 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2023-11-16T16:49:36.959 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6800", 2023-11-16T16:49:36.959 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4199419371 2023-11-16T16:49:36.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:36.959 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:36.959 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2023-11-16T16:49:36.960 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6801", 2023-11-16T16:49:36.960 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4199419371 2023-11-16T16:49:36.960 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:36.960 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2023-11-16T16:49:36.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:36.960 INFO:teuthology.orchestra.run.smithi001.stdout: "active_addr": "172.21.15.1:6801/4199419371", 2023-11-16T16:49:36.960 INFO:teuthology.orchestra.run.smithi001.stdout: "active_change": "2023-11-16T16:39:09.360451+0000", 2023-11-16T16:49:36.960 INFO:teuthology.orchestra.run.smithi001.stdout: "active_mgr_features": 4540138322906710015, 2023-11-16T16:49:36.960 INFO:teuthology.orchestra.run.smithi001.stdout: "available": true, 2023-11-16T16:49:36.961 INFO:teuthology.orchestra.run.smithi001.stdout: "standbys": [ 2023-11-16T16:49:36.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:36.961 INFO:teuthology.orchestra.run.smithi001.stdout: "gid": 25076, 2023-11-16T16:49:36.961 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "y", 2023-11-16T16:49:36.961 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_features": 4540138322906710015, 2023-11-16T16:49:36.961 INFO:teuthology.orchestra.run.smithi001.stdout: "available_modules": [ 2023-11-16T16:49:36.961 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:36.961 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "alerts", 2023-11-16T16:49:36.962 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:36.962 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:36.962 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:36.962 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2023-11-16T16:49:36.962 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2023-11-16T16:49:36.962 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:36.962 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:36.962 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:36.963 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2023-11-16T16:49:36.963 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:36.963 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:36.963 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:36.963 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How frequently to reexamine health status", 2023-11-16T16:49:36.963 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:36.963 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:36.963 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:36.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:36.964 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:36.964 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:36.964 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:36.964 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:36.964 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:36.964 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:36.964 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:36.965 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:36.965 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:36.965 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:36.965 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:36.965 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:36.965 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:36.965 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:36.965 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:36.965 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:36.966 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:36.966 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:36.966 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:36.966 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:36.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:36.966 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:36.966 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:36.966 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:36.967 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:36.967 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:36.967 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:36.967 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:36.967 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:36.967 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:36.967 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:36.967 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:36.968 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:36.968 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:36.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:36.968 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:36.968 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:36.968 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:36.968 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:36.968 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:36.968 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:36.969 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:36.969 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:36.969 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:36.969 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:36.969 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:36.969 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:36.969 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:36.969 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:36.970 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:36.970 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:36.970 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:36.970 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:36.971 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:36.971 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:36.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:36.971 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:36.972 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:36.972 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:36.972 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:36.972 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:36.973 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:36.973 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:36.973 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:36.973 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:36.974 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:36.974 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:36.974 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:36.975 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:36.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:36.975 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_destination": { 2023-11-16T16:49:36.975 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_destination", 2023-11-16T16:49:36.976 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:36.976 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:36.976 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:36.976 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:36.977 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:36.977 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:36.977 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:36.977 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email address to send alerts to", 2023-11-16T16:49:36.978 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:36.978 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:36.978 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:36.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:36.979 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_from_name": { 2023-11-16T16:49:36.979 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_from_name", 2023-11-16T16:49:36.979 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:36.980 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:36.980 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:36.980 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "Ceph", 2023-11-16T16:49:36.980 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:36.981 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:36.981 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:36.981 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email From: name", 2023-11-16T16:49:36.981 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:36.982 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:36.982 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:36.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:36.982 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_host": { 2023-11-16T16:49:36.983 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_host", 2023-11-16T16:49:36.983 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:36.983 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:36.983 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:36.984 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:36.984 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:36.984 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:36.984 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:36.984 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP server", 2023-11-16T16:49:36.985 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:36.985 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:36.985 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:36.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:36.985 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_password": { 2023-11-16T16:49:36.985 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_password", 2023-11-16T16:49:36.985 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:36.986 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:36.986 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:36.986 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:36.986 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:36.986 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:36.986 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:36.986 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Password to authenticate with", 2023-11-16T16:49:36.986 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:36.987 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:36.987 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:36.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:36.987 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_port": { 2023-11-16T16:49:36.987 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_port", 2023-11-16T16:49:36.987 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:36.987 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:36.987 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:36.988 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "465", 2023-11-16T16:49:36.988 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:36.988 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:36.988 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:36.988 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP port", 2023-11-16T16:49:36.988 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:36.988 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:36.988 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:36.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:36.989 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_sender": { 2023-11-16T16:49:36.989 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_sender", 2023-11-16T16:49:36.989 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:36.989 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:36.989 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:36.989 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:36.989 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:36.990 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:36.990 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:36.990 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP envelope sender", 2023-11-16T16:49:36.990 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:36.990 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:36.990 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:36.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:36.990 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_ssl": { 2023-11-16T16:49:36.991 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_ssl", 2023-11-16T16:49:36.991 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:36.991 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:36.991 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:36.991 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:36.991 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:36.991 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:36.992 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:36.992 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use SSL to connect to SMTP server", 2023-11-16T16:49:36.992 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:36.992 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:36.992 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:36.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:36.992 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_user": { 2023-11-16T16:49:36.992 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_user", 2023-11-16T16:49:36.992 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:36.993 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:36.993 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:36.993 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:36.993 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:36.993 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:36.993 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:36.993 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "User to authenticate as", 2023-11-16T16:49:36.994 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:36.994 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:36.994 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:36.994 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:36.994 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:36.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:36.994 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:36.994 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "balancer", 2023-11-16T16:49:36.995 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:36.995 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:36.995 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:36.995 INFO:teuthology.orchestra.run.smithi001.stdout: "active": { 2023-11-16T16:49:36.995 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "active", 2023-11-16T16:49:36.995 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:36.995 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:36.996 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:36.996 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:36.996 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:36.996 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:36.996 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:36.996 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically balance PGs across cluster", 2023-11-16T16:49:36.996 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:36.996 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:36.997 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:36.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:36.997 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_time": { 2023-11-16T16:49:36.997 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_time", 2023-11-16T16:49:36.997 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:36.997 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:36.997 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:36.997 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0000", 2023-11-16T16:49:36.998 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:36.998 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:36.998 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:36.998 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "beginning time of day to automatically balance", 2023-11-16T16:49:36.998 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-16T16:49:36.998 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:36.998 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:36.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:36.999 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_weekday": { 2023-11-16T16:49:36.999 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_weekday", 2023-11-16T16:49:36.999 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2023-11-16T16:49:36.999 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:36.999 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:36.999 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2023-11-16T16:49:36.999 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2023-11-16T16:49:37.000 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2023-11-16T16:49:37.000 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.000 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-11-16T16:49:37.000 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-16T16:49:37.000 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.000 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.001 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_max_iterations": { 2023-11-16T16:49:37.001 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_max_iterations", 2023-11-16T16:49:37.001 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2023-11-16T16:49:37.001 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.001 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.001 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "25", 2023-11-16T16:49:37.001 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2023-11-16T16:49:37.001 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "250", 2023-11-16T16:49:37.002 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.002 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-11-16T16:49:37.002 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.002 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.002 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.002 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_metrics": { 2023-11-16T16:49:37.002 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_metrics", 2023-11-16T16:49:37.003 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.003 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.003 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.003 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "pgs,objects,bytes", 2023-11-16T16:49:37.003 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.003 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.003 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.003 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-11-16T16:49:37.004 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-11-16T16:49:37.004 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.004 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.004 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_step": { 2023-11-16T16:49:37.004 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_step", 2023-11-16T16:49:37.004 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2023-11-16T16:49:37.004 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.005 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.005 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.5", 2023-11-16T16:49:37.005 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0.001", 2023-11-16T16:49:37.005 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "0.999", 2023-11-16T16:49:37.005 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.005 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "aggressiveness of optimization", 2023-11-16T16:49:37.005 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-11-16T16:49:37.006 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.006 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.006 INFO:teuthology.orchestra.run.smithi001.stdout: "end_time": { 2023-11-16T16:49:37.006 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_time", 2023-11-16T16:49:37.006 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.006 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.006 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.007 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2359", 2023-11-16T16:49:37.007 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.007 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.007 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.007 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ending time of day to automatically balance", 2023-11-16T16:49:37.007 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-16T16:49:37.007 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.008 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.008 INFO:teuthology.orchestra.run.smithi001.stdout: "end_weekday": { 2023-11-16T16:49:37.008 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_weekday", 2023-11-16T16:49:37.008 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2023-11-16T16:49:37.008 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.008 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.008 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2023-11-16T16:49:37.009 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2023-11-16T16:49:37.009 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2023-11-16T16:49:37.009 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.009 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-11-16T16:49:37.009 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-16T16:49:37.009 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.009 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.010 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:37.010 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:37.010 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.010 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.010 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.010 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.010 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.010 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.011 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.011 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.011 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.011 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.011 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.011 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.011 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.012 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.012 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.012 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.012 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.012 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.012 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:37.012 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:37.013 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.013 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.013 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.013 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.013 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.013 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.013 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.014 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.014 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.014 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.014 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.014 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:37.014 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:37.014 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.015 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.015 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.015 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:37.015 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.015 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.015 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.015 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.015 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.016 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.016 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.016 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.016 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.016 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.016 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.016 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.016 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.017 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.017 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:37.017 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:37.017 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.017 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.017 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.017 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.018 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.018 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.018 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.018 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.018 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.018 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.018 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.019 INFO:teuthology.orchestra.run.smithi001.stdout: "min_score": { 2023-11-16T16:49:37.019 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_score", 2023-11-16T16:49:37.019 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2023-11-16T16:49:37.019 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.019 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.019 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2023-11-16T16:49:37.019 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.019 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.020 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.020 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-11-16T16:49:37.020 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.020 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.020 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.020 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2023-11-16T16:49:37.020 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2023-11-16T16:49:37.021 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.021 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.021 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.021 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "upmap", 2023-11-16T16:49:37.021 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.021 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.021 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.021 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-compat", 2023-11-16T16:49:37.022 INFO:teuthology.orchestra.run.smithi001.stdout: "none", 2023-11-16T16:49:37.022 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap" 2023-11-16T16:49:37.022 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.022 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Balancer mode", 2023-11-16T16:49:37.022 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.022 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.022 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.023 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_ids": { 2023-11-16T16:49:37.023 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_ids", 2023-11-16T16:49:37.023 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.023 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.023 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.023 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.023 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.024 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.024 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.024 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-11-16T16:49:37.024 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.024 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.024 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.024 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2023-11-16T16:49:37.025 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2023-11-16T16:49:37.025 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:37.025 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.025 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.025 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2023-11-16T16:49:37.025 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.025 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.025 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.026 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-11-16T16:49:37.026 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.026 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.026 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.026 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_deviation": { 2023-11-16T16:49:37.026 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_deviation", 2023-11-16T16:49:37.026 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.027 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.027 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.027 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2023-11-16T16:49:37.027 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2023-11-16T16:49:37.027 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.027 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.027 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "deviation below which no optimization is attempted", 2023-11-16T16:49:37.027 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-11-16T16:49:37.028 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.028 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.028 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_optimizations": { 2023-11-16T16:49:37.028 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_optimizations", 2023-11-16T16:49:37.028 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2023-11-16T16:49:37.028 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.028 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.029 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2023-11-16T16:49:37.029 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.029 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.029 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.029 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-11-16T16:49:37.029 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.029 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.029 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.030 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.030 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.030 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:37.030 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cephadm", 2023-11-16T16:49:37.030 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:37.030 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:37.030 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:37.031 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_down_multiplier": { 2023-11-16T16:49:37.031 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_down_multiplier", 2023-11-16T16:49:37.031 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2023-11-16T16:49:37.031 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.031 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.031 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2023-11-16T16:49:37.031 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.031 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.032 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.032 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-11-16T16:49:37.032 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.032 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.032 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.032 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_refresh_rate": { 2023-11-16T16:49:37.033 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_refresh_rate", 2023-11-16T16:49:37.033 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:37.033 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.033 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.033 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "20", 2023-11-16T16:49:37.033 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.033 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.033 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.034 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-11-16T16:49:37.034 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.034 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.034 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.034 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_starting_port": { 2023-11-16T16:49:37.034 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_starting_port", 2023-11-16T16:49:37.034 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.035 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.035 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.035 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4721", 2023-11-16T16:49:37.035 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.035 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.035 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.035 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-11-16T16:49:37.035 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.036 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.036 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.036 INFO:teuthology.orchestra.run.smithi001.stdout: "alertmanager_web_password": { 2023-11-16T16:49:37.036 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "alertmanager_web_password", 2023-11-16T16:49:37.036 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.036 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.036 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.037 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2023-11-16T16:49:37.037 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.037 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.037 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.037 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Alertmanager web password", 2023-11-16T16:49:37.037 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.037 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.037 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.038 INFO:teuthology.orchestra.run.smithi001.stdout: "alertmanager_web_user": { 2023-11-16T16:49:37.038 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "alertmanager_web_user", 2023-11-16T16:49:37.038 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.038 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.038 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.038 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2023-11-16T16:49:37.038 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.039 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.039 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.039 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Alertmanager web user", 2023-11-16T16:49:37.039 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.039 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.039 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.039 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_ptrace": { 2023-11-16T16:49:37.040 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_ptrace", 2023-11-16T16:49:37.040 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.040 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.040 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.040 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.040 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.040 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.040 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.041 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-11-16T16:49:37.041 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-11-16T16:49:37.041 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.041 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.041 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_interval": { 2023-11-16T16:49:37.041 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_interval", 2023-11-16T16:49:37.041 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:37.042 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.042 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.042 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2023-11-16T16:49:37.042 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.042 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.042 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.042 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to autotune daemon memory", 2023-11-16T16:49:37.042 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.043 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.043 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.043 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_memory_target_ratio": { 2023-11-16T16:49:37.043 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_memory_target_ratio", 2023-11-16T16:49:37.043 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2023-11-16T16:49:37.043 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.044 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.044 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.7", 2023-11-16T16:49:37.044 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.044 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.044 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.044 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-11-16T16:49:37.044 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.044 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.045 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.045 INFO:teuthology.orchestra.run.smithi001.stdout: "cgroups_split": { 2023-11-16T16:49:37.045 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cgroups_split", 2023-11-16T16:49:37.045 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.045 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.045 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.045 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.046 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.046 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.046 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.046 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-11-16T16:49:37.046 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.046 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.046 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.047 INFO:teuthology.orchestra.run.smithi001.stdout: "config_checks_enabled": { 2023-11-16T16:49:37.047 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_checks_enabled", 2023-11-16T16:49:37.047 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.047 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.047 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.047 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.047 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.047 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.048 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.048 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-11-16T16:49:37.048 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.048 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.048 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.048 INFO:teuthology.orchestra.run.smithi001.stdout: "config_dashboard": { 2023-11-16T16:49:37.048 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_dashboard", 2023-11-16T16:49:37.049 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.049 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.049 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.049 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.049 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.049 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.049 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.049 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-11-16T16:49:37.050 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.050 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.050 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.050 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_alertmanager": { 2023-11-16T16:49:37.050 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_alertmanager", 2023-11-16T16:49:37.050 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.050 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.051 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.051 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-11-16T16:49:37.051 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.051 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.051 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.051 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2023-11-16T16:49:37.051 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.051 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.052 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.052 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_base": { 2023-11-16T16:49:37.052 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_base", 2023-11-16T16:49:37.052 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.052 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.052 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.052 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/ceph", 2023-11-16T16:49:37.053 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.053 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.053 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.053 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Container image name, without the tag", 2023-11-16T16:49:37.053 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.053 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.053 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.054 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_elasticsearch": { 2023-11-16T16:49:37.054 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_elasticsearch", 2023-11-16T16:49:37.054 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.054 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.054 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.054 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-11-16T16:49:37.054 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.054 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.055 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.055 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "elasticsearch container image", 2023-11-16T16:49:37.055 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.055 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.055 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.055 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_grafana": { 2023-11-16T16:49:37.056 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_grafana", 2023-11-16T16:49:37.056 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.056 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.056 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.056 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-11-16T16:49:37.056 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.056 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.056 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.057 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2023-11-16T16:49:37.057 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.057 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.057 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.057 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_haproxy": { 2023-11-16T16:49:37.057 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_haproxy", 2023-11-16T16:49:37.058 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.058 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.058 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.058 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-11-16T16:49:37.058 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.058 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.058 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.058 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "HAproxy container image", 2023-11-16T16:49:37.059 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.059 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.059 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.059 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_agent": { 2023-11-16T16:49:37.059 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_agent", 2023-11-16T16:49:37.059 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.059 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.060 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.060 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-11-16T16:49:37.060 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.060 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.060 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.060 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger agent container image", 2023-11-16T16:49:37.060 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.060 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.061 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.061 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_collector": { 2023-11-16T16:49:37.061 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_collector", 2023-11-16T16:49:37.061 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.061 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.061 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.061 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-11-16T16:49:37.062 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.062 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.062 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.062 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger collector container image", 2023-11-16T16:49:37.062 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.062 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.062 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.063 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_query": { 2023-11-16T16:49:37.063 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_query", 2023-11-16T16:49:37.063 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.063 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.063 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.063 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-11-16T16:49:37.063 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.063 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.064 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.064 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger query container image", 2023-11-16T16:49:37.064 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.064 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.064 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.064 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_keepalived": { 2023-11-16T16:49:37.064 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_keepalived", 2023-11-16T16:49:37.065 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.065 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.065 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.065 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-11-16T16:49:37.065 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.065 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.065 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.065 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Keepalived container image", 2023-11-16T16:49:37.066 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.066 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.066 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.066 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_loki": { 2023-11-16T16:49:37.066 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_loki", 2023-11-16T16:49:37.066 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.066 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.067 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.067 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-11-16T16:49:37.067 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.067 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.067 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.067 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Loki container image", 2023-11-16T16:49:37.067 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.067 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.068 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.068 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_node_exporter": { 2023-11-16T16:49:37.068 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_node_exporter", 2023-11-16T16:49:37.068 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.068 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.068 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.068 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-11-16T16:49:37.069 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.069 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.069 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.069 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2023-11-16T16:49:37.069 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.069 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.069 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.069 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.070 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_prometheus": { 2023-11-16T16:49:37.070 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_prometheus", 2023-11-16T16:49:37.070 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.070 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.070 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.070 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-11-16T16:49:37.070 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.070 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.071 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.071 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2023-11-16T16:49:37.071 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.071 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.071 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.071 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_promtail": { 2023-11-16T16:49:37.071 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_promtail", 2023-11-16T16:49:37.072 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.072 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.072 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.072 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-11-16T16:49:37.072 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.072 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.072 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.072 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Promtail container image", 2023-11-16T16:49:37.072 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.073 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.073 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.073 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_snmp_gateway": { 2023-11-16T16:49:37.073 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_snmp_gateway", 2023-11-16T16:49:37.073 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.073 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.074 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.074 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-11-16T16:49:37.074 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.074 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.074 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.074 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SNMP Gateway container image", 2023-11-16T16:49:37.074 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.074 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.075 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.075 INFO:teuthology.orchestra.run.smithi001.stdout: "container_init": { 2023-11-16T16:49:37.075 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_init", 2023-11-16T16:49:37.075 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.075 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.075 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.075 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.076 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.076 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.076 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.076 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Run podman/docker with `--init`", 2023-11-16T16:49:37.076 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.076 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.076 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.077 INFO:teuthology.orchestra.run.smithi001.stdout: "daemon_cache_timeout": { 2023-11-16T16:49:37.077 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "daemon_cache_timeout", 2023-11-16T16:49:37.077 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:37.077 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.077 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.077 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2023-11-16T16:49:37.077 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.077 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.078 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.078 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-11-16T16:49:37.078 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.078 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.078 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.078 INFO:teuthology.orchestra.run.smithi001.stdout: "default_cephadm_command_timeout": { 2023-11-16T16:49:37.078 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_cephadm_command_timeout", 2023-11-16T16:49:37.079 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:37.079 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.079 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.079 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "900", 2023-11-16T16:49:37.079 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.079 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.079 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.079 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-11-16T16:49:37.080 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.080 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.080 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.080 INFO:teuthology.orchestra.run.smithi001.stdout: "default_registry": { 2023-11-16T16:49:37.080 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_registry", 2023-11-16T16:49:37.080 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.081 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.081 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.081 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io", 2023-11-16T16:49:37.081 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.081 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.081 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.081 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-11-16T16:49:37.081 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.082 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.082 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.082 INFO:teuthology.orchestra.run.smithi001.stdout: "device_cache_timeout": { 2023-11-16T16:49:37.082 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_cache_timeout", 2023-11-16T16:49:37.082 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:37.082 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.082 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.083 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1800", 2023-11-16T16:49:37.083 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.083 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.083 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.083 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache device inventory", 2023-11-16T16:49:37.083 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.083 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.084 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.084 INFO:teuthology.orchestra.run.smithi001.stdout: "device_enhanced_scan": { 2023-11-16T16:49:37.084 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_enhanced_scan", 2023-11-16T16:49:37.084 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.084 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.084 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.085 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.085 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.085 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.085 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.085 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use libstoragemgmt during device scans", 2023-11-16T16:49:37.085 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.085 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.085 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.086 INFO:teuthology.orchestra.run.smithi001.stdout: "facts_cache_timeout": { 2023-11-16T16:49:37.086 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "facts_cache_timeout", 2023-11-16T16:49:37.086 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:37.086 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.086 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.086 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2023-11-16T16:49:37.086 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.087 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.087 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.087 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache host facts data", 2023-11-16T16:49:37.087 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.087 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.087 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.087 INFO:teuthology.orchestra.run.smithi001.stdout: "host_check_interval": { 2023-11-16T16:49:37.088 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "host_check_interval", 2023-11-16T16:49:37.088 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:37.088 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.088 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.088 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2023-11-16T16:49:37.088 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.088 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.088 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.089 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to perform a host check", 2023-11-16T16:49:37.089 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.089 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.089 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.089 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:37.089 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:37.089 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.090 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.090 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.090 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.090 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.090 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.090 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.090 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.090 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.091 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.091 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.091 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.091 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.091 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.091 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.091 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.091 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.092 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.092 INFO:teuthology.orchestra.run.smithi001.stdout: "log_refresh_metadata": { 2023-11-16T16:49:37.092 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_refresh_metadata", 2023-11-16T16:49:37.092 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.092 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.092 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.092 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.093 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.093 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.093 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.093 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-11-16T16:49:37.093 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.093 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.093 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.094 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:37.094 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:37.094 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.094 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.094 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.094 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.094 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.095 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.095 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.095 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-11-16T16:49:37.095 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.095 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.095 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.095 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:37.096 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:37.096 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.096 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.096 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.096 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:37.096 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.096 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.097 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.097 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.097 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.097 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.097 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.097 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.097 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.097 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.098 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.098 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.098 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.098 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.098 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:37.098 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:37.098 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.099 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.099 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.099 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.099 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.099 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.099 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.099 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.099 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.100 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.100 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.100 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf": { 2023-11-16T16:49:37.100 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-11-16T16:49:37.100 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.100 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.100 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.101 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.101 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.101 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.101 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.101 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-11-16T16:49:37.101 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.101 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.102 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.102 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-11-16T16:49:37.102 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-11-16T16:49:37.102 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.102 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.102 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.102 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "*", 2023-11-16T16:49:37.103 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.103 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.103 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.103 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-11-16T16:49:37.103 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.103 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.103 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.104 INFO:teuthology.orchestra.run.smithi001.stdout: "max_count_per_host": { 2023-11-16T16:49:37.104 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_count_per_host", 2023-11-16T16:49:37.104 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.104 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.104 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.104 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2023-11-16T16:49:37.104 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.104 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.105 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.105 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of daemons per service per host", 2023-11-16T16:49:37.105 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.105 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.105 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.105 INFO:teuthology.orchestra.run.smithi001.stdout: "max_osd_draining_count": { 2023-11-16T16:49:37.105 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_osd_draining_count", 2023-11-16T16:49:37.106 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.106 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.106 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.106 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2023-11-16T16:49:37.106 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.106 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.106 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.107 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-11-16T16:49:37.107 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.107 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.107 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.107 INFO:teuthology.orchestra.run.smithi001.stdout: "migration_current": { 2023-11-16T16:49:37.107 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "migration_current", 2023-11-16T16:49:37.108 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.108 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.108 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.108 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.108 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.108 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.108 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.108 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "internal - do not modify", 2023-11-16T16:49:37.108 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.109 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.109 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.109 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2023-11-16T16:49:37.109 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2023-11-16T16:49:37.109 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.109 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.110 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.110 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "root", 2023-11-16T16:49:37.110 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.110 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.110 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.110 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm-package", 2023-11-16T16:49:37.110 INFO:teuthology.orchestra.run.smithi001.stdout: "root" 2023-11-16T16:49:37.110 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.111 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "mode for remote execution of cephadm", 2023-11-16T16:49:37.111 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.111 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.111 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.111 INFO:teuthology.orchestra.run.smithi001.stdout: "prometheus_alerts_path": { 2023-11-16T16:49:37.111 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus_alerts_path", 2023-11-16T16:49:37.112 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.112 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.112 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.112 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-11-16T16:49:37.112 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.112 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.112 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.112 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-11-16T16:49:37.113 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.113 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.113 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.113 INFO:teuthology.orchestra.run.smithi001.stdout: "prometheus_web_password": { 2023-11-16T16:49:37.113 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus_web_password", 2023-11-16T16:49:37.113 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.113 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.114 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.114 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2023-11-16T16:49:37.114 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.114 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.114 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.114 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus web password", 2023-11-16T16:49:37.114 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.114 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.115 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.115 INFO:teuthology.orchestra.run.smithi001.stdout: "prometheus_web_user": { 2023-11-16T16:49:37.115 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus_web_user", 2023-11-16T16:49:37.115 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.115 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.115 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.115 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2023-11-16T16:49:37.116 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.116 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.116 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.116 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus web user", 2023-11-16T16:49:37.116 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.116 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.116 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.117 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_insecure": { 2023-11-16T16:49:37.117 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_insecure", 2023-11-16T16:49:37.117 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.117 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.117 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.117 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.117 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.117 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.118 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.118 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-11-16T16:49:37.118 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.118 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.118 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.119 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_password": { 2023-11-16T16:49:37.119 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_password", 2023-11-16T16:49:37.119 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.119 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.119 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.119 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.119 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.120 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.120 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.120 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-11-16T16:49:37.120 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.120 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.120 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.120 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_url": { 2023-11-16T16:49:37.121 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_url", 2023-11-16T16:49:37.121 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.121 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.121 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.121 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.121 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.121 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.121 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.122 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-11-16T16:49:37.122 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.122 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.122 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.122 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_username": { 2023-11-16T16:49:37.122 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_username", 2023-11-16T16:49:37.122 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.123 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.123 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.123 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.123 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.123 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.123 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.123 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-11-16T16:49:37.123 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.124 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.124 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.124 INFO:teuthology.orchestra.run.smithi001.stdout: "secure_monitoring_stack": { 2023-11-16T16:49:37.124 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "secure_monitoring_stack", 2023-11-16T16:49:37.124 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.124 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.125 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.125 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.125 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.125 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.125 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.125 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-11-16T16:49:37.125 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.125 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.126 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.126 INFO:teuthology.orchestra.run.smithi001.stdout: "service_discovery_port": { 2023-11-16T16:49:37.126 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "service_discovery_port", 2023-11-16T16:49:37.126 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.126 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.126 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.127 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8765", 2023-11-16T16:49:37.127 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.127 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.127 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.127 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "cephadm service discovery port", 2023-11-16T16:49:37.127 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.127 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.127 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.128 INFO:teuthology.orchestra.run.smithi001.stdout: "ssh_config_file": { 2023-11-16T16:49:37.128 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssh_config_file", 2023-11-16T16:49:37.128 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.128 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.128 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.128 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.128 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.129 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.129 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.129 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-11-16T16:49:37.129 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.129 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.129 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.129 INFO:teuthology.orchestra.run.smithi001.stdout: "use_agent": { 2023-11-16T16:49:37.130 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_agent", 2023-11-16T16:49:37.130 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.130 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.130 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.130 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.130 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.130 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.130 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.131 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-11-16T16:49:37.131 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.131 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.131 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.131 INFO:teuthology.orchestra.run.smithi001.stdout: "use_repo_digest": { 2023-11-16T16:49:37.131 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_repo_digest", 2023-11-16T16:49:37.131 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.132 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.132 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.132 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.132 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.132 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.132 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.132 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-11-16T16:49:37.132 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.133 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.133 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.133 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_failed_host_check": { 2023-11-16T16:49:37.133 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_failed_host_check", 2023-11-16T16:49:37.133 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.133 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.133 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.133 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.134 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.134 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.134 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.134 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if the host check fails", 2023-11-16T16:49:37.134 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.134 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.134 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.135 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_daemons": { 2023-11-16T16:49:37.135 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_daemons", 2023-11-16T16:49:37.135 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.135 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.135 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.135 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.135 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.135 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.136 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.136 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-11-16T16:49:37.136 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.136 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.136 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.136 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_hosts": { 2023-11-16T16:49:37.136 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_hosts", 2023-11-16T16:49:37.137 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.137 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.137 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.137 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.137 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.137 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.137 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.137 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-11-16T16:49:37.137 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.138 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.138 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.138 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.138 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.138 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:37.138 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crash", 2023-11-16T16:49:37.138 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:37.139 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:37.139 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:37.139 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:37.139 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:37.139 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.139 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.139 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.139 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.140 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.140 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.140 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.140 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.140 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.140 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.140 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.140 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.140 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.141 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.141 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.141 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.141 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.141 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.141 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:37.141 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:37.142 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.142 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.142 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.142 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.142 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.142 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.142 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.142 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.143 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.143 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.143 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.143 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:37.143 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:37.143 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.143 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.144 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.144 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:37.144 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.144 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.144 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.144 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.144 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.144 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.145 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.145 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.145 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.145 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.145 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.145 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.145 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.145 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.146 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:37.146 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:37.146 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.146 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.146 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.146 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.147 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.147 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.147 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.147 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.147 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.147 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.147 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.148 INFO:teuthology.orchestra.run.smithi001.stdout: "retain_interval": { 2023-11-16T16:49:37.148 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retain_interval", 2023-11-16T16:49:37.148 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:37.148 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.148 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.148 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "31536000", 2023-11-16T16:49:37.148 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.148 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.149 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.149 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain crashes before pruning them", 2023-11-16T16:49:37.149 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.149 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.149 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.149 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_recent_interval": { 2023-11-16T16:49:37.149 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_recent_interval", 2023-11-16T16:49:37.150 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:37.150 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.150 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.150 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1209600", 2023-11-16T16:49:37.150 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.150 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.150 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.150 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "time interval in which to warn about recent crashes", 2023-11-16T16:49:37.151 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.151 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.151 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.151 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.151 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.151 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:37.151 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dashboard", 2023-11-16T16:49:37.152 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:37.152 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:37.152 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:37.152 INFO:teuthology.orchestra.run.smithi001.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-11-16T16:49:37.152 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-11-16T16:49:37.152 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.152 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.152 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.153 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2023-11-16T16:49:37.153 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.153 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.153 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.153 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.153 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.153 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.153 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.154 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_HOST": { 2023-11-16T16:49:37.154 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_HOST", 2023-11-16T16:49:37.154 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.154 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.154 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.154 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.154 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.155 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.155 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.155 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.155 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.155 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.155 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.156 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-11-16T16:49:37.156 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-11-16T16:49:37.156 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.156 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.156 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.156 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.156 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.156 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.157 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.157 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.157 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.157 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.157 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.157 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_ENABLED": { 2023-11-16T16:49:37.158 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_ENABLED", 2023-11-16T16:49:37.158 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.158 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.158 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.158 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.158 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.158 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.158 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.159 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.159 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.159 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.159 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.159 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-11-16T16:49:37.159 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-11-16T16:49:37.159 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.160 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.160 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.160 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.160 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.160 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.160 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.160 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.160 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.161 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.161 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.161 INFO:teuthology.orchestra.run.smithi001.stdout: "ENABLE_BROWSABLE_API": { 2023-11-16T16:49:37.161 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ENABLE_BROWSABLE_API", 2023-11-16T16:49:37.161 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.161 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.161 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.162 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.162 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.162 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.162 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.162 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.162 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.162 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.162 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.163 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-11-16T16:49:37.163 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-11-16T16:49:37.163 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.163 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.163 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.163 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.163 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.164 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.164 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.164 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.164 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.164 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.164 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.164 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-11-16T16:49:37.165 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-11-16T16:49:37.165 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.165 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.165 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.165 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.165 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.165 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.165 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.166 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.166 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.166 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.166 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.166 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-11-16T16:49:37.166 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-11-16T16:49:37.166 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.167 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.167 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.167 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.167 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.167 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.167 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.167 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.167 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.168 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.168 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.168 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-11-16T16:49:37.168 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-11-16T16:49:37.168 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.168 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.168 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.169 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.169 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.169 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.169 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.169 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.169 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.169 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.169 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.170 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_NFS": { 2023-11-16T16:49:37.170 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-11-16T16:49:37.170 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.170 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.170 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.170 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.171 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.171 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.171 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.171 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.171 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.171 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.171 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.172 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RBD": { 2023-11-16T16:49:37.172 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-11-16T16:49:37.172 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.172 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.172 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.172 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.172 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.172 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.173 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.173 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.173 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.173 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.173 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.173 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RGW": { 2023-11-16T16:49:37.173 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-11-16T16:49:37.174 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.174 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.174 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.174 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.174 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.174 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.174 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.174 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.175 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.175 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.175 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.175 INFO:teuthology.orchestra.run.smithi001.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-11-16T16:49:37.175 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-11-16T16:49:37.175 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.176 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.176 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.176 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.176 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.176 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.176 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.176 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.176 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.176 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.177 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.177 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_PASSWORD": { 2023-11-16T16:49:37.177 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_PASSWORD", 2023-11-16T16:49:37.177 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.177 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.177 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.178 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2023-11-16T16:49:37.178 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.178 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.178 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.178 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.178 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.178 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.178 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.179 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-11-16T16:49:37.179 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-11-16T16:49:37.179 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.179 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.179 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.179 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.179 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.180 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.180 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.180 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.180 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.180 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.180 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.180 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_URL": { 2023-11-16T16:49:37.181 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_URL", 2023-11-16T16:49:37.181 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.181 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.181 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.181 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.181 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.181 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.181 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.182 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.182 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.182 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.182 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.182 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_USERNAME": { 2023-11-16T16:49:37.182 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_USERNAME", 2023-11-16T16:49:37.182 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.183 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.183 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.183 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2023-11-16T16:49:37.183 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.183 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.183 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.183 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.184 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.184 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.184 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.184 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-11-16T16:49:37.184 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-11-16T16:49:37.184 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.185 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.185 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.185 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.185 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.185 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.185 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.185 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.185 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.186 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.186 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.186 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-11-16T16:49:37.186 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-11-16T16:49:37.186 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.186 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.186 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.187 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.187 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.187 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.187 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.187 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.187 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.187 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.187 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.188 INFO:teuthology.orchestra.run.smithi001.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-11-16T16:49:37.188 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-11-16T16:49:37.188 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.188 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.188 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.188 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.188 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.189 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.189 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.189 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.189 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.189 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.189 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.189 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.189 INFO:teuthology.orchestra.run.smithi001.stdout: "ISSUE_TRACKER_API_KEY": { 2023-11-16T16:49:37.190 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-11-16T16:49:37.190 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.190 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.190 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.190 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.190 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.190 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.190 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.191 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.191 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.191 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.191 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.191 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_HOST": { 2023-11-16T16:49:37.191 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_HOST", 2023-11-16T16:49:37.191 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.192 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.192 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.192 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.192 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.192 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.192 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.192 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.192 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.193 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.193 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.193 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-11-16T16:49:37.193 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-11-16T16:49:37.193 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.193 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.194 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.194 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.194 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.194 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.194 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.194 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.194 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.194 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.195 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.195 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-11-16T16:49:37.195 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-11-16T16:49:37.195 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.195 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.195 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.195 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.196 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.196 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.196 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.196 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.196 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.196 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.196 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.197 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-11-16T16:49:37.197 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-11-16T16:49:37.197 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.197 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.197 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.197 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.197 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.198 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.198 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.198 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.198 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.198 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.198 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.198 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-11-16T16:49:37.199 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-11-16T16:49:37.199 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.199 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.199 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.199 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.199 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.199 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.199 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.200 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.200 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.200 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.200 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.200 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-11-16T16:49:37.200 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-11-16T16:49:37.200 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.201 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.201 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.201 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.201 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.201 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.201 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.201 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.201 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.202 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.202 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.202 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-11-16T16:49:37.202 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-11-16T16:49:37.202 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.202 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.203 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.203 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.203 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.203 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.203 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.203 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.203 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.203 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.204 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.204 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-11-16T16:49:37.204 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-11-16T16:49:37.204 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.204 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.204 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.204 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.205 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.205 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.205 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.205 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.205 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.205 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.205 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.206 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-11-16T16:49:37.206 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-11-16T16:49:37.206 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.206 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.207 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.207 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.207 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.207 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.208 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.208 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.208 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.208 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.208 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.208 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_ENABLED": { 2023-11-16T16:49:37.208 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_ENABLED", 2023-11-16T16:49:37.209 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.209 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.209 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.209 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.210 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.210 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.210 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.210 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.211 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.211 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.211 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.212 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-11-16T16:49:37.212 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-11-16T16:49:37.212 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.212 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.213 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.213 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-11-16T16:49:37.213 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.213 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.213 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.213 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.213 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.214 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.214 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.214 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-11-16T16:49:37.214 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-11-16T16:49:37.214 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.214 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.214 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.215 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2023-11-16T16:49:37.215 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.215 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.215 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.215 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.215 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.215 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.215 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.216 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-11-16T16:49:37.216 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-11-16T16:49:37.216 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.216 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.216 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.216 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8", 2023-11-16T16:49:37.216 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.217 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.217 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.217 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.217 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.217 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.217 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.217 INFO:teuthology.orchestra.run.smithi001.stdout: "REST_REQUESTS_TIMEOUT": { 2023-11-16T16:49:37.218 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-11-16T16:49:37.218 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.218 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.218 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.218 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "45", 2023-11-16T16:49:37.218 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.218 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.218 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.219 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.219 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.219 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.219 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.219 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ACCESS_KEY": { 2023-11-16T16:49:37.219 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ACCESS_KEY", 2023-11-16T16:49:37.219 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.220 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.220 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.220 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.220 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.220 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.220 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.220 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.220 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.221 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.221 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.221 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-11-16T16:49:37.221 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-11-16T16:49:37.221 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.221 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.221 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.222 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2023-11-16T16:49:37.222 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.222 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.222 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.222 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.222 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.222 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.222 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.223 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SECRET_KEY": { 2023-11-16T16:49:37.223 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SECRET_KEY", 2023-11-16T16:49:37.223 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.223 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.223 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.223 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.224 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.224 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.224 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.224 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.224 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.224 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.224 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.225 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SSL_VERIFY": { 2023-11-16T16:49:37.225 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SSL_VERIFY", 2023-11-16T16:49:37.225 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.225 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.225 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.225 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.225 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.225 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.226 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.226 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.226 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.226 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.226 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.226 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-11-16T16:49:37.226 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-11-16T16:49:37.227 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.227 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.227 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.227 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2023-11-16T16:49:37.227 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.227 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.227 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.227 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.228 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.228 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.228 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.229 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-11-16T16:49:37.229 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-11-16T16:49:37.229 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.229 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.230 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.230 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2023-11-16T16:49:37.230 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.230 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.230 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.231 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.231 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.231 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.231 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.231 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-11-16T16:49:37.231 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-11-16T16:49:37.231 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.232 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.232 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.232 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2023-11-16T16:49:37.232 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.232 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.232 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.232 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.232 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.233 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.233 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.233 INFO:teuthology.orchestra.run.smithi001.stdout: "cross_origin_url": { 2023-11-16T16:49:37.233 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cross_origin_url", 2023-11-16T16:49:37.233 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.233 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.233 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.234 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.234 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.234 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.234 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.234 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.234 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.234 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.234 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.235 INFO:teuthology.orchestra.run.smithi001.stdout: "crt_file": { 2023-11-16T16:49:37.235 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crt_file", 2023-11-16T16:49:37.235 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.235 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.236 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.236 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.236 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.236 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.236 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.236 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.237 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.237 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.237 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.237 INFO:teuthology.orchestra.run.smithi001.stdout: "debug": { 2023-11-16T16:49:37.237 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "debug", 2023-11-16T16:49:37.237 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.237 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.238 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.238 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.238 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.238 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.238 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.238 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable/disable debug options", 2023-11-16T16:49:37.238 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.238 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.239 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.239 INFO:teuthology.orchestra.run.smithi001.stdout: "jwt_token_ttl": { 2023-11-16T16:49:37.239 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "jwt_token_ttl", 2023-11-16T16:49:37.239 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.239 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.239 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.239 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "28800", 2023-11-16T16:49:37.240 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.240 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.240 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.240 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.240 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.240 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.240 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.241 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2023-11-16T16:49:37.241 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2023-11-16T16:49:37.241 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.241 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.241 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.241 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.241 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.241 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.242 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.242 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.242 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.242 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.242 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.242 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:37.242 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:37.243 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.243 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.243 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.243 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.243 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.243 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.243 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.243 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.244 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.244 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.244 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.244 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.244 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.244 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.244 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.245 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.245 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.245 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.245 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:37.245 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:37.245 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.245 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.246 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.246 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.246 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.246 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.246 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.246 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.246 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.246 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.247 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.247 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:37.247 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:37.247 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.247 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.247 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.247 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:37.248 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.248 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.248 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.248 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.248 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.248 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.248 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.248 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.249 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.249 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.249 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.249 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.249 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.249 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.249 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:37.250 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:37.250 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.250 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.250 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.250 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.250 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.250 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.251 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.251 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.251 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.251 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.251 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.251 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.251 INFO:teuthology.orchestra.run.smithi001.stdout: "motd": { 2023-11-16T16:49:37.251 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "motd", 2023-11-16T16:49:37.252 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.252 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.252 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.252 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.252 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.252 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.252 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.252 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "The message of the day", 2023-11-16T16:49:37.252 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.253 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.253 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.253 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect_resolve_ip_addr": { 2023-11-16T16:49:37.253 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "redirect_resolve_ip_addr", 2023-11-16T16:49:37.253 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.253 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.253 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.254 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.254 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.254 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.254 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.254 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.254 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.254 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.255 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.255 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2023-11-16T16:49:37.255 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2023-11-16T16:49:37.255 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.255 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.255 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.255 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2023-11-16T16:49:37.256 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.256 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.256 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.256 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.256 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.256 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.256 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.257 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2023-11-16T16:49:37.257 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2023-11-16T16:49:37.257 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.257 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.257 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.257 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8080", 2023-11-16T16:49:37.257 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.258 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.258 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.258 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.258 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.258 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.258 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.258 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2023-11-16T16:49:37.259 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2023-11-16T16:49:37.259 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.259 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.259 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.259 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.259 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.259 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.259 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.260 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.260 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.260 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.260 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.260 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl_server_port": { 2023-11-16T16:49:37.260 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl_server_port", 2023-11-16T16:49:37.260 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.261 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.261 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.261 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8443", 2023-11-16T16:49:37.261 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.261 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.261 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.261 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.261 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.262 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.262 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.262 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2023-11-16T16:49:37.262 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2023-11-16T16:49:37.262 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.262 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.262 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.263 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "redirect", 2023-11-16T16:49:37.263 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.263 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.263 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.263 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.263 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect" 2023-11-16T16:49:37.263 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.264 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.264 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.264 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.264 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.264 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2023-11-16T16:49:37.264 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2023-11-16T16:49:37.264 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.265 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.265 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.265 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2023-11-16T16:49:37.265 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2023-11-16T16:49:37.265 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2023-11-16T16:49:37.265 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.265 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.265 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.266 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.266 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.266 INFO:teuthology.orchestra.run.smithi001.stdout: "url_prefix": { 2023-11-16T16:49:37.266 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url_prefix", 2023-11-16T16:49:37.266 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.266 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.266 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.267 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.267 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.267 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.267 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.267 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.267 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.267 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.267 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.268 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.268 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:37.268 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "devicehealth", 2023-11-16T16:49:37.268 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:37.268 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:37.268 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:37.269 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_monitoring": { 2023-11-16T16:49:37.269 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_monitoring", 2023-11-16T16:49:37.269 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.269 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.269 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.269 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.269 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.270 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.270 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.270 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "monitor device health metrics", 2023-11-16T16:49:37.270 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.270 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.270 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.270 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:37.271 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:37.271 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.271 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.271 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.271 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.271 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.271 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.271 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.272 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.272 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.272 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.272 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.272 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.272 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.272 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.272 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.273 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.273 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.273 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.273 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:37.273 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:37.273 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.273 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.274 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.274 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.274 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.274 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.274 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.274 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.274 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.274 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.275 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.275 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:37.275 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:37.275 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.275 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.275 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.275 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:37.276 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.276 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.276 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.276 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.276 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.276 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.276 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.276 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.277 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.277 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.277 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.277 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.277 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.277 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.278 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:37.278 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:37.278 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.278 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.278 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.278 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.278 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.278 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.279 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.279 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.279 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.279 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.279 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.279 INFO:teuthology.orchestra.run.smithi001.stdout: "mark_out_threshold": { 2023-11-16T16:49:37.279 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mark_out_threshold", 2023-11-16T16:49:37.280 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:37.280 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.280 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.280 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2419200", 2023-11-16T16:49:37.280 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.280 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.280 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.280 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-11-16T16:49:37.281 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.281 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.281 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.281 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": { 2023-11-16T16:49:37.281 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_name", 2023-11-16T16:49:37.281 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.281 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.282 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.282 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "device_health_metrics", 2023-11-16T16:49:37.282 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.282 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.282 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.282 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name of pool in which to store device health metrics", 2023-11-16T16:49:37.282 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.282 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.283 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.283 INFO:teuthology.orchestra.run.smithi001.stdout: "retention_period": { 2023-11-16T16:49:37.283 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retention_period", 2023-11-16T16:49:37.283 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:37.283 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.283 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.283 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15552000", 2023-11-16T16:49:37.284 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.284 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.284 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.284 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain device health metrics", 2023-11-16T16:49:37.284 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.284 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.284 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.285 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_frequency": { 2023-11-16T16:49:37.285 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_frequency", 2023-11-16T16:49:37.285 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:37.285 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.285 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.285 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2023-11-16T16:49:37.286 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.286 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.286 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.286 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to scrape device health metrics", 2023-11-16T16:49:37.286 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.286 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.286 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.287 INFO:teuthology.orchestra.run.smithi001.stdout: "self_heal": { 2023-11-16T16:49:37.287 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "self_heal", 2023-11-16T16:49:37.287 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.287 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.287 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.287 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.287 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.287 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.288 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.288 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-11-16T16:49:37.288 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.288 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.288 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.288 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2023-11-16T16:49:37.288 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2023-11-16T16:49:37.288 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:37.289 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.289 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.289 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2023-11-16T16:49:37.289 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.289 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.289 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.289 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and check device health", 2023-11-16T16:49:37.289 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.290 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.290 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.290 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_threshold": { 2023-11-16T16:49:37.290 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_threshold", 2023-11-16T16:49:37.290 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:37.290 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.291 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.291 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7257600", 2023-11-16T16:49:37.291 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.291 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.291 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.291 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-11-16T16:49:37.291 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.291 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.292 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.292 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.292 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.292 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:37.292 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "diskprediction_local", 2023-11-16T16:49:37.292 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:37.292 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:37.293 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:37.293 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:37.293 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:37.293 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.293 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.293 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.293 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.294 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.294 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.294 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.294 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.294 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.294 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.294 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.294 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.295 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.295 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.295 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.295 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.295 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.295 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.295 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:37.296 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:37.296 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.296 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.296 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.296 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.296 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.296 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.296 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.297 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.297 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.297 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.297 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.297 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:37.297 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:37.298 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.298 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.298 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.298 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:37.298 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.298 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.298 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.298 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.299 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.299 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.299 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.299 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.299 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.299 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.299 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.299 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.300 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.300 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.300 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:37.300 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:37.300 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.300 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.300 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.301 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.301 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.301 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.301 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.301 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.301 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.301 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.302 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.302 INFO:teuthology.orchestra.run.smithi001.stdout: "predict_interval": { 2023-11-16T16:49:37.302 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predict_interval", 2023-11-16T16:49:37.302 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.302 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.302 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.302 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2023-11-16T16:49:37.303 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.303 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.303 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.303 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.303 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.303 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.303 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.304 INFO:teuthology.orchestra.run.smithi001.stdout: "predictor_model": { 2023-11-16T16:49:37.304 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predictor_model", 2023-11-16T16:49:37.304 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.304 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.304 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.304 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "prophetstor", 2023-11-16T16:49:37.304 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.304 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.305 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.305 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.305 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.305 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.305 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.305 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2023-11-16T16:49:37.306 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2023-11-16T16:49:37.306 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.306 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.306 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.306 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2023-11-16T16:49:37.306 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.306 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.306 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.307 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.307 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.307 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.307 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.307 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.307 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.307 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:37.308 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "influx", 2023-11-16T16:49:37.308 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": false, 2023-11-16T16:49:37.308 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "influxdb python module not found", 2023-11-16T16:49:37.308 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:37.308 INFO:teuthology.orchestra.run.smithi001.stdout: "batch_size": { 2023-11-16T16:49:37.308 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "batch_size", 2023-11-16T16:49:37.308 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.308 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.309 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.309 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5000", 2023-11-16T16:49:37.309 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.309 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.309 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.309 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-11-16T16:49:37.309 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.309 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.310 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.310 INFO:teuthology.orchestra.run.smithi001.stdout: "database": { 2023-11-16T16:49:37.310 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "database", 2023-11-16T16:49:37.310 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.310 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.310 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.310 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "ceph", 2023-11-16T16:49:37.311 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.313 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.313 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.313 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-11-16T16:49:37.313 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.314 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.314 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.314 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": { 2023-11-16T16:49:37.314 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "hostname", 2023-11-16T16:49:37.314 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.314 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.315 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.315 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.315 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.315 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.315 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.315 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server hostname", 2023-11-16T16:49:37.315 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.315 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.316 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.316 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.316 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2023-11-16T16:49:37.316 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2023-11-16T16:49:37.316 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:37.316 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.316 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.316 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "30", 2023-11-16T16:49:37.317 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "5", 2023-11-16T16:49:37.317 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.317 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.317 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-11-16T16:49:37.317 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.317 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.317 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.318 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:37.318 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:37.318 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.318 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.318 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.318 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.318 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.318 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.319 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.319 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.319 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.319 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.319 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.319 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.319 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.319 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.320 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.320 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.320 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.320 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.320 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:37.320 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:37.320 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.321 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.321 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.321 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.321 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.321 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.321 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.321 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.322 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.322 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.322 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.322 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:37.322 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:37.322 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.322 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.323 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.323 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:37.323 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.323 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.323 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.323 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.323 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.323 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.324 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.324 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.324 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.324 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.324 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.324 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.324 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.325 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.325 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:37.325 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:37.325 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.325 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.325 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.325 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.326 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.326 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.326 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.326 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.326 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.326 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.326 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.327 INFO:teuthology.orchestra.run.smithi001.stdout: "password": { 2023-11-16T16:49:37.327 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "password", 2023-11-16T16:49:37.327 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.327 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.327 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.327 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.327 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.328 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.328 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.328 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "password of InfluxDB server user", 2023-11-16T16:49:37.328 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.328 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.328 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.328 INFO:teuthology.orchestra.run.smithi001.stdout: "port": { 2023-11-16T16:49:37.329 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "port", 2023-11-16T16:49:37.329 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.329 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.329 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.329 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8086", 2023-11-16T16:49:37.329 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.329 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.329 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.330 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server port", 2023-11-16T16:49:37.330 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.330 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.330 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.330 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2023-11-16T16:49:37.330 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2023-11-16T16:49:37.330 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.331 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.331 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.331 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "false", 2023-11-16T16:49:37.331 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.331 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.331 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.331 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-11-16T16:49:37.331 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.332 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.332 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.332 INFO:teuthology.orchestra.run.smithi001.stdout: "threads": { 2023-11-16T16:49:37.332 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threads", 2023-11-16T16:49:37.332 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.332 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.332 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.333 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2023-11-16T16:49:37.333 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2023-11-16T16:49:37.333 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "32", 2023-11-16T16:49:37.333 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.333 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-11-16T16:49:37.333 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.333 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.333 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.334 INFO:teuthology.orchestra.run.smithi001.stdout: "username": { 2023-11-16T16:49:37.334 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "username", 2023-11-16T16:49:37.334 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.334 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.334 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.334 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.334 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.335 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.335 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.335 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "username of InfluxDB server user", 2023-11-16T16:49:37.335 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.335 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.335 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.335 INFO:teuthology.orchestra.run.smithi001.stdout: "verify_ssl": { 2023-11-16T16:49:37.336 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "verify_ssl", 2023-11-16T16:49:37.336 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.336 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.336 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.336 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "true", 2023-11-16T16:49:37.336 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.336 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.337 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.337 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-11-16T16:49:37.337 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.337 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.337 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.337 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.337 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.338 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:37.338 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "insights", 2023-11-16T16:49:37.338 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:37.338 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:37.338 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:37.338 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:37.338 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:37.339 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.339 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.339 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.339 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.339 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.339 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.339 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.339 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.340 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.340 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.340 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.340 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.340 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.340 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.340 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.340 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.340 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.341 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.341 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:37.341 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:37.341 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.341 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.341 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.341 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.342 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.342 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.342 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.342 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.342 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.342 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.342 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.343 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:37.343 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:37.343 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.343 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.343 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.343 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:37.343 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.344 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.344 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.344 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.344 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.344 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.344 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.344 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.344 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.345 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.345 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.345 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.345 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.345 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.345 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:37.345 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:37.346 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.346 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.346 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.346 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.346 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.346 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.346 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.347 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.347 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.347 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.347 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.347 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.347 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.347 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:37.347 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "iostat", 2023-11-16T16:49:37.348 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:37.348 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:37.348 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:37.348 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:37.348 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:37.348 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.348 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.348 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.349 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.349 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.349 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.349 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.349 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.349 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.349 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.349 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.350 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.350 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.350 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.350 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.350 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.350 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.350 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.351 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:37.351 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:37.351 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.351 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.351 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.351 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.351 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.352 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.352 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.352 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.352 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.352 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.352 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.352 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:37.353 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:37.353 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.353 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.353 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.353 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:37.353 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.353 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.353 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.354 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.354 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.354 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.354 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.354 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.354 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.354 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.354 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.355 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.355 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.355 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.355 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:37.355 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:37.355 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.356 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.356 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.356 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.356 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.356 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.356 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.356 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.356 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.357 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.357 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.357 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.357 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.357 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:37.357 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "k8sevents", 2023-11-16T16:49:37.358 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:37.358 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:37.358 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:37.358 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_event_retention_days": { 2023-11-16T16:49:37.358 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ceph_event_retention_days", 2023-11-16T16:49:37.358 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.358 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.358 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.359 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7", 2023-11-16T16:49:37.359 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.359 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.359 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.359 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Days to hold ceph event information within local cache", 2023-11-16T16:49:37.359 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.359 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.360 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.360 INFO:teuthology.orchestra.run.smithi001.stdout: "config_check_secs": { 2023-11-16T16:49:37.360 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_check_secs", 2023-11-16T16:49:37.360 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.360 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.360 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.360 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2023-11-16T16:49:37.361 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "10", 2023-11-16T16:49:37.361 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.361 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.361 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-11-16T16:49:37.361 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.361 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.361 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.362 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:37.362 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:37.362 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.362 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.362 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.362 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.362 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.362 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.363 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.363 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.363 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.363 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.363 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.363 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.363 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.363 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.364 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.364 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.364 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.364 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.364 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:37.364 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:37.364 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.365 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.365 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.365 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.365 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.365 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.365 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.365 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.365 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.366 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.366 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.366 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:37.366 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:37.366 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.366 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.366 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.366 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:37.367 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.367 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.367 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.367 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.367 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.367 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.367 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.367 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.368 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.368 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.368 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.368 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.368 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.368 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.368 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:37.369 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:37.369 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.369 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.369 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.369 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.369 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.369 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.369 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.370 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.370 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.370 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.370 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.370 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.370 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.370 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:37.371 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "localpool", 2023-11-16T16:49:37.371 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:37.371 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:37.371 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:37.371 INFO:teuthology.orchestra.run.smithi001.stdout: "failure_domain": { 2023-11-16T16:49:37.371 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "failure_domain", 2023-11-16T16:49:37.371 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.372 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.372 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.372 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "host", 2023-11-16T16:49:37.372 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.372 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.372 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.372 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "failure domain for any created local pool", 2023-11-16T16:49:37.372 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-11-16T16:49:37.373 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.373 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.373 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:37.373 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:37.373 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.373 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.373 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.374 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.374 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.374 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.374 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.374 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.374 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.374 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.374 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.375 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.375 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.375 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.375 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.375 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.375 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.375 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.375 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:37.376 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:37.376 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.376 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.376 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.376 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.376 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.376 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.376 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.377 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.377 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.377 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.377 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.377 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:37.377 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:37.377 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.378 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.378 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.378 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:37.378 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.378 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.378 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.378 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.379 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.379 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.379 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.379 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.379 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.379 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.379 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.379 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.379 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.380 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.380 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:37.380 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:37.380 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.380 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.380 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.380 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.380 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.381 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.381 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.381 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.381 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.381 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.381 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.381 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": { 2023-11-16T16:49:37.382 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_size", 2023-11-16T16:49:37.382 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.382 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.382 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.382 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.382 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.382 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.382 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.383 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default min_size for any created local pool", 2023-11-16T16:49:37.383 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-11-16T16:49:37.383 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.383 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.383 INFO:teuthology.orchestra.run.smithi001.stdout: "num_rep": { 2023-11-16T16:49:37.383 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "num_rep", 2023-11-16T16:49:37.383 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.384 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.384 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.384 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3", 2023-11-16T16:49:37.384 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.384 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.384 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.384 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default replica count for any created local pool", 2023-11-16T16:49:37.384 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.385 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.385 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.385 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": { 2023-11-16T16:49:37.385 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_num", 2023-11-16T16:49:37.385 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.385 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.386 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.386 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "128", 2023-11-16T16:49:37.386 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.386 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.386 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.386 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default pg_num for any created local pool", 2023-11-16T16:49:37.386 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.386 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.387 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.387 INFO:teuthology.orchestra.run.smithi001.stdout: "prefix": { 2023-11-16T16:49:37.387 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prefix", 2023-11-16T16:49:37.387 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.387 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.387 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.387 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.388 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.388 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.388 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.388 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name prefix for any created local pool", 2023-11-16T16:49:37.388 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.388 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.388 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.389 INFO:teuthology.orchestra.run.smithi001.stdout: "subtree": { 2023-11-16T16:49:37.389 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "subtree", 2023-11-16T16:49:37.389 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.389 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.389 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.389 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "rack", 2023-11-16T16:49:37.389 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.389 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.390 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.390 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "CRUSH level for which to create a local pool", 2023-11-16T16:49:37.390 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-11-16T16:49:37.390 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.390 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.390 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.390 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:37.391 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mds_autoscaler", 2023-11-16T16:49:37.391 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:37.391 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:37.391 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:37.391 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:37.391 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:37.392 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.392 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.392 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.392 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.392 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.392 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.392 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.392 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.393 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.393 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.393 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.393 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.393 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.393 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.393 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.393 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.394 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.394 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.394 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:37.394 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:37.394 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.394 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.395 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.395 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.395 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.395 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.395 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.395 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.395 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.395 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.396 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.396 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:37.396 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:37.396 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.396 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.396 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.396 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:37.397 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.397 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.397 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.397 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.397 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.397 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.397 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.397 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.398 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.398 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.398 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.398 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.398 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.398 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.399 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:37.399 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:37.399 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.399 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.399 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.399 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.399 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.399 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.400 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.400 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.400 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.400 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.400 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.400 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.400 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.401 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:37.401 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirroring", 2023-11-16T16:49:37.401 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:37.401 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:37.401 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:37.401 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:37.401 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:37.401 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.402 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.402 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.402 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.402 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.402 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.402 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.402 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.403 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.403 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.403 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.403 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.403 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.403 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.403 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.404 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.404 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.404 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.404 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:37.404 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:37.404 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.404 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.405 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.405 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.405 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.405 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.405 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.405 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.405 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.405 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.405 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.406 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:37.406 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:37.406 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.406 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.406 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.406 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:37.406 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.407 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.407 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.407 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.407 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.407 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.407 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.407 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.408 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.408 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.408 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.408 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.408 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.408 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.409 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:37.409 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:37.409 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.409 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.409 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.409 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.409 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.409 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.410 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.410 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.410 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.410 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.410 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.410 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.410 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:37.411 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "nfs", 2023-11-16T16:49:37.411 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:37.411 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:37.411 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:37.411 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:37.411 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:37.411 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.411 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.412 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.412 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.412 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.412 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.412 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.412 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.412 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.412 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.413 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.413 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.413 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.413 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.413 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.413 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.413 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.413 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.414 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:37.414 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:37.414 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.414 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.414 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.414 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.415 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.415 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.415 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.415 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.415 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.415 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.415 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.416 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:37.416 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:37.416 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.416 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.416 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.416 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:37.416 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.417 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.417 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.417 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.417 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.417 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.417 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.417 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.417 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.418 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.418 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.418 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.418 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.418 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.418 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:37.418 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:37.419 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.419 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.419 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.419 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.419 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.419 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.419 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.419 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.420 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.420 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.420 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.420 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.420 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.420 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:37.420 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2023-11-16T16:49:37.421 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:37.421 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:37.421 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:37.421 INFO:teuthology.orchestra.run.smithi001.stdout: "fail_fs": { 2023-11-16T16:49:37.421 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "fail_fs", 2023-11-16T16:49:37.421 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.421 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.421 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.422 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.422 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.422 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.422 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.422 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-11-16T16:49:37.422 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.422 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.423 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.423 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:37.423 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:37.423 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.423 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.423 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.423 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.424 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.424 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.424 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.424 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.424 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.424 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.424 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.425 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.425 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.425 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.425 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.425 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.425 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.425 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.426 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:37.426 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:37.426 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.426 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.426 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.426 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.426 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.426 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.427 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.427 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.427 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.427 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.427 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.427 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:37.428 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:37.428 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.428 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.428 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.428 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:37.428 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.428 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.428 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.429 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.429 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.429 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.429 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.429 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.429 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.429 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.429 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.430 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.430 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.430 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.430 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:37.430 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:37.430 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.430 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.431 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.431 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.431 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.431 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.431 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.431 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.431 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.431 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.432 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.432 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator": { 2023-11-16T16:49:37.432 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2023-11-16T16:49:37.432 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.432 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.432 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.433 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.433 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.433 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.433 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.433 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm", 2023-11-16T16:49:37.433 INFO:teuthology.orchestra.run.smithi001.stdout: "rook", 2023-11-16T16:49:37.433 INFO:teuthology.orchestra.run.smithi001.stdout: "test_orchestrator" 2023-11-16T16:49:37.433 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.434 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Orchestrator backend", 2023-11-16T16:49:37.434 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.434 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.434 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.434 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.434 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.434 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:37.435 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_perf_query", 2023-11-16T16:49:37.435 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:37.435 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:37.435 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:37.435 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:37.435 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:37.435 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.435 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.436 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.436 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.436 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.436 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.436 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.436 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.436 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.437 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.437 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.437 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.437 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.437 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.437 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.437 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.437 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.438 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.438 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:37.438 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:37.438 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.438 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.438 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.439 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.439 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.439 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.439 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.439 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.439 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.439 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.439 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.440 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:37.440 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:37.440 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.440 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.440 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.440 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:37.440 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.441 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.441 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.441 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.441 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.441 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.441 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.441 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.441 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.442 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.442 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.442 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.442 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.442 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.442 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:37.442 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:37.443 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.443 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.443 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.443 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.443 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.443 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.443 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.443 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.444 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.444 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.444 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.444 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.444 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.444 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:37.444 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_support", 2023-11-16T16:49:37.445 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:37.445 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:37.445 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:37.445 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:37.445 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:37.445 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.445 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.446 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.446 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.446 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.446 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.446 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.446 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.446 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.446 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.447 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.447 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.447 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.447 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.447 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.448 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.448 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.448 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.449 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:37.449 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:37.449 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.449 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.450 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.450 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.450 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.451 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.451 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.451 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.451 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.452 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.452 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.452 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:37.453 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:37.453 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.453 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.453 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.454 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:37.454 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.454 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.454 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.455 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.455 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.455 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.456 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.456 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.456 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.456 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.457 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.457 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.457 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.457 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.458 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:37.458 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:37.458 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.459 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.459 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.459 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.459 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.460 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.460 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.460 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.461 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.461 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.461 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.461 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.461 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:37.461 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_autoscaler", 2023-11-16T16:49:37.462 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:37.462 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:37.462 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:37.462 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:37.462 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:37.462 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.462 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.462 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.463 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.463 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.463 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.463 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.463 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.463 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.463 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.463 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.464 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.464 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.464 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.464 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.464 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.464 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.464 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.465 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:37.465 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:37.465 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.465 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.465 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.465 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.465 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.465 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.466 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.466 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.466 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.466 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.466 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.466 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:37.466 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:37.467 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.467 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.467 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.467 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:37.467 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.467 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.467 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.467 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.468 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.468 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.468 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.468 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.468 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.468 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.468 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.468 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.469 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.469 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.469 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:37.469 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:37.469 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.469 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.469 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.470 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.470 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.470 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.470 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.470 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.470 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.470 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.470 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.471 INFO:teuthology.orchestra.run.smithi001.stdout: "noautoscale": { 2023-11-16T16:49:37.471 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "noautoscale", 2023-11-16T16:49:37.471 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.471 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.471 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.471 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.471 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.472 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.472 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.472 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "global autoscale flag", 2023-11-16T16:49:37.472 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-11-16T16:49:37.472 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.472 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.472 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2023-11-16T16:49:37.473 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2023-11-16T16:49:37.473 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:37.473 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.473 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.473 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2023-11-16T16:49:37.473 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.473 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.473 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.474 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.474 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.474 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.474 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.474 INFO:teuthology.orchestra.run.smithi001.stdout: "threshold": { 2023-11-16T16:49:37.474 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threshold", 2023-11-16T16:49:37.475 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2023-11-16T16:49:37.475 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.475 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.475 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2023-11-16T16:49:37.475 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1.0", 2023-11-16T16:49:37.475 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.475 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.475 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "scaling threshold", 2023-11-16T16:49:37.476 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-11-16T16:49:37.476 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.476 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.476 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.476 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.476 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:37.476 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "progress", 2023-11-16T16:49:37.477 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:37.477 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:37.477 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:37.477 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_pg_recovery_event": { 2023-11-16T16:49:37.477 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_pg_recovery_event", 2023-11-16T16:49:37.477 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.477 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.477 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.478 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.478 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.478 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.478 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.478 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow the module to show pg recovery progress", 2023-11-16T16:49:37.478 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.478 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.478 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.479 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2023-11-16T16:49:37.479 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2023-11-16T16:49:37.479 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.479 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.479 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.479 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.480 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.480 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.480 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.480 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.480 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.480 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.480 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.481 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:37.481 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:37.481 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.481 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.481 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.481 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.481 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.481 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.481 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.482 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.482 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.482 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.482 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.482 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.482 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.482 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.482 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.483 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.483 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.483 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.483 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:37.483 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:37.483 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.483 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.484 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.484 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.484 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.484 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.484 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.484 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.484 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.484 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.485 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.485 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:37.485 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:37.485 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.485 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.485 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.486 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:37.486 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.486 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.486 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.486 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.486 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.486 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.487 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.487 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.487 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.487 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.487 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.487 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.487 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.487 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.488 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:37.488 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:37.488 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.488 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.488 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.488 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.488 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.489 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.489 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.489 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.489 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.489 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.489 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.489 INFO:teuthology.orchestra.run.smithi001.stdout: "max_completed_events": { 2023-11-16T16:49:37.489 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_completed_events", 2023-11-16T16:49:37.490 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.490 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.490 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.490 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "50", 2023-11-16T16:49:37.490 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.490 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.490 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.490 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "number of past completed events to remember", 2023-11-16T16:49:37.491 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.491 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.491 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.491 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2023-11-16T16:49:37.491 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2023-11-16T16:49:37.491 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:37.491 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.492 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.492 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2023-11-16T16:49:37.492 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.492 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.492 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.492 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long the module is going to sleep", 2023-11-16T16:49:37.492 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.492 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.493 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.493 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.493 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.493 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.493 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:37.493 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus", 2023-11-16T16:49:37.493 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:37.494 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:37.494 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:37.494 INFO:teuthology.orchestra.run.smithi001.stdout: "cache": { 2023-11-16T16:49:37.494 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cache", 2023-11-16T16:49:37.494 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.494 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.494 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.494 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.495 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.495 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.495 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.495 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.495 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.495 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.495 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.495 INFO:teuthology.orchestra.run.smithi001.stdout: "exclude_perf_counters": { 2023-11-16T16:49:37.496 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "exclude_perf_counters", 2023-11-16T16:49:37.496 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.496 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.496 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.496 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.496 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.496 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.496 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.497 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-11-16T16:49:37.497 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-11-16T16:49:37.497 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.497 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.497 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:37.497 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:37.498 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.498 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.498 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.498 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.498 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.498 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.498 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.498 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.499 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.499 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.499 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.499 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.499 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.499 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.499 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.500 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.500 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.500 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.500 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:37.500 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:37.500 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.500 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.501 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.501 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.501 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.501 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.501 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.501 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.501 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.501 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.502 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.502 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:37.502 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:37.502 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.502 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.502 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.503 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:37.503 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.503 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.503 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.503 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.503 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.503 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.503 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.503 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.504 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.504 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.504 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.504 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.504 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.504 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.504 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:37.505 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:37.505 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.505 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.505 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.505 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.505 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.505 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.506 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.506 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.506 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.506 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.506 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.506 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools": { 2023-11-16T16:49:37.506 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools", 2023-11-16T16:49:37.507 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.507 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.507 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.507 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.507 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.507 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.507 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.507 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.507 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.508 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.508 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.508 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools_refresh_interval": { 2023-11-16T16:49:37.508 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-11-16T16:49:37.508 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.508 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.508 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.509 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "300", 2023-11-16T16:49:37.509 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.509 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.509 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.509 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.509 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.509 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.510 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.510 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_interval": { 2023-11-16T16:49:37.510 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_interval", 2023-11-16T16:49:37.510 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2023-11-16T16:49:37.510 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.510 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.510 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15.0", 2023-11-16T16:49:37.511 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.511 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.511 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.511 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.511 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.511 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.511 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.512 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2023-11-16T16:49:37.512 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2023-11-16T16:49:37.512 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.512 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.512 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.512 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2023-11-16T16:49:37.512 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.512 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.513 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.513 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-11-16T16:49:37.513 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.513 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.513 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.513 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2023-11-16T16:49:37.513 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2023-11-16T16:49:37.514 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.514 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.514 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.514 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "9283", 2023-11-16T16:49:37.514 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.514 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.514 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.515 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-11-16T16:49:37.515 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.515 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.515 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.515 INFO:teuthology.orchestra.run.smithi001.stdout: "stale_cache_strategy": { 2023-11-16T16:49:37.515 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stale_cache_strategy", 2023-11-16T16:49:37.515 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.516 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.516 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.516 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "log", 2023-11-16T16:49:37.516 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.516 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.516 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.516 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.516 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.517 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.517 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.517 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2023-11-16T16:49:37.517 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2023-11-16T16:49:37.517 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.517 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.517 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.518 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "default", 2023-11-16T16:49:37.518 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.518 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.518 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.518 INFO:teuthology.orchestra.run.smithi001.stdout: "default", 2023-11-16T16:49:37.518 INFO:teuthology.orchestra.run.smithi001.stdout: "error" 2023-11-16T16:49:37.518 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.518 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.519 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.519 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.519 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.519 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2023-11-16T16:49:37.519 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2023-11-16T16:49:37.519 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.519 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.520 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.520 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2023-11-16T16:49:37.520 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2023-11-16T16:49:37.520 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2023-11-16T16:49:37.520 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.520 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.520 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.520 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.521 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.521 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.521 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.521 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:37.521 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_support", 2023-11-16T16:49:37.521 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:37.522 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:37.522 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:37.522 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:37.522 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:37.522 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.522 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.522 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.522 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.523 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.523 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.523 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.523 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.523 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.523 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.523 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.523 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.524 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.524 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.524 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.524 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.524 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.524 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.524 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:37.525 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:37.525 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.525 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.525 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.525 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.525 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.525 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.526 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.526 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.526 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.526 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.526 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.526 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:37.527 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:37.527 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.527 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.527 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.527 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:37.527 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.527 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.527 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.528 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.528 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.528 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.528 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.528 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.528 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.528 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.528 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.529 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.529 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.529 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.529 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:37.529 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:37.529 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.529 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.530 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.530 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.530 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.530 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.530 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.530 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.530 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.530 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.531 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.531 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_snap_create": { 2023-11-16T16:49:37.531 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_snap_create", 2023-11-16T16:49:37.531 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.531 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.531 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.531 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2023-11-16T16:49:37.532 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.532 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.532 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.532 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.532 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.532 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.532 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.533 INFO:teuthology.orchestra.run.smithi001.stdout: "mirror_snapshot_schedule": { 2023-11-16T16:49:37.533 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirror_snapshot_schedule", 2023-11-16T16:49:37.533 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.533 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.533 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.533 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.533 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.534 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.534 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.534 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.534 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.534 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.534 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.534 INFO:teuthology.orchestra.run.smithi001.stdout: "trash_purge_schedule": { 2023-11-16T16:49:37.535 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "trash_purge_schedule", 2023-11-16T16:49:37.535 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.535 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.535 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.535 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.535 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.535 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.535 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.536 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.536 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.536 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.536 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.536 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.536 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.536 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:37.536 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "restful", 2023-11-16T16:49:37.537 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:37.537 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:37.537 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:37.537 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_auth": { 2023-11-16T16:49:37.537 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_auth", 2023-11-16T16:49:37.537 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.537 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.537 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.538 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.538 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.538 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.538 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.538 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.538 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.538 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.538 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.539 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2023-11-16T16:49:37.539 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2023-11-16T16:49:37.539 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.539 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.539 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.539 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.539 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.540 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.540 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.540 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.540 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.540 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.540 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.540 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:37.541 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:37.541 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.541 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.541 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.541 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.541 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.541 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.541 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.542 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.542 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.542 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.542 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.542 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.542 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.542 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.542 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.543 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.543 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.543 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.543 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:37.543 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:37.543 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.543 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.544 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.544 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.544 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.544 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.544 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.544 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.544 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.544 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.544 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.545 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:37.545 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:37.545 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.545 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.545 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.546 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:37.546 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.546 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.546 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.546 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.546 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.546 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.547 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.547 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.547 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.547 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.547 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.547 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.547 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.548 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.548 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:37.548 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:37.548 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.548 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.548 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.548 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.549 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.549 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.549 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.549 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.549 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.549 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.549 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.550 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2023-11-16T16:49:37.550 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2023-11-16T16:49:37.550 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.550 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.550 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.550 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.550 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.550 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.551 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.551 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.551 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.551 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.551 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.551 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2023-11-16T16:49:37.551 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2023-11-16T16:49:37.552 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.552 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.552 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.552 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.552 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.552 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.552 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.552 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.553 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.553 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.553 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.553 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.553 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:37.553 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rgw", 2023-11-16T16:49:37.553 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:37.554 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:37.554 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:37.554 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:37.554 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:37.554 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.554 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.554 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.555 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.555 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.555 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.555 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.555 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.555 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.555 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.555 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.556 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.556 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.556 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.556 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.556 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.556 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.556 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.557 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:37.557 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:37.557 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.557 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.557 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.557 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.557 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.558 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.558 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.558 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.558 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.558 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.558 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.558 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:37.559 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:37.559 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.559 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.559 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.559 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:37.559 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.559 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.559 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.560 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.560 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.560 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.560 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.560 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.560 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.560 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.560 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.561 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.561 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.561 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.561 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:37.561 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:37.561 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.562 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.562 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.562 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.562 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.562 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.562 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.562 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.563 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.563 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.563 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.563 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.563 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.563 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:37.563 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rook", 2023-11-16T16:49:37.564 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:37.564 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:37.564 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:37.564 INFO:teuthology.orchestra.run.smithi001.stdout: "drive_group_interval": { 2023-11-16T16:49:37.564 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "drive_group_interval", 2023-11-16T16:49:37.564 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2023-11-16T16:49:37.564 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.564 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.565 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "300.0", 2023-11-16T16:49:37.565 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.565 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.565 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.565 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-11-16T16:49:37.565 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.565 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.565 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.566 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:37.566 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:37.566 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.566 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.566 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.566 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.567 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.567 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.567 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.567 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.567 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.567 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.567 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.567 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.568 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.568 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.568 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.568 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.568 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.568 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.568 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:37.569 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:37.569 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.569 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.569 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.569 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.569 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.569 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.569 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.570 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.570 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.570 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.570 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.570 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:37.570 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:37.570 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.571 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.571 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.571 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:37.571 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.571 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.571 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.571 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.571 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.572 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.572 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.572 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.572 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.572 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.572 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.572 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.572 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.573 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.573 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:37.573 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:37.573 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.573 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.573 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.573 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.574 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.574 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.574 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.574 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.574 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.574 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.574 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.575 INFO:teuthology.orchestra.run.smithi001.stdout: "storage_class": { 2023-11-16T16:49:37.575 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "storage_class", 2023-11-16T16:49:37.575 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.575 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.575 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.575 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "local", 2023-11-16T16:49:37.575 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.575 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.576 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.576 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-11-16T16:49:37.576 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.576 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.576 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.576 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.576 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.577 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:37.577 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "selftest", 2023-11-16T16:49:37.577 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:37.577 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:37.577 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:37.577 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:37.577 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:37.577 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.578 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.578 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.578 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.578 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.578 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.578 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.578 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.578 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.579 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.579 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.579 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.579 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.579 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.579 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.579 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.579 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.580 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.580 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:37.580 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:37.580 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.580 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.580 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.581 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.581 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.581 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.581 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.581 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.581 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.581 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.581 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.582 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:37.582 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:37.582 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.582 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.582 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.582 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:37.582 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.582 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.583 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.583 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.583 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.583 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.583 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.583 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.583 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.583 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.584 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.584 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.584 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.584 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.584 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:37.584 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:37.584 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.585 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.585 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.585 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.585 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.585 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.585 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.585 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.585 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.586 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.586 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.586 INFO:teuthology.orchestra.run.smithi001.stdout: "roption1": { 2023-11-16T16:49:37.586 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption1", 2023-11-16T16:49:37.586 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.587 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.587 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.587 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.587 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.587 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.587 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.587 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.587 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.588 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.588 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.588 INFO:teuthology.orchestra.run.smithi001.stdout: "roption2": { 2023-11-16T16:49:37.588 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption2", 2023-11-16T16:49:37.588 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.588 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.588 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.589 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "xyz", 2023-11-16T16:49:37.589 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.589 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.589 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.589 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.589 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.589 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.589 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.590 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption1": { 2023-11-16T16:49:37.590 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption1", 2023-11-16T16:49:37.590 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.590 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.590 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.590 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.591 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.591 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.591 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.591 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.591 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.591 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.591 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.592 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption2": { 2023-11-16T16:49:37.592 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption2", 2023-11-16T16:49:37.592 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.592 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.592 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.592 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.592 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.592 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.593 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.593 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.593 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.593 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.593 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.593 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption3": { 2023-11-16T16:49:37.594 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption3", 2023-11-16T16:49:37.594 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2023-11-16T16:49:37.594 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.594 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.594 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.594 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.594 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.594 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.595 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.595 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.595 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.595 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.595 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption4": { 2023-11-16T16:49:37.595 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption4", 2023-11-16T16:49:37.595 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.596 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.596 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.596 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.596 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.596 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.596 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.596 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.597 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.597 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.597 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.597 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption5": { 2023-11-16T16:49:37.597 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption5", 2023-11-16T16:49:37.597 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.597 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.598 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.598 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.598 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.598 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.598 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.598 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.598 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.598 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.599 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.599 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption6": { 2023-11-16T16:49:37.599 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption6", 2023-11-16T16:49:37.599 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.599 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.599 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.600 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.600 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.600 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.600 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.600 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.600 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.600 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.600 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.601 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption7": { 2023-11-16T16:49:37.601 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption7", 2023-11-16T16:49:37.601 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.601 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.601 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.601 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.601 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2023-11-16T16:49:37.602 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "42", 2023-11-16T16:49:37.602 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.602 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.602 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.602 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.602 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.603 INFO:teuthology.orchestra.run.smithi001.stdout: "testkey": { 2023-11-16T16:49:37.603 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testkey", 2023-11-16T16:49:37.603 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.603 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.603 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.603 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.603 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.603 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.604 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.604 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.604 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.604 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.604 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.604 INFO:teuthology.orchestra.run.smithi001.stdout: "testlkey": { 2023-11-16T16:49:37.604 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testlkey", 2023-11-16T16:49:37.605 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.605 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.605 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.605 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.605 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.605 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.605 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.605 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.606 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.606 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.606 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.606 INFO:teuthology.orchestra.run.smithi001.stdout: "testnewline": { 2023-11-16T16:49:37.606 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testnewline", 2023-11-16T16:49:37.606 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.606 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.607 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.607 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.607 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.607 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.607 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.607 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.607 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.607 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.608 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.608 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.608 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.608 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:37.608 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snap_schedule", 2023-11-16T16:49:37.608 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:37.608 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:37.609 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:37.609 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_m_granularity": { 2023-11-16T16:49:37.609 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_m_granularity", 2023-11-16T16:49:37.609 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.609 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.609 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.610 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.610 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.610 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.610 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.610 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow minute scheduled snapshots", 2023-11-16T16:49:37.610 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.610 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.610 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.611 INFO:teuthology.orchestra.run.smithi001.stdout: "dump_on_update": { 2023-11-16T16:49:37.611 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dump_on_update", 2023-11-16T16:49:37.611 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.611 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.611 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.611 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.611 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.612 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.612 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.612 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "dump database to debug log on update", 2023-11-16T16:49:37.612 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.612 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.612 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.612 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:37.613 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:37.613 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.613 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.613 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.613 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.613 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.613 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.613 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.614 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.614 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.614 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.614 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.614 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.614 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.614 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.614 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.615 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.615 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.615 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.615 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:37.615 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:37.615 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.616 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.616 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.616 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.616 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.616 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.616 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.616 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.616 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.617 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.617 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.617 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:37.617 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:37.617 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.617 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.617 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.617 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:37.618 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.618 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.618 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.618 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.618 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.618 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.618 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.618 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.619 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.619 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.619 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.619 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.619 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.619 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.619 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:37.620 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:37.620 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.620 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.620 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.620 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.620 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.620 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.620 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.621 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.621 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.621 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.621 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.621 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.621 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.621 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:37.622 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stats", 2023-11-16T16:49:37.622 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:37.622 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:37.622 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:37.622 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:37.622 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:37.622 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.623 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.623 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.623 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.623 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.623 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.623 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.623 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.623 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.623 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.624 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.624 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.624 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.624 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.624 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.624 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.624 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.624 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.625 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:37.625 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:37.625 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.625 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.625 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.626 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.626 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.626 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.626 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.626 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.626 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.626 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.626 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.627 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:37.627 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:37.627 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.627 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.627 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.627 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:37.627 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.628 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.628 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.628 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.628 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.628 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.628 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.628 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.628 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.629 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.629 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.629 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.629 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.629 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.629 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:37.630 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:37.630 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.630 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.630 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.631 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.631 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.631 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.631 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.631 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.631 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.631 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.631 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.632 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.632 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.632 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:37.632 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "status", 2023-11-16T16:49:37.632 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:37.632 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:37.632 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:37.633 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:37.633 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:37.633 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.633 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.633 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.633 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.633 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.633 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.634 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.634 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.634 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.634 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.634 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.634 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.634 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.635 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.635 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.635 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.635 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.635 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.636 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:37.636 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:37.636 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.636 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.636 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.636 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.636 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.636 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.637 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.637 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.637 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.637 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.637 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.637 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:37.638 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:37.638 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.638 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.638 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.638 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:37.638 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.638 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.639 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.639 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.639 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.639 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.639 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.639 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.640 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.640 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.640 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.640 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.640 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.640 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.641 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:37.641 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:37.641 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.642 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.642 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.642 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.642 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.643 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.643 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.643 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.643 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.644 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.644 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.644 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.644 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.645 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:37.645 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telegraf", 2023-11-16T16:49:37.645 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:37.645 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:37.645 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:37.645 INFO:teuthology.orchestra.run.smithi001.stdout: "address": { 2023-11-16T16:49:37.645 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "address", 2023-11-16T16:49:37.645 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.646 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.646 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.646 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-11-16T16:49:37.646 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.646 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.646 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.646 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.646 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.647 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.647 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.647 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2023-11-16T16:49:37.647 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2023-11-16T16:49:37.647 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:37.647 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.647 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.647 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15", 2023-11-16T16:49:37.648 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.648 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.648 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.648 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.648 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.648 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.648 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.649 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:37.649 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:37.649 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.649 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.649 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.649 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.649 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.649 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.650 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.650 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.650 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.650 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.650 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.650 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.650 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.651 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.651 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.651 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.651 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.651 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.651 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:37.651 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:37.652 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.652 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.652 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.652 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.652 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.652 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.652 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.652 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.653 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.653 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.653 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.653 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:37.653 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:37.653 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.653 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.654 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.654 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:37.654 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.654 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.654 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.654 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.654 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.654 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.655 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.655 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.655 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.655 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.655 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.655 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.655 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.655 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.656 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:37.656 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:37.656 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.656 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.656 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.656 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.657 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.657 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.657 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.657 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.657 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.657 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.657 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.657 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.658 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:37.658 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telemetry", 2023-11-16T16:49:37.658 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:37.658 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:37.658 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:37.659 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_basic": { 2023-11-16T16:49:37.659 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_basic", 2023-11-16T16:49:37.659 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.659 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.659 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.659 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.659 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.659 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.660 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.660 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share basic cluster information (size, version)", 2023-11-16T16:49:37.660 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.660 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.660 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.660 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_crash": { 2023-11-16T16:49:37.660 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_crash", 2023-11-16T16:49:37.661 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.661 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.661 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.661 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.661 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.661 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.661 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.661 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-11-16T16:49:37.662 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.662 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.662 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.662 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_device": { 2023-11-16T16:49:37.662 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_device", 2023-11-16T16:49:37.663 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.663 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.663 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.663 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.663 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.663 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.663 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.663 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-11-16T16:49:37.664 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.664 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.664 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.664 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_ident": { 2023-11-16T16:49:37.664 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_ident", 2023-11-16T16:49:37.664 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.664 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.665 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.665 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.665 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.665 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.665 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.665 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-11-16T16:49:37.665 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.665 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.666 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.666 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_perf": { 2023-11-16T16:49:37.666 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_perf", 2023-11-16T16:49:37.666 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.666 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.666 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.666 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.667 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.667 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.667 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.667 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share various performance metrics of a cluster", 2023-11-16T16:49:37.667 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.667 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.667 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.668 INFO:teuthology.orchestra.run.smithi001.stdout: "contact": { 2023-11-16T16:49:37.668 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "contact", 2023-11-16T16:49:37.668 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.668 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.668 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.668 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.669 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.669 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.669 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.669 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.669 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.669 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.669 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.670 INFO:teuthology.orchestra.run.smithi001.stdout: "description": { 2023-11-16T16:49:37.670 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "description", 2023-11-16T16:49:37.670 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.670 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.670 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.670 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.670 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.671 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.671 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.671 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.671 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.671 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.671 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.671 INFO:teuthology.orchestra.run.smithi001.stdout: "device_url": { 2023-11-16T16:49:37.672 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_url", 2023-11-16T16:49:37.672 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.672 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.672 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.672 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-11-16T16:49:37.672 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.672 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.672 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.672 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.673 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.673 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.673 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.673 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2023-11-16T16:49:37.673 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2023-11-16T16:49:37.673 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.673 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.674 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.674 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.674 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.674 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.674 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.674 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.674 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.674 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.675 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.675 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2023-11-16T16:49:37.675 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2023-11-16T16:49:37.675 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.675 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.675 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.675 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "24", 2023-11-16T16:49:37.676 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "8", 2023-11-16T16:49:37.676 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.676 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.676 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.676 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.676 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.676 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.676 INFO:teuthology.orchestra.run.smithi001.stdout: "last_opt_revision": { 2023-11-16T16:49:37.677 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "last_opt_revision", 2023-11-16T16:49:37.677 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.677 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.677 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.677 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1", 2023-11-16T16:49:37.677 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.677 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.677 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.677 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.678 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.678 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.678 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.678 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard": { 2023-11-16T16:49:37.678 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard", 2023-11-16T16:49:37.678 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.679 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.679 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.679 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.679 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.679 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.679 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.679 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.679 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.680 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.680 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.680 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard_description": { 2023-11-16T16:49:37.680 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard_description", 2023-11-16T16:49:37.680 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.680 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.680 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.681 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.681 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.681 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.681 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.681 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.681 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.681 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.682 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.682 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:37.682 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:37.682 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.682 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.682 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.682 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.683 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.683 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.683 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.683 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.683 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.683 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.683 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.683 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.684 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.684 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.684 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.684 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.684 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.684 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.685 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:37.685 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:37.685 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.685 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.685 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.685 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.685 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.685 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.686 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.686 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.686 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.686 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.686 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.686 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:37.687 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:37.687 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.687 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.687 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.687 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:37.687 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.687 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.688 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.688 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.688 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.688 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.688 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.688 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.688 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.689 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.689 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.689 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.689 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.689 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.689 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:37.689 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:37.690 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.690 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.690 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.690 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.690 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.690 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.690 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.691 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.691 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.691 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.691 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.691 INFO:teuthology.orchestra.run.smithi001.stdout: "organization": { 2023-11-16T16:49:37.691 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "organization", 2023-11-16T16:49:37.691 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.692 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.692 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.692 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.692 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.692 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.693 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.693 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.693 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.694 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.694 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.694 INFO:teuthology.orchestra.run.smithi001.stdout: "proxy": { 2023-11-16T16:49:37.694 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "proxy", 2023-11-16T16:49:37.694 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.695 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.695 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.695 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.695 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.696 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.696 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.696 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.697 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.697 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.697 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.698 INFO:teuthology.orchestra.run.smithi001.stdout: "url": { 2023-11-16T16:49:37.698 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url", 2023-11-16T16:49:37.698 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.698 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.698 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.698 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-11-16T16:49:37.699 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.699 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.699 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.699 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.699 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.699 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.699 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.699 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.700 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:37.700 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "test_orchestrator", 2023-11-16T16:49:37.700 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:37.700 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:37.700 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:37.701 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:37.701 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:37.701 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.701 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.701 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.701 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.701 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.701 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.702 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.702 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.702 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.702 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.702 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.702 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.702 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.703 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.703 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.703 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.703 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.703 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.703 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:37.703 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:37.704 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.704 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.704 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.704 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.704 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.704 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.704 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.704 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.705 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.705 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.705 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.705 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:37.705 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:37.705 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.705 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.706 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.706 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:37.706 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.706 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.706 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.706 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.706 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.707 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.707 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.707 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.707 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.707 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.707 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.707 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.707 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.708 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.708 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:37.708 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:37.708 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.708 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.708 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.708 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.709 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.709 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.709 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.709 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.709 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.709 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.710 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.710 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.710 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.710 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:37.710 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "volumes", 2023-11-16T16:49:37.710 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:37.711 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:37.711 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:37.711 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:37.711 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:37.711 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.711 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.711 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.712 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.712 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.712 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.712 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.712 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.712 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.712 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.712 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.713 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.713 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.713 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.713 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.713 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.713 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.713 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.714 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:37.714 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:37.714 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.714 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.714 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.714 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.714 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.714 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.715 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.715 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.715 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.715 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.715 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.716 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:37.716 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:37.717 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.717 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.717 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.717 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:37.717 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.718 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.718 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.718 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.718 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.718 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.718 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.718 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.719 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.719 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.719 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.719 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.719 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.719 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.719 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:37.720 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:37.720 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.720 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.720 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.720 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.720 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.720 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.721 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.721 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.721 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.721 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.721 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.722 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_clones": { 2023-11-16T16:49:37.722 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_clones", 2023-11-16T16:49:37.722 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.722 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.722 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.722 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4", 2023-11-16T16:49:37.723 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.723 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.723 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.723 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Number of asynchronous cloner threads", 2023-11-16T16:49:37.723 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.723 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.723 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.724 INFO:teuthology.orchestra.run.smithi001.stdout: "snapshot_clone_delay": { 2023-11-16T16:49:37.724 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snapshot_clone_delay", 2023-11-16T16:49:37.724 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.724 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.724 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.724 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2023-11-16T16:49:37.724 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.724 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.725 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.725 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-11-16T16:49:37.725 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.725 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.725 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.725 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.725 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.726 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:37.726 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix", 2023-11-16T16:49:37.726 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:37.726 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:37.726 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:37.726 INFO:teuthology.orchestra.run.smithi001.stdout: "discovery_interval": { 2023-11-16T16:49:37.726 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "discovery_interval", 2023-11-16T16:49:37.727 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2023-11-16T16:49:37.727 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.727 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.727 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "100", 2023-11-16T16:49:37.727 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.727 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.727 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.727 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.728 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.728 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.728 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.728 INFO:teuthology.orchestra.run.smithi001.stdout: "identifier": { 2023-11-16T16:49:37.728 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "identifier", 2023-11-16T16:49:37.728 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.728 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.729 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.729 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.729 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.729 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.729 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.729 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.729 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.730 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.730 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.730 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2023-11-16T16:49:37.730 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2023-11-16T16:49:37.730 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:37.730 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.730 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.731 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2023-11-16T16:49:37.731 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.731 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.731 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.731 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.731 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.731 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.731 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.732 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:37.732 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:37.732 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.732 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.732 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.732 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.732 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.733 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.733 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.733 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.733 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.733 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.733 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.733 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.733 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.734 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.734 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.734 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.734 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.734 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.734 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:37.734 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:37.735 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.735 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.735 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.735 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.735 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.735 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.735 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.735 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.736 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.736 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.736 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.736 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:37.736 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:37.736 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.737 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.737 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.737 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:37.737 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.737 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.737 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.737 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.737 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.738 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.738 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.738 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.738 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.738 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.738 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.738 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.738 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.739 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.739 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:37.739 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:37.739 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.739 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.739 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.739 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.740 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.740 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.740 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.740 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.740 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.740 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.740 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.741 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_host": { 2023-11-16T16:49:37.741 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_host", 2023-11-16T16:49:37.741 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.741 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.741 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.741 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.741 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.741 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.742 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.742 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.742 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.742 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.742 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.742 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_port": { 2023-11-16T16:49:37.742 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_port", 2023-11-16T16:49:37.743 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.743 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.743 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.743 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10051", 2023-11-16T16:49:37.743 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.743 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.743 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.743 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.744 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.744 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.744 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.744 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_sender": { 2023-11-16T16:49:37.744 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_sender", 2023-11-16T16:49:37.744 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.744 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.745 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.745 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-11-16T16:49:37.745 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.745 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.745 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.745 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.745 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.745 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.746 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.746 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.746 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.746 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.746 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2023-11-16T16:49:37.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.746 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:37.747 INFO:teuthology.orchestra.run.smithi001.stdout: "gid": 25084, 2023-11-16T16:49:37.747 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "z", 2023-11-16T16:49:37.747 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_features": 4540138322906710015, 2023-11-16T16:49:37.747 INFO:teuthology.orchestra.run.smithi001.stdout: "available_modules": [ 2023-11-16T16:49:37.747 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:37.747 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "alerts", 2023-11-16T16:49:37.747 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:37.747 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:37.748 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:37.748 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2023-11-16T16:49:37.748 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2023-11-16T16:49:37.748 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:37.748 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.748 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.748 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2023-11-16T16:49:37.748 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.748 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.749 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.749 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How frequently to reexamine health status", 2023-11-16T16:49:37.749 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.749 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.749 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.749 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:37.750 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:37.750 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.750 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.750 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.750 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.750 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.750 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.750 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.751 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.751 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.751 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.751 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.751 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.751 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.751 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.751 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.752 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.752 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.752 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.752 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:37.752 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:37.752 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.752 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.753 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.753 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.753 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.753 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.753 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.753 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.753 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.754 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.754 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.754 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:37.754 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:37.754 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.754 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.754 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.755 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:37.755 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.755 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.755 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.755 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.755 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.755 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.755 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.756 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.756 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.756 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.756 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.756 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.756 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.756 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.757 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:37.757 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:37.757 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.757 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.757 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.757 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.757 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.758 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.758 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.758 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.758 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.758 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.758 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.758 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_destination": { 2023-11-16T16:49:37.759 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_destination", 2023-11-16T16:49:37.759 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.759 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.759 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.759 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.759 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.759 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.760 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.760 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email address to send alerts to", 2023-11-16T16:49:37.760 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.760 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.760 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.760 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_from_name": { 2023-11-16T16:49:37.760 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_from_name", 2023-11-16T16:49:37.761 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.761 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.761 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.761 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "Ceph", 2023-11-16T16:49:37.761 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.761 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.761 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.761 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email From: name", 2023-11-16T16:49:37.762 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.762 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.762 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.762 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_host": { 2023-11-16T16:49:37.762 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_host", 2023-11-16T16:49:37.762 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.762 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.763 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.763 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.763 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.763 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.763 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.763 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP server", 2023-11-16T16:49:37.763 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.763 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.764 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.764 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_password": { 2023-11-16T16:49:37.764 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_password", 2023-11-16T16:49:37.764 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.764 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.764 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.764 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.765 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.765 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.765 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.765 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Password to authenticate with", 2023-11-16T16:49:37.765 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.765 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.765 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.766 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_port": { 2023-11-16T16:49:37.766 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_port", 2023-11-16T16:49:37.766 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.766 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.766 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.766 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "465", 2023-11-16T16:49:37.766 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.767 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.767 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.767 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP port", 2023-11-16T16:49:37.767 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.767 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.767 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.767 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_sender": { 2023-11-16T16:49:37.768 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_sender", 2023-11-16T16:49:37.768 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.768 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.768 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.768 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.768 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.768 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.768 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.769 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP envelope sender", 2023-11-16T16:49:37.769 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.769 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.769 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.769 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_ssl": { 2023-11-16T16:49:37.769 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_ssl", 2023-11-16T16:49:37.769 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.770 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.770 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.770 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.770 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.770 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.770 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.770 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use SSL to connect to SMTP server", 2023-11-16T16:49:37.770 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.771 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.771 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.771 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_user": { 2023-11-16T16:49:37.771 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_user", 2023-11-16T16:49:37.771 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.771 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.771 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.772 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.772 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.772 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.772 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.772 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "User to authenticate as", 2023-11-16T16:49:37.772 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.772 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.772 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.773 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.773 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:37.773 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "balancer", 2023-11-16T16:49:37.773 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:37.773 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:37.774 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:37.774 INFO:teuthology.orchestra.run.smithi001.stdout: "active": { 2023-11-16T16:49:37.774 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "active", 2023-11-16T16:49:37.774 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.774 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.774 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.774 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.774 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.775 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.775 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.775 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically balance PGs across cluster", 2023-11-16T16:49:37.775 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.775 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.775 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.775 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_time": { 2023-11-16T16:49:37.776 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_time", 2023-11-16T16:49:37.776 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.776 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.776 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.776 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0000", 2023-11-16T16:49:37.776 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.776 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.776 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.777 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "beginning time of day to automatically balance", 2023-11-16T16:49:37.777 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-16T16:49:37.777 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.777 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.777 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_weekday": { 2023-11-16T16:49:37.777 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_weekday", 2023-11-16T16:49:37.777 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2023-11-16T16:49:37.778 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.778 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.778 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2023-11-16T16:49:37.778 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2023-11-16T16:49:37.778 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2023-11-16T16:49:37.778 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.778 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-11-16T16:49:37.779 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-16T16:49:37.779 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.779 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.779 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_max_iterations": { 2023-11-16T16:49:37.779 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_max_iterations", 2023-11-16T16:49:37.779 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2023-11-16T16:49:37.779 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.780 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.780 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "25", 2023-11-16T16:49:37.780 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2023-11-16T16:49:37.780 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "250", 2023-11-16T16:49:37.780 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.780 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-11-16T16:49:37.780 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.780 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.781 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.781 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_metrics": { 2023-11-16T16:49:37.781 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_metrics", 2023-11-16T16:49:37.781 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.781 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.781 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.781 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "pgs,objects,bytes", 2023-11-16T16:49:37.782 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.782 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.782 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.782 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-11-16T16:49:37.782 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-11-16T16:49:37.782 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.782 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.783 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_step": { 2023-11-16T16:49:37.783 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_step", 2023-11-16T16:49:37.783 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2023-11-16T16:49:37.783 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.783 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.783 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.5", 2023-11-16T16:49:37.783 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0.001", 2023-11-16T16:49:37.784 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "0.999", 2023-11-16T16:49:37.784 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.784 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "aggressiveness of optimization", 2023-11-16T16:49:37.784 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-11-16T16:49:37.784 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.784 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.784 INFO:teuthology.orchestra.run.smithi001.stdout: "end_time": { 2023-11-16T16:49:37.785 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_time", 2023-11-16T16:49:37.785 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.785 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.785 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.785 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2359", 2023-11-16T16:49:37.785 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.785 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.785 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.786 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ending time of day to automatically balance", 2023-11-16T16:49:37.786 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-16T16:49:37.786 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.786 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.786 INFO:teuthology.orchestra.run.smithi001.stdout: "end_weekday": { 2023-11-16T16:49:37.786 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_weekday", 2023-11-16T16:49:37.786 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2023-11-16T16:49:37.787 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.787 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.787 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2023-11-16T16:49:37.787 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2023-11-16T16:49:37.787 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2023-11-16T16:49:37.787 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.788 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-11-16T16:49:37.788 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-16T16:49:37.788 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.788 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.788 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:37.788 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:37.788 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.789 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.789 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.789 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.789 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.789 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.789 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.789 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.789 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.790 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.790 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.790 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.790 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.790 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.790 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.790 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.790 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.791 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.791 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:37.791 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:37.791 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.791 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.791 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.791 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.791 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.792 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.792 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.792 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.792 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.792 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.792 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.792 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:37.793 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:37.793 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.793 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.793 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.793 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:37.793 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.793 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.793 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.793 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.794 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.794 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.794 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.794 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.794 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.794 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.794 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.795 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.795 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.795 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.795 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:37.795 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:37.795 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.796 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.796 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.796 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.796 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.796 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.796 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.796 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.796 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.797 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.797 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.797 INFO:teuthology.orchestra.run.smithi001.stdout: "min_score": { 2023-11-16T16:49:37.797 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_score", 2023-11-16T16:49:37.797 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2023-11-16T16:49:37.797 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.797 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.798 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2023-11-16T16:49:37.798 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.798 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.798 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.798 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-11-16T16:49:37.798 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.798 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.798 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.799 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2023-11-16T16:49:37.799 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2023-11-16T16:49:37.799 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.799 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.799 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.799 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "upmap", 2023-11-16T16:49:37.800 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.800 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.800 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.800 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-compat", 2023-11-16T16:49:37.800 INFO:teuthology.orchestra.run.smithi001.stdout: "none", 2023-11-16T16:49:37.800 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap" 2023-11-16T16:49:37.800 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.800 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Balancer mode", 2023-11-16T16:49:37.801 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.801 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.801 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.801 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_ids": { 2023-11-16T16:49:37.801 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_ids", 2023-11-16T16:49:37.801 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.802 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.802 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.802 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.802 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.802 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.802 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.802 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-11-16T16:49:37.802 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.803 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.803 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.803 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2023-11-16T16:49:37.803 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2023-11-16T16:49:37.803 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:37.803 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.803 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.804 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2023-11-16T16:49:37.804 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.804 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.804 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.804 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-11-16T16:49:37.804 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.804 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.804 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.805 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_deviation": { 2023-11-16T16:49:37.805 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_deviation", 2023-11-16T16:49:37.805 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.805 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.805 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.805 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2023-11-16T16:49:37.806 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2023-11-16T16:49:37.806 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.806 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.806 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "deviation below which no optimization is attempted", 2023-11-16T16:49:37.806 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-11-16T16:49:37.806 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.806 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.807 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_optimizations": { 2023-11-16T16:49:37.807 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_optimizations", 2023-11-16T16:49:37.807 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2023-11-16T16:49:37.807 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.807 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.807 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2023-11-16T16:49:37.807 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.807 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.808 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.808 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-11-16T16:49:37.808 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.808 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.808 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.808 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.808 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.809 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:37.809 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cephadm", 2023-11-16T16:49:37.809 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:37.809 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:37.809 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:37.809 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_down_multiplier": { 2023-11-16T16:49:37.809 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_down_multiplier", 2023-11-16T16:49:37.809 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2023-11-16T16:49:37.810 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.810 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.810 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2023-11-16T16:49:37.810 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.810 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.810 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.810 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-11-16T16:49:37.810 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.811 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.811 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.811 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_refresh_rate": { 2023-11-16T16:49:37.811 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_refresh_rate", 2023-11-16T16:49:37.811 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:37.811 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.812 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.812 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "20", 2023-11-16T16:49:37.812 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.812 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.812 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.812 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-11-16T16:49:37.812 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.812 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.813 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.813 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_starting_port": { 2023-11-16T16:49:37.813 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_starting_port", 2023-11-16T16:49:37.813 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.813 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.813 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.813 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4721", 2023-11-16T16:49:37.814 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.814 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.814 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.814 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-11-16T16:49:37.814 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.814 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.814 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.815 INFO:teuthology.orchestra.run.smithi001.stdout: "alertmanager_web_password": { 2023-11-16T16:49:37.815 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "alertmanager_web_password", 2023-11-16T16:49:37.815 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.815 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.815 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.815 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2023-11-16T16:49:37.815 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.816 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.816 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.816 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Alertmanager web password", 2023-11-16T16:49:37.816 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.816 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.816 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.817 INFO:teuthology.orchestra.run.smithi001.stdout: "alertmanager_web_user": { 2023-11-16T16:49:37.817 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "alertmanager_web_user", 2023-11-16T16:49:37.817 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.817 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.817 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.817 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2023-11-16T16:49:37.817 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.818 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.818 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.818 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Alertmanager web user", 2023-11-16T16:49:37.818 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.818 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.818 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.818 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_ptrace": { 2023-11-16T16:49:37.819 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_ptrace", 2023-11-16T16:49:37.819 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.819 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.819 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.819 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.819 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.819 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.819 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.819 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-11-16T16:49:37.820 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-11-16T16:49:37.820 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.820 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.820 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_interval": { 2023-11-16T16:49:37.820 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_interval", 2023-11-16T16:49:37.820 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:37.820 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.820 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.821 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2023-11-16T16:49:37.821 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.821 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.821 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.821 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to autotune daemon memory", 2023-11-16T16:49:37.821 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.821 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.821 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.822 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_memory_target_ratio": { 2023-11-16T16:49:37.822 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_memory_target_ratio", 2023-11-16T16:49:37.822 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2023-11-16T16:49:37.822 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.822 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.823 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.7", 2023-11-16T16:49:37.823 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.823 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.823 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.823 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-11-16T16:49:37.823 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.823 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.823 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.824 INFO:teuthology.orchestra.run.smithi001.stdout: "cgroups_split": { 2023-11-16T16:49:37.824 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cgroups_split", 2023-11-16T16:49:37.824 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.824 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.824 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.824 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.824 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.825 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.825 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.825 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-11-16T16:49:37.825 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.825 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.825 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.826 INFO:teuthology.orchestra.run.smithi001.stdout: "config_checks_enabled": { 2023-11-16T16:49:37.826 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_checks_enabled", 2023-11-16T16:49:37.826 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.826 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.826 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.826 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.826 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.826 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.827 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.827 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-11-16T16:49:37.827 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.827 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.827 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.827 INFO:teuthology.orchestra.run.smithi001.stdout: "config_dashboard": { 2023-11-16T16:49:37.827 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_dashboard", 2023-11-16T16:49:37.828 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.828 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.828 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.828 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.828 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.828 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.828 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.828 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-11-16T16:49:37.829 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.829 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.829 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.829 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_alertmanager": { 2023-11-16T16:49:37.829 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_alertmanager", 2023-11-16T16:49:37.829 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.829 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.830 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.830 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-11-16T16:49:37.830 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.830 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.830 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.830 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2023-11-16T16:49:37.830 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.830 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.831 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.831 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_base": { 2023-11-16T16:49:37.831 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_base", 2023-11-16T16:49:37.831 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.831 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.831 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.831 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/ceph", 2023-11-16T16:49:37.832 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.832 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.832 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.832 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Container image name, without the tag", 2023-11-16T16:49:37.832 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.832 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.832 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.833 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_elasticsearch": { 2023-11-16T16:49:37.833 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_elasticsearch", 2023-11-16T16:49:37.833 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.833 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.833 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.833 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-11-16T16:49:37.833 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.833 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.834 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.834 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "elasticsearch container image", 2023-11-16T16:49:37.834 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.834 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.834 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.834 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_grafana": { 2023-11-16T16:49:37.834 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_grafana", 2023-11-16T16:49:37.835 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.835 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.835 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.835 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-11-16T16:49:37.835 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.835 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.835 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.835 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2023-11-16T16:49:37.836 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.836 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.836 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.836 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_haproxy": { 2023-11-16T16:49:37.836 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_haproxy", 2023-11-16T16:49:37.836 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.836 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.837 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.837 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-11-16T16:49:37.837 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.837 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.837 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.837 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "HAproxy container image", 2023-11-16T16:49:37.837 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.837 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.837 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.838 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_agent": { 2023-11-16T16:49:37.838 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_agent", 2023-11-16T16:49:37.838 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.838 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.838 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.838 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-11-16T16:49:37.839 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.839 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.839 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.839 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger agent container image", 2023-11-16T16:49:37.839 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.839 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.839 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.840 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_collector": { 2023-11-16T16:49:37.840 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_collector", 2023-11-16T16:49:37.840 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.840 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.840 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.840 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-11-16T16:49:37.840 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.840 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.841 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.841 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger collector container image", 2023-11-16T16:49:37.841 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.841 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.841 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.841 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_query": { 2023-11-16T16:49:37.841 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_query", 2023-11-16T16:49:37.842 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.842 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.842 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.842 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-11-16T16:49:37.842 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.842 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.842 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.842 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger query container image", 2023-11-16T16:49:37.843 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.843 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.843 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.843 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_keepalived": { 2023-11-16T16:49:37.843 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_keepalived", 2023-11-16T16:49:37.843 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.843 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.844 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.844 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-11-16T16:49:37.844 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.844 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.844 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.844 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Keepalived container image", 2023-11-16T16:49:37.844 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.844 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.845 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.845 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_loki": { 2023-11-16T16:49:37.845 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_loki", 2023-11-16T16:49:37.845 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.845 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.845 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.845 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-11-16T16:49:37.846 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.846 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.846 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.846 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Loki container image", 2023-11-16T16:49:37.846 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.846 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.846 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.847 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_node_exporter": { 2023-11-16T16:49:37.847 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_node_exporter", 2023-11-16T16:49:37.847 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.847 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.847 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.847 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-11-16T16:49:37.847 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.847 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.848 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.848 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2023-11-16T16:49:37.848 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.848 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.848 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.848 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_prometheus": { 2023-11-16T16:49:37.848 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_prometheus", 2023-11-16T16:49:37.849 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.849 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.849 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.849 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-11-16T16:49:37.849 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.849 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.849 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.849 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2023-11-16T16:49:37.850 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.850 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.850 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.850 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_promtail": { 2023-11-16T16:49:37.850 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_promtail", 2023-11-16T16:49:37.850 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.850 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.851 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.851 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-11-16T16:49:37.851 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.851 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.851 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.851 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Promtail container image", 2023-11-16T16:49:37.851 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.851 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.852 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.852 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_snmp_gateway": { 2023-11-16T16:49:37.852 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_snmp_gateway", 2023-11-16T16:49:37.852 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.852 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.852 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.852 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-11-16T16:49:37.853 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.853 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.853 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.853 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SNMP Gateway container image", 2023-11-16T16:49:37.853 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.853 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.853 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.854 INFO:teuthology.orchestra.run.smithi001.stdout: "container_init": { 2023-11-16T16:49:37.854 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_init", 2023-11-16T16:49:37.854 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.854 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.854 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.854 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.854 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.854 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.855 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.855 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Run podman/docker with `--init`", 2023-11-16T16:49:37.855 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.855 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.855 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.855 INFO:teuthology.orchestra.run.smithi001.stdout: "daemon_cache_timeout": { 2023-11-16T16:49:37.855 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "daemon_cache_timeout", 2023-11-16T16:49:37.856 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:37.856 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.856 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.856 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2023-11-16T16:49:37.856 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.856 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.856 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.856 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-11-16T16:49:37.856 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.857 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.857 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.857 INFO:teuthology.orchestra.run.smithi001.stdout: "default_cephadm_command_timeout": { 2023-11-16T16:49:37.857 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_cephadm_command_timeout", 2023-11-16T16:49:37.857 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:37.857 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.857 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.858 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "900", 2023-11-16T16:49:37.858 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.858 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.858 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.858 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-11-16T16:49:37.858 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.858 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.859 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.859 INFO:teuthology.orchestra.run.smithi001.stdout: "default_registry": { 2023-11-16T16:49:37.859 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_registry", 2023-11-16T16:49:37.859 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.859 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.859 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.859 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io", 2023-11-16T16:49:37.859 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.860 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.860 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.860 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-11-16T16:49:37.860 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.860 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.860 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.861 INFO:teuthology.orchestra.run.smithi001.stdout: "device_cache_timeout": { 2023-11-16T16:49:37.861 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_cache_timeout", 2023-11-16T16:49:37.861 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:37.861 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.861 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.861 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1800", 2023-11-16T16:49:37.861 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.861 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.862 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.862 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache device inventory", 2023-11-16T16:49:37.862 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.862 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.862 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.862 INFO:teuthology.orchestra.run.smithi001.stdout: "device_enhanced_scan": { 2023-11-16T16:49:37.862 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_enhanced_scan", 2023-11-16T16:49:37.863 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.863 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.863 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.863 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.863 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.863 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.863 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.863 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use libstoragemgmt during device scans", 2023-11-16T16:49:37.864 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.864 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.864 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.864 INFO:teuthology.orchestra.run.smithi001.stdout: "facts_cache_timeout": { 2023-11-16T16:49:37.864 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "facts_cache_timeout", 2023-11-16T16:49:37.864 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:37.864 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.865 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.865 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2023-11-16T16:49:37.865 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.865 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.865 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.865 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache host facts data", 2023-11-16T16:49:37.865 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.866 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.866 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.866 INFO:teuthology.orchestra.run.smithi001.stdout: "host_check_interval": { 2023-11-16T16:49:37.866 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "host_check_interval", 2023-11-16T16:49:37.866 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:37.866 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.867 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.867 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2023-11-16T16:49:37.867 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.867 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.867 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.867 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to perform a host check", 2023-11-16T16:49:37.867 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.867 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.868 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.868 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:37.868 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:37.868 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.868 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.868 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.868 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.869 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.869 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.869 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.869 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.869 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.869 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.869 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.869 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.870 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.870 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.870 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.870 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.870 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.870 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.871 INFO:teuthology.orchestra.run.smithi001.stdout: "log_refresh_metadata": { 2023-11-16T16:49:37.871 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_refresh_metadata", 2023-11-16T16:49:37.871 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.871 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.871 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.871 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.871 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.871 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.872 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.872 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-11-16T16:49:37.872 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.872 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.872 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.872 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:37.872 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:37.873 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.873 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.873 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.873 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.873 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.873 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.873 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.874 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-11-16T16:49:37.874 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.874 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.874 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.874 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:37.874 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:37.875 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.875 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.875 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.875 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:37.875 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.875 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.875 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.875 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.876 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.876 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.876 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.876 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.876 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.876 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.876 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.876 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.877 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.877 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.877 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:37.877 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:37.877 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.877 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.877 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.878 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.878 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.878 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.878 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.878 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.878 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.878 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.878 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.879 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf": { 2023-11-16T16:49:37.879 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-11-16T16:49:37.879 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.879 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.879 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.879 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.880 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.880 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.880 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.880 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-11-16T16:49:37.880 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.880 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.880 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.881 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-11-16T16:49:37.881 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-11-16T16:49:37.881 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.881 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.881 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.881 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "*", 2023-11-16T16:49:37.881 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.881 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.882 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.882 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-11-16T16:49:37.882 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.882 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.882 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.882 INFO:teuthology.orchestra.run.smithi001.stdout: "max_count_per_host": { 2023-11-16T16:49:37.882 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_count_per_host", 2023-11-16T16:49:37.883 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.883 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.883 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.883 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2023-11-16T16:49:37.883 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.883 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.883 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.883 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of daemons per service per host", 2023-11-16T16:49:37.884 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.884 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.884 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.884 INFO:teuthology.orchestra.run.smithi001.stdout: "max_osd_draining_count": { 2023-11-16T16:49:37.884 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_osd_draining_count", 2023-11-16T16:49:37.884 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.884 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.885 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.885 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2023-11-16T16:49:37.885 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.885 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.885 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.885 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-11-16T16:49:37.885 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.885 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.886 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.886 INFO:teuthology.orchestra.run.smithi001.stdout: "migration_current": { 2023-11-16T16:49:37.886 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "migration_current", 2023-11-16T16:49:37.886 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.886 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.886 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.886 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.887 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.887 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.887 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.887 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "internal - do not modify", 2023-11-16T16:49:37.887 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.887 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.887 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.888 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2023-11-16T16:49:37.888 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2023-11-16T16:49:37.888 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.888 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.888 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.889 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "root", 2023-11-16T16:49:37.889 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.889 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.889 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.889 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm-package", 2023-11-16T16:49:37.889 INFO:teuthology.orchestra.run.smithi001.stdout: "root" 2023-11-16T16:49:37.889 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.889 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "mode for remote execution of cephadm", 2023-11-16T16:49:37.890 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.890 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.890 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.890 INFO:teuthology.orchestra.run.smithi001.stdout: "prometheus_alerts_path": { 2023-11-16T16:49:37.890 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus_alerts_path", 2023-11-16T16:49:37.890 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.891 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.891 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.891 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-11-16T16:49:37.891 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.891 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.891 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.891 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-11-16T16:49:37.891 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.892 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.892 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.892 INFO:teuthology.orchestra.run.smithi001.stdout: "prometheus_web_password": { 2023-11-16T16:49:37.892 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus_web_password", 2023-11-16T16:49:37.892 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.892 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.892 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.893 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2023-11-16T16:49:37.893 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.893 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.893 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.893 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus web password", 2023-11-16T16:49:37.893 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.893 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.893 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.894 INFO:teuthology.orchestra.run.smithi001.stdout: "prometheus_web_user": { 2023-11-16T16:49:37.894 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus_web_user", 2023-11-16T16:49:37.894 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.894 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.894 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.894 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2023-11-16T16:49:37.894 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.895 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.895 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.895 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus web user", 2023-11-16T16:49:37.895 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.895 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.895 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.896 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_insecure": { 2023-11-16T16:49:37.896 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_insecure", 2023-11-16T16:49:37.896 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.896 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.896 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.896 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.896 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.896 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.897 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.897 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-11-16T16:49:37.897 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.897 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.897 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.897 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_password": { 2023-11-16T16:49:37.897 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_password", 2023-11-16T16:49:37.898 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.898 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.898 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.898 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.898 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.898 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.898 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.898 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-11-16T16:49:37.899 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.899 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.899 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.899 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_url": { 2023-11-16T16:49:37.899 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_url", 2023-11-16T16:49:37.899 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.900 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.900 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.900 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.900 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.900 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.901 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.901 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-11-16T16:49:37.901 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.901 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.901 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.901 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_username": { 2023-11-16T16:49:37.902 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_username", 2023-11-16T16:49:37.902 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.902 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.902 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.902 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.902 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.902 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.902 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.903 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-11-16T16:49:37.903 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.903 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.903 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.903 INFO:teuthology.orchestra.run.smithi001.stdout: "secure_monitoring_stack": { 2023-11-16T16:49:37.903 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "secure_monitoring_stack", 2023-11-16T16:49:37.903 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.904 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.904 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.904 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.904 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.904 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.904 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.905 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-11-16T16:49:37.905 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.905 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.905 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.905 INFO:teuthology.orchestra.run.smithi001.stdout: "service_discovery_port": { 2023-11-16T16:49:37.905 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "service_discovery_port", 2023-11-16T16:49:37.905 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.906 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.906 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.906 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8765", 2023-11-16T16:49:37.906 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.906 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.906 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.906 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "cephadm service discovery port", 2023-11-16T16:49:37.906 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.906 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.907 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.907 INFO:teuthology.orchestra.run.smithi001.stdout: "ssh_config_file": { 2023-11-16T16:49:37.907 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssh_config_file", 2023-11-16T16:49:37.907 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.907 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.907 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.907 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.908 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.908 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.908 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.908 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-11-16T16:49:37.908 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.908 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.908 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.909 INFO:teuthology.orchestra.run.smithi001.stdout: "use_agent": { 2023-11-16T16:49:37.909 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_agent", 2023-11-16T16:49:37.909 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.909 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.909 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.909 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.909 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.909 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.910 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.910 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-11-16T16:49:37.910 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.910 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.910 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.910 INFO:teuthology.orchestra.run.smithi001.stdout: "use_repo_digest": { 2023-11-16T16:49:37.911 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_repo_digest", 2023-11-16T16:49:37.911 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.911 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.911 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.911 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.911 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.911 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.912 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.912 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-11-16T16:49:37.912 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.912 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.912 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.912 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_failed_host_check": { 2023-11-16T16:49:37.912 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_failed_host_check", 2023-11-16T16:49:37.913 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.913 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.913 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.913 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.913 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.913 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.913 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.913 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if the host check fails", 2023-11-16T16:49:37.913 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.914 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.914 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.914 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_daemons": { 2023-11-16T16:49:37.914 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_daemons", 2023-11-16T16:49:37.914 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.914 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.914 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.915 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.915 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.915 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.915 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.915 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-11-16T16:49:37.915 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.915 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.915 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.916 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_hosts": { 2023-11-16T16:49:37.916 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_hosts", 2023-11-16T16:49:37.916 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.916 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.916 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.916 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.916 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.917 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.917 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.917 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-11-16T16:49:37.917 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.917 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.917 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.917 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.918 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:37.918 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crash", 2023-11-16T16:49:37.918 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:37.918 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:37.918 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:37.918 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:37.919 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:37.919 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.919 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.919 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.919 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.919 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.919 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.919 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.920 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.920 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.920 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.920 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.920 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.920 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.920 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.921 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.921 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.921 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.921 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.921 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:37.921 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:37.922 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.922 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.922 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.922 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.922 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.922 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.922 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.922 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.923 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.923 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.923 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.923 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:37.923 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:37.923 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.924 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.924 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.924 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:37.924 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.924 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.924 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:37.924 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:37.924 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:37.925 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:37.925 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:37.925 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:37.925 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:37.925 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:37.925 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.925 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.926 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.926 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.926 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:37.926 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:37.926 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.926 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.927 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.927 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.927 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.927 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.927 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.927 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.927 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.927 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.928 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.928 INFO:teuthology.orchestra.run.smithi001.stdout: "retain_interval": { 2023-11-16T16:49:37.928 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retain_interval", 2023-11-16T16:49:37.928 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:37.928 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.928 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.928 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "31536000", 2023-11-16T16:49:37.929 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.929 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.929 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.929 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain crashes before pruning them", 2023-11-16T16:49:37.929 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.929 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.929 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.930 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_recent_interval": { 2023-11-16T16:49:37.930 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_recent_interval", 2023-11-16T16:49:37.930 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:37.930 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.930 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:37.930 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1209600", 2023-11-16T16:49:37.930 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.931 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.931 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.931 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "time interval in which to warn about recent crashes", 2023-11-16T16:49:37.931 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.931 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.931 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.931 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.931 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:37.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.932 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:37.932 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dashboard", 2023-11-16T16:49:37.932 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:37.932 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:37.932 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:37.932 INFO:teuthology.orchestra.run.smithi001.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-11-16T16:49:37.932 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-11-16T16:49:37.932 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:37.933 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.933 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.933 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2023-11-16T16:49:37.933 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.933 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.933 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.933 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.933 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.934 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.934 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.934 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_HOST": { 2023-11-16T16:49:37.934 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_HOST", 2023-11-16T16:49:37.934 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.935 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.935 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.935 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.935 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.936 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.936 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.936 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.937 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.937 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.937 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.938 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-11-16T16:49:37.938 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-11-16T16:49:37.938 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.939 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.939 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.939 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.939 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.940 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.940 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.940 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.940 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.941 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.941 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.941 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_ENABLED": { 2023-11-16T16:49:37.942 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_ENABLED", 2023-11-16T16:49:37.942 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.942 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.942 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.943 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.943 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.943 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.943 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.944 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.944 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.944 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.944 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.945 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-11-16T16:49:37.945 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-11-16T16:49:37.945 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.946 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.946 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.946 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.947 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.947 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.947 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.947 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.947 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.948 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.948 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.948 INFO:teuthology.orchestra.run.smithi001.stdout: "ENABLE_BROWSABLE_API": { 2023-11-16T16:49:37.948 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ENABLE_BROWSABLE_API", 2023-11-16T16:49:37.948 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.948 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.948 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.949 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.949 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.949 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.949 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.949 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.949 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.949 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.949 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.950 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-11-16T16:49:37.950 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-11-16T16:49:37.950 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.950 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.950 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.950 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.951 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.951 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.951 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.951 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.951 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.951 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.951 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.952 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-11-16T16:49:37.952 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-11-16T16:49:37.952 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.952 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.952 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.952 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.952 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.952 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.953 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.953 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.953 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.953 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.953 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.953 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-11-16T16:49:37.953 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-11-16T16:49:37.953 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.954 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.954 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.954 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.954 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.954 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.954 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.954 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.955 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.955 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.955 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.955 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-11-16T16:49:37.955 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-11-16T16:49:37.955 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.955 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.956 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.956 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.956 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.956 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.956 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.956 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.956 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.956 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.957 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.957 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_NFS": { 2023-11-16T16:49:37.957 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-11-16T16:49:37.957 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.957 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.957 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.958 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.958 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.958 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.958 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.958 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.958 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.958 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.959 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.959 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RBD": { 2023-11-16T16:49:37.959 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-11-16T16:49:37.959 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.959 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.959 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.960 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.960 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.960 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.960 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.960 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.960 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.960 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.961 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.961 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RGW": { 2023-11-16T16:49:37.961 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-11-16T16:49:37.961 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.961 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.961 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.961 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.962 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.962 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.962 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.962 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.962 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.962 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.962 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.963 INFO:teuthology.orchestra.run.smithi001.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-11-16T16:49:37.963 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-11-16T16:49:37.963 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.963 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.963 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.963 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.963 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.964 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.964 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.964 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.964 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.964 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.964 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.964 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_PASSWORD": { 2023-11-16T16:49:37.965 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_PASSWORD", 2023-11-16T16:49:37.965 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.965 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.965 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.965 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2023-11-16T16:49:37.965 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.965 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.965 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.966 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.966 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.966 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.966 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.966 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-11-16T16:49:37.966 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-11-16T16:49:37.966 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.967 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.967 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.967 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.967 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.967 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.967 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.967 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.968 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.968 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.968 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.968 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_URL": { 2023-11-16T16:49:37.968 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_URL", 2023-11-16T16:49:37.968 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.968 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.969 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.969 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.969 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.969 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.969 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.969 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.969 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.969 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.970 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.970 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_USERNAME": { 2023-11-16T16:49:37.970 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_USERNAME", 2023-11-16T16:49:37.970 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.970 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.970 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.970 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2023-11-16T16:49:37.971 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.971 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.971 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.971 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.971 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.971 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.971 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.972 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-11-16T16:49:37.972 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-11-16T16:49:37.972 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.972 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.972 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.972 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.972 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.972 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.973 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.973 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.973 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.973 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.973 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.973 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-11-16T16:49:37.973 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-11-16T16:49:37.974 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.974 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.974 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.974 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.974 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.974 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.974 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.974 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.975 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.975 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.975 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.975 INFO:teuthology.orchestra.run.smithi001.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-11-16T16:49:37.975 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-11-16T16:49:37.975 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.975 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.976 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.976 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.976 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.976 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.976 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.976 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.977 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.977 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.977 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.978 INFO:teuthology.orchestra.run.smithi001.stdout: "ISSUE_TRACKER_API_KEY": { 2023-11-16T16:49:37.978 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-11-16T16:49:37.978 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.978 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.978 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.978 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.978 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.979 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.979 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.979 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.979 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.979 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.979 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.979 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_HOST": { 2023-11-16T16:49:37.980 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_HOST", 2023-11-16T16:49:37.980 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.980 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.980 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.980 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:37.980 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.980 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.980 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.981 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.981 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.981 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.981 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.981 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-11-16T16:49:37.981 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-11-16T16:49:37.981 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.982 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.982 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.982 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.982 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.982 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.982 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.982 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.983 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.983 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.983 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.983 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-11-16T16:49:37.983 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-11-16T16:49:37.983 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.983 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.984 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.984 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.984 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.984 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.984 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.984 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.984 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.985 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.985 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.985 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-11-16T16:49:37.985 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-11-16T16:49:37.986 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.986 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.986 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.986 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.986 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.986 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.986 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.987 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.987 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.987 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.987 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.987 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-11-16T16:49:37.987 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-11-16T16:49:37.987 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.988 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.988 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.988 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.988 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.988 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.988 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.988 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.989 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.989 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.989 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.989 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-11-16T16:49:37.989 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-11-16T16:49:37.989 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.989 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.990 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.990 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.990 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.990 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.990 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.990 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.990 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.990 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.991 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.991 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-11-16T16:49:37.991 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-11-16T16:49:37.991 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.991 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.991 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.991 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.992 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.992 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.992 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.992 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.992 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.992 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.992 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.993 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-11-16T16:49:37.993 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-11-16T16:49:37.993 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.993 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.993 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.993 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.993 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.994 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.994 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.994 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.994 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.994 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.994 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.995 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-11-16T16:49:37.995 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-11-16T16:49:37.995 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.995 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.995 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.995 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:37.996 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.996 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.996 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.996 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.996 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.996 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.996 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.997 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_ENABLED": { 2023-11-16T16:49:37.997 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_ENABLED", 2023-11-16T16:49:37.997 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:37.997 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.997 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.997 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:37.997 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.997 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.998 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.998 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:37.998 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:37.998 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:37.998 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:37.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:37.998 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-11-16T16:49:37.998 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-11-16T16:49:37.999 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:37.999 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:37.999 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:37.999 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-11-16T16:49:37.999 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:37.999 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:37.999 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:37.999 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.000 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.000 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.000 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.000 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-11-16T16:49:38.000 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-11-16T16:49:38.000 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.000 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.001 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.001 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2023-11-16T16:49:38.001 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.001 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.001 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.001 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.001 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.001 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.002 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.002 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-11-16T16:49:38.002 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-11-16T16:49:38.002 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.002 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.002 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.003 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8", 2023-11-16T16:49:38.003 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.003 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.003 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.003 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.003 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.003 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.003 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.004 INFO:teuthology.orchestra.run.smithi001.stdout: "REST_REQUESTS_TIMEOUT": { 2023-11-16T16:49:38.004 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-11-16T16:49:38.004 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.004 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.004 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.004 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "45", 2023-11-16T16:49:38.004 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.005 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.005 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.005 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.005 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.005 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.005 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.005 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ACCESS_KEY": { 2023-11-16T16:49:38.006 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ACCESS_KEY", 2023-11-16T16:49:38.006 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.006 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.006 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.006 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.006 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.006 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.006 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.007 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.007 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.007 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.007 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.007 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-11-16T16:49:38.007 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-11-16T16:49:38.007 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.008 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.008 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.008 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2023-11-16T16:49:38.008 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.008 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.008 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.008 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.008 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.009 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.009 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.009 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SECRET_KEY": { 2023-11-16T16:49:38.009 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SECRET_KEY", 2023-11-16T16:49:38.009 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.009 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.009 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.010 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.010 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.010 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.010 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.010 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.010 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.010 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.010 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.011 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SSL_VERIFY": { 2023-11-16T16:49:38.011 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SSL_VERIFY", 2023-11-16T16:49:38.011 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.011 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.011 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.011 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.012 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.012 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.012 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.012 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.012 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.012 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.012 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.013 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-11-16T16:49:38.013 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-11-16T16:49:38.013 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.013 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.013 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.013 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2023-11-16T16:49:38.013 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.013 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.014 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.014 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.014 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.014 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.014 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.014 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-11-16T16:49:38.014 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-11-16T16:49:38.015 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.015 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.015 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.015 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2023-11-16T16:49:38.015 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.015 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.015 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.015 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.016 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.016 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.016 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.016 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-11-16T16:49:38.016 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-11-16T16:49:38.016 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.016 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.017 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.017 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2023-11-16T16:49:38.017 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.017 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.017 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.017 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.017 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.018 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.018 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.018 INFO:teuthology.orchestra.run.smithi001.stdout: "cross_origin_url": { 2023-11-16T16:49:38.018 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cross_origin_url", 2023-11-16T16:49:38.018 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.018 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.019 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.019 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.019 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.019 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.019 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.019 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.019 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.019 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.020 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.020 INFO:teuthology.orchestra.run.smithi001.stdout: "crt_file": { 2023-11-16T16:49:38.020 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crt_file", 2023-11-16T16:49:38.020 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.020 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.020 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.020 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.021 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.021 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.021 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.021 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.021 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.021 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.021 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.022 INFO:teuthology.orchestra.run.smithi001.stdout: "debug": { 2023-11-16T16:49:38.022 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "debug", 2023-11-16T16:49:38.022 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.022 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.022 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.022 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.022 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.022 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.023 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.023 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable/disable debug options", 2023-11-16T16:49:38.023 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.023 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.023 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.023 INFO:teuthology.orchestra.run.smithi001.stdout: "jwt_token_ttl": { 2023-11-16T16:49:38.024 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "jwt_token_ttl", 2023-11-16T16:49:38.024 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.024 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.024 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.024 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "28800", 2023-11-16T16:49:38.024 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.024 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.024 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.025 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.025 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.025 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.025 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.025 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2023-11-16T16:49:38.025 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2023-11-16T16:49:38.025 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.026 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.026 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.026 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.026 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.026 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.026 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.026 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.027 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.027 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.027 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.027 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.027 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.027 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.027 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.027 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.028 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.028 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.028 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.028 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.028 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.028 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.028 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.029 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.029 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.029 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.029 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.029 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.029 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.029 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.029 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.030 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.030 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.030 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.030 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.030 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.030 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.030 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.031 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.031 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.031 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.031 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.031 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.031 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.031 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.032 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.032 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.032 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.032 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.032 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.032 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.032 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.032 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.033 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.033 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.033 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.033 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.033 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.033 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.033 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.033 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.034 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.034 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.034 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.034 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.034 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.034 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.035 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.035 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.035 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.035 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.035 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.035 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.035 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.035 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.035 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.036 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.036 INFO:teuthology.orchestra.run.smithi001.stdout: "motd": { 2023-11-16T16:49:38.036 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "motd", 2023-11-16T16:49:38.036 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.036 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.036 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.036 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.037 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.037 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.037 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.037 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "The message of the day", 2023-11-16T16:49:38.037 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.037 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.037 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.038 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect_resolve_ip_addr": { 2023-11-16T16:49:38.038 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "redirect_resolve_ip_addr", 2023-11-16T16:49:38.038 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.038 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.038 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.038 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.038 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.038 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.039 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.039 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.039 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.039 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.039 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.039 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2023-11-16T16:49:38.039 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2023-11-16T16:49:38.040 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.040 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.040 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.040 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2023-11-16T16:49:38.040 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.040 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.040 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.040 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.041 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.041 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.041 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.041 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2023-11-16T16:49:38.041 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2023-11-16T16:49:38.041 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.041 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.042 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.042 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8080", 2023-11-16T16:49:38.042 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.042 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.042 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.042 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.042 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.042 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.043 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.043 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2023-11-16T16:49:38.043 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2023-11-16T16:49:38.043 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.043 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.043 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.043 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.044 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.044 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.044 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.044 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.044 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.044 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.044 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.044 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl_server_port": { 2023-11-16T16:49:38.045 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl_server_port", 2023-11-16T16:49:38.045 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.045 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.045 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.045 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8443", 2023-11-16T16:49:38.045 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.045 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.045 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.046 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.046 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.046 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.046 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.046 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2023-11-16T16:49:38.046 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2023-11-16T16:49:38.046 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.047 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.047 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.047 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "redirect", 2023-11-16T16:49:38.047 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.047 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.047 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.047 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.048 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect" 2023-11-16T16:49:38.048 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.048 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.048 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.048 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.048 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.048 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2023-11-16T16:49:38.049 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2023-11-16T16:49:38.049 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.049 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.049 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.049 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2023-11-16T16:49:38.049 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2023-11-16T16:49:38.049 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2023-11-16T16:49:38.049 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.050 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.050 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.050 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.050 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.050 INFO:teuthology.orchestra.run.smithi001.stdout: "url_prefix": { 2023-11-16T16:49:38.050 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url_prefix", 2023-11-16T16:49:38.050 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.051 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.051 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.051 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.051 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.051 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.051 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.051 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.051 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.052 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.052 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.052 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.052 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.052 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.052 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "devicehealth", 2023-11-16T16:49:38.052 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.053 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.053 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.053 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_monitoring": { 2023-11-16T16:49:38.053 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_monitoring", 2023-11-16T16:49:38.053 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.053 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.053 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.053 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.053 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.054 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.054 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.054 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "monitor device health metrics", 2023-11-16T16:49:38.054 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.054 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.054 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.054 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.055 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.055 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.055 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.055 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.055 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.055 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.055 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.055 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.056 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.056 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.056 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.056 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.056 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.056 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.056 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.056 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.057 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.057 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.057 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.057 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.057 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.057 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.057 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.058 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.058 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.058 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.058 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.058 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.058 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.058 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.058 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.059 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.059 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.059 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.059 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.059 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.059 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.060 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.060 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.060 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.060 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.060 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.060 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.060 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.061 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.061 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.061 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.061 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.061 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.061 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.061 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.061 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.062 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.062 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.062 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.062 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.062 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.062 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.062 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.063 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.063 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.063 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.063 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.063 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.063 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.063 INFO:teuthology.orchestra.run.smithi001.stdout: "mark_out_threshold": { 2023-11-16T16:49:38.064 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mark_out_threshold", 2023-11-16T16:49:38.064 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:38.064 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.064 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.064 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2419200", 2023-11-16T16:49:38.064 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.064 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.064 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.065 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-11-16T16:49:38.065 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.065 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.065 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.065 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": { 2023-11-16T16:49:38.065 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_name", 2023-11-16T16:49:38.065 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.066 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.066 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.066 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "device_health_metrics", 2023-11-16T16:49:38.066 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.066 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.066 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.066 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name of pool in which to store device health metrics", 2023-11-16T16:49:38.066 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.067 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.067 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.067 INFO:teuthology.orchestra.run.smithi001.stdout: "retention_period": { 2023-11-16T16:49:38.067 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retention_period", 2023-11-16T16:49:38.067 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:38.067 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.067 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.068 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15552000", 2023-11-16T16:49:38.068 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.068 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.068 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.068 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain device health metrics", 2023-11-16T16:49:38.068 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.068 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.068 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.069 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.069 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_frequency": { 2023-11-16T16:49:38.069 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_frequency", 2023-11-16T16:49:38.069 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:38.069 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.069 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.069 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2023-11-16T16:49:38.069 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.070 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.070 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.070 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to scrape device health metrics", 2023-11-16T16:49:38.070 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.070 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.070 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.070 INFO:teuthology.orchestra.run.smithi001.stdout: "self_heal": { 2023-11-16T16:49:38.071 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "self_heal", 2023-11-16T16:49:38.071 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.071 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.071 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.071 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.071 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.071 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.071 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.072 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-11-16T16:49:38.072 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.072 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.072 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.072 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2023-11-16T16:49:38.072 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2023-11-16T16:49:38.072 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:38.073 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.073 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.073 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2023-11-16T16:49:38.073 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.073 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.073 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.073 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and check device health", 2023-11-16T16:49:38.073 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.074 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.074 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.074 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_threshold": { 2023-11-16T16:49:38.074 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_threshold", 2023-11-16T16:49:38.074 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:38.074 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.074 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.074 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7257600", 2023-11-16T16:49:38.075 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.075 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.075 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.075 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-11-16T16:49:38.075 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.075 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.075 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.075 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.076 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.076 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.076 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "diskprediction_local", 2023-11-16T16:49:38.076 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.076 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.076 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.076 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.077 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.077 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.077 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.077 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.077 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.077 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.078 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.078 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.078 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.078 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.078 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.078 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.078 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.078 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.079 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.079 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.079 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.079 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.079 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.079 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.079 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.080 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.080 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.080 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.080 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.080 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.080 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.080 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.080 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.081 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.081 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.081 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.081 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.081 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.081 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.081 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.082 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.082 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.082 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.082 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.082 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.082 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.082 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.082 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.083 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.083 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.083 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.083 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.083 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.083 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.083 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.083 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.084 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.084 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.084 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.084 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.084 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.084 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.084 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.085 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.085 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.085 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.085 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.085 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.085 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.085 INFO:teuthology.orchestra.run.smithi001.stdout: "predict_interval": { 2023-11-16T16:49:38.086 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predict_interval", 2023-11-16T16:49:38.086 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.086 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.086 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.086 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2023-11-16T16:49:38.086 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.086 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.086 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.087 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.087 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.087 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.087 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.087 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.087 INFO:teuthology.orchestra.run.smithi001.stdout: "predictor_model": { 2023-11-16T16:49:38.087 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predictor_model", 2023-11-16T16:49:38.087 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.088 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.088 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.088 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "prophetstor", 2023-11-16T16:49:38.088 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.088 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.088 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.088 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.088 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.089 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.089 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.089 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2023-11-16T16:49:38.089 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2023-11-16T16:49:38.089 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.089 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.089 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.090 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2023-11-16T16:49:38.090 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.090 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.090 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.090 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.090 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.090 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.090 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.091 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.091 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.091 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.091 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "influx", 2023-11-16T16:49:38.091 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": false, 2023-11-16T16:49:38.091 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "influxdb python module not found", 2023-11-16T16:49:38.091 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.092 INFO:teuthology.orchestra.run.smithi001.stdout: "batch_size": { 2023-11-16T16:49:38.092 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "batch_size", 2023-11-16T16:49:38.092 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.092 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.092 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.092 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5000", 2023-11-16T16:49:38.092 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.092 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.093 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.093 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-11-16T16:49:38.093 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.093 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.093 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.093 INFO:teuthology.orchestra.run.smithi001.stdout: "database": { 2023-11-16T16:49:38.093 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "database", 2023-11-16T16:49:38.094 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.094 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.094 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.094 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "ceph", 2023-11-16T16:49:38.094 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.094 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.094 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.094 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-11-16T16:49:38.095 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.095 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.095 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.095 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": { 2023-11-16T16:49:38.095 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "hostname", 2023-11-16T16:49:38.095 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.095 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.096 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.096 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.096 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.096 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.096 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.096 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server hostname", 2023-11-16T16:49:38.096 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.096 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.097 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.097 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2023-11-16T16:49:38.097 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2023-11-16T16:49:38.097 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:38.097 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.097 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.097 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "30", 2023-11-16T16:49:38.097 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "5", 2023-11-16T16:49:38.098 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.098 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.098 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-11-16T16:49:38.098 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.098 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.098 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.098 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.099 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.099 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.099 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.099 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.099 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.099 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.099 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.099 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.100 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.100 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.100 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.100 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.100 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.100 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.100 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.100 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.101 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.101 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.101 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.101 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.101 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.101 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.101 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.102 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.102 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.102 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.102 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.102 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.102 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.103 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.103 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.103 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.103 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.103 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.103 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.103 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.104 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.104 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.104 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.104 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.104 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.104 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.104 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.105 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.105 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.105 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.105 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.105 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.105 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.105 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.105 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.105 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.106 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.106 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.106 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.106 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.106 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.106 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.107 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.107 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.107 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.107 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.107 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.107 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.107 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.108 INFO:teuthology.orchestra.run.smithi001.stdout: "password": { 2023-11-16T16:49:38.108 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "password", 2023-11-16T16:49:38.108 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.108 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.108 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.108 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.108 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.108 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.108 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.109 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "password of InfluxDB server user", 2023-11-16T16:49:38.109 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.109 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.109 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.109 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.109 INFO:teuthology.orchestra.run.smithi001.stdout: "port": { 2023-11-16T16:49:38.109 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "port", 2023-11-16T16:49:38.110 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.110 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.110 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.110 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8086", 2023-11-16T16:49:38.110 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.110 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.110 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.111 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server port", 2023-11-16T16:49:38.111 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.111 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.111 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.111 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2023-11-16T16:49:38.111 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2023-11-16T16:49:38.111 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.111 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.112 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.112 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "false", 2023-11-16T16:49:38.112 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.112 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.112 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.112 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-11-16T16:49:38.112 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.113 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.113 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.113 INFO:teuthology.orchestra.run.smithi001.stdout: "threads": { 2023-11-16T16:49:38.113 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threads", 2023-11-16T16:49:38.113 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.113 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.113 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.114 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2023-11-16T16:49:38.114 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2023-11-16T16:49:38.114 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "32", 2023-11-16T16:49:38.114 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.114 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-11-16T16:49:38.114 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.114 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.114 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.115 INFO:teuthology.orchestra.run.smithi001.stdout: "username": { 2023-11-16T16:49:38.115 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "username", 2023-11-16T16:49:38.115 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.115 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.115 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.115 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.115 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.116 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.116 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.116 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "username of InfluxDB server user", 2023-11-16T16:49:38.116 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.116 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.116 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.116 INFO:teuthology.orchestra.run.smithi001.stdout: "verify_ssl": { 2023-11-16T16:49:38.117 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "verify_ssl", 2023-11-16T16:49:38.117 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.117 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.117 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.117 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "true", 2023-11-16T16:49:38.117 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.117 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.117 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.118 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-11-16T16:49:38.118 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.118 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.118 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.119 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.119 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.119 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.120 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "insights", 2023-11-16T16:49:38.120 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.120 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.120 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.120 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.120 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.120 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.120 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.121 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.121 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.121 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.121 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.121 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.121 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.121 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.121 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.122 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.122 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.122 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.122 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.122 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.122 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.122 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.122 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.123 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.123 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.123 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.123 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.123 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.123 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.123 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.123 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.124 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.124 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.124 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.124 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.124 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.124 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.124 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.125 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.125 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.125 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.125 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.125 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.125 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.125 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.125 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.126 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.126 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.126 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.126 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.126 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.126 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.126 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.127 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.127 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.127 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.127 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.127 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.127 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.127 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.128 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.128 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.128 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.128 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.128 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.128 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.128 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.128 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.129 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.129 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.129 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.129 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.129 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "iostat", 2023-11-16T16:49:38.129 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.129 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.130 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.130 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.130 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.130 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.130 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.130 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.130 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.130 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.131 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.131 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.131 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.131 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.131 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.131 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.131 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.131 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.132 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.132 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.132 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.132 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.132 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.132 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.132 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.133 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.133 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.133 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.133 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.133 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.133 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.133 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.133 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.134 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.134 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.134 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.134 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.134 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.134 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.134 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.135 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.135 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.135 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.135 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.135 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.135 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.135 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.135 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.136 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.136 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.136 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.136 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.136 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.136 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.136 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.136 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.137 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.137 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.137 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.137 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.137 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.137 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.137 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.137 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.138 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.138 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.138 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.138 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.138 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.138 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.138 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.139 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.139 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "k8sevents", 2023-11-16T16:49:38.139 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.139 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.139 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.139 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_event_retention_days": { 2023-11-16T16:49:38.139 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ceph_event_retention_days", 2023-11-16T16:49:38.140 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.140 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.140 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.140 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7", 2023-11-16T16:49:38.140 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.140 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.140 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.140 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Days to hold ceph event information within local cache", 2023-11-16T16:49:38.141 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.141 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.141 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.141 INFO:teuthology.orchestra.run.smithi001.stdout: "config_check_secs": { 2023-11-16T16:49:38.141 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_check_secs", 2023-11-16T16:49:38.141 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.141 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.142 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.142 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2023-11-16T16:49:38.142 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "10", 2023-11-16T16:49:38.142 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.142 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.142 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-11-16T16:49:38.142 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.142 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.143 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.143 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.143 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.143 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.143 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.143 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.143 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.143 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.144 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.144 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.144 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.144 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.144 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.144 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.145 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.145 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.145 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.145 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.146 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.146 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.146 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.146 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.146 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.147 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.147 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.147 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.147 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.147 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.147 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.147 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.147 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.148 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.148 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.148 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.148 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.148 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.148 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.148 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.148 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.148 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.149 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.149 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.149 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.149 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.149 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.149 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.149 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.150 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.150 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.150 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.150 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.150 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.150 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.150 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.150 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.151 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.151 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.151 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.151 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.151 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.151 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.151 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.151 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.152 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.152 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.152 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.152 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.152 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.152 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.152 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.153 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.153 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "localpool", 2023-11-16T16:49:38.153 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.153 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.153 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.153 INFO:teuthology.orchestra.run.smithi001.stdout: "failure_domain": { 2023-11-16T16:49:38.153 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "failure_domain", 2023-11-16T16:49:38.153 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.154 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.154 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.154 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "host", 2023-11-16T16:49:38.154 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.154 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.154 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.154 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "failure domain for any created local pool", 2023-11-16T16:49:38.154 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-11-16T16:49:38.155 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.155 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.155 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.155 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.155 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.155 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.155 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.156 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.156 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.156 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.156 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.156 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.156 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.156 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.156 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.157 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.157 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.157 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.157 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.157 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.157 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.158 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.158 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.158 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.159 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.159 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.159 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.159 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.159 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.159 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.160 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.160 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.160 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.160 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.160 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.161 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.161 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.161 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.162 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.162 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.162 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.163 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.163 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.163 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.163 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.163 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.164 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.164 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.164 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.164 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.164 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.164 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.164 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.164 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.165 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.165 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.165 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.165 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.165 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.165 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.165 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.165 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.166 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.166 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.166 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.166 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.166 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.166 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.166 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": { 2023-11-16T16:49:38.167 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_size", 2023-11-16T16:49:38.167 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.167 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.167 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.167 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.167 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.167 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.167 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.168 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default min_size for any created local pool", 2023-11-16T16:49:38.168 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-11-16T16:49:38.168 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.168 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.168 INFO:teuthology.orchestra.run.smithi001.stdout: "num_rep": { 2023-11-16T16:49:38.168 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "num_rep", 2023-11-16T16:49:38.168 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.169 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.169 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.169 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3", 2023-11-16T16:49:38.169 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.169 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.169 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.169 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default replica count for any created local pool", 2023-11-16T16:49:38.169 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.170 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.170 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.170 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": { 2023-11-16T16:49:38.170 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_num", 2023-11-16T16:49:38.170 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.170 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.170 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.171 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "128", 2023-11-16T16:49:38.171 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.171 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.171 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.171 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default pg_num for any created local pool", 2023-11-16T16:49:38.171 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.171 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.171 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.172 INFO:teuthology.orchestra.run.smithi001.stdout: "prefix": { 2023-11-16T16:49:38.172 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prefix", 2023-11-16T16:49:38.172 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.172 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.172 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.172 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.172 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.173 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.173 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.173 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name prefix for any created local pool", 2023-11-16T16:49:38.173 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.173 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.173 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.173 INFO:teuthology.orchestra.run.smithi001.stdout: "subtree": { 2023-11-16T16:49:38.174 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "subtree", 2023-11-16T16:49:38.174 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.174 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.174 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.174 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "rack", 2023-11-16T16:49:38.174 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.174 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.175 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.175 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "CRUSH level for which to create a local pool", 2023-11-16T16:49:38.175 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-11-16T16:49:38.175 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.175 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.175 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.175 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.176 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.176 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mds_autoscaler", 2023-11-16T16:49:38.176 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.176 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.176 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.176 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.176 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.176 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.177 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.177 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.177 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.177 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.177 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.177 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.177 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.178 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.178 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.178 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.178 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.178 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.178 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.178 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.178 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.179 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.179 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.179 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.179 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.179 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.180 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.180 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.180 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.180 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.181 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.181 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.181 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.181 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.182 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.182 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.182 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.182 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.182 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.183 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.183 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.183 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.183 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.183 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.183 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.183 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.183 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.184 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.184 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.184 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.184 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.184 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.184 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.184 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.184 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.185 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.185 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.185 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.185 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.185 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.185 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.186 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.186 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.186 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.186 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.186 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.187 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.187 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.187 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.187 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.187 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.188 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirroring", 2023-11-16T16:49:38.188 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.188 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.188 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.188 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.188 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.189 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.189 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.189 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.189 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.189 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.189 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.189 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.189 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.190 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.190 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.190 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.190 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.190 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.190 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.190 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.190 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.191 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.191 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.191 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.191 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.191 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.191 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.191 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.192 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.192 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.192 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.192 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.192 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.192 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.192 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.192 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.193 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.193 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.193 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.193 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.193 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.193 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.193 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.194 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.194 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.194 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.194 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.194 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.194 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.194 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.194 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.195 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.195 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.195 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.195 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.195 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.195 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.195 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.196 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.196 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.196 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.196 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.196 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.196 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.196 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.196 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.197 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.197 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.197 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.197 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.197 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.197 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.197 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "nfs", 2023-11-16T16:49:38.198 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.198 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.198 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.198 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.198 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.198 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.198 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.198 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.199 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.199 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.199 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.199 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.199 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.199 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.199 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.200 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.200 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.200 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.200 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.200 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.200 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.200 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.200 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.201 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.201 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.201 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.201 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.201 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.201 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.201 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.202 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.202 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.202 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.202 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.202 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.202 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.202 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.203 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.203 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.203 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.203 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.203 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.203 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.203 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.203 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.204 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.204 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.204 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.204 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.204 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.204 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.204 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.204 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.205 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.205 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.205 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.205 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.205 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.205 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.205 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.206 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.206 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.206 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.206 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.206 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.206 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.207 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.207 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.207 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.207 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.207 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.208 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.208 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2023-11-16T16:49:38.208 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.208 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.208 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.209 INFO:teuthology.orchestra.run.smithi001.stdout: "fail_fs": { 2023-11-16T16:49:38.209 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "fail_fs", 2023-11-16T16:49:38.209 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.209 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.210 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.210 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.210 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.210 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.210 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.211 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-11-16T16:49:38.211 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.211 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.211 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.212 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.212 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.212 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.212 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.212 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.213 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.213 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.213 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.213 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.213 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.214 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.214 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.214 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.214 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.214 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.215 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.215 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.215 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.215 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.215 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.216 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.216 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.216 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.216 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.217 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.217 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.217 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.217 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.217 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.218 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.218 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.218 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.218 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.218 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.219 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.219 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.219 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.219 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.219 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.220 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.220 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.220 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.220 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.220 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.221 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.221 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.221 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.221 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.221 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.222 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.222 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.222 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.222 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.222 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.223 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.223 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.223 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.223 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.224 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.224 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.224 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.224 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.224 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.224 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.225 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.225 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.225 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.225 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator": { 2023-11-16T16:49:38.226 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2023-11-16T16:49:38.226 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.226 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.226 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.226 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.227 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.227 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.227 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.227 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm", 2023-11-16T16:49:38.228 INFO:teuthology.orchestra.run.smithi001.stdout: "rook", 2023-11-16T16:49:38.228 INFO:teuthology.orchestra.run.smithi001.stdout: "test_orchestrator" 2023-11-16T16:49:38.228 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.228 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Orchestrator backend", 2023-11-16T16:49:38.228 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.229 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.229 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.229 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.229 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.230 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.230 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_perf_query", 2023-11-16T16:49:38.230 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.230 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.230 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.231 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.231 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.231 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.231 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.231 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.232 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.232 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.232 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.232 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.232 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.233 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.233 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.233 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.233 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.233 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.234 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.234 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.234 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.234 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.234 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.235 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.235 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.235 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.235 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.236 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.236 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.236 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.236 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.236 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.237 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.237 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.237 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.237 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.238 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.238 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.238 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.238 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.238 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.239 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.239 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.239 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.239 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.239 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.240 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.240 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.240 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.240 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.240 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.241 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.241 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.241 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.241 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.241 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.242 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.242 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.242 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.242 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.243 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.243 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.243 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.243 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.243 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.244 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.244 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.244 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.244 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.244 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.245 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.245 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.245 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_support", 2023-11-16T16:49:38.245 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.246 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.246 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.246 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.246 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.246 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.246 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.246 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.246 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.247 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.247 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.247 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.247 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.247 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.247 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.247 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.247 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.248 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.248 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.248 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.248 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.248 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.248 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.248 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.249 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.249 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.249 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.249 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.249 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.249 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.249 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.249 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.250 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.250 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.250 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.250 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.250 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.250 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.251 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.251 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.251 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.251 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.251 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.251 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.251 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.251 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.252 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.252 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.252 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.252 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.252 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.252 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.252 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.253 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.253 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.253 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.253 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.254 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.254 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.254 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.254 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.254 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.255 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.255 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.255 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.255 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.255 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.256 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.256 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.256 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.256 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.257 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.257 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_autoscaler", 2023-11-16T16:49:38.257 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.257 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.258 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.258 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.258 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.258 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.258 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.259 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.259 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.259 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.259 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.259 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.260 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.260 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.260 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.260 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.260 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.261 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.261 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.261 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.261 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.261 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.262 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.262 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.262 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.263 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.263 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.263 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.263 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.263 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.264 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.264 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.264 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.264 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.264 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.265 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.265 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.265 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.265 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.266 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.266 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.266 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.266 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.266 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.267 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.267 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.267 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.267 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.267 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.268 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.268 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.268 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.268 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.268 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.269 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.269 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.269 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.269 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.270 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.270 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.270 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.270 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.270 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.271 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.271 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.271 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.271 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.271 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.272 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.272 INFO:teuthology.orchestra.run.smithi001.stdout: "noautoscale": { 2023-11-16T16:49:38.272 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "noautoscale", 2023-11-16T16:49:38.272 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.272 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.273 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.273 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.273 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.273 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.273 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.274 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "global autoscale flag", 2023-11-16T16:49:38.274 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-11-16T16:49:38.274 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.274 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.275 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2023-11-16T16:49:38.275 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2023-11-16T16:49:38.275 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:38.275 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.275 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.276 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2023-11-16T16:49:38.276 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.276 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.276 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.276 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.277 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.277 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.277 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.277 INFO:teuthology.orchestra.run.smithi001.stdout: "threshold": { 2023-11-16T16:49:38.278 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threshold", 2023-11-16T16:49:38.278 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2023-11-16T16:49:38.278 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.278 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.278 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2023-11-16T16:49:38.279 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1.0", 2023-11-16T16:49:38.279 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.279 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.279 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "scaling threshold", 2023-11-16T16:49:38.279 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-11-16T16:49:38.279 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.280 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.280 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.280 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.280 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.280 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "progress", 2023-11-16T16:49:38.280 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.280 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.281 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.281 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_pg_recovery_event": { 2023-11-16T16:49:38.281 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_pg_recovery_event", 2023-11-16T16:49:38.281 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.281 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.281 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.281 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.281 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.282 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.282 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.282 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow the module to show pg recovery progress", 2023-11-16T16:49:38.282 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.282 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.282 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.283 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2023-11-16T16:49:38.283 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2023-11-16T16:49:38.283 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.283 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.283 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.283 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.283 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.283 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.284 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.284 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.284 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.284 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.284 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.284 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.284 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.285 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.285 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.285 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.285 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.285 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.285 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.285 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.285 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.285 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.286 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.286 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.286 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.286 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.286 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.286 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.286 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.287 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.287 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.287 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.287 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.287 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.287 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.287 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.288 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.288 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.288 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.288 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.288 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.288 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.288 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.288 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.289 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.289 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.289 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.289 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.289 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.289 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.289 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.289 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.290 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.290 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.290 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.290 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.290 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.290 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.290 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.290 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.291 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.291 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.291 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.291 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.291 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.291 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.292 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.292 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.292 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.292 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.292 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.292 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.292 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.292 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.293 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.293 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.293 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.293 INFO:teuthology.orchestra.run.smithi001.stdout: "max_completed_events": { 2023-11-16T16:49:38.293 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_completed_events", 2023-11-16T16:49:38.293 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.293 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.294 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.294 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "50", 2023-11-16T16:49:38.294 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.294 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.294 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.294 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "number of past completed events to remember", 2023-11-16T16:49:38.294 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.294 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.295 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.295 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2023-11-16T16:49:38.295 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2023-11-16T16:49:38.295 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:38.295 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.295 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.295 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2023-11-16T16:49:38.296 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.296 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.296 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.296 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long the module is going to sleep", 2023-11-16T16:49:38.296 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.296 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.296 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.296 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.297 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.297 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.297 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus", 2023-11-16T16:49:38.297 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.297 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.297 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.297 INFO:teuthology.orchestra.run.smithi001.stdout: "cache": { 2023-11-16T16:49:38.298 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cache", 2023-11-16T16:49:38.298 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.298 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.298 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.298 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.298 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.298 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.298 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.299 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.299 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.299 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.299 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.299 INFO:teuthology.orchestra.run.smithi001.stdout: "exclude_perf_counters": { 2023-11-16T16:49:38.299 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "exclude_perf_counters", 2023-11-16T16:49:38.299 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.300 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.300 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.300 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.300 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.300 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.300 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.300 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-11-16T16:49:38.300 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-11-16T16:49:38.301 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.301 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.301 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.301 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.301 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.301 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.301 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.301 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.302 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.302 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.302 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.302 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.302 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.302 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.302 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.303 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.303 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.303 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.303 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.303 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.303 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.303 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.304 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.304 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.304 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.304 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.304 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.304 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.304 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.304 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.305 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.305 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.305 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.305 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.305 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.305 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.305 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.306 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.306 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.306 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.306 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.306 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.306 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.306 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.306 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.307 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.307 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.307 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.307 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.307 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.307 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.307 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.307 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.308 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.308 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.308 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.308 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.308 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.308 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.308 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.309 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.309 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.309 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.309 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.309 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.309 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.309 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.310 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.310 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools": { 2023-11-16T16:49:38.310 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools", 2023-11-16T16:49:38.310 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.310 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.310 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.310 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.311 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.311 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.311 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.311 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.311 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.311 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.311 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.312 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools_refresh_interval": { 2023-11-16T16:49:38.312 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-11-16T16:49:38.312 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.312 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.312 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.312 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "300", 2023-11-16T16:49:38.312 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.312 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.313 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.313 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.313 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.313 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.313 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.313 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_interval": { 2023-11-16T16:49:38.313 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_interval", 2023-11-16T16:49:38.314 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2023-11-16T16:49:38.314 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.314 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.314 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15.0", 2023-11-16T16:49:38.314 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.314 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.314 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.314 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.315 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.315 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.315 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.315 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2023-11-16T16:49:38.315 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2023-11-16T16:49:38.315 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.315 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.316 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.316 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2023-11-16T16:49:38.316 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.316 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.316 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.316 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-11-16T16:49:38.316 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.316 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.317 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.317 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2023-11-16T16:49:38.317 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2023-11-16T16:49:38.317 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.317 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.317 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.317 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "9283", 2023-11-16T16:49:38.318 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.318 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.318 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.318 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-11-16T16:49:38.318 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.318 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.318 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.319 INFO:teuthology.orchestra.run.smithi001.stdout: "stale_cache_strategy": { 2023-11-16T16:49:38.319 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stale_cache_strategy", 2023-11-16T16:49:38.319 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.319 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.319 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.319 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "log", 2023-11-16T16:49:38.319 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.319 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.320 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.320 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.320 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.320 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.320 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.320 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2023-11-16T16:49:38.321 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2023-11-16T16:49:38.321 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.321 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.321 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.321 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "default", 2023-11-16T16:49:38.321 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.321 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.321 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.321 INFO:teuthology.orchestra.run.smithi001.stdout: "default", 2023-11-16T16:49:38.322 INFO:teuthology.orchestra.run.smithi001.stdout: "error" 2023-11-16T16:49:38.322 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.322 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.322 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.322 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.322 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.323 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2023-11-16T16:49:38.323 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2023-11-16T16:49:38.323 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.323 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.323 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.323 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2023-11-16T16:49:38.323 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2023-11-16T16:49:38.323 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2023-11-16T16:49:38.323 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.324 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.324 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.324 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.324 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.324 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.324 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.324 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.325 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_support", 2023-11-16T16:49:38.325 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.325 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.325 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.325 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.325 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.325 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.325 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.326 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.326 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.326 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.326 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.326 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.326 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.326 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.327 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.327 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.327 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.327 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.327 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.327 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.327 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.327 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.328 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.328 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.328 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.328 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.328 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.328 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.328 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.329 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.329 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.329 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.329 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.329 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.329 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.329 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.330 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.330 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.330 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.330 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.330 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.330 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.330 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.331 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.331 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.331 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.334 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.334 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.334 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.334 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.334 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.334 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.335 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.335 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.335 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.335 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.335 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.335 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.335 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.336 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.336 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.336 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.336 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.336 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.336 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.336 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.336 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.337 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.337 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.337 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_snap_create": { 2023-11-16T16:49:38.337 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_snap_create", 2023-11-16T16:49:38.337 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.337 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.337 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.338 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2023-11-16T16:49:38.338 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.338 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.338 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.338 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.338 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.338 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.338 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.339 INFO:teuthology.orchestra.run.smithi001.stdout: "mirror_snapshot_schedule": { 2023-11-16T16:49:38.339 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirror_snapshot_schedule", 2023-11-16T16:49:38.339 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.339 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.339 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.339 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.340 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.340 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.340 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.340 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.340 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.340 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.340 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.340 INFO:teuthology.orchestra.run.smithi001.stdout: "trash_purge_schedule": { 2023-11-16T16:49:38.341 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "trash_purge_schedule", 2023-11-16T16:49:38.341 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.341 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.341 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.341 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.341 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.341 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.341 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.342 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.342 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.342 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.342 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.342 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.342 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.342 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.343 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "restful", 2023-11-16T16:49:38.343 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.343 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.343 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.343 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_auth": { 2023-11-16T16:49:38.343 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_auth", 2023-11-16T16:49:38.343 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.343 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.344 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.344 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.344 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.344 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.344 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.344 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.344 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.344 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.345 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.345 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2023-11-16T16:49:38.345 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2023-11-16T16:49:38.345 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.345 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.345 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.345 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.346 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.346 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.346 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.346 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.346 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.346 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.346 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.347 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.347 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.347 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.347 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.347 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.347 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.347 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.347 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.348 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.348 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.348 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.348 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.348 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.348 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.348 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.348 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.349 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.349 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.349 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.349 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.349 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.349 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.349 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.350 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.350 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.350 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.350 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.350 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.350 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.350 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.351 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.351 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.351 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.351 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.351 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.351 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.351 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.351 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.352 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.352 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.352 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.352 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.352 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.352 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.352 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.352 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.353 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.353 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.353 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.353 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.353 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.353 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.353 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.354 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.354 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.354 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.354 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.354 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.354 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.354 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.354 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.355 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.355 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.355 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.355 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.355 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.355 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2023-11-16T16:49:38.355 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2023-11-16T16:49:38.355 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.356 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.356 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.356 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.356 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.356 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.356 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.356 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.356 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.357 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.357 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.357 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2023-11-16T16:49:38.357 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2023-11-16T16:49:38.357 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.357 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.357 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.358 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.358 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.358 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.358 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.358 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.358 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.358 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.358 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.359 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.359 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.359 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.359 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rgw", 2023-11-16T16:49:38.359 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.359 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.360 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.360 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.360 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.360 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.360 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.360 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.360 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.360 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.361 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.361 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.361 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.361 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.361 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.361 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.361 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.361 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.362 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.362 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.362 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.362 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.362 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.362 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.362 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.363 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.363 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.363 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.363 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.363 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.363 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.363 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.364 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.364 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.364 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.364 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.364 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.364 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.364 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.364 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.365 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.365 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.365 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.365 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.365 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.365 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.365 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.365 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.366 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.366 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.366 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.366 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.366 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.366 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.366 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.366 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.367 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.367 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.367 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.367 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.367 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.367 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.368 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.368 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.368 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.368 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.368 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.368 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.368 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.368 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.368 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.369 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.369 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rook", 2023-11-16T16:49:38.369 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.369 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.369 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.369 INFO:teuthology.orchestra.run.smithi001.stdout: "drive_group_interval": { 2023-11-16T16:49:38.370 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "drive_group_interval", 2023-11-16T16:49:38.370 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2023-11-16T16:49:38.370 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.370 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.370 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "300.0", 2023-11-16T16:49:38.370 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.370 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.370 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.371 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-11-16T16:49:38.371 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.371 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.371 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.371 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.371 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.371 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.372 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.372 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.372 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.372 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.372 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.372 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.372 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.372 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.373 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.373 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.373 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.373 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.373 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.373 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.373 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.373 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.374 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.374 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.374 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.374 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.374 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.374 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.374 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.375 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.375 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.375 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.375 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.375 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.375 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.375 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.376 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.376 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.376 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.376 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.376 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.376 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.376 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.376 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.376 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.377 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.377 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.377 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.377 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.377 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.377 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.377 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.377 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.378 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.378 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.378 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.378 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.378 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.378 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.379 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.379 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.379 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.379 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.379 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.379 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.379 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.380 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.380 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.380 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.380 INFO:teuthology.orchestra.run.smithi001.stdout: "storage_class": { 2023-11-16T16:49:38.380 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "storage_class", 2023-11-16T16:49:38.380 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.380 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.381 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.381 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "local", 2023-11-16T16:49:38.381 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.381 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.381 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.381 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-11-16T16:49:38.381 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.381 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.382 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.382 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.382 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.382 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.382 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "selftest", 2023-11-16T16:49:38.382 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.382 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.383 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.383 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.383 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.383 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.383 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.383 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.383 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.383 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.383 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.384 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.384 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.384 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.384 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.384 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.384 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.384 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.384 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.385 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.385 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.385 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.385 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.385 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.385 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.385 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.386 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.386 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.386 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.386 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.386 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.386 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.386 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.387 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.387 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.387 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.387 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.387 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.387 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.388 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.388 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.388 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.388 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.388 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.388 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.388 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.388 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.389 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.389 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.389 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.389 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.389 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.389 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.389 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.389 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.390 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.390 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.390 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.390 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.390 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.390 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.390 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.391 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.391 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.391 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.391 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.391 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.391 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.391 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.392 INFO:teuthology.orchestra.run.smithi001.stdout: "roption1": { 2023-11-16T16:49:38.392 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption1", 2023-11-16T16:49:38.392 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.392 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.392 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.392 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.392 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.393 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.393 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.393 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.393 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.393 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.393 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.393 INFO:teuthology.orchestra.run.smithi001.stdout: "roption2": { 2023-11-16T16:49:38.394 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption2", 2023-11-16T16:49:38.394 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.394 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.394 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.394 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "xyz", 2023-11-16T16:49:38.394 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.394 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.395 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.395 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.395 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.395 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.395 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.395 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption1": { 2023-11-16T16:49:38.395 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption1", 2023-11-16T16:49:38.396 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.396 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.396 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.396 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.396 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.396 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.396 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.396 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.397 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.397 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.397 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.397 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption2": { 2023-11-16T16:49:38.397 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption2", 2023-11-16T16:49:38.397 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.397 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.398 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.398 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.398 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.398 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.398 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.398 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.398 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.398 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.399 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.399 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption3": { 2023-11-16T16:49:38.399 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption3", 2023-11-16T16:49:38.399 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2023-11-16T16:49:38.399 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.399 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.399 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.400 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.400 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.400 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.400 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.400 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.400 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.400 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.401 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption4": { 2023-11-16T16:49:38.401 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption4", 2023-11-16T16:49:38.401 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.401 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.401 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.401 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.401 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.401 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.402 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.402 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.402 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.402 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.402 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.402 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption5": { 2023-11-16T16:49:38.403 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption5", 2023-11-16T16:49:38.403 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.403 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.403 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.403 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.403 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.403 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.404 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.404 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.404 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.404 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.404 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.404 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption6": { 2023-11-16T16:49:38.404 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption6", 2023-11-16T16:49:38.405 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.405 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.405 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.405 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.405 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.405 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.405 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.405 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.405 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.406 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.406 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.406 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption7": { 2023-11-16T16:49:38.406 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption7", 2023-11-16T16:49:38.406 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.406 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.407 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.407 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.407 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2023-11-16T16:49:38.407 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "42", 2023-11-16T16:49:38.407 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.407 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.407 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.407 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.408 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.408 INFO:teuthology.orchestra.run.smithi001.stdout: "testkey": { 2023-11-16T16:49:38.408 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testkey", 2023-11-16T16:49:38.408 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.408 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.408 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.408 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.408 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.409 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.409 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.409 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.409 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.409 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.409 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.409 INFO:teuthology.orchestra.run.smithi001.stdout: "testlkey": { 2023-11-16T16:49:38.410 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testlkey", 2023-11-16T16:49:38.410 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.410 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.410 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.410 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.410 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.410 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.411 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.411 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.411 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.411 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.411 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.411 INFO:teuthology.orchestra.run.smithi001.stdout: "testnewline": { 2023-11-16T16:49:38.411 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testnewline", 2023-11-16T16:49:38.412 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.412 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.412 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.412 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.412 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.412 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.412 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.412 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.413 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.413 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.413 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.413 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.413 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.413 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.413 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snap_schedule", 2023-11-16T16:49:38.414 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.414 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.414 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.414 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_m_granularity": { 2023-11-16T16:49:38.414 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_m_granularity", 2023-11-16T16:49:38.414 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.414 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.414 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.415 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.415 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.415 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.415 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.415 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow minute scheduled snapshots", 2023-11-16T16:49:38.415 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.415 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.415 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.416 INFO:teuthology.orchestra.run.smithi001.stdout: "dump_on_update": { 2023-11-16T16:49:38.416 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dump_on_update", 2023-11-16T16:49:38.416 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.416 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.416 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.416 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.416 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.417 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.417 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.417 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "dump database to debug log on update", 2023-11-16T16:49:38.417 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.417 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.417 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.417 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.418 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.418 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.418 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.418 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.418 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.418 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.418 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.418 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.419 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.419 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.419 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.419 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.419 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.419 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.419 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.419 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.420 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.420 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.420 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.420 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.420 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.420 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.420 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.421 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.421 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.421 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.421 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.421 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.421 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.421 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.421 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.422 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.422 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.422 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.422 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.422 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.422 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.422 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.423 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.423 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.423 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.423 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.423 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.423 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.423 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.423 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.424 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.424 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.424 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.424 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.424 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.424 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.424 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.425 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.425 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.425 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.425 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.425 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.425 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.425 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.425 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.426 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.426 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.426 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.426 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.426 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.426 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.427 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.427 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stats", 2023-11-16T16:49:38.427 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.427 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.427 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.427 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.427 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.427 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.428 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.428 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.428 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.428 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.428 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.428 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.428 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.428 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.429 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.429 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.429 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.429 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.429 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.429 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.429 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.430 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.430 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.430 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.430 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.430 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.430 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.430 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.431 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.431 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.431 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.431 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.431 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.431 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.431 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.431 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.432 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.432 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.432 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.432 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.432 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.432 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.432 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.433 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.433 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.433 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.433 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.433 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.433 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.433 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.433 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.434 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.434 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.434 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.434 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.434 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.434 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.434 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.435 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.435 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.435 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.435 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.435 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.435 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.435 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.435 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.436 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.436 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.436 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.436 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.436 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.436 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.437 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "status", 2023-11-16T16:49:38.437 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.437 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.437 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.437 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.437 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.437 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.437 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.437 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.438 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.438 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.438 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.438 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.438 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.438 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.438 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.438 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.439 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.439 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.439 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.439 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.439 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.439 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.439 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.440 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.440 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.440 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.440 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.440 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.440 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.440 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.441 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.441 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.441 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.441 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.441 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.441 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.441 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.441 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.442 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.442 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.442 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.442 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.442 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.442 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.442 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.443 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.443 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.443 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.443 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.443 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.443 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.443 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.443 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.443 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.444 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.444 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.444 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.444 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.444 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.444 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.444 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.445 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.445 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.445 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.445 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.445 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.445 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.445 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.445 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.446 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.446 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.446 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telegraf", 2023-11-16T16:49:38.446 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.446 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.446 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.447 INFO:teuthology.orchestra.run.smithi001.stdout: "address": { 2023-11-16T16:49:38.447 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "address", 2023-11-16T16:49:38.447 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.447 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.447 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.447 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-11-16T16:49:38.447 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.447 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.448 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.448 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.448 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.448 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.448 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.448 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2023-11-16T16:49:38.448 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2023-11-16T16:49:38.449 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:38.449 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.449 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.449 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15", 2023-11-16T16:49:38.449 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.449 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.449 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.449 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.450 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.450 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.450 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.450 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.450 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.450 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.451 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.451 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.451 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.451 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.451 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.451 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.451 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.451 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.452 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.452 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.452 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.452 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.452 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.452 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.452 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.452 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.453 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.453 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.453 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.453 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.453 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.453 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.453 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.454 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.454 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.454 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.454 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.454 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.454 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.454 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.454 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.455 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.455 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.455 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.455 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.455 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.455 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.455 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.455 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.456 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.456 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.456 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.456 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.456 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.456 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.456 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.456 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.457 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.457 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.457 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.457 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.457 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.457 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.457 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.458 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.458 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.458 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.458 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.458 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.458 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.458 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.458 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.459 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.459 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.459 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.459 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.459 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telemetry", 2023-11-16T16:49:38.459 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.459 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.460 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.460 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_basic": { 2023-11-16T16:49:38.460 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_basic", 2023-11-16T16:49:38.460 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.460 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.460 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.460 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.460 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.461 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.461 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.461 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share basic cluster information (size, version)", 2023-11-16T16:49:38.461 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.461 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.461 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.461 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_crash": { 2023-11-16T16:49:38.462 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_crash", 2023-11-16T16:49:38.462 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.462 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.462 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.462 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.462 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.462 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.462 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.463 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-11-16T16:49:38.463 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.463 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.463 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.463 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_device": { 2023-11-16T16:49:38.463 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_device", 2023-11-16T16:49:38.463 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.464 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.464 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.464 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.464 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.464 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.464 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.464 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-11-16T16:49:38.464 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.465 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.465 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.465 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_ident": { 2023-11-16T16:49:38.465 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_ident", 2023-11-16T16:49:38.465 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.465 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.465 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.466 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.466 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.466 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.466 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.466 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-11-16T16:49:38.466 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.466 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.466 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.467 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_perf": { 2023-11-16T16:49:38.467 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_perf", 2023-11-16T16:49:38.467 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.467 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.467 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.467 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.467 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.468 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.468 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.468 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share various performance metrics of a cluster", 2023-11-16T16:49:38.468 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.468 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.468 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.469 INFO:teuthology.orchestra.run.smithi001.stdout: "contact": { 2023-11-16T16:49:38.469 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "contact", 2023-11-16T16:49:38.469 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.469 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.469 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.469 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.469 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.469 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.470 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.470 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.470 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.470 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.470 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.470 INFO:teuthology.orchestra.run.smithi001.stdout: "description": { 2023-11-16T16:49:38.470 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "description", 2023-11-16T16:49:38.471 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.471 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.471 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.471 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.471 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.471 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.471 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.471 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.472 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.472 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.472 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.472 INFO:teuthology.orchestra.run.smithi001.stdout: "device_url": { 2023-11-16T16:49:38.472 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_url", 2023-11-16T16:49:38.472 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.472 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.472 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.473 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-11-16T16:49:38.473 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.473 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.473 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.473 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.473 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.473 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.473 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.474 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2023-11-16T16:49:38.474 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2023-11-16T16:49:38.474 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.474 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.474 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.474 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.474 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.475 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.475 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.475 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.475 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.475 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.475 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.475 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2023-11-16T16:49:38.476 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2023-11-16T16:49:38.476 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.476 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.476 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.476 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "24", 2023-11-16T16:49:38.476 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "8", 2023-11-16T16:49:38.476 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.476 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.477 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.477 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.477 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.477 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.477 INFO:teuthology.orchestra.run.smithi001.stdout: "last_opt_revision": { 2023-11-16T16:49:38.477 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "last_opt_revision", 2023-11-16T16:49:38.477 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.478 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.478 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.478 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1", 2023-11-16T16:49:38.478 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.478 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.478 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.478 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.478 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.479 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.479 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.479 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard": { 2023-11-16T16:49:38.479 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard", 2023-11-16T16:49:38.479 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.479 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.479 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.480 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.480 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.480 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.480 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.480 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.480 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.480 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.481 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.481 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard_description": { 2023-11-16T16:49:38.481 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard_description", 2023-11-16T16:49:38.481 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.481 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.481 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.481 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.482 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.482 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.482 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.482 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.482 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.482 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.482 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.483 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.483 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.483 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.483 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.483 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.483 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.483 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.484 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.484 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.484 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.484 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.484 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.484 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.484 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.484 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.485 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.485 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.485 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.485 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.485 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.485 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.485 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.486 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.486 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.486 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.486 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.486 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.486 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.486 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.486 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.486 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.487 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.487 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.487 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.487 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.487 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.487 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.487 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.488 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.488 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.488 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.488 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.488 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.488 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.488 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.489 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.489 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.489 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.489 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.489 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.489 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.489 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.490 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.490 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.490 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.490 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.490 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.490 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.490 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.491 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.491 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.491 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.491 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.491 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.491 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.491 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.492 INFO:teuthology.orchestra.run.smithi001.stdout: "organization": { 2023-11-16T16:49:38.492 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "organization", 2023-11-16T16:49:38.492 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.492 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.492 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.492 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.492 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.492 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.493 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.493 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.493 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.493 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.493 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.493 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.493 INFO:teuthology.orchestra.run.smithi001.stdout: "proxy": { 2023-11-16T16:49:38.493 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "proxy", 2023-11-16T16:49:38.494 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.494 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.494 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.494 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.494 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.494 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.494 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.494 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.495 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.495 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.495 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.495 INFO:teuthology.orchestra.run.smithi001.stdout: "url": { 2023-11-16T16:49:38.495 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url", 2023-11-16T16:49:38.495 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.495 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.496 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.496 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-11-16T16:49:38.496 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.496 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.496 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.496 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.496 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.496 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.496 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.497 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.497 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.497 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.497 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "test_orchestrator", 2023-11-16T16:49:38.497 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.497 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.498 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.498 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.498 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.498 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.498 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.498 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.498 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.498 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.499 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.499 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.499 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.499 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.499 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.499 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.499 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.500 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.500 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.500 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.500 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.500 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.500 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.500 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.501 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.501 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.501 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.501 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.501 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.501 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.501 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.501 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.501 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.502 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.502 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.502 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.502 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.502 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.502 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.503 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.503 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.503 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.503 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.503 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.503 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.503 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.504 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.504 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.504 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.504 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.504 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.504 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.504 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.504 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.505 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.505 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.505 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.505 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.505 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.505 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.505 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.506 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.506 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.506 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.506 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.506 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.506 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.506 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.506 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.507 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.507 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.507 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.507 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "volumes", 2023-11-16T16:49:38.507 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.507 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.507 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.508 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.508 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.508 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.508 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.508 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.508 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.508 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.508 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.509 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.509 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.509 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.509 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.509 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.509 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.509 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.509 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.510 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.510 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.510 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.510 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.510 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.510 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.510 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.511 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.511 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.511 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.511 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.511 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.511 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.511 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.511 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.512 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.512 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.512 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.512 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.512 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.512 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.512 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.513 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.513 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.513 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.513 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.513 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.513 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.513 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.513 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.514 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.514 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.514 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.514 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.514 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.514 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.514 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.515 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.515 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.515 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.515 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.515 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.515 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.515 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.515 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.516 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.516 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.516 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.516 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.516 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.516 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_clones": { 2023-11-16T16:49:38.516 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_clones", 2023-11-16T16:49:38.517 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.517 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.517 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.517 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4", 2023-11-16T16:49:38.517 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.517 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.517 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.517 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Number of asynchronous cloner threads", 2023-11-16T16:49:38.518 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.518 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.518 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.518 INFO:teuthology.orchestra.run.smithi001.stdout: "snapshot_clone_delay": { 2023-11-16T16:49:38.518 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snapshot_clone_delay", 2023-11-16T16:49:38.518 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.518 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.519 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.519 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2023-11-16T16:49:38.519 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.519 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.519 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.519 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-11-16T16:49:38.519 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.519 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.520 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.520 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.520 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.520 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.520 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix", 2023-11-16T16:49:38.520 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.521 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.521 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.521 INFO:teuthology.orchestra.run.smithi001.stdout: "discovery_interval": { 2023-11-16T16:49:38.521 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "discovery_interval", 2023-11-16T16:49:38.521 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2023-11-16T16:49:38.521 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.521 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.521 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "100", 2023-11-16T16:49:38.522 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.522 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.522 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.522 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.522 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.522 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.522 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.523 INFO:teuthology.orchestra.run.smithi001.stdout: "identifier": { 2023-11-16T16:49:38.523 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "identifier", 2023-11-16T16:49:38.523 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.523 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.523 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.523 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.523 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.523 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.524 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.524 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.524 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.524 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.524 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.524 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2023-11-16T16:49:38.524 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2023-11-16T16:49:38.525 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:38.525 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.525 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.525 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2023-11-16T16:49:38.525 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.525 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.525 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.525 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.526 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.526 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.526 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.526 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.526 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.526 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.526 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.527 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.527 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.527 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.527 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.527 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.527 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.527 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.527 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.528 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.528 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.528 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.528 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.528 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.528 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.528 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.528 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.529 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.529 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.529 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.529 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.529 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.529 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.529 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.530 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.530 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.530 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.530 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.530 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.530 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.531 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.531 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.531 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.531 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.531 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.531 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.531 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.531 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.532 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.532 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.532 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.532 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.532 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.532 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.532 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.532 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.533 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.533 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.533 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.533 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.533 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.533 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.533 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.534 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.534 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.534 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.534 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.534 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.534 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.534 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.535 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.535 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.535 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.535 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_host": { 2023-11-16T16:49:38.535 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_host", 2023-11-16T16:49:38.535 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.535 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.536 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.536 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.536 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.536 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.536 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.536 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.536 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.536 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.537 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.537 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_port": { 2023-11-16T16:49:38.537 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_port", 2023-11-16T16:49:38.537 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.537 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.537 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.537 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10051", 2023-11-16T16:49:38.538 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.538 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.538 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.538 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.538 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.538 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.538 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.539 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_sender": { 2023-11-16T16:49:38.539 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_sender", 2023-11-16T16:49:38.539 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.539 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.539 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.539 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-11-16T16:49:38.539 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.539 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.540 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.540 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.540 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.540 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.540 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.540 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.540 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.540 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.541 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2023-11-16T16:49:38.541 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.541 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.541 INFO:teuthology.orchestra.run.smithi001.stdout: "modules": [ 2023-11-16T16:49:38.541 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm", 2023-11-16T16:49:38.541 INFO:teuthology.orchestra.run.smithi001.stdout: "dashboard", 2023-11-16T16:49:38.541 INFO:teuthology.orchestra.run.smithi001.stdout: "iostat", 2023-11-16T16:49:38.541 INFO:teuthology.orchestra.run.smithi001.stdout: "nfs", 2023-11-16T16:49:38.542 INFO:teuthology.orchestra.run.smithi001.stdout: "restful", 2023-11-16T16:49:38.542 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_schedule" 2023-11-16T16:49:38.542 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.542 INFO:teuthology.orchestra.run.smithi001.stdout: "available_modules": [ 2023-11-16T16:49:38.542 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.542 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "alerts", 2023-11-16T16:49:38.542 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.542 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.543 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.543 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2023-11-16T16:49:38.543 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2023-11-16T16:49:38.543 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:38.543 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.543 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.543 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2023-11-16T16:49:38.543 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.544 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.544 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.544 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How frequently to reexamine health status", 2023-11-16T16:49:38.544 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.544 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.544 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.544 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.545 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.545 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.545 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.545 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.545 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.545 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.545 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.545 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.546 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.546 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.546 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.546 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.546 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.546 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.546 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.546 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.547 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.547 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.547 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.547 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.547 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.547 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.547 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.548 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.548 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.548 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.548 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.548 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.548 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.548 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.548 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.549 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.549 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.549 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.549 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.549 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.549 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.549 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.550 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.550 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.550 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.550 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.550 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.550 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.550 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.550 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.551 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.551 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.551 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.551 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.551 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.551 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.551 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.552 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.552 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.552 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.552 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.552 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.552 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.552 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.552 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.553 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.553 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.553 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.553 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.553 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_destination": { 2023-11-16T16:49:38.553 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_destination", 2023-11-16T16:49:38.553 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.553 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.554 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.554 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.554 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.554 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.554 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.554 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email address to send alerts to", 2023-11-16T16:49:38.554 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.554 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.555 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.555 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_from_name": { 2023-11-16T16:49:38.555 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_from_name", 2023-11-16T16:49:38.555 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.555 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.555 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.556 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "Ceph", 2023-11-16T16:49:38.556 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.556 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.556 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.556 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email From: name", 2023-11-16T16:49:38.556 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.556 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.556 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.557 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_host": { 2023-11-16T16:49:38.557 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_host", 2023-11-16T16:49:38.557 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.557 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.557 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.557 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.557 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.558 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.558 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.558 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP server", 2023-11-16T16:49:38.558 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.558 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.558 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.558 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_password": { 2023-11-16T16:49:38.559 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_password", 2023-11-16T16:49:38.559 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.559 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.559 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.559 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.559 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.559 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.559 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.560 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Password to authenticate with", 2023-11-16T16:49:38.560 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.560 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.560 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.560 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_port": { 2023-11-16T16:49:38.560 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_port", 2023-11-16T16:49:38.560 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.561 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.561 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.561 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "465", 2023-11-16T16:49:38.561 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.561 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.561 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.561 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP port", 2023-11-16T16:49:38.561 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.562 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.562 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.562 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_sender": { 2023-11-16T16:49:38.562 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_sender", 2023-11-16T16:49:38.562 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.562 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.562 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.563 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.563 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.563 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.563 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.563 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP envelope sender", 2023-11-16T16:49:38.563 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.563 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.563 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.564 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_ssl": { 2023-11-16T16:49:38.564 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_ssl", 2023-11-16T16:49:38.564 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.564 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.564 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.564 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.564 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.565 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.565 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.565 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use SSL to connect to SMTP server", 2023-11-16T16:49:38.565 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.565 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.565 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.565 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_user": { 2023-11-16T16:49:38.566 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_user", 2023-11-16T16:49:38.566 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.566 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.566 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.566 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.566 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.566 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.566 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.567 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "User to authenticate as", 2023-11-16T16:49:38.567 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.567 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.567 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.567 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.567 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.568 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "balancer", 2023-11-16T16:49:38.568 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.568 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.568 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.568 INFO:teuthology.orchestra.run.smithi001.stdout: "active": { 2023-11-16T16:49:38.568 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "active", 2023-11-16T16:49:38.568 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.568 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.569 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.569 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.569 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.569 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.569 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.569 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically balance PGs across cluster", 2023-11-16T16:49:38.569 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.569 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.570 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.570 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_time": { 2023-11-16T16:49:38.570 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_time", 2023-11-16T16:49:38.570 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.570 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.570 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.570 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0000", 2023-11-16T16:49:38.571 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.571 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.571 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.571 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "beginning time of day to automatically balance", 2023-11-16T16:49:38.571 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-16T16:49:38.571 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.571 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.572 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_weekday": { 2023-11-16T16:49:38.572 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_weekday", 2023-11-16T16:49:38.572 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2023-11-16T16:49:38.572 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.572 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.572 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2023-11-16T16:49:38.572 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2023-11-16T16:49:38.572 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2023-11-16T16:49:38.572 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.573 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-11-16T16:49:38.573 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-16T16:49:38.573 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.573 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.573 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_max_iterations": { 2023-11-16T16:49:38.573 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_max_iterations", 2023-11-16T16:49:38.573 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2023-11-16T16:49:38.574 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.574 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.574 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "25", 2023-11-16T16:49:38.574 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2023-11-16T16:49:38.574 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "250", 2023-11-16T16:49:38.574 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.574 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-11-16T16:49:38.574 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.575 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.575 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.575 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_metrics": { 2023-11-16T16:49:38.575 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_metrics", 2023-11-16T16:49:38.575 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.575 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.575 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.576 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "pgs,objects,bytes", 2023-11-16T16:49:38.576 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.576 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.576 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.576 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-11-16T16:49:38.576 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-11-16T16:49:38.576 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.576 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.577 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_step": { 2023-11-16T16:49:38.577 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_step", 2023-11-16T16:49:38.577 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2023-11-16T16:49:38.577 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.577 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.577 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.5", 2023-11-16T16:49:38.577 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0.001", 2023-11-16T16:49:38.578 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "0.999", 2023-11-16T16:49:38.578 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.578 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "aggressiveness of optimization", 2023-11-16T16:49:38.578 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-11-16T16:49:38.578 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.578 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.578 INFO:teuthology.orchestra.run.smithi001.stdout: "end_time": { 2023-11-16T16:49:38.579 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_time", 2023-11-16T16:49:38.579 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.579 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.579 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.579 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2359", 2023-11-16T16:49:38.579 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.579 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.579 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.580 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ending time of day to automatically balance", 2023-11-16T16:49:38.580 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-16T16:49:38.580 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.580 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.580 INFO:teuthology.orchestra.run.smithi001.stdout: "end_weekday": { 2023-11-16T16:49:38.580 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_weekday", 2023-11-16T16:49:38.581 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2023-11-16T16:49:38.581 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.581 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.581 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2023-11-16T16:49:38.581 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2023-11-16T16:49:38.581 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2023-11-16T16:49:38.581 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.581 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-11-16T16:49:38.582 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-16T16:49:38.582 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.582 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.582 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.582 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.582 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.583 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.583 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.583 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.583 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.583 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.583 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.583 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.583 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.584 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.584 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.584 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.584 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.584 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.584 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.584 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.584 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.585 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.585 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.585 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.585 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.585 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.585 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.585 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.586 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.586 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.586 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.586 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.586 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.586 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.586 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.587 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.587 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.587 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.587 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.587 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.587 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.587 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.587 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.588 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.588 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.588 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.588 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.588 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.588 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.588 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.589 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.589 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.589 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.589 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.589 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.589 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.590 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.590 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.590 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.590 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.590 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.590 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.590 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.590 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.591 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.591 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.591 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.591 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.591 INFO:teuthology.orchestra.run.smithi001.stdout: "min_score": { 2023-11-16T16:49:38.591 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_score", 2023-11-16T16:49:38.591 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2023-11-16T16:49:38.592 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.592 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.592 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2023-11-16T16:49:38.592 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.592 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.592 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.592 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-11-16T16:49:38.592 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.593 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.593 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.593 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2023-11-16T16:49:38.593 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2023-11-16T16:49:38.593 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.593 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.593 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.594 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "upmap", 2023-11-16T16:49:38.594 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.594 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.594 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.594 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-compat", 2023-11-16T16:49:38.594 INFO:teuthology.orchestra.run.smithi001.stdout: "none", 2023-11-16T16:49:38.594 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap" 2023-11-16T16:49:38.594 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.595 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Balancer mode", 2023-11-16T16:49:38.595 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.595 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.595 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.595 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_ids": { 2023-11-16T16:49:38.595 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_ids", 2023-11-16T16:49:38.595 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.596 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.596 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.596 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.596 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.596 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.596 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.596 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-11-16T16:49:38.596 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.597 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.597 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.597 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2023-11-16T16:49:38.597 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2023-11-16T16:49:38.597 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:38.597 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.597 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.597 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2023-11-16T16:49:38.598 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.598 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.598 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.598 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-11-16T16:49:38.598 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.598 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.598 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.599 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_deviation": { 2023-11-16T16:49:38.599 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_deviation", 2023-11-16T16:49:38.599 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.599 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.599 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.599 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2023-11-16T16:49:38.599 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2023-11-16T16:49:38.600 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.600 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.600 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "deviation below which no optimization is attempted", 2023-11-16T16:49:38.600 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-11-16T16:49:38.600 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.600 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.600 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_optimizations": { 2023-11-16T16:49:38.601 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_optimizations", 2023-11-16T16:49:38.601 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2023-11-16T16:49:38.601 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.601 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.601 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2023-11-16T16:49:38.601 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.601 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.601 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.601 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-11-16T16:49:38.602 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.602 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.602 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.602 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.602 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.603 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cephadm", 2023-11-16T16:49:38.603 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.603 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.603 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.603 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_down_multiplier": { 2023-11-16T16:49:38.603 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_down_multiplier", 2023-11-16T16:49:38.603 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2023-11-16T16:49:38.603 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.604 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.604 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2023-11-16T16:49:38.604 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.604 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.604 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.604 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-11-16T16:49:38.604 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.605 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.605 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.605 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_refresh_rate": { 2023-11-16T16:49:38.605 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_refresh_rate", 2023-11-16T16:49:38.605 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:38.605 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.605 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.606 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "20", 2023-11-16T16:49:38.606 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.606 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.606 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.606 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-11-16T16:49:38.606 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.606 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.606 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.607 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_starting_port": { 2023-11-16T16:49:38.607 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_starting_port", 2023-11-16T16:49:38.607 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.607 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.607 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.607 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4721", 2023-11-16T16:49:38.607 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.608 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.608 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.608 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-11-16T16:49:38.608 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.608 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.608 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.608 INFO:teuthology.orchestra.run.smithi001.stdout: "alertmanager_web_password": { 2023-11-16T16:49:38.609 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "alertmanager_web_password", 2023-11-16T16:49:38.609 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.609 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.609 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.609 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2023-11-16T16:49:38.609 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.609 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.610 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.610 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Alertmanager web password", 2023-11-16T16:49:38.610 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.610 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.610 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.610 INFO:teuthology.orchestra.run.smithi001.stdout: "alertmanager_web_user": { 2023-11-16T16:49:38.610 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "alertmanager_web_user", 2023-11-16T16:49:38.611 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.611 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.611 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.611 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2023-11-16T16:49:38.611 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.611 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.611 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.611 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Alertmanager web user", 2023-11-16T16:49:38.612 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.612 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.612 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.612 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_ptrace": { 2023-11-16T16:49:38.612 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_ptrace", 2023-11-16T16:49:38.612 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.612 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.613 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.613 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.613 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.613 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.613 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.613 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-11-16T16:49:38.613 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-11-16T16:49:38.613 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.614 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.614 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_interval": { 2023-11-16T16:49:38.614 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_interval", 2023-11-16T16:49:38.614 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:38.614 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.614 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.614 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2023-11-16T16:49:38.615 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.615 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.615 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.615 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to autotune daemon memory", 2023-11-16T16:49:38.615 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.615 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.615 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.616 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_memory_target_ratio": { 2023-11-16T16:49:38.616 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_memory_target_ratio", 2023-11-16T16:49:38.616 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2023-11-16T16:49:38.616 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.616 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.616 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.7", 2023-11-16T16:49:38.616 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.616 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.617 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.617 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-11-16T16:49:38.617 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.617 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.617 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.617 INFO:teuthology.orchestra.run.smithi001.stdout: "cgroups_split": { 2023-11-16T16:49:38.617 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cgroups_split", 2023-11-16T16:49:38.618 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.618 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.618 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.618 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.618 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.618 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.618 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.618 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-11-16T16:49:38.619 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.619 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.619 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.619 INFO:teuthology.orchestra.run.smithi001.stdout: "config_checks_enabled": { 2023-11-16T16:49:38.619 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_checks_enabled", 2023-11-16T16:49:38.619 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.619 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.620 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.620 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.620 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.620 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.620 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.620 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-11-16T16:49:38.620 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.620 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.621 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.621 INFO:teuthology.orchestra.run.smithi001.stdout: "config_dashboard": { 2023-11-16T16:49:38.621 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_dashboard", 2023-11-16T16:49:38.621 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.621 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.621 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.621 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.622 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.622 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.622 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.622 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-11-16T16:49:38.622 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.622 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.622 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.623 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_alertmanager": { 2023-11-16T16:49:38.623 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_alertmanager", 2023-11-16T16:49:38.623 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.624 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.624 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.624 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-11-16T16:49:38.624 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.625 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.625 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.625 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2023-11-16T16:49:38.625 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.625 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.626 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.626 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_base": { 2023-11-16T16:49:38.626 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_base", 2023-11-16T16:49:38.627 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.627 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.627 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.628 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/ceph", 2023-11-16T16:49:38.628 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.628 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.628 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.628 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Container image name, without the tag", 2023-11-16T16:49:38.629 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.629 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.629 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.630 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_elasticsearch": { 2023-11-16T16:49:38.630 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_elasticsearch", 2023-11-16T16:49:38.630 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.630 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.630 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.631 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-11-16T16:49:38.631 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.631 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.631 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.631 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "elasticsearch container image", 2023-11-16T16:49:38.631 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.631 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.631 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.632 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_grafana": { 2023-11-16T16:49:38.632 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_grafana", 2023-11-16T16:49:38.632 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.632 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.632 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.632 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-11-16T16:49:38.632 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.633 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.633 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.633 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2023-11-16T16:49:38.633 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.633 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.633 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.633 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_haproxy": { 2023-11-16T16:49:38.634 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_haproxy", 2023-11-16T16:49:38.634 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.634 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.634 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.634 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-11-16T16:49:38.635 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.635 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.635 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.635 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "HAproxy container image", 2023-11-16T16:49:38.636 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.636 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.636 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.636 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_agent": { 2023-11-16T16:49:38.636 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_agent", 2023-11-16T16:49:38.637 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.637 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.637 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.637 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-11-16T16:49:38.638 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.638 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.638 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.638 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger agent container image", 2023-11-16T16:49:38.639 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.639 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.639 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.640 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_collector": { 2023-11-16T16:49:38.640 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_collector", 2023-11-16T16:49:38.640 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.640 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.641 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.641 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-11-16T16:49:38.641 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.641 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.641 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.641 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger collector container image", 2023-11-16T16:49:38.642 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.642 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.642 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.642 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_query": { 2023-11-16T16:49:38.642 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_query", 2023-11-16T16:49:38.642 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.643 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.643 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.643 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-11-16T16:49:38.643 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.643 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.643 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.643 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger query container image", 2023-11-16T16:49:38.643 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.644 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.644 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.644 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_keepalived": { 2023-11-16T16:49:38.645 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_keepalived", 2023-11-16T16:49:38.645 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.645 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.645 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.645 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-11-16T16:49:38.645 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.645 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.646 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.646 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Keepalived container image", 2023-11-16T16:49:38.646 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.646 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.646 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.646 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_loki": { 2023-11-16T16:49:38.646 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_loki", 2023-11-16T16:49:38.647 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.647 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.647 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.647 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-11-16T16:49:38.647 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.647 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.647 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.648 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Loki container image", 2023-11-16T16:49:38.648 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.648 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.648 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.648 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_node_exporter": { 2023-11-16T16:49:38.648 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_node_exporter", 2023-11-16T16:49:38.649 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.649 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.649 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.649 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-11-16T16:49:38.649 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.649 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.649 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.649 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2023-11-16T16:49:38.650 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.650 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.650 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.650 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_prometheus": { 2023-11-16T16:49:38.650 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_prometheus", 2023-11-16T16:49:38.650 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.651 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.651 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.651 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-11-16T16:49:38.651 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.651 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.651 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.651 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2023-11-16T16:49:38.651 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.652 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.652 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.652 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_promtail": { 2023-11-16T16:49:38.652 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_promtail", 2023-11-16T16:49:38.652 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.652 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.652 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.653 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-11-16T16:49:38.653 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.653 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.653 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.653 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Promtail container image", 2023-11-16T16:49:38.653 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.653 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.653 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.654 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_snmp_gateway": { 2023-11-16T16:49:38.654 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_snmp_gateway", 2023-11-16T16:49:38.654 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.654 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.654 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.654 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-11-16T16:49:38.654 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.655 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.655 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.655 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SNMP Gateway container image", 2023-11-16T16:49:38.655 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.655 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.655 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.655 INFO:teuthology.orchestra.run.smithi001.stdout: "container_init": { 2023-11-16T16:49:38.656 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_init", 2023-11-16T16:49:38.656 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.656 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.656 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.656 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.656 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.656 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.656 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.657 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Run podman/docker with `--init`", 2023-11-16T16:49:38.657 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.657 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.657 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.657 INFO:teuthology.orchestra.run.smithi001.stdout: "daemon_cache_timeout": { 2023-11-16T16:49:38.657 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "daemon_cache_timeout", 2023-11-16T16:49:38.657 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:38.658 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.658 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.658 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2023-11-16T16:49:38.658 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.658 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.658 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.658 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-11-16T16:49:38.658 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.659 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.659 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.659 INFO:teuthology.orchestra.run.smithi001.stdout: "default_cephadm_command_timeout": { 2023-11-16T16:49:38.659 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_cephadm_command_timeout", 2023-11-16T16:49:38.659 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:38.659 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.659 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.660 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "900", 2023-11-16T16:49:38.660 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.660 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.660 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.660 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-11-16T16:49:38.660 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.660 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.661 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.661 INFO:teuthology.orchestra.run.smithi001.stdout: "default_registry": { 2023-11-16T16:49:38.661 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_registry", 2023-11-16T16:49:38.661 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.661 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.661 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.661 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io", 2023-11-16T16:49:38.662 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.662 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.662 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.662 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-11-16T16:49:38.662 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.662 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.662 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.663 INFO:teuthology.orchestra.run.smithi001.stdout: "device_cache_timeout": { 2023-11-16T16:49:38.663 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_cache_timeout", 2023-11-16T16:49:38.663 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:38.663 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.663 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.663 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1800", 2023-11-16T16:49:38.663 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.664 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.664 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.664 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache device inventory", 2023-11-16T16:49:38.664 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.664 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.664 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.664 INFO:teuthology.orchestra.run.smithi001.stdout: "device_enhanced_scan": { 2023-11-16T16:49:38.665 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_enhanced_scan", 2023-11-16T16:49:38.665 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.665 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.665 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.665 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.665 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.665 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.665 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.666 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use libstoragemgmt during device scans", 2023-11-16T16:49:38.666 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.666 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.666 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.666 INFO:teuthology.orchestra.run.smithi001.stdout: "facts_cache_timeout": { 2023-11-16T16:49:38.666 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "facts_cache_timeout", 2023-11-16T16:49:38.666 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:38.667 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.667 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.667 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2023-11-16T16:49:38.667 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.667 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.667 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.667 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache host facts data", 2023-11-16T16:49:38.668 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.668 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.668 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.668 INFO:teuthology.orchestra.run.smithi001.stdout: "host_check_interval": { 2023-11-16T16:49:38.668 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "host_check_interval", 2023-11-16T16:49:38.668 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:38.668 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.669 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.669 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2023-11-16T16:49:38.669 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.669 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.669 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.669 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to perform a host check", 2023-11-16T16:49:38.669 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.670 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.670 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.670 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.670 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.670 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.670 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.670 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.671 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.671 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.671 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.671 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.671 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.671 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.671 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.671 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.672 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.672 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.672 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.672 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.672 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.672 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.672 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.673 INFO:teuthology.orchestra.run.smithi001.stdout: "log_refresh_metadata": { 2023-11-16T16:49:38.673 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_refresh_metadata", 2023-11-16T16:49:38.673 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.673 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.673 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.673 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.673 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.673 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.673 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.674 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-11-16T16:49:38.674 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.674 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.674 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.674 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.674 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.675 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.675 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.675 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.675 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.675 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.675 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.675 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.675 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-11-16T16:49:38.676 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.676 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.676 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.676 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.676 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.676 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.676 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.677 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.677 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.677 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.677 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.677 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.677 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.677 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.677 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.678 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.678 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.678 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.678 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.678 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.678 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.678 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.678 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.679 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.679 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.679 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.679 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.679 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.679 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.679 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.680 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.680 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.680 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.680 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.680 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.680 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.680 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf": { 2023-11-16T16:49:38.681 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-11-16T16:49:38.681 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.681 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.681 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.681 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.681 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.681 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.682 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.682 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-11-16T16:49:38.682 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.682 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.682 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.682 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-11-16T16:49:38.683 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-11-16T16:49:38.683 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.683 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.683 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.683 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "*", 2023-11-16T16:49:38.683 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.683 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.683 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.684 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-11-16T16:49:38.684 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.684 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.684 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.684 INFO:teuthology.orchestra.run.smithi001.stdout: "max_count_per_host": { 2023-11-16T16:49:38.684 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_count_per_host", 2023-11-16T16:49:38.684 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.685 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.685 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.685 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2023-11-16T16:49:38.685 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.685 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.685 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.685 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of daemons per service per host", 2023-11-16T16:49:38.685 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.686 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.686 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.686 INFO:teuthology.orchestra.run.smithi001.stdout: "max_osd_draining_count": { 2023-11-16T16:49:38.686 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_osd_draining_count", 2023-11-16T16:49:38.686 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.686 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.686 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.687 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2023-11-16T16:49:38.687 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.687 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.687 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.687 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-11-16T16:49:38.687 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.687 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.687 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.688 INFO:teuthology.orchestra.run.smithi001.stdout: "migration_current": { 2023-11-16T16:49:38.688 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "migration_current", 2023-11-16T16:49:38.688 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.688 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.688 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.688 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.688 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.689 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.689 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.689 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "internal - do not modify", 2023-11-16T16:49:38.689 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.689 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.689 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.690 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2023-11-16T16:49:38.690 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2023-11-16T16:49:38.690 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.690 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.690 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.690 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "root", 2023-11-16T16:49:38.690 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.690 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.691 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.691 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm-package", 2023-11-16T16:49:38.691 INFO:teuthology.orchestra.run.smithi001.stdout: "root" 2023-11-16T16:49:38.691 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.691 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "mode for remote execution of cephadm", 2023-11-16T16:49:38.691 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.691 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.691 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.692 INFO:teuthology.orchestra.run.smithi001.stdout: "prometheus_alerts_path": { 2023-11-16T16:49:38.692 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus_alerts_path", 2023-11-16T16:49:38.692 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.692 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.692 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.692 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-11-16T16:49:38.692 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.693 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.693 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.693 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-11-16T16:49:38.693 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.693 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.693 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.693 INFO:teuthology.orchestra.run.smithi001.stdout: "prometheus_web_password": { 2023-11-16T16:49:38.694 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus_web_password", 2023-11-16T16:49:38.694 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.694 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.694 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.694 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2023-11-16T16:49:38.694 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.694 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.694 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.695 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus web password", 2023-11-16T16:49:38.695 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.695 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.695 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.695 INFO:teuthology.orchestra.run.smithi001.stdout: "prometheus_web_user": { 2023-11-16T16:49:38.695 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus_web_user", 2023-11-16T16:49:38.695 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.696 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.696 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.696 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2023-11-16T16:49:38.696 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.696 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.696 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.696 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus web user", 2023-11-16T16:49:38.696 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.697 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.697 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.697 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_insecure": { 2023-11-16T16:49:38.697 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_insecure", 2023-11-16T16:49:38.697 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.697 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.697 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.697 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.698 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.698 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.698 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.698 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-11-16T16:49:38.698 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.698 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.698 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.699 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_password": { 2023-11-16T16:49:38.699 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_password", 2023-11-16T16:49:38.699 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.699 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.699 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.699 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.699 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.700 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.700 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.700 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-11-16T16:49:38.700 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.700 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.700 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.700 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_url": { 2023-11-16T16:49:38.701 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_url", 2023-11-16T16:49:38.701 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.701 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.701 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.701 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.701 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.701 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.701 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.702 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-11-16T16:49:38.702 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.702 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.702 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.702 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_username": { 2023-11-16T16:49:38.702 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_username", 2023-11-16T16:49:38.703 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.703 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.703 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.703 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.703 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.703 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.703 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.703 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-11-16T16:49:38.704 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.704 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.704 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.704 INFO:teuthology.orchestra.run.smithi001.stdout: "secure_monitoring_stack": { 2023-11-16T16:49:38.704 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "secure_monitoring_stack", 2023-11-16T16:49:38.704 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.705 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.705 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.705 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.705 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.705 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.705 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.705 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-11-16T16:49:38.705 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.706 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.706 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.706 INFO:teuthology.orchestra.run.smithi001.stdout: "service_discovery_port": { 2023-11-16T16:49:38.706 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "service_discovery_port", 2023-11-16T16:49:38.706 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.706 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.706 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.707 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8765", 2023-11-16T16:49:38.707 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.707 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.707 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.707 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "cephadm service discovery port", 2023-11-16T16:49:38.707 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.707 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.707 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.708 INFO:teuthology.orchestra.run.smithi001.stdout: "ssh_config_file": { 2023-11-16T16:49:38.708 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssh_config_file", 2023-11-16T16:49:38.708 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.708 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.708 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.708 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.708 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.709 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.709 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.709 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-11-16T16:49:38.709 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.709 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.709 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.709 INFO:teuthology.orchestra.run.smithi001.stdout: "use_agent": { 2023-11-16T16:49:38.710 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_agent", 2023-11-16T16:49:38.710 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.710 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.710 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.710 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.710 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.710 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.710 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.711 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-11-16T16:49:38.711 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.711 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.711 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.711 INFO:teuthology.orchestra.run.smithi001.stdout: "use_repo_digest": { 2023-11-16T16:49:38.711 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_repo_digest", 2023-11-16T16:49:38.711 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.712 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.712 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.712 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.712 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.712 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.712 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.712 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-11-16T16:49:38.712 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.713 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.713 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.713 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_failed_host_check": { 2023-11-16T16:49:38.713 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_failed_host_check", 2023-11-16T16:49:38.713 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.713 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.713 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.714 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.714 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.714 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.714 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.714 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if the host check fails", 2023-11-16T16:49:38.714 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.714 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.714 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.715 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_daemons": { 2023-11-16T16:49:38.715 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_daemons", 2023-11-16T16:49:38.715 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.715 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.715 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.715 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.715 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.716 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.716 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.716 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-11-16T16:49:38.716 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.716 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.716 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.716 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_hosts": { 2023-11-16T16:49:38.717 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_hosts", 2023-11-16T16:49:38.717 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.717 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.717 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.717 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.717 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.717 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.717 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.718 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-11-16T16:49:38.718 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.718 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.718 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.718 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.718 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.718 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.719 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crash", 2023-11-16T16:49:38.719 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.719 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.719 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.719 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.719 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.719 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.719 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.720 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.720 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.720 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.720 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.720 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.720 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.720 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.720 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.721 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.721 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.721 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.721 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.721 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.721 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.721 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.721 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.722 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.722 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.722 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.722 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.722 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.722 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.722 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.723 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.723 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.723 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.723 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.723 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.723 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.723 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.724 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.724 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.724 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.724 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.724 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.724 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.724 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.724 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.725 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.725 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.725 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.725 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.725 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.725 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.725 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.725 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.726 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.726 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.726 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.726 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.726 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.726 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.726 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.727 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.727 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.727 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.727 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.727 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.727 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.727 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.727 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.728 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.728 INFO:teuthology.orchestra.run.smithi001.stdout: "retain_interval": { 2023-11-16T16:49:38.728 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retain_interval", 2023-11-16T16:49:38.728 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:38.728 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.728 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.728 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "31536000", 2023-11-16T16:49:38.729 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.729 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.729 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.729 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain crashes before pruning them", 2023-11-16T16:49:38.729 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.729 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.729 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.730 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_recent_interval": { 2023-11-16T16:49:38.730 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_recent_interval", 2023-11-16T16:49:38.730 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:38.730 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.730 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.730 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1209600", 2023-11-16T16:49:38.730 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.731 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.731 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.731 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "time interval in which to warn about recent crashes", 2023-11-16T16:49:38.731 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.731 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.731 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.731 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.731 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.732 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.732 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dashboard", 2023-11-16T16:49:38.732 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.732 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.732 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.732 INFO:teuthology.orchestra.run.smithi001.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-11-16T16:49:38.732 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-11-16T16:49:38.733 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.733 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.733 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.733 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2023-11-16T16:49:38.733 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.733 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.733 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.733 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.734 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.734 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.734 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.734 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_HOST": { 2023-11-16T16:49:38.734 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_HOST", 2023-11-16T16:49:38.734 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.734 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.735 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.735 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.735 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.735 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.735 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.735 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.735 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.735 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.736 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.736 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-11-16T16:49:38.736 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-11-16T16:49:38.736 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.736 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.736 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.736 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.736 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.737 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.737 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.737 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.737 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.737 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.737 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.737 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_ENABLED": { 2023-11-16T16:49:38.738 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_ENABLED", 2023-11-16T16:49:38.738 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.738 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.738 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.738 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.738 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.738 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.738 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.739 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.739 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.739 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.739 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.739 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-11-16T16:49:38.739 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-11-16T16:49:38.739 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.740 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.740 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.740 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.740 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.740 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.740 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.740 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.740 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.741 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.741 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.741 INFO:teuthology.orchestra.run.smithi001.stdout: "ENABLE_BROWSABLE_API": { 2023-11-16T16:49:38.741 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ENABLE_BROWSABLE_API", 2023-11-16T16:49:38.741 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.741 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.741 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.742 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.742 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.742 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.742 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.742 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.742 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.742 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.742 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.743 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-11-16T16:49:38.743 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-11-16T16:49:38.743 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.743 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.743 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.743 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.743 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.744 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.744 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.744 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.744 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.744 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.744 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.744 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.744 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-11-16T16:49:38.745 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-11-16T16:49:38.745 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.745 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.745 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.745 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.745 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.745 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.745 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.746 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.746 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.746 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.746 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.746 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-11-16T16:49:38.746 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-11-16T16:49:38.747 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.747 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.747 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.747 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.747 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.747 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.747 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.747 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.747 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.748 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.748 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.748 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-11-16T16:49:38.748 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-11-16T16:49:38.748 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.748 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.749 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.749 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.749 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.749 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.749 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.749 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.749 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.749 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.750 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.750 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_NFS": { 2023-11-16T16:49:38.750 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-11-16T16:49:38.750 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.750 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.750 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.750 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.751 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.751 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.751 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.751 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.751 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.751 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.751 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.751 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.752 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RBD": { 2023-11-16T16:49:38.752 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-11-16T16:49:38.752 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.752 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.752 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.752 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.752 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.752 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.753 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.753 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.753 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.753 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.753 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.753 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RGW": { 2023-11-16T16:49:38.753 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-11-16T16:49:38.754 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.754 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.754 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.754 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.754 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.754 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.754 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.754 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.755 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.755 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.755 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.755 INFO:teuthology.orchestra.run.smithi001.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-11-16T16:49:38.755 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-11-16T16:49:38.755 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.755 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.756 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.756 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.756 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.756 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.756 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.756 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.756 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.756 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.757 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.757 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_PASSWORD": { 2023-11-16T16:49:38.757 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_PASSWORD", 2023-11-16T16:49:38.757 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.757 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.757 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.757 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2023-11-16T16:49:38.758 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.758 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.758 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.758 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.758 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.758 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.758 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.759 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-11-16T16:49:38.759 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-11-16T16:49:38.759 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.759 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.759 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.759 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.759 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.760 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.760 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.760 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.760 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.760 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.760 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.761 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_URL": { 2023-11-16T16:49:38.761 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_URL", 2023-11-16T16:49:38.761 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.761 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.761 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.761 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.761 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.761 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.762 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.762 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.762 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.762 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.762 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.762 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.762 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_USERNAME": { 2023-11-16T16:49:38.762 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_USERNAME", 2023-11-16T16:49:38.763 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.763 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.763 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.763 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2023-11-16T16:49:38.763 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.763 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.763 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.764 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.764 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.764 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.764 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.764 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-11-16T16:49:38.764 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-11-16T16:49:38.764 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.765 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.765 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.765 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.765 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.765 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.765 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.765 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.765 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.765 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.766 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.766 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-11-16T16:49:38.766 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-11-16T16:49:38.766 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.766 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.766 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.767 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.767 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.767 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.767 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.767 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.767 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.767 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.767 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.768 INFO:teuthology.orchestra.run.smithi001.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-11-16T16:49:38.768 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-11-16T16:49:38.768 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.768 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.768 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.768 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.769 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.769 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.769 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.769 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.769 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.769 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.769 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.770 INFO:teuthology.orchestra.run.smithi001.stdout: "ISSUE_TRACKER_API_KEY": { 2023-11-16T16:49:38.770 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-11-16T16:49:38.770 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.770 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.770 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.770 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.770 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.771 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.771 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.771 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.771 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.771 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.771 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.771 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_HOST": { 2023-11-16T16:49:38.772 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_HOST", 2023-11-16T16:49:38.772 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.772 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.772 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.772 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.772 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.772 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.772 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.773 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.773 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.773 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.773 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.773 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-11-16T16:49:38.773 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-11-16T16:49:38.773 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.774 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.774 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.774 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.774 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.774 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.774 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.774 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.774 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.775 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.775 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.775 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-11-16T16:49:38.775 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-11-16T16:49:38.775 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.775 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.775 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.776 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.776 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.776 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.776 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.776 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.776 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.776 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.776 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.777 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-11-16T16:49:38.777 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-11-16T16:49:38.777 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.777 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.777 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.777 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.777 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.778 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.778 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.778 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.778 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.778 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.778 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.778 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-11-16T16:49:38.779 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-11-16T16:49:38.779 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.779 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.779 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.779 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.779 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.779 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.779 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.780 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.780 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.780 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.780 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.780 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-11-16T16:49:38.780 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-11-16T16:49:38.780 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.781 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.781 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.781 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.781 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.781 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.781 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.781 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.781 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.782 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.782 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.782 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-11-16T16:49:38.782 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-11-16T16:49:38.782 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.782 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.783 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.783 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.783 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.783 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.783 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.783 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.783 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.783 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.784 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.784 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-11-16T16:49:38.784 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-11-16T16:49:38.784 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.784 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.784 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.784 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.785 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.785 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.785 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.785 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.785 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.785 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.785 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.786 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-11-16T16:49:38.786 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-11-16T16:49:38.786 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.786 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.786 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.786 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.786 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.786 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.787 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.787 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.787 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.787 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.787 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.787 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_ENABLED": { 2023-11-16T16:49:38.787 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_ENABLED", 2023-11-16T16:49:38.788 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.788 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.788 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.788 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.788 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.788 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.788 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.788 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.789 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.789 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.789 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.789 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-11-16T16:49:38.789 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-11-16T16:49:38.789 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.789 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.790 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.790 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-11-16T16:49:38.790 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.790 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.790 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.790 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.790 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.791 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.791 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.791 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-11-16T16:49:38.791 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-11-16T16:49:38.791 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.791 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.791 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.792 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2023-11-16T16:49:38.792 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.792 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.792 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.792 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.792 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.792 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.792 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.793 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-11-16T16:49:38.793 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-11-16T16:49:38.793 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.793 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.793 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.793 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8", 2023-11-16T16:49:38.793 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.794 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.794 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.794 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.794 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.794 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.794 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.794 INFO:teuthology.orchestra.run.smithi001.stdout: "REST_REQUESTS_TIMEOUT": { 2023-11-16T16:49:38.795 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-11-16T16:49:38.795 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.795 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.795 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.795 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "45", 2023-11-16T16:49:38.795 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.795 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.795 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.796 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.796 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.796 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.796 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.796 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ACCESS_KEY": { 2023-11-16T16:49:38.796 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ACCESS_KEY", 2023-11-16T16:49:38.796 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.797 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.797 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.797 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.797 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.797 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.797 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.797 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.797 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.798 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.798 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.798 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-11-16T16:49:38.798 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-11-16T16:49:38.798 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.798 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.798 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.799 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2023-11-16T16:49:38.799 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.799 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.799 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.799 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.799 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.799 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.799 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.800 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SECRET_KEY": { 2023-11-16T16:49:38.800 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SECRET_KEY", 2023-11-16T16:49:38.800 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.800 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.800 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.800 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.800 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.801 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.801 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.801 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.801 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.801 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.801 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.801 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SSL_VERIFY": { 2023-11-16T16:49:38.802 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SSL_VERIFY", 2023-11-16T16:49:38.802 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.802 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.802 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.802 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.802 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.802 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.802 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.803 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.803 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.803 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.803 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.803 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-11-16T16:49:38.803 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-11-16T16:49:38.803 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.804 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.804 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.804 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2023-11-16T16:49:38.804 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.804 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.804 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.804 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.804 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.805 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.805 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.805 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-11-16T16:49:38.805 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-11-16T16:49:38.805 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.805 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.806 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.806 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2023-11-16T16:49:38.806 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.806 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.806 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.806 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.806 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.806 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.807 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.807 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-11-16T16:49:38.807 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-11-16T16:49:38.807 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.807 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.807 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.807 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2023-11-16T16:49:38.808 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.808 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.808 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.808 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.808 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.808 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.808 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.809 INFO:teuthology.orchestra.run.smithi001.stdout: "cross_origin_url": { 2023-11-16T16:49:38.809 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cross_origin_url", 2023-11-16T16:49:38.809 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.809 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.809 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.809 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.809 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.809 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.810 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.810 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.810 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.810 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.810 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.810 INFO:teuthology.orchestra.run.smithi001.stdout: "crt_file": { 2023-11-16T16:49:38.811 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crt_file", 2023-11-16T16:49:38.811 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.811 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.811 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.811 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.811 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.811 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.811 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.811 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.812 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.812 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.812 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.812 INFO:teuthology.orchestra.run.smithi001.stdout: "debug": { 2023-11-16T16:49:38.812 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "debug", 2023-11-16T16:49:38.812 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.812 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.813 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.813 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.813 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.813 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.813 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.813 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable/disable debug options", 2023-11-16T16:49:38.813 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.813 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.814 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.814 INFO:teuthology.orchestra.run.smithi001.stdout: "jwt_token_ttl": { 2023-11-16T16:49:38.814 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "jwt_token_ttl", 2023-11-16T16:49:38.814 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.814 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.814 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.815 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "28800", 2023-11-16T16:49:38.815 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.815 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.815 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.815 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.815 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.815 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.815 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.816 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2023-11-16T16:49:38.816 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2023-11-16T16:49:38.816 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.816 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.816 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.816 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.816 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.817 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.817 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.817 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.817 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.817 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.817 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.817 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.818 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.818 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.818 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.818 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.818 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.818 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.818 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.818 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.819 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.819 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.819 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.819 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.819 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.819 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.819 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.819 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.820 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.820 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.820 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.820 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.820 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.820 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.821 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.821 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.821 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.821 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.821 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.821 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.821 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.821 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.821 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.822 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.822 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.822 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.822 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.822 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.822 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.823 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.823 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.823 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.823 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.823 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.823 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.823 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.823 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.824 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.824 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.824 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.824 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.824 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.824 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.824 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.825 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.825 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.825 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.825 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.825 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.825 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.825 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.825 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.826 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.826 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.826 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.826 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.826 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.826 INFO:teuthology.orchestra.run.smithi001.stdout: "motd": { 2023-11-16T16:49:38.826 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "motd", 2023-11-16T16:49:38.827 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.827 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.827 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.827 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.827 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.827 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.827 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.827 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "The message of the day", 2023-11-16T16:49:38.828 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.828 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.828 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.828 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect_resolve_ip_addr": { 2023-11-16T16:49:38.828 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "redirect_resolve_ip_addr", 2023-11-16T16:49:38.828 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.828 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.829 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.829 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.829 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.829 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.829 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.829 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.829 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.830 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.830 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.830 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2023-11-16T16:49:38.830 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2023-11-16T16:49:38.830 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.830 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.830 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.831 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2023-11-16T16:49:38.831 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.831 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.831 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.831 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.831 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.831 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.831 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.832 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2023-11-16T16:49:38.832 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2023-11-16T16:49:38.832 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.832 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.832 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.832 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8080", 2023-11-16T16:49:38.832 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.833 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.833 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.833 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.833 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.833 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.833 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.833 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2023-11-16T16:49:38.834 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2023-11-16T16:49:38.834 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.834 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.834 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.834 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.834 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.834 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.834 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.835 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.835 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.835 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.835 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.835 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl_server_port": { 2023-11-16T16:49:38.835 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl_server_port", 2023-11-16T16:49:38.835 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.836 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.836 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.836 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8443", 2023-11-16T16:49:38.836 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.836 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.836 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.836 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.836 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.837 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.837 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.837 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2023-11-16T16:49:38.837 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2023-11-16T16:49:38.837 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.837 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.837 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.838 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "redirect", 2023-11-16T16:49:38.838 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.838 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.838 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.838 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.838 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect" 2023-11-16T16:49:38.838 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.838 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.839 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.839 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.839 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.839 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2023-11-16T16:49:38.839 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2023-11-16T16:49:38.839 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.840 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.840 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.840 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2023-11-16T16:49:38.840 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2023-11-16T16:49:38.840 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2023-11-16T16:49:38.840 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.840 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.840 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.841 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.841 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.841 INFO:teuthology.orchestra.run.smithi001.stdout: "url_prefix": { 2023-11-16T16:49:38.841 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url_prefix", 2023-11-16T16:49:38.841 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.841 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.841 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.841 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.842 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.842 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.842 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.842 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.842 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.842 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.842 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.842 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.843 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.843 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "devicehealth", 2023-11-16T16:49:38.843 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.843 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.843 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.843 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_monitoring": { 2023-11-16T16:49:38.844 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_monitoring", 2023-11-16T16:49:38.844 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.844 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.844 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.844 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.844 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.844 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.844 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.845 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "monitor device health metrics", 2023-11-16T16:49:38.845 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.845 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.845 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.845 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.845 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.845 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.846 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.846 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.846 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.846 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.846 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.846 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.846 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.847 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.847 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.847 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.847 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.847 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.847 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.847 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.847 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.847 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.848 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.848 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.848 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.848 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.848 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.848 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.849 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.849 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.849 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.849 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.849 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.849 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.849 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.849 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.850 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.850 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.850 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.850 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.850 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.850 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.850 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.851 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.851 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.851 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.851 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.851 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.851 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.851 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.851 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.852 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.852 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.852 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.852 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.852 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.852 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.852 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.853 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.853 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.853 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.853 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.853 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.853 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.853 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.853 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.853 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.854 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.854 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.854 INFO:teuthology.orchestra.run.smithi001.stdout: "mark_out_threshold": { 2023-11-16T16:49:38.854 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mark_out_threshold", 2023-11-16T16:49:38.854 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:38.854 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.854 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.855 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2419200", 2023-11-16T16:49:38.855 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.855 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.855 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.855 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-11-16T16:49:38.855 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.855 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.856 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.856 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": { 2023-11-16T16:49:38.856 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_name", 2023-11-16T16:49:38.856 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.856 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.856 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.856 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "device_health_metrics", 2023-11-16T16:49:38.857 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.857 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.857 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.857 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name of pool in which to store device health metrics", 2023-11-16T16:49:38.857 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.857 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.857 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.857 INFO:teuthology.orchestra.run.smithi001.stdout: "retention_period": { 2023-11-16T16:49:38.858 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retention_period", 2023-11-16T16:49:38.858 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:38.858 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.858 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.858 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15552000", 2023-11-16T16:49:38.858 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.858 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.859 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.859 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain device health metrics", 2023-11-16T16:49:38.859 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.859 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.859 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.859 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_frequency": { 2023-11-16T16:49:38.859 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_frequency", 2023-11-16T16:49:38.860 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:38.860 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.860 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.860 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2023-11-16T16:49:38.860 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.860 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.860 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.860 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to scrape device health metrics", 2023-11-16T16:49:38.861 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.861 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.861 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.861 INFO:teuthology.orchestra.run.smithi001.stdout: "self_heal": { 2023-11-16T16:49:38.861 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "self_heal", 2023-11-16T16:49:38.861 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.862 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.862 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.862 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:38.862 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.862 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.862 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.862 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-11-16T16:49:38.862 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.863 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.863 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.863 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2023-11-16T16:49:38.863 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2023-11-16T16:49:38.863 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:38.863 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.863 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.864 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2023-11-16T16:49:38.864 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.864 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.864 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.864 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and check device health", 2023-11-16T16:49:38.864 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.864 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.864 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.865 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_threshold": { 2023-11-16T16:49:38.865 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_threshold", 2023-11-16T16:49:38.865 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:38.865 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.865 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.865 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7257600", 2023-11-16T16:49:38.865 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.866 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.866 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.866 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-11-16T16:49:38.866 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.866 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.866 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.866 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.866 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.867 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.867 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "diskprediction_local", 2023-11-16T16:49:38.867 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.867 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.867 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.867 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.867 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.868 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.868 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.868 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.868 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.868 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.868 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.868 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.869 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.869 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.869 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.869 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.869 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.869 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.869 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.869 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.870 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.870 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.870 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.870 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.870 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.870 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.871 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.871 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.871 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.871 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.871 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.871 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.871 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.872 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.872 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.872 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.872 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.872 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.872 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.872 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.873 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.873 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.873 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.873 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.873 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.873 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.873 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.874 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.874 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.874 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.874 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.874 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.874 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.874 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.875 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.875 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.875 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.875 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.875 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.875 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.875 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.875 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.876 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.876 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.876 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.876 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.876 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.876 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.876 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.877 INFO:teuthology.orchestra.run.smithi001.stdout: "predict_interval": { 2023-11-16T16:49:38.877 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predict_interval", 2023-11-16T16:49:38.877 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.877 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.877 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.877 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2023-11-16T16:49:38.877 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.877 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.878 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.878 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.878 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.878 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.878 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.878 INFO:teuthology.orchestra.run.smithi001.stdout: "predictor_model": { 2023-11-16T16:49:38.878 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predictor_model", 2023-11-16T16:49:38.879 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.879 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.879 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.879 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "prophetstor", 2023-11-16T16:49:38.879 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.879 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.879 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.879 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.880 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.880 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.880 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.880 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2023-11-16T16:49:38.880 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2023-11-16T16:49:38.880 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.880 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.881 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.881 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2023-11-16T16:49:38.881 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.881 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.881 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.881 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.881 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.881 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.882 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.882 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.882 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.882 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.882 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "influx", 2023-11-16T16:49:38.882 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": false, 2023-11-16T16:49:38.883 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "influxdb python module not found", 2023-11-16T16:49:38.883 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.883 INFO:teuthology.orchestra.run.smithi001.stdout: "batch_size": { 2023-11-16T16:49:38.883 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "batch_size", 2023-11-16T16:49:38.883 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.883 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.883 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.884 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5000", 2023-11-16T16:49:38.884 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.884 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.884 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.884 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-11-16T16:49:38.884 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.884 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.884 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.885 INFO:teuthology.orchestra.run.smithi001.stdout: "database": { 2023-11-16T16:49:38.885 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "database", 2023-11-16T16:49:38.885 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.885 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.885 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.885 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "ceph", 2023-11-16T16:49:38.885 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.885 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.886 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.886 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-11-16T16:49:38.886 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.886 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.886 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.886 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": { 2023-11-16T16:49:38.886 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "hostname", 2023-11-16T16:49:38.887 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.887 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.887 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.887 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.887 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.887 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.887 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.887 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server hostname", 2023-11-16T16:49:38.888 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.888 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.888 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.888 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2023-11-16T16:49:38.888 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2023-11-16T16:49:38.888 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:38.888 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.889 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.889 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "30", 2023-11-16T16:49:38.889 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "5", 2023-11-16T16:49:38.889 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.889 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.889 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-11-16T16:49:38.889 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.890 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.890 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.890 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.890 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.890 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.890 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.891 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.891 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.891 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.891 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.891 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.891 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.891 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.891 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.892 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.892 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.892 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.892 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.892 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.892 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.892 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.892 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.893 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.893 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.893 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.893 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.893 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.893 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.893 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.893 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.894 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.894 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.894 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.894 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.894 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.894 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.894 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.895 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.895 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.895 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.895 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.895 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.895 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.895 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.895 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.896 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.896 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.896 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.896 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.896 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.896 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.896 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.896 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.897 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.897 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.897 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.897 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.897 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.897 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.897 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.897 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.898 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.898 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.898 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.898 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.898 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.898 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.898 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.899 INFO:teuthology.orchestra.run.smithi001.stdout: "password": { 2023-11-16T16:49:38.899 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "password", 2023-11-16T16:49:38.899 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.899 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.899 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.899 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.899 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.900 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.900 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.900 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "password of InfluxDB server user", 2023-11-16T16:49:38.900 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.900 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.900 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.900 INFO:teuthology.orchestra.run.smithi001.stdout: "port": { 2023-11-16T16:49:38.900 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "port", 2023-11-16T16:49:38.901 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.901 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.901 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.901 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8086", 2023-11-16T16:49:38.901 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.901 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.902 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.902 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server port", 2023-11-16T16:49:38.902 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.902 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.902 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.902 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2023-11-16T16:49:38.903 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2023-11-16T16:49:38.903 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.903 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.903 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.903 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "false", 2023-11-16T16:49:38.903 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.903 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.903 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.904 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-11-16T16:49:38.904 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.904 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.904 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.904 INFO:teuthology.orchestra.run.smithi001.stdout: "threads": { 2023-11-16T16:49:38.904 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threads", 2023-11-16T16:49:38.904 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.905 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.905 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.905 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2023-11-16T16:49:38.905 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2023-11-16T16:49:38.905 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "32", 2023-11-16T16:49:38.905 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.905 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-11-16T16:49:38.905 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.906 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.906 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.906 INFO:teuthology.orchestra.run.smithi001.stdout: "username": { 2023-11-16T16:49:38.906 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "username", 2023-11-16T16:49:38.906 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.906 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.907 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.907 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.907 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.907 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.907 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.907 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "username of InfluxDB server user", 2023-11-16T16:49:38.907 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.907 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.908 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.908 INFO:teuthology.orchestra.run.smithi001.stdout: "verify_ssl": { 2023-11-16T16:49:38.908 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "verify_ssl", 2023-11-16T16:49:38.908 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.908 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.908 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.908 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "true", 2023-11-16T16:49:38.909 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.909 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.909 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.909 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-11-16T16:49:38.909 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.909 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.909 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.909 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.910 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.910 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.910 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "insights", 2023-11-16T16:49:38.910 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.910 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.910 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.911 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.911 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.911 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.911 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.911 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.911 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.911 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.911 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.912 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.912 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.912 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.912 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.912 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.912 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.912 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.912 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.912 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.913 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.913 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.913 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.913 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.913 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.913 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.913 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.914 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.914 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.914 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.914 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.914 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.914 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.914 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.914 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.915 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.915 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.915 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.915 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.915 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.915 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.915 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.916 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.916 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.916 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.916 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.916 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.916 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.916 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.916 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.917 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.917 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.917 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.917 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.917 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.917 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.917 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.918 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.918 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.918 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.918 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.918 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.918 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.918 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.918 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.919 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.919 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.919 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.919 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.919 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.919 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.919 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.920 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "iostat", 2023-11-16T16:49:38.920 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.920 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.920 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.920 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.920 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.920 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.920 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.921 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.921 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.921 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.921 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.921 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.921 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.921 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.921 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.922 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.922 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.922 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.922 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.922 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.922 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.922 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.922 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.923 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.923 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.923 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.923 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.923 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.923 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.923 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.924 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.924 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.924 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.924 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.924 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.924 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.924 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.924 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.925 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.925 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.925 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.925 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.925 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.925 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.925 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.926 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.926 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.926 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.926 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.926 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.926 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.926 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.926 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.927 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.927 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.927 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.927 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.927 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.927 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.927 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.928 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.928 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.928 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.928 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.928 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.928 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.928 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.928 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.928 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.929 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.929 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.929 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.929 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "k8sevents", 2023-11-16T16:49:38.929 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.929 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.930 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.930 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_event_retention_days": { 2023-11-16T16:49:38.930 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ceph_event_retention_days", 2023-11-16T16:49:38.930 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.930 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.930 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.930 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7", 2023-11-16T16:49:38.930 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.931 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.931 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.931 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Days to hold ceph event information within local cache", 2023-11-16T16:49:38.931 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.931 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.931 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.931 INFO:teuthology.orchestra.run.smithi001.stdout: "config_check_secs": { 2023-11-16T16:49:38.932 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_check_secs", 2023-11-16T16:49:38.932 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.932 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.932 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.932 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2023-11-16T16:49:38.932 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "10", 2023-11-16T16:49:38.932 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.933 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.933 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-11-16T16:49:38.933 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.933 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.933 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.933 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.933 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.933 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.934 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.934 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.934 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.934 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.934 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.934 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.934 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.934 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.935 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.935 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.935 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.935 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.935 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.935 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.935 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.935 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.936 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.936 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.936 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.936 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.936 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.936 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.936 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.937 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.937 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.937 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.937 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.937 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.937 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.937 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.938 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.938 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.938 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.938 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.938 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.938 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.938 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.938 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.939 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.939 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.939 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.939 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.939 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.939 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.939 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.939 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.940 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.940 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.940 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.940 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.940 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.940 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.940 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.941 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.941 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.941 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.941 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.941 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.941 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.941 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.941 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.942 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.942 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.942 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.942 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.942 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "localpool", 2023-11-16T16:49:38.943 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.943 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.943 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.943 INFO:teuthology.orchestra.run.smithi001.stdout: "failure_domain": { 2023-11-16T16:49:38.943 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "failure_domain", 2023-11-16T16:49:38.943 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.943 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.943 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.943 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "host", 2023-11-16T16:49:38.944 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.944 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.944 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.944 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "failure domain for any created local pool", 2023-11-16T16:49:38.944 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-11-16T16:49:38.944 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.944 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.945 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.945 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.945 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.945 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.945 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.945 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.945 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.945 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.946 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.946 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.946 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.946 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.946 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.946 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.946 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.946 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.947 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.947 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.947 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.947 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.947 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.947 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.948 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.948 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.948 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.948 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.948 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.948 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.948 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.948 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.948 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.949 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.949 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.949 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.949 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.949 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.949 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.950 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.950 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.950 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.950 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.950 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.950 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.950 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.950 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.951 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.951 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.951 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.951 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.951 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.951 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.951 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.951 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.952 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.952 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.952 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.952 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.952 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.952 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.952 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.952 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.953 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.953 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.953 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.953 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.953 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.953 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": { 2023-11-16T16:49:38.953 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_size", 2023-11-16T16:49:38.954 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.954 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.954 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.954 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.954 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.954 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.954 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.955 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default min_size for any created local pool", 2023-11-16T16:49:38.955 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-11-16T16:49:38.955 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.955 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.955 INFO:teuthology.orchestra.run.smithi001.stdout: "num_rep": { 2023-11-16T16:49:38.955 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "num_rep", 2023-11-16T16:49:38.955 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.955 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.956 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.956 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3", 2023-11-16T16:49:38.956 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.956 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.956 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.956 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default replica count for any created local pool", 2023-11-16T16:49:38.956 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.956 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.957 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.957 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": { 2023-11-16T16:49:38.957 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_num", 2023-11-16T16:49:38.957 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:38.957 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.957 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.957 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "128", 2023-11-16T16:49:38.958 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.958 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.958 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.958 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default pg_num for any created local pool", 2023-11-16T16:49:38.958 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.958 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.958 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.959 INFO:teuthology.orchestra.run.smithi001.stdout: "prefix": { 2023-11-16T16:49:38.959 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prefix", 2023-11-16T16:49:38.959 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.959 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.959 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.959 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.959 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.959 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.960 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.960 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name prefix for any created local pool", 2023-11-16T16:49:38.960 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.960 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.960 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.960 INFO:teuthology.orchestra.run.smithi001.stdout: "subtree": { 2023-11-16T16:49:38.960 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "subtree", 2023-11-16T16:49:38.961 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.961 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.961 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.961 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "rack", 2023-11-16T16:49:38.961 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.961 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.961 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.961 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "CRUSH level for which to create a local pool", 2023-11-16T16:49:38.962 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-11-16T16:49:38.962 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.962 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.962 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.962 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.962 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.963 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mds_autoscaler", 2023-11-16T16:49:38.963 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.963 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.963 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.963 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.963 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.963 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.963 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.964 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.964 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.964 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.964 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.964 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.964 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.964 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.964 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.965 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.965 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.965 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.965 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.965 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.965 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.965 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.965 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.966 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.966 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.966 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.966 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.966 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.966 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.967 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.967 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.967 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.967 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.967 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.967 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.967 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.967 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.968 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.968 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.968 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.968 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.968 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.968 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.968 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.969 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.969 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.969 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.969 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.969 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.969 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.969 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.970 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.970 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.970 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.970 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.970 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.970 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.970 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.971 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.971 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.971 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.971 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.971 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.971 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.971 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.972 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.972 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.972 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.972 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.972 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.972 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.972 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.973 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirroring", 2023-11-16T16:49:38.973 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.973 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.973 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.973 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.973 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.973 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.973 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.974 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.974 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.974 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.974 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.974 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.974 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.974 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.974 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.975 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.975 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.975 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.975 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.975 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.975 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.975 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.975 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.976 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.976 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.976 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.976 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.976 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.976 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.976 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.977 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.977 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.977 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.977 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.977 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.977 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.977 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.978 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.978 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.978 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.978 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.978 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.978 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.978 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.978 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.979 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.979 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.979 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.979 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.979 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.979 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.979 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.979 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.980 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.980 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.980 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.980 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.980 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.980 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.980 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.981 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.981 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.981 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.981 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.981 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.981 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.981 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.981 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.982 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.982 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.982 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.982 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "nfs", 2023-11-16T16:49:38.982 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.982 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.983 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.983 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.983 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.983 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.983 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.983 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.983 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.984 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.984 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.984 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.984 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.984 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.984 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.984 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.984 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.985 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.985 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.985 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.985 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.985 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.985 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.985 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.986 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.986 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.986 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.986 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.986 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.986 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.986 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.986 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.987 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.987 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.987 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.987 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.987 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.987 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.987 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.988 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.988 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.988 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.988 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.988 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.988 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.988 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.988 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.989 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.989 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.989 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.989 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.989 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.989 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.989 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.989 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.990 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.990 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:38.990 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:38.990 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.990 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.990 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.990 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.991 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.991 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.991 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.991 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.991 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.991 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.991 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.992 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.992 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:38.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.992 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:38.992 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2023-11-16T16:49:38.992 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:38.992 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:38.992 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:38.993 INFO:teuthology.orchestra.run.smithi001.stdout: "fail_fs": { 2023-11-16T16:49:38.993 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "fail_fs", 2023-11-16T16:49:38.993 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.993 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.993 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:38.993 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.993 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.993 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.993 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.994 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-11-16T16:49:38.994 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.994 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.994 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.994 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:38.994 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:38.995 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.995 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.995 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.995 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:38.995 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.995 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.995 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:38.995 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:38.995 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:38.996 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:38.996 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:38.996 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:38.996 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:38.996 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:38.996 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.996 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.996 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.997 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.997 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:38.997 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:38.997 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:38.997 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.997 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.997 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:38.998 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.998 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:38.998 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:38.998 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:38.998 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:38.998 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:38.998 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:38.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:38.999 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:38.999 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:38.999 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:38.999 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:38.999 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:38.999 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:38.999 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:38.999 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.000 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:39.000 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:39.000 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:39.000 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:39.000 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:39.000 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:39.000 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:39.000 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.001 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.001 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.001 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.001 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.001 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:39.001 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:39.001 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.002 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.002 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.002 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.002 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.002 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.002 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.002 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.002 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.003 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.003 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.003 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.003 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator": { 2023-11-16T16:49:39.003 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2023-11-16T16:49:39.003 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.003 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.003 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.004 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.004 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.004 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.004 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:39.004 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm", 2023-11-16T16:49:39.004 INFO:teuthology.orchestra.run.smithi001.stdout: "rook", 2023-11-16T16:49:39.004 INFO:teuthology.orchestra.run.smithi001.stdout: "test_orchestrator" 2023-11-16T16:49:39.004 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.005 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Orchestrator backend", 2023-11-16T16:49:39.005 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.005 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.005 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.005 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.005 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.005 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:39.006 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_perf_query", 2023-11-16T16:49:39.006 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:39.006 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:39.006 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:39.006 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:39.006 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:39.006 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.007 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.007 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.007 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.007 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.007 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.007 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:39.007 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:39.007 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:39.008 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:39.008 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:39.008 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:39.008 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:39.008 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.008 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.008 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.009 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.009 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.009 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:39.009 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:39.009 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.009 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.009 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.010 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.010 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.010 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.010 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.010 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.010 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.010 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.010 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.011 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:39.011 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:39.011 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.011 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.011 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.011 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:39.011 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.012 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.012 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:39.012 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:39.012 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:39.012 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:39.012 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:39.012 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:39.012 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:39.013 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.013 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.013 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.013 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.013 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.013 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:39.013 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:39.014 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.014 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.014 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.014 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.014 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.014 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.014 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.014 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.015 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.015 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.015 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.015 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.015 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.015 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:39.015 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_support", 2023-11-16T16:49:39.016 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:39.016 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:39.016 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:39.016 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:39.016 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:39.016 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.016 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.016 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.017 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.017 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.017 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.017 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:39.017 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:39.017 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:39.017 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:39.017 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:39.018 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:39.018 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:39.018 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.018 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.018 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.018 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.018 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.019 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:39.019 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:39.019 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.019 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.019 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.019 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.019 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.019 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.020 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.020 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.020 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.020 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.020 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.020 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:39.020 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:39.021 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.021 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.021 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.021 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:39.021 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.021 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.021 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:39.021 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:39.022 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:39.022 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:39.022 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:39.022 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:39.022 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:39.022 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.022 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.022 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.023 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.023 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.023 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:39.023 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:39.023 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.023 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.023 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.024 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.025 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.025 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.025 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:39.025 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_autoscaler", 2023-11-16T16:49:39.025 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:39.025 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:39.025 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:39.026 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:39.026 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:39.026 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.026 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.026 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.026 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.026 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.027 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.027 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:39.027 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:39.027 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:39.027 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:39.027 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:39.027 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:39.027 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:39.028 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.028 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.028 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.028 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.028 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.028 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:39.028 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:39.029 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.029 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.029 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.029 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.029 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.029 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.029 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.029 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.030 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.030 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.030 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.030 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:39.030 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:39.030 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.030 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.031 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.031 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:39.031 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.031 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.031 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:39.031 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:39.031 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:39.032 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:39.032 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:39.032 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:39.032 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:39.032 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.032 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.032 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.032 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.033 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.033 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:39.033 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:39.033 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.033 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.033 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.033 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.033 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.034 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.034 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.034 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.034 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.034 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.034 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.034 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.035 INFO:teuthology.orchestra.run.smithi001.stdout: "noautoscale": { 2023-11-16T16:49:39.035 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "noautoscale", 2023-11-16T16:49:39.035 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.035 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.035 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.035 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.035 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.035 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.036 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.036 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "global autoscale flag", 2023-11-16T16:49:39.036 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-11-16T16:49:39.036 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.036 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.036 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2023-11-16T16:49:39.036 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2023-11-16T16:49:39.037 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:39.037 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.037 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.037 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2023-11-16T16:49:39.037 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.037 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.037 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.037 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.038 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.038 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.038 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.038 INFO:teuthology.orchestra.run.smithi001.stdout: "threshold": { 2023-11-16T16:49:39.038 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threshold", 2023-11-16T16:49:39.038 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2023-11-16T16:49:39.038 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.039 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.039 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2023-11-16T16:49:39.039 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1.0", 2023-11-16T16:49:39.039 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.039 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.039 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "scaling threshold", 2023-11-16T16:49:39.039 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-11-16T16:49:39.039 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.040 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.040 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.040 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.040 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:39.040 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "progress", 2023-11-16T16:49:39.040 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:39.040 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:39.041 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:39.041 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_pg_recovery_event": { 2023-11-16T16:49:39.041 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_pg_recovery_event", 2023-11-16T16:49:39.041 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.041 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.041 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.041 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.041 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.042 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.042 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.042 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow the module to show pg recovery progress", 2023-11-16T16:49:39.042 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.042 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.042 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.042 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.042 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2023-11-16T16:49:39.043 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2023-11-16T16:49:39.043 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.043 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.043 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.043 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:39.043 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.043 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.043 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.044 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.044 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.044 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.044 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.044 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:39.044 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:39.044 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.044 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.045 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.045 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.045 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.045 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.045 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:39.045 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:39.045 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:39.046 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:39.046 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:39.046 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:39.046 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:39.046 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.046 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.046 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.046 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.047 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.047 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:39.047 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:39.047 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.047 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.047 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.047 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.048 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.048 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.048 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.048 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.048 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.048 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.048 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.049 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:39.049 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:39.049 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.049 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.049 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.049 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:39.049 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.049 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.050 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:39.050 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:39.050 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:39.050 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:39.050 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:39.050 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:39.050 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:39.050 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.051 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.051 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.051 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.051 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.051 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:39.051 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:39.051 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.052 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.052 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.052 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.052 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.052 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.052 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.052 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.052 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.053 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.053 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.053 INFO:teuthology.orchestra.run.smithi001.stdout: "max_completed_events": { 2023-11-16T16:49:39.053 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_completed_events", 2023-11-16T16:49:39.053 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:39.053 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.053 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.054 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "50", 2023-11-16T16:49:39.054 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.054 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.054 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.054 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "number of past completed events to remember", 2023-11-16T16:49:39.054 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.054 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.055 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.055 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2023-11-16T16:49:39.055 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2023-11-16T16:49:39.055 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:39.055 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.055 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.055 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2023-11-16T16:49:39.056 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.056 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.056 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.056 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long the module is going to sleep", 2023-11-16T16:49:39.056 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.056 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.056 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.056 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.057 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.057 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:39.057 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus", 2023-11-16T16:49:39.057 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:39.057 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:39.057 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:39.057 INFO:teuthology.orchestra.run.smithi001.stdout: "cache": { 2023-11-16T16:49:39.057 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cache", 2023-11-16T16:49:39.058 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.058 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.058 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.058 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:39.058 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.058 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.058 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.059 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.059 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.059 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.059 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.059 INFO:teuthology.orchestra.run.smithi001.stdout: "exclude_perf_counters": { 2023-11-16T16:49:39.059 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "exclude_perf_counters", 2023-11-16T16:49:39.059 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.060 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.060 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.060 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:39.060 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.060 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.060 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.060 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-11-16T16:49:39.060 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-11-16T16:49:39.061 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.061 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.061 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:39.061 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:39.061 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.061 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.061 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.062 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.062 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.062 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.062 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:39.062 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:39.062 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:39.062 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:39.063 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:39.063 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:39.063 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:39.063 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.063 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.063 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.063 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.063 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.064 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:39.064 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:39.064 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.064 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.064 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.064 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.064 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.064 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.065 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.065 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.065 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.065 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.065 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.065 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:39.065 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:39.066 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.066 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.066 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.066 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:39.066 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.066 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.066 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:39.066 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:39.067 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:39.067 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:39.067 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:39.067 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:39.067 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:39.067 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.067 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.068 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.068 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.068 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.068 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:39.068 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:39.068 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.068 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.069 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.069 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.069 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.069 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.069 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.069 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.069 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.070 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.070 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.070 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools": { 2023-11-16T16:49:39.070 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools", 2023-11-16T16:49:39.070 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.070 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.070 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.071 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.071 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.071 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.071 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.071 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.071 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.071 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.071 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.072 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools_refresh_interval": { 2023-11-16T16:49:39.072 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-11-16T16:49:39.072 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:39.072 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.072 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.072 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "300", 2023-11-16T16:49:39.073 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.073 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.073 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.073 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.073 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.073 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.073 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.073 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_interval": { 2023-11-16T16:49:39.074 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_interval", 2023-11-16T16:49:39.074 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2023-11-16T16:49:39.074 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.074 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.074 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15.0", 2023-11-16T16:49:39.074 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.074 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.075 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.075 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.075 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.075 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.075 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.075 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.075 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2023-11-16T16:49:39.075 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2023-11-16T16:49:39.075 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.076 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.076 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.076 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2023-11-16T16:49:39.076 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.076 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.076 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.076 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-11-16T16:49:39.076 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.077 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.077 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.077 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.077 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2023-11-16T16:49:39.077 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2023-11-16T16:49:39.077 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:39.077 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.077 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.078 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "9283", 2023-11-16T16:49:39.078 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.078 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.078 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.078 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-11-16T16:49:39.078 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.078 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.078 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.079 INFO:teuthology.orchestra.run.smithi001.stdout: "stale_cache_strategy": { 2023-11-16T16:49:39.079 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stale_cache_strategy", 2023-11-16T16:49:39.079 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.079 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.079 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.079 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "log", 2023-11-16T16:49:39.079 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.080 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.080 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.080 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.080 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.080 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.080 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.081 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2023-11-16T16:49:39.081 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2023-11-16T16:49:39.081 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.081 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.081 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.081 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "default", 2023-11-16T16:49:39.081 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.081 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.082 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:39.082 INFO:teuthology.orchestra.run.smithi001.stdout: "default", 2023-11-16T16:49:39.082 INFO:teuthology.orchestra.run.smithi001.stdout: "error" 2023-11-16T16:49:39.082 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.082 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.082 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.082 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.082 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.083 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2023-11-16T16:49:39.083 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2023-11-16T16:49:39.083 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:39.083 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.083 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.083 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2023-11-16T16:49:39.084 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2023-11-16T16:49:39.084 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2023-11-16T16:49:39.084 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.084 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.084 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.084 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.084 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.085 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.085 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:39.085 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_support", 2023-11-16T16:49:39.085 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:39.085 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:39.085 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:39.086 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:39.086 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:39.086 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.086 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.086 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.086 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.086 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.086 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.087 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:39.087 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:39.087 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:39.087 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:39.087 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:39.087 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:39.087 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:39.087 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.088 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.088 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.088 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.088 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.088 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:39.088 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:39.088 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.089 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.089 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.089 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.089 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.089 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.089 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.089 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.089 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.090 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.090 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.090 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:39.090 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:39.090 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.090 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.090 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.091 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:39.091 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.091 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.091 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:39.091 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:39.091 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:39.091 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:39.091 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:39.092 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:39.092 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:39.092 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.092 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.092 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.092 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.092 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.093 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:39.093 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:39.093 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.093 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.093 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.093 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.093 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.094 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.094 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.094 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.094 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.094 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.094 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.095 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_snap_create": { 2023-11-16T16:49:39.095 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_snap_create", 2023-11-16T16:49:39.095 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:39.095 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.095 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.095 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2023-11-16T16:49:39.095 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.095 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.096 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.096 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.096 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.096 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.096 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.096 INFO:teuthology.orchestra.run.smithi001.stdout: "mirror_snapshot_schedule": { 2023-11-16T16:49:39.096 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirror_snapshot_schedule", 2023-11-16T16:49:39.096 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.097 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.097 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.097 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.097 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.097 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.097 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.097 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.097 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.098 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.098 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.098 INFO:teuthology.orchestra.run.smithi001.stdout: "trash_purge_schedule": { 2023-11-16T16:49:39.098 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "trash_purge_schedule", 2023-11-16T16:49:39.098 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.098 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.098 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.099 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.099 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.099 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.099 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.099 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.099 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.099 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.099 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.100 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.100 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.100 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:39.100 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "restful", 2023-11-16T16:49:39.100 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:39.100 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:39.100 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:39.101 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_auth": { 2023-11-16T16:49:39.101 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_auth", 2023-11-16T16:49:39.101 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.101 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.101 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.101 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:39.101 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.101 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.102 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.102 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.102 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.102 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.102 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.102 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2023-11-16T16:49:39.102 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2023-11-16T16:49:39.103 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.103 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.103 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.103 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.103 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.103 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.103 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.103 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.104 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.104 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.104 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.104 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:39.104 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:39.104 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.104 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.105 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.105 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.105 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.105 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.105 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:39.105 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:39.105 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:39.105 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:39.106 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:39.106 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:39.106 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:39.106 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.106 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.106 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.106 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.106 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.107 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:39.107 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:39.107 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.107 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.107 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.107 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.108 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.108 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.108 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.108 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.108 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.108 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.108 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.109 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:39.109 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:39.109 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.109 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.109 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.109 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:39.109 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.109 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.110 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:39.110 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:39.110 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:39.110 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:39.110 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:39.110 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:39.110 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:39.111 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.111 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.111 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.111 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.111 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.111 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:39.111 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:39.112 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.112 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.112 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.112 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.112 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.112 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.112 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.112 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.113 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.113 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.113 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.113 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2023-11-16T16:49:39.114 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2023-11-16T16:49:39.114 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.114 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.114 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.114 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.114 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.115 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.115 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.115 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.115 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.115 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.115 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.115 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2023-11-16T16:49:39.116 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2023-11-16T16:49:39.116 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.116 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.116 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.116 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.116 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.116 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.117 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.117 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.117 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.117 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.117 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.117 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.117 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:39.118 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rgw", 2023-11-16T16:49:39.118 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:39.118 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:39.118 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:39.118 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:39.118 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:39.118 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.119 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.119 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.119 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.119 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.120 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.120 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:39.120 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:39.120 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:39.120 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:39.120 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:39.120 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:39.120 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:39.121 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.121 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.121 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.121 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.121 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.121 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:39.121 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:39.122 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.122 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.122 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.122 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.122 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.122 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.122 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.122 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.123 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.123 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.123 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.123 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:39.123 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:39.123 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.123 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.124 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.124 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:39.124 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.124 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.124 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:39.124 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:39.124 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:39.124 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:39.125 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:39.125 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:39.125 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:39.125 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.125 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.125 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.125 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.125 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.126 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:39.126 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:39.126 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.126 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.126 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.126 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.126 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.127 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.127 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.127 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.127 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.127 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.127 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.127 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.127 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.128 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:39.128 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rook", 2023-11-16T16:49:39.128 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:39.128 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:39.128 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:39.128 INFO:teuthology.orchestra.run.smithi001.stdout: "drive_group_interval": { 2023-11-16T16:49:39.128 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "drive_group_interval", 2023-11-16T16:49:39.129 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2023-11-16T16:49:39.129 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.129 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.129 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "300.0", 2023-11-16T16:49:39.129 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.129 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.129 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.129 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-11-16T16:49:39.130 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.130 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.130 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.130 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:39.130 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:39.130 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.130 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.131 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.131 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.131 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.131 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.131 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:39.131 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:39.131 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:39.131 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:39.132 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:39.132 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:39.132 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:39.132 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.132 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.132 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.132 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.132 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.133 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:39.133 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:39.133 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.133 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.133 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.133 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.133 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.134 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.134 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.134 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.134 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.134 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.134 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.134 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:39.135 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:39.135 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.135 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.135 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.135 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:39.135 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.135 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.135 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:39.136 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:39.136 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:39.136 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:39.136 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:39.136 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:39.136 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:39.136 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.136 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.137 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.137 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.137 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.137 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:39.137 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:39.137 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.137 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.138 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.138 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.138 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.138 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.138 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.138 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.138 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.138 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.139 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.139 INFO:teuthology.orchestra.run.smithi001.stdout: "storage_class": { 2023-11-16T16:49:39.139 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "storage_class", 2023-11-16T16:49:39.139 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.139 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.139 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.139 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "local", 2023-11-16T16:49:39.140 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.140 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.140 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.140 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-11-16T16:49:39.140 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.140 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.140 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.140 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.141 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.141 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:39.141 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "selftest", 2023-11-16T16:49:39.141 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:39.141 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:39.141 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:39.141 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:39.142 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:39.142 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.142 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.142 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.142 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.142 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.142 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.142 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:39.143 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:39.143 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:39.143 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:39.143 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:39.143 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:39.143 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:39.143 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.143 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.144 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.144 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.144 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.144 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:39.144 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:39.144 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.145 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.145 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.145 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.145 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.146 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.146 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.146 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.146 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.146 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.146 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.147 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:39.147 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:39.147 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.147 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.147 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.147 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:39.147 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.147 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.147 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:39.148 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:39.148 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:39.148 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:39.148 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:39.148 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:39.148 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:39.148 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.148 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.149 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.149 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.149 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.149 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:39.149 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:39.149 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.149 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.150 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.150 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.150 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.150 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.150 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.150 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.150 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.151 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.151 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.151 INFO:teuthology.orchestra.run.smithi001.stdout: "roption1": { 2023-11-16T16:49:39.151 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption1", 2023-11-16T16:49:39.151 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.151 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.151 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.152 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.152 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.152 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.152 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.152 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.152 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.152 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.152 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.153 INFO:teuthology.orchestra.run.smithi001.stdout: "roption2": { 2023-11-16T16:49:39.153 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption2", 2023-11-16T16:49:39.153 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.153 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.153 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.153 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "xyz", 2023-11-16T16:49:39.153 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.154 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.154 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.154 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.154 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.154 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.154 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.155 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption1": { 2023-11-16T16:49:39.155 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption1", 2023-11-16T16:49:39.155 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.155 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.155 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.156 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.156 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.156 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.156 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.156 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.156 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.156 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.156 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.157 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption2": { 2023-11-16T16:49:39.157 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption2", 2023-11-16T16:49:39.157 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:39.157 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.157 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.157 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.158 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.158 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.158 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.158 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.158 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.158 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.158 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.159 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption3": { 2023-11-16T16:49:39.159 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption3", 2023-11-16T16:49:39.159 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2023-11-16T16:49:39.159 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.159 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.159 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.159 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.159 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.160 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.160 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.160 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.160 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.160 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.160 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption4": { 2023-11-16T16:49:39.160 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption4", 2023-11-16T16:49:39.161 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.161 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.161 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.161 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.161 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.161 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.161 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.161 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.162 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.162 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.162 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.162 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.162 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption5": { 2023-11-16T16:49:39.162 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption5", 2023-11-16T16:49:39.162 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.162 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.163 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.163 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.163 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.163 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.163 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.163 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.163 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.163 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.164 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.164 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption6": { 2023-11-16T16:49:39.164 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption6", 2023-11-16T16:49:39.164 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.164 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.164 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.164 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:39.165 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.165 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.165 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.165 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.165 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.165 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.165 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.166 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption7": { 2023-11-16T16:49:39.166 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption7", 2023-11-16T16:49:39.166 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:39.166 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.166 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.166 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.166 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2023-11-16T16:49:39.166 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "42", 2023-11-16T16:49:39.167 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.167 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.167 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.167 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.167 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.167 INFO:teuthology.orchestra.run.smithi001.stdout: "testkey": { 2023-11-16T16:49:39.167 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testkey", 2023-11-16T16:49:39.168 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.168 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.168 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.168 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.168 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.168 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.168 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.168 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.169 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.169 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.169 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.169 INFO:teuthology.orchestra.run.smithi001.stdout: "testlkey": { 2023-11-16T16:49:39.169 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testlkey", 2023-11-16T16:49:39.169 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.169 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.170 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.170 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.170 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.170 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.170 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.170 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.170 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.171 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.171 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.171 INFO:teuthology.orchestra.run.smithi001.stdout: "testnewline": { 2023-11-16T16:49:39.171 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testnewline", 2023-11-16T16:49:39.171 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.171 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.171 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.172 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.172 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.172 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.172 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.172 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.172 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.172 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.172 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.173 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.173 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.173 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:39.173 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snap_schedule", 2023-11-16T16:49:39.173 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:39.173 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:39.173 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:39.174 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_m_granularity": { 2023-11-16T16:49:39.174 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_m_granularity", 2023-11-16T16:49:39.174 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.174 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.174 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.174 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.174 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.174 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.175 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.175 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow minute scheduled snapshots", 2023-11-16T16:49:39.175 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.175 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.175 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.175 INFO:teuthology.orchestra.run.smithi001.stdout: "dump_on_update": { 2023-11-16T16:49:39.175 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dump_on_update", 2023-11-16T16:49:39.176 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.176 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.176 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.176 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.176 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.176 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.176 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.176 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "dump database to debug log on update", 2023-11-16T16:49:39.177 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.177 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.177 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.177 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:39.177 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:39.177 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.177 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.178 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.178 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.178 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.178 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.178 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:39.178 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:39.179 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:39.179 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:39.179 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:39.179 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:39.179 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:39.179 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.179 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.179 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.180 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.180 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.180 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:39.180 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:39.180 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.180 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.180 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.181 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.181 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.181 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.181 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.181 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.181 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.181 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.181 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.182 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:39.182 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:39.182 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.182 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.182 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.182 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:39.182 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.183 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.183 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:39.183 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:39.183 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:39.183 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:39.183 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:39.183 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:39.184 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:39.184 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.184 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.184 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.184 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.184 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.184 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:39.185 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:39.185 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.185 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.185 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.185 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.185 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.185 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.186 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.186 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.186 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.186 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.186 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.186 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.186 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.187 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:39.187 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stats", 2023-11-16T16:49:39.187 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:39.187 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:39.187 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:39.187 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:39.187 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:39.187 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.188 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.188 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.188 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.188 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.188 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.188 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:39.188 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:39.188 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:39.189 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:39.189 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:39.189 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:39.189 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:39.189 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.189 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.189 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.189 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.190 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.190 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:39.190 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:39.190 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.190 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.190 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.190 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.191 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.191 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.191 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.191 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.191 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.191 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.191 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.192 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:39.192 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:39.192 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.192 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.192 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.192 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:39.192 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.193 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.193 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:39.193 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:39.193 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:39.193 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:39.193 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:39.193 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:39.193 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:39.194 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.194 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.194 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.194 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.194 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.194 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:39.197 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:39.198 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.198 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.198 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.198 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.198 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.198 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.198 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.198 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.199 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.199 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.199 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.199 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.199 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.199 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:39.199 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "status", 2023-11-16T16:49:39.200 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:39.200 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:39.200 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:39.200 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:39.200 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:39.200 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.200 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.200 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.201 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.201 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.201 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.201 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:39.201 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:39.201 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:39.201 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:39.201 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:39.202 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:39.202 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:39.202 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.202 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.202 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.202 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.202 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.203 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:39.203 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:39.203 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.203 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.203 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.203 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.203 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.203 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.204 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.204 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.204 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.204 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.204 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.204 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:39.204 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:39.205 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.205 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.205 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.205 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:39.205 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.205 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.205 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:39.205 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:39.206 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:39.206 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:39.206 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:39.206 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:39.206 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:39.206 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.206 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.207 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.207 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.207 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.207 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:39.207 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:39.207 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.207 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.208 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.208 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.208 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.208 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.208 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.208 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.208 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.209 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.209 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.209 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.209 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:39.209 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telegraf", 2023-11-16T16:49:39.209 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:39.210 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:39.210 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:39.210 INFO:teuthology.orchestra.run.smithi001.stdout: "address": { 2023-11-16T16:49:39.210 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "address", 2023-11-16T16:49:39.210 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.210 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.210 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.210 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-11-16T16:49:39.211 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.211 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.211 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.211 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.211 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.211 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.211 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.212 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2023-11-16T16:49:39.212 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2023-11-16T16:49:39.212 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:39.212 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.212 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.212 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15", 2023-11-16T16:49:39.212 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.212 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.213 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.213 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.213 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.213 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.213 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.213 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:39.213 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:39.213 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.214 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.214 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.214 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.214 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.214 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.214 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:39.214 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:39.215 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:39.215 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:39.215 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:39.215 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:39.215 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:39.215 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.215 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.215 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.216 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.216 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.216 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:39.216 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:39.216 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.216 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.216 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.216 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.217 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.217 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.217 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.217 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.217 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.217 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.217 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.218 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:39.218 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:39.218 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.218 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.218 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.218 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:39.218 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.218 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.219 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:39.219 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:39.219 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:39.219 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:39.219 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:39.219 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:39.219 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:39.220 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.220 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.220 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.220 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.220 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.220 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:39.221 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:39.221 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.221 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.221 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.221 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.221 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.221 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.221 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.221 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.222 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.222 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.222 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.222 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.222 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.222 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:39.222 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telemetry", 2023-11-16T16:49:39.223 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:39.223 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:39.223 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:39.223 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_basic": { 2023-11-16T16:49:39.223 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_basic", 2023-11-16T16:49:39.223 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.223 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.223 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.224 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:39.224 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.224 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.224 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.224 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share basic cluster information (size, version)", 2023-11-16T16:49:39.224 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.224 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.224 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.225 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_crash": { 2023-11-16T16:49:39.225 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_crash", 2023-11-16T16:49:39.225 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.225 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.225 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.225 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:39.225 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.226 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.226 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.226 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-11-16T16:49:39.226 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.226 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.226 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.226 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_device": { 2023-11-16T16:49:39.227 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_device", 2023-11-16T16:49:39.227 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.227 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.227 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.227 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2023-11-16T16:49:39.227 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.227 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.227 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.228 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-11-16T16:49:39.228 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.228 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.228 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.228 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_ident": { 2023-11-16T16:49:39.228 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_ident", 2023-11-16T16:49:39.228 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.229 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.229 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.229 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.229 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.229 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.229 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.229 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-11-16T16:49:39.229 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.229 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.230 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.230 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_perf": { 2023-11-16T16:49:39.230 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_perf", 2023-11-16T16:49:39.230 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.230 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.230 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.230 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.231 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.231 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.231 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.231 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share various performance metrics of a cluster", 2023-11-16T16:49:39.231 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.231 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.231 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.232 INFO:teuthology.orchestra.run.smithi001.stdout: "contact": { 2023-11-16T16:49:39.232 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "contact", 2023-11-16T16:49:39.232 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.232 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.232 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.232 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.232 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.232 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.232 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.233 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.233 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.233 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.233 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.233 INFO:teuthology.orchestra.run.smithi001.stdout: "description": { 2023-11-16T16:49:39.233 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "description", 2023-11-16T16:49:39.233 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.234 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.234 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.234 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.234 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.234 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.234 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.234 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.234 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.235 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.235 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.235 INFO:teuthology.orchestra.run.smithi001.stdout: "device_url": { 2023-11-16T16:49:39.235 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_url", 2023-11-16T16:49:39.235 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.235 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.235 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.236 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-11-16T16:49:39.236 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.236 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.236 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.236 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.236 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.236 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.236 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.237 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2023-11-16T16:49:39.237 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2023-11-16T16:49:39.237 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.237 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.237 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.237 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.237 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.238 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.238 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.238 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.238 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.238 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.238 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.238 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2023-11-16T16:49:39.238 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2023-11-16T16:49:39.239 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:39.239 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.239 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.239 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "24", 2023-11-16T16:49:39.239 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "8", 2023-11-16T16:49:39.239 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.239 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.239 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.240 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.240 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.240 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.240 INFO:teuthology.orchestra.run.smithi001.stdout: "last_opt_revision": { 2023-11-16T16:49:39.240 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "last_opt_revision", 2023-11-16T16:49:39.240 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:39.240 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.240 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.241 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1", 2023-11-16T16:49:39.241 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.241 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.241 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.241 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.241 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.241 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.241 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.242 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard": { 2023-11-16T16:49:39.242 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard", 2023-11-16T16:49:39.242 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.242 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.242 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.242 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.242 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.243 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.243 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.243 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.243 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.243 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.243 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.243 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard_description": { 2023-11-16T16:49:39.244 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard_description", 2023-11-16T16:49:39.244 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.244 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.244 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.244 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.244 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.244 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.244 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.244 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.245 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.245 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.245 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.245 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:39.245 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:39.245 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.245 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.246 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.246 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.246 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.246 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.246 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:39.246 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:39.246 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:39.246 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:39.247 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:39.247 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:39.247 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:39.247 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.247 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.247 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.247 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.247 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.248 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:39.248 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:39.248 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.248 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.248 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.248 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.248 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.249 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.249 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.249 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.249 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.249 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.249 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.249 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:39.250 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:39.250 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.250 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.250 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.250 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:39.250 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.250 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.250 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:39.251 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:39.251 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:39.251 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:39.251 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:39.251 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:39.251 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:39.251 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.251 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.252 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.252 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.252 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.252 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:39.252 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:39.252 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.252 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.253 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.253 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.253 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.253 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.253 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.253 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.253 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.253 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.253 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.254 INFO:teuthology.orchestra.run.smithi001.stdout: "organization": { 2023-11-16T16:49:39.254 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "organization", 2023-11-16T16:49:39.254 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.254 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.254 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.254 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.254 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.255 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.255 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.255 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.255 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.255 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.255 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.255 INFO:teuthology.orchestra.run.smithi001.stdout: "proxy": { 2023-11-16T16:49:39.256 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "proxy", 2023-11-16T16:49:39.256 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.256 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.256 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.256 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.256 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.256 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.257 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.257 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.257 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.257 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.257 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.257 INFO:teuthology.orchestra.run.smithi001.stdout: "url": { 2023-11-16T16:49:39.257 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url", 2023-11-16T16:49:39.258 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.258 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.258 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.258 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-11-16T16:49:39.258 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.258 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.258 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.258 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.259 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.259 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.259 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.259 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.259 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:39.259 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "test_orchestrator", 2023-11-16T16:49:39.260 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:39.260 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:39.260 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:39.260 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:39.260 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:39.260 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.260 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.260 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.261 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.261 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.261 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.261 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:39.261 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:39.261 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:39.261 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:39.261 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:39.262 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:39.262 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:39.262 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.262 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.262 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.262 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.262 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.263 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:39.263 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:39.263 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.263 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.263 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.263 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.263 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.263 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.263 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.264 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.264 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.264 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.264 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.264 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:39.265 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:39.265 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.265 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.265 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.265 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:39.265 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.265 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.266 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:39.266 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:39.266 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:39.266 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:39.266 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:39.266 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:39.266 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:39.267 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.267 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.267 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.267 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.267 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.267 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:39.267 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:39.267 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.268 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.268 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.268 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.268 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.268 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.268 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.268 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.268 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.269 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.269 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.269 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.269 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.269 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:39.269 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "volumes", 2023-11-16T16:49:39.269 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:39.269 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:39.270 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:39.270 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:39.270 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:39.270 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.270 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.270 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.270 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.270 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.271 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.271 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:39.271 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:39.271 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:39.271 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:39.271 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:39.271 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:39.271 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:39.272 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.272 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.272 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.272 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.272 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.272 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:39.272 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:39.273 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.273 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.273 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.273 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.273 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.273 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.273 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.273 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.273 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.274 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.274 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.274 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:39.274 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:39.274 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.274 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.274 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.275 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:39.275 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.275 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.275 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:39.275 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:39.275 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:39.275 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:39.275 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:39.276 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:39.276 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:39.276 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.276 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.276 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.276 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.276 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.277 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:39.277 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:39.277 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.277 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.277 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.277 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.277 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.277 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.278 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.278 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.278 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.278 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.278 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.278 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_clones": { 2023-11-16T16:49:39.279 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_clones", 2023-11-16T16:49:39.279 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:39.279 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.279 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.279 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4", 2023-11-16T16:49:39.279 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.279 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.279 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.280 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Number of asynchronous cloner threads", 2023-11-16T16:49:39.280 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.280 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.280 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.280 INFO:teuthology.orchestra.run.smithi001.stdout: "snapshot_clone_delay": { 2023-11-16T16:49:39.280 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snapshot_clone_delay", 2023-11-16T16:49:39.281 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:39.281 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.281 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.281 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2023-11-16T16:49:39.281 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.281 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.281 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.281 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-11-16T16:49:39.282 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.282 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.282 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.282 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.282 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:39.282 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix", 2023-11-16T16:49:39.283 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2023-11-16T16:49:39.283 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2023-11-16T16:49:39.283 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2023-11-16T16:49:39.283 INFO:teuthology.orchestra.run.smithi001.stdout: "discovery_interval": { 2023-11-16T16:49:39.283 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "discovery_interval", 2023-11-16T16:49:39.283 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2023-11-16T16:49:39.283 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.283 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.284 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "100", 2023-11-16T16:49:39.284 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.284 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.284 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.284 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.284 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.284 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.284 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.285 INFO:teuthology.orchestra.run.smithi001.stdout: "identifier": { 2023-11-16T16:49:39.285 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "identifier", 2023-11-16T16:49:39.285 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.285 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.285 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.285 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.286 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.286 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.286 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.286 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.286 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.286 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.286 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.287 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2023-11-16T16:49:39.287 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2023-11-16T16:49:39.287 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2023-11-16T16:49:39.287 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.287 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.287 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2023-11-16T16:49:39.287 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.287 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.288 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.288 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.288 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.288 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.288 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.288 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2023-11-16T16:49:39.288 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2023-11-16T16:49:39.289 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.289 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.289 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.289 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.289 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.289 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.289 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:39.289 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:39.289 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:39.290 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:39.290 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:39.290 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:39.290 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:39.290 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.290 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.290 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.291 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.291 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.291 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2023-11-16T16:49:39.291 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2023-11-16T16:49:39.291 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.291 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.291 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.292 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.292 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.292 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.292 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.292 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.292 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.292 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.292 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.293 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2023-11-16T16:49:39.293 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2023-11-16T16:49:39.293 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.293 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.293 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.293 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2023-11-16T16:49:39.293 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.294 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.294 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2023-11-16T16:49:39.294 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2023-11-16T16:49:39.294 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2023-11-16T16:49:39.294 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2023-11-16T16:49:39.294 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2023-11-16T16:49:39.294 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2023-11-16T16:49:39.294 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2023-11-16T16:49:39.295 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.295 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.295 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.295 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.295 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.295 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2023-11-16T16:49:39.296 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2023-11-16T16:49:39.296 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2023-11-16T16:49:39.296 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.296 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2023-11-16T16:49:39.296 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2023-11-16T16:49:39.296 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.296 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.296 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.297 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.297 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.297 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.297 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.297 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_host": { 2023-11-16T16:49:39.297 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_host", 2023-11-16T16:49:39.297 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.298 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.298 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.298 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2023-11-16T16:49:39.298 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.298 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.298 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.298 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.298 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.299 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.299 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.299 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_port": { 2023-11-16T16:49:39.299 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_port", 2023-11-16T16:49:39.299 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2023-11-16T16:49:39.299 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.299 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.300 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10051", 2023-11-16T16:49:39.300 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.300 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.300 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.300 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.300 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.300 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.300 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.301 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_sender": { 2023-11-16T16:49:39.301 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_sender", 2023-11-16T16:49:39.301 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2023-11-16T16:49:39.301 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2023-11-16T16:49:39.301 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2023-11-16T16:49:39.301 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-11-16T16:49:39.301 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2023-11-16T16:49:39.301 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2023-11-16T16:49:39.302 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2023-11-16T16:49:39.302 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2023-11-16T16:49:39.302 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2023-11-16T16:49:39.302 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2023-11-16T16:49:39.302 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2023-11-16T16:49:39.302 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.302 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.303 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.303 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.303 INFO:teuthology.orchestra.run.smithi001.stdout: "services": { 2023-11-16T16:49:39.303 INFO:teuthology.orchestra.run.smithi001.stdout: "dashboard": "https://172.21.15.1:8443/" 2023-11-16T16:49:39.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.303 INFO:teuthology.orchestra.run.smithi001.stdout: "always_on_modules": { 2023-11-16T16:49:39.303 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus": [ 2023-11-16T16:49:39.303 INFO:teuthology.orchestra.run.smithi001.stdout: "balancer", 2023-11-16T16:49:39.303 INFO:teuthology.orchestra.run.smithi001.stdout: "crash", 2023-11-16T16:49:39.304 INFO:teuthology.orchestra.run.smithi001.stdout: "devicehealth", 2023-11-16T16:49:39.304 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator", 2023-11-16T16:49:39.304 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscaler", 2023-11-16T16:49:39.304 INFO:teuthology.orchestra.run.smithi001.stdout: "progress", 2023-11-16T16:49:39.304 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_support", 2023-11-16T16:49:39.304 INFO:teuthology.orchestra.run.smithi001.stdout: "status", 2023-11-16T16:49:39.304 INFO:teuthology.orchestra.run.smithi001.stdout: "telemetry", 2023-11-16T16:49:39.304 INFO:teuthology.orchestra.run.smithi001.stdout: "volumes" 2023-11-16T16:49:39.305 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.305 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific": [ 2023-11-16T16:49:39.305 INFO:teuthology.orchestra.run.smithi001.stdout: "balancer", 2023-11-16T16:49:39.305 INFO:teuthology.orchestra.run.smithi001.stdout: "crash", 2023-11-16T16:49:39.305 INFO:teuthology.orchestra.run.smithi001.stdout: "devicehealth", 2023-11-16T16:49:39.305 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator", 2023-11-16T16:49:39.305 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscaler", 2023-11-16T16:49:39.305 INFO:teuthology.orchestra.run.smithi001.stdout: "progress", 2023-11-16T16:49:39.306 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_support", 2023-11-16T16:49:39.306 INFO:teuthology.orchestra.run.smithi001.stdout: "status", 2023-11-16T16:49:39.306 INFO:teuthology.orchestra.run.smithi001.stdout: "telemetry", 2023-11-16T16:49:39.306 INFO:teuthology.orchestra.run.smithi001.stdout: "volumes" 2023-11-16T16:49:39.306 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.306 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy": [ 2023-11-16T16:49:39.306 INFO:teuthology.orchestra.run.smithi001.stdout: "balancer", 2023-11-16T16:49:39.306 INFO:teuthology.orchestra.run.smithi001.stdout: "crash", 2023-11-16T16:49:39.307 INFO:teuthology.orchestra.run.smithi001.stdout: "devicehealth", 2023-11-16T16:49:39.307 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator", 2023-11-16T16:49:39.307 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscaler", 2023-11-16T16:49:39.307 INFO:teuthology.orchestra.run.smithi001.stdout: "progress", 2023-11-16T16:49:39.307 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_support", 2023-11-16T16:49:39.307 INFO:teuthology.orchestra.run.smithi001.stdout: "status", 2023-11-16T16:49:39.307 INFO:teuthology.orchestra.run.smithi001.stdout: "telemetry", 2023-11-16T16:49:39.307 INFO:teuthology.orchestra.run.smithi001.stdout: "volumes" 2023-11-16T16:49:39.308 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2023-11-16T16:49:39.308 INFO:teuthology.orchestra.run.smithi001.stdout: "reef": [ 2023-11-16T16:49:39.308 INFO:teuthology.orchestra.run.smithi001.stdout: "balancer", 2023-11-16T16:49:39.308 INFO:teuthology.orchestra.run.smithi001.stdout: "crash", 2023-11-16T16:49:39.308 INFO:teuthology.orchestra.run.smithi001.stdout: "devicehealth", 2023-11-16T16:49:39.308 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator", 2023-11-16T16:49:39.308 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscaler", 2023-11-16T16:49:39.309 INFO:teuthology.orchestra.run.smithi001.stdout: "progress", 2023-11-16T16:49:39.309 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_support", 2023-11-16T16:49:39.309 INFO:teuthology.orchestra.run.smithi001.stdout: "status", 2023-11-16T16:49:39.309 INFO:teuthology.orchestra.run.smithi001.stdout: "telemetry", 2023-11-16T16:49:39.309 INFO:teuthology.orchestra.run.smithi001.stdout: "volumes" 2023-11-16T16:49:39.309 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2023-11-16T16:49:39.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.309 INFO:teuthology.orchestra.run.smithi001.stdout: "last_failure_osd_epoch": 87, 2023-11-16T16:49:39.310 INFO:teuthology.orchestra.run.smithi001.stdout: "active_clients": [ 2023-11-16T16:49:39.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:39.310 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "devicehealth", 2023-11-16T16:49:39.310 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2023-11-16T16:49:39.310 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:39.310 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2023-11-16T16:49:39.310 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:0", 2023-11-16T16:49:39.310 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 3010095125 2023-11-16T16:49:39.311 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.311 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2023-11-16T16:49:39.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.311 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:39.311 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "libcephsqlite", 2023-11-16T16:49:39.311 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2023-11-16T16:49:39.311 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:39.311 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2023-11-16T16:49:39.312 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:0", 2023-11-16T16:49:39.312 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1863378890 2023-11-16T16:49:39.312 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.312 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2023-11-16T16:49:39.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:39.312 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_support", 2023-11-16T16:49:39.312 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2023-11-16T16:49:39.312 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:39.313 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2023-11-16T16:49:39.313 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:0", 2023-11-16T16:49:39.313 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4119867098 2023-11-16T16:49:39.313 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.313 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2023-11-16T16:49:39.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.313 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:39.313 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snap_schedule", 2023-11-16T16:49:39.314 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2023-11-16T16:49:39.314 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:39.314 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2023-11-16T16:49:39.314 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:0", 2023-11-16T16:49:39.314 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2734955863 2023-11-16T16:49:39.314 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.314 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2023-11-16T16:49:39.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:39.315 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snap_schedule", 2023-11-16T16:49:39.315 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2023-11-16T16:49:39.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:39.315 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2023-11-16T16:49:39.315 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:0", 2023-11-16T16:49:39.315 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2464142517 2023-11-16T16:49:39.315 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.316 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2023-11-16T16:49:39.316 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:39.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:39.316 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "volumes", 2023-11-16T16:49:39.316 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2023-11-16T16:49:39.316 INFO:teuthology.orchestra.run.smithi001.stdout: { 2023-11-16T16:49:39.316 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2023-11-16T16:49:39.316 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:0", 2023-11-16T16:49:39.317 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1404554491 2023-11-16T16:49:39.317 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.317 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2023-11-16T16:49:39.317 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:39.317 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2023-11-16T16:49:39.317 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:49:39.319 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:36 smithi053 ceph-mon[100638]: pgmap v324: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:49:39.319 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:36 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/886397378' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:49:39.319 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:36 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1183631661' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:49:39.320 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:36 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/4083684091' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:49:39.320 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:37 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/3860561352' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-11-16T16:49:39.320 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:37 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:49:39.320 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:37 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:49:39.320 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:37 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:49:39.320 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:37 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:49:39.320 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:38 smithi053 ceph-mon[100638]: pgmap v325: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:49:39.321 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:38 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:49:39.321 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:38 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:49:39.321 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:38 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:49:39.321 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:38 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:49:39.321 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:38 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:49:39.321 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:38 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:49:39.321 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:38 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:49:39.322 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:38 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:49:39.322 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:38 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:49:39.322 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:38 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:49:39.322 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:36 smithi026 ceph-mon[100112]: pgmap v324: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:49:39.322 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:36 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/886397378' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:49:39.323 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:36 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1183631661' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:49:39.323 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:36 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/4083684091' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:49:39.323 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:37 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/3860561352' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-11-16T16:49:39.324 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:37 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:49:39.324 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:37 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:49:39.324 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:37 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:49:39.324 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:37 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:49:39.325 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:38 smithi026 ceph-mon[100112]: pgmap v325: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:49:39.325 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:38 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:49:39.325 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:38 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:49:39.325 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:38 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:49:39.326 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:38 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:49:39.326 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:38 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:49:39.326 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:38 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:49:39.326 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:38 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:49:39.327 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:38 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:49:39.327 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:38 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:49:39.327 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:38 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:49:39.328 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.952+0000 7f9214e8e700 1 -- 172.21.15.1:0/3860561352 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f91f80a1340 msgr2=0x7f91f80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:39.328 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.952+0000 7f9214e8e700 1 --2- 172.21.15.1:0/3860561352 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f91f80a1340 0x7f91f80a37f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f920000bc10 tx=0x7f920000b5f0 comp rx=0 tx=0).stop 2023-11-16T16:49:39.328 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.952+0000 7f9214e8e700 1 -- 172.21.15.1:0/3860561352 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9210104470 msgr2=0x7f92101a2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:49:39.329 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.952+0000 7f9214e8e700 1 --2- 172.21.15.1:0/3860561352 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9210104470 0x7f92101a2330 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f9208007ee0 tx=0x7f920800d3b0 comp rx=0 tx=0).stop 2023-11-16T16:49:39.329 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.953+0000 7f9214e8e700 1 -- 172.21.15.1:0/3860561352 shutdown_connections 2023-11-16T16:49:39.329 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.953+0000 7f9214e8e700 1 --2- 172.21.15.1:0/3860561352 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f91f80a1340 0x7f91f80a37f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:39.329 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.953+0000 7f9214e8e700 1 --2- 172.21.15.1:0/3860561352 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9210104470 0x7f92101a2330 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:39.330 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.953+0000 7f9214e8e700 1 --2- 172.21.15.1:0/3860561352 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f92100ff430 0x7f92101972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:39.330 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.953+0000 7f9214e8e700 1 --2- 172.21.15.1:0/3860561352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92100ffdd0 0x7f92101977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:49:39.330 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.953+0000 7f9214e8e700 1 -- 172.21.15.1:0/3860561352 >> 172.21.15.1:0/3860561352 conn(0x7f9210077020 msgr2=0x7f92100fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:49:39.331 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.953+0000 7f9214e8e700 1 -- 172.21.15.1:0/3860561352 shutdown_connections 2023-11-16T16:49:39.331 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:49:36.953+0000 7f9214e8e700 1 -- 172.21.15.1:0/3860561352 wait complete. 2023-11-16T16:49:39.339 DEBUG:tasks.check_counter:Getting stats from a 2023-11-16T16:49:39.339 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph daemon mds.a perf dump 2023-11-16T16:49:39.340 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:37 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3860561352' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-11-16T16:49:39.340 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:37 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:49:39.340 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:37 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:49:39.340 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:37 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:49:39.341 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:37 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:49:39.341 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:38 smithi001 ceph-mon[98800]: pgmap v325: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:49:39.341 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:38 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:49:39.341 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:38 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:49:39.341 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:38 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:49:39.341 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:38 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:49:39.341 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:38 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:49:39.341 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:38 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:49:39.341 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:38 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:49:39.342 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:38 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:49:39.342 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:38 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:49:39.342 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:38 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:49:39.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:39 smithi001 ceph-mon[98800]: Adjusting osd_memory_target on smithi053 to 197.2M 2023-11-16T16:49:39.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:39 smithi001 ceph-mon[98800]: Unable to set osd_memory_target on smithi053 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-11-16T16:49:39.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:39 smithi001 ceph-mon[98800]: Adjusting osd_memory_target on smithi026 to 197.2M 2023-11-16T16:49:39.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:39 smithi001 ceph-mon[98800]: Unable to set osd_memory_target on smithi026 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-11-16T16:49:39.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:39 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:49:39.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:39 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:49:39.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:39 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:49:39.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:39 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:49:39.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:39 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:49:39.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:39 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:49:39.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:39 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:49:40.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:39 smithi026 ceph-mon[100112]: Adjusting osd_memory_target on smithi053 to 197.2M 2023-11-16T16:49:40.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:39 smithi026 ceph-mon[100112]: Unable to set osd_memory_target on smithi053 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-11-16T16:49:40.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:39 smithi026 ceph-mon[100112]: Adjusting osd_memory_target on smithi026 to 197.2M 2023-11-16T16:49:40.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:39 smithi026 ceph-mon[100112]: Unable to set osd_memory_target on smithi026 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-11-16T16:49:40.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:39 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:49:40.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:39 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:49:40.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:39 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:49:40.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:39 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:49:40.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:39 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:49:40.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:39 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:49:40.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:39 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:49:40.029 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:49:40.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:39 smithi053 ceph-mon[100638]: Adjusting osd_memory_target on smithi053 to 197.2M 2023-11-16T16:49:40.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:39 smithi053 ceph-mon[100638]: Unable to set osd_memory_target on smithi053 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-11-16T16:49:40.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:39 smithi053 ceph-mon[100638]: Adjusting osd_memory_target on smithi026 to 197.2M 2023-11-16T16:49:40.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:39 smithi053 ceph-mon[100638]: Unable to set osd_memory_target on smithi026 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-11-16T16:49:40.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:39 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:49:40.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:39 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:49:40.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:39 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:49:40.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:39 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-16T16:49:40.082 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:39 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-16T16:49:40.082 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:39 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-16T16:49:40.082 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:39 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' 2023-11-16T16:49:40.737 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:40 smithi001 ceph-mon[98800]: Adjusting osd_memory_target on smithi001 to 197.2M 2023-11-16T16:49:40.737 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:40 smithi001 ceph-mon[98800]: Unable to set osd_memory_target on smithi001 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-11-16T16:49:40.737 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:40 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:49:40.737 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:40 smithi001 ceph-mon[98800]: Service mds.all has no related daemon already placed. 2023-11-16T16:49:40.737 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:40 smithi001 ceph-mon[98800]: Service mgr has no related daemon already placed. 2023-11-16T16:49:40.738 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:40 smithi001 ceph-mon[98800]: Service mon has no related daemon already placed. 2023-11-16T16:49:40.738 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:40 smithi001 ceph-mon[98800]: pgmap v326: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:49:41.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:40 smithi026 ceph-mon[100112]: Adjusting osd_memory_target on smithi001 to 197.2M 2023-11-16T16:49:41.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:40 smithi026 ceph-mon[100112]: Unable to set osd_memory_target on smithi001 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-11-16T16:49:41.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:40 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:49:41.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:40 smithi026 ceph-mon[100112]: Service mds.all has no related daemon already placed. 2023-11-16T16:49:41.008 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:40 smithi026 ceph-mon[100112]: Service mgr has no related daemon already placed. 2023-11-16T16:49:41.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:40 smithi026 ceph-mon[100112]: Service mon has no related daemon already placed. 2023-11-16T16:49:41.009 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:40 smithi026 ceph-mon[100112]: pgmap v326: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:49:41.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:40 smithi053 ceph-mon[100638]: Adjusting osd_memory_target on smithi001 to 197.2M 2023-11-16T16:49:41.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:40 smithi053 ceph-mon[100638]: Unable to set osd_memory_target on smithi001 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-11-16T16:49:41.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:40 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:49:41.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:40 smithi053 ceph-mon[100638]: Service mds.all has no related daemon already placed. 2023-11-16T16:49:41.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:40 smithi053 ceph-mon[100638]: Service mgr has no related daemon already placed. 2023-11-16T16:49:41.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:40 smithi053 ceph-mon[100638]: Service mon has no related daemon already placed. 2023-11-16T16:49:41.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:40 smithi053 ceph-mon[100638]: pgmap v326: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:49:41.720 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:49:41.720 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-0": { 2023-11-16T16:49:41.720 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 217, 2023-11-16T16:49:41.721 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 205, 2023-11-16T16:49:41.721 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 1422256, 2023-11-16T16:49:41.721 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 116048, 2023-11-16T16:49:41.721 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 4, 2023-11-16T16:49:41.721 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 0, 2023-11-16T16:49:41.721 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.041364866, 2023-11-16T16:49:41.721 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.013042260, 2023-11-16T16:49:41.721 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.030260310, 2023-11-16T16:49:41.722 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.004879875, 2023-11-16T16:49:41.722 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:49:41.722 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 205, 2023-11-16T16:49:41.722 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.011127394, 2023-11-16T16:49:41.722 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000054279 2023-11-16T16:49:41.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:41.722 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:49:41.722 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2023-11-16T16:49:41.723 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2023-11-16T16:49:41.723 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2023-11-16T16:49:41.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:41.723 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 1422256, 2023-11-16T16:49:41.723 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 116048 2023-11-16T16:49:41.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:41.723 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-1": { 2023-11-16T16:49:41.723 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 2, 2023-11-16T16:49:41.724 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 155, 2023-11-16T16:49:41.724 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 320, 2023-11-16T16:49:41.724 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 178976, 2023-11-16T16:49:41.724 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 5, 2023-11-16T16:49:41.724 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 3, 2023-11-16T16:49:41.724 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.012639823, 2023-11-16T16:49:41.724 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.008037579, 2023-11-16T16:49:41.724 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.002268791, 2023-11-16T16:49:41.725 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-16T16:49:41.725 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:49:41.725 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 152, 2023-11-16T16:49:41.725 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.017006190, 2023-11-16T16:49:41.725 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000111882 2023-11-16T16:49:41.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:41.726 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:49:41.726 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2023-11-16T16:49:41.726 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2023-11-16T16:49:41.726 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2023-11-16T16:49:41.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:41.726 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 320, 2023-11-16T16:49:41.727 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 178976 2023-11-16T16:49:41.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:41.727 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-2": { 2023-11-16T16:49:41.727 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 0, 2023-11-16T16:49:41.727 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 2, 2023-11-16T16:49:41.727 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 0, 2023-11-16T16:49:41.727 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 0, 2023-11-16T16:49:41.727 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 2, 2023-11-16T16:49:41.727 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 1, 2023-11-16T16:49:41.728 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.000322939, 2023-11-16T16:49:41.728 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.000000000, 2023-11-16T16:49:41.728 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.000236195, 2023-11-16T16:49:41.728 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-16T16:49:41.728 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:49:41.728 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2023-11-16T16:49:41.728 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2023-11-16T16:49:41.728 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2023-11-16T16:49:41.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:41.729 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:49:41.729 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2023-11-16T16:49:41.729 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2023-11-16T16:49:41.729 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2023-11-16T16:49:41.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:41.729 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-16T16:49:41.729 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 0 2023-11-16T16:49:41.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:41.730 INFO:teuthology.orchestra.run.smithi001.stdout: "cct": { 2023-11-16T16:49:41.730 INFO:teuthology.orchestra.run.smithi001.stdout: "total_workers": 0, 2023-11-16T16:49:41.730 INFO:teuthology.orchestra.run.smithi001.stdout: "unhealthy_workers": 0 2023-11-16T16:49:41.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:41.730 INFO:teuthology.orchestra.run.smithi001.stdout: "mempool": { 2023-11-16T16:49:41.730 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_bytes": 0, 2023-11-16T16:49:41.730 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_items": 0, 2023-11-16T16:49:41.731 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_bytes": 0, 2023-11-16T16:49:41.731 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_items": 0, 2023-11-16T16:49:41.731 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_bytes": 0, 2023-11-16T16:49:41.731 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_items": 0, 2023-11-16T16:49:41.731 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-16T16:49:41.731 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_items": 0, 2023-11-16T16:49:41.731 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-16T16:49:41.731 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_items": 0, 2023-11-16T16:49:41.732 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_bytes": 0, 2023-11-16T16:49:41.732 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_items": 0, 2023-11-16T16:49:41.732 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-16T16:49:41.732 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_items": 0, 2023-11-16T16:49:41.732 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_bytes": 0, 2023-11-16T16:49:41.732 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_items": 0, 2023-11-16T16:49:41.732 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_bytes": 0, 2023-11-16T16:49:41.732 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_items": 0, 2023-11-16T16:49:41.732 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-16T16:49:41.733 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_items": 0, 2023-11-16T16:49:41.733 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-16T16:49:41.733 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_items": 0, 2023-11-16T16:49:41.733 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_bytes": 0, 2023-11-16T16:49:41.733 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_items": 0, 2023-11-16T16:49:41.733 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_bytes": 0, 2023-11-16T16:49:41.733 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_items": 0, 2023-11-16T16:49:41.733 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-16T16:49:41.734 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_items": 0, 2023-11-16T16:49:41.734 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_bytes": 0, 2023-11-16T16:49:41.734 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_items": 0, 2023-11-16T16:49:41.734 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_bytes": 0, 2023-11-16T16:49:41.734 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_items": 0, 2023-11-16T16:49:41.735 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_bytes": 0, 2023-11-16T16:49:41.735 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_items": 0, 2023-11-16T16:49:41.735 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_bytes": 0, 2023-11-16T16:49:41.735 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_items": 0, 2023-11-16T16:49:41.735 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_bytes": 148977, 2023-11-16T16:49:41.735 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_items": 30, 2023-11-16T16:49:41.735 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_bytes": 0, 2023-11-16T16:49:41.736 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_items": 0, 2023-11-16T16:49:41.736 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bytes": 0, 2023-11-16T16:49:41.736 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_items": 0, 2023-11-16T16:49:41.736 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_bytes": 0, 2023-11-16T16:49:41.736 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_items": 0, 2023-11-16T16:49:41.736 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_bytes": 0, 2023-11-16T16:49:41.736 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_items": 0, 2023-11-16T16:49:41.736 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_bytes": 0, 2023-11-16T16:49:41.737 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_items": 0, 2023-11-16T16:49:41.737 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_bytes": 0, 2023-11-16T16:49:41.737 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_items": 0, 2023-11-16T16:49:41.737 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_bytes": 0, 2023-11-16T16:49:41.737 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_items": 0, 2023-11-16T16:49:41.737 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_bytes": 928, 2023-11-16T16:49:41.737 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_items": 2, 2023-11-16T16:49:41.737 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_bytes": 0, 2023-11-16T16:49:41.737 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_items": 0, 2023-11-16T16:49:41.738 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_bytes": 0, 2023-11-16T16:49:41.738 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_items": 0 2023-11-16T16:49:41.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:41.738 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-16T16:49:41.738 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2023-11-16T16:49:41.738 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 104857600, 2023-11-16T16:49:41.738 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2023-11-16T16:49:41.739 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 215, 2023-11-16T16:49:41.739 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 1389199, 2023-11-16T16:49:41.739 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2023-11-16T16:49:41.739 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 215, 2023-11-16T16:49:41.739 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2023-11-16T16:49:41.739 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2023-11-16T16:49:41.739 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 215, 2023-11-16T16:49:41.739 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 1389199, 2023-11-16T16:49:41.739 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2023-11-16T16:49:41.740 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2023-11-16T16:49:41.740 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2023-11-16T16:49:41.740 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2023-11-16T16:49:41.740 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:41.740 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:41.740 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:49:42.097 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2023-11-16T16:49:42.097 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2023-11-16T16:49:42.098 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2023-11-16T16:49:42.098 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2023-11-16T16:49:42.098 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2023-11-16T16:49:42.098 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2023-11-16T16:49:42.098 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2023-11-16T16:49:42.098 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2023-11-16T16:49:42.099 DEBUG:tasks.check_counter:Getting stats from d 2023-11-16T16:49:42.099 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph daemon mds.d perf dump 2023-11-16T16:49:42.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:42 smithi001 ceph-mon[98800]: pgmap v327: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:49:42.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:42 smithi026 ceph-mon[100112]: pgmap v327: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:49:42.766 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:49:42.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:42 smithi053 ceph-mon[100638]: pgmap v327: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:49:43.887 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:43 smithi001 ceph-mon[98800]: pgmap v328: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:49:44.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:43 smithi053 ceph-mon[100638]: pgmap v328: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:49:44.125 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:49:44.125 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-0": { 2023-11-16T16:49:44.125 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 0, 2023-11-16T16:49:44.126 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 2, 2023-11-16T16:49:44.126 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 0, 2023-11-16T16:49:44.126 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 0, 2023-11-16T16:49:44.126 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 4, 2023-11-16T16:49:44.126 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 0, 2023-11-16T16:49:44.127 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.001583721, 2023-11-16T16:49:44.127 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.000000000, 2023-11-16T16:49:44.127 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.000434675, 2023-11-16T16:49:44.127 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-16T16:49:44.127 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:49:44.128 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2023-11-16T16:49:44.128 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2023-11-16T16:49:44.128 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2023-11-16T16:49:44.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:44.128 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:49:44.129 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2023-11-16T16:49:44.129 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2023-11-16T16:49:44.129 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2023-11-16T16:49:44.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:44.129 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-16T16:49:44.130 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 0 2023-11-16T16:49:44.130 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:44.130 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-1": { 2023-11-16T16:49:44.130 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 6, 2023-11-16T16:49:44.130 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 160, 2023-11-16T16:49:44.130 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 3312, 2023-11-16T16:49:44.131 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 179952, 2023-11-16T16:49:44.131 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 5, 2023-11-16T16:49:44.131 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 3, 2023-11-16T16:49:44.131 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.013616597, 2023-11-16T16:49:44.131 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.009125895, 2023-11-16T16:49:44.132 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.002139840, 2023-11-16T16:49:44.132 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-16T16:49:44.132 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:49:44.132 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 158, 2023-11-16T16:49:44.132 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.013811474, 2023-11-16T16:49:44.133 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000087414 2023-11-16T16:49:44.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:44.133 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:49:44.133 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2023-11-16T16:49:44.133 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2023-11-16T16:49:44.133 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2023-11-16T16:49:44.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:44.134 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 3312, 2023-11-16T16:49:44.134 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 179952 2023-11-16T16:49:44.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:44.134 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-2": { 2023-11-16T16:49:44.135 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 218, 2023-11-16T16:49:44.135 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 208, 2023-11-16T16:49:44.135 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 1420624, 2023-11-16T16:49:44.135 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 118560, 2023-11-16T16:49:44.135 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 2, 2023-11-16T16:49:44.136 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 1, 2023-11-16T16:49:44.136 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.043267283, 2023-11-16T16:49:44.136 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.014276841, 2023-11-16T16:49:44.136 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.032602380, 2023-11-16T16:49:44.136 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.004887101, 2023-11-16T16:49:44.136 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:49:44.137 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 207, 2023-11-16T16:49:44.137 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.009757895, 2023-11-16T16:49:44.137 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000047139 2023-11-16T16:49:44.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:44.137 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:49:44.138 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2023-11-16T16:49:44.138 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2023-11-16T16:49:44.138 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2023-11-16T16:49:44.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:44.138 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 1420624, 2023-11-16T16:49:44.138 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 118560 2023-11-16T16:49:44.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:44.139 INFO:teuthology.orchestra.run.smithi001.stdout: "cct": { 2023-11-16T16:49:44.139 INFO:teuthology.orchestra.run.smithi001.stdout: "total_workers": 0, 2023-11-16T16:49:44.139 INFO:teuthology.orchestra.run.smithi001.stdout: "unhealthy_workers": 0 2023-11-16T16:49:44.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:44.140 INFO:teuthology.orchestra.run.smithi001.stdout: "mempool": { 2023-11-16T16:49:44.140 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_bytes": 0, 2023-11-16T16:49:44.140 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_items": 0, 2023-11-16T16:49:44.140 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_bytes": 0, 2023-11-16T16:49:44.140 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_items": 0, 2023-11-16T16:49:44.140 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_bytes": 0, 2023-11-16T16:49:44.141 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_items": 0, 2023-11-16T16:49:44.141 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-16T16:49:44.141 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_items": 0, 2023-11-16T16:49:44.141 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-16T16:49:44.141 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_items": 0, 2023-11-16T16:49:44.141 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_bytes": 0, 2023-11-16T16:49:44.142 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_items": 0, 2023-11-16T16:49:44.142 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-16T16:49:44.142 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_items": 0, 2023-11-16T16:49:44.142 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_bytes": 0, 2023-11-16T16:49:44.142 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_items": 0, 2023-11-16T16:49:44.143 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_bytes": 0, 2023-11-16T16:49:44.143 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_items": 0, 2023-11-16T16:49:44.143 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-16T16:49:44.143 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_items": 0, 2023-11-16T16:49:44.143 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-16T16:49:44.143 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_items": 0, 2023-11-16T16:49:44.144 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_bytes": 0, 2023-11-16T16:49:44.144 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_items": 0, 2023-11-16T16:49:44.144 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_bytes": 0, 2023-11-16T16:49:44.144 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_items": 0, 2023-11-16T16:49:44.144 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-16T16:49:44.144 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_items": 0, 2023-11-16T16:49:44.145 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_bytes": 0, 2023-11-16T16:49:44.145 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_items": 0, 2023-11-16T16:49:44.145 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_bytes": 0, 2023-11-16T16:49:44.145 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_items": 0, 2023-11-16T16:49:44.145 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_bytes": 0, 2023-11-16T16:49:44.145 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_items": 0, 2023-11-16T16:49:44.146 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_bytes": 0, 2023-11-16T16:49:44.146 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_items": 0, 2023-11-16T16:49:44.146 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_bytes": 149047, 2023-11-16T16:49:44.146 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_items": 31, 2023-11-16T16:49:44.146 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_bytes": 0, 2023-11-16T16:49:44.146 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_items": 0, 2023-11-16T16:49:44.147 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bytes": 0, 2023-11-16T16:49:44.147 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_items": 0, 2023-11-16T16:49:44.147 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_bytes": 0, 2023-11-16T16:49:44.147 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_items": 0, 2023-11-16T16:49:44.147 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_bytes": 0, 2023-11-16T16:49:44.148 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_items": 0, 2023-11-16T16:49:44.148 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_bytes": 0, 2023-11-16T16:49:44.148 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_items": 0, 2023-11-16T16:49:44.148 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_bytes": 0, 2023-11-16T16:49:44.148 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_items": 0, 2023-11-16T16:49:44.148 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_bytes": 0, 2023-11-16T16:49:44.149 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_items": 0, 2023-11-16T16:49:44.149 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_bytes": 928, 2023-11-16T16:49:44.149 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_items": 2, 2023-11-16T16:49:44.149 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_bytes": 0, 2023-11-16T16:49:44.149 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_items": 0, 2023-11-16T16:49:44.149 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_bytes": 0, 2023-11-16T16:49:44.149 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_items": 0 2023-11-16T16:49:44.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:44.150 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-16T16:49:44.150 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2023-11-16T16:49:44.150 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 104857600, 2023-11-16T16:49:44.150 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2023-11-16T16:49:44.151 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 220, 2023-11-16T16:49:44.151 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 1389849, 2023-11-16T16:49:44.151 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2023-11-16T16:49:44.151 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 220, 2023-11-16T16:49:44.151 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2023-11-16T16:49:44.151 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2023-11-16T16:49:44.152 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 220, 2023-11-16T16:49:44.152 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 1389849, 2023-11-16T16:49:44.152 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2023-11-16T16:49:44.152 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2023-11-16T16:49:44.152 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2023-11-16T16:49:44.153 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2023-11-16T16:49:44.153 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:44.153 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:44.153 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:49:44.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:43 smithi026 ceph-mon[100112]: pgmap v328: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:49:44.520 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-11-16T16:49:44.520 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-11-16T16:49:44.520 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2023-11-16T16:49:44.520 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2023-11-16T16:49:44.521 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2023-11-16T16:49:44.521 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2023-11-16T16:49:44.521 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2023-11-16T16:49:44.521 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2023-11-16T16:49:44.521 DEBUG:tasks.check_counter:Getting stats from g 2023-11-16T16:49:44.521 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph daemon mds.g perf dump 2023-11-16T16:49:45.174 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:49:46.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:45 smithi053 ceph-mon[100638]: pgmap v329: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:49:46.181 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:45 smithi001 ceph-mon[98800]: pgmap v329: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:49:46.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:45 smithi026 ceph-mon[100112]: pgmap v329: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:49:46.521 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:49:46.521 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-0": { 2023-11-16T16:49:46.521 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 0, 2023-11-16T16:49:46.521 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 2, 2023-11-16T16:49:46.522 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 0, 2023-11-16T16:49:46.522 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 0, 2023-11-16T16:49:46.522 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 4, 2023-11-16T16:49:46.522 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 0, 2023-11-16T16:49:46.522 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.001591255, 2023-11-16T16:49:46.522 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.000000000, 2023-11-16T16:49:46.522 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.000346613, 2023-11-16T16:49:46.523 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-16T16:49:46.523 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:49:46.523 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2023-11-16T16:49:46.523 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2023-11-16T16:49:46.523 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2023-11-16T16:49:46.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:46.523 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:49:46.523 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2023-11-16T16:49:46.524 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2023-11-16T16:49:46.524 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2023-11-16T16:49:46.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:46.524 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-16T16:49:46.524 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 0 2023-11-16T16:49:46.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:46.524 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-1": { 2023-11-16T16:49:46.524 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 215, 2023-11-16T16:49:46.525 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 205, 2023-11-16T16:49:46.525 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 1419808, 2023-11-16T16:49:46.525 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 116784, 2023-11-16T16:49:46.525 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 2, 2023-11-16T16:49:46.525 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 1, 2023-11-16T16:49:46.525 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.039979504, 2023-11-16T16:49:46.525 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.012524210, 2023-11-16T16:49:46.525 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.031140336, 2023-11-16T16:49:46.526 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.004795151, 2023-11-16T16:49:46.526 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:49:46.526 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 204, 2023-11-16T16:49:46.526 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.010146932, 2023-11-16T16:49:46.526 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000049739 2023-11-16T16:49:46.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:46.526 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:49:46.527 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2023-11-16T16:49:46.527 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2023-11-16T16:49:46.527 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2023-11-16T16:49:46.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:46.527 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 1419808, 2023-11-16T16:49:46.527 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 116784 2023-11-16T16:49:46.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:46.527 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-2": { 2023-11-16T16:49:46.527 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 6, 2023-11-16T16:49:46.528 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 159, 2023-11-16T16:49:46.528 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 3312, 2023-11-16T16:49:46.528 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 179776, 2023-11-16T16:49:46.528 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 5, 2023-11-16T16:49:46.528 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 3, 2023-11-16T16:49:46.528 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.012267200, 2023-11-16T16:49:46.528 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.007739321, 2023-11-16T16:49:46.528 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.002337296, 2023-11-16T16:49:46.529 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-16T16:49:46.529 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:49:46.529 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 157, 2023-11-16T16:49:46.529 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.013288894, 2023-11-16T16:49:46.529 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000084642 2023-11-16T16:49:46.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:46.529 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:49:46.530 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2023-11-16T16:49:46.530 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2023-11-16T16:49:46.530 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2023-11-16T16:49:46.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:46.530 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 3312, 2023-11-16T16:49:46.530 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 179776 2023-11-16T16:49:46.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:46.530 INFO:teuthology.orchestra.run.smithi001.stdout: "cct": { 2023-11-16T16:49:46.531 INFO:teuthology.orchestra.run.smithi001.stdout: "total_workers": 0, 2023-11-16T16:49:46.531 INFO:teuthology.orchestra.run.smithi001.stdout: "unhealthy_workers": 0 2023-11-16T16:49:46.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:46.531 INFO:teuthology.orchestra.run.smithi001.stdout: "mempool": { 2023-11-16T16:49:46.531 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_bytes": 0, 2023-11-16T16:49:46.531 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_items": 0, 2023-11-16T16:49:46.531 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_bytes": 0, 2023-11-16T16:49:46.531 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_items": 0, 2023-11-16T16:49:46.532 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_bytes": 0, 2023-11-16T16:49:46.532 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_items": 0, 2023-11-16T16:49:46.532 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-16T16:49:46.532 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_items": 0, 2023-11-16T16:49:46.532 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-16T16:49:46.532 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_items": 0, 2023-11-16T16:49:46.532 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_bytes": 0, 2023-11-16T16:49:46.532 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_items": 0, 2023-11-16T16:49:46.533 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-16T16:49:46.533 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_items": 0, 2023-11-16T16:49:46.533 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_bytes": 0, 2023-11-16T16:49:46.533 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_items": 0, 2023-11-16T16:49:46.533 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_bytes": 0, 2023-11-16T16:49:46.533 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_items": 0, 2023-11-16T16:49:46.533 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-16T16:49:46.533 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_items": 0, 2023-11-16T16:49:46.534 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-16T16:49:46.534 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_items": 0, 2023-11-16T16:49:46.534 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_bytes": 0, 2023-11-16T16:49:46.534 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_items": 0, 2023-11-16T16:49:46.534 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_bytes": 0, 2023-11-16T16:49:46.534 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_items": 0, 2023-11-16T16:49:46.534 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-16T16:49:46.534 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_items": 0, 2023-11-16T16:49:46.535 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_bytes": 0, 2023-11-16T16:49:46.535 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_items": 0, 2023-11-16T16:49:46.535 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_bytes": 0, 2023-11-16T16:49:46.535 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_items": 0, 2023-11-16T16:49:46.535 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_bytes": 0, 2023-11-16T16:49:46.535 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_items": 0, 2023-11-16T16:49:46.535 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_bytes": 0, 2023-11-16T16:49:46.535 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_items": 0, 2023-11-16T16:49:46.536 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_bytes": 149034, 2023-11-16T16:49:46.536 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_items": 32, 2023-11-16T16:49:46.536 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_bytes": 0, 2023-11-16T16:49:46.536 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_items": 0, 2023-11-16T16:49:46.536 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bytes": 0, 2023-11-16T16:49:46.536 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_items": 0, 2023-11-16T16:49:46.536 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_bytes": 0, 2023-11-16T16:49:46.536 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_items": 0, 2023-11-16T16:49:46.537 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_bytes": 0, 2023-11-16T16:49:46.537 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_items": 0, 2023-11-16T16:49:46.537 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_bytes": 0, 2023-11-16T16:49:46.537 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_items": 0, 2023-11-16T16:49:46.537 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_bytes": 0, 2023-11-16T16:49:46.537 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_items": 0, 2023-11-16T16:49:46.537 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_bytes": 0, 2023-11-16T16:49:46.538 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_items": 0, 2023-11-16T16:49:46.538 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_bytes": 928, 2023-11-16T16:49:46.538 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_items": 2, 2023-11-16T16:49:46.538 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_bytes": 0, 2023-11-16T16:49:46.538 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_items": 0, 2023-11-16T16:49:46.538 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_bytes": 0, 2023-11-16T16:49:46.538 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_items": 0 2023-11-16T16:49:46.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:46.539 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-16T16:49:46.539 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2023-11-16T16:49:46.539 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 104857600, 2023-11-16T16:49:46.539 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2023-11-16T16:49:46.539 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 217, 2023-11-16T16:49:46.539 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 1389459, 2023-11-16T16:49:46.539 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2023-11-16T16:49:46.539 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 217, 2023-11-16T16:49:46.540 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2023-11-16T16:49:46.540 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2023-11-16T16:49:46.540 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 217, 2023-11-16T16:49:46.540 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 1389459, 2023-11-16T16:49:46.540 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2023-11-16T16:49:46.540 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2023-11-16T16:49:46.540 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2023-11-16T16:49:46.540 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2023-11-16T16:49:46.541 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:46.541 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:46.541 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:49:46.936 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2023-11-16T16:49:46.937 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2023-11-16T16:49:46.937 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2023-11-16T16:49:46.937 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2023-11-16T16:49:46.937 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2023-11-16T16:49:46.937 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2023-11-16T16:49:46.937 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2023-11-16T16:49:46.937 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2023-11-16T16:49:46.937 DEBUG:tasks.check_counter:Getting stats from j 2023-11-16T16:49:46.938 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph daemon mds.j perf dump 2023-11-16T16:49:47.590 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.a/config 2023-11-16T16:49:48.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:47 smithi053 ceph-mon[100638]: pgmap v330: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:49:48.088 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:47 smithi001 ceph-mon[98800]: pgmap v330: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:49:48.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:47 smithi026 ceph-mon[100112]: pgmap v330: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:49:49.109 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2023-11-16T16:49:49.109 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-0": { 2023-11-16T16:49:49.109 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 4, 2023-11-16T16:49:49.109 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 4, 2023-11-16T16:49:49.109 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 2992, 2023-11-16T16:49:49.110 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 448, 2023-11-16T16:49:49.110 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 4, 2023-11-16T16:49:49.110 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 0, 2023-11-16T16:49:49.110 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.001689989, 2023-11-16T16:49:49.110 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.000041848, 2023-11-16T16:49:49.110 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.000637655, 2023-11-16T16:49:49.110 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-16T16:49:49.111 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:49:49.111 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 3, 2023-11-16T16:49:49.111 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000743039, 2023-11-16T16:49:49.111 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000247679 2023-11-16T16:49:49.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:49.111 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:49:49.111 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2023-11-16T16:49:49.111 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2023-11-16T16:49:49.112 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2023-11-16T16:49:49.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:49.112 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-11-16T16:49:49.112 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 448 2023-11-16T16:49:49.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:49.112 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-1": { 2023-11-16T16:49:49.112 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 220, 2023-11-16T16:49:49.112 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 210, 2023-11-16T16:49:49.113 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 1421168, 2023-11-16T16:49:49.113 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 119744, 2023-11-16T16:49:49.113 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 2, 2023-11-16T16:49:49.113 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 1, 2023-11-16T16:49:49.113 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.041061191, 2023-11-16T16:49:49.113 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.013073554, 2023-11-16T16:49:49.113 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.032114288, 2023-11-16T16:49:49.113 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.004902053, 2023-11-16T16:49:49.114 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:49:49.114 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 209, 2023-11-16T16:49:49.114 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.010296860, 2023-11-16T16:49:49.114 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000049267 2023-11-16T16:49:49.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:49.114 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:49:49.114 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2023-11-16T16:49:49.114 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2023-11-16T16:49:49.115 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2023-11-16T16:49:49.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:49.115 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 1421168, 2023-11-16T16:49:49.115 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 119744 2023-11-16T16:49:49.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:49.115 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-2": { 2023-11-16T16:49:49.115 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 2, 2023-11-16T16:49:49.115 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 160, 2023-11-16T16:49:49.116 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 320, 2023-11-16T16:49:49.116 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 179856, 2023-11-16T16:49:49.116 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 5, 2023-11-16T16:49:49.116 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 3, 2023-11-16T16:49:49.116 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.013080333, 2023-11-16T16:49:49.116 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.008890965, 2023-11-16T16:49:49.116 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.001931816, 2023-11-16T16:49:49.117 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-16T16:49:49.117 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:49:49.117 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 157, 2023-11-16T16:49:49.117 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.014965459, 2023-11-16T16:49:49.117 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000095321 2023-11-16T16:49:49.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:49.117 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:49:49.117 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2023-11-16T16:49:49.118 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2023-11-16T16:49:49.118 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2023-11-16T16:49:49.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:49.118 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 320, 2023-11-16T16:49:49.118 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 179856 2023-11-16T16:49:49.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:49.118 INFO:teuthology.orchestra.run.smithi001.stdout: "cct": { 2023-11-16T16:49:49.118 INFO:teuthology.orchestra.run.smithi001.stdout: "total_workers": 0, 2023-11-16T16:49:49.119 INFO:teuthology.orchestra.run.smithi001.stdout: "unhealthy_workers": 0 2023-11-16T16:49:49.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:49.119 INFO:teuthology.orchestra.run.smithi001.stdout: "mempool": { 2023-11-16T16:49:49.119 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_bytes": 0, 2023-11-16T16:49:49.119 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_items": 0, 2023-11-16T16:49:49.119 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_bytes": 0, 2023-11-16T16:49:49.119 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_items": 0, 2023-11-16T16:49:49.120 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_bytes": 0, 2023-11-16T16:49:49.120 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_items": 0, 2023-11-16T16:49:49.120 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-16T16:49:49.120 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_items": 0, 2023-11-16T16:49:49.120 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-16T16:49:49.120 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_items": 0, 2023-11-16T16:49:49.120 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_bytes": 0, 2023-11-16T16:49:49.120 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_items": 0, 2023-11-16T16:49:49.120 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-16T16:49:49.121 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_items": 0, 2023-11-16T16:49:49.121 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_bytes": 0, 2023-11-16T16:49:49.121 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_items": 0, 2023-11-16T16:49:49.121 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_bytes": 0, 2023-11-16T16:49:49.121 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_items": 0, 2023-11-16T16:49:49.121 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-16T16:49:49.121 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_items": 0, 2023-11-16T16:49:49.121 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-16T16:49:49.122 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_items": 0, 2023-11-16T16:49:49.122 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_bytes": 0, 2023-11-16T16:49:49.122 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_items": 0, 2023-11-16T16:49:49.122 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_bytes": 0, 2023-11-16T16:49:49.122 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_items": 0, 2023-11-16T16:49:49.122 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-16T16:49:49.122 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_items": 0, 2023-11-16T16:49:49.122 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_bytes": 0, 2023-11-16T16:49:49.123 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_items": 0, 2023-11-16T16:49:49.123 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_bytes": 0, 2023-11-16T16:49:49.123 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_items": 0, 2023-11-16T16:49:49.123 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_bytes": 0, 2023-11-16T16:49:49.123 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_items": 0, 2023-11-16T16:49:49.123 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_bytes": 0, 2023-11-16T16:49:49.123 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_items": 0, 2023-11-16T16:49:49.123 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_bytes": 149017, 2023-11-16T16:49:49.124 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_items": 31, 2023-11-16T16:49:49.124 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_bytes": 0, 2023-11-16T16:49:49.124 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_items": 0, 2023-11-16T16:49:49.124 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bytes": 0, 2023-11-16T16:49:49.124 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_items": 0, 2023-11-16T16:49:49.124 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_bytes": 0, 2023-11-16T16:49:49.124 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_items": 0, 2023-11-16T16:49:49.124 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_bytes": 0, 2023-11-16T16:49:49.125 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_items": 0, 2023-11-16T16:49:49.125 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_bytes": 0, 2023-11-16T16:49:49.125 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_items": 0, 2023-11-16T16:49:49.125 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_bytes": 0, 2023-11-16T16:49:49.125 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_items": 0, 2023-11-16T16:49:49.125 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_bytes": 0, 2023-11-16T16:49:49.125 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_items": 0, 2023-11-16T16:49:49.125 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_bytes": 928, 2023-11-16T16:49:49.126 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_items": 2, 2023-11-16T16:49:49.126 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_bytes": 0, 2023-11-16T16:49:49.126 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_items": 0, 2023-11-16T16:49:49.126 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_bytes": 0, 2023-11-16T16:49:49.126 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_items": 0 2023-11-16T16:49:49.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2023-11-16T16:49:49.126 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-16T16:49:49.126 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2023-11-16T16:49:49.127 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 104857600, 2023-11-16T16:49:49.127 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2023-11-16T16:49:49.127 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 222, 2023-11-16T16:49:49.127 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 1390109, 2023-11-16T16:49:49.127 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2023-11-16T16:49:49.127 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 222, 2023-11-16T16:49:49.127 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2023-11-16T16:49:49.127 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2023-11-16T16:49:49.128 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 222, 2023-11-16T16:49:49.128 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 1390109, 2023-11-16T16:49:49.128 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2023-11-16T16:49:49.128 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2023-11-16T16:49:49.128 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2023-11-16T16:49:49.128 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2023-11-16T16:49:49.128 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:49.129 INFO:teuthology.orchestra.run.smithi001.stdout: } 2023-11-16T16:49:49.129 INFO:teuthology.orchestra.run.smithi001.stdout:} 2023-11-16T16:49:49.546 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-11-16T16:49:49.546 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-11-16T16:49:49.546 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2023-11-16T16:49:49.546 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2023-11-16T16:49:49.546 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2023-11-16T16:49:49.546 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2023-11-16T16:49:49.546 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2023-11-16T16:49:49.547 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2023-11-16T16:49:49.547 DEBUG:tasks.check_counter:Getting stats from b 2023-11-16T16:49:49.547 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph daemon mds.b perf dump 2023-11-16T16:49:49.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:49 smithi001 ceph-mon[98800]: pgmap v331: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:49:50.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:49 smithi026 ceph-mon[100112]: pgmap v331: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:49:50.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:49 smithi053 ceph-mon[100638]: pgmap v331: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:49:50.222 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.b/config 2023-11-16T16:49:51.456 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-11-16T16:49:51.457 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-0": { 2023-11-16T16:49:51.457 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 96699, 2023-11-16T16:49:51.457 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 184409, 2023-11-16T16:49:51.457 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 33748723, 2023-11-16T16:49:51.457 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 190918957, 2023-11-16T16:49:51.457 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 122, 2023-11-16T16:49:51.458 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 7, 2023-11-16T16:49:51.458 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 5.667424066, 2023-11-16T16:49:51.458 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 2.849035392, 2023-11-16T16:49:51.458 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 62.113263164, 2023-11-16T16:49:51.458 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.394282092, 2023-11-16T16:49:51.458 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:49:51.458 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 184404, 2023-11-16T16:49:51.458 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 7.194150307, 2023-11-16T16:49:51.459 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000039012 2023-11-16T16:49:51.459 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.459 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:49:51.459 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 159231, 2023-11-16T16:49:51.459 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.037687654, 2023-11-16T16:49:51.459 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000236 2023-11-16T16:49:51.459 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.460 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 1594640, 2023-11-16T16:49:51.460 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 172736 2023-11-16T16:49:51.460 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.460 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-1": { 2023-11-16T16:49:51.460 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 120530, 2023-11-16T16:49:51.460 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 121797, 2023-11-16T16:49:51.460 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 48336630, 2023-11-16T16:49:51.460 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 81467007, 2023-11-16T16:49:51.461 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 11, 2023-11-16T16:49:51.461 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 5, 2023-11-16T16:49:51.461 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 7.461434676, 2023-11-16T16:49:51.461 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 3.113448917, 2023-11-16T16:49:51.461 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 4.663044039, 2023-11-16T16:49:51.461 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.543488349, 2023-11-16T16:49:51.461 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:49:51.461 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 121798, 2023-11-16T16:49:51.462 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 4.252723087, 2023-11-16T16:49:51.462 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000034916 2023-11-16T16:49:51.462 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.462 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:49:51.462 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 220206, 2023-11-16T16:49:51.462 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.032815944, 2023-11-16T16:49:51.462 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000149 2023-11-16T16:49:51.462 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.463 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 42528, 2023-11-16T16:49:51.463 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 438704 2023-11-16T16:49:51.463 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.463 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-2": { 2023-11-16T16:49:51.463 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 18807, 2023-11-16T16:49:51.463 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 20235, 2023-11-16T16:49:51.463 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 78262186, 2023-11-16T16:49:51.463 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 104241999, 2023-11-16T16:49:51.464 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 19, 2023-11-16T16:49:51.464 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 7, 2023-11-16T16:49:51.464 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 1.671491460, 2023-11-16T16:49:51.464 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.635766910, 2023-11-16T16:49:51.464 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.631832773, 2023-11-16T16:49:51.464 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.415692953, 2023-11-16T16:49:51.464 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:49:51.464 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 20234, 2023-11-16T16:49:51.465 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 3.777638442, 2023-11-16T16:49:51.465 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000186697 2023-11-16T16:49:51.465 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.465 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:49:51.465 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 18058, 2023-11-16T16:49:51.465 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.003791987, 2023-11-16T16:49:51.465 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000209 2023-11-16T16:49:51.465 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.466 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 311264, 2023-11-16T16:49:51.466 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 87936 2023-11-16T16:49:51.466 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.466 INFO:teuthology.orchestra.run.smithi026.stdout: "cct": { 2023-11-16T16:49:51.466 INFO:teuthology.orchestra.run.smithi026.stdout: "total_workers": 1, 2023-11-16T16:49:51.466 INFO:teuthology.orchestra.run.smithi026.stdout: "unhealthy_workers": 0 2023-11-16T16:49:51.466 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.466 INFO:teuthology.orchestra.run.smithi026.stdout: "finisher-MDSRank": { 2023-11-16T16:49:51.467 INFO:teuthology.orchestra.run.smithi026.stdout: "queue_len": 0, 2023-11-16T16:49:51.467 INFO:teuthology.orchestra.run.smithi026.stdout: "complete_latency": { 2023-11-16T16:49:51.467 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 5816, 2023-11-16T16:49:51.467 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 98.613855072, 2023-11-16T16:49:51.467 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.016955614 2023-11-16T16:49:51.467 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-11-16T16:49:51.467 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.467 INFO:teuthology.orchestra.run.smithi026.stdout: "finisher-PurgeQueue": { 2023-11-16T16:49:51.468 INFO:teuthology.orchestra.run.smithi026.stdout: "queue_len": 0, 2023-11-16T16:49:51.468 INFO:teuthology.orchestra.run.smithi026.stdout: "complete_latency": { 2023-11-16T16:49:51.468 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 21001, 2023-11-16T16:49:51.468 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 19.530886522, 2023-11-16T16:49:51.468 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000929997 2023-11-16T16:49:51.468 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-11-16T16:49:51.468 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.468 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": { 2023-11-16T16:49:51.469 INFO:teuthology.orchestra.run.smithi026.stdout: "request": 63294, 2023-11-16T16:49:51.469 INFO:teuthology.orchestra.run.smithi026.stdout: "reply": 55254, 2023-11-16T16:49:51.469 INFO:teuthology.orchestra.run.smithi026.stdout: "reply_latency": { 2023-11-16T16:49:51.469 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 55254, 2023-11-16T16:49:51.469 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 253.723937707, 2023-11-16T16:49:51.469 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.004591956 2023-11-16T16:49:51.469 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.469 INFO:teuthology.orchestra.run.smithi026.stdout: "slow_reply": 0, 2023-11-16T16:49:51.470 INFO:teuthology.orchestra.run.smithi026.stdout: "forward": 8040, 2023-11-16T16:49:51.470 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_fetch_complete": 840, 2023-11-16T16:49:51.470 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_fetch_keys": 0, 2023-11-16T16:49:51.470 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_commit": 1041, 2023-11-16T16:49:51.470 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_split": 128, 2023-11-16T16:49:51.470 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_merge": 0, 2023-11-16T16:49:51.470 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes": 53970, 2023-11-16T16:49:51.470 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_top": 3440, 2023-11-16T16:49:51.471 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_bottom": 26445, 2023-11-16T16:49:51.471 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_pin_tail": 24085, 2023-11-16T16:49:51.471 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_pinned": 49055, 2023-11-16T16:49:51.471 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_expired": 37559, 2023-11-16T16:49:51.471 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_with_caps": 5, 2023-11-16T16:49:51.471 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": 6, 2023-11-16T16:49:51.471 INFO:teuthology.orchestra.run.smithi026.stdout: "subtrees": 4, 2023-11-16T16:49:51.471 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse": 79552, 2023-11-16T16:49:51.472 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_hit": 71058, 2023-11-16T16:49:51.472 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_forward": 7925, 2023-11-16T16:49:51.472 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_discover": 0, 2023-11-16T16:49:51.472 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_dir_fetch": 0, 2023-11-16T16:49:51.472 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_remote_ino": 0, 2023-11-16T16:49:51.472 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_lock": 0, 2023-11-16T16:49:51.472 INFO:teuthology.orchestra.run.smithi026.stdout: "load_cent": 0, 2023-11-16T16:49:51.472 INFO:teuthology.orchestra.run.smithi026.stdout: "q": 0, 2023-11-16T16:49:51.473 INFO:teuthology.orchestra.run.smithi026.stdout: "exported": 475, 2023-11-16T16:49:51.473 INFO:teuthology.orchestra.run.smithi026.stdout: "exported_inodes": 34413, 2023-11-16T16:49:51.473 INFO:teuthology.orchestra.run.smithi026.stdout: "imported": 342, 2023-11-16T16:49:51.473 INFO:teuthology.orchestra.run.smithi026.stdout: "imported_inodes": 0, 2023-11-16T16:49:51.473 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_dir_fetch": 0, 2023-11-16T16:49:51.473 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_backtrace_fetch": 6330, 2023-11-16T16:49:51.473 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_peer_discover": 0, 2023-11-16T16:49:51.473 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rfiles": 4, 2023-11-16T16:49:51.474 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rbytes": 246, 2023-11-16T16:49:51.474 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rsnaps": 6, 2023-11-16T16:49:51.474 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_backtrace_fetch": 6330, 2023-11-16T16:49:51.474 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_set_tag": 0, 2023-11-16T16:49:51.474 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_backtrace_repaired": 0, 2023-11-16T16:49:51.474 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_inotable_repaired": 0, 2023-11-16T16:49:51.474 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dir_inodes": 132, 2023-11-16T16:49:51.474 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dir_base_inodes": 13, 2023-11-16T16:49:51.475 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dirfrag_rstats": 119, 2023-11-16T16:49:51.475 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_file_inodes": 6198, 2023-11-16T16:49:51.475 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_inode_file_caps": 4, 2023-11-16T16:49:51.475 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_revoke": 277, 2023-11-16T16:49:51.475 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_grant": 38049, 2023-11-16T16:49:51.475 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_trunc": 1, 2023-11-16T16:49:51.475 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_flushsnap_ack": 33, 2023-11-16T16:49:51.475 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-16T16:49:51.475 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_caps": 24293, 2023-11-16T16:49:51.476 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_caps_dirty": 24014, 2023-11-16T16:49:51.476 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_cap_release": 674, 2023-11-16T16:49:51.476 INFO:teuthology.orchestra.run.smithi026.stdout: "process_request_cap_release": 32630 2023-11-16T16:49:51.476 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.476 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_cache": { 2023-11-16T16:49:51.476 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_update": 1348, 2023-11-16T16:49:51.476 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_update_receipt": 506, 2023-11-16T16:49:51.476 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_try_discover": 253, 2023-11-16T16:49:51.477 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_send_discover": 285, 2023-11-16T16:49:51.477 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_handle_discover": 1520, 2023-11-16T16:49:51.477 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays": 8, 2023-11-16T16:49:51.477 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays_delayed": 0, 2023-11-16T16:49:51.477 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays_enqueuing": 0, 2023-11-16T16:49:51.477 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_created": 15703, 2023-11-16T16:49:51.477 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_enqueued": 15695, 2023-11-16T16:49:51.477 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_reintegrated": 0, 2023-11-16T16:49:51.478 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_migrated": 0, 2023-11-16T16:49:51.478 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_processing": 0, 2023-11-16T16:49:51.478 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_enqueued": 0, 2023-11-16T16:49:51.478 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_prioritized": 0, 2023-11-16T16:49:51.478 INFO:teuthology.orchestra.run.smithi026.stdout: "recovery_started": 0, 2023-11-16T16:49:51.478 INFO:teuthology.orchestra.run.smithi026.stdout: "recovery_completed": 0, 2023-11-16T16:49:51.478 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_enqueue_scrub": 13, 2023-11-16T16:49:51.478 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_exportdir": 478, 2023-11-16T16:49:51.479 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_flush": 0, 2023-11-16T16:49:51.479 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_fragmentdir": 131, 2023-11-16T16:49:51.479 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_fragstats": 0, 2023-11-16T16:49:51.479 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_inodestats": 0 2023-11-16T16:49:51.479 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.479 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_log": { 2023-11-16T16:49:51.479 INFO:teuthology.orchestra.run.smithi026.stdout: "evadd": 86355, 2023-11-16T16:49:51.479 INFO:teuthology.orchestra.run.smithi026.stdout: "evex": 0, 2023-11-16T16:49:51.480 INFO:teuthology.orchestra.run.smithi026.stdout: "evtrm": 0, 2023-11-16T16:49:51.480 INFO:teuthology.orchestra.run.smithi026.stdout: "ev": 86355, 2023-11-16T16:49:51.480 INFO:teuthology.orchestra.run.smithi026.stdout: "evexg": 0, 2023-11-16T16:49:51.480 INFO:teuthology.orchestra.run.smithi026.stdout: "evexd": 0, 2023-11-16T16:49:51.480 INFO:teuthology.orchestra.run.smithi026.stdout: "segadd": 118, 2023-11-16T16:49:51.480 INFO:teuthology.orchestra.run.smithi026.stdout: "segex": 0, 2023-11-16T16:49:51.480 INFO:teuthology.orchestra.run.smithi026.stdout: "segtrm": 0, 2023-11-16T16:49:51.480 INFO:teuthology.orchestra.run.smithi026.stdout: "seg": 118, 2023-11-16T16:49:51.481 INFO:teuthology.orchestra.run.smithi026.stdout: "segexg": 0, 2023-11-16T16:49:51.481 INFO:teuthology.orchestra.run.smithi026.stdout: "segexd": 0, 2023-11-16T16:49:51.481 INFO:teuthology.orchestra.run.smithi026.stdout: "expos": 4194304, 2023-11-16T16:49:51.481 INFO:teuthology.orchestra.run.smithi026.stdout: "wrpos": 276134106, 2023-11-16T16:49:51.481 INFO:teuthology.orchestra.run.smithi026.stdout: "rdpos": 0, 2023-11-16T16:49:51.481 INFO:teuthology.orchestra.run.smithi026.stdout: "jlat": { 2023-11-16T16:49:51.481 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 1979, 2023-11-16T16:49:51.481 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 190.671039074, 2023-11-16T16:49:51.481 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.096347164 2023-11-16T16:49:51.482 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.482 INFO:teuthology.orchestra.run.smithi026.stdout: "replayed": 0 2023-11-16T16:49:51.482 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.482 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_mem": { 2023-11-16T16:49:51.482 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 33959, 2023-11-16T16:49:51.482 INFO:teuthology.orchestra.run.smithi026.stdout: "ino+": 78124, 2023-11-16T16:49:51.482 INFO:teuthology.orchestra.run.smithi026.stdout: "ino-": 44165, 2023-11-16T16:49:51.483 INFO:teuthology.orchestra.run.smithi026.stdout: "dir": 1055, 2023-11-16T16:49:51.483 INFO:teuthology.orchestra.run.smithi026.stdout: "dir+": 3290, 2023-11-16T16:49:51.483 INFO:teuthology.orchestra.run.smithi026.stdout: "dir-": 2235, 2023-11-16T16:49:51.483 INFO:teuthology.orchestra.run.smithi026.stdout: "dn": 53970, 2023-11-16T16:49:51.483 INFO:teuthology.orchestra.run.smithi026.stdout: "dn+": 107231, 2023-11-16T16:49:51.483 INFO:teuthology.orchestra.run.smithi026.stdout: "dn-": 53261, 2023-11-16T16:49:51.483 INFO:teuthology.orchestra.run.smithi026.stdout: "cap": 6, 2023-11-16T16:49:51.483 INFO:teuthology.orchestra.run.smithi026.stdout: "cap+": 48605, 2023-11-16T16:49:51.483 INFO:teuthology.orchestra.run.smithi026.stdout: "cap-": 48599, 2023-11-16T16:49:51.484 INFO:teuthology.orchestra.run.smithi026.stdout: "rss": 297316, 2023-11-16T16:49:51.484 INFO:teuthology.orchestra.run.smithi026.stdout: "heap": 207156 2023-11-16T16:49:51.484 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.484 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_server": { 2023-11-16T16:49:51.484 INFO:teuthology.orchestra.run.smithi026.stdout: "dispatch_client_request": 64384, 2023-11-16T16:49:51.484 INFO:teuthology.orchestra.run.smithi026.stdout: "dispatch_server_request": 32, 2023-11-16T16:49:51.484 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_request": 63294, 2023-11-16T16:49:51.484 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_session": 37, 2023-11-16T16:49:51.485 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_peer_request": 141, 2023-11-16T16:49:51.485 INFO:teuthology.orchestra.run.smithi026.stdout: "req_create_latency": { 2023-11-16T16:49:51.485 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 22316, 2023-11-16T16:49:51.485 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 110.905644607, 2023-11-16T16:49:51.485 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.004969781 2023-11-16T16:49:51.485 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.485 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getattr_latency": { 2023-11-16T16:49:51.485 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 113, 2023-11-16T16:49:51.486 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.236144225, 2023-11-16T16:49:51.486 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.002089771 2023-11-16T16:49:51.486 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.486 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getfilelock_latency": { 2023-11-16T16:49:51.486 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:51.486 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:51.486 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:51.486 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.487 INFO:teuthology.orchestra.run.smithi026.stdout: "req_link_latency": { 2023-11-16T16:49:51.487 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:51.487 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:51.487 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:51.487 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.487 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookup_latency": { 2023-11-16T16:49:51.487 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 15663, 2023-11-16T16:49:51.487 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 41.245220781, 2023-11-16T16:49:51.488 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.002633289 2023-11-16T16:49:51.488 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.488 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookuphash_latency": { 2023-11-16T16:49:51.488 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:51.488 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:51.488 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:51.488 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.488 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupino_latency": { 2023-11-16T16:49:51.489 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 1, 2023-11-16T16:49:51.489 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000358922, 2023-11-16T16:49:51.489 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000358922 2023-11-16T16:49:51.489 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.489 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupname_latency": { 2023-11-16T16:49:51.489 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:51.489 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:51.489 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:51.490 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.490 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupparent_latency": { 2023-11-16T16:49:51.490 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:51.490 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:51.490 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:51.490 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.490 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupsnap_latency": { 2023-11-16T16:49:51.490 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:51.491 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:51.491 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:51.491 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.491 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lssnap_latency": { 2023-11-16T16:49:51.491 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 10, 2023-11-16T16:49:51.491 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.008618783, 2023-11-16T16:49:51.491 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000861878 2023-11-16T16:49:51.491 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.492 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mkdir_latency": { 2023-11-16T16:49:51.492 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 125, 2023-11-16T16:49:51.492 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.175503808, 2023-11-16T16:49:51.492 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.001404030 2023-11-16T16:49:51.492 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.492 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mknod_latency": { 2023-11-16T16:49:51.492 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:51.492 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:51.493 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:51.493 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.493 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mksnap_latency": { 2023-11-16T16:49:51.493 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 10, 2023-11-16T16:49:51.493 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.682308440, 2023-11-16T16:49:51.493 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.068230844 2023-11-16T16:49:51.493 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.493 INFO:teuthology.orchestra.run.smithi026.stdout: "req_open_latency": { 2023-11-16T16:49:51.494 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:51.494 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:51.494 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:51.494 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.494 INFO:teuthology.orchestra.run.smithi026.stdout: "req_readdir_latency": { 2023-11-16T16:49:51.494 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 1264, 2023-11-16T16:49:51.494 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 3.659123054, 2023-11-16T16:49:51.494 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.002894875 2023-11-16T16:49:51.495 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.495 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rename_latency": { 2023-11-16T16:49:51.495 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 2, 2023-11-16T16:49:51.495 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.003579278, 2023-11-16T16:49:51.495 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.001789639 2023-11-16T16:49:51.495 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.495 INFO:teuthology.orchestra.run.smithi026.stdout: "req_renamesnap_latency": { 2023-11-16T16:49:51.495 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:51.496 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:51.496 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:51.496 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.496 INFO:teuthology.orchestra.run.smithi026.stdout: "req_snapdiff_latency": { 2023-11-16T16:49:51.496 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:51.496 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:51.496 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:51.496 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.497 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmdir_latency": { 2023-11-16T16:49:51.497 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 113, 2023-11-16T16:49:51.497 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.597386919, 2023-11-16T16:49:51.497 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.005286609 2023-11-16T16:49:51.497 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.497 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmsnap_latency": { 2023-11-16T16:49:51.497 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 4, 2023-11-16T16:49:51.497 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.023442251, 2023-11-16T16:49:51.498 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.005860562 2023-11-16T16:49:51.498 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.498 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmxattr_latency": { 2023-11-16T16:49:51.498 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 3, 2023-11-16T16:49:51.498 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.003744868, 2023-11-16T16:49:51.498 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.001248289 2023-11-16T16:49:51.498 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.498 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setattr_latency": { 2023-11-16T16:49:51.499 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 3, 2023-11-16T16:49:51.499 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.016245566, 2023-11-16T16:49:51.499 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.005415188 2023-11-16T16:49:51.499 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.499 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setdirlayout_latency": { 2023-11-16T16:49:51.499 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:51.499 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:51.499 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:51.500 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.500 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setfilelock_latency": { 2023-11-16T16:49:51.500 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 22, 2023-11-16T16:49:51.500 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 10.023063537, 2023-11-16T16:49:51.500 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.455593797 2023-11-16T16:49:51.500 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.500 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setlayout_latency": { 2023-11-16T16:49:51.500 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:51.501 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:51.501 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:51.501 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.501 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setxattr_latency": { 2023-11-16T16:49:51.501 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 15, 2023-11-16T16:49:51.501 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.038160385, 2023-11-16T16:49:51.501 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.002544025 2023-11-16T16:49:51.501 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.502 INFO:teuthology.orchestra.run.smithi026.stdout: "req_symlink_latency": { 2023-11-16T16:49:51.502 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:51.502 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:51.502 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:51.502 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.502 INFO:teuthology.orchestra.run.smithi026.stdout: "req_unlink_latency": { 2023-11-16T16:49:51.502 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 15590, 2023-11-16T16:49:51.502 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 86.105392283, 2023-11-16T16:49:51.503 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.005523116 2023-11-16T16:49:51.503 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.503 INFO:teuthology.orchestra.run.smithi026.stdout: "cap_revoke_eviction": 0, 2023-11-16T16:49:51.503 INFO:teuthology.orchestra.run.smithi026.stdout: "cap_acquisition_throttle": 0, 2023-11-16T16:49:51.503 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getvxattr_latency": { 2023-11-16T16:49:51.503 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:51.503 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:51.503 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:51.504 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-11-16T16:49:51.504 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.504 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_sessions": { 2023-11-16T16:49:51.504 INFO:teuthology.orchestra.run.smithi026.stdout: "session_count": 2, 2023-11-16T16:49:51.504 INFO:teuthology.orchestra.run.smithi026.stdout: "session_add": 5, 2023-11-16T16:49:51.504 INFO:teuthology.orchestra.run.smithi026.stdout: "session_remove": 3, 2023-11-16T16:49:51.504 INFO:teuthology.orchestra.run.smithi026.stdout: "sessions_open": 2, 2023-11-16T16:49:51.505 INFO:teuthology.orchestra.run.smithi026.stdout: "sessions_stale": 0, 2023-11-16T16:49:51.505 INFO:teuthology.orchestra.run.smithi026.stdout: "total_load": 1797, 2023-11-16T16:49:51.505 INFO:teuthology.orchestra.run.smithi026.stdout: "average_load": 898, 2023-11-16T16:49:51.505 INFO:teuthology.orchestra.run.smithi026.stdout: "avg_session_uptime": 622 2023-11-16T16:49:51.505 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.505 INFO:teuthology.orchestra.run.smithi026.stdout: "mempool": { 2023-11-16T16:49:51.505 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_bytes": 0, 2023-11-16T16:49:51.505 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_items": 0, 2023-11-16T16:49:51.505 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_bytes": 0, 2023-11-16T16:49:51.506 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_items": 0, 2023-11-16T16:49:51.506 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_bytes": 0, 2023-11-16T16:49:51.506 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_items": 0, 2023-11-16T16:49:51.506 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-16T16:49:51.506 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_items": 0, 2023-11-16T16:49:51.506 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-16T16:49:51.506 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_items": 0, 2023-11-16T16:49:51.506 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_bytes": 0, 2023-11-16T16:49:51.507 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_items": 0, 2023-11-16T16:49:51.507 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-16T16:49:51.507 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_items": 0, 2023-11-16T16:49:51.507 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_bytes": 0, 2023-11-16T16:49:51.507 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_items": 0, 2023-11-16T16:49:51.507 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_bytes": 0, 2023-11-16T16:49:51.507 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_items": 0, 2023-11-16T16:49:51.507 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-16T16:49:51.507 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_items": 0, 2023-11-16T16:49:51.508 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-16T16:49:51.508 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_items": 0, 2023-11-16T16:49:51.508 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_bytes": 0, 2023-11-16T16:49:51.508 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_items": 0, 2023-11-16T16:49:51.508 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_bytes": 0, 2023-11-16T16:49:51.508 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_items": 0, 2023-11-16T16:49:51.508 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-16T16:49:51.508 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_items": 0, 2023-11-16T16:49:51.509 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_bytes": 0, 2023-11-16T16:49:51.509 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_items": 0, 2023-11-16T16:49:51.509 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_bytes": 0, 2023-11-16T16:49:51.509 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_items": 0, 2023-11-16T16:49:51.509 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_bytes": 0, 2023-11-16T16:49:51.509 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_items": 0, 2023-11-16T16:49:51.509 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_bytes": 0, 2023-11-16T16:49:51.509 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_items": 0, 2023-11-16T16:49:51.509 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_bytes": 1407222, 2023-11-16T16:49:51.510 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_items": 30375, 2023-11-16T16:49:51.510 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_bytes": 1144, 2023-11-16T16:49:51.510 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_items": 13, 2023-11-16T16:49:51.510 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_bytes": 0, 2023-11-16T16:49:51.510 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_items": 0, 2023-11-16T16:49:51.510 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_bytes": 0, 2023-11-16T16:49:51.510 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_items": 0, 2023-11-16T16:49:51.510 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_bytes": 0, 2023-11-16T16:49:51.511 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_items": 0, 2023-11-16T16:49:51.511 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_bytes": 8128, 2023-11-16T16:49:51.511 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_items": 169, 2023-11-16T16:49:51.511 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_bytes": 0, 2023-11-16T16:49:51.511 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_items": 0, 2023-11-16T16:49:51.511 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_bytes": 0, 2023-11-16T16:49:51.511 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_items": 0, 2023-11-16T16:49:51.511 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_bytes": 114754839, 2023-11-16T16:49:51.511 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_items": 1012232, 2023-11-16T16:49:51.512 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_bytes": 0, 2023-11-16T16:49:51.512 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_items": 0, 2023-11-16T16:49:51.512 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_bytes": 0, 2023-11-16T16:49:51.512 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_items": 0 2023-11-16T16:49:51.512 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.512 INFO:teuthology.orchestra.run.smithi026.stdout: "objecter": { 2023-11-16T16:49:51.512 INFO:teuthology.orchestra.run.smithi026.stdout: "op_active": 0, 2023-11-16T16:49:51.512 INFO:teuthology.orchestra.run.smithi026.stdout: "op_laggy": 0, 2023-11-16T16:49:51.513 INFO:teuthology.orchestra.run.smithi026.stdout: "op_send": 28321, 2023-11-16T16:49:51.513 INFO:teuthology.orchestra.run.smithi026.stdout: "op_send_bytes": 286954606, 2023-11-16T16:49:51.513 INFO:teuthology.orchestra.run.smithi026.stdout: "op_resend": 19, 2023-11-16T16:49:51.513 INFO:teuthology.orchestra.run.smithi026.stdout: "op_reply": 28321, 2023-11-16T16:49:51.513 INFO:teuthology.orchestra.run.smithi026.stdout: "op_latency": { 2023-11-16T16:49:51.513 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 28321, 2023-11-16T16:49:51.513 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 97.478510055, 2023-11-16T16:49:51.513 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.003441916 2023-11-16T16:49:51.514 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.514 INFO:teuthology.orchestra.run.smithi026.stdout: "op_inflight": 0, 2023-11-16T16:49:51.514 INFO:teuthology.orchestra.run.smithi026.stdout: "oplen_avg": { 2023-11-16T16:49:51.514 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 28321, 2023-11-16T16:49:51.514 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 30336 2023-11-16T16:49:51.514 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.514 INFO:teuthology.orchestra.run.smithi026.stdout: "op": 28321, 2023-11-16T16:49:51.514 INFO:teuthology.orchestra.run.smithi026.stdout: "op_r": 8381, 2023-11-16T16:49:51.515 INFO:teuthology.orchestra.run.smithi026.stdout: "op_w": 19940, 2023-11-16T16:49:51.515 INFO:teuthology.orchestra.run.smithi026.stdout: "op_rmw": 0, 2023-11-16T16:49:51.515 INFO:teuthology.orchestra.run.smithi026.stdout: "op_pg": 0, 2023-11-16T16:49:51.515 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_stat": 0, 2023-11-16T16:49:51.515 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_create": 11, 2023-11-16T16:49:51.515 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_read": 297, 2023-11-16T16:49:51.515 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_write": 2264, 2023-11-16T16:49:51.515 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_writefull": 143, 2023-11-16T16:49:51.516 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_writesame": 0, 2023-11-16T16:49:51.516 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_append": 0, 2023-11-16T16:49:51.516 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_zero": 0, 2023-11-16T16:49:51.516 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_truncate": 5, 2023-11-16T16:49:51.516 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_delete": 16394, 2023-11-16T16:49:51.516 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_mapext": 0, 2023-11-16T16:49:51.516 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_sparse_read": 0, 2023-11-16T16:49:51.516 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_clonerange": 0, 2023-11-16T16:49:51.517 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_getxattr": 6343, 2023-11-16T16:49:51.517 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_setxattr": 22, 2023-11-16T16:49:51.517 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_cmpxattr": 0, 2023-11-16T16:49:51.517 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_rmxattr": 0, 2023-11-16T16:49:51.517 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_resetxattrs": 0, 2023-11-16T16:49:51.517 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_call": 0, 2023-11-16T16:49:51.517 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_watch": 0, 2023-11-16T16:49:51.517 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_notify": 0, 2023-11-16T16:49:51.518 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_src_cmpxattr": 0, 2023-11-16T16:49:51.518 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_pgls": 0, 2023-11-16T16:49:51.518 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_pgls_filter": 0, 2023-11-16T16:49:51.518 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_other": 1, 2023-11-16T16:49:51.518 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_active": 0, 2023-11-16T16:49:51.518 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_send": 0, 2023-11-16T16:49:51.518 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_resend": 0, 2023-11-16T16:49:51.518 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_ping": 0, 2023-11-16T16:49:51.519 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_active": 0, 2023-11-16T16:49:51.519 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_send": 0, 2023-11-16T16:49:51.519 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_resend": 0, 2023-11-16T16:49:51.519 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_active": 0, 2023-11-16T16:49:51.519 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_send": 0, 2023-11-16T16:49:51.519 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_resend": 0, 2023-11-16T16:49:51.519 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_active": 0, 2023-11-16T16:49:51.519 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_send": 0, 2023-11-16T16:49:51.520 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_resend": 0, 2023-11-16T16:49:51.520 INFO:teuthology.orchestra.run.smithi026.stdout: "command_active": 0, 2023-11-16T16:49:51.520 INFO:teuthology.orchestra.run.smithi026.stdout: "command_send": 0, 2023-11-16T16:49:51.520 INFO:teuthology.orchestra.run.smithi026.stdout: "command_resend": 0, 2023-11-16T16:49:51.520 INFO:teuthology.orchestra.run.smithi026.stdout: "map_epoch": 95, 2023-11-16T16:49:51.520 INFO:teuthology.orchestra.run.smithi026.stdout: "map_full": 0, 2023-11-16T16:49:51.520 INFO:teuthology.orchestra.run.smithi026.stdout: "map_inc": 12, 2023-11-16T16:49:51.520 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_sessions": 12, 2023-11-16T16:49:51.521 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_session_open": 12, 2023-11-16T16:49:51.521 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_session_close": 0, 2023-11-16T16:49:51.521 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_laggy": 0, 2023-11-16T16:49:51.521 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_wr": 2218, 2023-11-16T16:49:51.521 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_rd": 2581, 2023-11-16T16:49:51.521 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_del": 57 2023-11-16T16:49:51.521 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.521 INFO:teuthology.orchestra.run.smithi026.stdout: "oft": { 2023-11-16T16:49:51.521 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_objs": 1, 2023-11-16T16:49:51.522 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_kv_pairs": 0, 2023-11-16T16:49:51.522 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_updates": 0, 2023-11-16T16:49:51.522 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_removes": 0 2023-11-16T16:49:51.522 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.522 INFO:teuthology.orchestra.run.smithi026.stdout: "osd-slow-ops": { 2023-11-16T16:49:51.522 INFO:teuthology.orchestra.run.smithi026.stdout: "slow_ops_count": 63963 2023-11-16T16:49:51.522 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.523 INFO:teuthology.orchestra.run.smithi026.stdout: "purge_queue": { 2023-11-16T16:49:51.523 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_ops": 0, 2023-11-16T16:49:51.523 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_ops_high_water": 10, 2023-11-16T16:49:51.523 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing": 0, 2023-11-16T16:49:51.523 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_high_water": 10, 2023-11-16T16:49:51.523 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executed": 15695, 2023-11-16T16:49:51.523 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_item_in_journal": 0 2023-11-16T16:49:51.523 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.523 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-16T16:49:51.524 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2023-11-16T16:49:51.524 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2023-11-16T16:49:51.524 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2023-11-16T16:49:51.524 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 236032, 2023-11-16T16:49:51.524 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 140458137, 2023-11-16T16:49:51.524 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2023-11-16T16:49:51.524 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 236032, 2023-11-16T16:49:51.524 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2023-11-16T16:49:51.525 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2023-11-16T16:49:51.525 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 236032, 2023-11-16T16:49:51.525 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 140458137, 2023-11-16T16:49:51.525 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2023-11-16T16:49:51.525 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:51.525 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:51.525 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:51.525 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-11-16T16:49:51.526 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.526 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-objecter_bytes": { 2023-11-16T16:49:51.526 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2023-11-16T16:49:51.526 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2023-11-16T16:49:51.526 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2023-11-16T16:49:51.526 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2023-11-16T16:49:51.526 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2023-11-16T16:49:51.526 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2023-11-16T16:49:51.527 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2023-11-16T16:49:51.527 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 28321, 2023-11-16T16:49:51.527 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 288789231, 2023-11-16T16:49:51.527 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 28321, 2023-11-16T16:49:51.527 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 288789231, 2023-11-16T16:49:51.527 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2023-11-16T16:49:51.527 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:51.527 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:51.528 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:51.528 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-11-16T16:49:51.528 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.528 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-objecter_ops": { 2023-11-16T16:49:51.528 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2023-11-16T16:49:51.528 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1024, 2023-11-16T16:49:51.528 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2023-11-16T16:49:51.529 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2023-11-16T16:49:51.529 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2023-11-16T16:49:51.529 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2023-11-16T16:49:51.529 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2023-11-16T16:49:51.529 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 28321, 2023-11-16T16:49:51.529 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 28321, 2023-11-16T16:49:51.529 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 28321, 2023-11-16T16:49:51.529 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 28321, 2023-11-16T16:49:51.530 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2023-11-16T16:49:51.530 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:51.530 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:51.530 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:51.530 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-11-16T16:49:51.530 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.530 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-write_buf_throttle": { 2023-11-16T16:49:51.530 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2023-11-16T16:49:51.531 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 3758096384, 2023-11-16T16:49:51.531 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2023-11-16T16:49:51.531 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 15695, 2023-11-16T16:49:51.531 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 1873595, 2023-11-16T16:49:51.531 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2023-11-16T16:49:51.531 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 15695, 2023-11-16T16:49:51.531 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2023-11-16T16:49:51.531 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2023-11-16T16:49:51.532 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 284, 2023-11-16T16:49:51.532 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 1873595, 2023-11-16T16:49:51.532 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2023-11-16T16:49:51.532 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:51.532 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:51.532 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:51.532 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-11-16T16:49:51.532 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:51.532 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-write_buf_throttle-0x558a5dc35cc0": { 2023-11-16T16:49:51.533 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2023-11-16T16:49:51.533 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 3758096384, 2023-11-16T16:49:51.533 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2023-11-16T16:49:51.533 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 86355, 2023-11-16T16:49:51.533 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 271939802, 2023-11-16T16:49:51.533 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2023-11-16T16:49:51.533 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 86355, 2023-11-16T16:49:51.533 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2023-11-16T16:49:51.534 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2023-11-16T16:49:51.534 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 1979, 2023-11-16T16:49:51.534 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 271939802, 2023-11-16T16:49:51.534 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2023-11-16T16:49:51.534 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:51.534 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:51.534 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:51.534 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-11-16T16:49:51.535 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-11-16T16:49:51.535 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-11-16T16:49:51.978 INFO:tasks.check_counter:Daemon mds.b mds.exported=475 2023-11-16T16:49:51.978 INFO:tasks.check_counter:Daemon mds.b mds.imported=342 2023-11-16T16:49:51.978 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=1348 2023-11-16T16:49:51.978 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=506 2023-11-16T16:49:51.979 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2023-11-16T16:49:51.979 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2023-11-16T16:49:51.979 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2023-11-16T16:49:51.979 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=128 2023-11-16T16:49:51.979 DEBUG:tasks.check_counter:Getting stats from e 2023-11-16T16:49:51.979 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph daemon mds.e perf dump 2023-11-16T16:49:52.630 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.b/config 2023-11-16T16:49:52.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:52 smithi001 ceph-mon[98800]: pgmap v332: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:49:52.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:52 smithi026 ceph-mon[100112]: pgmap v332: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:49:52.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:52 smithi053 ceph-mon[100638]: pgmap v332: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:49:53.917 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-11-16T16:49:53.918 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-0": { 2023-11-16T16:49:53.918 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 1933, 2023-11-16T16:49:53.918 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 1935, 2023-11-16T16:49:53.918 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 155909602, 2023-11-16T16:49:53.918 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 601170, 2023-11-16T16:49:53.918 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 8, 2023-11-16T16:49:53.918 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 4, 2023-11-16T16:49:53.919 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.247754569, 2023-11-16T16:49:53.919 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.050958452, 2023-11-16T16:49:53.919 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.118170622, 2023-11-16T16:49:53.919 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.062727495, 2023-11-16T16:49:53.919 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:49:53.919 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 1933, 2023-11-16T16:49:53.919 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.067192530, 2023-11-16T16:49:53.920 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000034760 2023-11-16T16:49:53.920 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.920 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:49:53.920 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.920 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.920 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.920 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.920 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-16T16:49:53.921 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 0 2023-11-16T16:49:53.921 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.921 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-1": { 2023-11-16T16:49:53.921 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 533, 2023-11-16T16:49:53.921 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 511, 2023-11-16T16:49:53.921 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 106448485, 2023-11-16T16:49:53.921 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 211035, 2023-11-16T16:49:53.921 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 6, 2023-11-16T16:49:53.922 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 5, 2023-11-16T16:49:53.922 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.110297845, 2023-11-16T16:49:53.922 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.018366571, 2023-11-16T16:49:53.922 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.085018024, 2023-11-16T16:49:53.922 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.013571998, 2023-11-16T16:49:53.922 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:49:53.922 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 511, 2023-11-16T16:49:53.922 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.022027448, 2023-11-16T16:49:53.923 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000043106 2023-11-16T16:49:53.923 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.923 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:49:53.923 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.923 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.923 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.923 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.923 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 1485456, 2023-11-16T16:49:53.924 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 122960 2023-11-16T16:49:53.924 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.924 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-2": { 2023-11-16T16:49:53.924 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 610, 2023-11-16T16:49:53.924 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 768, 2023-11-16T16:49:53.924 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 115256590, 2023-11-16T16:49:53.924 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 617792, 2023-11-16T16:49:53.924 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 8, 2023-11-16T16:49:53.925 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 6, 2023-11-16T16:49:53.925 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.110909401, 2023-11-16T16:49:53.925 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.018521493, 2023-11-16T16:49:53.925 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.068497596, 2023-11-16T16:49:53.925 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.017093891, 2023-11-16T16:49:53.925 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:49:53.925 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 765, 2023-11-16T16:49:53.925 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.032410679, 2023-11-16T16:49:53.926 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000042366 2023-11-16T16:49:53.926 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.926 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:49:53.926 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.926 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.926 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.926 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.926 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 320, 2023-11-16T16:49:53.927 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 428704 2023-11-16T16:49:53.927 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.927 INFO:teuthology.orchestra.run.smithi026.stdout: "cct": { 2023-11-16T16:49:53.927 INFO:teuthology.orchestra.run.smithi026.stdout: "total_workers": 1, 2023-11-16T16:49:53.927 INFO:teuthology.orchestra.run.smithi026.stdout: "unhealthy_workers": 0 2023-11-16T16:49:53.927 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.927 INFO:teuthology.orchestra.run.smithi026.stdout: "finisher-MDSRank": { 2023-11-16T16:49:53.927 INFO:teuthology.orchestra.run.smithi026.stdout: "queue_len": 0, 2023-11-16T16:49:53.928 INFO:teuthology.orchestra.run.smithi026.stdout: "complete_latency": { 2023-11-16T16:49:53.928 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 4395, 2023-11-16T16:49:53.928 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.128139722, 2023-11-16T16:49:53.928 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000029155 2023-11-16T16:49:53.928 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-11-16T16:49:53.928 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.928 INFO:teuthology.orchestra.run.smithi026.stdout: "finisher-PurgeQueue": { 2023-11-16T16:49:53.928 INFO:teuthology.orchestra.run.smithi026.stdout: "queue_len": 0, 2023-11-16T16:49:53.929 INFO:teuthology.orchestra.run.smithi026.stdout: "complete_latency": { 2023-11-16T16:49:53.929 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.929 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.929 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.929 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-11-16T16:49:53.929 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.929 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": { 2023-11-16T16:49:53.929 INFO:teuthology.orchestra.run.smithi026.stdout: "request": 0, 2023-11-16T16:49:53.930 INFO:teuthology.orchestra.run.smithi026.stdout: "reply": 0, 2023-11-16T16:49:53.930 INFO:teuthology.orchestra.run.smithi026.stdout: "reply_latency": { 2023-11-16T16:49:53.930 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.930 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.930 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.930 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.930 INFO:teuthology.orchestra.run.smithi026.stdout: "slow_reply": 0, 2023-11-16T16:49:53.930 INFO:teuthology.orchestra.run.smithi026.stdout: "forward": 0, 2023-11-16T16:49:53.931 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_fetch_complete": 0, 2023-11-16T16:49:53.931 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_fetch_keys": 0, 2023-11-16T16:49:53.931 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_commit": 0, 2023-11-16T16:49:53.931 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_split": 0, 2023-11-16T16:49:53.931 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_merge": 0, 2023-11-16T16:49:53.931 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes": 69174, 2023-11-16T16:49:53.931 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_top": 626, 2023-11-16T16:49:53.931 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_bottom": 44004, 2023-11-16T16:49:53.932 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_pin_tail": 24544, 2023-11-16T16:49:53.932 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_pinned": 68279, 2023-11-16T16:49:53.932 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_expired": 0, 2023-11-16T16:49:53.932 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_with_caps": 0, 2023-11-16T16:49:53.932 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": 0, 2023-11-16T16:49:53.932 INFO:teuthology.orchestra.run.smithi026.stdout: "subtrees": 1, 2023-11-16T16:49:53.933 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse": 0, 2023-11-16T16:49:53.933 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_hit": 0, 2023-11-16T16:49:53.933 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_forward": 0, 2023-11-16T16:49:53.933 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_discover": 0, 2023-11-16T16:49:53.933 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_dir_fetch": 0, 2023-11-16T16:49:53.933 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_remote_ino": 0, 2023-11-16T16:49:53.933 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_lock": 0, 2023-11-16T16:49:53.933 INFO:teuthology.orchestra.run.smithi026.stdout: "load_cent": 0, 2023-11-16T16:49:53.934 INFO:teuthology.orchestra.run.smithi026.stdout: "q": 0, 2023-11-16T16:49:53.934 INFO:teuthology.orchestra.run.smithi026.stdout: "exported": 0, 2023-11-16T16:49:53.934 INFO:teuthology.orchestra.run.smithi026.stdout: "exported_inodes": 0, 2023-11-16T16:49:53.934 INFO:teuthology.orchestra.run.smithi026.stdout: "imported": 0, 2023-11-16T16:49:53.934 INFO:teuthology.orchestra.run.smithi026.stdout: "imported_inodes": 0, 2023-11-16T16:49:53.934 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_dir_fetch": 0, 2023-11-16T16:49:53.934 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_backtrace_fetch": 0, 2023-11-16T16:49:53.934 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_peer_discover": 0, 2023-11-16T16:49:53.935 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rfiles": 0, 2023-11-16T16:49:53.935 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rbytes": 0, 2023-11-16T16:49:53.935 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rsnaps": 0, 2023-11-16T16:49:53.935 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_backtrace_fetch": 0, 2023-11-16T16:49:53.935 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_set_tag": 0, 2023-11-16T16:49:53.935 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_backtrace_repaired": 0, 2023-11-16T16:49:53.935 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_inotable_repaired": 0, 2023-11-16T16:49:53.936 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dir_inodes": 0, 2023-11-16T16:49:53.936 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dir_base_inodes": 0, 2023-11-16T16:49:53.936 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dirfrag_rstats": 0, 2023-11-16T16:49:53.936 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_file_inodes": 0, 2023-11-16T16:49:53.936 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_inode_file_caps": 0, 2023-11-16T16:49:53.936 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_revoke": 0, 2023-11-16T16:49:53.936 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_grant": 0, 2023-11-16T16:49:53.936 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_trunc": 0, 2023-11-16T16:49:53.937 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-16T16:49:53.937 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-16T16:49:53.937 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_caps": 0, 2023-11-16T16:49:53.937 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_caps_dirty": 0, 2023-11-16T16:49:53.937 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_cap_release": 0, 2023-11-16T16:49:53.937 INFO:teuthology.orchestra.run.smithi026.stdout: "process_request_cap_release": 0 2023-11-16T16:49:53.937 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.937 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_cache": { 2023-11-16T16:49:53.938 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_update": 0, 2023-11-16T16:49:53.938 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_update_receipt": 0, 2023-11-16T16:49:53.938 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_try_discover": 0, 2023-11-16T16:49:53.938 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_send_discover": 0, 2023-11-16T16:49:53.938 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_handle_discover": 0, 2023-11-16T16:49:53.938 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays": 0, 2023-11-16T16:49:53.938 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays_delayed": 0, 2023-11-16T16:49:53.938 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays_enqueuing": 0, 2023-11-16T16:49:53.939 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_created": 0, 2023-11-16T16:49:53.939 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_enqueued": 0, 2023-11-16T16:49:53.939 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_reintegrated": 0, 2023-11-16T16:49:53.939 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_migrated": 0, 2023-11-16T16:49:53.939 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_processing": 0, 2023-11-16T16:49:53.939 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_enqueued": 0, 2023-11-16T16:49:53.939 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_prioritized": 0, 2023-11-16T16:49:53.939 INFO:teuthology.orchestra.run.smithi026.stdout: "recovery_started": 0, 2023-11-16T16:49:53.940 INFO:teuthology.orchestra.run.smithi026.stdout: "recovery_completed": 0, 2023-11-16T16:49:53.940 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_enqueue_scrub": 0, 2023-11-16T16:49:53.940 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_exportdir": 0, 2023-11-16T16:49:53.940 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_flush": 0, 2023-11-16T16:49:53.940 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_fragmentdir": 0, 2023-11-16T16:49:53.940 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_fragstats": 0, 2023-11-16T16:49:53.940 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_inodestats": 0 2023-11-16T16:49:53.940 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.941 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_log": { 2023-11-16T16:49:53.941 INFO:teuthology.orchestra.run.smithi026.stdout: "evadd": 0, 2023-11-16T16:49:53.941 INFO:teuthology.orchestra.run.smithi026.stdout: "evex": 0, 2023-11-16T16:49:53.941 INFO:teuthology.orchestra.run.smithi026.stdout: "evtrm": 0, 2023-11-16T16:49:53.941 INFO:teuthology.orchestra.run.smithi026.stdout: "ev": 0, 2023-11-16T16:49:53.941 INFO:teuthology.orchestra.run.smithi026.stdout: "evexg": 0, 2023-11-16T16:49:53.941 INFO:teuthology.orchestra.run.smithi026.stdout: "evexd": 0, 2023-11-16T16:49:53.941 INFO:teuthology.orchestra.run.smithi026.stdout: "segadd": 0, 2023-11-16T16:49:53.942 INFO:teuthology.orchestra.run.smithi026.stdout: "segex": 0, 2023-11-16T16:49:53.942 INFO:teuthology.orchestra.run.smithi026.stdout: "segtrm": 0, 2023-11-16T16:49:53.942 INFO:teuthology.orchestra.run.smithi026.stdout: "seg": 130, 2023-11-16T16:49:53.942 INFO:teuthology.orchestra.run.smithi026.stdout: "segexg": 0, 2023-11-16T16:49:53.942 INFO:teuthology.orchestra.run.smithi026.stdout: "segexd": 0, 2023-11-16T16:49:53.942 INFO:teuthology.orchestra.run.smithi026.stdout: "expos": 62917679, 2023-11-16T16:49:53.942 INFO:teuthology.orchestra.run.smithi026.stdout: "wrpos": 0, 2023-11-16T16:49:53.942 INFO:teuthology.orchestra.run.smithi026.stdout: "rdpos": 379307106, 2023-11-16T16:49:53.943 INFO:teuthology.orchestra.run.smithi026.stdout: "jlat": { 2023-11-16T16:49:53.943 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.943 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.943 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.943 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.943 INFO:teuthology.orchestra.run.smithi026.stdout: "replayed": 105041 2023-11-16T16:49:53.943 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.943 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_mem": { 2023-11-16T16:49:53.944 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 30788, 2023-11-16T16:49:53.944 INFO:teuthology.orchestra.run.smithi026.stdout: "ino+": 70983, 2023-11-16T16:49:53.944 INFO:teuthology.orchestra.run.smithi026.stdout: "ino-": 40195, 2023-11-16T16:49:53.944 INFO:teuthology.orchestra.run.smithi026.stdout: "dir": 1035, 2023-11-16T16:49:53.944 INFO:teuthology.orchestra.run.smithi026.stdout: "dir+": 1835, 2023-11-16T16:49:53.944 INFO:teuthology.orchestra.run.smithi026.stdout: "dir-": 800, 2023-11-16T16:49:53.944 INFO:teuthology.orchestra.run.smithi026.stdout: "dn": 69174, 2023-11-16T16:49:53.944 INFO:teuthology.orchestra.run.smithi026.stdout: "dn+": 94373, 2023-11-16T16:49:53.945 INFO:teuthology.orchestra.run.smithi026.stdout: "dn-": 25199, 2023-11-16T16:49:53.945 INFO:teuthology.orchestra.run.smithi026.stdout: "cap": 0, 2023-11-16T16:49:53.945 INFO:teuthology.orchestra.run.smithi026.stdout: "cap+": 0, 2023-11-16T16:49:53.945 INFO:teuthology.orchestra.run.smithi026.stdout: "cap-": 0, 2023-11-16T16:49:53.945 INFO:teuthology.orchestra.run.smithi026.stdout: "rss": 201804, 2023-11-16T16:49:53.945 INFO:teuthology.orchestra.run.smithi026.stdout: "heap": 207156 2023-11-16T16:49:53.945 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.945 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_server": { 2023-11-16T16:49:53.946 INFO:teuthology.orchestra.run.smithi026.stdout: "dispatch_client_request": 0, 2023-11-16T16:49:53.946 INFO:teuthology.orchestra.run.smithi026.stdout: "dispatch_server_request": 0, 2023-11-16T16:49:53.946 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_request": 0, 2023-11-16T16:49:53.946 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_session": 0, 2023-11-16T16:49:53.946 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_peer_request": 0, 2023-11-16T16:49:53.946 INFO:teuthology.orchestra.run.smithi026.stdout: "req_create_latency": { 2023-11-16T16:49:53.946 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.946 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.947 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.947 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.947 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getattr_latency": { 2023-11-16T16:49:53.947 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.947 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.947 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.947 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.947 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getfilelock_latency": { 2023-11-16T16:49:53.948 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.948 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.948 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.948 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.948 INFO:teuthology.orchestra.run.smithi026.stdout: "req_link_latency": { 2023-11-16T16:49:53.948 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.948 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.948 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.949 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.949 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookup_latency": { 2023-11-16T16:49:53.949 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.949 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.949 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.949 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.949 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookuphash_latency": { 2023-11-16T16:49:53.949 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.949 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.950 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.950 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.950 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupino_latency": { 2023-11-16T16:49:53.950 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.950 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.950 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.950 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.951 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupname_latency": { 2023-11-16T16:49:53.951 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.951 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.951 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.951 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.951 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupparent_latency": { 2023-11-16T16:49:53.951 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.951 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.952 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.952 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.952 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupsnap_latency": { 2023-11-16T16:49:53.952 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.952 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.952 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.952 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.952 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lssnap_latency": { 2023-11-16T16:49:53.952 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.953 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.953 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.953 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.953 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mkdir_latency": { 2023-11-16T16:49:53.953 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.953 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.953 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.953 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.954 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mknod_latency": { 2023-11-16T16:49:53.954 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.954 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.954 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.954 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.954 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mksnap_latency": { 2023-11-16T16:49:53.954 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.954 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.955 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.955 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.955 INFO:teuthology.orchestra.run.smithi026.stdout: "req_open_latency": { 2023-11-16T16:49:53.955 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.955 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.955 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.955 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.955 INFO:teuthology.orchestra.run.smithi026.stdout: "req_readdir_latency": { 2023-11-16T16:49:53.956 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.956 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.956 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.956 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.956 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rename_latency": { 2023-11-16T16:49:53.956 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.956 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.956 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.957 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.957 INFO:teuthology.orchestra.run.smithi026.stdout: "req_renamesnap_latency": { 2023-11-16T16:49:53.957 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.957 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.957 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.957 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.957 INFO:teuthology.orchestra.run.smithi026.stdout: "req_snapdiff_latency": { 2023-11-16T16:49:53.957 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.958 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.958 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.958 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.958 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmdir_latency": { 2023-11-16T16:49:53.958 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.958 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.958 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.958 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.959 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmsnap_latency": { 2023-11-16T16:49:53.959 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.959 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.959 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.959 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.959 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmxattr_latency": { 2023-11-16T16:49:53.959 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.959 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.960 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.960 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.960 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setattr_latency": { 2023-11-16T16:49:53.960 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.960 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.960 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.960 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.960 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setdirlayout_latency": { 2023-11-16T16:49:53.961 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.961 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.961 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.961 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.961 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setfilelock_latency": { 2023-11-16T16:49:53.961 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.961 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.961 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.962 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.962 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setlayout_latency": { 2023-11-16T16:49:53.962 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.962 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.962 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.962 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.962 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setxattr_latency": { 2023-11-16T16:49:53.962 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.963 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.963 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.963 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.963 INFO:teuthology.orchestra.run.smithi026.stdout: "req_symlink_latency": { 2023-11-16T16:49:53.963 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.963 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.963 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.963 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.964 INFO:teuthology.orchestra.run.smithi026.stdout: "req_unlink_latency": { 2023-11-16T16:49:53.964 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.964 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.964 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.964 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.964 INFO:teuthology.orchestra.run.smithi026.stdout: "cap_revoke_eviction": 0, 2023-11-16T16:49:53.964 INFO:teuthology.orchestra.run.smithi026.stdout: "cap_acquisition_throttle": 0, 2023-11-16T16:49:53.964 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getvxattr_latency": { 2023-11-16T16:49:53.964 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.965 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.965 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.965 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-11-16T16:49:53.965 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.965 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_sessions": { 2023-11-16T16:49:53.965 INFO:teuthology.orchestra.run.smithi026.stdout: "session_count": 1, 2023-11-16T16:49:53.965 INFO:teuthology.orchestra.run.smithi026.stdout: "session_add": 3, 2023-11-16T16:49:53.965 INFO:teuthology.orchestra.run.smithi026.stdout: "session_remove": 2, 2023-11-16T16:49:53.966 INFO:teuthology.orchestra.run.smithi026.stdout: "sessions_open": 2, 2023-11-16T16:49:53.966 INFO:teuthology.orchestra.run.smithi026.stdout: "sessions_stale": 0, 2023-11-16T16:49:53.966 INFO:teuthology.orchestra.run.smithi026.stdout: "total_load": 0, 2023-11-16T16:49:53.966 INFO:teuthology.orchestra.run.smithi026.stdout: "average_load": 0, 2023-11-16T16:49:53.966 INFO:teuthology.orchestra.run.smithi026.stdout: "avg_session_uptime": 3641 2023-11-16T16:49:53.966 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.966 INFO:teuthology.orchestra.run.smithi026.stdout: "mempool": { 2023-11-16T16:49:53.967 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_bytes": 0, 2023-11-16T16:49:53.967 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_items": 0, 2023-11-16T16:49:53.967 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_bytes": 0, 2023-11-16T16:49:53.967 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_items": 0, 2023-11-16T16:49:53.967 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_bytes": 0, 2023-11-16T16:49:53.967 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_items": 0, 2023-11-16T16:49:53.967 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-16T16:49:53.967 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_items": 0, 2023-11-16T16:49:53.968 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-16T16:49:53.968 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_items": 0, 2023-11-16T16:49:53.968 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_bytes": 0, 2023-11-16T16:49:53.968 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_items": 0, 2023-11-16T16:49:53.968 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-16T16:49:53.968 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_items": 0, 2023-11-16T16:49:53.968 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_bytes": 0, 2023-11-16T16:49:53.968 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_items": 0, 2023-11-16T16:49:53.969 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_bytes": 0, 2023-11-16T16:49:53.969 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_items": 0, 2023-11-16T16:49:53.969 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-16T16:49:53.969 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_items": 0, 2023-11-16T16:49:53.969 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-16T16:49:53.969 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_items": 0, 2023-11-16T16:49:53.969 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_bytes": 0, 2023-11-16T16:49:53.969 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_items": 0, 2023-11-16T16:49:53.970 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_bytes": 0, 2023-11-16T16:49:53.970 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_items": 0, 2023-11-16T16:49:53.970 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-16T16:49:53.970 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_items": 0, 2023-11-16T16:49:53.970 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_bytes": 0, 2023-11-16T16:49:53.970 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_items": 0, 2023-11-16T16:49:53.970 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_bytes": 0, 2023-11-16T16:49:53.970 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_items": 0, 2023-11-16T16:49:53.971 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_bytes": 0, 2023-11-16T16:49:53.971 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_items": 0, 2023-11-16T16:49:53.971 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_bytes": 0, 2023-11-16T16:49:53.971 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_items": 0, 2023-11-16T16:49:53.971 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_bytes": 1285319, 2023-11-16T16:49:53.971 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_items": 30861, 2023-11-16T16:49:53.971 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_bytes": 0, 2023-11-16T16:49:53.971 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_items": 0, 2023-11-16T16:49:53.972 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_bytes": 0, 2023-11-16T16:49:53.972 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_items": 0, 2023-11-16T16:49:53.972 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_bytes": 0, 2023-11-16T16:49:53.972 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_items": 0, 2023-11-16T16:49:53.972 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_bytes": 0, 2023-11-16T16:49:53.972 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_items": 0, 2023-11-16T16:49:53.972 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_bytes": 8128, 2023-11-16T16:49:53.972 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_items": 169, 2023-11-16T16:49:53.973 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_bytes": 0, 2023-11-16T16:49:53.973 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_items": 0, 2023-11-16T16:49:53.973 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_bytes": 0, 2023-11-16T16:49:53.973 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_items": 0, 2023-11-16T16:49:53.973 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_bytes": 118151610, 2023-11-16T16:49:53.973 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_items": 952573, 2023-11-16T16:49:53.973 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_bytes": 0, 2023-11-16T16:49:53.973 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_items": 0, 2023-11-16T16:49:53.974 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_bytes": 0, 2023-11-16T16:49:53.974 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_items": 0 2023-11-16T16:49:53.974 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.974 INFO:teuthology.orchestra.run.smithi026.stdout: "objecter": { 2023-11-16T16:49:53.974 INFO:teuthology.orchestra.run.smithi026.stdout: "op_active": 0, 2023-11-16T16:49:53.974 INFO:teuthology.orchestra.run.smithi026.stdout: "op_laggy": 0, 2023-11-16T16:49:53.974 INFO:teuthology.orchestra.run.smithi026.stdout: "op_send": 2824, 2023-11-16T16:49:53.974 INFO:teuthology.orchestra.run.smithi026.stdout: "op_send_bytes": 21, 2023-11-16T16:49:53.975 INFO:teuthology.orchestra.run.smithi026.stdout: "op_resend": 3, 2023-11-16T16:49:53.975 INFO:teuthology.orchestra.run.smithi026.stdout: "op_reply": 2824, 2023-11-16T16:49:53.975 INFO:teuthology.orchestra.run.smithi026.stdout: "op_latency": { 2023-11-16T16:49:53.975 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 2824, 2023-11-16T16:49:53.975 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 1.994969459, 2023-11-16T16:49:53.975 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000706433 2023-11-16T16:49:53.975 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.975 INFO:teuthology.orchestra.run.smithi026.stdout: "op_inflight": 0, 2023-11-16T16:49:53.976 INFO:teuthology.orchestra.run.smithi026.stdout: "oplen_avg": { 2023-11-16T16:49:53.976 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 2824, 2023-11-16T16:49:53.976 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 2825 2023-11-16T16:49:53.976 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.976 INFO:teuthology.orchestra.run.smithi026.stdout: "op": 2824, 2023-11-16T16:49:53.976 INFO:teuthology.orchestra.run.smithi026.stdout: "op_r": 2824, 2023-11-16T16:49:53.976 INFO:teuthology.orchestra.run.smithi026.stdout: "op_w": 0, 2023-11-16T16:49:53.976 INFO:teuthology.orchestra.run.smithi026.stdout: "op_rmw": 0, 2023-11-16T16:49:53.977 INFO:teuthology.orchestra.run.smithi026.stdout: "op_pg": 0, 2023-11-16T16:49:53.977 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_stat": 1809, 2023-11-16T16:49:53.977 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_create": 0, 2023-11-16T16:49:53.977 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_read": 1013, 2023-11-16T16:49:53.977 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_write": 0, 2023-11-16T16:49:53.977 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_writefull": 0, 2023-11-16T16:49:53.977 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_writesame": 0, 2023-11-16T16:49:53.977 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_append": 0, 2023-11-16T16:49:53.977 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_zero": 0, 2023-11-16T16:49:53.978 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_truncate": 0, 2023-11-16T16:49:53.978 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_delete": 0, 2023-11-16T16:49:53.978 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_mapext": 0, 2023-11-16T16:49:53.978 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_sparse_read": 0, 2023-11-16T16:49:53.978 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_clonerange": 0, 2023-11-16T16:49:53.978 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_getxattr": 1, 2023-11-16T16:49:53.978 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_setxattr": 0, 2023-11-16T16:49:53.979 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_cmpxattr": 0, 2023-11-16T16:49:53.979 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_rmxattr": 0, 2023-11-16T16:49:53.979 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_resetxattrs": 0, 2023-11-16T16:49:53.979 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_call": 0, 2023-11-16T16:49:53.979 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_watch": 0, 2023-11-16T16:49:53.979 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_notify": 0, 2023-11-16T16:49:53.979 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_src_cmpxattr": 0, 2023-11-16T16:49:53.979 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_pgls": 0, 2023-11-16T16:49:53.980 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_pgls_filter": 0, 2023-11-16T16:49:53.980 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_other": 0, 2023-11-16T16:49:53.980 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_active": 0, 2023-11-16T16:49:53.980 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_send": 0, 2023-11-16T16:49:53.980 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_resend": 0, 2023-11-16T16:49:53.980 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_ping": 0, 2023-11-16T16:49:53.980 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_active": 0, 2023-11-16T16:49:53.980 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_send": 0, 2023-11-16T16:49:53.981 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_resend": 0, 2023-11-16T16:49:53.981 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_active": 0, 2023-11-16T16:49:53.981 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_send": 0, 2023-11-16T16:49:53.981 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_resend": 0, 2023-11-16T16:49:53.981 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_active": 0, 2023-11-16T16:49:53.981 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_send": 0, 2023-11-16T16:49:53.981 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_resend": 0, 2023-11-16T16:49:53.981 INFO:teuthology.orchestra.run.smithi026.stdout: "command_active": 0, 2023-11-16T16:49:53.982 INFO:teuthology.orchestra.run.smithi026.stdout: "command_send": 0, 2023-11-16T16:49:53.982 INFO:teuthology.orchestra.run.smithi026.stdout: "command_resend": 0, 2023-11-16T16:49:53.982 INFO:teuthology.orchestra.run.smithi026.stdout: "map_epoch": 95, 2023-11-16T16:49:53.982 INFO:teuthology.orchestra.run.smithi026.stdout: "map_full": 0, 2023-11-16T16:49:53.982 INFO:teuthology.orchestra.run.smithi026.stdout: "map_inc": 11, 2023-11-16T16:49:53.982 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_sessions": 11, 2023-11-16T16:49:53.982 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_session_open": 11, 2023-11-16T16:49:53.982 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_session_close": 0, 2023-11-16T16:49:53.983 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_laggy": 0, 2023-11-16T16:49:53.983 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_wr": 0, 2023-11-16T16:49:53.983 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_rd": 2, 2023-11-16T16:49:53.983 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_del": 0 2023-11-16T16:49:53.983 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.983 INFO:teuthology.orchestra.run.smithi026.stdout: "oft": { 2023-11-16T16:49:53.983 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_objs": 0, 2023-11-16T16:49:53.983 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_kv_pairs": 0, 2023-11-16T16:49:53.984 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_updates": 0, 2023-11-16T16:49:53.984 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_removes": 0 2023-11-16T16:49:53.984 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.984 INFO:teuthology.orchestra.run.smithi026.stdout: "osd-slow-ops": { 2023-11-16T16:49:53.984 INFO:teuthology.orchestra.run.smithi026.stdout: "slow_ops_count": 0 2023-11-16T16:49:53.984 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.984 INFO:teuthology.orchestra.run.smithi026.stdout: "purge_queue": { 2023-11-16T16:49:53.984 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_ops": 0, 2023-11-16T16:49:53.985 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_ops_high_water": 0, 2023-11-16T16:49:53.985 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing": 0, 2023-11-16T16:49:53.985 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_high_water": 0, 2023-11-16T16:49:53.985 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executed": 0, 2023-11-16T16:49:53.985 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_item_in_journal": 0 2023-11-16T16:49:53.985 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.985 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-16T16:49:53.985 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2023-11-16T16:49:53.986 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2023-11-16T16:49:53.986 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2023-11-16T16:49:53.986 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 3072, 2023-11-16T16:49:53.986 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 377322585, 2023-11-16T16:49:53.986 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2023-11-16T16:49:53.986 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 3072, 2023-11-16T16:49:53.986 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2023-11-16T16:49:53.986 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2023-11-16T16:49:53.987 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 3072, 2023-11-16T16:49:53.987 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 377322585, 2023-11-16T16:49:53.987 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2023-11-16T16:49:53.987 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.987 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.987 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.987 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-11-16T16:49:53.987 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.988 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-objecter_bytes": { 2023-11-16T16:49:53.988 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2023-11-16T16:49:53.988 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2023-11-16T16:49:53.988 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2023-11-16T16:49:53.988 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2023-11-16T16:49:53.988 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2023-11-16T16:49:53.988 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2023-11-16T16:49:53.988 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2023-11-16T16:49:53.989 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 2824, 2023-11-16T16:49:53.989 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 375347201, 2023-11-16T16:49:53.989 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 2824, 2023-11-16T16:49:53.989 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 375347201, 2023-11-16T16:49:53.989 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2023-11-16T16:49:53.989 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.989 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.990 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.990 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-11-16T16:49:53.990 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.990 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-objecter_ops": { 2023-11-16T16:49:53.990 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2023-11-16T16:49:53.990 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1024, 2023-11-16T16:49:53.990 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2023-11-16T16:49:53.990 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2023-11-16T16:49:53.991 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2023-11-16T16:49:53.991 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2023-11-16T16:49:53.991 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2023-11-16T16:49:53.991 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 2824, 2023-11-16T16:49:53.991 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 2824, 2023-11-16T16:49:53.991 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 2824, 2023-11-16T16:49:53.991 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 2824, 2023-11-16T16:49:53.991 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2023-11-16T16:49:53.992 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.992 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.992 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.992 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-11-16T16:49:53.992 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.992 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-write_buf_throttle": { 2023-11-16T16:49:53.992 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2023-11-16T16:49:53.992 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 3758096384, 2023-11-16T16:49:53.993 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2023-11-16T16:49:53.993 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2023-11-16T16:49:53.993 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2023-11-16T16:49:53.993 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2023-11-16T16:49:53.993 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2023-11-16T16:49:53.993 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2023-11-16T16:49:53.993 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2023-11-16T16:49:53.993 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 0, 2023-11-16T16:49:53.994 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 0, 2023-11-16T16:49:53.994 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2023-11-16T16:49:53.994 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.994 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.994 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.994 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-11-16T16:49:53.994 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:53.994 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-write_buf_throttle-0x561531f38000": { 2023-11-16T16:49:53.995 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2023-11-16T16:49:53.995 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 3758096384, 2023-11-16T16:49:53.995 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2023-11-16T16:49:53.995 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2023-11-16T16:49:53.995 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2023-11-16T16:49:53.995 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2023-11-16T16:49:53.995 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2023-11-16T16:49:53.995 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2023-11-16T16:49:53.996 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2023-11-16T16:49:53.996 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 0, 2023-11-16T16:49:53.996 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 0, 2023-11-16T16:49:53.996 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2023-11-16T16:49:53.996 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:53.996 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:53.996 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:53.996 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-11-16T16:49:53.996 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-11-16T16:49:53.997 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-11-16T16:49:54.007 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:53 smithi026 ceph-mon[100112]: pgmap v333: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:49:54.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:53 smithi053 ceph-mon[100638]: pgmap v333: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:49:54.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:53 smithi001 ceph-mon[98800]: pgmap v333: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:49:54.277 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2023-11-16T16:49:54.277 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2023-11-16T16:49:54.277 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2023-11-16T16:49:54.277 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2023-11-16T16:49:54.278 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2023-11-16T16:49:54.278 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2023-11-16T16:49:54.278 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2023-11-16T16:49:54.278 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2023-11-16T16:49:54.278 DEBUG:tasks.check_counter:Getting stats from h 2023-11-16T16:49:54.278 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph daemon mds.h perf dump 2023-11-16T16:49:54.929 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.b/config 2023-11-16T16:49:56.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:55 smithi053 ceph-mon[100638]: pgmap v334: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:49:56.095 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:55 smithi026 ceph-mon[100112]: pgmap v334: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:49:56.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:55 smithi001 ceph-mon[98800]: pgmap v334: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:49:56.339 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-11-16T16:49:56.339 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-0": { 2023-11-16T16:49:56.340 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 77761, 2023-11-16T16:49:56.340 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 130210, 2023-11-16T16:49:56.340 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 25694326, 2023-11-16T16:49:56.340 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 131662016, 2023-11-16T16:49:56.340 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 35, 2023-11-16T16:49:56.340 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 4, 2023-11-16T16:49:56.340 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 4.174444484, 2023-11-16T16:49:56.340 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 2.134465104, 2023-11-16T16:49:56.341 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 5.168579530, 2023-11-16T16:49:56.341 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.303218686, 2023-11-16T16:49:56.341 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:49:56.341 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 130208, 2023-11-16T16:49:56.341 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 5.352176046, 2023-11-16T16:49:56.341 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000041104 2023-11-16T16:49:56.341 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.342 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:49:56.342 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 104141, 2023-11-16T16:49:56.342 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.025967066, 2023-11-16T16:49:56.342 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000249 2023-11-16T16:49:56.342 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.342 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 912, 2023-11-16T16:49:56.342 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 928 2023-11-16T16:49:56.342 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.343 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-1": { 2023-11-16T16:49:56.343 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 29624, 2023-11-16T16:49:56.343 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 34262, 2023-11-16T16:49:56.343 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 83057278, 2023-11-16T16:49:56.343 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 109721370, 2023-11-16T16:49:56.343 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 7, 2023-11-16T16:49:56.343 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 6, 2023-11-16T16:49:56.343 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 2.109913722, 2023-11-16T16:49:56.344 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.872370999, 2023-11-16T16:49:56.344 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 1.069492223, 2023-11-16T16:49:56.344 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.295444847, 2023-11-16T16:49:56.344 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:49:56.344 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 34261, 2023-11-16T16:49:56.344 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 1.997790596, 2023-11-16T16:49:56.344 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000058310 2023-11-16T16:49:56.344 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.345 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:49:56.345 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 45842, 2023-11-16T16:49:56.345 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.007742111, 2023-11-16T16:49:56.345 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000168 2023-11-16T16:49:56.345 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.345 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 1493712, 2023-11-16T16:49:56.345 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 283360 2023-11-16T16:49:56.345 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.346 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-2": { 2023-11-16T16:49:56.346 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 121681, 2023-11-16T16:49:56.346 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 120636, 2023-11-16T16:49:56.346 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 36675085, 2023-11-16T16:49:56.346 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 76000342, 2023-11-16T16:49:56.346 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 17, 2023-11-16T16:49:56.346 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 6, 2023-11-16T16:49:56.346 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 7.048119841, 2023-11-16T16:49:56.347 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 2.999062742, 2023-11-16T16:49:56.347 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 4.557776000, 2023-11-16T16:49:56.347 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.346692024, 2023-11-16T16:49:56.347 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:49:56.347 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 120634, 2023-11-16T16:49:56.347 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 3.087526952, 2023-11-16T16:49:56.347 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000025594 2023-11-16T16:49:56.348 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.348 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:49:56.348 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 219707, 2023-11-16T16:49:56.348 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.031582656, 2023-11-16T16:49:56.348 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000143 2023-11-16T16:49:56.348 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.348 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 5424, 2023-11-16T16:49:56.348 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 437488 2023-11-16T16:49:56.349 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.349 INFO:teuthology.orchestra.run.smithi026.stdout: "cct": { 2023-11-16T16:49:56.349 INFO:teuthology.orchestra.run.smithi026.stdout: "total_workers": 1, 2023-11-16T16:49:56.349 INFO:teuthology.orchestra.run.smithi026.stdout: "unhealthy_workers": 0 2023-11-16T16:49:56.349 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.349 INFO:teuthology.orchestra.run.smithi026.stdout: "finisher-MDSRank": { 2023-11-16T16:49:56.349 INFO:teuthology.orchestra.run.smithi026.stdout: "queue_len": 0, 2023-11-16T16:49:56.349 INFO:teuthology.orchestra.run.smithi026.stdout: "complete_latency": { 2023-11-16T16:49:56.350 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 5531, 2023-11-16T16:49:56.350 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 80.906536244, 2023-11-16T16:49:56.350 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.014627831 2023-11-16T16:49:56.350 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-11-16T16:49:56.350 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.350 INFO:teuthology.orchestra.run.smithi026.stdout: "finisher-PurgeQueue": { 2023-11-16T16:49:56.350 INFO:teuthology.orchestra.run.smithi026.stdout: "queue_len": 0, 2023-11-16T16:49:56.350 INFO:teuthology.orchestra.run.smithi026.stdout: "complete_latency": { 2023-11-16T16:49:56.351 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 21771, 2023-11-16T16:49:56.351 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 17.767118825, 2023-11-16T16:49:56.351 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000816091 2023-11-16T16:49:56.351 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-11-16T16:49:56.351 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.351 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": { 2023-11-16T16:49:56.351 INFO:teuthology.orchestra.run.smithi026.stdout: "request": 48086, 2023-11-16T16:49:56.351 INFO:teuthology.orchestra.run.smithi026.stdout: "reply": 40167, 2023-11-16T16:49:56.352 INFO:teuthology.orchestra.run.smithi026.stdout: "reply_latency": { 2023-11-16T16:49:56.352 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 40167, 2023-11-16T16:49:56.352 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 201.908847032, 2023-11-16T16:49:56.352 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.005026734 2023-11-16T16:49:56.352 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.352 INFO:teuthology.orchestra.run.smithi026.stdout: "slow_reply": 0, 2023-11-16T16:49:56.352 INFO:teuthology.orchestra.run.smithi026.stdout: "forward": 7919, 2023-11-16T16:49:56.352 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_fetch_complete": 694, 2023-11-16T16:49:56.353 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_fetch_keys": 0, 2023-11-16T16:49:56.353 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_commit": 419, 2023-11-16T16:49:56.353 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_split": 51, 2023-11-16T16:49:56.353 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_merge": 0, 2023-11-16T16:49:56.353 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes": 25709, 2023-11-16T16:49:56.353 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_top": 1899, 2023-11-16T16:49:56.353 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_bottom": 867, 2023-11-16T16:49:56.353 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_pin_tail": 22943, 2023-11-16T16:49:56.354 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_pinned": 22994, 2023-11-16T16:49:56.354 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_expired": 34535, 2023-11-16T16:49:56.354 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_with_caps": 0, 2023-11-16T16:49:56.354 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": 0, 2023-11-16T16:49:56.354 INFO:teuthology.orchestra.run.smithi026.stdout: "subtrees": 4, 2023-11-16T16:49:56.354 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse": 65528, 2023-11-16T16:49:56.354 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_hit": 57399, 2023-11-16T16:49:56.354 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_forward": 7588, 2023-11-16T16:49:56.355 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_discover": 10, 2023-11-16T16:49:56.355 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_dir_fetch": 14, 2023-11-16T16:49:56.355 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_remote_ino": 0, 2023-11-16T16:49:56.355 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_lock": 0, 2023-11-16T16:49:56.355 INFO:teuthology.orchestra.run.smithi026.stdout: "load_cent": 0, 2023-11-16T16:49:56.355 INFO:teuthology.orchestra.run.smithi026.stdout: "q": 0, 2023-11-16T16:49:56.355 INFO:teuthology.orchestra.run.smithi026.stdout: "exported": 85, 2023-11-16T16:49:56.356 INFO:teuthology.orchestra.run.smithi026.stdout: "exported_inodes": 30862, 2023-11-16T16:49:56.356 INFO:teuthology.orchestra.run.smithi026.stdout: "imported": 502, 2023-11-16T16:49:56.356 INFO:teuthology.orchestra.run.smithi026.stdout: "imported_inodes": 0, 2023-11-16T16:49:56.356 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_dir_fetch": 0, 2023-11-16T16:49:56.356 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_backtrace_fetch": 7635, 2023-11-16T16:49:56.356 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_peer_discover": 0, 2023-11-16T16:49:56.356 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rfiles": 4, 2023-11-16T16:49:56.357 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rbytes": 246, 2023-11-16T16:49:56.357 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rsnaps": 6, 2023-11-16T16:49:56.357 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_backtrace_fetch": 7635, 2023-11-16T16:49:56.357 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_set_tag": 0, 2023-11-16T16:49:56.357 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_backtrace_repaired": 0, 2023-11-16T16:49:56.357 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_inotable_repaired": 0, 2023-11-16T16:49:56.357 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dir_inodes": 3, 2023-11-16T16:49:56.357 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dir_base_inodes": 0, 2023-11-16T16:49:56.358 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dirfrag_rstats": 3, 2023-11-16T16:49:56.358 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_file_inodes": 7632, 2023-11-16T16:49:56.358 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_inode_file_caps": 8, 2023-11-16T16:49:56.358 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_revoke": 1565, 2023-11-16T16:49:56.358 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_grant": 23534, 2023-11-16T16:49:56.358 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_trunc": 0, 2023-11-16T16:49:56.358 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_flushsnap_ack": 104, 2023-11-16T16:49:56.358 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-16T16:49:56.359 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_caps": 20560, 2023-11-16T16:49:56.359 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_caps_dirty": 18996, 2023-11-16T16:49:56.359 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_cap_release": 620, 2023-11-16T16:49:56.359 INFO:teuthology.orchestra.run.smithi026.stdout: "process_request_cap_release": 33441 2023-11-16T16:49:56.359 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.359 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_cache": { 2023-11-16T16:49:56.359 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_update": 293, 2023-11-16T16:49:56.359 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_update_receipt": 2902, 2023-11-16T16:49:56.360 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_try_discover": 1453, 2023-11-16T16:49:56.360 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_send_discover": 1667, 2023-11-16T16:49:56.360 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_handle_discover": 707, 2023-11-16T16:49:56.360 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays": 1, 2023-11-16T16:49:56.360 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays_delayed": 0, 2023-11-16T16:49:56.360 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays_enqueuing": 0, 2023-11-16T16:49:56.360 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_created": 16034, 2023-11-16T16:49:56.360 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_enqueued": 16033, 2023-11-16T16:49:56.361 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_reintegrated": 0, 2023-11-16T16:49:56.361 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_migrated": 0, 2023-11-16T16:49:56.361 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_processing": 0, 2023-11-16T16:49:56.361 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_enqueued": 0, 2023-11-16T16:49:56.361 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_prioritized": 0, 2023-11-16T16:49:56.361 INFO:teuthology.orchestra.run.smithi026.stdout: "recovery_started": 0, 2023-11-16T16:49:56.361 INFO:teuthology.orchestra.run.smithi026.stdout: "recovery_completed": 0, 2023-11-16T16:49:56.361 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_enqueue_scrub": 0, 2023-11-16T16:49:56.362 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_exportdir": 91, 2023-11-16T16:49:56.362 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_flush": 0, 2023-11-16T16:49:56.362 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_fragmentdir": 54, 2023-11-16T16:49:56.362 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_fragstats": 0, 2023-11-16T16:49:56.362 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_inodestats": 0 2023-11-16T16:49:56.362 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.362 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_log": { 2023-11-16T16:49:56.362 INFO:teuthology.orchestra.run.smithi026.stdout: "evadd": 63477, 2023-11-16T16:49:56.363 INFO:teuthology.orchestra.run.smithi026.stdout: "evex": 0, 2023-11-16T16:49:56.363 INFO:teuthology.orchestra.run.smithi026.stdout: "evtrm": 0, 2023-11-16T16:49:56.363 INFO:teuthology.orchestra.run.smithi026.stdout: "ev": 63477, 2023-11-16T16:49:56.363 INFO:teuthology.orchestra.run.smithi026.stdout: "evexg": 0, 2023-11-16T16:49:56.363 INFO:teuthology.orchestra.run.smithi026.stdout: "evexd": 0, 2023-11-16T16:49:56.363 INFO:teuthology.orchestra.run.smithi026.stdout: "segadd": 97, 2023-11-16T16:49:56.363 INFO:teuthology.orchestra.run.smithi026.stdout: "segex": 0, 2023-11-16T16:49:56.363 INFO:teuthology.orchestra.run.smithi026.stdout: "segtrm": 0, 2023-11-16T16:49:56.364 INFO:teuthology.orchestra.run.smithi026.stdout: "seg": 97, 2023-11-16T16:49:56.364 INFO:teuthology.orchestra.run.smithi026.stdout: "segexg": 0, 2023-11-16T16:49:56.364 INFO:teuthology.orchestra.run.smithi026.stdout: "segexd": 0, 2023-11-16T16:49:56.364 INFO:teuthology.orchestra.run.smithi026.stdout: "expos": 4194304, 2023-11-16T16:49:56.364 INFO:teuthology.orchestra.run.smithi026.stdout: "wrpos": 265413695, 2023-11-16T16:49:56.364 INFO:teuthology.orchestra.run.smithi026.stdout: "rdpos": 0, 2023-11-16T16:49:56.364 INFO:teuthology.orchestra.run.smithi026.stdout: "jlat": { 2023-11-16T16:49:56.364 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 1024, 2023-11-16T16:49:56.365 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 127.160429657, 2023-11-16T16:49:56.365 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.124180107 2023-11-16T16:49:56.365 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.365 INFO:teuthology.orchestra.run.smithi026.stdout: "replayed": 0 2023-11-16T16:49:56.365 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.365 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_mem": { 2023-11-16T16:49:56.365 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 15712, 2023-11-16T16:49:56.365 INFO:teuthology.orchestra.run.smithi026.stdout: "ino+": 54583, 2023-11-16T16:49:56.366 INFO:teuthology.orchestra.run.smithi026.stdout: "ino-": 38871, 2023-11-16T16:49:56.366 INFO:teuthology.orchestra.run.smithi026.stdout: "dir": 533, 2023-11-16T16:49:56.366 INFO:teuthology.orchestra.run.smithi026.stdout: "dir+": 2673, 2023-11-16T16:49:56.366 INFO:teuthology.orchestra.run.smithi026.stdout: "dir-": 2140, 2023-11-16T16:49:56.366 INFO:teuthology.orchestra.run.smithi026.stdout: "dn": 25709, 2023-11-16T16:49:56.366 INFO:teuthology.orchestra.run.smithi026.stdout: "dn+": 76277, 2023-11-16T16:49:56.366 INFO:teuthology.orchestra.run.smithi026.stdout: "dn-": 50568, 2023-11-16T16:49:56.366 INFO:teuthology.orchestra.run.smithi026.stdout: "cap": 0, 2023-11-16T16:49:56.367 INFO:teuthology.orchestra.run.smithi026.stdout: "cap+": 35326, 2023-11-16T16:49:56.367 INFO:teuthology.orchestra.run.smithi026.stdout: "cap-": 35326, 2023-11-16T16:49:56.367 INFO:teuthology.orchestra.run.smithi026.stdout: "rss": 280468, 2023-11-16T16:49:56.367 INFO:teuthology.orchestra.run.smithi026.stdout: "heap": 198964 2023-11-16T16:49:56.367 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.367 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_server": { 2023-11-16T16:49:56.367 INFO:teuthology.orchestra.run.smithi026.stdout: "dispatch_client_request": 48841, 2023-11-16T16:49:56.367 INFO:teuthology.orchestra.run.smithi026.stdout: "dispatch_server_request": 46, 2023-11-16T16:49:56.368 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_request": 48086, 2023-11-16T16:49:56.368 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_session": 16, 2023-11-16T16:49:56.368 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_peer_request": 131, 2023-11-16T16:49:56.368 INFO:teuthology.orchestra.run.smithi026.stdout: "req_create_latency": { 2023-11-16T16:49:56.368 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 7061, 2023-11-16T16:49:56.368 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 40.992558521, 2023-11-16T16:49:56.368 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.005805489 2023-11-16T16:49:56.369 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.369 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getattr_latency": { 2023-11-16T16:49:56.369 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 2, 2023-11-16T16:49:56.369 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.009481976, 2023-11-16T16:49:56.369 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.004740988 2023-11-16T16:49:56.369 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.369 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getfilelock_latency": { 2023-11-16T16:49:56.369 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:56.369 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:56.370 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:56.370 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.370 INFO:teuthology.orchestra.run.smithi026.stdout: "req_link_latency": { 2023-11-16T16:49:56.370 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:56.370 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:56.370 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:56.370 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.371 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookup_latency": { 2023-11-16T16:49:56.371 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 15733, 2023-11-16T16:49:56.371 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 49.665959184, 2023-11-16T16:49:56.371 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.003156801 2023-11-16T16:49:56.371 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.371 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookuphash_latency": { 2023-11-16T16:49:56.371 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:56.372 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:56.372 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:56.372 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.372 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupino_latency": { 2023-11-16T16:49:56.372 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:56.372 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:56.372 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:56.372 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.372 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupname_latency": { 2023-11-16T16:49:56.373 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:56.373 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:56.373 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:56.373 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.373 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupparent_latency": { 2023-11-16T16:49:56.373 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:56.373 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:56.373 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:56.374 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.374 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupsnap_latency": { 2023-11-16T16:49:56.374 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:56.374 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:56.374 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:56.374 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.374 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lssnap_latency": { 2023-11-16T16:49:56.374 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:56.375 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:56.375 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:56.375 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.375 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mkdir_latency": { 2023-11-16T16:49:56.375 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:56.375 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:56.375 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:56.375 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.376 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mknod_latency": { 2023-11-16T16:49:56.376 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:56.376 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:56.376 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:56.376 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.376 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mksnap_latency": { 2023-11-16T16:49:56.376 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:56.376 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:56.377 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:56.377 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.377 INFO:teuthology.orchestra.run.smithi026.stdout: "req_open_latency": { 2023-11-16T16:49:56.377 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:56.377 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:56.377 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:56.377 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.377 INFO:teuthology.orchestra.run.smithi026.stdout: "req_readdir_latency": { 2023-11-16T16:49:56.378 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 1337, 2023-11-16T16:49:56.378 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 3.128164812, 2023-11-16T16:49:56.378 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.002339689 2023-11-16T16:49:56.378 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.378 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rename_latency": { 2023-11-16T16:49:56.378 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:56.378 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:56.379 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:56.379 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.379 INFO:teuthology.orchestra.run.smithi026.stdout: "req_renamesnap_latency": { 2023-11-16T16:49:56.379 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:56.379 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:56.379 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:56.379 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.379 INFO:teuthology.orchestra.run.smithi026.stdout: "req_snapdiff_latency": { 2023-11-16T16:49:56.380 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:56.380 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:56.380 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:56.380 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.380 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmdir_latency": { 2023-11-16T16:49:56.380 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 1, 2023-11-16T16:49:56.380 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.144621674, 2023-11-16T16:49:56.380 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.144621674 2023-11-16T16:49:56.381 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.381 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmsnap_latency": { 2023-11-16T16:49:56.381 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:56.381 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:56.381 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:56.381 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.381 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmxattr_latency": { 2023-11-16T16:49:56.381 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:56.382 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:56.382 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:56.382 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.382 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setattr_latency": { 2023-11-16T16:49:56.382 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:56.382 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:56.382 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:56.382 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.383 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setdirlayout_latency": { 2023-11-16T16:49:56.383 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:56.383 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:56.383 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:56.383 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.383 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setfilelock_latency": { 2023-11-16T16:49:56.383 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:56.383 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:56.384 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:56.384 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.384 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setlayout_latency": { 2023-11-16T16:49:56.384 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:56.384 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:56.384 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:56.384 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.384 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setxattr_latency": { 2023-11-16T16:49:56.385 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:56.385 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:56.385 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:56.385 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.385 INFO:teuthology.orchestra.run.smithi026.stdout: "req_symlink_latency": { 2023-11-16T16:49:56.385 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:56.385 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:56.385 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:56.386 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.386 INFO:teuthology.orchestra.run.smithi026.stdout: "req_unlink_latency": { 2023-11-16T16:49:56.386 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 16033, 2023-11-16T16:49:56.386 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 107.968060865, 2023-11-16T16:49:56.386 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.006734114 2023-11-16T16:49:56.386 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.386 INFO:teuthology.orchestra.run.smithi026.stdout: "cap_revoke_eviction": 0, 2023-11-16T16:49:56.387 INFO:teuthology.orchestra.run.smithi026.stdout: "cap_acquisition_throttle": 0, 2023-11-16T16:49:56.387 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getvxattr_latency": { 2023-11-16T16:49:56.387 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:56.387 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:56.387 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:56.387 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-11-16T16:49:56.387 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.387 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_sessions": { 2023-11-16T16:49:56.388 INFO:teuthology.orchestra.run.smithi026.stdout: "session_count": 1, 2023-11-16T16:49:56.388 INFO:teuthology.orchestra.run.smithi026.stdout: "session_add": 3, 2023-11-16T16:49:56.388 INFO:teuthology.orchestra.run.smithi026.stdout: "session_remove": 2, 2023-11-16T16:49:56.388 INFO:teuthology.orchestra.run.smithi026.stdout: "sessions_open": 1, 2023-11-16T16:49:56.388 INFO:teuthology.orchestra.run.smithi026.stdout: "sessions_stale": 0, 2023-11-16T16:49:56.388 INFO:teuthology.orchestra.run.smithi026.stdout: "total_load": 16822, 2023-11-16T16:49:56.388 INFO:teuthology.orchestra.run.smithi026.stdout: "average_load": 16822, 2023-11-16T16:49:56.388 INFO:teuthology.orchestra.run.smithi026.stdout: "avg_session_uptime": 575 2023-11-16T16:49:56.389 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.389 INFO:teuthology.orchestra.run.smithi026.stdout: "mempool": { 2023-11-16T16:49:56.389 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_bytes": 0, 2023-11-16T16:49:56.389 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_items": 0, 2023-11-16T16:49:56.389 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_bytes": 0, 2023-11-16T16:49:56.389 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_items": 0, 2023-11-16T16:49:56.389 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_bytes": 0, 2023-11-16T16:49:56.389 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_items": 0, 2023-11-16T16:49:56.390 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-16T16:49:56.390 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_items": 0, 2023-11-16T16:49:56.390 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-16T16:49:56.390 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_items": 0, 2023-11-16T16:49:56.390 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_bytes": 0, 2023-11-16T16:49:56.390 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_items": 0, 2023-11-16T16:49:56.390 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-16T16:49:56.390 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_items": 0, 2023-11-16T16:49:56.391 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_bytes": 0, 2023-11-16T16:49:56.391 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_items": 0, 2023-11-16T16:49:56.391 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_bytes": 0, 2023-11-16T16:49:56.391 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_items": 0, 2023-11-16T16:49:56.391 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-16T16:49:56.391 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_items": 0, 2023-11-16T16:49:56.391 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-16T16:49:56.391 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_items": 0, 2023-11-16T16:49:56.392 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_bytes": 0, 2023-11-16T16:49:56.392 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_items": 0, 2023-11-16T16:49:56.392 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_bytes": 0, 2023-11-16T16:49:56.392 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_items": 0, 2023-11-16T16:49:56.392 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-16T16:49:56.392 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_items": 0, 2023-11-16T16:49:56.392 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_bytes": 0, 2023-11-16T16:49:56.392 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_items": 0, 2023-11-16T16:49:56.393 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_bytes": 0, 2023-11-16T16:49:56.393 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_items": 0, 2023-11-16T16:49:56.393 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_bytes": 0, 2023-11-16T16:49:56.393 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_items": 0, 2023-11-16T16:49:56.393 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_bytes": 0, 2023-11-16T16:49:56.393 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_items": 0, 2023-11-16T16:49:56.393 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_bytes": 761619, 2023-11-16T16:49:56.393 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_items": 13687, 2023-11-16T16:49:56.394 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_bytes": 352, 2023-11-16T16:49:56.394 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_items": 4, 2023-11-16T16:49:56.394 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_bytes": 0, 2023-11-16T16:49:56.394 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_items": 0, 2023-11-16T16:49:56.394 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_bytes": 0, 2023-11-16T16:49:56.394 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_items": 0, 2023-11-16T16:49:56.394 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_bytes": 0, 2023-11-16T16:49:56.395 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_items": 0, 2023-11-16T16:49:56.395 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_bytes": 8128, 2023-11-16T16:49:56.395 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_items": 169, 2023-11-16T16:49:56.395 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_bytes": 0, 2023-11-16T16:49:56.395 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_items": 0, 2023-11-16T16:49:56.395 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_bytes": 0, 2023-11-16T16:49:56.395 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_items": 0, 2023-11-16T16:49:56.395 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_bytes": 53748671, 2023-11-16T16:49:56.396 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_items": 484517, 2023-11-16T16:49:56.396 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_bytes": 0, 2023-11-16T16:49:56.396 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_items": 0, 2023-11-16T16:49:56.396 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_bytes": 0, 2023-11-16T16:49:56.396 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_items": 0 2023-11-16T16:49:56.396 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.396 INFO:teuthology.orchestra.run.smithi026.stdout: "objecter": { 2023-11-16T16:49:56.396 INFO:teuthology.orchestra.run.smithi026.stdout: "op_active": 0, 2023-11-16T16:49:56.397 INFO:teuthology.orchestra.run.smithi026.stdout: "op_laggy": 0, 2023-11-16T16:49:56.397 INFO:teuthology.orchestra.run.smithi026.stdout: "op_send": 27289, 2023-11-16T16:49:56.397 INFO:teuthology.orchestra.run.smithi026.stdout: "op_send_bytes": 267862930, 2023-11-16T16:49:56.397 INFO:teuthology.orchestra.run.smithi026.stdout: "op_resend": 33, 2023-11-16T16:49:56.397 INFO:teuthology.orchestra.run.smithi026.stdout: "op_reply": 27289, 2023-11-16T16:49:56.397 INFO:teuthology.orchestra.run.smithi026.stdout: "op_latency": { 2023-11-16T16:49:56.397 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 27289, 2023-11-16T16:49:56.397 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 69.302936867, 2023-11-16T16:49:56.398 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.002539592 2023-11-16T16:49:56.398 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.398 INFO:teuthology.orchestra.run.smithi026.stdout: "op_inflight": 0, 2023-11-16T16:49:56.398 INFO:teuthology.orchestra.run.smithi026.stdout: "oplen_avg": { 2023-11-16T16:49:56.398 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 27289, 2023-11-16T16:49:56.398 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 28495 2023-11-16T16:49:56.398 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.398 INFO:teuthology.orchestra.run.smithi026.stdout: "op": 27289, 2023-11-16T16:49:56.399 INFO:teuthology.orchestra.run.smithi026.stdout: "op_r": 9249, 2023-11-16T16:49:56.399 INFO:teuthology.orchestra.run.smithi026.stdout: "op_w": 18040, 2023-11-16T16:49:56.399 INFO:teuthology.orchestra.run.smithi026.stdout: "op_rmw": 0, 2023-11-16T16:49:56.399 INFO:teuthology.orchestra.run.smithi026.stdout: "op_pg": 0, 2023-11-16T16:49:56.399 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_stat": 0, 2023-11-16T16:49:56.399 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_create": 10, 2023-11-16T16:49:56.399 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_read": 267, 2023-11-16T16:49:56.399 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_write": 1291, 2023-11-16T16:49:56.400 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_writefull": 112, 2023-11-16T16:49:56.400 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_writesame": 0, 2023-11-16T16:49:56.400 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_append": 0, 2023-11-16T16:49:56.400 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_zero": 0, 2023-11-16T16:49:56.400 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_truncate": 0, 2023-11-16T16:49:56.400 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_delete": 16158, 2023-11-16T16:49:56.400 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_mapext": 0, 2023-11-16T16:49:56.400 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_sparse_read": 0, 2023-11-16T16:49:56.400 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_clonerange": 0, 2023-11-16T16:49:56.401 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_getxattr": 7635, 2023-11-16T16:49:56.401 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_setxattr": 20, 2023-11-16T16:49:56.401 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_cmpxattr": 0, 2023-11-16T16:49:56.401 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_rmxattr": 0, 2023-11-16T16:49:56.401 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_resetxattrs": 0, 2023-11-16T16:49:56.401 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_call": 0, 2023-11-16T16:49:56.401 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_watch": 0, 2023-11-16T16:49:56.401 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_notify": 0, 2023-11-16T16:49:56.402 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_src_cmpxattr": 0, 2023-11-16T16:49:56.402 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_pgls": 0, 2023-11-16T16:49:56.402 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_pgls_filter": 0, 2023-11-16T16:49:56.402 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_other": 0, 2023-11-16T16:49:56.402 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_active": 0, 2023-11-16T16:49:56.402 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_send": 0, 2023-11-16T16:49:56.402 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_resend": 0, 2023-11-16T16:49:56.402 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_ping": 0, 2023-11-16T16:49:56.403 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_active": 0, 2023-11-16T16:49:56.403 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_send": 0, 2023-11-16T16:49:56.403 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_resend": 0, 2023-11-16T16:49:56.403 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_active": 0, 2023-11-16T16:49:56.403 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_send": 0, 2023-11-16T16:49:56.403 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_resend": 0, 2023-11-16T16:49:56.403 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_active": 0, 2023-11-16T16:49:56.403 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_send": 0, 2023-11-16T16:49:56.404 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_resend": 0, 2023-11-16T16:49:56.404 INFO:teuthology.orchestra.run.smithi026.stdout: "command_active": 0, 2023-11-16T16:49:56.404 INFO:teuthology.orchestra.run.smithi026.stdout: "command_send": 0, 2023-11-16T16:49:56.404 INFO:teuthology.orchestra.run.smithi026.stdout: "command_resend": 0, 2023-11-16T16:49:56.404 INFO:teuthology.orchestra.run.smithi026.stdout: "map_epoch": 95, 2023-11-16T16:49:56.404 INFO:teuthology.orchestra.run.smithi026.stdout: "map_full": 0, 2023-11-16T16:49:56.404 INFO:teuthology.orchestra.run.smithi026.stdout: "map_inc": 11, 2023-11-16T16:49:56.404 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_sessions": 12, 2023-11-16T16:49:56.405 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_session_open": 12, 2023-11-16T16:49:56.405 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_session_close": 0, 2023-11-16T16:49:56.405 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_laggy": 0, 2023-11-16T16:49:56.405 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_wr": 923, 2023-11-16T16:49:56.405 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_rd": 2041, 2023-11-16T16:49:56.405 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_del": 38 2023-11-16T16:49:56.405 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.405 INFO:teuthology.orchestra.run.smithi026.stdout: "oft": { 2023-11-16T16:49:56.406 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_objs": 1, 2023-11-16T16:49:56.406 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_kv_pairs": 0, 2023-11-16T16:49:56.406 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_updates": 0, 2023-11-16T16:49:56.406 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_removes": 0 2023-11-16T16:49:56.406 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.406 INFO:teuthology.orchestra.run.smithi026.stdout: "osd-slow-ops": { 2023-11-16T16:49:56.406 INFO:teuthology.orchestra.run.smithi026.stdout: "slow_ops_count": 48279 2023-11-16T16:49:56.407 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.407 INFO:teuthology.orchestra.run.smithi026.stdout: "purge_queue": { 2023-11-16T16:49:56.407 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_ops": 0, 2023-11-16T16:49:56.407 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_ops_high_water": 10, 2023-11-16T16:49:56.407 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing": 0, 2023-11-16T16:49:56.407 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_high_water": 10, 2023-11-16T16:49:56.407 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executed": 16033, 2023-11-16T16:49:56.407 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_item_in_journal": 0 2023-11-16T16:49:56.408 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.408 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-16T16:49:56.408 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2023-11-16T16:49:56.408 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2023-11-16T16:49:56.408 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2023-11-16T16:49:56.408 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 229062, 2023-11-16T16:49:56.408 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 125903648, 2023-11-16T16:49:56.408 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2023-11-16T16:49:56.409 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 229062, 2023-11-16T16:49:56.409 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2023-11-16T16:49:56.409 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2023-11-16T16:49:56.409 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 229062, 2023-11-16T16:49:56.409 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 125903648, 2023-11-16T16:49:56.409 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2023-11-16T16:49:56.409 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:56.409 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:56.410 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:56.410 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-11-16T16:49:56.410 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.410 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-objecter_bytes": { 2023-11-16T16:49:56.410 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2023-11-16T16:49:56.410 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2023-11-16T16:49:56.410 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2023-11-16T16:49:56.410 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2023-11-16T16:49:56.411 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2023-11-16T16:49:56.411 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2023-11-16T16:49:56.411 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2023-11-16T16:49:56.411 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 27289, 2023-11-16T16:49:56.411 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 269739883, 2023-11-16T16:49:56.411 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 27289, 2023-11-16T16:49:56.411 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 269739883, 2023-11-16T16:49:56.411 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2023-11-16T16:49:56.412 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:56.412 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:56.412 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:56.412 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-11-16T16:49:56.412 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.412 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-objecter_ops": { 2023-11-16T16:49:56.412 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2023-11-16T16:49:56.412 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1024, 2023-11-16T16:49:56.412 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2023-11-16T16:49:56.413 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2023-11-16T16:49:56.413 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2023-11-16T16:49:56.413 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2023-11-16T16:49:56.413 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2023-11-16T16:49:56.413 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 27289, 2023-11-16T16:49:56.413 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 27289, 2023-11-16T16:49:56.413 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 27289, 2023-11-16T16:49:56.413 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 27289, 2023-11-16T16:49:56.414 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2023-11-16T16:49:56.414 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:56.414 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:56.414 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:56.414 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-11-16T16:49:56.414 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.414 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-write_buf_throttle": { 2023-11-16T16:49:56.414 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2023-11-16T16:49:56.415 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 3758096384, 2023-11-16T16:49:56.415 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2023-11-16T16:49:56.415 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 16033, 2023-11-16T16:49:56.415 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 1906445, 2023-11-16T16:49:56.415 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2023-11-16T16:49:56.415 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 16033, 2023-11-16T16:49:56.415 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2023-11-16T16:49:56.415 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2023-11-16T16:49:56.416 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 267, 2023-11-16T16:49:56.416 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 1906445, 2023-11-16T16:49:56.416 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2023-11-16T16:49:56.416 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:56.416 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:56.416 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:56.416 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-11-16T16:49:56.416 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:56.417 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-write_buf_throttle-0x56048ec13cc0": { 2023-11-16T16:49:56.417 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2023-11-16T16:49:56.417 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 3758096384, 2023-11-16T16:49:56.417 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2023-11-16T16:49:56.417 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 63477, 2023-11-16T16:49:56.417 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 261219391, 2023-11-16T16:49:56.417 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2023-11-16T16:49:56.417 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 63477, 2023-11-16T16:49:56.418 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2023-11-16T16:49:56.418 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2023-11-16T16:49:56.418 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 1024, 2023-11-16T16:49:56.418 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 261219391, 2023-11-16T16:49:56.418 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2023-11-16T16:49:56.418 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:56.418 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:56.418 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:56.419 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-11-16T16:49:56.419 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-11-16T16:49:56.419 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-11-16T16:49:56.779 INFO:tasks.check_counter:Daemon mds.h mds.exported=85 2023-11-16T16:49:56.779 INFO:tasks.check_counter:Daemon mds.h mds.imported=502 2023-11-16T16:49:56.779 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=293 2023-11-16T16:49:56.779 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=2902 2023-11-16T16:49:56.780 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2023-11-16T16:49:56.780 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2023-11-16T16:49:56.780 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2023-11-16T16:49:56.780 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=51 2023-11-16T16:49:56.780 DEBUG:tasks.check_counter:Getting stats from k 2023-11-16T16:49:56.780 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph daemon mds.k perf dump 2023-11-16T16:49:57.429 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.b/config 2023-11-16T16:49:57.930 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:57 smithi026 ceph-mon[100112]: pgmap v335: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:49:58.080 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:57 smithi053 ceph-mon[100638]: pgmap v335: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:49:58.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:57 smithi001 ceph-mon[98800]: pgmap v335: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:49:58.794 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-11-16T16:49:58.795 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-0": { 2023-11-16T16:49:58.795 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 227, 2023-11-16T16:49:58.795 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 215, 2023-11-16T16:49:58.795 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 1424976, 2023-11-16T16:49:58.795 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 121968, 2023-11-16T16:49:58.795 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 4, 2023-11-16T16:49:58.796 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 0, 2023-11-16T16:49:58.796 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.045505562, 2023-11-16T16:49:58.796 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.012359684, 2023-11-16T16:49:58.796 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.035295421, 2023-11-16T16:49:58.796 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.006297780, 2023-11-16T16:49:58.796 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:49:58.796 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 215, 2023-11-16T16:49:58.796 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.011626742, 2023-11-16T16:49:58.797 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000054077 2023-11-16T16:49:58.797 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:58.797 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:49:58.797 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:58.797 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:58.797 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:58.797 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:58.797 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 1424976, 2023-11-16T16:49:58.798 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 121968 2023-11-16T16:49:58.798 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:58.798 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-1": { 2023-11-16T16:49:58.798 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 2, 2023-11-16T16:49:58.798 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 162, 2023-11-16T16:49:58.798 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 320, 2023-11-16T16:49:58.798 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 180208, 2023-11-16T16:49:58.798 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 5, 2023-11-16T16:49:58.799 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 3, 2023-11-16T16:49:58.799 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.010350336, 2023-11-16T16:49:58.799 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.006572976, 2023-11-16T16:49:58.799 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.001416592, 2023-11-16T16:49:58.799 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-16T16:49:58.799 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:49:58.799 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 159, 2023-11-16T16:49:58.800 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.015524122, 2023-11-16T16:49:58.800 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000097635 2023-11-16T16:49:58.800 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:58.800 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:49:58.800 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:58.800 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:58.800 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:58.800 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:58.800 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 320, 2023-11-16T16:49:58.801 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 180208 2023-11-16T16:49:58.801 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:58.801 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-2": { 2023-11-16T16:49:58.801 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 0, 2023-11-16T16:49:58.801 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 2, 2023-11-16T16:49:58.801 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 0, 2023-11-16T16:49:58.801 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 0, 2023-11-16T16:49:58.801 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 2, 2023-11-16T16:49:58.802 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 1, 2023-11-16T16:49:58.802 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.000334342, 2023-11-16T16:49:58.802 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.000000000, 2023-11-16T16:49:58.802 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.000218110, 2023-11-16T16:49:58.802 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-16T16:49:58.802 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:49:58.802 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:58.802 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:58.803 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:58.803 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:58.803 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:49:58.803 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:58.803 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:58.803 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:58.803 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:58.804 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-16T16:49:58.804 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 0 2023-11-16T16:49:58.804 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:58.804 INFO:teuthology.orchestra.run.smithi026.stdout: "cct": { 2023-11-16T16:49:58.804 INFO:teuthology.orchestra.run.smithi026.stdout: "total_workers": 0, 2023-11-16T16:49:58.804 INFO:teuthology.orchestra.run.smithi026.stdout: "unhealthy_workers": 0 2023-11-16T16:49:58.804 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:58.804 INFO:teuthology.orchestra.run.smithi026.stdout: "mempool": { 2023-11-16T16:49:58.805 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_bytes": 0, 2023-11-16T16:49:58.805 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_items": 0, 2023-11-16T16:49:58.805 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_bytes": 0, 2023-11-16T16:49:58.805 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_items": 0, 2023-11-16T16:49:58.805 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_bytes": 0, 2023-11-16T16:49:58.805 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_items": 0, 2023-11-16T16:49:58.805 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-16T16:49:58.805 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_items": 0, 2023-11-16T16:49:58.805 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-16T16:49:58.806 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_items": 0, 2023-11-16T16:49:58.806 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_bytes": 0, 2023-11-16T16:49:58.806 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_items": 0, 2023-11-16T16:49:58.806 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-16T16:49:58.806 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_items": 0, 2023-11-16T16:49:58.806 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_bytes": 0, 2023-11-16T16:49:58.806 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_items": 0, 2023-11-16T16:49:58.806 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_bytes": 0, 2023-11-16T16:49:58.807 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_items": 0, 2023-11-16T16:49:58.807 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-16T16:49:58.807 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_items": 0, 2023-11-16T16:49:58.807 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-16T16:49:58.807 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_items": 0, 2023-11-16T16:49:58.807 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_bytes": 0, 2023-11-16T16:49:58.807 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_items": 0, 2023-11-16T16:49:58.807 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_bytes": 0, 2023-11-16T16:49:58.808 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_items": 0, 2023-11-16T16:49:58.808 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-16T16:49:58.808 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_items": 0, 2023-11-16T16:49:58.808 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_bytes": 0, 2023-11-16T16:49:58.808 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_items": 0, 2023-11-16T16:49:58.808 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_bytes": 0, 2023-11-16T16:49:58.808 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_items": 0, 2023-11-16T16:49:58.808 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_bytes": 0, 2023-11-16T16:49:58.809 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_items": 0, 2023-11-16T16:49:58.809 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_bytes": 0, 2023-11-16T16:49:58.809 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_items": 0, 2023-11-16T16:49:58.809 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_bytes": 148985, 2023-11-16T16:49:58.809 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_items": 30, 2023-11-16T16:49:58.809 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_bytes": 0, 2023-11-16T16:49:58.809 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_items": 0, 2023-11-16T16:49:58.809 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_bytes": 0, 2023-11-16T16:49:58.810 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_items": 0, 2023-11-16T16:49:58.810 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_bytes": 0, 2023-11-16T16:49:58.810 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_items": 0, 2023-11-16T16:49:58.810 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_bytes": 0, 2023-11-16T16:49:58.810 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_items": 0, 2023-11-16T16:49:58.810 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_bytes": 0, 2023-11-16T16:49:58.810 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_items": 0, 2023-11-16T16:49:58.810 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_bytes": 0, 2023-11-16T16:49:58.811 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_items": 0, 2023-11-16T16:49:58.811 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_bytes": 0, 2023-11-16T16:49:58.811 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_items": 0, 2023-11-16T16:49:58.811 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_bytes": 928, 2023-11-16T16:49:58.811 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_items": 2, 2023-11-16T16:49:58.811 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_bytes": 0, 2023-11-16T16:49:58.811 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_items": 0, 2023-11-16T16:49:58.811 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_bytes": 0, 2023-11-16T16:49:58.812 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_items": 0 2023-11-16T16:49:58.812 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-11-16T16:49:58.812 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-16T16:49:58.812 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2023-11-16T16:49:58.812 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2023-11-16T16:49:58.812 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2023-11-16T16:49:58.812 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 225, 2023-11-16T16:49:58.812 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 1390499, 2023-11-16T16:49:58.812 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2023-11-16T16:49:58.813 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 225, 2023-11-16T16:49:58.813 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2023-11-16T16:49:58.813 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2023-11-16T16:49:58.813 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 225, 2023-11-16T16:49:58.813 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 1390499, 2023-11-16T16:49:58.813 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2023-11-16T16:49:58.813 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-11-16T16:49:58.813 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-11-16T16:49:58.814 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-11-16T16:49:58.814 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-11-16T16:49:58.814 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-11-16T16:49:58.814 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-11-16T16:49:59.179 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2023-11-16T16:49:59.179 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2023-11-16T16:49:59.179 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2023-11-16T16:49:59.179 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2023-11-16T16:49:59.179 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2023-11-16T16:49:59.179 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2023-11-16T16:49:59.180 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2023-11-16T16:49:59.180 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2023-11-16T16:49:59.180 DEBUG:tasks.check_counter:Getting stats from c 2023-11-16T16:49:59.180 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph daemon mds.c perf dump 2023-11-16T16:49:59.859 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-16T16:50:00.033 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:49:59 smithi053 ceph-mon[100638]: pgmap v336: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:50:00.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:49:59 smithi001 ceph-mon[98800]: pgmap v336: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:50:00.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:49:59 smithi026 ceph-mon[100112]: pgmap v336: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:50:01.064 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-16T16:50:01.064 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2023-11-16T16:50:01.064 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 1484, 2023-11-16T16:50:01.064 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 1484, 2023-11-16T16:50:01.064 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 105402422, 2023-11-16T16:50:01.065 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 460738, 2023-11-16T16:50:01.065 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 8, 2023-11-16T16:50:01.065 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 4, 2023-11-16T16:50:01.065 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.196887385, 2023-11-16T16:50:01.065 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.039310465, 2023-11-16T16:50:01.065 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.089330553, 2023-11-16T16:50:01.065 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.053315363, 2023-11-16T16:50:01.065 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:50:01.066 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1483, 2023-11-16T16:50:01.066 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.061987906, 2023-11-16T16:50:01.066 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000041798 2023-11-16T16:50:01.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.066 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:50:01.066 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.066 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.066 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.067 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-11-16T16:50:01.067 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 448 2023-11-16T16:50:01.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.067 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2023-11-16T16:50:01.067 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 868, 2023-11-16T16:50:01.067 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 1027, 2023-11-16T16:50:01.067 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 96023138, 2023-11-16T16:50:01.068 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 701376, 2023-11-16T16:50:01.068 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 9, 2023-11-16T16:50:01.068 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 7, 2023-11-16T16:50:01.068 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.123410523, 2023-11-16T16:50:01.068 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.026095812, 2023-11-16T16:50:01.068 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.065516882, 2023-11-16T16:50:01.068 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.024505849, 2023-11-16T16:50:01.068 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:50:01.069 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1024, 2023-11-16T16:50:01.069 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.049765871, 2023-11-16T16:50:01.069 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000048599 2023-11-16T16:50:01.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.069 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:50:01.069 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.069 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.069 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.070 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 320, 2023-11-16T16:50:01.070 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 431984 2023-11-16T16:50:01.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.070 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2023-11-16T16:50:01.070 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 738, 2023-11-16T16:50:01.070 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 718, 2023-11-16T16:50:01.070 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 72927472, 2023-11-16T16:50:01.071 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 275374, 2023-11-16T16:50:01.071 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 5, 2023-11-16T16:50:01.071 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 4, 2023-11-16T16:50:01.071 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.117693161, 2023-11-16T16:50:01.071 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.027972980, 2023-11-16T16:50:01.071 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.072587557, 2023-11-16T16:50:01.071 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.021657802, 2023-11-16T16:50:01.071 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:50:01.072 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 717, 2023-11-16T16:50:01.072 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.031381787, 2023-11-16T16:50:01.072 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000043768 2023-11-16T16:50:01.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.072 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:50:01.072 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.072 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.072 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.073 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 1482624, 2023-11-16T16:50:01.073 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 122672 2023-11-16T16:50:01.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.073 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2023-11-16T16:50:01.073 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 1, 2023-11-16T16:50:01.073 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2023-11-16T16:50:01.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.073 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-MDSRank": { 2023-11-16T16:50:01.074 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2023-11-16T16:50:01.074 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2023-11-16T16:50:01.074 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 4524, 2023-11-16T16:50:01.074 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.137184207, 2023-11-16T16:50:01.074 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000030323 2023-11-16T16:50:01.074 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-16T16:50:01.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.074 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-PurgeQueue": { 2023-11-16T16:50:01.075 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2023-11-16T16:50:01.075 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2023-11-16T16:50:01.075 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.075 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.075 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.075 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-16T16:50:01.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.075 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": { 2023-11-16T16:50:01.076 INFO:teuthology.orchestra.run.smithi053.stdout: "request": 0, 2023-11-16T16:50:01.076 INFO:teuthology.orchestra.run.smithi053.stdout: "reply": 0, 2023-11-16T16:50:01.076 INFO:teuthology.orchestra.run.smithi053.stdout: "reply_latency": { 2023-11-16T16:50:01.076 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.076 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.076 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.076 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_reply": 0, 2023-11-16T16:50:01.076 INFO:teuthology.orchestra.run.smithi053.stdout: "forward": 0, 2023-11-16T16:50:01.077 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_complete": 0, 2023-11-16T16:50:01.077 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_keys": 0, 2023-11-16T16:50:01.077 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_commit": 0, 2023-11-16T16:50:01.077 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_split": 0, 2023-11-16T16:50:01.077 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_merge": 0, 2023-11-16T16:50:01.077 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes": 68329, 2023-11-16T16:50:01.077 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_top": 1095, 2023-11-16T16:50:01.077 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_bottom": 46022, 2023-11-16T16:50:01.078 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pin_tail": 21212, 2023-11-16T16:50:01.078 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pinned": 66764, 2023-11-16T16:50:01.078 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_expired": 0, 2023-11-16T16:50:01.078 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_with_caps": 0, 2023-11-16T16:50:01.078 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": 0, 2023-11-16T16:50:01.078 INFO:teuthology.orchestra.run.smithi053.stdout: "subtrees": 2, 2023-11-16T16:50:01.078 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse": 0, 2023-11-16T16:50:01.079 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_hit": 0, 2023-11-16T16:50:01.079 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_forward": 0, 2023-11-16T16:50:01.079 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_discover": 0, 2023-11-16T16:50:01.079 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_dir_fetch": 0, 2023-11-16T16:50:01.079 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_remote_ino": 0, 2023-11-16T16:50:01.079 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_lock": 0, 2023-11-16T16:50:01.079 INFO:teuthology.orchestra.run.smithi053.stdout: "load_cent": 0, 2023-11-16T16:50:01.079 INFO:teuthology.orchestra.run.smithi053.stdout: "q": 0, 2023-11-16T16:50:01.080 INFO:teuthology.orchestra.run.smithi053.stdout: "exported": 0, 2023-11-16T16:50:01.080 INFO:teuthology.orchestra.run.smithi053.stdout: "exported_inodes": 0, 2023-11-16T16:50:01.080 INFO:teuthology.orchestra.run.smithi053.stdout: "imported": 0, 2023-11-16T16:50:01.080 INFO:teuthology.orchestra.run.smithi053.stdout: "imported_inodes": 0, 2023-11-16T16:50:01.080 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_dir_fetch": 0, 2023-11-16T16:50:01.080 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_backtrace_fetch": 0, 2023-11-16T16:50:01.080 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_peer_discover": 0, 2023-11-16T16:50:01.080 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rfiles": 4, 2023-11-16T16:50:01.081 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rbytes": 246, 2023-11-16T16:50:01.081 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rsnaps": 6, 2023-11-16T16:50:01.081 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_fetch": 0, 2023-11-16T16:50:01.081 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_set_tag": 0, 2023-11-16T16:50:01.081 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_repaired": 0, 2023-11-16T16:50:01.081 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_inotable_repaired": 0, 2023-11-16T16:50:01.081 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_inodes": 0, 2023-11-16T16:50:01.081 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_base_inodes": 0, 2023-11-16T16:50:01.082 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dirfrag_rstats": 0, 2023-11-16T16:50:01.082 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_file_inodes": 0, 2023-11-16T16:50:01.082 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_inode_file_caps": 0, 2023-11-16T16:50:01.082 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_revoke": 0, 2023-11-16T16:50:01.082 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_grant": 0, 2023-11-16T16:50:01.082 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_trunc": 0, 2023-11-16T16:50:01.082 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-16T16:50:01.082 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-16T16:50:01.083 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps": 0, 2023-11-16T16:50:01.083 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps_dirty": 0, 2023-11-16T16:50:01.083 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_cap_release": 0, 2023-11-16T16:50:01.083 INFO:teuthology.orchestra.run.smithi053.stdout: "process_request_cap_release": 0 2023-11-16T16:50:01.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.083 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache": { 2023-11-16T16:50:01.083 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update": 0, 2023-11-16T16:50:01.083 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update_receipt": 0, 2023-11-16T16:50:01.084 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_try_discover": 0, 2023-11-16T16:50:01.084 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_send_discover": 0, 2023-11-16T16:50:01.084 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_handle_discover": 0, 2023-11-16T16:50:01.084 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays": 0, 2023-11-16T16:50:01.084 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_delayed": 0, 2023-11-16T16:50:01.084 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_enqueuing": 0, 2023-11-16T16:50:01.084 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_created": 0, 2023-11-16T16:50:01.084 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_enqueued": 0, 2023-11-16T16:50:01.084 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_reintegrated": 0, 2023-11-16T16:50:01.085 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_migrated": 0, 2023-11-16T16:50:01.085 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_processing": 0, 2023-11-16T16:50:01.085 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_enqueued": 0, 2023-11-16T16:50:01.085 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_prioritized": 0, 2023-11-16T16:50:01.085 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_started": 0, 2023-11-16T16:50:01.085 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_completed": 0, 2023-11-16T16:50:01.085 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_enqueue_scrub": 0, 2023-11-16T16:50:01.086 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_exportdir": 0, 2023-11-16T16:50:01.086 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_flush": 0, 2023-11-16T16:50:01.086 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragmentdir": 0, 2023-11-16T16:50:01.086 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragstats": 0, 2023-11-16T16:50:01.086 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_inodestats": 0 2023-11-16T16:50:01.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.086 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log": { 2023-11-16T16:50:01.086 INFO:teuthology.orchestra.run.smithi053.stdout: "evadd": 0, 2023-11-16T16:50:01.087 INFO:teuthology.orchestra.run.smithi053.stdout: "evex": 0, 2023-11-16T16:50:01.087 INFO:teuthology.orchestra.run.smithi053.stdout: "evtrm": 0, 2023-11-16T16:50:01.087 INFO:teuthology.orchestra.run.smithi053.stdout: "ev": 0, 2023-11-16T16:50:01.087 INFO:teuthology.orchestra.run.smithi053.stdout: "evexg": 0, 2023-11-16T16:50:01.087 INFO:teuthology.orchestra.run.smithi053.stdout: "evexd": 0, 2023-11-16T16:50:01.087 INFO:teuthology.orchestra.run.smithi053.stdout: "segadd": 0, 2023-11-16T16:50:01.087 INFO:teuthology.orchestra.run.smithi053.stdout: "segex": 0, 2023-11-16T16:50:01.087 INFO:teuthology.orchestra.run.smithi053.stdout: "segtrm": 0, 2023-11-16T16:50:01.088 INFO:teuthology.orchestra.run.smithi053.stdout: "seg": 118, 2023-11-16T16:50:01.088 INFO:teuthology.orchestra.run.smithi053.stdout: "segexg": 0, 2023-11-16T16:50:01.088 INFO:teuthology.orchestra.run.smithi053.stdout: "segexd": 0, 2023-11-16T16:50:01.088 INFO:teuthology.orchestra.run.smithi053.stdout: "expos": 4194304, 2023-11-16T16:50:01.088 INFO:teuthology.orchestra.run.smithi053.stdout: "wrpos": 0, 2023-11-16T16:50:01.088 INFO:teuthology.orchestra.run.smithi053.stdout: "rdpos": 276133994, 2023-11-16T16:50:01.088 INFO:teuthology.orchestra.run.smithi053.stdout: "jlat": { 2023-11-16T16:50:01.088 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.088 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.089 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.089 INFO:teuthology.orchestra.run.smithi053.stdout: "replayed": 86355 2023-11-16T16:50:01.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.089 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_mem": { 2023-11-16T16:50:01.089 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 32621, 2023-11-16T16:50:01.089 INFO:teuthology.orchestra.run.smithi053.stdout: "ino+": 74517, 2023-11-16T16:50:01.090 INFO:teuthology.orchestra.run.smithi053.stdout: "ino-": 41896, 2023-11-16T16:50:01.090 INFO:teuthology.orchestra.run.smithi053.stdout: "dir": 1051, 2023-11-16T16:50:01.090 INFO:teuthology.orchestra.run.smithi053.stdout: "dir+": 2046, 2023-11-16T16:50:01.090 INFO:teuthology.orchestra.run.smithi053.stdout: "dir-": 995, 2023-11-16T16:50:01.090 INFO:teuthology.orchestra.run.smithi053.stdout: "dn": 68329, 2023-11-16T16:50:01.090 INFO:teuthology.orchestra.run.smithi053.stdout: "dn+": 104327, 2023-11-16T16:50:01.090 INFO:teuthology.orchestra.run.smithi053.stdout: "dn-": 35998, 2023-11-16T16:50:01.090 INFO:teuthology.orchestra.run.smithi053.stdout: "cap": 0, 2023-11-16T16:50:01.091 INFO:teuthology.orchestra.run.smithi053.stdout: "cap+": 0, 2023-11-16T16:50:01.091 INFO:teuthology.orchestra.run.smithi053.stdout: "cap-": 0, 2023-11-16T16:50:01.091 INFO:teuthology.orchestra.run.smithi053.stdout: "rss": 193668, 2023-11-16T16:50:01.091 INFO:teuthology.orchestra.run.smithi053.stdout: "heap": 207156 2023-11-16T16:50:01.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.091 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_server": { 2023-11-16T16:50:01.091 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_client_request": 0, 2023-11-16T16:50:01.091 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_server_request": 0, 2023-11-16T16:50:01.092 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_request": 0, 2023-11-16T16:50:01.092 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_session": 0, 2023-11-16T16:50:01.092 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_peer_request": 0, 2023-11-16T16:50:01.092 INFO:teuthology.orchestra.run.smithi053.stdout: "req_create_latency": { 2023-11-16T16:50:01.092 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.092 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.092 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.093 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getattr_latency": { 2023-11-16T16:50:01.093 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.093 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.093 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.093 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getfilelock_latency": { 2023-11-16T16:50:01.093 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.093 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.093 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.094 INFO:teuthology.orchestra.run.smithi053.stdout: "req_link_latency": { 2023-11-16T16:50:01.094 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.094 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.094 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.094 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookup_latency": { 2023-11-16T16:50:01.095 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.095 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.095 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.095 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookuphash_latency": { 2023-11-16T16:50:01.095 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.095 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.095 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.096 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupino_latency": { 2023-11-16T16:50:01.096 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.096 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.096 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.096 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupname_latency": { 2023-11-16T16:50:01.096 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.096 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.097 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.097 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupparent_latency": { 2023-11-16T16:50:01.097 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.097 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.097 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.097 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupsnap_latency": { 2023-11-16T16:50:01.097 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.098 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.098 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.098 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lssnap_latency": { 2023-11-16T16:50:01.098 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.098 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.098 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.099 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mkdir_latency": { 2023-11-16T16:50:01.099 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.099 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.099 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.099 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mknod_latency": { 2023-11-16T16:50:01.099 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.100 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.100 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.100 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mksnap_latency": { 2023-11-16T16:50:01.100 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.100 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.100 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.101 INFO:teuthology.orchestra.run.smithi053.stdout: "req_open_latency": { 2023-11-16T16:50:01.101 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.101 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.101 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.101 INFO:teuthology.orchestra.run.smithi053.stdout: "req_readdir_latency": { 2023-11-16T16:50:01.101 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.102 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.102 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.102 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rename_latency": { 2023-11-16T16:50:01.102 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.102 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.102 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.103 INFO:teuthology.orchestra.run.smithi053.stdout: "req_renamesnap_latency": { 2023-11-16T16:50:01.103 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.103 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.103 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.103 INFO:teuthology.orchestra.run.smithi053.stdout: "req_snapdiff_latency": { 2023-11-16T16:50:01.103 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.103 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.104 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.104 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmdir_latency": { 2023-11-16T16:50:01.104 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.104 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.104 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.104 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmsnap_latency": { 2023-11-16T16:50:01.104 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.105 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.105 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.105 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmxattr_latency": { 2023-11-16T16:50:01.105 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.105 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.105 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.106 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setattr_latency": { 2023-11-16T16:50:01.106 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.106 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.106 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.106 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setdirlayout_latency": { 2023-11-16T16:50:01.106 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.106 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.107 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.107 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setfilelock_latency": { 2023-11-16T16:50:01.107 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.107 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.107 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.107 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setlayout_latency": { 2023-11-16T16:50:01.108 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.108 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.108 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.108 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setxattr_latency": { 2023-11-16T16:50:01.108 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.108 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.108 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.109 INFO:teuthology.orchestra.run.smithi053.stdout: "req_symlink_latency": { 2023-11-16T16:50:01.109 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.109 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.109 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.109 INFO:teuthology.orchestra.run.smithi053.stdout: "req_unlink_latency": { 2023-11-16T16:50:01.109 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.110 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.110 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.110 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_revoke_eviction": 0, 2023-11-16T16:50:01.110 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_acquisition_throttle": 0, 2023-11-16T16:50:01.110 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getvxattr_latency": { 2023-11-16T16:50:01.110 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.110 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.111 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.111 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-16T16:50:01.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.111 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_sessions": { 2023-11-16T16:50:01.111 INFO:teuthology.orchestra.run.smithi053.stdout: "session_count": 2, 2023-11-16T16:50:01.111 INFO:teuthology.orchestra.run.smithi053.stdout: "session_add": 5, 2023-11-16T16:50:01.111 INFO:teuthology.orchestra.run.smithi053.stdout: "session_remove": 3, 2023-11-16T16:50:01.111 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_open": 4, 2023-11-16T16:50:01.112 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_stale": 0, 2023-11-16T16:50:01.112 INFO:teuthology.orchestra.run.smithi053.stdout: "total_load": 0, 2023-11-16T16:50:01.112 INFO:teuthology.orchestra.run.smithi053.stdout: "average_load": 0, 2023-11-16T16:50:01.112 INFO:teuthology.orchestra.run.smithi053.stdout: "avg_session_uptime": 3642 2023-11-16T16:50:01.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.112 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2023-11-16T16:50:01.112 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 0, 2023-11-16T16:50:01.112 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 0, 2023-11-16T16:50:01.112 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2023-11-16T16:50:01.113 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2023-11-16T16:50:01.113 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2023-11-16T16:50:01.113 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2023-11-16T16:50:01.113 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-16T16:50:01.113 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2023-11-16T16:50:01.113 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-16T16:50:01.113 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2023-11-16T16:50:01.113 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2023-11-16T16:50:01.114 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2023-11-16T16:50:01.114 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-16T16:50:01.114 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2023-11-16T16:50:01.114 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2023-11-16T16:50:01.114 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2023-11-16T16:50:01.114 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2023-11-16T16:50:01.114 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2023-11-16T16:50:01.115 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-16T16:50:01.115 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2023-11-16T16:50:01.115 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-16T16:50:01.115 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2023-11-16T16:50:01.115 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2023-11-16T16:50:01.115 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2023-11-16T16:50:01.115 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2023-11-16T16:50:01.115 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2023-11-16T16:50:01.115 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-16T16:50:01.116 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2023-11-16T16:50:01.116 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2023-11-16T16:50:01.116 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2023-11-16T16:50:01.116 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2023-11-16T16:50:01.116 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2023-11-16T16:50:01.116 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2023-11-16T16:50:01.116 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2023-11-16T16:50:01.116 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2023-11-16T16:50:01.117 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2023-11-16T16:50:01.117 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 1348863, 2023-11-16T16:50:01.117 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 32708, 2023-11-16T16:50:01.117 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 0, 2023-11-16T16:50:01.117 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 0, 2023-11-16T16:50:01.117 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2023-11-16T16:50:01.117 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2023-11-16T16:50:01.118 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2023-11-16T16:50:01.118 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2023-11-16T16:50:01.118 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2023-11-16T16:50:01.118 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2023-11-16T16:50:01.118 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 8128, 2023-11-16T16:50:01.118 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 169, 2023-11-16T16:50:01.118 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2023-11-16T16:50:01.118 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2023-11-16T16:50:01.119 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2023-11-16T16:50:01.119 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2023-11-16T16:50:01.119 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 121607742, 2023-11-16T16:50:01.119 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 981735, 2023-11-16T16:50:01.119 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2023-11-16T16:50:01.119 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2023-11-16T16:50:01.119 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2023-11-16T16:50:01.119 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2023-11-16T16:50:01.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.120 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter": { 2023-11-16T16:50:01.120 INFO:teuthology.orchestra.run.smithi053.stdout: "op_active": 0, 2023-11-16T16:50:01.120 INFO:teuthology.orchestra.run.smithi053.stdout: "op_laggy": 0, 2023-11-16T16:50:01.120 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send": 2837, 2023-11-16T16:50:01.120 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send_bytes": 26, 2023-11-16T16:50:01.120 INFO:teuthology.orchestra.run.smithi053.stdout: "op_resend": 4, 2023-11-16T16:50:01.120 INFO:teuthology.orchestra.run.smithi053.stdout: "op_reply": 2837, 2023-11-16T16:50:01.120 INFO:teuthology.orchestra.run.smithi053.stdout: "op_latency": { 2023-11-16T16:50:01.121 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2837, 2023-11-16T16:50:01.121 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 1.828964325, 2023-11-16T16:50:01.121 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000644682 2023-11-16T16:50:01.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.121 INFO:teuthology.orchestra.run.smithi053.stdout: "op_inflight": 0, 2023-11-16T16:50:01.121 INFO:teuthology.orchestra.run.smithi053.stdout: "oplen_avg": { 2023-11-16T16:50:01.121 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2837, 2023-11-16T16:50:01.121 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 2838 2023-11-16T16:50:01.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.122 INFO:teuthology.orchestra.run.smithi053.stdout: "op": 2837, 2023-11-16T16:50:01.122 INFO:teuthology.orchestra.run.smithi053.stdout: "op_r": 2837, 2023-11-16T16:50:01.122 INFO:teuthology.orchestra.run.smithi053.stdout: "op_w": 0, 2023-11-16T16:50:01.122 INFO:teuthology.orchestra.run.smithi053.stdout: "op_rmw": 0, 2023-11-16T16:50:01.122 INFO:teuthology.orchestra.run.smithi053.stdout: "op_pg": 0, 2023-11-16T16:50:01.122 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_stat": 1766, 2023-11-16T16:50:01.122 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_create": 0, 2023-11-16T16:50:01.123 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_read": 1068, 2023-11-16T16:50:01.123 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_write": 0, 2023-11-16T16:50:01.123 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writefull": 0, 2023-11-16T16:50:01.123 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writesame": 0, 2023-11-16T16:50:01.123 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_append": 0, 2023-11-16T16:50:01.123 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_zero": 0, 2023-11-16T16:50:01.123 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_truncate": 0, 2023-11-16T16:50:01.123 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_delete": 0, 2023-11-16T16:50:01.123 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_mapext": 0, 2023-11-16T16:50:01.124 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_sparse_read": 0, 2023-11-16T16:50:01.124 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_clonerange": 0, 2023-11-16T16:50:01.124 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_getxattr": 2, 2023-11-16T16:50:01.124 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_setxattr": 0, 2023-11-16T16:50:01.124 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_cmpxattr": 0, 2023-11-16T16:50:01.124 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_rmxattr": 0, 2023-11-16T16:50:01.124 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_resetxattrs": 0, 2023-11-16T16:50:01.124 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_call": 0, 2023-11-16T16:50:01.125 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_watch": 0, 2023-11-16T16:50:01.125 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_notify": 0, 2023-11-16T16:50:01.125 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_src_cmpxattr": 0, 2023-11-16T16:50:01.125 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls": 0, 2023-11-16T16:50:01.125 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls_filter": 0, 2023-11-16T16:50:01.125 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_other": 0, 2023-11-16T16:50:01.125 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_active": 0, 2023-11-16T16:50:01.125 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_send": 0, 2023-11-16T16:50:01.126 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_resend": 0, 2023-11-16T16:50:01.126 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_ping": 0, 2023-11-16T16:50:01.126 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_active": 0, 2023-11-16T16:50:01.126 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_send": 0, 2023-11-16T16:50:01.126 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_resend": 0, 2023-11-16T16:50:01.126 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_active": 0, 2023-11-16T16:50:01.126 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_send": 0, 2023-11-16T16:50:01.126 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_resend": 0, 2023-11-16T16:50:01.127 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_active": 0, 2023-11-16T16:50:01.127 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_send": 0, 2023-11-16T16:50:01.127 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_resend": 0, 2023-11-16T16:50:01.127 INFO:teuthology.orchestra.run.smithi053.stdout: "command_active": 0, 2023-11-16T16:50:01.127 INFO:teuthology.orchestra.run.smithi053.stdout: "command_send": 0, 2023-11-16T16:50:01.127 INFO:teuthology.orchestra.run.smithi053.stdout: "command_resend": 0, 2023-11-16T16:50:01.127 INFO:teuthology.orchestra.run.smithi053.stdout: "map_epoch": 95, 2023-11-16T16:50:01.127 INFO:teuthology.orchestra.run.smithi053.stdout: "map_full": 0, 2023-11-16T16:50:01.128 INFO:teuthology.orchestra.run.smithi053.stdout: "map_inc": 11, 2023-11-16T16:50:01.128 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_sessions": 11, 2023-11-16T16:50:01.128 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_open": 11, 2023-11-16T16:50:01.128 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_close": 0, 2023-11-16T16:50:01.128 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_laggy": 0, 2023-11-16T16:50:01.128 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_wr": 0, 2023-11-16T16:50:01.128 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_rd": 2, 2023-11-16T16:50:01.128 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_del": 0 2023-11-16T16:50:01.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.129 INFO:teuthology.orchestra.run.smithi053.stdout: "oft": { 2023-11-16T16:50:01.129 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_objs": 0, 2023-11-16T16:50:01.129 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_kv_pairs": 0, 2023-11-16T16:50:01.129 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_updates": 0, 2023-11-16T16:50:01.129 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_removes": 0 2023-11-16T16:50:01.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.129 INFO:teuthology.orchestra.run.smithi053.stdout: "osd-slow-ops": { 2023-11-16T16:50:01.130 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_ops_count": 0 2023-11-16T16:50:01.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.130 INFO:teuthology.orchestra.run.smithi053.stdout: "purge_queue": { 2023-11-16T16:50:01.130 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops": 0, 2023-11-16T16:50:01.130 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops_high_water": 0, 2023-11-16T16:50:01.130 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing": 0, 2023-11-16T16:50:01.130 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_high_water": 0, 2023-11-16T16:50:01.131 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed": 0, 2023-11-16T16:50:01.131 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_item_in_journal": 0 2023-11-16T16:50:01.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.131 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-16T16:50:01.131 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2023-11-16T16:50:01.131 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2023-11-16T16:50:01.131 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2023-11-16T16:50:01.131 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 3086, 2023-11-16T16:50:01.132 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 274059634, 2023-11-16T16:50:01.132 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2023-11-16T16:50:01.132 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 3086, 2023-11-16T16:50:01.132 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2023-11-16T16:50:01.132 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2023-11-16T16:50:01.132 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 3086, 2023-11-16T16:50:01.132 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 274059634, 2023-11-16T16:50:01.132 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2023-11-16T16:50:01.133 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.133 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.133 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.133 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-16T16:50:01.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.133 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_bytes": { 2023-11-16T16:50:01.133 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2023-11-16T16:50:01.133 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2023-11-16T16:50:01.133 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2023-11-16T16:50:01.134 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2023-11-16T16:50:01.134 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2023-11-16T16:50:01.134 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2023-11-16T16:50:01.134 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2023-11-16T16:50:01.134 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 2837, 2023-11-16T16:50:01.134 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 272081690, 2023-11-16T16:50:01.134 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 2837, 2023-11-16T16:50:01.134 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 272081690, 2023-11-16T16:50:01.135 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2023-11-16T16:50:01.135 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.135 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.135 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.135 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-16T16:50:01.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.135 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_ops": { 2023-11-16T16:50:01.135 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2023-11-16T16:50:01.136 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 1024, 2023-11-16T16:50:01.136 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2023-11-16T16:50:01.136 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2023-11-16T16:50:01.136 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2023-11-16T16:50:01.136 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2023-11-16T16:50:01.136 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2023-11-16T16:50:01.136 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 2837, 2023-11-16T16:50:01.136 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 2837, 2023-11-16T16:50:01.137 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 2837, 2023-11-16T16:50:01.137 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 2837, 2023-11-16T16:50:01.137 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2023-11-16T16:50:01.137 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.137 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.137 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.137 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-16T16:50:01.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.137 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle": { 2023-11-16T16:50:01.138 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2023-11-16T16:50:01.138 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2023-11-16T16:50:01.138 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2023-11-16T16:50:01.138 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2023-11-16T16:50:01.138 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2023-11-16T16:50:01.138 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2023-11-16T16:50:01.138 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2023-11-16T16:50:01.139 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2023-11-16T16:50:01.139 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2023-11-16T16:50:01.139 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 0, 2023-11-16T16:50:01.139 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 0, 2023-11-16T16:50:01.139 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2023-11-16T16:50:01.139 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.139 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.139 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.140 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-16T16:50:01.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:01.140 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle-0x563ee55ae000": { 2023-11-16T16:50:01.140 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2023-11-16T16:50:01.140 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2023-11-16T16:50:01.140 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2023-11-16T16:50:01.140 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2023-11-16T16:50:01.140 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2023-11-16T16:50:01.141 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2023-11-16T16:50:01.141 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2023-11-16T16:50:01.141 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2023-11-16T16:50:01.141 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2023-11-16T16:50:01.141 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 0, 2023-11-16T16:50:01.141 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 0, 2023-11-16T16:50:01.141 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2023-11-16T16:50:01.141 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:01.141 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:01.142 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:01.142 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-16T16:50:01.142 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-16T16:50:01.142 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-16T16:50:01.142 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:00 smithi053 ceph-mon[100638]: overall HEALTH_OK 2023-11-16T16:50:01.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:00 smithi001 ceph-mon[98800]: overall HEALTH_OK 2023-11-16T16:50:01.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:00 smithi026 ceph-mon[100112]: overall HEALTH_OK 2023-11-16T16:50:01.418 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2023-11-16T16:50:01.418 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2023-11-16T16:50:01.418 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2023-11-16T16:50:01.419 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2023-11-16T16:50:01.419 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2023-11-16T16:50:01.419 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2023-11-16T16:50:01.419 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2023-11-16T16:50:01.419 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2023-11-16T16:50:01.419 DEBUG:tasks.check_counter:Getting stats from f 2023-11-16T16:50:01.419 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph daemon mds.f perf dump 2023-11-16T16:50:02.070 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-16T16:50:02.580 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:02 smithi053 ceph-mon[100638]: pgmap v337: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:50:02.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:02 smithi001 ceph-mon[98800]: pgmap v337: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:50:02.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:02 smithi026 ceph-mon[100112]: pgmap v337: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-16T16:50:03.273 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-16T16:50:03.273 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2023-11-16T16:50:03.273 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 474, 2023-11-16T16:50:03.273 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 454, 2023-11-16T16:50:03.273 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 64472640, 2023-11-16T16:50:03.274 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 193707, 2023-11-16T16:50:03.274 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 7, 2023-11-16T16:50:03.274 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 3, 2023-11-16T16:50:03.274 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.089916228, 2023-11-16T16:50:03.274 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.018295844, 2023-11-16T16:50:03.274 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.066337484, 2023-11-16T16:50:03.274 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.011925234, 2023-11-16T16:50:03.274 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:50:03.275 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 453, 2023-11-16T16:50:03.275 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.018701005, 2023-11-16T16:50:03.275 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000041282 2023-11-16T16:50:03.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.275 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:50:03.275 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.275 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.275 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.276 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 1482736, 2023-11-16T16:50:03.276 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 123104 2023-11-16T16:50:03.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.276 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2023-11-16T16:50:03.276 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 1070, 2023-11-16T16:50:03.276 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 1228, 2023-11-16T16:50:03.277 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 94292831, 2023-11-16T16:50:03.277 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 765069, 2023-11-16T16:50:03.277 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 8, 2023-11-16T16:50:03.277 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 6, 2023-11-16T16:50:03.277 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.140424555, 2023-11-16T16:50:03.277 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.028709224, 2023-11-16T16:50:03.277 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.066885173, 2023-11-16T16:50:03.277 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.034422808, 2023-11-16T16:50:03.277 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:50:03.278 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1226, 2023-11-16T16:50:03.278 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.056236835, 2023-11-16T16:50:03.278 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000045870 2023-11-16T16:50:03.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.278 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:50:03.278 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.278 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.279 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.279 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 3312, 2023-11-16T16:50:03.279 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 434160 2023-11-16T16:50:03.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.279 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2023-11-16T16:50:03.279 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 1422, 2023-11-16T16:50:03.280 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 1424, 2023-11-16T16:50:03.280 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 107899475, 2023-11-16T16:50:03.280 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 442300, 2023-11-16T16:50:03.280 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 6, 2023-11-16T16:50:03.280 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 5, 2023-11-16T16:50:03.280 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.186560737, 2023-11-16T16:50:03.280 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.038844595, 2023-11-16T16:50:03.280 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.082914575, 2023-11-16T16:50:03.281 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.050748704, 2023-11-16T16:50:03.281 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:50:03.281 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1422, 2023-11-16T16:50:03.281 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.058069061, 2023-11-16T16:50:03.281 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000040836 2023-11-16T16:50:03.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.281 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:50:03.281 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.282 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.282 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.282 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-16T16:50:03.282 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 0 2023-11-16T16:50:03.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.282 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2023-11-16T16:50:03.283 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 1, 2023-11-16T16:50:03.283 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2023-11-16T16:50:03.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.283 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-MDSRank": { 2023-11-16T16:50:03.283 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2023-11-16T16:50:03.283 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2023-11-16T16:50:03.283 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 4342, 2023-11-16T16:50:03.283 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.130610155, 2023-11-16T16:50:03.283 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000030080 2023-11-16T16:50:03.284 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-16T16:50:03.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.284 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-PurgeQueue": { 2023-11-16T16:50:03.284 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2023-11-16T16:50:03.284 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2023-11-16T16:50:03.284 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.284 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.285 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.285 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-16T16:50:03.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.285 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": { 2023-11-16T16:50:03.285 INFO:teuthology.orchestra.run.smithi053.stdout: "request": 0, 2023-11-16T16:50:03.285 INFO:teuthology.orchestra.run.smithi053.stdout: "reply": 0, 2023-11-16T16:50:03.285 INFO:teuthology.orchestra.run.smithi053.stdout: "reply_latency": { 2023-11-16T16:50:03.285 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.286 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.286 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.286 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_reply": 0, 2023-11-16T16:50:03.286 INFO:teuthology.orchestra.run.smithi053.stdout: "forward": 0, 2023-11-16T16:50:03.286 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_complete": 0, 2023-11-16T16:50:03.286 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_keys": 0, 2023-11-16T16:50:03.287 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_commit": 0, 2023-11-16T16:50:03.287 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_split": 0, 2023-11-16T16:50:03.287 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_merge": 0, 2023-11-16T16:50:03.287 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes": 41367, 2023-11-16T16:50:03.287 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_top": 1463, 2023-11-16T16:50:03.287 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_bottom": 39150, 2023-11-16T16:50:03.287 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pin_tail": 754, 2023-11-16T16:50:03.287 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pinned": 39276, 2023-11-16T16:50:03.288 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_expired": 0, 2023-11-16T16:50:03.288 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_with_caps": 0, 2023-11-16T16:50:03.288 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": 0, 2023-11-16T16:50:03.288 INFO:teuthology.orchestra.run.smithi053.stdout: "subtrees": 1, 2023-11-16T16:50:03.288 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse": 0, 2023-11-16T16:50:03.288 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_hit": 0, 2023-11-16T16:50:03.288 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_forward": 0, 2023-11-16T16:50:03.288 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_discover": 0, 2023-11-16T16:50:03.289 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_dir_fetch": 0, 2023-11-16T16:50:03.289 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_remote_ino": 0, 2023-11-16T16:50:03.289 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_lock": 0, 2023-11-16T16:50:03.289 INFO:teuthology.orchestra.run.smithi053.stdout: "load_cent": 0, 2023-11-16T16:50:03.289 INFO:teuthology.orchestra.run.smithi053.stdout: "q": 0, 2023-11-16T16:50:03.289 INFO:teuthology.orchestra.run.smithi053.stdout: "exported": 0, 2023-11-16T16:50:03.289 INFO:teuthology.orchestra.run.smithi053.stdout: "exported_inodes": 0, 2023-11-16T16:50:03.289 INFO:teuthology.orchestra.run.smithi053.stdout: "imported": 0, 2023-11-16T16:50:03.290 INFO:teuthology.orchestra.run.smithi053.stdout: "imported_inodes": 0, 2023-11-16T16:50:03.290 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_dir_fetch": 0, 2023-11-16T16:50:03.290 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_backtrace_fetch": 0, 2023-11-16T16:50:03.290 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_peer_discover": 0, 2023-11-16T16:50:03.290 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rfiles": 0, 2023-11-16T16:50:03.290 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rbytes": 0, 2023-11-16T16:50:03.290 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rsnaps": 0, 2023-11-16T16:50:03.290 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_fetch": 0, 2023-11-16T16:50:03.291 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_set_tag": 0, 2023-11-16T16:50:03.291 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_repaired": 0, 2023-11-16T16:50:03.291 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_inotable_repaired": 0, 2023-11-16T16:50:03.291 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_inodes": 0, 2023-11-16T16:50:03.291 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_base_inodes": 0, 2023-11-16T16:50:03.291 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dirfrag_rstats": 0, 2023-11-16T16:50:03.291 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_file_inodes": 0, 2023-11-16T16:50:03.292 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_inode_file_caps": 0, 2023-11-16T16:50:03.292 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_revoke": 0, 2023-11-16T16:50:03.292 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_grant": 0, 2023-11-16T16:50:03.292 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_trunc": 0, 2023-11-16T16:50:03.292 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-16T16:50:03.292 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-16T16:50:03.292 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps": 0, 2023-11-16T16:50:03.292 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps_dirty": 0, 2023-11-16T16:50:03.293 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_cap_release": 0, 2023-11-16T16:50:03.293 INFO:teuthology.orchestra.run.smithi053.stdout: "process_request_cap_release": 0 2023-11-16T16:50:03.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.293 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache": { 2023-11-16T16:50:03.293 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update": 0, 2023-11-16T16:50:03.293 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update_receipt": 0, 2023-11-16T16:50:03.293 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_try_discover": 0, 2023-11-16T16:50:03.293 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_send_discover": 0, 2023-11-16T16:50:03.294 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_handle_discover": 0, 2023-11-16T16:50:03.294 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays": 0, 2023-11-16T16:50:03.294 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_delayed": 0, 2023-11-16T16:50:03.294 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_enqueuing": 0, 2023-11-16T16:50:03.294 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_created": 0, 2023-11-16T16:50:03.294 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_enqueued": 0, 2023-11-16T16:50:03.294 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_reintegrated": 0, 2023-11-16T16:50:03.295 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_migrated": 0, 2023-11-16T16:50:03.295 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_processing": 0, 2023-11-16T16:50:03.295 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_enqueued": 0, 2023-11-16T16:50:03.295 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_prioritized": 0, 2023-11-16T16:50:03.295 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_started": 0, 2023-11-16T16:50:03.295 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_completed": 0, 2023-11-16T16:50:03.295 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_enqueue_scrub": 0, 2023-11-16T16:50:03.295 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_exportdir": 0, 2023-11-16T16:50:03.295 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_flush": 0, 2023-11-16T16:50:03.296 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragmentdir": 0, 2023-11-16T16:50:03.296 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragstats": 0, 2023-11-16T16:50:03.296 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_inodestats": 0 2023-11-16T16:50:03.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.296 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log": { 2023-11-16T16:50:03.296 INFO:teuthology.orchestra.run.smithi053.stdout: "evadd": 0, 2023-11-16T16:50:03.296 INFO:teuthology.orchestra.run.smithi053.stdout: "evex": 0, 2023-11-16T16:50:03.297 INFO:teuthology.orchestra.run.smithi053.stdout: "evtrm": 0, 2023-11-16T16:50:03.297 INFO:teuthology.orchestra.run.smithi053.stdout: "ev": 0, 2023-11-16T16:50:03.297 INFO:teuthology.orchestra.run.smithi053.stdout: "evexg": 0, 2023-11-16T16:50:03.297 INFO:teuthology.orchestra.run.smithi053.stdout: "evexd": 0, 2023-11-16T16:50:03.297 INFO:teuthology.orchestra.run.smithi053.stdout: "segadd": 0, 2023-11-16T16:50:03.297 INFO:teuthology.orchestra.run.smithi053.stdout: "segex": 0, 2023-11-16T16:50:03.297 INFO:teuthology.orchestra.run.smithi053.stdout: "segtrm": 0, 2023-11-16T16:50:03.297 INFO:teuthology.orchestra.run.smithi053.stdout: "seg": 97, 2023-11-16T16:50:03.298 INFO:teuthology.orchestra.run.smithi053.stdout: "segexg": 0, 2023-11-16T16:50:03.298 INFO:teuthology.orchestra.run.smithi053.stdout: "segexd": 0, 2023-11-16T16:50:03.298 INFO:teuthology.orchestra.run.smithi053.stdout: "expos": 4194304, 2023-11-16T16:50:03.298 INFO:teuthology.orchestra.run.smithi053.stdout: "wrpos": 0, 2023-11-16T16:50:03.298 INFO:teuthology.orchestra.run.smithi053.stdout: "rdpos": 265233908, 2023-11-16T16:50:03.298 INFO:teuthology.orchestra.run.smithi053.stdout: "jlat": { 2023-11-16T16:50:03.298 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.298 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.299 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.299 INFO:teuthology.orchestra.run.smithi053.stdout: "replayed": 63477 2023-11-16T16:50:03.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.299 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_mem": { 2023-11-16T16:50:03.299 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 15335, 2023-11-16T16:50:03.299 INFO:teuthology.orchestra.run.smithi053.stdout: "ino+": 51020, 2023-11-16T16:50:03.299 INFO:teuthology.orchestra.run.smithi053.stdout: "ino-": 35685, 2023-11-16T16:50:03.300 INFO:teuthology.orchestra.run.smithi053.stdout: "dir": 523, 2023-11-16T16:50:03.300 INFO:teuthology.orchestra.run.smithi053.stdout: "dir+": 1227, 2023-11-16T16:50:03.300 INFO:teuthology.orchestra.run.smithi053.stdout: "dir-": 704, 2023-11-16T16:50:03.300 INFO:teuthology.orchestra.run.smithi053.stdout: "dn": 41367, 2023-11-16T16:50:03.300 INFO:teuthology.orchestra.run.smithi053.stdout: "dn+": 72801, 2023-11-16T16:50:03.300 INFO:teuthology.orchestra.run.smithi053.stdout: "dn-": 31434, 2023-11-16T16:50:03.300 INFO:teuthology.orchestra.run.smithi053.stdout: "cap": 0, 2023-11-16T16:50:03.300 INFO:teuthology.orchestra.run.smithi053.stdout: "cap+": 0, 2023-11-16T16:50:03.301 INFO:teuthology.orchestra.run.smithi053.stdout: "cap-": 0, 2023-11-16T16:50:03.301 INFO:teuthology.orchestra.run.smithi053.stdout: "rss": 177520, 2023-11-16T16:50:03.301 INFO:teuthology.orchestra.run.smithi053.stdout: "heap": 207156 2023-11-16T16:50:03.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.301 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_server": { 2023-11-16T16:50:03.301 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_client_request": 0, 2023-11-16T16:50:03.301 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_server_request": 0, 2023-11-16T16:50:03.301 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_request": 0, 2023-11-16T16:50:03.302 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_session": 0, 2023-11-16T16:50:03.302 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_peer_request": 0, 2023-11-16T16:50:03.302 INFO:teuthology.orchestra.run.smithi053.stdout: "req_create_latency": { 2023-11-16T16:50:03.302 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.302 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.302 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.303 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getattr_latency": { 2023-11-16T16:50:03.303 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.303 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.303 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.303 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getfilelock_latency": { 2023-11-16T16:50:03.303 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.303 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.304 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.304 INFO:teuthology.orchestra.run.smithi053.stdout: "req_link_latency": { 2023-11-16T16:50:03.304 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.304 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.304 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.304 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookup_latency": { 2023-11-16T16:50:03.305 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.305 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.305 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.305 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookuphash_latency": { 2023-11-16T16:50:03.305 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.305 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.305 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.306 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupino_latency": { 2023-11-16T16:50:03.306 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.306 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.306 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.306 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupname_latency": { 2023-11-16T16:50:03.306 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.307 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.307 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.307 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupparent_latency": { 2023-11-16T16:50:03.307 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.307 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.307 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.308 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupsnap_latency": { 2023-11-16T16:50:03.308 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.308 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.308 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.308 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lssnap_latency": { 2023-11-16T16:50:03.308 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.309 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.309 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.309 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mkdir_latency": { 2023-11-16T16:50:03.309 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.309 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.309 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.310 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mknod_latency": { 2023-11-16T16:50:03.310 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.310 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.310 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.310 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mksnap_latency": { 2023-11-16T16:50:03.310 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.311 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.311 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.311 INFO:teuthology.orchestra.run.smithi053.stdout: "req_open_latency": { 2023-11-16T16:50:03.311 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.311 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.311 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.312 INFO:teuthology.orchestra.run.smithi053.stdout: "req_readdir_latency": { 2023-11-16T16:50:03.312 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.312 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.312 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.312 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rename_latency": { 2023-11-16T16:50:03.312 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.312 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.313 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.313 INFO:teuthology.orchestra.run.smithi053.stdout: "req_renamesnap_latency": { 2023-11-16T16:50:03.313 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.313 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.313 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.313 INFO:teuthology.orchestra.run.smithi053.stdout: "req_snapdiff_latency": { 2023-11-16T16:50:03.314 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.314 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.314 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.314 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmdir_latency": { 2023-11-16T16:50:03.314 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.314 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.314 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.315 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmsnap_latency": { 2023-11-16T16:50:03.315 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.315 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.315 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.315 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmxattr_latency": { 2023-11-16T16:50:03.315 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.316 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.316 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.316 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setattr_latency": { 2023-11-16T16:50:03.316 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.316 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.316 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.317 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setdirlayout_latency": { 2023-11-16T16:50:03.317 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.317 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.317 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.317 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setfilelock_latency": { 2023-11-16T16:50:03.317 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.317 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.318 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.318 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setlayout_latency": { 2023-11-16T16:50:03.318 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.318 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.318 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.319 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setxattr_latency": { 2023-11-16T16:50:03.319 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.319 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.319 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.319 INFO:teuthology.orchestra.run.smithi053.stdout: "req_symlink_latency": { 2023-11-16T16:50:03.319 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.319 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.320 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.320 INFO:teuthology.orchestra.run.smithi053.stdout: "req_unlink_latency": { 2023-11-16T16:50:03.320 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.320 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.320 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.321 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_revoke_eviction": 0, 2023-11-16T16:50:03.321 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_acquisition_throttle": 0, 2023-11-16T16:50:03.321 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getvxattr_latency": { 2023-11-16T16:50:03.321 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.321 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.321 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.321 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-16T16:50:03.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.322 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_sessions": { 2023-11-16T16:50:03.322 INFO:teuthology.orchestra.run.smithi053.stdout: "session_count": 1, 2023-11-16T16:50:03.322 INFO:teuthology.orchestra.run.smithi053.stdout: "session_add": 3, 2023-11-16T16:50:03.322 INFO:teuthology.orchestra.run.smithi053.stdout: "session_remove": 2, 2023-11-16T16:50:03.322 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_open": 3, 2023-11-16T16:50:03.322 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_stale": 0, 2023-11-16T16:50:03.322 INFO:teuthology.orchestra.run.smithi053.stdout: "total_load": 0, 2023-11-16T16:50:03.322 INFO:teuthology.orchestra.run.smithi053.stdout: "average_load": 0, 2023-11-16T16:50:03.323 INFO:teuthology.orchestra.run.smithi053.stdout: "avg_session_uptime": 5142 2023-11-16T16:50:03.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.323 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2023-11-16T16:50:03.323 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 0, 2023-11-16T16:50:03.323 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 0, 2023-11-16T16:50:03.323 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2023-11-16T16:50:03.323 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2023-11-16T16:50:03.323 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2023-11-16T16:50:03.323 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2023-11-16T16:50:03.324 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-16T16:50:03.324 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2023-11-16T16:50:03.324 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-16T16:50:03.324 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2023-11-16T16:50:03.324 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2023-11-16T16:50:03.324 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2023-11-16T16:50:03.324 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-16T16:50:03.324 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2023-11-16T16:50:03.325 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2023-11-16T16:50:03.325 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2023-11-16T16:50:03.325 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2023-11-16T16:50:03.325 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2023-11-16T16:50:03.325 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-16T16:50:03.325 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2023-11-16T16:50:03.325 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-16T16:50:03.325 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2023-11-16T16:50:03.326 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2023-11-16T16:50:03.326 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2023-11-16T16:50:03.326 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2023-11-16T16:50:03.326 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2023-11-16T16:50:03.326 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-16T16:50:03.326 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2023-11-16T16:50:03.326 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2023-11-16T16:50:03.326 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2023-11-16T16:50:03.327 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2023-11-16T16:50:03.327 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2023-11-16T16:50:03.327 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2023-11-16T16:50:03.327 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2023-11-16T16:50:03.327 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2023-11-16T16:50:03.327 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2023-11-16T16:50:03.327 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 752492, 2023-11-16T16:50:03.327 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 15402, 2023-11-16T16:50:03.328 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 0, 2023-11-16T16:50:03.328 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 0, 2023-11-16T16:50:03.328 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2023-11-16T16:50:03.328 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2023-11-16T16:50:03.328 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2023-11-16T16:50:03.328 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2023-11-16T16:50:03.328 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2023-11-16T16:50:03.328 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2023-11-16T16:50:03.329 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 8128, 2023-11-16T16:50:03.329 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 169, 2023-11-16T16:50:03.329 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2023-11-16T16:50:03.329 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2023-11-16T16:50:03.329 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2023-11-16T16:50:03.329 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2023-11-16T16:50:03.329 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 63703139, 2023-11-16T16:50:03.329 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 497740, 2023-11-16T16:50:03.330 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2023-11-16T16:50:03.330 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2023-11-16T16:50:03.330 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2023-11-16T16:50:03.330 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2023-11-16T16:50:03.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.330 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter": { 2023-11-16T16:50:03.330 INFO:teuthology.orchestra.run.smithi053.stdout: "op_active": 0, 2023-11-16T16:50:03.331 INFO:teuthology.orchestra.run.smithi053.stdout: "op_laggy": 0, 2023-11-16T16:50:03.331 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send": 2713, 2023-11-16T16:50:03.331 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send_bytes": 21, 2023-11-16T16:50:03.331 INFO:teuthology.orchestra.run.smithi053.stdout: "op_resend": 3, 2023-11-16T16:50:03.331 INFO:teuthology.orchestra.run.smithi053.stdout: "op_reply": 2713, 2023-11-16T16:50:03.331 INFO:teuthology.orchestra.run.smithi053.stdout: "op_latency": { 2023-11-16T16:50:03.331 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2713, 2023-11-16T16:50:03.331 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 1.744966011, 2023-11-16T16:50:03.332 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000643186 2023-11-16T16:50:03.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.332 INFO:teuthology.orchestra.run.smithi053.stdout: "op_inflight": 0, 2023-11-16T16:50:03.332 INFO:teuthology.orchestra.run.smithi053.stdout: "oplen_avg": { 2023-11-16T16:50:03.332 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2713, 2023-11-16T16:50:03.332 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 2714 2023-11-16T16:50:03.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.332 INFO:teuthology.orchestra.run.smithi053.stdout: "op": 2713, 2023-11-16T16:50:03.333 INFO:teuthology.orchestra.run.smithi053.stdout: "op_r": 2713, 2023-11-16T16:50:03.333 INFO:teuthology.orchestra.run.smithi053.stdout: "op_w": 0, 2023-11-16T16:50:03.333 INFO:teuthology.orchestra.run.smithi053.stdout: "op_rmw": 0, 2023-11-16T16:50:03.333 INFO:teuthology.orchestra.run.smithi053.stdout: "op_pg": 0, 2023-11-16T16:50:03.333 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_stat": 1714, 2023-11-16T16:50:03.333 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_create": 0, 2023-11-16T16:50:03.333 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_read": 997, 2023-11-16T16:50:03.333 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_write": 0, 2023-11-16T16:50:03.334 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writefull": 0, 2023-11-16T16:50:03.334 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writesame": 0, 2023-11-16T16:50:03.334 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_append": 0, 2023-11-16T16:50:03.334 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_zero": 0, 2023-11-16T16:50:03.334 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_truncate": 0, 2023-11-16T16:50:03.334 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_delete": 0, 2023-11-16T16:50:03.334 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_mapext": 0, 2023-11-16T16:50:03.334 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_sparse_read": 0, 2023-11-16T16:50:03.335 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_clonerange": 0, 2023-11-16T16:50:03.335 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_getxattr": 1, 2023-11-16T16:50:03.335 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_setxattr": 0, 2023-11-16T16:50:03.335 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_cmpxattr": 0, 2023-11-16T16:50:03.335 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_rmxattr": 0, 2023-11-16T16:50:03.335 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_resetxattrs": 0, 2023-11-16T16:50:03.335 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_call": 0, 2023-11-16T16:50:03.335 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_watch": 0, 2023-11-16T16:50:03.336 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_notify": 0, 2023-11-16T16:50:03.336 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_src_cmpxattr": 0, 2023-11-16T16:50:03.336 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls": 0, 2023-11-16T16:50:03.336 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls_filter": 0, 2023-11-16T16:50:03.336 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_other": 0, 2023-11-16T16:50:03.336 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_active": 0, 2023-11-16T16:50:03.336 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_send": 0, 2023-11-16T16:50:03.336 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_resend": 0, 2023-11-16T16:50:03.337 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_ping": 0, 2023-11-16T16:50:03.337 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_active": 0, 2023-11-16T16:50:03.337 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_send": 0, 2023-11-16T16:50:03.337 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_resend": 0, 2023-11-16T16:50:03.337 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_active": 0, 2023-11-16T16:50:03.337 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_send": 0, 2023-11-16T16:50:03.337 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_resend": 0, 2023-11-16T16:50:03.338 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_active": 0, 2023-11-16T16:50:03.338 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_send": 0, 2023-11-16T16:50:03.338 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_resend": 0, 2023-11-16T16:50:03.338 INFO:teuthology.orchestra.run.smithi053.stdout: "command_active": 0, 2023-11-16T16:50:03.338 INFO:teuthology.orchestra.run.smithi053.stdout: "command_send": 0, 2023-11-16T16:50:03.338 INFO:teuthology.orchestra.run.smithi053.stdout: "command_resend": 0, 2023-11-16T16:50:03.338 INFO:teuthology.orchestra.run.smithi053.stdout: "map_epoch": 95, 2023-11-16T16:50:03.338 INFO:teuthology.orchestra.run.smithi053.stdout: "map_full": 0, 2023-11-16T16:50:03.339 INFO:teuthology.orchestra.run.smithi053.stdout: "map_inc": 11, 2023-11-16T16:50:03.339 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_sessions": 10, 2023-11-16T16:50:03.339 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_open": 10, 2023-11-16T16:50:03.339 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_close": 0, 2023-11-16T16:50:03.339 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_laggy": 0, 2023-11-16T16:50:03.339 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_wr": 0, 2023-11-16T16:50:03.339 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_rd": 2, 2023-11-16T16:50:03.339 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_del": 0 2023-11-16T16:50:03.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.340 INFO:teuthology.orchestra.run.smithi053.stdout: "oft": { 2023-11-16T16:50:03.340 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_objs": 0, 2023-11-16T16:50:03.340 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_kv_pairs": 0, 2023-11-16T16:50:03.340 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_updates": 0, 2023-11-16T16:50:03.340 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_removes": 0 2023-11-16T16:50:03.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.340 INFO:teuthology.orchestra.run.smithi053.stdout: "osd-slow-ops": { 2023-11-16T16:50:03.340 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_ops_count": 0 2023-11-16T16:50:03.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.341 INFO:teuthology.orchestra.run.smithi053.stdout: "purge_queue": { 2023-11-16T16:50:03.341 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops": 0, 2023-11-16T16:50:03.341 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops_high_water": 0, 2023-11-16T16:50:03.341 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing": 0, 2023-11-16T16:50:03.341 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_high_water": 0, 2023-11-16T16:50:03.341 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed": 0, 2023-11-16T16:50:03.341 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_item_in_journal": 0 2023-11-16T16:50:03.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.342 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-16T16:50:03.342 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2023-11-16T16:50:03.342 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2023-11-16T16:50:03.342 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2023-11-16T16:50:03.342 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 2962, 2023-11-16T16:50:03.342 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 266382702, 2023-11-16T16:50:03.342 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2023-11-16T16:50:03.343 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 2962, 2023-11-16T16:50:03.343 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2023-11-16T16:50:03.343 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2023-11-16T16:50:03.343 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 2962, 2023-11-16T16:50:03.343 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 266382702, 2023-11-16T16:50:03.343 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2023-11-16T16:50:03.343 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.343 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.344 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.344 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-16T16:50:03.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.344 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_bytes": { 2023-11-16T16:50:03.344 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2023-11-16T16:50:03.344 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2023-11-16T16:50:03.344 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2023-11-16T16:50:03.344 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2023-11-16T16:50:03.345 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2023-11-16T16:50:03.345 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2023-11-16T16:50:03.345 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2023-11-16T16:50:03.345 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 2713, 2023-11-16T16:50:03.345 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 264424132, 2023-11-16T16:50:03.345 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 2713, 2023-11-16T16:50:03.345 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 264424132, 2023-11-16T16:50:03.345 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2023-11-16T16:50:03.345 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.346 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.346 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.346 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-16T16:50:03.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.346 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_ops": { 2023-11-16T16:50:03.346 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2023-11-16T16:50:03.346 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 1024, 2023-11-16T16:50:03.346 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2023-11-16T16:50:03.347 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2023-11-16T16:50:03.347 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2023-11-16T16:50:03.347 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2023-11-16T16:50:03.347 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2023-11-16T16:50:03.347 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 2713, 2023-11-16T16:50:03.347 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 2713, 2023-11-16T16:50:03.347 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 2713, 2023-11-16T16:50:03.347 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 2713, 2023-11-16T16:50:03.347 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2023-11-16T16:50:03.348 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.348 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.348 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.348 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-16T16:50:03.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.348 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle": { 2023-11-16T16:50:03.348 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2023-11-16T16:50:03.348 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2023-11-16T16:50:03.349 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2023-11-16T16:50:03.349 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2023-11-16T16:50:03.349 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2023-11-16T16:50:03.349 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2023-11-16T16:50:03.349 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2023-11-16T16:50:03.349 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2023-11-16T16:50:03.349 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2023-11-16T16:50:03.349 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 0, 2023-11-16T16:50:03.350 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 0, 2023-11-16T16:50:03.350 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2023-11-16T16:50:03.350 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.350 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.350 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.350 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-16T16:50:03.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:03.351 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle-0x55fb9b1a2000": { 2023-11-16T16:50:03.351 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2023-11-16T16:50:03.351 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2023-11-16T16:50:03.351 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2023-11-16T16:50:03.351 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2023-11-16T16:50:03.351 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2023-11-16T16:50:03.351 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2023-11-16T16:50:03.351 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2023-11-16T16:50:03.351 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2023-11-16T16:50:03.352 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2023-11-16T16:50:03.352 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 0, 2023-11-16T16:50:03.352 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 0, 2023-11-16T16:50:03.352 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2023-11-16T16:50:03.352 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:03.352 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:03.352 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:03.352 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-16T16:50:03.353 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-16T16:50:03.353 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-16T16:50:03.719 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2023-11-16T16:50:03.719 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2023-11-16T16:50:03.719 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2023-11-16T16:50:03.719 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2023-11-16T16:50:03.719 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2023-11-16T16:50:03.719 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2023-11-16T16:50:03.720 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2023-11-16T16:50:03.720 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2023-11-16T16:50:03.720 DEBUG:tasks.check_counter:Getting stats from i 2023-11-16T16:50:03.720 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph daemon mds.i perf dump 2023-11-16T16:50:04.081 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:03 smithi053 ceph-mon[100638]: pgmap v338: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:50:04.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:03 smithi001 ceph-mon[98800]: pgmap v338: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:50:04.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:03 smithi026 ceph-mon[100112]: pgmap v338: 97 pgs: 97 active+clean; 817 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-16T16:50:04.371 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-16T16:50:05.577 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-16T16:50:05.626 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2023-11-16T16:50:05.626 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 35614, 2023-11-16T16:50:05.626 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 31092, 2023-11-16T16:50:05.626 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 19761211, 2023-11-16T16:50:05.626 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 200893784, 2023-11-16T16:50:05.627 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 28, 2023-11-16T16:50:05.627 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 6, 2023-11-16T16:50:05.627 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 2.089691972, 2023-11-16T16:50:05.627 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.895834710, 2023-11-16T16:50:05.627 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.985385059, 2023-11-16T16:50:05.627 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.312959743, 2023-11-16T16:50:05.627 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:50:05.627 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 31089, 2023-11-16T16:50:05.628 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 3.109987998, 2023-11-16T16:50:05.628 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000100034 2023-11-16T16:50:05.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.628 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:50:05.628 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 46349, 2023-11-16T16:50:05.628 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.006771381, 2023-11-16T16:50:05.628 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000146 2023-11-16T16:50:05.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.629 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 3152, 2023-11-16T16:50:05.629 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 315744 2023-11-16T16:50:05.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.629 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2023-11-16T16:50:05.629 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 27712, 2023-11-16T16:50:05.629 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 25749, 2023-11-16T16:50:05.630 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 31221580, 2023-11-16T16:50:05.630 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 186223909, 2023-11-16T16:50:05.630 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 21, 2023-11-16T16:50:05.630 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 5, 2023-11-16T16:50:05.630 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 1.893855846, 2023-11-16T16:50:05.630 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.753425110, 2023-11-16T16:50:05.630 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.645663135, 2023-11-16T16:50:05.630 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.496815657, 2023-11-16T16:50:05.631 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:50:05.631 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 25748, 2023-11-16T16:50:05.631 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 4.019792947, 2023-11-16T16:50:05.631 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000156120 2023-11-16T16:50:05.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.631 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:50:05.631 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 17459, 2023-11-16T16:50:05.632 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.002959964, 2023-11-16T16:50:05.632 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000169 2023-11-16T16:50:05.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.632 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 2272, 2023-11-16T16:50:05.632 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 124880 2023-11-16T16:50:05.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.632 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2023-11-16T16:50:05.632 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 127912, 2023-11-16T16:50:05.633 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 223180, 2023-11-16T16:50:05.633 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 47741576, 2023-11-16T16:50:05.633 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 220449112, 2023-11-16T16:50:05.633 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 10, 2023-11-16T16:50:05.633 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 6, 2023-11-16T16:50:05.633 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 6.407579935, 2023-11-16T16:50:05.633 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 3.288037653, 2023-11-16T16:50:05.633 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 91.042878205, 2023-11-16T16:50:05.634 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.393521734, 2023-11-16T16:50:05.634 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:50:05.634 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 223179, 2023-11-16T16:50:05.634 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 11.359301714, 2023-11-16T16:50:05.634 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000050897 2023-11-16T16:50:05.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.634 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:50:05.635 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 187870, 2023-11-16T16:50:05.635 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.040908758, 2023-11-16T16:50:05.635 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000217 2023-11-16T16:50:05.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.635 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 1497600, 2023-11-16T16:50:05.635 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 131152 2023-11-16T16:50:05.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.635 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2023-11-16T16:50:05.636 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 1, 2023-11-16T16:50:05.636 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2023-11-16T16:50:05.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.636 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-MDSRank": { 2023-11-16T16:50:05.636 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2023-11-16T16:50:05.636 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2023-11-16T16:50:05.636 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 7328, 2023-11-16T16:50:05.636 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 131.969456373, 2023-11-16T16:50:05.637 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.018008932 2023-11-16T16:50:05.637 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-16T16:50:05.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.637 INFO:teuthology.orchestra.run.smithi053.stdout: "finisher-PurgeQueue": { 2023-11-16T16:50:05.637 INFO:teuthology.orchestra.run.smithi053.stdout: "queue_len": 0, 2023-11-16T16:50:05.637 INFO:teuthology.orchestra.run.smithi053.stdout: "complete_latency": { 2023-11-16T16:50:05.637 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 28140, 2023-11-16T16:50:05.637 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 15.448633495, 2023-11-16T16:50:05.638 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000548991 2023-11-16T16:50:05.638 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-16T16:50:05.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.638 INFO:teuthology.orchestra.run.smithi053.stdout: "mds": { 2023-11-16T16:50:05.638 INFO:teuthology.orchestra.run.smithi053.stdout: "request": 65911, 2023-11-16T16:50:05.638 INFO:teuthology.orchestra.run.smithi053.stdout: "reply": 59144, 2023-11-16T16:50:05.638 INFO:teuthology.orchestra.run.smithi053.stdout: "reply_latency": { 2023-11-16T16:50:05.639 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 59144, 2023-11-16T16:50:05.639 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 395.910741727, 2023-11-16T16:50:05.639 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.006694013 2023-11-16T16:50:05.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.639 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_reply": 0, 2023-11-16T16:50:05.639 INFO:teuthology.orchestra.run.smithi053.stdout: "forward": 6766, 2023-11-16T16:50:05.639 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_complete": 1593, 2023-11-16T16:50:05.640 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_fetch_keys": 0, 2023-11-16T16:50:05.640 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_commit": 1508, 2023-11-16T16:50:05.640 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_split": 187, 2023-11-16T16:50:05.640 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_merge": 1, 2023-11-16T16:50:05.640 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes": 51257, 2023-11-16T16:50:05.640 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_top": 3785, 2023-11-16T16:50:05.640 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_bottom": 47460, 2023-11-16T16:50:05.640 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pin_tail": 12, 2023-11-16T16:50:05.641 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_pinned": 45848, 2023-11-16T16:50:05.641 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_expired": 25749, 2023-11-16T16:50:05.641 INFO:teuthology.orchestra.run.smithi053.stdout: "inodes_with_caps": 1, 2023-11-16T16:50:05.641 INFO:teuthology.orchestra.run.smithi053.stdout: "caps": 1, 2023-11-16T16:50:05.641 INFO:teuthology.orchestra.run.smithi053.stdout: "subtrees": 4, 2023-11-16T16:50:05.641 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse": 85854, 2023-11-16T16:50:05.641 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_hit": 78167, 2023-11-16T16:50:05.642 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_forward": 6153, 2023-11-16T16:50:05.642 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_discover": 21, 2023-11-16T16:50:05.642 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_dir_fetch": 735, 2023-11-16T16:50:05.642 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_remote_ino": 0, 2023-11-16T16:50:05.642 INFO:teuthology.orchestra.run.smithi053.stdout: "traverse_lock": 0, 2023-11-16T16:50:05.642 INFO:teuthology.orchestra.run.smithi053.stdout: "load_cent": 0, 2023-11-16T16:50:05.642 INFO:teuthology.orchestra.run.smithi053.stdout: "q": 0, 2023-11-16T16:50:05.642 INFO:teuthology.orchestra.run.smithi053.stdout: "exported": 511, 2023-11-16T16:50:05.643 INFO:teuthology.orchestra.run.smithi053.stdout: "exported_inodes": 25114, 2023-11-16T16:50:05.643 INFO:teuthology.orchestra.run.smithi053.stdout: "imported": 227, 2023-11-16T16:50:05.643 INFO:teuthology.orchestra.run.smithi053.stdout: "imported_inodes": 0, 2023-11-16T16:50:05.643 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_dir_fetch": 0, 2023-11-16T16:50:05.643 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_backtrace_fetch": 11917, 2023-11-16T16:50:05.643 INFO:teuthology.orchestra.run.smithi053.stdout: "openino_peer_discover": 0, 2023-11-16T16:50:05.643 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rfiles": 4, 2023-11-16T16:50:05.643 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rbytes": 246, 2023-11-16T16:50:05.644 INFO:teuthology.orchestra.run.smithi053.stdout: "root_rsnaps": 6, 2023-11-16T16:50:05.644 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_fetch": 11917, 2023-11-16T16:50:05.644 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_set_tag": 0, 2023-11-16T16:50:05.644 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_backtrace_repaired": 0, 2023-11-16T16:50:05.644 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_inotable_repaired": 0, 2023-11-16T16:50:05.644 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_inodes": 4, 2023-11-16T16:50:05.644 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dir_base_inodes": 0, 2023-11-16T16:50:05.644 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_dirfrag_rstats": 4, 2023-11-16T16:50:05.645 INFO:teuthology.orchestra.run.smithi053.stdout: "scrub_file_inodes": 11913, 2023-11-16T16:50:05.645 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_inode_file_caps": 9, 2023-11-16T16:50:05.645 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_revoke": 38, 2023-11-16T16:50:05.645 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_grant": 39561, 2023-11-16T16:50:05.645 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_trunc": 0, 2023-11-16T16:50:05.645 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flushsnap_ack": 95, 2023-11-16T16:50:05.645 INFO:teuthology.orchestra.run.smithi053.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-16T16:50:05.646 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps": 48439, 2023-11-16T16:50:05.646 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_caps_dirty": 48400, 2023-11-16T16:50:05.646 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_cap_release": 347, 2023-11-16T16:50:05.646 INFO:teuthology.orchestra.run.smithi053.stdout: "process_request_cap_release": 38375 2023-11-16T16:50:05.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.646 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache": { 2023-11-16T16:50:05.646 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update": 976, 2023-11-16T16:50:05.646 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_update_receipt": 1834, 2023-11-16T16:50:05.647 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_try_discover": 919, 2023-11-16T16:50:05.647 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_send_discover": 1389, 2023-11-16T16:50:05.647 INFO:teuthology.orchestra.run.smithi053.stdout: "dir_handle_discover": 1114, 2023-11-16T16:50:05.647 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays": 2, 2023-11-16T16:50:05.647 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_delayed": 0, 2023-11-16T16:50:05.647 INFO:teuthology.orchestra.run.smithi053.stdout: "num_strays_enqueuing": 0, 2023-11-16T16:50:05.647 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_created": 18387, 2023-11-16T16:50:05.648 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_enqueued": 18385, 2023-11-16T16:50:05.648 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_reintegrated": 0, 2023-11-16T16:50:05.648 INFO:teuthology.orchestra.run.smithi053.stdout: "strays_migrated": 0, 2023-11-16T16:50:05.648 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_processing": 0, 2023-11-16T16:50:05.648 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_enqueued": 0, 2023-11-16T16:50:05.648 INFO:teuthology.orchestra.run.smithi053.stdout: "num_recovering_prioritized": 0, 2023-11-16T16:50:05.648 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_started": 0, 2023-11-16T16:50:05.648 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_completed": 0, 2023-11-16T16:50:05.649 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_enqueue_scrub": 0, 2023-11-16T16:50:05.649 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_exportdir": 565, 2023-11-16T16:50:05.649 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_flush": 0, 2023-11-16T16:50:05.649 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragmentdir": 211, 2023-11-16T16:50:05.649 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_fragstats": 0, 2023-11-16T16:50:05.649 INFO:teuthology.orchestra.run.smithi053.stdout: "ireq_inodestats": 0 2023-11-16T16:50:05.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.649 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log": { 2023-11-16T16:50:05.650 INFO:teuthology.orchestra.run.smithi053.stdout: "evadd": 105041, 2023-11-16T16:50:05.650 INFO:teuthology.orchestra.run.smithi053.stdout: "evex": 20769, 2023-11-16T16:50:05.650 INFO:teuthology.orchestra.run.smithi053.stdout: "evtrm": 20769, 2023-11-16T16:50:05.650 INFO:teuthology.orchestra.run.smithi053.stdout: "ev": 84272, 2023-11-16T16:50:05.650 INFO:teuthology.orchestra.run.smithi053.stdout: "evexg": 0, 2023-11-16T16:50:05.650 INFO:teuthology.orchestra.run.smithi053.stdout: "evexd": 547, 2023-11-16T16:50:05.650 INFO:teuthology.orchestra.run.smithi053.stdout: "segadd": 155, 2023-11-16T16:50:05.650 INFO:teuthology.orchestra.run.smithi053.stdout: "segex": 27, 2023-11-16T16:50:05.651 INFO:teuthology.orchestra.run.smithi053.stdout: "segtrm": 27, 2023-11-16T16:50:05.651 INFO:teuthology.orchestra.run.smithi053.stdout: "seg": 129, 2023-11-16T16:50:05.651 INFO:teuthology.orchestra.run.smithi053.stdout: "segexg": 0, 2023-11-16T16:50:05.651 INFO:teuthology.orchestra.run.smithi053.stdout: "segexd": 1, 2023-11-16T16:50:05.651 INFO:teuthology.orchestra.run.smithi053.stdout: "expos": 62917679, 2023-11-16T16:50:05.651 INFO:teuthology.orchestra.run.smithi053.stdout: "wrpos": 379312076, 2023-11-16T16:50:05.651 INFO:teuthology.orchestra.run.smithi053.stdout: "rdpos": 0, 2023-11-16T16:50:05.651 INFO:teuthology.orchestra.run.smithi053.stdout: "jlat": { 2023-11-16T16:50:05.652 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1696, 2023-11-16T16:50:05.652 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 178.780470442, 2023-11-16T16:50:05.652 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.105413013 2023-11-16T16:50:05.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.652 INFO:teuthology.orchestra.run.smithi053.stdout: "replayed": 0 2023-11-16T16:50:05.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.652 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_mem": { 2023-11-16T16:50:05.652 INFO:teuthology.orchestra.run.smithi053.stdout: "ino": 31258, 2023-11-16T16:50:05.653 INFO:teuthology.orchestra.run.smithi053.stdout: "ino+": 74363, 2023-11-16T16:50:05.653 INFO:teuthology.orchestra.run.smithi053.stdout: "ino-": 43105, 2023-11-16T16:50:05.653 INFO:teuthology.orchestra.run.smithi053.stdout: "dir": 1057, 2023-11-16T16:50:05.653 INFO:teuthology.orchestra.run.smithi053.stdout: "dir+": 3287, 2023-11-16T16:50:05.653 INFO:teuthology.orchestra.run.smithi053.stdout: "dir-": 2230, 2023-11-16T16:50:05.653 INFO:teuthology.orchestra.run.smithi053.stdout: "dn": 51257, 2023-11-16T16:50:05.653 INFO:teuthology.orchestra.run.smithi053.stdout: "dn+": 95277, 2023-11-16T16:50:05.653 INFO:teuthology.orchestra.run.smithi053.stdout: "dn-": 44020, 2023-11-16T16:50:05.654 INFO:teuthology.orchestra.run.smithi053.stdout: "cap": 1, 2023-11-16T16:50:05.654 INFO:teuthology.orchestra.run.smithi053.stdout: "cap+": 49028, 2023-11-16T16:50:05.654 INFO:teuthology.orchestra.run.smithi053.stdout: "cap-": 49027, 2023-11-16T16:50:05.654 INFO:teuthology.orchestra.run.smithi053.stdout: "rss": 291876, 2023-11-16T16:50:05.654 INFO:teuthology.orchestra.run.smithi053.stdout: "heap": 198964 2023-11-16T16:50:05.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.654 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_server": { 2023-11-16T16:50:05.655 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_client_request": 68188, 2023-11-16T16:50:05.655 INFO:teuthology.orchestra.run.smithi053.stdout: "dispatch_server_request": 46, 2023-11-16T16:50:05.655 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_request": 65911, 2023-11-16T16:50:05.655 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_client_session": 370, 2023-11-16T16:50:05.655 INFO:teuthology.orchestra.run.smithi053.stdout: "handle_peer_request": 105, 2023-11-16T16:50:05.655 INFO:teuthology.orchestra.run.smithi053.stdout: "req_create_latency": { 2023-11-16T16:50:05.655 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 20635, 2023-11-16T16:50:05.655 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 239.764873545, 2023-11-16T16:50:05.656 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.011619329 2023-11-16T16:50:05.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.656 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getattr_latency": { 2023-11-16T16:50:05.656 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 10, 2023-11-16T16:50:05.656 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.054483086, 2023-11-16T16:50:05.656 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.005448308 2023-11-16T16:50:05.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.656 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getfilelock_latency": { 2023-11-16T16:50:05.656 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:05.657 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:05.657 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:05.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.657 INFO:teuthology.orchestra.run.smithi053.stdout: "req_link_latency": { 2023-11-16T16:50:05.657 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:05.657 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:05.657 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:05.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.658 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookup_latency": { 2023-11-16T16:50:05.658 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 18095, 2023-11-16T16:50:05.658 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 42.866791033, 2023-11-16T16:50:05.658 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.002368985 2023-11-16T16:50:05.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.658 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookuphash_latency": { 2023-11-16T16:50:05.658 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:05.659 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:05.659 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:05.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.659 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupino_latency": { 2023-11-16T16:50:05.659 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 1, 2023-11-16T16:50:05.659 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000180892, 2023-11-16T16:50:05.659 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000180892 2023-11-16T16:50:05.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.660 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupname_latency": { 2023-11-16T16:50:05.660 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:05.660 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:05.660 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:05.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.660 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupparent_latency": { 2023-11-16T16:50:05.660 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:05.660 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:05.661 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:05.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.661 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lookupsnap_latency": { 2023-11-16T16:50:05.661 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:05.661 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:05.661 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:05.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.661 INFO:teuthology.orchestra.run.smithi053.stdout: "req_lssnap_latency": { 2023-11-16T16:50:05.662 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:05.662 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:05.662 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:05.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.662 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mkdir_latency": { 2023-11-16T16:50:05.662 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:05.662 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:05.662 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:05.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.663 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mknod_latency": { 2023-11-16T16:50:05.663 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:05.663 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:05.663 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:05.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.663 INFO:teuthology.orchestra.run.smithi053.stdout: "req_mksnap_latency": { 2023-11-16T16:50:05.663 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:05.664 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:05.664 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:05.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.664 INFO:teuthology.orchestra.run.smithi053.stdout: "req_open_latency": { 2023-11-16T16:50:05.664 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:05.664 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:05.664 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:05.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.665 INFO:teuthology.orchestra.run.smithi053.stdout: "req_readdir_latency": { 2023-11-16T16:50:05.665 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2016, 2023-11-16T16:50:05.665 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 26.221810801, 2023-11-16T16:50:05.665 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.013006850 2023-11-16T16:50:05.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.665 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rename_latency": { 2023-11-16T16:50:05.665 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:05.665 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:05.666 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:05.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.666 INFO:teuthology.orchestra.run.smithi053.stdout: "req_renamesnap_latency": { 2023-11-16T16:50:05.666 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:05.666 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:05.666 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:05.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.666 INFO:teuthology.orchestra.run.smithi053.stdout: "req_snapdiff_latency": { 2023-11-16T16:50:05.667 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:05.667 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:05.667 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:05.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.667 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmdir_latency": { 2023-11-16T16:50:05.667 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 2, 2023-11-16T16:50:05.667 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.174414968, 2023-11-16T16:50:05.667 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.087207484 2023-11-16T16:50:05.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.668 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmsnap_latency": { 2023-11-16T16:50:05.668 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:05.668 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:05.668 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:05.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.668 INFO:teuthology.orchestra.run.smithi053.stdout: "req_rmxattr_latency": { 2023-11-16T16:50:05.669 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:05.669 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:05.669 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:05.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.669 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setattr_latency": { 2023-11-16T16:50:05.669 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:05.669 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:05.670 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:05.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.670 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setdirlayout_latency": { 2023-11-16T16:50:05.670 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:05.670 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:05.670 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:05.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.670 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setfilelock_latency": { 2023-11-16T16:50:05.671 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:05.671 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:05.671 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:05.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.671 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setlayout_latency": { 2023-11-16T16:50:05.671 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:05.671 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:05.671 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:05.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.672 INFO:teuthology.orchestra.run.smithi053.stdout: "req_setxattr_latency": { 2023-11-16T16:50:05.672 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:05.672 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:05.672 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:05.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.672 INFO:teuthology.orchestra.run.smithi053.stdout: "req_symlink_latency": { 2023-11-16T16:50:05.672 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:05.673 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:05.673 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:05.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.673 INFO:teuthology.orchestra.run.smithi053.stdout: "req_unlink_latency": { 2023-11-16T16:50:05.673 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 18385, 2023-11-16T16:50:05.673 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 86.828187402, 2023-11-16T16:50:05.673 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.004722773 2023-11-16T16:50:05.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.674 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_revoke_eviction": 0, 2023-11-16T16:50:05.674 INFO:teuthology.orchestra.run.smithi053.stdout: "cap_acquisition_throttle": 0, 2023-11-16T16:50:05.674 INFO:teuthology.orchestra.run.smithi053.stdout: "req_getvxattr_latency": { 2023-11-16T16:50:05.674 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:05.674 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:05.674 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:05.674 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-16T16:50:05.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.675 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_sessions": { 2023-11-16T16:50:05.675 INFO:teuthology.orchestra.run.smithi053.stdout: "session_count": 1, 2023-11-16T16:50:05.675 INFO:teuthology.orchestra.run.smithi053.stdout: "session_add": 3, 2023-11-16T16:50:05.675 INFO:teuthology.orchestra.run.smithi053.stdout: "session_remove": 2, 2023-11-16T16:50:05.675 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_open": 1, 2023-11-16T16:50:05.675 INFO:teuthology.orchestra.run.smithi053.stdout: "sessions_stale": 0, 2023-11-16T16:50:05.675 INFO:teuthology.orchestra.run.smithi053.stdout: "total_load": 20110, 2023-11-16T16:50:05.676 INFO:teuthology.orchestra.run.smithi053.stdout: "average_load": 20110, 2023-11-16T16:50:05.676 INFO:teuthology.orchestra.run.smithi053.stdout: "avg_session_uptime": 669 2023-11-16T16:50:05.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.676 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2023-11-16T16:50:05.676 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 0, 2023-11-16T16:50:05.676 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 0, 2023-11-16T16:50:05.676 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2023-11-16T16:50:05.676 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2023-11-16T16:50:05.677 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2023-11-16T16:50:05.677 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2023-11-16T16:50:05.677 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-16T16:50:05.677 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2023-11-16T16:50:05.677 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-16T16:50:05.677 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2023-11-16T16:50:05.677 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2023-11-16T16:50:05.677 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2023-11-16T16:50:05.678 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-16T16:50:05.678 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2023-11-16T16:50:05.678 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2023-11-16T16:50:05.678 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2023-11-16T16:50:05.678 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2023-11-16T16:50:05.678 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2023-11-16T16:50:05.678 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-16T16:50:05.679 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2023-11-16T16:50:05.679 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-16T16:50:05.679 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2023-11-16T16:50:05.679 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2023-11-16T16:50:05.679 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2023-11-16T16:50:05.679 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2023-11-16T16:50:05.679 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2023-11-16T16:50:05.679 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-16T16:50:05.680 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2023-11-16T16:50:05.680 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2023-11-16T16:50:05.680 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2023-11-16T16:50:05.680 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2023-11-16T16:50:05.680 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2023-11-16T16:50:05.680 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2023-11-16T16:50:05.680 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2023-11-16T16:50:05.680 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2023-11-16T16:50:05.681 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2023-11-16T16:50:05.681 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 1151719, 2023-11-16T16:50:05.681 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 24516, 2023-11-16T16:50:05.681 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 1232, 2023-11-16T16:50:05.681 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 14, 2023-11-16T16:50:05.681 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2023-11-16T16:50:05.681 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2023-11-16T16:50:05.681 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2023-11-16T16:50:05.682 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2023-11-16T16:50:05.682 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2023-11-16T16:50:05.682 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2023-11-16T16:50:05.682 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 8128, 2023-11-16T16:50:05.682 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 169, 2023-11-16T16:50:05.682 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2023-11-16T16:50:05.682 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2023-11-16T16:50:05.682 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2023-11-16T16:50:05.683 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2023-11-16T16:50:05.683 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 106561327, 2023-11-16T16:50:05.683 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 927760, 2023-11-16T16:50:05.683 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2023-11-16T16:50:05.683 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2023-11-16T16:50:05.683 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2023-11-16T16:50:05.683 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2023-11-16T16:50:05.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.684 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter": { 2023-11-16T16:50:05.684 INFO:teuthology.orchestra.run.smithi053.stdout: "op_active": 0, 2023-11-16T16:50:05.684 INFO:teuthology.orchestra.run.smithi053.stdout: "op_laggy": 0, 2023-11-16T16:50:05.684 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send": 40825, 2023-11-16T16:50:05.684 INFO:teuthology.orchestra.run.smithi053.stdout: "op_send_bytes": 395047106, 2023-11-16T16:50:05.684 INFO:teuthology.orchestra.run.smithi053.stdout: "op_resend": 27, 2023-11-16T16:50:05.684 INFO:teuthology.orchestra.run.smithi053.stdout: "op_reply": 40825, 2023-11-16T16:50:05.684 INFO:teuthology.orchestra.run.smithi053.stdout: "op_latency": { 2023-11-16T16:50:05.684 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 40825, 2023-11-16T16:50:05.685 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 574.541913412, 2023-11-16T16:50:05.685 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.014073286 2023-11-16T16:50:05.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.685 INFO:teuthology.orchestra.run.smithi053.stdout: "op_inflight": 0, 2023-11-16T16:50:05.685 INFO:teuthology.orchestra.run.smithi053.stdout: "oplen_avg": { 2023-11-16T16:50:05.685 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 40825, 2023-11-16T16:50:05.685 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 50259 2023-11-16T16:50:05.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.686 INFO:teuthology.orchestra.run.smithi053.stdout: "op": 40825, 2023-11-16T16:50:05.686 INFO:teuthology.orchestra.run.smithi053.stdout: "op_r": 15377, 2023-11-16T16:50:05.686 INFO:teuthology.orchestra.run.smithi053.stdout: "op_w": 25448, 2023-11-16T16:50:05.686 INFO:teuthology.orchestra.run.smithi053.stdout: "op_rmw": 0, 2023-11-16T16:50:05.686 INFO:teuthology.orchestra.run.smithi053.stdout: "op_pg": 0, 2023-11-16T16:50:05.686 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_stat": 0, 2023-11-16T16:50:05.686 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_create": 3129, 2023-11-16T16:50:05.686 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_read": 244, 2023-11-16T16:50:05.687 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_write": 1940, 2023-11-16T16:50:05.687 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writefull": 117, 2023-11-16T16:50:05.687 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_writesame": 0, 2023-11-16T16:50:05.687 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_append": 0, 2023-11-16T16:50:05.687 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_zero": 0, 2023-11-16T16:50:05.687 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_truncate": 1, 2023-11-16T16:50:05.687 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_delete": 18694, 2023-11-16T16:50:05.687 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_mapext": 0, 2023-11-16T16:50:05.688 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_sparse_read": 0, 2023-11-16T16:50:05.688 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_clonerange": 0, 2023-11-16T16:50:05.688 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_getxattr": 11917, 2023-11-16T16:50:05.688 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_setxattr": 6258, 2023-11-16T16:50:05.688 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_cmpxattr": 0, 2023-11-16T16:50:05.688 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_rmxattr": 0, 2023-11-16T16:50:05.688 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_resetxattrs": 0, 2023-11-16T16:50:05.688 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_call": 0, 2023-11-16T16:50:05.689 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_watch": 0, 2023-11-16T16:50:05.689 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_notify": 0, 2023-11-16T16:50:05.689 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_src_cmpxattr": 0, 2023-11-16T16:50:05.689 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls": 0, 2023-11-16T16:50:05.689 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_pgls_filter": 0, 2023-11-16T16:50:05.689 INFO:teuthology.orchestra.run.smithi053.stdout: "osdop_other": 0, 2023-11-16T16:50:05.689 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_active": 0, 2023-11-16T16:50:05.689 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_send": 0, 2023-11-16T16:50:05.690 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_resend": 0, 2023-11-16T16:50:05.690 INFO:teuthology.orchestra.run.smithi053.stdout: "linger_ping": 0, 2023-11-16T16:50:05.690 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_active": 0, 2023-11-16T16:50:05.690 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_send": 0, 2023-11-16T16:50:05.690 INFO:teuthology.orchestra.run.smithi053.stdout: "poolop_resend": 0, 2023-11-16T16:50:05.690 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_active": 0, 2023-11-16T16:50:05.690 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_send": 0, 2023-11-16T16:50:05.691 INFO:teuthology.orchestra.run.smithi053.stdout: "poolstat_resend": 0, 2023-11-16T16:50:05.691 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_active": 0, 2023-11-16T16:50:05.691 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_send": 0, 2023-11-16T16:50:05.691 INFO:teuthology.orchestra.run.smithi053.stdout: "statfs_resend": 0, 2023-11-16T16:50:05.691 INFO:teuthology.orchestra.run.smithi053.stdout: "command_active": 0, 2023-11-16T16:50:05.691 INFO:teuthology.orchestra.run.smithi053.stdout: "command_send": 0, 2023-11-16T16:50:05.691 INFO:teuthology.orchestra.run.smithi053.stdout: "command_resend": 0, 2023-11-16T16:50:05.691 INFO:teuthology.orchestra.run.smithi053.stdout: "map_epoch": 95, 2023-11-16T16:50:05.692 INFO:teuthology.orchestra.run.smithi053.stdout: "map_full": 0, 2023-11-16T16:50:05.692 INFO:teuthology.orchestra.run.smithi053.stdout: "map_inc": 11, 2023-11-16T16:50:05.692 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_sessions": 12, 2023-11-16T16:50:05.692 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_open": 12, 2023-11-16T16:50:05.692 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_session_close": 0, 2023-11-16T16:50:05.692 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_laggy": 0, 2023-11-16T16:50:05.692 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_wr": 3111, 2023-11-16T16:50:05.692 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_rd": 4809, 2023-11-16T16:50:05.693 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_del": 39 2023-11-16T16:50:05.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.693 INFO:teuthology.orchestra.run.smithi053.stdout: "oft": { 2023-11-16T16:50:05.693 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_objs": 1, 2023-11-16T16:50:05.693 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_kv_pairs": 0, 2023-11-16T16:50:05.693 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_updates": 0, 2023-11-16T16:50:05.693 INFO:teuthology.orchestra.run.smithi053.stdout: "omap_total_removes": 0 2023-11-16T16:50:05.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.694 INFO:teuthology.orchestra.run.smithi053.stdout: "osd-slow-ops": { 2023-11-16T16:50:05.694 INFO:teuthology.orchestra.run.smithi053.stdout: "slow_ops_count": 66733 2023-11-16T16:50:05.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.694 INFO:teuthology.orchestra.run.smithi053.stdout: "purge_queue": { 2023-11-16T16:50:05.694 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops": 0, 2023-11-16T16:50:05.694 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_ops_high_water": 10, 2023-11-16T16:50:05.694 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing": 0, 2023-11-16T16:50:05.695 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executing_high_water": 10, 2023-11-16T16:50:05.695 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_executed": 18385, 2023-11-16T16:50:05.695 INFO:teuthology.orchestra.run.smithi053.stdout: "pq_item_in_journal": 0 2023-11-16T16:50:05.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.695 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-16T16:50:05.695 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2023-11-16T16:50:05.695 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2023-11-16T16:50:05.695 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2023-11-16T16:50:05.696 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 191234, 2023-11-16T16:50:05.696 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 83329636, 2023-11-16T16:50:05.696 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2023-11-16T16:50:05.696 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 191234, 2023-11-16T16:50:05.696 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2023-11-16T16:50:05.696 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2023-11-16T16:50:05.696 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 191234, 2023-11-16T16:50:05.696 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 83329636, 2023-11-16T16:50:05.697 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2023-11-16T16:50:05.697 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:05.697 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:05.697 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:05.697 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-16T16:50:05.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.697 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_bytes": { 2023-11-16T16:50:05.697 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2023-11-16T16:50:05.698 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2023-11-16T16:50:05.698 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2023-11-16T16:50:05.698 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2023-11-16T16:50:05.698 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2023-11-16T16:50:05.698 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2023-11-16T16:50:05.698 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2023-11-16T16:50:05.698 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 40825, 2023-11-16T16:50:05.699 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 397148709, 2023-11-16T16:50:05.699 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 40825, 2023-11-16T16:50:05.699 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 397148709, 2023-11-16T16:50:05.699 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2023-11-16T16:50:05.699 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:05.699 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:05.699 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:05.699 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-16T16:50:05.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.700 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-objecter_ops": { 2023-11-16T16:50:05.700 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2023-11-16T16:50:05.700 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 1024, 2023-11-16T16:50:05.700 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2023-11-16T16:50:05.700 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 0, 2023-11-16T16:50:05.700 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 0, 2023-11-16T16:50:05.700 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2023-11-16T16:50:05.700 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 0, 2023-11-16T16:50:05.701 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 40825, 2023-11-16T16:50:05.701 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 40825, 2023-11-16T16:50:05.701 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 40825, 2023-11-16T16:50:05.701 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 40825, 2023-11-16T16:50:05.701 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2023-11-16T16:50:05.701 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:05.701 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:05.701 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:05.702 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-16T16:50:05.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.702 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle": { 2023-11-16T16:50:05.702 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2023-11-16T16:50:05.702 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2023-11-16T16:50:05.702 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2023-11-16T16:50:05.702 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 18385, 2023-11-16T16:50:05.702 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 2172757, 2023-11-16T16:50:05.703 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2023-11-16T16:50:05.703 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 18385, 2023-11-16T16:50:05.703 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2023-11-16T16:50:05.703 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2023-11-16T16:50:05.703 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 244, 2023-11-16T16:50:05.703 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 2172757, 2023-11-16T16:50:05.703 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2023-11-16T16:50:05.703 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:05.704 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:05.704 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:05.704 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-16T16:50:05.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:05.704 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-write_buf_throttle-0x562f3d86fcc0": { 2023-11-16T16:50:05.704 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2023-11-16T16:50:05.704 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 3758096384, 2023-11-16T16:50:05.704 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2023-11-16T16:50:05.705 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 105041, 2023-11-16T16:50:05.705 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 375117772, 2023-11-16T16:50:05.705 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2023-11-16T16:50:05.705 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 105041, 2023-11-16T16:50:05.705 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2023-11-16T16:50:05.705 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2023-11-16T16:50:05.705 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 1696, 2023-11-16T16:50:05.705 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 375117772, 2023-11-16T16:50:05.705 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2023-11-16T16:50:05.706 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:05.706 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:05.706 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:05.706 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-16T16:50:05.706 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-16T16:50:05.706 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-16T16:50:05.913 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:05 smithi053 ceph-mon[100638]: pgmap v339: 97 pgs: 97 active+clean; 817 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-11-16T16:50:05.932 INFO:tasks.check_counter:Daemon mds.i mds.exported=511 2023-11-16T16:50:05.932 INFO:tasks.check_counter:Daemon mds.i mds.imported=227 2023-11-16T16:50:05.932 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=976 2023-11-16T16:50:05.932 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=1834 2023-11-16T16:50:05.932 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2023-11-16T16:50:05.932 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2023-11-16T16:50:05.932 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2023-11-16T16:50:05.932 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=187 2023-11-16T16:50:05.933 DEBUG:tasks.check_counter:Getting stats from l 2023-11-16T16:50:05.933 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5e4a743ad7fd1422ebd6c4dc950a21a303279207 shell --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 -- ceph daemon mds.l perf dump 2023-11-16T16:50:06.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:05 smithi001 ceph-mon[98800]: pgmap v339: 97 pgs: 97 active+clean; 817 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-11-16T16:50:06.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:05 smithi026 ceph-mon[100112]: pgmap v339: 97 pgs: 97 active+clean; 817 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2023-11-16T16:50:06.583 INFO:teuthology.orchestra.run.smithi053.stderr:Inferring config /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/mon.c/config 2023-11-16T16:50:07.775 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2023-11-16T16:50:07.775 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-0": { 2023-11-16T16:50:07.775 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 225, 2023-11-16T16:50:07.776 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 215, 2023-11-16T16:50:07.776 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 1422528, 2023-11-16T16:50:07.776 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 122704, 2023-11-16T16:50:07.776 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 4, 2023-11-16T16:50:07.776 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 0, 2023-11-16T16:50:07.776 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.045784353, 2023-11-16T16:50:07.776 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.013044376, 2023-11-16T16:50:07.776 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.034012123, 2023-11-16T16:50:07.777 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.006575566, 2023-11-16T16:50:07.777 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:50:07.777 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 214, 2023-11-16T16:50:07.777 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.010616910, 2023-11-16T16:50:07.777 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000049611 2023-11-16T16:50:07.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:07.777 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:50:07.777 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:07.778 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:07.778 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:07.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:07.778 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 1422528, 2023-11-16T16:50:07.778 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 122704 2023-11-16T16:50:07.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:07.778 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-1": { 2023-11-16T16:50:07.779 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 2, 2023-11-16T16:50:07.779 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 165, 2023-11-16T16:50:07.779 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 320, 2023-11-16T16:50:07.779 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 180736, 2023-11-16T16:50:07.779 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 5, 2023-11-16T16:50:07.779 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 3, 2023-11-16T16:50:07.779 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.011228229, 2023-11-16T16:50:07.779 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.006886716, 2023-11-16T16:50:07.780 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.001807698, 2023-11-16T16:50:07.780 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-16T16:50:07.780 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:50:07.780 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 162, 2023-11-16T16:50:07.780 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.015248015, 2023-11-16T16:50:07.780 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000094123 2023-11-16T16:50:07.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:07.780 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:50:07.781 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:07.781 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:07.781 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:07.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:07.781 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 320, 2023-11-16T16:50:07.781 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 180736 2023-11-16T16:50:07.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:07.781 INFO:teuthology.orchestra.run.smithi053.stdout: "AsyncMessenger::Worker-2": { 2023-11-16T16:50:07.781 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_messages": 4, 2023-11-16T16:50:07.782 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages": 4, 2023-11-16T16:50:07.782 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_bytes": 2992, 2023-11-16T16:50:07.782 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_bytes": 448, 2023-11-16T16:50:07.782 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_created_connections": 2, 2023-11-16T16:50:07.782 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_active_connections": 1, 2023-11-16T16:50:07.782 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_total_time": 0.000835192, 2023-11-16T16:50:07.782 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_send_time": 0.000047262, 2023-11-16T16:50:07.783 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_recv_time": 0.000758782, 2023-11-16T16:50:07.783 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-16T16:50:07.783 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_messages_queue_lat": { 2023-11-16T16:50:07.783 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 3, 2023-11-16T16:50:07.783 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000741138, 2023-11-16T16:50:07.783 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000247046 2023-11-16T16:50:07.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:07.783 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_handle_ack_lat": { 2023-11-16T16:50:07.784 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:07.784 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:07.784 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:07.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:07.784 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-11-16T16:50:07.784 INFO:teuthology.orchestra.run.smithi053.stdout: "msgr_send_encrypted_bytes": 448 2023-11-16T16:50:07.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:07.785 INFO:teuthology.orchestra.run.smithi053.stdout: "cct": { 2023-11-16T16:50:07.785 INFO:teuthology.orchestra.run.smithi053.stdout: "total_workers": 0, 2023-11-16T16:50:07.785 INFO:teuthology.orchestra.run.smithi053.stdout: "unhealthy_workers": 0 2023-11-16T16:50:07.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:07.785 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool": { 2023-11-16T16:50:07.785 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_bytes": 0, 2023-11-16T16:50:07.785 INFO:teuthology.orchestra.run.smithi053.stdout: "bloom_filter_items": 0, 2023-11-16T16:50:07.785 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_bytes": 0, 2023-11-16T16:50:07.786 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_items": 0, 2023-11-16T16:50:07.786 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_bytes": 0, 2023-11-16T16:50:07.786 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_data_items": 0, 2023-11-16T16:50:07.786 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-16T16:50:07.786 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_onode_items": 0, 2023-11-16T16:50:07.786 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-16T16:50:07.786 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_items": 0, 2023-11-16T16:50:07.786 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_bytes": 0, 2023-11-16T16:50:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_other_items": 0, 2023-11-16T16:50:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-16T16:50:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_buffer_items": 0, 2023-11-16T16:50:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_bytes": 0, 2023-11-16T16:50:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_items": 0, 2023-11-16T16:50:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_bytes": 0, 2023-11-16T16:50:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blob_items": 0, 2023-11-16T16:50:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-16T16:50:07.788 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_shared_blob_items": 0, 2023-11-16T16:50:07.788 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-16T16:50:07.788 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_inline_bl_items": 0, 2023-11-16T16:50:07.788 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_bytes": 0, 2023-11-16T16:50:07.788 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_items": 0, 2023-11-16T16:50:07.788 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_bytes": 0, 2023-11-16T16:50:07.788 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_txc_items": 0, 2023-11-16T16:50:07.788 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-16T16:50:07.789 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_deferred_items": 0, 2023-11-16T16:50:07.789 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_bytes": 0, 2023-11-16T16:50:07.789 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_writing_items": 0, 2023-11-16T16:50:07.789 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_bytes": 0, 2023-11-16T16:50:07.789 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_items": 0, 2023-11-16T16:50:07.789 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_bytes": 0, 2023-11-16T16:50:07.789 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_reader_items": 0, 2023-11-16T16:50:07.790 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_bytes": 0, 2023-11-16T16:50:07.790 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_file_writer_items": 0, 2023-11-16T16:50:07.790 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_bytes": 149017, 2023-11-16T16:50:07.790 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_anon_items": 31, 2023-11-16T16:50:07.790 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_bytes": 0, 2023-11-16T16:50:07.790 INFO:teuthology.orchestra.run.smithi053.stdout: "buffer_meta_items": 0, 2023-11-16T16:50:07.790 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bytes": 0, 2023-11-16T16:50:07.791 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_items": 0, 2023-11-16T16:50:07.791 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_bytes": 0, 2023-11-16T16:50:07.791 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mapbl_items": 0, 2023-11-16T16:50:07.791 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_bytes": 0, 2023-11-16T16:50:07.791 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pglog_items": 0, 2023-11-16T16:50:07.791 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_bytes": 0, 2023-11-16T16:50:07.791 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_items": 0, 2023-11-16T16:50:07.791 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_bytes": 0, 2023-11-16T16:50:07.792 INFO:teuthology.orchestra.run.smithi053.stdout: "osdmap_mapping_items": 0, 2023-11-16T16:50:07.792 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_bytes": 0, 2023-11-16T16:50:07.792 INFO:teuthology.orchestra.run.smithi053.stdout: "pgmap_items": 0, 2023-11-16T16:50:07.792 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_bytes": 928, 2023-11-16T16:50:07.792 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_co_items": 2, 2023-11-16T16:50:07.792 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_bytes": 0, 2023-11-16T16:50:07.792 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_1_items": 0, 2023-11-16T16:50:07.792 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_bytes": 0, 2023-11-16T16:50:07.793 INFO:teuthology.orchestra.run.smithi053.stdout: "unittest_2_items": 0 2023-11-16T16:50:07.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2023-11-16T16:50:07.793 INFO:teuthology.orchestra.run.smithi053.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-16T16:50:07.793 INFO:teuthology.orchestra.run.smithi053.stdout: "val": 0, 2023-11-16T16:50:07.793 INFO:teuthology.orchestra.run.smithi053.stdout: "max": 104857600, 2023-11-16T16:50:07.793 INFO:teuthology.orchestra.run.smithi053.stdout: "get_started": 0, 2023-11-16T16:50:07.793 INFO:teuthology.orchestra.run.smithi053.stdout: "get": 227, 2023-11-16T16:50:07.793 INFO:teuthology.orchestra.run.smithi053.stdout: "get_sum": 1390759, 2023-11-16T16:50:07.794 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_fail": 0, 2023-11-16T16:50:07.794 INFO:teuthology.orchestra.run.smithi053.stdout: "get_or_fail_success": 227, 2023-11-16T16:50:07.794 INFO:teuthology.orchestra.run.smithi053.stdout: "take": 0, 2023-11-16T16:50:07.794 INFO:teuthology.orchestra.run.smithi053.stdout: "take_sum": 0, 2023-11-16T16:50:07.794 INFO:teuthology.orchestra.run.smithi053.stdout: "put": 227, 2023-11-16T16:50:07.794 INFO:teuthology.orchestra.run.smithi053.stdout: "put_sum": 1390759, 2023-11-16T16:50:07.794 INFO:teuthology.orchestra.run.smithi053.stdout: "wait": { 2023-11-16T16:50:07.795 INFO:teuthology.orchestra.run.smithi053.stdout: "avgcount": 0, 2023-11-16T16:50:07.795 INFO:teuthology.orchestra.run.smithi053.stdout: "sum": 0.000000000, 2023-11-16T16:50:07.795 INFO:teuthology.orchestra.run.smithi053.stdout: "avgtime": 0.000000000 2023-11-16T16:50:07.795 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-16T16:50:07.795 INFO:teuthology.orchestra.run.smithi053.stdout: } 2023-11-16T16:50:07.795 INFO:teuthology.orchestra.run.smithi053.stdout:} 2023-11-16T16:50:07.801 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:07 smithi053 ceph-mon[100638]: pgmap v340: 97 pgs: 97 active+clean; 817 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-11-16T16:50:08.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:07 smithi001 ceph-mon[98800]: pgmap v340: 97 pgs: 97 active+clean; 817 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-11-16T16:50:08.257 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:07 smithi026 ceph-mon[100112]: pgmap v340: 97 pgs: 97 active+clean; 817 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-11-16T16:50:08.333 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2023-11-16T16:50:08.334 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2023-11-16T16:50:08.334 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2023-11-16T16:50:08.334 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2023-11-16T16:50:08.334 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2023-11-16T16:50:08.334 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2023-11-16T16:50:08.334 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2023-11-16T16:50:08.335 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2023-11-16T16:50:08.335 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-11-16T16:50:08.346 INFO:tasks.kclient:Unmounting kernel clients... 2023-11-16T16:50:08.346 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2023-11-16T16:50:08.347 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-16T16:50:08.386 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2023-11-16T16:50:08.387 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-16T16:50:08.449 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-16T16:50:08.630 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.628+0000 7fd53a68b700 1 -- 172.21.15.1:0/4173679267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd534105380 msgr2=0x7fd534105750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:08.630 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.628+0000 7fd53a68b700 1 --2- 172.21.15.1:0/4173679267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd534105380 0x7fd534105750 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fd5240099c0 tx=0x7fd524009cd0 comp rx=0 tx=0).stop 2023-11-16T16:50:08.630 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.628+0000 7fd53a68b700 1 -- 172.21.15.1:0/4173679267 shutdown_connections 2023-11-16T16:50:08.630 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.628+0000 7fd53a68b700 1 --2- 172.21.15.1:0/4173679267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5341003e0 0x7fd53410c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:08.630 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.628+0000 7fd53a68b700 1 --2- 172.21.15.1:0/4173679267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd534105380 0x7fd534105750 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:08.631 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.628+0000 7fd53a68b700 1 --2- 172.21.15.1:0/4173679267 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5340ffa30 0x7fd5340ffea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:08.631 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.628+0000 7fd53a68b700 1 -- 172.21.15.1:0/4173679267 >> 172.21.15.1:0/4173679267 conn(0x7fd534075470 msgr2=0x7fd534075870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:08.631 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.628+0000 7fd53a68b700 1 -- 172.21.15.1:0/4173679267 shutdown_connections 2023-11-16T16:50:08.631 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.628+0000 7fd53a68b700 1 -- 172.21.15.1:0/4173679267 wait complete. 2023-11-16T16:50:08.631 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.628+0000 7fd53a68b700 1 Processor -- start 2023-11-16T16:50:08.631 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.628+0000 7fd53a68b700 1 -- start start 2023-11-16T16:50:08.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.629+0000 7fd53a68b700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5340ffa30 0x7fd5341963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:08.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.629+0000 7fd53a68b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5341003e0 0x7fd5341968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:08.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.629+0000 7fd53a68b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd534105380 0x7fd53419ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:08.633 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.629+0000 7fd53a68b700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53419b390 con 0x7fd534105380 2023-11-16T16:50:08.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.629+0000 7fd53a68b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53419b500 con 0x7fd5341003e0 2023-11-16T16:50:08.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.629+0000 7fd53a68b700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53419b6c0 con 0x7fd5340ffa30 2023-11-16T16:50:08.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.629+0000 7fd538c28700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd534105380 0x7fd53419ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:08.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.629+0000 7fd538c28700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd534105380 0x7fd53419ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47474/0 (socket says 172.21.15.1:47474) 2023-11-16T16:50:08.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.629+0000 7fd538c28700 1 -- 172.21.15.1:0/3750866202 learned_addr learned my addr 172.21.15.1:0/3750866202 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:50:08.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.629+0000 7fd533fff700 1 --2- 172.21.15.1:0/3750866202 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5340ffa30 0x7fd5341963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:08.634 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.629+0000 7fd52bfff700 1 --2- 172.21.15.1:0/3750866202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5341003e0 0x7fd5341968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:08.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.629+0000 7fd538c28700 1 -- 172.21.15.1:0/3750866202 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5340ffa30 msgr2=0x7fd5341963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:08.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.629+0000 7fd538c28700 1 --2- 172.21.15.1:0/3750866202 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5340ffa30 0x7fd5341963b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:08.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.629+0000 7fd538c28700 1 -- 172.21.15.1:0/3750866202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5341003e0 msgr2=0x7fd5341968f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:08.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.629+0000 7fd538c28700 1 --2- 172.21.15.1:0/3750866202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5341003e0 0x7fd5341968f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:08.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.629+0000 7fd538c28700 1 -- 172.21.15.1:0/3750866202 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5240095f0 con 0x7fd534105380 2023-11-16T16:50:08.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.629+0000 7fd538c28700 1 --2- 172.21.15.1:0/3750866202 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd534105380 0x7fd53419ad50 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fd52c007ee0 tx=0x7fd52c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:08.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.629+0000 7fd531ffb700 1 -- 172.21.15.1:0/3750866202 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52c017070 con 0x7fd534105380 2023-11-16T16:50:08.635 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.630+0000 7fd531ffb700 1 -- 172.21.15.1:0/3750866202 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd52c00f040 con 0x7fd534105380 2023-11-16T16:50:08.636 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.630+0000 7fd53a68b700 1 -- 172.21.15.1:0/3750866202 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5341104c0 con 0x7fd534105380 2023-11-16T16:50:08.636 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.630+0000 7fd531ffb700 1 -- 172.21.15.1:0/3750866202 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52c005740 con 0x7fd534105380 2023-11-16T16:50:08.636 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.630+0000 7fd53a68b700 1 -- 172.21.15.1:0/3750866202 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd534110330 con 0x7fd534105380 2023-11-16T16:50:08.636 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.631+0000 7fd53a68b700 1 -- 172.21.15.1:0/3750866202 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd534110650 con 0x7fd534105380 2023-11-16T16:50:08.636 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.632+0000 7fd531ffb700 1 -- 172.21.15.1:0/3750866202 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fd52c00dc60 con 0x7fd534105380 2023-11-16T16:50:08.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.632+0000 7fd531ffb700 1 --2- 172.21.15.1:0/3750866202 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fd5200a12c0 0x7fd5200a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:08.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.632+0000 7fd533fff700 1 --2- 172.21.15.1:0/3750866202 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fd5200a12c0 0x7fd5200a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:08.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.632+0000 7fd531ffb700 1 -- 172.21.15.1:0/3750866202 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fd52c0cc570 con 0x7fd534105380 2023-11-16T16:50:08.637 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.633+0000 7fd533fff700 1 --2- 172.21.15.1:0/3750866202 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fd5200a12c0 0x7fd5200a3770 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd524006010 tx=0x7fd524003680 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:08.638 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.637+0000 7fd531ffb700 1 -- 172.21.15.1:0/3750866202 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd52c095970 con 0x7fd534105380 2023-11-16T16:50:08.841 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.840+0000 7fd53a68b700 1 -- 172.21.15.1:0/3750866202 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd534067f40 con 0x7fd534105380 2023-11-16T16:50:08.843 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.841+0000 7fd531ffb700 1 -- 172.21.15.1:0/3750866202 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+22931 (secure 0 0 0) 0x7fd52c098f90 con 0x7fd534105380 2023-11-16T16:50:08.843 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:50:08.843 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":95,"fsid":"35d9b010-849d-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:30:06.217778+0000","modified":"2023-11-16T16:49:16.352495+0000","last_up_change":"2023-11-16T16:36:31.735403+0000","last_in_change":"2023-11-16T16:36:14.083021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:33:15.916723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-16T16:37:12.158031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-16T16:37:13.272343+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":94,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a8480507-5275-4e52-953b-626110beb436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6803","nonce":2469671609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6805","nonce":2469671609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6809","nonce":2469671609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6807","nonce":2469671609}]},"public_addr":"172.21.15.1:6803/2469671609","cluster_addr":"172.21.15.1:6805/2469671609","heartbeat_back_addr":"172.21.15.1:6809/2469671609","heartbeat_front_addr":"172.21.15.1:6807/2469671609","state":["exists","up"]},{"osd":1,"uuid":"1d011db2-99d1-4575-bdc5-51f88a308fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6801","nonce":247994796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6803","nonce":247994796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6807","nonce":247994796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6805","nonce":247994796}]},"public_addr":"172.21.15.26:6801/247994796","cluster_addr":"172.21.15.26:6803/247994796","heartbeat_back_addr":"172.21.15.26:6807/247994796","heartbeat_front_addr":"172.21.15.26:6805/247994796","state":["exists","up"]},{"osd":2,"uuid":"4d7a2bd1-3c8e-4546-a243-f0354473b031","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6801","nonce":1659316290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6803","nonce":1659316290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6807","nonce":1659316290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6805","nonce":1659316290}]},"public_addr":"172.21.15.53:6801/1659316290","cluster_addr":"172.21.15.53:6803/1659316290","heartbeat_back_addr":"172.21.15.53:6807/1659316290","heartbeat_front_addr":"172.21.15.53:6805/1659316290","state":["exists","up"]},{"osd":3,"uuid":"9c290fd9-e121-4a81-9687-b0eb93debd86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6811","nonce":2728987281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6813","nonce":2728987281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6817","nonce":2728987281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6815","nonce":2728987281}]},"public_addr":"172.21.15.1:6811/2728987281","cluster_addr":"172.21.15.1:6813/2728987281","heartbeat_back_addr":"172.21.15.1:6817/2728987281","heartbeat_front_addr":"172.21.15.1:6815/2728987281","state":["exists","up"]},{"osd":4,"uuid":"533226bc-cc57-4f15-9e29-820294d57e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6809","nonce":2954740024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6811","nonce":2954740024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6815","nonce":2954740024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6813","nonce":2954740024}]},"public_addr":"172.21.15.26:6809/2954740024","cluster_addr":"172.21.15.26:6811/2954740024","heartbeat_back_addr":"172.21.15.26:6815/2954740024","heartbeat_front_addr":"172.21.15.26:6813/2954740024","state":["exists","up"]},{"osd":5,"uuid":"d3ab25eb-e1d4-4557-bc89-efc9072401ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6809","nonce":3966666318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6811","nonce":3966666318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6815","nonce":3966666318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6813","nonce":3966666318}]},"public_addr":"172.21.15.53:6809/3966666318","cluster_addr":"172.21.15.53:6811/3966666318","heartbeat_back_addr":"172.21.15.53:6815/3966666318","heartbeat_front_addr":"172.21.15.53:6813/3966666318","state":["exists","up"]},{"osd":6,"uuid":"180f506e-afb8-4043-b0b4-2dc86f6d4bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6819","nonce":3364032646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6821","nonce":3364032646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6825","nonce":3364032646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6823","nonce":3364032646}]},"public_addr":"172.21.15.1:6819/3364032646","cluster_addr":"172.21.15.1:6821/3364032646","heartbeat_back_addr":"172.21.15.1:6825/3364032646","heartbeat_front_addr":"172.21.15.1:6823/3364032646","state":["exists","up"]},{"osd":7,"uuid":"0d837bac-194f-498d-b7fb-b3147e992329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6817","nonce":3766901391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6819","nonce":3766901391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6823","nonce":3766901391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6821","nonce":3766901391}]},"public_addr":"172.21.15.26:6817/3766901391","cluster_addr":"172.21.15.26:6819/3766901391","heartbeat_back_addr":"172.21.15.26:6823/3766901391","heartbeat_front_addr":"172.21.15.26:6821/3766901391","state":["exists","up"]},{"osd":8,"uuid":"933f231a-7454-47b0-9291-f1fa64f0baec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6817","nonce":3224335871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6819","nonce":3224335871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6823","nonce":3224335871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6821","nonce":3224335871}]},"public_addr":"172.21.15.53:6817/3224335871","cluster_addr":"172.21.15.53:6819/3224335871","heartbeat_back_addr":"172.21.15.53:6823/3224335871","heartbeat_front_addr":"172.21.15.53:6821/3224335871","state":["exists","up"]},{"osd":9,"uuid":"245fcbb3-c651-4703-ab80-309dc98bd983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6827","nonce":1391199844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6829","nonce":1391199844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6833","nonce":1391199844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6831","nonce":1391199844}]},"public_addr":"172.21.15.1:6827/1391199844","cluster_addr":"172.21.15.1:6829/1391199844","heartbeat_back_addr":"172.21.15.1:6833/1391199844","heartbeat_front_addr":"172.21.15.1:6831/1391199844","state":["exists","up"]},{"osd":10,"uuid":"2c090e7a-5240-4d52-ae4f-e94fe05383d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6825","nonce":1773545447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6827","nonce":1773545447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6831","nonce":1773545447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6829","nonce":1773545447}]},"public_addr":"172.21.15.26:6825/1773545447","cluster_addr":"172.21.15.26:6827/1773545447","heartbeat_back_addr":"172.21.15.26:6831/1773545447","heartbeat_front_addr":"172.21.15.26:6829/1773545447","state":["exists","up"]},{"osd":11,"uuid":"4511e32d-e469-45e1-8310-0ff400cb7b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6825","nonce":2564669546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6827","nonce":2564669546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6831","nonce":2564669546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6829","nonce":2564669546}]},"public_addr":"172.21.15.53:6825/2564669546","cluster_addr":"172.21.15.53:6827/2564669546","heartbeat_back_addr":"172.21.15.53:6831/2564669546","heartbeat_front_addr":"172.21.15.53:6829/2564669546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:28.140250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:49.885461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:12.521487+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:34.926197+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:56.815594+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:18.387751+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:40.587106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:02.319370+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:23.832298+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:46.338040+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:08.730414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:30.050070+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1210074029":"2023-11-17T16:39:09.360183+0000","172.21.15.1:0/2308375878":"2023-11-17T16:39:09.360183+0000","172.21.15.1:0/3270932790":"2023-11-17T16:39:09.360183+0000","172.21.15.1:6801/2490729488":"2023-11-17T16:39:09.360183+0000","172.21.15.1:6800/2490729488":"2023-11-17T16:39:09.360183+0000","172.21.15.1:0/1494336948":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6801/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6800/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/1768923877":"2023-11-17T16:39:09.360183+0000","172.21.15.1:6800/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/1641976669":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/2895923638":"2023-11-17T16:30:35.064289+0000","172.21.15.1:6801/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/78811711":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/4131957708":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/344403698":"2023-11-17T16:30:35.064289+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:50:08.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.843+0000 7fd53a68b700 1 -- 172.21.15.1:0/3750866202 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fd5200a12c0 msgr2=0x7fd5200a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:08.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.843+0000 7fd53a68b700 1 --2- 172.21.15.1:0/3750866202 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fd5200a12c0 0x7fd5200a3770 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd524006010 tx=0x7fd524003680 comp rx=0 tx=0).stop 2023-11-16T16:50:08.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.843+0000 7fd53a68b700 1 -- 172.21.15.1:0/3750866202 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd534105380 msgr2=0x7fd53419ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:08.845 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.843+0000 7fd53a68b700 1 --2- 172.21.15.1:0/3750866202 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd534105380 0x7fd53419ad50 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fd52c007ee0 tx=0x7fd52c00d3b0 comp rx=0 tx=0).stop 2023-11-16T16:50:08.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.843+0000 7fd53a68b700 1 -- 172.21.15.1:0/3750866202 shutdown_connections 2023-11-16T16:50:08.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.843+0000 7fd53a68b700 1 --2- 172.21.15.1:0/3750866202 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fd5200a12c0 0x7fd5200a3770 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:08.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.843+0000 7fd53a68b700 1 --2- 172.21.15.1:0/3750866202 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd534105380 0x7fd53419ad50 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:08.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.843+0000 7fd53a68b700 1 --2- 172.21.15.1:0/3750866202 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd5340ffa30 0x7fd5341963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:08.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.843+0000 7fd53a68b700 1 --2- 172.21.15.1:0/3750866202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5341003e0 0x7fd5341968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:08.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.843+0000 7fd53a68b700 1 -- 172.21.15.1:0/3750866202 >> 172.21.15.1:0/3750866202 conn(0x7fd534075470 msgr2=0x7fd5340fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:08.846 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.843+0000 7fd53a68b700 1 -- 172.21.15.1:0/3750866202 shutdown_connections 2023-11-16T16:50:08.847 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.843+0000 7fd53a68b700 1 -- 172.21.15.1:0/3750866202 wait complete. 2023-11-16T16:50:08.858 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-16T16:33:15.916723+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-16T16:37:12.158031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-16T16:37:13.272343+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 94, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-16T16:50:08.858 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-16T16:50:08.988 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.986+0000 7fd34d510700 1 -- 172.21.15.1:0/416004326 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd348103490 msgr2=0x7fd34810c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:08.988 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.986+0000 7fd34d510700 1 --2- 172.21.15.1:0/416004326 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd348103490 0x7fd34810c100 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fd34000b0d0 tx=0x7fd34000b3e0 comp rx=0 tx=0).stop 2023-11-16T16:50:08.989 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.986+0000 7fd34d510700 1 -- 172.21.15.1:0/416004326 shutdown_connections 2023-11-16T16:50:08.989 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.986+0000 7fd34d510700 1 --2- 172.21.15.1:0/416004326 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd348103490 0x7fd34810c100 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:08.989 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.986+0000 7fd34d510700 1 --2- 172.21.15.1:0/416004326 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3480ffcf0 0x7fd3481000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:08.989 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.986+0000 7fd34d510700 1 --2- 172.21.15.1:0/416004326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd348100690 0x7fd348102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:08.989 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.986+0000 7fd34d510700 1 -- 172.21.15.1:0/416004326 >> 172.21.15.1:0/416004326 conn(0x7fd3480763a0 msgr2=0x7fd3480767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:08.989 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.986+0000 7fd34d510700 1 -- 172.21.15.1:0/416004326 shutdown_connections 2023-11-16T16:50:08.989 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.987+0000 7fd34d510700 1 -- 172.21.15.1:0/416004326 wait complete. 2023-11-16T16:50:08.990 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.987+0000 7fd34d510700 1 Processor -- start 2023-11-16T16:50:08.992 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.987+0000 7fd34d510700 1 -- start start 2023-11-16T16:50:08.992 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.987+0000 7fd34d510700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3480ffcf0 0x7fd348100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:08.992 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.987+0000 7fd34d510700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd348100690 0x7fd348101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:08.992 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.987+0000 7fd34d510700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd348103490 0x7fd3481a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:08.992 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.987+0000 7fd34d510700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd348101b80 con 0x7fd348103490 2023-11-16T16:50:08.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.987+0000 7fd34d510700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3481a4bb0 con 0x7fd348100690 2023-11-16T16:50:08.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.987+0000 7fd34d510700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3481a4cf0 con 0x7fd3480ffcf0 2023-11-16T16:50:08.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.988+0000 7fd3477fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd348103490 0x7fd3481a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:08.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.988+0000 7fd3477fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd348103490 0x7fd3481a4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47488/0 (socket says 172.21.15.1:47488) 2023-11-16T16:50:08.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.988+0000 7fd3477fe700 1 -- 172.21.15.1:0/2680329556 learned_addr learned my addr 172.21.15.1:0/2680329556 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:50:08.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.988+0000 7fd3467fc700 1 --2- 172.21.15.1:0/2680329556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd348100690 0x7fd348101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:08.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.988+0000 7fd346ffd700 1 --2- 172.21.15.1:0/2680329556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3480ffcf0 0x7fd348100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:08.993 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.988+0000 7fd3477fe700 1 -- 172.21.15.1:0/2680329556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3480ffcf0 msgr2=0x7fd348100e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:08.994 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.988+0000 7fd3477fe700 1 --2- 172.21.15.1:0/2680329556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3480ffcf0 0x7fd348100e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:08.994 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.988+0000 7fd3477fe700 1 -- 172.21.15.1:0/2680329556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd348100690 msgr2=0x7fd348101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:08.994 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.988+0000 7fd3477fe700 1 --2- 172.21.15.1:0/2680329556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd348100690 0x7fd348101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:08.994 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.988+0000 7fd3477fe700 1 -- 172.21.15.1:0/2680329556 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd340009d00 con 0x7fd348103490 2023-11-16T16:50:08.994 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.988+0000 7fd346ffd700 1 --2- 172.21.15.1:0/2680329556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3480ffcf0 0x7fd348100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:50:08.994 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.988+0000 7fd3477fe700 1 --2- 172.21.15.1:0/2680329556 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd348103490 0x7fd3481a4670 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fd340007820 tx=0x7fd340007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:08.994 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.988+0000 7fd32ffff700 1 -- 172.21.15.1:0/2680329556 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34000e040 con 0x7fd348103490 2023-11-16T16:50:08.994 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.988+0000 7fd34d510700 1 -- 172.21.15.1:0/2680329556 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd34810f340 con 0x7fd348103490 2023-11-16T16:50:08.995 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.988+0000 7fd32ffff700 1 -- 172.21.15.1:0/2680329556 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd34000bdb0 con 0x7fd348103490 2023-11-16T16:50:08.995 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.988+0000 7fd32ffff700 1 -- 172.21.15.1:0/2680329556 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34001d470 con 0x7fd348103490 2023-11-16T16:50:08.995 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.988+0000 7fd34d510700 1 -- 172.21.15.1:0/2680329556 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd34810f1b0 con 0x7fd348103490 2023-11-16T16:50:08.995 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.989+0000 7fd34d510700 1 -- 172.21.15.1:0/2680329556 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd34810f4d0 con 0x7fd348103490 2023-11-16T16:50:08.995 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.990+0000 7fd32ffff700 1 -- 172.21.15.1:0/2680329556 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fd3400050b0 con 0x7fd348103490 2023-11-16T16:50:08.996 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.991+0000 7fd32ffff700 1 --2- 172.21.15.1:0/2680329556 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fd3300a12c0 0x7fd3300a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:08.996 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.991+0000 7fd32ffff700 1 -- 172.21.15.1:0/2680329556 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7fd3400ce210 con 0x7fd348103490 2023-11-16T16:50:08.996 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.991+0000 7fd346ffd700 1 --2- 172.21.15.1:0/2680329556 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fd3300a12c0 0x7fd3300a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:08.996 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.991+0000 7fd346ffd700 1 --2- 172.21.15.1:0/2680329556 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fd3300a12c0 0x7fd3300a3770 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd338009be0 tx=0x7fd338009260 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:08.997 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:08.994+0000 7fd32ffff700 1 -- 172.21.15.1:0/2680329556 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd3400976a0 con 0x7fd348103490 2023-11-16T16:50:09.182 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:08 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3750866202' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:50:09.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.193+0000 7fd34d510700 1 -- 172.21.15.1:0/2680329556 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd3481a5660 con 0x7fd348103490 2023-11-16T16:50:09.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.194+0000 7fd32ffff700 1 -- 172.21.15.1:0/2680329556 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd34009acc0 con 0x7fd348103490 2023-11-16T16:50:09.195 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2023-11-16T16:50:09.197 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.196+0000 7fd34d510700 1 -- 172.21.15.1:0/2680329556 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fd3300a12c0 msgr2=0x7fd3300a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:09.198 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.196+0000 7fd34d510700 1 --2- 172.21.15.1:0/2680329556 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fd3300a12c0 0x7fd3300a3770 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd338009be0 tx=0x7fd338009260 comp rx=0 tx=0).stop 2023-11-16T16:50:09.198 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.196+0000 7fd34d510700 1 -- 172.21.15.1:0/2680329556 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd348103490 msgr2=0x7fd3481a4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:09.198 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.196+0000 7fd34d510700 1 --2- 172.21.15.1:0/2680329556 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd348103490 0x7fd3481a4670 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fd340007820 tx=0x7fd340007900 comp rx=0 tx=0).stop 2023-11-16T16:50:09.198 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.196+0000 7fd34d510700 1 -- 172.21.15.1:0/2680329556 shutdown_connections 2023-11-16T16:50:09.198 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.196+0000 7fd34d510700 1 --2- 172.21.15.1:0/2680329556 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fd3300a12c0 0x7fd3300a3770 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:09.198 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.196+0000 7fd34d510700 1 --2- 172.21.15.1:0/2680329556 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd348103490 0x7fd3481a4670 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:09.198 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.196+0000 7fd34d510700 1 --2- 172.21.15.1:0/2680329556 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fd3480ffcf0 0x7fd348100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:09.199 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.196+0000 7fd34d510700 1 --2- 172.21.15.1:0/2680329556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd348100690 0x7fd348101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:09.199 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.196+0000 7fd34d510700 1 -- 172.21.15.1:0/2680329556 >> 172.21.15.1:0/2680329556 conn(0x7fd3480763a0 msgr2=0x7fd3480690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:09.199 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.196+0000 7fd34d510700 1 -- 172.21.15.1:0/2680329556 shutdown_connections 2023-11-16T16:50:09.199 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.196+0000 7fd34d510700 1 -- 172.21.15.1:0/2680329556 wait complete. 2023-11-16T16:50:09.210 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-16T16:50:09.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:08 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/3750866202' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:50:09.330 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:08 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/3750866202' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:50:09.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.338+0000 7f2aeeb02700 1 -- 172.21.15.1:0/1706729101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ae8074610 msgr2=0x7f2ae80749e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:09.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.338+0000 7f2aeeb02700 1 --2- 172.21.15.1:0/1706729101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ae8074610 0x7f2ae80749e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2adc009990 tx=0x7f2adc009ca0 comp rx=0 tx=0).stop 2023-11-16T16:50:09.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.338+0000 7f2aeeb02700 1 -- 172.21.15.1:0/1706729101 shutdown_connections 2023-11-16T16:50:09.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.338+0000 7f2aeeb02700 1 --2- 172.21.15.1:0/1706729101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ae8073900 0x7f2ae810eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:09.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.338+0000 7f2aeeb02700 1 --2- 172.21.15.1:0/1706729101 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ae8074610 0x7f2ae80749e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:09.340 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.338+0000 7f2aeeb02700 1 --2- 172.21.15.1:0/1706729101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ae8072db0 0x7f2ae8073220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:09.341 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.338+0000 7f2aeeb02700 1 -- 172.21.15.1:0/1706729101 >> 172.21.15.1:0/1706729101 conn(0x7f2ae8077560 msgr2=0x7f2ae8077960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:09.341 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.338+0000 7f2aeeb02700 1 -- 172.21.15.1:0/1706729101 shutdown_connections 2023-11-16T16:50:09.341 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.338+0000 7f2aeeb02700 1 -- 172.21.15.1:0/1706729101 wait complete. 2023-11-16T16:50:09.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.339+0000 7f2aeeb02700 1 Processor -- start 2023-11-16T16:50:09.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.339+0000 7f2aeeb02700 1 -- start start 2023-11-16T16:50:09.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.339+0000 7f2aeeb02700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ae8072db0 0x7f2ae8198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:09.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.339+0000 7f2aeeb02700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ae8073900 0x7f2ae81989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:09.343 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.339+0000 7f2aeeb02700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ae8074610 0x7f2ae819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:09.344 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.339+0000 7f2aeeb02700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae819d470 con 0x7f2ae8073900 2023-11-16T16:50:09.344 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.339+0000 7f2aeeb02700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae819d5e0 con 0x7f2ae8074610 2023-11-16T16:50:09.344 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.339+0000 7f2aeeb02700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae819d7a0 con 0x7f2ae8072db0 2023-11-16T16:50:09.344 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.339+0000 7f2aec89e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ae8072db0 0x7f2ae8198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:09.344 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.339+0000 7f2ae7fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ae8073900 0x7f2ae81989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:09.344 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.339+0000 7f2aec89e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ae8072db0 0x7f2ae8198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:44180/0 (socket says 172.21.15.1:44180) 2023-11-16T16:50:09.345 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.339+0000 7f2aec89e700 1 -- 172.21.15.1:0/1609611500 learned_addr learned my addr 172.21.15.1:0/1609611500 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:50:09.345 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.339+0000 7f2aed09f700 1 --2- 172.21.15.1:0/1609611500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ae8074610 0x7f2ae819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:09.345 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.339+0000 7f2aec89e700 1 -- 172.21.15.1:0/1609611500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ae8074610 msgr2=0x7f2ae819ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:09.345 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.339+0000 7f2aec89e700 1 --2- 172.21.15.1:0/1609611500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ae8074610 0x7f2ae819ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:09.345 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.339+0000 7f2aec89e700 1 -- 172.21.15.1:0/1609611500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ae8073900 msgr2=0x7f2ae81989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:09.345 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.339+0000 7f2aec89e700 1 --2- 172.21.15.1:0/1609611500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ae8073900 0x7f2ae81989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:09.345 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.339+0000 7f2aec89e700 1 -- 172.21.15.1:0/1609611500 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad4009520 con 0x7f2ae8072db0 2023-11-16T16:50:09.345 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.339+0000 7f2ae7fff700 1 --2- 172.21.15.1:0/1609611500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ae8073900 0x7f2ae81989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-16T16:50:09.346 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.339+0000 7f2aed09f700 1 --2- 172.21.15.1:0/1609611500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ae8074610 0x7f2ae819ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:50:09.346 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.340+0000 7f2aec89e700 1 --2- 172.21.15.1:0/1609611500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ae8072db0 0x7f2ae8198490 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2adc000c00 tx=0x7f2adc004790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:09.346 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.340+0000 7f2ae5ffb700 1 -- 172.21.15.1:0/1609611500 <== mon.1 v2:172.21.15.53:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc01d070 con 0x7f2ae8072db0 2023-11-16T16:50:09.346 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.340+0000 7f2ae5ffb700 1 -- 172.21.15.1:0/1609611500 <== mon.1 v2:172.21.15.53:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2adc021410 con 0x7f2ae8072db0 2023-11-16T16:50:09.346 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.340+0000 7f2aeeb02700 1 -- 172.21.15.1:0/1609611500 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2adc0095f0 con 0x7f2ae8072db0 2023-11-16T16:50:09.346 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.340+0000 7f2ae5ffb700 1 -- 172.21.15.1:0/1609611500 <== mon.1 v2:172.21.15.53:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc017650 con 0x7f2ae8072db0 2023-11-16T16:50:09.346 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.340+0000 7f2aeeb02700 1 -- 172.21.15.1:0/1609611500 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ae81124f0 con 0x7f2ae8072db0 2023-11-16T16:50:09.346 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.341+0000 7f2aeeb02700 1 -- 172.21.15.1:0/1609611500 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ae8112360 con 0x7f2ae8072db0 2023-11-16T16:50:09.347 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.342+0000 7f2ae5ffb700 1 -- 172.21.15.1:0/1609611500 <== mon.1 v2:172.21.15.53:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f2adc021580 con 0x7f2ae8072db0 2023-11-16T16:50:09.347 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.343+0000 7f2ae5ffb700 1 --2- 172.21.15.1:0/1609611500 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f2ad80a12c0 0x7f2ad80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:09.348 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.343+0000 7f2ae7fff700 1 --2- 172.21.15.1:0/1609611500 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f2ad80a12c0 0x7f2ad80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:09.348 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.343+0000 7f2ae5ffb700 1 -- 172.21.15.1:0/1609611500 <== mon.1 v2:172.21.15.53:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f2adc0ce7f0 con 0x7f2ae8072db0 2023-11-16T16:50:09.348 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.343+0000 7f2ae7fff700 1 --2- 172.21.15.1:0/1609611500 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f2ad80a12c0 0x7f2ad80a3770 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4009fd0 tx=0x7f2ad4009260 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:09.348 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.347+0000 7f2ae5ffb700 1 -- 172.21.15.1:0/1609611500 <== mon.1 v2:172.21.15.53:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2adc097c20 con 0x7f2ae8072db0 2023-11-16T16:50:09.551 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.549+0000 7f2aeeb02700 1 -- 172.21.15.1:0/1609611500 --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2ae8112680 con 0x7f2ae8072db0 2023-11-16T16:50:09.551 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.549+0000 7f2ae5ffb700 1 -- 172.21.15.1:0/1609611500 <== mon.1 v2:172.21.15.53:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2adc09b240 con 0x7f2ae8072db0 2023-11-16T16:50:09.552 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2023-11-16T16:50:09.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.552+0000 7f2aeeb02700 1 -- 172.21.15.1:0/1609611500 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f2ad80a12c0 msgr2=0x7f2ad80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:09.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.552+0000 7f2aeeb02700 1 --2- 172.21.15.1:0/1609611500 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f2ad80a12c0 0x7f2ad80a3770 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4009fd0 tx=0x7f2ad4009260 comp rx=0 tx=0).stop 2023-11-16T16:50:09.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.552+0000 7f2aeeb02700 1 -- 172.21.15.1:0/1609611500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ae8072db0 msgr2=0x7f2ae8198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:09.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.552+0000 7f2aeeb02700 1 --2- 172.21.15.1:0/1609611500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ae8072db0 0x7f2ae8198490 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2adc000c00 tx=0x7f2adc004790 comp rx=0 tx=0).stop 2023-11-16T16:50:09.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.552+0000 7f2aeeb02700 1 -- 172.21.15.1:0/1609611500 shutdown_connections 2023-11-16T16:50:09.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.552+0000 7f2aeeb02700 1 --2- 172.21.15.1:0/1609611500 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f2ad80a12c0 0x7f2ad80a3770 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:09.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.552+0000 7f2aeeb02700 1 --2- 172.21.15.1:0/1609611500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ae8074610 0x7f2ae819ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:09.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.552+0000 7f2aeeb02700 1 --2- 172.21.15.1:0/1609611500 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f2ae8072db0 0x7f2ae8198490 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:09.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.552+0000 7f2aeeb02700 1 --2- 172.21.15.1:0/1609611500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ae8073900 0x7f2ae81989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:09.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.552+0000 7f2aeeb02700 1 -- 172.21.15.1:0/1609611500 >> 172.21.15.1:0/1609611500 conn(0x7f2ae8077560 msgr2=0x7f2ae8109010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:09.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.552+0000 7f2aeeb02700 1 -- 172.21.15.1:0/1609611500 shutdown_connections 2023-11-16T16:50:09.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.552+0000 7f2aeeb02700 1 -- 172.21.15.1:0/1609611500 wait complete. 2023-11-16T16:50:09.566 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-16T16:50:09.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.693+0000 7f2683034700 1 -- 172.21.15.1:0/2618363793 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f267c100cc0 msgr2=0x7f267c101090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:09.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.693+0000 7f2683034700 1 --2- 172.21.15.1:0/2618363793 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f267c100cc0 0x7f267c101090 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f26700099c0 tx=0x7f2670009cd0 comp rx=0 tx=0).stop 2023-11-16T16:50:09.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.693+0000 7f2683034700 1 -- 172.21.15.1:0/2618363793 shutdown_connections 2023-11-16T16:50:09.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.693+0000 7f2683034700 1 --2- 172.21.15.1:0/2618363793 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f267c1055c0 0x7f267c10d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:09.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.693+0000 7f2683034700 1 --2- 172.21.15.1:0/2618363793 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f267c100cc0 0x7f267c101090 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:09.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.693+0000 7f2683034700 1 --2- 172.21.15.1:0/2618363793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f267c101660 0x7f267c104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:09.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.693+0000 7f2683034700 1 -- 172.21.15.1:0/2618363793 >> 172.21.15.1:0/2618363793 conn(0x7f267c077560 msgr2=0x7f267c077960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:09.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.693+0000 7f2683034700 1 -- 172.21.15.1:0/2618363793 shutdown_connections 2023-11-16T16:50:09.696 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.693+0000 7f2683034700 1 -- 172.21.15.1:0/2618363793 wait complete. 2023-11-16T16:50:09.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.694+0000 7f2683034700 1 Processor -- start 2023-11-16T16:50:09.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.694+0000 7f2683034700 1 -- start start 2023-11-16T16:50:09.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.694+0000 7f2683034700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f267c100cc0 0x7f267c196350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:09.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.694+0000 7f2683034700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f267c101660 0x7f267c196890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:09.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.694+0000 7f2683034700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f267c1055c0 0x7f267c19acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:09.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.694+0000 7f2683034700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267c19b330 con 0x7f267c1055c0 2023-11-16T16:50:09.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.694+0000 7f2683034700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267c19b4a0 con 0x7f267c100cc0 2023-11-16T16:50:09.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.694+0000 7f2683034700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267c19b660 con 0x7f267c101660 2023-11-16T16:50:09.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.695+0000 7f26815d1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f267c1055c0 0x7f267c19acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:09.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.695+0000 7f26815d1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f267c1055c0 0x7f267c19acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47516/0 (socket says 172.21.15.1:47516) 2023-11-16T16:50:09.700 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.695+0000 7f26815d1700 1 -- 172.21.15.1:0/2934129294 learned_addr learned my addr 172.21.15.1:0/2934129294 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:50:09.700 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.695+0000 7f267bfff700 1 --2- 172.21.15.1:0/2934129294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f267c101660 0x7f267c196890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:09.700 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.695+0000 7f2680dd0700 1 --2- 172.21.15.1:0/2934129294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f267c100cc0 0x7f267c196350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:09.700 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.695+0000 7f26815d1700 1 -- 172.21.15.1:0/2934129294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f267c101660 msgr2=0x7f267c196890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:09.700 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.695+0000 7f26815d1700 1 --2- 172.21.15.1:0/2934129294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f267c101660 0x7f267c196890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:09.700 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.695+0000 7f26815d1700 1 -- 172.21.15.1:0/2934129294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f267c100cc0 msgr2=0x7f267c196350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:09.700 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.695+0000 7f26815d1700 1 --2- 172.21.15.1:0/2934129294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f267c100cc0 0x7f267c196350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:09.700 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.695+0000 7f26815d1700 1 -- 172.21.15.1:0/2934129294 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26700095f0 con 0x7f267c1055c0 2023-11-16T16:50:09.701 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.695+0000 7f26815d1700 1 --2- 172.21.15.1:0/2934129294 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f267c1055c0 0x7f267c19acf0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f267400ada0 tx=0x7f267400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:09.701 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.695+0000 7f2679ffb700 1 -- 172.21.15.1:0/2934129294 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267400f070 con 0x7f267c1055c0 2023-11-16T16:50:09.701 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.695+0000 7f2679ffb700 1 -- 172.21.15.1:0/2934129294 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2674007b30 con 0x7f267c1055c0 2023-11-16T16:50:09.701 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.695+0000 7f2683034700 1 -- 172.21.15.1:0/2934129294 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f267c110330 con 0x7f267c1055c0 2023-11-16T16:50:09.701 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.695+0000 7f2679ffb700 1 -- 172.21.15.1:0/2934129294 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2674005280 con 0x7f267c1055c0 2023-11-16T16:50:09.701 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.696+0000 7f2683034700 1 -- 172.21.15.1:0/2934129294 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f267c1101a0 con 0x7f267c1055c0 2023-11-16T16:50:09.701 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.697+0000 7f2683034700 1 -- 172.21.15.1:0/2934129294 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f267c1104c0 con 0x7f267c1055c0 2023-11-16T16:50:09.703 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.697+0000 7f2679ffb700 1 -- 172.21.15.1:0/2934129294 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f267400c8d0 con 0x7f267c1055c0 2023-11-16T16:50:09.703 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.698+0000 7f2679ffb700 1 --2- 172.21.15.1:0/2934129294 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f266c0a12c0 0x7f266c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:09.703 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.698+0000 7f2679ffb700 1 -- 172.21.15.1:0/2934129294 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8542+0+0 (secure 0 0 0) 0x7f26740cc290 con 0x7f267c1055c0 2023-11-16T16:50:09.703 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.698+0000 7f2680dd0700 1 --2- 172.21.15.1:0/2934129294 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f266c0a12c0 0x7f266c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:09.703 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.699+0000 7f2680dd0700 1 --2- 172.21.15.1:0/2934129294 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f266c0a12c0 0x7f266c0a3770 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2670000c00 tx=0x7f2670005c40 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:09.704 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.702+0000 7f2679ffb700 1 -- 172.21.15.1:0/2934129294 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f26740956c0 con 0x7f267c1055c0 2023-11-16T16:50:09.902 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.900+0000 7f2683034700 1 -- 172.21.15.1:0/2934129294 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f267c1a33c0 con 0x7f267c1055c0 2023-11-16T16:50:09.902 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.901+0000 7f2679ffb700 1 -- 172.21.15.1:0/2934129294 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2674098ce0 con 0x7f267c1055c0 2023-11-16T16:50:09.902 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2023-11-16T16:50:09.915 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.913+0000 7f2683034700 1 -- 172.21.15.1:0/2934129294 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f266c0a12c0 msgr2=0x7f266c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:09.915 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.913+0000 7f2683034700 1 --2- 172.21.15.1:0/2934129294 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f266c0a12c0 0x7f266c0a3770 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2670000c00 tx=0x7f2670005c40 comp rx=0 tx=0).stop 2023-11-16T16:50:09.915 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.913+0000 7f2683034700 1 -- 172.21.15.1:0/2934129294 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f267c1055c0 msgr2=0x7f267c19acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:09.916 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.913+0000 7f2683034700 1 --2- 172.21.15.1:0/2934129294 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f267c1055c0 0x7f267c19acf0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f267400ada0 tx=0x7f267400c330 comp rx=0 tx=0).stop 2023-11-16T16:50:09.917 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.916+0000 7f2683034700 1 -- 172.21.15.1:0/2934129294 shutdown_connections 2023-11-16T16:50:09.917 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.916+0000 7f2683034700 1 --2- 172.21.15.1:0/2934129294 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f266c0a12c0 0x7f266c0a3770 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:09.917 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.916+0000 7f2683034700 1 --2- 172.21.15.1:0/2934129294 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f267c1055c0 0x7f267c19acf0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:09.917 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.916+0000 7f2683034700 1 --2- 172.21.15.1:0/2934129294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f267c100cc0 0x7f267c196350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:09.918 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.916+0000 7f2683034700 1 --2- 172.21.15.1:0/2934129294 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f267c101660 0x7f267c196890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:09.918 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.916+0000 7f2683034700 1 -- 172.21.15.1:0/2934129294 >> 172.21.15.1:0/2934129294 conn(0x7f267c077560 msgr2=0x7f267c103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:09.920 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.918+0000 7f2683034700 1 -- 172.21.15.1:0/2934129294 shutdown_connections 2023-11-16T16:50:09.920 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:09.918+0000 7f2683034700 1 -- 172.21.15.1:0/2934129294 wait complete. 2023-11-16T16:50:09.942 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-16T16:50:10.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:09 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/2680329556' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:50:10.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:09 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1609611500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:50:10.183 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:09 smithi001 ceph-mon[98800]: pgmap v341: 97 pgs: 97 active+clean; 817 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-11-16T16:50:10.188 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.186+0000 7f8fdcf96700 1 -- 172.21.15.1:0/4290822575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fd81113f0 msgr2=0x7f8fd81137d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:10.188 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.186+0000 7f8fdcf96700 1 --2- 172.21.15.1:0/4290822575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fd81113f0 0x7f8fd81137d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0009f90 tx=0x7f8fd000b330 comp rx=0 tx=0).stop 2023-11-16T16:50:10.189 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.187+0000 7f8fdcf96700 1 -- 172.21.15.1:0/4290822575 shutdown_connections 2023-11-16T16:50:10.189 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.187+0000 7f8fdcf96700 1 --2- 172.21.15.1:0/4290822575 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fd81113f0 0x7f8fd81137d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.189 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.187+0000 7f8fdcf96700 1 --2- 172.21.15.1:0/4290822575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fd80ff4a0 0x7f8fd8106500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.189 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.187+0000 7f8fdcf96700 1 --2- 172.21.15.1:0/4290822575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fd80ff870 0x7f8fd80ffce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.189 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.187+0000 7f8fdcf96700 1 -- 172.21.15.1:0/4290822575 >> 172.21.15.1:0/4290822575 conn(0x7f8fd8074a50 msgr2=0x7f8fd8074e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:10.194 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.187+0000 7f8fdcf96700 1 -- 172.21.15.1:0/4290822575 shutdown_connections 2023-11-16T16:50:10.194 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.188+0000 7f8fdcf96700 1 -- 172.21.15.1:0/4290822575 wait complete. 2023-11-16T16:50:10.194 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.189+0000 7f8fdcf96700 1 Processor -- start 2023-11-16T16:50:10.194 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.189+0000 7f8fdcf96700 1 -- start start 2023-11-16T16:50:10.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.190+0000 7f8fdcf96700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fd80ff4a0 0x7f8fd8107660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:10.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.190+0000 7f8fdcf96700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fd80ff870 0x7f8fd8107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:10.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.190+0000 7f8fdcf96700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fd81113f0 0x7f8fd81b0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:10.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.190+0000 7f8fdcf96700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd81b13c0 con 0x7f8fd80ff4a0 2023-11-16T16:50:10.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.190+0000 7f8fdcf96700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd806a540 con 0x7f8fd80ff870 2023-11-16T16:50:10.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.190+0000 7f8fdcf96700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd806a700 con 0x7f8fd81113f0 2023-11-16T16:50:10.195 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.190+0000 7f8fd5d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fd80ff870 0x7f8fd8107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:10.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.190+0000 7f8fd659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fd80ff4a0 0x7f8fd8107660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:10.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.190+0000 7f8fd659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fd80ff4a0 0x7f8fd8107660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47524/0 (socket says 172.21.15.1:47524) 2023-11-16T16:50:10.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.190+0000 7f8fd659c700 1 -- 172.21.15.1:0/2523104243 learned_addr learned my addr 172.21.15.1:0/2523104243 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:50:10.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.191+0000 7f8fd659c700 1 -- 172.21.15.1:0/2523104243 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fd81113f0 msgr2=0x7f8fd81b0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:10.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.191+0000 7f8fd659c700 1 --2- 172.21.15.1:0/2523104243 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fd81113f0 0x7f8fd81b0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.191+0000 7f8fd659c700 1 -- 172.21.15.1:0/2523104243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fd80ff870 msgr2=0x7f8fd8107ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:10.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.191+0000 7f8fd659c700 1 --2- 172.21.15.1:0/2523104243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fd80ff870 0x7f8fd8107ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.196 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.191+0000 7f8fd659c700 1 -- 172.21.15.1:0/2523104243 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd0009c40 con 0x7f8fd80ff4a0 2023-11-16T16:50:10.197 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.191+0000 7f8fd659c700 1 --2- 172.21.15.1:0/2523104243 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fd80ff4a0 0x7f8fd8107660 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f8fc800b4d0 tx=0x7f8fc800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:10.197 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.192+0000 7f8fc77fe700 1 -- 172.21.15.1:0/2523104243 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc8011830 con 0x7f8fd80ff4a0 2023-11-16T16:50:10.197 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.192+0000 7f8fdcf96700 1 -- 172.21.15.1:0/2523104243 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fd8116960 con 0x7f8fd80ff4a0 2023-11-16T16:50:10.197 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.192+0000 7f8fdcf96700 1 -- 172.21.15.1:0/2523104243 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fd81167d0 con 0x7f8fd80ff4a0 2023-11-16T16:50:10.197 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.192+0000 7f8fc77fe700 1 -- 172.21.15.1:0/2523104243 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fc8009340 con 0x7f8fd80ff4a0 2023-11-16T16:50:10.197 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.192+0000 7f8fc77fe700 1 -- 172.21.15.1:0/2523104243 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc80193b0 con 0x7f8fd80ff4a0 2023-11-16T16:50:10.198 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.193+0000 7f8fdcf96700 1 -- 172.21.15.1:0/2523104243 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fb80052e0 con 0x7f8fd80ff4a0 2023-11-16T16:50:10.202 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.197+0000 7f8fc77fe700 1 -- 172.21.15.1:0/2523104243 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f8fc801a400 con 0x7f8fd80ff4a0 2023-11-16T16:50:10.202 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.198+0000 7f8fc77fe700 1 --2- 172.21.15.1:0/2523104243 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f8fc00a1400 0x7f8fc00a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:10.202 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.199+0000 7f8fc77fe700 1 -- 172.21.15.1:0/2523104243 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8586+0+0 (secure 0 0 0) 0x7f8fc80cc5f0 con 0x7f8fd80ff4a0 2023-11-16T16:50:10.202 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.199+0000 7f8fd5d9b700 1 --2- 172.21.15.1:0/2523104243 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f8fc00a1400 0x7f8fc00a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:10.203 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.201+0000 7f8fd5d9b700 1 --2- 172.21.15.1:0/2523104243 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f8fc00a1400 0x7f8fc00a38b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8108c70 tx=0x7f8fcc009310 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:10.213 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.211+0000 7f8fc77fe700 1 -- 172.21.15.1:0/2523104243 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8fc8095a70 con 0x7f8fd80ff4a0 2023-11-16T16:50:10.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:09 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/2680329556' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:50:10.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:09 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1609611500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:50:10.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:09 smithi026 ceph-mon[100112]: pgmap v341: 97 pgs: 97 active+clean; 817 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-11-16T16:50:10.330 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:09 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/2680329556' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-16T16:50:10.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:09 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1609611500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-16T16:50:10.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:09 smithi053 ceph-mon[100638]: pgmap v341: 97 pgs: 97 active+clean; 817 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2023-11-16T16:50:10.638 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/1210074029 2023-11-17T16:39:09.360183+0000 2023-11-16T16:50:10.638 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/2308375878 2023-11-17T16:39:09.360183+0000 2023-11-16T16:50:10.638 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/3270932790 2023-11-17T16:39:09.360183+0000 2023-11-16T16:50:10.638 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6801/2490729488 2023-11-17T16:39:09.360183+0000 2023-11-16T16:50:10.638 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6800/2490729488 2023-11-17T16:39:09.360183+0000 2023-11-16T16:50:10.638 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/1494336948 2023-11-17T16:31:01.391197+0000 2023-11-16T16:50:10.639 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6801/3137425292 2023-11-17T16:31:01.391197+0000 2023-11-16T16:50:10.639 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6800/3137425292 2023-11-17T16:31:01.391197+0000 2023-11-16T16:50:10.639 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/1768923877 2023-11-17T16:39:09.360183+0000 2023-11-16T16:50:10.639 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6800/1726282374 2023-11-17T16:30:35.064289+0000 2023-11-16T16:50:10.639 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/1641976669 2023-11-17T16:30:35.064289+0000 2023-11-16T16:50:10.639 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/2895923638 2023-11-17T16:30:35.064289+0000 2023-11-16T16:50:10.639 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6801/1726282374 2023-11-17T16:30:35.064289+0000 2023-11-16T16:50:10.639 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/78811711 2023-11-17T16:31:01.391197+0000 2023-11-16T16:50:10.640 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/4131957708 2023-11-17T16:31:01.391197+0000 2023-11-16T16:50:10.640 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/344403698 2023-11-17T16:30:35.064289+0000 2023-11-16T16:50:10.640 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.635+0000 7f8fdcf96700 1 -- 172.21.15.1:0/2523104243 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8fb8005150 con 0x7f8fd80ff4a0 2023-11-16T16:50:10.640 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.635+0000 7f8fc77fe700 1 -- 172.21.15.1:0/2523104243 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v96) v1 ==== 81+0+927 (secure 0 0 0) 0x7f8fc8099090 con 0x7f8fd80ff4a0 2023-11-16T16:50:10.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.639+0000 7f8fdcf96700 1 -- 172.21.15.1:0/2523104243 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f8fc00a1400 msgr2=0x7f8fc00a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:10.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.639+0000 7f8fdcf96700 1 --2- 172.21.15.1:0/2523104243 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f8fc00a1400 0x7f8fc00a38b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8108c70 tx=0x7f8fcc009310 comp rx=0 tx=0).stop 2023-11-16T16:50:10.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.639+0000 7f8fdcf96700 1 -- 172.21.15.1:0/2523104243 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fd80ff4a0 msgr2=0x7f8fd8107660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:10.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.639+0000 7f8fdcf96700 1 --2- 172.21.15.1:0/2523104243 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fd80ff4a0 0x7f8fd8107660 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f8fc800b4d0 tx=0x7f8fc800b890 comp rx=0 tx=0).stop 2023-11-16T16:50:10.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.639+0000 7f8fdcf96700 1 -- 172.21.15.1:0/2523104243 shutdown_connections 2023-11-16T16:50:10.641 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.639+0000 7f8fdcf96700 1 --2- 172.21.15.1:0/2523104243 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f8fc00a1400 0x7f8fc00a38b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.639+0000 7f8fdcf96700 1 --2- 172.21.15.1:0/2523104243 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f8fd81113f0 0x7f8fd81b0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.639+0000 7f8fdcf96700 1 --2- 172.21.15.1:0/2523104243 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fd80ff4a0 0x7f8fd8107660 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.639+0000 7f8fdcf96700 1 --2- 172.21.15.1:0/2523104243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fd80ff870 0x7f8fd8107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:10.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.639+0000 7f8fdcf96700 1 -- 172.21.15.1:0/2523104243 >> 172.21.15.1:0/2523104243 conn(0x7f8fd8074a50 msgr2=0x7f8fd81117c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:10.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.639+0000 7f8fdcf96700 1 -- 172.21.15.1:0/2523104243 shutdown_connections 2023-11-16T16:50:10.642 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:10.639+0000 7f8fdcf96700 1 -- 172.21.15.1:0/2523104243 wait complete. 2023-11-16T16:50:10.649 INFO:teuthology.orchestra.run.smithi001.stderr:listed 16 entries 2023-11-16T16:50:10.668 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-16T16:50:10.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:50:10.668 DEBUG:teuthology.orchestra.run.smithi001:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-16T16:50:10.767 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2023-11-16T16:50:10.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:50:10.767 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-16T16:50:10.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:50:10.875 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2023-11-16T16:50:10.931 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:10 smithi001 ceph-mon[98800]: osdmap e96: 12 total, 12 up, 12 in 2023-11-16T16:50:10.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:10 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/2934129294' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:50:10.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:10 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/2523104243' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-16T16:50:10.953 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-16T16:50:10.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:50:10.954 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-16T16:50:10.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-16T16:50:10.996 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link delete ceph-brx 2023-11-16T16:50:11.112 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-11-16T16:50:11.122 INFO:tasks.cephadm:Teardown begin 2023-11-16T16:50:11.122 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-16T16:50:11.123 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:50:11.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:10 smithi026 ceph-mon[100112]: osdmap e96: 12 total, 12 up, 12 in 2023-11-16T16:50:11.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:10 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/2934129294' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:50:11.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:10 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/2523104243' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-16T16:50:11.300 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.296+0000 7fae140bb700 1 -- 172.21.15.1:0/1201995581 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae0c0ffdd0 msgr2=0x7fae0c103d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:11.300 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.296+0000 7fae140bb700 1 --2- 172.21.15.1:0/1201995581 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae0c0ffdd0 0x7fae0c103d90 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fae080099c0 tx=0x7fae08009cd0 comp rx=0 tx=0).stop 2023-11-16T16:50:11.300 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.296+0000 7fae140bb700 1 -- 172.21.15.1:0/1201995581 shutdown_connections 2023-11-16T16:50:11.300 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.296+0000 7fae140bb700 1 --2- 172.21.15.1:0/1201995581 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae0c104470 0x7fae0c10c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:11.300 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.296+0000 7fae140bb700 1 --2- 172.21.15.1:0/1201995581 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae0c0ff430 0x7fae0c0ff800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:11.300 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.296+0000 7fae140bb700 1 --2- 172.21.15.1:0/1201995581 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae0c0ffdd0 0x7fae0c103d90 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:11.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.296+0000 7fae140bb700 1 -- 172.21.15.1:0/1201995581 >> 172.21.15.1:0/1201995581 conn(0x7fae0c077060 msgr2=0x7fae0c077460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:11.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.296+0000 7fae140bb700 1 -- 172.21.15.1:0/1201995581 shutdown_connections 2023-11-16T16:50:11.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.296+0000 7fae140bb700 1 -- 172.21.15.1:0/1201995581 wait complete. 2023-11-16T16:50:11.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.297+0000 7fae140bb700 1 Processor -- start 2023-11-16T16:50:11.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.297+0000 7fae140bb700 1 -- start start 2023-11-16T16:50:11.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.297+0000 7fae140bb700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae0c0ff430 0x7fae0c195320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:11.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.297+0000 7fae140bb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae0c0ffdd0 0x7fae0c195860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:11.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.297+0000 7fae140bb700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae0c104470 0x7fae0c199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:11.302 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.297+0000 7fae140bb700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c19a300 con 0x7fae0c0ff430 2023-11-16T16:50:11.302 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.297+0000 7fae140bb700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c19a470 con 0x7fae0c0ffdd0 2023-11-16T16:50:11.302 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.297+0000 7fae140bb700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c19a630 con 0x7fae0c104470 2023-11-16T16:50:11.302 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.297+0000 7fae11e57700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae0c0ff430 0x7fae0c195320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:11.302 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.297+0000 7fae11e57700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae0c0ff430 0x7fae0c195320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47538/0 (socket says 172.21.15.1:47538) 2023-11-16T16:50:11.302 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.297+0000 7fae11e57700 1 -- 172.21.15.1:0/3704149942 learned_addr learned my addr 172.21.15.1:0/3704149942 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:50:11.302 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.297+0000 7fae11656700 1 --2- 172.21.15.1:0/3704149942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae0c0ffdd0 0x7fae0c195860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:11.302 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.297+0000 7fae12658700 1 --2- 172.21.15.1:0/3704149942 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae0c104470 0x7fae0c199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:11.303 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.298+0000 7fae11e57700 1 -- 172.21.15.1:0/3704149942 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae0c104470 msgr2=0x7fae0c199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:11.303 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.298+0000 7fae11e57700 1 --2- 172.21.15.1:0/3704149942 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae0c104470 0x7fae0c199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:11.303 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.298+0000 7fae11e57700 1 -- 172.21.15.1:0/3704149942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae0c0ffdd0 msgr2=0x7fae0c195860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:11.303 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.298+0000 7fae11e57700 1 --2- 172.21.15.1:0/3704149942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae0c0ffdd0 0x7fae0c195860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:11.303 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.298+0000 7fae11e57700 1 -- 172.21.15.1:0/3704149942 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae080095f0 con 0x7fae0c0ff430 2023-11-16T16:50:11.303 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.298+0000 7fae11e57700 1 --2- 172.21.15.1:0/3704149942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae0c0ff430 0x7fae0c195320 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fae0000e9a0 tx=0x7fae0000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:11.303 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.298+0000 7fadfeffd700 1 -- 172.21.15.1:0/3704149942 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae00009790 con 0x7fae0c0ff430 2023-11-16T16:50:11.304 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.298+0000 7fadfeffd700 1 -- 172.21.15.1:0/3704149942 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae00010430 con 0x7fae0c0ff430 2023-11-16T16:50:11.304 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.298+0000 7fadfeffd700 1 -- 172.21.15.1:0/3704149942 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae00018680 con 0x7fae0c0ff430 2023-11-16T16:50:11.304 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.298+0000 7fae140bb700 1 -- 172.21.15.1:0/3704149942 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae0c10f1f0 con 0x7fae0c0ff430 2023-11-16T16:50:11.304 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.298+0000 7fae140bb700 1 -- 172.21.15.1:0/3704149942 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae0c10f060 con 0x7fae0c0ff430 2023-11-16T16:50:11.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.299+0000 7fae140bb700 1 -- 172.21.15.1:0/3704149942 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae0c10f380 con 0x7fae0c0ff430 2023-11-16T16:50:11.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.300+0000 7fadfeffd700 1 -- 172.21.15.1:0/3704149942 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fae000105a0 con 0x7fae0c0ff430 2023-11-16T16:50:11.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.301+0000 7fadfeffd700 1 --2- 172.21.15.1:0/3704149942 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fadf80a12c0 0x7fadf80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:11.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.301+0000 7fae11656700 1 --2- 172.21.15.1:0/3704149942 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fadf80a12c0 0x7fadf80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:11.305 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.301+0000 7fadfeffd700 1 -- 172.21.15.1:0/3704149942 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8586+0+0 (secure 0 0 0) 0x7fae00014070 con 0x7fae0c0ff430 2023-11-16T16:50:11.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.301+0000 7fae11656700 1 --2- 172.21.15.1:0/3704149942 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fadf80a12c0 0x7fadf80a3770 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fae080098e0 tx=0x7fae08000bc0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:11.306 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.304+0000 7fadfeffd700 1 -- 172.21.15.1:0/3704149942 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fae000964e0 con 0x7fae0c0ff430 2023-11-16T16:50:11.330 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:10 smithi053 ceph-mon[100638]: osdmap e96: 12 total, 12 up, 12 in 2023-11-16T16:50:11.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:10 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/2934129294' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-16T16:50:11.331 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:10 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/2523104243' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-16T16:50:11.552 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.550+0000 7fae140bb700 1 -- 172.21.15.1:0/3704149942 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae0c067af0 con 0x7fae0c0ff430 2023-11-16T16:50:11.552 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.550+0000 7fadfeffd700 1 -- 172.21.15.1:0/3704149942 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+10774 (secure 0 0 0) 0x7fae00026090 con 0x7fae0c0ff430 2023-11-16T16:50:11.552 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:50:11.553 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2587563684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2587563684},{"type":"v1","addr":"172.21.15.1:6835","nonce":2587563684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/277936168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":277936168},{"type":"v1","addr":"172.21.15.1:6837","nonce":277936168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/1172575045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1172575045},{"type":"v1","addr":"172.21.15.1:6839","nonce":1172575045}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/2631175899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2631175899},{"type":"v1","addr":"172.21.15.1:6841","nonce":2631175899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3693905267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3693905267},{"type":"v1","addr":"172.21.15.53:6833","nonce":3693905267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/164668033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":164668033},{"type":"v1","addr":"172.21.15.26:6833","nonce":164668033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-16T16:37:13.918943+0000","modified":"2023-11-16T16:49:15.365148+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2140542192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2140542192},{"type":"v1","addr":"172.21.15.53:6835","nonce":2140542192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1010760275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1010760275},{"type":"v1","addr":"172.21.15.26:6835","nonce":1010760275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3197060302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3197060302},{"type":"v1","addr":"172.21.15.53:6837","nonce":3197060302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6837/1453504863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1453504863},{"type":"v1","addr":"172.21.15.26:6837","nonce":1453504863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.53:6839/1044753024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1044753024},{"type":"v1","addr":"172.21.15.53:6839","nonce":1044753024}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6839/2192205949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2192205949},{"type":"v1","addr":"172.21.15.26:6839","nonce":2192205949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-16T16:50:11.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.553+0000 7fae140bb700 1 -- 172.21.15.1:0/3704149942 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fadf80a12c0 msgr2=0x7fadf80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:11.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.553+0000 7fae140bb700 1 --2- 172.21.15.1:0/3704149942 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fadf80a12c0 0x7fadf80a3770 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fae080098e0 tx=0x7fae08000bc0 comp rx=0 tx=0).stop 2023-11-16T16:50:11.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.553+0000 7fae140bb700 1 -- 172.21.15.1:0/3704149942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae0c0ff430 msgr2=0x7fae0c195320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:11.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.553+0000 7fae140bb700 1 --2- 172.21.15.1:0/3704149942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae0c0ff430 0x7fae0c195320 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fae0000e9a0 tx=0x7fae0000ecb0 comp rx=0 tx=0).stop 2023-11-16T16:50:11.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.553+0000 7fae140bb700 1 -- 172.21.15.1:0/3704149942 shutdown_connections 2023-11-16T16:50:11.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.553+0000 7fae140bb700 1 --2- 172.21.15.1:0/3704149942 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fadf80a12c0 0x7fadf80a3770 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:11.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.553+0000 7fae140bb700 1 --2- 172.21.15.1:0/3704149942 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fae0c104470 0x7fae0c199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:11.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.553+0000 7fae140bb700 1 --2- 172.21.15.1:0/3704149942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae0c0ff430 0x7fae0c195320 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:11.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.553+0000 7fae140bb700 1 --2- 172.21.15.1:0/3704149942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae0c0ffdd0 0x7fae0c195860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:11.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.553+0000 7fae140bb700 1 -- 172.21.15.1:0/3704149942 >> 172.21.15.1:0/3704149942 conn(0x7fae0c077060 msgr2=0x7fae0c0fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:11.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.553+0000 7fae140bb700 1 -- 172.21.15.1:0/3704149942 shutdown_connections 2023-11-16T16:50:11.556 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.553+0000 7fae140bb700 1 -- 172.21.15.1:0/3704149942 wait complete. 2023-11-16T16:50:11.557 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 33 2023-11-16T16:50:11.567 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-16T16:50:11.694 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.692+0000 7f259b59e700 1 -- 172.21.15.1:0/4216124504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f259c100ce0 msgr2=0x7f259c104ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:11.694 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.692+0000 7f259b59e700 1 --2- 172.21.15.1:0/4216124504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f259c100ce0 0x7f259c104ca0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f25840099c0 tx=0x7f2584009cd0 comp rx=0 tx=0).stop 2023-11-16T16:50:11.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.693+0000 7f259b59e700 1 -- 172.21.15.1:0/4216124504 shutdown_connections 2023-11-16T16:50:11.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.693+0000 7f259b59e700 1 --2- 172.21.15.1:0/4216124504 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f259c105380 0x7f259c10cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:11.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.693+0000 7f259b59e700 1 --2- 172.21.15.1:0/4216124504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f259c100340 0x7f259c100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:11.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.693+0000 7f259b59e700 1 --2- 172.21.15.1:0/4216124504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f259c100ce0 0x7f259c104ca0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:11.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.693+0000 7f259b59e700 1 -- 172.21.15.1:0/4216124504 >> 172.21.15.1:0/4216124504 conn(0x7f259c0fbbd0 msgr2=0x7f259c0fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:11.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.693+0000 7f259b59e700 1 -- 172.21.15.1:0/4216124504 shutdown_connections 2023-11-16T16:50:11.695 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.693+0000 7f259b59e700 1 -- 172.21.15.1:0/4216124504 wait complete. 2023-11-16T16:50:11.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.693+0000 7f259b59e700 1 Processor -- start 2023-11-16T16:50:11.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.693+0000 7f259b59e700 1 -- start start 2023-11-16T16:50:11.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.694+0000 7f259b59e700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f259c100340 0x7f259c196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:11.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.694+0000 7f259b59e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f259c100ce0 0x7f259c1966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:11.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.694+0000 7f259b59e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f259c105380 0x7f259c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:11.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.694+0000 7f259b59e700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259c19b170 con 0x7f259c105380 2023-11-16T16:50:11.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.694+0000 7f259b59e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259c19b2e0 con 0x7f259c100ce0 2023-11-16T16:50:11.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.694+0000 7f259b59e700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259c19b4a0 con 0x7f259c100340 2023-11-16T16:50:11.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.694+0000 7f259ad9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f259c105380 0x7f259c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:11.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.694+0000 7f259ad9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f259c105380 0x7f259c19ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47556/0 (socket says 172.21.15.1:47556) 2023-11-16T16:50:11.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.694+0000 7f259ad9d700 1 -- 172.21.15.1:0/2035033057 learned_addr learned my addr 172.21.15.1:0/2035033057 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:50:11.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.694+0000 7f2599d9b700 1 --2- 172.21.15.1:0/2035033057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f259c100ce0 0x7f259c1966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:11.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.694+0000 7f259a59c700 1 --2- 172.21.15.1:0/2035033057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f259c100340 0x7f259c196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:11.699 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.694+0000 7f259ad9d700 1 -- 172.21.15.1:0/2035033057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f259c100340 msgr2=0x7f259c196190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:11.700 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.694+0000 7f259ad9d700 1 --2- 172.21.15.1:0/2035033057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f259c100340 0x7f259c196190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:11.700 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.694+0000 7f259ad9d700 1 -- 172.21.15.1:0/2035033057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f259c100ce0 msgr2=0x7f259c1966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:11.700 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.694+0000 7f259ad9d700 1 --2- 172.21.15.1:0/2035033057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f259c100ce0 0x7f259c1966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:11.700 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.694+0000 7f259ad9d700 1 -- 172.21.15.1:0/2035033057 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25840095f0 con 0x7f259c105380 2023-11-16T16:50:11.700 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.694+0000 7f259a59c700 1 --2- 172.21.15.1:0/2035033057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f259c100340 0x7f259c196190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:50:11.700 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.694+0000 7f259ad9d700 1 --2- 172.21.15.1:0/2035033057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f259c105380 0x7f259c19ab30 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f2594007ee0 tx=0x7f259400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:11.700 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.694+0000 7f25937fe700 1 -- 172.21.15.1:0/2035033057 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2594017070 con 0x7f259c105380 2023-11-16T16:50:11.700 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.694+0000 7f25937fe700 1 -- 172.21.15.1:0/2035033057 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f259400f040 con 0x7f259c105380 2023-11-16T16:50:11.701 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.694+0000 7f259b59e700 1 -- 172.21.15.1:0/2035033057 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f259c110100 con 0x7f259c105380 2023-11-16T16:50:11.701 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.695+0000 7f25937fe700 1 -- 172.21.15.1:0/2035033057 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2594005740 con 0x7f259c105380 2023-11-16T16:50:11.701 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.695+0000 7f259b59e700 1 -- 172.21.15.1:0/2035033057 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f259c10ff70 con 0x7f259c105380 2023-11-16T16:50:11.701 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.695+0000 7f259b59e700 1 -- 172.21.15.1:0/2035033057 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f259c110290 con 0x7f259c105380 2023-11-16T16:50:11.702 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.697+0000 7f25937fe700 1 -- 172.21.15.1:0/2035033057 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f259400dc60 con 0x7f259c105380 2023-11-16T16:50:11.702 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.698+0000 7f25937fe700 1 --2- 172.21.15.1:0/2035033057 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f25880a12c0 0x7f25880a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:11.702 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.698+0000 7f25937fe700 1 -- 172.21.15.1:0/2035033057 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8586+0+0 (secure 0 0 0) 0x7f25940ccb30 con 0x7f259c105380 2023-11-16T16:50:11.702 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.698+0000 7f259a59c700 1 --2- 172.21.15.1:0/2035033057 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f25880a12c0 0x7f25880a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:11.702 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.698+0000 7f259a59c700 1 --2- 172.21.15.1:0/2035033057 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f25880a12c0 0x7f25880a3770 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f258c005950 tx=0x7f258c0058e0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:11.703 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.701+0000 7f25937fe700 1 -- 172.21.15.1:0/2035033057 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2594095ea0 con 0x7f259c105380 2023-11-16T16:50:11.946 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.944+0000 7f259b59e700 1 -- 172.21.15.1:0/2035033057 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f259c0670e0 con 0x7f259c105380 2023-11-16T16:50:11.946 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.945+0000 7f25937fe700 1 -- 172.21.15.1:0/2035033057 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+10774 (secure 0 0 0) 0x7f25940994c0 con 0x7f259c105380 2023-11-16T16:50:11.947 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:50:11.947 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2587563684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2587563684},{"type":"v1","addr":"172.21.15.1:6835","nonce":2587563684}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/277936168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":277936168},{"type":"v1","addr":"172.21.15.1:6837","nonce":277936168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14613,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/1172575045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1172575045},{"type":"v1","addr":"172.21.15.1:6839","nonce":1172575045}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/2631175899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2631175899},{"type":"v1","addr":"172.21.15.1:6841","nonce":2631175899}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.53:6833/3693905267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6832","nonce":3693905267},{"type":"v1","addr":"172.21.15.53:6833","nonce":3693905267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/164668033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":164668033},{"type":"v1","addr":"172.21.15.26:6833","nonce":164668033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-16T16:37:13.918943+0000","modified":"2023-11-16T16:49:15.365148+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6835/2140542192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6834","nonce":2140542192},{"type":"v1","addr":"172.21.15.53:6835","nonce":2140542192}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6835/1010760275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1010760275},{"type":"v1","addr":"172.21.15.26:6835","nonce":1010760275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.53:6837/3197060302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6836","nonce":3197060302},{"type":"v1","addr":"172.21.15.53:6837","nonce":3197060302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6837/1453504863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":1453504863},{"type":"v1","addr":"172.21.15.26:6837","nonce":1453504863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.53:6839/1044753024","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6838","nonce":1044753024},{"type":"v1","addr":"172.21.15.53:6839","nonce":1044753024}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6839/2192205949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":2192205949},{"type":"v1","addr":"172.21.15.26:6839","nonce":2192205949}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-16T16:50:11.949 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.947+0000 7f259b59e700 1 -- 172.21.15.1:0/2035033057 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f25880a12c0 msgr2=0x7f25880a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:11.949 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.947+0000 7f259b59e700 1 --2- 172.21.15.1:0/2035033057 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f25880a12c0 0x7f25880a3770 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f258c005950 tx=0x7f258c0058e0 comp rx=0 tx=0).stop 2023-11-16T16:50:11.949 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.947+0000 7f259b59e700 1 -- 172.21.15.1:0/2035033057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f259c105380 msgr2=0x7f259c19ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:11.949 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.947+0000 7f259b59e700 1 --2- 172.21.15.1:0/2035033057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f259c105380 0x7f259c19ab30 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f2594007ee0 tx=0x7f259400d3b0 comp rx=0 tx=0).stop 2023-11-16T16:50:11.950 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.947+0000 7f259b59e700 1 -- 172.21.15.1:0/2035033057 shutdown_connections 2023-11-16T16:50:11.950 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.947+0000 7f259b59e700 1 --2- 172.21.15.1:0/2035033057 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f25880a12c0 0x7f25880a3770 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:11.950 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.947+0000 7f259b59e700 1 --2- 172.21.15.1:0/2035033057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f259c105380 0x7f259c19ab30 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:11.950 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.947+0000 7f259b59e700 1 --2- 172.21.15.1:0/2035033057 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f259c100340 0x7f259c196190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:11.950 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.947+0000 7f259b59e700 1 --2- 172.21.15.1:0/2035033057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f259c100ce0 0x7f259c1966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:11.950 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.947+0000 7f259b59e700 1 -- 172.21.15.1:0/2035033057 >> 172.21.15.1:0/2035033057 conn(0x7f259c0fbbd0 msgr2=0x7f259c0fc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:11.950 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.947+0000 7f259b59e700 1 -- 172.21.15.1:0/2035033057 shutdown_connections 2023-11-16T16:50:11.951 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:11.947+0000 7f259b59e700 1 -- 172.21.15.1:0/2035033057 wait complete. 2023-11-16T16:50:11.951 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 33 2023-11-16T16:50:11.961 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-16T16:50:12.088 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.086+0000 7feb1b4fb700 1 -- 172.21.15.1:0/3839513205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb14100610 msgr2=0x7feb1410c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:12.088 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.086+0000 7feb1b4fb700 1 --2- 172.21.15.1:0/3839513205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb14100610 0x7feb1410c2a0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7feb0c00b0a0 tx=0x7feb0c00b3b0 comp rx=0 tx=0).stop 2023-11-16T16:50:12.088 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.086+0000 7feb1b4fb700 1 -- 172.21.15.1:0/3839513205 shutdown_connections 2023-11-16T16:50:12.088 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.086+0000 7feb1b4fb700 1 --2- 172.21.15.1:0/3839513205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb14100610 0x7feb1410c2a0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:12.088 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.086+0000 7feb1b4fb700 1 --2- 172.21.15.1:0/3839513205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb14105470 0x7feb14105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:12.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.086+0000 7feb1b4fb700 1 --2- 172.21.15.1:0/3839513205 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb140ffc60 0x7feb141000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:12.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.086+0000 7feb1b4fb700 1 -- 172.21.15.1:0/3839513205 >> 172.21.15.1:0/3839513205 conn(0x7feb14074950 msgr2=0x7feb14074d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:12.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.086+0000 7feb1b4fb700 1 -- 172.21.15.1:0/3839513205 shutdown_connections 2023-11-16T16:50:12.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.086+0000 7feb1b4fb700 1 -- 172.21.15.1:0/3839513205 wait complete. 2023-11-16T16:50:12.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.086+0000 7feb1b4fb700 1 Processor -- start 2023-11-16T16:50:12.089 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.087+0000 7feb1b4fb700 1 -- start start 2023-11-16T16:50:12.091 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.087+0000 7feb1b4fb700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb140ffc60 0x7feb141955d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:12.092 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.087+0000 7feb1b4fb700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb14100610 0x7feb14195b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:12.092 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.087+0000 7feb1b4fb700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb14105470 0x7feb14199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:12.092 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.087+0000 7feb1b4fb700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb1419a5b0 con 0x7feb14105470 2023-11-16T16:50:12.092 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.087+0000 7feb1b4fb700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb1419a720 con 0x7feb14100610 2023-11-16T16:50:12.092 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.087+0000 7feb1b4fb700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb1419a8e0 con 0x7feb140ffc60 2023-11-16T16:50:12.092 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.087+0000 7feb19a98700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb14105470 0x7feb14199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:12.092 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.087+0000 7feb19297700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb140ffc60 0x7feb141955d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:12.093 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.087+0000 7feb19297700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb140ffc60 0x7feb141955d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.53:3300/0 says I am v2:172.21.15.1:44248/0 (socket says 172.21.15.1:44248) 2023-11-16T16:50:12.093 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.087+0000 7feb19a98700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb14105470 0x7feb14199f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47570/0 (socket says 172.21.15.1:47570) 2023-11-16T16:50:12.093 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.087+0000 7feb19297700 1 -- 172.21.15.1:0/2296290725 learned_addr learned my addr 172.21.15.1:0/2296290725 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:50:12.093 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.087+0000 7feb18a96700 1 --2- 172.21.15.1:0/2296290725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb14100610 0x7feb14195b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:12.093 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.087+0000 7feb19a98700 1 -- 172.21.15.1:0/2296290725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb140ffc60 msgr2=0x7feb141955d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:12.093 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.087+0000 7feb19a98700 1 --2- 172.21.15.1:0/2296290725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb140ffc60 0x7feb141955d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:12.093 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.087+0000 7feb19a98700 1 -- 172.21.15.1:0/2296290725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb14100610 msgr2=0x7feb14195b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:12.093 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.087+0000 7feb19a98700 1 --2- 172.21.15.1:0/2296290725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb14100610 0x7feb14195b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:12.094 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.087+0000 7feb19a98700 1 -- 172.21.15.1:0/2296290725 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb0c009d00 con 0x7feb14105470 2023-11-16T16:50:12.094 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.087+0000 7feb18a96700 1 --2- 172.21.15.1:0/2296290725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb14100610 0x7feb14195b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:50:12.094 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.087+0000 7feb19a98700 1 --2- 172.21.15.1:0/2296290725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb14105470 0x7feb14199f70 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7feb0c009fd0 tx=0x7feb0c007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:12.094 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.088+0000 7feb0a7fc700 1 -- 172.21.15.1:0/2296290725 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0c00e040 con 0x7feb14105470 2023-11-16T16:50:12.094 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.088+0000 7feb0a7fc700 1 -- 172.21.15.1:0/2296290725 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb0c004c50 con 0x7feb14105470 2023-11-16T16:50:12.094 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.088+0000 7feb1b4fb700 1 -- 172.21.15.1:0/2296290725 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb1410f4b0 con 0x7feb14105470 2023-11-16T16:50:12.094 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.088+0000 7feb0a7fc700 1 -- 172.21.15.1:0/2296290725 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0c005040 con 0x7feb14105470 2023-11-16T16:50:12.095 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.088+0000 7feb1b4fb700 1 -- 172.21.15.1:0/2296290725 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb1410f320 con 0x7feb14105470 2023-11-16T16:50:12.095 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.089+0000 7feb1b4fb700 1 -- 172.21.15.1:0/2296290725 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb1410f640 con 0x7feb14105470 2023-11-16T16:50:12.095 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.090+0000 7feb0a7fc700 1 -- 172.21.15.1:0/2296290725 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7feb0c01d470 con 0x7feb14105470 2023-11-16T16:50:12.095 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.091+0000 7feb0a7fc700 1 --2- 172.21.15.1:0/2296290725 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7feb000a1340 0x7feb000a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:12.096 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.091+0000 7feb19297700 1 --2- 172.21.15.1:0/2296290725 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7feb000a1340 0x7feb000a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:12.096 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.091+0000 7feb0a7fc700 1 -- 172.21.15.1:0/2296290725 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8586+0+0 (secure 0 0 0) 0x7feb0c0ce880 con 0x7feb14105470 2023-11-16T16:50:12.096 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.091+0000 7feb19297700 1 --2- 172.21.15.1:0/2296290725 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7feb000a1340 0x7feb000a37f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7feb04009ca0 tx=0x7feb04009260 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:12.096 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.094+0000 7feb0a7fc700 1 -- 172.21.15.1:0/2296290725 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7feb0c097ce0 con 0x7feb14105470 2023-11-16T16:50:12.297 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.295+0000 7feb1b4fb700 1 -- 172.21.15.1:0/2296290725 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb14196920 con 0x7feb14105470 2023-11-16T16:50:12.299 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.297+0000 7feb0a7fc700 1 -- 172.21.15.1:0/2296290725 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+22998 (secure 0 0 0) 0x7feb0c09b300 con 0x7feb14105470 2023-11-16T16:50:12.299 INFO:teuthology.orchestra.run.smithi001.stdout: 2023-11-16T16:50:12.299 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":96,"fsid":"35d9b010-849d-11ee-8dbe-212e2dc638e7","created":"2023-11-16T16:30:06.217778+0000","modified":"2023-11-16T16:50:09.894437+0000","last_up_change":"2023-11-16T16:36:31.735403+0000","last_in_change":"2023-11-16T16:36:14.083021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-16T16:33:15.916723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-16T16:37:12.158031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-16T16:37:13.272343+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":96,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a8480507-5275-4e52-953b-626110beb436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6803","nonce":2469671609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6805","nonce":2469671609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6809","nonce":2469671609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2469671609},{"type":"v1","addr":"172.21.15.1:6807","nonce":2469671609}]},"public_addr":"172.21.15.1:6803/2469671609","cluster_addr":"172.21.15.1:6805/2469671609","heartbeat_back_addr":"172.21.15.1:6809/2469671609","heartbeat_front_addr":"172.21.15.1:6807/2469671609","state":["exists","up"]},{"osd":1,"uuid":"1d011db2-99d1-4575-bdc5-51f88a308fdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6801","nonce":247994796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6803","nonce":247994796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6807","nonce":247994796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":247994796},{"type":"v1","addr":"172.21.15.26:6805","nonce":247994796}]},"public_addr":"172.21.15.26:6801/247994796","cluster_addr":"172.21.15.26:6803/247994796","heartbeat_back_addr":"172.21.15.26:6807/247994796","heartbeat_front_addr":"172.21.15.26:6805/247994796","state":["exists","up"]},{"osd":2,"uuid":"4d7a2bd1-3c8e-4546-a243-f0354473b031","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6801","nonce":1659316290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6803","nonce":1659316290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6807","nonce":1659316290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":1659316290},{"type":"v1","addr":"172.21.15.53:6805","nonce":1659316290}]},"public_addr":"172.21.15.53:6801/1659316290","cluster_addr":"172.21.15.53:6803/1659316290","heartbeat_back_addr":"172.21.15.53:6807/1659316290","heartbeat_front_addr":"172.21.15.53:6805/1659316290","state":["exists","up"]},{"osd":3,"uuid":"9c290fd9-e121-4a81-9687-b0eb93debd86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6811","nonce":2728987281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6813","nonce":2728987281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6817","nonce":2728987281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":2728987281},{"type":"v1","addr":"172.21.15.1:6815","nonce":2728987281}]},"public_addr":"172.21.15.1:6811/2728987281","cluster_addr":"172.21.15.1:6813/2728987281","heartbeat_back_addr":"172.21.15.1:6817/2728987281","heartbeat_front_addr":"172.21.15.1:6815/2728987281","state":["exists","up"]},{"osd":4,"uuid":"533226bc-cc57-4f15-9e29-820294d57e25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6809","nonce":2954740024}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6811","nonce":2954740024}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6815","nonce":2954740024}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":2954740024},{"type":"v1","addr":"172.21.15.26:6813","nonce":2954740024}]},"public_addr":"172.21.15.26:6809/2954740024","cluster_addr":"172.21.15.26:6811/2954740024","heartbeat_back_addr":"172.21.15.26:6815/2954740024","heartbeat_front_addr":"172.21.15.26:6813/2954740024","state":["exists","up"]},{"osd":5,"uuid":"d3ab25eb-e1d4-4557-bc89-efc9072401ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6809","nonce":3966666318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6811","nonce":3966666318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6815","nonce":3966666318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":3966666318},{"type":"v1","addr":"172.21.15.53:6813","nonce":3966666318}]},"public_addr":"172.21.15.53:6809/3966666318","cluster_addr":"172.21.15.53:6811/3966666318","heartbeat_back_addr":"172.21.15.53:6815/3966666318","heartbeat_front_addr":"172.21.15.53:6813/3966666318","state":["exists","up"]},{"osd":6,"uuid":"180f506e-afb8-4043-b0b4-2dc86f6d4bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6819","nonce":3364032646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6821","nonce":3364032646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6825","nonce":3364032646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":3364032646},{"type":"v1","addr":"172.21.15.1:6823","nonce":3364032646}]},"public_addr":"172.21.15.1:6819/3364032646","cluster_addr":"172.21.15.1:6821/3364032646","heartbeat_back_addr":"172.21.15.1:6825/3364032646","heartbeat_front_addr":"172.21.15.1:6823/3364032646","state":["exists","up"]},{"osd":7,"uuid":"0d837bac-194f-498d-b7fb-b3147e992329","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6817","nonce":3766901391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6819","nonce":3766901391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6823","nonce":3766901391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":3766901391},{"type":"v1","addr":"172.21.15.26:6821","nonce":3766901391}]},"public_addr":"172.21.15.26:6817/3766901391","cluster_addr":"172.21.15.26:6819/3766901391","heartbeat_back_addr":"172.21.15.26:6823/3766901391","heartbeat_front_addr":"172.21.15.26:6821/3766901391","state":["exists","up"]},{"osd":8,"uuid":"933f231a-7454-47b0-9291-f1fa64f0baec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6817","nonce":3224335871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6819","nonce":3224335871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6823","nonce":3224335871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":3224335871},{"type":"v1","addr":"172.21.15.53:6821","nonce":3224335871}]},"public_addr":"172.21.15.53:6817/3224335871","cluster_addr":"172.21.15.53:6819/3224335871","heartbeat_back_addr":"172.21.15.53:6823/3224335871","heartbeat_front_addr":"172.21.15.53:6821/3224335871","state":["exists","up"]},{"osd":9,"uuid":"245fcbb3-c651-4703-ab80-309dc98bd983","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6827","nonce":1391199844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6829","nonce":1391199844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6833","nonce":1391199844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":1391199844},{"type":"v1","addr":"172.21.15.1:6831","nonce":1391199844}]},"public_addr":"172.21.15.1:6827/1391199844","cluster_addr":"172.21.15.1:6829/1391199844","heartbeat_back_addr":"172.21.15.1:6833/1391199844","heartbeat_front_addr":"172.21.15.1:6831/1391199844","state":["exists","up"]},{"osd":10,"uuid":"2c090e7a-5240-4d52-ae4f-e94fe05383d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6825","nonce":1773545447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6827","nonce":1773545447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6831","nonce":1773545447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":1773545447},{"type":"v1","addr":"172.21.15.26:6829","nonce":1773545447}]},"public_addr":"172.21.15.26:6825/1773545447","cluster_addr":"172.21.15.26:6827/1773545447","heartbeat_back_addr":"172.21.15.26:6831/1773545447","heartbeat_front_addr":"172.21.15.26:6829/1773545447","state":["exists","up"]},{"osd":11,"uuid":"4511e32d-e469-45e1-8310-0ff400cb7b54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6825","nonce":2564669546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6827","nonce":2564669546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6831","nonce":2564669546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":2564669546},{"type":"v1","addr":"172.21.15.53:6829","nonce":2564669546}]},"public_addr":"172.21.15.53:6825/2564669546","cluster_addr":"172.21.15.53:6827/2564669546","heartbeat_back_addr":"172.21.15.53:6831/2564669546","heartbeat_front_addr":"172.21.15.53:6829/2564669546","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:28.140250+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:32:49.885461+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:12.521487+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:34.926197+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:33:56.815594+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:18.387751+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:34:40.587106+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:02.319370+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:23.832298+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:35:46.338040+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:08.730414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-16T16:36:30.050070+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/1210074029":"2023-11-17T16:39:09.360183+0000","172.21.15.1:0/2308375878":"2023-11-17T16:39:09.360183+0000","172.21.15.1:0/3270932790":"2023-11-17T16:39:09.360183+0000","172.21.15.1:6801/2490729488":"2023-11-17T16:39:09.360183+0000","172.21.15.1:6800/2490729488":"2023-11-17T16:39:09.360183+0000","172.21.15.1:0/1494336948":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6801/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:6800/3137425292":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/1768923877":"2023-11-17T16:39:09.360183+0000","172.21.15.1:6800/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/1641976669":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/2895923638":"2023-11-17T16:30:35.064289+0000","172.21.15.1:6801/1726282374":"2023-11-17T16:30:35.064289+0000","172.21.15.1:0/78811711":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/4131957708":"2023-11-17T16:31:01.391197+0000","172.21.15.1:0/344403698":"2023-11-17T16:30:35.064289+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-16T16:50:12.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.299+0000 7feb1b4fb700 1 -- 172.21.15.1:0/2296290725 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7feb000a1340 msgr2=0x7feb000a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:12.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.299+0000 7feb1b4fb700 1 --2- 172.21.15.1:0/2296290725 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7feb000a1340 0x7feb000a37f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7feb04009ca0 tx=0x7feb04009260 comp rx=0 tx=0).stop 2023-11-16T16:50:12.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.299+0000 7feb1b4fb700 1 -- 172.21.15.1:0/2296290725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb14105470 msgr2=0x7feb14199f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:12.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.299+0000 7feb1b4fb700 1 --2- 172.21.15.1:0/2296290725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb14105470 0x7feb14199f70 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7feb0c009fd0 tx=0x7feb0c007930 comp rx=0 tx=0).stop 2023-11-16T16:50:12.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.299+0000 7feb1b4fb700 1 -- 172.21.15.1:0/2296290725 shutdown_connections 2023-11-16T16:50:12.301 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.299+0000 7feb1b4fb700 1 --2- 172.21.15.1:0/2296290725 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7feb000a1340 0x7feb000a37f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:12.302 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.299+0000 7feb1b4fb700 1 --2- 172.21.15.1:0/2296290725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feb14105470 0x7feb14199f70 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:12.302 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.299+0000 7feb1b4fb700 1 --2- 172.21.15.1:0/2296290725 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7feb140ffc60 0x7feb141955d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:12.302 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.299+0000 7feb1b4fb700 1 --2- 172.21.15.1:0/2296290725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb14100610 0x7feb14195b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:12.302 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.299+0000 7feb1b4fb700 1 -- 172.21.15.1:0/2296290725 >> 172.21.15.1:0/2296290725 conn(0x7feb14074950 msgr2=0x7feb140fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:12.302 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.299+0000 7feb1b4fb700 1 -- 172.21.15.1:0/2296290725 shutdown_connections 2023-11-16T16:50:12.302 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.299+0000 7feb1b4fb700 1 -- 172.21.15.1:0/2296290725 wait complete. 2023-11-16T16:50:12.313 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-16T16:50:12.443 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.440+0000 7f7175fd3700 1 -- 172.21.15.1:0/2556440507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71700ffb60 msgr2=0x7f71700fffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:12.443 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.440+0000 7f7175fd3700 1 --2- 172.21.15.1:0/2556440507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71700ffb60 0x7f71700fffb0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f7164009990 tx=0x7f7164009ca0 comp rx=0 tx=0).stop 2023-11-16T16:50:12.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.440+0000 7f7175fd3700 1 -- 172.21.15.1:0/2556440507 shutdown_connections 2023-11-16T16:50:12.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.440+0000 7f7175fd3700 1 --2- 172.21.15.1:0/2556440507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71701004f0 0x7f717010c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:12.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.440+0000 7f7175fd3700 1 --2- 172.21.15.1:0/2556440507 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f7170105250 0x7f7170105620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:12.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.440+0000 7f7175fd3700 1 --2- 172.21.15.1:0/2556440507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71700ffb60 0x7f71700fffb0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:12.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.440+0000 7f7175fd3700 1 -- 172.21.15.1:0/2556440507 >> 172.21.15.1:0/2556440507 conn(0x7f71700748b0 msgr2=0x7f7170074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:12.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.440+0000 7f7175fd3700 1 -- 172.21.15.1:0/2556440507 shutdown_connections 2023-11-16T16:50:12.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.440+0000 7f7175fd3700 1 -- 172.21.15.1:0/2556440507 wait complete. 2023-11-16T16:50:12.444 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.440+0000 7f7175fd3700 1 Processor -- start 2023-11-16T16:50:12.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.440+0000 7f7175fd3700 1 -- start start 2023-11-16T16:50:12.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.441+0000 7f7175fd3700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71700ffb60 0x7f71701971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:12.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.441+0000 7f7175fd3700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71701004f0 0x7f71701976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:12.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.441+0000 7f7175fd3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7170105250 0x7f71701a2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:12.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.441+0000 7f7175fd3700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7170198000 con 0x7f71701004f0 2023-11-16T16:50:12.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.441+0000 7f7175fd3700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71701a2770 con 0x7f7170105250 2023-11-16T16:50:12.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.441+0000 7f7175fd3700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71701a28e0 con 0x7f71700ffb60 2023-11-16T16:50:12.445 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.441+0000 7f716effd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71701004f0 0x7f71701976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:12.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.441+0000 7f716effd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71701004f0 0x7f71701976f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47582/0 (socket says 172.21.15.1:47582) 2023-11-16T16:50:12.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.441+0000 7f716effd700 1 -- 172.21.15.1:0/1497554452 learned_addr learned my addr 172.21.15.1:0/1497554452 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:50:12.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.441+0000 7f716f7fe700 1 --2- 172.21.15.1:0/1497554452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71700ffb60 0x7f71701971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:12.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.441+0000 7f716ffff700 1 --2- 172.21.15.1:0/1497554452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7170105250 0x7f71701a2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:12.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.441+0000 7f716effd700 1 -- 172.21.15.1:0/1497554452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71700ffb60 msgr2=0x7f71701971b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:12.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.441+0000 7f716effd700 1 --2- 172.21.15.1:0/1497554452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71700ffb60 0x7f71701971b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:12.446 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.441+0000 7f716effd700 1 -- 172.21.15.1:0/1497554452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7170105250 msgr2=0x7f71701a2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:12.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.441+0000 7f716effd700 1 --2- 172.21.15.1:0/1497554452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7170105250 0x7f71701a2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:12.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.441+0000 7f716effd700 1 -- 172.21.15.1:0/1497554452 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71640095f0 con 0x7f71701004f0 2023-11-16T16:50:12.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.441+0000 7f716ffff700 1 --2- 172.21.15.1:0/1497554452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7170105250 0x7f71701a2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-16T16:50:12.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.441+0000 7f716effd700 1 --2- 172.21.15.1:0/1497554452 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71701004f0 0x7f71701976f0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f71640095c0 tx=0x7f716400f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:12.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.441+0000 7f716cff9700 1 -- 172.21.15.1:0/1497554452 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716401d070 con 0x7f71701004f0 2023-11-16T16:50:12.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.441+0000 7f716cff9700 1 -- 172.21.15.1:0/1497554452 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f716400fb50 con 0x7f71701004f0 2023-11-16T16:50:12.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.441+0000 7f716cff9700 1 -- 172.21.15.1:0/1497554452 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71640173f0 con 0x7f71701004f0 2023-11-16T16:50:12.447 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.441+0000 7f7175fd3700 1 -- 172.21.15.1:0/1497554452 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f717010f250 con 0x7f71701004f0 2023-11-16T16:50:12.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.442+0000 7f7175fd3700 1 -- 172.21.15.1:0/1497554452 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f717010f0c0 con 0x7f71701004f0 2023-11-16T16:50:12.448 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.442+0000 7f7175fd3700 1 -- 172.21.15.1:0/1497554452 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f717010f3e0 con 0x7f71701004f0 2023-11-16T16:50:12.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.444+0000 7f716cff9700 1 -- 172.21.15.1:0/1497554452 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f716400fcc0 con 0x7f71701004f0 2023-11-16T16:50:12.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.444+0000 7f716cff9700 1 --2- 172.21.15.1:0/1497554452 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f71580a12c0 0x7f71580a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:12.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.445+0000 7f716f7fe700 1 --2- 172.21.15.1:0/1497554452 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f71580a12c0 0x7f71580a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:12.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.445+0000 7f716cff9700 1 -- 172.21.15.1:0/1497554452 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8586+0+0 (secure 0 0 0) 0x7f71640ce2a0 con 0x7f71701004f0 2023-11-16T16:50:12.449 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.445+0000 7f716f7fe700 1 --2- 172.21.15.1:0/1497554452 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f71580a12c0 0x7f71580a3770 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7160005950 tx=0x7f716000b410 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:12.450 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.448+0000 7f716cff9700 1 -- 172.21.15.1:0/1497554452 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7164097700 con 0x7f71701004f0 2023-11-16T16:50:12.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:12 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3704149942' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:50:12.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:12 smithi001 ceph-mon[98800]: pgmap v343: 97 pgs: 16 active+clean+snaptrim, 49 active+clean, 32 active+clean+snaptrim_wait; 817 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-11-16T16:50:12.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:12 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/2035033057' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:50:12.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:12 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/2296290725' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:50:12.698 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:12.697+0000 7f7175fd3700 1 -- 172.21.15.1:0/1497554452 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7170067150 con 0x7f71701004f0 2023-11-16T16:50:12.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:12 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/3704149942' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:50:12.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:12 smithi026 ceph-mon[100112]: pgmap v343: 97 pgs: 16 active+clean+snaptrim, 49 active+clean, 32 active+clean+snaptrim_wait; 817 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-11-16T16:50:12.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:12 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/2035033057' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:50:12.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:12 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/2296290725' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:50:12.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:12 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/3704149942' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:50:12.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:12 smithi053 ceph-mon[100638]: pgmap v343: 97 pgs: 16 active+clean+snaptrim, 49 active+clean, 32 active+clean+snaptrim_wait; 817 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 23 op/s 2023-11-16T16:50:12.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:12 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/2035033057' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-16T16:50:12.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:12 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/2296290725' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-16T16:50:13.388 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.386+0000 7f716cff9700 1 -- 172.21.15.1:0/1497554452 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v34) v1 ==== 161+0+0 (secure 0 0 0) 0x7f716409ad20 con 0x7f71701004f0 2023-11-16T16:50:13.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.389+0000 7f7175fd3700 1 -- 172.21.15.1:0/1497554452 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f71580a12c0 msgr2=0x7f71580a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:13.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.389+0000 7f7175fd3700 1 --2- 172.21.15.1:0/1497554452 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f71580a12c0 0x7f71580a3770 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7160005950 tx=0x7f716000b410 comp rx=0 tx=0).stop 2023-11-16T16:50:13.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.389+0000 7f7175fd3700 1 -- 172.21.15.1:0/1497554452 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71701004f0 msgr2=0x7f71701976f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:13.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.389+0000 7f7175fd3700 1 --2- 172.21.15.1:0/1497554452 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71701004f0 0x7f71701976f0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f71640095c0 tx=0x7f716400f710 comp rx=0 tx=0).stop 2023-11-16T16:50:13.391 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.389+0000 7f7175fd3700 1 -- 172.21.15.1:0/1497554452 shutdown_connections 2023-11-16T16:50:13.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.389+0000 7f7175fd3700 1 --2- 172.21.15.1:0/1497554452 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f71580a12c0 0x7f71580a3770 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:13.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.389+0000 7f7175fd3700 1 --2- 172.21.15.1:0/1497554452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7170105250 0x7f71701a2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:13.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.389+0000 7f7175fd3700 1 --2- 172.21.15.1:0/1497554452 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f71700ffb60 0x7f71701971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:13.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.389+0000 7f7175fd3700 1 --2- 172.21.15.1:0/1497554452 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71701004f0 0x7f71701976f0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:13.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.389+0000 7f7175fd3700 1 -- 172.21.15.1:0/1497554452 >> 172.21.15.1:0/1497554452 conn(0x7f71700748b0 msgr2=0x7f71700fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:13.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.389+0000 7f7175fd3700 1 -- 172.21.15.1:0/1497554452 shutdown_connections 2023-11-16T16:50:13.392 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.389+0000 7f7175fd3700 1 -- 172.21.15.1:0/1497554452 wait complete. 2023-11-16T16:50:13.393 INFO:teuthology.orchestra.run.smithi001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-16T16:50:13.407 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-16T16:50:13.544 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.543+0000 7fc9e7406700 1 -- 172.21.15.1:0/2992688128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9e00ffa70 msgr2=0x7fc9e00ffee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:13.545 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.543+0000 7fc9e7406700 1 --2- 172.21.15.1:0/2992688128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9e00ffa70 0x7fc9e00ffee0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc009990 tx=0x7fc9dc009ca0 comp rx=0 tx=0).stop 2023-11-16T16:50:13.545 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.543+0000 7fc9e7406700 1 -- 172.21.15.1:0/2992688128 shutdown_connections 2023-11-16T16:50:13.545 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.543+0000 7fc9e7406700 1 --2- 172.21.15.1:0/2992688128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9e0100420 0x7fc9e010bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:13.545 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.543+0000 7fc9e7406700 1 --2- 172.21.15.1:0/2992688128 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9e0105250 0x7fc9e0105620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:13.545 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.543+0000 7fc9e7406700 1 --2- 172.21.15.1:0/2992688128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9e00ffa70 0x7fc9e00ffee0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:13.545 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.543+0000 7fc9e7406700 1 -- 172.21.15.1:0/2992688128 >> 172.21.15.1:0/2992688128 conn(0x7fc9e00763a0 msgr2=0x7fc9e00767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:13.546 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.543+0000 7fc9e7406700 1 -- 172.21.15.1:0/2992688128 shutdown_connections 2023-11-16T16:50:13.546 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.543+0000 7fc9e7406700 1 -- 172.21.15.1:0/2992688128 wait complete. 2023-11-16T16:50:13.548 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.544+0000 7fc9e7406700 1 Processor -- start 2023-11-16T16:50:13.548 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.544+0000 7fc9e7406700 1 -- start start 2023-11-16T16:50:13.548 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.544+0000 7fc9e7406700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9e00ffa70 0x7fc9e0195260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:13.548 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.544+0000 7fc9e7406700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9e0100420 0x7fc9e01957a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:13.549 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.544+0000 7fc9e7406700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9e0105250 0x7fc9e0199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:13.549 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.544+0000 7fc9e7406700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e019a240 con 0x7fc9e00ffa70 2023-11-16T16:50:13.549 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.544+0000 7fc9e7406700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e019a3b0 con 0x7fc9e0105250 2023-11-16T16:50:13.549 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.544+0000 7fc9e7406700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e019a570 con 0x7fc9e0100420 2023-11-16T16:50:13.549 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.544+0000 7fc9e51a2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9e00ffa70 0x7fc9e0195260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:13.549 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.544+0000 7fc9e51a2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9e00ffa70 0x7fc9e0195260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47594/0 (socket says 172.21.15.1:47594) 2023-11-16T16:50:13.549 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.544+0000 7fc9e51a2700 1 -- 172.21.15.1:0/1402433324 learned_addr learned my addr 172.21.15.1:0/1402433324 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:50:13.550 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.544+0000 7fc9e49a1700 1 --2- 172.21.15.1:0/1402433324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9e0100420 0x7fc9e01957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:13.550 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.544+0000 7fc9e59a3700 1 --2- 172.21.15.1:0/1402433324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9e0105250 0x7fc9e0199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:13.550 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.545+0000 7fc9e51a2700 1 -- 172.21.15.1:0/1402433324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9e0100420 msgr2=0x7fc9e01957a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:13.550 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.545+0000 7fc9e51a2700 1 --2- 172.21.15.1:0/1402433324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9e0100420 0x7fc9e01957a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:13.550 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.545+0000 7fc9e51a2700 1 -- 172.21.15.1:0/1402433324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9e0105250 msgr2=0x7fc9e0199c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:13.550 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.545+0000 7fc9e51a2700 1 --2- 172.21.15.1:0/1402433324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9e0105250 0x7fc9e0199c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:13.550 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.545+0000 7fc9e51a2700 1 -- 172.21.15.1:0/1402433324 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9dc0095f0 con 0x7fc9e00ffa70 2023-11-16T16:50:13.551 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.545+0000 7fc9e51a2700 1 --2- 172.21.15.1:0/1402433324 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9e00ffa70 0x7fc9e0195260 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fc9d000d950 tx=0x7fc9d000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:13.551 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.545+0000 7fc9d67fc700 1 -- 172.21.15.1:0/1402433324 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d000f890 con 0x7fc9e00ffa70 2023-11-16T16:50:13.551 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.545+0000 7fc9d67fc700 1 -- 172.21.15.1:0/1402433324 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9d0010460 con 0x7fc9e00ffa70 2023-11-16T16:50:13.551 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.545+0000 7fc9e7406700 1 -- 172.21.15.1:0/1402433324 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9e010f1e0 con 0x7fc9e00ffa70 2023-11-16T16:50:13.551 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.545+0000 7fc9d67fc700 1 -- 172.21.15.1:0/1402433324 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d000e610 con 0x7fc9e00ffa70 2023-11-16T16:50:13.551 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.545+0000 7fc9e7406700 1 -- 172.21.15.1:0/1402433324 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e010f050 con 0x7fc9e00ffa70 2023-11-16T16:50:13.551 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.546+0000 7fc9e7406700 1 -- 172.21.15.1:0/1402433324 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9e010f370 con 0x7fc9e00ffa70 2023-11-16T16:50:13.552 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.547+0000 7fc9d67fc700 1 -- 172.21.15.1:0/1402433324 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fc9d00108d0 con 0x7fc9e00ffa70 2023-11-16T16:50:13.552 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.548+0000 7fc9d67fc700 1 --2- 172.21.15.1:0/1402433324 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fc9cc0a1340 0x7fc9cc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:13.553 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.548+0000 7fc9d67fc700 1 -- 172.21.15.1:0/1402433324 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8800+0+0 (secure 0 0 0) 0x7fc9d00ccab0 con 0x7fc9e00ffa70 2023-11-16T16:50:13.553 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.548+0000 7fc9e49a1700 1 --2- 172.21.15.1:0/1402433324 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fc9cc0a1340 0x7fc9cc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:13.553 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.548+0000 7fc9e49a1700 1 --2- 172.21.15.1:0/1402433324 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fc9cc0a1340 0x7fc9cc0a37f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0196870 tx=0x7fc9dc005fd0 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:13.553 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.551+0000 7fc9d67fc700 1 -- 172.21.15.1:0/1402433324 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc9d0095db0 con 0x7fc9e00ffa70 2023-11-16T16:50:13.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:13 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1497554452' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-11-16T16:50:13.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:13 smithi001 ceph-mon[98800]: osdmap e97: 12 total, 12 up, 12 in 2023-11-16T16:50:13.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:13 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mon-a[98776]: 2023-11-16T16:50:13.377+0000 7f85bc7f4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-16T16:50:13.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:13 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1497554452' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-11-16T16:50:13.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:13 smithi026 ceph-mon[100112]: osdmap e97: 12 total, 12 up, 12 in 2023-11-16T16:50:13.758 INFO:journalctl@ceph.mds.b.smithi026.stdout:Nov 16 16:50:13 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-b[125397]: ignoring --setuser ceph since I am not root 2023-11-16T16:50:13.759 INFO:journalctl@ceph.mds.b.smithi026.stdout:Nov 16 16:50:13 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-b[125397]: ignoring --setgroup ceph since I am not root 2023-11-16T16:50:13.759 INFO:journalctl@ceph.mds.b.smithi026.stdout:Nov 16 16:50:13 smithi026 ceph-mds[125422]: -- 172.21.15.26:0/3379518846 <== mon.1 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5632f48d3520 con 0x5632f48a8800 2023-11-16T16:50:13.759 INFO:journalctl@ceph.mds.b.smithi026.stdout:Nov 16 16:50:13 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-b[125397]: starting mds.b at 2023-11-16T16:50:13.760 INFO:journalctl@ceph.mds.e.smithi026.stdout:Nov 16 16:50:13 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-e[124717]: ignoring --setuser ceph since I am not root 2023-11-16T16:50:13.760 INFO:journalctl@ceph.mds.e.smithi026.stdout:Nov 16 16:50:13 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-e[124717]: ignoring --setgroup ceph since I am not root 2023-11-16T16:50:13.760 INFO:journalctl@ceph.mds.e.smithi026.stdout:Nov 16 16:50:13 smithi026 ceph-mds[124741]: -- 172.21.15.26:0/2397943901 <== mon.0 v2:172.21.15.1:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d04ef89520 con 0x55d04ef5e800 2023-11-16T16:50:13.760 INFO:journalctl@ceph.mds.e.smithi026.stdout:Nov 16 16:50:13 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-e[124717]: starting mds.e at 2023-11-16T16:50:13.760 INFO:journalctl@ceph.mds.h.smithi026.stdout:Nov 16 16:50:13 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-h[125059]: ignoring --setuser ceph since I am not root 2023-11-16T16:50:13.761 INFO:journalctl@ceph.mds.h.smithi026.stdout:Nov 16 16:50:13 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-h[125059]: ignoring --setgroup ceph since I am not root 2023-11-16T16:50:13.761 INFO:journalctl@ceph.mds.h.smithi026.stdout:Nov 16 16:50:13 smithi026 ceph-mds[125083]: -- 172.21.15.26:0/152711108 <== mon.0 v2:172.21.15.1:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562eb451b520 con 0x562eb44f0800 2023-11-16T16:50:13.761 INFO:journalctl@ceph.mds.h.smithi026.stdout:Nov 16 16:50:13 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-h[125059]: starting mds.h at 2023-11-16T16:50:13.799 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:13.798+0000 7fc9e7406700 1 -- 172.21.15.1:0/1402433324 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc9e01966b0 con 0x7fc9e00ffa70 2023-11-16T16:50:13.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:13 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1497554452' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-11-16T16:50:13.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:13 smithi053 ceph-mon[100638]: osdmap e97: 12 total, 12 up, 12 in 2023-11-16T16:50:13.831 INFO:journalctl@ceph.mds.c.smithi053.stdout:Nov 16 16:50:13 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-c[126266]: ignoring --setuser ceph since I am not root 2023-11-16T16:50:13.831 INFO:journalctl@ceph.mds.c.smithi053.stdout:Nov 16 16:50:13 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-c[126266]: ignoring --setgroup ceph since I am not root 2023-11-16T16:50:13.831 INFO:journalctl@ceph.mds.c.smithi053.stdout:Nov 16 16:50:13 smithi053 ceph-mds[126290]: -- 172.21.15.53:0/3863187442 <== mon.2 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56392afc1520 con 0x56392af96800 2023-11-16T16:50:13.832 INFO:journalctl@ceph.mds.c.smithi053.stdout:Nov 16 16:50:13 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-c[126266]: starting mds.c at 2023-11-16T16:50:13.832 INFO:journalctl@ceph.mds.f.smithi053.stdout:Nov 16 16:50:13 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-f[125922]: ignoring --setuser ceph since I am not root 2023-11-16T16:50:13.832 INFO:journalctl@ceph.mds.f.smithi053.stdout:Nov 16 16:50:13 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-f[125922]: ignoring --setgroup ceph since I am not root 2023-11-16T16:50:13.832 INFO:journalctl@ceph.mds.f.smithi053.stdout:Nov 16 16:50:13 smithi053 ceph-mds[125946]: -- 172.21.15.53:0/1866709553 <== mon.1 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5616ee349520 con 0x5616ee31f000 2023-11-16T16:50:13.832 INFO:journalctl@ceph.mds.f.smithi053.stdout:Nov 16 16:50:13 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-f[125922]: starting mds.f at 2023-11-16T16:50:13.833 INFO:journalctl@ceph.mds.i.smithi053.stdout:Nov 16 16:50:13 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-i[126607]: ignoring --setuser ceph since I am not root 2023-11-16T16:50:13.833 INFO:journalctl@ceph.mds.i.smithi053.stdout:Nov 16 16:50:13 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-i[126607]: ignoring --setgroup ceph since I am not root 2023-11-16T16:50:13.833 INFO:journalctl@ceph.mds.i.smithi053.stdout:Nov 16 16:50:13 smithi053 ceph-mds[126631]: -- 172.21.15.53:0/1836203004 <== mon.1 v2:172.21.15.53:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d0fe45f520 con 0x55d0fe434800 2023-11-16T16:50:13.833 INFO:journalctl@ceph.mds.i.smithi053.stdout:Nov 16 16:50:13 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-i[126607]: starting mds.i at 2023-11-16T16:50:14.389 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.387+0000 7fc9d67fc700 1 -- 172.21.15.1:0/1402433324 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v35) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc9d00160a0 con 0x7fc9e00ffa70 2023-11-16T16:50:14.393 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.391+0000 7fc9e7406700 1 -- 172.21.15.1:0/1402433324 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fc9cc0a1340 msgr2=0x7fc9cc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:14.393 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.391+0000 7fc9e7406700 1 --2- 172.21.15.1:0/1402433324 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fc9cc0a1340 0x7fc9cc0a37f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0196870 tx=0x7fc9dc005fd0 comp rx=0 tx=0).stop 2023-11-16T16:50:14.393 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.391+0000 7fc9e7406700 1 -- 172.21.15.1:0/1402433324 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9e00ffa70 msgr2=0x7fc9e0195260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:14.393 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.391+0000 7fc9e7406700 1 --2- 172.21.15.1:0/1402433324 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9e00ffa70 0x7fc9e0195260 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fc9d000d950 tx=0x7fc9d000dc60 comp rx=0 tx=0).stop 2023-11-16T16:50:14.393 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.391+0000 7fc9e7406700 1 -- 172.21.15.1:0/1402433324 shutdown_connections 2023-11-16T16:50:14.393 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.391+0000 7fc9e7406700 1 --2- 172.21.15.1:0/1402433324 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fc9cc0a1340 0x7fc9cc0a37f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:14.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.391+0000 7fc9e7406700 1 --2- 172.21.15.1:0/1402433324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9e0105250 0x7fc9e0199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:14.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.391+0000 7fc9e7406700 1 --2- 172.21.15.1:0/1402433324 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9e00ffa70 0x7fc9e0195260 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:14.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.391+0000 7fc9e7406700 1 --2- 172.21.15.1:0/1402433324 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fc9e0100420 0x7fc9e01957a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:14.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.391+0000 7fc9e7406700 1 -- 172.21.15.1:0/1402433324 >> 172.21.15.1:0/1402433324 conn(0x7fc9e00763a0 msgr2=0x7fc9e00fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:14.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.391+0000 7fc9e7406700 1 -- 172.21.15.1:0/1402433324 shutdown_connections 2023-11-16T16:50:14.394 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.391+0000 7fc9e7406700 1 -- 172.21.15.1:0/1402433324 wait complete. 2023-11-16T16:50:14.408 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-16T16:50:14.546 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.544+0000 7fe8be471700 1 -- 172.21.15.1:0/540403863 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe8b81014b0 msgr2=0x7fe8b8101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:14.546 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.544+0000 7fe8be471700 1 --2- 172.21.15.1:0/540403863 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe8b81014b0 0x7fe8b8101880 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8009990 tx=0x7fe8a8009ca0 comp rx=0 tx=0).stop 2023-11-16T16:50:14.546 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.544+0000 7fe8be471700 1 -- 172.21.15.1:0/540403863 shutdown_connections 2023-11-16T16:50:14.546 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.544+0000 7fe8be471700 1 --2- 172.21.15.1:0/540403863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8b810a880 0x7fe8b810ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:14.546 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.544+0000 7fe8be471700 1 --2- 172.21.15.1:0/540403863 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe8b81014b0 0x7fe8b8101880 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:14.546 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.544+0000 7fe8be471700 1 --2- 172.21.15.1:0/540403863 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8b8101dc0 0x7fe8b810a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:14.547 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.544+0000 7fe8be471700 1 -- 172.21.15.1:0/540403863 >> 172.21.15.1:0/540403863 conn(0x7fe8b80763a0 msgr2=0x7fe8b80767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:14.547 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.544+0000 7fe8be471700 1 -- 172.21.15.1:0/540403863 shutdown_connections 2023-11-16T16:50:14.547 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.544+0000 7fe8be471700 1 -- 172.21.15.1:0/540403863 wait complete. 2023-11-16T16:50:14.549 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.544+0000 7fe8be471700 1 Processor -- start 2023-11-16T16:50:14.549 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.545+0000 7fe8be471700 1 -- start start 2023-11-16T16:50:14.550 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.545+0000 7fe8be471700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8b81014b0 0x7fe8b80ff100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:14.550 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.545+0000 7fe8be471700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe8b8101dc0 0x7fe8b80ff640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:14.550 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.545+0000 7fe8be471700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8b810a880 0x7fe8b81a4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:14.550 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.545+0000 7fe8be471700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b80ffe70 con 0x7fe8b8101dc0 2023-11-16T16:50:14.550 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.545+0000 7fe8be471700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b81a4a80 con 0x7fe8b81014b0 2023-11-16T16:50:14.550 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.545+0000 7fe8be471700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b81a4bc0 con 0x7fe8b810a880 2023-11-16T16:50:14.550 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.545+0000 7fe8b77fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe8b8101dc0 0x7fe8b80ff640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:14.551 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.545+0000 7fe8b77fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe8b8101dc0 0x7fe8b80ff640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47608/0 (socket says 172.21.15.1:47608) 2023-11-16T16:50:14.551 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.545+0000 7fe8b77fe700 1 -- 172.21.15.1:0/3186701805 learned_addr learned my addr 172.21.15.1:0/3186701805 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:50:14.551 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.545+0000 7fe8b7fff700 1 --2- 172.21.15.1:0/3186701805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8b81014b0 0x7fe8b80ff100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:14.551 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.545+0000 7fe8bca0e700 1 --2- 172.21.15.1:0/3186701805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8b810a880 0x7fe8b81a4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:14.551 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.545+0000 7fe8b77fe700 1 -- 172.21.15.1:0/3186701805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8b810a880 msgr2=0x7fe8b81a4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:14.551 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.545+0000 7fe8b77fe700 1 --2- 172.21.15.1:0/3186701805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8b810a880 0x7fe8b81a4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:14.551 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.545+0000 7fe8b77fe700 1 -- 172.21.15.1:0/3186701805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8b81014b0 msgr2=0x7fe8b80ff100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:14.551 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.546+0000 7fe8b77fe700 1 --2- 172.21.15.1:0/3186701805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8b81014b0 0x7fe8b80ff100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:14.552 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.546+0000 7fe8b77fe700 1 -- 172.21.15.1:0/3186701805 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a80095f0 con 0x7fe8b8101dc0 2023-11-16T16:50:14.552 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.546+0000 7fe8b7fff700 1 --2- 172.21.15.1:0/3186701805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8b81014b0 0x7fe8b80ff100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-16T16:50:14.552 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.546+0000 7fe8b77fe700 1 --2- 172.21.15.1:0/3186701805 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe8b8101dc0 0x7fe8b80ff640 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00d980 tx=0x7fe8ac00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:14.552 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.546+0000 7fe8b57fa700 1 -- 172.21.15.1:0/3186701805 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac009b40 con 0x7fe8b8101dc0 2023-11-16T16:50:14.552 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.546+0000 7fe8b57fa700 1 -- 172.21.15.1:0/3186701805 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8ac00de60 con 0x7fe8b8101dc0 2023-11-16T16:50:14.552 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.546+0000 7fe8be471700 1 -- 172.21.15.1:0/3186701805 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b8110320 con 0x7fe8b8101dc0 2023-11-16T16:50:14.552 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.546+0000 7fe8b57fa700 1 -- 172.21.15.1:0/3186701805 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac005580 con 0x7fe8b8101dc0 2023-11-16T16:50:14.552 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.546+0000 7fe8be471700 1 -- 172.21.15.1:0/3186701805 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b8110190 con 0x7fe8b8101dc0 2023-11-16T16:50:14.553 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.547+0000 7fe8be471700 1 -- 172.21.15.1:0/3186701805 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8b81104b0 con 0x7fe8b8101dc0 2023-11-16T16:50:14.553 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.548+0000 7fe8b57fa700 1 -- 172.21.15.1:0/3186701805 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fe8ac004750 con 0x7fe8b8101dc0 2023-11-16T16:50:14.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.549+0000 7fe8b57fa700 1 --2- 172.21.15.1:0/3186701805 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fe8a00a1340 0x7fe8a00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:14.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.549+0000 7fe8b7fff700 1 --2- 172.21.15.1:0/3186701805 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fe8a00a1340 0x7fe8a00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:14.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.549+0000 7fe8b57fa700 1 -- 172.21.15.1:0/3186701805 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8800+0+0 (secure 0 0 0) 0x7fe8ac0cd1d0 con 0x7fe8b8101dc0 2023-11-16T16:50:14.554 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.549+0000 7fe8b7fff700 1 --2- 172.21.15.1:0/3186701805 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fe8a00a1340 0x7fe8a00a37f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8006010 tx=0x7fe8a800b540 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:14.555 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.553+0000 7fe8b57fa700 1 -- 172.21.15.1:0/3186701805 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe8ac096510 con 0x7fe8b8101dc0 2023-11-16T16:50:14.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:14 smithi001 ceph-mon[98800]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-11-16T16:50:14.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:14 smithi001 ceph-mon[98800]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-16T16:50:14.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:14 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1497554452' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-11-16T16:50:14.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:14 smithi001 ceph-mon[98800]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-11-16T16:50:14.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:14 smithi001 ceph-mon[98800]: pgmap v345: 97 pgs: 16 active+clean+snaptrim, 49 active+clean, 32 active+clean+snaptrim_wait; 817 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2023-11-16T16:50:14.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:14 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1402433324' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-11-16T16:50:14.756 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:14.755+0000 7fe8be471700 1 -- 172.21.15.1:0/3186701805 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe8b8100720 con 0x7fe8b8101dc0 2023-11-16T16:50:14.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:14 smithi026 ceph-mon[100112]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-11-16T16:50:14.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:14 smithi026 ceph-mon[100112]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-16T16:50:14.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:14 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1497554452' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-11-16T16:50:14.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:14 smithi026 ceph-mon[100112]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-11-16T16:50:14.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:14 smithi026 ceph-mon[100112]: pgmap v345: 97 pgs: 16 active+clean+snaptrim, 49 active+clean, 32 active+clean+snaptrim_wait; 817 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2023-11-16T16:50:14.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:14 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1402433324' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-11-16T16:50:14.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:14 smithi053 ceph-mon[100638]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-11-16T16:50:14.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:14 smithi053 ceph-mon[100638]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-16T16:50:14.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:14 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1497554452' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-11-16T16:50:14.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:14 smithi053 ceph-mon[100638]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-11-16T16:50:14.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:14 smithi053 ceph-mon[100638]: pgmap v345: 97 pgs: 16 active+clean+snaptrim, 49 active+clean, 32 active+clean+snaptrim_wait; 817 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2023-11-16T16:50:14.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:14 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1402433324' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-11-16T16:50:15.488 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.487+0000 7fe8b57fa700 1 -- 172.21.15.1:0/3186701805 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v98) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe8ac017030 con 0x7fe8b8101dc0 2023-11-16T16:50:15.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.490+0000 7fe8be471700 1 -- 172.21.15.1:0/3186701805 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fe8a00a1340 msgr2=0x7fe8a00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:15.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.490+0000 7fe8be471700 1 --2- 172.21.15.1:0/3186701805 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fe8a00a1340 0x7fe8a00a37f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8006010 tx=0x7fe8a800b540 comp rx=0 tx=0).stop 2023-11-16T16:50:15.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.490+0000 7fe8be471700 1 -- 172.21.15.1:0/3186701805 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe8b8101dc0 msgr2=0x7fe8b80ff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:15.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.490+0000 7fe8be471700 1 --2- 172.21.15.1:0/3186701805 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe8b8101dc0 0x7fe8b80ff640 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00d980 tx=0x7fe8ac00dd40 comp rx=0 tx=0).stop 2023-11-16T16:50:15.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.491+0000 7fe8be471700 1 -- 172.21.15.1:0/3186701805 shutdown_connections 2023-11-16T16:50:15.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.491+0000 7fe8be471700 1 --2- 172.21.15.1:0/3186701805 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7fe8a00a1340 0x7fe8a00a37f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:15.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.491+0000 7fe8be471700 1 --2- 172.21.15.1:0/3186701805 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7fe8b810a880 0x7fe8b81a4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:15.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.491+0000 7fe8be471700 1 --2- 172.21.15.1:0/3186701805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8b81014b0 0x7fe8b80ff100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:15.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.491+0000 7fe8be471700 1 --2- 172.21.15.1:0/3186701805 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe8b8101dc0 0x7fe8b80ff640 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:15.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.491+0000 7fe8be471700 1 -- 172.21.15.1:0/3186701805 >> 172.21.15.1:0/3186701805 conn(0x7fe8b80763a0 msgr2=0x7fe8b8102250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:15.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.491+0000 7fe8be471700 1 -- 172.21.15.1:0/3186701805 shutdown_connections 2023-11-16T16:50:15.494 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.491+0000 7fe8be471700 1 -- 172.21.15.1:0/3186701805 wait complete. 2023-11-16T16:50:15.495 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_metadata' removed 2023-11-16T16:50:15.514 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-16T16:50:15.652 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.648+0000 7f6dff63d700 1 -- 172.21.15.1:0/2889121115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6df8105250 msgr2=0x7f6df8105620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:15.652 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.648+0000 7f6dff63d700 1 --2- 172.21.15.1:0/2889121115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6df8105250 0x7f6df8105620 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f6df4009990 tx=0x7f6df4009ca0 comp rx=0 tx=0).stop 2023-11-16T16:50:15.652 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.648+0000 7f6dff63d700 1 -- 172.21.15.1:0/2889121115 shutdown_connections 2023-11-16T16:50:15.652 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.648+0000 7f6dff63d700 1 --2- 172.21.15.1:0/2889121115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6df8100420 0x7f6df810bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:15.653 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.648+0000 7f6dff63d700 1 --2- 172.21.15.1:0/2889121115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6df8105250 0x7f6df8105620 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:15.653 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.648+0000 7f6dff63d700 1 --2- 172.21.15.1:0/2889121115 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6df80ffa70 0x7f6df80ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:15.653 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.648+0000 7f6dff63d700 1 -- 172.21.15.1:0/2889121115 >> 172.21.15.1:0/2889121115 conn(0x7f6df80763a0 msgr2=0x7f6df80767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:15.653 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.648+0000 7f6dff63d700 1 -- 172.21.15.1:0/2889121115 shutdown_connections 2023-11-16T16:50:15.653 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.648+0000 7f6dff63d700 1 -- 172.21.15.1:0/2889121115 wait complete. 2023-11-16T16:50:15.653 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.649+0000 7f6dff63d700 1 Processor -- start 2023-11-16T16:50:15.653 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.649+0000 7f6dff63d700 1 -- start start 2023-11-16T16:50:15.654 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.649+0000 7f6dff63d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6df80ffa70 0x7f6df81963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:15.654 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.649+0000 7f6dff63d700 1 --2- >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6df8100420 0x7f6df81968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:15.654 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.649+0000 7f6dff63d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6df8105250 0x7f6df819ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:15.654 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.649+0000 7f6dff63d700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df819b380 con 0x7f6df80ffa70 2023-11-16T16:50:15.654 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.649+0000 7f6dff63d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df819b4f0 con 0x7f6df8105250 2023-11-16T16:50:15.654 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.649+0000 7f6dff63d700 1 -- --> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df819b6b0 con 0x7f6df8100420 2023-11-16T16:50:15.654 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.649+0000 7f6dfd3d9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6df80ffa70 0x7f6df81963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:15.655 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.649+0000 7f6dfd3d9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6df80ffa70 0x7f6df81963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47620/0 (socket says 172.21.15.1:47620) 2023-11-16T16:50:15.655 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.649+0000 7f6dfd3d9700 1 -- 172.21.15.1:0/3539890791 learned_addr learned my addr 172.21.15.1:0/3539890791 (peer_addr_for_me v2:172.21.15.1:0/0) 2023-11-16T16:50:15.655 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.649+0000 7f6dfcbd8700 1 --2- 172.21.15.1:0/3539890791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6df8100420 0x7f6df81968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:15.655 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.649+0000 7f6dfd3d9700 1 -- 172.21.15.1:0/3539890791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6df8100420 msgr2=0x7f6df81968e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:15.655 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.649+0000 7f6dfd3d9700 1 --2- 172.21.15.1:0/3539890791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6df8100420 0x7f6df81968e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:15.655 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.649+0000 7f6dfd3d9700 1 -- 172.21.15.1:0/3539890791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6df8105250 msgr2=0x7f6df819ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:15.655 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.649+0000 7f6dfdbda700 1 --2- 172.21.15.1:0/3539890791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6df8105250 0x7f6df819ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:15.655 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.649+0000 7f6dfd3d9700 1 --2- 172.21.15.1:0/3539890791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6df8105250 0x7f6df819ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:15.656 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.649+0000 7f6dfd3d9700 1 -- 172.21.15.1:0/3539890791 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df40095f0 con 0x7f6df80ffa70 2023-11-16T16:50:15.656 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.650+0000 7f6dfd3d9700 1 --2- 172.21.15.1:0/3539890791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6df80ffa70 0x7f6df81963a0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f6df4000c00 tx=0x7f6df400f7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:15.656 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.650+0000 7f6dea7fc700 1 -- 172.21.15.1:0/3539890791 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df401d070 con 0x7f6df80ffa70 2023-11-16T16:50:15.656 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.650+0000 7f6dea7fc700 1 -- 172.21.15.1:0/3539890791 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6df400fcb0 con 0x7f6df80ffa70 2023-11-16T16:50:15.656 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.650+0000 7f6dea7fc700 1 -- 172.21.15.1:0/3539890791 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df4017630 con 0x7f6df80ffa70 2023-11-16T16:50:15.656 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.650+0000 7f6dff63d700 1 -- 172.21.15.1:0/3539890791 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6df810f230 con 0x7f6df80ffa70 2023-11-16T16:50:15.656 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.650+0000 7f6dff63d700 1 -- 172.21.15.1:0/3539890791 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df810f0a0 con 0x7f6df80ffa70 2023-11-16T16:50:15.657 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.651+0000 7f6dff63d700 1 -- 172.21.15.1:0/3539890791 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6df810f3c0 con 0x7f6df80ffa70 2023-11-16T16:50:15.657 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.652+0000 7f6dea7fc700 1 -- 172.21.15.1:0/3539890791 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f6df4022470 con 0x7f6df80ffa70 2023-11-16T16:50:15.657 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.652+0000 7f6dea7fc700 1 --2- 172.21.15.1:0/3539890791 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f6de40a1340 0x7f6de40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-16T16:50:15.658 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.653+0000 7f6dea7fc700 1 -- 172.21.15.1:0/3539890791 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8246+0+0 (secure 0 0 0) 0x7f6df40ce310 con 0x7f6df80ffa70 2023-11-16T16:50:15.658 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.653+0000 7f6dfcbd8700 1 --2- 172.21.15.1:0/3539890791 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f6de40a1340 0x7f6de40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-16T16:50:15.658 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.653+0000 7f6dfcbd8700 1 --2- 172.21.15.1:0/3539890791 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f6de40a1340 0x7f6de40a37f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6dec006fd0 tx=0x7f6dec008040 comp rx=0 tx=0).ready entity=mgr.15432 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-16T16:50:15.658 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.656+0000 7f6dea7fc700 1 -- 172.21.15.1:0/3539890791 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6df40978c0 con 0x7f6df80ffa70 2023-11-16T16:50:15.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:15 smithi001 ceph-mon[98800]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-11-16T16:50:15.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:15 smithi001 ceph-mon[98800]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-11-16T16:50:15.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:15 smithi001 ceph-mon[98800]: Cluster is now healthy 2023-11-16T16:50:15.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:15 smithi001 ceph-mon[98800]: mds.? [v2:172.21.15.26:6835/872590738,v1:172.21.15.26:6837/872590738] up:boot 2023-11-16T16:50:15.682 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:15 smithi001 ceph-mon[98800]: mds.? [v2:172.21.15.53:6834/4121613725,v1:172.21.15.53:6835/4121613725] up:boot 2023-11-16T16:50:15.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:15 smithi001 ceph-mon[98800]: mds.? [v2:172.21.15.53:6836/3662489397,v1:172.21.15.53:6838/3662489397] up:boot 2023-11-16T16:50:15.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:15 smithi001 ceph-mon[98800]: mds.? [v2:172.21.15.53:6837/1192817471,v1:172.21.15.53:6839/1192817471] up:boot 2023-11-16T16:50:15.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:15 smithi001 ceph-mon[98800]: mds.? [v2:172.21.15.26:6834/1356098572,v1:172.21.15.26:6836/1356098572] up:boot 2023-11-16T16:50:15.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:15 smithi001 ceph-mon[98800]: mds.? [v2:172.21.15.26:6838/445592542,v1:172.21.15.26:6839/445592542] up:boot 2023-11-16T16:50:15.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:15 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/1402433324' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-11-16T16:50:15.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:15 smithi001 ceph-mon[98800]: fsmap 12 up:standby 2023-11-16T16:50:15.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:15 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-16T16:50:15.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:15 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-16T16:50:15.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:15 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-16T16:50:15.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:15 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-16T16:50:15.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:15 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-16T16:50:15.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:15 smithi001 ceph-mon[98800]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-16T16:50:15.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:15 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3186701805' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-16T16:50:15.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:15 smithi026 ceph-mon[100112]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-11-16T16:50:15.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:15 smithi026 ceph-mon[100112]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-11-16T16:50:15.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:15 smithi026 ceph-mon[100112]: Cluster is now healthy 2023-11-16T16:50:15.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:15 smithi026 ceph-mon[100112]: mds.? [v2:172.21.15.26:6835/872590738,v1:172.21.15.26:6837/872590738] up:boot 2023-11-16T16:50:15.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:15 smithi026 ceph-mon[100112]: mds.? [v2:172.21.15.53:6834/4121613725,v1:172.21.15.53:6835/4121613725] up:boot 2023-11-16T16:50:15.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:15 smithi026 ceph-mon[100112]: mds.? [v2:172.21.15.53:6836/3662489397,v1:172.21.15.53:6838/3662489397] up:boot 2023-11-16T16:50:15.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:15 smithi026 ceph-mon[100112]: mds.? [v2:172.21.15.53:6837/1192817471,v1:172.21.15.53:6839/1192817471] up:boot 2023-11-16T16:50:15.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:15 smithi026 ceph-mon[100112]: mds.? [v2:172.21.15.26:6834/1356098572,v1:172.21.15.26:6836/1356098572] up:boot 2023-11-16T16:50:15.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:15 smithi026 ceph-mon[100112]: mds.? [v2:172.21.15.26:6838/445592542,v1:172.21.15.26:6839/445592542] up:boot 2023-11-16T16:50:15.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:15 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/1402433324' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-11-16T16:50:15.759 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:15 smithi026 ceph-mon[100112]: fsmap 12 up:standby 2023-11-16T16:50:15.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:15 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-16T16:50:15.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:15 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-16T16:50:15.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:15 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-16T16:50:15.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:15 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-16T16:50:15.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:15 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-16T16:50:15.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:15 smithi026 ceph-mon[100112]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-16T16:50:15.760 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:15 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/3186701805' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-16T16:50:15.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:15 smithi053 ceph-mon[100638]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-11-16T16:50:15.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:15 smithi053 ceph-mon[100638]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-11-16T16:50:15.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:15 smithi053 ceph-mon[100638]: Cluster is now healthy 2023-11-16T16:50:15.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:15 smithi053 ceph-mon[100638]: mds.? [v2:172.21.15.26:6835/872590738,v1:172.21.15.26:6837/872590738] up:boot 2023-11-16T16:50:15.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:15 smithi053 ceph-mon[100638]: mds.? [v2:172.21.15.53:6834/4121613725,v1:172.21.15.53:6835/4121613725] up:boot 2023-11-16T16:50:15.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:15 smithi053 ceph-mon[100638]: mds.? [v2:172.21.15.53:6836/3662489397,v1:172.21.15.53:6838/3662489397] up:boot 2023-11-16T16:50:15.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:15 smithi053 ceph-mon[100638]: mds.? [v2:172.21.15.53:6837/1192817471,v1:172.21.15.53:6839/1192817471] up:boot 2023-11-16T16:50:15.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:15 smithi053 ceph-mon[100638]: mds.? [v2:172.21.15.26:6834/1356098572,v1:172.21.15.26:6836/1356098572] up:boot 2023-11-16T16:50:15.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:15 smithi053 ceph-mon[100638]: mds.? [v2:172.21.15.26:6838/445592542,v1:172.21.15.26:6839/445592542] up:boot 2023-11-16T16:50:15.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:15 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/1402433324' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-11-16T16:50:15.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:15 smithi053 ceph-mon[100638]: fsmap 12 up:standby 2023-11-16T16:50:15.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:15 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-16T16:50:15.832 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:15 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-16T16:50:15.833 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:15 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-16T16:50:15.833 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:15 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-16T16:50:15.833 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:15 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-16T16:50:15.833 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:15 smithi053 ceph-mon[100638]: from='mgr.15432 172.21.15.1:0/637138521' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-16T16:50:15.833 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:15 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/3186701805' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-16T16:50:15.855 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:15.853+0000 7f6dff63d700 1 -- 172.21.15.1:0/3539890791 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6df8067a50 con 0x7f6df80ffa70 2023-11-16T16:50:16.493 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:16.491+0000 7f6dea7fc700 1 -- 172.21.15.1:0/3539890791 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v99) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6df409aee0 con 0x7f6df80ffa70 2023-11-16T16:50:16.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:16.495+0000 7f6dff63d700 1 -- 172.21.15.1:0/3539890791 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f6de40a1340 msgr2=0x7f6de40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:16.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:16.495+0000 7f6dff63d700 1 --2- 172.21.15.1:0/3539890791 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f6de40a1340 0x7f6de40a37f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6dec006fd0 tx=0x7f6dec008040 comp rx=0 tx=0).stop 2023-11-16T16:50:16.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:16.495+0000 7f6dff63d700 1 -- 172.21.15.1:0/3539890791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6df80ffa70 msgr2=0x7f6df81963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-16T16:50:16.497 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:16.495+0000 7f6dff63d700 1 --2- 172.21.15.1:0/3539890791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6df80ffa70 0x7f6df81963a0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f6df4000c00 tx=0x7f6df400f7c0 comp rx=0 tx=0).stop 2023-11-16T16:50:16.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:16.495+0000 7f6dff63d700 1 -- 172.21.15.1:0/3539890791 shutdown_connections 2023-11-16T16:50:16.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:16.495+0000 7f6dff63d700 1 --2- 172.21.15.1:0/3539890791 >> [v2:172.21.15.1:6800/4199419371,v1:172.21.15.1:6801/4199419371] conn(0x7f6de40a1340 0x7f6de40a37f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:16.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:16.495+0000 7f6dff63d700 1 --2- 172.21.15.1:0/3539890791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6df8105250 0x7f6df819ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:16.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:16.495+0000 7f6dff63d700 1 --2- 172.21.15.1:0/3539890791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6df80ffa70 0x7f6df81963a0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:16.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:16.495+0000 7f6dff63d700 1 --2- 172.21.15.1:0/3539890791 >> [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] conn(0x7f6df8100420 0x7f6df81968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-16T16:50:16.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:16.495+0000 7f6dff63d700 1 -- 172.21.15.1:0/3539890791 >> 172.21.15.1:0/3539890791 conn(0x7f6df80763a0 msgr2=0x7f6df80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-16T16:50:16.498 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:16.495+0000 7f6dff63d700 1 -- 172.21.15.1:0/3539890791 shutdown_connections 2023-11-16T16:50:16.499 INFO:teuthology.orchestra.run.smithi001.stderr:2023-11-16T16:50:16.496+0000 7f6dff63d700 1 -- 172.21.15.1:0/3539890791 wait complete. 2023-11-16T16:50:16.499 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data' removed 2023-11-16T16:50:16.517 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-16T16:50:16.548 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-16T16:50:16.584 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-16T16:50:16.617 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-11-16T16:50:16.618 DEBUG:teuthology.orchestra.run.smithi001:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-11-16T16:50:16.636 DEBUG:teuthology.orchestra.run.smithi026:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-11-16T16:50:16.662 DEBUG:teuthology.orchestra.run.smithi053:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-11-16T16:50:16.691 INFO:tasks.cephadm:Stopping all daemons... 2023-11-16T16:50:16.692 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-11-16T16:50:16.692 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mon.a 2023-11-16T16:50:16.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:16 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/3186701805' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-11-16T16:50:16.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:16 smithi026 ceph-mon[100112]: osdmap e98: 12 total, 12 up, 12 in 2023-11-16T16:50:16.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:16 smithi026 ceph-mon[100112]: pgmap v347: 65 pgs: 65 active+clean; 577 KiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:50:16.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:16 smithi026 ceph-mon[100112]: from='client.? 172.21.15.1:0/3539890791' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-16T16:50:16.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:16 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/3186701805' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-11-16T16:50:16.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:16 smithi053 ceph-mon[100638]: osdmap e98: 12 total, 12 up, 12 in 2023-11-16T16:50:16.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:16 smithi053 ceph-mon[100638]: pgmap v347: 65 pgs: 65 active+clean; 577 KiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:50:16.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:16 smithi053 ceph-mon[100638]: from='client.? 172.21.15.1:0/3539890791' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-16T16:50:16.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:16 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3186701805' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-11-16T16:50:16.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:16 smithi001 ceph-mon[98800]: osdmap e98: 12 total, 12 up, 12 in 2023-11-16T16:50:16.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:16 smithi001 ceph-mon[98800]: pgmap v347: 65 pgs: 65 active+clean; 577 KiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail 2023-11-16T16:50:16.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:16 smithi001 ceph-mon[98800]: from='client.? 172.21.15.1:0/3539890791' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-16T16:50:16.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:16 smithi001 systemd[1]: Stopping Ceph mon.a for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:50:17.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:17 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mon-a[98776]: 2023-11-16T16:50:17.088+0000 7f85bf7fa700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:50:17.432 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:17 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mon-a[98776]: 2023-11-16T16:50:17.088+0000 7f85bf7fa700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-11-16T16:50:17.854 INFO:journalctl@ceph.mon.a.smithi001.stdout:Nov 16 16:50:17 smithi001 bash[262402]: ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mon-a 2023-11-16T16:50:17.861 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mon.a.service' 2023-11-16T16:50:17.911 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:50:17.911 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-11-16T16:50:17.911 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-11-16T16:50:17.911 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mon.b 2023-11-16T16:50:18.258 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:17 smithi026 systemd[1]: Stopping Ceph mon.b for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:50:18.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:18 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mon-b[100088]: 2023-11-16T16:50:18.304+0000 7fb7d157e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:50:18.758 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:18 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mon-b[100088]: 2023-11-16T16:50:18.304+0000 7fb7d157e700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-11-16T16:50:19.046 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mon.b.service' 2023-11-16T16:50:19.071 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:18 smithi026 bash[131321]: ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mon-b 2023-11-16T16:50:19.071 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:19 smithi026 systemd[1]: ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mon.b.service: Succeeded. 2023-11-16T16:50:19.071 INFO:journalctl@ceph.mon.b.smithi026.stdout:Nov 16 16:50:19 smithi026 systemd[1]: Stopped Ceph mon.b for 35d9b010-849d-11ee-8dbe-212e2dc638e7. 2023-11-16T16:50:19.094 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:50:19.094 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-11-16T16:50:19.095 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-11-16T16:50:19.095 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mon.c 2023-11-16T16:50:19.475 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:19 smithi053 systemd[1]: Stopping Ceph mon.c for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:50:19.830 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:19 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mon-c[100594]: 2023-11-16T16:50:19.473+0000 7fedd2bce700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:50:19.831 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:19 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mon-c[100594]: 2023-11-16T16:50:19.473+0000 7fedd2bce700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-11-16T16:50:20.124 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mon.c.service' 2023-11-16T16:50:20.151 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:19 smithi053 bash[132791]: ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mon-c 2023-11-16T16:50:20.151 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:20 smithi053 systemd[1]: ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mon.c.service: Succeeded. 2023-11-16T16:50:20.151 INFO:journalctl@ceph.mon.c.smithi053.stdout:Nov 16 16:50:20 smithi053 systemd[1]: Stopped Ceph mon.c for 35d9b010-849d-11ee-8dbe-212e2dc638e7. 2023-11-16T16:50:20.172 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:50:20.173 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-11-16T16:50:20.173 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-11-16T16:50:20.173 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mgr.x 2023-11-16T16:50:20.526 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:50:20 smithi001 systemd[1]: Stopping Ceph mgr.x for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:50:20.849 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:50:20 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x[99043]: 2023-11-16T16:50:20.523+0000 7f4b9765a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:50:21.076 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mgr.x.service' 2023-11-16T16:50:21.101 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:50:20 smithi001 bash[262548]: ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-x 2023-11-16T16:50:21.101 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:50:21 smithi001 systemd[1]: ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mgr.x.service: Succeeded. 2023-11-16T16:50:21.101 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Nov 16 16:50:21 smithi001 systemd[1]: Stopped Ceph mgr.x for 35d9b010-849d-11ee-8dbe-212e2dc638e7. 2023-11-16T16:50:21.114 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:50:21.114 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-11-16T16:50:21.114 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-11-16T16:50:21.114 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mgr.y 2023-11-16T16:50:21.507 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:50:21 smithi026 systemd[1]: Stopping Ceph mgr.y for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:50:22.008 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:50:21 smithi026 bash[131469]: ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-y 2023-11-16T16:50:22.008 INFO:journalctl@ceph.mgr.y.smithi026.stdout:Nov 16 16:50:21 smithi026 systemd[1]: ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-11-16T16:50:22.127 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mgr.y.service' 2023-11-16T16:50:22.166 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:50:22.167 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-11-16T16:50:22.167 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-11-16T16:50:22.167 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mgr.z 2023-11-16T16:50:22.580 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:50:22 smithi053 systemd[1]: Stopping Ceph mgr.z for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:50:23.171 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:50:22 smithi053 bash[132932]: ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mgr-z 2023-11-16T16:50:23.172 INFO:journalctl@ceph.mgr.z.smithi053.stdout:Nov 16 16:50:23 smithi053 systemd[1]: ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-11-16T16:50:23.176 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mgr.z.service' 2023-11-16T16:50:23.214 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:50:23.214 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-11-16T16:50:23.214 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-11-16T16:50:23.214 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@osd.0 2023-11-16T16:50:23.576 INFO:journalctl@ceph.osd.0.smithi001.stdout:Nov 16 16:50:23 smithi001 systemd[1]: Stopping Ceph osd.0 for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:50:23.932 INFO:journalctl@ceph.osd.0.smithi001.stdout:Nov 16 16:50:23 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-0[111670]: 2023-11-16T16:50:23.574+0000 7faafce89700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:50:23.932 INFO:journalctl@ceph.osd.0.smithi001.stdout:Nov 16 16:50:23 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-0[111670]: 2023-11-16T16:50:23.574+0000 7faafce89700 -1 osd.0 99 *** Got signal Terminated *** 2023-11-16T16:50:23.932 INFO:journalctl@ceph.osd.0.smithi001.stdout:Nov 16 16:50:23 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-0[111670]: 2023-11-16T16:50:23.574+0000 7faafce89700 -1 osd.0 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-16T16:50:29.683 INFO:journalctl@ceph.osd.0.smithi001.stdout:Nov 16 16:50:29 smithi001 bash[262693]: ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-0 2023-11-16T16:50:32.180 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@osd.0.service' 2023-11-16T16:50:32.226 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:50:32.226 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-11-16T16:50:32.227 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-11-16T16:50:32.227 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@osd.1 2023-11-16T16:50:32.600 INFO:journalctl@ceph.osd.1.smithi026.stdout:Nov 16 16:50:32 smithi026 systemd[1]: Stopping Ceph osd.1 for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:50:33.008 INFO:journalctl@ceph.osd.1.smithi026.stdout:Nov 16 16:50:32 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-1[105219]: 2023-11-16T16:50:32.597+0000 7fee21830700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:50:33.008 INFO:journalctl@ceph.osd.1.smithi026.stdout:Nov 16 16:50:32 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-1[105219]: 2023-11-16T16:50:32.597+0000 7fee21830700 -1 osd.1 99 *** Got signal Terminated *** 2023-11-16T16:50:33.008 INFO:journalctl@ceph.osd.1.smithi026.stdout:Nov 16 16:50:32 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-1[105219]: 2023-11-16T16:50:32.597+0000 7fee21830700 -1 osd.1 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-16T16:50:38.508 INFO:journalctl@ceph.osd.1.smithi026.stdout:Nov 16 16:50:38 smithi026 bash[131617]: ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-1 2023-11-16T16:50:39.945 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@osd.1.service' 2023-11-16T16:50:39.987 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:50:39.988 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-11-16T16:50:39.988 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-11-16T16:50:39.988 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@osd.2 2023-11-16T16:50:40.331 INFO:journalctl@ceph.osd.2.smithi053.stdout:Nov 16 16:50:40 smithi053 systemd[1]: Stopping Ceph osd.2 for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:50:40.831 INFO:journalctl@ceph.osd.2.smithi053.stdout:Nov 16 16:50:40 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-2[106394]: 2023-11-16T16:50:40.370+0000 7f44c2024700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:50:40.831 INFO:journalctl@ceph.osd.2.smithi053.stdout:Nov 16 16:50:40 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-2[106394]: 2023-11-16T16:50:40.370+0000 7f44c2024700 -1 osd.2 99 *** Got signal Terminated *** 2023-11-16T16:50:40.831 INFO:journalctl@ceph.osd.2.smithi053.stdout:Nov 16 16:50:40 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-2[106394]: 2023-11-16T16:50:40.370+0000 7f44c2024700 -1 osd.2 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-16T16:50:46.330 INFO:journalctl@ceph.osd.2.smithi053.stdout:Nov 16 16:50:46 smithi053 bash[133087]: ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-2 2023-11-16T16:50:47.639 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@osd.2.service' 2023-11-16T16:50:47.681 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:50:47.681 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-11-16T16:50:47.681 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-11-16T16:50:47.681 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@osd.3 2023-11-16T16:50:48.049 INFO:journalctl@ceph.osd.3.smithi001.stdout:Nov 16 16:50:47 smithi001 systemd[1]: Stopping Ceph osd.3 for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:50:48.431 INFO:journalctl@ceph.osd.3.smithi001.stdout:Nov 16 16:50:48 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-3[117210]: 2023-11-16T16:50:48.046+0000 7fa708b60700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:50:48.432 INFO:journalctl@ceph.osd.3.smithi001.stdout:Nov 16 16:50:48 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-3[117210]: 2023-11-16T16:50:48.046+0000 7fa708b60700 -1 osd.3 99 *** Got signal Terminated *** 2023-11-16T16:50:48.432 INFO:journalctl@ceph.osd.3.smithi001.stdout:Nov 16 16:50:48 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-3[117210]: 2023-11-16T16:50:48.046+0000 7fa708b60700 -1 osd.3 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-16T16:50:50.580 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:50:50 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:50:50.206+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:50:50.682 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:50:50 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:50:50.283+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:26.032803+0000 front 2023-11-16T16:50:26.032692+0000 (oldest deadline 2023-11-16T16:50:49.532862+0000) 2023-11-16T16:50:50.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:50:50 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:50:50.201+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:50:51.432 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:50:51 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:50:51.161+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:50:51.432 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:50:51 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:50:51.322+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:26.032803+0000 front 2023-11-16T16:50:26.032692+0000 (oldest deadline 2023-11-16T16:50:49.532862+0000) 2023-11-16T16:50:51.580 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:50:51 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:50:51.161+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:50:52.580 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:50:52 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:50:52.136+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:50:52.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:50:52 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:50:52.194+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:50:52.682 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:50:52 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:50:52.311+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:26.032803+0000 front 2023-11-16T16:50:26.032692+0000 (oldest deadline 2023-11-16T16:50:49.532862+0000) 2023-11-16T16:50:53.515 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:50:53 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:50:53.152+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:50:53.548 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:50:53 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:50:53.195+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:50:53.549 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:50:53 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:50:53.359+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:26.032803+0000 front 2023-11-16T16:50:26.032692+0000 (oldest deadline 2023-11-16T16:50:49.532862+0000) 2023-11-16T16:50:53.825 INFO:journalctl@ceph.osd.3.smithi001.stdout:Nov 16 16:50:53 smithi001 bash[262980]: ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-3 2023-11-16T16:50:53.830 INFO:journalctl@ceph.osd.5.smithi053.stdout:Nov 16 16:50:53 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-5[111878]: 2023-11-16T16:50:53.513+0000 7fdbc5967700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.816257+0000 front 2023-11-16T16:50:27.816098+0000 (oldest deadline 2023-11-16T16:50:53.116098+0000) 2023-11-16T16:50:54.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:50:53 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:50:53.658+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:50:54.553 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:50:54 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:50:54.159+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:50:54.682 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:50:54 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:50:54.392+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:26.032803+0000 front 2023-11-16T16:50:26.032692+0000 (oldest deadline 2023-11-16T16:50:49.532862+0000) 2023-11-16T16:50:54.683 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:50:54 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:50:54.238+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:50:54.830 INFO:journalctl@ceph.osd.5.smithi053.stdout:Nov 16 16:50:54 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-5[111878]: 2023-11-16T16:50:54.552+0000 7fdbc5967700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.816257+0000 front 2023-11-16T16:50:27.816098+0000 (oldest deadline 2023-11-16T16:50:53.116098+0000) 2023-11-16T16:50:55.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:50:54 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:50:54.615+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:50:55.198 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@osd.3.service' 2023-11-16T16:50:55.241 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:50:55.241 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-11-16T16:50:55.241 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-11-16T16:50:55.241 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@osd.4 2023-11-16T16:50:55.569 INFO:journalctl@ceph.osd.4.smithi026.stdout:Nov 16 16:50:55 smithi026 systemd[1]: Stopping Ceph osd.4 for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:50:55.579 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:50:55 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:50:55.118+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:50:55.682 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:50:55 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:50:55.440+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:26.032803+0000 front 2023-11-16T16:50:26.032692+0000 (oldest deadline 2023-11-16T16:50:49.532862+0000) 2023-11-16T16:50:55.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:50:55 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:50:55.287+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:50:55.830 INFO:journalctl@ceph.osd.5.smithi053.stdout:Nov 16 16:50:55 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-5[111878]: 2023-11-16T16:50:55.578+0000 7fdbc5967700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.816257+0000 front 2023-11-16T16:50:27.816098+0000 (oldest deadline 2023-11-16T16:50:53.116098+0000) 2023-11-16T16:50:56.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:50:55 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:50:55.567+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:50:56.008 INFO:journalctl@ceph.osd.4.smithi026.stdout:Nov 16 16:50:55 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-4[110709]: 2023-11-16T16:50:55.602+0000 7f842443d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:50:56.008 INFO:journalctl@ceph.osd.4.smithi026.stdout:Nov 16 16:50:55 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-4[110709]: 2023-11-16T16:50:55.602+0000 7f842443d700 -1 osd.4 99 *** Got signal Terminated *** 2023-11-16T16:50:56.009 INFO:journalctl@ceph.osd.4.smithi026.stdout:Nov 16 16:50:55 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-4[110709]: 2023-11-16T16:50:55.602+0000 7f842443d700 -1 osd.4 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-16T16:50:56.330 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:50:56 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:50:56.073+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:50:56.682 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:50:56 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:50:56.457+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:26.032803+0000 front 2023-11-16T16:50:26.032692+0000 (oldest deadline 2023-11-16T16:50:49.532862+0000) 2023-11-16T16:50:56.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:50:56 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:50:56.330+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:50:57.007 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:50:56 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:50:56.571+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:50:57.080 INFO:journalctl@ceph.osd.5.smithi053.stdout:Nov 16 16:50:56 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-5[111878]: 2023-11-16T16:50:56.616+0000 7fdbc5967700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.816257+0000 front 2023-11-16T16:50:27.816098+0000 (oldest deadline 2023-11-16T16:50:53.116098+0000) 2023-11-16T16:50:57.581 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:50:57 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:50:57.084+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:50:57.682 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:50:57 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:50:57.467+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:26.032803+0000 front 2023-11-16T16:50:26.032692+0000 (oldest deadline 2023-11-16T16:50:49.532862+0000) 2023-11-16T16:50:57.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:50:57 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:50:57.312+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:50:58.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:50:57 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:50:57.604+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:50:58.080 INFO:journalctl@ceph.osd.5.smithi053.stdout:Nov 16 16:50:57 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-5[111878]: 2023-11-16T16:50:57.606+0000 7fdbc5967700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.816257+0000 front 2023-11-16T16:50:27.816098+0000 (oldest deadline 2023-11-16T16:50:53.116098+0000) 2023-11-16T16:50:58.580 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:50:58 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:50:58.132+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:50:58.682 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:50:58 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:50:58.422+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:26.032803+0000 front 2023-11-16T16:50:26.032692+0000 (oldest deadline 2023-11-16T16:50:49.532862+0000) 2023-11-16T16:50:58.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:50:58 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:50:58.330+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:50:59.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:50:58 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:50:58.653+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:50:59.080 INFO:journalctl@ceph.osd.5.smithi053.stdout:Nov 16 16:50:58 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-5[111878]: 2023-11-16T16:50:58.587+0000 7fdbc5967700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.816257+0000 front 2023-11-16T16:50:27.816098+0000 (oldest deadline 2023-11-16T16:50:53.116098+0000) 2023-11-16T16:50:59.580 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:50:59 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:50:59.093+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:50:59.581 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:50:59 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:50:59.093+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:50:59.632 INFO:journalctl@ceph.osd.4.smithi026.stdout:Nov 16 16:50:59 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-4[110709]: 2023-11-16T16:50:59.350+0000 7f841d22c700 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:33.870388+0000 front 2023-11-16T16:50:33.870430+0000 (oldest deadline 2023-11-16T16:50:59.170543+0000) 2023-11-16T16:50:59.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:50:59 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:50:59.331+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:50:59.682 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:50:59 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:50:59.388+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:26.032803+0000 front 2023-11-16T16:50:26.032692+0000 (oldest deadline 2023-11-16T16:50:49.532862+0000) 2023-11-16T16:51:00.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:50:59 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:50:59.631+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:00.080 INFO:journalctl@ceph.osd.5.smithi053.stdout:Nov 16 16:50:59 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-5[111878]: 2023-11-16T16:50:59.632+0000 7fdbc5967700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.816257+0000 front 2023-11-16T16:50:27.816098+0000 (oldest deadline 2023-11-16T16:50:53.116098+0000) 2023-11-16T16:51:00.580 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:00 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:00.125+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:00.581 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:00 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:00.125+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:00.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:00 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:00.328+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:51:00.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:00 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:00.328+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.372509+0000 front 2023-11-16T16:50:34.372445+0000 (oldest deadline 2023-11-16T16:50:59.672610+0000) 2023-11-16T16:51:00.682 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:00 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:00.358+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:26.032803+0000 front 2023-11-16T16:50:26.032692+0000 (oldest deadline 2023-11-16T16:50:49.532862+0000) 2023-11-16T16:51:00.683 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:00 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:00.358+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.635668+0000 front 2023-11-16T16:50:34.635569+0000 (oldest deadline 2023-11-16T16:50:59.935677+0000) 2023-11-16T16:51:00.758 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:00 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:00.624+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:00.758 INFO:journalctl@ceph.osd.4.smithi026.stdout:Nov 16 16:51:00 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-4[110709]: 2023-11-16T16:51:00.379+0000 7f841d22c700 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:33.870388+0000 front 2023-11-16T16:50:33.870430+0000 (oldest deadline 2023-11-16T16:50:59.170543+0000) 2023-11-16T16:51:01.080 INFO:journalctl@ceph.osd.5.smithi053.stdout:Nov 16 16:51:00 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-5[111878]: 2023-11-16T16:51:00.619+0000 7fdbc5967700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.816257+0000 front 2023-11-16T16:50:27.816098+0000 (oldest deadline 2023-11-16T16:50:53.116098+0000) 2023-11-16T16:51:01.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:01 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:01.271+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:36.811405+0000 front 2023-11-16T16:50:36.811513+0000 (oldest deadline 2023-11-16T16:51:00.911616+0000) 2023-11-16T16:51:01.581 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:01 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:01.084+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:01.581 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:01 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:01.084+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:01.663 INFO:journalctl@ceph.osd.4.smithi026.stdout:Nov 16 16:51:01 smithi026 bash[131887]: ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-4 2023-11-16T16:51:01.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:01 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:01.356+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:51:01.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:01 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:01.356+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.372509+0000 front 2023-11-16T16:50:34.372445+0000 (oldest deadline 2023-11-16T16:50:59.672610+0000) 2023-11-16T16:51:01.682 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:01 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:01.363+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:26.032803+0000 front 2023-11-16T16:50:26.032692+0000 (oldest deadline 2023-11-16T16:50:49.532862+0000) 2023-11-16T16:51:01.682 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:01 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:01.363+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.635668+0000 front 2023-11-16T16:50:34.635569+0000 (oldest deadline 2023-11-16T16:50:59.935677+0000) 2023-11-16T16:51:02.008 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:01 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:01.661+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:37.550795+0000 front 2023-11-16T16:50:37.550812+0000 (oldest deadline 2023-11-16T16:51:01.650942+0000) 2023-11-16T16:51:02.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:01 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:01.669+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:02.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:01 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:01.669+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:02.044 INFO:journalctl@ceph.osd.5.smithi053.stdout:Nov 16 16:51:01 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-5[111878]: 2023-11-16T16:51:01.626+0000 7fdbc5967700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.816257+0000 front 2023-11-16T16:50:27.816098+0000 (oldest deadline 2023-11-16T16:50:53.116098+0000) 2023-11-16T16:51:02.044 INFO:journalctl@ceph.osd.5.smithi053.stdout:Nov 16 16:51:01 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-5[111878]: 2023-11-16T16:51:01.626+0000 7fdbc5967700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.417376+0000 front 2023-11-16T16:50:35.417212+0000 (oldest deadline 2023-11-16T16:51:00.717232+0000) 2023-11-16T16:51:02.330 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:02 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:02.260+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:36.811405+0000 front 2023-11-16T16:50:36.811513+0000 (oldest deadline 2023-11-16T16:51:00.911616+0000) 2023-11-16T16:51:02.330 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:02 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:02.043+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:02.331 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:02 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:02.043+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:02.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:02 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:02.338+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:51:02.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:02 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:02.338+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.372509+0000 front 2023-11-16T16:50:34.372445+0000 (oldest deadline 2023-11-16T16:50:59.672610+0000) 2023-11-16T16:51:02.682 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:02 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:02.402+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:26.032803+0000 front 2023-11-16T16:50:26.032692+0000 (oldest deadline 2023-11-16T16:50:49.532862+0000) 2023-11-16T16:51:02.683 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:02 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:02.402+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.635668+0000 front 2023-11-16T16:50:34.635569+0000 (oldest deadline 2023-11-16T16:50:59.935677+0000) 2023-11-16T16:51:02.713 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:02 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:02.633+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:37.550795+0000 front 2023-11-16T16:50:37.550812+0000 (oldest deadline 2023-11-16T16:51:01.650942+0000) 2023-11-16T16:51:02.745 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@osd.4.service' 2023-11-16T16:51:02.786 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:51:02.787 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-11-16T16:51:02.787 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-11-16T16:51:02.787 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@osd.5 2023-11-16T16:51:02.997 INFO:journalctl@ceph.osd.5.smithi053.stdout:Nov 16 16:51:02 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-5[111878]: 2023-11-16T16:51:02.647+0000 7fdbc5967700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.816257+0000 front 2023-11-16T16:50:27.816098+0000 (oldest deadline 2023-11-16T16:50:53.116098+0000) 2023-11-16T16:51:02.997 INFO:journalctl@ceph.osd.5.smithi053.stdout:Nov 16 16:51:02 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-5[111878]: 2023-11-16T16:51:02.647+0000 7fdbc5967700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.417376+0000 front 2023-11-16T16:50:35.417212+0000 (oldest deadline 2023-11-16T16:51:00.717232+0000) 2023-11-16T16:51:02.998 INFO:journalctl@ceph.osd.5.smithi053.stdout:Nov 16 16:51:02 smithi053 systemd[1]: Stopping Ceph osd.5 for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:51:03.007 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:02 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:02.712+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:03.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:02 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:02.712+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:03.330 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:03 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:03.232+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:36.811405+0000 front 2023-11-16T16:50:36.811513+0000 (oldest deadline 2023-11-16T16:51:00.911616+0000) 2023-11-16T16:51:03.331 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:02 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:02.996+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:03.331 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:02 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:02.996+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:03.331 INFO:journalctl@ceph.osd.5.smithi053.stdout:Nov 16 16:51:03 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-5[111878]: 2023-11-16T16:51:03.159+0000 7fdbc9371700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:51:03.331 INFO:journalctl@ceph.osd.5.smithi053.stdout:Nov 16 16:51:03 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-5[111878]: 2023-11-16T16:51:03.159+0000 7fdbc9371700 -1 osd.5 99 *** Got signal Terminated *** 2023-11-16T16:51:03.331 INFO:journalctl@ceph.osd.5.smithi053.stdout:Nov 16 16:51:03 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-5[111878]: 2023-11-16T16:51:03.159+0000 7fdbc9371700 -1 osd.5 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-16T16:51:03.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:03 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:03.312+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:51:03.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:03 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:03.312+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.372509+0000 front 2023-11-16T16:50:34.372445+0000 (oldest deadline 2023-11-16T16:50:59.672610+0000) 2023-11-16T16:51:03.682 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:03 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:03.408+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:26.032803+0000 front 2023-11-16T16:50:26.032692+0000 (oldest deadline 2023-11-16T16:50:49.532862+0000) 2023-11-16T16:51:03.682 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:03 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:03.408+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.635668+0000 front 2023-11-16T16:50:34.635569+0000 (oldest deadline 2023-11-16T16:50:59.935677+0000) 2023-11-16T16:51:04.008 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:03 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:03.676+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:37.550795+0000 front 2023-11-16T16:50:37.550812+0000 (oldest deadline 2023-11-16T16:51:01.650942+0000) 2023-11-16T16:51:04.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:03 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:03.726+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:04.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:03 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:03.726+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:04.025 INFO:journalctl@ceph.osd.5.smithi053.stdout:Nov 16 16:51:03 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-5[111878]: 2023-11-16T16:51:03.664+0000 7fdbc5967700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.816257+0000 front 2023-11-16T16:50:27.816098+0000 (oldest deadline 2023-11-16T16:50:53.116098+0000) 2023-11-16T16:51:04.025 INFO:journalctl@ceph.osd.5.smithi053.stdout:Nov 16 16:51:03 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-5[111878]: 2023-11-16T16:51:03.664+0000 7fdbc5967700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.417376+0000 front 2023-11-16T16:50:35.417212+0000 (oldest deadline 2023-11-16T16:51:00.717232+0000) 2023-11-16T16:51:04.283 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:04 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:04.024+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:04.283 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:04 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:04.024+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:04.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:04 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:04.282+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:36.811405+0000 front 2023-11-16T16:50:36.811513+0000 (oldest deadline 2023-11-16T16:51:00.911616+0000) 2023-11-16T16:51:04.682 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:04 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:04.398+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:26.032803+0000 front 2023-11-16T16:50:26.032692+0000 (oldest deadline 2023-11-16T16:50:49.532862+0000) 2023-11-16T16:51:04.682 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:04 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:04.398+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.635668+0000 front 2023-11-16T16:50:34.635569+0000 (oldest deadline 2023-11-16T16:50:59.935677+0000) 2023-11-16T16:51:04.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:04 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:04.303+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:51:04.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:04 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:04.303+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.372509+0000 front 2023-11-16T16:50:34.372445+0000 (oldest deadline 2023-11-16T16:50:59.672610+0000) 2023-11-16T16:51:05.008 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:04 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:04.723+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:37.550795+0000 front 2023-11-16T16:50:37.550812+0000 (oldest deadline 2023-11-16T16:51:01.650942+0000) 2023-11-16T16:51:05.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:04 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:04.728+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:05.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:04 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:04.728+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:05.068 INFO:journalctl@ceph.osd.5.smithi053.stdout:Nov 16 16:51:04 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-5[111878]: 2023-11-16T16:51:04.712+0000 7fdbc5967700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.816257+0000 front 2023-11-16T16:50:27.816098+0000 (oldest deadline 2023-11-16T16:50:53.116098+0000) 2023-11-16T16:51:05.068 INFO:journalctl@ceph.osd.5.smithi053.stdout:Nov 16 16:51:04 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-5[111878]: 2023-11-16T16:51:04.712+0000 7fdbc5967700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.417376+0000 front 2023-11-16T16:50:35.417212+0000 (oldest deadline 2023-11-16T16:51:00.717232+0000) 2023-11-16T16:51:05.330 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:05 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:05.066+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:05.330 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:05 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:05.066+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:05.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:05 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:05.329+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:36.811405+0000 front 2023-11-16T16:50:36.811513+0000 (oldest deadline 2023-11-16T16:51:00.911616+0000) 2023-11-16T16:51:05.682 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:05 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:05.375+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:26.032803+0000 front 2023-11-16T16:50:26.032692+0000 (oldest deadline 2023-11-16T16:50:49.532862+0000) 2023-11-16T16:51:05.682 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:05 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:05.375+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.635668+0000 front 2023-11-16T16:50:34.635569+0000 (oldest deadline 2023-11-16T16:50:59.935677+0000) 2023-11-16T16:51:05.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:05 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:05.300+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:51:05.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:05 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:05.300+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.372509+0000 front 2023-11-16T16:50:34.372445+0000 (oldest deadline 2023-11-16T16:50:59.672610+0000) 2023-11-16T16:51:06.008 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:05 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:05.723+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:37.550795+0000 front 2023-11-16T16:50:37.550812+0000 (oldest deadline 2023-11-16T16:51:01.650942+0000) 2023-11-16T16:51:06.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:05 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:05.746+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:06.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:05 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:05.746+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:06.042 INFO:journalctl@ceph.osd.5.smithi053.stdout:Nov 16 16:51:05 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-5[111878]: 2023-11-16T16:51:05.741+0000 7fdbc5967700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.816257+0000 front 2023-11-16T16:50:27.816098+0000 (oldest deadline 2023-11-16T16:50:53.116098+0000) 2023-11-16T16:51:06.042 INFO:journalctl@ceph.osd.5.smithi053.stdout:Nov 16 16:51:05 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-5[111878]: 2023-11-16T16:51:05.741+0000 7fdbc5967700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.417376+0000 front 2023-11-16T16:50:35.417212+0000 (oldest deadline 2023-11-16T16:51:00.717232+0000) 2023-11-16T16:51:06.330 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:06 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:06.041+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:06.330 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:06 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:06.041+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:06.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:06 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:06.329+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:36.811405+0000 front 2023-11-16T16:50:36.811513+0000 (oldest deadline 2023-11-16T16:51:00.911616+0000) 2023-11-16T16:51:06.682 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:06 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:06.423+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:26.032803+0000 front 2023-11-16T16:50:26.032692+0000 (oldest deadline 2023-11-16T16:50:49.532862+0000) 2023-11-16T16:51:06.682 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:06 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:06.423+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.635668+0000 front 2023-11-16T16:50:34.635569+0000 (oldest deadline 2023-11-16T16:50:59.935677+0000) 2023-11-16T16:51:06.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:06 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:06.304+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:51:06.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:06 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:06.304+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.372509+0000 front 2023-11-16T16:50:34.372445+0000 (oldest deadline 2023-11-16T16:50:59.672610+0000) 2023-11-16T16:51:07.000 INFO:journalctl@ceph.osd.5.smithi053.stdout:Nov 16 16:51:06 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-5[111878]: 2023-11-16T16:51:06.748+0000 7fdbc5967700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.816257+0000 front 2023-11-16T16:50:27.816098+0000 (oldest deadline 2023-11-16T16:50:53.116098+0000) 2023-11-16T16:51:07.000 INFO:journalctl@ceph.osd.5.smithi053.stdout:Nov 16 16:51:06 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-5[111878]: 2023-11-16T16:51:06.748+0000 7fdbc5967700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.417376+0000 front 2023-11-16T16:50:35.417212+0000 (oldest deadline 2023-11-16T16:51:00.717232+0000) 2023-11-16T16:51:07.000 INFO:journalctl@ceph.osd.5.smithi053.stdout:Nov 16 16:51:06 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-5[111878]: 2023-11-16T16:51:06.748+0000 7fdbc5967700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:44.218213+0000 front 2023-11-16T16:50:44.218235+0000 (oldest deadline 2023-11-16T16:51:06.518326+0000) 2023-11-16T16:51:07.008 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:06 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:06.756+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:37.550795+0000 front 2023-11-16T16:50:37.550812+0000 (oldest deadline 2023-11-16T16:51:01.650942+0000) 2023-11-16T16:51:07.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:06 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:06.736+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:07.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:06 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:06.736+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:07.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:06 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:06.736+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:07.329 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:06 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:06.999+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:07.330 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:06 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:06.999+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:07.330 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:06 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:06.999+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:07.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:07 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:07.328+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:36.811405+0000 front 2023-11-16T16:50:36.811513+0000 (oldest deadline 2023-11-16T16:51:00.911616+0000) 2023-11-16T16:51:07.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:07 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:07.352+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:51:07.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:07 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:07.352+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.372509+0000 front 2023-11-16T16:50:34.372445+0000 (oldest deadline 2023-11-16T16:50:59.672610+0000) 2023-11-16T16:51:07.682 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:07 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:07.459+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:26.032803+0000 front 2023-11-16T16:50:26.032692+0000 (oldest deadline 2023-11-16T16:50:49.532862+0000) 2023-11-16T16:51:07.682 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:07 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:07.459+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.635668+0000 front 2023-11-16T16:50:34.635569+0000 (oldest deadline 2023-11-16T16:50:59.935677+0000) 2023-11-16T16:51:07.682 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:07 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:07.459+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:45.137710+0000 front 2023-11-16T16:50:45.137733+0000 (oldest deadline 2023-11-16T16:51:07.437843+0000) 2023-11-16T16:51:08.008 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:07 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:07.726+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:37.550795+0000 front 2023-11-16T16:50:37.550812+0000 (oldest deadline 2023-11-16T16:51:01.650942+0000) 2023-11-16T16:51:08.008 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:07 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:07.726+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:41.651596+0000 front 2023-11-16T16:50:41.651409+0000 (oldest deadline 2023-11-16T16:51:07.551550+0000) 2023-11-16T16:51:08.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:07 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:07.720+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:08.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:07 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:07.720+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:08.009 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:07 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:07.720+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:08.080 INFO:journalctl@ceph.osd.5.smithi053.stdout:Nov 16 16:51:07 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-5[111878]: 2023-11-16T16:51:07.791+0000 7fdbc5967700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.816257+0000 front 2023-11-16T16:50:27.816098+0000 (oldest deadline 2023-11-16T16:50:53.116098+0000) 2023-11-16T16:51:08.080 INFO:journalctl@ceph.osd.5.smithi053.stdout:Nov 16 16:51:07 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-5[111878]: 2023-11-16T16:51:07.791+0000 7fdbc5967700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.417376+0000 front 2023-11-16T16:50:35.417212+0000 (oldest deadline 2023-11-16T16:51:00.717232+0000) 2023-11-16T16:51:08.081 INFO:journalctl@ceph.osd.5.smithi053.stdout:Nov 16 16:51:07 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-5[111878]: 2023-11-16T16:51:07.791+0000 7fdbc5967700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:44.218213+0000 front 2023-11-16T16:50:44.218235+0000 (oldest deadline 2023-11-16T16:51:06.518326+0000) 2023-11-16T16:51:08.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:08 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:08.027+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:08.082 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:08 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:08.027+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:08.082 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:08 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:08.027+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:08.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:08 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:08.313+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:36.811405+0000 front 2023-11-16T16:50:36.811513+0000 (oldest deadline 2023-11-16T16:51:00.911616+0000) 2023-11-16T16:51:08.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:08 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:08.313+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.212686+0000 front 2023-11-16T16:50:43.212712+0000 (oldest deadline 2023-11-16T16:51:07.913040+0000) 2023-11-16T16:51:08.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:08 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:08.314+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:51:08.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:08 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:08.314+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.372509+0000 front 2023-11-16T16:50:34.372445+0000 (oldest deadline 2023-11-16T16:50:59.672610+0000) 2023-11-16T16:51:08.682 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:08 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:08.494+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:26.032803+0000 front 2023-11-16T16:50:26.032692+0000 (oldest deadline 2023-11-16T16:50:49.532862+0000) 2023-11-16T16:51:08.683 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:08 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:08.494+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.635668+0000 front 2023-11-16T16:50:34.635569+0000 (oldest deadline 2023-11-16T16:50:59.935677+0000) 2023-11-16T16:51:08.683 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:08 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:08.494+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:45.137710+0000 front 2023-11-16T16:50:45.137733+0000 (oldest deadline 2023-11-16T16:51:07.437843+0000) 2023-11-16T16:51:09.008 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:08 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:08.681+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:37.550795+0000 front 2023-11-16T16:50:37.550812+0000 (oldest deadline 2023-11-16T16:51:01.650942+0000) 2023-11-16T16:51:09.008 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:08 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:08.681+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:41.651596+0000 front 2023-11-16T16:50:41.651409+0000 (oldest deadline 2023-11-16T16:51:07.551550+0000) 2023-11-16T16:51:09.009 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:08 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:08.707+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:09.009 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:08 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:08.707+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:09.010 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:08 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:08.707+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:09.035 INFO:journalctl@ceph.osd.5.smithi053.stdout:Nov 16 16:51:08 smithi053 bash[133359]: ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-5 2023-11-16T16:51:09.307 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:09 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:09.044+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:09.308 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:09 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:09.044+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:09.308 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:09 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:09.044+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:09.581 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:09 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:09.306+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:36.811405+0000 front 2023-11-16T16:50:36.811513+0000 (oldest deadline 2023-11-16T16:51:00.911616+0000) 2023-11-16T16:51:09.581 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:09 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:09.306+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.212686+0000 front 2023-11-16T16:50:43.212712+0000 (oldest deadline 2023-11-16T16:51:07.913040+0000) 2023-11-16T16:51:09.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:09 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:09.331+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:51:09.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:09 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:09.332+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.372509+0000 front 2023-11-16T16:50:34.372445+0000 (oldest deadline 2023-11-16T16:50:59.672610+0000) 2023-11-16T16:51:09.682 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:09 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:09.513+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:26.032803+0000 front 2023-11-16T16:50:26.032692+0000 (oldest deadline 2023-11-16T16:50:49.532862+0000) 2023-11-16T16:51:09.682 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:09 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:09.513+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.635668+0000 front 2023-11-16T16:50:34.635569+0000 (oldest deadline 2023-11-16T16:50:59.935677+0000) 2023-11-16T16:51:09.683 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:09 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:09.513+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:45.137710+0000 front 2023-11-16T16:50:45.137733+0000 (oldest deadline 2023-11-16T16:51:07.437843+0000) 2023-11-16T16:51:10.008 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:09 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:09.658+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:37.550795+0000 front 2023-11-16T16:50:37.550812+0000 (oldest deadline 2023-11-16T16:51:01.650942+0000) 2023-11-16T16:51:10.009 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:09 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:09.658+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:41.651596+0000 front 2023-11-16T16:50:41.651409+0000 (oldest deadline 2023-11-16T16:51:07.551550+0000) 2023-11-16T16:51:10.009 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:09 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:09.668+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:10.009 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:09 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:09.668+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:10.009 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:09 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:09.668+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:10.267 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:10 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:10.002+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:10.267 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:10 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:10.002+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:10.268 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:10 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:10.002+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:10.309 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@osd.5.service' 2023-11-16T16:51:10.350 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:51:10.351 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-11-16T16:51:10.351 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-11-16T16:51:10.351 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@osd.6 2023-11-16T16:51:10.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:10 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:10.266+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:36.811405+0000 front 2023-11-16T16:50:36.811513+0000 (oldest deadline 2023-11-16T16:51:00.911616+0000) 2023-11-16T16:51:10.581 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:10 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:10.266+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.212686+0000 front 2023-11-16T16:50:43.212712+0000 (oldest deadline 2023-11-16T16:51:07.913040+0000) 2023-11-16T16:51:10.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:10 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:10.335+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:51:10.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:10 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:10.335+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.372509+0000 front 2023-11-16T16:50:34.372445+0000 (oldest deadline 2023-11-16T16:50:59.672610+0000) 2023-11-16T16:51:10.683 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:10 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:10.335+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:44.974145+0000 front 2023-11-16T16:50:44.973914+0000 (oldest deadline 2023-11-16T16:51:09.673929+0000) 2023-11-16T16:51:10.683 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:10 smithi001 systemd[1]: Stopping Ceph osd.6 for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:51:10.683 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:10 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:10.528+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:26.032803+0000 front 2023-11-16T16:50:26.032692+0000 (oldest deadline 2023-11-16T16:50:49.532862+0000) 2023-11-16T16:51:10.684 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:10 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:10.528+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.635668+0000 front 2023-11-16T16:50:34.635569+0000 (oldest deadline 2023-11-16T16:50:59.935677+0000) 2023-11-16T16:51:10.684 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:10 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:10.528+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:45.137710+0000 front 2023-11-16T16:50:45.137733+0000 (oldest deadline 2023-11-16T16:51:07.437843+0000) 2023-11-16T16:51:11.008 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:10 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:10.630+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:37.550795+0000 front 2023-11-16T16:50:37.550812+0000 (oldest deadline 2023-11-16T16:51:01.650942+0000) 2023-11-16T16:51:11.008 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:10 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:10.630+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:41.651596+0000 front 2023-11-16T16:50:41.651409+0000 (oldest deadline 2023-11-16T16:51:07.551550+0000) 2023-11-16T16:51:11.009 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:10 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:10.664+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:11.009 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:10 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:10.664+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:11.009 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:10 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:10.664+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:11.181 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:10 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:10.709+0000 7fc3e37a3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:51:11.182 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:10 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:10.709+0000 7fc3e37a3700 -1 osd.6 99 *** Got signal Terminated *** 2023-11-16T16:51:11.182 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:10 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:10.709+0000 7fc3e37a3700 -1 osd.6 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-16T16:51:11.263 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:10 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:10.985+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:11.263 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:10 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:10.985+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:11.263 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:10 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:10.985+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:11.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:11 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:11.262+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:36.811405+0000 front 2023-11-16T16:50:36.811513+0000 (oldest deadline 2023-11-16T16:51:00.911616+0000) 2023-11-16T16:51:11.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:11 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:11.262+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.212686+0000 front 2023-11-16T16:50:43.212712+0000 (oldest deadline 2023-11-16T16:51:07.913040+0000) 2023-11-16T16:51:11.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:11 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:11.292+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:51:11.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:11 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:11.292+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.372509+0000 front 2023-11-16T16:50:34.372445+0000 (oldest deadline 2023-11-16T16:50:59.672610+0000) 2023-11-16T16:51:11.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:11 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:11.292+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:44.974145+0000 front 2023-11-16T16:50:44.973914+0000 (oldest deadline 2023-11-16T16:51:09.673929+0000) 2023-11-16T16:51:11.682 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:11 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:11.489+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:26.032803+0000 front 2023-11-16T16:50:26.032692+0000 (oldest deadline 2023-11-16T16:50:49.532862+0000) 2023-11-16T16:51:11.682 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:11 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:11.489+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.635668+0000 front 2023-11-16T16:50:34.635569+0000 (oldest deadline 2023-11-16T16:50:59.935677+0000) 2023-11-16T16:51:11.683 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:11 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:11.489+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:45.137710+0000 front 2023-11-16T16:50:45.137733+0000 (oldest deadline 2023-11-16T16:51:07.437843+0000) 2023-11-16T16:51:12.008 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:11 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:11.590+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:37.550795+0000 front 2023-11-16T16:50:37.550812+0000 (oldest deadline 2023-11-16T16:51:01.650942+0000) 2023-11-16T16:51:12.008 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:11 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:11.590+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:41.651596+0000 front 2023-11-16T16:50:41.651409+0000 (oldest deadline 2023-11-16T16:51:07.551550+0000) 2023-11-16T16:51:12.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:11 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:11.710+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:12.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:11 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:11.710+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:12.009 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:11 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:11.710+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:12.272 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:12 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:12.003+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:12.283 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:12 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:12.003+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:12.283 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:12 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:12.003+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:12.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:12 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:12.271+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:36.811405+0000 front 2023-11-16T16:50:36.811513+0000 (oldest deadline 2023-11-16T16:51:00.911616+0000) 2023-11-16T16:51:12.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:12 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:12.271+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.212686+0000 front 2023-11-16T16:50:43.212712+0000 (oldest deadline 2023-11-16T16:51:07.913040+0000) 2023-11-16T16:51:12.682 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:12 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:12.511+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:26.032803+0000 front 2023-11-16T16:50:26.032692+0000 (oldest deadline 2023-11-16T16:50:49.532862+0000) 2023-11-16T16:51:12.682 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:12 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:12.511+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.635668+0000 front 2023-11-16T16:50:34.635569+0000 (oldest deadline 2023-11-16T16:50:59.935677+0000) 2023-11-16T16:51:12.682 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:12 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:12.511+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:45.137710+0000 front 2023-11-16T16:50:45.137733+0000 (oldest deadline 2023-11-16T16:51:07.437843+0000) 2023-11-16T16:51:12.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:12 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:12.290+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:51:12.683 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:12 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:12.290+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.372509+0000 front 2023-11-16T16:50:34.372445+0000 (oldest deadline 2023-11-16T16:50:59.672610+0000) 2023-11-16T16:51:12.683 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:12 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:12.290+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:44.974145+0000 front 2023-11-16T16:50:44.973914+0000 (oldest deadline 2023-11-16T16:51:09.673929+0000) 2023-11-16T16:51:13.008 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:12 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:12.588+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:37.550795+0000 front 2023-11-16T16:50:37.550812+0000 (oldest deadline 2023-11-16T16:51:01.650942+0000) 2023-11-16T16:51:13.008 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:12 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:12.588+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:41.651596+0000 front 2023-11-16T16:50:41.651409+0000 (oldest deadline 2023-11-16T16:51:07.551550+0000) 2023-11-16T16:51:13.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:12 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:12.707+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:13.009 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:12 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:12.707+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:13.009 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:12 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:12.707+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:13.250 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:12 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:12.998+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:13.250 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:12 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:12.998+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:13.250 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:12 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:12.998+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:13.521 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:13 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:13.269+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:51:13.521 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:13 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:13.269+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.372509+0000 front 2023-11-16T16:50:34.372445+0000 (oldest deadline 2023-11-16T16:50:59.672610+0000) 2023-11-16T16:51:13.522 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:13 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:13.269+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:44.974145+0000 front 2023-11-16T16:50:44.973914+0000 (oldest deadline 2023-11-16T16:51:09.673929+0000) 2023-11-16T16:51:13.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:13 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:13.249+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:36.811405+0000 front 2023-11-16T16:50:36.811513+0000 (oldest deadline 2023-11-16T16:51:00.911616+0000) 2023-11-16T16:51:13.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:13 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:13.249+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.212686+0000 front 2023-11-16T16:50:43.212712+0000 (oldest deadline 2023-11-16T16:51:07.913040+0000) 2023-11-16T16:51:13.932 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:13 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:13.521+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:26.032803+0000 front 2023-11-16T16:50:26.032692+0000 (oldest deadline 2023-11-16T16:50:49.532862+0000) 2023-11-16T16:51:13.932 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:13 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:13.521+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.635668+0000 front 2023-11-16T16:50:34.635569+0000 (oldest deadline 2023-11-16T16:50:59.935677+0000) 2023-11-16T16:51:13.932 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:13 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:13.521+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:45.137710+0000 front 2023-11-16T16:50:45.137733+0000 (oldest deadline 2023-11-16T16:51:07.437843+0000) 2023-11-16T16:51:14.008 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:13 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:13.635+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:37.550795+0000 front 2023-11-16T16:50:37.550812+0000 (oldest deadline 2023-11-16T16:51:01.650942+0000) 2023-11-16T16:51:14.008 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:13 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:13.635+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:41.651596+0000 front 2023-11-16T16:50:41.651409+0000 (oldest deadline 2023-11-16T16:51:07.551550+0000) 2023-11-16T16:51:14.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:13 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:13.698+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:14.009 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:13 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:13.698+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:14.009 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:13 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:13.698+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:14.009 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:13 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:13.698+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:51.414772+0000 front 2023-11-16T16:50:51.414692+0000 (oldest deadline 2023-11-16T16:51:13.615360+0000) 2023-11-16T16:51:14.330 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:14 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:14.201+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:36.811405+0000 front 2023-11-16T16:50:36.811513+0000 (oldest deadline 2023-11-16T16:51:00.911616+0000) 2023-11-16T16:51:14.330 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:14 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:14.201+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.212686+0000 front 2023-11-16T16:50:43.212712+0000 (oldest deadline 2023-11-16T16:51:07.913040+0000) 2023-11-16T16:51:14.331 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:14 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:14.201+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:47.913771+0000 front 2023-11-16T16:50:47.913677+0000 (oldest deadline 2023-11-16T16:51:13.714459+0000) 2023-11-16T16:51:14.331 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:14 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:14.033+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:14.331 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:14 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:14.033+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:14.331 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:14 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:14.033+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:14.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:14 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:14.294+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:51:14.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:14 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:14.294+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.372509+0000 front 2023-11-16T16:50:34.372445+0000 (oldest deadline 2023-11-16T16:50:59.672610+0000) 2023-11-16T16:51:14.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:14 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:14.294+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:44.974145+0000 front 2023-11-16T16:50:44.973914+0000 (oldest deadline 2023-11-16T16:51:09.673929+0000) 2023-11-16T16:51:14.683 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:14 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:14.482+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:26.032803+0000 front 2023-11-16T16:50:26.032692+0000 (oldest deadline 2023-11-16T16:50:49.532862+0000) 2023-11-16T16:51:14.683 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:14 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:14.482+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.635668+0000 front 2023-11-16T16:50:34.635569+0000 (oldest deadline 2023-11-16T16:50:59.935677+0000) 2023-11-16T16:51:14.683 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:14 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:14.482+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:45.137710+0000 front 2023-11-16T16:50:45.137733+0000 (oldest deadline 2023-11-16T16:51:07.437843+0000) 2023-11-16T16:51:15.007 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:14 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:14.651+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:37.550795+0000 front 2023-11-16T16:50:37.550812+0000 (oldest deadline 2023-11-16T16:51:01.650942+0000) 2023-11-16T16:51:15.008 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:14 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:14.651+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:41.651596+0000 front 2023-11-16T16:50:41.651409+0000 (oldest deadline 2023-11-16T16:51:07.551550+0000) 2023-11-16T16:51:15.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:14 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:14.656+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:15.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:14 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:14.656+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:15.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:14 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:14.656+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:15.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:14 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:14.656+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:51.414772+0000 front 2023-11-16T16:50:51.414692+0000 (oldest deadline 2023-11-16T16:51:13.615360+0000) 2023-11-16T16:51:15.330 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:15 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:15.026+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:15.330 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:15 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:15.026+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:15.331 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:15 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:15.026+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:15.331 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:15 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:15.219+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:36.811405+0000 front 2023-11-16T16:50:36.811513+0000 (oldest deadline 2023-11-16T16:51:00.911616+0000) 2023-11-16T16:51:15.331 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:15 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:15.219+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.212686+0000 front 2023-11-16T16:50:43.212712+0000 (oldest deadline 2023-11-16T16:51:07.913040+0000) 2023-11-16T16:51:15.331 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:15 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:15.219+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:47.913771+0000 front 2023-11-16T16:50:47.913677+0000 (oldest deadline 2023-11-16T16:51:13.714459+0000) 2023-11-16T16:51:15.682 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:15 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:15.497+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:26.032803+0000 front 2023-11-16T16:50:26.032692+0000 (oldest deadline 2023-11-16T16:50:49.532862+0000) 2023-11-16T16:51:15.682 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:15 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:15.497+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.635668+0000 front 2023-11-16T16:50:34.635569+0000 (oldest deadline 2023-11-16T16:50:59.935677+0000) 2023-11-16T16:51:15.682 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:15 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6[122686]: 2023-11-16T16:51:15.497+0000 7fc3ddd95700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:45.137710+0000 front 2023-11-16T16:50:45.137733+0000 (oldest deadline 2023-11-16T16:51:07.437843+0000) 2023-11-16T16:51:15.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:15 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:15.279+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:51:15.683 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:15 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:15.279+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.372509+0000 front 2023-11-16T16:50:34.372445+0000 (oldest deadline 2023-11-16T16:50:59.672610+0000) 2023-11-16T16:51:15.683 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:15 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:15.279+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:44.974145+0000 front 2023-11-16T16:50:44.973914+0000 (oldest deadline 2023-11-16T16:51:09.673929+0000) 2023-11-16T16:51:15.683 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:15 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:15.279+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:49.674280+0000 front 2023-11-16T16:50:49.674295+0000 (oldest deadline 2023-11-16T16:51:14.374459+0000) 2023-11-16T16:51:16.008 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:15 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:15.644+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:37.550795+0000 front 2023-11-16T16:50:37.550812+0000 (oldest deadline 2023-11-16T16:51:01.650942+0000) 2023-11-16T16:51:16.008 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:15 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:15.644+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:41.651596+0000 front 2023-11-16T16:50:41.651409+0000 (oldest deadline 2023-11-16T16:51:07.551550+0000) 2023-11-16T16:51:16.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:15 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:15.607+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:16.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:15 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:15.607+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:16.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:15 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:15.607+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:16.009 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:15 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:15.607+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:51.414772+0000 front 2023-11-16T16:50:51.414692+0000 (oldest deadline 2023-11-16T16:51:13.615360+0000) 2023-11-16T16:51:16.330 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:16 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:16.242+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:36.811405+0000 front 2023-11-16T16:50:36.811513+0000 (oldest deadline 2023-11-16T16:51:00.911616+0000) 2023-11-16T16:51:16.331 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:16 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:16.242+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.212686+0000 front 2023-11-16T16:50:43.212712+0000 (oldest deadline 2023-11-16T16:51:07.913040+0000) 2023-11-16T16:51:16.331 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:16 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:16.242+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:47.913771+0000 front 2023-11-16T16:50:47.913677+0000 (oldest deadline 2023-11-16T16:51:13.714459+0000) 2023-11-16T16:51:16.331 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:16 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:16.023+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:16.331 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:16 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:16.023+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:16.331 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:16 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:16.023+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:16.432 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:16 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:16.279+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:51:16.432 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:16 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:16.279+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.372509+0000 front 2023-11-16T16:50:34.372445+0000 (oldest deadline 2023-11-16T16:50:59.672610+0000) 2023-11-16T16:51:16.432 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:16 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:16.279+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:44.974145+0000 front 2023-11-16T16:50:44.973914+0000 (oldest deadline 2023-11-16T16:51:09.673929+0000) 2023-11-16T16:51:16.432 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:16 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:16.279+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:49.674280+0000 front 2023-11-16T16:50:49.674295+0000 (oldest deadline 2023-11-16T16:51:14.374459+0000) 2023-11-16T16:51:16.932 INFO:journalctl@ceph.osd.6.smithi001.stdout:Nov 16 16:51:16 smithi001 bash[263245]: ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-6 2023-11-16T16:51:17.008 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:16 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:16.641+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:37.550795+0000 front 2023-11-16T16:50:37.550812+0000 (oldest deadline 2023-11-16T16:51:01.650942+0000) 2023-11-16T16:51:17.008 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:16 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:16.641+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:41.651596+0000 front 2023-11-16T16:50:41.651409+0000 (oldest deadline 2023-11-16T16:51:07.551550+0000) 2023-11-16T16:51:17.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:16 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:16.622+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:17.009 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:16 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:16.622+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:17.009 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:16 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:16.622+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:17.009 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:16 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:16.622+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:51.414772+0000 front 2023-11-16T16:50:51.414692+0000 (oldest deadline 2023-11-16T16:51:13.615360+0000) 2023-11-16T16:51:17.256 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:16 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:16.985+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:17.256 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:16 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:16.985+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:17.257 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:16 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:16.985+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:17.432 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:17 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:17.316+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:51:17.432 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:17 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:17.316+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.372509+0000 front 2023-11-16T16:50:34.372445+0000 (oldest deadline 2023-11-16T16:50:59.672610+0000) 2023-11-16T16:51:17.432 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:17 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:17.316+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:44.974145+0000 front 2023-11-16T16:50:44.973914+0000 (oldest deadline 2023-11-16T16:51:09.673929+0000) 2023-11-16T16:51:17.432 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:17 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:17.316+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:49.674280+0000 front 2023-11-16T16:50:49.674295+0000 (oldest deadline 2023-11-16T16:51:14.374459+0000) 2023-11-16T16:51:17.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:17 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:17.255+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:36.811405+0000 front 2023-11-16T16:50:36.811513+0000 (oldest deadline 2023-11-16T16:51:00.911616+0000) 2023-11-16T16:51:17.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:17 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:17.255+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.212686+0000 front 2023-11-16T16:50:43.212712+0000 (oldest deadline 2023-11-16T16:51:07.913040+0000) 2023-11-16T16:51:17.581 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:17 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:17.255+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:47.913771+0000 front 2023-11-16T16:50:47.913677+0000 (oldest deadline 2023-11-16T16:51:13.714459+0000) 2023-11-16T16:51:17.867 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@osd.6.service' 2023-11-16T16:51:17.908 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:51:17.909 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-11-16T16:51:17.909 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-11-16T16:51:17.909 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@osd.7 2023-11-16T16:51:17.940 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:17 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:17.670+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:17.940 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:17 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:17.670+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:17.940 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:17 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:17.670+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:17.941 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:17 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:17.670+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:51.414772+0000 front 2023-11-16T16:50:51.414692+0000 (oldest deadline 2023-11-16T16:51:13.615360+0000) 2023-11-16T16:51:17.941 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:17 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:17.683+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:37.550795+0000 front 2023-11-16T16:50:37.550812+0000 (oldest deadline 2023-11-16T16:51:01.650942+0000) 2023-11-16T16:51:17.941 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:17 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:17.683+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:41.651596+0000 front 2023-11-16T16:50:41.651409+0000 (oldest deadline 2023-11-16T16:51:07.551550+0000) 2023-11-16T16:51:17.941 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:17 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:17.683+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:51.453642+0000 front 2023-11-16T16:50:51.453714+0000 (oldest deadline 2023-11-16T16:51:17.353527+0000) 2023-11-16T16:51:18.258 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:17 smithi026 systemd[1]: Stopping Ceph osd.7 for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:51:18.330 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:18 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:18.210+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:36.811405+0000 front 2023-11-16T16:50:36.811513+0000 (oldest deadline 2023-11-16T16:51:00.911616+0000) 2023-11-16T16:51:18.331 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:18 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:18.210+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.212686+0000 front 2023-11-16T16:50:43.212712+0000 (oldest deadline 2023-11-16T16:51:07.913040+0000) 2023-11-16T16:51:18.331 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:18 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:18.210+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:47.913771+0000 front 2023-11-16T16:50:47.913677+0000 (oldest deadline 2023-11-16T16:51:13.714459+0000) 2023-11-16T16:51:18.331 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:18 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:18.012+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:18.332 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:18 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:18.012+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:18.332 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:18 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:18.012+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:18.676 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:18 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:18.282+0000 7fb328b1a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:51:18.676 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:18 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:18.282+0000 7fb328b1a700 -1 osd.7 99 *** Got signal Terminated *** 2023-11-16T16:51:18.677 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:18 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:18.282+0000 7fb328b1a700 -1 osd.7 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-16T16:51:18.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:18 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:18.325+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:51:18.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:18 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:18.325+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.372509+0000 front 2023-11-16T16:50:34.372445+0000 (oldest deadline 2023-11-16T16:50:59.672610+0000) 2023-11-16T16:51:18.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:18 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:18.325+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:44.974145+0000 front 2023-11-16T16:50:44.973914+0000 (oldest deadline 2023-11-16T16:51:09.673929+0000) 2023-11-16T16:51:18.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:18 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:18.325+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:49.674280+0000 front 2023-11-16T16:50:49.674295+0000 (oldest deadline 2023-11-16T16:51:14.374459+0000) 2023-11-16T16:51:19.008 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:18 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:18.725+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:37.550795+0000 front 2023-11-16T16:50:37.550812+0000 (oldest deadline 2023-11-16T16:51:01.650942+0000) 2023-11-16T16:51:19.008 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:18 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:18.725+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:41.651596+0000 front 2023-11-16T16:50:41.651409+0000 (oldest deadline 2023-11-16T16:51:07.551550+0000) 2023-11-16T16:51:19.008 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:18 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:18.725+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:51.453642+0000 front 2023-11-16T16:50:51.453714+0000 (oldest deadline 2023-11-16T16:51:17.353527+0000) 2023-11-16T16:51:19.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:18 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:18.675+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:19.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:18 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:18.675+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:19.009 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:18 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:18.675+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:19.009 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:18 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:18.675+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:51.414772+0000 front 2023-11-16T16:50:51.414692+0000 (oldest deadline 2023-11-16T16:51:13.615360+0000) 2023-11-16T16:51:19.331 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:18 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:18.981+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:19.331 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:18 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:18.981+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:19.331 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:18 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:18.981+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:19.332 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:19 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:19.227+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:36.811405+0000 front 2023-11-16T16:50:36.811513+0000 (oldest deadline 2023-11-16T16:51:00.911616+0000) 2023-11-16T16:51:19.332 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:19 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:19.227+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.212686+0000 front 2023-11-16T16:50:43.212712+0000 (oldest deadline 2023-11-16T16:51:07.913040+0000) 2023-11-16T16:51:19.332 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:19 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:19.227+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:47.913771+0000 front 2023-11-16T16:50:47.913677+0000 (oldest deadline 2023-11-16T16:51:13.714459+0000) 2023-11-16T16:51:19.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:19 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:19.369+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:51:19.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:19 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:19.369+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.372509+0000 front 2023-11-16T16:50:34.372445+0000 (oldest deadline 2023-11-16T16:50:59.672610+0000) 2023-11-16T16:51:19.683 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:19 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:19.369+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:44.974145+0000 front 2023-11-16T16:50:44.973914+0000 (oldest deadline 2023-11-16T16:51:09.673929+0000) 2023-11-16T16:51:19.683 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:19 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:19.369+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:49.674280+0000 front 2023-11-16T16:50:49.674295+0000 (oldest deadline 2023-11-16T16:51:14.374459+0000) 2023-11-16T16:51:20.008 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:19 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:19.697+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:37.550795+0000 front 2023-11-16T16:50:37.550812+0000 (oldest deadline 2023-11-16T16:51:01.650942+0000) 2023-11-16T16:51:20.008 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:19 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:19.697+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:41.651596+0000 front 2023-11-16T16:50:41.651409+0000 (oldest deadline 2023-11-16T16:51:07.551550+0000) 2023-11-16T16:51:20.008 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:19 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:19.697+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:51.453642+0000 front 2023-11-16T16:50:51.453714+0000 (oldest deadline 2023-11-16T16:51:17.353527+0000) 2023-11-16T16:51:20.009 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:19 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:19.679+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:20.009 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:19 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:19.679+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:20.009 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:19 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:19.679+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:20.009 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:19 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:19.679+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:51.414772+0000 front 2023-11-16T16:50:51.414692+0000 (oldest deadline 2023-11-16T16:51:13.615360+0000) 2023-11-16T16:51:20.258 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:19 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:19.947+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:20.259 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:19 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:19.947+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:20.259 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:19 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:19.947+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:20.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:20 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:20.257+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:36.811405+0000 front 2023-11-16T16:50:36.811513+0000 (oldest deadline 2023-11-16T16:51:00.911616+0000) 2023-11-16T16:51:20.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:20 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:20.257+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.212686+0000 front 2023-11-16T16:50:43.212712+0000 (oldest deadline 2023-11-16T16:51:07.913040+0000) 2023-11-16T16:51:20.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:20 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:20.257+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:47.913771+0000 front 2023-11-16T16:50:47.913677+0000 (oldest deadline 2023-11-16T16:51:13.714459+0000) 2023-11-16T16:51:20.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:20 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:20.393+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:51:20.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:20 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:20.393+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.372509+0000 front 2023-11-16T16:50:34.372445+0000 (oldest deadline 2023-11-16T16:50:59.672610+0000) 2023-11-16T16:51:20.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:20 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:20.393+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:44.974145+0000 front 2023-11-16T16:50:44.973914+0000 (oldest deadline 2023-11-16T16:51:09.673929+0000) 2023-11-16T16:51:20.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:20 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:20.393+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:49.674280+0000 front 2023-11-16T16:50:49.674295+0000 (oldest deadline 2023-11-16T16:51:14.374459+0000) 2023-11-16T16:51:21.008 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:20 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:20.735+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:37.550795+0000 front 2023-11-16T16:50:37.550812+0000 (oldest deadline 2023-11-16T16:51:01.650942+0000) 2023-11-16T16:51:21.008 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:20 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:20.735+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:41.651596+0000 front 2023-11-16T16:50:41.651409+0000 (oldest deadline 2023-11-16T16:51:07.551550+0000) 2023-11-16T16:51:21.008 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:20 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:20.735+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:51.453642+0000 front 2023-11-16T16:50:51.453714+0000 (oldest deadline 2023-11-16T16:51:17.353527+0000) 2023-11-16T16:51:21.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:20 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:20.719+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:21.009 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:20 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:20.719+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:21.009 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:20 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:20.719+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:21.009 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:20 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:20.719+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:51.414772+0000 front 2023-11-16T16:50:51.414692+0000 (oldest deadline 2023-11-16T16:51:13.615360+0000) 2023-11-16T16:51:21.263 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:20 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:20.937+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:21.263 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:20 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:20.937+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:21.263 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:20 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:20.937+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:21.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:21 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:21.262+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:36.811405+0000 front 2023-11-16T16:50:36.811513+0000 (oldest deadline 2023-11-16T16:51:00.911616+0000) 2023-11-16T16:51:21.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:21 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:21.262+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.212686+0000 front 2023-11-16T16:50:43.212712+0000 (oldest deadline 2023-11-16T16:51:07.913040+0000) 2023-11-16T16:51:21.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:21 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:21.262+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:47.913771+0000 front 2023-11-16T16:50:47.913677+0000 (oldest deadline 2023-11-16T16:51:13.714459+0000) 2023-11-16T16:51:21.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:21 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:21.414+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:51:21.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:21 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:21.414+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.372509+0000 front 2023-11-16T16:50:34.372445+0000 (oldest deadline 2023-11-16T16:50:59.672610+0000) 2023-11-16T16:51:21.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:21 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:21.414+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:44.974145+0000 front 2023-11-16T16:50:44.973914+0000 (oldest deadline 2023-11-16T16:51:09.673929+0000) 2023-11-16T16:51:21.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:21 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:21.414+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:49.674280+0000 front 2023-11-16T16:50:49.674295+0000 (oldest deadline 2023-11-16T16:51:14.374459+0000) 2023-11-16T16:51:22.008 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:21 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:21.730+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:37.550795+0000 front 2023-11-16T16:50:37.550812+0000 (oldest deadline 2023-11-16T16:51:01.650942+0000) 2023-11-16T16:51:22.008 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:21 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:21.730+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:41.651596+0000 front 2023-11-16T16:50:41.651409+0000 (oldest deadline 2023-11-16T16:51:07.551550+0000) 2023-11-16T16:51:22.008 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:21 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:21.730+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:51.453642+0000 front 2023-11-16T16:50:51.453714+0000 (oldest deadline 2023-11-16T16:51:17.353527+0000) 2023-11-16T16:51:22.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:21 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:21.692+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:22.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:21 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:21.692+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:22.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:21 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:21.692+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:22.009 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:21 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:21.692+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:51.414772+0000 front 2023-11-16T16:50:51.414692+0000 (oldest deadline 2023-11-16T16:51:13.615360+0000) 2023-11-16T16:51:22.009 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:21 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:21.692+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:51:00.017293+0000 front 2023-11-16T16:51:00.017318+0000 (oldest deadline 2023-11-16T16:51:21.117420+0000) 2023-11-16T16:51:22.298 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:21 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:21.930+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:22.299 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:21 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:21.930+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:22.299 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:21 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:21.930+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:22.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:22 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:22.297+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:36.811405+0000 front 2023-11-16T16:50:36.811513+0000 (oldest deadline 2023-11-16T16:51:00.911616+0000) 2023-11-16T16:51:22.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:22 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:22.297+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.212686+0000 front 2023-11-16T16:50:43.212712+0000 (oldest deadline 2023-11-16T16:51:07.913040+0000) 2023-11-16T16:51:22.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:22 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:22.297+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:47.913771+0000 front 2023-11-16T16:50:47.913677+0000 (oldest deadline 2023-11-16T16:51:13.714459+0000) 2023-11-16T16:51:22.932 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:22 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:22.434+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:51:22.932 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:22 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:22.434+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.372509+0000 front 2023-11-16T16:50:34.372445+0000 (oldest deadline 2023-11-16T16:50:59.672610+0000) 2023-11-16T16:51:22.932 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:22 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:22.434+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:44.974145+0000 front 2023-11-16T16:50:44.973914+0000 (oldest deadline 2023-11-16T16:51:09.673929+0000) 2023-11-16T16:51:22.932 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:22 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:22.434+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:49.674280+0000 front 2023-11-16T16:50:49.674295+0000 (oldest deadline 2023-11-16T16:51:14.374459+0000) 2023-11-16T16:51:22.932 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:22 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:22.434+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.675926+0000 front 2023-11-16T16:50:56.675856+0000 (oldest deadline 2023-11-16T16:51:21.975628+0000) 2023-11-16T16:51:23.008 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:22 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:22.734+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:37.550795+0000 front 2023-11-16T16:50:37.550812+0000 (oldest deadline 2023-11-16T16:51:01.650942+0000) 2023-11-16T16:51:23.008 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:22 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:22.734+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:41.651596+0000 front 2023-11-16T16:50:41.651409+0000 (oldest deadline 2023-11-16T16:51:07.551550+0000) 2023-11-16T16:51:23.008 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:22 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:22.734+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:51.453642+0000 front 2023-11-16T16:50:51.453714+0000 (oldest deadline 2023-11-16T16:51:17.353527+0000) 2023-11-16T16:51:23.008 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:22 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7[116105]: 2023-11-16T16:51:22.734+0000 7fb321909700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:57.353921+0000 front 2023-11-16T16:50:57.353898+0000 (oldest deadline 2023-11-16T16:51:22.654010+0000) 2023-11-16T16:51:23.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:22 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:22.712+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:23.009 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:22 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:22.712+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:23.009 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:22 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:22.712+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:23.009 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:22 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:22.712+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:51.414772+0000 front 2023-11-16T16:50:51.414692+0000 (oldest deadline 2023-11-16T16:51:13.615360+0000) 2023-11-16T16:51:23.009 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:22 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:22.712+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:51:00.017293+0000 front 2023-11-16T16:51:00.017318+0000 (oldest deadline 2023-11-16T16:51:21.117420+0000) 2023-11-16T16:51:23.309 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:22 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:22.924+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:23.310 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:22 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:22.924+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:23.310 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:22 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:22.924+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:23.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:23 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:23.308+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:36.811405+0000 front 2023-11-16T16:50:36.811513+0000 (oldest deadline 2023-11-16T16:51:00.911616+0000) 2023-11-16T16:51:23.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:23 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:23.308+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.212686+0000 front 2023-11-16T16:50:43.212712+0000 (oldest deadline 2023-11-16T16:51:07.913040+0000) 2023-11-16T16:51:23.581 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:23 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:23.308+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:47.913771+0000 front 2023-11-16T16:50:47.913677+0000 (oldest deadline 2023-11-16T16:51:13.714459+0000) 2023-11-16T16:51:23.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:23 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:23.427+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:51:23.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:23 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:23.427+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.372509+0000 front 2023-11-16T16:50:34.372445+0000 (oldest deadline 2023-11-16T16:50:59.672610+0000) 2023-11-16T16:51:23.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:23 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:23.427+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:44.974145+0000 front 2023-11-16T16:50:44.973914+0000 (oldest deadline 2023-11-16T16:51:09.673929+0000) 2023-11-16T16:51:23.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:23 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:23.427+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:49.674280+0000 front 2023-11-16T16:50:49.674295+0000 (oldest deadline 2023-11-16T16:51:14.374459+0000) 2023-11-16T16:51:23.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:23 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:23.427+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.675926+0000 front 2023-11-16T16:50:56.675856+0000 (oldest deadline 2023-11-16T16:51:21.975628+0000) 2023-11-16T16:51:24.022 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:23 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:23.761+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:24.022 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:23 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:23.761+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:24.023 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:23 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:23.761+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:24.023 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:23 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:23.761+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:51.414772+0000 front 2023-11-16T16:50:51.414692+0000 (oldest deadline 2023-11-16T16:51:13.615360+0000) 2023-11-16T16:51:24.023 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:23 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:23.762+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:51:00.017293+0000 front 2023-11-16T16:51:00.017318+0000 (oldest deadline 2023-11-16T16:51:21.117420+0000) 2023-11-16T16:51:24.330 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:23 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:23.882+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:24.330 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:23 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:23.882+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:24.330 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:23 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:23.882+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:24.331 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:23 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:23.882+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.747074+0000 front 2023-11-16T16:50:56.746975+0000 (oldest deadline 2023-11-16T16:51:23.147359+0000) 2023-11-16T16:51:24.507 INFO:journalctl@ceph.osd.7.smithi026.stdout:Nov 16 16:51:24 smithi026 bash[132160]: ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-7 2023-11-16T16:51:24.830 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:24 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:24.329+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:36.811405+0000 front 2023-11-16T16:50:36.811513+0000 (oldest deadline 2023-11-16T16:51:00.911616+0000) 2023-11-16T16:51:24.831 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:24 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:24.329+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.212686+0000 front 2023-11-16T16:50:43.212712+0000 (oldest deadline 2023-11-16T16:51:07.913040+0000) 2023-11-16T16:51:24.831 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:24 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:24.329+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:47.913771+0000 front 2023-11-16T16:50:47.913677+0000 (oldest deadline 2023-11-16T16:51:13.714459+0000) 2023-11-16T16:51:24.831 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:24 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:24.329+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:57.617094+0000 front 2023-11-16T16:50:57.616978+0000 (oldest deadline 2023-11-16T16:51:23.517035+0000) 2023-11-16T16:51:24.932 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:24 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:24.452+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:51:24.932 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:24 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:24.452+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.372509+0000 front 2023-11-16T16:50:34.372445+0000 (oldest deadline 2023-11-16T16:50:59.672610+0000) 2023-11-16T16:51:24.932 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:24 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:24.452+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:44.974145+0000 front 2023-11-16T16:50:44.973914+0000 (oldest deadline 2023-11-16T16:51:09.673929+0000) 2023-11-16T16:51:24.932 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:24 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:24.452+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:49.674280+0000 front 2023-11-16T16:50:49.674295+0000 (oldest deadline 2023-11-16T16:51:14.374459+0000) 2023-11-16T16:51:24.932 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:24 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:24.452+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.675926+0000 front 2023-11-16T16:50:56.675856+0000 (oldest deadline 2023-11-16T16:51:21.975628+0000) 2023-11-16T16:51:25.228 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:24 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:24.772+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:25.228 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:24 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:24.772+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:25.228 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:24 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:24.772+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:25.228 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:24 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:24.772+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:51.414772+0000 front 2023-11-16T16:50:51.414692+0000 (oldest deadline 2023-11-16T16:51:13.615360+0000) 2023-11-16T16:51:25.228 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:24 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:24.772+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:51:00.017293+0000 front 2023-11-16T16:51:00.017318+0000 (oldest deadline 2023-11-16T16:51:21.117420+0000) 2023-11-16T16:51:25.283 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:24 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:24.897+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:25.283 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:24 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:24.897+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:25.283 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:24 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:24.897+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:25.283 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:24 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:24.897+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.747074+0000 front 2023-11-16T16:50:56.746975+0000 (oldest deadline 2023-11-16T16:51:23.147359+0000) 2023-11-16T16:51:25.560 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@osd.7.service' 2023-11-16T16:51:25.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:25 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:25.281+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:36.811405+0000 front 2023-11-16T16:50:36.811513+0000 (oldest deadline 2023-11-16T16:51:00.911616+0000) 2023-11-16T16:51:25.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:25 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:25.281+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.212686+0000 front 2023-11-16T16:50:43.212712+0000 (oldest deadline 2023-11-16T16:51:07.913040+0000) 2023-11-16T16:51:25.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:25 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:25.281+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:47.913771+0000 front 2023-11-16T16:50:47.913677+0000 (oldest deadline 2023-11-16T16:51:13.714459+0000) 2023-11-16T16:51:25.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:25 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:25.281+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:57.617094+0000 front 2023-11-16T16:50:57.616978+0000 (oldest deadline 2023-11-16T16:51:23.517035+0000) 2023-11-16T16:51:25.642 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:51:25.643 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-11-16T16:51:25.643 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-11-16T16:51:25.643 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@osd.8 2023-11-16T16:51:25.931 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:25 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:25.444+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:51:25.932 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:25 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:25.444+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.372509+0000 front 2023-11-16T16:50:34.372445+0000 (oldest deadline 2023-11-16T16:50:59.672610+0000) 2023-11-16T16:51:25.932 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:25 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:25.444+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:44.974145+0000 front 2023-11-16T16:50:44.973914+0000 (oldest deadline 2023-11-16T16:51:09.673929+0000) 2023-11-16T16:51:25.932 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:25 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:25.444+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:49.674280+0000 front 2023-11-16T16:50:49.674295+0000 (oldest deadline 2023-11-16T16:51:14.374459+0000) 2023-11-16T16:51:25.932 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:25 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:25.444+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.675926+0000 front 2023-11-16T16:50:56.675856+0000 (oldest deadline 2023-11-16T16:51:21.975628+0000) 2023-11-16T16:51:26.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:25 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:25.738+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:26.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:25 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:25.738+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:26.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:25 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:25.738+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:26.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:25 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:25.738+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:51.414772+0000 front 2023-11-16T16:50:51.414692+0000 (oldest deadline 2023-11-16T16:51:13.615360+0000) 2023-11-16T16:51:26.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:25 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:25.738+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:51:00.017293+0000 front 2023-11-16T16:51:00.017318+0000 (oldest deadline 2023-11-16T16:51:21.117420+0000) 2023-11-16T16:51:26.016 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:25 smithi053 systemd[1]: Stopping Ceph osd.8 for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:51:26.016 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:25 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:25.894+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:26.016 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:25 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:25.894+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:26.016 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:25 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:25.894+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:26.016 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:25 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:25.894+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.747074+0000 front 2023-11-16T16:50:56.746975+0000 (oldest deadline 2023-11-16T16:51:23.147359+0000) 2023-11-16T16:51:26.309 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:26 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:26.013+0000 7fc02f047700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:51:26.309 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:26 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:26.013+0000 7fc02f047700 -1 osd.8 99 *** Got signal Terminated *** 2023-11-16T16:51:26.309 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:26 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:26.013+0000 7fc02f047700 -1 osd.8 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-16T16:51:26.309 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:26 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:26.306+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:36.811405+0000 front 2023-11-16T16:50:36.811513+0000 (oldest deadline 2023-11-16T16:51:00.911616+0000) 2023-11-16T16:51:26.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:26 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:26.306+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.212686+0000 front 2023-11-16T16:50:43.212712+0000 (oldest deadline 2023-11-16T16:51:07.913040+0000) 2023-11-16T16:51:26.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:26 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:26.306+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:47.913771+0000 front 2023-11-16T16:50:47.913677+0000 (oldest deadline 2023-11-16T16:51:13.714459+0000) 2023-11-16T16:51:26.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:26 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:26.306+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:57.617094+0000 front 2023-11-16T16:50:57.616978+0000 (oldest deadline 2023-11-16T16:51:23.517035+0000) 2023-11-16T16:51:26.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:26 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:26.424+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:51:26.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:26 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:26.424+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.372509+0000 front 2023-11-16T16:50:34.372445+0000 (oldest deadline 2023-11-16T16:50:59.672610+0000) 2023-11-16T16:51:26.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:26 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:26.424+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:44.974145+0000 front 2023-11-16T16:50:44.973914+0000 (oldest deadline 2023-11-16T16:51:09.673929+0000) 2023-11-16T16:51:26.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:26 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:26.424+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:49.674280+0000 front 2023-11-16T16:50:49.674295+0000 (oldest deadline 2023-11-16T16:51:14.374459+0000) 2023-11-16T16:51:26.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:26 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:26.424+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.675926+0000 front 2023-11-16T16:50:56.675856+0000 (oldest deadline 2023-11-16T16:51:21.975628+0000) 2023-11-16T16:51:27.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:26 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:26.786+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:27.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:26 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:26.786+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:27.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:26 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:26.786+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:27.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:26 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:26.786+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:51.414772+0000 front 2023-11-16T16:50:51.414692+0000 (oldest deadline 2023-11-16T16:51:13.615360+0000) 2023-11-16T16:51:27.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:26 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:26.786+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:51:00.017293+0000 front 2023-11-16T16:51:00.017318+0000 (oldest deadline 2023-11-16T16:51:21.117420+0000) 2023-11-16T16:51:27.307 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:26 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:26.845+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:27.308 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:26 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:26.845+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:27.308 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:26 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:26.845+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:27.308 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:26 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:26.845+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.747074+0000 front 2023-11-16T16:50:56.746975+0000 (oldest deadline 2023-11-16T16:51:23.147359+0000) 2023-11-16T16:51:27.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:27 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:27.306+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:36.811405+0000 front 2023-11-16T16:50:36.811513+0000 (oldest deadline 2023-11-16T16:51:00.911616+0000) 2023-11-16T16:51:27.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:27 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:27.306+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.212686+0000 front 2023-11-16T16:50:43.212712+0000 (oldest deadline 2023-11-16T16:51:07.913040+0000) 2023-11-16T16:51:27.581 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:27 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:27.306+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:47.913771+0000 front 2023-11-16T16:50:47.913677+0000 (oldest deadline 2023-11-16T16:51:13.714459+0000) 2023-11-16T16:51:27.581 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:27 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:27.306+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:57.617094+0000 front 2023-11-16T16:50:57.616978+0000 (oldest deadline 2023-11-16T16:51:23.517035+0000) 2023-11-16T16:51:27.932 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:27 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:27.452+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:51:27.932 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:27 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:27.452+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.372509+0000 front 2023-11-16T16:50:34.372445+0000 (oldest deadline 2023-11-16T16:50:59.672610+0000) 2023-11-16T16:51:27.932 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:27 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:27.452+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:44.974145+0000 front 2023-11-16T16:50:44.973914+0000 (oldest deadline 2023-11-16T16:51:09.673929+0000) 2023-11-16T16:51:27.932 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:27 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:27.452+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:49.674280+0000 front 2023-11-16T16:50:49.674295+0000 (oldest deadline 2023-11-16T16:51:14.374459+0000) 2023-11-16T16:51:27.932 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:27 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:27.452+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.675926+0000 front 2023-11-16T16:50:56.675856+0000 (oldest deadline 2023-11-16T16:51:21.975628+0000) 2023-11-16T16:51:28.080 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:27 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:27.817+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:28.080 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:27 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:27.817+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:28.080 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:27 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:27.817+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:28.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:27 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:27.817+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.747074+0000 front 2023-11-16T16:50:56.746975+0000 (oldest deadline 2023-11-16T16:51:23.147359+0000) 2023-11-16T16:51:28.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:27 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:27.832+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:28.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:27 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:27.832+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:28.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:27 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:27.832+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:28.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:27 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:27.832+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:51.414772+0000 front 2023-11-16T16:50:51.414692+0000 (oldest deadline 2023-11-16T16:51:13.615360+0000) 2023-11-16T16:51:28.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:27 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:27.832+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:51:00.017293+0000 front 2023-11-16T16:51:00.017318+0000 (oldest deadline 2023-11-16T16:51:21.117420+0000) 2023-11-16T16:51:28.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:28 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:28.322+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:36.811405+0000 front 2023-11-16T16:50:36.811513+0000 (oldest deadline 2023-11-16T16:51:00.911616+0000) 2023-11-16T16:51:28.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:28 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:28.322+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.212686+0000 front 2023-11-16T16:50:43.212712+0000 (oldest deadline 2023-11-16T16:51:07.913040+0000) 2023-11-16T16:51:28.581 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:28 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:28.322+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:47.913771+0000 front 2023-11-16T16:50:47.913677+0000 (oldest deadline 2023-11-16T16:51:13.714459+0000) 2023-11-16T16:51:28.581 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:28 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:28.322+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:57.617094+0000 front 2023-11-16T16:50:57.616978+0000 (oldest deadline 2023-11-16T16:51:23.517035+0000) 2023-11-16T16:51:28.932 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:28 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:28.452+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:51:28.932 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:28 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:28.452+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.372509+0000 front 2023-11-16T16:50:34.372445+0000 (oldest deadline 2023-11-16T16:50:59.672610+0000) 2023-11-16T16:51:28.933 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:28 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:28.452+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:44.974145+0000 front 2023-11-16T16:50:44.973914+0000 (oldest deadline 2023-11-16T16:51:09.673929+0000) 2023-11-16T16:51:28.933 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:28 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:28.452+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:49.674280+0000 front 2023-11-16T16:50:49.674295+0000 (oldest deadline 2023-11-16T16:51:14.374459+0000) 2023-11-16T16:51:28.933 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:28 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:28.452+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.675926+0000 front 2023-11-16T16:50:56.675856+0000 (oldest deadline 2023-11-16T16:51:21.975628+0000) 2023-11-16T16:51:29.080 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:28 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:28.822+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:29.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:28 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:28.822+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:29.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:28 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:28.822+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:29.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:28 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:28.822+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.747074+0000 front 2023-11-16T16:50:56.746975+0000 (oldest deadline 2023-11-16T16:51:23.147359+0000) 2023-11-16T16:51:29.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:28 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:28.827+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:29.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:28 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:28.827+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:29.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:28 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:28.827+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:29.259 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:28 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:28.827+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:51.414772+0000 front 2023-11-16T16:50:51.414692+0000 (oldest deadline 2023-11-16T16:51:13.615360+0000) 2023-11-16T16:51:29.259 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:28 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:28.827+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:51:00.017293+0000 front 2023-11-16T16:51:00.017318+0000 (oldest deadline 2023-11-16T16:51:21.117420+0000) 2023-11-16T16:51:29.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:29 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:29.294+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:36.811405+0000 front 2023-11-16T16:50:36.811513+0000 (oldest deadline 2023-11-16T16:51:00.911616+0000) 2023-11-16T16:51:29.581 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:29 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:29.294+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.212686+0000 front 2023-11-16T16:50:43.212712+0000 (oldest deadline 2023-11-16T16:51:07.913040+0000) 2023-11-16T16:51:29.581 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:29 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:29.294+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:47.913771+0000 front 2023-11-16T16:50:47.913677+0000 (oldest deadline 2023-11-16T16:51:13.714459+0000) 2023-11-16T16:51:29.581 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:29 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:29.294+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:57.617094+0000 front 2023-11-16T16:50:57.616978+0000 (oldest deadline 2023-11-16T16:51:23.517035+0000) 2023-11-16T16:51:29.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:29 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:29.419+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:51:29.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:29 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:29.419+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.372509+0000 front 2023-11-16T16:50:34.372445+0000 (oldest deadline 2023-11-16T16:50:59.672610+0000) 2023-11-16T16:51:29.683 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:29 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:29.419+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:44.974145+0000 front 2023-11-16T16:50:44.973914+0000 (oldest deadline 2023-11-16T16:51:09.673929+0000) 2023-11-16T16:51:29.683 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:29 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:29.419+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:49.674280+0000 front 2023-11-16T16:50:49.674295+0000 (oldest deadline 2023-11-16T16:51:14.374459+0000) 2023-11-16T16:51:29.683 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:29 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:29.419+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.675926+0000 front 2023-11-16T16:50:56.675856+0000 (oldest deadline 2023-11-16T16:51:21.975628+0000) 2023-11-16T16:51:30.080 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:29 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:29.781+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:30.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:29 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:29.781+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:30.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:29 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:29.781+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:30.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:29 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:29.781+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.747074+0000 front 2023-11-16T16:50:56.746975+0000 (oldest deadline 2023-11-16T16:51:23.147359+0000) 2023-11-16T16:51:30.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:29 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:29.781+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:05.348634+0000 front 2023-11-16T16:51:05.348649+0000 (oldest deadline 2023-11-16T16:51:28.848866+0000) 2023-11-16T16:51:30.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:29 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:29.794+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:30.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:29 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:29.794+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:30.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:29 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:29.794+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:30.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:29 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:29.794+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:51.414772+0000 front 2023-11-16T16:50:51.414692+0000 (oldest deadline 2023-11-16T16:51:13.615360+0000) 2023-11-16T16:51:30.259 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:29 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:29.794+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:51:00.017293+0000 front 2023-11-16T16:51:00.017318+0000 (oldest deadline 2023-11-16T16:51:21.117420+0000) 2023-11-16T16:51:30.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:30 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:30.305+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:36.811405+0000 front 2023-11-16T16:50:36.811513+0000 (oldest deadline 2023-11-16T16:51:00.911616+0000) 2023-11-16T16:51:30.580 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:30 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:30.305+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.212686+0000 front 2023-11-16T16:50:43.212712+0000 (oldest deadline 2023-11-16T16:51:07.913040+0000) 2023-11-16T16:51:30.581 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:30 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:30.305+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:47.913771+0000 front 2023-11-16T16:50:47.913677+0000 (oldest deadline 2023-11-16T16:51:13.714459+0000) 2023-11-16T16:51:30.581 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:30 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:30.305+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:57.617094+0000 front 2023-11-16T16:50:57.616978+0000 (oldest deadline 2023-11-16T16:51:23.517035+0000) 2023-11-16T16:51:30.581 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:30 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8[117302]: 2023-11-16T16:51:30.305+0000 7fc02ae3c700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:03.517484+0000 front 2023-11-16T16:51:03.517532+0000 (oldest deadline 2023-11-16T16:51:29.417652+0000) 2023-11-16T16:51:30.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:30 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:30.377+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:51:30.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:30 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:30.377+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.372509+0000 front 2023-11-16T16:50:34.372445+0000 (oldest deadline 2023-11-16T16:50:59.672610+0000) 2023-11-16T16:51:30.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:30 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:30.377+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:44.974145+0000 front 2023-11-16T16:50:44.973914+0000 (oldest deadline 2023-11-16T16:51:09.673929+0000) 2023-11-16T16:51:30.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:30 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:30.377+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:49.674280+0000 front 2023-11-16T16:50:49.674295+0000 (oldest deadline 2023-11-16T16:51:14.374459+0000) 2023-11-16T16:51:30.683 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:30 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:30.377+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.675926+0000 front 2023-11-16T16:50:56.675856+0000 (oldest deadline 2023-11-16T16:51:21.975628+0000) 2023-11-16T16:51:30.683 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:30 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:30.377+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:04.877051+0000 front 2023-11-16T16:51:04.877101+0000 (oldest deadline 2023-11-16T16:51:30.176905+0000) 2023-11-16T16:51:31.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:30 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:30.745+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:31.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:30 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:30.745+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:31.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:30 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:30.745+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:31.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:30 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:30.745+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:51.414772+0000 front 2023-11-16T16:50:51.414692+0000 (oldest deadline 2023-11-16T16:51:13.615360+0000) 2023-11-16T16:51:31.009 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:30 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:30.745+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:51:00.017293+0000 front 2023-11-16T16:51:00.017318+0000 (oldest deadline 2023-11-16T16:51:21.117420+0000) 2023-11-16T16:51:31.080 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:30 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:30.775+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:31.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:30 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:30.775+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:31.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:30 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:30.775+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:31.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:30 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:30.775+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.747074+0000 front 2023-11-16T16:50:56.746975+0000 (oldest deadline 2023-11-16T16:51:23.147359+0000) 2023-11-16T16:51:31.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:30 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:30.775+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:05.348634+0000 front 2023-11-16T16:51:05.348649+0000 (oldest deadline 2023-11-16T16:51:28.848866+0000) 2023-11-16T16:51:31.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:31 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:31.392+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:51:31.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:31 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:31.392+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.372509+0000 front 2023-11-16T16:50:34.372445+0000 (oldest deadline 2023-11-16T16:50:59.672610+0000) 2023-11-16T16:51:31.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:31 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:31.392+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:44.974145+0000 front 2023-11-16T16:50:44.973914+0000 (oldest deadline 2023-11-16T16:51:09.673929+0000) 2023-11-16T16:51:31.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:31 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:31.392+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:49.674280+0000 front 2023-11-16T16:50:49.674295+0000 (oldest deadline 2023-11-16T16:51:14.374459+0000) 2023-11-16T16:51:31.683 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:31 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:31.392+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.675926+0000 front 2023-11-16T16:50:56.675856+0000 (oldest deadline 2023-11-16T16:51:21.975628+0000) 2023-11-16T16:51:31.683 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:31 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:31.392+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:04.877051+0000 front 2023-11-16T16:51:04.877101+0000 (oldest deadline 2023-11-16T16:51:30.176905+0000) 2023-11-16T16:51:31.790 INFO:journalctl@ceph.osd.8.smithi053.stdout:Nov 16 16:51:31 smithi053 bash[133634]: ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-8 2023-11-16T16:51:32.007 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:31 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:31.742+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:32.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:31 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:31.742+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:32.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:31 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:31.742+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:32.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:31 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:31.742+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:51.414772+0000 front 2023-11-16T16:50:51.414692+0000 (oldest deadline 2023-11-16T16:51:13.615360+0000) 2023-11-16T16:51:32.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:31 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:31.742+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:51:00.017293+0000 front 2023-11-16T16:51:00.017318+0000 (oldest deadline 2023-11-16T16:51:21.117420+0000) 2023-11-16T16:51:32.080 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:31 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:31.788+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:32.080 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:31 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:31.788+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:32.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:31 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:31.788+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:32.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:31 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:31.788+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.747074+0000 front 2023-11-16T16:50:56.746975+0000 (oldest deadline 2023-11-16T16:51:23.147359+0000) 2023-11-16T16:51:32.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:31 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:31.788+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:05.348634+0000 front 2023-11-16T16:51:05.348649+0000 (oldest deadline 2023-11-16T16:51:28.848866+0000) 2023-11-16T16:51:32.932 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:32 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:32.431+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:51:32.932 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:32 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:32.431+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.372509+0000 front 2023-11-16T16:50:34.372445+0000 (oldest deadline 2023-11-16T16:50:59.672610+0000) 2023-11-16T16:51:32.933 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:32 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:32.431+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:44.974145+0000 front 2023-11-16T16:50:44.973914+0000 (oldest deadline 2023-11-16T16:51:09.673929+0000) 2023-11-16T16:51:32.933 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:32 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:32.431+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:49.674280+0000 front 2023-11-16T16:50:49.674295+0000 (oldest deadline 2023-11-16T16:51:14.374459+0000) 2023-11-16T16:51:32.933 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:32 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:32.431+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.675926+0000 front 2023-11-16T16:50:56.675856+0000 (oldest deadline 2023-11-16T16:51:21.975628+0000) 2023-11-16T16:51:32.933 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:32 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:32.431+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:04.877051+0000 front 2023-11-16T16:51:04.877101+0000 (oldest deadline 2023-11-16T16:51:30.176905+0000) 2023-11-16T16:51:33.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:32 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:32.742+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:33.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:32 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:32.742+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:33.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:32 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:32.742+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:33.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:32 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:32.742+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:51.414772+0000 front 2023-11-16T16:50:51.414692+0000 (oldest deadline 2023-11-16T16:51:13.615360+0000) 2023-11-16T16:51:33.009 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:32 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:32.742+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:51:00.017293+0000 front 2023-11-16T16:51:00.017318+0000 (oldest deadline 2023-11-16T16:51:21.117420+0000) 2023-11-16T16:51:33.009 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:32 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:32.742+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:06.719803+0000 front 2023-11-16T16:51:06.719838+0000 (oldest deadline 2023-11-16T16:51:32.019846+0000) 2023-11-16T16:51:33.080 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:32 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:32.778+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:33.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:32 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:32.778+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:33.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:32 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:32.778+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:33.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:32 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:32.778+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.747074+0000 front 2023-11-16T16:50:56.746975+0000 (oldest deadline 2023-11-16T16:51:23.147359+0000) 2023-11-16T16:51:33.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:32 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:32.778+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:05.348634+0000 front 2023-11-16T16:51:05.348649+0000 (oldest deadline 2023-11-16T16:51:28.848866+0000) 2023-11-16T16:51:33.371 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@osd.8.service' 2023-11-16T16:51:33.415 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:51:33.415 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-11-16T16:51:33.415 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-11-16T16:51:33.415 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@osd.9 2023-11-16T16:51:33.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:33 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:33.389+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:51:33.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:33 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:33.389+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.372509+0000 front 2023-11-16T16:50:34.372445+0000 (oldest deadline 2023-11-16T16:50:59.672610+0000) 2023-11-16T16:51:33.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:33 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:33.389+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:44.974145+0000 front 2023-11-16T16:50:44.973914+0000 (oldest deadline 2023-11-16T16:51:09.673929+0000) 2023-11-16T16:51:33.683 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:33 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:33.389+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:49.674280+0000 front 2023-11-16T16:50:49.674295+0000 (oldest deadline 2023-11-16T16:51:14.374459+0000) 2023-11-16T16:51:33.683 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:33 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:33.389+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.675926+0000 front 2023-11-16T16:50:56.675856+0000 (oldest deadline 2023-11-16T16:51:21.975628+0000) 2023-11-16T16:51:33.683 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:33 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:33.389+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:04.877051+0000 front 2023-11-16T16:51:04.877101+0000 (oldest deadline 2023-11-16T16:51:30.176905+0000) 2023-11-16T16:51:33.683 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:33 smithi001 systemd[1]: Stopping Ceph osd.9 for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:51:34.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:33 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:33.744+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:34.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:33 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:33.744+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:34.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:33 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:33.744+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:34.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:33 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:33.744+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:51.414772+0000 front 2023-11-16T16:50:51.414692+0000 (oldest deadline 2023-11-16T16:51:13.615360+0000) 2023-11-16T16:51:34.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:33 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:33.744+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:51:00.017293+0000 front 2023-11-16T16:51:00.017318+0000 (oldest deadline 2023-11-16T16:51:21.117420+0000) 2023-11-16T16:51:34.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:33 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:33.744+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:06.719803+0000 front 2023-11-16T16:51:06.719838+0000 (oldest deadline 2023-11-16T16:51:32.019846+0000) 2023-11-16T16:51:34.080 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:33 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:33.766+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:34.080 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:33 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:33.766+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:34.080 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:33 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:33.766+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:34.080 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:33 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:33.766+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.747074+0000 front 2023-11-16T16:50:56.746975+0000 (oldest deadline 2023-11-16T16:51:23.147359+0000) 2023-11-16T16:51:34.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:33 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:33.766+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:05.348634+0000 front 2023-11-16T16:51:05.348649+0000 (oldest deadline 2023-11-16T16:51:28.848866+0000) 2023-11-16T16:51:34.182 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:33 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:33.773+0000 7f9eec98a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:51:34.182 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:33 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:33.773+0000 7f9eec98a700 -1 osd.9 99 *** Got signal Terminated *** 2023-11-16T16:51:34.182 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:33 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:33.773+0000 7f9eec98a700 -1 osd.9 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-16T16:51:34.932 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:34 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:34.436+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:51:34.932 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:34 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:34.436+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.372509+0000 front 2023-11-16T16:50:34.372445+0000 (oldest deadline 2023-11-16T16:50:59.672610+0000) 2023-11-16T16:51:34.932 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:34 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:34.436+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:44.974145+0000 front 2023-11-16T16:50:44.973914+0000 (oldest deadline 2023-11-16T16:51:09.673929+0000) 2023-11-16T16:51:34.932 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:34 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:34.436+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:49.674280+0000 front 2023-11-16T16:50:49.674295+0000 (oldest deadline 2023-11-16T16:51:14.374459+0000) 2023-11-16T16:51:34.933 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:34 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:34.436+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.675926+0000 front 2023-11-16T16:50:56.675856+0000 (oldest deadline 2023-11-16T16:51:21.975628+0000) 2023-11-16T16:51:34.933 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:34 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:34.436+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:04.877051+0000 front 2023-11-16T16:51:04.877101+0000 (oldest deadline 2023-11-16T16:51:30.176905+0000) 2023-11-16T16:51:35.080 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:34 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:34.745+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:35.080 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:34 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:34.745+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:35.080 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:34 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:34.745+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:35.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:34 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:34.745+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.747074+0000 front 2023-11-16T16:50:56.746975+0000 (oldest deadline 2023-11-16T16:51:23.147359+0000) 2023-11-16T16:51:35.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:34 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:34.745+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:05.348634+0000 front 2023-11-16T16:51:05.348649+0000 (oldest deadline 2023-11-16T16:51:28.848866+0000) 2023-11-16T16:51:35.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:34 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:34.765+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:35.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:34 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:34.765+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:35.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:34 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:34.765+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:35.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:34 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:34.765+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:51.414772+0000 front 2023-11-16T16:50:51.414692+0000 (oldest deadline 2023-11-16T16:51:13.615360+0000) 2023-11-16T16:51:35.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:34 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:34.765+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:51:00.017293+0000 front 2023-11-16T16:51:00.017318+0000 (oldest deadline 2023-11-16T16:51:21.117420+0000) 2023-11-16T16:51:35.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:34 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:34.765+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:06.719803+0000 front 2023-11-16T16:51:06.719838+0000 (oldest deadline 2023-11-16T16:51:32.019846+0000) 2023-11-16T16:51:35.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:35 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:35.412+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:51:35.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:35 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:35.412+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.372509+0000 front 2023-11-16T16:50:34.372445+0000 (oldest deadline 2023-11-16T16:50:59.672610+0000) 2023-11-16T16:51:35.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:35 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:35.412+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:44.974145+0000 front 2023-11-16T16:50:44.973914+0000 (oldest deadline 2023-11-16T16:51:09.673929+0000) 2023-11-16T16:51:35.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:35 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:35.412+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:49.674280+0000 front 2023-11-16T16:50:49.674295+0000 (oldest deadline 2023-11-16T16:51:14.374459+0000) 2023-11-16T16:51:35.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:35 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:35.412+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.675926+0000 front 2023-11-16T16:50:56.675856+0000 (oldest deadline 2023-11-16T16:51:21.975628+0000) 2023-11-16T16:51:35.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:35 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:35.412+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:04.877051+0000 front 2023-11-16T16:51:04.877101+0000 (oldest deadline 2023-11-16T16:51:30.176905+0000) 2023-11-16T16:51:36.080 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:35 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:35.743+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:36.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:35 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:35.743+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:36.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:35 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:35.743+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:36.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:35 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:35.743+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.747074+0000 front 2023-11-16T16:50:56.746975+0000 (oldest deadline 2023-11-16T16:51:23.147359+0000) 2023-11-16T16:51:36.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:35 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:35.743+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:05.348634+0000 front 2023-11-16T16:51:05.348649+0000 (oldest deadline 2023-11-16T16:51:28.848866+0000) 2023-11-16T16:51:36.257 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:35 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:35.765+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:36.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:35 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:35.765+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:36.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:35 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:35.765+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:36.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:35 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:35.765+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:51.414772+0000 front 2023-11-16T16:50:51.414692+0000 (oldest deadline 2023-11-16T16:51:13.615360+0000) 2023-11-16T16:51:36.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:35 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:35.765+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:51:00.017293+0000 front 2023-11-16T16:51:00.017318+0000 (oldest deadline 2023-11-16T16:51:21.117420+0000) 2023-11-16T16:51:36.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:35 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:35.765+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:06.719803+0000 front 2023-11-16T16:51:06.719838+0000 (oldest deadline 2023-11-16T16:51:32.019846+0000) 2023-11-16T16:51:36.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:36 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:36.367+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:51:36.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:36 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:36.367+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.372509+0000 front 2023-11-16T16:50:34.372445+0000 (oldest deadline 2023-11-16T16:50:59.672610+0000) 2023-11-16T16:51:36.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:36 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:36.367+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:44.974145+0000 front 2023-11-16T16:50:44.973914+0000 (oldest deadline 2023-11-16T16:51:09.673929+0000) 2023-11-16T16:51:36.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:36 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:36.367+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:49.674280+0000 front 2023-11-16T16:50:49.674295+0000 (oldest deadline 2023-11-16T16:51:14.374459+0000) 2023-11-16T16:51:36.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:36 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:36.367+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.675926+0000 front 2023-11-16T16:50:56.675856+0000 (oldest deadline 2023-11-16T16:51:21.975628+0000) 2023-11-16T16:51:36.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:36 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:36.367+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:04.877051+0000 front 2023-11-16T16:51:04.877101+0000 (oldest deadline 2023-11-16T16:51:30.176905+0000) 2023-11-16T16:51:37.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:36 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:36.725+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:37.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:36 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:36.725+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:37.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:36 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:36.725+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:37.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:36 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:36.725+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:51.414772+0000 front 2023-11-16T16:50:51.414692+0000 (oldest deadline 2023-11-16T16:51:13.615360+0000) 2023-11-16T16:51:37.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:36 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:36.725+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:51:00.017293+0000 front 2023-11-16T16:51:00.017318+0000 (oldest deadline 2023-11-16T16:51:21.117420+0000) 2023-11-16T16:51:37.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:36 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:36.725+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:06.719803+0000 front 2023-11-16T16:51:06.719838+0000 (oldest deadline 2023-11-16T16:51:32.019846+0000) 2023-11-16T16:51:37.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:36 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:36.725+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2023-11-16T16:51:13.720913+0000 front 2023-11-16T16:51:13.721009+0000 (oldest deadline 2023-11-16T16:51:36.620862+0000) 2023-11-16T16:51:37.080 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:36 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:36.762+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:37.080 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:36 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:36.762+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:37.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:36 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:36.762+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:37.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:36 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:36.762+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.747074+0000 front 2023-11-16T16:50:56.746975+0000 (oldest deadline 2023-11-16T16:51:23.147359+0000) 2023-11-16T16:51:37.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:36 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:36.762+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:05.348634+0000 front 2023-11-16T16:51:05.348649+0000 (oldest deadline 2023-11-16T16:51:28.848866+0000) 2023-11-16T16:51:37.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:36 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:36.762+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2023-11-16T16:51:10.550045+0000 front 2023-11-16T16:51:10.550066+0000 (oldest deadline 2023-11-16T16:51:36.449919+0000) 2023-11-16T16:51:37.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:37 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:37.404+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:51:37.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:37 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:37.404+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.372509+0000 front 2023-11-16T16:50:34.372445+0000 (oldest deadline 2023-11-16T16:50:59.672610+0000) 2023-11-16T16:51:37.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:37 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:37.404+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:44.974145+0000 front 2023-11-16T16:50:44.973914+0000 (oldest deadline 2023-11-16T16:51:09.673929+0000) 2023-11-16T16:51:37.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:37 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:37.404+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:49.674280+0000 front 2023-11-16T16:50:49.674295+0000 (oldest deadline 2023-11-16T16:51:14.374459+0000) 2023-11-16T16:51:37.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:37 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:37.404+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.675926+0000 front 2023-11-16T16:50:56.675856+0000 (oldest deadline 2023-11-16T16:51:21.975628+0000) 2023-11-16T16:51:37.683 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:37 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:37.404+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:04.877051+0000 front 2023-11-16T16:51:04.877101+0000 (oldest deadline 2023-11-16T16:51:30.176905+0000) 2023-11-16T16:51:37.683 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:37 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:37.404+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2023-11-16T16:51:15.477597+0000 front 2023-11-16T16:51:15.477615+0000 (oldest deadline 2023-11-16T16:51:36.577764+0000) 2023-11-16T16:51:38.007 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:37 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:37.751+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:38.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:37 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:37.751+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:38.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:37 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:37.751+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:38.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:37 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:37.751+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:51.414772+0000 front 2023-11-16T16:50:51.414692+0000 (oldest deadline 2023-11-16T16:51:13.615360+0000) 2023-11-16T16:51:38.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:37 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:37.751+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:51:00.017293+0000 front 2023-11-16T16:51:00.017318+0000 (oldest deadline 2023-11-16T16:51:21.117420+0000) 2023-11-16T16:51:38.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:37 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:37.751+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:06.719803+0000 front 2023-11-16T16:51:06.719838+0000 (oldest deadline 2023-11-16T16:51:32.019846+0000) 2023-11-16T16:51:38.008 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:37 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:37.751+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2023-11-16T16:51:13.720913+0000 front 2023-11-16T16:51:13.721009+0000 (oldest deadline 2023-11-16T16:51:36.620862+0000) 2023-11-16T16:51:38.080 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:37 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:37.782+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:38.080 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:37 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:37.782+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:38.080 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:37 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:37.782+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:38.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:37 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:37.782+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.747074+0000 front 2023-11-16T16:50:56.746975+0000 (oldest deadline 2023-11-16T16:51:23.147359+0000) 2023-11-16T16:51:38.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:37 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:37.782+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:05.348634+0000 front 2023-11-16T16:51:05.348649+0000 (oldest deadline 2023-11-16T16:51:28.848866+0000) 2023-11-16T16:51:38.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:37 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:37.782+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2023-11-16T16:51:10.550045+0000 front 2023-11-16T16:51:10.550066+0000 (oldest deadline 2023-11-16T16:51:36.449919+0000) 2023-11-16T16:51:38.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:38 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:38.415+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:24.470567+0000 front 2023-11-16T16:50:24.470754+0000 (oldest deadline 2023-11-16T16:50:49.770708+0000) 2023-11-16T16:51:38.682 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:38 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:38.415+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.372509+0000 front 2023-11-16T16:50:34.372445+0000 (oldest deadline 2023-11-16T16:50:59.672610+0000) 2023-11-16T16:51:38.683 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:38 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:38.415+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:44.974145+0000 front 2023-11-16T16:50:44.973914+0000 (oldest deadline 2023-11-16T16:51:09.673929+0000) 2023-11-16T16:51:38.683 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:38 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:38.415+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:49.674280+0000 front 2023-11-16T16:50:49.674295+0000 (oldest deadline 2023-11-16T16:51:14.374459+0000) 2023-11-16T16:51:38.683 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:38 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:38.415+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.675926+0000 front 2023-11-16T16:50:56.675856+0000 (oldest deadline 2023-11-16T16:51:21.975628+0000) 2023-11-16T16:51:38.683 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:38 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:38.415+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:04.877051+0000 front 2023-11-16T16:51:04.877101+0000 (oldest deadline 2023-11-16T16:51:30.176905+0000) 2023-11-16T16:51:38.684 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:38 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9[128181]: 2023-11-16T16:51:38.415+0000 7f9ee4f78700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2023-11-16T16:51:15.477597+0000 front 2023-11-16T16:51:15.477615+0000 (oldest deadline 2023-11-16T16:51:36.577764+0000) 2023-11-16T16:51:39.080 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:38 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:38.782+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:39.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:38 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:38.782+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:39.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:38 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:38.782+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:39.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:38 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:38.782+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.747074+0000 front 2023-11-16T16:50:56.746975+0000 (oldest deadline 2023-11-16T16:51:23.147359+0000) 2023-11-16T16:51:39.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:38 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:38.782+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:05.348634+0000 front 2023-11-16T16:51:05.348649+0000 (oldest deadline 2023-11-16T16:51:28.848866+0000) 2023-11-16T16:51:39.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:38 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:38.782+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2023-11-16T16:51:10.550045+0000 front 2023-11-16T16:51:10.550066+0000 (oldest deadline 2023-11-16T16:51:36.449919+0000) 2023-11-16T16:51:39.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:38 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:38.782+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:39.259 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:38 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:38.782+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:39.259 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:38 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:38.782+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:39.259 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:38 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:38.782+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:51.414772+0000 front 2023-11-16T16:50:51.414692+0000 (oldest deadline 2023-11-16T16:51:13.615360+0000) 2023-11-16T16:51:39.259 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:38 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:38.782+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:51:00.017293+0000 front 2023-11-16T16:51:00.017318+0000 (oldest deadline 2023-11-16T16:51:21.117420+0000) 2023-11-16T16:51:39.260 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:38 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:38.782+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:06.719803+0000 front 2023-11-16T16:51:06.719838+0000 (oldest deadline 2023-11-16T16:51:32.019846+0000) 2023-11-16T16:51:39.260 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:38 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:38.782+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2023-11-16T16:51:13.720913+0000 front 2023-11-16T16:51:13.721009+0000 (oldest deadline 2023-11-16T16:51:36.620862+0000) 2023-11-16T16:51:39.562 INFO:journalctl@ceph.osd.9.smithi001.stdout:Nov 16 16:51:39 smithi001 bash[263511]: ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-9 2023-11-16T16:51:40.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:39 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:39.747+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:40.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:39 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:39.747+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:40.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:39 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:39.747+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:40.082 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:39 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:39.747+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.747074+0000 front 2023-11-16T16:50:56.746975+0000 (oldest deadline 2023-11-16T16:51:23.147359+0000) 2023-11-16T16:51:40.082 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:39 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:39.747+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:05.348634+0000 front 2023-11-16T16:51:05.348649+0000 (oldest deadline 2023-11-16T16:51:28.848866+0000) 2023-11-16T16:51:40.082 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:39 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:39.747+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2023-11-16T16:51:10.550045+0000 front 2023-11-16T16:51:10.550066+0000 (oldest deadline 2023-11-16T16:51:36.449919+0000) 2023-11-16T16:51:40.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:39 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:39.799+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:40.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:39 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:39.799+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:40.259 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:39 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:39.799+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:40.259 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:39 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:39.799+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:51.414772+0000 front 2023-11-16T16:50:51.414692+0000 (oldest deadline 2023-11-16T16:51:13.615360+0000) 2023-11-16T16:51:40.259 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:39 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:39.799+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:51:00.017293+0000 front 2023-11-16T16:51:00.017318+0000 (oldest deadline 2023-11-16T16:51:21.117420+0000) 2023-11-16T16:51:40.259 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:39 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:39.799+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:06.719803+0000 front 2023-11-16T16:51:06.719838+0000 (oldest deadline 2023-11-16T16:51:32.019846+0000) 2023-11-16T16:51:40.260 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:39 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:39.799+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2023-11-16T16:51:13.720913+0000 front 2023-11-16T16:51:13.721009+0000 (oldest deadline 2023-11-16T16:51:36.620862+0000) 2023-11-16T16:51:40.942 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@osd.9.service' 2023-11-16T16:51:40.985 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:51:40.986 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-11-16T16:51:40.986 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-11-16T16:51:40.986 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@osd.10 2023-11-16T16:51:41.080 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:40 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:40.713+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:41.080 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:40 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:40.713+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:41.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:40 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:40.713+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:41.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:40 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:40.713+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.747074+0000 front 2023-11-16T16:50:56.746975+0000 (oldest deadline 2023-11-16T16:51:23.147359+0000) 2023-11-16T16:51:41.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:40 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:40.713+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:05.348634+0000 front 2023-11-16T16:51:05.348649+0000 (oldest deadline 2023-11-16T16:51:28.848866+0000) 2023-11-16T16:51:41.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:40 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:40.713+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2023-11-16T16:51:10.550045+0000 front 2023-11-16T16:51:10.550066+0000 (oldest deadline 2023-11-16T16:51:36.449919+0000) 2023-11-16T16:51:41.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:40 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:40.821+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:41.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:40 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:40.821+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:41.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:40 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:40.821+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:41.259 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:40 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:40.821+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:51.414772+0000 front 2023-11-16T16:50:51.414692+0000 (oldest deadline 2023-11-16T16:51:13.615360+0000) 2023-11-16T16:51:41.259 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:40 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:40.821+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:51:00.017293+0000 front 2023-11-16T16:51:00.017318+0000 (oldest deadline 2023-11-16T16:51:21.117420+0000) 2023-11-16T16:51:41.259 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:40 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:40.821+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:06.719803+0000 front 2023-11-16T16:51:06.719838+0000 (oldest deadline 2023-11-16T16:51:32.019846+0000) 2023-11-16T16:51:41.260 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:40 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:40.821+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2023-11-16T16:51:13.720913+0000 front 2023-11-16T16:51:13.721009+0000 (oldest deadline 2023-11-16T16:51:36.620862+0000) 2023-11-16T16:51:41.260 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:41 smithi026 systemd[1]: Stopping Ceph osd.10 for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:51:41.758 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:41 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:41.344+0000 7f4bb4b69700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:51:41.758 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:41 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:41.344+0000 7f4bb4b69700 -1 osd.10 99 *** Got signal Terminated *** 2023-11-16T16:51:41.758 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:41 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:41.344+0000 7f4bb4b69700 -1 osd.10 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-16T16:51:42.080 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:41 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:41.707+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:42.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:41 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:41.707+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:42.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:41 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:41.707+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:42.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:41 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:41.707+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.747074+0000 front 2023-11-16T16:50:56.746975+0000 (oldest deadline 2023-11-16T16:51:23.147359+0000) 2023-11-16T16:51:42.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:41 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:41.707+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:05.348634+0000 front 2023-11-16T16:51:05.348649+0000 (oldest deadline 2023-11-16T16:51:28.848866+0000) 2023-11-16T16:51:42.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:41 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:41.707+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2023-11-16T16:51:10.550045+0000 front 2023-11-16T16:51:10.550066+0000 (oldest deadline 2023-11-16T16:51:36.449919+0000) 2023-11-16T16:51:42.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:41 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:41.858+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:42.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:41 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:41.858+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:42.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:41 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:41.858+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:42.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:41 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:41.858+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:51.414772+0000 front 2023-11-16T16:50:51.414692+0000 (oldest deadline 2023-11-16T16:51:13.615360+0000) 2023-11-16T16:51:42.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:41 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:41.858+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:51:00.017293+0000 front 2023-11-16T16:51:00.017318+0000 (oldest deadline 2023-11-16T16:51:21.117420+0000) 2023-11-16T16:51:42.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:41 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:41.858+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:06.719803+0000 front 2023-11-16T16:51:06.719838+0000 (oldest deadline 2023-11-16T16:51:32.019846+0000) 2023-11-16T16:51:42.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:41 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:41.858+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2023-11-16T16:51:13.720913+0000 front 2023-11-16T16:51:13.721009+0000 (oldest deadline 2023-11-16T16:51:36.620862+0000) 2023-11-16T16:51:43.080 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:42 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:42.725+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:43.745 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:42 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:42.725+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:43.746 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:42 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:42.725+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:43.746 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:42 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:42.725+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.747074+0000 front 2023-11-16T16:50:56.746975+0000 (oldest deadline 2023-11-16T16:51:23.147359+0000) 2023-11-16T16:51:43.746 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:42 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:42.725+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:05.348634+0000 front 2023-11-16T16:51:05.348649+0000 (oldest deadline 2023-11-16T16:51:28.848866+0000) 2023-11-16T16:51:43.746 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:42 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:42.725+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2023-11-16T16:51:10.550045+0000 front 2023-11-16T16:51:10.550066+0000 (oldest deadline 2023-11-16T16:51:36.449919+0000) 2023-11-16T16:51:43.747 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:42 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:42.825+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:43.747 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:42 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:42.825+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:43.747 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:42 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:42.825+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:43.747 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:42 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:42.825+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:51.414772+0000 front 2023-11-16T16:50:51.414692+0000 (oldest deadline 2023-11-16T16:51:13.615360+0000) 2023-11-16T16:51:43.747 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:42 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:42.825+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:51:00.017293+0000 front 2023-11-16T16:51:00.017318+0000 (oldest deadline 2023-11-16T16:51:21.117420+0000) 2023-11-16T16:51:43.747 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:42 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:42.825+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:06.719803+0000 front 2023-11-16T16:51:06.719838+0000 (oldest deadline 2023-11-16T16:51:32.019846+0000) 2023-11-16T16:51:43.747 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:42 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:42.825+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2023-11-16T16:51:13.720913+0000 front 2023-11-16T16:51:13.721009+0000 (oldest deadline 2023-11-16T16:51:36.620862+0000) 2023-11-16T16:51:44.080 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:43 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:43.771+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:44.080 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:43 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:43.771+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:44.080 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:43 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:43.771+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:44.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:43 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:43.771+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.747074+0000 front 2023-11-16T16:50:56.746975+0000 (oldest deadline 2023-11-16T16:51:23.147359+0000) 2023-11-16T16:51:44.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:43 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:43.771+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:05.348634+0000 front 2023-11-16T16:51:05.348649+0000 (oldest deadline 2023-11-16T16:51:28.848866+0000) 2023-11-16T16:51:44.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:43 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:43.771+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2023-11-16T16:51:10.550045+0000 front 2023-11-16T16:51:10.550066+0000 (oldest deadline 2023-11-16T16:51:36.449919+0000) 2023-11-16T16:51:44.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:43 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:43.776+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:44.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:43 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:43.776+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:44.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:43 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:43.776+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:44.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:43 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:43.776+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:51.414772+0000 front 2023-11-16T16:50:51.414692+0000 (oldest deadline 2023-11-16T16:51:13.615360+0000) 2023-11-16T16:51:44.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:43 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:43.776+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:51:00.017293+0000 front 2023-11-16T16:51:00.017318+0000 (oldest deadline 2023-11-16T16:51:21.117420+0000) 2023-11-16T16:51:44.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:43 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:43.776+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:06.719803+0000 front 2023-11-16T16:51:06.719838+0000 (oldest deadline 2023-11-16T16:51:32.019846+0000) 2023-11-16T16:51:44.259 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:43 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:43.776+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2023-11-16T16:51:13.720913+0000 front 2023-11-16T16:51:13.721009+0000 (oldest deadline 2023-11-16T16:51:36.620862+0000) 2023-11-16T16:51:45.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:44 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:44.757+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:45.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:44 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:44.757+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:45.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:44 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:44.757+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:45.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:44 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:44.757+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.747074+0000 front 2023-11-16T16:50:56.746975+0000 (oldest deadline 2023-11-16T16:51:23.147359+0000) 2023-11-16T16:51:45.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:44 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:44.757+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:05.348634+0000 front 2023-11-16T16:51:05.348649+0000 (oldest deadline 2023-11-16T16:51:28.848866+0000) 2023-11-16T16:51:45.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:44 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:44.757+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2023-11-16T16:51:10.550045+0000 front 2023-11-16T16:51:10.550066+0000 (oldest deadline 2023-11-16T16:51:36.449919+0000) 2023-11-16T16:51:45.082 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:44 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:44.757+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6820 osd.7 since back 2023-11-16T16:51:19.851428+0000 front 2023-11-16T16:51:19.851402+0000 (oldest deadline 2023-11-16T16:51:44.551095+0000) 2023-11-16T16:51:45.257 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:44 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:44.768+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:45.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:44 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:44.768+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:45.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:44 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:44.768+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:45.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:44 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:44.768+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:51.414772+0000 front 2023-11-16T16:50:51.414692+0000 (oldest deadline 2023-11-16T16:51:13.615360+0000) 2023-11-16T16:51:45.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:44 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:44.768+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:51:00.017293+0000 front 2023-11-16T16:51:00.017318+0000 (oldest deadline 2023-11-16T16:51:21.117420+0000) 2023-11-16T16:51:45.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:44 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:44.768+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:06.719803+0000 front 2023-11-16T16:51:06.719838+0000 (oldest deadline 2023-11-16T16:51:32.019846+0000) 2023-11-16T16:51:45.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:44 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:44.768+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2023-11-16T16:51:13.720913+0000 front 2023-11-16T16:51:13.721009+0000 (oldest deadline 2023-11-16T16:51:36.620862+0000) 2023-11-16T16:51:46.080 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:45 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:45.722+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:46.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:45 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:45.722+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:46.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:45 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:45.722+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:46.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:45 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:45.722+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.747074+0000 front 2023-11-16T16:50:56.746975+0000 (oldest deadline 2023-11-16T16:51:23.147359+0000) 2023-11-16T16:51:46.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:45 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:45.722+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:05.348634+0000 front 2023-11-16T16:51:05.348649+0000 (oldest deadline 2023-11-16T16:51:28.848866+0000) 2023-11-16T16:51:46.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:45 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:45.722+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2023-11-16T16:51:10.550045+0000 front 2023-11-16T16:51:10.550066+0000 (oldest deadline 2023-11-16T16:51:36.449919+0000) 2023-11-16T16:51:46.082 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:45 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:45.722+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6820 osd.7 since back 2023-11-16T16:51:19.851428+0000 front 2023-11-16T16:51:19.851402+0000 (oldest deadline 2023-11-16T16:51:44.551095+0000) 2023-11-16T16:51:46.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:45 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:45.772+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:27.510261+0000 front 2023-11-16T16:50:27.510301+0000 (oldest deadline 2023-11-16T16:50:53.311122+0000) 2023-11-16T16:51:46.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:45 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:45.772+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:34.912771+0000 front 2023-11-16T16:50:34.912710+0000 (oldest deadline 2023-11-16T16:51:00.813117+0000) 2023-11-16T16:51:46.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:45 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:45.772+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:40.813936+0000 front 2023-11-16T16:50:40.813817+0000 (oldest deadline 2023-11-16T16:51:06.713780+0000) 2023-11-16T16:51:46.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:45 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:45.772+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2023-11-16T16:50:51.414772+0000 front 2023-11-16T16:50:51.414692+0000 (oldest deadline 2023-11-16T16:51:13.615360+0000) 2023-11-16T16:51:46.259 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:45 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:45.772+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:51:00.017293+0000 front 2023-11-16T16:51:00.017318+0000 (oldest deadline 2023-11-16T16:51:21.117420+0000) 2023-11-16T16:51:46.259 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:45 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:45.772+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:06.719803+0000 front 2023-11-16T16:51:06.719838+0000 (oldest deadline 2023-11-16T16:51:32.019846+0000) 2023-11-16T16:51:46.259 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:45 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10[121603]: 2023-11-16T16:51:45.772+0000 7f4bad157700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2023-11-16T16:51:13.720913+0000 front 2023-11-16T16:51:13.721009+0000 (oldest deadline 2023-11-16T16:51:36.620862+0000) 2023-11-16T16:51:47.080 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:46 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:46.695+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:47.080 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:46 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:46.695+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:47.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:46 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:46.695+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:47.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:46 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:46.695+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.747074+0000 front 2023-11-16T16:50:56.746975+0000 (oldest deadline 2023-11-16T16:51:23.147359+0000) 2023-11-16T16:51:47.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:46 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:46.695+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:05.348634+0000 front 2023-11-16T16:51:05.348649+0000 (oldest deadline 2023-11-16T16:51:28.848866+0000) 2023-11-16T16:51:47.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:46 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:46.695+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2023-11-16T16:51:10.550045+0000 front 2023-11-16T16:51:10.550066+0000 (oldest deadline 2023-11-16T16:51:36.449919+0000) 2023-11-16T16:51:47.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:46 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:46.695+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6820 osd.7 since back 2023-11-16T16:51:19.851428+0000 front 2023-11-16T16:51:19.851402+0000 (oldest deadline 2023-11-16T16:51:44.551095+0000) 2023-11-16T16:51:47.258 INFO:journalctl@ceph.osd.10.smithi026.stdout:Nov 16 16:51:47 smithi026 bash[132437]: ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-10 2023-11-16T16:51:48.080 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:47 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:47.658+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:48.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:47 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:47.658+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:48.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:47 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:47.658+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:48.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:47 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:47.658+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.747074+0000 front 2023-11-16T16:50:56.746975+0000 (oldest deadline 2023-11-16T16:51:23.147359+0000) 2023-11-16T16:51:48.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:47 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:47.658+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:05.348634+0000 front 2023-11-16T16:51:05.348649+0000 (oldest deadline 2023-11-16T16:51:28.848866+0000) 2023-11-16T16:51:48.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:47 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:47.658+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2023-11-16T16:51:10.550045+0000 front 2023-11-16T16:51:10.550066+0000 (oldest deadline 2023-11-16T16:51:36.449919+0000) 2023-11-16T16:51:48.082 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:47 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:47.658+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6820 osd.7 since back 2023-11-16T16:51:19.851428+0000 front 2023-11-16T16:51:19.851402+0000 (oldest deadline 2023-11-16T16:51:44.551095+0000) 2023-11-16T16:51:48.421 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@osd.10.service' 2023-11-16T16:51:48.462 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:51:48.462 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-11-16T16:51:48.462 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-11-16T16:51:48.463 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@osd.11 2023-11-16T16:51:48.829 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:48 smithi053 systemd[1]: Stopping Ceph osd.11 for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:51:48.830 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:48 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:48.698+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:48.830 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:48 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:48.698+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:48.830 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:48 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:48.698+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:48.830 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:48 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:48.698+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.747074+0000 front 2023-11-16T16:50:56.746975+0000 (oldest deadline 2023-11-16T16:51:23.147359+0000) 2023-11-16T16:51:48.830 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:48 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:48.698+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:05.348634+0000 front 2023-11-16T16:51:05.348649+0000 (oldest deadline 2023-11-16T16:51:28.848866+0000) 2023-11-16T16:51:48.830 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:48 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:48.698+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2023-11-16T16:51:10.550045+0000 front 2023-11-16T16:51:10.550066+0000 (oldest deadline 2023-11-16T16:51:36.449919+0000) 2023-11-16T16:51:48.830 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:48 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:48.698+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6820 osd.7 since back 2023-11-16T16:51:19.851428+0000 front 2023-11-16T16:51:19.851402+0000 (oldest deadline 2023-11-16T16:51:44.551095+0000) 2023-11-16T16:51:48.830 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:48 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:48.828+0000 7f22e6553700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:51:48.831 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:48 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:48.828+0000 7f22e6553700 -1 osd.11 99 *** Got signal Terminated *** 2023-11-16T16:51:48.831 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:48 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:48.828+0000 7f22e6553700 -1 osd.11 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-16T16:51:50.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:49 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:49.690+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:50.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:49 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:49.690+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:50.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:49 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:49.690+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:50.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:49 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:49.690+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.747074+0000 front 2023-11-16T16:50:56.746975+0000 (oldest deadline 2023-11-16T16:51:23.147359+0000) 2023-11-16T16:51:50.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:49 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:49.690+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:05.348634+0000 front 2023-11-16T16:51:05.348649+0000 (oldest deadline 2023-11-16T16:51:28.848866+0000) 2023-11-16T16:51:50.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:49 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:49.690+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2023-11-16T16:51:10.550045+0000 front 2023-11-16T16:51:10.550066+0000 (oldest deadline 2023-11-16T16:51:36.449919+0000) 2023-11-16T16:51:50.082 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:49 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:49.690+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6820 osd.7 since back 2023-11-16T16:51:19.851428+0000 front 2023-11-16T16:51:19.851402+0000 (oldest deadline 2023-11-16T16:51:44.551095+0000) 2023-11-16T16:51:51.080 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:50 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:50.642+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:51.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:50 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:50.642+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:51.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:50 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:50.642+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:51.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:50 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:50.642+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.747074+0000 front 2023-11-16T16:50:56.746975+0000 (oldest deadline 2023-11-16T16:51:23.147359+0000) 2023-11-16T16:51:51.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:50 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:50.642+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:05.348634+0000 front 2023-11-16T16:51:05.348649+0000 (oldest deadline 2023-11-16T16:51:28.848866+0000) 2023-11-16T16:51:51.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:50 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:50.642+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2023-11-16T16:51:10.550045+0000 front 2023-11-16T16:51:10.550066+0000 (oldest deadline 2023-11-16T16:51:36.449919+0000) 2023-11-16T16:51:51.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:50 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:50.642+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6820 osd.7 since back 2023-11-16T16:51:19.851428+0000 front 2023-11-16T16:51:19.851402+0000 (oldest deadline 2023-11-16T16:51:44.551095+0000) 2023-11-16T16:51:52.080 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:51 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:51.647+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:52.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:51 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:51.647+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:52.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:51 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:51.647+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:52.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:51 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:51.647+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.747074+0000 front 2023-11-16T16:50:56.746975+0000 (oldest deadline 2023-11-16T16:51:23.147359+0000) 2023-11-16T16:51:52.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:51 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:51.647+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:05.348634+0000 front 2023-11-16T16:51:05.348649+0000 (oldest deadline 2023-11-16T16:51:28.848866+0000) 2023-11-16T16:51:52.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:51 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:51.647+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2023-11-16T16:51:10.550045+0000 front 2023-11-16T16:51:10.550066+0000 (oldest deadline 2023-11-16T16:51:36.449919+0000) 2023-11-16T16:51:52.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:51 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:51.647+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6820 osd.7 since back 2023-11-16T16:51:19.851428+0000 front 2023-11-16T16:51:19.851402+0000 (oldest deadline 2023-11-16T16:51:44.551095+0000) 2023-11-16T16:51:53.080 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:52 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:52.653+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:53.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:52 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:52.653+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:53.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:52 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:52.653+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:53.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:52 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:52.653+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.747074+0000 front 2023-11-16T16:50:56.746975+0000 (oldest deadline 2023-11-16T16:51:23.147359+0000) 2023-11-16T16:51:53.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:52 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:52.653+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:05.348634+0000 front 2023-11-16T16:51:05.348649+0000 (oldest deadline 2023-11-16T16:51:28.848866+0000) 2023-11-16T16:51:53.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:52 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:52.653+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2023-11-16T16:51:10.550045+0000 front 2023-11-16T16:51:10.550066+0000 (oldest deadline 2023-11-16T16:51:36.449919+0000) 2023-11-16T16:51:53.082 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:52 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:52.653+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6820 osd.7 since back 2023-11-16T16:51:19.851428+0000 front 2023-11-16T16:51:19.851402+0000 (oldest deadline 2023-11-16T16:51:44.551095+0000) 2023-11-16T16:51:54.080 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:53 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:53.678+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2023-11-16T16:50:25.340686+0000 front 2023-11-16T16:50:25.340657+0000 (oldest deadline 2023-11-16T16:50:50.040696+0000) 2023-11-16T16:51:54.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:53 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:53.678+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6804 osd.1 since back 2023-11-16T16:50:35.842817+0000 front 2023-11-16T16:50:35.842623+0000 (oldest deadline 2023-11-16T16:50:58.742705+0000) 2023-11-16T16:51:54.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:53 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:53.678+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6804 osd.2 since back 2023-11-16T16:50:43.943999+0000 front 2023-11-16T16:50:43.944022+0000 (oldest deadline 2023-11-16T16:51:06.844190+0000) 2023-11-16T16:51:54.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:53 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:53.678+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6812 osd.4 since back 2023-11-16T16:50:56.747074+0000 front 2023-11-16T16:50:56.746975+0000 (oldest deadline 2023-11-16T16:51:23.147359+0000) 2023-11-16T16:51:54.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:53 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:53.678+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.53:6812 osd.5 since back 2023-11-16T16:51:05.348634+0000 front 2023-11-16T16:51:05.348649+0000 (oldest deadline 2023-11-16T16:51:28.848866+0000) 2023-11-16T16:51:54.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:53 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:53.678+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2023-11-16T16:51:10.550045+0000 front 2023-11-16T16:51:10.550066+0000 (oldest deadline 2023-11-16T16:51:36.449919+0000) 2023-11-16T16:51:54.081 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:53 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11[122773]: 2023-11-16T16:51:53.678+0000 7f22e1346700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.26:6820 osd.7 since back 2023-11-16T16:51:19.851428+0000 front 2023-11-16T16:51:19.851402+0000 (oldest deadline 2023-11-16T16:51:44.551095+0000) 2023-11-16T16:51:54.580 INFO:journalctl@ceph.osd.11.smithi053.stdout:Nov 16 16:51:54 smithi053 bash[133894]: ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-osd-11 2023-11-16T16:51:55.985 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@osd.11.service' 2023-11-16T16:51:56.027 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:51:56.027 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-11-16T16:51:56.027 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-11-16T16:51:56.027 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mds.a 2023-11-16T16:51:56.397 INFO:journalctl@ceph.mds.a.smithi001.stdout:Nov 16 16:51:56 smithi001 systemd[1]: Stopping Ceph mds.a for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:51:56.682 INFO:journalctl@ceph.mds.a.smithi001.stdout:Nov 16 16:51:56 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-a[133544]: 2023-11-16T16:51:56.395+0000 7f21a318c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:51:56.682 INFO:journalctl@ceph.mds.a.smithi001.stdout:Nov 16 16:51:56 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-a[133544]: 2023-11-16T16:51:56.395+0000 7f21a318c700 -1 mds.a *** got signal Terminated *** 2023-11-16T16:51:59.255 INFO:journalctl@ceph.mds.a.smithi001.stdout:Nov 16 16:51:59 smithi001 bash[263779]: ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-a 2023-11-16T16:51:59.454 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mds.a.service' 2023-11-16T16:51:59.491 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:51:59.492 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-11-16T16:51:59.492 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-11-16T16:51:59.493 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mds.d 2023-11-16T16:51:59.880 INFO:journalctl@ceph.mds.d.smithi001.stdout:Nov 16 16:51:59 smithi001 systemd[1]: Stopping Ceph mds.d for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:52:00.182 INFO:journalctl@ceph.mds.d.smithi001.stdout:Nov 16 16:51:59 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-d[132569]: 2023-11-16T16:51:59.878+0000 7fabc5316700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:52:00.182 INFO:journalctl@ceph.mds.d.smithi001.stdout:Nov 16 16:51:59 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-d[132569]: 2023-11-16T16:51:59.878+0000 7fabc5316700 -1 mds.d *** got signal Terminated *** 2023-11-16T16:52:03.143 INFO:journalctl@ceph.mds.d.smithi001.stdout:Nov 16 16:52:03 smithi001 bash[263919]: ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-d 2023-11-16T16:52:03.338 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mds.d.service' 2023-11-16T16:52:03.373 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:52:03.373 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-11-16T16:52:03.373 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-11-16T16:52:03.373 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mds.g 2023-11-16T16:52:03.682 INFO:journalctl@ceph.mds.g.smithi001.stdout:Nov 16 16:52:03 smithi001 systemd[1]: Stopping Ceph mds.g for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:52:04.182 INFO:journalctl@ceph.mds.g.smithi001.stdout:Nov 16 16:52:03 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-g[133153]: 2023-11-16T16:52:03.752+0000 7f859eb83700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:52:04.182 INFO:journalctl@ceph.mds.g.smithi001.stdout:Nov 16 16:52:03 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-g[133153]: 2023-11-16T16:52:03.752+0000 7f859eb83700 -1 mds.g *** got signal Terminated *** 2023-11-16T16:52:08.182 INFO:journalctl@ceph.mds.g.smithi001.stdout:Nov 16 16:52:07 smithi001 bash[264059]: ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-g 2023-11-16T16:52:08.339 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mds.g.service' 2023-11-16T16:52:08.375 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:52:08.375 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-11-16T16:52:08.375 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-11-16T16:52:08.375 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mds.j 2023-11-16T16:52:08.682 INFO:journalctl@ceph.mds.j.smithi001.stdout:Nov 16 16:52:08 smithi001 systemd[1]: Stopping Ceph mds.j for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:52:09.181 INFO:journalctl@ceph.mds.j.smithi001.stdout:Nov 16 16:52:08 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-j[132205]: 2023-11-16T16:52:08.748+0000 7f62f2ba7700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:52:09.182 INFO:journalctl@ceph.mds.j.smithi001.stdout:Nov 16 16:52:08 smithi001 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-j[132205]: 2023-11-16T16:52:08.748+0000 7f62f2ba7700 -1 mds.j *** got signal Terminated *** 2023-11-16T16:52:13.182 INFO:journalctl@ceph.mds.j.smithi001.stdout:Nov 16 16:52:12 smithi001 bash[264206]: ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-j 2023-11-16T16:52:16.246 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mds.j.service' 2023-11-16T16:52:16.248 INFO:journalctl@ceph.mds.j.smithi001.stdout:Nov 16 16:52:13 smithi001 systemd[1]: ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mds.j.service: Succeeded. 2023-11-16T16:52:16.248 INFO:journalctl@ceph.mds.j.smithi001.stdout:Nov 16 16:52:13 smithi001 systemd[1]: Stopped Ceph mds.j for 35d9b010-849d-11ee-8dbe-212e2dc638e7. 2023-11-16T16:52:16.291 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:52:16.291 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-11-16T16:52:16.291 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-11-16T16:52:16.292 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mds.b 2023-11-16T16:52:16.649 INFO:journalctl@ceph.mds.b.smithi026.stdout:Nov 16 16:52:16 smithi026 systemd[1]: Stopping Ceph mds.b for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:52:17.008 INFO:journalctl@ceph.mds.b.smithi026.stdout:Nov 16 16:52:16 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-b[125397]: 2023-11-16T16:52:16.647+0000 7fe7cc0bd700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:52:17.008 INFO:journalctl@ceph.mds.b.smithi026.stdout:Nov 16 16:52:16 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-b[125397]: 2023-11-16T16:52:16.647+0000 7fe7cc0bd700 -1 mds.b *** got signal Terminated *** 2023-11-16T16:52:21.198 INFO:journalctl@ceph.mds.b.smithi026.stdout:Nov 16 16:52:20 smithi026 bash[132706]: ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-b 2023-11-16T16:52:21.204 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mds.b.service' 2023-11-16T16:52:21.242 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:52:21.242 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-11-16T16:52:21.242 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-11-16T16:52:21.242 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mds.e 2023-11-16T16:52:21.508 INFO:journalctl@ceph.mds.e.smithi026.stdout:Nov 16 16:52:21 smithi026 systemd[1]: Stopping Ceph mds.e for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:52:22.008 INFO:journalctl@ceph.mds.e.smithi026.stdout:Nov 16 16:52:21 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-e[124717]: 2023-11-16T16:52:21.613+0000 7f2f440db700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:52:22.008 INFO:journalctl@ceph.mds.e.smithi026.stdout:Nov 16 16:52:21 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-e[124717]: 2023-11-16T16:52:21.613+0000 7f2f440db700 -1 mds.e *** got signal Terminated *** 2023-11-16T16:52:25.892 INFO:journalctl@ceph.mds.e.smithi026.stdout:Nov 16 16:52:25 smithi026 bash[132861]: ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-e 2023-11-16T16:52:25.897 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mds.e.service' 2023-11-16T16:52:25.934 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:52:25.934 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-11-16T16:52:25.934 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-11-16T16:52:25.935 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mds.h 2023-11-16T16:52:26.257 INFO:journalctl@ceph.mds.h.smithi026.stdout:Nov 16 16:52:26 smithi026 systemd[1]: Stopping Ceph mds.h for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:52:26.758 INFO:journalctl@ceph.mds.h.smithi026.stdout:Nov 16 16:52:26 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-h[125059]: 2023-11-16T16:52:26.302+0000 7f1a97426700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:52:26.758 INFO:journalctl@ceph.mds.h.smithi026.stdout:Nov 16 16:52:26 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-h[125059]: 2023-11-16T16:52:26.302+0000 7f1a97426700 -1 mds.h *** got signal Terminated *** 2023-11-16T16:52:30.005 INFO:journalctl@ceph.mds.h.smithi026.stdout:Nov 16 16:52:29 smithi026 bash[133000]: ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-h 2023-11-16T16:52:30.011 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mds.h.service' 2023-11-16T16:52:30.045 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:52:30.046 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-11-16T16:52:30.046 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-11-16T16:52:30.046 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mds.k 2023-11-16T16:52:30.258 INFO:journalctl@ceph.mds.k.smithi026.stdout:Nov 16 16:52:30 smithi026 systemd[1]: Stopping Ceph mds.k for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:52:30.758 INFO:journalctl@ceph.mds.k.smithi026.stdout:Nov 16 16:52:30 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-k[124375]: 2023-11-16T16:52:30.418+0000 7f4947586700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:52:30.758 INFO:journalctl@ceph.mds.k.smithi026.stdout:Nov 16 16:52:30 smithi026 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-k[124375]: 2023-11-16T16:52:30.418+0000 7f4947586700 -1 mds.k *** got signal Terminated *** 2023-11-16T16:52:34.746 INFO:journalctl@ceph.mds.k.smithi026.stdout:Nov 16 16:52:34 smithi026 bash[133155]: ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-k 2023-11-16T16:52:34.871 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mds.k.service' 2023-11-16T16:52:34.905 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:52:34.905 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-11-16T16:52:34.905 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-11-16T16:52:34.905 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mds.c 2023-11-16T16:52:35.282 INFO:journalctl@ceph.mds.c.smithi053.stdout:Nov 16 16:52:34 smithi053 systemd[1]: Stopping Ceph mds.c for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:52:35.580 INFO:journalctl@ceph.mds.c.smithi053.stdout:Nov 16 16:52:35 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-c[126266]: 2023-11-16T16:52:35.280+0000 7faf87e41700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:52:35.581 INFO:journalctl@ceph.mds.c.smithi053.stdout:Nov 16 16:52:35 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-c[126266]: 2023-11-16T16:52:35.280+0000 7faf87e41700 -1 mds.c *** got signal Terminated *** 2023-11-16T16:52:38.024 INFO:journalctl@ceph.mds.c.smithi053.stdout:Nov 16 16:52:37 smithi053 bash[134166]: ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-c 2023-11-16T16:52:38.031 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mds.c.service' 2023-11-16T16:52:38.068 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:52:38.068 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-11-16T16:52:38.069 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-11-16T16:52:38.069 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mds.f 2023-11-16T16:52:38.330 INFO:journalctl@ceph.mds.f.smithi053.stdout:Nov 16 16:52:38 smithi053 systemd[1]: Stopping Ceph mds.f for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:52:38.830 INFO:journalctl@ceph.mds.f.smithi053.stdout:Nov 16 16:52:38 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-f[125922]: 2023-11-16T16:52:38.455+0000 7fbb68ffb700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:52:38.831 INFO:journalctl@ceph.mds.f.smithi053.stdout:Nov 16 16:52:38 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-f[125922]: 2023-11-16T16:52:38.455+0000 7fbb68ffb700 -1 mds.f *** got signal Terminated *** 2023-11-16T16:52:41.822 INFO:journalctl@ceph.mds.f.smithi053.stdout:Nov 16 16:52:41 smithi053 bash[134312]: ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-f 2023-11-16T16:52:42.022 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mds.f.service' 2023-11-16T16:52:42.058 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:52:42.058 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-11-16T16:52:42.058 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-11-16T16:52:42.059 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mds.i 2023-11-16T16:52:42.330 INFO:journalctl@ceph.mds.i.smithi053.stdout:Nov 16 16:52:42 smithi053 systemd[1]: Stopping Ceph mds.i for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:52:42.830 INFO:journalctl@ceph.mds.i.smithi053.stdout:Nov 16 16:52:42 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-i[126607]: 2023-11-16T16:52:42.446+0000 7f239665a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:52:42.831 INFO:journalctl@ceph.mds.i.smithi053.stdout:Nov 16 16:52:42 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-i[126607]: 2023-11-16T16:52:42.446+0000 7f239665a700 -1 mds.i *** got signal Terminated *** 2023-11-16T16:52:46.008 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mds.i.service' 2023-11-16T16:52:46.043 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:52:46.043 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-11-16T16:52:46.044 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-11-16T16:52:46.044 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mds.l 2023-11-16T16:52:46.438 INFO:journalctl@ceph.mds.l.smithi053.stdout:Nov 16 16:52:46 smithi053 systemd[1]: Stopping Ceph mds.l for 35d9b010-849d-11ee-8dbe-212e2dc638e7... 2023-11-16T16:52:46.830 INFO:journalctl@ceph.mds.l.smithi053.stdout:Nov 16 16:52:46 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-l[125578]: 2023-11-16T16:52:46.436+0000 7f20acf93700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-16T16:52:46.831 INFO:journalctl@ceph.mds.l.smithi053.stdout:Nov 16 16:52:46 smithi053 ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-l[125578]: 2023-11-16T16:52:46.436+0000 7f20acf93700 -1 mds.l *** got signal Terminated *** 2023-11-16T16:52:48.386 INFO:journalctl@ceph.mds.l.smithi053.stdout:Nov 16 16:52:48 smithi053 bash[134592]: ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7-mds-l 2023-11-16T16:52:48.593 DEBUG:teuthology.orchestra.run.smithi053:> sudo pkill -f 'journalctl -f -n 0 -u ceph-35d9b010-849d-11ee-8dbe-212e2dc638e7@mds.l.service' 2023-11-16T16:52:48.628 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-16T16:52:48.628 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-11-16T16:52:48.628 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 --force --keep-logs 2023-11-16T16:52:51.247 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 --force --keep-logs 2023-11-16T16:52:53.319 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 --force --keep-logs 2023-11-16T16:52:55.369 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-16T16:52:55.405 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-16T16:52:55.442 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-16T16:52:55.473 INFO:tasks.cephadm:Archiving crash dumps... 2023-11-16T16:52:55.475 DEBUG:teuthology.misc:Transferring archived files from smithi001:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/crash to /home/teuthworker/archive/choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi/7460919/remote/smithi001/crash 2023-11-16T16:52:55.475 DEBUG:teuthology.orchestra.run.smithi001:> sudo tar c -f - -C /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/crash -- . 2023-11-16T16:52:55.518 INFO:teuthology.orchestra.run.smithi001.stderr:tar: /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/crash: Cannot open: No such file or directory 2023-11-16T16:52:55.518 INFO:teuthology.orchestra.run.smithi001.stderr:tar: Error is not recoverable: exiting now 2023-11-16T16:52:55.520 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/crash to /home/teuthworker/archive/choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi/7460919/remote/smithi026/crash 2023-11-16T16:52:55.520 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/crash -- . 2023-11-16T16:52:55.563 INFO:teuthology.orchestra.run.smithi026.stderr:tar: /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/crash: Cannot open: No such file or directory 2023-11-16T16:52:55.564 INFO:teuthology.orchestra.run.smithi026.stderr:tar: Error is not recoverable: exiting now 2023-11-16T16:52:55.565 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/crash to /home/teuthworker/archive/choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi/7460919/remote/smithi053/crash 2023-11-16T16:52:55.566 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/crash -- . 2023-11-16T16:52:55.614 INFO:teuthology.orchestra.run.smithi053.stderr:tar: /var/lib/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/crash: Cannot open: No such file or directory 2023-11-16T16:52:55.614 INFO:teuthology.orchestra.run.smithi053.stderr:tar: Error is not recoverable: exiting now 2023-11-16T16:52:55.615 INFO:tasks.cephadm:Checking cluster log for badness... 2023-11-16T16:52:55.616 DEBUG:teuthology.orchestra.run.smithi001:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-11-16T16:52:55.652 INFO:teuthology.orchestra.run.smithi001.stderr:grep: /var/log/ceph/35d9b010-849d-11ee-8dbe-212e2dc638e7/ceph.log: No such file or directory 2023-11-16T16:52:55.660 INFO:tasks.cephadm:Compressing logs... 2023-11-16T16:52:55.660 DEBUG:teuthology.orchestra.run.smithi001:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-16T16:52:55.703 DEBUG:teuthology.orchestra.run.smithi026:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-16T16:52:55.705 DEBUG:teuthology.orchestra.run.smithi053:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-16T16:52:55.732 INFO:teuthology.orchestra.run.smithi053.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-11-16T16:52:55.736 INFO:teuthology.orchestra.run.smithi026.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-11-16T16:52:55.746 INFO:teuthology.orchestra.run.smithi001.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-11-16T16:56:46.015 INFO:tasks.cephadm:Archiving logs... 2023-11-16T16:56:46.016 DEBUG:teuthology.misc:Transferring archived files from smithi001:/var/log/ceph to /home/teuthworker/archive/choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi/7460919/remote/smithi001/log 2023-11-16T16:56:46.018 DEBUG:teuthology.orchestra.run.smithi001:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-16T16:56:49.584 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/log/ceph to /home/teuthworker/archive/choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi/7460919/remote/smithi026/log 2023-11-16T16:56:49.585 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-16T16:57:10.430 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/log/ceph to /home/teuthworker/archive/choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi/7460919/remote/smithi053/log 2023-11-16T16:57:10.431 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-16T16:57:25.200 INFO:tasks.cephadm:Removing cluster... 2023-11-16T16:57:25.201 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 --force 2023-11-16T16:57:26.122 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 --force 2023-11-16T16:57:27.177 DEBUG:teuthology.orchestra.run.smithi053:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 35d9b010-849d-11ee-8dbe-212e2dc638e7 --force 2023-11-16T16:57:28.185 INFO:tasks.cephadm:Removing cephadm ... 2023-11-16T16:57:28.186 DEBUG:teuthology.orchestra.run.smithi001:> rm -rf /home/ubuntu/cephtest/cephadm 2023-11-16T16:57:28.208 DEBUG:teuthology.orchestra.run.smithi026:> rm -rf /home/ubuntu/cephtest/cephadm 2023-11-16T16:57:28.232 DEBUG:teuthology.orchestra.run.smithi053:> rm -rf /home/ubuntu/cephtest/cephadm 2023-11-16T16:57:28.250 INFO:tasks.cephadm:Teardown complete 2023-11-16T16:57:28.251 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-16T16:57:28.263 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-16T16:57:28.263 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-16T16:57:28.266 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-16T16:57:28.275 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-16T16:57:28.392 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-16T16:57:28.403 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-16T16:57:28.403 DEBUG:teuthology.orchestra.run.smithi001:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-16T16:57:28.407 DEBUG:teuthology.orchestra.run.smithi026:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-16T16:57:28.410 DEBUG:teuthology.orchestra.run.smithi053:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-16T16:57:28.424 INFO:teuthology.orchestra.run.smithi001.stderr:bash: ntpq: command not found 2023-11-16T16:57:28.426 INFO:teuthology.orchestra.run.smithi053.stderr:bash: ntpq: command not found 2023-11-16T16:57:28.427 INFO:teuthology.orchestra.run.smithi026.stderr:bash: ntpq: command not found 2023-11-16T16:57:28.431 INFO:teuthology.orchestra.run.smithi053.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-16T16:57:28.431 INFO:teuthology.orchestra.run.smithi053.stdout:=============================================================================== 2023-11-16T16:57:28.431 INFO:teuthology.orchestra.run.smithi053.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 38 +1011us[+1009us] +/- 87ms 2023-11-16T16:57:28.431 INFO:teuthology.orchestra.run.smithi053.stdout:^* hv02.front.sepia.ceph.com 3 6 377 36 -464us[ -465us] +/- 65ms 2023-11-16T16:57:28.432 INFO:teuthology.orchestra.run.smithi053.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 40 -481us[ -483us] +/- 99ms 2023-11-16T16:57:28.432 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T16:57:28.435 INFO:teuthology.orchestra.run.smithi026.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-16T16:57:28.435 INFO:teuthology.orchestra.run.smithi026.stdout:=============================================================================== 2023-11-16T16:57:28.435 INFO:teuthology.orchestra.run.smithi026.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 39 +1008us[+1008us] +/- 87ms 2023-11-16T16:57:28.435 INFO:teuthology.orchestra.run.smithi026.stdout:^* hv02.front.sepia.ceph.com 3 8 377 42 -423us[ -425us] +/- 65ms 2023-11-16T16:57:28.435 INFO:teuthology.orchestra.run.smithi026.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 38 -484us[ -484us] +/- 99ms 2023-11-16T16:57:28.436 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T16:57:28.438 INFO:teuthology.orchestra.run.smithi001.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-16T16:57:28.439 INFO:teuthology.orchestra.run.smithi001.stdout:=============================================================================== 2023-11-16T16:57:28.439 INFO:teuthology.orchestra.run.smithi001.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 38 +1040us[+1040us] +/- 87ms 2023-11-16T16:57:28.439 INFO:teuthology.orchestra.run.smithi001.stdout:^* hv02.front.sepia.ceph.com 3 6 377 39 -433us[ -408us] +/- 65ms 2023-11-16T16:57:28.440 INFO:teuthology.orchestra.run.smithi001.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 -448us[ -424us] +/- 99ms 2023-11-16T16:57:28.440 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-11-16T16:57:28.441 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-16T16:57:28.461 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-16T16:57:28.462 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-16T16:57:28.527 DEBUG:teuthology.orchestra.run.smithi001:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-16T16:57:28.751 DEBUG:teuthology.orchestra.run.smithi026:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-16T16:57:28.857 DEBUG:teuthology.orchestra.run.smithi053:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-16T16:57:28.960 DEBUG:teuthology.orchestra.run.smithi001:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-16T16:57:28.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-16T16:57:28.997 DEBUG:teuthology.orchestra.run.smithi026:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-16T16:57:29.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-16T16:57:29.031 DEBUG:teuthology.orchestra.run.smithi053:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-16T16:57:29.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-16T16:57:29.065 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-16T16:57:29.076 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-16T16:57:29.130 INFO:teuthology.task.internal:Duration was 2525.673512 seconds 2023-11-16T16:57:29.130 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-16T16:57:29.210 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-16T16:57:29.210 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-16T16:57:29.213 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-16T16:57:29.216 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-16T16:57:29.266 INFO:teuthology.orchestra.run.smithi053.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-16T16:57:29.268 INFO:teuthology.orchestra.run.smithi026.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-16T16:57:29.310 INFO:teuthology.orchestra.run.smithi001.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-16T16:57:29.750 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-16T16:57:29.751 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi001.front.sepia.ceph.com 2023-11-16T16:57:29.751 DEBUG:teuthology.orchestra.run.smithi001:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-16T16:57:29.801 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi026.front.sepia.ceph.com 2023-11-16T16:57:29.802 DEBUG:teuthology.orchestra.run.smithi026:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-16T16:57:29.829 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi053.front.sepia.ceph.com 2023-11-16T16:57:29.830 DEBUG:teuthology.orchestra.run.smithi053:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-16T16:57:29.855 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-16T16:57:29.855 DEBUG:teuthology.orchestra.run.smithi001:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-16T16:57:29.858 DEBUG:teuthology.orchestra.run.smithi026:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-16T16:57:29.873 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-16T16:57:29.949 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-16T16:57:29.949 DEBUG:teuthology.orchestra.run.smithi001:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-16T16:57:29.952 DEBUG:teuthology.orchestra.run.smithi026:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-16T16:57:29.954 DEBUG:teuthology.orchestra.run.smithi053:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-16T16:57:30.093 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-16T16:57:30.105 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-16T16:57:30.106 DEBUG:teuthology.orchestra.run.smithi001:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-16T16:57:30.147 DEBUG:teuthology.orchestra.run.smithi026:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-16T16:57:30.176 DEBUG:teuthology.orchestra.run.smithi053:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-16T16:57:30.203 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-16T16:57:30.215 DEBUG:teuthology.orchestra.run.smithi001:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-16T16:57:30.218 DEBUG:teuthology.orchestra.run.smithi026:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-16T16:57:30.220 DEBUG:teuthology.orchestra.run.smithi053:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-16T16:57:30.245 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern = core 2023-11-16T16:57:30.255 INFO:teuthology.orchestra.run.smithi001.stdout:kernel.core_pattern = core 2023-11-16T16:57:30.270 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = core 2023-11-16T16:57:30.298 DEBUG:teuthology.orchestra.run.smithi001:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-16T16:57:30.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-16T16:57:30.355 DEBUG:teuthology.orchestra.run.smithi026:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-16T16:57:30.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-16T16:57:30.372 DEBUG:teuthology.orchestra.run.smithi053:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-16T16:57:30.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-16T16:57:30.389 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-16T16:57:30.400 INFO:teuthology.task.internal:Transferring archived files... 2023-11-16T16:57:30.401 DEBUG:teuthology.misc:Transferring archived files from smithi001:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi/7460919/remote/smithi001 2023-11-16T16:57:30.401 DEBUG:teuthology.orchestra.run.smithi001:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-16T16:57:30.444 DEBUG:teuthology.misc:Transferring archived files from smithi026:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi/7460919/remote/smithi026 2023-11-16T16:57:30.444 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-16T16:57:30.485 DEBUG:teuthology.misc:Transferring archived files from smithi053:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi/7460919/remote/smithi053 2023-11-16T16:57:30.485 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-16T16:57:30.526 INFO:teuthology.task.internal:Removing archive directory... 2023-11-16T16:57:30.527 DEBUG:teuthology.orchestra.run.smithi001:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-16T16:57:30.529 DEBUG:teuthology.orchestra.run.smithi026:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-16T16:57:30.531 DEBUG:teuthology.orchestra.run.smithi053:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-16T16:57:30.579 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-16T16:57:30.592 INFO:teuthology.task.internal:Not uploading archives. 2023-11-16T16:57:30.592 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-16T16:57:30.635 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-16T16:57:30.635 DEBUG:teuthology.orchestra.run.smithi001:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-16T16:57:30.637 DEBUG:teuthology.orchestra.run.smithi026:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-16T16:57:30.640 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-16T16:57:30.652 INFO:teuthology.orchestra.run.smithi001.stdout: 262307 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 16 16:57 /home/ubuntu/cephtest 2023-11-16T16:57:30.654 INFO:teuthology.orchestra.run.smithi026.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 16 16:57 /home/ubuntu/cephtest 2023-11-16T16:57:30.656 INFO:teuthology.orchestra.run.smithi053.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 16 16:57 /home/ubuntu/cephtest 2023-11-16T16:57:30.658 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-16T16:57:30.702 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-16T16:57:30.819 INFO:teuthology.nuke:Checking targets against current locks 2023-11-16T16:57:30.846 DEBUG:teuthology.nuke:shortname: smithi001 2023-11-16T16:57:30.847 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-16T16:57:30.870 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi001.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi/7460919', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-16 16:10:26.161957', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-16T16:57:30.874 DEBUG:teuthology.nuke:shortname: smithi026 2023-11-16T16:57:30.874 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-16T16:57:30.896 DEBUG:teuthology.nuke:shortname: smithi053 2023-11-16T16:57:30.896 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-16T16:57:30.899 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi026.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi/7460919', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-16 16:10:26.160345', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-16T16:57:30.912 INFO:teuthology.orchestra.console.smithi001:Power off 2023-11-16T16:57:30.913 DEBUG:teuthology.orchestra.console.smithi001:pexpect command: ipmitool -H smithi001.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-16T16:57:30.923 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi053.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-11-16_16:09:17-fs:workload-wip-choffman-62793-distro-default-smithi/7460919', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-16 16:10:26.158524', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-16T16:57:30.935 DEBUG:teuthology.orchestra.console.smithi001:power off output: Chassis Power Control: Down/Off 2023-11-16T16:57:30.935 DEBUG:teuthology.orchestra.console.smithi001:pexpect command: ipmitool -H smithi001.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-16T16:57:30.952 INFO:teuthology.orchestra.console.smithi026:Power off 2023-11-16T16:57:30.953 DEBUG:teuthology.orchestra.console.smithi026:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-16T16:57:30.964 DEBUG:teuthology.orchestra.console.smithi001:check power output: Chassis Power is on 2023-11-16T16:57:30.974 DEBUG:teuthology.orchestra.console.smithi026:power off output: Chassis Power Control: Down/Off 2023-11-16T16:57:30.974 DEBUG:teuthology.orchestra.console.smithi026:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-16T16:57:30.992 INFO:teuthology.orchestra.console.smithi053:Power off 2023-11-16T16:57:30.992 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-16T16:57:31.001 DEBUG:teuthology.orchestra.console.smithi026:check power output: Chassis Power is on 2023-11-16T16:57:31.016 DEBUG:teuthology.orchestra.console.smithi053:power off output: Chassis Power Control: Down/Off 2023-11-16T16:57:31.016 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-16T16:57:31.036 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is on 2023-11-16T16:57:34.966 DEBUG:teuthology.orchestra.console.smithi001:pexpect command: ipmitool -H smithi001.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-16T16:57:35.002 DEBUG:teuthology.orchestra.console.smithi026:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-16T16:57:35.037 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-16T16:57:35.080 DEBUG:teuthology.orchestra.console.smithi001:check power output: Chassis Power is on 2023-11-16T16:57:35.113 DEBUG:teuthology.orchestra.console.smithi026:check power output: Chassis Power is on 2023-11-16T16:57:35.148 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is on 2023-11-16T16:57:39.086 DEBUG:teuthology.orchestra.console.smithi001:pexpect command: ipmitool -H smithi001.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-16T16:57:39.115 DEBUG:teuthology.orchestra.console.smithi026:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-16T16:57:39.149 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-16T16:57:39.196 DEBUG:teuthology.orchestra.console.smithi001:check power output: Chassis Power is off 2023-11-16T16:57:39.224 DEBUG:teuthology.orchestra.console.smithi026:check power output: Chassis Power is off 2023-11-16T16:57:39.257 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is off 2023-11-16T16:57:39.297 INFO:teuthology.orchestra.console.smithi001:Power off completed 2023-11-16T16:57:39.324 INFO:teuthology.orchestra.console.smithi026:Power off completed 2023-11-16T16:57:39.358 INFO:teuthology.orchestra.console.smithi053:Power off completed 2023-11-16T16:57:39.465 INFO:teuthology.run:Summary data: description: fs:workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/fs/misc}} duration: 2525.6735122203827 failure_reason: 'Command failed (workunit test fs/misc/general_vxattrs.sh) on smithi001 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/general_vxattrs.sh''' flavor: default owner: scheduled_choffman@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=e78be233e0634c3cbfd81992005a60b1 status: fail success: false 2023-11-16T16:57:39.465 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-16T16:57:39.547 INFO:teuthology.run:FAIL